From patchwork Mon Jan 7 08:33:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 10750119 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A8FBB14DE for ; Mon, 7 Jan 2019 08:33:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8A2BA28606 for ; Mon, 7 Jan 2019 08:33:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 781B22868D; Mon, 7 Jan 2019 08:33:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2670928606 for ; Mon, 7 Jan 2019 08:33:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726448AbfAGIdy (ORCPT ); Mon, 7 Jan 2019 03:33:54 -0500 Received: from cloud.peff.net ([104.130.231.41]:56308 "HELO cloud.peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1725550AbfAGIdy (ORCPT ); Mon, 7 Jan 2019 03:33:54 -0500 Received: (qmail 1112 invoked by uid 109); 7 Jan 2019 08:33:55 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with SMTP; Mon, 07 Jan 2019 08:33:55 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 684 invoked by uid 111); 7 Jan 2019 08:33:33 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) by peff.net (qpsmtpd/0.94) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) SMTP; Mon, 07 Jan 2019 03:33:33 -0500 Authentication-Results: peff.net; auth=none Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Mon, 07 Jan 2019 03:33:52 -0500 Date: Mon, 7 Jan 2019 03:33:52 -0500 From: Jeff King To: =?utf-8?b?UmVuw6k=?= Scharfe Cc: git@vger.kernel.org, Junio C Hamano , =?utf-8?b?w4Z2?= =?utf-8?b?YXIgQXJuZmrDtnLDsA==?= Bjarmason Subject: [PATCH 01/11] sha1-file: fix outdated sha1 comment references Message-ID: <20190107083352.GA29431@sigill.intra.peff.net> References: <20190107083150.GC21362@sigill.intra.peff.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190107083150.GC21362@sigill.intra.peff.net> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Commit 17e65451e3 (sha1_file: convert check_sha1_signature to struct object_id, 2018-03-12) switched to using the name "oid", but forgot to update the variable name in the comment. Likewise, b4f5aca40e (sha1_file: convert read_sha1_file to struct object_id, 2018-03-12) dropped the name read_sha1_file(), but missed a comment which mentions it. Signed-off-by: Jeff King --- sha1-file.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sha1-file.c b/sha1-file.c index c3c6e50704..e86bb28320 100644 --- a/sha1-file.c +++ b/sha1-file.c @@ -124,7 +124,7 @@ const char *empty_blob_oid_hex(void) /* * This is meant to hold a *small* number of objects that you would - * want read_sha1_file() to be able to return, but yet you do not want + * want read_object_file() to be able to return, but yet you do not want * to write them into the object store (e.g. a browse-only * application). */ @@ -798,8 +798,8 @@ void *xmmap(void *start, size_t length, /* * With an in-core object data in "map", rehash it to make sure the - * object name actually matches "sha1" to detect object corruption. - * With "map" == NULL, try reading the object named with "sha1" using + * object name actually matches "oid" to detect object corruption. + * With "map" == NULL, try reading the object named with "oid" using * the streaming interface and rehash it to do the same. */ int check_object_signature(const struct object_id *oid, void *map, From patchwork Mon Jan 7 08:34:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 10750121 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA5A914DE for ; Mon, 7 Jan 2019 08:34:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B9CDA28606 for ; Mon, 7 Jan 2019 08:34:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ADE832868D; Mon, 7 Jan 2019 08:34:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D22D28606 for ; Mon, 7 Jan 2019 08:34:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726516AbfAGIeO (ORCPT ); Mon, 7 Jan 2019 03:34:14 -0500 Received: from cloud.peff.net ([104.130.231.41]:56318 "HELO cloud.peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1725550AbfAGIeO (ORCPT ); Mon, 7 Jan 2019 03:34:14 -0500 Received: (qmail 1182 invoked by uid 109); 7 Jan 2019 08:34:14 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with SMTP; Mon, 07 Jan 2019 08:34:14 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 703 invoked by uid 111); 7 Jan 2019 08:33:52 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) by peff.net (qpsmtpd/0.94) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) SMTP; Mon, 07 Jan 2019 03:33:52 -0500 Authentication-Results: peff.net; auth=none Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Mon, 07 Jan 2019 03:34:12 -0500 Date: Mon, 7 Jan 2019 03:34:12 -0500 From: Jeff King To: =?utf-8?b?UmVuw6k=?= Scharfe Cc: git@vger.kernel.org, Junio C Hamano , =?utf-8?b?w4Z2?= =?utf-8?b?YXIgQXJuZmrDtnLDsA==?= Bjarmason Subject: [PATCH 02/11] update comment references to sha1_object_info() Message-ID: <20190107083412.GB29431@sigill.intra.peff.net> References: <20190107083150.GC21362@sigill.intra.peff.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190107083150.GC21362@sigill.intra.peff.net> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Commit abef9020e3 (sha1_file: convert sha1_object_info* to object_id, 2018-03-12) renamed the function to oid_object_info(), but missed some comments which mention it. Signed-off-by: Jeff King --- builtin/cat-file.c | 6 +++--- builtin/pack-objects.c | 4 ++-- cache.h | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 2ca56fd086..baaafbdcf3 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -209,14 +209,14 @@ struct expand_data { /* * After a mark_query run, this object_info is set up to be - * passed to sha1_object_info_extended. It will point to the data + * passed to oid_object_info_extended. It will point to the data * elements above, so you can retrieve the response from there. */ struct object_info info; /* * This flag will be true if the requested batch format and options - * don't require us to call sha1_object_info, which can then be + * don't require us to call oid_object_info, which can then be * optimized out. */ unsigned skip_object_info : 1; @@ -490,7 +490,7 @@ static int batch_objects(struct batch_options *opt) /* * Expand once with our special mark_query flag, which will prime the - * object_info to be handed to sha1_object_info_extended for each + * object_info to be handed to oid_object_info_extended for each * object. */ memset(&data, 0, sizeof(data)); diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 24bba8147f..d4c3987f3e 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -1642,7 +1642,7 @@ static void check_object(struct object_entry *entry) /* * No choice but to fall back to the recursive delta walk - * with sha1_object_info() to find about the object type + * with oid_object_info() to find about the object type * at this point... */ give_up: @@ -1718,7 +1718,7 @@ static void drop_reused_delta(struct object_entry *entry) if (packed_object_info(the_repository, IN_PACK(entry), entry->in_pack_offset, &oi) < 0) { /* * We failed to get the info from this pack for some reason; - * fall back to sha1_object_info, which may find another copy. + * fall back to oid_object_info, which may find another copy. * And if that fails, the error will be recorded in oe_type(entry) * and dealt with in prepare_pack(). */ diff --git a/cache.h b/cache.h index ca36b44ee0..587512747b 100644 --- a/cache.h +++ b/cache.h @@ -1592,7 +1592,7 @@ extern int odb_mkstemp(struct strbuf *temp_filename, const char *pattern); extern int odb_pack_keep(const char *name); /* - * Set this to 0 to prevent sha1_object_info_extended() from fetching missing + * Set this to 0 to prevent oid_object_info_extended() from fetching missing * blobs. This has a difference only if extensions.partialClone is set. * * Its default value is 1. From patchwork Mon Jan 7 08:34:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 10750123 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 991AF746 for ; Mon, 7 Jan 2019 08:34:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8779B28A1E for ; Mon, 7 Jan 2019 08:34:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7BB8128A22; Mon, 7 Jan 2019 08:34:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 08D0028A1E for ; Mon, 7 Jan 2019 08:34:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726542AbfAGIen (ORCPT ); Mon, 7 Jan 2019 03:34:43 -0500 Received: from cloud.peff.net ([104.130.231.41]:56334 "HELO cloud.peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1725550AbfAGIem (ORCPT ); Mon, 7 Jan 2019 03:34:42 -0500 Received: (qmail 1225 invoked by uid 109); 7 Jan 2019 08:34:43 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with SMTP; Mon, 07 Jan 2019 08:34:43 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 723 invoked by uid 111); 7 Jan 2019 08:34:21 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) by peff.net (qpsmtpd/0.94) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) SMTP; Mon, 07 Jan 2019 03:34:21 -0500 Authentication-Results: peff.net; auth=none Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Mon, 07 Jan 2019 03:34:40 -0500 Date: Mon, 7 Jan 2019 03:34:40 -0500 From: Jeff King To: =?utf-8?b?UmVuw6k=?= Scharfe Cc: git@vger.kernel.org, Junio C Hamano , =?utf-8?b?w4Z2?= =?utf-8?b?YXIgQXJuZmrDtnLDsA==?= Bjarmason Subject: [PATCH 03/11] http: use struct object_id instead of bare sha1 Message-ID: <20190107083440.GC29431@sigill.intra.peff.net> References: <20190107083150.GC21362@sigill.intra.peff.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190107083150.GC21362@sigill.intra.peff.net> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The dumb-http walker code still passes around and stores object ids as "unsigned char *sha1". Let's modernize it. There's probably still more work to be done to handle dumb-http fetches with a new, larger hash. But that can wait; this is enough that we can now convert some of the low-level object routines that we call into from here (and in fact, some of the "oid.hash" references added here will be further improved in the next patch). Signed-off-by: Jeff King --- http-push.c | 2 +- http-walker.c | 6 +++--- http.c | 14 +++++++------- http.h | 6 +++--- 4 files changed, 14 insertions(+), 14 deletions(-) diff --git a/http-push.c b/http-push.c index cd48590912..0141b0ad53 100644 --- a/http-push.c +++ b/http-push.c @@ -255,7 +255,7 @@ static void start_fetch_loose(struct transfer_request *request) struct active_request_slot *slot; struct http_object_request *obj_req; - obj_req = new_http_object_request(repo->url, request->obj->oid.hash); + obj_req = new_http_object_request(repo->url, &request->obj->oid); if (obj_req == NULL) { request->state = ABORTED; return; diff --git a/http-walker.c b/http-walker.c index 0a392c85b6..856716c63d 100644 --- a/http-walker.c +++ b/http-walker.c @@ -58,7 +58,7 @@ static void start_object_request(struct walker *walker, struct active_request_slot *slot; struct http_object_request *req; - req = new_http_object_request(obj_req->repo->base, obj_req->oid.hash); + req = new_http_object_request(obj_req->repo->base, &obj_req->oid); if (req == NULL) { obj_req->state = ABORTED; return; @@ -543,11 +543,11 @@ static int fetch_object(struct walker *walker, unsigned char *sha1) } else if (req->zret != Z_STREAM_END) { walker->corrupt_object_found++; ret = error("File %s (%s) corrupt", hex, req->url); - } else if (!hasheq(obj_req->oid.hash, req->real_sha1)) { + } else if (!oideq(&obj_req->oid, &req->real_oid)) { ret = error("File %s has bad hash", hex); } else if (req->rename < 0) { struct strbuf buf = STRBUF_INIT; - loose_object_path(the_repository, &buf, req->sha1); + loose_object_path(the_repository, &buf, req->oid.hash); ret = error("unable to write sha1 filename %s", buf.buf); strbuf_release(&buf); } diff --git a/http.c b/http.c index 0b6807cef9..8d42154792 100644 --- a/http.c +++ b/http.c @@ -2337,9 +2337,9 @@ static size_t fwrite_sha1_file(char *ptr, size_t eltsize, size_t nmemb, } struct http_object_request *new_http_object_request(const char *base_url, - unsigned char *sha1) + const struct object_id *oid) { - char *hex = sha1_to_hex(sha1); + char *hex = oid_to_hex(oid); struct strbuf filename = STRBUF_INIT; struct strbuf prevfile = STRBUF_INIT; int prevlocal; @@ -2350,10 +2350,10 @@ struct http_object_request *new_http_object_request(const char *base_url, freq = xcalloc(1, sizeof(*freq)); strbuf_init(&freq->tmpfile, 0); - hashcpy(freq->sha1, sha1); + oidcpy(&freq->oid, oid); freq->localfile = -1; - loose_object_path(the_repository, &filename, sha1); + loose_object_path(the_repository, &filename, oid->hash); strbuf_addf(&freq->tmpfile, "%s.temp", filename.buf); strbuf_addf(&prevfile, "%s.prev", filename.buf); @@ -2495,16 +2495,16 @@ int finish_http_object_request(struct http_object_request *freq) } git_inflate_end(&freq->stream); - git_SHA1_Final(freq->real_sha1, &freq->c); + git_SHA1_Final(freq->real_oid.hash, &freq->c); if (freq->zret != Z_STREAM_END) { unlink_or_warn(freq->tmpfile.buf); return -1; } - if (!hasheq(freq->sha1, freq->real_sha1)) { + if (!oideq(&freq->oid, &freq->real_oid)) { unlink_or_warn(freq->tmpfile.buf); return -1; } - loose_object_path(the_repository, &filename, freq->sha1); + loose_object_path(the_repository, &filename, freq->oid.hash); freq->rename = finalize_object_file(freq->tmpfile.buf, filename.buf); strbuf_release(&filename); diff --git a/http.h b/http.h index d305ca1dc7..66c52b2e1e 100644 --- a/http.h +++ b/http.h @@ -224,8 +224,8 @@ struct http_object_request { CURLcode curl_result; char errorstr[CURL_ERROR_SIZE]; long http_code; - unsigned char sha1[20]; - unsigned char real_sha1[20]; + struct object_id oid; + struct object_id real_oid; git_SHA_CTX c; git_zstream stream; int zret; @@ -234,7 +234,7 @@ struct http_object_request { }; extern struct http_object_request *new_http_object_request( - const char *base_url, unsigned char *sha1); + const char *base_url, const struct object_id *oid); extern void process_http_object_request(struct http_object_request *freq); extern int finish_http_object_request(struct http_object_request *freq); extern void abort_http_object_request(struct http_object_request *freq); From patchwork Mon Jan 7 08:35:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 10750125 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5280C14DE for ; Mon, 7 Jan 2019 08:35:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3D8BD28A1E for ; Mon, 7 Jan 2019 08:35:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 31A1C28A24; Mon, 7 Jan 2019 08:35:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4247428A1E for ; Mon, 7 Jan 2019 08:35:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726511AbfAGIfq (ORCPT ); Mon, 7 Jan 2019 03:35:46 -0500 Received: from cloud.peff.net ([104.130.231.41]:56344 "HELO cloud.peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1725550AbfAGIfq (ORCPT ); Mon, 7 Jan 2019 03:35:46 -0500 Received: (qmail 1327 invoked by uid 109); 7 Jan 2019 08:35:45 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with SMTP; Mon, 07 Jan 2019 08:35:45 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 742 invoked by uid 111); 7 Jan 2019 08:35:23 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) by peff.net (qpsmtpd/0.94) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) SMTP; Mon, 07 Jan 2019 03:35:23 -0500 Authentication-Results: peff.net; auth=none Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Mon, 07 Jan 2019 03:35:43 -0500 Date: Mon, 7 Jan 2019 03:35:42 -0500 From: Jeff King To: =?utf-8?b?UmVuw6k=?= Scharfe Cc: git@vger.kernel.org, Junio C Hamano , =?utf-8?b?w4Z2?= =?utf-8?b?YXIgQXJuZmrDtnLDsA==?= Bjarmason Subject: [PATCH 04/11] sha1-file: modernize loose object file functions Message-ID: <20190107083542.GD29431@sigill.intra.peff.net> References: <20190107083150.GC21362@sigill.intra.peff.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190107083150.GC21362@sigill.intra.peff.net> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The loose object access code in sha1-file.c is some of the oldest in Git, and could use some modernizing. It mostly uses "unsigned char *" for object ids, which these days should be "struct object_id". It also uses the term "sha1_file" in many functions, which is confusing. The term "loose_objects" is much better. It clearly distinguishes them from packed objects (which didn't even exist back when the name "sha1_file" came into being). And it also distinguishes it from the checksummed-file concept in csum-file.c (which until recently was actually called "struct sha1file"!). This patch converts the functions {open,close,map,stat}_sha1_file() into open_loose_object(), etc, and switches their sha1 arguments for object_id structs. Similarly, path functions like fill_sha1_path() become fill_loose_path() and use object_ids. The function sha1_loose_object_info() already says "loose", so we can just drop the "sha1" (and teach it to use object_id). Signed-off-by: Jeff King --- http-walker.c | 2 +- http.c | 4 +-- object-store.h | 8 +++-- sha1-file.c | 81 +++++++++++++++++++++++++------------------------- streaming.c | 4 +-- 5 files changed, 51 insertions(+), 48 deletions(-) diff --git a/http-walker.c b/http-walker.c index 856716c63d..29b59e2fe0 100644 --- a/http-walker.c +++ b/http-walker.c @@ -547,7 +547,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1) ret = error("File %s has bad hash", hex); } else if (req->rename < 0) { struct strbuf buf = STRBUF_INIT; - loose_object_path(the_repository, &buf, req->oid.hash); + loose_object_path(the_repository, &buf, &req->oid); ret = error("unable to write sha1 filename %s", buf.buf); strbuf_release(&buf); } diff --git a/http.c b/http.c index 8d42154792..43d06dd074 100644 --- a/http.c +++ b/http.c @@ -2353,7 +2353,7 @@ struct http_object_request *new_http_object_request(const char *base_url, oidcpy(&freq->oid, oid); freq->localfile = -1; - loose_object_path(the_repository, &filename, oid->hash); + loose_object_path(the_repository, &filename, oid); strbuf_addf(&freq->tmpfile, "%s.temp", filename.buf); strbuf_addf(&prevfile, "%s.prev", filename.buf); @@ -2504,7 +2504,7 @@ int finish_http_object_request(struct http_object_request *freq) unlink_or_warn(freq->tmpfile.buf); return -1; } - loose_object_path(the_repository, &filename, freq->oid.hash); + loose_object_path(the_repository, &filename, &freq->oid); freq->rename = finalize_object_file(freq->tmpfile.buf, filename.buf); strbuf_release(&filename); diff --git a/object-store.h b/object-store.h index 2fb6c0e4db..6b1c408753 100644 --- a/object-store.h +++ b/object-store.h @@ -161,11 +161,13 @@ void raw_object_store_clear(struct raw_object_store *o); /* * Put in `buf` the name of the file in the local object database that - * would be used to store a loose object with the specified sha1. + * would be used to store a loose object with the specified oid. */ -const char *loose_object_path(struct repository *r, struct strbuf *buf, const unsigned char *sha1); +const char *loose_object_path(struct repository *r, struct strbuf *buf, + const struct object_id *oid); -void *map_sha1_file(struct repository *r, const unsigned char *sha1, unsigned long *size); +void *map_loose_object(struct repository *r, const struct object_id *oid, + unsigned long *size); extern void *read_object_file_extended(const struct object_id *oid, enum object_type *type, diff --git a/sha1-file.c b/sha1-file.c index e86bb28320..cd8e5f005a 100644 --- a/sha1-file.c +++ b/sha1-file.c @@ -333,12 +333,12 @@ int raceproof_create_file(const char *path, create_file_fn fn, void *cb) return ret; } -static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1) +static void fill_loose_path(struct strbuf *buf, const struct object_id *oid) { int i; for (i = 0; i < the_hash_algo->rawsz; i++) { static char hex[] = "0123456789abcdef"; - unsigned int val = sha1[i]; + unsigned int val = oid->hash[i]; strbuf_addch(buf, hex[val >> 4]); strbuf_addch(buf, hex[val & 0xf]); if (!i) @@ -348,19 +348,19 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1) static const char *odb_loose_path(struct object_directory *odb, struct strbuf *buf, - const unsigned char *sha1) + const struct object_id *oid) { strbuf_reset(buf); strbuf_addstr(buf, odb->path); strbuf_addch(buf, '/'); - fill_sha1_path(buf, sha1); + fill_loose_path(buf, oid); return buf->buf; } const char *loose_object_path(struct repository *r, struct strbuf *buf, - const unsigned char *sha1) + const struct object_id *oid) { - return odb_loose_path(r->objects->odb, buf, sha1); + return odb_loose_path(r->objects->odb, buf, oid); } /* @@ -721,7 +721,7 @@ static int check_and_freshen_odb(struct object_directory *odb, int freshen) { static struct strbuf path = STRBUF_INIT; - odb_loose_path(odb, &path, oid->hash); + odb_loose_path(odb, &path, oid); return check_and_freshen_file(path.buf, freshen); } @@ -872,22 +872,22 @@ int git_open_cloexec(const char *name, int flags) } /* - * Find "sha1" as a loose object in the local repository or in an alternate. + * Find "oid" as a loose object in the local repository or in an alternate. * Returns 0 on success, negative on failure. * * The "path" out-parameter will give the path of the object we found (if any). * Note that it may point to static storage and is only valid until another - * call to stat_sha1_file(). + * call to stat_loose_object(). */ -static int stat_sha1_file(struct repository *r, const unsigned char *sha1, - struct stat *st, const char **path) +static int stat_loose_object(struct repository *r, const struct object_id *oid, + struct stat *st, const char **path) { struct object_directory *odb; static struct strbuf buf = STRBUF_INIT; prepare_alt_odb(r); for (odb = r->objects->odb; odb; odb = odb->next) { - *path = odb_loose_path(odb, &buf, sha1); + *path = odb_loose_path(odb, &buf, oid); if (!lstat(*path, st)) return 0; } @@ -896,11 +896,11 @@ static int stat_sha1_file(struct repository *r, const unsigned char *sha1, } /* - * Like stat_sha1_file(), but actually open the object and return the + * Like stat_loose_object(), but actually open the object and return the * descriptor. See the caveats on the "path" parameter above. */ -static int open_sha1_file(struct repository *r, - const unsigned char *sha1, const char **path) +static int open_loose_object(struct repository *r, + const struct object_id *oid, const char **path) { int fd; struct object_directory *odb; @@ -909,7 +909,7 @@ static int open_sha1_file(struct repository *r, prepare_alt_odb(r); for (odb = r->objects->odb; odb; odb = odb->next) { - *path = odb_loose_path(odb, &buf, sha1); + *path = odb_loose_path(odb, &buf, oid); fd = git_open(*path); if (fd >= 0) return fd; @@ -939,10 +939,10 @@ static int quick_has_loose(struct repository *r, /* * Map the loose object at "path" if it is not NULL, or the path found by - * searching for a loose object named "sha1". + * searching for a loose object named "oid". */ -static void *map_sha1_file_1(struct repository *r, const char *path, - const unsigned char *sha1, unsigned long *size) +static void *map_loose_object_1(struct repository *r, const char *path, + const struct object_id *oid, unsigned long *size) { void *map; int fd; @@ -950,7 +950,7 @@ static void *map_sha1_file_1(struct repository *r, const char *path, if (path) fd = git_open(path); else - fd = open_sha1_file(r, sha1, &path); + fd = open_loose_object(r, oid, &path); map = NULL; if (fd >= 0) { struct stat st; @@ -969,10 +969,11 @@ static void *map_sha1_file_1(struct repository *r, const char *path, return map; } -void *map_sha1_file(struct repository *r, - const unsigned char *sha1, unsigned long *size) +void *map_loose_object(struct repository *r, + const struct object_id *oid, + unsigned long *size) { - return map_sha1_file_1(r, NULL, sha1, size); + return map_loose_object_1(r, NULL, oid, size); } static int unpack_sha1_short_header(git_zstream *stream, @@ -1161,9 +1162,9 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep) return parse_sha1_header_extended(hdr, &oi, 0); } -static int sha1_loose_object_info(struct repository *r, - const unsigned char *sha1, - struct object_info *oi, int flags) +static int loose_object_info(struct repository *r, + const struct object_id *oid, + struct object_info *oi, int flags) { int status = 0; unsigned long mapsize; @@ -1188,15 +1189,15 @@ static int sha1_loose_object_info(struct repository *r, const char *path; struct stat st; if (!oi->disk_sizep && (flags & OBJECT_INFO_QUICK)) - return quick_has_loose(r, sha1) ? 0 : -1; - if (stat_sha1_file(r, sha1, &st, &path) < 0) + return quick_has_loose(r, oid->hash) ? 0 : -1; + if (stat_loose_object(r, oid, &st, &path) < 0) return -1; if (oi->disk_sizep) *oi->disk_sizep = st.st_size; return 0; } - map = map_sha1_file(r, sha1, &mapsize); + map = map_loose_object(r, oid, &mapsize); if (!map) return -1; @@ -1208,22 +1209,22 @@ static int sha1_loose_object_info(struct repository *r, if ((flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE)) { if (unpack_sha1_header_to_strbuf(&stream, map, mapsize, hdr, sizeof(hdr), &hdrbuf) < 0) status = error(_("unable to unpack %s header with --allow-unknown-type"), - sha1_to_hex(sha1)); + oid_to_hex(oid)); } else if (unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) status = error(_("unable to unpack %s header"), - sha1_to_hex(sha1)); + oid_to_hex(oid)); if (status < 0) ; /* Do nothing */ else if (hdrbuf.len) { if ((status = parse_sha1_header_extended(hdrbuf.buf, oi, flags)) < 0) status = error(_("unable to parse %s header with --allow-unknown-type"), - sha1_to_hex(sha1)); + oid_to_hex(oid)); } else if ((status = parse_sha1_header_extended(hdr, oi, flags)) < 0) - status = error(_("unable to parse %s header"), sha1_to_hex(sha1)); + status = error(_("unable to parse %s header"), oid_to_hex(oid)); if (status >= 0 && oi->contentp) { *oi->contentp = unpack_sha1_rest(&stream, hdr, - *oi->sizep, sha1); + *oi->sizep, oid->hash); if (!*oi->contentp) { git_inflate_end(&stream); status = -1; @@ -1289,7 +1290,7 @@ int oid_object_info_extended(struct repository *r, const struct object_id *oid, return -1; /* Most likely it's a loose object. */ - if (!sha1_loose_object_info(r, real->hash, oi, flags)) + if (!loose_object_info(r, real, oi, flags)) return 0; /* Not a loose object; someone else may have just packed it. */ @@ -1417,7 +1418,7 @@ void *read_object_file_extended(const struct object_id *oid, die(_("replacement %s not found for %s"), oid_to_hex(repl), oid_to_hex(oid)); - if (!stat_sha1_file(the_repository, repl->hash, &st, &path)) + if (!stat_loose_object(the_repository, repl, &st, &path)) die(_("loose object %s (stored in %s) is corrupt"), oid_to_hex(repl), path); @@ -1552,7 +1553,7 @@ int hash_object_file(const void *buf, unsigned long len, const char *type, } /* Finalize a file on disk, and close it. */ -static void close_sha1_file(int fd) +static void close_loose_object(int fd) { if (fsync_object_files) fsync_or_die(fd, "sha1 file"); @@ -1617,7 +1618,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr, static struct strbuf tmp_file = STRBUF_INIT; static struct strbuf filename = STRBUF_INIT; - loose_object_path(the_repository, &filename, oid->hash); + loose_object_path(the_repository, &filename, oid); fd = create_tmpfile(&tmp_file, filename.buf); if (fd < 0) { @@ -1665,7 +1666,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr, die(_("confused by unstable object source data for %s"), oid_to_hex(oid)); - close_sha1_file(fd); + close_loose_object(fd); if (mtime) { struct utimbuf utb; @@ -2260,7 +2261,7 @@ int read_loose_object(const char *path, *contents = NULL; - map = map_sha1_file_1(the_repository, path, NULL, &mapsize); + map = map_loose_object_1(the_repository, path, NULL, &mapsize); if (!map) { error_errno(_("unable to mmap %s"), path); goto out; diff --git a/streaming.c b/streaming.c index ac7c7a22f9..9049146bc1 100644 --- a/streaming.c +++ b/streaming.c @@ -338,8 +338,8 @@ static struct stream_vtbl loose_vtbl = { static open_method_decl(loose) { - st->u.loose.mapped = map_sha1_file(the_repository, - oid->hash, &st->u.loose.mapsize); + st->u.loose.mapped = map_loose_object(the_repository, + oid, &st->u.loose.mapsize); if (!st->u.loose.mapped) return -1; if ((unpack_sha1_header(&st->z, From patchwork Mon Jan 7 08:37:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 10750127 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9817D746 for ; Mon, 7 Jan 2019 08:37:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 848F028970 for ; Mon, 7 Jan 2019 08:37:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 785E728A20; Mon, 7 Jan 2019 08:37:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B204428970 for ; Mon, 7 Jan 2019 08:37:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726448AbfAGIhG (ORCPT ); Mon, 7 Jan 2019 03:37:06 -0500 Received: from cloud.peff.net ([104.130.231.41]:56354 "HELO cloud.peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1726257AbfAGIhG (ORCPT ); Mon, 7 Jan 2019 03:37:06 -0500 Received: (qmail 1455 invoked by uid 109); 7 Jan 2019 08:37:04 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with SMTP; Mon, 07 Jan 2019 08:37:04 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 761 invoked by uid 111); 7 Jan 2019 08:36:42 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) by peff.net (qpsmtpd/0.94) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) SMTP; Mon, 07 Jan 2019 03:36:42 -0500 Authentication-Results: peff.net; auth=none Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Mon, 07 Jan 2019 03:37:02 -0500 Date: Mon, 7 Jan 2019 03:37:02 -0500 From: Jeff King To: =?utf-8?b?UmVuw6k=?= Scharfe Cc: git@vger.kernel.org, Junio C Hamano , =?utf-8?b?w4Z2?= =?utf-8?b?YXIgQXJuZmrDtnLDsA==?= Bjarmason Subject: [PATCH 05/11] sha1-file: modernize loose header/stream functions Message-ID: <20190107083702.GE29431@sigill.intra.peff.net> References: <20190107083150.GC21362@sigill.intra.peff.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190107083150.GC21362@sigill.intra.peff.net> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP As with the open/map/close functions for loose objects that were recently converted, the functions for parsing the loose object stream use the name "sha1" and a bare "unsigned char *". Let's fix that so that unpack_sha1_header() becomes unpack_loose_header(), etc. These conversions are less clear-cut than the file access functions. You could argue that the they are parsing Git's canonical object format (i.e., "type size\0contents", over which we compute the hash), which is not strictly tied to loose storage. But in practice these functions are used only for loose objects, and using the term "loose_header" (instead of "object_header") distinguishes it from the object header found in packfiles (which contains the same information in a different format). Signed-off-by: Jeff King --- Of course "loose_object_header" would be even more exact, but is quite long. ;) cache.h | 4 +-- sha1-file.c | 84 +++++++++++++++++++++++++++-------------------------- streaming.c | 12 ++++---- 3 files changed, 51 insertions(+), 49 deletions(-) diff --git a/cache.h b/cache.h index 587512747b..653c36d0b7 100644 --- a/cache.h +++ b/cache.h @@ -1269,8 +1269,8 @@ extern char *xdg_cache_home(const char *filename); extern int git_open_cloexec(const char *name, int flags); #define git_open(name) git_open_cloexec(name, O_RDONLY) -extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz); -extern int parse_sha1_header(const char *hdr, unsigned long *sizep); +extern int unpack_loose_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz); +extern int parse_loose_header(const char *hdr, unsigned long *sizep); extern int check_object_signature(const struct object_id *oid, void *buf, unsigned long size, const char *type); diff --git a/sha1-file.c b/sha1-file.c index cd8e5f005a..4938258ed1 100644 --- a/sha1-file.c +++ b/sha1-file.c @@ -976,9 +976,9 @@ void *map_loose_object(struct repository *r, return map_loose_object_1(r, NULL, oid, size); } -static int unpack_sha1_short_header(git_zstream *stream, - unsigned char *map, unsigned long mapsize, - void *buffer, unsigned long bufsiz) +static int unpack_loose_short_header(git_zstream *stream, + unsigned char *map, unsigned long mapsize, + void *buffer, unsigned long bufsiz) { /* Get the data stream */ memset(stream, 0, sizeof(*stream)); @@ -991,12 +991,12 @@ static int unpack_sha1_short_header(git_zstream *stream, return git_inflate(stream, 0); } -int unpack_sha1_header(git_zstream *stream, - unsigned char *map, unsigned long mapsize, - void *buffer, unsigned long bufsiz) +int unpack_loose_header(git_zstream *stream, + unsigned char *map, unsigned long mapsize, + void *buffer, unsigned long bufsiz) { - int status = unpack_sha1_short_header(stream, map, mapsize, - buffer, bufsiz); + int status = unpack_loose_short_header(stream, map, mapsize, + buffer, bufsiz); if (status < Z_OK) return status; @@ -1007,13 +1007,13 @@ int unpack_sha1_header(git_zstream *stream, return 0; } -static int unpack_sha1_header_to_strbuf(git_zstream *stream, unsigned char *map, - unsigned long mapsize, void *buffer, - unsigned long bufsiz, struct strbuf *header) +static int unpack_loose_header_to_strbuf(git_zstream *stream, unsigned char *map, + unsigned long mapsize, void *buffer, + unsigned long bufsiz, struct strbuf *header) { int status; - status = unpack_sha1_short_header(stream, map, mapsize, buffer, bufsiz); + status = unpack_loose_short_header(stream, map, mapsize, buffer, bufsiz); if (status < Z_OK) return -1; @@ -1043,7 +1043,9 @@ static int unpack_sha1_header_to_strbuf(git_zstream *stream, unsigned char *map, return -1; } -static void *unpack_sha1_rest(git_zstream *stream, void *buffer, unsigned long size, const unsigned char *sha1) +static void *unpack_loose_rest(git_zstream *stream, + void *buffer, unsigned long size, + const struct object_id *oid) { int bytes = strlen(buffer) + 1; unsigned char *buf = xmallocz(size); @@ -1080,10 +1082,10 @@ static void *unpack_sha1_rest(git_zstream *stream, void *buffer, unsigned long s } if (status < 0) - error(_("corrupt loose object '%s'"), sha1_to_hex(sha1)); + error(_("corrupt loose object '%s'"), oid_to_hex(oid)); else if (stream->avail_in) error(_("garbage at end of loose object '%s'"), - sha1_to_hex(sha1)); + oid_to_hex(oid)); free(buf); return NULL; } @@ -1093,8 +1095,8 @@ static void *unpack_sha1_rest(git_zstream *stream, void *buffer, unsigned long s * too permissive for what we want to check. So do an anal * object header parse by hand. */ -static int parse_sha1_header_extended(const char *hdr, struct object_info *oi, - unsigned int flags) +static int parse_loose_header_extended(const char *hdr, struct object_info *oi, + unsigned int flags) { const char *type_buf = hdr; unsigned long size; @@ -1154,12 +1156,12 @@ static int parse_sha1_header_extended(const char *hdr, struct object_info *oi, return *hdr ? -1 : type; } -int parse_sha1_header(const char *hdr, unsigned long *sizep) +int parse_loose_header(const char *hdr, unsigned long *sizep) { struct object_info oi = OBJECT_INFO_INIT; oi.sizep = sizep; - return parse_sha1_header_extended(hdr, &oi, 0); + return parse_loose_header_extended(hdr, &oi, 0); } static int loose_object_info(struct repository *r, @@ -1207,24 +1209,24 @@ static int loose_object_info(struct repository *r, if (oi->disk_sizep) *oi->disk_sizep = mapsize; if ((flags & OBJECT_INFO_ALLOW_UNKNOWN_TYPE)) { - if (unpack_sha1_header_to_strbuf(&stream, map, mapsize, hdr, sizeof(hdr), &hdrbuf) < 0) + if (unpack_loose_header_to_strbuf(&stream, map, mapsize, hdr, sizeof(hdr), &hdrbuf) < 0) status = error(_("unable to unpack %s header with --allow-unknown-type"), oid_to_hex(oid)); - } else if (unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) + } else if (unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) status = error(_("unable to unpack %s header"), oid_to_hex(oid)); if (status < 0) ; /* Do nothing */ else if (hdrbuf.len) { - if ((status = parse_sha1_header_extended(hdrbuf.buf, oi, flags)) < 0) + if ((status = parse_loose_header_extended(hdrbuf.buf, oi, flags)) < 0) status = error(_("unable to parse %s header with --allow-unknown-type"), oid_to_hex(oid)); - } else if ((status = parse_sha1_header_extended(hdr, oi, flags)) < 0) + } else if ((status = parse_loose_header_extended(hdr, oi, flags)) < 0) status = error(_("unable to parse %s header"), oid_to_hex(oid)); if (status >= 0 && oi->contentp) { - *oi->contentp = unpack_sha1_rest(&stream, hdr, - *oi->sizep, oid->hash); + *oi->contentp = unpack_loose_rest(&stream, hdr, + *oi->sizep, oid); if (!*oi->contentp) { git_inflate_end(&stream); status = -1; @@ -2189,14 +2191,14 @@ void odb_clear_loose_cache(struct object_directory *odb) sizeof(odb->loose_objects_subdir_seen)); } -static int check_stream_sha1(git_zstream *stream, - const char *hdr, - unsigned long size, - const char *path, - const unsigned char *expected_sha1) +static int check_stream_oid(git_zstream *stream, + const char *hdr, + unsigned long size, + const char *path, + const struct object_id *expected_oid) { git_hash_ctx c; - unsigned char real_sha1[GIT_MAX_RAWSZ]; + struct object_id real_oid; unsigned char buf[4096]; unsigned long total_read; int status = Z_OK; @@ -2212,7 +2214,7 @@ static int check_stream_sha1(git_zstream *stream, /* * This size comparison must be "<=" to read the final zlib packets; - * see the comment in unpack_sha1_rest for details. + * see the comment in unpack_loose_rest for details. */ while (total_read <= size && (status == Z_OK || @@ -2228,19 +2230,19 @@ static int check_stream_sha1(git_zstream *stream, git_inflate_end(stream); if (status != Z_STREAM_END) { - error(_("corrupt loose object '%s'"), sha1_to_hex(expected_sha1)); + error(_("corrupt loose object '%s'"), oid_to_hex(expected_oid)); return -1; } if (stream->avail_in) { error(_("garbage at end of loose object '%s'"), - sha1_to_hex(expected_sha1)); + oid_to_hex(expected_oid)); return -1; } - the_hash_algo->final_fn(real_sha1, &c); - if (!hasheq(expected_sha1, real_sha1)) { + the_hash_algo->final_fn(real_oid.hash, &c); + if (!oideq(expected_oid, &real_oid)) { error(_("sha1 mismatch for %s (expected %s)"), path, - sha1_to_hex(expected_sha1)); + oid_to_hex(expected_oid)); return -1; } @@ -2267,12 +2269,12 @@ int read_loose_object(const char *path, goto out; } - if (unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) { + if (unpack_loose_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) { error(_("unable to unpack header of %s"), path); goto out; } - *type = parse_sha1_header(hdr, size); + *type = parse_loose_header(hdr, size); if (*type < 0) { error(_("unable to parse header of %s"), path); git_inflate_end(&stream); @@ -2280,10 +2282,10 @@ int read_loose_object(const char *path, } if (*type == OBJ_BLOB && *size > big_file_threshold) { - if (check_stream_sha1(&stream, hdr, *size, path, expected_oid->hash) < 0) + if (check_stream_oid(&stream, hdr, *size, path, expected_oid) < 0) goto out; } else { - *contents = unpack_sha1_rest(&stream, hdr, *size, expected_oid->hash); + *contents = unpack_loose_rest(&stream, hdr, *size, expected_oid); if (!*contents) { error(_("unable to unpack contents of %s"), path); git_inflate_end(&stream); diff --git a/streaming.c b/streaming.c index 9049146bc1..998e6285d7 100644 --- a/streaming.c +++ b/streaming.c @@ -342,12 +342,12 @@ static open_method_decl(loose) oid, &st->u.loose.mapsize); if (!st->u.loose.mapped) return -1; - if ((unpack_sha1_header(&st->z, - st->u.loose.mapped, - st->u.loose.mapsize, - st->u.loose.hdr, - sizeof(st->u.loose.hdr)) < 0) || - (parse_sha1_header(st->u.loose.hdr, &st->size) < 0)) { + if ((unpack_loose_header(&st->z, + st->u.loose.mapped, + st->u.loose.mapsize, + st->u.loose.hdr, + sizeof(st->u.loose.hdr)) < 0) || + (parse_loose_header(st->u.loose.hdr, &st->size) < 0)) { git_inflate_end(&st->z); munmap(st->u.loose.mapped, st->u.loose.mapsize); return -1; From patchwork Mon Jan 7 08:37:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 10750129 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7EB9114DE for ; Mon, 7 Jan 2019 08:37:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6C78628970 for ; Mon, 7 Jan 2019 08:37:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5E39C28A20; Mon, 7 Jan 2019 08:37:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F312328970 for ; Mon, 7 Jan 2019 08:37:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726534AbfAGIhc (ORCPT ); Mon, 7 Jan 2019 03:37:32 -0500 Received: from cloud.peff.net ([104.130.231.41]:56366 "HELO cloud.peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1725550AbfAGIhb (ORCPT ); Mon, 7 Jan 2019 03:37:31 -0500 Received: (qmail 1524 invoked by uid 109); 7 Jan 2019 08:37:32 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with SMTP; Mon, 07 Jan 2019 08:37:32 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 780 invoked by uid 111); 7 Jan 2019 08:37:10 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) by peff.net (qpsmtpd/0.94) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) SMTP; Mon, 07 Jan 2019 03:37:10 -0500 Authentication-Results: peff.net; auth=none Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Mon, 07 Jan 2019 03:37:29 -0500 Date: Mon, 7 Jan 2019 03:37:29 -0500 From: Jeff King To: =?utf-8?b?UmVuw6k=?= Scharfe Cc: git@vger.kernel.org, Junio C Hamano , =?utf-8?b?w4Z2?= =?utf-8?b?YXIgQXJuZmrDtnLDsA==?= Bjarmason Subject: [PATCH 06/11] sha1-file: convert pass-through functions to object_id Message-ID: <20190107083729.GF29431@sigill.intra.peff.net> References: <20190107083150.GC21362@sigill.intra.peff.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190107083150.GC21362@sigill.intra.peff.net> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP These two static functions, read_object() and quick_has_loose(), both have to hashcpy() their bare-sha1 arguments into object_id structs to pass them along. Since all of their callers actually have object_id structs in the first place, we can eliminate the copying by adjusting their input parameters. Signed-off-by: Jeff King --- sha1-file.c | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-) diff --git a/sha1-file.c b/sha1-file.c index 4938258ed1..589a666686 100644 --- a/sha1-file.c +++ b/sha1-file.c @@ -922,16 +922,13 @@ static int open_loose_object(struct repository *r, } static int quick_has_loose(struct repository *r, - const unsigned char *sha1) + const struct object_id *oid) { - struct object_id oid; struct object_directory *odb; - hashcpy(oid.hash, sha1); - prepare_alt_odb(r); for (odb = r->objects->odb; odb; odb = odb->next) { - if (oid_array_lookup(odb_loose_cache(odb, &oid), &oid) >= 0) + if (oid_array_lookup(odb_loose_cache(odb, oid), oid) >= 0) return 1; } return 0; @@ -1191,7 +1188,7 @@ static int loose_object_info(struct repository *r, const char *path; struct stat st; if (!oi->disk_sizep && (flags & OBJECT_INFO_QUICK)) - return quick_has_loose(r, oid->hash) ? 0 : -1; + return quick_has_loose(r, oid) ? 0 : -1; if (stat_loose_object(r, oid, &st, &path) < 0) return -1; if (oi->disk_sizep) @@ -1355,19 +1352,16 @@ int oid_object_info(struct repository *r, return type; } -static void *read_object(const unsigned char *sha1, enum object_type *type, +static void *read_object(const struct object_id *oid, enum object_type *type, unsigned long *size) { - struct object_id oid; struct object_info oi = OBJECT_INFO_INIT; void *content; oi.typep = type; oi.sizep = size; oi.contentp = &content; - hashcpy(oid.hash, sha1); - - if (oid_object_info_extended(the_repository, &oid, &oi, 0) < 0) + if (oid_object_info_extended(the_repository, oid, &oi, 0) < 0) return NULL; return content; } @@ -1408,7 +1402,7 @@ void *read_object_file_extended(const struct object_id *oid, lookup_replace_object(the_repository, oid) : oid; errno = 0; - data = read_object(repl->hash, type, size); + data = read_object(repl, type, size); if (data) return data; @@ -1748,7 +1742,7 @@ int force_object_loose(const struct object_id *oid, time_t mtime) if (has_loose_object(oid)) return 0; - buf = read_object(oid->hash, &type, &len); + buf = read_object(oid, &type, &len); if (!buf) return error(_("cannot read sha1_file for %s"), oid_to_hex(oid)); hdrlen = xsnprintf(hdr, sizeof(hdr), "%s %"PRIuMAX , type_name(type), (uintmax_t)len) + 1; From patchwork Mon Jan 7 08:37:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 10750131 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4895214DE for ; Mon, 7 Jan 2019 08:38:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 319C628970 for ; Mon, 7 Jan 2019 08:38:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 23B2C28A24; Mon, 7 Jan 2019 08:38:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 76BFC28970 for ; Mon, 7 Jan 2019 08:37:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726540AbfAGIh6 (ORCPT ); Mon, 7 Jan 2019 03:37:58 -0500 Received: from cloud.peff.net ([104.130.231.41]:56382 "HELO cloud.peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1725550AbfAGIh6 (ORCPT ); Mon, 7 Jan 2019 03:37:58 -0500 Received: (qmail 1564 invoked by uid 109); 7 Jan 2019 08:37:56 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with SMTP; Mon, 07 Jan 2019 08:37:56 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 800 invoked by uid 111); 7 Jan 2019 08:37:34 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) by peff.net (qpsmtpd/0.94) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) SMTP; Mon, 07 Jan 2019 03:37:34 -0500 Authentication-Results: peff.net; auth=none Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Mon, 07 Jan 2019 03:37:54 -0500 Date: Mon, 7 Jan 2019 03:37:54 -0500 From: Jeff King To: =?utf-8?b?UmVuw6k=?= Scharfe Cc: git@vger.kernel.org, Junio C Hamano , =?utf-8?b?w4Z2?= =?utf-8?b?YXIgQXJuZmrDtnLDsA==?= Bjarmason Subject: [PATCH 07/11] convert has_sha1_file() callers to has_object_file() Message-ID: <20190107083754.GG29431@sigill.intra.peff.net> References: <20190107083150.GC21362@sigill.intra.peff.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190107083150.GC21362@sigill.intra.peff.net> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The only remaining callers of has_sha1_file() actually have an object_id already. They can use the "object" variant, rather than dereferencing the hash themselves. The code changes here were completely generated by the included coccinelle patch. Signed-off-by: Jeff King --- apply.c | 2 +- builtin/fetch.c | 7 +++---- builtin/index-pack.c | 2 +- builtin/reflog.c | 2 +- builtin/show-ref.c | 2 +- bulk-checkin.c | 2 +- cache-tree.c | 4 ++-- contrib/coccinelle/object_id.cocci | 32 ++++++++++++++++++++++++++++++ http-walker.c | 4 ++-- refs.c | 2 +- send-pack.c | 2 +- sha1-file.c | 2 +- 12 files changed, 47 insertions(+), 16 deletions(-) diff --git a/apply.c b/apply.c index 01793d6126..b8e257ead2 100644 --- a/apply.c +++ b/apply.c @@ -3183,7 +3183,7 @@ static int apply_binary(struct apply_state *state, return 0; /* deletion patch */ } - if (has_sha1_file(oid.hash)) { + if (has_object_file(&oid)) { /* We already have the postimage */ enum object_type type; unsigned long size; diff --git a/builtin/fetch.c b/builtin/fetch.c index e0140327aa..57f35c6a0a 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -317,8 +317,7 @@ static void find_non_local_tags(const struct ref *refs, !has_object_file_with_flags(&ref->old_oid, OBJECT_INFO_QUICK) && !will_fetch(head, ref->old_oid.hash) && - !has_sha1_file_with_flags(item->oid.hash, - OBJECT_INFO_QUICK) && + !has_object_file_with_flags(&item->oid, OBJECT_INFO_QUICK) && !will_fetch(head, item->oid.hash)) oidclr(&item->oid); item = NULL; @@ -332,7 +331,7 @@ static void find_non_local_tags(const struct ref *refs, * fetch. */ if (item && - !has_sha1_file_with_flags(item->oid.hash, OBJECT_INFO_QUICK) && + !has_object_file_with_flags(&item->oid, OBJECT_INFO_QUICK) && !will_fetch(head, item->oid.hash)) oidclr(&item->oid); @@ -353,7 +352,7 @@ static void find_non_local_tags(const struct ref *refs, * checked to see if it needs fetching. */ if (item && - !has_sha1_file_with_flags(item->oid.hash, OBJECT_INFO_QUICK) && + !has_object_file_with_flags(&item->oid, OBJECT_INFO_QUICK) && !will_fetch(head, item->oid.hash)) oidclr(&item->oid); diff --git a/builtin/index-pack.c b/builtin/index-pack.c index ac1f4ea9a7..31046c7a0a 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -772,7 +772,7 @@ static void sha1_object(const void *data, struct object_entry *obj_entry, if (startup_info->have_repository) { read_lock(); collision_test_needed = - has_sha1_file_with_flags(oid->hash, OBJECT_INFO_QUICK); + has_object_file_with_flags(oid, OBJECT_INFO_QUICK); read_unlock(); } diff --git a/builtin/reflog.c b/builtin/reflog.c index 64a8df4f25..45e9e15006 100644 --- a/builtin/reflog.c +++ b/builtin/reflog.c @@ -94,7 +94,7 @@ static int tree_is_complete(const struct object_id *oid) init_tree_desc(&desc, tree->buffer, tree->size); complete = 1; while (tree_entry(&desc, &entry)) { - if (!has_sha1_file(entry.oid->hash) || + if (!has_object_file(entry.oid) || (S_ISDIR(entry.mode) && !tree_is_complete(entry.oid))) { tree->object.flags |= INCOMPLETE; complete = 0; diff --git a/builtin/show-ref.c b/builtin/show-ref.c index ed888ffa48..6a706c02a6 100644 --- a/builtin/show-ref.c +++ b/builtin/show-ref.c @@ -23,7 +23,7 @@ static void show_one(const char *refname, const struct object_id *oid) const char *hex; struct object_id peeled; - if (!has_sha1_file(oid->hash)) + if (!has_object_file(oid)) die("git show-ref: bad ref %s (%s)", refname, oid_to_hex(oid)); diff --git a/bulk-checkin.c b/bulk-checkin.c index 409ecb566b..39ee7d6107 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -67,7 +67,7 @@ static int already_written(struct bulk_checkin_state *state, struct object_id *o int i; /* The object may already exist in the repository */ - if (has_sha1_file(oid->hash)) + if (has_object_file(oid)) return 1; /* Might want to keep the list sorted */ diff --git a/cache-tree.c b/cache-tree.c index 190c6e5aa6..47f3464a1f 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -225,7 +225,7 @@ int cache_tree_fully_valid(struct cache_tree *it) int i; if (!it) return 0; - if (it->entry_count < 0 || !has_sha1_file(it->oid.hash)) + if (it->entry_count < 0 || !has_object_file(&it->oid)) return 0; for (i = 0; i < it->subtree_nr; i++) { if (!cache_tree_fully_valid(it->down[i]->cache_tree)) @@ -253,7 +253,7 @@ static int update_one(struct cache_tree *it, *skip_count = 0; - if (0 <= it->entry_count && has_sha1_file(it->oid.hash)) + if (0 <= it->entry_count && has_object_file(&it->oid)) return it->entry_count; /* diff --git a/contrib/coccinelle/object_id.cocci b/contrib/coccinelle/object_id.cocci index 6a7cf3e02d..73886ae583 100644 --- a/contrib/coccinelle/object_id.cocci +++ b/contrib/coccinelle/object_id.cocci @@ -147,3 +147,35 @@ expression E1, E2; - hashcmp(E1, E2) != 0 + !hasheq(E1, E2) ...>} + +@@ +struct object_id OID; +@@ +- has_sha1_file(OID.hash) ++ has_object_file(&OID) + +@@ +identifier f != has_object_file; +struct object_id *OIDPTR; +@@ + f(...) {<... +- has_sha1_file(OIDPTR->hash) ++ has_object_file(OIDPTR) + ...>} + +@@ +struct object_id OID; +expression E; +@@ +- has_sha1_file_with_flags(OID.hash, E) ++ has_object_file_with_flags(&OID, E) + +@@ +identifier f != has_object_file_with_flags; +struct object_id *OIDPTR; +expression E; +@@ + f(...) {<... +- has_sha1_file_with_flags(OIDPTR->hash, E) ++ has_object_file_with_flags(OIDPTR, E) + ...>} diff --git a/http-walker.c b/http-walker.c index 29b59e2fe0..8ae5d76c6a 100644 --- a/http-walker.c +++ b/http-walker.c @@ -131,7 +131,7 @@ static int fill_active_slot(struct walker *walker) list_for_each_safe(pos, tmp, head) { obj_req = list_entry(pos, struct object_request, node); if (obj_req->state == WAITING) { - if (has_sha1_file(obj_req->oid.hash)) + if (has_object_file(&obj_req->oid)) obj_req->state = COMPLETE; else { start_object_request(walker, obj_req); @@ -489,7 +489,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1) if (obj_req == NULL) return error("Couldn't find request for %s in the queue", hex); - if (has_sha1_file(obj_req->oid.hash)) { + if (has_object_file(&obj_req->oid)) { if (obj_req->req != NULL) abort_http_object_request(obj_req->req); abort_object_request(obj_req); diff --git a/refs.c b/refs.c index f9936355cd..142888a40a 100644 --- a/refs.c +++ b/refs.c @@ -188,7 +188,7 @@ int ref_resolves_to_object(const char *refname, { if (flags & REF_ISBROKEN) return 0; - if (!has_sha1_file(oid->hash)) { + if (!has_object_file(oid)) { error(_("%s does not point to a valid object!"), refname); return 0; } diff --git a/send-pack.c b/send-pack.c index f692686770..c673d3ed06 100644 --- a/send-pack.c +++ b/send-pack.c @@ -40,7 +40,7 @@ int option_parse_push_signed(const struct option *opt, static void feed_object(const struct object_id *oid, FILE *fh, int negative) { - if (negative && !has_sha1_file(oid->hash)) + if (negative && !has_object_file(oid)) return; if (negative) diff --git a/sha1-file.c b/sha1-file.c index 589a666686..449456f2ad 100644 --- a/sha1-file.c +++ b/sha1-file.c @@ -1372,7 +1372,7 @@ int pretend_object_file(void *buf, unsigned long len, enum object_type type, struct cached_object *co; hash_object_file(buf, len, type_name(type), oid); - if (has_sha1_file(oid->hash) || find_cached_object(oid)) + if (has_object_file(oid) || find_cached_object(oid)) return 0; ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc); co = &cached_objects[cached_object_nr++]; From patchwork Mon Jan 7 08:39:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 10750139 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 35DB114DE for ; Mon, 7 Jan 2019 08:39:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 23A7F283C3 for ; Mon, 7 Jan 2019 08:39:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 172BC2842B; Mon, 7 Jan 2019 08:39:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A728A283C3 for ; Mon, 7 Jan 2019 08:39:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726441AbfAGIjC (ORCPT ); Mon, 7 Jan 2019 03:39:02 -0500 Received: from cloud.peff.net ([104.130.231.41]:56390 "HELO cloud.peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1726257AbfAGIjC (ORCPT ); Mon, 7 Jan 2019 03:39:02 -0500 Received: (qmail 1711 invoked by uid 109); 7 Jan 2019 08:39:02 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with SMTP; Mon, 07 Jan 2019 08:39:02 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 819 invoked by uid 111); 7 Jan 2019 08:38:40 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) by peff.net (qpsmtpd/0.94) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) SMTP; Mon, 07 Jan 2019 03:38:40 -0500 Authentication-Results: peff.net; auth=none Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Mon, 07 Jan 2019 03:39:00 -0500 Date: Mon, 7 Jan 2019 03:39:00 -0500 From: Jeff King To: =?utf-8?b?UmVuw6k=?= Scharfe Cc: git@vger.kernel.org, Junio C Hamano , =?utf-8?b?w4Z2?= =?utf-8?b?YXIgQXJuZmrDtnLDsA==?= Bjarmason Subject: [PATCH 08/11] sha1-file: drop has_sha1_file() Message-ID: <20190107083900.GH29431@sigill.intra.peff.net> References: <20190107083150.GC21362@sigill.intra.peff.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190107083150.GC21362@sigill.intra.peff.net> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP There are no callers left of has_sha1_file() or its with_flags() variant. Let's drop them, and convert has_object_file() from a wrapper into the "real" function. Ironically, the sha1 variant was just copying into an object_id internally, so the resulting code is actually shorter! We can also drop the coccinelle rules for catching has_sha1_file() callers. Since the function no longer exists, the compiler will do that for us. Signed-off-by: Jeff King --- contrib/coccinelle/object_id.cocci | 32 ------------------------------ object-store.h | 12 ++++------- sha1-file.c | 16 ++------------- 3 files changed, 6 insertions(+), 54 deletions(-) diff --git a/contrib/coccinelle/object_id.cocci b/contrib/coccinelle/object_id.cocci index 73886ae583..6a7cf3e02d 100644 --- a/contrib/coccinelle/object_id.cocci +++ b/contrib/coccinelle/object_id.cocci @@ -147,35 +147,3 @@ expression E1, E2; - hashcmp(E1, E2) != 0 + !hasheq(E1, E2) ...>} - -@@ -struct object_id OID; -@@ -- has_sha1_file(OID.hash) -+ has_object_file(&OID) - -@@ -identifier f != has_object_file; -struct object_id *OIDPTR; -@@ - f(...) {<... -- has_sha1_file(OIDPTR->hash) -+ has_object_file(OIDPTR) - ...>} - -@@ -struct object_id OID; -expression E; -@@ -- has_sha1_file_with_flags(OID.hash, E) -+ has_object_file_with_flags(&OID, E) - -@@ -identifier f != has_object_file_with_flags; -struct object_id *OIDPTR; -expression E; -@@ - f(...) {<... -- has_sha1_file_with_flags(OIDPTR->hash, E) -+ has_object_file_with_flags(OIDPTR, E) - ...>} diff --git a/object-store.h b/object-store.h index 6b1c408753..9e8ac6a1d4 100644 --- a/object-store.h +++ b/object-store.h @@ -209,20 +209,16 @@ int read_loose_object(const char *path, void **contents); /* - * Convenience for sha1_object_info_extended() with a NULL struct + * Convenience for oid_object_info_extended() with a NULL struct * object_info. OBJECT_INFO_SKIP_CACHED is automatically set; pass * nonzero flags to also set other flags. */ -extern int has_sha1_file_with_flags(const unsigned char *sha1, int flags); -static inline int has_sha1_file(const unsigned char *sha1) +int has_object_file_with_flags(const struct object_id *oid, int flags); +static inline int has_object_file(const struct object_id *oid) { - return has_sha1_file_with_flags(sha1, 0); + return has_object_file_with_flags(oid, 0); } -/* Same as the above, except for struct object_id. */ -extern int has_object_file(const struct object_id *oid); -extern int has_object_file_with_flags(const struct object_id *oid, int flags); - /* * Return true iff an alternate object database has a loose object * with the specified name. This function does not respect replace diff --git a/sha1-file.c b/sha1-file.c index 449456f2ad..da6d78976f 100644 --- a/sha1-file.c +++ b/sha1-file.c @@ -1752,26 +1752,14 @@ int force_object_loose(const struct object_id *oid, time_t mtime) return ret; } -int has_sha1_file_with_flags(const unsigned char *sha1, int flags) +int has_object_file_with_flags(const struct object_id *oid, int flags) { - struct object_id oid; if (!startup_info->have_repository) return 0; - hashcpy(oid.hash, sha1); - return oid_object_info_extended(the_repository, &oid, NULL, + return oid_object_info_extended(the_repository, oid, NULL, flags | OBJECT_INFO_SKIP_CACHED) >= 0; } -int has_object_file(const struct object_id *oid) -{ - return has_sha1_file(oid->hash); -} - -int has_object_file_with_flags(const struct object_id *oid, int flags) -{ - return has_sha1_file_with_flags(oid->hash, flags); -} - static void check_tree(const void *buf, size_t size) { struct tree_desc desc; From patchwork Mon Jan 7 08:39:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 10750141 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 71400746 for ; Mon, 7 Jan 2019 08:39:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5FB442864E for ; Mon, 7 Jan 2019 08:39:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 540562866C; Mon, 7 Jan 2019 08:39:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0035D2864E for ; Mon, 7 Jan 2019 08:39:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726535AbfAGIj1 (ORCPT ); Mon, 7 Jan 2019 03:39:27 -0500 Received: from cloud.peff.net ([104.130.231.41]:56406 "HELO cloud.peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1726448AbfAGIj0 (ORCPT ); Mon, 7 Jan 2019 03:39:26 -0500 Received: (qmail 1774 invoked by uid 109); 7 Jan 2019 08:39:27 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with SMTP; Mon, 07 Jan 2019 08:39:27 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 838 invoked by uid 111); 7 Jan 2019 08:39:05 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) by peff.net (qpsmtpd/0.94) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) SMTP; Mon, 07 Jan 2019 03:39:05 -0500 Authentication-Results: peff.net; auth=none Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Mon, 07 Jan 2019 03:39:24 -0500 Date: Mon, 7 Jan 2019 03:39:24 -0500 From: Jeff King To: =?utf-8?b?UmVuw6k=?= Scharfe Cc: git@vger.kernel.org, Junio C Hamano , =?utf-8?b?w4Z2?= =?utf-8?b?YXIgQXJuZmrDtnLDsA==?= Bjarmason Subject: [PATCH 09/11] sha1-file: prefer "loose object file" to "sha1 file" in messages Message-ID: <20190107083924.GI29431@sigill.intra.peff.net> References: <20190107083150.GC21362@sigill.intra.peff.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190107083150.GC21362@sigill.intra.peff.net> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When we're reporting an error for a loose object, let's use that term. It's more consistent with other parts of Git, and it is future-proof against changes to the hash function. Signed-off-by: Jeff King --- sha1-file.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sha1-file.c b/sha1-file.c index da6d78976f..07cc9b548b 100644 --- a/sha1-file.c +++ b/sha1-file.c @@ -1552,9 +1552,9 @@ int hash_object_file(const void *buf, unsigned long len, const char *type, static void close_loose_object(int fd) { if (fsync_object_files) - fsync_or_die(fd, "sha1 file"); + fsync_or_die(fd, "loose object file"); if (close(fd) != 0) - die_errno(_("error when closing sha1 file")); + die_errno(_("error when closing loose object file")); } /* Size of directory component, including the ending '/' */ @@ -1645,7 +1645,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr, ret = git_deflate(&stream, Z_FINISH); the_hash_algo->update_fn(&c, in0, stream.next_in - in0); if (write_buffer(fd, compressed, stream.next_out - compressed) < 0) - die(_("unable to write sha1 file")); + die(_("unable to write loose object file")); stream.next_out = compressed; stream.avail_out = sizeof(compressed); } while (ret == Z_OK); From patchwork Mon Jan 7 08:39:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 10750143 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B1278746 for ; Mon, 7 Jan 2019 08:39:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9EB2F2864E for ; Mon, 7 Jan 2019 08:39:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 92C492866C; Mon, 7 Jan 2019 08:39:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3A10A2864E for ; Mon, 7 Jan 2019 08:39:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726540AbfAGIjf (ORCPT ); Mon, 7 Jan 2019 03:39:35 -0500 Received: from cloud.peff.net ([104.130.231.41]:56414 "HELO cloud.peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1725550AbfAGIjf (ORCPT ); Mon, 7 Jan 2019 03:39:35 -0500 Received: (qmail 1794 invoked by uid 109); 7 Jan 2019 08:39:35 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with SMTP; Mon, 07 Jan 2019 08:39:35 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 857 invoked by uid 111); 7 Jan 2019 08:39:13 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) by peff.net (qpsmtpd/0.94) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) SMTP; Mon, 07 Jan 2019 03:39:13 -0500 Authentication-Results: peff.net; auth=none Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Mon, 07 Jan 2019 03:39:33 -0500 Date: Mon, 7 Jan 2019 03:39:33 -0500 From: Jeff King To: =?utf-8?b?UmVuw6k=?= Scharfe Cc: git@vger.kernel.org, Junio C Hamano , =?utf-8?b?w4Z2?= =?utf-8?b?YXIgQXJuZmrDtnLDsA==?= Bjarmason Subject: [PATCH 10/11] sha1-file: avoid "sha1 file" for generic use in messages Message-ID: <20190107083933.GJ29431@sigill.intra.peff.net> References: <20190107083150.GC21362@sigill.intra.peff.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190107083150.GC21362@sigill.intra.peff.net> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP These error messages say "sha1 file", which is vague and not common in user-facing documentation. Unlike the conversions from the previous commit, these do not always refer to loose objects. In finalize_object_file() we could be dealing with a packfile. Let's just say "unable to write file" instead; since we include the filename, the nature of the file is clear from the rest of the message. In force_object_loose(), we're calling into read_object(), which could actually be _any_ type of object. Just say "object". Signed-off-by: Jeff King --- sha1-file.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sha1-file.c b/sha1-file.c index 07cc9b548b..55a4782844 100644 --- a/sha1-file.c +++ b/sha1-file.c @@ -1521,7 +1521,7 @@ int finalize_object_file(const char *tmpfile, const char *filename) unlink_or_warn(tmpfile); if (ret) { if (ret != EEXIST) { - return error_errno(_("unable to write sha1 filename %s"), filename); + return error_errno(_("unable to write file %s"), filename); } /* FIXME!!! Collision check here ? */ } @@ -1744,7 +1744,7 @@ int force_object_loose(const struct object_id *oid, time_t mtime) return 0; buf = read_object(oid, &type, &len); if (!buf) - return error(_("cannot read sha1_file for %s"), oid_to_hex(oid)); + return error(_("cannot read object for %s"), oid_to_hex(oid)); hdrlen = xsnprintf(hdr, sizeof(hdr), "%s %"PRIuMAX , type_name(type), (uintmax_t)len) + 1; ret = write_loose_object(oid, hdr, hdrlen, buf, len, mtime); free(buf); From patchwork Mon Jan 7 08:40:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 10750145 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB2FD746 for ; Mon, 7 Jan 2019 08:40:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D8D5628A20 for ; Mon, 7 Jan 2019 08:40:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CD69228A26; Mon, 7 Jan 2019 08:40:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5C3EF28A20 for ; Mon, 7 Jan 2019 08:40:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726453AbfAGIkg (ORCPT ); Mon, 7 Jan 2019 03:40:36 -0500 Received: from cloud.peff.net ([104.130.231.41]:56426 "HELO cloud.peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1726404AbfAGIkg (ORCPT ); Mon, 7 Jan 2019 03:40:36 -0500 Received: (qmail 1904 invoked by uid 109); 7 Jan 2019 08:40:36 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with SMTP; Mon, 07 Jan 2019 08:40:36 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 877 invoked by uid 111); 7 Jan 2019 08:40:14 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) by peff.net (qpsmtpd/0.94) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) SMTP; Mon, 07 Jan 2019 03:40:14 -0500 Authentication-Results: peff.net; auth=none Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Mon, 07 Jan 2019 03:40:34 -0500 Date: Mon, 7 Jan 2019 03:40:34 -0500 From: Jeff King To: =?utf-8?b?UmVuw6k=?= Scharfe Cc: git@vger.kernel.org, Junio C Hamano , =?utf-8?b?w4Z2?= =?utf-8?b?YXIgQXJuZmrDtnLDsA==?= Bjarmason Subject: [PATCH 11/11] prefer "hash mismatch" to "sha1 mismatch" Message-ID: <20190107084033.GK29431@sigill.intra.peff.net> References: <20190107083150.GC21362@sigill.intra.peff.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190107083150.GC21362@sigill.intra.peff.net> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP To future-proof ourselves against a change in the hash, let's use the more generic "hash mismatch" to refer to integrity problems. Note that we do advertise this exact string in git-fsck(1). However, the message itself is marked for translation, meaning we do not expect it to be machine-readable. While we're touching that documentation, let's also update it for grammar and clarity. Signed-off-by: Jeff King --- I'm actually a little nervous that we _shouldn't_ have marked the messages that fsck produces for translation (and nor should we change them here, but then we're stuck with the word "sha1" forever). I actually think fsck ought to have a machine-readable output format, but of course that does not help any existing scripts. Documentation/git-fsck.txt | 6 +++--- object.c | 4 ++-- sha1-file.c | 4 ++-- t/t1450-fsck.sh | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Documentation/git-fsck.txt b/Documentation/git-fsck.txt index ab9a93fb9b..55950d9eea 100644 --- a/Documentation/git-fsck.txt +++ b/Documentation/git-fsck.txt @@ -140,9 +140,9 @@ dangling :: The object , is present in the database but never 'directly' used. A dangling commit could be a root node. -sha1 mismatch :: - The database has an object who's sha1 doesn't match the - database value. +hash mismatch :: + The database has an object whose hash doesn't match the + object database value. This indicates a serious data integrity problem. Environment Variables diff --git a/object.c b/object.c index a5c5cf830f..df72914bdc 100644 --- a/object.c +++ b/object.c @@ -263,7 +263,7 @@ struct object *parse_object(struct repository *r, const struct object_id *oid) (!obj && has_object_file(oid) && oid_object_info(r, oid, NULL) == OBJ_BLOB)) { if (check_object_signature(repl, NULL, 0, NULL) < 0) { - error(_("sha1 mismatch %s"), oid_to_hex(oid)); + error(_("hash mismatch %s"), oid_to_hex(oid)); return NULL; } parse_blob_buffer(lookup_blob(r, oid), NULL, 0); @@ -274,7 +274,7 @@ struct object *parse_object(struct repository *r, const struct object_id *oid) if (buffer) { if (check_object_signature(repl, buffer, size, type_name(type)) < 0) { free(buffer); - error(_("sha1 mismatch %s"), oid_to_hex(repl)); + error(_("hash mismatch %s"), oid_to_hex(repl)); return NULL; } diff --git a/sha1-file.c b/sha1-file.c index 55a4782844..a5726c3e73 100644 --- a/sha1-file.c +++ b/sha1-file.c @@ -2223,7 +2223,7 @@ static int check_stream_oid(git_zstream *stream, the_hash_algo->final_fn(real_oid.hash, &c); if (!oideq(expected_oid, &real_oid)) { - error(_("sha1 mismatch for %s (expected %s)"), path, + error(_("hash mismatch for %s (expected %s)"), path, oid_to_hex(expected_oid)); return -1; } @@ -2275,7 +2275,7 @@ int read_loose_object(const char *path, } if (check_object_signature(expected_oid, *contents, *size, type_name(*type))) { - error(_("sha1 mismatch for %s (expected %s)"), path, + error(_("hash mismatch for %s (expected %s)"), path, oid_to_hex(expected_oid)); free(*contents); goto out; diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh index 2e5e979336..c61f972141 100755 --- a/t/t1450-fsck.sh +++ b/t/t1450-fsck.sh @@ -406,7 +406,7 @@ test_expect_success 'rev-list --verify-objects with bad sha1' ' test_might_fail git rev-list --verify-objects refs/heads/bogus >/dev/null 2>out && cat out && - test_i18ngrep -q "error: sha1 mismatch 63ffffffffffffffffffffffffffffffffffffff" out + test_i18ngrep -q "error: hash mismatch 63ffffffffffffffffffffffffffffffffffffff" out ' test_expect_success 'force fsck to ignore double author' '