From patchwork Fri Jan 17 22:03:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13943994 Received: from mail-yb1-f177.google.com (mail-yb1-f177.google.com [209.85.219.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 711461ACED2 for ; Fri, 17 Jan 2025 22:03:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151395; cv=none; b=EPiRcIT+ep9hGrAFF4L6riUul/WjOKjhNJN57t9P2bGxGN4jiTLtwDZl5NKgdIemYO5R6btmTo+C5FJMbD1fbr4624vyrGtKP4apBscWeJRBb910TJ/3TbUzk25Dgq7oWtr/tdY398iQ6zbupDtf5tVEHPPmdlLhCxDJT0GAqNI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151395; c=relaxed/simple; bh=jbPcViVQHxoayWAQ1lou2ZsbjQo0KUGNLUNBWiR0PFE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=K3ED4BTHHZ9CkkVMKmPZvuW6cNXpL+aWSpJ9oC/F9Z1hHKr7aluN8cQSfTeglGqwv1folZW2tKq0I6XCk7iwHnZaEOvg/6JvL4nrwyithmveIKk8Tr9vzzs22ztAZPd5lYhUn2RmyQnZTLDopADGctIKap3Y9EqCiaHfZfFuFro= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=n4HXqy+Y; arc=none smtp.client-ip=209.85.219.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="n4HXqy+Y" Received: by mail-yb1-f177.google.com with SMTP id 3f1490d57ef6-e3a0acba5feso3783824276.2 for ; Fri, 17 Jan 2025 14:03:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737151392; x=1737756192; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Uwx/Hv6AEC9pPJADNyNzVM/LHaZ+FmSV5k9M6I4Z3Rw=; b=n4HXqy+Yi7VvRn2qM/608F8N0+xzzTUaagpWP4o7hN4GdqGxgVNmAEKYxn8g6z5Lsb 4okmaS49Nft3oA28r9AyzlGOrpruXTxXQdES7iVfMLNIQs6kVGySmrfcbhJNCeyo0XKS EXZXsCYibcX3aP8nwwFXEfWdexQqD1x77U7Ly5LwThDzsvO3iXsDZ5L3eBp0h2XmnTxc WG484t0l5lAREljCTxHkooscnnnfHD4bGJP5PtaKoyaFJ3YXmo0wLi5cUD1yn/RljCJv CyMeHu1r4tQCChsMhkErCd9MVVHP8mU9KAU9UA+b+qB4Nh5jhMiVanyGFwoQVeQW5h9P oK/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737151392; x=1737756192; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Uwx/Hv6AEC9pPJADNyNzVM/LHaZ+FmSV5k9M6I4Z3Rw=; b=bqi/cZaPWwGrX5ehmv6WuvPCeAB6V2NeBPE0l50QFA+SZXswatF3XQ/87h/CNds7t6 9urrYlOD9s98XisP4vQX6+PAj/NMj0YqP8OBr62sStzTlt424JO5qzUY17scd4rJBACE 0heFsbRcmpioSOFMbV/KxguXmUn0DOGbqon6a0EFlreK+8Q5RVjPFkoTo7YAuW2xJ/0Q mxVN9YsHlRMrzHz0EHLBNK3pjnGeaLFU8DAyKM1N/6tSptz3hROJXF9JgLgBbTonCYtr xjA8VFS52+btLf83L8W+BSB+hTjGF37rYvOZL7yT9cnhZssjdeQbFFTtaqSJXJk2bsCV SEHQ== X-Gm-Message-State: AOJu0YyYpxzH7LdlEtoo9/EfrB8k30193w38Kl+osHceX/nrZ8bwnD0c N44+YNCgXkUNN2wQJWeFuWPwf76weUb3xsY9SsHMVaPvugx+h9dmLcNu3PqZfs0nAg4SPcNS/Rp EXfI= X-Gm-Gg: ASbGncu/+GL+ntWCidI9braKjlz4+crwevHVWW8a7WNGZN1mp4NLQU83Zwbbz7C68tt iG2GpM4RtfiJd3adk8wkEILFIVN8CnMmEPfFz5WFboH4sxcmakXpjh54R+UGhGTTV/qfKzDGGmS vuZGwcUG6rabzExOJh0bhtSyl/Nyk8i2b58qkNAB7n6GjfIAuEBD3WBV91Kxx4Ve1i1mH2voTYy aeouMr8H6lqvLCWULQZP0JgkXBNUz9iYGmLAry+rJ/EybfG1MhbdRBCYq47QFX+3ocLPsqOwPyv Mm0hEMJnplbTESAC2xpdP6pBvD1a X-Google-Smtp-Source: AGHT+IECln+/MrMAOi5fHC4HDsFvV/ma1BliZn1WVBjNDnUmXLwRDVFbkUlht6VVZRcsZgFWU9utow== X-Received: by 2002:a05:690c:4d06:b0:6ef:9017:3cd8 with SMTP id 00721157ae682-6f6eb938a1emr38856277b3.35.1737151391984; Fri, 17 Jan 2025 14:03:11 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f6e63fd82bsm5894277b3.31.2025.01.17.14.03.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Jan 2025 14:03:11 -0800 (PST) Date: Fri, 17 Jan 2025 17:03:10 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v3 1/8] t/helper/test-tool: implement sha1-unsafe helper Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: With the new "unsafe" SHA-1 build knob, it is convenient to have a test-tool that can exercise Git's unsafe SHA-1 wrappers for testing, similar to 't/helper/test-tool sha1'. Implement that helper by altering the implementation of that test-tool (in cmd_hash_impl(), which is generic and parameterized over different hash functions) to conditionally run the unsafe variants of the chosen hash function, and expose the new behavior via a new 'sha1-unsafe' test helper. Signed-off-by: Taylor Blau --- t/helper/test-hash.c | 17 +++++++++++++---- t/helper/test-sha1.c | 7 ++++++- t/helper/test-sha1.sh | 38 ++++++++++++++++++++++---------------- t/helper/test-sha256.c | 2 +- t/helper/test-tool.c | 1 + t/helper/test-tool.h | 3 ++- 6 files changed, 45 insertions(+), 23 deletions(-) diff --git a/t/helper/test-hash.c b/t/helper/test-hash.c index 45d829c908f..d0ee668df95 100644 --- a/t/helper/test-hash.c +++ b/t/helper/test-hash.c @@ -1,7 +1,7 @@ #include "test-tool.h" #include "hex.h" -int cmd_hash_impl(int ac, const char **av, int algo) +int cmd_hash_impl(int ac, const char **av, int algo, int unsafe) { git_hash_ctx ctx; unsigned char hash[GIT_MAX_HEXSZ]; @@ -27,7 +27,10 @@ int cmd_hash_impl(int ac, const char **av, int algo) die("OOPS"); } - algop->init_fn(&ctx); + if (unsafe) + algop->unsafe_init_fn(&ctx); + else + algop->init_fn(&ctx); while (1) { ssize_t sz, this_sz; @@ -46,9 +49,15 @@ int cmd_hash_impl(int ac, const char **av, int algo) } if (this_sz == 0) break; - algop->update_fn(&ctx, buffer, this_sz); + if (unsafe) + algop->unsafe_update_fn(&ctx, buffer, this_sz); + else + algop->update_fn(&ctx, buffer, this_sz); } - algop->final_fn(hash, &ctx); + if (unsafe) + algop->unsafe_final_fn(hash, &ctx); + else + algop->final_fn(hash, &ctx); if (binary) fwrite(hash, 1, algop->rawsz, stdout); diff --git a/t/helper/test-sha1.c b/t/helper/test-sha1.c index e60d000c039..349540c4df8 100644 --- a/t/helper/test-sha1.c +++ b/t/helper/test-sha1.c @@ -3,7 +3,7 @@ int cmd__sha1(int ac, const char **av) { - return cmd_hash_impl(ac, av, GIT_HASH_SHA1); + return cmd_hash_impl(ac, av, GIT_HASH_SHA1, 0); } int cmd__sha1_is_sha1dc(int argc UNUSED, const char **argv UNUSED) @@ -13,3 +13,8 @@ int cmd__sha1_is_sha1dc(int argc UNUSED, const char **argv UNUSED) #endif return 1; } + +int cmd__sha1_unsafe(int ac, const char **av) +{ + return cmd_hash_impl(ac, av, GIT_HASH_SHA1, 1); +} diff --git a/t/helper/test-sha1.sh b/t/helper/test-sha1.sh index 84594885c70..bf387d3db14 100755 --- a/t/helper/test-sha1.sh +++ b/t/helper/test-sha1.sh @@ -3,25 +3,31 @@ dd if=/dev/zero bs=1048576 count=100 2>/dev/null | /usr/bin/time t/helper/test-tool sha1 >/dev/null +dd if=/dev/zero bs=1048576 count=100 2>/dev/null | +/usr/bin/time t/helper/test-tool sha1-unsafe >/dev/null + while read expect cnt pfx do case "$expect" in '#'*) continue ;; esac - actual=$( - { - test -z "$pfx" || echo "$pfx" - dd if=/dev/zero bs=1048576 count=$cnt 2>/dev/null | - perl -pe 'y/\000/g/' - } | ./t/helper/test-tool sha1 $cnt - ) - if test "$expect" = "$actual" - then - echo "OK: $expect $cnt $pfx" - else - echo >&2 "OOPS: $cnt" - echo >&2 "expect: $expect" - echo >&2 "actual: $actual" - exit 1 - fi + for sha1 in sha1 sha1-unsafe + do + actual=$( + { + test -z "$pfx" || echo "$pfx" + dd if=/dev/zero bs=1048576 count=$cnt 2>/dev/null | + perl -pe 'y/\000/g/' + } | ./t/helper/test-tool $sha1 $cnt + ) + if test "$expect" = "$actual" + then + echo "OK ($sha1): $expect $cnt $pfx" + else + echo >&2 "OOPS ($sha1): $cnt" + echo >&2 "expect ($sha1): $expect" + echo >&2 "actual ($sha1): $actual" + exit 1 + fi + done done < X-Patchwork-Id: 13943995 Received: from mail-yb1-f179.google.com (mail-yb1-f179.google.com [209.85.219.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E9851ACED2 for ; Fri, 17 Jan 2025 22:03:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151398; cv=none; b=FXRKAOUMNBZu2L/cuOhM4kcr6OHIoYy8kAcZNwK1TG6Pvdxb7uMy6ShQoNAbvdhoitfAp0Dus2qsc0XGpvbXDww6tl6QWwfehb0mOkptEXPU13K4fy2FOctA7QsQH2MagPvqbqWlL9kN+fIL5cc+3N13PYA5yXTEYg6nvFjpuoM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151398; c=relaxed/simple; bh=6c2efNIDHQ/emvsO1GdUvmmphqDHzyBDErb84N+2EUc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Rl9VpJbFC0AplOd/Mv0UH25EIXjHa9htqeoM1/czACJtg+UdY6cLM1YYSrAS5HbesnJfW9N2UTxnm+NP2dvHiDxc7Jdwa+jozC9J54vSImhhXrBAfaoS9/WJbB+ebpNkV2ofwsLeVsSwiXUxxTo5Qz3MDvwA1jURsIslqeKCzEE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=pYmFnGYH; arc=none smtp.client-ip=209.85.219.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="pYmFnGYH" Received: by mail-yb1-f179.google.com with SMTP id 3f1490d57ef6-e399e904940so4102579276.2 for ; Fri, 17 Jan 2025 14:03:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737151395; x=1737756195; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=o3I104F98Ae3K86wspkO/urGXHLPbOx0f488xCA5RkY=; b=pYmFnGYHfeQ6/9mjO3jvQDyVo0M49YVKm4Aa1/bTTO9FCPeV9V4Sg1z/Fm/naQFQOz 8/mm2kHckekYbVcPCc+l9J8nEFnGoUXklgJkF6oYZp2kd7jWeLilXL0QP4Ly4ZhKtmYo h7NIfbRzymxnr5cRswkgNFXNrqlNQ0q3Qe1gaDwJekYUedhawfY1de0nk4Ki95ZZJyCZ WKwN1mFdzRanHtTEVx/iwYKJugewccwAVGDMIsFU/H8IApy+F5v1vY8Ru3EaOTjEhN23 ocVA8w1RuZlcyF6VtdQYXgkDB0OCgufuOhq2i2yIzBSboEceVgVjXqgh1nlv7VNYLis8 DGPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737151395; x=1737756195; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=o3I104F98Ae3K86wspkO/urGXHLPbOx0f488xCA5RkY=; b=TbKGkUYY5jKChC1LMH6SkbWju0CEAonJaMFMTw62YP2/1o0BGPQ9B+IMnyTfz0Dznf 74HXIjBresHhVpo9D1pQgZK8seiZ0ZFesCnDewL1pN8nqCgjv1hZ6aCJKTKYSyOHjuc5 hQcAWlW9T9+Vel0gzl5J35I3R3DIgki/hYdrgEFX5t0KgWhJxCwmkjVn+YBi/A84yjtl Egyn5zQcUOJWYLE2AnF+KSXeAtmTweyUr4vmYWEXp7vN9UamJclLQdC/Vhcx8yFNt55a uN/GJfWtEs3SLd7r/fdqcpE20JYahQgcTALfIMtt3DiLWFo48E3AL8rtEayXjBKF2Sh7 KOnA== X-Gm-Message-State: AOJu0YzQZu2fzhU6VXHFtQGAHVicComcquzAB1f1/cjF+7mU9oIV+ntu VM5cPMW+3k9atPYQg2Toq6byiB9XguHsgwrO1kd9lxvNkADFu17AzN3e0cIgIlVGAExQn0idRL0 9qlA= X-Gm-Gg: ASbGncsYbdn9hmf5pXNbKZL7giYaC8Jvbb/MulpTdZJDk/iMvb8z3xrfGZc9ze+w/eP Ee0tC7s0t1BbBAcuQLqqB2KjIAyj5Cs1IXpQytmrgcQIx/u1aYJar0yOmLBaX33vO6e5Clu0N0E /gHggxXXOVOoX9h24xeD4VLN91Qt1imETiD9DAR+6OXB3+kYfnQNwO4dq7sz9C3Zdn8QZbvrlQx tLEiRDtPq3O5KtanwPlrOPFLPZ0XB93kE14kogi6mwD5JfiKklTr+UAUy2qBsLZFYf/6sS7+EFi K7ckL7vnqEqvhwOzUtJWi3MEcKIY X-Google-Smtp-Source: AGHT+IFbe+u2VKhSfPGWsQfKkdC+DZjYm8uo76kFswP6+w8n8pTf4LWVUDvA9RpHQsnwPYQtb3T2mg== X-Received: by 2002:a05:690c:4d87:b0:6ef:6c57:ddb6 with SMTP id 00721157ae682-6f6eb90d7aemr40967777b3.29.1737151395066; Fri, 17 Jan 2025 14:03:15 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f6e63ab380sm5924797b3.7.2025.01.17.14.03.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Jan 2025 14:03:14 -0800 (PST) Date: Fri, 17 Jan 2025 17:03:13 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v3 2/8] csum-file: store the hash algorithm as a struct field Message-ID: <2b79c76e4714145057a526876b9ad581c497af0f.1737151386.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Throughout the hashfile API, we rely on a reference to 'the_hash_algo', and call its _unsafe function variants directly. Prepare for a future change where we may use a different 'git_hash_algo' pointer (instead of just relying on 'the_hash_algo' throughout) by making the 'git_hash_algo' pointer a member of the 'hashfile' structure itself. Signed-off-by: Taylor Blau --- csum-file.c | 20 +++++++++++--------- csum-file.h | 1 + 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/csum-file.c b/csum-file.c index 5716016e12e..b28cd047e3f 100644 --- a/csum-file.c +++ b/csum-file.c @@ -50,7 +50,7 @@ void hashflush(struct hashfile *f) if (offset) { if (!f->skip_hash) - the_hash_algo->unsafe_update_fn(&f->ctx, f->buffer, offset); + f->algop->unsafe_update_fn(&f->ctx, f->buffer, offset); flush(f, f->buffer, offset); f->offset = 0; } @@ -71,14 +71,14 @@ int finalize_hashfile(struct hashfile *f, unsigned char *result, hashflush(f); if (f->skip_hash) - hashclr(f->buffer, the_repository->hash_algo); + hashclr(f->buffer, f->algop); else - the_hash_algo->unsafe_final_fn(f->buffer, &f->ctx); + f->algop->unsafe_final_fn(f->buffer, &f->ctx); if (result) - hashcpy(result, f->buffer, the_repository->hash_algo); + hashcpy(result, f->buffer, f->algop); if (flags & CSUM_HASH_IN_STREAM) - flush(f, f->buffer, the_hash_algo->rawsz); + flush(f, f->buffer, f->algop->rawsz); if (flags & CSUM_FSYNC) fsync_component_or_die(component, f->fd, f->name); if (flags & CSUM_CLOSE) { @@ -128,7 +128,7 @@ void hashwrite(struct hashfile *f, const void *buf, unsigned int count) * f->offset is necessarily zero. */ if (!f->skip_hash) - the_hash_algo->unsafe_update_fn(&f->ctx, buf, nr); + f->algop->unsafe_update_fn(&f->ctx, buf, nr); flush(f, buf, nr); } else { /* @@ -174,7 +174,9 @@ static struct hashfile *hashfd_internal(int fd, const char *name, f->name = name; f->do_crc = 0; f->skip_hash = 0; - the_hash_algo->unsafe_init_fn(&f->ctx); + + f->algop = the_hash_algo; + f->algop->unsafe_init_fn(&f->ctx); f->buffer_len = buffer_len; f->buffer = xmalloc(buffer_len); @@ -208,7 +210,7 @@ void hashfile_checkpoint(struct hashfile *f, struct hashfile_checkpoint *checkpo { hashflush(f); checkpoint->offset = f->total; - the_hash_algo->unsafe_clone_fn(&checkpoint->ctx, &f->ctx); + f->algop->unsafe_clone_fn(&checkpoint->ctx, &f->ctx); } int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint) @@ -219,7 +221,7 @@ int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint lseek(f->fd, offset, SEEK_SET) != offset) return -1; f->total = offset; - the_hash_algo->unsafe_clone_fn(&f->ctx, &checkpoint->ctx); + f->algop->unsafe_clone_fn(&f->ctx, &checkpoint->ctx); f->offset = 0; /* hashflush() was called in checkpoint */ return 0; } diff --git a/csum-file.h b/csum-file.h index 7c73da0a40a..2b45f4673a2 100644 --- a/csum-file.h +++ b/csum-file.h @@ -20,6 +20,7 @@ struct hashfile { size_t buffer_len; unsigned char *buffer; unsigned char *check_buffer; + const struct git_hash_algo *algop; /** * If non-zero, skip_hash indicates that we should From patchwork Fri Jan 17 22:03:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13943996 Received: from mail-yb1-f170.google.com (mail-yb1-f170.google.com [209.85.219.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9BEEA1D90D7 for ; Fri, 17 Jan 2025 22:03:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151401; cv=none; b=uq/21muBS7NoA2QO7qBAd8faoiV54DpKq94ORM/V3/6gZDj7zh5NHvdDxXOx6EhGK+OpX290RvoeCM2J2f1tyEWDMbSuKY+3Fn4pXRmVUS1vNYIBH7M2xsEQj3fh+UThXmnA9fyeMeo0aHmHoBiO+X/AaPKzyVgaLcQo85r6hbc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151401; c=relaxed/simple; bh=5TeyP+2I7e1QumaevDVX52+qaxCl8MPjc2R+DIooTo0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=iYaalCA0plU8kFPtFPNIDoVQBfad68W6UFMkvT7vCHHCP/wdc+rcLjV95RcDBPY8tzhZR5zLX+SCbjgJVsE6cmnZ5bEfFsUATkBjc95LfpbvQ87kV3B2+BprydWBBmFWi1iUhvKP/YmpVTK2sI+UQke8VAwbsucWLifKvAHAJ5s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=Gqlp6yFt; arc=none smtp.client-ip=209.85.219.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="Gqlp6yFt" Received: by mail-yb1-f170.google.com with SMTP id 3f1490d57ef6-e549b0f8d57so4639844276.3 for ; Fri, 17 Jan 2025 14:03:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737151398; x=1737756198; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=zZx2rtUEUNJp8BLqSjRVtuWYRqiV1muUjTPFRHkYlH0=; b=Gqlp6yFtKy+iPrUJp70GZCSIHXmv/cddx64LhRXeUz4Pn+HkAcTEHGj0WIpl47SvyH y37/AhnqiDv4qQNJedFSXYxaYPwremf97iaHtt2jabvkBtaQd4b06dknDKALavFwfQWF JEvNvDVh3JGHpBOdVqL8Tbq97QBfglzKF5yA3omE8c+BIJ+ebdpthadSUo2LrfzYi/70 HQL2edXBeErX90khXOIYLMNqzzs7Wc3OS3ZHM0ETPi5gyXhkQqH9O+vvWYlrDgUeohfM rJLSSt/ULg7cVx/dIcN0dDEfHC1wfkQ7nY/2bsfBc33sE/9WUhoyG5kLKuUbwzqTFGbX GmHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737151398; x=1737756198; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=zZx2rtUEUNJp8BLqSjRVtuWYRqiV1muUjTPFRHkYlH0=; b=a32pnjbZNIHxSbUHvk2qpKZheJjZXO1aGZr42B0SW6Lf2dKvhowNsIJjWUquDLrTxR rYKUjv6zVrUNpnOqbPpYBVvAQ4ZWDyzdKb5/CyS9c3sVslRQG0yK2OzM2D/ubsC8Y9Yk 28CK4ok48L1N7z+LPvF1KGCkcM2ufgpAkmirqVqxMWk1+ca9n6vB6FrwwCMdmGhfOpU8 U6aVtjfPhNWIKTelycH4oxmRzVg7yMXDKK38FgU4dGaCKtEVGT+FTpnYndljDy/bXymL fXPK7Id0FNtUCNxezyht3LVLR0OmaRQ4Gs82T2RgQkHb0jQIqiGGkvhroJG13KhPNzpG rIfQ== X-Gm-Message-State: AOJu0Yytk/Im/4aBcEoET0h41T86yUvDPnLjcpmQnn2pmoTvGk4p7xh2 bDRzxG84cvfLgKKN8dAiDN/xcuLdHR09OACr8X6ZpJE/b9uBAeadKex4TEDTvdoq1h3qlqzOjSo 8C4I= X-Gm-Gg: ASbGncsJUgFrZG2CZoZ58VJEa9fMKsivnEHajTwuPAz6mmU9S2p1ntnsvh19PteMWFE bRUhaULLXcmvVqfJsYlctTezpevc4dOTPe/wuEQl5SQhaF6C6UYCJp+jWprxBTDPsDxJj2w5Tra rtXlzUXSM3ITcuYcCaUoeFUcpSmeZ0637YGtvw6aK80J+RnuZlyqGJq2cb3R7vSvXonmyFpq0nj 8BeQf+cgAnoKFPhMIZlg9wxVTBDBUaXYNYUcg5BB5hpG26FtycuY3vM2HaQEzh6eO9hQuvcx3xd k8skFxe7sJJFiR2M+AdSQZoDmd1P X-Google-Smtp-Source: AGHT+IGXaWt1wiSlwIzouVNDP5ndlQLT8VBXTETzqM+gRCXa3Mq2lFYN69O8Bt/m4/oYOUYCRsAYyw== X-Received: by 2002:a05:6902:18c5:b0:e39:826b:4c14 with SMTP id 3f1490d57ef6-e57b132671cmr2952932276.32.1737151398212; Fri, 17 Jan 2025 14:03:18 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e57ab47ab22sm579160276.57.2025.01.17.14.03.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Jan 2025 14:03:17 -0800 (PST) Date: Fri, 17 Jan 2025 17:03:16 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v3 3/8] csum-file.c: extract algop from hashfile_checksum_valid() Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Perform a similar transformation as in the previous commit, but focused instead on hashfile_checksum_valid(). This function does not work with a hashfile structure itself, and instead validates the raw contents of a file written using the hashfile API. We'll want to be prepared for a similar change to this function in the future, so prepare ourselves for that by extracting 'the_hash_algo' into its own field for use within this function. Signed-off-by: Taylor Blau --- csum-file.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/csum-file.c b/csum-file.c index b28cd047e3f..7a71121e340 100644 --- a/csum-file.c +++ b/csum-file.c @@ -242,14 +242,15 @@ int hashfile_checksum_valid(const unsigned char *data, size_t total_len) { unsigned char got[GIT_MAX_RAWSZ]; git_hash_ctx ctx; - size_t data_len = total_len - the_hash_algo->rawsz; + const struct git_hash_algo *algop = the_hash_algo; + size_t data_len = total_len - algop->rawsz; - if (total_len < the_hash_algo->rawsz) + if (total_len < algop->rawsz) return 0; /* say "too short"? */ - the_hash_algo->unsafe_init_fn(&ctx); - the_hash_algo->unsafe_update_fn(&ctx, data, data_len); - the_hash_algo->unsafe_final_fn(got, &ctx); + algop->unsafe_init_fn(&ctx); + algop->unsafe_update_fn(&ctx, data, data_len); + algop->unsafe_final_fn(got, &ctx); - return hasheq(got, data + data_len, the_repository->hash_algo); + return hasheq(got, data + data_len, algop); } From patchwork Fri Jan 17 22:03:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13943997 Received: from mail-yb1-f175.google.com (mail-yb1-f175.google.com [209.85.219.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A706C1DE8A2 for ; Fri, 17 Jan 2025 22:03:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151404; cv=none; b=PBaAHTNFtRO5RQ6aw6/CdHSEGoIQ5pbZkqiXAn5EnLFun/efsLFqfbRj6Y/0ryft4B5qgPZ/rkUEC3HNzQOSLRMe05IAVUkHSkCmVWV0xBq9yjmG6YjiR4sOgjkQ4KtEtLHSZNrk/QF9fxYhO33ozs3AzfUergfUkCTDJwuFYaM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151404; c=relaxed/simple; bh=UuKVjRxNWzII24LfmOG9qyDwdfx0WBYGreOP/x6qCMs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=CIMMTevDPXufekKkfsEe1pdZaVoZN1cZnvhJBgrgK8ukSU2WUw3hhPjyk1AxxhzZ/M/LbbG15VpCBc2kBl6xOSWXG0Ek4F7izTHVZQ+v2vW2XiYhe5lCc0FHxsdvAva9ps12ULT9aHN1XqKlTTT9o5L8Zc2l5ZBiTIw9bjckGYM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=ojuqGphg; arc=none smtp.client-ip=209.85.219.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="ojuqGphg" Received: by mail-yb1-f175.google.com with SMTP id 3f1490d57ef6-e3a26de697fso4320554276.3 for ; Fri, 17 Jan 2025 14:03:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737151401; x=1737756201; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=3nU3sFVreh/uBgnQRMHuOup+HRxvtN28BUpzFT9vj+I=; b=ojuqGphgvPRglTiEzt229i7PFlijqQoeQK5Xn89U0G8RA32Ovr9dnQWCEVwhm1NWn0 h1JCjix1MJLFrEddZzZTQImp0vMhWxdAM7IfuzeviQ44EAhocr9Y9Mu8Nm4yuMHJoscu Wp44O1WJzThAM6FK9TTBBbF7gkmuBHE1i1yRgEkW7oHzixNtwejhDmfu00hwKHT2j6Bw Z5IBHuvmgFCGUCU6IIFaZZp13cOYVGEooCW8wcVl05TrnGrurFbRio27fquaWS7naoqZ E3/xYU9MBZ1BCWMEZvOBATSYU+2/6BKuiaFwj+YPZMjeKPQn8zhpqYLb2EhgGSNzvJHK eE6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737151401; x=1737756201; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=3nU3sFVreh/uBgnQRMHuOup+HRxvtN28BUpzFT9vj+I=; b=IP/jADIxIIAE3wiHauC47FQpcftuZ6ZfsUnmukvt4uMsTaAWB87I+NagM32bi4aQ72 niJZvBm0k6tqHh1Y2daIdgrwIdoKie8QOHGAYxZiJPAtA9Bu57TXPKi6GOonBfO80slr Nej290uxNb8hTLnlimFQbTL6Bxr7ksAscKtD2FWagZCMk8XPc5HuZypEcctc5JnxQDHp lwBWWSPO3mvCkYmrRBzFTZ0I0pReVnjkr92A6TBk5+UD0/oD0mud1TyYahtDzaoeJqqC 9k3iHEY6DM8CaDBBDa9xLJ6hca7I12iQW7gOvCAWHr/kxXEbtmsStyOYXE3vgTKfUw+a 6Lpw== X-Gm-Message-State: AOJu0YxDR3924whxaXqmaa7FPd4r07fC7BPGgtf+HLtdSUaVVUER/g1E COzTQC9pixZ1fGFbJT0T4hQrmluthWNAupXc8A4oZl1vmBfXRAYTkLLq7cmNwUFDACnZPRdRHaa ekFI= X-Gm-Gg: ASbGncsHbBWODq02VXcbDfI2oYcVPWm+I4U6skyjwftqvzKQK1hnjot4LWn8Zg6Fud6 NU5+Eab/EoG3++FNxUd+CfoRUbcuy6PHNM5Jyx2LZAn7HRyH5icoFnS1EODktqKg8a8/J2u4DMR guBtK+814lE6RdcwZcZ7j9cv0x5bKxBHPB5oaZ1EWru2icDR5mplnYWGr1WyaPxDIsdbwR9bMAJ bAQic5oKEmrLEME/kFmnY8mv8I3M0Gf82OwC42r0KO9xjvv30mwGb28/H+6SUqw4tOdjRyjkGvp S1RA/91zWxfoitDNB0iRbMKYX3PZLht1X8CxrHcuMQ== X-Google-Smtp-Source: AGHT+IFX/N0J+bYx6+EOtQ/+IlnGKQN7qSRAZPEPkf28oR2oQkyR43UvpJLS6IzPwI4je1cH+g1H9A== X-Received: by 2002:a05:6902:2788:b0:e28:ee2b:34fa with SMTP id 3f1490d57ef6-e57b0e28dadmr3583053276.0.1737151401393; Fri, 17 Jan 2025 14:03:21 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e57ab4767a5sm576778276.47.2025.01.17.14.03.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Jan 2025 14:03:21 -0800 (PST) Date: Fri, 17 Jan 2025 17:03:19 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v3 4/8] hash.h: introduce `unsafe_hash_algo()` Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: In 253ed9ecff (hash.h: scaffolding for _unsafe hashing variants, 2024-09-26), we introduced "unsafe" variants of the SHA-1 hashing functions by introducing new functions like "unsafe_init_fn()" and so on. This approach has a major shortcoming that callers must remember to consistently use one variant or the other. Failing to consistently use (or not use) the unsafe variants can lead to crashes at best, or subtle memory corruption issues at worst. In the hashfile API, this isn't difficult to achieve, but verifying that all callers consistently use the unsafe variants is somewhat of a chore given how spread out all of the callers are. In the sha1 and sha1-unsafe test helpers, all of the calls to various hash functions are guarded by an "if (unsafe)" conditional, which is repetitive and cumbersome. Address these issues by introducing a new pattern whereby one 'git_hash_algo' can return a pointer to another 'git_hash_algo' that represents the unsafe version of itself. So instead of having something like: if (unsafe) the_hash_algo->init_fn(...); the_hash_algo->update_fn(...); the_hash_algo->final_fn(...); else the_hash_algo->unsafe_init_fn(...); the_hash_algo->unsafe_update_fn(...); the_hash_algo->unsafe_final_fn(...); we can instead write: struct git_hash_algo *algop = the_hash_algo; if (unsafe) algop = unsafe_hash_algo(algop); algop->init_fn(...); algop->update_fn(...); algop->final_fn(...); This removes the existing shortcoming by no longer forcing the caller to "remember" which variant of the hash functions it wants to call, only to hold onto a 'struct git_hash_algo' pointer that is initialized once. Similarly, while there currently is still a way to "mix" safe and unsafe functions, this too will go away after subsequent commits remove all direct calls to the unsafe_ variants. Note that hash_algo_by_ptr() needs an adjustment to allow passing in the unsafe variant of a hash function. All other query functions on the hash_algos array will continue to return the safe variants of any function. Suggested-by: Jeff King Signed-off-by: Taylor Blau --- hash.h | 13 ++++++++++++- object-file.c | 26 ++++++++++++++++++++++++++ 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/hash.h b/hash.h index 756166ce5e8..a68a2b6a161 100644 --- a/hash.h +++ b/hash.h @@ -305,6 +305,9 @@ struct git_hash_algo { /* The all-zeros OID. */ const struct object_id *null_oid; + + /* The unsafe variant of this hash function, if one exists. */ + const struct git_hash_algo *unsafe; }; extern const struct git_hash_algo hash_algos[GIT_HASH_NALGOS]; @@ -320,9 +323,17 @@ int hash_algo_by_length(int len); /* Identical, except for a pointer to struct git_hash_algo. */ static inline int hash_algo_by_ptr(const struct git_hash_algo *p) { - return p - hash_algos; + size_t i; + for (i = 0; i < GIT_HASH_NALGOS; i++) { + const struct git_hash_algo *algop = &hash_algos[i]; + if (p == algop || (algop->unsafe && p == algop->unsafe)) + return i; + } + return GIT_HASH_UNKNOWN; } +const struct git_hash_algo *unsafe_hash_algo(const struct git_hash_algo *algop); + const struct object_id *null_oid(void); static inline int hashcmp(const unsigned char *sha1, const unsigned char *sha2, const struct git_hash_algo *algop) diff --git a/object-file.c b/object-file.c index 5b792b3dd42..43efa4ca361 100644 --- a/object-file.c +++ b/object-file.c @@ -202,6 +202,22 @@ static void git_hash_unknown_final_oid(struct object_id *oid UNUSED, BUG("trying to finalize unknown hash"); } +static const struct git_hash_algo sha1_unsafe_algo = { + .name = "sha1", + .format_id = GIT_SHA1_FORMAT_ID, + .rawsz = GIT_SHA1_RAWSZ, + .hexsz = GIT_SHA1_HEXSZ, + .blksz = GIT_SHA1_BLKSZ, + .init_fn = git_hash_sha1_init_unsafe, + .clone_fn = git_hash_sha1_clone_unsafe, + .update_fn = git_hash_sha1_update_unsafe, + .final_fn = git_hash_sha1_final_unsafe, + .final_oid_fn = git_hash_sha1_final_oid_unsafe, + .empty_tree = &empty_tree_oid, + .empty_blob = &empty_blob_oid, + .null_oid = &null_oid_sha1, +}; + const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { { .name = NULL, @@ -239,6 +255,7 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { .unsafe_update_fn = git_hash_sha1_update_unsafe, .unsafe_final_fn = git_hash_sha1_final_unsafe, .unsafe_final_oid_fn = git_hash_sha1_final_oid_unsafe, + .unsafe = &sha1_unsafe_algo, .empty_tree = &empty_tree_oid, .empty_blob = &empty_blob_oid, .null_oid = &null_oid_sha1, @@ -305,6 +322,15 @@ int hash_algo_by_length(int len) return GIT_HASH_UNKNOWN; } +const struct git_hash_algo *unsafe_hash_algo(const struct git_hash_algo *algop) +{ + /* If we have a faster "unsafe" implementation, use that. */ + if (algop->unsafe) + return algop->unsafe; + /* Otherwise use the default one. */ + return algop; +} + /* * This is meant to hold a *small* number of objects that you would * want repo_read_object_file() to be able to return, but yet you do not want From patchwork Fri Jan 17 22:03:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13943998 Received: from mail-yb1-f175.google.com (mail-yb1-f175.google.com [209.85.219.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 02A021DE8A2 for ; Fri, 17 Jan 2025 22:03:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151407; cv=none; b=d4Nha/dX5IP9To8ser4ko4qY5GjfeGNbrkatmPoeGg5H03yMzLoxdNdBAqeJuplX2e4RWX5oeNWP8x9eDHvnVSJ4XlWnNdX0f00OgsT8yPN0gezBYpEVw8FwwnAkWou3ewcbN83SwmosjIhvzUXi0GKgJoJAd6QowsAD3xTBEi4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151407; c=relaxed/simple; bh=1c8UGk8NIpUPvvZEDL/j+/Tl0Vnvf+F9EzYy5gaDyLc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=GWf17AEL7KzZk9+VWBupgmMj9DpanHL4EU3jObEMI+yiScKT4Jpkqn9/eqH3WYANx0LkXKAkg528eQ75XMOb1QXDG/DRWsBEbvbciosELif5NvwO4AkQJJDfTqyGW+qUp9snHvXMf/8sktZ1Ap9RHl9jwhEzq6EoRctaB+XHl4E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=1IowE5th; arc=none smtp.client-ip=209.85.219.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="1IowE5th" Received: by mail-yb1-f175.google.com with SMTP id 3f1490d57ef6-e3c9ec344efso4000227276.2 for ; Fri, 17 Jan 2025 14:03:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737151404; x=1737756204; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=T5RdfVzpkWSjVoMBALnctho7ROnLcrWsck32SjVs6L4=; b=1IowE5thtsSd/Myd+VqcTG+Wi1Oyo2yYL6ZomMyDe7jh3PDx5Evk0K2lUhbYPr8a1S 2L9KjFyKigkA3MpsMAJ97ie61HKSX2v5iuYrDcR2LJ1mIYKTfJ6A+enY6IxaWHCVbr1c Z9nyQeJC/MAbeoPvR4Vp3q95BYqDW2yw5Viyln4qGyP/U/RIWLauj/b29QE0nnNUISZR JS+lalwwFtbe4CiTdFE+AyaOE6snnnamX+GWEOgfHQjRgrMpaAGiQAy83DhAB/JoRvTW ApZuGXrCbI7ipQtL4hDvIhm40kbl9A2tXtoDE0oyA+CwE4j2uJhLUmzQZPWnX7rwQAb9 4YhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737151404; x=1737756204; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=T5RdfVzpkWSjVoMBALnctho7ROnLcrWsck32SjVs6L4=; b=QH2QyLhDHjUgxUGOOFi8ODd/bO1oc5HkTM2uh2F6RnNUl+zVXRt4BpS97ONzL08S6/ eTMAN6q4G47/uIidK2MAUDKwpI4Bt4eDj3yYcXALKK/NnE1hsMwimS1hzYXlOQSEYMAN MVQwhgZlICgmmkjSdzJQ5XUF9cxRJJFftW5Uq8kqlhTHN0R0Kb9Tsohbbaz6VSqHK3+l VA00j24Ws4zJK9N/Y1fV6S8poEA/HoangvFZVzf+xxq3RUWFeqlLoPbkXYrMLV/1WKtH PdeO2wZLbEm+eEnXy7hAM3BEaybyeo75K15o8Hn4vpKLjosfZV23m/3X7UuN7IiaH/v0 hTBw== X-Gm-Message-State: AOJu0YxFQ+pWZJTsiJu2pNN4gRrErjlcIM1gEp3GynZYrxxaiXQQlkfY jCnwO79LUAEZIS+pS/TYD+NdknHh71dDo0jW7ISq+it83Iqm22G2J1hBP3cwwKnVmmfPXv2CTZz TR/Q= X-Gm-Gg: ASbGnctjilHsHxE40/1rg71OMV+oPYnpoWLeM+PXo0q+nq9uKzxpJo0n5ZKcctvcFmj RtUHpmT9qWNHr8Fumw0eBpk/D2JgtHRK5mXKZgu7ZBGEjFoQDGLvgGj4InjSWgO6+6WUWtb6nUJ Gew19IvPDIHtj0hHoKoJoC33Bxkxn7nsr8pwKeDIEtKb8eavG7iH4sQ7ZLmVPaEygU2v1x4zpi2 oagyp6LHrt16whSCQnf5TpHNmUy4ef0ND9Vfw31U0nVTRftXOzrzS+AgTuuoonh6S/dvtKre0+E P1RZzddSZyQVWCewLgIBMxtajF77 X-Google-Smtp-Source: AGHT+IGCVnfMDQ9QrrfMwj9ZDO21QRAJuYXyqz20Oe+7BWs4/4Cwze0wqfVSxN6nymX0fFHVO2j3Kg== X-Received: by 2002:a05:6902:2749:b0:e55:12e2:62c4 with SMTP id 3f1490d57ef6-e57b132efd2mr3453600276.35.1737151404638; Fri, 17 Jan 2025 14:03:24 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e57ab43aa46sm579916276.45.2025.01.17.14.03.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Jan 2025 14:03:24 -0800 (PST) Date: Fri, 17 Jan 2025 17:03:23 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v3 5/8] csum-file.c: use unsafe_hash_algo() Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Instead of calling the unsafe_ hash function variants directly, make use of the shared 'algop' pointer by initializing it to: f->algop = unsafe_hash_algo(the_hash_algo); , thus making all calls use the unsafe variants directly. Signed-off-by: Taylor Blau --- csum-file.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/csum-file.c b/csum-file.c index 7a71121e340..ebffc80ef71 100644 --- a/csum-file.c +++ b/csum-file.c @@ -50,7 +50,7 @@ void hashflush(struct hashfile *f) if (offset) { if (!f->skip_hash) - f->algop->unsafe_update_fn(&f->ctx, f->buffer, offset); + f->algop->update_fn(&f->ctx, f->buffer, offset); flush(f, f->buffer, offset); f->offset = 0; } @@ -73,7 +73,7 @@ int finalize_hashfile(struct hashfile *f, unsigned char *result, if (f->skip_hash) hashclr(f->buffer, f->algop); else - f->algop->unsafe_final_fn(f->buffer, &f->ctx); + f->algop->final_fn(f->buffer, &f->ctx); if (result) hashcpy(result, f->buffer, f->algop); @@ -128,7 +128,7 @@ void hashwrite(struct hashfile *f, const void *buf, unsigned int count) * f->offset is necessarily zero. */ if (!f->skip_hash) - f->algop->unsafe_update_fn(&f->ctx, buf, nr); + f->algop->update_fn(&f->ctx, buf, nr); flush(f, buf, nr); } else { /* @@ -175,8 +175,8 @@ static struct hashfile *hashfd_internal(int fd, const char *name, f->do_crc = 0; f->skip_hash = 0; - f->algop = the_hash_algo; - f->algop->unsafe_init_fn(&f->ctx); + f->algop = unsafe_hash_algo(the_hash_algo); + f->algop->init_fn(&f->ctx); f->buffer_len = buffer_len; f->buffer = xmalloc(buffer_len); @@ -210,7 +210,7 @@ void hashfile_checkpoint(struct hashfile *f, struct hashfile_checkpoint *checkpo { hashflush(f); checkpoint->offset = f->total; - f->algop->unsafe_clone_fn(&checkpoint->ctx, &f->ctx); + f->algop->clone_fn(&checkpoint->ctx, &f->ctx); } int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint) @@ -221,7 +221,7 @@ int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint lseek(f->fd, offset, SEEK_SET) != offset) return -1; f->total = offset; - f->algop->unsafe_clone_fn(&f->ctx, &checkpoint->ctx); + f->algop->clone_fn(&f->ctx, &checkpoint->ctx); f->offset = 0; /* hashflush() was called in checkpoint */ return 0; } @@ -242,15 +242,15 @@ int hashfile_checksum_valid(const unsigned char *data, size_t total_len) { unsigned char got[GIT_MAX_RAWSZ]; git_hash_ctx ctx; - const struct git_hash_algo *algop = the_hash_algo; + const struct git_hash_algo *algop = unsafe_hash_algo(the_hash_algo); size_t data_len = total_len - algop->rawsz; if (total_len < algop->rawsz) return 0; /* say "too short"? */ - algop->unsafe_init_fn(&ctx); - algop->unsafe_update_fn(&ctx, data, data_len); - algop->unsafe_final_fn(got, &ctx); + algop->init_fn(&ctx); + algop->update_fn(&ctx, data, data_len); + algop->final_fn(got, &ctx); return hasheq(got, data + data_len, algop); } From patchwork Fri Jan 17 22:03:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13943999 Received: from mail-yb1-f171.google.com (mail-yb1-f171.google.com [209.85.219.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E46A1DE8A2 for ; Fri, 17 Jan 2025 22:03:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151410; cv=none; b=FK10kFeTvZWZ7SUGIc/N2YaHk3LFA36C0v//aVo38mjUiOQklALMNXBgk6vlegNx1SlmcUWIUFUUdSHGUTsaRlK0rSU1QbS8I45R/80V9qkik4gRHfcb1OAkluI7B3X9EChupfH0xwzzxL2+mDRnNhhu85qhg7j4x4LiNvLEPCM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151410; c=relaxed/simple; bh=6Sr5D0vo9c5IeE3wtXQeheOl4L07Dh4wqzZMQifXjrM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Byq+t9ykQi69cwV/NsMO9yy1Th55dcUbBVZkARYHy/OjWaPTKmonaMb18AMna2yGhi1UwJ/RKJaW53UPXhJgY7t/Z70Jyz5M4elkAE/Obmw0TIRtZd2NwQdcwU/wDL3yG4qPxXmN23LH3OOe/bpBWcix116AUXpKfGOEdrBv9GI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=wRzyYZ6W; arc=none smtp.client-ip=209.85.219.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="wRzyYZ6W" Received: by mail-yb1-f171.google.com with SMTP id 3f1490d57ef6-e54bd61e793so4999169276.2 for ; Fri, 17 Jan 2025 14:03:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737151408; x=1737756208; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=uoN10/GjMDdlRdDA4HFTLrdK38qB77pmy38zO60vFhQ=; b=wRzyYZ6WL+gQGl6r1oWP+cTx/RRotWnI2ZiVmlUQzeaFbrfA7J45LIxT4FOvUqVUt0 /sZqT/e/reFzCRQETTzhWkKYmTOj5xALRS5Ro4fIelH7GHR9DNb86A/WftCd4OGgawep ZV3xwzf4rIjNeLjP90HhQXVIWS2JPKJ1PtzqfghaW3ZuYiyRkkazaM2v1+7kW1SWm9dQ kd+57xGQyILT41hAayMEvyDoi1gVPzIojiWCPoQ2fL3Fa/2KWVoc+ruWjkrA6HHW6IxL ueIog/Ru9REOSUjsudSr0jKYIcJGM3e/TMHTk5cGLk7GAkJONXxooHcJ5A6AYeFibKZc tDlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737151408; x=1737756208; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=uoN10/GjMDdlRdDA4HFTLrdK38qB77pmy38zO60vFhQ=; b=kcWD/qgQFKvfkXSli7i8xHT0soH1QjGOyU8HfCzkB3g6EkO9tKEXz/wI8XPHUT0ivQ 5mM0vVEFW0gh+TGPZnimAX30rk0pRiX2ISz4QXnpqu+3Ldzls1aGS7hYF/yGBvj/CeXF /pqZZGqsUAbi4o18NrZ0fXgzhYp/wGFH+QIOzFvvkrIBcLalLMS99BbjmamGWuIfknCG 6oB7rfbZJOPdvqzLvJkCq6FwTM6IBqteumwf7vbbbYpHRGRS1tIUQHbdLCLk4X0DTHle SpSH0dvcRUZchQ7wFU1eM1u5iekeHPqhpRduzMs3vgOoIE1+1l7WH+TfXoJPK8pB/NxS PV6Q== X-Gm-Message-State: AOJu0Yyg/rCAtY6RujOWFeE4vrhXfONubsLPE9twJUWtg44ra33KbdiV 5bIsvfstDbe2zyoOLwvSvSlTIHrot18D+prC/MOYsX8z4c7cTPjF79Hypkzx6HvBu3U6Xz1OVDk geic= X-Gm-Gg: ASbGncu8Pn4DLbAWHcAk6sSeXeEDPGPzuq1M0CLypJWam7TE5RrTG4iDIE9eT9hGL9R GLSurCkUFQBRlX+3O8MfSlp3AD+16PcXq/spvZ2Pz8+WRxgdUSne3gFrWAmTT9XlwExn3DYdU2n LZn3bNSUb9WOx2CCIfkfYnmYrCGMuBWesy7wMmRMErkDk3CcCU7UvgJPyfmS44yBGikZyE0IyLF S235EOTmBTZm5Bl30nC+lqPzQScMHWjwsBHHxA7nNUa6dTmdxGC8jxhOICf3Ljhd6jDplEu5T05 9ePLSmHtDCjyi1voyyzgE424T0iu X-Google-Smtp-Source: AGHT+IGhqS7P5Xot+K2cQUl/ngK3IN1iax/cYJ95greo1qi2i7RTNAq2NK1kSqedGoBqfUGnPGrE7w== X-Received: by 2002:a05:6902:102e:b0:e57:9250:b722 with SMTP id 3f1490d57ef6-e57b1036009mr3694324276.2.1737151407920; Fri, 17 Jan 2025 14:03:27 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e57ab2a7665sm584334276.5.2025.01.17.14.03.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Jan 2025 14:03:27 -0800 (PST) Date: Fri, 17 Jan 2025 17:03:26 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v3 6/8] t/helper/test-hash.c: use unsafe_hash_algo() Message-ID: <21b175b07ff46471acdaa1afe5fa8b0c2749248b.1737151386.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Remove a series of conditionals within the shared cmd_hash_impl() helper that powers the 'sha1' and 'sha1-unsafe' helpers. Instead, replace them with a single conditional that transforms the specified hash algorithm into its unsafe variant. Then all subsequent calls can directly use whatever function it wants to call without having to decide between the safe and unsafe variants. Signed-off-by: Taylor Blau --- t/helper/test-hash.c | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/t/helper/test-hash.c b/t/helper/test-hash.c index d0ee668df95..aa82638c621 100644 --- a/t/helper/test-hash.c +++ b/t/helper/test-hash.c @@ -9,6 +9,8 @@ int cmd_hash_impl(int ac, const char **av, int algo, int unsafe) int binary = 0; char *buffer; const struct git_hash_algo *algop = &hash_algos[algo]; + if (unsafe) + algop = unsafe_hash_algo(algop); if (ac == 2) { if (!strcmp(av[1], "-b")) @@ -27,10 +29,7 @@ int cmd_hash_impl(int ac, const char **av, int algo, int unsafe) die("OOPS"); } - if (unsafe) - algop->unsafe_init_fn(&ctx); - else - algop->init_fn(&ctx); + algop->init_fn(&ctx); while (1) { ssize_t sz, this_sz; @@ -49,15 +48,9 @@ int cmd_hash_impl(int ac, const char **av, int algo, int unsafe) } if (this_sz == 0) break; - if (unsafe) - algop->unsafe_update_fn(&ctx, buffer, this_sz); - else - algop->update_fn(&ctx, buffer, this_sz); + algop->update_fn(&ctx, buffer, this_sz); } - if (unsafe) - algop->unsafe_final_fn(hash, &ctx); - else - algop->final_fn(hash, &ctx); + algop->final_fn(hash, &ctx); if (binary) fwrite(hash, 1, algop->rawsz, stdout); From patchwork Fri Jan 17 22:03:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13944000 Received: from mail-yb1-f169.google.com (mail-yb1-f169.google.com [209.85.219.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 448691D86C7 for ; Fri, 17 Jan 2025 22:03:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151413; cv=none; b=Fi4PxEi12zuL1KqsVXHHIrVaFUjsGejHBMPpx66ckgZw5ufjzKOLZtMOgMvbBZWXmmPrkv3YJo7PAjz7NvrtuNtiFvKheQ2r4tNlL/IjdY+SQyhbdw3H0dxM49buT5yIYUXpme0KMMJme6L124JknyDP5i6hkZAyR7YQQdZ1MVc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151413; c=relaxed/simple; bh=4tEx831bUDWTLq/a/Jj+gaAObDrweYwFVXGjx+prZAM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=bpq3XnXvAzk7yW2eVvUXslmH8x1tE9v3yXN2xpR4LSy8upSNmsTewEZQmoxSNZwv2BeIgIikc2EqERXpy/vtDRxHhvKAJqN/2e33kbTuuHYiocv4yZ4NzdEy1AB81aikycTFzbj97bTG1pfA3VQ9766RGqdl256A7SJo+Xx/aWA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=G3uu5f97; arc=none smtp.client-ip=209.85.219.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="G3uu5f97" Received: by mail-yb1-f169.google.com with SMTP id 3f1490d57ef6-e398484b60bso4172186276.1 for ; Fri, 17 Jan 2025 14:03:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737151411; x=1737756211; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=qBY741sXZOT0WTbq40IXRM3lwOB19DkD28587LMhXH4=; b=G3uu5f97un0GTv803Gy7PVXW8Vwod8LmeMlylcwfBVurbHLhhTm1eL6hGC0EZRoVhj p3DxqE7Q+Dgj3NJLmiga1sHcx+POrx9rH96XN4uGzMAkB/UUWd8jw/hm8hESnhyTbiSw 4KvzpQq1YigA2pidE6/vzX+uXDreMSTb2sbSShCYzj0hQyeEti5A97qT0zkCrbHWixne V49CF9eIdFv3uDTpUxhNDO8JwvwFl6IHeB64ts86gw84PuRtOK2PHLivLm+YG9So1dBM YPWWH5Ll5GMshrRQ7UPetVGIabpuQ7CHtBTFotgO3r+hZ+T7zpFtXtbdoJKUq94Ep8FJ +VEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737151411; x=1737756211; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=qBY741sXZOT0WTbq40IXRM3lwOB19DkD28587LMhXH4=; b=Qgv+8KshegHugYIfEdfm3MkV6jIqn0Xrj06MjMsgRfsvLzFVJfbDJQ1+dDlltIV9NR a+kn0/HAVLRGythAxmLVTk6VER8eymPh2hgSR1o2M1Sfeo3zSUmlzwyFdxADuTHH8t2f AQcNwCWEz3TpNpiSlL12hL7nDyr/2OpqynajXs3YxEbRYSW/cDskzkUKtnRhl9/8HwwG iDVgnLTzHC4BDbJcoI31NJRhvnpFQ3lXRPJlzberzvfzJ2zA6xGfHBhKuos7xMApsszD SHlSAN8LiOWtPPGO6wZDvIkLhswcCnnzb1KWF9BmPN5+L9aO71Do9boPA2jzmYEAtsnY WTcw== X-Gm-Message-State: AOJu0YxO6/9oA8pYz4KXx4amoI/osQJmXBME/2oLEU0cXNOQaxQ9VukZ k8LMEIMLu4q3O7QiDfZKo0jzvncGuNv26z8VL5f/sDX27My5ahZi2df9a1k91fATAUudu+/gaTQ cjWw= X-Gm-Gg: ASbGncu8xJMF27o7H5LrtYBxc2JROs5BdZ1s9yfXxsNy4RXGVPsrIMBHGUhE+zZKwGe 1AOjYyn1fIH1ZFUdZT/I2TBThmtFEen8PbHtMOu3sru7z8DPPz3JoE1VYPwSP+1BGJ3XyFxFjWj TexXeXP5UYe2u76vLJgQ+tHez50vvINQ2g8s8JfcGZdqvg9SGdkHbMUeZgTKCLJoIC2IgciIfGM zsWp5wucvOhL48w5tjlh7ZZYHi1/7W977+SNIpPRq9x1sT+lc34ZWhOjlLm2AakHkH7V63eDbTp QBw1s1rZMXVZgvvka09C5YJSvmGA X-Google-Smtp-Source: AGHT+IFP33FaCX8IgLV25j1L5/34Qn7HEW7TBou1Ft+JQSD5Hn4oL0LSqwqRbuFAlkQGwoEKFVH90Q== X-Received: by 2002:a05:6902:2683:b0:e57:2ff6:945a with SMTP id 3f1490d57ef6-e57b0e17ee2mr3796411276.0.1737151410951; Fri, 17 Jan 2025 14:03:30 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e57ab479729sm584524276.54.2025.01.17.14.03.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Jan 2025 14:03:30 -0800 (PST) Date: Fri, 17 Jan 2025 17:03:29 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v3 7/8] csum-file: introduce hashfile_checkpoint_init() Message-ID: <850d4f407dbb3999e194bf094d0fd7b95a9c29d8.1737151386.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: In 106140a99f (builtin/fast-import: fix segfault with unsafe SHA1 backend, 2024-12-30) and 9218c0bfe1 (bulk-checkin: fix segfault with unsafe SHA1 backend, 2024-12-30), we observed the effects of failing to initialize a hashfile_checkpoint with the same hash function implementation as is used by the hashfile it is used to checkpoint. While both 106140a99f and 9218c0bfe1 work around the immediate crash, changing the hash function implementation within the hashfile API to, for example, the non-unsafe variant would re-introduce the crash. This is a result of the tight coupling between initializing hashfiles and hashfile_checkpoints. Introduce and use a new function which ensures that both parts of a hashfile and hashfile_checkpoint pair use the same hash function implementation to avoid such crashes. A few things worth noting: - In the change to builtin/fast-import.c::stream_blob(), we can see that by removing the explicit reference to 'the_hash_algo->unsafe_init_fn()', we are hardened against the hashfile API changing away from the_hash_algo (or its unsafe variant) in the future. - The bulk-checkin code no longer needs to explicitly zero-initialize the hashfile_checkpoint, since it is now done as a result of calling 'hashfile_checkpoint_init()'. - Also in the bulk-checkin code, we add an additional call to prepare_to_stream() outside of the main loop in order to initialize 'state->f' so we know which hash function implementation to use when calling 'hashfile_checkpoint_init()'. This is OK, since subsequent 'prepare_to_stream()' calls are noops. However, we only need to call 'prepare_to_stream()' when we have the HASH_WRITE_OBJECT bit set in our flags. Without that bit, calling 'prepare_to_stream()' does not assign 'state->f', so we have nothing to initialize. - Other uses of the 'checkpoint' in 'deflate_blob_to_pack()' are appropriately guarded. Helped-by: Patrick Steinhardt Signed-off-by: Taylor Blau --- builtin/fast-import.c | 2 +- bulk-checkin.c | 9 ++++++--- csum-file.c | 7 +++++++ csum-file.h | 1 + 4 files changed, 15 insertions(+), 4 deletions(-) diff --git a/builtin/fast-import.c b/builtin/fast-import.c index 0f86392761a..4a6c7ab52ac 100644 --- a/builtin/fast-import.c +++ b/builtin/fast-import.c @@ -1106,7 +1106,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark) || (pack_size + PACK_SIZE_THRESHOLD + len) < pack_size) cycle_packfile(); - the_hash_algo->unsafe_init_fn(&checkpoint.ctx); + hashfile_checkpoint_init(pack_file, &checkpoint); hashfile_checkpoint(pack_file, &checkpoint); offset = checkpoint.offset; diff --git a/bulk-checkin.c b/bulk-checkin.c index 433070a3bda..892176d23d2 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -261,7 +261,7 @@ static int deflate_blob_to_pack(struct bulk_checkin_packfile *state, git_hash_ctx ctx; unsigned char obuf[16384]; unsigned header_len; - struct hashfile_checkpoint checkpoint = {0}; + struct hashfile_checkpoint checkpoint; struct pack_idx_entry *idx = NULL; seekback = lseek(fd, 0, SEEK_CUR); @@ -272,12 +272,15 @@ static int deflate_blob_to_pack(struct bulk_checkin_packfile *state, OBJ_BLOB, size); the_hash_algo->init_fn(&ctx); the_hash_algo->update_fn(&ctx, obuf, header_len); - the_hash_algo->unsafe_init_fn(&checkpoint.ctx); /* Note: idx is non-NULL when we are writing */ - if ((flags & HASH_WRITE_OBJECT) != 0) + if ((flags & HASH_WRITE_OBJECT) != 0) { CALLOC_ARRAY(idx, 1); + prepare_to_stream(state, flags); + hashfile_checkpoint_init(state->f, &checkpoint); + } + already_hashed_to = 0; while (1) { diff --git a/csum-file.c b/csum-file.c index ebffc80ef71..232121f415f 100644 --- a/csum-file.c +++ b/csum-file.c @@ -206,6 +206,13 @@ struct hashfile *hashfd_throughput(int fd, const char *name, struct progress *tp return hashfd_internal(fd, name, tp, 8 * 1024); } +void hashfile_checkpoint_init(struct hashfile *f, + struct hashfile_checkpoint *checkpoint) +{ + memset(checkpoint, 0, sizeof(*checkpoint)); + f->algop->init_fn(&checkpoint->ctx); +} + void hashfile_checkpoint(struct hashfile *f, struct hashfile_checkpoint *checkpoint) { hashflush(f); diff --git a/csum-file.h b/csum-file.h index 2b45f4673a2..b7475f16c20 100644 --- a/csum-file.h +++ b/csum-file.h @@ -36,6 +36,7 @@ struct hashfile_checkpoint { git_hash_ctx ctx; }; +void hashfile_checkpoint_init(struct hashfile *, struct hashfile_checkpoint *); void hashfile_checkpoint(struct hashfile *, struct hashfile_checkpoint *); int hashfile_truncate(struct hashfile *, struct hashfile_checkpoint *); From patchwork Fri Jan 17 22:03:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13944001 Received: from mail-yb1-f176.google.com (mail-yb1-f176.google.com [209.85.219.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 864FB1D86C7 for ; Fri, 17 Jan 2025 22:03:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151417; cv=none; b=gNAIkiozW9A4tY9X/gFXzOBTERxSpzifNcJ/xvU1trYPxLCloCskOh1v2uX3xipAdlgbfgGdYllfe9TX7Qi9C4kTkh5ku4BBn+qNc2OBmiLnKI78dOiENkunuttOPMgmm6eUMfzBDCZTjMBh9bQ+tcidgHGBKTg5OW2ZXsL0NyU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737151417; c=relaxed/simple; bh=Q+pHBQNslBienbX/Wz9J/TCqDekok0lDXoQcJ8Rhaiw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=nlGn9Pggd5vY2f9ZGnwFg8ZIbN/KNoUN5TboRnhAyXjx6M3YsqcaBpWtPqGDdawGwYkqCnTU/kpXVtci1AYRRMJliXKh376KEQJcWYjGZaEZbWU9/MZlWGFovlFpz3RrDmcroS58OBaSckmsHkiEpKljCaBNwf0P253xKt7+A5I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=vm+IiSQJ; arc=none smtp.client-ip=209.85.219.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="vm+IiSQJ" Received: by mail-yb1-f176.google.com with SMTP id 3f1490d57ef6-e545c1e8a15so4289927276.1 for ; Fri, 17 Jan 2025 14:03:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737151414; x=1737756214; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=xJIDRo5Ke0OvXwV3YjkKD7Fj8WeUZtCAkJVeWYEnzdY=; b=vm+IiSQJ6p198brt/uFPdOU/svbUBgdxcDc6xfq3aA4hldomlaQmkSvggQfHKU0M8e V2LQPA6rX4wAPWlqWi8uqS2Dqy7HBtu3JhvVIY4qm3o4R4UutaUVldB7tjSEkEeyzdzI 5He0qXHgEc364dHyaB+N9wtxnUNSapmwfHS0PNzGl0YvGOJRMz7z4/o7Q0lYgKcjRbqh xVN1VtKgiw4tASP7VICHUyCPEQ0JWCQpU0kiJS70ii8H6UkQMBD2sBoe/Dyy4KuFecrd OQVj6Jqlyjg2yElDJEgD0Atsyc6uOkyBCTW0uoDE5hS2yIw9aTZrMQtiFuT0emLrFmip fgeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737151414; x=1737756214; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=xJIDRo5Ke0OvXwV3YjkKD7Fj8WeUZtCAkJVeWYEnzdY=; b=KCjbR9O4Kq3ACqvsRBWCXb/xLd3m43KcpPcp+2Pz48VsGItqOLn3kMYrA9Zbysr9iX ZEe7yXRNfxJqzgXOuhoWucNYw6If6osQRnXDt51fLwGrxgh5dwV1cC6WjOa2QppgKP24 lutw8gkETpGG6PVUIrUNrrfY4AaI/OxI6PZaf91XlkINTGGuIGjcgtLGsSR4ywEYTrAV Ix5II59QGoTIrDjQmpUzsQdqhQavCAv2vHIQoWhUycZphvHpgX7rMNcNJjXSh7Ck5Zyz 1qH78UoPlCKvaTc787LD+bsQBsVmhrIK8AHZWerTx+yGKjQtPiJFCunTkDqhsvCR9D75 FVgQ== X-Gm-Message-State: AOJu0YwpbfrWG3PrB8nJbkMni0gAlfdRB725qLyT19j5woMJ1Y9DL9I9 dzykdD3lA88AqS9908kIzz6ZF4e3wvQ7/C/Hwnu9ULFgkhtUyWR0+5+lhye13/G29aAbyWFSfnR P4JU= X-Gm-Gg: ASbGncvgnzJfT9PBzfG78XKe5IoAcWOn3/PC+VO2oSVTCCI2nYdVASP034L12Oelhzj 2FIdC8AP+yfX8sPafEJaB28++3IMcNG7qMd1PTbP7muexUfNEFCmOpuxgt/XlBxyDjqnGLaGQwE dMYH9Grc+ThdgPNWhYqjddklu1DCK2nExnHPFHcI+21Ots872DqQ9Y1EaarnFJkxIByVRHCWUga WYuQkuCvhKCeoUXrhiqgCwgRnpXsh0LQOUbNyd6dN1ERKiPLuEZZc8Gj9d7C6Jrc+Wivu/3O3Xi rWW5IYt0OF1xAtiOqspgm9YCNPQ9 X-Google-Smtp-Source: AGHT+IEQrrT+pam3toALyYjhzlFUp9Oc21L0Dt+gAchRP6Pb0HP6Vhypt6qa+Y4JDMJ2oCxBr17/CA== X-Received: by 2002:a05:6902:2b11:b0:e57:90b7:e37d with SMTP id 3f1490d57ef6-e57b13afdc0mr2791313276.33.1737151414269; Fri, 17 Jan 2025 14:03:34 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e57ab479729sm584543276.54.2025.01.17.14.03.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Jan 2025 14:03:33 -0800 (PST) Date: Fri, 17 Jan 2025 17:03:32 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v3 8/8] hash.h: drop unsafe_ function variants Message-ID: <0c4d006e6e8005c10387d10fe0224aec5c5613e5.1737151386.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Now that all callers have been converted from: the_hash_algo->unsafe_init_fn(); to unsafe_hash_algo(the_hash_algo)->init_fn(); and similar, we can remove the scaffolding for the unsafe_ function variants and force callers to use the new unsafe_hash_algo() mechanic instead. Signed-off-by: Taylor Blau --- hash.h | 15 --------------- object-file.c | 15 --------------- 2 files changed, 30 deletions(-) diff --git a/hash.h b/hash.h index a68a2b6a161..68d4292e6da 100644 --- a/hash.h +++ b/hash.h @@ -282,21 +282,6 @@ struct git_hash_algo { /* The hash finalization function for object IDs. */ git_hash_final_oid_fn final_oid_fn; - /* The non-cryptographic hash initialization function. */ - git_hash_init_fn unsafe_init_fn; - - /* The non-cryptographic hash context cloning function. */ - git_hash_clone_fn unsafe_clone_fn; - - /* The non-cryptographic hash update function. */ - git_hash_update_fn unsafe_update_fn; - - /* The non-cryptographic hash finalization function. */ - git_hash_final_fn unsafe_final_fn; - - /* The non-cryptographic hash finalization function. */ - git_hash_final_oid_fn unsafe_final_oid_fn; - /* The OID of the empty tree. */ const struct object_id *empty_tree; diff --git a/object-file.c b/object-file.c index 43efa4ca361..c4b42dd4be9 100644 --- a/object-file.c +++ b/object-file.c @@ -230,11 +230,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { .update_fn = git_hash_unknown_update, .final_fn = git_hash_unknown_final, .final_oid_fn = git_hash_unknown_final_oid, - .unsafe_init_fn = git_hash_unknown_init, - .unsafe_clone_fn = git_hash_unknown_clone, - .unsafe_update_fn = git_hash_unknown_update, - .unsafe_final_fn = git_hash_unknown_final, - .unsafe_final_oid_fn = git_hash_unknown_final_oid, .empty_tree = NULL, .empty_blob = NULL, .null_oid = NULL, @@ -250,11 +245,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { .update_fn = git_hash_sha1_update, .final_fn = git_hash_sha1_final, .final_oid_fn = git_hash_sha1_final_oid, - .unsafe_init_fn = git_hash_sha1_init_unsafe, - .unsafe_clone_fn = git_hash_sha1_clone_unsafe, - .unsafe_update_fn = git_hash_sha1_update_unsafe, - .unsafe_final_fn = git_hash_sha1_final_unsafe, - .unsafe_final_oid_fn = git_hash_sha1_final_oid_unsafe, .unsafe = &sha1_unsafe_algo, .empty_tree = &empty_tree_oid, .empty_blob = &empty_blob_oid, @@ -271,11 +261,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { .update_fn = git_hash_sha256_update, .final_fn = git_hash_sha256_final, .final_oid_fn = git_hash_sha256_final_oid, - .unsafe_init_fn = git_hash_sha256_init, - .unsafe_clone_fn = git_hash_sha256_clone, - .unsafe_update_fn = git_hash_sha256_update, - .unsafe_final_fn = git_hash_sha256_final, - .unsafe_final_oid_fn = git_hash_sha256_final_oid, .empty_tree = &empty_tree_oid_sha256, .empty_blob = &empty_blob_oid_sha256, .null_oid = &null_oid_sha256,