From patchwork Thu Jan 23 17:34:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13948505 Received: from mail-yb1-f176.google.com (mail-yb1-f176.google.com [209.85.219.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4588149C51 for ; Thu, 23 Jan 2025 17:34:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653664; cv=none; b=OMc69qGU81pNF90Fo/hmWNf+Y/QSztU70OaSOrSDbd+CjLXx6tvS5wLUCNqN9YI4mY6S4V1MHk4jp0Kf2V2SAO0epuPYPXOZsVp+xLvtbDD58mJiXFNCe4gsXaYsMPuRtqQSxbSM0+9EFhNYpuQc/+mwBefj3Y40u/ovx6cc0II= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653664; c=relaxed/simple; bh=cEu5aB8gosLgPyDEAsZkmKTyVLh6W9D6E2/Qy9Bpc00=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=j8mlsi0wtiKh+0pk/A8WGsM5fIJlvwQp3Ok0OFfjjraxnDtlH8OI0y13wCjdh/Cd2P7WvIzRi3huTt04sn4rYog+xeetTEenGTvqcGUJXTuYdEt998cil+31JoGM6JZc8kuvfPQnFaQw0IZSkRaM4tuhwja8oo3a4gKa11dYIZo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=kjsSn7nH; arc=none smtp.client-ip=209.85.219.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="kjsSn7nH" Received: by mail-yb1-f176.google.com with SMTP id 3f1490d57ef6-e46ebe19368so1742713276.0 for ; Thu, 23 Jan 2025 09:34:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737653661; x=1738258461; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=QYUcXoDGE1VZui4RG1S9LRYAFXHI4AAnswfZspxdGi4=; b=kjsSn7nHI3eojoHmprCYNu+QqhxrpNohByVERCRd6TWQreuKtzHPaPR0Ae5zmp24wk YI3SXbV4ZeCmJfLQ4m1MRQ0Pa0YUzT56jBKoIsEyPUHTU7DZ+dNJper/VdBQqeYyoDzg Lx7gZxJUHbXNFFuwQ9K+He1CHJUJ24jTluKcEXX8xpG0g6Cu92ZTYZVJaMwsv/K62sWB wnPdlOwEP/qKlK8TVf1qSSPfc1qe4pQMVofPm01LspRlfeQQ+YyIuyO61kUOF5uPoBO2 G1nWbgmoydDyyx8JjBU+ba375EAuCNiHtfLyUyq3vpiPyf7nxNZaPqDtZI8i4TkTw+/C sWjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737653661; x=1738258461; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=QYUcXoDGE1VZui4RG1S9LRYAFXHI4AAnswfZspxdGi4=; b=vao+A/bhAH1aBZ9tglwAYGZ4bZ8YGz0TEn5MrWUj5fyLJGHdPI8DzWIBhp7xOZZyVg hxiFaPu/k1ogz/Mpz5yUstrmVb4qZermLm9z2RqJgqy4he+meXWEPSGMiQShkY4+mq0h du+CboHx5ObiTkWUxnNYC6bvgLqf8k0NQEgHX500HNJQY8UUWD4dwjGIjFWTxlQuQB3a xNox1kfT6R1tO1pihH2bdG3Tuuv2tzoSxRu+PMFqxz5nMh7mZ3yXvqs+SDxdgC4UpWoy FGCEqtTVFC6BkOLdg8/DGflq9eBuHVQHXdnMwOYofsOnuogkjSvOex2g1SwCAbMDyBS/ A6/A== X-Gm-Message-State: AOJu0YyzorSWVrYw1mmwdFwj0Jc8DtJ7TWFdTtBW0zqgADj2GK0gArit e2Hu8cdGJiZoEeibfABmjPcDPO9SVXKrtxJ/4Dq6MOlEDlfsSBv/Y7Q3AaXVIRl27E44oUSJsiv 3QMM= X-Gm-Gg: ASbGncuXUkf3k3yJ1hkqBHl2KGppiGGZ/KoOJF0vWVFqUqNyvN8s5ciBhVzzABTxUtj wiqY8scpBtXOVzSZlZf5O7+w93RFQ8dt3FeINLMeRoGg3uaJh9zZZ/48S87pderdEPukIbAjxTQ ljy2sAcx56clSYkTu54rHmNdheEzfWZ/ppCsp8IMdd9rAV1qXMj9TTNt7kQUvJuYXQUPr1gOd3C J8BAdyoQdMl+F2+zbD4SzlPf/r6vNk1JnyKd04JlFIIJGud/aNlwf2t4FjTZu77j11TtUJolKZx BzTHHPEcypR4vS0UsBoThDWDJdE4Mbx7cAPi6ZbQZE6PYbB+qZoh X-Google-Smtp-Source: AGHT+IHWQ3hYy/maz14YmeGzgS3SUoheE68H2u6Smuk0stJj7KH6gyNa4bur/eoVa9u/qJll7p80/Q== X-Received: by 2002:a05:690c:6e07:b0:6ef:6a71:aa55 with SMTP id 00721157ae682-6f6eb51a3b4mr217935697b3.0.1737653661254; Thu, 23 Jan 2025 09:34:21 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f757a2de28sm442807b3.111.2025.01.23.09.34.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jan 2025 09:34:20 -0800 (PST) Date: Thu, 23 Jan 2025 12:34:19 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v4 1/8] t/helper/test-tool: implement sha1-unsafe helper Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: With the new "unsafe" SHA-1 build knob, it is convenient to have a test-tool that can exercise Git's unsafe SHA-1 wrappers for testing, similar to 't/helper/test-tool sha1'. Implement that helper by altering the implementation of that test-tool (in cmd_hash_impl(), which is generic and parameterized over different hash functions) to conditionally run the unsafe variants of the chosen hash function, and expose the new behavior via a new 'sha1-unsafe' test helper. Signed-off-by: Taylor Blau --- t/helper/test-hash.c | 17 +++++++++++++---- t/helper/test-sha1.c | 7 ++++++- t/helper/test-sha1.sh | 38 ++++++++++++++++++++++---------------- t/helper/test-sha256.c | 2 +- t/helper/test-tool.c | 1 + t/helper/test-tool.h | 3 ++- 6 files changed, 45 insertions(+), 23 deletions(-) diff --git a/t/helper/test-hash.c b/t/helper/test-hash.c index 45d829c908f..d0ee668df95 100644 --- a/t/helper/test-hash.c +++ b/t/helper/test-hash.c @@ -1,7 +1,7 @@ #include "test-tool.h" #include "hex.h" -int cmd_hash_impl(int ac, const char **av, int algo) +int cmd_hash_impl(int ac, const char **av, int algo, int unsafe) { git_hash_ctx ctx; unsigned char hash[GIT_MAX_HEXSZ]; @@ -27,7 +27,10 @@ int cmd_hash_impl(int ac, const char **av, int algo) die("OOPS"); } - algop->init_fn(&ctx); + if (unsafe) + algop->unsafe_init_fn(&ctx); + else + algop->init_fn(&ctx); while (1) { ssize_t sz, this_sz; @@ -46,9 +49,15 @@ int cmd_hash_impl(int ac, const char **av, int algo) } if (this_sz == 0) break; - algop->update_fn(&ctx, buffer, this_sz); + if (unsafe) + algop->unsafe_update_fn(&ctx, buffer, this_sz); + else + algop->update_fn(&ctx, buffer, this_sz); } - algop->final_fn(hash, &ctx); + if (unsafe) + algop->unsafe_final_fn(hash, &ctx); + else + algop->final_fn(hash, &ctx); if (binary) fwrite(hash, 1, algop->rawsz, stdout); diff --git a/t/helper/test-sha1.c b/t/helper/test-sha1.c index e60d000c039..349540c4df8 100644 --- a/t/helper/test-sha1.c +++ b/t/helper/test-sha1.c @@ -3,7 +3,7 @@ int cmd__sha1(int ac, const char **av) { - return cmd_hash_impl(ac, av, GIT_HASH_SHA1); + return cmd_hash_impl(ac, av, GIT_HASH_SHA1, 0); } int cmd__sha1_is_sha1dc(int argc UNUSED, const char **argv UNUSED) @@ -13,3 +13,8 @@ int cmd__sha1_is_sha1dc(int argc UNUSED, const char **argv UNUSED) #endif return 1; } + +int cmd__sha1_unsafe(int ac, const char **av) +{ + return cmd_hash_impl(ac, av, GIT_HASH_SHA1, 1); +} diff --git a/t/helper/test-sha1.sh b/t/helper/test-sha1.sh index 84594885c70..bf387d3db14 100755 --- a/t/helper/test-sha1.sh +++ b/t/helper/test-sha1.sh @@ -3,25 +3,31 @@ dd if=/dev/zero bs=1048576 count=100 2>/dev/null | /usr/bin/time t/helper/test-tool sha1 >/dev/null +dd if=/dev/zero bs=1048576 count=100 2>/dev/null | +/usr/bin/time t/helper/test-tool sha1-unsafe >/dev/null + while read expect cnt pfx do case "$expect" in '#'*) continue ;; esac - actual=$( - { - test -z "$pfx" || echo "$pfx" - dd if=/dev/zero bs=1048576 count=$cnt 2>/dev/null | - perl -pe 'y/\000/g/' - } | ./t/helper/test-tool sha1 $cnt - ) - if test "$expect" = "$actual" - then - echo "OK: $expect $cnt $pfx" - else - echo >&2 "OOPS: $cnt" - echo >&2 "expect: $expect" - echo >&2 "actual: $actual" - exit 1 - fi + for sha1 in sha1 sha1-unsafe + do + actual=$( + { + test -z "$pfx" || echo "$pfx" + dd if=/dev/zero bs=1048576 count=$cnt 2>/dev/null | + perl -pe 'y/\000/g/' + } | ./t/helper/test-tool $sha1 $cnt + ) + if test "$expect" = "$actual" + then + echo "OK ($sha1): $expect $cnt $pfx" + else + echo >&2 "OOPS ($sha1): $cnt" + echo >&2 "expect ($sha1): $expect" + echo >&2 "actual ($sha1): $actual" + exit 1 + fi + done done < X-Patchwork-Id: 13948506 Received: from mail-yb1-f177.google.com (mail-yb1-f177.google.com [209.85.219.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E368156228 for ; Thu, 23 Jan 2025 17:34:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653667; cv=none; b=VC+fvyPyzmQr8w+1X1KSBPM6IgWf6WM7fdW9N2/wMm3MT2BlKPoQDicLidRgdW51TAmE55I5Kb6a3wHFPzGRwJ941u15Z8TRMHkiex0x/qF+W9xYCSKeSdSU+rjMPcnuTRj+4b++w46LJ9vi/C9D1E2zE2qdHdaMEYnuXyae2cA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653667; c=relaxed/simple; bh=UQ83yp+q3k6rgPWWWwZQz6/MGKIShYG7JE9HOegkIHM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=qVSSofEnqyVvdG9u67ClVm1rHXXf/RjjT0Wfso9yA+d+flyFhjEFOQWaC8WsuHV8Ye7WxbV2N5npaEICARFXnMcqeNX1GbqPy/WZCIa5ITlwaq5dX1aZduV5PJIweEUdmqgcv0FGB1x15m+FuWLji/2XQYBMW6idT2qYxnA/Cis= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=DJvXW+M1; arc=none smtp.client-ip=209.85.219.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="DJvXW+M1" Received: by mail-yb1-f177.google.com with SMTP id 3f1490d57ef6-e53ef7462b6so2001270276.3 for ; Thu, 23 Jan 2025 09:34:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737653665; x=1738258465; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=dSIVIjVOLWmCFq1Gaxvc2NuA2Lkh9yI4zKMvBPhFhFI=; b=DJvXW+M15Wg7inaCTMsG4Nf/ul4R8Z85XNbKrJNFp4KMXAuZ3/dKAZ/Plt/2VIXeUk GgGzQEQoFFICHOq2479craFrmpiSktJXAPLLoQ2JiugURij0WYCSzbJEwvfOxPS7grfQ Y4ZIevN8Fri9CjC+RpCETkuJ4Hf8xXoaDIGf9500hUWTnqVzxv4VQl7wu5APuaWHtPFS WNCKOqZ7zf2KeTQ+hV/8joAx4lA3Pr6+ilRw26AgcUl4dKCCOgHMHyzuJZiExXQ1Ht/0 zyNL8RbrvemAWFoTMdweooQMiyHk1xgxVLnlbKxntdLgl7d2aSx4KjcIENG3+n809ItD Lxmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737653665; x=1738258465; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=dSIVIjVOLWmCFq1Gaxvc2NuA2Lkh9yI4zKMvBPhFhFI=; b=XZcB/fCjcxH8L5JlwdfGuASCiokzhRxy9oSAoFTyaS1fdRDXmHgiaMppOlUC9oUmuU OuTTfY7fvp8cB0V+KiQMNzgG2eSnxvV/zcIL4Lp9PsSOjMZhhNYsEQgb7ugBKnh5tuj2 26JeN2mCD7xcOW/kDzXGdZCO3QoCsvp57meEDgeRxOaQEWvAXe85A/R8ybHnQpz/b6ru 4aeeu8i+nCUx5ib9HMDEpYJQtb7qiWMVbZgLWEfVFa/3ngXku39zmlco+4TnkrmOC+Hp 7UmrFo5yj/AK8twmR5JpCWvxWMlxiLsnIa1HRecJ0H/0R/3GXV6CJRwCBfrO14y5CtDd GH+w== X-Gm-Message-State: AOJu0YzjlbXeBD0ipo6Okmt/jvkvqNrdu8SflBWvs3RvZX3HXz21uJ8J hLQZlNuQ8HXepzCzhvD0uXUZQTZyYB9/QrQkmFpzeU6ZpCK5+uRrZccsjoocuJK8cf5OV5Oxu0B /EcQ= X-Gm-Gg: ASbGncvVKpwZvKTCSbh3XV1SPwBV9TDHoZ6T+bllrhUlNVWj8an4MJUnVaDkLgS7znG 72x4D9NBITc6pjL4Rl7Mn4FQn5VuG/VVEWQE3MZfl2xIsnxHwoLYaEIgFUJOMQjyzLJHIWpJCm8 aPKqp71ezl8Ng8bHyiwa6nq/XncVJ7pu7forYWw7iuz71hGoWeUt5ScTTKy7+VCMO5Dafr8wHdc 3a5IfPEl/ytjc7t/ZL/GRObZBoSmZp38dD/DgSBb51x0B+SASkrJEwfrlcLd0ms/Aycfibw3AZ0 0D0fqSM8bhKfHhHarnmtDS2hCF9eYLPvVucY8Mz+qL/XsrGcOTyP X-Google-Smtp-Source: AGHT+IH3i4n3dMuIm1W2rBn5X1cS1l3y/17HJaIIVWm25Z9tZ+JjWduDLx4VrmdNNkBCR2Jcq26cuQ== X-Received: by 2002:a05:690c:2506:b0:6f7:5868:86ea with SMTP id 00721157ae682-6f758688a75mr1967467b3.8.1737653664787; Thu, 23 Jan 2025 09:34:24 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f7578a1cdasm477557b3.53.2025.01.23.09.34.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jan 2025 09:34:24 -0800 (PST) Date: Thu, 23 Jan 2025 12:34:23 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v4 2/8] csum-file: store the hash algorithm as a struct field Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Throughout the hashfile API, we rely on a reference to 'the_hash_algo', and call its _unsafe function variants directly. Prepare for a future change where we may use a different 'git_hash_algo' pointer (instead of just relying on 'the_hash_algo' throughout) by making the 'git_hash_algo' pointer a member of the 'hashfile' structure itself. Signed-off-by: Taylor Blau --- csum-file.c | 20 +++++++++++--------- csum-file.h | 1 + 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/csum-file.c b/csum-file.c index 5716016e12e..b28cd047e3f 100644 --- a/csum-file.c +++ b/csum-file.c @@ -50,7 +50,7 @@ void hashflush(struct hashfile *f) if (offset) { if (!f->skip_hash) - the_hash_algo->unsafe_update_fn(&f->ctx, f->buffer, offset); + f->algop->unsafe_update_fn(&f->ctx, f->buffer, offset); flush(f, f->buffer, offset); f->offset = 0; } @@ -71,14 +71,14 @@ int finalize_hashfile(struct hashfile *f, unsigned char *result, hashflush(f); if (f->skip_hash) - hashclr(f->buffer, the_repository->hash_algo); + hashclr(f->buffer, f->algop); else - the_hash_algo->unsafe_final_fn(f->buffer, &f->ctx); + f->algop->unsafe_final_fn(f->buffer, &f->ctx); if (result) - hashcpy(result, f->buffer, the_repository->hash_algo); + hashcpy(result, f->buffer, f->algop); if (flags & CSUM_HASH_IN_STREAM) - flush(f, f->buffer, the_hash_algo->rawsz); + flush(f, f->buffer, f->algop->rawsz); if (flags & CSUM_FSYNC) fsync_component_or_die(component, f->fd, f->name); if (flags & CSUM_CLOSE) { @@ -128,7 +128,7 @@ void hashwrite(struct hashfile *f, const void *buf, unsigned int count) * f->offset is necessarily zero. */ if (!f->skip_hash) - the_hash_algo->unsafe_update_fn(&f->ctx, buf, nr); + f->algop->unsafe_update_fn(&f->ctx, buf, nr); flush(f, buf, nr); } else { /* @@ -174,7 +174,9 @@ static struct hashfile *hashfd_internal(int fd, const char *name, f->name = name; f->do_crc = 0; f->skip_hash = 0; - the_hash_algo->unsafe_init_fn(&f->ctx); + + f->algop = the_hash_algo; + f->algop->unsafe_init_fn(&f->ctx); f->buffer_len = buffer_len; f->buffer = xmalloc(buffer_len); @@ -208,7 +210,7 @@ void hashfile_checkpoint(struct hashfile *f, struct hashfile_checkpoint *checkpo { hashflush(f); checkpoint->offset = f->total; - the_hash_algo->unsafe_clone_fn(&checkpoint->ctx, &f->ctx); + f->algop->unsafe_clone_fn(&checkpoint->ctx, &f->ctx); } int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint) @@ -219,7 +221,7 @@ int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint lseek(f->fd, offset, SEEK_SET) != offset) return -1; f->total = offset; - the_hash_algo->unsafe_clone_fn(&f->ctx, &checkpoint->ctx); + f->algop->unsafe_clone_fn(&f->ctx, &checkpoint->ctx); f->offset = 0; /* hashflush() was called in checkpoint */ return 0; } diff --git a/csum-file.h b/csum-file.h index 7c73da0a40a..2b45f4673a2 100644 --- a/csum-file.h +++ b/csum-file.h @@ -20,6 +20,7 @@ struct hashfile { size_t buffer_len; unsigned char *buffer; unsigned char *check_buffer; + const struct git_hash_algo *algop; /** * If non-zero, skip_hash indicates that we should From patchwork Thu Jan 23 17:34:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13948507 Received: from mail-yb1-f181.google.com (mail-yb1-f181.google.com [209.85.219.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6836B1581E1 for ; Thu, 23 Jan 2025 17:34:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653671; cv=none; b=eQRlynxmsSSNw7C8QYb4HfgZ6JV39URrKxmhDUb+4xnTxkLSrWq+E8Yo8TMZepuEEpGtcZuflzdGXkrbUzfqBn0fNK3t0S30lunrfWFwZ+MrlEpiO+tctS1zAfgRLqLBNezlMQBq2kI2HD8/VKHhAf4PdwBJtZa9LVGovzdjAv0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653671; c=relaxed/simple; bh=Wtrq82Dmo+EUjsd8WWGRiJGHR+KqEWsJQXT3dd2gY0E=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=bodnE1YPRCfbqEeCD504w3820bac0vBPKKNxi7f5JnSgaun6lnPtfSJLD2Jm8of58OT9+m2x84yP7d65OJdqNmA8TNFBeMLJnwviLzkVAvAXZnpmar9mO4Yo7k8rMtsfecsoJ7qMsM8brkMvBF1SpEMksN8bXsjQ1CsUm0394BY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=zqDhpjQW; arc=none smtp.client-ip=209.85.219.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="zqDhpjQW" Received: by mail-yb1-f181.google.com with SMTP id 3f1490d57ef6-e398484b60bso1892240276.1 for ; Thu, 23 Jan 2025 09:34:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737653668; x=1738258468; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=XIAuF2Rpp2rrzkoZXCrVOsrlLB1cwByQdFvBqaOvOVw=; b=zqDhpjQWDGx5qYGoK3fde4oh79sW0ogcTJX/QC7ZkqswbF/3yB3jZvAf6gNfvKi2Dq TYeDP0t0fyoQjzGs7k3dZzkuDXRqhhkX3y/0pmkUct6PYA96Gm6cc3amE9IvRMhkWSzY rKUzTbu4kmW/vH4ARA2T4UA7Nu4ju37JetF6oqi1ipii39Tzxg4UQ44nD8LhVeWd06D1 AfCgHVddUCXAErgrQ5uiJCEKc478lLlkxZ3pkOk85038HoJUv1TWkTGgVRPXoUbLNYrL HbtgeXcPB65k5efNsg7/kCgZeFqZu5ag5uiSxACA75xCNP/+qv1FfVdYxWOD7sBgnRhg e6dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737653668; x=1738258468; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=XIAuF2Rpp2rrzkoZXCrVOsrlLB1cwByQdFvBqaOvOVw=; b=CP6JU6RcTzKFR9DYoi4atYFJk37M3ijQ5rmub+m67TvJrRPvMOuYFZa6FTyTUnrCxm nBq2psxcJwFYLsBFkOZQQdgS+xZMBP7ffRmO8Osh9C6CxVojsOMj+EcGxz94dGlGdCYH r8RvwTzmr8uNMLuAzhqosfDB0vOMi/FniQctsFUnXsFFiR7g+ck9axWkDXIPDIM6kmG4 v8LUpq3FnCDGDMQIMZB5/XH1wDbYaCY8QpVdd4pdA1fK377wifFi3MOc/QIUW0oRIpqA NwP53vpGk1z7GUaXXBT1adSOHZfVcSlZ7dFTYhd2r2G0L8qeJRylsz+cWnVulwwb07xu 98NQ== X-Gm-Message-State: AOJu0Yzl8RK76STa4UprMdP4uziKy5sErWUZSzKMGjmKd62VAh6XY1Nq uNiBAAauEVVAuXLKSQZv2x3U6HKaYSrWIYjBAuz+rDI0cHixpstZqV/UhXNhP0p8/IO/NgYjSzk PoFo= X-Gm-Gg: ASbGncuJQghTNJdVmIkqpitGC2FDaw9HzlSTo7OPRl72g3u31p/hGeyNLC0wD+GCSDm 0wjDre8rxR2OKzVYmqCV63vFYEttlHslhQoEFJiPjOAYyHK3kH+766Y5IcED96eQgEBPfu71Hhd +9nYWC/OAmwpKWCgpljSzE643vi7MljZeyHJIpyMXzB4OBvLrQPiG9BbRcaVy0dbh1xkCn7TIyN bIrvyDjrrBe7eqrmzmNH4IYNTAJvz5K81veONQBIm3b3zQly2kGmVhWyHixmJZj5Sj5KoRzXzlv 2BmZyjrDPFxXo9mXCkt6mqDNV/bkqVJahtQVQxrfQArcWlFbcy/L X-Google-Smtp-Source: AGHT+IExUx/u/z+9paHConXsWTCw1RwCUXl3iXcNj+PW48g0zIbVz8cYVcn4T2DcJ7P9ATLpGkcnzg== X-Received: by 2002:a05:6902:2683:b0:e57:2ff6:945a with SMTP id 3f1490d57ef6-e57b0e17ee2mr20520143276.0.1737653668070; Thu, 23 Jan 2025 09:34:28 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e583b7669b7sm32849276.14.2025.01.23.09.34.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jan 2025 09:34:27 -0800 (PST) Date: Thu, 23 Jan 2025 12:34:26 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v4 3/8] csum-file.c: extract algop from hashfile_checksum_valid() Message-ID: <73554c3b8814b4864a53469e6e20c2170a9f58ec.1737653640.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Perform a similar transformation as in the previous commit, but focused instead on hashfile_checksum_valid(). This function does not work with a hashfile structure itself, and instead validates the raw contents of a file written using the hashfile API. We'll want to be prepared for a similar change to this function in the future, so prepare ourselves for that by extracting 'the_hash_algo' into its own field for use within this function. Signed-off-by: Taylor Blau --- csum-file.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/csum-file.c b/csum-file.c index b28cd047e3f..7a71121e340 100644 --- a/csum-file.c +++ b/csum-file.c @@ -242,14 +242,15 @@ int hashfile_checksum_valid(const unsigned char *data, size_t total_len) { unsigned char got[GIT_MAX_RAWSZ]; git_hash_ctx ctx; - size_t data_len = total_len - the_hash_algo->rawsz; + const struct git_hash_algo *algop = the_hash_algo; + size_t data_len = total_len - algop->rawsz; - if (total_len < the_hash_algo->rawsz) + if (total_len < algop->rawsz) return 0; /* say "too short"? */ - the_hash_algo->unsafe_init_fn(&ctx); - the_hash_algo->unsafe_update_fn(&ctx, data, data_len); - the_hash_algo->unsafe_final_fn(got, &ctx); + algop->unsafe_init_fn(&ctx); + algop->unsafe_update_fn(&ctx, data, data_len); + algop->unsafe_final_fn(got, &ctx); - return hasheq(got, data + data_len, the_repository->hash_algo); + return hasheq(got, data + data_len, algop); } From patchwork Thu Jan 23 17:34:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13948508 Received: from mail-yb1-f175.google.com (mail-yb1-f175.google.com [209.85.219.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2024A1581E1 for ; Thu, 23 Jan 2025 17:34:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653674; cv=none; b=L5h9WxepUGTRzOMG9qUm5vNWaViWlHhitMRD/XHbIOLPvNb8Jy+6XFSswqJKObTnDIx/q0ONVxK+6wNbZBTPbpiatmzMhYhx3EbddFHGIwoQGKTzi3HVpZPhO0i/p5sZJilKsKzivvggpZxFdJdCq/kNkBnKX+9pk8tEzF+8QaA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653674; c=relaxed/simple; bh=tKuokJvo/WLcpKuTv5D85uir6Kce7cwhYiN7194yR2A=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=OVMaJPxMPexJYudiWXlR9o06P2xNvQbNtRx7qz0TFl4/jbw3lfuOs9Vdj7+YVBby6BGTDZeflApiqdexbhdVKI+n5GcZU0tjlN656VJzvPFcmk8Rw87NsefJpIV+UktvvCeQmpXH7d71L1oUfVyK6ElAHqDzWMTo2ALG4J2YTB8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=2wi79pxU; arc=none smtp.client-ip=209.85.219.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="2wi79pxU" Received: by mail-yb1-f175.google.com with SMTP id 3f1490d57ef6-e573136107bso2013470276.3 for ; Thu, 23 Jan 2025 09:34:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737653672; x=1738258472; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yoym53bWBwApkNdptCke/Beu/+I3YNdcEMeZRNHNY5Q=; b=2wi79pxUfBuvyEIsKFQbqTygP6Sp1TM8zjfGsOrYWFXDLPDEdl/DYkkgHx0CMLhBuh mGTrM1c2yXmK819m6VxLXmsUXntWyN+VGaYx9dH7NdVrfJqYg0Yea6AZuE3LLAzwqWTg rWu56FuJFRBGCBgKwpM40P+7wYlT1Ic+73BM0ygWUo0h2sn2P1WQHxcfLFgnnlOdnEtC b/cw8iGN1PL+oqpUV/xi/aWJYso5phGXrIMe7773OlJ76UP4THVX2z5EESbkiB+cntpb GbzYjt7UulcZq5cfpVU6n6FesZ/BVARV00vXV7UUTyjnkANAuR06GcSvYHd8P2A5iudj FiHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737653672; x=1738258472; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=yoym53bWBwApkNdptCke/Beu/+I3YNdcEMeZRNHNY5Q=; b=o6Y3JG/b+/hjG1P1Ome1kPI/U5Dx/7WrE5Y5kkDFHf5DA/0ofdoKaucpreXd7rzyyU lPNsYW93iAL3B3deVqJnDLwwKlBsrRv2osSIVTbuPHJTX21elGQXiAmuS+A54rQaNc/A oGNFKripT2prPuumkaDt+5GnyQ0oP+xNKdszbbev6rqfKeGeoM/La6TeKzmxroTO4mT5 ixKpSBq2vW1eNfVo8AH8SLgy5ocql/HAmhXsF/XfprjwnEkdaoBn22Lqj0F4ukoKzzmO ioQ5E6+Zrj5pRP8wsKltXMSy24JFCCS7JLts40GmGFOfjPBwCXq4nZzji7W99FRHgJyR Bi0g== X-Gm-Message-State: AOJu0YwBf54fCzquC6gxG6+6D2/Vgd4/tZu+HJDq2a4+AajsdWKpUwPT lkng+ARQLs+CZ8r5onRoXqvxE7FTWBV7uhBa0hP9cIGS5WGXlSwjlGeTqVIFHq9yIB79/mB+SWa Ms3k= X-Gm-Gg: ASbGnctU9/JkbBY21QpiVFR8rvmUqpSCmYS+Re9oHT0nOJcDR1Q8Ys17bS6MmWfwtNl MtW6zel15pK1gmEaUUoyiNpfoJxTzXqrLiZQxc2/Sa0r0hu2pLwVQftyJFHs7PLmlzibkj2J61D MDkeeT8tN7cOxor1BKDik075duk0B2sdhms3grurioc2TxflN/sqFidr47q19mRE8Ww01H3VwpZ Qt6h8Uergah0zIHCSKQVUkiiHv4HthsV5XCnJLfqrvPZMNkREw07Gsm3fsU/3RxkLKvQaOp8i3H to3Csn3OfTZRUnHiIQRckF9fQgdqNRLJhp5t7QPwQX0VInRN5rnz X-Google-Smtp-Source: AGHT+IFnwGFCkFn2XUEDC56BhtTWMSGuTd0kKDsXkLTcaK2Zei/T8kxFaaI2HaZGPNtYWBMjdjrw0g== X-Received: by 2002:a05:690c:4c13:b0:6ef:8e0b:8c63 with SMTP id 00721157ae682-6f6eb677dddmr205259517b3.14.1737653671595; Thu, 23 Jan 2025 09:34:31 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f75787710dsm487167b3.4.2025.01.23.09.34.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jan 2025 09:34:30 -0800 (PST) Date: Thu, 23 Jan 2025 12:34:29 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v4 4/8] hash.h: introduce `unsafe_hash_algo()` Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: In 253ed9ecff (hash.h: scaffolding for _unsafe hashing variants, 2024-09-26), we introduced "unsafe" variants of the SHA-1 hashing functions by introducing new functions like "unsafe_init_fn()" and so on. This approach has a major shortcoming that callers must remember to consistently use one variant or the other. Failing to consistently use (or not use) the unsafe variants can lead to crashes at best, or subtle memory corruption issues at worst. In the hashfile API, this isn't difficult to achieve, but verifying that all callers consistently use the unsafe variants is somewhat of a chore given how spread out all of the callers are. In the sha1 and sha1-unsafe test helpers, all of the calls to various hash functions are guarded by an "if (unsafe)" conditional, which is repetitive and cumbersome. Address these issues by introducing a new pattern whereby one 'git_hash_algo' can return a pointer to another 'git_hash_algo' that represents the unsafe version of itself. So instead of having something like: if (unsafe) the_hash_algo->init_fn(...); the_hash_algo->update_fn(...); the_hash_algo->final_fn(...); else the_hash_algo->unsafe_init_fn(...); the_hash_algo->unsafe_update_fn(...); the_hash_algo->unsafe_final_fn(...); we can instead write: struct git_hash_algo *algop = the_hash_algo; if (unsafe) algop = unsafe_hash_algo(algop); algop->init_fn(...); algop->update_fn(...); algop->final_fn(...); This removes the existing shortcoming by no longer forcing the caller to "remember" which variant of the hash functions it wants to call, only to hold onto a 'struct git_hash_algo' pointer that is initialized once. Similarly, while there currently is still a way to "mix" safe and unsafe functions, this too will go away after subsequent commits remove all direct calls to the unsafe_ variants. Note that hash_algo_by_ptr() needs an adjustment to allow passing in the unsafe variant of a hash function. All other query functions on the hash_algos array will continue to return the safe variants of any function. Suggested-by: Jeff King Signed-off-by: Taylor Blau --- hash.h | 13 ++++++++++++- object-file.c | 26 ++++++++++++++++++++++++++ 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/hash.h b/hash.h index 756166ce5e8..0bf63cedfa4 100644 --- a/hash.h +++ b/hash.h @@ -305,6 +305,9 @@ struct git_hash_algo { /* The all-zeros OID. */ const struct object_id *null_oid; + + /* The unsafe variant of this hash function, if one exists. */ + const struct git_hash_algo *unsafe; }; extern const struct git_hash_algo hash_algos[GIT_HASH_NALGOS]; @@ -320,9 +323,17 @@ int hash_algo_by_length(int len); /* Identical, except for a pointer to struct git_hash_algo. */ static inline int hash_algo_by_ptr(const struct git_hash_algo *p) { - return p - hash_algos; + size_t i; + for (i = 0; i < GIT_HASH_NALGOS; i++) { + const struct git_hash_algo *algop = &hash_algos[i]; + if (p == algop) + return i; + } + return GIT_HASH_UNKNOWN; } +const struct git_hash_algo *unsafe_hash_algo(const struct git_hash_algo *algop); + const struct object_id *null_oid(void); static inline int hashcmp(const unsigned char *sha1, const unsigned char *sha2, const struct git_hash_algo *algop) diff --git a/object-file.c b/object-file.c index 5b792b3dd42..43efa4ca361 100644 --- a/object-file.c +++ b/object-file.c @@ -202,6 +202,22 @@ static void git_hash_unknown_final_oid(struct object_id *oid UNUSED, BUG("trying to finalize unknown hash"); } +static const struct git_hash_algo sha1_unsafe_algo = { + .name = "sha1", + .format_id = GIT_SHA1_FORMAT_ID, + .rawsz = GIT_SHA1_RAWSZ, + .hexsz = GIT_SHA1_HEXSZ, + .blksz = GIT_SHA1_BLKSZ, + .init_fn = git_hash_sha1_init_unsafe, + .clone_fn = git_hash_sha1_clone_unsafe, + .update_fn = git_hash_sha1_update_unsafe, + .final_fn = git_hash_sha1_final_unsafe, + .final_oid_fn = git_hash_sha1_final_oid_unsafe, + .empty_tree = &empty_tree_oid, + .empty_blob = &empty_blob_oid, + .null_oid = &null_oid_sha1, +}; + const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { { .name = NULL, @@ -239,6 +255,7 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { .unsafe_update_fn = git_hash_sha1_update_unsafe, .unsafe_final_fn = git_hash_sha1_final_unsafe, .unsafe_final_oid_fn = git_hash_sha1_final_oid_unsafe, + .unsafe = &sha1_unsafe_algo, .empty_tree = &empty_tree_oid, .empty_blob = &empty_blob_oid, .null_oid = &null_oid_sha1, @@ -305,6 +322,15 @@ int hash_algo_by_length(int len) return GIT_HASH_UNKNOWN; } +const struct git_hash_algo *unsafe_hash_algo(const struct git_hash_algo *algop) +{ + /* If we have a faster "unsafe" implementation, use that. */ + if (algop->unsafe) + return algop->unsafe; + /* Otherwise use the default one. */ + return algop; +} + /* * This is meant to hold a *small* number of objects that you would * want repo_read_object_file() to be able to return, but yet you do not want From patchwork Thu Jan 23 17:34:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13948509 Received: from mail-yb1-f173.google.com (mail-yb1-f173.google.com [209.85.219.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 050F215CD41 for ; Thu, 23 Jan 2025 17:34:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653677; cv=none; b=iwx/Y4fsiB6sK9DlVb2DMSydLGjszDxM7f2DJu4KA2MJQO6rquXzwBmKY4spIVB0CiyGUpYmiKWw753Kz+5VR2Xv9/ODkkzWHn7NgfzkIm7XEjfX0g83mJBbT/0OXdWaDR4ZrDLcR6RgSfd3Z9hp1vXs+0+FaifvEDAKxPw4ldk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653677; c=relaxed/simple; bh=Q6l+NBYTp6zJ75VFRnmoiRxGhf5iSth8xzw3xRCUxd4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=l6mgzcb87vjsxJD67AW7kmRFZKIeh9TnHiFZ9hmrU+BtnVoi/qKzeOc6YQSCMorQkkDLgP7og8pAgrYvbg7gBlIiMGX1IvvwLt1TFZJ6IJ85aikQ4m7PaQ8BMyW28J6tMHv1m/wp8grsNxun1uE+5RmFYueGFB0IKGQEOmuUoyo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=aiYx6m/y; arc=none smtp.client-ip=209.85.219.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="aiYx6m/y" Received: by mail-yb1-f173.google.com with SMTP id 3f1490d57ef6-e5447fae695so2150029276.2 for ; Thu, 23 Jan 2025 09:34:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737653674; x=1738258474; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=gT7XGZHCsVln4HE+Q6fBXpLgxkC3FR/cb3zgeiUKGbA=; b=aiYx6m/yNaH7bOzzDKs+J8xL/CuHeKgumayPXNEb/73mwcWH/eWGMU0Wqw6IRmJF+M pgWLYru3UpE6dMPq2wjjBdbtos7qgMFd5zICFJsmYouUAkjCOms4RXQu1kUOFhG402FH GinUAGD/MFMmPriWO8aYCgN7abJ3LJtQL/MfokgQweSQVxKvWIs6fZExgOjemKoI5PLc cIJ/KwGRV2j6G2rVQ7FD7tuBa/4FJWCvddzIvbZaZJ7FvMMJzZvw58DkDIC5zcJGobg9 4leV2uyq7rcGmG1CaJM3bE87exk4296Nh5qNQQY/6wOA09SOBBPFNLrk3cYePYwMh5V3 +xFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737653674; x=1738258474; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=gT7XGZHCsVln4HE+Q6fBXpLgxkC3FR/cb3zgeiUKGbA=; b=Rcgj43BedjFJkSlJQuy8K0u2rTgC32zy38iCwIarhPk5Oj09/wFFcbWG95AlbiJMFO gWo5TNb2oYZ8022gIWtCl3sAEz/764rqyPrK7sUZrmbA5ktmZuoVkg2cdzG6CXrZLVVI 21U7Rr/9toIfPaqPR71xeo7h1+/tvEKT+WgFv9AFUACrCS+G/mXHoFrc0LV2lZ9jlBO4 4xbiDN+fovmOQ26OUIuEJH5hxajZAW4W/ilwbSb4qvq2ruffjTZuZUaG6Uq3fmxphsPx DAW1ncPWa8952t3RpEwwkn9c+W8wiZHT5R1Bmj9vSoRTHTCQbvVktti0yQck0iE34WUN L4RQ== X-Gm-Message-State: AOJu0Yy4DiAbmMZR/8Et95IxyvRe69cEdTRPe6TrDSZw9jF4Vdt6aoEY u8lcOt4gdC6RbXQ4FhJM41FS6U4DjuiH84krUpv++4W2VQHy4d8bos9YZY/Z2m5FMTK7ZJnSsA9 +8sQ= X-Gm-Gg: ASbGncufBErt9JoOJM2pyYJPnpmfPZaFvh3KEnFha78m4h0SIY07mPeCokLMBhwtdnE oisnHFLuOsiqNPrKZBMRQFFd0z6eMSUxKrvkQL+Wap+ytIl7/392ZMpnFe+SdU8dO5qmkH1YrnU QlKUCv8O3Ch09TGVmcfxjpEpttJoW/eYU0fWqgEQVuCAeZnnBAgSL7lgdI+QzmwD1wj5nTcy4Gi PJK+W4zRSLWgFmR7OzLz9KNcClGy83QZcXOw1kRfz+5UNGbBslI8ngttnkoESlz2gT9jAk29mdF eBV3z3uRw12cZ7EMWabhnRQM6ypJ3hAnN/fcVQw6pYfJRZ9R8E46 X-Google-Smtp-Source: AGHT+IEDwQijrDq++X85oZP9e7uOLlMi/huBO7q1eq9Uqphj57NissZ/B/xCLZDfGBYtq84GeSEwaw== X-Received: by 2002:a05:690c:9692:b0:6f2:8e62:d8b2 with SMTP id 00721157ae682-6f6eb922360mr201850107b3.29.1737653674669; Thu, 23 Jan 2025 09:34:34 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f757a33e41sm439397b3.115.2025.01.23.09.34.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jan 2025 09:34:34 -0800 (PST) Date: Thu, 23 Jan 2025 12:34:33 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v4 5/8] csum-file.c: use unsafe_hash_algo() Message-ID: <64a850c77ae23b92698acf89761f0e7513b6d35e.1737653640.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Instead of calling the unsafe_ hash function variants directly, make use of the shared 'algop' pointer by initializing it to: f->algop = unsafe_hash_algo(the_hash_algo); , thus making all calls use the unsafe variants directly. Signed-off-by: Taylor Blau --- csum-file.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/csum-file.c b/csum-file.c index 7a71121e340..ebffc80ef71 100644 --- a/csum-file.c +++ b/csum-file.c @@ -50,7 +50,7 @@ void hashflush(struct hashfile *f) if (offset) { if (!f->skip_hash) - f->algop->unsafe_update_fn(&f->ctx, f->buffer, offset); + f->algop->update_fn(&f->ctx, f->buffer, offset); flush(f, f->buffer, offset); f->offset = 0; } @@ -73,7 +73,7 @@ int finalize_hashfile(struct hashfile *f, unsigned char *result, if (f->skip_hash) hashclr(f->buffer, f->algop); else - f->algop->unsafe_final_fn(f->buffer, &f->ctx); + f->algop->final_fn(f->buffer, &f->ctx); if (result) hashcpy(result, f->buffer, f->algop); @@ -128,7 +128,7 @@ void hashwrite(struct hashfile *f, const void *buf, unsigned int count) * f->offset is necessarily zero. */ if (!f->skip_hash) - f->algop->unsafe_update_fn(&f->ctx, buf, nr); + f->algop->update_fn(&f->ctx, buf, nr); flush(f, buf, nr); } else { /* @@ -175,8 +175,8 @@ static struct hashfile *hashfd_internal(int fd, const char *name, f->do_crc = 0; f->skip_hash = 0; - f->algop = the_hash_algo; - f->algop->unsafe_init_fn(&f->ctx); + f->algop = unsafe_hash_algo(the_hash_algo); + f->algop->init_fn(&f->ctx); f->buffer_len = buffer_len; f->buffer = xmalloc(buffer_len); @@ -210,7 +210,7 @@ void hashfile_checkpoint(struct hashfile *f, struct hashfile_checkpoint *checkpo { hashflush(f); checkpoint->offset = f->total; - f->algop->unsafe_clone_fn(&checkpoint->ctx, &f->ctx); + f->algop->clone_fn(&checkpoint->ctx, &f->ctx); } int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint) @@ -221,7 +221,7 @@ int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint lseek(f->fd, offset, SEEK_SET) != offset) return -1; f->total = offset; - f->algop->unsafe_clone_fn(&f->ctx, &checkpoint->ctx); + f->algop->clone_fn(&f->ctx, &checkpoint->ctx); f->offset = 0; /* hashflush() was called in checkpoint */ return 0; } @@ -242,15 +242,15 @@ int hashfile_checksum_valid(const unsigned char *data, size_t total_len) { unsigned char got[GIT_MAX_RAWSZ]; git_hash_ctx ctx; - const struct git_hash_algo *algop = the_hash_algo; + const struct git_hash_algo *algop = unsafe_hash_algo(the_hash_algo); size_t data_len = total_len - algop->rawsz; if (total_len < algop->rawsz) return 0; /* say "too short"? */ - algop->unsafe_init_fn(&ctx); - algop->unsafe_update_fn(&ctx, data, data_len); - algop->unsafe_final_fn(got, &ctx); + algop->init_fn(&ctx); + algop->update_fn(&ctx, data, data_len); + algop->final_fn(got, &ctx); return hasheq(got, data + data_len, algop); } From patchwork Thu Jan 23 17:34:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13948510 Received: from mail-yb1-f182.google.com (mail-yb1-f182.google.com [209.85.219.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15BFD15ECD7 for ; Thu, 23 Jan 2025 17:34:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653680; cv=none; b=d3DF0NX8CNYjTRO8/ETl+MdqkHt5zezmfF/JgA3p9dPJP0TME0SnoOaqqTMRTd/T+rZpMLs+LTula4hghC3ZPIZ953jfVhye8ESAK0eC2Yr1nEXnnmkoTkcUY3E0zmQrmPRBLsZX3unDrdsPqpLowfHgmZEU/o5e2/sVshxsMuI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653680; c=relaxed/simple; bh=iaYLhVr5ITdgKklU3wMRBe6kRZ8OVjP8Gufp/dLGIak=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=B0QEnMUeQzAj3Z9vtXqRWOXkLYnvntSNgSFjzJUw0aEoe8xG3LrFnV78KA0tLLpKu4DIw5Wa0FkF6j/8opVymvqWeMJPUnNlrIx9kgPXIf5tuy58wtNBBMe+GRZzx+WdaRz7xyZ9Ix1tnzWAYRPd0SCeaSs97EAVOb9j2F72OQ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=2Vcg+jb+; arc=none smtp.client-ip=209.85.219.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="2Vcg+jb+" Received: by mail-yb1-f182.google.com with SMTP id 3f1490d57ef6-e39f43344c5so1705002276.1 for ; Thu, 23 Jan 2025 09:34:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737653678; x=1738258478; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yvl2c/EZuDTHNncUogAJwA1jcWgV73J4Dfo5ZA9n33E=; b=2Vcg+jb+nGse5n8hpyu4hfEpMs8iJOIFUPSE/MxHL2wDmVHCXe0z3UVlsfoI1Xvot1 qlX6WLtX0AQJB/pNdjNPVFAWCNDleh5yRIbI5pelnKfRxINrGd7qCG26gyC6+2Ufuhmx dkyJ1VxaSj8NUb+ZGMbbBSYU+Qt/rS8I9kwhospth9B1NtS5XRHMGzbQO3rB7x0LQuxb 7B2z6/cx6VWGAAcgXdtMmJU4+z2JsSLMwuyku4EIT68/Z+8FRgV8cAQfGJXr66Zi58yC SCHvD1gVb0qAW4TgXVj7jbxUZr8sd46pTjNw+kCJueY/CU+ObPBo5sKIzrA2y3k7yrG3 HmlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737653678; x=1738258478; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=yvl2c/EZuDTHNncUogAJwA1jcWgV73J4Dfo5ZA9n33E=; b=mXrjhI2H5xJlRB1Q4zZG5/JQb/9jw92iisoDvGt0pLN2gE2S0Z2lmBwHCkzI1SyUYm 0cTcoAeoqHJLUfK1B9keEO3Hhpp/AQ579Z/+/JD3jNY8MhKFr27oJPg/MX35JeaNU9J1 7n+6bMmH6FRJFTC0pkt/QKKQ2JEMg4E5nFYWNkZwgRLbEBKv+/18RBlfT3hUKdEJX0xC 4OmwbyP279KggS3ofEvSvtonubIopEgokRFipA9aT+IknAdKwdK6IkOBSLD/HRJsvnRy ofgAwrjfswo7d7CBc+ZTVZuthLxGzZUJ9xCyzEoWwlEyemOscSGHnFIPoUYIl9/BHM2N PZkg== X-Gm-Message-State: AOJu0YyxItUHjDsamOc8N1L4kszUAakzaMr+mkeTrDZ95DADL3VDlq9A Y/kSC82IwUBvRCB7RQ0GddckxsokYRH2hEyHhRjyy9q4T+Adk+Sgq3fGjviGx15tLWFYyK82MAM e2N4= X-Gm-Gg: ASbGncutZ4ml8TqQ03QD5flsuiQxJVTpQ7O814cqu+9YAgWsI/GhtoMKSUzDRDh+5oG FEl0H0+Amm/eAkvV/OgHnaTSzeHkYvClAapyCqDc/9F9EdccZ0jd2tXbJtvQT/Rn//btoutZUBU aF3Icg7moaRkQMd3yr55EDsxl+V9/5L4PdIHBOQjPA90LDmulWuzylxdJFNnarDpmEeSGFnlzqU /Qn3qsjhUCC57+2LOMYKIeKkxXrB4BQ4bmD264qgGqywq9+XMoTnmnsFTcxxR2kWTZ3NOlEtpKp CUxCwFWD/qqqJefIhDDoYZT0Pzq+FKLkZPROAC1Ku/g5/uXps9Au X-Google-Smtp-Source: AGHT+IHjr2YfPhiDbvTppSX2G5yUZzu42US4mbyLI07ffKi0SnIyZcymdRMMbVNYOuFFf2pPhErXxQ== X-Received: by 2002:a05:690c:7202:b0:6ef:4fba:8137 with SMTP id 00721157ae682-6f6eb65c4f7mr216443517b3.6.1737653677780; Thu, 23 Jan 2025 09:34:37 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f757a0124fsm453387b3.79.2025.01.23.09.34.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jan 2025 09:34:37 -0800 (PST) Date: Thu, 23 Jan 2025 12:34:36 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v4 6/8] t/helper/test-hash.c: use unsafe_hash_algo() Message-ID: <3dcccccf75275d62d533773db4e027ef051d5388.1737653640.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Remove a series of conditionals within the shared cmd_hash_impl() helper that powers the 'sha1' and 'sha1-unsafe' helpers. Instead, replace them with a single conditional that transforms the specified hash algorithm into its unsafe variant. Then all subsequent calls can directly use whatever function it wants to call without having to decide between the safe and unsafe variants. Signed-off-by: Taylor Blau --- t/helper/test-hash.c | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/t/helper/test-hash.c b/t/helper/test-hash.c index d0ee668df95..aa82638c621 100644 --- a/t/helper/test-hash.c +++ b/t/helper/test-hash.c @@ -9,6 +9,8 @@ int cmd_hash_impl(int ac, const char **av, int algo, int unsafe) int binary = 0; char *buffer; const struct git_hash_algo *algop = &hash_algos[algo]; + if (unsafe) + algop = unsafe_hash_algo(algop); if (ac == 2) { if (!strcmp(av[1], "-b")) @@ -27,10 +29,7 @@ int cmd_hash_impl(int ac, const char **av, int algo, int unsafe) die("OOPS"); } - if (unsafe) - algop->unsafe_init_fn(&ctx); - else - algop->init_fn(&ctx); + algop->init_fn(&ctx); while (1) { ssize_t sz, this_sz; @@ -49,15 +48,9 @@ int cmd_hash_impl(int ac, const char **av, int algo, int unsafe) } if (this_sz == 0) break; - if (unsafe) - algop->unsafe_update_fn(&ctx, buffer, this_sz); - else - algop->update_fn(&ctx, buffer, this_sz); + algop->update_fn(&ctx, buffer, this_sz); } - if (unsafe) - algop->unsafe_final_fn(hash, &ctx); - else - algop->final_fn(hash, &ctx); + algop->final_fn(hash, &ctx); if (binary) fwrite(hash, 1, algop->rawsz, stdout); From patchwork Thu Jan 23 17:34:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13948511 Received: from mail-yb1-f181.google.com (mail-yb1-f181.google.com [209.85.219.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 070F4166F32 for ; Thu, 23 Jan 2025 17:34:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653683; cv=none; b=S+iza9WTOvDGEWM1qW3UEpOvz1bnvKqeyNZsyy8/Jy9faltch+J5raXl4+nbs56d46ZIfeSBcyligNt2DrypZPUXSrfpirrYoeJBw4Wxu6k0mdGu/tkxnMmNpuLjuzkH8ivA87Cmv7c507Wdni2ZqepY7qPCwnkTMjUtMZWpqQY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653683; c=relaxed/simple; bh=XX9CayRwodWzzPoMzTsJqWyYVvYPKGyCntRvQaeBaJQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=nfC0bkZU6y++55GAcAKOrNpWIshTHeBj/T8lCK8hy67h2Y8GamjBx9+l5P5nm3ZU1WShKecnlWjZxI0ydy2LaX7h0Xtbl/Ujwdsdcp38vQjTsJZomUiVT1CK7A62w7W/51pbCHnNJx/hWdA2JK/qH94eBCiEr60D+MYocOFPvPg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=eOshSluP; arc=none smtp.client-ip=209.85.219.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="eOshSluP" Received: by mail-yb1-f181.google.com with SMTP id 3f1490d57ef6-e53a91756e5so2153906276.1 for ; Thu, 23 Jan 2025 09:34:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737653681; x=1738258481; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=3XfbhZ4NJq/su2IZlAhXveMpldbkSQsCFX7OvTBk/uk=; b=eOshSluP5GfYViMr6FonUOV7Rym5o1oz4qsStt52C4PQg7h8TtjRf4fboXbkQF0iW+ BNgCucYlWzA1kU8PIt41acl5Y+orGDFVlubefwD8ROBRs3atKHe+wQpnNXeNu0aBEvXG n5Jd1mP/SZmM9nK6PJ3RYPWRWmLjBJEkrXRVMy3YelKdzIM7pK2ZFzopi02tr1u12yBk NaYhA/fol5/TvnGS+zWECeF+gO4iryP+9XWDXRqo4Z1FgiOJYyOhxhfdh54R2eSC4LOt FgGvENtKcY5xIyF9n90EIL+svtihCr3xyoxQRiVsRL5naRnFSjlyFnalqOrQXMUbN46T 9sww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737653681; x=1738258481; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=3XfbhZ4NJq/su2IZlAhXveMpldbkSQsCFX7OvTBk/uk=; b=ZvX3ZcuhxaUvPYqhLxRZr53nGIECibA/nA/GpPAGVY+AXRNFfXtzDv1TdayA54xyfY RllWXKPsOkkXPp2Lf2Ii8vFTy3OHYVHmHtgmGpLZnDLONR5rey/w81OJy1pQuQuS3fPK HFC4MUqg00Uj2Kyu1SRwqGRCT/yuRI6XulJjocsSd89cQiT7rB/wY+dxN7GKurOtJSQF rw8NV0m982qjtjtoSgQD/st4J5WMf+88P/6mTc+67LCRlLiML9/ZS3w5eab7H8WaYrh0 GqtN6hf+78TouXPCEqcPS6lnhL1smzM/5QVWu/kc3+ZBx8qIxPbCqq1jkZviSd3IQeSC Zw1w== X-Gm-Message-State: AOJu0YzQenBGxJB2e396TWY8mh9wA8y3bc81qlSFNk+y3kMZafUB29CN aIB6NzhVCg6tzw+aWD9I9pyT1G29nKHSsxOdwtsC+ObefrBdVH1LgTJv0Apk55CaGgwUVFkDK1D hSQk= X-Gm-Gg: ASbGncuWNpEItk7Rgkfrkvpzbj707bqyel0JZLqfJ4MuB+2MqLsnvFMclOyfdgv+UT/ xlZTvyJhQoyD4GAfJp2LKd+rKCLHLXGMPNY5d4X0FoVzgK7kDjeiBKSrSuR5RXnyAneawxiF7PL 4OUqG70gzg1yxh0FE4K0H1KZZa7TxpFk2VnRIhTxYLElebU6LFBwKXiWxw5TDzaL+P1/fb8DyEz FSCH5ONBJXIvbr8d6TYPzr+sYO7q98qhb04DUvC2GnMMlmmkLCh/Kayf5p2mQ+kTwFcFweQ3qTe +e0QJSh2IpwJ0j/N/agtTNIF15dwnOOkX3vh2vjDYFXwnm9CUQsa X-Google-Smtp-Source: AGHT+IEZPzO8cQYdu3ZrqizZz1Xa8Bq5Woa735CCoG7j1vzFjtK07oLZgBdT4JoF9gMh4GBucjQuzg== X-Received: by 2002:a05:690c:6812:b0:6f6:cba2:a881 with SMTP id 00721157ae682-6f6eb93f1e7mr215946327b3.33.1737653680867; Thu, 23 Jan 2025 09:34:40 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f757a0554csm455377b3.93.2025.01.23.09.34.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jan 2025 09:34:40 -0800 (PST) Date: Thu, 23 Jan 2025 12:34:39 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v4 7/8] csum-file: introduce hashfile_checkpoint_init() Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: In 106140a99f (builtin/fast-import: fix segfault with unsafe SHA1 backend, 2024-12-30) and 9218c0bfe1 (bulk-checkin: fix segfault with unsafe SHA1 backend, 2024-12-30), we observed the effects of failing to initialize a hashfile_checkpoint with the same hash function implementation as is used by the hashfile it is used to checkpoint. While both 106140a99f and 9218c0bfe1 work around the immediate crash, changing the hash function implementation within the hashfile API to, for example, the non-unsafe variant would re-introduce the crash. This is a result of the tight coupling between initializing hashfiles and hashfile_checkpoints. Introduce and use a new function which ensures that both parts of a hashfile and hashfile_checkpoint pair use the same hash function implementation to avoid such crashes. A few things worth noting: - In the change to builtin/fast-import.c::stream_blob(), we can see that by removing the explicit reference to 'the_hash_algo->unsafe_init_fn()', we are hardened against the hashfile API changing away from the_hash_algo (or its unsafe variant) in the future. - The bulk-checkin code no longer needs to explicitly zero-initialize the hashfile_checkpoint, since it is now done as a result of calling 'hashfile_checkpoint_init()'. - Also in the bulk-checkin code, we add an additional call to prepare_to_stream() outside of the main loop in order to initialize 'state->f' so we know which hash function implementation to use when calling 'hashfile_checkpoint_init()'. This is OK, since subsequent 'prepare_to_stream()' calls are noops. However, we only need to call 'prepare_to_stream()' when we have the HASH_WRITE_OBJECT bit set in our flags. Without that bit, calling 'prepare_to_stream()' does not assign 'state->f', so we have nothing to initialize. - Other uses of the 'checkpoint' in 'deflate_blob_to_pack()' are appropriately guarded. Helped-by: Patrick Steinhardt Signed-off-by: Taylor Blau --- builtin/fast-import.c | 2 +- bulk-checkin.c | 9 ++++++--- csum-file.c | 7 +++++++ csum-file.h | 1 + 4 files changed, 15 insertions(+), 4 deletions(-) diff --git a/builtin/fast-import.c b/builtin/fast-import.c index 0f86392761a..4a6c7ab52ac 100644 --- a/builtin/fast-import.c +++ b/builtin/fast-import.c @@ -1106,7 +1106,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark) || (pack_size + PACK_SIZE_THRESHOLD + len) < pack_size) cycle_packfile(); - the_hash_algo->unsafe_init_fn(&checkpoint.ctx); + hashfile_checkpoint_init(pack_file, &checkpoint); hashfile_checkpoint(pack_file, &checkpoint); offset = checkpoint.offset; diff --git a/bulk-checkin.c b/bulk-checkin.c index 433070a3bda..892176d23d2 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -261,7 +261,7 @@ static int deflate_blob_to_pack(struct bulk_checkin_packfile *state, git_hash_ctx ctx; unsigned char obuf[16384]; unsigned header_len; - struct hashfile_checkpoint checkpoint = {0}; + struct hashfile_checkpoint checkpoint; struct pack_idx_entry *idx = NULL; seekback = lseek(fd, 0, SEEK_CUR); @@ -272,12 +272,15 @@ static int deflate_blob_to_pack(struct bulk_checkin_packfile *state, OBJ_BLOB, size); the_hash_algo->init_fn(&ctx); the_hash_algo->update_fn(&ctx, obuf, header_len); - the_hash_algo->unsafe_init_fn(&checkpoint.ctx); /* Note: idx is non-NULL when we are writing */ - if ((flags & HASH_WRITE_OBJECT) != 0) + if ((flags & HASH_WRITE_OBJECT) != 0) { CALLOC_ARRAY(idx, 1); + prepare_to_stream(state, flags); + hashfile_checkpoint_init(state->f, &checkpoint); + } + already_hashed_to = 0; while (1) { diff --git a/csum-file.c b/csum-file.c index ebffc80ef71..232121f415f 100644 --- a/csum-file.c +++ b/csum-file.c @@ -206,6 +206,13 @@ struct hashfile *hashfd_throughput(int fd, const char *name, struct progress *tp return hashfd_internal(fd, name, tp, 8 * 1024); } +void hashfile_checkpoint_init(struct hashfile *f, + struct hashfile_checkpoint *checkpoint) +{ + memset(checkpoint, 0, sizeof(*checkpoint)); + f->algop->init_fn(&checkpoint->ctx); +} + void hashfile_checkpoint(struct hashfile *f, struct hashfile_checkpoint *checkpoint) { hashflush(f); diff --git a/csum-file.h b/csum-file.h index 2b45f4673a2..b7475f16c20 100644 --- a/csum-file.h +++ b/csum-file.h @@ -36,6 +36,7 @@ struct hashfile_checkpoint { git_hash_ctx ctx; }; +void hashfile_checkpoint_init(struct hashfile *, struct hashfile_checkpoint *); void hashfile_checkpoint(struct hashfile *, struct hashfile_checkpoint *); int hashfile_truncate(struct hashfile *, struct hashfile_checkpoint *); From patchwork Thu Jan 23 17:34:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13948512 Received: from mail-yb1-f173.google.com (mail-yb1-f173.google.com [209.85.219.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9991F13D882 for ; Thu, 23 Jan 2025 17:34:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653688; cv=none; b=ZG0p9BKV+Ce/ASmhJu20hQTd9BvynTHgvbzWpuKaoZ9Lp914joLgOxgeL3ch15A8UWwDfn/rDr4Z9qooFRODla8NlU6imxUhsQfrIgQMlzbo4CjCYxcQp6QKlnuuD4yrBB/dhJM/O9ikEOT2zArPnWlec7oa+KCatYY+ocfMp+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737653688; c=relaxed/simple; bh=ZxufP7yCO8MOsTX4t0D2bm0+KT8jpBLSNFAIQRpzwW4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=nK9Xy0HfUzzfSahQ3vd4xctsrk+YE4sa6DTGc8rQIPPkCj8hX4I54dswaknIIZydob1hEOKVMRJSVabUQJ4ZQjGFuVs9uZHFoAU6T/jAy2eEMYHE8xIV7PWRusmy7pK22be1aN4AXPOL3Gsfem09JLIhjpWXDdfV/XMqTuJ/Qkk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=emkVgVWG; arc=none smtp.client-ip=209.85.219.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="emkVgVWG" Received: by mail-yb1-f173.google.com with SMTP id 3f1490d57ef6-e53ef7462b6so2001806276.3 for ; Thu, 23 Jan 2025 09:34:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1737653684; x=1738258484; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=QA4d8lONclLoGTt0Pu9UVbPowqoU8qd1DWCHz0he9Zc=; b=emkVgVWGDz2ukpMwtHIPyWO2u8JdOBsgvYZHT/8RgRAt3dSKJADwFJplywM+4hqpRx 8ZECw2ztn+bno/C+MT2UGFg6UmVOD81Fb2Ut6nR5T+Tr6bQ3dwcRbvQkjZzBaGT35Hhb zGfYNqQIKUBLt+xhmTqrUAGBQw+JcaNg67w5WC047e6b6ppfdfYj7nq3LPkXpynTQ7sl AFljzz8xcLfiz1CsbMXZw+s4Tt2GDm4QTY5CarsgTAmsnBu6MfXIX4ZmC2Z5yFF1vQiq NpWU7MWKHcKBRbv6dK4AIzcyAl5Y1NnMFB2Bx4TwAt+F4ODHo8elP64k0hhdZKeUuoPb M1JA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737653684; x=1738258484; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=QA4d8lONclLoGTt0Pu9UVbPowqoU8qd1DWCHz0he9Zc=; b=moJ7K64ZlCge4EBXym4JkRPtBmjJ/C9yc3VaNnR3Lda8J5bTksn2galCJ6n1aRI/F7 8lAAHrMhZCYeDNyq7tMRNo/tr10dWQwhB9C/zzfemYP74/EaSso7/CiZMlx6OW/dqNp6 ywePF9xS3tLoUdkei3QVwhlhLQPrPzWXPY4n57TACJ9gkeP+1EYRp+pHIgTGaA38MViB fdma042CwYJ0wUFv8lVIUtVsvtxAd956xfXBFiADjYInty3R+o4xH4Fe3kMvVMiAo/ak nbZOdc7bPRH5+ajQtsUHFG1fy9Z4kZ0Fd2fvnIZk5wvTKaB10BjruXOkURfMV5vq0Q8d kQKA== X-Gm-Message-State: AOJu0Yz6ualSh6FCTx84nExvXrC74m0S6zHoBM/cY2ZinzIfc6S17vTI ruSTnnKzp3zoPZzQXDXLLzvEakISFg4bjhqJBa0QABzkGxUioN7b1DTP1UCSzEcL5CIDa4djv4V rpCs= X-Gm-Gg: ASbGncu+5wdlfLvVyFToZiatjZ2+idpJT5BpWfBUAGLaWCWzHoA55+koeid7tcxMuO0 J+QMWTkwoXuV4I85x1cTVwllhmFKKGYJKDWXBW5b8pysbx9dVNBpAIxhxo4fWM+KdPzdIjE1YDT 7uKD8gWNSznPbBN9I35OYN0und8OWC8/Nh055CRaxRODzm30UPucHvR7YKcgnSWNgbHEnKxqctw qoRDwcANj2jpmxKj5ebN2aYezZaDZX6h4vrjy1oNd4p/zxLIqC797AzlZ1A+8tSRZqwFDJNRuQm NH1ZCcTY7uIkqt8sUbO57Pd+rLphbg6+mRvTVyChfNwyz+/wChZoWmkpsR679F0= X-Google-Smtp-Source: AGHT+IFOCx4Ug2PR8TPown5J9ppR9MKRRuzk0jV9O/HiMXzaKt8bKdF7vJs7uDCrIUN9v+lKX/l5QQ== X-Received: by 2002:a05:690c:6603:b0:6ef:c5f6:2ac8 with SMTP id 00721157ae682-6f6eb904750mr219145267b3.27.1737653684031; Thu, 23 Jan 2025 09:34:44 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f757891e2esm483877b3.45.2025.01.23.09.34.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jan 2025 09:34:43 -0800 (PST) Date: Thu, 23 Jan 2025 12:34:42 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v4 8/8] hash.h: drop unsafe_ function variants Message-ID: <0ba27182b5ed666cb9f49c674f2597eec7572b90.1737653640.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Now that all callers have been converted from: the_hash_algo->unsafe_init_fn(); to unsafe_hash_algo(the_hash_algo)->init_fn(); and similar, we can remove the scaffolding for the unsafe_ function variants and force callers to use the new unsafe_hash_algo() mechanic instead. Signed-off-by: Taylor Blau --- hash.h | 15 --------------- object-file.c | 15 --------------- 2 files changed, 30 deletions(-) diff --git a/hash.h b/hash.h index 0bf63cedfa4..ad2c919991c 100644 --- a/hash.h +++ b/hash.h @@ -282,21 +282,6 @@ struct git_hash_algo { /* The hash finalization function for object IDs. */ git_hash_final_oid_fn final_oid_fn; - /* The non-cryptographic hash initialization function. */ - git_hash_init_fn unsafe_init_fn; - - /* The non-cryptographic hash context cloning function. */ - git_hash_clone_fn unsafe_clone_fn; - - /* The non-cryptographic hash update function. */ - git_hash_update_fn unsafe_update_fn; - - /* The non-cryptographic hash finalization function. */ - git_hash_final_fn unsafe_final_fn; - - /* The non-cryptographic hash finalization function. */ - git_hash_final_oid_fn unsafe_final_oid_fn; - /* The OID of the empty tree. */ const struct object_id *empty_tree; diff --git a/object-file.c b/object-file.c index 43efa4ca361..c4b42dd4be9 100644 --- a/object-file.c +++ b/object-file.c @@ -230,11 +230,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { .update_fn = git_hash_unknown_update, .final_fn = git_hash_unknown_final, .final_oid_fn = git_hash_unknown_final_oid, - .unsafe_init_fn = git_hash_unknown_init, - .unsafe_clone_fn = git_hash_unknown_clone, - .unsafe_update_fn = git_hash_unknown_update, - .unsafe_final_fn = git_hash_unknown_final, - .unsafe_final_oid_fn = git_hash_unknown_final_oid, .empty_tree = NULL, .empty_blob = NULL, .null_oid = NULL, @@ -250,11 +245,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { .update_fn = git_hash_sha1_update, .final_fn = git_hash_sha1_final, .final_oid_fn = git_hash_sha1_final_oid, - .unsafe_init_fn = git_hash_sha1_init_unsafe, - .unsafe_clone_fn = git_hash_sha1_clone_unsafe, - .unsafe_update_fn = git_hash_sha1_update_unsafe, - .unsafe_final_fn = git_hash_sha1_final_unsafe, - .unsafe_final_oid_fn = git_hash_sha1_final_oid_unsafe, .unsafe = &sha1_unsafe_algo, .empty_tree = &empty_tree_oid, .empty_blob = &empty_blob_oid, @@ -271,11 +261,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { .update_fn = git_hash_sha256_update, .final_fn = git_hash_sha256_final, .final_oid_fn = git_hash_sha256_final_oid, - .unsafe_init_fn = git_hash_sha256_init, - .unsafe_clone_fn = git_hash_sha256_clone, - .unsafe_update_fn = git_hash_sha256_update, - .unsafe_final_fn = git_hash_sha256_final, - .unsafe_final_oid_fn = git_hash_sha256_final_oid, .empty_tree = &empty_tree_oid_sha256, .empty_blob = &empty_blob_oid_sha256, .null_oid = &null_oid_sha256,