From patchwork Fri Oct 13 19:25:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13421621 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 875D8208DE for ; Fri, 13 Oct 2023 19:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="rmTEuGap" Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7CA295 for ; Fri, 13 Oct 2023 12:25:20 -0700 (PDT) Received: by mail-qk1-x732.google.com with SMTP id af79cd13be357-77575233636so163973085a.2 for ; Fri, 13 Oct 2023 12:25:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1697225120; x=1697829920; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=XG4WCoWmp1vyMlGAqXq8uO3Pg6j11C+mEDyN6QSKRiQ=; b=rmTEuGapUWCO+rqaOAPuYFZWljm0qTqUg72qDXe9BdboqEBG0xvPvpVELf5KZneJUj sRgvsVVqI5rm8vL3C70vDPd/nYnFh3SyDQVEB0Ko/6Aq6LvD11/2P2DNq8ov+OgvjwKH bUvszgvS/2KNPy44pawG8mDqcV4X+2cmD7sv8A79FKIc35bdTZBM8T52JdV5TmfHgG6b xOJndSONWTSdFePMW6xT9amXXhuAaSQCGw3cbXRoCUiZMtck+fkFP8u3GORQUQfROZfD Su5x+ez4yYYs3VAwfJ4Cn0OmN2bRxBk8tDnBXxfeytypKAVvcE19eezIiyQIVma3SdRz K3gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697225120; x=1697829920; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=XG4WCoWmp1vyMlGAqXq8uO3Pg6j11C+mEDyN6QSKRiQ=; b=hJjdUedKhIqgVP/X2pL8NizmeZ2jfJhfvPrfl2AZ9VC87zIW2ACAl2lpe/pF2gVbA8 qgpoHq0nHhjTyrkHnYi1YC71GbZ+q5r1SPmRlJ8Q3W365pOGVL5OZWk4o6FPQmLPOq0P +pouq3dsh1qrvbIFmX9Z7u5GC31Xz/LyngWjFOG37RMshcvMjl/MDik6K4ktsB4FejcE 0SBCz4rsaKqx8ltpdNUcJzGlcX8EA/wWFUTSSuIkkPBCK2EHLvQ/V1eADCOiGcGEYdkN 18Z1MhgjtQuk+GQBNBoNs2ZZ7RWZTartx1q+jrZYlxtQfIC++S5+41msv02qiw4SENTp kYZw== X-Gm-Message-State: AOJu0YzlndcC70C/bsulDmON/H6VZ7x0IkmGJZzWUaxgdd5QwaGDq1eR VSzBevAQDCtH18rwJLiRPv23pbz6uJmpMZrRHjycKA== X-Google-Smtp-Source: AGHT+IHXWn6Vez0wGZ+dnXybWWjAIhj2x7iqMCSCpk6Yleg8OCJdCvX01eWqx24k3slEAOZR9Kx0fQ== X-Received: by 2002:a05:620a:410c:b0:768:1e00:76c4 with SMTP id j12-20020a05620a410c00b007681e0076c4mr32317866qko.51.1697225119678; Fri, 13 Oct 2023 12:25:19 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id dt52-20020a05620a47b400b00767721aebc0sm881069qkb.32.2023.10.13.12.25.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 12:25:19 -0700 (PDT) Date: Fri, 13 Oct 2023 15:25:18 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King Subject: [PATCH 1/8] chunk-format: introduce `pair_chunk_expect()` helper Message-ID: <6fa20034b2246599b3e1cd49accfc421b0fba4fb.1697225110.git.me@ttaylorr.com> References: <20231009205544.GA3281950@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net In previous commits, the pair_chunk() interface grew a required "size" pointer, so that the caller is forced to at least have a handle on the actual size of the given chunk. Many callers were converted to the new interface. A handful were instead converted from the unsafe version of pair_chunk() to read_chunk() so that they could check their expected size. This led to a lot of code like: static int graph_read_oid_fanout(const unsigned char *chunk_start, size_t chunk_size, void *data) { struct commit_graph *g = data; if (chunk_size != 256 * sizeof(uint32_t)) return error("commit-graph oid fanout chunk is wrong size"); g->chunk_oid_fanout = (const uint32_t *)chunk_start; return 0; } , leaving the caller to use read_chunk(), like so: read_chunk(cf, GRAPH_CHUNKID_OIDFANOUT, graph_read_oid_fanout, graph); The callback to read_chunk() (in the above, `graph_read_oid_fanout()`) does nothing more than (a) assign a pointer to the location of the start of the chunk in the mmap'd file, and (b) assert that it has the correct size. For callers that know the expected size of their chunk(s) up-front, we can simplify this by teaching the chunk-format API itself to validate the expected size for us. This is wrapped in a new function, called `pair_chunk_expect()` which takes a "size_t" instead of a "size_t *", and validates that the given chunk matches the expected size as given. This will allow us to reduce the number of lines of code it takes to perform these basic read_chunk() operations, by taking the above and replacing it with something like: if (pair_chunk_expect(cf, GRAPH_CHUNKID_OIDFANOUT, (const unsigned char **)&graph->chunk_oid_fanout, 256 * sizeof(uint32_t))) error(_("commit-graph oid fanout chunk is wrong size")); We will perform those transformations in the following commits. Signed-off-by: Taylor Blau --- chunk-format.c | 22 ++++++++++++++++++++++ chunk-format.h | 12 +++++++++++- 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/chunk-format.c b/chunk-format.c index cdc7f39b70..9550f15699 100644 --- a/chunk-format.c +++ b/chunk-format.c @@ -163,6 +163,8 @@ int read_table_of_contents(struct chunkfile *cf, struct pair_chunk_data { const unsigned char **p; size_t *size; + + size_t expected_size; }; static int pair_chunk_fn(const unsigned char *chunk_start, @@ -175,6 +177,17 @@ static int pair_chunk_fn(const unsigned char *chunk_start, return 0; } +static int pair_chunk_expect_fn(const unsigned char *chunk_start, + size_t chunk_size, + void *data) +{ + struct pair_chunk_data *pcd = data; + if (pcd->expected_size != chunk_size) + return -1; + *pcd->p = chunk_start; + return 0; +} + int pair_chunk(struct chunkfile *cf, uint32_t chunk_id, const unsigned char **p, @@ -184,6 +197,15 @@ int pair_chunk(struct chunkfile *cf, return read_chunk(cf, chunk_id, pair_chunk_fn, &pcd); } +int pair_chunk_expect(struct chunkfile *cf, + uint32_t chunk_id, + const unsigned char **p, + size_t expected_size) +{ + struct pair_chunk_data pcd = { .p = p, .expected_size = expected_size }; + return read_chunk(cf, chunk_id, pair_chunk_expect_fn, &pcd); +} + int read_chunk(struct chunkfile *cf, uint32_t chunk_id, chunk_read_fn fn, diff --git a/chunk-format.h b/chunk-format.h index 14b76180ef..92c529d7ab 100644 --- a/chunk-format.h +++ b/chunk-format.h @@ -17,7 +17,8 @@ struct chunkfile; * * If reading a file, use a NULL 'struct hashfile *' and then call * read_table_of_contents(). Supply the memory-mapped data to the - * pair_chunk() or read_chunk() methods, as appropriate. + * pair_chunk(), pair_chunk_expect(), or read_chunk() methods, as + * appropriate. * * DO NOT MIX THESE MODES. Use different 'struct chunkfile' instances * for reading and writing. @@ -54,6 +55,15 @@ int pair_chunk(struct chunkfile *cf, const unsigned char **p, size_t *size); +/* + * Similar to 'pair_chunk', but used for callers who have an expected + * size for the given 'chunk_id' in advance. + */ +int pair_chunk_expect(struct chunkfile *cf, + uint32_t chunk_id, + const unsigned char **p, + size_t expected_size); + typedef int (*chunk_read_fn)(const unsigned char *chunk_start, size_t chunk_size, void *data); /* From patchwork Fri Oct 13 19:25:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13421622 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D77C32376D for ; Fri, 13 Oct 2023 19:25:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="ceEpWXH3" Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com [IPv6:2607:f8b0:4864:20::829]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95007A9 for ; Fri, 13 Oct 2023 12:25:23 -0700 (PDT) Received: by mail-qt1-x829.google.com with SMTP id d75a77b69052e-4197bb0a0d9so14382911cf.3 for ; Fri, 13 Oct 2023 12:25:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1697225122; x=1697829922; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=fAq6mbBAohEzrtPePLq4Lqw8rDkaGGY2VM65qOtCsyg=; b=ceEpWXH3jDtECf1juLAgniQag7vI3DeT6mw3iqkUjth1QKcpDP4W/fg3QN2czMuD0+ JOU7B8HY1gPy+PhmXbgrT/6k0/V9df2aT4XAmu8vi25QxyUoDxXNfak5npc2nI0LJANb gZfDTUriFbp7VRCUZEUDjP+Sks0Y+NVfoQoIXR7NqhR0KezwlXKvmQjIZR+Rp5Mky7x2 6O9C6vSqD+3lxToAQrv7iFJ8XOA+rC0nQDqzDY9HxCDZnCg/MmdbMVbf9gSWI38PfVl5 Sgq6llthz3fw3Y/nSLQ0McgMrWhUez0/pZAqtGPdeD/eGcMkEJzUtdL66wbBYJGKQHQM S7xA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697225122; x=1697829922; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=fAq6mbBAohEzrtPePLq4Lqw8rDkaGGY2VM65qOtCsyg=; b=CnfhrBv2IKAqzmAruDIq613lwmLMXViW3laNLwGKZzt69nslqeYN9H5ikrdzopekJt F6XVUV4OONMTZiPC6nF1dFoVIzrGFknfegWNen2B2/A4SXAeik8xvay+2m9UWpFPIO+0 eXbRMbl7ROvReoN25OiIAqXB6VlVqQaSsMsM+DXr4orkqnIRo0eY3qND44DxBKG++Wy4 2YUtPr3EY4uk4RCyGIt0gmnUFS25/OaHJcrcXkBFK/jAoiS7LP2n3qJuQPi+IFz1PupX ZvCSu2+81MzgxP6roN4dzeSg0KNXp+nTi7X8ELvBlNGUS4p0/ZW0HDsnfBimwfwBAEYM TCyw== X-Gm-Message-State: AOJu0YyR1e0ZKoBBG8TWSWUMVJvf71Vz4TZPyxOAI7lESj3WOCkIYB0K cv01jTevw75SK7dUwJJlIriUX/2KRzyNnovrqiTWdg== X-Google-Smtp-Source: AGHT+IGahuI7lpI21I8obhTJbQJueq8BXg7SCqKNC6VpraiGTd4mgwIA8dDCWi5+xU3hcb14+kv8Tw== X-Received: by 2002:ac8:578d:0:b0:412:c2a:eaef with SMTP id v13-20020ac8578d000000b004120c2aeaefmr34938086qta.11.1697225122579; Fri, 13 Oct 2023 12:25:22 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id h3-20020ac85483000000b004194c21ee85sm832536qtq.79.2023.10.13.12.25.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 12:25:22 -0700 (PDT) Date: Fri, 13 Oct 2023 15:25:21 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King Subject: [PATCH 2/8] commit-graph: read `OIDF` chunk with `pair_chunk_expect()` Message-ID: <3e0969d794a775f9167d7a20fe6aeb17edc60d8a.1697225110.git.me@ttaylorr.com> References: <20231009205544.GA3281950@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net The `OIDF` chunk can benefit from the new chunk-format API function described in the previous commit. Convert it to `pair_chunk_expect()` accordingly. While we're at it, let's mark the error() string for translation. Signed-off-by: Taylor Blau --- commit-graph.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 1f334987b5..cdefd7f926 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -321,16 +321,6 @@ static int verify_commit_graph_lite(struct commit_graph *g) return 0; } -static int graph_read_oid_fanout(const unsigned char *chunk_start, - size_t chunk_size, void *data) -{ - struct commit_graph *g = data; - if (chunk_size != 256 * sizeof(uint32_t)) - return error("commit-graph oid fanout chunk is wrong size"); - g->chunk_oid_fanout = (const uint32_t *)chunk_start; - return 0; -} - static int graph_read_oid_lookup(const unsigned char *chunk_start, size_t chunk_size, void *data) { @@ -462,7 +452,10 @@ struct commit_graph *parse_commit_graph(struct repo_settings *s, GRAPH_HEADER_SIZE, graph->num_chunks, 1)) goto free_and_return; - read_chunk(cf, GRAPH_CHUNKID_OIDFANOUT, graph_read_oid_fanout, graph); + if (pair_chunk_expect(cf, GRAPH_CHUNKID_OIDFANOUT, + (const unsigned char **)&graph->chunk_oid_fanout, + 256 * sizeof(uint32_t))) + error(_("commit-graph oid fanout chunk is wrong size")); read_chunk(cf, GRAPH_CHUNKID_OIDLOOKUP, graph_read_oid_lookup, graph); read_chunk(cf, GRAPH_CHUNKID_DATA, graph_read_commit_data, graph); pair_chunk(cf, GRAPH_CHUNKID_EXTRAEDGES, &graph->chunk_extra_edges, From patchwork Fri Oct 13 19:25:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13421623 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80493241FE for ; Fri, 13 Oct 2023 19:25:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="Chgf/1C+" Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7D515A9 for ; Fri, 13 Oct 2023 12:25:26 -0700 (PDT) Received: by mail-qk1-x735.google.com with SMTP id af79cd13be357-7757523362fso160009985a.0 for ; Fri, 13 Oct 2023 12:25:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1697225125; x=1697829925; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yFYevAyqGF2fzM/byjA8sl8kvwPTdFxSbuVDY0sx25w=; b=Chgf/1C+YDaxf7EGUYdDGYqcRdLab6MhpaA2sEUrapRM6s5S/F6WPj6qrPX0he8FUI 8OXjpFS24gN3ICkPLev51D7DOjlfTK+bBpwIEKPx/Z2EwbvL1sKqBQy+MlslbjAo8SNQ qWvXZ8jhFT9NecFkF+SPC9eSVCAMyQmukAFvlk3GqMfZlgBEudIQ/wGBdL0P24O45Hvb 6bTWOQ7xFRLabbfYOWxa1lxbeJlbN13F8Jt/fPbZg40+6f20vcrSB6ftZFDBt1nJ1oVd QGwUeTP+hE7HY77rA6eFFUD16f7sdd29bkKss1ZKzHCaKRhAgTnIAstl8Zx9FEjaMqgk GO/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697225125; x=1697829925; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=yFYevAyqGF2fzM/byjA8sl8kvwPTdFxSbuVDY0sx25w=; b=N0p1f2L9NAMJvUAbqxjlgDztuA6ENfobV5ER9Bf8xF9SvPQArvAxigpZ5u6iCzFzmd 661LhQvvMKWvkPGx6xSLqR4n0LLuVPp1uLZAamolVM576Ujo/pbkmAU1cHf1d+9v2q2m M5RbDWzH9VEi/uX8R6yHBa07ibs1xcXq2Vic7dVbKtiEyKoFQm13EYtZPn/8dZaHzxNp jbLTxUp29ikf/JUNQfm5ILZTf0gHAVFqYZYRPql67uFgZefGf5Y7RfOAcejvYqcmgoLE VH8tjXApNGlYBhoYD7RiNq/tqQai/RMD9jLbdLeVwZAoc3HT3C4IJfpWSNVSYJRngCeu QEbg== X-Gm-Message-State: AOJu0YzAASbTyjUGsJoPsaO3oMIv/GXXgSRzQrBsQytIwU+/vKYUpTvz 1MXAxl4od3xwQxPja0WHE0vEPAwawzgTmmQ6XxRXJA== X-Google-Smtp-Source: AGHT+IHCN0tL8xzUP+nJrw5caGM1UlU2N6I9IHXKhuwbc18LXWD88beWIKveW6cab8WGb4T2OHc4gw== X-Received: by 2002:a0c:b551:0:b0:66d:e4a:f2eb with SMTP id w17-20020a0cb551000000b0066d0e4af2ebmr7233933qvd.36.1697225125545; Fri, 13 Oct 2023 12:25:25 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id a10-20020a0ccdca000000b0065afe284b3csm897030qvn.125.2023.10.13.12.25.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 12:25:25 -0700 (PDT) Date: Fri, 13 Oct 2023 15:25:24 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King Subject: [PATCH 3/8] commit-graph: read `CDAT` chunk with `pair_chunk_expect()` Message-ID: <141ab00a1eec89666f3fb23e9bb428667f774e14.1697225110.git.me@ttaylorr.com> References: <20231009205544.GA3281950@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Perform a similar conversion as in the previous commit read the CDAT bits. While we're here, mark the error() string for translation, and guard against overflow when computing the expected size by wrapping it in an st_mult() call. Note that the pre-image of this patch was already sufficiently guarded against overflow, since GRAPH_DATA_WIDTH is defined as (the_hash_algo->rawsz + 16), so the expression in the parenthesis would get performed as a size_t, and then g->num_commits would be promoted to the width of size_t for the purposes of evaluating this expression. But let's make it explicitly clear that this computation is safe by wrapping it in an st_mult() call. Signed-off-by: Taylor Blau --- commit-graph.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index cdefd7f926..97d4824673 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -330,16 +330,6 @@ static int graph_read_oid_lookup(const unsigned char *chunk_start, return 0; } -static int graph_read_commit_data(const unsigned char *chunk_start, - size_t chunk_size, void *data) -{ - struct commit_graph *g = data; - if (chunk_size != g->num_commits * GRAPH_DATA_WIDTH) - return error("commit-graph commit data chunk is wrong size"); - g->chunk_commit_data = chunk_start; - return 0; -} - static int graph_read_generation_data(const unsigned char *chunk_start, size_t chunk_size, void *data) { @@ -457,7 +447,10 @@ struct commit_graph *parse_commit_graph(struct repo_settings *s, 256 * sizeof(uint32_t))) error(_("commit-graph oid fanout chunk is wrong size")); read_chunk(cf, GRAPH_CHUNKID_OIDLOOKUP, graph_read_oid_lookup, graph); - read_chunk(cf, GRAPH_CHUNKID_DATA, graph_read_commit_data, graph); + if (pair_chunk_expect(cf, GRAPH_CHUNKID_DATA, + &graph->chunk_commit_data, + st_mult(graph->num_commits, GRAPH_DATA_WIDTH))) + error(_("commit-graph commit data chunk is wrong size")); pair_chunk(cf, GRAPH_CHUNKID_EXTRAEDGES, &graph->chunk_extra_edges, &graph->chunk_extra_edges_size); pair_chunk(cf, GRAPH_CHUNKID_BASE, &graph->chunk_base_graphs, From patchwork Fri Oct 13 19:25:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13421624 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24429250F6 for ; Fri, 13 Oct 2023 19:25:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="XyvAvn7J" Received: from mail-yb1-xb2a.google.com (mail-yb1-xb2a.google.com [IPv6:2607:f8b0:4864:20::b2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8286AA9 for ; Fri, 13 Oct 2023 12:25:29 -0700 (PDT) Received: by mail-yb1-xb2a.google.com with SMTP id 3f1490d57ef6-d9a3d737d66so2544267276.2 for ; Fri, 13 Oct 2023 12:25:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1697225128; x=1697829928; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=SPe4DvdLmwUNmMitt5VqvfCSMM5fb9QIx28EK/d4RTM=; b=XyvAvn7JqL/hqSe8tymCzvp6m+8VOSOh1bKpLf8yhEcGbG11UdF7h02kvNhl/+2h+v B5S26ilKD6AiErWEa9FwcJPEDsUeUOik0V/hFfmSbdNZahn5IBudJWjjbVsUGXF14D+q Zj7x6um3vZtHq7BRydV3t89VZYHpmXlKhf6JxiUv5qAlyIP+MJJXg/i5Klq/h8sPlbdQ m65v1tpiLpGmuuQG4LE/CYWNZLpOgXmI9ES/mfr5vvSJvbVLFu3sAvjeEmAmxDyy7Uhw 7YspiL5wy9Kw5n81DpwHODcM3EXQ5xKixRm3I0Mr3/mJJ3y+v5/2dmPzDw87owlgJeQK 2eUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697225128; x=1697829928; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=SPe4DvdLmwUNmMitt5VqvfCSMM5fb9QIx28EK/d4RTM=; b=et32WoWOv4nuADBJrxW3NkzAYH74eqWqTwpGyaZwC9gpzM2z94crVXJgD2flFbpKt1 5B2t7JcgeHiKJXgT91yUndpXnNrQKa7nylH60t6HkiV62RbhzWoxhcV/fCb2qBQLS8FA oLNGFOl2rbdwxs/qi2wcWxoYgBSs0d90a0ETor2xrjsyv9o0RTKWafWDHD5FC2mf5M2J lDpyt9igoMeoJXpKv/qdAEUL20TSjZFMcvR2g8SbZdGTaxeymoN4xlxlaDirljZ+Up3T S+CV8rISro7I9kr/7KSqyQysuaZdBxcPsCEo7hSMrx5e9Gn1ftxk6EpY9T+HkI1Gf2bv yAWA== X-Gm-Message-State: AOJu0YwHEPJY4BfGYSzA/cs1umcdvR5E+LXqudNu8XI0zbOcmdaNnxW3 kvn4aM4vNbxfSWfIlqkXcExXwV5xUirKeZlnHehdIA== X-Google-Smtp-Source: AGHT+IEbKzC6zu6yiQ2FpzalXPEGn4uvRaCtTAbeNXfaCpkcYHGOBZm5Wo62EGwMgLFEqDVQURwauQ== X-Received: by 2002:a25:ab8f:0:b0:d9a:fe8c:c29c with SMTP id v15-20020a25ab8f000000b00d9afe8cc29cmr3444045ybi.58.1697225128393; Fri, 13 Oct 2023 12:25:28 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id ph3-20020a0562144a4300b006585c7f64a3sm908617qvb.14.2023.10.13.12.25.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 12:25:28 -0700 (PDT) Date: Fri, 13 Oct 2023 15:25:27 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King Subject: [PATCH 4/8] commit-graph: read `GDAT` chunk with `pair_chunk_expect()` Message-ID: <898b5bdb033518f2022d41b3cb5f03f71cc90070.1697225110.git.me@ttaylorr.com> References: <20231009205544.GA3281950@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Perform an identical conversion as in previous commits to read the GDAT chunk. Signed-off-by: Taylor Blau --- commit-graph.c | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 97d4824673..0fab99f5dd 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -330,16 +330,6 @@ static int graph_read_oid_lookup(const unsigned char *chunk_start, return 0; } -static int graph_read_generation_data(const unsigned char *chunk_start, - size_t chunk_size, void *data) -{ - struct commit_graph *g = data; - if (chunk_size != g->num_commits * sizeof(uint32_t)) - return error("commit-graph generations chunk is wrong size"); - g->chunk_generation_data = chunk_start; - return 0; -} - static int graph_read_bloom_index(const unsigned char *chunk_start, size_t chunk_size, void *data) { @@ -457,8 +447,11 @@ struct commit_graph *parse_commit_graph(struct repo_settings *s, &graph->chunk_base_graphs_size); if (s->commit_graph_generation_version >= 2) { - read_chunk(cf, GRAPH_CHUNKID_GENERATION_DATA, - graph_read_generation_data, graph); + if (pair_chunk_expect(cf, GRAPH_CHUNKID_GENERATION_DATA, + &graph->chunk_generation_data, + st_mult(graph->num_commits, + sizeof(uint32_t)))) + error(_("commit-graph generations chunk is wrong size")); pair_chunk(cf, GRAPH_CHUNKID_GENERATION_DATA_OVERFLOW, &graph->chunk_generation_data_overflow, &graph->chunk_generation_data_overflow_size); From patchwork Fri Oct 13 19:25:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13421625 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19C7F250F9 for ; Fri, 13 Oct 2023 19:25:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="Turjq6Yf" Received: from mail-yb1-xb32.google.com (mail-yb1-xb32.google.com [IPv6:2607:f8b0:4864:20::b32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64DAADD for ; Fri, 13 Oct 2023 12:25:32 -0700 (PDT) Received: by mail-yb1-xb32.google.com with SMTP id 3f1490d57ef6-d849df4f1ffso2751421276.0 for ; Fri, 13 Oct 2023 12:25:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1697225131; x=1697829931; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=IFDjdGK3sxCO5YC4i/yKtN+uSRJwHYxudzUbjnxuTLs=; b=Turjq6Yfpz3iUDQ9kbrymolK8NYhKqdHYxOuFd1yo5KRLBmZd8ODI9zC7NJlI6xhO9 A/dlzlBXGm/OONMF7tSK56doyjHCBNvSQiFAsnoLzxOBnxVaj7I3g0Iwu/zgYFlRaSiw aVYTEQhsxn8CMuFjPFf/1i+n6gfJ/ZtQ42L2RvCtt24dc0JdfrJ99qYpFhrhZ9i5qfli rqhQxiAv/ZDj5wZ6ecU+0Uxw8dswpXcg2elW8s5zeKc7BrE4Lvt+6n10dCZGUVT2/tuk emrLCNax8F/v76hH6/oNTQLlmo2K3XJWwEtpLKh+PRIzSO1Zxx13C806IR0rKfUt5aM0 4iCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697225131; x=1697829931; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=IFDjdGK3sxCO5YC4i/yKtN+uSRJwHYxudzUbjnxuTLs=; b=GijrW0wvVapMlNAuRGCVj9p2KN9jcVPJ+sG+GfCOi7Ra1J9e5scgSjds/tBePISHyc eHYNEg7JJ9h2U/ot9oguIS21uBQIv8cPhcnEVoh6tNPf7knhYtoxmaAlxjTrjM+EHwzD +jBUWAWOh9NkST+1LDtMiTSDbBZoFKzE4TIF+Nvt/4STcTExD0aaOQ7bqedYQV4rgXIq vbj27R7KDira653ZJRBzpM9NUO/B52GunwAqRX1vlUGjY4k3VetqKvdLwzKPIjZv/v6b xB50IJsX9Q6XGySL9hjAZVXs0tuns57Xmr+8qfP36as6sndZkYzSLgKy3tH/RoLDFxm3 kFOg== X-Gm-Message-State: AOJu0YzmpxWNcHQEo53vDNRlyD+x5sVBx8rTcKLqarClPTaw+CLNBKJ9 Py1e6+zGnxHCqsN8SG+lA0pmNEqSXqVbdW6GMMtYew== X-Google-Smtp-Source: AGHT+IFSq2UjE1DWCqskohNnX6RkQtgPxXA+t3v80yu+1NwCaRGeknDDCCisP0h9i8l1GdOge30f5g== X-Received: by 2002:a25:8901:0:b0:d7e:8175:4fa9 with SMTP id e1-20020a258901000000b00d7e81754fa9mr25970795ybl.4.1697225131192; Fri, 13 Oct 2023 12:25:31 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id d11-20020a05621416cb00b0066d15e2b73fsm880967qvz.145.2023.10.13.12.25.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 12:25:31 -0700 (PDT) Date: Fri, 13 Oct 2023 15:25:30 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King Subject: [PATCH 5/8] commit-graph: read `BIDX` chunk with `pair_chunk_expect()` Message-ID: <45cac29403e63483951f7766c6da3c022c68d9f0.1697225110.git.me@ttaylorr.com> References: <20231009205544.GA3281950@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Perform an identical conversion as in previous commits to read the BIDX chunk. Signed-off-by: Taylor Blau Signed-off-by: Taylor Blau --- commit-graph.c | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 0fab99f5dd..ad98f6334d 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -330,18 +330,6 @@ static int graph_read_oid_lookup(const unsigned char *chunk_start, return 0; } -static int graph_read_bloom_index(const unsigned char *chunk_start, - size_t chunk_size, void *data) -{ - struct commit_graph *g = data; - if (chunk_size != g->num_commits * 4) { - warning("commit-graph changed-path index chunk is too small"); - return -1; - } - g->chunk_bloom_indexes = chunk_start; - return 0; -} - static int graph_read_bloom_data(const unsigned char *chunk_start, size_t chunk_size, void *data) { @@ -461,8 +449,10 @@ struct commit_graph *parse_commit_graph(struct repo_settings *s, } if (s->commit_graph_read_changed_paths) { - read_chunk(cf, GRAPH_CHUNKID_BLOOMINDEXES, - graph_read_bloom_index, graph); + if (pair_chunk_expect(cf, GRAPH_CHUNKID_BLOOMINDEXES, + &graph->chunk_bloom_indexes, + st_mult(graph->num_commits, 4)) == -1) + warning(_("commit-graph changed-path index chunk is too small (%d)"), graph->num_commits * 4); read_chunk(cf, GRAPH_CHUNKID_BLOOMDATA, graph_read_bloom_data, graph); } From patchwork Fri Oct 13 19:25:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13421626 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97E3B224E0 for ; Fri, 13 Oct 2023 19:25:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="hO9BLJrF" Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com [IPv6:2607:f8b0:4864:20::82f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9BA9E6 for ; Fri, 13 Oct 2023 12:25:34 -0700 (PDT) Received: by mail-qt1-x82f.google.com with SMTP id d75a77b69052e-4195fddd6d7so20450841cf.0 for ; Fri, 13 Oct 2023 12:25:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1697225134; x=1697829934; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=bgFolJu8Y3Md8X+JvcQ8Z0v/knZLWnQV9pSSHP3t8FM=; b=hO9BLJrF9LRl9NUL9VbF2YHjC/l8DEG2nGG5NDiIhRq27jp/z0CKvS0vET7dEwqTBG 1rROjsXiDACVY6nX5VdYTQHf8gG9TER5hL4brqVzFOQqKL1W2k/eYnlZj7KUTB7jWBUx MecBqXdiXAKwGv44wEkoUCvStfb9Qd+P4Vv3Is3wmGY1fiRqrix9d40U7oUhTdlwdsO8 XzB1Q0dKKZA8sPZYH2grIQ00tGYBrdsQIlzzc7677OxK/hmpAv9kgcuoMZHUhKbigUza DqAC+xK8Wh1INXDalyeEJ3pxCm3NXWXrQw1krYGmFyBZqVm0mv3fl2TfTOaA2Tfz64uk aRBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697225134; x=1697829934; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=bgFolJu8Y3Md8X+JvcQ8Z0v/knZLWnQV9pSSHP3t8FM=; b=t180DCk74n2HIh4bL0vsB/u7JgXOFBUFjMcbQ7hiFI34jnRg3Z99tO2udHvYpFDe4o C1C1gd2iUrglrW8S8eLHxpkWEPQ9/q+kC/RAwsPTvJx3jeV0bKw8etxdu8NJ5P8uzO4V PXY5Kkzlmt+77tsPiBUEeMNK+24Y3UMJNEBx8phxGknSPJYXu+bCDReRgThZ7JiruNy9 Mhou6SJ+GIba9h5bEgsDUKh9p71m1bsJjulwq+224SGCs0H0kunD/QfP7X1r9RO0XBT0 YpBReRSMFqHV4Pvg0ahFgr9+wqGsGM6cydpvYuXISzSlh7CWj8eeN5rIlJRCbgyeQsxO iYAg== X-Gm-Message-State: AOJu0YySkkxUGEQSMqUdV2Bt+M9tbKbNO1LqQzc4MjFvAalEwYyWRoBL 8731SdhVVxS2Sr1mmB8ZhzAdUbbj10J9wU+mo3soBQ== X-Google-Smtp-Source: AGHT+IFayh2I28V4Bduz5ZfRDad1/yoyEJlQ5Qw8XTuJ569ZoGdlyqahIhAMfJhLebWUH8WvEAXuFQ== X-Received: by 2002:ac8:588c:0:b0:408:392e:2aa5 with SMTP id t12-20020ac8588c000000b00408392e2aa5mr1749128qta.20.1697225133909; Fri, 13 Oct 2023 12:25:33 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id jk3-20020a05622a748300b0041aff9339a2sm821638qtb.22.2023.10.13.12.25.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 12:25:33 -0700 (PDT) Date: Fri, 13 Oct 2023 15:25:32 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King Subject: [PATCH 6/8] midx: read `OIDF` chunk with `pair_chunk_expect()` Message-ID: References: <20231009205544.GA3281950@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Perform an identical conversion as in previous commits to read the OIDF chunk in the MIDX machinery. Signed-off-by: Taylor Blau --- midx.c | 21 ++++++--------------- 1 file changed, 6 insertions(+), 15 deletions(-) diff --git a/midx.c b/midx.c index 2f3863c936..38bf816cce 100644 --- a/midx.c +++ b/midx.c @@ -61,20 +61,6 @@ void get_midx_rev_filename(struct strbuf *out, struct multi_pack_index *m) strbuf_addf(out, "-%s.rev", hash_to_hex(get_midx_checksum(m))); } -static int midx_read_oid_fanout(const unsigned char *chunk_start, - size_t chunk_size, void *data) -{ - struct multi_pack_index *m = data; - m->chunk_oid_fanout = (uint32_t *)chunk_start; - - if (chunk_size != 4 * 256) { - error(_("multi-pack-index OID fanout is of the wrong size")); - return 1; - } - m->num_objects = ntohl(m->chunk_oid_fanout[255]); - return 0; -} - static int midx_read_oid_lookup(const unsigned char *chunk_start, size_t chunk_size, void *data) { @@ -173,8 +159,13 @@ struct multi_pack_index *load_multi_pack_index(const char *object_dir, int local if (pair_chunk(cf, MIDX_CHUNKID_PACKNAMES, &m->chunk_pack_names, &m->chunk_pack_names_len)) die(_("multi-pack-index required pack-name chunk missing or corrupted")); - if (read_chunk(cf, MIDX_CHUNKID_OIDFANOUT, midx_read_oid_fanout, m)) + if (pair_chunk_expect(cf, MIDX_CHUNKID_OIDFANOUT, + (const unsigned char **)&m->chunk_oid_fanout, + 256 * sizeof(uint32_t))) { + error(_("multi-pack-index OID fanout is of the wrong size")); die(_("multi-pack-index required OID fanout chunk missing or corrupted")); + } + m->num_objects = ntohl(m->chunk_oid_fanout[255]); if (read_chunk(cf, MIDX_CHUNKID_OIDLOOKUP, midx_read_oid_lookup, m)) die(_("multi-pack-index required OID lookup chunk missing or corrupted")); if (read_chunk(cf, MIDX_CHUNKID_OBJECTOFFSETS, midx_read_object_offsets, m)) From patchwork Fri Oct 13 19:25:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13421627 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68C63241E4 for ; Fri, 13 Oct 2023 19:25:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="zPoZZENF" Received: from mail-yb1-xb35.google.com (mail-yb1-xb35.google.com [IPv6:2607:f8b0:4864:20::b35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7DBB5F0 for ; Fri, 13 Oct 2023 12:25:37 -0700 (PDT) Received: by mail-yb1-xb35.google.com with SMTP id 3f1490d57ef6-d77ad095f13so2462644276.2 for ; Fri, 13 Oct 2023 12:25:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1697225136; x=1697829936; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=w2zc6g4RWcbBE0i46CrILLQ31UNkbOvaTior+gb4iU4=; b=zPoZZENFKDRrMA92Xl4M5DT+c6OpXuI/Am4pbGHmnjNrQpW4UOi8vEFwlCyEDpyWUk uR9fpEoDenogIgTH9Iii0AOXo7cmtlbfolMQyeSJHAQeLv9RzcB6PN3z8Ali7yxcl+PV zLUABrFPyAQk4IfTCp1ralxPjQCsBo2Uv12RN3crcW/WkAR2uaucWCQHsPQplejrS93C 3eLnZkkENEW16U8T+x9txDAgvLhBbQjDQaxRqUAbZuiEizWSF7C5LjurIKsGj1rX2Png IfV/9Gjis3W0n9GgZEJHKaUWvbGMraw3+cl1SvYHuCMvKNE2TBCSZCKlAm2mNTSiJVsv kWnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697225136; x=1697829936; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=w2zc6g4RWcbBE0i46CrILLQ31UNkbOvaTior+gb4iU4=; b=oSJk3ut+QkjyaBTppqipRJyVcAjdAIIPjXZHgaPaW/NShOsQUSa2DltBAU45KhkNfS twdCRfQXt7+KZDLIva+CuOOGnCAqwuoFR3kyuVqCDvr1fAeskdJGeXsDqYbF4Q5hGI3X 7M5C4KIzU4ABcan5BI/4fhDOoEKDxDGvqGhFFJyhVs4J+WOBQVyD+Gl/jFQyjIlx2+pu MhKimoEOdOrnzPNHcvd+PyVBaT7XwKG3D8F0JN/L50o1eg3yxEyB/XZ+ACXKiSur3MVQ dbRZNxzuvZo7wKLL1r4bH1vv2xzos3Mv8RuT2x4Fvb7s7QzEQEfdcP1aoQROxxp2qjFY YlpQ== X-Gm-Message-State: AOJu0YzadUpzqJPYnQKoFnKH54w8h87sPLwRsGo0UWM/is8QEeynB/VE RlUJtwWEzqz8+8TFBJrbfhchrTOd32171n2tQjCT7Q== X-Google-Smtp-Source: AGHT+IGDDvozOCDa76ufpb6oqdvzkS1m+Vj3gDj9+0ofrIBpG0JEMPb1JRa+EwNE7MY5wLro39emzA== X-Received: by 2002:a25:4e05:0:b0:d69:8faa:5a28 with SMTP id c5-20020a254e05000000b00d698faa5a28mr25353026ybb.55.1697225136533; Fri, 13 Oct 2023 12:25:36 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id a10-20020a0ce90a000000b006577e289d37sm908408qvo.2.2023.10.13.12.25.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 12:25:36 -0700 (PDT) Date: Fri, 13 Oct 2023 15:25:35 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King Subject: [PATCH 7/8] midx: read `OIDL` chunk with `pair_chunk_expect()` Message-ID: <5844dd9383076b158eba20b9e43019280b74dd76.1697225110.git.me@ttaylorr.com> References: <20231009205544.GA3281950@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Perform an identical conversion as in previous commits to read the OIDL chunk in the MIDX machinery. Signed-off-by: Taylor Blau --- midx.c | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/midx.c b/midx.c index 38bf816cce..74167b8fdb 100644 --- a/midx.c +++ b/midx.c @@ -61,19 +61,6 @@ void get_midx_rev_filename(struct strbuf *out, struct multi_pack_index *m) strbuf_addf(out, "-%s.rev", hash_to_hex(get_midx_checksum(m))); } -static int midx_read_oid_lookup(const unsigned char *chunk_start, - size_t chunk_size, void *data) -{ - struct multi_pack_index *m = data; - m->chunk_oid_lookup = chunk_start; - - if (chunk_size != st_mult(m->hash_len, m->num_objects)) { - error(_("multi-pack-index OID lookup chunk is the wrong size")); - return 1; - } - return 0; -} - static int midx_read_object_offsets(const unsigned char *chunk_start, size_t chunk_size, void *data) { @@ -166,8 +153,11 @@ struct multi_pack_index *load_multi_pack_index(const char *object_dir, int local die(_("multi-pack-index required OID fanout chunk missing or corrupted")); } m->num_objects = ntohl(m->chunk_oid_fanout[255]); - if (read_chunk(cf, MIDX_CHUNKID_OIDLOOKUP, midx_read_oid_lookup, m)) + if (pair_chunk_expect(cf, MIDX_CHUNKID_OIDLOOKUP, &m->chunk_oid_lookup, + st_mult(m->hash_len, m->num_objects))) { + error(_("multi-pack-index OID lookup chunk is the wrong size")); die(_("multi-pack-index required OID lookup chunk missing or corrupted")); + } if (read_chunk(cf, MIDX_CHUNKID_OBJECTOFFSETS, midx_read_object_offsets, m)) die(_("multi-pack-index required object offsets chunk missing or corrupted")); From patchwork Fri Oct 13 19:25:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13421628 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E5AF262BF for ; Fri, 13 Oct 2023 19:25:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="EUCQND5a" Received: from mail-oi1-x22f.google.com (mail-oi1-x22f.google.com [IPv6:2607:f8b0:4864:20::22f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22DF0CE for ; Fri, 13 Oct 2023 12:25:40 -0700 (PDT) Received: by mail-oi1-x22f.google.com with SMTP id 5614622812f47-3af5fd13004so1607048b6e.0 for ; Fri, 13 Oct 2023 12:25:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1697225139; x=1697829939; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=M85EuHX7xw/EodNm4yfeLAowWD6UW8lizY0TRLvsMVE=; b=EUCQND5aA+Oet7XhZ3f+gG1LQnQhB+W8Euvc1H7i7xP1gESs5mHyN+gw57n6+AbkwW 2yZCU/g9EDpMvVtYWT0NftVLSgqlXbPVeqIedOh3hRllqLiIgO1PSoEi5ToptOTrVsOZ snPMQ7moEtzdOb/MARFdbV21YxVzamfTxeuDbNBHkIKo2ya//dtaxbV4H5SJa6aPU7r1 a6x3TVv+VC42Pt6nomko+LI/D7lkrsT7LkBpF9fCCbyP5TYFkpZWL95400rEbacK4cBV Xl4mhES634LNuW5aL0tZ3y3HxRQa94m0uhyuYVqiy3p1vYrOtgbO/oDtjdhTE/gcEvH3 ZQOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697225139; x=1697829939; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=M85EuHX7xw/EodNm4yfeLAowWD6UW8lizY0TRLvsMVE=; b=GE+o2fWDP3ElIjphaLZjeuAWBbiwtnPQb80w+y14CAc4wOmWXCEk4tLLIqnFFb4+VG AZOUmdZHkpLvvdeCDiBZAovLXvNEGcs4npUZiflLPAHKDPlVoHpqQk41JLtSqvES5Fi7 UUDmBirg5wa7Ei3gG8jScjK7XwiR4eVrhbRYwnjheJDKUck3JOIzcGZB0hjCJjmtKKf0 ZMXUDbPYlSNzi+v3mjiYVp5oBeL4XlOgn35auHqnRIfEktTO6lZ9ofOgrLbvcrjLsTx3 tiEr9+WFjRhvPOdnrTNDfxpHu+Uwt0yxfFD2A+S/pwtOoBBq0k7+inANvDCFlIzpx1l+ Js9g== X-Gm-Message-State: AOJu0Yy0mdtnYTHoL6qT8AbMqkBujAxHEMI8s4MQV1eKpLhxPQaG0zQ4 w8764mZiwxkxnpR+8kAf30IyUP+GSiOFE/TRzfsHFQ== X-Google-Smtp-Source: AGHT+IHGAsZzqBRmvNTNpdzMUEy2NXexfFtchUrfLpiicSZDvxih8B+XVYq22bUfthYCtw1T3/6qvQ== X-Received: by 2002:a05:6808:152a:b0:3ad:f4cc:79a5 with SMTP id u42-20020a056808152a00b003adf4cc79a5mr36111957oiw.33.1697225139357; Fri, 13 Oct 2023 12:25:39 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id hg3-20020a05622a610300b004166905aa2asm821240qtb.28.2023.10.13.12.25.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 12:25:39 -0700 (PDT) Date: Fri, 13 Oct 2023 15:25:38 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King Subject: [PATCH 8/8] midx: read `OOFF` chunk with `pair_chunk_expect()` Message-ID: References: <20231009205544.GA3281950@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Perform an identical conversion as in previous commits to read the OOFF chunk in the MIDX machinery. Signed-off-by: Taylor Blau --- midx.c | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-) diff --git a/midx.c b/midx.c index 74167b8fdb..4aeadd5e00 100644 --- a/midx.c +++ b/midx.c @@ -61,19 +61,6 @@ void get_midx_rev_filename(struct strbuf *out, struct multi_pack_index *m) strbuf_addf(out, "-%s.rev", hash_to_hex(get_midx_checksum(m))); } -static int midx_read_object_offsets(const unsigned char *chunk_start, - size_t chunk_size, void *data) -{ - struct multi_pack_index *m = data; - m->chunk_object_offsets = chunk_start; - - if (chunk_size != st_mult(m->num_objects, MIDX_CHUNK_OFFSET_WIDTH)) { - error(_("multi-pack-index object offset chunk is the wrong size")); - return 1; - } - return 0; -} - struct multi_pack_index *load_multi_pack_index(const char *object_dir, int local) { struct multi_pack_index *m = NULL; @@ -158,8 +145,12 @@ struct multi_pack_index *load_multi_pack_index(const char *object_dir, int local error(_("multi-pack-index OID lookup chunk is the wrong size")); die(_("multi-pack-index required OID lookup chunk missing or corrupted")); } - if (read_chunk(cf, MIDX_CHUNKID_OBJECTOFFSETS, midx_read_object_offsets, m)) + if (pair_chunk_expect(cf, MIDX_CHUNKID_OBJECTOFFSETS, + &m->chunk_object_offsets, + st_mult(m->num_objects, MIDX_CHUNK_OFFSET_WIDTH))) { + error(_("multi-pack-index object offset chunk is the wrong size")); die(_("multi-pack-index required object offsets chunk missing or corrupted")); + } pair_chunk(cf, MIDX_CHUNKID_LARGEOFFSETS, &m->chunk_large_offsets, &m->chunk_large_offsets_len);