From patchwork Wed Jan 31 08:01:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13538869 Received: from fhigh2-smtp.messagingengine.com (fhigh2-smtp.messagingengine.com [103.168.172.153]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 07A0A55C18 for ; Wed, 31 Jan 2024 08:01:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.153 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688070; cv=none; b=mJwCz5tturplwVYkyaAUiO53zN/m1yBmxT9FKQ6rSkdFbBtXyCF1810IcsFyI1Z2hpPv5RkIV4vS07L8sX8dMgX2n5+4tg1sNqdWtqI1RTYH0wRF1sETP3lUqR88UWL5BvX9skRkow2F28bcOyFk5XLYORTZscHzcHvsIjbILJs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688070; c=relaxed/simple; bh=UXL9C4hP9c/jzW33/QVl6MmVLM13/RfPKCgoFlgBVNM=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ODlu+rit5SiaW/4H5zURn59fkmsSnwp8gXsHuPwQbj0tk+m7ftgLXt5ioHziWnmZgsSoSzihA+/xAvEHkQgIZAQnIOUUTF31XO0L+xDFBRxUjN5P7ic7GTOyBzBj2XEpg4ot2rcRB9BvjfEOFsaJZ9IRWkK2Vw+I6FkGnSl5sBE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=P5JuNWEd; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=LDhmB9eb; arc=none smtp.client-ip=103.168.172.153 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="P5JuNWEd"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="LDhmB9eb" Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfhigh.nyi.internal (Postfix) with ESMTP id F06741140060 for ; Wed, 31 Jan 2024 03:01:05 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Wed, 31 Jan 2024 03:01:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1706688065; x=1706774465; bh=jM7qfkpP2t rg9sNMFBA19sQCKvDjXI80sDU092Pnwos=; b=P5JuNWEdp0Ixz41Cb5OQlA3zjM Y/kl9a0JLUghTE0yzVSRr+k8kq3yAAdAJR6IaWCO8oPZ0uWLRmrwL+eHCGxHNkhi Zvwj/RHCssWt91Xu2IjAvxiQlcGCLotaOCbwppDuignMWrGEmh8rpebJHympU1x6 mz3C0xpVCdI0P4qNjAhNMJZzYTtrLDqEsOJyO5HUeq2H5XOLgjRzDTZkTZ4SE4Ry XDJBk0H4BbQ7QV7nmHCpEioEcVgqF4DIMbB9RKVVDd7RZkSi1mohB6ujOOF9MTDD BJLxa698PzRNqM0De9wM85JKq2IpKG7BgTwZQqShQZXF5ShVCiLFtNE5OE3A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1706688065; x=1706774465; bh=jM7qfkpP2trg9sNMFBA19sQCKvDj XI80sDU092Pnwos=; b=LDhmB9ebWZf06zVortuIGqpvWurIpDVfk5McVun0zm/S Mjy1LSXDeApCDRJE6VHKAmcmp8OdOsdHxRo8zE39K+xwEi3S1BTRIKBQZYbQameW DrO0lUDOcRzGFD+NLyNJsqAceq4Xgn3Rh27iPKoClV9mLRTZnAm6zVR0FSBh0wy8 v4p/zxM6FqEEwmYmeAzUuyJ+ChKPS+irnhpBp9zLvFWCm4FpIK31V8b2HUwag5Hh 0NBpS/KcLX1I8MfiRDkQ0wm7WUEek48dEGh5O2tljxd17xFePCofG5Q1dtO6zJYK SF3ZkHpTqZOJ56GIZ2tNZwGkaxUq4kMP6G/6GmCV+g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrfedtkedguddufecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtd erredttdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshes phhkshdrihhmqeenucggtffrrghtthgvrhhnpeehgfejueevjeetudehgffffeffvdejfe ejiedvkeffgfekuefgheevteeufeelkeenucevlhhushhtvghrufhiiigvpedtnecurfgr rhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 31 Jan 2024 03:01:05 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id e01ffe87 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 31 Jan 2024 07:57:46 +0000 (UTC) Date: Wed, 31 Jan 2024 09:01:03 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 1/9] reftable: introduce macros to grow arrays Message-ID: <0597e6944a1a65720d050f47bc82766d5bcf860b.1706687982.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Throughout the reftable library we have many cases where we need to grow arrays. In order to avoid too many reallocations, we roughly double the capacity of the array on each iteration. The resulting code pattern is thus duplicated across many sites. We have similar patterns in our main codebase, which is why we have eventually introduced an `ALLOC_GROW()` macro to abstract it away and avoid some code duplication. We cannot easily reuse this macro here though because `ALLOC_GROW()` uses `REALLOC_ARRAY()`, which in turn will call realloc(3P) to grow the array. The reftable code is structured as a library though (even if the boundaries are fuzzy), and one property this brings with it is that it is possible to plug in your own allocators. So instead of using realloc(3P), we need to use `reftable_realloc()` that knows to use the user-provided implementation. So let's introduce two new macros `REFTABLE_REALLOC_ARRAY()` and `REFTABLE_ALLOC_GROW()` that mirror what we do in our main codebase, with two modifications: - They use `reftable_realloc()`, as explained above. - They use a different growth factor of `2 * cap + 1` instead of `(cap + 16) * 3 / 2`. The second change is because we know a bit more about the allocation patterns in the reftable library. For In most cases, we end up only having a single item in the array, so the initial capacity that our global growth factor uses (which is 24), significantly overallocates in a lot of code paths. This effect is indeed measurable: Benchmark 1: update-ref: create many refs (growth factor = 2 * cap + 1) Time (mean ± σ): 4.834 s ± 0.020 s [User: 2.219 s, System: 2.614 s] Range (min … max): 4.793 s … 4.871 s 20 runs Benchmark 2: update-ref: create many refs (growth factor = (cap + 16) * 3 + 2) Time (mean ± σ): 4.933 s ± 0.021 s [User: 2.325 s, System: 2.607 s] Range (min … max): 4.889 s … 4.962 s 20 runs Summary update-ref: create many refs (growth factor = 2 * cap + 1) ran 1.02 ± 0.01 times faster than update-ref: create many refs (growth factor = (cap + 16) * 3 + 2) Convert the reftable library to use these new macros. Signed-off-by: Patrick Steinhardt --- reftable/basics.c | 8 ++------ reftable/basics.h | 11 +++++++++++ reftable/block.c | 7 +------ reftable/merged_test.c | 20 ++++++-------------- reftable/pq.c | 8 ++------ reftable/stack.c | 29 ++++++++++++----------------- reftable/writer.c | 14 ++------------ 7 files changed, 36 insertions(+), 61 deletions(-) diff --git a/reftable/basics.c b/reftable/basics.c index f761e48028..af9004cec2 100644 --- a/reftable/basics.c +++ b/reftable/basics.c @@ -89,17 +89,13 @@ void parse_names(char *buf, int size, char ***namesp) next = end; } if (p < next) { - if (names_len == names_cap) { - names_cap = 2 * names_cap + 1; - names = reftable_realloc( - names, names_cap * sizeof(*names)); - } + REFTABLE_ALLOC_GROW(names, names_len + 1, names_cap); names[names_len++] = xstrdup(p); } p = next + 1; } - names = reftable_realloc(names, (names_len + 1) * sizeof(*names)); + REFTABLE_REALLOC_ARRAY(names, names_len + 1); names[names_len] = NULL; *namesp = names; } diff --git a/reftable/basics.h b/reftable/basics.h index 096b36862b..2f855cd724 100644 --- a/reftable/basics.h +++ b/reftable/basics.h @@ -53,6 +53,17 @@ void *reftable_realloc(void *p, size_t sz); void reftable_free(void *p); void *reftable_calloc(size_t sz); +#define REFTABLE_REALLOC_ARRAY(x, alloc) (x) = reftable_realloc((x), st_mult(sizeof(*(x)), (alloc))) +#define REFTABLE_ALLOC_GROW(x, nr, alloc) \ + do { \ + if ((nr) > alloc) { \ + alloc = 2 * (alloc) + 1; \ + if (alloc < (nr)) \ + alloc = (nr); \ + REFTABLE_REALLOC_ARRAY(x, alloc); \ + } \ + } while (0) + /* Find the longest shared prefix size of `a` and `b` */ struct strbuf; int common_prefix_size(struct strbuf *a, struct strbuf *b); diff --git a/reftable/block.c b/reftable/block.c index 1df3d8a0f0..6952d0facf 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -51,12 +51,7 @@ static int block_writer_register_restart(struct block_writer *w, int n, if (2 + 3 * rlen + n > w->block_size - w->next) return -1; if (is_restart) { - if (w->restart_len == w->restart_cap) { - w->restart_cap = w->restart_cap * 2 + 1; - w->restarts = reftable_realloc( - w->restarts, sizeof(uint32_t) * w->restart_cap); - } - + REFTABLE_ALLOC_GROW(w->restarts, w->restart_len + 1, w->restart_cap); w->restarts[w->restart_len++] = w->next; } diff --git a/reftable/merged_test.c b/reftable/merged_test.c index 46908f738f..e05351e035 100644 --- a/reftable/merged_test.c +++ b/reftable/merged_test.c @@ -231,14 +231,10 @@ static void test_merged(void) while (len < 100) { /* cap loops/recursion. */ struct reftable_ref_record ref = { NULL }; int err = reftable_iterator_next_ref(&it, &ref); - if (err > 0) { + if (err > 0) break; - } - if (len == cap) { - cap = 2 * cap + 1; - out = reftable_realloc( - out, sizeof(struct reftable_ref_record) * cap); - } + + REFTABLE_ALLOC_GROW(out, len + 1, cap); out[len++] = ref; } reftable_iterator_destroy(&it); @@ -368,14 +364,10 @@ static void test_merged_logs(void) while (len < 100) { /* cap loops/recursion. */ struct reftable_log_record log = { NULL }; int err = reftable_iterator_next_log(&it, &log); - if (err > 0) { + if (err > 0) break; - } - if (len == cap) { - cap = 2 * cap + 1; - out = reftable_realloc( - out, sizeof(struct reftable_log_record) * cap); - } + + REFTABLE_ALLOC_GROW(out, len + 1, cap); out[len++] = log; } reftable_iterator_destroy(&it); diff --git a/reftable/pq.c b/reftable/pq.c index dcefeb793a..2461daf5ff 100644 --- a/reftable/pq.c +++ b/reftable/pq.c @@ -75,13 +75,9 @@ void merged_iter_pqueue_add(struct merged_iter_pqueue *pq, const struct pq_entry { int i = 0; - if (pq->len == pq->cap) { - pq->cap = 2 * pq->cap + 1; - pq->heap = reftable_realloc(pq->heap, - pq->cap * sizeof(struct pq_entry)); - } - + REFTABLE_ALLOC_GROW(pq->heap, pq->len + 1, pq->cap); pq->heap[pq->len++] = *e; + i = pq->len - 1; while (i > 0) { int j = (i - 1) / 2; diff --git a/reftable/stack.c b/reftable/stack.c index bf3869ce70..1dfab99e96 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -551,7 +551,7 @@ struct reftable_addition { struct reftable_stack *stack; char **new_tables; - int new_tables_len; + size_t new_tables_len, new_tables_cap; uint64_t next_update_index; }; @@ -602,8 +602,9 @@ static int reftable_stack_init_addition(struct reftable_addition *add, static void reftable_addition_close(struct reftable_addition *add) { - int i = 0; struct strbuf nm = STRBUF_INIT; + size_t i; + for (i = 0; i < add->new_tables_len; i++) { stack_filename(&nm, add->stack, add->new_tables[i]); unlink(nm.buf); @@ -613,6 +614,7 @@ static void reftable_addition_close(struct reftable_addition *add) reftable_free(add->new_tables); add->new_tables = NULL; add->new_tables_len = 0; + add->new_tables_cap = 0; delete_tempfile(&add->lock_file); strbuf_release(&nm); @@ -631,8 +633,8 @@ int reftable_addition_commit(struct reftable_addition *add) { struct strbuf table_list = STRBUF_INIT; int lock_file_fd = get_tempfile_fd(add->lock_file); - int i = 0; int err = 0; + size_t i; if (add->new_tables_len == 0) goto done; @@ -660,12 +662,12 @@ int reftable_addition_commit(struct reftable_addition *add) } /* success, no more state to clean up. */ - for (i = 0; i < add->new_tables_len; i++) { + for (i = 0; i < add->new_tables_len; i++) reftable_free(add->new_tables[i]); - } reftable_free(add->new_tables); add->new_tables = NULL; add->new_tables_len = 0; + add->new_tables_cap = 0; err = reftable_stack_reload_maybe_reuse(add->stack, 1); if (err) @@ -792,11 +794,9 @@ int reftable_addition_add(struct reftable_addition *add, goto done; } - add->new_tables = reftable_realloc(add->new_tables, - sizeof(*add->new_tables) * - (add->new_tables_len + 1)); - add->new_tables[add->new_tables_len] = strbuf_detach(&next_name, NULL); - add->new_tables_len++; + REFTABLE_ALLOC_GROW(add->new_tables, add->new_tables_len + 1, + add->new_tables_cap); + add->new_tables[add->new_tables_len++] = strbuf_detach(&next_name, NULL); done: if (tab_fd > 0) { close(tab_fd); @@ -1367,17 +1367,12 @@ static int stack_check_addition(struct reftable_stack *st, while (1) { struct reftable_ref_record ref = { NULL }; err = reftable_iterator_next_ref(&it, &ref); - if (err > 0) { + if (err > 0) break; - } if (err < 0) goto done; - if (len >= cap) { - cap = 2 * cap + 1; - refs = reftable_realloc(refs, cap * sizeof(refs[0])); - } - + REFTABLE_ALLOC_GROW(refs, len + 1, cap); refs[len++] = ref; } diff --git a/reftable/writer.c b/reftable/writer.c index ee4590e20f..4483bb21c3 100644 --- a/reftable/writer.c +++ b/reftable/writer.c @@ -200,12 +200,7 @@ static void writer_index_hash(struct reftable_writer *w, struct strbuf *hash) return; } - if (key->offset_len == key->offset_cap) { - key->offset_cap = 2 * key->offset_cap + 1; - key->offsets = reftable_realloc( - key->offsets, sizeof(uint64_t) * key->offset_cap); - } - + REFTABLE_ALLOC_GROW(key->offsets, key->offset_len + 1, key->offset_cap); key->offsets[key->offset_len++] = off; } @@ -674,12 +669,7 @@ static int writer_flush_nonempty_block(struct reftable_writer *w) if (err < 0) return err; - if (w->index_cap == w->index_len) { - w->index_cap = 2 * w->index_cap + 1; - w->index = reftable_realloc( - w->index, - sizeof(struct reftable_index_record) * w->index_cap); - } + REFTABLE_ALLOC_GROW(w->index, w->index_len + 1, w->index_cap); ir.offset = w->next; strbuf_reset(&ir.last_key); From patchwork Wed Jan 31 08:01:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13538870 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF79058108 for ; Wed, 31 Jan 2024 08:01:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=66.111.4.25 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688074; cv=none; b=aMo9b+ON3VbsmXYr3lP6jU7MVu0Y99s5BRNoZ9bl7R5aCJtU6pwPms5apy0BDdDzN7IkX8D9uQsDlZg2Hv+rLueACfhtXtRi9GLQo05QjhlcSo0bya8DYY2gvjRNrX6ojLjYR6M5b/u2ovsWw6EZKK/UFaZtscpeg7LDnB3X48Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688074; c=relaxed/simple; bh=D7shY26lhx7frn8WhXSOFh6GLE9qBOtuSbYOAKAIujM=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=VnacEZYDXLR/kxHdk2UT9F6wpd/4p+ss7+a2XZf9c8cjgWQ6RFxWg5dfCYaFQy7oU52/H8OW+3UnKR7OEqtaZs2QiQHJ+ItH4Jsr5VN3QranGvsdHpiu+ULm+HdEtXxWGE0awjVyg9OVUbwDoDw+NR0roeICdLPWtYighIq5/Zs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=hgKYldto; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=BQ3qotGn; arc=none smtp.client-ip=66.111.4.25 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="hgKYldto"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="BQ3qotGn" Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id F33E25C005F for ; Wed, 31 Jan 2024 03:01:10 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Wed, 31 Jan 2024 03:01:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1706688070; x=1706774470; bh=w8qsTRGGdC lI+ODuk+LMHYhIJKrq8BprQQQt4A/Lj6g=; b=hgKYldtoNHq+ZoHNkktLoabNvC OEjsO+mQykpAqIcYNHqIzlJxin0x/4yRVAtB80ryq6Vh8e6vUUo/2HNIL0+d2iT/ jOJZcX1x0wXQlx8TFUN3N685cy6rlyjkYFm98yJMhoKJklSGjz8X5sKdOfcYb5u5 KC9jWbMsCtnBdDUv/R1VvCFhj1TVhUm/KmvgMaTZrLhqq205ETop9idKJH2bCAmt uRI4Xjbfi7jUk5X1zquCDqcAkj3RmqovG9jx33R267aZhyx3UfBwMsAln4icggCf Pr9NxYT7f53YjF+8U87rzae0/OdqaEIoEQwwjCbG6kVeVs/c65z4aw4ZM+xQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1706688070; x=1706774470; bh=w8qsTRGGdClI+ODuk+LMHYhIJKrq 8BprQQQt4A/Lj6g=; b=BQ3qotGnO/sG/JPYXf9wk3pvg/fgym7YdaZe0iT3jW0S HVUxEVeR/iuXy0OuvTCMSeS3jdzNdDa6zW2SqClcAM2IIvOSqX0RE9PjL+Swz6BH 5SGNTmN4aR2QuVP2zeTTktCI9qN6LyvGAiSjkcjqfCbfq/rp0GpZaHbLEX+gv7Ik fTeHqus6OpOwyzB/7r4Nm3Xq7c9Mts7FIJjW/f7MZxtlxVk9BdcwK33e7HAlwk+A zp+jwr1CvNNo8/yuemIdqEu08i0FN/aLe/mMDzvDviNm6L0RHJB7jvkb48HOojSb wcwpdlv7eqxysXa443Ev1MhPWuUE3Iji1GUJB4aUog== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrfedtkedgudduvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtd erredttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshes phhkshdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtd fhvdejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedtnecurfgr rhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 31 Jan 2024 03:01:10 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 50cc7257 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 31 Jan 2024 07:57:50 +0000 (UTC) Date: Wed, 31 Jan 2024 09:01:07 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 2/9] reftable: introduce macros to allocate arrays Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Similar to the preceding commit, let's carry over macros to allocate arrays with `REFTABLE_ALLOC_ARRAY()` and `REFTABLE_CALLOC_ARRAY()`. This requires us to change the signature of `reftable_calloc()`, which only takes a single argument right now and thus puts the burden on the caller to calculate the final array's size. This is a net improvement though as it means that we can now provide proper overflow checks when multiplying the array size with the member size. Convert callsites of `reftable_calloc()` to the new signature, using the new macros where possible. Signed-off-by: Patrick Steinhardt --- reftable/basics.h | 4 +++- reftable/block_test.c | 2 +- reftable/blocksource.c | 4 ++-- reftable/iter.c | 3 +-- reftable/merged.c | 4 ++-- reftable/merged_test.c | 22 +++++++++++++--------- reftable/publicbasics.c | 3 ++- reftable/reader.c | 8 +++----- reftable/readwrite_test.c | 8 +++++--- reftable/record_test.c | 4 ++-- reftable/refname.c | 4 ++-- reftable/stack.c | 26 ++++++++++++-------------- reftable/tree.c | 4 ++-- reftable/writer.c | 7 +++---- 14 files changed, 53 insertions(+), 50 deletions(-) diff --git a/reftable/basics.h b/reftable/basics.h index 2f855cd724..4c3ac963a3 100644 --- a/reftable/basics.h +++ b/reftable/basics.h @@ -51,8 +51,10 @@ int names_length(char **names); void *reftable_malloc(size_t sz); void *reftable_realloc(void *p, size_t sz); void reftable_free(void *p); -void *reftable_calloc(size_t sz); +void *reftable_calloc(size_t nelem, size_t elsize); +#define REFTABLE_ALLOC_ARRAY(x, alloc) (x) = reftable_malloc(st_mult(sizeof(*(x)), (alloc))) +#define REFTABLE_CALLOC_ARRAY(x, alloc) (x) = reftable_calloc((alloc), sizeof(*(x))) #define REFTABLE_REALLOC_ARRAY(x, alloc) (x) = reftable_realloc((x), st_mult(sizeof(*(x)), (alloc))) #define REFTABLE_ALLOC_GROW(x, nr, alloc) \ do { \ diff --git a/reftable/block_test.c b/reftable/block_test.c index dedb05c7d8..e162c6e33f 100644 --- a/reftable/block_test.c +++ b/reftable/block_test.c @@ -36,7 +36,7 @@ static void test_block_read_write(void) int j = 0; struct strbuf want = STRBUF_INIT; - block.data = reftable_calloc(block_size); + REFTABLE_CALLOC_ARRAY(block.data, block_size); block.len = block_size; block.source = malloc_block_source(); block_writer_init(&bw, BLOCK_TYPE_REF, block.data, block_size, diff --git a/reftable/blocksource.c b/reftable/blocksource.c index 8c41e3c70f..eeed254ba9 100644 --- a/reftable/blocksource.c +++ b/reftable/blocksource.c @@ -29,7 +29,7 @@ static int strbuf_read_block(void *v, struct reftable_block *dest, uint64_t off, { struct strbuf *b = v; assert(off + size <= b->len); - dest->data = reftable_calloc(size); + REFTABLE_CALLOC_ARRAY(dest->data, size); memcpy(dest->data, b->buf + off, size); dest->len = size; return size; @@ -132,7 +132,7 @@ int reftable_block_source_from_file(struct reftable_block_source *bs, return REFTABLE_IO_ERROR; } - p = reftable_calloc(sizeof(*p)); + REFTABLE_CALLOC_ARRAY(p, 1); p->size = st.st_size; p->data = xmmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0); close(fd); diff --git a/reftable/iter.c b/reftable/iter.c index a8d174c040..8b5ebf6183 100644 --- a/reftable/iter.c +++ b/reftable/iter.c @@ -160,8 +160,7 @@ int new_indexed_table_ref_iter(struct indexed_table_ref_iter **dest, int oid_len, uint64_t *offsets, int offset_len) { struct indexed_table_ref_iter empty = INDEXED_TABLE_REF_ITER_INIT; - struct indexed_table_ref_iter *itr = - reftable_calloc(sizeof(struct indexed_table_ref_iter)); + struct indexed_table_ref_iter *itr = reftable_calloc(1, sizeof(*itr)); int err = 0; *itr = empty; diff --git a/reftable/merged.c b/reftable/merged.c index c258ce953e..2031fd51b4 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -190,7 +190,7 @@ int reftable_new_merged_table(struct reftable_merged_table **dest, } } - m = reftable_calloc(sizeof(struct reftable_merged_table)); + REFTABLE_CALLOC_ARRAY(m, 1); m->stack = stack; m->stack_len = n; m->min = first_min; @@ -240,7 +240,7 @@ static int merged_table_seek_record(struct reftable_merged_table *mt, struct reftable_record *rec) { struct reftable_iterator *iters = reftable_calloc( - sizeof(struct reftable_iterator) * mt->stack_len); + mt->stack_len, sizeof(*iters)); struct merged_iter merged = { .stack = iters, .typ = reftable_record_type(rec), diff --git a/reftable/merged_test.c b/reftable/merged_test.c index e05351e035..e233a9d581 100644 --- a/reftable/merged_test.c +++ b/reftable/merged_test.c @@ -93,10 +93,12 @@ merged_table_from_records(struct reftable_ref_record **refs, int i = 0; struct reftable_merged_table *mt = NULL; int err; - struct reftable_table *tabs = - reftable_calloc(n * sizeof(struct reftable_table)); - *readers = reftable_calloc(n * sizeof(struct reftable_reader *)); - *source = reftable_calloc(n * sizeof(**source)); + struct reftable_table *tabs; + + REFTABLE_CALLOC_ARRAY(tabs, n); + REFTABLE_CALLOC_ARRAY(*readers, n); + REFTABLE_CALLOC_ARRAY(*source, n); + for (i = 0; i < n; i++) { write_test_table(&buf[i], refs[i], sizes[i]); block_source_from_strbuf(&(*source)[i], &buf[i]); @@ -266,10 +268,12 @@ merged_table_from_log_records(struct reftable_log_record **logs, int i = 0; struct reftable_merged_table *mt = NULL; int err; - struct reftable_table *tabs = - reftable_calloc(n * sizeof(struct reftable_table)); - *readers = reftable_calloc(n * sizeof(struct reftable_reader *)); - *source = reftable_calloc(n * sizeof(**source)); + struct reftable_table *tabs; + + REFTABLE_CALLOC_ARRAY(tabs, n); + REFTABLE_CALLOC_ARRAY(*readers, n); + REFTABLE_CALLOC_ARRAY(*source, n); + for (i = 0; i < n; i++) { write_test_log_table(&buf[i], logs[i], sizes[i], i + 1); block_source_from_strbuf(&(*source)[i], &buf[i]); @@ -412,7 +416,7 @@ static void test_default_write_opts(void) }; int err; struct reftable_block_source source = { NULL }; - struct reftable_table *tab = reftable_calloc(sizeof(*tab) * 1); + struct reftable_table *tab = reftable_calloc(1, sizeof(*tab)); uint32_t hash_id; struct reftable_reader *rd = NULL; struct reftable_merged_table *merged = NULL; diff --git a/reftable/publicbasics.c b/reftable/publicbasics.c index bcb82530d6..44b84a125e 100644 --- a/reftable/publicbasics.c +++ b/reftable/publicbasics.c @@ -37,8 +37,9 @@ void reftable_free(void *p) free(p); } -void *reftable_calloc(size_t sz) +void *reftable_calloc(size_t nelem, size_t elsize) { + size_t sz = st_mult(nelem, elsize); void *p = reftable_malloc(sz); memset(p, 0, sz); return p; diff --git a/reftable/reader.c b/reftable/reader.c index 64dc366fb1..3e0de5e8ad 100644 --- a/reftable/reader.c +++ b/reftable/reader.c @@ -539,8 +539,7 @@ static int reader_seek_indexed(struct reftable_reader *r, if (err == 0) { struct table_iter empty = TABLE_ITER_INIT; - struct table_iter *malloced = - reftable_calloc(sizeof(struct table_iter)); + struct table_iter *malloced = reftable_calloc(1, sizeof(*malloced)); *malloced = empty; table_iter_copy_from(malloced, &next); iterator_from_table_iter(it, malloced); @@ -635,8 +634,7 @@ void reader_close(struct reftable_reader *r) int reftable_new_reader(struct reftable_reader **p, struct reftable_block_source *src, char const *name) { - struct reftable_reader *rd = - reftable_calloc(sizeof(struct reftable_reader)); + struct reftable_reader *rd = reftable_calloc(1, sizeof(*rd)); int err = init_reader(rd, src, name); if (err == 0) { *p = rd; @@ -711,7 +709,7 @@ static int reftable_reader_refs_for_unindexed(struct reftable_reader *r, uint8_t *oid) { struct table_iter ti_empty = TABLE_ITER_INIT; - struct table_iter *ti = reftable_calloc(sizeof(struct table_iter)); + struct table_iter *ti = reftable_calloc(1, sizeof(*ti)); struct filtering_ref_iterator *filter = NULL; struct filtering_ref_iterator empty = FILTERING_REF_ITERATOR_INIT; int oid_len = hash_size(r->hash_id); diff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c index b8a3224016..91ea7a10ef 100644 --- a/reftable/readwrite_test.c +++ b/reftable/readwrite_test.c @@ -56,7 +56,9 @@ static void write_table(char ***names, struct strbuf *buf, int N, int i = 0, n; struct reftable_log_record log = { NULL }; const struct reftable_stats *stats = NULL; - *names = reftable_calloc(sizeof(char *) * (N + 1)); + + REFTABLE_CALLOC_ARRAY(*names, N + 1); + reftable_writer_set_limits(w, update_index, update_index); for (i = 0; i < N; i++) { char name[100]; @@ -188,7 +190,7 @@ static void test_log_overflow(void) static void test_log_write_read(void) { int N = 2; - char **names = reftable_calloc(sizeof(char *) * (N + 1)); + char **names = reftable_calloc(N + 1, sizeof(*names)); int err; struct reftable_write_options opts = { .block_size = 256, @@ -519,7 +521,7 @@ static void test_table_read_write_seek_index(void) static void test_table_refs_for(int indexed) { int N = 50; - char **want_names = reftable_calloc(sizeof(char *) * (N + 1)); + char **want_names = reftable_calloc(N + 1, sizeof(*want_names)); int want_names_len = 0; uint8_t want_hash[GIT_SHA1_RAWSZ]; diff --git a/reftable/record_test.c b/reftable/record_test.c index 2876db7d27..999366ad46 100644 --- a/reftable/record_test.c +++ b/reftable/record_test.c @@ -231,8 +231,8 @@ static void test_reftable_log_record_roundtrip(void) .value_type = REFTABLE_LOG_UPDATE, .value = { .update = { - .new_hash = reftable_calloc(GIT_SHA1_RAWSZ), - .old_hash = reftable_calloc(GIT_SHA1_RAWSZ), + .new_hash = reftable_calloc(GIT_SHA1_RAWSZ, 1), + .old_hash = reftable_calloc(GIT_SHA1_RAWSZ, 1), .name = xstrdup("old name"), .email = xstrdup("old@email"), .message = xstrdup("old message"), diff --git a/reftable/refname.c b/reftable/refname.c index 9573496932..7570e4acf9 100644 --- a/reftable/refname.c +++ b/reftable/refname.c @@ -140,8 +140,8 @@ int validate_ref_record_addition(struct reftable_table tab, { struct modification mod = { .tab = tab, - .add = reftable_calloc(sizeof(char *) * sz), - .del = reftable_calloc(sizeof(char *) * sz), + .add = reftable_calloc(sz, sizeof(*mod.add)), + .del = reftable_calloc(sz, sizeof(*mod.del)), }; int i = 0; int err = 0; diff --git a/reftable/stack.c b/reftable/stack.c index 1dfab99e96..d084823a92 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -50,8 +50,7 @@ static ssize_t reftable_fd_write(void *arg, const void *data, size_t sz) int reftable_new_stack(struct reftable_stack **dest, const char *dir, struct reftable_write_options config) { - struct reftable_stack *p = - reftable_calloc(sizeof(struct reftable_stack)); + struct reftable_stack *p = reftable_calloc(1, sizeof(*p)); struct strbuf list_file_name = STRBUF_INIT; int err = 0; @@ -114,7 +113,7 @@ int read_lines(const char *filename, char ***namesp) int err = 0; if (fd < 0) { if (errno == ENOENT) { - *namesp = reftable_calloc(sizeof(char *)); + REFTABLE_CALLOC_ARRAY(*namesp, 1); return 0; } @@ -191,8 +190,7 @@ void reftable_stack_destroy(struct reftable_stack *st) static struct reftable_reader **stack_copy_readers(struct reftable_stack *st, int cur_len) { - struct reftable_reader **cur = - reftable_calloc(sizeof(struct reftable_reader *) * cur_len); + struct reftable_reader **cur = reftable_calloc(cur_len, sizeof(*cur)); int i = 0; for (i = 0; i < cur_len; i++) { cur[i] = st->readers[i]; @@ -208,9 +206,9 @@ static int reftable_stack_reload_once(struct reftable_stack *st, char **names, int err = 0; int names_len = names_length(names); struct reftable_reader **new_readers = - reftable_calloc(sizeof(struct reftable_reader *) * names_len); + reftable_calloc(names_len, sizeof(*new_readers)); struct reftable_table *new_tables = - reftable_calloc(sizeof(struct reftable_table) * names_len); + reftable_calloc(names_len, sizeof(*new_tables)); int new_readers_len = 0; struct reftable_merged_table *new_merged = NULL; struct strbuf table_path = STRBUF_INIT; @@ -344,7 +342,7 @@ static int reftable_stack_reload_maybe_reuse(struct reftable_stack *st, goto out; } - names = reftable_calloc(sizeof(char *)); + REFTABLE_CALLOC_ARRAY(names, 1); } else { err = fd_read_lines(fd, &names); if (err < 0) @@ -686,7 +684,7 @@ int reftable_stack_new_addition(struct reftable_addition **dest, { int err = 0; struct reftable_addition empty = REFTABLE_ADDITION_INIT; - *dest = reftable_calloc(sizeof(**dest)); + REFTABLE_CALLOC_ARRAY(*dest, 1); **dest = empty; err = reftable_stack_init_addition(*dest, st); if (err) { @@ -871,7 +869,7 @@ static int stack_write_compact(struct reftable_stack *st, { int subtabs_len = last - first + 1; struct reftable_table *subtabs = reftable_calloc( - sizeof(struct reftable_table) * (last - first + 1)); + last - first + 1, sizeof(*subtabs)); struct reftable_merged_table *mt = NULL; int err = 0; struct reftable_iterator it = { NULL }; @@ -979,9 +977,9 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last, int compact_count = last - first + 1; char **listp = NULL; char **delete_on_success = - reftable_calloc(sizeof(char *) * (compact_count + 1)); + reftable_calloc(compact_count + 1, sizeof(*delete_on_success)); char **subtable_locks = - reftable_calloc(sizeof(char *) * (compact_count + 1)); + reftable_calloc(compact_count + 1, sizeof(*subtable_locks)); int i = 0; int j = 0; int is_empty_table = 0; @@ -1204,7 +1202,7 @@ int fastlog2(uint64_t sz) struct segment *sizes_to_segments(int *seglen, uint64_t *sizes, int n) { - struct segment *segs = reftable_calloc(sizeof(struct segment) * n); + struct segment *segs = reftable_calloc(n, sizeof(*segs)); int next = 0; struct segment cur = { 0 }; int i = 0; @@ -1268,7 +1266,7 @@ struct segment suggest_compaction_segment(uint64_t *sizes, int n) static uint64_t *stack_table_sizes_for_compaction(struct reftable_stack *st) { uint64_t *sizes = - reftable_calloc(sizeof(uint64_t) * st->merged->stack_len); + reftable_calloc(st->merged->stack_len, sizeof(*sizes)); int version = (st->config.hash_id == GIT_SHA1_FORMAT_ID) ? 1 : 2; int overhead = header_size(version) - 1; int i = 0; diff --git a/reftable/tree.c b/reftable/tree.c index a5bf880985..528f33ae38 100644 --- a/reftable/tree.c +++ b/reftable/tree.c @@ -20,8 +20,8 @@ struct tree_node *tree_search(void *key, struct tree_node **rootp, if (!insert) { return NULL; } else { - struct tree_node *n = - reftable_calloc(sizeof(struct tree_node)); + struct tree_node *n; + REFTABLE_CALLOC_ARRAY(n, 1); n->key = key; *rootp = n; return *rootp; diff --git a/reftable/writer.c b/reftable/writer.c index 4483bb21c3..47b3448132 100644 --- a/reftable/writer.c +++ b/reftable/writer.c @@ -49,7 +49,7 @@ static int padded_write(struct reftable_writer *w, uint8_t *data, size_t len, { int n = 0; if (w->pending_padding > 0) { - uint8_t *zeroed = reftable_calloc(w->pending_padding); + uint8_t *zeroed = reftable_calloc(w->pending_padding, sizeof(*zeroed)); int n = w->write(w->write_arg, zeroed, w->pending_padding); if (n < 0) return n; @@ -123,8 +123,7 @@ struct reftable_writer * reftable_new_writer(ssize_t (*writer_func)(void *, const void *, size_t), void *writer_arg, struct reftable_write_options *opts) { - struct reftable_writer *wp = - reftable_calloc(sizeof(struct reftable_writer)); + struct reftable_writer *wp = reftable_calloc(1, sizeof(*wp)); strbuf_init(&wp->block_writer_data.last_key, 0); options_set_defaults(opts); if (opts->block_size >= (1 << 24)) { @@ -132,7 +131,7 @@ reftable_new_writer(ssize_t (*writer_func)(void *, const void *, size_t), abort(); } wp->last_key = reftable_empty_strbuf; - wp->block = reftable_calloc(opts->block_size); + REFTABLE_CALLOC_ARRAY(wp->block, opts->block_size); wp->write = writer_func; wp->write_arg = writer_arg; wp->opts = *opts; From patchwork Wed Jan 31 08:01:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13538871 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0D08B59149 for ; Wed, 31 Jan 2024 08:01:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=66.111.4.25 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688078; cv=none; b=NEFo2DXg4SfJkZs7AZuC2/368UMiCP3c1HwSgUjQ66Y8BeBDgspJ2KDBf1geyVgylFRoDjpe8aH4ULFIcwvCRnp5edQeuv9X+gJ9dS8qR40ef4k6LRkhwFms05bUPNlTTT9RwT7O4rdd6+Ttp3YqZVZtD8yNIgGOYaK4ofz5aYg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688078; c=relaxed/simple; bh=G/biQ/V3dor4eyT7nJxcuS5TfgRNoAuDAyjTXCcNPyo=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=idRhU1c9GL9n13LqQSbZxWUAaZNH4xFNj+blQYW2ULwbK/bjVngt6FhAuaAQhJkopR4rvrB0MC+WnBbuNEOpddsThhMcjzB/DWpKB/qIo29PrUQLTypR/0jHKc10ZZVcoRY1Io/vA4GLMhIIRPy69ykxMhpc5B7NUbIar/Xuc3Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=onx4a92h; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=deMPPExJ; arc=none smtp.client-ip=66.111.4.25 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="onx4a92h"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="deMPPExJ" Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id DFCA75C005F for ; Wed, 31 Jan 2024 03:01:14 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Wed, 31 Jan 2024 03:01:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1706688074; x=1706774474; bh=cTjH54JZp9 inPl1mxR7p0G9wgwhDutk8HiItgwz3F/I=; b=onx4a92hJXivwemcBss1EUHLDL xbfyRL8zuCvnyVWJSte9BQl+nI6nqAXUWootnZHJHvyfzlBJ9KnJMrrV2H0LCiTl nq/fovp9rhwols3jx/toXFNfhvwbdKM8jA93W/dK3zkBCfk/FgrY6UdHHDasZH04 9/MMD2/70cn+tNnyiIjLBNT4uYsSTOSA+2r3FBC26rtEgEhSMRJqPnrf8JIwZvX/ 4EWZGeff36ltDKodaNGfE5r04kTqvvBnIr+e6WC6/BnbHTp0m8MAVXRWHe14rpEI xdOlv4Ch/Ob/ViklGA2tsEP0zsKl4wjsidU9QOkBxqKDQfHuGhtd1B3097jg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1706688074; x=1706774474; bh=cTjH54JZp9inPl1mxR7p0G9wgwhD utk8HiItgwz3F/I=; b=deMPPExJOIDhxZRp2V/g2uQBPc+5vzWLrH/ZQHIj/ac9 Bv0eFKqdrIO6gM258RNatTWCSNWncVG02oyIZCEaYev6ESdaTZOkKClgZEkXaNve M+xRkE4JJNZMouztT9wRTkBN9sKYEJahWnbQq5vN46Mk7TLxhlhWrjpN2rR3wI+7 pG2h6NJYixM78XufeiMZE5hTtmmn5luuttwc8nSfuLXA8ln4aSOW9hn08vR1D4Zg FGda1z4RDWXNpjThfUcNGXSj+5OpTgVIb2Iqp0Og85+ML/LCePQnOLKfaXfl8Wrv 9cxuinFzwlEyxdcWQhE4nwudDBJCEm83O8KOEBlBog== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrfedtkedgudduvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtd erredttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshes phhkshdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtd fhvdejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedunecurfgr rhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 31 Jan 2024 03:01:14 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 03f54737 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 31 Jan 2024 07:57:54 +0000 (UTC) Date: Wed, 31 Jan 2024 09:01:11 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 3/9] reftable/stack: fix parameter validation when compacting range Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The `stack_compact_range()` function receives a "first" and "last" index that indicates which tables of the reftable stack should be compacted. Naturally, "first" must be smaller than "last" in order to identify a proper range of tables to compress, which we indeed also assert in the function. But the validations happens after we have already allocated arrays with a size of `last - first + 1`, leading to an underflow and thus an invalid allocation size. Fix this by reordering the array allocations to happen after we have validated parameters. While at it, convert the array allocations to use the newly introduced macros. Note that the relevant variables pointing into arrays should also be converted to use `size_t` instead of `int`. This is left for a later commit in this series. Signed-off-by: Patrick Steinhardt --- reftable/stack.c | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/reftable/stack.c b/reftable/stack.c index d084823a92..b6b24c90bf 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -966,6 +966,7 @@ static int stack_write_compact(struct reftable_stack *st, static int stack_compact_range(struct reftable_stack *st, int first, int last, struct reftable_log_expiry_config *expiry) { + char **delete_on_success = NULL, **subtable_locks = NULL, **listp = NULL; struct strbuf temp_tab_file_name = STRBUF_INIT; struct strbuf new_table_name = STRBUF_INIT; struct strbuf lock_file_name = STRBUF_INIT; @@ -974,12 +975,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last, int err = 0; int have_lock = 0; int lock_file_fd = -1; - int compact_count = last - first + 1; - char **listp = NULL; - char **delete_on_success = - reftable_calloc(compact_count + 1, sizeof(*delete_on_success)); - char **subtable_locks = - reftable_calloc(compact_count + 1, sizeof(*subtable_locks)); + int compact_count; int i = 0; int j = 0; int is_empty_table = 0; @@ -989,6 +985,10 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last, goto done; } + compact_count = last - first + 1; + REFTABLE_CALLOC_ARRAY(delete_on_success, compact_count + 1); + REFTABLE_CALLOC_ARRAY(subtable_locks, compact_count + 1); + st->stats.attempts++; strbuf_reset(&lock_file_name); @@ -1146,12 +1146,14 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last, done: free_names(delete_on_success); - listp = subtable_locks; - while (*listp) { - unlink(*listp); - listp++; + if (subtable_locks) { + listp = subtable_locks; + while (*listp) { + unlink(*listp); + listp++; + } + free_names(subtable_locks); } - free_names(subtable_locks); if (lock_file_fd >= 0) { close(lock_file_fd); lock_file_fd = -1; From patchwork Wed Jan 31 08:01:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13538872 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8CEB5D8E5 for ; Wed, 31 Jan 2024 08:01:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=66.111.4.25 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688081; cv=none; b=lSopoYFtpjmqVyx9PLVwpx3dY9EwSUewZ/E/7Zv7ngr4zZCZFBlEk9KjAhiyg///2JDedgceIe4qOvQoSkCvpWaP76E7WXJqtbneCbo1WjyafGG3m7mgYrucB7eYc21XIS+p4FLnoMOq1dY1AoMAf+ZY4LORks4axkB6vlXi/GE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688081; c=relaxed/simple; bh=jZDKEfs1WpeRna99wyKahiaGerjFe6ZW6IpPH2yc4+g=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=LU7b0sujJ8WpPkFk8mQaOnzWB/lN3sByhRp2UR7iACCisZIQcRMc/cW8VKgSZ4vBPFEKTGKpiQf6SnR6dT0H+oYLmGxvmDgB0HOQnKzsiCT+FCFcWuOy6EpAqWJwNcCU1APLttLh76jij04US6xfHy0qpCEy8N34iOaXtJUqUZY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=IJYFq16Z; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=OH3ZZ3ed; arc=none smtp.client-ip=66.111.4.25 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="IJYFq16Z"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="OH3ZZ3ed" Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id CF0845C005F for ; Wed, 31 Jan 2024 03:01:18 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Wed, 31 Jan 2024 03:01:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1706688078; x=1706774478; bh=qKgnGdCUyf L2nihbtgl+5EvbKhr4gSHKzebbkKNmuiU=; b=IJYFq16ZNPNcPmKJsjvuB7oY3P EBiqoFHXWhC8mnDkIPOFI5wOY/POIQIjBUadPstXylhls8NpK/p7Zam2hICh+MMG nRFwG7mV7cfm58AOMAaTdYnVaKYMOnIFBDYir3jH1/Fm2tJsF5i/ziazk564E4ex wcon8TxRorZPM6/T/zTM6byRa5RyHCj1xXyPwh2wiULuuqxqeXcK4ErlDcAcFESN mmOaYZh+B9BtW+EMIQwM/iahxAngufaihOJ3nIU+JEM8IByuKSRcKREqnK67bcId 5McIzgH1Mq2fmZdGTNXx6XZEGG5VMhrwU5sddH2XxFJwB49vCeASYdkK99aQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1706688078; x=1706774478; bh=qKgnGdCUyfL2nihbtgl+5EvbKhr4 gSHKzebbkKNmuiU=; b=OH3ZZ3ed4OmV7G6FOS2j+Uk11fhH93P12nyPwS/77KzN NNOQYG4bwnK4YGJDWP8Ur9gNB1OrYCCHs52cTEVjEZjoTGQF0BRzlODVEqpTdHDq 8f7iX7KLLoELpsHvyGLKYW0cuv9K5vtxz4aDnLydWO254ekajLWU96OUvr6GMHta dUETlcfUd4U5mE3XwuERb/UlGVI3m/0XsKWbnARQKcgoOlu3qMUsxYAyeeCxhNOJ OgggB9QLl8wpr5OVCrxpOFJ4BxAV7xPSFsOZ0uJ6vlsCtV54rnXnNd9YWZKYhpyw BqRFrFhyiukuBKhO3sw5RAxB5OyoTxPq63+cFp3qaQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrfedtkedguddufecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtd erredttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshes phhkshdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtd fhvdejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedtnecurfgr rhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 31 Jan 2024 03:01:18 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 7e75cbd8 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 31 Jan 2024 07:57:58 +0000 (UTC) Date: Wed, 31 Jan 2024 09:01:15 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 4/9] reftable/stack: index segments with `size_t` Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We use `int`s to index into arrays of segments and track the length of them, which is considered to be a code smell in the Git project. Convert the code to use `size_t` instead. Signed-off-by: Patrick Steinhardt --- reftable/stack.c | 25 +++++++++++-------------- reftable/stack.h | 6 +++--- reftable/stack_test.c | 7 +++---- 3 files changed, 17 insertions(+), 21 deletions(-) diff --git a/reftable/stack.c b/reftable/stack.c index b6b24c90bf..2be3d1e4ba 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -1202,12 +1202,11 @@ int fastlog2(uint64_t sz) return l - 1; } -struct segment *sizes_to_segments(int *seglen, uint64_t *sizes, int n) +struct segment *sizes_to_segments(size_t *seglen, uint64_t *sizes, size_t n) { struct segment *segs = reftable_calloc(n, sizeof(*segs)); - int next = 0; struct segment cur = { 0 }; - int i = 0; + size_t next = 0, i; if (n == 0) { *seglen = 0; @@ -1233,29 +1232,27 @@ struct segment *sizes_to_segments(int *seglen, uint64_t *sizes, int n) return segs; } -struct segment suggest_compaction_segment(uint64_t *sizes, int n) +struct segment suggest_compaction_segment(uint64_t *sizes, size_t n) { - int seglen = 0; - struct segment *segs = sizes_to_segments(&seglen, sizes, n); struct segment min_seg = { .log = 64, }; - int i = 0; + struct segment *segs; + size_t seglen = 0, i; + + segs = sizes_to_segments(&seglen, sizes, n); for (i = 0; i < seglen; i++) { - if (segment_size(&segs[i]) == 1) { + if (segment_size(&segs[i]) == 1) continue; - } - if (segs[i].log < min_seg.log) { + if (segs[i].log < min_seg.log) min_seg = segs[i]; - } } while (min_seg.start > 0) { - int prev = min_seg.start - 1; - if (fastlog2(min_seg.bytes) < fastlog2(sizes[prev])) { + size_t prev = min_seg.start - 1; + if (fastlog2(min_seg.bytes) < fastlog2(sizes[prev])) break; - } min_seg.start = prev; min_seg.bytes += sizes[prev]; diff --git a/reftable/stack.h b/reftable/stack.h index c1e3efa899..d919455669 100644 --- a/reftable/stack.h +++ b/reftable/stack.h @@ -32,13 +32,13 @@ struct reftable_stack { int read_lines(const char *filename, char ***lines); struct segment { - int start, end; + size_t start, end; int log; uint64_t bytes; }; int fastlog2(uint64_t sz); -struct segment *sizes_to_segments(int *seglen, uint64_t *sizes, int n); -struct segment suggest_compaction_segment(uint64_t *sizes, int n); +struct segment *sizes_to_segments(size_t *seglen, uint64_t *sizes, size_t n); +struct segment suggest_compaction_segment(uint64_t *sizes, size_t n); #endif diff --git a/reftable/stack_test.c b/reftable/stack_test.c index 289e902146..2d5b24e5c5 100644 --- a/reftable/stack_test.c +++ b/reftable/stack_test.c @@ -711,7 +711,7 @@ static void test_sizes_to_segments(void) uint64_t sizes[] = { 2, 3, 4, 5, 7, 9 }; /* .................0 1 2 3 4 5 */ - int seglen = 0; + size_t seglen = 0; struct segment *segs = sizes_to_segments(&seglen, sizes, ARRAY_SIZE(sizes)); EXPECT(segs[2].log == 3); @@ -726,7 +726,7 @@ static void test_sizes_to_segments(void) static void test_sizes_to_segments_empty(void) { - int seglen = 0; + size_t seglen = 0; struct segment *segs = sizes_to_segments(&seglen, NULL, 0); EXPECT(seglen == 0); reftable_free(segs); @@ -735,8 +735,7 @@ static void test_sizes_to_segments_empty(void) static void test_sizes_to_segments_all_equal(void) { uint64_t sizes[] = { 5, 5 }; - - int seglen = 0; + size_t seglen = 0; struct segment *segs = sizes_to_segments(&seglen, sizes, ARRAY_SIZE(sizes)); EXPECT(seglen == 1); From patchwork Wed Jan 31 08:01:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13538873 Received: from fout8-smtp.messagingengine.com (fout8-smtp.messagingengine.com [103.168.172.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8CEF5DF17 for ; Wed, 31 Jan 2024 08:01:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.151 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688085; cv=none; b=HIT8rRfnrwfQyLIfYdrPiGE5V+zx4cGtH6dCoFwdR62qodZKeYMZwmSiTXoihFPRT8clsZ0pYMBmy/YlK06VHE61YFkubZRklurqtfYIV8f5rurQsNTn3/h8SzLxBV0S9JdTMUUBQ0BHJ66W7YWwFIVET4cN58GaJdpSveoEonk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688085; c=relaxed/simple; bh=4nB/NsZP/xYrzYmKfNzHdV/3nm71/H1xcUlc5rygZxM=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=nPP5OdyaNJp6nNTdcD2mVrbJ5JxSkw+JElOTmExjPc8ZRhX5cO/w0jVVtKHOGa6qzpMdQw9M4ruvXa013J+1YCN+rYuorrDpzCjmMAoX6F3K6Bajmlu66SHmpt4hs7Do1oagIKzEFYd3FDAKKCSS0wxqsF9lytVqa+/35fH/iFI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=GIdGwD+N; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=MD3JqVQ6; arc=none smtp.client-ip=103.168.172.151 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="GIdGwD+N"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="MD3JqVQ6" Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfout.nyi.internal (Postfix) with ESMTP id C90361380055 for ; Wed, 31 Jan 2024 03:01:22 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Wed, 31 Jan 2024 03:01:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1706688082; x=1706774482; bh=mt0GRmci8m rza66qAbRK7bruz6akpl9rEp7CbueN0mY=; b=GIdGwD+NlKN3g3urVs6Yw/ztH/ l8VFNcztOdMG7U7jVKKcn6+kDO8pn+a4AZYiKF/HlmyWOe5Y4zQT3JsUOQu3NsW0 PXKGl8HcRwU4NwtmZQz/eYDR3rsRJvW9Y4/PQWMbWYs6UfTMVHqgl9apa1TuBF0c fosc+oArf0ITlCX7pHmjp0BeInUGTib0aMLL9THOXn0Jv1XjUR4cayTL53t4gpwq beweKarcRM442g3QSRAQKGbN6qfcC1SPSDz1v4SOh4EnNuK5U/YozItsdKGytxAX DaBl0diEwZ7oQViZx0xrK8KKh2FBCkLad6GPwFwVht3WwaCtMG0/s6OWs4jg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1706688082; x=1706774482; bh=mt0GRmci8mrza66qAbRK7bruz6ak pl9rEp7CbueN0mY=; b=MD3JqVQ6xoRaYt/b5AajelgPv5mz/lnlwlqhpnObM+9Y fEwtEAJYguB8hoU4gkDIpwaoYZDmPvbKDvq4SftCFlFV+y6M9HEP7tHIjQwIXEJJ cQDI3/pODR//wixVUfgtfE6/JhERN9ZF0kGArEVYqNOHKZOwgP7oMIBBHXuDgg3J SKL6Ff8vzNzFsPjW1PvMd2feuzj560m8bI5+LAKK5hHEVLovFlLcHGwQehh0jyHF hU2dbr7SpsJfubg2BhdzoAIVRKy36TcYv2Nfp4rkKBJniyOUOoAN9PmMicCPFVE8 krQ2zAL1JeDEEydAweKi69aNOKar3AreGEc5hqT89g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrfedtkedgudduvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtd erredttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshes phhkshdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtd fhvdejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedtnecurfgr rhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 31 Jan 2024 03:01:22 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id e771f8fa (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 31 Jan 2024 07:58:02 +0000 (UTC) Date: Wed, 31 Jan 2024 09:01:19 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 5/9] reftable/stack: use `size_t` to track stack slices during compaction Message-ID: <2f6a69aa14ddb865548bafa5f3005d0579ea09f9.1706687982.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We use `int`s to track reftable slices when compacting the reftable stack, which is considered to be a code smell in the Git project. Convert the code to use `size_t` instead. Signed-off-by: Patrick Steinhardt --- reftable/stack.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/reftable/stack.c b/reftable/stack.c index 2be3d1e4ba..c1f8cf1cef 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -24,7 +24,8 @@ static int stack_try_add(struct reftable_stack *st, void *arg), void *arg); static int stack_write_compact(struct reftable_stack *st, - struct reftable_writer *wr, int first, int last, + struct reftable_writer *wr, + size_t first, size_t last, struct reftable_log_expiry_config *config); static int stack_check_addition(struct reftable_stack *st, const char *new_tab_name); @@ -820,7 +821,8 @@ uint64_t reftable_stack_next_update_index(struct reftable_stack *st) return 1; } -static int stack_compact_locked(struct reftable_stack *st, int first, int last, +static int stack_compact_locked(struct reftable_stack *st, + size_t first, size_t last, struct strbuf *temp_tab, struct reftable_log_expiry_config *config) { @@ -864,22 +866,21 @@ static int stack_compact_locked(struct reftable_stack *st, int first, int last, } static int stack_write_compact(struct reftable_stack *st, - struct reftable_writer *wr, int first, int last, + struct reftable_writer *wr, + size_t first, size_t last, struct reftable_log_expiry_config *config) { int subtabs_len = last - first + 1; struct reftable_table *subtabs = reftable_calloc( last - first + 1, sizeof(*subtabs)); struct reftable_merged_table *mt = NULL; - int err = 0; struct reftable_iterator it = { NULL }; struct reftable_ref_record ref = { NULL }; struct reftable_log_record log = { NULL }; - uint64_t entries = 0; + int err = 0; - int i = 0, j = 0; - for (i = first, j = 0; i <= last; i++) { + for (size_t i = first, j = 0; i <= last; i++) { struct reftable_reader *t = st->readers[i]; reftable_table_from_reader(&subtabs[j++], t); st->stats.bytes += t->size; @@ -963,7 +964,8 @@ static int stack_write_compact(struct reftable_stack *st, } /* < 0: error. 0 == OK, > 0 attempt failed; could retry. */ -static int stack_compact_range(struct reftable_stack *st, int first, int last, +static int stack_compact_range(struct reftable_stack *st, + size_t first, size_t last, struct reftable_log_expiry_config *expiry) { char **delete_on_success = NULL, **subtable_locks = NULL, **listp = NULL; @@ -972,12 +974,10 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last, struct strbuf lock_file_name = STRBUF_INIT; struct strbuf ref_list_contents = STRBUF_INIT; struct strbuf new_table_path = STRBUF_INIT; + size_t i, j, compact_count; int err = 0; int have_lock = 0; int lock_file_fd = -1; - int compact_count; - int i = 0; - int j = 0; int is_empty_table = 0; if (first > last || (!expiry && first == last)) { @@ -1172,17 +1172,17 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last, int reftable_stack_compact_all(struct reftable_stack *st, struct reftable_log_expiry_config *config) { - return stack_compact_range(st, 0, st->merged->stack_len - 1, config); + return stack_compact_range(st, 0, st->merged->stack_len ? + st->merged->stack_len - 1 : 0, config); } -static int stack_compact_range_stats(struct reftable_stack *st, int first, - int last, +static int stack_compact_range_stats(struct reftable_stack *st, + size_t first, size_t last, struct reftable_log_expiry_config *config) { int err = stack_compact_range(st, first, last, config); - if (err > 0) { + if (err > 0) st->stats.failures++; - } return err; } From patchwork Wed Jan 31 08:01:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13538874 Received: from fout8-smtp.messagingengine.com (fout8-smtp.messagingengine.com [103.168.172.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D2A43605DC for ; Wed, 31 Jan 2024 08:01:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.151 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688089; cv=none; b=lrb0N+HH+qosuyIbOgvJGD4qkpYHG4s5LhCZJnfWYVnDEZLh3kjO6EY8mvJWsiLYbqkiJCHan+8zsiRToqHsZzbbk5pMAshC83KrtsThwZIiRl2fSrMgrUSS8gvveSlmBxNcanJXl1qPtQGMr1DSnb5ZGAXFiTFTGWpLxIjtFFQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688089; c=relaxed/simple; bh=IlWLkimwu7hDayLS0ixmfdfTQwJTSjJ2SHYVJf1wadI=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=U2K/8wVGIrNXmNdph1+6Fouj7q1UcrcUPqa5uWajJT0qdTuxDDmN0ykcFa4mB9jvvR7iYyIsteS6xXDVXPou9L+4uzDuN4CVVwurQps2Tqwuao+K5nUIb6rEOUeQP6sCsACcS4TOze2H+lTJX++8Z2usKnS8iGrRhGUfVcYA3CI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=eFwNA/3o; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=wLH9YROy; arc=none smtp.client-ip=103.168.172.151 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="eFwNA/3o"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="wLH9YROy" Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfout.nyi.internal (Postfix) with ESMTP id C3C481380055 for ; Wed, 31 Jan 2024 03:01:26 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Wed, 31 Jan 2024 03:01:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1706688086; x=1706774486; bh=MYUNbpIQs8 u/2pZEKJB8VAF30s2V9E4S0hPAU82TYP4=; b=eFwNA/3ouMvpkydezaRr14dFWw McRFoXk42xCq+t0HKF44B5uq+YOk53Oj/qcFKMmqeSNbxsL6d71lI8RXz6i2hDHi eGwV/TYiA7hseZZiOj8mmxKgvVJZAFGhTPOvWX3IwWEXRcIuNLyzBSZ34UvlQTVm 00Yt5vllPgVfLpQCooebavg4OWtFroUl69OtDMfBu9T2z2tRpvu9GppiJXFitHfj VqSb11novxKNtT8EyR1olwygWXu7z1koI836biiipUdVfUZFsfL4RM0xjYtRSLb5 eZpAfIpZ721II/TfrhxHg8KXd+HAnw/IYYtsd14t35JSrc8TarAroyydVg6g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1706688086; x=1706774486; bh=MYUNbpIQs8u/2pZEKJB8VAF30s2V 9E4S0hPAU82TYP4=; b=wLH9YROySNmDJq4ph7eNdTQ60o1FfO0KG7Ngxwpj616y lQcbPEROufDeFLmsKaZOKS8yV3tLAMMjna6cb290ujQuSzmyIf2G9TMAC+XjqYAK UcZIG5EJ1RWbCHHz2d2k7qEjEfqGijCJ4uN/hyQMY7ATN6kglqsG7RxGf5ImLeVC VF6vvO5SWe+v37F/WOfCed3//MGnwkSY0DBPxJUdSb2njuyXgDp95urwJ7KrVyJ/ BAxKOgMGgqUIKyiMeXtLnWFmZa/Z0xYaHXyvjAaNNDSBoUC7IBzEwdujp5Vl9u3T xGiewblo23DCzaRyytzLSWsvRi6c4nDojRU6WGqK/g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrfedtkedguddufecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtd erredttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshes phhkshdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtd fhvdejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedunecurfgr rhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 31 Jan 2024 03:01:25 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id ac0059d9 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 31 Jan 2024 07:58:06 +0000 (UTC) Date: Wed, 31 Jan 2024 09:01:23 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 6/9] reftable/stack: use `size_t` to track stack length Message-ID: <1ee9a4477f1bf49258a4d5e9bc446c6c7dae85fb.1706687982.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: While the stack length is already stored as `size_t`, we frequently use `int`s to refer to those stacks throughout the reftable library. Convert those cases to use `size_t` instead to make things consistent. Signed-off-by: Patrick Steinhardt --- reftable/basics.c | 7 +++---- reftable/basics.h | 2 +- reftable/merged.c | 11 +++++------ reftable/merged_test.c | 14 ++++++-------- reftable/reftable-merged.h | 2 +- reftable/stack.c | 21 ++++++++++----------- 6 files changed, 26 insertions(+), 31 deletions(-) diff --git a/reftable/basics.c b/reftable/basics.c index af9004cec2..0785aff941 100644 --- a/reftable/basics.c +++ b/reftable/basics.c @@ -64,12 +64,11 @@ void free_names(char **a) reftable_free(a); } -int names_length(char **names) +size_t names_length(char **names) { char **p = names; - for (; *p; p++) { - /* empty */ - } + while (*p) + p++; return p - names; } diff --git a/reftable/basics.h b/reftable/basics.h index 4c3ac963a3..91f3533efe 100644 --- a/reftable/basics.h +++ b/reftable/basics.h @@ -44,7 +44,7 @@ void parse_names(char *buf, int size, char ***namesp); int names_equal(char **a, char **b); /* returns the array size of a NULL-terminated array of strings. */ -int names_length(char **names); +size_t names_length(char **names); /* Allocation routines; they invoke the functions set through * reftable_set_alloc() */ diff --git a/reftable/merged.c b/reftable/merged.c index 2031fd51b4..e2c6253324 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -45,11 +45,10 @@ static int merged_iter_init(struct merged_iter *mi) static void merged_iter_close(void *p) { struct merged_iter *mi = p; - int i = 0; + merged_iter_pqueue_release(&mi->pq); - for (i = 0; i < mi->stack_len; i++) { + for (size_t i = 0; i < mi->stack_len; i++) reftable_iterator_destroy(&mi->stack[i]); - } reftable_free(mi->stack); strbuf_release(&mi->key); strbuf_release(&mi->entry_key); @@ -168,14 +167,14 @@ static void iterator_from_merged_iter(struct reftable_iterator *it, } int reftable_new_merged_table(struct reftable_merged_table **dest, - struct reftable_table *stack, int n, + struct reftable_table *stack, size_t n, uint32_t hash_id) { struct reftable_merged_table *m = NULL; uint64_t last_max = 0; uint64_t first_min = 0; - int i = 0; - for (i = 0; i < n; i++) { + + for (size_t i = 0; i < n; i++) { uint64_t min = reftable_table_min_update_index(&stack[i]); uint64_t max = reftable_table_max_update_index(&stack[i]); diff --git a/reftable/merged_test.c b/reftable/merged_test.c index e233a9d581..442917cc83 100644 --- a/reftable/merged_test.c +++ b/reftable/merged_test.c @@ -88,18 +88,17 @@ static struct reftable_merged_table * merged_table_from_records(struct reftable_ref_record **refs, struct reftable_block_source **source, struct reftable_reader ***readers, int *sizes, - struct strbuf *buf, int n) + struct strbuf *buf, size_t n) { - int i = 0; struct reftable_merged_table *mt = NULL; - int err; struct reftable_table *tabs; + int err; REFTABLE_CALLOC_ARRAY(tabs, n); REFTABLE_CALLOC_ARRAY(*readers, n); REFTABLE_CALLOC_ARRAY(*source, n); - for (i = 0; i < n; i++) { + for (size_t i = 0; i < n; i++) { write_test_table(&buf[i], refs[i], sizes[i]); block_source_from_strbuf(&(*source)[i], &buf[i]); @@ -263,18 +262,17 @@ static struct reftable_merged_table * merged_table_from_log_records(struct reftable_log_record **logs, struct reftable_block_source **source, struct reftable_reader ***readers, int *sizes, - struct strbuf *buf, int n) + struct strbuf *buf, size_t n) { - int i = 0; struct reftable_merged_table *mt = NULL; - int err; struct reftable_table *tabs; + int err; REFTABLE_CALLOC_ARRAY(tabs, n); REFTABLE_CALLOC_ARRAY(*readers, n); REFTABLE_CALLOC_ARRAY(*source, n); - for (i = 0; i < n; i++) { + for (size_t i = 0; i < n; i++) { write_test_log_table(&buf[i], logs[i], sizes[i], i + 1); block_source_from_strbuf(&(*source)[i], &buf[i]); diff --git a/reftable/reftable-merged.h b/reftable/reftable-merged.h index 1a6d16915a..c91a2d83a2 100644 --- a/reftable/reftable-merged.h +++ b/reftable/reftable-merged.h @@ -33,7 +33,7 @@ struct reftable_table; the stack array. */ int reftable_new_merged_table(struct reftable_merged_table **dest, - struct reftable_table *stack, int n, + struct reftable_table *stack, size_t n, uint32_t hash_id); /* returns an iterator positioned just before 'name' */ diff --git a/reftable/stack.c b/reftable/stack.c index c1f8cf1cef..079ba7fde8 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -202,18 +202,18 @@ static struct reftable_reader **stack_copy_readers(struct reftable_stack *st, static int reftable_stack_reload_once(struct reftable_stack *st, char **names, int reuse_open) { - int cur_len = !st->merged ? 0 : st->merged->stack_len; + size_t cur_len = !st->merged ? 0 : st->merged->stack_len; struct reftable_reader **cur = stack_copy_readers(st, cur_len); - int err = 0; - int names_len = names_length(names); + size_t names_len = names_length(names); struct reftable_reader **new_readers = reftable_calloc(names_len, sizeof(*new_readers)); struct reftable_table *new_tables = reftable_calloc(names_len, sizeof(*new_tables)); - int new_readers_len = 0; + size_t new_readers_len = 0; struct reftable_merged_table *new_merged = NULL; struct strbuf table_path = STRBUF_INIT; - int i; + int err = 0; + size_t i; while (*names) { struct reftable_reader *rd = NULL; @@ -221,11 +221,10 @@ static int reftable_stack_reload_once(struct reftable_stack *st, char **names, /* this is linear; we assume compaction keeps the number of tables under control so this is not quadratic. */ - int j = 0; - for (j = 0; reuse_open && j < cur_len; j++) { - if (cur[j] && 0 == strcmp(cur[j]->name, name)) { - rd = cur[j]; - cur[j] = NULL; + for (i = 0; reuse_open && i < cur_len; i++) { + if (cur[i] && 0 == strcmp(cur[i]->name, name)) { + rd = cur[i]; + cur[i] = NULL; break; } } @@ -870,7 +869,7 @@ static int stack_write_compact(struct reftable_stack *st, size_t first, size_t last, struct reftable_log_expiry_config *config) { - int subtabs_len = last - first + 1; + size_t subtabs_len = last - first + 1; struct reftable_table *subtabs = reftable_calloc( last - first + 1, sizeof(*subtabs)); struct reftable_merged_table *mt = NULL; From patchwork Wed Jan 31 08:01:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13538875 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A87E261696 for ; Wed, 31 Jan 2024 08:01:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=66.111.4.25 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688093; cv=none; b=EdnE4yXuJye2Z3e/H4iT5GfC86ankVUfCCTA6UyP/B2yOVOoQhq+zz9t3ANPos976sONNvsc55CkROVtrOd2PZplFg6qE+UAURLJOphF8w/Dvxgsb16TZHLrnX3C9hAEoGR6thzQMp7a68A7HBCdzQCx0aXOb3yk5L58KctCyFM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688093; c=relaxed/simple; bh=HOSxfe73dGqmqAgq0NJD2FgO9lQXECB5cZE1QHuIKkg=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=tvCu+R2j2Nm2tDsMIN6Hdqm21REi4FY4kL7NyiOB8ffwWBGz7v4Um1pV/Nv2T7DmZbTBmeh97UR2r1dHknLYF2AEvEyT8kshN61G3rdOF4hH1SV0SAioOO1LRXsjZRnM+Wfd+lIw96pDIS8X0QAAvoLVQIs9fexPuyyUDzDNhao= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=CqtU06Mh; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=dBaKiGTk; arc=none smtp.client-ip=66.111.4.25 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="CqtU06Mh"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="dBaKiGTk" Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id BC4605C0077 for ; Wed, 31 Jan 2024 03:01:30 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Wed, 31 Jan 2024 03:01:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1706688090; x=1706774490; bh=ep7cVr0apq WP9wylJEYLwhu4Ud7E9zYIyCPMRgkeLtM=; b=CqtU06Mh9spEntnehgb44y2ek+ K8/lUO76ptVC5ASfBSkHrQGEUQd8Sd4jSmDMTRgx2N5VuHU4Coi/iaLDa9Gv2uyN Rc9GJ8o3XyENgqw17RuhQ5TqRjUq7VTxnX7mf8n0XlWPwVikDiw+qI9kDkUzjk56 Ip0qAi2REO+UEULibsOZ49i2lePoR77slDUI5VOZLQhrNoF96an+8MYC2wDrlVeW 5WuTYiBqu+WTLAPVN3rtpCeVThSrTeCl47uMqLI3qgLVnX2e7ZjfeaaH9P8OVkpr KDF8xu5Xeym0jD2IUBg0JhuEJMM1Q2XROciTjpnkqHPvbwlmCaTwP1P6+jTA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1706688090; x=1706774490; bh=ep7cVr0apqWP9wylJEYLwhu4Ud7E 9zYIyCPMRgkeLtM=; b=dBaKiGTkjpfGZehz257vlmeFmltsSPF6e4zUS97h0pIs TMIJEUpcaNqFB5wCHUZdd9mDgzkFt4fUcXx7RbUW9qAYcL18Gsmw87bFfFevKXHC kiJ50aycDb/CMT+A03YYTHYvXLvS00PtX9ggJ4/d2jFNDi4wJu8N33vA2onZgxWu rA3NmqaUexIguDv8Tmgv3UUHCXRFQAosn/3ByeL71rcIcn6A0ogtkjlhFBadM/bA lvrDXNGDibeE3Z95QkNXUhKu7OmO5t3YZwfvgMzATEe71VE8W6KKg7xyPhrk8Rlh w5HvxB+/8PnWedFF7+FLwnwbQYhkOviHs1pIbv1/hQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrfedtkedgudduvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtd erredttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshes phhkshdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtd fhvdejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedtnecurfgr rhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 31 Jan 2024 03:01:29 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 937fc14e (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 31 Jan 2024 07:58:10 +0000 (UTC) Date: Wed, 31 Jan 2024 09:01:28 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 7/9] reftable/merged: refactor seeking of records Message-ID: <00aeaeee63dc1371d3338b72f779c6f165bd14f7.1706687982.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The code to seek reftable records in the merged table code is quite hard to read and does not conform to our coding style in multiple ways: - We have multiple exit paths where we release resources even though that is not really necessary - We use a scoped error variable `e` which is hard to reason about. This variable is not required at all. - We allocate memory in the variable declarations, which is easy to miss. Refactor the function so that it becomes more maintainable in the future. Signed-off-by: Patrick Steinhardt --- reftable/merged.c | 54 ++++++++++++++++++----------------------------- 1 file changed, 21 insertions(+), 33 deletions(-) diff --git a/reftable/merged.c b/reftable/merged.c index e2c6253324..0abcda26e8 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -238,50 +238,38 @@ static int merged_table_seek_record(struct reftable_merged_table *mt, struct reftable_iterator *it, struct reftable_record *rec) { - struct reftable_iterator *iters = reftable_calloc( - mt->stack_len, sizeof(*iters)); struct merged_iter merged = { - .stack = iters, .typ = reftable_record_type(rec), .hash_id = mt->hash_id, .suppress_deletions = mt->suppress_deletions, .key = STRBUF_INIT, .entry_key = STRBUF_INIT, }; - int n = 0; - int err = 0; - int i = 0; - for (i = 0; i < mt->stack_len && err == 0; i++) { - int e = reftable_table_seek_record(&mt->stack[i], &iters[n], - rec); - if (e < 0) { - err = e; - } - if (e == 0) { - n++; - } - } - if (err < 0) { - int i = 0; - for (i = 0; i < n; i++) { - reftable_iterator_destroy(&iters[i]); - } - reftable_free(iters); - return err; + struct merged_iter *p; + int err; + + REFTABLE_CALLOC_ARRAY(merged.stack, mt->stack_len); + for (size_t i = 0; i < mt->stack_len; i++) { + err = reftable_table_seek_record(&mt->stack[i], + &merged.stack[merged.stack_len], rec); + if (err < 0) + goto out; + if (!err) + merged.stack_len++; } - merged.stack_len = n; err = merged_iter_init(&merged); - if (err < 0) { + if (err < 0) + goto out; + + p = reftable_malloc(sizeof(struct merged_iter)); + *p = merged; + iterator_from_merged_iter(it, p); + +out: + if (err < 0) merged_iter_close(&merged); - return err; - } else { - struct merged_iter *p = - reftable_malloc(sizeof(struct merged_iter)); - *p = merged; - iterator_from_merged_iter(it, p); - } - return 0; + return err; } int reftable_merged_table_seek_ref(struct reftable_merged_table *mt, From patchwork Wed Jan 31 08:01:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13538876 Received: from fout8-smtp.messagingengine.com (fout8-smtp.messagingengine.com [103.168.172.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F01A464CC8 for ; Wed, 31 Jan 2024 08:01:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.151 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688097; cv=none; b=ueK44uL4WT95bqbkWn6oOXJFPUNZ7IqMz9pIKgsezUWcickCD5OsORUg0lyPmErYPzqiFmq0qww2Juf7wiDkWdaMTrIN1vlPPL/7qBELWrQDNKc9snqXyyUv+anixGfEzdD1hoxJjaUDlfNpiWW+rvhMt6FNCyTw6KLiuAp+SAk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688097; c=relaxed/simple; bh=+GLYGqEilEO+U4jzAMvFtz/R8Hotjjlqb42hciZJSsQ=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Dl044vupc3uDi8weW541nPN4DPKnMDIW0Oko5EpFTJMZwOC8Z03KwFUOP1gR7iNgw1Jj7LSNCl7ZzuNvW6Iksv9vUIzzb8XgZqryUtWKYppXnJBKhKUkl7t4ULVk8S0pNVBc0Q5CxpOopoRwChnLH+L2BosXKbe0fc47ooSMdQ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=foF6D6n8; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=DcPvMwje; arc=none smtp.client-ip=103.168.172.151 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="foF6D6n8"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="DcPvMwje" Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfout.nyi.internal (Postfix) with ESMTP id B92B01380055 for ; Wed, 31 Jan 2024 03:01:34 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Wed, 31 Jan 2024 03:01:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1706688094; x=1706774494; bh=iQ86pfNUUM Run/a1kr3yZoYgaMu3ANPbW2p688YEu1w=; b=foF6D6n8VHalBFSCRB5gwV2pUb oJrrAuyAYg/GJNCWUTREvQaqnXx7ango058Zv39Qk5V70NrbBowo12/nYm9DXy04 Oz75OXGz5gNQI58DwVLQBS8xNnx/yfVwg8WsBfOUoEfe7G936pe0ius76uptNd83 OVnVscCx5GfEOt6ZO05IkOEYVi0rI2pCiNlnOEG2dNwaAuNCx+PIBESEIiIjlW8g dd2dfXg/HlbLcH8YiSZ0jepp+GFvEsFbJKCt3/2DfBFGitjs2M1l2qAYjJz9JZf1 Ce/m5t3G1sGrPVqQiAdBUutvmLay2FgpsPvFSRmqyuclb/15LSSDRS5QxzkQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1706688094; x=1706774494; bh=iQ86pfNUUMRun/a1kr3yZoYgaMu3 ANPbW2p688YEu1w=; b=DcPvMwjeat05mjl5dC6bTi8xsVz1GrffVr+LmxQPNmWs OJvjJ7yoBmyXBC3FBZdXaZ968Y3HcCiE85LFGqzJA1I+qSNogRgW3yRrTcPNe1I1 WkQCf/f547q4ZSYNahAAvt6UC0cLFd0PBtSNGkrh8qxBV2yfLTLqwgqoZRIz0Evl xugA60mggmyU7NCvXtN62GFTtoUXtyb9DAJK5fnbcCIX33sGx2fjSQ3HkOdaNrM6 BxLJCZfsHXhk+OrRrX2rV1xP0HDkK3Qy9lAZLJ3KRd0cRotlsJRZ3Y8aCMbj/6Ee 6mExlmEJS9WnDhXZUSTWdJCjrpEOYVYPnWQAvWd6Mg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrfedtkedguddufecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtd erredttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshes phhkshdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtd fhvdejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedvnecurfgr rhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 31 Jan 2024 03:01:33 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 8cfa7b36 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 31 Jan 2024 07:58:14 +0000 (UTC) Date: Wed, 31 Jan 2024 09:01:32 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 8/9] reftable/merged: refactor initialization of iterators Message-ID: <31864740e996368f7b87cdd47e828502fe0e80ad.1706687982.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Refactor the initialization of the merged iterator to fit our code style better. This refactoring prepares the code for a refactoring of how records are being initialized. Signed-off-by: Patrick Steinhardt --- reftable/merged.c | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/reftable/merged.c b/reftable/merged.c index 0abcda26e8..0e60e2a39b 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -19,24 +19,23 @@ license that can be found in the LICENSE file or at static int merged_iter_init(struct merged_iter *mi) { - int i = 0; - for (i = 0; i < mi->stack_len; i++) { - struct reftable_record rec = reftable_new_record(mi->typ); - int err = iterator_next(&mi->stack[i], &rec); - if (err < 0) { + for (size_t i = 0; i < mi->stack_len; i++) { + struct pq_entry e = { + .rec = reftable_new_record(mi->typ), + .index = i, + }; + int err; + + err = iterator_next(&mi->stack[i], &e.rec); + if (err < 0) return err; - } - if (err > 0) { reftable_iterator_destroy(&mi->stack[i]); - reftable_record_release(&rec); - } else { - struct pq_entry e = { - .rec = rec, - .index = i, - }; - merged_iter_pqueue_add(&mi->pq, &e); + reftable_record_release(&e.rec); + continue; } + + merged_iter_pqueue_add(&mi->pq, &e); } return 0; From patchwork Wed Jan 31 08:01:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13538877 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1514664CF for ; Wed, 31 Jan 2024 08:01:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=66.111.4.25 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688101; cv=none; b=by2+Tty/qpp24CY1+u8qSvs9aJMtrmFKDxvQDavWhBAEpWgExtAwG1P1FeYu6XGST1fyzLAfphlG/xW7gbMgdyTuV0jXYx1lFyveeswZ7isHINKWiUULrO3bMbHysD+XV/l+E9gYVOEX1icLUJmyN0aSGJ/ibbaXZ/N690NIIvk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706688101; c=relaxed/simple; bh=xTYOTPIfubl7CzDC15qp8y4ivyhMT0QUPSQnZkEBCE0=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=L4BkVB1M11J8nyUdnHBh3SCbBjifqu79x/0V7BpouETC/DcACtPNo8Xuk/RbrSBRkBFbzvJthVaGxq7p+8IcT7580EAC4ye+r4BaayhwvV+/t3+8yOUZv15muV/e04J8lCKHALqQpgFDXVkMR3gImqf69VMq3kmaoQkVhIN8xmI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=k/fXeLuo; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=JxnTZo4G; arc=none smtp.client-ip=66.111.4.25 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="k/fXeLuo"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="JxnTZo4G" Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id B9E725C00A3 for ; Wed, 31 Jan 2024 03:01:38 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Wed, 31 Jan 2024 03:01:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1706688098; x=1706774498; bh=TRADLeOCBg Bv+izm7XRZ5EE4kxu8BIj7IfgZpIRAb/w=; b=k/fXeLuos5Y4idv3uh6HRpcLqF gtGZNEzU/lwQeXL9WUr57D9zAJe5A7BwsYvUmFC0/jJNHrLIhxZ+8kn2l/MAWxEI T5Vy1OOMr5rkvraWmoHYHevMkU5pv4ZICG0UonqbCsRAWrZdJ2UIzdhV/0mwhIpi bGCF7VV+4UzdqRT8I141RSYg+zGCty6+RlTOtf0p41blb2xPq0SWLyxM3T1IapIm e19WM/c4eWaitqzI8andoM0NFtQyHXKNCROjB27vMuWEF6rXPgWD64Dac4HTbt4t doE8wwK7GHfw0+FWFhKR+hKhmfIli5cS6fY26slmDEt3PqIZ0WIXYDqHxeJA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1706688098; x=1706774498; bh=TRADLeOCBgBv+izm7XRZ5EE4kxu8 BIj7IfgZpIRAb/w=; b=JxnTZo4G7PotUtRYl/mgN+PF83RviBZv4uTmE+hpA03S +7+T4O+Hx26ds9IbfCB416sRyDIyISaOUjhSnCM39h3/+/pASYi6mlUZVLef4IIm eX7XL+p0aeqPETQcgk+M8NjHOWnY4zctZm+g3DvPjQQo58q8gkDmuMt7UNuWFAre +sA0RH5WcDK1bbU69pHVa+nVFtExpWJOABNvbryiS+p/Xh4ARCXWWtvmP6w1decU Tpdkz/sshO5uj1/sFPL2qiPyK24SygAqSbckm24bMqv3yvg59I1YmgQy9p3nY7Z4 ToBAoDNfBm1ZtI0aE1n5zmx+W1zj7UIJ2TyTgyEclA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrfedtkedguddufecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtd erredttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshes phhkshdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtd fhvdejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedvnecurfgr rhgrmhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 31 Jan 2024 03:01:37 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 39946f3e (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 31 Jan 2024 07:58:18 +0000 (UTC) Date: Wed, 31 Jan 2024 09:01:36 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 9/9] reftable/record: improve semantics when initializing records Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: According to our usual coding style, the `reftable_new_record()` function would indicate that it is allocating a new record. This is not the case though as the function merely initializes records without allocating any memory. Replace `reftable_new_record()` with a new `reftable_record_init()` function that takes a record pointer as input and initializes it accordingly. Signed-off-by: Patrick Steinhardt --- reftable/block.c | 18 +++++++++--------- reftable/merged.c | 8 +++++--- reftable/reader.c | 4 ++-- reftable/record.c | 43 ++++++++++-------------------------------- reftable/record.h | 10 +++++----- reftable/record_test.c | 4 ++-- 6 files changed, 33 insertions(+), 54 deletions(-) diff --git a/reftable/block.c b/reftable/block.c index 6952d0facf..9ad220747e 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -380,23 +380,23 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it, .key = *want, .r = br, }; - struct reftable_record rec = reftable_new_record(block_reader_type(br)); - int err = 0; struct block_iter next = BLOCK_ITER_INIT; + struct reftable_record rec; + int err = 0, i; - int i = binsearch(br->restart_count, &restart_key_less, &args); if (args.error) { err = REFTABLE_FORMAT_ERROR; goto done; } - it->br = br; - if (i > 0) { - i--; - it->next_off = block_reader_restart_offset(br, i); - } else { + i = binsearch(br->restart_count, &restart_key_less, &args); + if (i > 0) + it->next_off = block_reader_restart_offset(br, i - 1); + else it->next_off = br->header_off + 4; - } + it->br = br; + + reftable_record_init(&rec, block_reader_type(br)); /* We're looking for the last entry less/equal than the wanted key, so we have to go one entry too far and then back up. diff --git a/reftable/merged.c b/reftable/merged.c index 0e60e2a39b..a0f222e07b 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -21,11 +21,11 @@ static int merged_iter_init(struct merged_iter *mi) { for (size_t i = 0; i < mi->stack_len; i++) { struct pq_entry e = { - .rec = reftable_new_record(mi->typ), .index = i, }; int err; + reftable_record_init(&e.rec, mi->typ); err = iterator_next(&mi->stack[i], &e.rec); if (err < 0) return err; @@ -57,10 +57,12 @@ static int merged_iter_advance_nonnull_subiter(struct merged_iter *mi, size_t idx) { struct pq_entry e = { - .rec = reftable_new_record(mi->typ), .index = idx, }; - int err = iterator_next(&mi->stack[idx], &e.rec); + int err; + + reftable_record_init(&e.rec, mi->typ); + err = iterator_next(&mi->stack[idx], &e.rec); if (err < 0) return err; diff --git a/reftable/reader.c b/reftable/reader.c index 3e0de5e8ad..5e6c8f30a1 100644 --- a/reftable/reader.c +++ b/reftable/reader.c @@ -444,13 +444,13 @@ static int reader_start(struct reftable_reader *r, struct table_iter *ti, static int reader_seek_linear(struct table_iter *ti, struct reftable_record *want) { - struct reftable_record rec = - reftable_new_record(reftable_record_type(want)); struct strbuf want_key = STRBUF_INIT; struct strbuf got_key = STRBUF_INIT; struct table_iter next = TABLE_ITER_INIT; + struct reftable_record rec; int err = -1; + reftable_record_init(&rec, reftable_record_type(want)); reftable_record_key(want, &want_key); while (1) { diff --git a/reftable/record.c b/reftable/record.c index 5c3fbb7b2a..8b84b8157f 100644 --- a/reftable/record.c +++ b/reftable/record.c @@ -1257,45 +1257,22 @@ reftable_record_vtable(struct reftable_record *rec) abort(); } -struct reftable_record reftable_new_record(uint8_t typ) +void reftable_record_init(struct reftable_record *rec, uint8_t typ) { - struct reftable_record clean = { - .type = typ, - }; + memset(rec, 0, sizeof(*rec)); + rec->type = typ; - /* the following is involved, but the naive solution (just return - * `clean` as is, except for BLOCK_TYPE_INDEX), returns a garbage - * clean.u.obj.offsets pointer on Windows VS CI. Go figure. - */ switch (typ) { - case BLOCK_TYPE_OBJ: - { - struct reftable_obj_record obj = { 0 }; - clean.u.obj = obj; - break; - } - case BLOCK_TYPE_INDEX: - { - struct reftable_index_record idx = { - .last_key = STRBUF_INIT, - }; - clean.u.idx = idx; - break; - } case BLOCK_TYPE_REF: - { - struct reftable_ref_record ref = { 0 }; - clean.u.ref = ref; - break; - } case BLOCK_TYPE_LOG: - { - struct reftable_log_record log = { 0 }; - clean.u.log = log; - break; - } + case BLOCK_TYPE_OBJ: + return; + case BLOCK_TYPE_INDEX: + strbuf_init(&rec->u.idx.last_key, 0); + return; + default: + BUG("unhandled record type"); } - return clean; } void reftable_record_print(struct reftable_record *rec, int hash_size) diff --git a/reftable/record.h b/reftable/record.h index fd80cd451d..e64ed30c80 100644 --- a/reftable/record.h +++ b/reftable/record.h @@ -69,9 +69,6 @@ struct reftable_record_vtable { /* returns true for recognized block types. Block start with the block type. */ int reftable_is_block_type(uint8_t typ); -/* return an initialized record for the given type */ -struct reftable_record reftable_new_record(uint8_t typ); - /* Encode `key` into `dest`. Sets `is_restart` to indicate a restart. Returns * number of bytes written. */ int reftable_encode_key(int *is_restart, struct string_view dest, @@ -100,8 +97,8 @@ struct reftable_obj_record { /* record is a generic wrapper for different types of records. It is normally * created on the stack, or embedded within another struct. If the type is * known, a fresh instance can be initialized explicitly. Otherwise, use - * reftable_new_record() to initialize generically (as the index_record is not - * valid as 0-initialized structure) + * `reftable_record_init()` to initialize generically (as the index_record is + * not valid as 0-initialized structure) */ struct reftable_record { uint8_t type; @@ -113,6 +110,9 @@ struct reftable_record { } u; }; +/* Initialize the reftable record for the given type */ +void reftable_record_init(struct reftable_record *rec, uint8_t typ); + /* see struct record_vtable */ int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size); void reftable_record_print(struct reftable_record *rec, int hash_size); diff --git a/reftable/record_test.c b/reftable/record_test.c index 999366ad46..a86cff5526 100644 --- a/reftable/record_test.c +++ b/reftable/record_test.c @@ -16,11 +16,11 @@ static void test_copy(struct reftable_record *rec) { - struct reftable_record copy = { 0 }; + struct reftable_record copy; uint8_t typ; typ = reftable_record_type(rec); - copy = reftable_new_record(typ); + reftable_record_init(©, typ); reftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ); /* do it twice to catch memory leaks */ reftable_record_copy_from(©, rec, GIT_SHA1_RAWSZ);