From patchwork Mon Apr 7 13:16:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040500 Received: from fout-a1-smtp.messagingengine.com (fout-a1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1172C253B77 for ; Mon, 7 Apr 2025 13:16:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031788; cv=none; b=q6NF3feQzhtdEvl2dDTD20JwHBuXtfb/v0pKyibW1sWScPuJsZCyijWoo0sgqn9V4RUGCpxTUhmgrdgz8ZlWnP48E5x1H7hnZo/Nbklt/kc34hK5HOk8waw3Xhzb7W3buzzg4E6fzb1s03PTtBX4Gpn47rJDIAIK9833Y2/LQJI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031788; c=relaxed/simple; bh=GLyRQ4e+dTHSNrhNHOIVlTILH9u3BSUx03TMNOsITcg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=gP2/8X70fm0Sv5iIRaPJ6f33fdbFW4CrhzQlUIMPARmnHkOwBqC6C03KpQWTmGoZe2+dO0qFHK5VyC4KGEtPAElFCr7iIvIuFEkAb5gRDZmCy6rZhqlfoZYGe47BMrsPLX1kV2XSryiBDbG95iXrEVeJVDI0O1nz2LY5qi+z5/s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=nE2K5QUS; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=pJrw1Deh; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="nE2K5QUS"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="pJrw1Deh" Received: from phl-compute-09.internal (phl-compute-09.phl.internal [10.202.2.49]) by mailfout.phl.internal (Postfix) with ESMTP id 0DACA1380228; Mon, 7 Apr 2025 09:16:25 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-09.internal (MEProxy); Mon, 07 Apr 2025 09:16:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031785; x=1744118185; bh=mNfctF1n7F9ALuXmHRaEGIW+LNfS6gGc78rc2CAT3S4=; b= nE2K5QUSbGWoC6eF7l5mrNoCPWtTBuDK8b1Ff2Ipb+3GD7Sq0+Y/tQ619a/xwojz C3c8P7MQ7K2fc6cLveauBK+iTzIx1fF6teZ2NAlLvMSarP1e1p9QMwvPYzacs62i tXgXwWyo4NRhMCRJt96F01X/uWyQ82dDi+fwvX/vK7rhP/fcnPNccBJgQuJBFowP nbcYONm5e21Zi/MapMNbb0orOQEjkLVO9h9xyy5AwS523pCQRTyigzbfQafakrdD EVwhWthbqbqfwFDX5X9eu04m0eW33aXX+CooosJE80sf0Ps/x7hwlIqZ2vUdKH9q JFAT2Fbk/7kwG0jz1Qxhyw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031785; x= 1744118185; bh=mNfctF1n7F9ALuXmHRaEGIW+LNfS6gGc78rc2CAT3S4=; b=p Jrw1DehwbjWz3otCnumybCO/hWTPlD185/6vEnNzKk7zjfkyXZdUm9smV5t2PIAS ibBb3zTH/GDO65Y1FApD6vvqEMPRTyhzFJmBonB3KSgEyLBdPgXyMnqPuIVIqtbw ey64h40I4WcQkXRGbiZAe05SmfaDKIH2TiLkpdmF0ChDCJRcKnsuRMzz6GdXPJJS nPFdNiJLE/WX4lza3QBINEa5IjfZp9dSq5fKhTRspWC8KkJy/EgO5BPN6ZBgxCHR HWRLCwP5d0ZR/cO/EGDWc5ZOIRce07STuowvceFlzEAeLTEX21oGZetjojwjHHyZ r40rQ7cekRR8ea5KSBf4Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdelucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpedthfegfeejuefgieffvefhvedvfedtffdtgffh heevgfdtjedukefgkefgfeevtdenucffohhmrghinhepghhoohhglhgvrdgtohhmnecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhs rdhimhdpnhgspghrtghpthhtohepgedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtoh epjhhlthhosghlvghrsehgmhgrihhlrdgtohhmpdhrtghpthhtohepghhithhsthgvrhes phhosghogidrtghomhdprhgtphhtthhopehkrghrthhhihhkrddukeeksehgmhgrihhlrd gtohhmpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlhdrohhrgh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:23 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 35e24e36 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:21 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:13 +0200 Subject: [PATCH v2 01/16] reftable: fix formatting of the license header Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-1-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 The license headers used across the reftable library doesn't follow our typical coding style for multi-line comments. Fix it. Signed-off-by: Patrick Steinhardt --- reftable/basics.c | 12 ++++++------ reftable/basics.h | 12 ++++++------ reftable/block.c | 12 ++++++------ reftable/block.h | 12 ++++++------ reftable/blocksource.c | 12 ++++++------ reftable/blocksource.h | 12 ++++++------ reftable/constants.h | 12 ++++++------ reftable/error.c | 12 ++++++------ reftable/iter.c | 12 ++++++------ reftable/iter.h | 12 ++++++------ reftable/merged.c | 12 ++++++------ reftable/merged.h | 12 ++++++------ reftable/pq.c | 12 ++++++------ reftable/pq.h | 12 ++++++------ reftable/reader.c | 12 ++++++------ reftable/reader.h | 12 ++++++------ reftable/record.c | 12 ++++++------ reftable/record.h | 12 ++++++------ reftable/reftable-basics.h | 2 +- reftable/reftable-blocksource.h | 12 ++++++------ reftable/reftable-error.h | 12 ++++++------ reftable/reftable-iterator.h | 12 ++++++------ reftable/reftable-merged.h | 12 ++++++------ reftable/reftable-reader.h | 12 ++++++------ reftable/reftable-record.h | 12 ++++++------ reftable/reftable-stack.h | 12 ++++++------ reftable/reftable-writer.h | 12 ++++++------ reftable/stack.c | 12 ++++++------ reftable/stack.h | 12 ++++++------ reftable/system.h | 12 ++++++------ reftable/tree.c | 12 ++++++------ reftable/tree.h | 12 ++++++------ reftable/writer.c | 12 ++++++------ reftable/writer.h | 12 ++++++------ 34 files changed, 199 insertions(+), 199 deletions(-) diff --git a/reftable/basics.c b/reftable/basics.c index 8c4a4433e41..9988ebd635e 100644 --- a/reftable/basics.c +++ b/reftable/basics.c @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #define REFTABLE_ALLOW_BANNED_ALLOCATORS #include "basics.h" diff --git a/reftable/basics.h b/reftable/basics.h index fd59cbb7720..96a2f0d3829 100644 --- a/reftable/basics.h +++ b/reftable/basics.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef BASICS_H #define BASICS_H diff --git a/reftable/block.c b/reftable/block.c index 53b5e044690..a5734d44415 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #include "block.h" diff --git a/reftable/block.h b/reftable/block.h index bef2b8a4c5c..eaeffdffc90 100644 --- a/reftable/block.h +++ b/reftable/block.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef BLOCK_H #define BLOCK_H diff --git a/reftable/blocksource.c b/reftable/blocksource.c index 78c1be23373..1397cbe7800 100644 --- a/reftable/blocksource.c +++ b/reftable/blocksource.c @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #include "system.h" diff --git a/reftable/blocksource.h b/reftable/blocksource.h index a84a3ccd891..7b67898ae22 100644 --- a/reftable/blocksource.h +++ b/reftable/blocksource.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef BLOCKSOURCE_H #define BLOCKSOURCE_H diff --git a/reftable/constants.h b/reftable/constants.h index f6beb843ebf..091728cf033 100644 --- a/reftable/constants.h +++ b/reftable/constants.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef CONSTANTS_H #define CONSTANTS_H diff --git a/reftable/error.c b/reftable/error.c index 660d0296170..c7cab2dbc42 100644 --- a/reftable/error.c +++ b/reftable/error.c @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #include "system.h" #include "reftable-error.h" diff --git a/reftable/iter.c b/reftable/iter.c index f520382e700..44a155e756e 100644 --- a/reftable/iter.c +++ b/reftable/iter.c @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #include "iter.h" diff --git a/reftable/iter.h b/reftable/iter.h index 40f98893b85..bd217405f55 100644 --- a/reftable/iter.h +++ b/reftable/iter.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef ITER_H #define ITER_H diff --git a/reftable/merged.c b/reftable/merged.c index 4ff1553772a..1829a081756 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #include "merged.h" diff --git a/reftable/merged.h b/reftable/merged.h index 0b7d939e92b..ae92409d342 100644 --- a/reftable/merged.h +++ b/reftable/merged.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef MERGED_H #define MERGED_H diff --git a/reftable/pq.c b/reftable/pq.c index 82394a972db..9a79f5c5eec 100644 --- a/reftable/pq.c +++ b/reftable/pq.c @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #include "pq.h" diff --git a/reftable/pq.h b/reftable/pq.h index ff39016445b..42310670b09 100644 --- a/reftable/pq.h +++ b/reftable/pq.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef PQ_H #define PQ_H diff --git a/reftable/reader.c b/reftable/reader.c index 172aff2c10b..155a26c30ff 100644 --- a/reftable/reader.c +++ b/reftable/reader.c @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #include "reader.h" diff --git a/reftable/reader.h b/reftable/reader.h index bb72108a6f1..c9dccbd6c97 100644 --- a/reftable/reader.h +++ b/reftable/reader.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef READER_H #define READER_H diff --git a/reftable/record.c b/reftable/record.c index 142853d5070..26cd834d405 100644 --- a/reftable/record.c +++ b/reftable/record.c @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ /* record.c - methods for different types of records. */ diff --git a/reftable/record.h b/reftable/record.h index 867810a9328..7953f352a3c 100644 --- a/reftable/record.h +++ b/reftable/record.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef RECORD_H #define RECORD_H diff --git a/reftable/reftable-basics.h b/reftable/reftable-basics.h index e0397ed5836..ed7c7c9ac2c 100644 --- a/reftable/reftable-basics.h +++ b/reftable/reftable-basics.h @@ -4,7 +4,7 @@ * Use of this source code is governed by a BSD-style * license that can be found in the LICENSE file or at * https://developers.google.com/open-source/licenses/bsd -*/ + */ #ifndef REFTABLE_BASICS_H #define REFTABLE_BASICS_H diff --git a/reftable/reftable-blocksource.h b/reftable/reftable-blocksource.h index 6b326aa5ea5..8692cd017e9 100644 --- a/reftable/reftable-blocksource.h +++ b/reftable/reftable-blocksource.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef REFTABLE_BLOCKSOURCE_H #define REFTABLE_BLOCKSOURCE_H diff --git a/reftable/reftable-error.h b/reftable/reftable-error.h index a7e33d964d0..d100e0df927 100644 --- a/reftable/reftable-error.h +++ b/reftable/reftable-error.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef REFTABLE_ERROR_H #define REFTABLE_ERROR_H diff --git a/reftable/reftable-iterator.h b/reftable/reftable-iterator.h index e3bf688d53d..af582028c27 100644 --- a/reftable/reftable-iterator.h +++ b/reftable/reftable-iterator.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef REFTABLE_ITERATOR_H #define REFTABLE_ITERATOR_H diff --git a/reftable/reftable-merged.h b/reftable/reftable-merged.h index f2d01c3ef82..445756e475a 100644 --- a/reftable/reftable-merged.h +++ b/reftable/reftable-merged.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef REFTABLE_MERGED_H #define REFTABLE_MERGED_H diff --git a/reftable/reftable-reader.h b/reftable/reftable-reader.h index 0085fbb9032..9b6a8ac0716 100644 --- a/reftable/reftable-reader.h +++ b/reftable/reftable-reader.h @@ -1,10 +1,10 @@ /* - Copyright 2020 Google LLC - - Use of this source code is governed by a BSD-style - license that can be found in the LICENSE file or at - https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef REFTABLE_READER_H #define REFTABLE_READER_H diff --git a/reftable/reftable-record.h b/reftable/reftable-record.h index 931e5947441..385a74cc864 100644 --- a/reftable/reftable-record.h +++ b/reftable/reftable-record.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef REFTABLE_RECORD_H #define REFTABLE_RECORD_H diff --git a/reftable/reftable-stack.h b/reftable/reftable-stack.h index ae14270ea74..910ec6ef3a2 100644 --- a/reftable/reftable-stack.h +++ b/reftable/reftable-stack.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef REFTABLE_STACK_H #define REFTABLE_STACK_H diff --git a/reftable/reftable-writer.h b/reftable/reftable-writer.h index 1befe3b07cf..0fbeff17f46 100644 --- a/reftable/reftable-writer.h +++ b/reftable/reftable-writer.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef REFTABLE_WRITER_H #define REFTABLE_WRITER_H diff --git a/reftable/stack.c b/reftable/stack.c index 6dac015b473..bc3cfa91170 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #include "stack.h" diff --git a/reftable/stack.h b/reftable/stack.h index 5b45cff4f75..d91f5ce6072 100644 --- a/reftable/stack.h +++ b/reftable/stack.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef STACK_H #define STACK_H diff --git a/reftable/system.h b/reftable/system.h index 10055fbff2d..cf207aa0c53 100644 --- a/reftable/system.h +++ b/reftable/system.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef SYSTEM_H #define SYSTEM_H diff --git a/reftable/tree.c b/reftable/tree.c index f4dbe720901..a52f7c0c7d4 100644 --- a/reftable/tree.c +++ b/reftable/tree.c @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #include "system.h" #include "tree.h" diff --git a/reftable/tree.h b/reftable/tree.h index 9604453b6d5..2c9c4652994 100644 --- a/reftable/tree.h +++ b/reftable/tree.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef TREE_H #define TREE_H diff --git a/reftable/writer.c b/reftable/writer.c index ce55a1deb06..f0accfd0c32 100644 --- a/reftable/writer.c +++ b/reftable/writer.c @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #include "writer.h" diff --git a/reftable/writer.h b/reftable/writer.h index 1f4788a430c..9f53610b27a 100644 --- a/reftable/writer.h +++ b/reftable/writer.h @@ -1,10 +1,10 @@ /* -Copyright 2020 Google LLC - -Use of this source code is governed by a BSD-style -license that can be found in the LICENSE file or at -https://developers.google.com/open-source/licenses/bsd -*/ + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ #ifndef WRITER_H #define WRITER_H From patchwork Mon Apr 7 13:16:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040505 Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E39B253F00 for ; Mon, 7 Apr 2025 13:16:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031792; cv=none; b=YqpicbkVnJpuo2K5QqUBi9eqU1R29+al4KdcgK6epwfacKIDpfzZBPoqViboa9yld+IEt3LtUOFdiVUjVm89uuCcbORVRRSJcraszhaSYL03NPlko2Ls2+6wC91fKQBvL53ejhR8KcBG1dLuZ2kDNMyBcLJGp5sqkeidjBxq6t0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031792; c=relaxed/simple; bh=hTTQF4n1qza1rZaQ2c8ID4OZ7/hlX4LHCI6Vm6mHsRE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=o9McWrIQKi2G/p1PP1CV0RDz2TXF7N88xXQBT4fjDn2acUjv9XFq7CR3kubQSTadNnmuIvTU9/CS34S6e70h4IeoLyGXf7KVDMMWm8nhcfdNLKipJDuAYh4Wgs8jr+mZp9CW+dM2AE8n1/TvIH30xUdni+qmoulpOl8HMdkCwyI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=MF/r9A56; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=U+vpSv3P; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="MF/r9A56"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="U+vpSv3P" Received: from phl-compute-10.internal (phl-compute-10.phl.internal [10.202.2.50]) by mailfhigh.phl.internal (Postfix) with ESMTP id 5B6641140211; Mon, 7 Apr 2025 09:16:25 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-10.internal (MEProxy); Mon, 07 Apr 2025 09:16:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031785; x=1744118185; bh=CQeuuOsUfKAoEPCCsInI79uIUqYCS6zb5xweHtHSrCc=; b= MF/r9A56h3lQKmDzLRBqWkme9N0elJxp1gXoYJi0zKcitERvkM4/FJc9TsmNDnC+ R2tgyahCR/vE/8fqEgpHhK58tYPao7sU/iK9AHWxVIajk7NG0yddbRSpjsN+no/V 94nDEjeg0zK2H9mdbMq4aKPq7IUnHRbsx2ESERYXrZccJPm3gKpjxMFfik2tmuU5 p/0+/HFflq6zHuxPiP34jJOMFQvtxbmxZf3cnCwf0SGAfF22HW3NF3Q6AQxiHmXJ ob5wazv3qoMElClNmmrJc6fnrZvqRXjdNswJmHJ8LplbYw8hfYXFqkMCfYU54fmT gqPZ+VnhQMUfRlYldXEg1A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031785; x= 1744118185; bh=CQeuuOsUfKAoEPCCsInI79uIUqYCS6zb5xweHtHSrCc=; b=U +vpSv3PS1rhKsVuaY6UitA/HvwRIb6sTm1FyebM/XbvPyfpY2FJmmMRS12cbllkl WY/qKXQuu5RNGbbsiT1eGLPc0Rdrb5+C6/Dwfw828NuBojjLu/FCGGd+iw5ixBpx b5rgYH0iEhSBz+Eyl7w20LCwer6nJ3gqGKFwVmnLHQcr2lzyu77P2c8/XqIJvWZJ HhRiZjQPosbrvpOmbsuxCLjMrbK9B70tALqRsPbfZ25G99IC5h+vjXaCyOchkXs6 /7tYauDNigcGmsDqAXL1pnDM/tIzdktzcbKNpxm0eYaQiIFU9y5mN5wQyknu46h+ NFIAcFhKkqiY/C3tJyfxA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdelucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpedthfegfeejuefgieffvefhvedvfedtffdtgffh heevgfdtjedukefgkefgfeevtdenucffohhmrghinhepghhoohhglhgvrdgtohhmnecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhs rdhimhdpnhgspghrtghpthhtohepgedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtoh epghhithhsthgvrhesphhosghogidrtghomhdprhgtphhtthhopehkrghrthhhihhkrddu keeksehgmhgrihhlrdgtohhmpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlh drohhrghdprhgtphhtthhopehjlhhtohgslhgvrhesghhmrghilhdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:24 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id e77a8a5e (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:22 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:14 +0200 Subject: [PATCH v2 02/16] reftable/reader: rename data structure to "table" Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-2-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 The `struct reftable_reader` subsystem encapsulates a table that has been read from the disk. As such, the current name of that structure is somewhat hard to understand as it only talks about the fact that we read something from disk, without really giving an indicator _what_ that is. Furthermore, this naming schema doesn't really fit well into how the other structures are named: `reftable_merged_table`, `reftable_stack`, `reftable_block` and `reftable_record` are all named after what they encapsulate. Rename the subsystem to `reftable_table`, which directly gives a hint that the data structure is about handling the individual tables part of the stack. While this change results in a lot of churn, it prepares for us exposing the APIs to third-party callers now that the reftable library is a standalone library that can be linked against by other projects. Signed-off-by: Patrick Steinhardt --- Makefile | 4 +- meson.build | 2 +- reftable/iter.c | 10 +- reftable/iter.h | 4 +- reftable/merged.c | 26 +-- reftable/merged.h | 4 +- reftable/reftable-merged.h | 6 +- reftable/reftable-reader.h | 72 ------ reftable/reftable-table.h | 72 ++++++ reftable/stack.c | 168 +++++++------- reftable/stack.h | 4 +- reftable/{reader.c => table.c} | 254 ++++++++++----------- reftable/{reader.h => table.h} | 30 +-- t/helper/test-reftable.c | 14 +- t/meson.build | 2 +- t/unit-tests/t-reftable-merged.c | 74 +++--- t/unit-tests/t-reftable-readwrite.c | 96 ++++---- t/unit-tests/t-reftable-stack.c | 66 +++--- .../{t-reftable-reader.c => t-reftable-table.c} | 26 +-- 19 files changed, 467 insertions(+), 467 deletions(-) diff --git a/Makefile b/Makefile index 22d632da4ea..58bd9dc6452 100644 --- a/Makefile +++ b/Makefile @@ -1374,10 +1374,10 @@ UNIT_TEST_PROGRAMS += t-reftable-basics UNIT_TEST_PROGRAMS += t-reftable-block UNIT_TEST_PROGRAMS += t-reftable-merged UNIT_TEST_PROGRAMS += t-reftable-pq -UNIT_TEST_PROGRAMS += t-reftable-reader UNIT_TEST_PROGRAMS += t-reftable-readwrite UNIT_TEST_PROGRAMS += t-reftable-record UNIT_TEST_PROGRAMS += t-reftable-stack +UNIT_TEST_PROGRAMS += t-reftable-table UNIT_TEST_PROGRAMS += t-trailer UNIT_TEST_PROGRAMS += t-urlmatch-normalization UNIT_TEST_PROGS = $(patsubst %,$(UNIT_TEST_BIN)/%$X,$(UNIT_TEST_PROGRAMS)) @@ -2729,10 +2729,10 @@ REFTABLE_OBJS += reftable/blocksource.o REFTABLE_OBJS += reftable/iter.o REFTABLE_OBJS += reftable/merged.o REFTABLE_OBJS += reftable/pq.o -REFTABLE_OBJS += reftable/reader.o REFTABLE_OBJS += reftable/record.o REFTABLE_OBJS += reftable/stack.o REFTABLE_OBJS += reftable/system.o +REFTABLE_OBJS += reftable/table.o REFTABLE_OBJS += reftable/tree.o REFTABLE_OBJS += reftable/writer.o diff --git a/meson.build b/meson.build index efe2871c9db..61986cc48db 100644 --- a/meson.build +++ b/meson.build @@ -410,10 +410,10 @@ libgit_sources = [ 'reftable/iter.c', 'reftable/merged.c', 'reftable/pq.c', - 'reftable/reader.c', 'reftable/record.c', 'reftable/stack.c', 'reftable/system.c', + 'reftable/table.c', 'reftable/tree.c', 'reftable/writer.c', 'remote.c', diff --git a/reftable/iter.c b/reftable/iter.c index 44a155e756e..7376f263c99 100644 --- a/reftable/iter.c +++ b/reftable/iter.c @@ -12,8 +12,8 @@ #include "block.h" #include "constants.h" -#include "reader.h" #include "reftable-error.h" +#include "table.h" int iterator_seek(struct reftable_iterator *it, struct reftable_record *want) { @@ -130,8 +130,8 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it) reftable_block_done(&it->block_reader.block); off = it->offsets[it->offset_idx++]; - err = reader_init_block_reader(it->r, &it->block_reader, off, - BLOCK_TYPE_REF); + err = table_init_block_reader(it->table, &it->block_reader, off, + BLOCK_TYPE_REF); if (err < 0) { return err; } @@ -181,7 +181,7 @@ static int indexed_table_ref_iter_next(void *p, struct reftable_record *rec) } int indexed_table_ref_iter_new(struct indexed_table_ref_iter **dest, - struct reftable_reader *r, uint8_t *oid, + struct reftable_table *t, uint8_t *oid, int oid_len, uint64_t *offsets, int offset_len) { struct indexed_table_ref_iter empty = INDEXED_TABLE_REF_ITER_INIT; @@ -195,7 +195,7 @@ int indexed_table_ref_iter_new(struct indexed_table_ref_iter **dest, } *itr = empty; - itr->r = r; + itr->table = t; err = reftable_buf_add(&itr->oid, oid, oid_len); if (err < 0) diff --git a/reftable/iter.h b/reftable/iter.h index bd217405f55..add5c9472fc 100644 --- a/reftable/iter.h +++ b/reftable/iter.h @@ -59,7 +59,7 @@ void iterator_from_filtering_ref_iterator(struct reftable_iterator *, * but using the object index. */ struct indexed_table_ref_iter { - struct reftable_reader *r; + struct reftable_table *table; struct reftable_buf oid; /* mutable */ @@ -83,7 +83,7 @@ void iterator_from_indexed_table_ref_iter(struct reftable_iterator *it, /* Takes ownership of `offsets` */ int indexed_table_ref_iter_new(struct indexed_table_ref_iter **dest, - struct reftable_reader *r, uint8_t *oid, + struct reftable_table *t, uint8_t *oid, int oid_len, uint64_t *offsets, int offset_len); #endif diff --git a/reftable/merged.c b/reftable/merged.c index 1829a081756..d5b974d660e 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -11,11 +11,11 @@ #include "constants.h" #include "iter.h" #include "pq.h" -#include "reader.h" #include "record.h" #include "reftable-merged.h" #include "reftable-error.h" #include "system.h" +#include "table.h" struct merged_subiter { struct reftable_iterator iter; @@ -192,7 +192,7 @@ static void iterator_from_merged_iter(struct reftable_iterator *it, } int reftable_merged_table_new(struct reftable_merged_table **dest, - struct reftable_reader **readers, size_t n, + struct reftable_table **tables, size_t n, enum reftable_hash hash_id) { struct reftable_merged_table *m = NULL; @@ -200,10 +200,10 @@ int reftable_merged_table_new(struct reftable_merged_table **dest, uint64_t first_min = 0; for (size_t i = 0; i < n; i++) { - uint64_t min = reftable_reader_min_update_index(readers[i]); - uint64_t max = reftable_reader_max_update_index(readers[i]); + uint64_t min = reftable_table_min_update_index(tables[i]); + uint64_t max = reftable_table_max_update_index(tables[i]); - if (reftable_reader_hash_id(readers[i]) != hash_id) { + if (reftable_table_hash_id(tables[i]) != hash_id) { return REFTABLE_FORMAT_ERROR; } if (i == 0 || min < first_min) { @@ -218,8 +218,8 @@ int reftable_merged_table_new(struct reftable_merged_table **dest, if (!m) return REFTABLE_OUT_OF_MEMORY_ERROR; - m->readers = readers; - m->readers_len = n; + m->tables = tables; + m->tables_len = n; m->min = first_min; m->max = last_max; m->hash_id = hash_id; @@ -254,20 +254,20 @@ int merged_table_init_iter(struct reftable_merged_table *mt, struct merged_iter *mi = NULL; int ret; - if (mt->readers_len) { - REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len); + if (mt->tables_len) { + REFTABLE_CALLOC_ARRAY(subiters, mt->tables_len); if (!subiters) { ret = REFTABLE_OUT_OF_MEMORY_ERROR; goto out; } } - for (size_t i = 0; i < mt->readers_len; i++) { + for (size_t i = 0; i < mt->tables_len; i++) { ret = reftable_record_init(&subiters[i].rec, typ); if (ret < 0) goto out; - ret = reader_init_iter(mt->readers[i], &subiters[i].iter, typ); + ret = table_init_iter(mt->tables[i], &subiters[i].iter, typ); if (ret < 0) goto out; } @@ -280,14 +280,14 @@ int merged_table_init_iter(struct reftable_merged_table *mt, mi->advance_index = -1; mi->suppress_deletions = mt->suppress_deletions; mi->subiters = subiters; - mi->subiters_len = mt->readers_len; + mi->subiters_len = mt->tables_len; iterator_from_merged_iter(it, mi); ret = 0; out: if (ret < 0) { - for (size_t i = 0; subiters && i < mt->readers_len; i++) { + for (size_t i = 0; subiters && i < mt->tables_len; i++) { reftable_iterator_destroy(&subiters[i].iter); reftable_record_release(&subiters[i].rec); } diff --git a/reftable/merged.h b/reftable/merged.h index ae92409d342..4317e5f5f67 100644 --- a/reftable/merged.h +++ b/reftable/merged.h @@ -13,8 +13,8 @@ #include "reftable-basics.h" struct reftable_merged_table { - struct reftable_reader **readers; - size_t readers_len; + struct reftable_table **tables; + size_t tables_len; enum reftable_hash hash_id; /* If unset, produce deletions. This is useful for compaction. For the diff --git a/reftable/reftable-merged.h b/reftable/reftable-merged.h index 445756e475a..e5af846b32a 100644 --- a/reftable/reftable-merged.h +++ b/reftable/reftable-merged.h @@ -26,14 +26,14 @@ /* A merged table is implements seeking/iterating over a stack of tables. */ struct reftable_merged_table; -struct reftable_reader; +struct reftable_table; /* - * reftable_merged_table_new creates a new merged table. The readers must be + * reftable_merged_table_new creates a new merged table. The tables must be * kept alive as long as the merged table is still in use. */ int reftable_merged_table_new(struct reftable_merged_table **dest, - struct reftable_reader **readers, size_t n, + struct reftable_table **tables, size_t n, enum reftable_hash hash_id); /* Initialize a merged table iterator for reading refs. */ diff --git a/reftable/reftable-reader.h b/reftable/reftable-reader.h deleted file mode 100644 index 9b6a8ac0716..00000000000 --- a/reftable/reftable-reader.h +++ /dev/null @@ -1,72 +0,0 @@ -/* - * Copyright 2020 Google LLC - * - * Use of this source code is governed by a BSD-style - * license that can be found in the LICENSE file or at - * https://developers.google.com/open-source/licenses/bsd - */ - -#ifndef REFTABLE_READER_H -#define REFTABLE_READER_H - -#include "reftable-iterator.h" -#include "reftable-blocksource.h" - -/* - * Reading single tables - * - * The follow routines are for reading single files. For an - * application-level interface, skip ahead to struct - * reftable_merged_table and struct reftable_stack. - */ - -/* The reader struct is a handle to an open reftable file. */ -struct reftable_reader; - -/* reftable_reader_new opens a reftable for reading. If successful, - * returns 0 code and sets pp. The name is used for creating a - * stack. Typically, it is the basename of the file. The block source - * `src` is owned by the reader, and is closed on calling - * reftable_reader_destroy(). On error, the block source `src` is - * closed as well. - */ -int reftable_reader_new(struct reftable_reader **pp, - struct reftable_block_source *src, const char *name); - -/* - * Manage the reference count of the reftable reader. A newly initialized - * reader starts with a refcount of 1 and will be deleted once the refcount has - * reached 0. - * - * This is required because readers may have longer lifetimes than the stack - * they belong to. The stack may for example be reloaded while the old tables - * are still being accessed by an iterator. - */ -void reftable_reader_incref(struct reftable_reader *reader); -void reftable_reader_decref(struct reftable_reader *reader); - -/* Initialize a reftable iterator for reading refs. */ -int reftable_reader_init_ref_iterator(struct reftable_reader *r, - struct reftable_iterator *it); - -/* Initialize a reftable iterator for reading logs. */ -int reftable_reader_init_log_iterator(struct reftable_reader *r, - struct reftable_iterator *it); - -/* returns the hash ID used in this table. */ -enum reftable_hash reftable_reader_hash_id(struct reftable_reader *r); - -/* return an iterator for the refs pointing to `oid`. */ -int reftable_reader_refs_for(struct reftable_reader *r, - struct reftable_iterator *it, uint8_t *oid); - -/* return the max_update_index for a table */ -uint64_t reftable_reader_max_update_index(struct reftable_reader *r); - -/* return the min_update_index for a table */ -uint64_t reftable_reader_min_update_index(struct reftable_reader *r); - -/* print blocks onto stdout for debugging. */ -int reftable_reader_print_blocks(const char *tablename); - -#endif diff --git a/reftable/reftable-table.h b/reftable/reftable-table.h new file mode 100644 index 00000000000..94379026727 --- /dev/null +++ b/reftable/reftable-table.h @@ -0,0 +1,72 @@ +/* + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ + +#ifndef REFTABLE_TABLE_H +#define REFTABLE_TABLE_H + +#include "reftable-iterator.h" +#include "reftable-blocksource.h" + +/* + * Reading single tables + * + * The follow routines are for reading single files. For an + * application-level interface, skip ahead to struct + * reftable_merged_table and struct reftable_stack. + */ + +/* The table struct is a handle to an open reftable file. */ +struct reftable_table; + +/* reftable_table_new opens a reftable for reading. If successful, + * returns 0 code and sets pp. The name is used for creating a + * stack. Typically, it is the basename of the file. The block source + * `src` is owned by the table, and is closed on calling + * reftable_table_destroy(). On error, the block source `src` is + * closed as well. + */ +int reftable_table_new(struct reftable_table **out, + struct reftable_block_source *src, const char *name); + +/* + * Manage the reference count of the reftable table. A newly initialized + * table starts with a refcount of 1 and will be deleted once the refcount has + * reached 0. + * + * This is required because tables may have longer lifetimes than the stack + * they belong to. The stack may for example be reloaded while the old tables + * are still being accessed by an iterator. + */ +void reftable_table_incref(struct reftable_table *table); +void reftable_table_decref(struct reftable_table *table); + +/* Initialize a reftable iterator for reading refs. */ +int reftable_table_init_ref_iterator(struct reftable_table *t, + struct reftable_iterator *it); + +/* Initialize a reftable iterator for reading logs. */ +int reftable_table_init_log_iterator(struct reftable_table *t, + struct reftable_iterator *it); + +/* returns the hash ID used in this table. */ +enum reftable_hash reftable_table_hash_id(struct reftable_table *t); + +/* return an iterator for the refs pointing to `oid`. */ +int reftable_table_refs_for(struct reftable_table *t, + struct reftable_iterator *it, uint8_t *oid); + +/* return the max_update_index for a table */ +uint64_t reftable_table_max_update_index(struct reftable_table *t); + +/* return the min_update_index for a table */ +uint64_t reftable_table_min_update_index(struct reftable_table *t); + +/* print blocks onto stdout for debugging. */ +int reftable_table_print_blocks(const char *tablename); + +#endif diff --git a/reftable/stack.c b/reftable/stack.c index bc3cfa91170..cc48e725b14 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -11,10 +11,10 @@ #include "system.h" #include "constants.h" #include "merged.h" -#include "reader.h" #include "reftable-error.h" #include "reftable-record.h" #include "reftable-merged.h" +#include "table.h" #include "writer.h" static int stack_try_add(struct reftable_stack *st, @@ -248,11 +248,11 @@ void reftable_stack_destroy(struct reftable_stack *st) REFTABLE_FREE_AND_NULL(names); } - if (st->readers) { + if (st->tables) { struct reftable_buf filename = REFTABLE_BUF_INIT; - for (size_t i = 0; i < st->readers_len; i++) { - const char *name = reader_name(st->readers[i]); + for (size_t i = 0; i < st->tables_len; i++) { + const char *name = reftable_table_name(st->tables[i]); int try_unlinking = 1; reftable_buf_reset(&filename); @@ -260,7 +260,7 @@ void reftable_stack_destroy(struct reftable_stack *st) if (stack_filename(&filename, st, name) < 0) try_unlinking = 0; } - reftable_reader_decref(st->readers[i]); + reftable_table_decref(st->tables[i]); if (try_unlinking && filename.len) { /* On Windows, can only unlink after closing. */ @@ -269,8 +269,8 @@ void reftable_stack_destroy(struct reftable_stack *st) } reftable_buf_release(&filename); - st->readers_len = 0; - REFTABLE_FREE_AND_NULL(st->readers); + st->tables_len = 0; + REFTABLE_FREE_AND_NULL(st->tables); } if (st->list_fd >= 0) { @@ -284,14 +284,14 @@ void reftable_stack_destroy(struct reftable_stack *st) free_names(names); } -static struct reftable_reader **stack_copy_readers(struct reftable_stack *st, - size_t cur_len) +static struct reftable_table **stack_copy_tables(struct reftable_stack *st, + size_t cur_len) { - struct reftable_reader **cur = reftable_calloc(cur_len, sizeof(*cur)); + struct reftable_table **cur = reftable_calloc(cur_len, sizeof(*cur)); if (!cur) return NULL; for (size_t i = 0; i < cur_len; i++) - cur[i] = st->readers[i]; + cur[i] = st->tables[i]; return cur; } @@ -299,19 +299,19 @@ static int reftable_stack_reload_once(struct reftable_stack *st, const char **names, int reuse_open) { - size_t cur_len = !st->merged ? 0 : st->merged->readers_len; - struct reftable_reader **cur = NULL; - struct reftable_reader **reused = NULL; - struct reftable_reader **new_readers = NULL; + size_t cur_len = !st->merged ? 0 : st->merged->tables_len; + struct reftable_table **cur = NULL; + struct reftable_table **reused = NULL; + struct reftable_table **new_tables = NULL; size_t reused_len = 0, reused_alloc = 0, names_len; - size_t new_readers_len = 0; + size_t new_tables_len = 0; struct reftable_merged_table *new_merged = NULL; struct reftable_buf table_path = REFTABLE_BUF_INIT; int err = 0; size_t i; if (cur_len) { - cur = stack_copy_readers(st, cur_len); + cur = stack_copy_tables(st, cur_len); if (!cur) { err = REFTABLE_OUT_OF_MEMORY_ERROR; goto done; @@ -321,28 +321,28 @@ static int reftable_stack_reload_once(struct reftable_stack *st, names_len = names_length(names); if (names_len) { - new_readers = reftable_calloc(names_len, sizeof(*new_readers)); - if (!new_readers) { + new_tables = reftable_calloc(names_len, sizeof(*new_tables)); + if (!new_tables) { err = REFTABLE_OUT_OF_MEMORY_ERROR; goto done; } } while (*names) { - struct reftable_reader *rd = NULL; + struct reftable_table *table = NULL; const char *name = *names++; /* this is linear; we assume compaction keeps the number of tables under control so this is not quadratic. */ for (i = 0; reuse_open && i < cur_len; i++) { if (cur[i] && 0 == strcmp(cur[i]->name, name)) { - rd = cur[i]; + table = cur[i]; cur[i] = NULL; /* * When reloading the stack fails, we end up - * releasing all new readers. This also - * includes the reused readers, even though + * releasing all new tables. This also + * includes the reused tables, even though * they are still in used by the old stack. We * thus need to keep them alive here, which we * do by bumping their refcount. @@ -354,13 +354,13 @@ static int reftable_stack_reload_once(struct reftable_stack *st, err = REFTABLE_OUT_OF_MEMORY_ERROR; goto done; } - reused[reused_len++] = rd; - reftable_reader_incref(rd); + reused[reused_len++] = table; + reftable_table_incref(table); break; } } - if (!rd) { + if (!table) { struct reftable_block_source src = { NULL }; err = stack_filename(&table_path, st, name); @@ -372,36 +372,36 @@ static int reftable_stack_reload_once(struct reftable_stack *st, if (err < 0) goto done; - err = reftable_reader_new(&rd, &src, name); + err = reftable_table_new(&table, &src, name); if (err < 0) goto done; } - new_readers[new_readers_len] = rd; - new_readers_len++; + new_tables[new_tables_len] = table; + new_tables_len++; } /* success! */ - err = reftable_merged_table_new(&new_merged, new_readers, - new_readers_len, st->opts.hash_id); + err = reftable_merged_table_new(&new_merged, new_tables, + new_tables_len, st->opts.hash_id); if (err < 0) goto done; /* - * Close the old, non-reused readers and proactively try to unlink + * Close the old, non-reused tables and proactively try to unlink * them. This is done for systems like Windows, where the underlying - * file of such an open reader wouldn't have been possible to be + * file of such an open table wouldn't have been possible to be * unlinked by the compacting process. */ for (i = 0; i < cur_len; i++) { if (cur[i]) { - const char *name = reader_name(cur[i]); + const char *name = reftable_table_name(cur[i]); err = stack_filename(&table_path, st, name); if (err < 0) goto done; - reftable_reader_decref(cur[i]); + reftable_table_decref(cur[i]); unlink(table_path.buf); } } @@ -412,25 +412,25 @@ static int reftable_stack_reload_once(struct reftable_stack *st, new_merged->suppress_deletions = 1; st->merged = new_merged; - if (st->readers) - reftable_free(st->readers); - st->readers = new_readers; - st->readers_len = new_readers_len; - new_readers = NULL; - new_readers_len = 0; + if (st->tables) + reftable_free(st->tables); + st->tables = new_tables; + st->tables_len = new_tables_len; + new_tables = NULL; + new_tables_len = 0; /* - * Decrement the refcount of reused readers again. This only needs to + * Decrement the refcount of reused tables again. This only needs to * happen on the successful case, because on the unsuccessful one we - * decrement their refcount via `new_readers`. + * decrement their refcount via `new_tables`. */ for (i = 0; i < reused_len; i++) - reftable_reader_decref(reused[i]); + reftable_table_decref(reused[i]); done: - for (i = 0; i < new_readers_len; i++) - reftable_reader_decref(new_readers[i]); - reftable_free(new_readers); + for (i = 0; i < new_tables_len; i++) + reftable_table_decref(new_tables[i]); + reftable_free(new_tables); reftable_free(reused); reftable_free(cur); reftable_buf_release(&table_path); @@ -615,10 +615,10 @@ static int stack_uptodate(struct reftable_stack *st) /* * It's fine for "tables.list" to not exist. In that * case, we have to refresh when the loaded stack has - * any readers. + * any tables. */ if (errno == ENOENT) - return !!st->readers_len; + return !!st->tables_len; return REFTABLE_IO_ERROR; } @@ -637,19 +637,19 @@ static int stack_uptodate(struct reftable_stack *st) if (err < 0) return err; - for (size_t i = 0; i < st->readers_len; i++) { + for (size_t i = 0; i < st->tables_len; i++) { if (!names[i]) { err = 1; goto done; } - if (strcmp(st->readers[i]->name, names[i])) { + if (strcmp(st->tables[i]->name, names[i])) { err = 1; goto done; } } - if (names[st->merged->readers_len]) { + if (names[st->merged->tables_len]) { err = 1; goto done; } @@ -792,8 +792,8 @@ int reftable_addition_commit(struct reftable_addition *add) if (add->new_tables_len == 0) goto done; - for (i = 0; i < add->stack->merged->readers_len; i++) { - if ((err = reftable_buf_addstr(&table_list, add->stack->readers[i]->name)) < 0 || + for (i = 0; i < add->stack->merged->tables_len; i++) { + if ((err = reftable_buf_addstr(&table_list, add->stack->tables[i]->name)) < 0 || (err = reftable_buf_addstr(&table_list, "\n")) < 0) goto done; } @@ -1000,9 +1000,9 @@ int reftable_addition_add(struct reftable_addition *add, uint64_t reftable_stack_next_update_index(struct reftable_stack *st) { - int sz = st->merged->readers_len; + int sz = st->merged->tables_len; if (sz > 0) - return reftable_reader_max_update_index(st->readers[sz - 1]) + + return reftable_table_max_update_index(st->tables[sz - 1]) + 1; return 1; } @@ -1021,8 +1021,8 @@ static int stack_compact_locked(struct reftable_stack *st, struct reftable_tmpfile tab_file = REFTABLE_TMPFILE_INIT; int err = 0; - err = format_name(&next_name, reftable_reader_min_update_index(st->readers[first]), - reftable_reader_max_update_index(st->readers[last])); + err = format_name(&next_name, reftable_table_min_update_index(st->tables[first]), + reftable_table_max_update_index(st->tables[last])); if (err < 0) goto done; @@ -1087,13 +1087,13 @@ static int stack_write_compact(struct reftable_stack *st, int err = 0; for (size_t i = first; i <= last; i++) - st->stats.bytes += st->readers[i]->size; - err = reftable_writer_set_limits(wr, st->readers[first]->min_update_index, - st->readers[last]->max_update_index); + st->stats.bytes += st->tables[i]->size; + err = reftable_writer_set_limits(wr, st->tables[first]->min_update_index, + st->tables[last]->max_update_index); if (err < 0) goto done; - err = reftable_merged_table_new(&mt, st->readers + first, subtabs_len, + err = reftable_merged_table_new(&mt, st->tables + first, subtabs_len, st->opts.hash_id); if (err < 0) goto done; @@ -1250,7 +1250,7 @@ static int stack_compact_range(struct reftable_stack *st, table_locks[i] = REFTABLE_FLOCK_INIT; for (i = last + 1; i > first; i--) { - err = stack_filename(&table_name, st, reader_name(st->readers[i - 1])); + err = stack_filename(&table_name, st, reftable_table_name(st->tables[i - 1])); if (err < 0) goto done; @@ -1376,7 +1376,7 @@ static int stack_compact_range(struct reftable_stack *st, * compacted in the updated "tables.list" file. */ for (size_t i = 0; names[i]; i++) { - if (strcmp(names[i], st->readers[first]->name)) + if (strcmp(names[i], st->tables[first]->name)) continue; /* @@ -1386,8 +1386,8 @@ static int stack_compact_range(struct reftable_stack *st, * have compacted them. */ for (size_t j = 1; j < last - first + 1; j++) { - const char *old = first + j < st->merged->readers_len ? - st->readers[first + j]->name : NULL; + const char *old = first + j < st->merged->tables_len ? + st->tables[first + j]->name : NULL; const char *new = names[i + j]; /* @@ -1427,16 +1427,16 @@ static int stack_compact_range(struct reftable_stack *st, * `fd_read_lines()` uses a `NULL` sentinel to indicate that * the array is at its end. As we use `free_names()` to free * the array, we need to include this sentinel value here and - * thus have to allocate `readers_len + 1` many entries. + * thus have to allocate `tables_len + 1` many entries. */ - REFTABLE_CALLOC_ARRAY(names, st->merged->readers_len + 1); + REFTABLE_CALLOC_ARRAY(names, st->merged->tables_len + 1); if (!names) { err = REFTABLE_OUT_OF_MEMORY_ERROR; goto done; } - for (size_t i = 0; i < st->merged->readers_len; i++) { - names[i] = reftable_strdup(st->readers[i]->name); + for (size_t i = 0; i < st->merged->tables_len; i++) { + names[i] = reftable_strdup(st->tables[i]->name); if (!names[i]) { err = REFTABLE_OUT_OF_MEMORY_ERROR; goto done; @@ -1451,8 +1451,8 @@ static int stack_compact_range(struct reftable_stack *st, * it into place now. */ if (!is_empty_table) { - err = format_name(&new_table_name, st->readers[first]->min_update_index, - st->readers[last]->max_update_index); + err = format_name(&new_table_name, st->tables[first]->min_update_index, + st->tables[last]->max_update_index); if (err < 0) goto done; @@ -1559,7 +1559,7 @@ static int stack_compact_range(struct reftable_stack *st, int reftable_stack_compact_all(struct reftable_stack *st, struct reftable_log_expiry_config *config) { - size_t last = st->merged->readers_len ? st->merged->readers_len - 1 : 0; + size_t last = st->merged->tables_len ? st->merged->tables_len - 1 : 0; return stack_compact_range(st, 0, last, config, 0); } @@ -1650,12 +1650,12 @@ static uint64_t *stack_table_sizes_for_compaction(struct reftable_stack *st) int overhead = header_size(version) - 1; uint64_t *sizes; - REFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len); + REFTABLE_CALLOC_ARRAY(sizes, st->merged->tables_len); if (!sizes) return NULL; - for (size_t i = 0; i < st->merged->readers_len; i++) - sizes[i] = st->readers[i]->size - overhead; + for (size_t i = 0; i < st->merged->tables_len; i++) + sizes[i] = st->tables[i]->size - overhead; return sizes; } @@ -1665,14 +1665,14 @@ int reftable_stack_auto_compact(struct reftable_stack *st) struct segment seg; uint64_t *sizes; - if (st->merged->readers_len < 2) + if (st->merged->tables_len < 2) return 0; sizes = stack_table_sizes_for_compaction(st); if (!sizes) return REFTABLE_OUT_OF_MEMORY_ERROR; - seg = suggest_compaction_segment(sizes, st->merged->readers_len, + seg = suggest_compaction_segment(sizes, st->merged->tables_len, st->opts.auto_compaction_factor); reftable_free(sizes); @@ -1763,7 +1763,7 @@ static void remove_maybe_stale_table(struct reftable_stack *st, uint64_t max, int err = 0; uint64_t update_idx = 0; struct reftable_block_source src = { NULL }; - struct reftable_reader *rd = NULL; + struct reftable_table *table = NULL; struct reftable_buf table_path = REFTABLE_BUF_INIT; err = stack_filename(&table_path, st, name); @@ -1774,12 +1774,12 @@ static void remove_maybe_stale_table(struct reftable_stack *st, uint64_t max, if (err < 0) goto done; - err = reftable_reader_new(&rd, &src, name); + err = reftable_table_new(&table, &src, name); if (err < 0) goto done; - update_idx = reftable_reader_max_update_index(rd); - reftable_reader_decref(rd); + update_idx = reftable_table_max_update_index(table); + reftable_table_decref(table); if (update_idx <= max) { unlink(table_path.buf); @@ -1803,8 +1803,8 @@ static int reftable_stack_clean_locked(struct reftable_stack *st) if (!is_table_name(d->d_name)) continue; - for (size_t i = 0; !found && i < st->readers_len; i++) - found = !strcmp(reader_name(st->readers[i]), d->d_name); + for (size_t i = 0; !found && i < st->tables_len; i++) + found = !strcmp(reftable_table_name(st->tables[i]), d->d_name); if (found) continue; diff --git a/reftable/stack.h b/reftable/stack.h index d91f5ce6072..bc28f2998ac 100644 --- a/reftable/stack.h +++ b/reftable/stack.h @@ -22,8 +22,8 @@ struct reftable_stack { struct reftable_write_options opts; - struct reftable_reader **readers; - size_t readers_len; + struct reftable_table **tables; + size_t tables_len; struct reftable_merged_table *merged; struct reftable_compaction_stats stats; }; diff --git a/reftable/reader.c b/reftable/table.c similarity index 73% rename from reftable/reader.c rename to reftable/table.c index 155a26c30ff..440fb559ad1 100644 --- a/reftable/reader.c +++ b/reftable/table.c @@ -6,7 +6,7 @@ * https://developers.google.com/open-source/licenses/bsd */ -#include "reader.h" +#include "table.h" #include "system.h" #include "block.h" @@ -39,48 +39,48 @@ void block_source_close(struct reftable_block_source *source) source->ops = NULL; } -static struct reftable_reader_offsets * -reader_offsets_for(struct reftable_reader *r, uint8_t typ) +static struct reftable_table_offsets * +table_offsets_for(struct reftable_table *t, uint8_t typ) { switch (typ) { case BLOCK_TYPE_REF: - return &r->ref_offsets; + return &t->ref_offsets; case BLOCK_TYPE_LOG: - return &r->log_offsets; + return &t->log_offsets; case BLOCK_TYPE_OBJ: - return &r->obj_offsets; + return &t->obj_offsets; } abort(); } -static int reader_get_block(struct reftable_reader *r, - struct reftable_block *dest, uint64_t off, - uint32_t sz) +static int table_get_block(struct reftable_table *t, + struct reftable_block *dest, uint64_t off, + uint32_t sz) { ssize_t bytes_read; - if (off >= r->size) + if (off >= t->size) return 0; - if (off + sz > r->size) - sz = r->size - off; + if (off + sz > t->size) + sz = t->size - off; - bytes_read = block_source_read_block(&r->source, dest, off, sz); + bytes_read = block_source_read_block(&t->source, dest, off, sz); if (bytes_read < 0) return (int)bytes_read; return 0; } -enum reftable_hash reftable_reader_hash_id(struct reftable_reader *r) +enum reftable_hash reftable_table_hash_id(struct reftable_table *t) { - return r->hash_id; + return t->hash_id; } -const char *reader_name(struct reftable_reader *r) +const char *reftable_table_name(struct reftable_table *t) { - return r->name; + return t->name; } -static int parse_footer(struct reftable_reader *r, uint8_t *footer, +static int parse_footer(struct reftable_table *t, uint8_t *footer, uint8_t *header) { uint8_t *f = footer; @@ -95,29 +95,29 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer, } f += 4; - if (memcmp(footer, header, header_size(r->version))) { + if (memcmp(footer, header, header_size(t->version))) { err = REFTABLE_FORMAT_ERROR; goto done; } f++; - r->block_size = reftable_get_be24(f); + t->block_size = reftable_get_be24(f); f += 3; - r->min_update_index = reftable_get_be64(f); + t->min_update_index = reftable_get_be64(f); f += 8; - r->max_update_index = reftable_get_be64(f); + t->max_update_index = reftable_get_be64(f); f += 8; - if (r->version == 1) { - r->hash_id = REFTABLE_HASH_SHA1; + if (t->version == 1) { + t->hash_id = REFTABLE_HASH_SHA1; } else { switch (reftable_get_be32(f)) { case REFTABLE_FORMAT_ID_SHA1: - r->hash_id = REFTABLE_HASH_SHA1; + t->hash_id = REFTABLE_HASH_SHA1; break; case REFTABLE_FORMAT_ID_SHA256: - r->hash_id = REFTABLE_HASH_SHA256; + t->hash_id = REFTABLE_HASH_SHA256; break; default: err = REFTABLE_FORMAT_ERROR; @@ -127,20 +127,20 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer, f += 4; } - r->ref_offsets.index_offset = reftable_get_be64(f); + t->ref_offsets.index_offset = reftable_get_be64(f); f += 8; - r->obj_offsets.offset = reftable_get_be64(f); + t->obj_offsets.offset = reftable_get_be64(f); f += 8; - r->object_id_len = r->obj_offsets.offset & ((1 << 5) - 1); - r->obj_offsets.offset >>= 5; + t->object_id_len = t->obj_offsets.offset & ((1 << 5) - 1); + t->obj_offsets.offset >>= 5; - r->obj_offsets.index_offset = reftable_get_be64(f); + t->obj_offsets.index_offset = reftable_get_be64(f); f += 8; - r->log_offsets.offset = reftable_get_be64(f); + t->log_offsets.offset = reftable_get_be64(f); f += 8; - r->log_offsets.index_offset = reftable_get_be64(f); + t->log_offsets.index_offset = reftable_get_be64(f); f += 8; computed_crc = crc32(0, footer, f - footer); @@ -151,13 +151,13 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer, goto done; } - first_block_typ = header[header_size(r->version)]; - r->ref_offsets.is_present = (first_block_typ == BLOCK_TYPE_REF); - r->ref_offsets.offset = 0; - r->log_offsets.is_present = (first_block_typ == BLOCK_TYPE_LOG || - r->log_offsets.offset > 0); - r->obj_offsets.is_present = r->obj_offsets.offset > 0; - if (r->obj_offsets.is_present && !r->object_id_len) { + first_block_typ = header[header_size(t->version)]; + t->ref_offsets.is_present = (first_block_typ == BLOCK_TYPE_REF); + t->ref_offsets.offset = 0; + t->log_offsets.is_present = (first_block_typ == BLOCK_TYPE_LOG || + t->log_offsets.offset > 0); + t->obj_offsets.is_present = t->obj_offsets.offset > 0; + if (t->obj_offsets.is_present && !t->object_id_len) { err = REFTABLE_FORMAT_ERROR; goto done; } @@ -168,7 +168,7 @@ static int parse_footer(struct reftable_reader *r, uint8_t *footer, } struct table_iter { - struct reftable_reader *r; + struct reftable_table *table; uint8_t typ; uint64_t block_off; struct block_reader br; @@ -176,12 +176,12 @@ struct table_iter { int is_finished; }; -static int table_iter_init(struct table_iter *ti, struct reftable_reader *r) +static int table_iter_init(struct table_iter *ti, struct reftable_table *t) { struct block_iter bi = BLOCK_ITER_INIT; memset(ti, 0, sizeof(*ti)); - reftable_reader_incref(r); - ti->r = r; + reftable_table_incref(t); + ti->table = t; ti->bi = bi; return 0; } @@ -191,7 +191,7 @@ static int table_iter_next_in_block(struct table_iter *ti, { int res = block_iter_next(&ti->bi, rec); if (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) { - rec->u.ref.update_index += ti->r->min_update_index; + rec->u.ref.update_index += ti->table->min_update_index; } return res; @@ -219,26 +219,26 @@ static int32_t extract_block_size(uint8_t *data, uint8_t *typ, uint64_t off, return result; } -int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br, - uint64_t next_off, uint8_t want_typ) +int table_init_block_reader(struct reftable_table *t, struct block_reader *br, + uint64_t next_off, uint8_t want_typ) { - int32_t guess_block_size = r->block_size ? r->block_size : + int32_t guess_block_size = t->block_size ? t->block_size : DEFAULT_BLOCK_SIZE; struct reftable_block block = { NULL }; uint8_t block_typ = 0; int err = 0; - uint32_t header_off = next_off ? 0 : header_size(r->version); + uint32_t header_off = next_off ? 0 : header_size(t->version); int32_t block_size = 0; - if (next_off >= r->size) + if (next_off >= t->size) return 1; - err = reader_get_block(r, &block, next_off, guess_block_size); + err = table_get_block(t, &block, next_off, guess_block_size); if (err < 0) goto done; block_size = extract_block_size(block.data, &block_typ, next_off, - r->version); + t->version); if (block_size < 0) { err = block_size; goto done; @@ -250,14 +250,14 @@ int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br, if (block_size > guess_block_size) { reftable_block_done(&block); - err = reader_get_block(r, &block, next_off, block_size); + err = table_get_block(t, &block, next_off, block_size); if (err < 0) { goto done; } } - err = block_reader_init(br, &block, header_off, r->block_size, - hash_size(r->hash_id)); + err = block_reader_init(br, &block, header_off, t->block_size, + hash_size(t->hash_id)); done: reftable_block_done(&block); @@ -268,7 +268,7 @@ static void table_iter_close(struct table_iter *ti) { table_iter_block_done(ti); block_iter_close(&ti->bi); - reftable_reader_decref(ti->r); + reftable_table_decref(ti->table); } static int table_iter_next_block(struct table_iter *ti) @@ -276,7 +276,7 @@ static int table_iter_next_block(struct table_iter *ti) uint64_t next_block_off = ti->block_off + ti->br.full_block_size; int err; - err = reader_init_block_reader(ti->r, &ti->br, next_block_off, ti->typ); + err = table_init_block_reader(ti->table, &ti->br, next_block_off, ti->typ); if (err > 0) ti->is_finished = 1; if (err) @@ -326,7 +326,7 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ) { int err; - err = reader_init_block_reader(ti->r, &ti->br, off, typ); + err = table_init_block_reader(ti->table, &ti->br, off, typ); if (err != 0) return err; @@ -339,7 +339,7 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ) static int table_iter_seek_start(struct table_iter *ti, uint8_t typ, int index) { - struct reftable_reader_offsets *offs = reader_offsets_for(ti->r, typ); + struct reftable_table_offsets *offs = table_offsets_for(ti->table, typ); uint64_t off = offs->offset; if (index) { off = offs->index_offset; @@ -518,7 +518,7 @@ static int table_iter_seek(struct table_iter *ti, struct reftable_record *want) { uint8_t typ = reftable_record_type(want); - struct reftable_reader_offsets *offs = reader_offsets_for(ti->r, typ); + struct reftable_table_offsets *offs = table_offsets_for(ti->table, typ); int err; err = table_iter_seek_start(ti, reftable_record_type(want), @@ -566,11 +566,11 @@ static void iterator_from_table_iter(struct reftable_iterator *it, it->ops = &table_iter_vtable; } -int reader_init_iter(struct reftable_reader *r, - struct reftable_iterator *it, - uint8_t typ) +int table_init_iter(struct reftable_table *t, + struct reftable_iterator *it, + uint8_t typ) { - struct reftable_reader_offsets *offs = reader_offsets_for(r, typ); + struct reftable_table_offsets *offs = table_offsets_for(t, typ); if (offs->is_present) { struct table_iter *ti; @@ -578,7 +578,7 @@ int reader_init_iter(struct reftable_reader *r, if (!ti) return REFTABLE_OUT_OF_MEMORY_ERROR; - table_iter_init(ti, r); + table_iter_init(ti, t); iterator_from_table_iter(it, ti); } else { iterator_set_empty(it); @@ -587,31 +587,31 @@ int reader_init_iter(struct reftable_reader *r, return 0; } -int reftable_reader_init_ref_iterator(struct reftable_reader *r, - struct reftable_iterator *it) +int reftable_table_init_ref_iterator(struct reftable_table *t, + struct reftable_iterator *it) { - return reader_init_iter(r, it, BLOCK_TYPE_REF); + return table_init_iter(t, it, BLOCK_TYPE_REF); } -int reftable_reader_init_log_iterator(struct reftable_reader *r, - struct reftable_iterator *it) +int reftable_table_init_log_iterator(struct reftable_table *t, + struct reftable_iterator *it) { - return reader_init_iter(r, it, BLOCK_TYPE_LOG); + return table_init_iter(t, it, BLOCK_TYPE_LOG); } -int reftable_reader_new(struct reftable_reader **out, - struct reftable_block_source *source, char const *name) +int reftable_table_new(struct reftable_table **out, + struct reftable_block_source *source, char const *name) { struct reftable_block footer = { 0 }; struct reftable_block header = { 0 }; - struct reftable_reader *r; + struct reftable_table *t; uint64_t file_size = block_source_size(source); uint32_t read_size; ssize_t bytes_read; int err; - REFTABLE_CALLOC_ARRAY(r, 1); - if (!r) { + REFTABLE_CALLOC_ARRAY(t, 1); + if (!t) { err = REFTABLE_OUT_OF_MEMORY_ERROR; goto done; } @@ -636,72 +636,72 @@ int reftable_reader_new(struct reftable_reader **out, err = REFTABLE_FORMAT_ERROR; goto done; } - r->version = header.data[4]; - if (r->version != 1 && r->version != 2) { + t->version = header.data[4]; + if (t->version != 1 && t->version != 2) { err = REFTABLE_FORMAT_ERROR; goto done; } - r->size = file_size - footer_size(r->version); - r->source = *source; - r->name = reftable_strdup(name); - if (!r->name) { + t->size = file_size - footer_size(t->version); + t->source = *source; + t->name = reftable_strdup(name); + if (!t->name) { err = REFTABLE_OUT_OF_MEMORY_ERROR; goto done; } - r->hash_id = 0; - r->refcount = 1; + t->hash_id = 0; + t->refcount = 1; - bytes_read = block_source_read_block(source, &footer, r->size, - footer_size(r->version)); - if (bytes_read < 0 || (size_t)bytes_read != footer_size(r->version)) { + bytes_read = block_source_read_block(source, &footer, t->size, + footer_size(t->version)); + if (bytes_read < 0 || (size_t)bytes_read != footer_size(t->version)) { err = REFTABLE_IO_ERROR; goto done; } - err = parse_footer(r, footer.data, header.data); + err = parse_footer(t, footer.data, header.data); if (err) goto done; - *out = r; + *out = t; done: reftable_block_done(&footer); reftable_block_done(&header); if (err) { - if (r) - reftable_free(r->name); - reftable_free(r); + if (t) + reftable_free(t->name); + reftable_free(t); block_source_close(source); } return err; } -void reftable_reader_incref(struct reftable_reader *r) +void reftable_table_incref(struct reftable_table *t) { - r->refcount++; + t->refcount++; } -void reftable_reader_decref(struct reftable_reader *r) +void reftable_table_decref(struct reftable_table *t) { - if (!r) + if (!t) return; - if (--r->refcount) + if (--t->refcount) return; - block_source_close(&r->source); - REFTABLE_FREE_AND_NULL(r->name); - reftable_free(r); + block_source_close(&t->source); + REFTABLE_FREE_AND_NULL(t->name); + reftable_free(t); } -static int reftable_reader_refs_for_indexed(struct reftable_reader *r, - struct reftable_iterator *it, - uint8_t *oid) +static int reftable_table_refs_for_indexed(struct reftable_table *t, + struct reftable_iterator *it, + uint8_t *oid) { struct reftable_record want = { .type = BLOCK_TYPE_OBJ, .u.obj = { .hash_prefix = oid, - .hash_prefix_len = r->object_id_len, + .hash_prefix_len = t->object_id_len, }, }; struct reftable_iterator oit = { NULL }; @@ -713,7 +713,7 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r, struct indexed_table_ref_iter *itr = NULL; /* Look through the reverse index. */ - err = reader_init_iter(r, &oit, BLOCK_TYPE_OBJ); + err = table_init_iter(t, &oit, BLOCK_TYPE_OBJ); if (err < 0) goto done; @@ -727,14 +727,14 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r, goto done; if (err > 0 || memcmp(want.u.obj.hash_prefix, got.u.obj.hash_prefix, - r->object_id_len)) { + t->object_id_len)) { /* didn't find it; return empty iterator */ iterator_set_empty(it); err = 0; goto done; } - err = indexed_table_ref_iter_new(&itr, r, oid, hash_size(r->hash_id), + err = indexed_table_ref_iter_new(&itr, t, oid, hash_size(t->hash_id), got.u.obj.offsets, got.u.obj.offset_len); if (err < 0) @@ -748,14 +748,14 @@ static int reftable_reader_refs_for_indexed(struct reftable_reader *r, return err; } -static int reftable_reader_refs_for_unindexed(struct reftable_reader *r, - struct reftable_iterator *it, - uint8_t *oid) +static int reftable_table_refs_for_unindexed(struct reftable_table *t, + struct reftable_iterator *it, + uint8_t *oid) { struct table_iter *ti; struct filtering_ref_iterator *filter = NULL; struct filtering_ref_iterator empty = FILTERING_REF_ITERATOR_INIT; - uint32_t oid_len = hash_size(r->hash_id); + uint32_t oid_len = hash_size(t->hash_id); int err; REFTABLE_ALLOC_ARRAY(ti, 1); @@ -764,7 +764,7 @@ static int reftable_reader_refs_for_unindexed(struct reftable_reader *r, goto out; } - table_iter_init(ti, r); + table_iter_init(ti, t); err = table_iter_seek_start(ti, BLOCK_TYPE_REF, 0); if (err < 0) goto out; @@ -795,25 +795,25 @@ static int reftable_reader_refs_for_unindexed(struct reftable_reader *r, return err; } -int reftable_reader_refs_for(struct reftable_reader *r, - struct reftable_iterator *it, uint8_t *oid) +int reftable_table_refs_for(struct reftable_table *t, + struct reftable_iterator *it, uint8_t *oid) { - if (r->obj_offsets.is_present) - return reftable_reader_refs_for_indexed(r, it, oid); - return reftable_reader_refs_for_unindexed(r, it, oid); + if (t->obj_offsets.is_present) + return reftable_table_refs_for_indexed(t, it, oid); + return reftable_table_refs_for_unindexed(t, it, oid); } -uint64_t reftable_reader_max_update_index(struct reftable_reader *r) +uint64_t reftable_table_max_update_index(struct reftable_table *t) { - return r->max_update_index; + return t->max_update_index; } -uint64_t reftable_reader_min_update_index(struct reftable_reader *r) +uint64_t reftable_table_min_update_index(struct reftable_table *t) { - return r->min_update_index; + return t->min_update_index; } -int reftable_reader_print_blocks(const char *tablename) +int reftable_table_print_blocks(const char *tablename) { struct { const char *name; @@ -833,7 +833,7 @@ int reftable_reader_print_blocks(const char *tablename) }, }; struct reftable_block_source src = { 0 }; - struct reftable_reader *r = NULL; + struct reftable_table *table = NULL; struct table_iter ti = { 0 }; size_t i; int err; @@ -842,14 +842,14 @@ int reftable_reader_print_blocks(const char *tablename) if (err < 0) goto done; - err = reftable_reader_new(&r, &src, tablename); + err = reftable_table_new(&table, &src, tablename); if (err < 0) goto done; - table_iter_init(&ti, r); + table_iter_init(&ti, table); printf("header:\n"); - printf(" block_size: %d\n", r->block_size); + printf(" block_size: %d\n", table->block_size); for (i = 0; i < sizeof(sections) / sizeof(*sections); i++) { err = table_iter_seek_start(&ti, sections[i].type, 0); @@ -873,7 +873,7 @@ int reftable_reader_print_blocks(const char *tablename) } done: - reftable_reader_decref(r); + reftable_table_decref(table); table_iter_close(&ti); return err; } diff --git a/reftable/reader.h b/reftable/table.h similarity index 65% rename from reftable/reader.h rename to reftable/table.h index c9dccbd6c97..9cd8f80a207 100644 --- a/reftable/reader.h +++ b/reftable/table.h @@ -6,13 +6,13 @@ * https://developers.google.com/open-source/licenses/bsd */ -#ifndef READER_H -#define READER_H +#ifndef TABLE_H +#define TABLE_H #include "block.h" #include "record.h" #include "reftable-iterator.h" -#include "reftable-reader.h" +#include "reftable-table.h" uint64_t block_source_size(struct reftable_block_source *source); @@ -22,14 +22,14 @@ ssize_t block_source_read_block(struct reftable_block_source *source, void block_source_close(struct reftable_block_source *source); /* metadata for a block type */ -struct reftable_reader_offsets { +struct reftable_table_offsets { int is_present; uint64_t offset; uint64_t index_offset; }; /* The state for reading a reftable file. */ -struct reftable_reader { +struct reftable_table { /* for convenience, associate a name with the instance. */ char *name; struct reftable_block_source source; @@ -47,21 +47,21 @@ struct reftable_reader { int object_id_len; int version; - struct reftable_reader_offsets ref_offsets; - struct reftable_reader_offsets obj_offsets; - struct reftable_reader_offsets log_offsets; + struct reftable_table_offsets ref_offsets; + struct reftable_table_offsets obj_offsets; + struct reftable_table_offsets log_offsets; uint64_t refcount; }; -const char *reader_name(struct reftable_reader *r); +const char *reftable_table_name(struct reftable_table *t); -int reader_init_iter(struct reftable_reader *r, - struct reftable_iterator *it, - uint8_t typ); +int table_init_iter(struct reftable_table *t, + struct reftable_iterator *it, + uint8_t typ); -/* initialize a block reader to read from `r` */ -int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br, - uint64_t next_off, uint8_t want_typ); +/* initialize a block reader to read from `t` */ +int table_init_block_reader(struct reftable_table *t, struct block_reader *br, + uint64_t next_off, uint8_t want_typ); #endif diff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c index 3c72ed985b3..f8f1956f4f3 100644 --- a/t/helper/test-reftable.c +++ b/t/helper/test-reftable.c @@ -4,8 +4,8 @@ #include "reftable/system.h" #include "reftable/reftable-error.h" #include "reftable/reftable-merged.h" -#include "reftable/reftable-reader.h" #include "reftable/reftable-stack.h" +#include "reftable/reftable-table.h" #include "test-tool.h" static void print_help(void) @@ -126,19 +126,19 @@ static int dump_reftable(const char *tablename) { struct reftable_block_source src = { 0 }; struct reftable_merged_table *mt = NULL; - struct reftable_reader *r = NULL; + struct reftable_table *table = NULL; int err; err = reftable_block_source_from_file(&src, tablename); if (err < 0) goto done; - err = reftable_reader_new(&r, &src, tablename); + err = reftable_table_new(&table, &src, tablename); if (err < 0) goto done; - err = reftable_merged_table_new(&mt, &r, 1, - reftable_reader_hash_id(r)); + err = reftable_merged_table_new(&mt, &table, 1, + reftable_table_hash_id(table)); if (err < 0) goto done; @@ -146,7 +146,7 @@ static int dump_reftable(const char *tablename) done: reftable_merged_table_free(mt); - reftable_reader_decref(r); + reftable_table_decref(table); return err; } @@ -184,7 +184,7 @@ int cmd__dump_reftable(int argc, const char **argv) arg = argv[1]; if (opt_dump_blocks) { - err = reftable_reader_print_blocks(arg); + err = reftable_table_print_blocks(arg); } else if (opt_dump_table) { err = dump_reftable(arg); } else if (opt_dump_stack) { diff --git a/t/meson.build b/t/meson.build index a59da26be3f..ded928d9e6d 100644 --- a/t/meson.build +++ b/t/meson.build @@ -56,10 +56,10 @@ unit_test_programs = [ 'unit-tests/t-reftable-block.c', 'unit-tests/t-reftable-merged.c', 'unit-tests/t-reftable-pq.c', - 'unit-tests/t-reftable-reader.c', 'unit-tests/t-reftable-readwrite.c', 'unit-tests/t-reftable-record.c', 'unit-tests/t-reftable-stack.c', + 'unit-tests/t-reftable-table.c', 'unit-tests/t-trailer.c', 'unit-tests/t-urlmatch-normalization.c', ] diff --git a/t/unit-tests/t-reftable-merged.c b/t/unit-tests/t-reftable-merged.c index 60836f80d60..fed6beb85c0 100644 --- a/t/unit-tests/t-reftable-merged.c +++ b/t/unit-tests/t-reftable-merged.c @@ -11,7 +11,7 @@ license that can be found in the LICENSE file or at #include "reftable/blocksource.h" #include "reftable/constants.h" #include "reftable/merged.h" -#include "reftable/reader.h" +#include "reftable/table.h" #include "reftable/reftable-error.h" #include "reftable/reftable-merged.h" #include "reftable/reftable-writer.h" @@ -19,7 +19,7 @@ license that can be found in the LICENSE file or at static struct reftable_merged_table * merged_table_from_records(struct reftable_ref_record **refs, struct reftable_block_source **source, - struct reftable_reader ***readers, const size_t *sizes, + struct reftable_table ***tables, const size_t *sizes, struct reftable_buf *buf, const size_t n) { struct reftable_merged_table *mt = NULL; @@ -28,8 +28,8 @@ merged_table_from_records(struct reftable_ref_record **refs, }; int err; - REFTABLE_CALLOC_ARRAY(*readers, n); - check(*readers != NULL); + REFTABLE_CALLOC_ARRAY(*tables, n); + check(*tables != NULL); REFTABLE_CALLOC_ARRAY(*source, n); check(*source != NULL); @@ -37,21 +37,21 @@ merged_table_from_records(struct reftable_ref_record **refs, t_reftable_write_to_buf(&buf[i], refs[i], sizes[i], NULL, 0, &opts); block_source_from_buf(&(*source)[i], &buf[i]); - err = reftable_reader_new(&(*readers)[i], &(*source)[i], - "name"); + err = reftable_table_new(&(*tables)[i], &(*source)[i], + "name"); check(!err); } - err = reftable_merged_table_new(&mt, *readers, n, REFTABLE_HASH_SHA1); + err = reftable_merged_table_new(&mt, *tables, n, REFTABLE_HASH_SHA1); check(!err); return mt; } -static void readers_destroy(struct reftable_reader **readers, const size_t n) +static void tables_destroy(struct reftable_table **tables, const size_t n) { for (size_t i = 0; i < n; i++) - reftable_reader_decref(readers[i]); - reftable_free(readers); + reftable_table_decref(tables[i]); + reftable_free(tables); } static void t_merged_single_record(void) @@ -77,9 +77,9 @@ static void t_merged_single_record(void) size_t sizes[] = { ARRAY_SIZE(r1), ARRAY_SIZE(r2), ARRAY_SIZE(r3) }; struct reftable_buf bufs[3] = { REFTABLE_BUF_INIT, REFTABLE_BUF_INIT, REFTABLE_BUF_INIT }; struct reftable_block_source *bs = NULL; - struct reftable_reader **readers = NULL; + struct reftable_table **tables = NULL; struct reftable_merged_table *mt = - merged_table_from_records(refs, &bs, &readers, sizes, bufs, 3); + merged_table_from_records(refs, &bs, &tables, sizes, bufs, 3); struct reftable_ref_record ref = { 0 }; struct reftable_iterator it = { 0 }; int err; @@ -94,7 +94,7 @@ static void t_merged_single_record(void) check(reftable_ref_record_equal(&r2[0], &ref, REFTABLE_HASH_SIZE_SHA1)); reftable_ref_record_release(&ref); reftable_iterator_destroy(&it); - readers_destroy(readers, 3); + tables_destroy(tables, 3); reftable_merged_table_free(mt); for (size_t i = 0; i < ARRAY_SIZE(bufs); i++) reftable_buf_release(&bufs[i]); @@ -154,9 +154,9 @@ static void t_merged_refs(void) size_t sizes[3] = { ARRAY_SIZE(r1), ARRAY_SIZE(r2), ARRAY_SIZE(r3) }; struct reftable_buf bufs[3] = { REFTABLE_BUF_INIT, REFTABLE_BUF_INIT, REFTABLE_BUF_INIT }; struct reftable_block_source *bs = NULL; - struct reftable_reader **readers = NULL; + struct reftable_table **tables = NULL; struct reftable_merged_table *mt = - merged_table_from_records(refs, &bs, &readers, sizes, bufs, 3); + merged_table_from_records(refs, &bs, &tables, sizes, bufs, 3); struct reftable_iterator it = { 0 }; int err; struct reftable_ref_record *out = NULL; @@ -193,7 +193,7 @@ static void t_merged_refs(void) for (i = 0; i < 3; i++) reftable_buf_release(&bufs[i]); - readers_destroy(readers, 3); + tables_destroy(tables, 3); reftable_merged_table_free(mt); reftable_free(bs); } @@ -238,12 +238,12 @@ static void t_merged_seek_multiple_times(void) REFTABLE_BUF_INIT, REFTABLE_BUF_INIT, }; struct reftable_block_source *sources = NULL; - struct reftable_reader **readers = NULL; + struct reftable_table **tables = NULL; struct reftable_ref_record rec = { 0 }; struct reftable_iterator it = { 0 }; struct reftable_merged_table *mt; - mt = merged_table_from_records(refs, &sources, &readers, sizes, bufs, 2); + mt = merged_table_from_records(refs, &sources, &tables, sizes, bufs, 2); merged_table_init_iter(mt, &it, BLOCK_TYPE_REF); for (size_t i = 0; i < 5; i++) { @@ -266,7 +266,7 @@ static void t_merged_seek_multiple_times(void) for (size_t i = 0; i < ARRAY_SIZE(bufs); i++) reftable_buf_release(&bufs[i]); - readers_destroy(readers, ARRAY_SIZE(refs)); + tables_destroy(tables, ARRAY_SIZE(refs)); reftable_ref_record_release(&rec); reftable_iterator_destroy(&it); reftable_merged_table_free(mt); @@ -313,13 +313,13 @@ static void t_merged_seek_multiple_times_without_draining(void) REFTABLE_BUF_INIT, REFTABLE_BUF_INIT, }; struct reftable_block_source *sources = NULL; - struct reftable_reader **readers = NULL; + struct reftable_table **tables = NULL; struct reftable_ref_record rec = { 0 }; struct reftable_iterator it = { 0 }; struct reftable_merged_table *mt; int err; - mt = merged_table_from_records(refs, &sources, &readers, sizes, bufs, 2); + mt = merged_table_from_records(refs, &sources, &tables, sizes, bufs, 2); merged_table_init_iter(mt, &it, BLOCK_TYPE_REF); err = reftable_iterator_seek_ref(&it, "b"); @@ -338,7 +338,7 @@ static void t_merged_seek_multiple_times_without_draining(void) for (size_t i = 0; i < ARRAY_SIZE(bufs); i++) reftable_buf_release(&bufs[i]); - readers_destroy(readers, ARRAY_SIZE(refs)); + tables_destroy(tables, ARRAY_SIZE(refs)); reftable_ref_record_release(&rec); reftable_iterator_destroy(&it); reftable_merged_table_free(mt); @@ -348,7 +348,7 @@ static void t_merged_seek_multiple_times_without_draining(void) static struct reftable_merged_table * merged_table_from_log_records(struct reftable_log_record **logs, struct reftable_block_source **source, - struct reftable_reader ***readers, const size_t *sizes, + struct reftable_table ***tables, const size_t *sizes, struct reftable_buf *buf, const size_t n) { struct reftable_merged_table *mt = NULL; @@ -358,8 +358,8 @@ merged_table_from_log_records(struct reftable_log_record **logs, }; int err; - REFTABLE_CALLOC_ARRAY(*readers, n); - check(*readers != NULL); + REFTABLE_CALLOC_ARRAY(*tables, n); + check(*tables != NULL); REFTABLE_CALLOC_ARRAY(*source, n); check(*source != NULL); @@ -367,12 +367,12 @@ merged_table_from_log_records(struct reftable_log_record **logs, t_reftable_write_to_buf(&buf[i], NULL, 0, logs[i], sizes[i], &opts); block_source_from_buf(&(*source)[i], &buf[i]); - err = reftable_reader_new(&(*readers)[i], &(*source)[i], - "name"); + err = reftable_table_new(&(*tables)[i], &(*source)[i], + "name"); check(!err); } - err = reftable_merged_table_new(&mt, *readers, n, REFTABLE_HASH_SHA1); + err = reftable_merged_table_new(&mt, *tables, n, REFTABLE_HASH_SHA1); check(!err); return mt; } @@ -435,9 +435,9 @@ static void t_merged_logs(void) size_t sizes[3] = { ARRAY_SIZE(r1), ARRAY_SIZE(r2), ARRAY_SIZE(r3) }; struct reftable_buf bufs[3] = { REFTABLE_BUF_INIT, REFTABLE_BUF_INIT, REFTABLE_BUF_INIT }; struct reftable_block_source *bs = NULL; - struct reftable_reader **readers = NULL; + struct reftable_table **tables = NULL; struct reftable_merged_table *mt = merged_table_from_log_records( - logs, &bs, &readers, sizes, bufs, 3); + logs, &bs, &tables, sizes, bufs, 3); struct reftable_iterator it = { 0 }; int err; struct reftable_log_record *out = NULL; @@ -485,7 +485,7 @@ static void t_merged_logs(void) for (i = 0; i < 3; i++) reftable_buf_release(&bufs[i]); - readers_destroy(readers, 3); + tables_destroy(tables, 3); reftable_merged_table_free(mt); reftable_free(bs); } @@ -502,7 +502,7 @@ static void t_default_write_opts(void) int err; struct reftable_block_source source = { 0 }; uint32_t hash_id; - struct reftable_reader *rd = NULL; + struct reftable_table *table = NULL; struct reftable_merged_table *merged = NULL; reftable_writer_set_limits(w, 1, 1); @@ -516,18 +516,18 @@ static void t_default_write_opts(void) block_source_from_buf(&source, &buf); - err = reftable_reader_new(&rd, &source, "filename"); + err = reftable_table_new(&table, &source, "filename"); check(!err); - hash_id = reftable_reader_hash_id(rd); + hash_id = reftable_table_hash_id(table); check_int(hash_id, ==, REFTABLE_HASH_SHA1); - err = reftable_merged_table_new(&merged, &rd, 1, REFTABLE_HASH_SHA256); + err = reftable_merged_table_new(&merged, &table, 1, REFTABLE_HASH_SHA256); check_int(err, ==, REFTABLE_FORMAT_ERROR); - err = reftable_merged_table_new(&merged, &rd, 1, REFTABLE_HASH_SHA1); + err = reftable_merged_table_new(&merged, &table, 1, REFTABLE_HASH_SHA1); check(!err); - reftable_reader_decref(rd); + reftable_table_decref(table); reftable_merged_table_free(merged); reftable_buf_release(&buf); } diff --git a/t/unit-tests/t-reftable-readwrite.c b/t/unit-tests/t-reftable-readwrite.c index c9626831dac..c4c27242ba9 100644 --- a/t/unit-tests/t-reftable-readwrite.c +++ b/t/unit-tests/t-reftable-readwrite.c @@ -12,9 +12,9 @@ license that can be found in the LICENSE file or at #include "lib-reftable.h" #include "reftable/basics.h" #include "reftable/blocksource.h" -#include "reftable/reader.h" #include "reftable/reftable-error.h" #include "reftable/reftable-writer.h" +#include "reftable/table.h" #include "strbuf.h" static const int update_index = 5; @@ -204,7 +204,7 @@ static void t_log_write_read(void) struct reftable_ref_record ref = { 0 }; struct reftable_log_record log = { 0 }; struct reftable_iterator it = { 0 }; - struct reftable_reader *reader; + struct reftable_table *table; struct reftable_block_source source = { 0 }; struct reftable_buf buf = REFTABLE_BUF_INIT; struct reftable_writer *w = t_reftable_strbuf_writer(&buf, &opts); @@ -254,10 +254,10 @@ static void t_log_write_read(void) block_source_from_buf(&source, &buf); - err = reftable_reader_new(&reader, &source, "file.log"); + err = reftable_table_new(&table, &source, "file.log"); check(!err); - err = reftable_reader_init_ref_iterator(reader, &it); + err = reftable_table_init_ref_iterator(table, &it); check(!err); err = reftable_iterator_seek_ref(&it, names[N - 1]); @@ -273,7 +273,7 @@ static void t_log_write_read(void) reftable_iterator_destroy(&it); reftable_ref_record_release(&ref); - err = reftable_reader_init_log_iterator(reader, &it); + err = reftable_table_init_log_iterator(table, &it); check(!err); err = reftable_iterator_seek_log(&it, ""); check(!err); @@ -294,7 +294,7 @@ static void t_log_write_read(void) /* cleanup. */ reftable_buf_release(&buf); free_names(names); - reftable_reader_decref(reader); + reftable_table_decref(table); } static void t_log_zlib_corruption(void) @@ -303,7 +303,7 @@ static void t_log_zlib_corruption(void) .block_size = 256, }; struct reftable_iterator it = { 0 }; - struct reftable_reader *reader; + struct reftable_table *table; struct reftable_block_source source = { 0 }; struct reftable_buf buf = REFTABLE_BUF_INIT; struct reftable_writer *w = t_reftable_strbuf_writer(&buf, &opts); @@ -345,10 +345,10 @@ static void t_log_zlib_corruption(void) block_source_from_buf(&source, &buf); - err = reftable_reader_new(&reader, &source, "file.log"); + err = reftable_table_new(&table, &source, "file.log"); check(!err); - err = reftable_reader_init_log_iterator(reader, &it); + err = reftable_table_init_log_iterator(table, &it); check(!err); err = reftable_iterator_seek_log(&it, "refname"); check_int(err, ==, REFTABLE_ZLIB_ERROR); @@ -356,7 +356,7 @@ static void t_log_zlib_corruption(void) reftable_iterator_destroy(&it); /* cleanup. */ - reftable_reader_decref(reader); + reftable_table_decref(table); reftable_buf_release(&buf); } @@ -367,7 +367,7 @@ static void t_table_read_write_sequential(void) int N = 50; struct reftable_iterator it = { 0 }; struct reftable_block_source source = { 0 }; - struct reftable_reader *reader; + struct reftable_table *table; int err = 0; int j = 0; @@ -375,10 +375,10 @@ static void t_table_read_write_sequential(void) block_source_from_buf(&source, &buf); - err = reftable_reader_new(&reader, &source, "file.ref"); + err = reftable_table_new(&table, &source, "file.ref"); check(!err); - err = reftable_reader_init_ref_iterator(reader, &it); + err = reftable_table_init_ref_iterator(table, &it); check(!err); err = reftable_iterator_seek_ref(&it, ""); check(!err); @@ -396,7 +396,7 @@ static void t_table_read_write_sequential(void) check_int(j, ==, N); reftable_iterator_destroy(&it); - reftable_reader_decref(reader); + reftable_table_decref(table); reftable_buf_release(&buf); free_names(names); } @@ -417,7 +417,7 @@ static void t_table_read_api(void) char **names; struct reftable_buf buf = REFTABLE_BUF_INIT; int N = 50; - struct reftable_reader *reader; + struct reftable_table *table; struct reftable_block_source source = { 0 }; int err; struct reftable_log_record log = { 0 }; @@ -427,10 +427,10 @@ static void t_table_read_api(void) block_source_from_buf(&source, &buf); - err = reftable_reader_new(&reader, &source, "file.ref"); + err = reftable_table_new(&table, &source, "file.ref"); check(!err); - err = reftable_reader_init_ref_iterator(reader, &it); + err = reftable_table_init_ref_iterator(table, &it); check(!err); err = reftable_iterator_seek_ref(&it, names[0]); check(!err); @@ -441,7 +441,7 @@ static void t_table_read_api(void) reftable_buf_release(&buf); free_names(names); reftable_iterator_destroy(&it); - reftable_reader_decref(reader); + reftable_table_decref(table); reftable_buf_release(&buf); } @@ -450,7 +450,7 @@ static void t_table_read_write_seek(int index, enum reftable_hash hash_id) char **names; struct reftable_buf buf = REFTABLE_BUF_INIT; int N = 50; - struct reftable_reader *reader; + struct reftable_table *table; struct reftable_block_source source = { 0 }; int err; int i = 0; @@ -463,18 +463,18 @@ static void t_table_read_write_seek(int index, enum reftable_hash hash_id) block_source_from_buf(&source, &buf); - err = reftable_reader_new(&reader, &source, "file.ref"); + err = reftable_table_new(&table, &source, "file.ref"); check(!err); - check_int(hash_id, ==, reftable_reader_hash_id(reader)); + check_int(hash_id, ==, reftable_table_hash_id(table)); if (!index) { - reader->ref_offsets.index_offset = 0; + table->ref_offsets.index_offset = 0; } else { - check_int(reader->ref_offsets.index_offset, >, 0); + check_int(table->ref_offsets.index_offset, >, 0); } for (i = 1; i < N; i++) { - err = reftable_reader_init_ref_iterator(reader, &it); + err = reftable_table_init_ref_iterator(table, &it); check(!err); err = reftable_iterator_seek_ref(&it, names[i]); check(!err); @@ -491,7 +491,7 @@ static void t_table_read_write_seek(int index, enum reftable_hash hash_id) check(!reftable_buf_addstr(&pastLast, names[N - 1])); check(!reftable_buf_addstr(&pastLast, "/")); - err = reftable_reader_init_ref_iterator(reader, &it); + err = reftable_table_init_ref_iterator(table, &it); check(!err); err = reftable_iterator_seek_ref(&it, pastLast.buf); if (err == 0) { @@ -507,7 +507,7 @@ static void t_table_read_write_seek(int index, enum reftable_hash hash_id) reftable_buf_release(&buf); free_names(names); - reftable_reader_decref(reader); + reftable_table_decref(table); } static void t_table_read_write_seek_linear(void) @@ -535,7 +535,7 @@ static void t_table_refs_for(int indexed) .block_size = 256, }; struct reftable_ref_record ref = { 0 }; - struct reftable_reader *reader; + struct reftable_table *table; struct reftable_block_source source = { 0 }; struct reftable_buf buf = REFTABLE_BUF_INIT; struct reftable_writer *w = t_reftable_strbuf_writer(&buf, &opts); @@ -585,18 +585,18 @@ static void t_table_refs_for(int indexed) block_source_from_buf(&source, &buf); - err = reftable_reader_new(&reader, &source, "file.ref"); + err = reftable_table_new(&table, &source, "file.ref"); check(!err); if (!indexed) - reader->obj_offsets.is_present = 0; + table->obj_offsets.is_present = 0; - err = reftable_reader_init_ref_iterator(reader, &it); + err = reftable_table_init_ref_iterator(table, &it); check(!err); err = reftable_iterator_seek_ref(&it, ""); check(!err); reftable_iterator_destroy(&it); - err = reftable_reader_refs_for(reader, &it, want_hash); + err = reftable_table_refs_for(table, &it, want_hash); check(!err); for (j = 0; ; j++) { @@ -613,7 +613,7 @@ static void t_table_refs_for(int indexed) reftable_buf_release(&buf); free_names(want_names); reftable_iterator_destroy(&it); - reftable_reader_decref(reader); + reftable_table_decref(table); } static void t_table_refs_for_no_index(void) @@ -632,7 +632,7 @@ static void t_write_empty_table(void) struct reftable_buf buf = REFTABLE_BUF_INIT; struct reftable_writer *w = t_reftable_strbuf_writer(&buf, &opts); struct reftable_block_source source = { 0 }; - struct reftable_reader *rd = NULL; + struct reftable_table *table = NULL; struct reftable_ref_record rec = { 0 }; struct reftable_iterator it = { 0 }; int err; @@ -647,10 +647,10 @@ static void t_write_empty_table(void) block_source_from_buf(&source, &buf); - err = reftable_reader_new(&rd, &source, "filename"); + err = reftable_table_new(&table, &source, "filename"); check(!err); - err = reftable_reader_init_ref_iterator(rd, &it); + err = reftable_table_init_ref_iterator(table, &it); check(!err); err = reftable_iterator_seek_ref(&it, ""); check(!err); @@ -659,7 +659,7 @@ static void t_write_empty_table(void) check_int(err, >, 0); reftable_iterator_destroy(&it); - reftable_reader_decref(rd); + reftable_table_decref(table); reftable_buf_release(&buf); } @@ -803,7 +803,7 @@ static void t_write_multiple_indices(void) struct reftable_iterator it = { 0 }; const struct reftable_stats *stats; struct reftable_writer *writer; - struct reftable_reader *reader; + struct reftable_table *table; char buf[128]; int err, i; @@ -852,21 +852,21 @@ static void t_write_multiple_indices(void) check_int(stats->log_stats.index_offset, >, 0); block_source_from_buf(&source, &writer_buf); - err = reftable_reader_new(&reader, &source, "filename"); + err = reftable_table_new(&table, &source, "filename"); check(!err); /* * Seeking the log uses the log index now. In case there is any * confusion regarding indices we would notice here. */ - err = reftable_reader_init_log_iterator(reader, &it); + err = reftable_table_init_log_iterator(table, &it); check(!err); err = reftable_iterator_seek_log(&it, ""); check(!err); reftable_iterator_destroy(&it); reftable_writer_free(writer); - reftable_reader_decref(reader); + reftable_table_decref(table); reftable_buf_release(&writer_buf); } @@ -880,7 +880,7 @@ static void t_write_multi_level_index(void) struct reftable_iterator it = { 0 }; const struct reftable_stats *stats; struct reftable_writer *writer; - struct reftable_reader *reader; + struct reftable_table *table; int err; writer = t_reftable_strbuf_writer(&writer_buf, &opts); @@ -909,20 +909,20 @@ static void t_write_multi_level_index(void) check_int(stats->ref_stats.max_index_level, ==, 2); block_source_from_buf(&source, &writer_buf); - err = reftable_reader_new(&reader, &source, "filename"); + err = reftable_table_new(&table, &source, "filename"); check(!err); /* * Seeking the last ref should work as expected. */ - err = reftable_reader_init_ref_iterator(reader, &it); + err = reftable_table_init_ref_iterator(table, &it); check(!err); err = reftable_iterator_seek_ref(&it, "refs/heads/199"); check(!err); reftable_iterator_destroy(&it); reftable_writer_free(writer); - reftable_reader_decref(reader); + reftable_table_decref(table); reftable_buf_release(&writer_buf); reftable_buf_release(&buf); } @@ -931,11 +931,11 @@ static void t_corrupt_table_empty(void) { struct reftable_buf buf = REFTABLE_BUF_INIT; struct reftable_block_source source = { 0 }; - struct reftable_reader *reader; + struct reftable_table *table; int err; block_source_from_buf(&source, &buf); - err = reftable_reader_new(&reader, &source, "file.log"); + err = reftable_table_new(&table, &source, "file.log"); check_int(err, ==, REFTABLE_FORMAT_ERROR); } @@ -944,12 +944,12 @@ static void t_corrupt_table(void) uint8_t zeros[1024] = { 0 }; struct reftable_buf buf = REFTABLE_BUF_INIT; struct reftable_block_source source = { 0 }; - struct reftable_reader *reader; + struct reftable_table *table; int err; check(!reftable_buf_add(&buf, zeros, sizeof(zeros))); block_source_from_buf(&source, &buf); - err = reftable_reader_new(&reader, &source, "file.log"); + err = reftable_table_new(&table, &source, "file.log"); check_int(err, ==, REFTABLE_FORMAT_ERROR); reftable_buf_release(&buf); diff --git a/t/unit-tests/t-reftable-stack.c b/t/unit-tests/t-reftable-stack.c index c3f0059c346..2f49c975194 100644 --- a/t/unit-tests/t-reftable-stack.c +++ b/t/unit-tests/t-reftable-stack.c @@ -12,9 +12,9 @@ license that can be found in the LICENSE file or at #include "lib-reftable.h" #include "dir.h" #include "reftable/merged.h" -#include "reftable/reader.h" #include "reftable/reftable-error.h" #include "reftable/stack.h" +#include "reftable/table.h" #include "strbuf.h" #include "tempfile.h" #include @@ -176,7 +176,7 @@ static void t_reftable_stack_add_one(void) err = reftable_stack_read_ref(st, ref.refname, &dest); check(!err); check(reftable_ref_record_equal(&ref, &dest, REFTABLE_HASH_SIZE_SHA1)); - check_int(st->readers_len, >, 0); + check_int(st->tables_len, >, 0); #ifndef GIT_WINDOWS_NATIVE check(!reftable_buf_addstr(&scratch, dir)); @@ -189,7 +189,7 @@ static void t_reftable_stack_add_one(void) check(!reftable_buf_addstr(&scratch, dir)); check(!reftable_buf_addstr(&scratch, "/")); /* do not try at home; not an external API for reftable. */ - check(!reftable_buf_addstr(&scratch, st->readers[0]->name)); + check(!reftable_buf_addstr(&scratch, st->tables[0]->name)); err = stat(scratch.buf, &stat_result); check(!err); check_int((stat_result.st_mode & 0777), ==, opts.default_permissions); @@ -402,9 +402,9 @@ static void t_reftable_stack_transaction_api_performs_auto_compaction(void) * all tables in the stack. */ if (i != n) - check_int(st->merged->readers_len, ==, i + 1); + check_int(st->merged->tables_len, ==, i + 1); else - check_int(st->merged->readers_len, ==, 1); + check_int(st->merged->tables_len, ==, 1); } reftable_stack_destroy(st); @@ -430,7 +430,7 @@ static void t_reftable_stack_auto_compaction_fails_gracefully(void) err = reftable_stack_add(st, write_test_ref, &ref); check(!err); - check_int(st->merged->readers_len, ==, 1); + check_int(st->merged->tables_len, ==, 1); check_int(st->stats.attempts, ==, 0); check_int(st->stats.failures, ==, 0); @@ -441,14 +441,14 @@ static void t_reftable_stack_auto_compaction_fails_gracefully(void) */ check(!reftable_buf_addstr(&table_path, dir)); check(!reftable_buf_addstr(&table_path, "/")); - check(!reftable_buf_addstr(&table_path, st->readers[0]->name)); + check(!reftable_buf_addstr(&table_path, st->tables[0]->name)); check(!reftable_buf_addstr(&table_path, ".lock")); write_file_buf(table_path.buf, "", 0); ref.update_index = 2; err = reftable_stack_add(st, write_test_ref, &ref); check(!err); - check_int(st->merged->readers_len, ==, 2); + check_int(st->merged->tables_len, ==, 2); check_int(st->stats.attempts, ==, 1); check_int(st->stats.failures, ==, 1); @@ -592,7 +592,7 @@ static void t_reftable_stack_add(void) check(!reftable_buf_addstr(&path, dir)); check(!reftable_buf_addstr(&path, "/")); /* do not try at home; not an external API for reftable. */ - check(!reftable_buf_addstr(&path, st->readers[0]->name)); + check(!reftable_buf_addstr(&path, st->tables[0]->name)); err = stat(path.buf, &stat_result); check(!err); check_int((stat_result.st_mode & 0777), ==, opts.default_permissions); @@ -1026,7 +1026,7 @@ static void t_reftable_stack_auto_compaction(void) err = reftable_stack_auto_compact(st); check(!err); - check(i < 2 || st->merged->readers_len < 2 * fastlogN(i, 2)); + check(i < 2 || st->merged->tables_len < 2 * fastlogN(i, 2)); } check_int(reftable_stack_compaction_stats(st)->entries_written, <, @@ -1061,7 +1061,7 @@ static void t_reftable_stack_auto_compaction_factor(void) err = reftable_stack_add(st, &write_test_ref, &ref); check(!err); - check(i < 5 || st->merged->readers_len < 5 * fastlogN(i, 5)); + check(i < 5 || st->merged->tables_len < 5 * fastlogN(i, 5)); } reftable_stack_destroy(st); @@ -1082,7 +1082,7 @@ static void t_reftable_stack_auto_compaction_with_locked_tables(void) check(!err); write_n_ref_tables(st, 5); - check_int(st->merged->readers_len, ==, 5); + check_int(st->merged->tables_len, ==, 5); /* * Given that all tables we have written should be roughly the same @@ -1091,7 +1091,7 @@ static void t_reftable_stack_auto_compaction_with_locked_tables(void) */ check(!reftable_buf_addstr(&buf, dir)); check(!reftable_buf_addstr(&buf, "/")); - check(!reftable_buf_addstr(&buf, st->readers[2]->name)); + check(!reftable_buf_addstr(&buf, st->tables[2]->name)); check(!reftable_buf_addstr(&buf, ".lock")); write_file_buf(buf.buf, "", 0); @@ -1104,7 +1104,7 @@ static void t_reftable_stack_auto_compaction_with_locked_tables(void) err = reftable_stack_auto_compact(st); check(!err); check_int(st->stats.failures, ==, 0); - check_int(st->merged->readers_len, ==, 4); + check_int(st->merged->tables_len, ==, 4); reftable_stack_destroy(st); reftable_buf_release(&buf); @@ -1149,9 +1149,9 @@ static void t_reftable_stack_add_performs_auto_compaction(void) * all tables in the stack. */ if (i != n) - check_int(st->merged->readers_len, ==, i + 1); + check_int(st->merged->tables_len, ==, i + 1); else - check_int(st->merged->readers_len, ==, 1); + check_int(st->merged->tables_len, ==, 1); } reftable_stack_destroy(st); @@ -1172,12 +1172,12 @@ static void t_reftable_stack_compaction_with_locked_tables(void) check(!err); write_n_ref_tables(st, 3); - check_int(st->merged->readers_len, ==, 3); + check_int(st->merged->tables_len, ==, 3); /* Lock one of the tables that we're about to compact. */ check(!reftable_buf_addstr(&buf, dir)); check(!reftable_buf_addstr(&buf, "/")); - check(!reftable_buf_addstr(&buf, st->readers[1]->name)); + check(!reftable_buf_addstr(&buf, st->tables[1]->name)); check(!reftable_buf_addstr(&buf, ".lock")); write_file_buf(buf.buf, "", 0); @@ -1188,7 +1188,7 @@ static void t_reftable_stack_compaction_with_locked_tables(void) err = reftable_stack_compact_all(st, NULL); check_int(err, ==, REFTABLE_LOCK_ERROR); check_int(st->stats.failures, ==, 1); - check_int(st->merged->readers_len, ==, 3); + check_int(st->merged->tables_len, ==, 3); reftable_stack_destroy(st); reftable_buf_release(&buf); @@ -1222,10 +1222,10 @@ static void t_reftable_stack_compaction_concurrent(void) static void unclean_stack_close(struct reftable_stack *st) { /* break abstraction boundary to simulate unclean shutdown. */ - for (size_t i = 0; i < st->readers_len; i++) - reftable_reader_decref(st->readers[i]); - st->readers_len = 0; - REFTABLE_FREE_AND_NULL(st->readers); + for (size_t i = 0; i < st->tables_len; i++) + reftable_table_decref(st->tables[i]); + st->tables_len = 0; + REFTABLE_FREE_AND_NULL(st->tables); } static void t_reftable_stack_compaction_concurrent_clean(void) @@ -1275,7 +1275,7 @@ static void t_reftable_stack_read_across_reload(void) err = reftable_new_stack(&st1, dir, &opts); check(!err); write_n_ref_tables(st1, 2); - check_int(st1->merged->readers_len, ==, 2); + check_int(st1->merged->tables_len, ==, 2); reftable_stack_init_ref_iterator(st1, &it); err = reftable_iterator_seek_ref(&it, ""); check(!err); @@ -1283,10 +1283,10 @@ static void t_reftable_stack_read_across_reload(void) /* Set up a second stack for the same directory and compact it. */ err = reftable_new_stack(&st2, dir, &opts); check(!err); - check_int(st2->merged->readers_len, ==, 2); + check_int(st2->merged->tables_len, ==, 2); err = reftable_stack_compact_all(st2, NULL); check(!err); - check_int(st2->merged->readers_len, ==, 1); + check_int(st2->merged->tables_len, ==, 1); /* * Verify that we can continue to use the old iterator even after we @@ -1294,7 +1294,7 @@ static void t_reftable_stack_read_across_reload(void) */ err = reftable_stack_reload(st1); check(!err); - check_int(st1->merged->readers_len, ==, 1); + check_int(st1->merged->tables_len, ==, 1); err = reftable_iterator_next_ref(&it, &rec); check(!err); check_str(rec.refname, "refs/heads/branch-0000"); @@ -1325,19 +1325,19 @@ static void t_reftable_stack_reload_with_missing_table(void) err = reftable_new_stack(&st, dir, &opts); check(!err); write_n_ref_tables(st, 2); - check_int(st->merged->readers_len, ==, 2); + check_int(st->merged->tables_len, ==, 2); reftable_stack_init_ref_iterator(st, &it); err = reftable_iterator_seek_ref(&it, ""); check(!err); /* * Update the tables.list file with some garbage data, while reusing - * our old readers. This should trigger a partial reload of the stack, - * where we try to reuse our old readers. + * our old tables. This should trigger a partial reload of the stack, + * where we try to reuse our old tables. */ - check(!reftable_buf_addstr(&content, st->readers[0]->name)); + check(!reftable_buf_addstr(&content, st->tables[0]->name)); check(!reftable_buf_addstr(&content, "\n")); - check(!reftable_buf_addstr(&content, st->readers[1]->name)); + check(!reftable_buf_addstr(&content, st->tables[1]->name)); check(!reftable_buf_addstr(&content, "\n")); check(!reftable_buf_addstr(&content, "garbage\n")); check(!reftable_buf_addstr(&table_path, st->list_file)); @@ -1348,7 +1348,7 @@ static void t_reftable_stack_reload_with_missing_table(void) err = reftable_stack_reload(st); check_int(err, ==, -4); - check_int(st->merged->readers_len, ==, 2); + check_int(st->merged->tables_len, ==, 2); /* * Even though the reload has failed, we should be able to continue diff --git a/t/unit-tests/t-reftable-reader.c b/t/unit-tests/t-reftable-table.c similarity index 78% rename from t/unit-tests/t-reftable-reader.c rename to t/unit-tests/t-reftable-table.c index 546df6005e4..77c59dbf46d 100644 --- a/t/unit-tests/t-reftable-reader.c +++ b/t/unit-tests/t-reftable-table.c @@ -1,9 +1,9 @@ #include "test-lib.h" #include "lib-reftable.h" #include "reftable/blocksource.h" -#include "reftable/reader.h" +#include "reftable/table.h" -static int t_reader_seek_once(void) +static int t_table_seek_once(void) { struct reftable_ref_record records[] = { { @@ -15,17 +15,17 @@ static int t_reader_seek_once(void) struct reftable_block_source source = { 0 }; struct reftable_ref_record ref = { 0 }; struct reftable_iterator it = { 0 }; - struct reftable_reader *reader; + struct reftable_table *table; struct reftable_buf buf = REFTABLE_BUF_INIT; int ret; t_reftable_write_to_buf(&buf, records, ARRAY_SIZE(records), NULL, 0, NULL); block_source_from_buf(&source, &buf); - ret = reftable_reader_new(&reader, &source, "name"); + ret = reftable_table_new(&table, &source, "name"); check(!ret); - reftable_reader_init_ref_iterator(reader, &it); + reftable_table_init_ref_iterator(table, &it); ret = reftable_iterator_seek_ref(&it, ""); check(!ret); ret = reftable_iterator_next_ref(&it, &ref); @@ -39,12 +39,12 @@ static int t_reader_seek_once(void) reftable_ref_record_release(&ref); reftable_iterator_destroy(&it); - reftable_reader_decref(reader); + reftable_table_decref(table); reftable_buf_release(&buf); return 0; } -static int t_reader_reseek(void) +static int t_table_reseek(void) { struct reftable_ref_record records[] = { { @@ -56,17 +56,17 @@ static int t_reader_reseek(void) struct reftable_block_source source = { 0 }; struct reftable_ref_record ref = { 0 }; struct reftable_iterator it = { 0 }; - struct reftable_reader *reader; + struct reftable_table *table; struct reftable_buf buf = REFTABLE_BUF_INIT; int ret; t_reftable_write_to_buf(&buf, records, ARRAY_SIZE(records), NULL, 0, NULL); block_source_from_buf(&source, &buf); - ret = reftable_reader_new(&reader, &source, "name"); + ret = reftable_table_new(&table, &source, "name"); check(!ret); - reftable_reader_init_ref_iterator(reader, &it); + reftable_table_init_ref_iterator(table, &it); for (size_t i = 0; i < 5; i++) { ret = reftable_iterator_seek_ref(&it, ""); @@ -83,14 +83,14 @@ static int t_reader_reseek(void) reftable_ref_record_release(&ref); reftable_iterator_destroy(&it); - reftable_reader_decref(reader); + reftable_table_decref(table); reftable_buf_release(&buf); return 0; } int cmd_main(int argc UNUSED, const char *argv[] UNUSED) { - TEST(t_reader_seek_once(), "reader can seek once"); - TEST(t_reader_reseek(), "reader can reseek multiple times"); + TEST(t_table_seek_once(), "table can seek once"); + TEST(t_table_reseek(), "table can reseek multiple times"); return test_done(); } From patchwork Mon Apr 7 13:16:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040503 Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 249FF253B6E for ; Mon, 7 Apr 2025 13:16:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031790; cv=none; b=NTcczn9vyWuU7yUuYbhFpKQfFLxN7GUtg+kjpxhwJN2C1eI/mM0NnINHYenTj4YcRlkS6psYRYZr9/9JEkAmLLZSxlXz9g2yGqedfWOHSkWsCdxXRZFrYMQzTJ3mJR7g+gt8hgOMgLJzAD3N89Wqn8xNv/JzVjbQf0qDQoprv+0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031790; c=relaxed/simple; bh=/UFJcpSnVrSJbIZFgn+VeYQi9GkR+eYqYE+Zggt+hZo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=R8ZhkBXyJYiw0ITt5jDS/nA9CuKSEc2W9hBnhYE7+GesvavIWsD7qb/tlM4a6MDQRH5xKBMuCzx0qYFZYtL4eAL5yzzR7K4KDVYPQVIK+jsOCQfknen9qtlIAi0Rj7h0WvDAxwwyxK6z6vGwBWZE6myQ2q/wheLfYZRmmhiNBDc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=lhLYrUpB; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=s+GIFaZs; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="lhLYrUpB"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="s+GIFaZs" Received: from phl-compute-02.internal (phl-compute-02.phl.internal [10.202.2.42]) by mailfhigh.phl.internal (Postfix) with ESMTP id 45D4E114020A; Mon, 7 Apr 2025 09:16:26 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-02.internal (MEProxy); Mon, 07 Apr 2025 09:16:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031786; x=1744118186; bh=OFPL7tTwSeOdxD4koHsxZBUp4E8lC8vMlwcKnvxIW1w=; b= lhLYrUpBk8Ujh/qoZuyorDNYqHuDmey1FMYn2SW5Y056DchYFMi8WgctmGbiJ/tu CI1XnsaNktUQfsi9Vsfvpet57stYH+PhmyBNeFSO/2nBYTb5ua4nxo1/PZRz+cmf cDvMk1ID8xeKWYV3E+lijH/skrb6IeKmf27fjEVp3jNAlgaXsxk3KdVceFWa2S7Y whcrNF5rczb1EFuCLjcmYBSZq6CorNbN88sI2QsWZ86y7YoLxwkECrVdPbNAbv65 M+oWXWyKpBaWEgOmaUKsZpRRJwZ9mpGLjgTtAG0c01KerPysQlmCzJaASw9pD7oq pi4xXclTxbqkoyeZWLAgRA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031786; x= 1744118186; bh=OFPL7tTwSeOdxD4koHsxZBUp4E8lC8vMlwcKnvxIW1w=; b=s +GIFaZsg/YGfINGlIYWxnNl7jLdYLz/q5LOIqq/AFRJeVuYSvvNl0hxGv+P3CpUU 5s/JE7okJ+YtZRSX7hV76J3esj4znTc7Lx3M9Or/sXjD0jaoR1gxWWhquSVKc4/+ rJkjDA1gg+NJF7bb4lM54FkJgEgjL8dVysWgoKVsaYwFuo45t5ljYDhhdaNby5tU mHi1TScqaRD2jIX9IYJnLFb+rabWxk/7mumWpXxRK5WZVQp45ocNvp809fZykAID rVa1wPSJAF7UqpDlgxPHjn3MR2upTl3eqNm+bJKJpfZo0ebds4WwP6oILu8M/QNv dsJI61qX4aWBovVJ3jJtA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdekucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteek udehjeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorh hgpdhrtghpthhtohepjhhlthhosghlvghrsehgmhgrihhlrdgtohhmpdhrtghpthhtohep khgrrhhthhhikhdrudekkeesghhmrghilhdrtghomhdprhgtphhtthhopehgihhtshhtvg hrsehpohgsohigrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:25 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 5f1c24af (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:23 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:15 +0200 Subject: [PATCH v2 03/16] reftable/blocksource: consolidate code into a single file Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-3-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 The code that implements block sources is distributed across a couple of files. Consolidate all of it into "reftable/blocksource.c" and its accompanying header so that it is easier to locate and more self contained. While at it, rename some of the functions to have properly scoped names. Signed-off-by: Patrick Steinhardt --- reftable/block.c | 17 +++-------------- reftable/block.h | 3 --- reftable/blocksource.c | 35 +++++++++++++++++++++++++++++++++++ reftable/blocksource.h | 27 ++++++++++++++++++++++++++- reftable/iter.c | 5 +++-- reftable/reftable-blocksource.h | 3 ++- reftable/table.c | 33 +++++---------------------------- reftable/table.h | 7 ------- t/unit-tests/t-reftable-block.c | 8 ++++---- t/unit-tests/t-reftable-readwrite.c | 4 ++-- 10 files changed, 80 insertions(+), 62 deletions(-) diff --git a/reftable/block.c b/reftable/block.c index a5734d44415..97740187259 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -221,7 +221,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block, uint32_t restart_start = 0; uint8_t *restart_bytes = NULL; - reftable_block_done(&br->block); + block_source_return_block(&br->block); if (!reftable_is_block_type(typ)) { err = REFTABLE_FORMAT_ERROR; @@ -285,7 +285,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block, } /* We're done with the input data. */ - reftable_block_done(block); + block_source_return_block(block); block->data = br->uncompressed_data; block->len = sz; full_block_size = src_len + block_header_skip - br->zstream->avail_in; @@ -324,7 +324,7 @@ void block_reader_release(struct block_reader *br) inflateEnd(br->zstream); reftable_free(br->zstream); reftable_free(br->uncompressed_data); - reftable_block_done(&br->block); + block_source_return_block(&br->block); } uint8_t block_reader_type(const struct block_reader *r) @@ -570,14 +570,3 @@ void block_writer_release(struct block_writer *bw) reftable_buf_release(&bw->last_key); /* the block is not owned. */ } - -void reftable_block_done(struct reftable_block *blockp) -{ - struct reftable_block_source source = blockp->source; - if (blockp && source.ops) - source.ops->return_block(source.arg, blockp); - blockp->data = NULL; - blockp->len = 0; - blockp->source.ops = NULL; - blockp->source.arg = NULL; -} diff --git a/reftable/block.h b/reftable/block.h index eaeffdffc90..203b07d9a44 100644 --- a/reftable/block.h +++ b/reftable/block.h @@ -142,7 +142,4 @@ size_t header_size(int version); /* size of file footer, depending on format version */ size_t footer_size(int version); -/* returns a block to its source. */ -void reftable_block_done(struct reftable_block *ret); - #endif diff --git a/reftable/blocksource.c b/reftable/blocksource.c index 1397cbe7800..bc785506fb1 100644 --- a/reftable/blocksource.c +++ b/reftable/blocksource.c @@ -13,6 +13,41 @@ #include "reftable-blocksource.h" #include "reftable-error.h" +void block_source_return_block(struct reftable_block *block) +{ + struct reftable_block_source source = block->source; + if (block && source.ops) + source.ops->return_block(source.arg, block); + block->data = NULL; + block->len = 0; + block->source.ops = NULL; + block->source.arg = NULL; +} + +void block_source_close(struct reftable_block_source *source) +{ + if (!source->ops) { + return; + } + + source->ops->close(source->arg); + source->ops = NULL; +} + +ssize_t block_source_read_block(struct reftable_block_source *source, + struct reftable_block *dest, uint64_t off, + uint32_t size) +{ + ssize_t result = source->ops->read_block(source->arg, dest, off, size); + dest->source = *source; + return result; +} + +uint64_t block_source_size(struct reftable_block_source *source) +{ + return source->ops->size(source->arg); +} + static void reftable_buf_return_block(void *b REFTABLE_UNUSED, struct reftable_block *dest) { if (dest->len) diff --git a/reftable/blocksource.h b/reftable/blocksource.h index 7b67898ae22..639b9a1a3c5 100644 --- a/reftable/blocksource.h +++ b/reftable/blocksource.h @@ -12,9 +12,34 @@ #include "system.h" struct reftable_block_source; +struct reftable_block; struct reftable_buf; -/* Create an in-memory block source for reading reftables */ +/* + * Close the block source and the underlying resource. This is a no-op in case + * the block source is zero-initialized. + */ +void block_source_close(struct reftable_block_source *source); + +/* + * Read a block of length `size` from the source at the given `off`. + */ +ssize_t block_source_read_block(struct reftable_block_source *source, + struct reftable_block *dest, uint64_t off, + uint32_t size); + +/* + * Return the total length of the underlying resource. + */ +uint64_t block_source_size(struct reftable_block_source *source); + +/* + * Return a block to its original source, releasing any resources associated + * with it. + */ +void block_source_return_block(struct reftable_block *block); + +/* Create an in-memory block source for reading reftables. */ void block_source_from_buf(struct reftable_block_source *bs, struct reftable_buf *buf); diff --git a/reftable/iter.c b/reftable/iter.c index 7376f263c99..6af6eb49396 100644 --- a/reftable/iter.c +++ b/reftable/iter.c @@ -11,6 +11,7 @@ #include "system.h" #include "block.h" +#include "blocksource.h" #include "constants.h" #include "reftable-error.h" #include "table.h" @@ -113,7 +114,7 @@ static void indexed_table_ref_iter_close(void *p) { struct indexed_table_ref_iter *it = p; block_iter_close(&it->cur); - reftable_block_done(&it->block_reader.block); + block_source_return_block(&it->block_reader.block); reftable_free(it->offsets); reftable_buf_release(&it->oid); } @@ -127,7 +128,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it) return 1; } - reftable_block_done(&it->block_reader.block); + block_source_return_block(&it->block_reader.block); off = it->offsets[it->offset_idx++]; err = table_init_block_reader(it->table, &it->block_reader, off, diff --git a/reftable/reftable-blocksource.h b/reftable/reftable-blocksource.h index 8692cd017e9..96430b629e4 100644 --- a/reftable/reftable-blocksource.h +++ b/reftable/reftable-blocksource.h @@ -11,7 +11,8 @@ #include -/* block_source is a generic wrapper for a seekable readable file. +/* + * Generic wrapper for a seekable readable file. */ struct reftable_block_source { struct reftable_block_source_vtable *ops; diff --git a/reftable/table.c b/reftable/table.c index 440fb559ad1..d18e17b0d44 100644 --- a/reftable/table.c +++ b/reftable/table.c @@ -10,35 +10,12 @@ #include "system.h" #include "block.h" +#include "blocksource.h" #include "constants.h" #include "iter.h" #include "record.h" #include "reftable-error.h" -uint64_t block_source_size(struct reftable_block_source *source) -{ - return source->ops->size(source->arg); -} - -ssize_t block_source_read_block(struct reftable_block_source *source, - struct reftable_block *dest, uint64_t off, - uint32_t size) -{ - ssize_t result = source->ops->read_block(source->arg, dest, off, size); - dest->source = *source; - return result; -} - -void block_source_close(struct reftable_block_source *source) -{ - if (!source->ops) { - return; - } - - source->ops->close(source->arg); - source->ops = NULL; -} - static struct reftable_table_offsets * table_offsets_for(struct reftable_table *t, uint8_t typ) { @@ -249,7 +226,7 @@ int table_init_block_reader(struct reftable_table *t, struct block_reader *br, } if (block_size > guess_block_size) { - reftable_block_done(&block); + block_source_return_block(&block); err = table_get_block(t, &block, next_off, block_size); if (err < 0) { goto done; @@ -259,7 +236,7 @@ int table_init_block_reader(struct reftable_table *t, struct block_reader *br, err = block_reader_init(br, &block, header_off, t->block_size, hash_size(t->hash_id)); done: - reftable_block_done(&block); + block_source_return_block(&block); return err; } @@ -666,8 +643,8 @@ int reftable_table_new(struct reftable_table **out, *out = t; done: - reftable_block_done(&footer); - reftable_block_done(&header); + block_source_return_block(&footer); + block_source_return_block(&header); if (err) { if (t) reftable_free(t->name); diff --git a/reftable/table.h b/reftable/table.h index 9cd8f80a207..8d8dd2b413d 100644 --- a/reftable/table.h +++ b/reftable/table.h @@ -14,13 +14,6 @@ #include "reftable-iterator.h" #include "reftable-table.h" -uint64_t block_source_size(struct reftable_block_source *source); - -ssize_t block_source_read_block(struct reftable_block_source *source, - struct reftable_block *dest, uint64_t off, - uint32_t size); -void block_source_close(struct reftable_block_source *source); - /* metadata for a block type */ struct reftable_table_offsets { int is_present; diff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c index 22040aeefa5..8bb40482347 100644 --- a/t/unit-tests/t-reftable-block.c +++ b/t/unit-tests/t-reftable-block.c @@ -100,7 +100,7 @@ static void t_ref_block_read_write(void) block_reader_release(&br); block_iter_close(&it); reftable_record_release(&rec); - reftable_block_done(&br.block); + block_source_return_block(&br.block); reftable_buf_release(&want); reftable_buf_release(&buf); for (i = 0; i < N; i++) @@ -190,7 +190,7 @@ static void t_log_block_read_write(void) block_reader_release(&br); block_iter_close(&it); reftable_record_release(&rec); - reftable_block_done(&br.block); + block_source_return_block(&br.block); reftable_buf_release(&want); reftable_buf_release(&buf); for (i = 0; i < N; i++) @@ -273,7 +273,7 @@ static void t_obj_block_read_write(void) block_reader_release(&br); block_iter_close(&it); reftable_record_release(&rec); - reftable_block_done(&br.block); + block_source_return_block(&br.block); reftable_buf_release(&want); reftable_buf_release(&buf); for (i = 0; i < N; i++) @@ -365,7 +365,7 @@ static void t_index_block_read_write(void) block_reader_release(&br); block_iter_close(&it); reftable_record_release(&rec); - reftable_block_done(&br.block); + block_source_return_block(&br.block); reftable_buf_release(&want); reftable_buf_release(&buf); for (i = 0; i < N; i++) diff --git a/t/unit-tests/t-reftable-readwrite.c b/t/unit-tests/t-reftable-readwrite.c index c4c27242ba9..3fba888cdaa 100644 --- a/t/unit-tests/t-reftable-readwrite.c +++ b/t/unit-tests/t-reftable-readwrite.c @@ -32,13 +32,13 @@ static void t_buffer(void) n = block_source_read_block(&source, &out, 0, sizeof(in)); check_int(n, ==, sizeof(in)); check(!memcmp(in, out.data, n)); - reftable_block_done(&out); + block_source_return_block(&out); n = block_source_read_block(&source, &out, 1, 2); check_int(n, ==, 2); check(!memcmp(out.data, "el", 2)); - reftable_block_done(&out); + block_source_return_block(&out); block_source_close(&source); reftable_buf_release(&buf); } From patchwork Mon Apr 7 13:16:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040501 Received: from fout-a1-smtp.messagingengine.com (fout-a1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A443253B70 for ; Mon, 7 Apr 2025 13:16:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031789; cv=none; b=K5tHSAI9d9/q0bBVV/A1/dMcQbkU9TUMgieRpo7v/8TDHmR3qNXL71Un+zysCJliVm9BVTRt3ExCm5wW7fX1akgMPVBOXx36TM99hWTh5aSm9PpweiY1tdq1se7y9/dtF79t67VXBlErqib83P5bppqPsxyh5/mm0z2IvPzfO2g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031789; c=relaxed/simple; bh=wkYSel6L9yDN/33WiGeQXqO7Xhw56pNgmyEftp9rQJQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=o8y9ACrmQ39/uoP9pXspAQ8bYBgvErTqO0MoEre72+n0YzH23XEIBMFseyXeZ11f4h5d0vmxXt+8n974jbgciBCZhfMQRe+Gzr8jRkxVNWMrSJvKiqzLzPTsf2Q1TxWK7HrRqxpv3Ij7P9VlnwxnvuzwK94wLe1I1irWcadUxkc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=AjBI2Bdo; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=bbWUrkxE; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="AjBI2Bdo"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="bbWUrkxE" Received: from phl-compute-05.internal (phl-compute-05.phl.internal [10.202.2.45]) by mailfout.phl.internal (Postfix) with ESMTP id 86EC71380355; Mon, 7 Apr 2025 09:16:26 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-05.internal (MEProxy); Mon, 07 Apr 2025 09:16:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031786; x=1744118186; bh=T1xrQEvCmA3pvSu+/Dj7GAutcWCh7MZvFpACgM0P9II=; b= AjBI2Bdoi1Xff4jF+xmGgPxxClfk8VEOIZj2MKCF53wqarkEWvXok2JiltdlLdFe UpIIuqoAiEfIfYpKSO86+wI/l9qQuMgqX9Ei3i/h1veQzGP1I5ZOHBtOmAibivMw FPy8xU8Zv91+Bg2dYaJGhgdLncrwycRqTiA8XJIbFQZjFoagBPXJb6XHsfuLW59a GFKWzMsIyplECDi41sA3PT8CkZXeflkUkueY/UKbqVYPzus11Y7T1S8vt+9Am17t k/pIr2Zgf9/+jS4eqhnhx6NND0ZA1opY96umaU6mYj6IZ4ued9x77UKcs1emfJjD IYVuNgvP4Z+YumAj8HlvpA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031786; x= 1744118186; bh=T1xrQEvCmA3pvSu+/Dj7GAutcWCh7MZvFpACgM0P9II=; b=b bWUrkxEBf2G26f6V9WiJEiYxYNDAFFG5idxHTqwRHVrdESGpqvIqArqnx/z2kvJJ 5tn2FptO5NoiELU/w8amRGk5E6tKdNsga1cqacyFO+jkmA7Cw6D4ZLJ5LgpZZ5Ah k0swEPyULhQmAG5a2IRrpy/eEzt6bR7dXY2Tfma4fOrnavLl5tyi/EKJRLbagQQL U0n5F3ns4GFH0P0VCBHn69NU7i4J5Q5dZy/ZYdjxkzWG6dkErJ7thSls10VA0bve jREMbsHu4XKFsJSjJnG+WA8+A2zdaCtA6NMvox93JyM0VZGSZfQsqhg1jGP8DuRm sh10u4m0TJcEfAEQ1pm0Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdelucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteek udehjeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehkrghrthhhihhkrddukeeksehgmhgrihhlrd gtohhmpdhrtghpthhtohepjhhlthhosghlvghrsehgmhgrihhlrdgtohhmpdhrtghpthht ohepghhithhsthgvrhesphhosghogidrtghomhdprhgtphhtthhopehgihhtsehvghgvrh drkhgvrhhnvghlrdhorhhg X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:25 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 1752124a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:24 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:16 +0200 Subject: [PATCH v2 04/16] reftable/block: simplify how we track restart points Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-4-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 Restart points record the location of reftable records that do not use prefix compression and are used to perform a binary search inside of a block. These restart points are encoded at the end of a block, between the record data and the footer of a table. The block structure contains three different variables related to these restart points: - The block length contains the length of the reftable block up to the restart points. - The restart count contains the number of restart points contained in the block. - The restart bytes variable tracks where the restart point data begins. Tracking all three of these variables is unnecessary though as the data can be derived from one another: the block length without restart points is the exact same as the offset of the restart count data, which we already track via the `restart_bytes` data. Refactor the code so that we track the location of restart bytes not as a pointer, but instead as an offset. This allows us to trivially get rid of the `block_len` variable as described above. This avoids having the confusing `block_len` variable and allows us to do less bookkeeping overall. Signed-off-by: Patrick Steinhardt --- reftable/block.c | 25 ++++++++++++------------- reftable/block.h | 8 +++++--- reftable/table.c | 2 +- 3 files changed, 18 insertions(+), 17 deletions(-) diff --git a/reftable/block.c b/reftable/block.c index 97740187259..f2567a8f0fd 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -216,10 +216,9 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block, uint32_t full_block_size = table_block_size; uint8_t typ = block->data[header_off]; uint32_t sz = reftable_get_be24(block->data + header_off + 1); - int err = 0; - uint16_t restart_count = 0; - uint32_t restart_start = 0; - uint8_t *restart_bytes = NULL; + uint16_t restart_count; + uint32_t restart_off; + int err; block_source_return_block(&br->block); @@ -300,8 +299,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block, } restart_count = reftable_get_be16(block->data + sz - 2); - restart_start = sz - 2 - 3 * restart_count; - restart_bytes = block->data + restart_start; + restart_off = sz - 2 - 3 * restart_count; /* transfer ownership. */ br->block = *block; @@ -309,11 +307,12 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block, block->len = 0; br->hash_size = hash_size; - br->block_len = restart_start; + br->restart_off = restart_off; br->full_block_size = full_block_size; br->header_off = header_off; br->restart_count = restart_count; - br->restart_bytes = restart_bytes; + + err = 0; done: return err; @@ -337,7 +336,7 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k int off = br->header_off + 4, n; struct string_view in = { .buf = br->block.data + off, - .len = br->block_len - off, + .len = br->restart_off - off, }; uint8_t extra = 0; @@ -354,13 +353,13 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k static uint32_t block_reader_restart_offset(const struct block_reader *br, size_t idx) { - return reftable_get_be24(br->restart_bytes + 3 * idx); + return reftable_get_be24(br->block.data + br->restart_off + 3 * idx); } void block_iter_seek_start(struct block_iter *it, const struct block_reader *br) { it->block = br->block.data; - it->block_len = br->block_len; + it->block_len = br->restart_off; it->hash_size = br->hash_size; reftable_buf_reset(&it->last_key); it->next_off = br->header_off + 4; @@ -378,7 +377,7 @@ static int restart_needle_less(size_t idx, void *_args) uint32_t off = block_reader_restart_offset(args->reader, idx); struct string_view in = { .buf = args->reader->block.data + off, - .len = args->reader->block_len - off, + .len = args->reader->restart_off - off, }; uint64_t prefix_len, suffix_len; uint8_t extra; @@ -505,7 +504,7 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br, else it->next_off = br->header_off + 4; it->block = br->block.data; - it->block_len = br->block_len; + it->block_len = br->restart_off; it->hash_size = br->hash_size; err = reftable_record_init(&rec, block_reader_type(br)); diff --git a/reftable/block.h b/reftable/block.h index 203b07d9a44..b78f322e646 100644 --- a/reftable/block.h +++ b/reftable/block.h @@ -79,10 +79,12 @@ struct block_reader { unsigned char *uncompressed_data; size_t uncompressed_cap; - /* size of the data, excluding restart data. */ - uint32_t block_len; - uint8_t *restart_bytes; + /* + * Restart point data. Restart points are located after the block's + * record data. + */ uint16_t restart_count; + uint32_t restart_off; /* size of the data in the file. For log blocks, this is the compressed * size. */ diff --git a/reftable/table.c b/reftable/table.c index d18e17b0d44..ec84545707c 100644 --- a/reftable/table.c +++ b/reftable/table.c @@ -838,7 +838,7 @@ int reftable_table_print_blocks(const char *tablename) printf("%s:\n", sections[i].name); while (1) { - printf(" - length: %u\n", ti.br.block_len); + printf(" - length: %u\n", ti.br.restart_off); printf(" restarts: %u\n", ti.br.restart_count); err = table_iter_next_block(&ti); From patchwork Mon Apr 7 13:16:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040504 Received: from fout-a1-smtp.messagingengine.com (fout-a1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A86F253F14 for ; Mon, 7 Apr 2025 13:16:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031791; cv=none; b=I0g4jE6hbgMUqUTzvToLnqNEadjg3ifAZSant/SgvetV8NkL9sRarxVfZym0tqlAQHgN37stWn+PB26eFTRvkAfU0dDP/uPY6SiolVvmRifauL1LrxXjGkkNaLup2qxmaF8N7uT5ld1eJRBB7xx3hcCK5X4knSvyPX/vZDFYc24= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031791; c=relaxed/simple; bh=R4eJ+5MgZWRjklULR+2jKqkoW0093oP4ZS/hCQgydcM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=chr9H9tXETfeua+kdhEOCI/JygNejkHIasw3ip8EsB1tWZxaFKSaP91RDcs+SC6vJJIq/0jLg9HT0ff3J9Moyl+1Et7c7mHPCw6ykC0qJNEqRjcaWf96DZE5kFi6NBNaJ+44hUIcOSVi8uvRLV3NBsi61HAH1u1x4VUiGbIXndE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=n0mJgUUn; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=frkuakgE; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="n0mJgUUn"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="frkuakgE" Received: from phl-compute-10.internal (phl-compute-10.phl.internal [10.202.2.50]) by mailfout.phl.internal (Postfix) with ESMTP id 7B0AB1380345; Mon, 7 Apr 2025 09:16:27 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-10.internal (MEProxy); Mon, 07 Apr 2025 09:16:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031787; x=1744118187; bh=ijaAP/+yRVFoMgyN09Wfbpy6m6vGiQkOisFcpDZaSVU=; b= n0mJgUUnjkR8wq+4OCsxodUk5NB/Oi21kFq1zTyCDX0bJ1eljrp/ZNc82tr1Y447 QNmBrMAytiqcgHlMVK2Yy48Z7GXdhzoQQVd1LMsuwDgO+1DSWDqtrXgV9O47F63m 5ULDBF8eHtDWWNo7LL979Fsqz0Ge16iSbyml/76sTM7WBQLlu+NRZG/pjSrzvZ4t lgiyyPEmyePD+P0TqOS+kBcbVpwtwz1t4utWqRjmDGpRBq0wIKpOhmUQK2Q1Xros TR8dEvqPfyHXVFR025aeA1sDCZFoXUVyHf30N0/CA1p6AuX/cyiNhwMB0/nW/5EM 4RaQM2ZOs6CAwxanEMwG8w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031787; x= 1744118187; bh=ijaAP/+yRVFoMgyN09Wfbpy6m6vGiQkOisFcpDZaSVU=; b=f rkuakgEIId6JbZYajXLPFxAaU1RmfnnmjpZ2iYSKIJMuIRWEuwfDtG0WwT/UfYS6 PjGA0kqLBY9x/sc7B0ZMs3xXm9XTKLqNu2sXBMfdSRILDfnGzzUb3LIRwMCRlZ17 +0Iv7JaaFvcQkOYHIw5dHiLIvs5pYEK6UV4keLE6KeoL5pFsR+lJSRAuUDChOxVe Ba3V45r/NxlDP5fBo7GXLUmi0rPo13VIuoYvKQRd39C0tn92kzZ1u2x0wYr96i3C u6swYwZBzgDaDugR2sQj5mec3OoemR4U2ItmK+uJR//zLTeXJmFcExACo4Y6rU0u 6JbIwK35mB4k26NdaWLcw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdelucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteek udehjeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhmpd hrtghpthhtohepkhgrrhhthhhikhdrudekkeesghhmrghilhdrtghomhdprhgtphhtthho pehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepjhhlthhosghlvg hrsehgmhgrihhlrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:26 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 5ee1c0f1 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:25 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:17 +0200 Subject: [PATCH v2 05/16] reftable/table: move reading block into block reader Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-5-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 The logic to read blocks from a reftable is scattered across both the table and the block subsystems. Besides causing somewhat fuzzy responsibilities, it also means that we have to awkwardly pass around the ownership of blocks between the subsystems. Refactor the code so that we stop passing the block when initializing a reader, but instead by passing in the block source plus the offset at which we're supposed to read a block. Like this, the ownership of the block itself doesn't need to get handed over as the block reader is the one owning the block right from the start. Signed-off-by: Patrick Steinhardt --- reftable/block.c | 87 ++++++++++++++++++++++++++--------------- reftable/block.h | 8 ++-- reftable/table.c | 65 +++--------------------------- t/unit-tests/t-reftable-block.c | 76 ++++++++++++++++++----------------- 4 files changed, 107 insertions(+), 129 deletions(-) diff --git a/reftable/block.c b/reftable/block.c index f2567a8f0fd..2517108b8ef 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -209,31 +209,57 @@ int block_writer_finish(struct block_writer *w) return w->next; } -int block_reader_init(struct block_reader *br, struct reftable_block *block, - uint32_t header_off, uint32_t table_block_size, - uint32_t hash_size) +static int read_block(struct reftable_block_source *source, + struct reftable_block *dest, uint64_t off, + uint32_t sz) { + size_t size = block_source_size(source); + block_source_return_block(dest); + if (off >= size) + return 0; + if (off + sz > size) + sz = size - off; + return block_source_read_block(source, dest, off, sz); +} + +int block_reader_init(struct block_reader *br, + struct reftable_block_source *source, + uint32_t offset, uint32_t header_size, + uint32_t table_block_size, uint32_t hash_size) +{ + uint32_t guess_block_size = table_block_size ? + table_block_size : DEFAULT_BLOCK_SIZE; uint32_t full_block_size = table_block_size; - uint8_t typ = block->data[header_off]; - uint32_t sz = reftable_get_be24(block->data + header_off + 1); uint16_t restart_count; uint32_t restart_off; + uint32_t block_size; + uint8_t block_type; int err; - block_source_return_block(&br->block); + err = read_block(source, &br->block, offset, guess_block_size); + if (err < 0) + goto done; - if (!reftable_is_block_type(typ)) { - err = REFTABLE_FORMAT_ERROR; + block_type = br->block.data[header_size]; + if (!reftable_is_block_type(block_type)) { + err = REFTABLE_FORMAT_ERROR; goto done; } - if (typ == BLOCK_TYPE_LOG) { - uint32_t block_header_skip = 4 + header_off; - uLong dst_len = sz - block_header_skip; - uLong src_len = block->len - block_header_skip; + block_size = reftable_get_be24(br->block.data + header_size + 1); + if (block_size > guess_block_size) { + err = read_block(source, &br->block, offset, block_size); + if (err < 0) + goto done; + } + + if (block_type == BLOCK_TYPE_LOG) { + uint32_t block_header_skip = 4 + header_size; + uLong dst_len = block_size - block_header_skip; + uLong src_len = br->block.len - block_header_skip; /* Log blocks specify the *uncompressed* size in their header. */ - REFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, sz, + REFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, block_size, br->uncompressed_cap); if (!br->uncompressed_data) { err = REFTABLE_OUT_OF_MEMORY_ERROR; @@ -241,7 +267,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block, } /* Copy over the block header verbatim. It's not compressed. */ - memcpy(br->uncompressed_data, block->data, block_header_skip); + memcpy(br->uncompressed_data, br->block.data, block_header_skip); if (!br->zstream) { REFTABLE_CALLOC_ARRAY(br->zstream, 1); @@ -259,7 +285,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block, goto done; } - br->zstream->next_in = block->data + block_header_skip; + br->zstream->next_in = br->block.data + block_header_skip; br->zstream->avail_in = src_len; br->zstream->next_out = br->uncompressed_data + block_header_skip; br->zstream->avail_out = dst_len; @@ -278,43 +304,41 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block, } err = 0; - if (br->zstream->total_out + block_header_skip != sz) { + if (br->zstream->total_out + block_header_skip != block_size) { err = REFTABLE_FORMAT_ERROR; goto done; } /* We're done with the input data. */ - block_source_return_block(block); - block->data = br->uncompressed_data; - block->len = sz; + block_source_return_block(&br->block); + br->block.data = br->uncompressed_data; + br->block.len = block_size; full_block_size = src_len + block_header_skip - br->zstream->avail_in; } else if (full_block_size == 0) { - full_block_size = sz; - } else if (sz < full_block_size && sz < block->len && - block->data[sz] != 0) { + full_block_size = block_size; + } else if (block_size < full_block_size && block_size < br->block.len && + br->block.data[block_size] != 0) { /* If the block is smaller than the full block size, it is padded (data followed by '\0') or the next block is unaligned. */ - full_block_size = sz; + full_block_size = block_size; } - restart_count = reftable_get_be16(block->data + sz - 2); - restart_off = sz - 2 - 3 * restart_count; - - /* transfer ownership. */ - br->block = *block; - block->data = NULL; - block->len = 0; + restart_count = reftable_get_be16(br->block.data + block_size - 2); + restart_off = block_size - 2 - 3 * restart_count; + br->block_type = block_type; br->hash_size = hash_size; br->restart_off = restart_off; br->full_block_size = full_block_size; - br->header_off = header_off; + br->header_off = header_size; br->restart_count = restart_count; err = 0; done: + if (err < 0) + block_reader_release(br); return err; } @@ -324,6 +348,7 @@ void block_reader_release(struct block_reader *br) reftable_free(br->zstream); reftable_free(br->uncompressed_data); block_source_return_block(&br->block); + memset(br, 0, sizeof(*br)); } uint8_t block_reader_type(const struct block_reader *r) diff --git a/reftable/block.h b/reftable/block.h index b78f322e646..6afb1b2952a 100644 --- a/reftable/block.h +++ b/reftable/block.h @@ -89,12 +89,14 @@ struct block_reader { /* size of the data in the file. For log blocks, this is the compressed * size. */ uint32_t full_block_size; + uint8_t block_type; }; /* initializes a block reader. */ -int block_reader_init(struct block_reader *br, struct reftable_block *bl, - uint32_t header_off, uint32_t table_block_size, - uint32_t hash_size); +int block_reader_init(struct block_reader *br, + struct reftable_block_source *source, + uint32_t offset, uint32_t header_size, + uint32_t table_block_size, uint32_t hash_size); void block_reader_release(struct block_reader *br); diff --git a/reftable/table.c b/reftable/table.c index ec84545707c..7c0f1c9e6e3 100644 --- a/reftable/table.c +++ b/reftable/table.c @@ -30,23 +30,6 @@ table_offsets_for(struct reftable_table *t, uint8_t typ) abort(); } -static int table_get_block(struct reftable_table *t, - struct reftable_block *dest, uint64_t off, - uint32_t sz) -{ - ssize_t bytes_read; - if (off >= t->size) - return 0; - if (off + sz > t->size) - sz = t->size - off; - - bytes_read = block_source_read_block(&t->source, dest, off, sz); - if (bytes_read < 0) - return (int)bytes_read; - - return 0; -} - enum reftable_hash reftable_table_hash_id(struct reftable_table *t) { return t->hash_id; @@ -180,64 +163,28 @@ static void table_iter_block_done(struct table_iter *ti) block_iter_reset(&ti->bi); } -static int32_t extract_block_size(uint8_t *data, uint8_t *typ, uint64_t off, - int version) -{ - int32_t result = 0; - - if (off == 0) { - data += header_size(version); - } - - *typ = data[0]; - if (reftable_is_block_type(*typ)) { - result = reftable_get_be24(data + 1); - } - return result; -} - int table_init_block_reader(struct reftable_table *t, struct block_reader *br, uint64_t next_off, uint8_t want_typ) { - int32_t guess_block_size = t->block_size ? t->block_size : - DEFAULT_BLOCK_SIZE; - struct reftable_block block = { NULL }; - uint8_t block_typ = 0; - int err = 0; uint32_t header_off = next_off ? 0 : header_size(t->version); - int32_t block_size = 0; + int err; if (next_off >= t->size) return 1; - err = table_get_block(t, &block, next_off, guess_block_size); + err = block_reader_init(br, &t->source, next_off, header_off, + t->block_size, hash_size(t->hash_id)); if (err < 0) goto done; - block_size = extract_block_size(block.data, &block_typ, next_off, - t->version); - if (block_size < 0) { - err = block_size; - goto done; - } - if (want_typ != BLOCK_TYPE_ANY && block_typ != want_typ) { + if (want_typ != BLOCK_TYPE_ANY && br->block_type != want_typ) { err = 1; goto done; } - if (block_size > guess_block_size) { - block_source_return_block(&block); - err = table_get_block(t, &block, next_off, block_size); - if (err < 0) { - goto done; - } - } - - err = block_reader_init(br, &block, header_off, t->block_size, - hash_size(t->hash_id)); done: - block_source_return_block(&block); - + if (err) + block_reader_release(br); return err; } diff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c index 8bb40482347..dbec6a5d0d0 100644 --- a/t/unit-tests/t-reftable-block.c +++ b/t/unit-tests/t-reftable-block.c @@ -19,7 +19,7 @@ static void t_ref_block_read_write(void) struct reftable_record recs[30]; const size_t N = ARRAY_SIZE(recs); const size_t block_size = 1024; - struct reftable_block block = { 0 }; + struct reftable_block_source source = { 0 }; struct block_writer bw = { .last_key = REFTABLE_BUF_INIT, }; @@ -30,13 +30,14 @@ static void t_ref_block_read_write(void) int ret; struct block_reader br = { 0 }; struct block_iter it = BLOCK_ITER_INIT; - struct reftable_buf want = REFTABLE_BUF_INIT, buf = REFTABLE_BUF_INIT; + struct reftable_buf want = REFTABLE_BUF_INIT; + struct reftable_buf block = REFTABLE_BUF_INIT; - REFTABLE_CALLOC_ARRAY(block.data, block_size); - check(block.data != NULL); + REFTABLE_CALLOC_ARRAY(block.buf, block_size); + check(block.buf != NULL); block.len = block_size; - block_source_from_buf(&block.source ,&buf); - ret = block_writer_init(&bw, BLOCK_TYPE_REF, block.data, block_size, + + ret = block_writer_init(&bw, BLOCK_TYPE_REF, (uint8_t *) block.buf, block_size, header_off, hash_size(REFTABLE_HASH_SHA1)); check(!ret); @@ -62,7 +63,8 @@ static void t_ref_block_read_write(void) block_writer_release(&bw); - block_reader_init(&br, &block, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); + block_source_from_buf(&source ,&block); + block_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); block_iter_seek_start(&it, &br); @@ -100,9 +102,8 @@ static void t_ref_block_read_write(void) block_reader_release(&br); block_iter_close(&it); reftable_record_release(&rec); - block_source_return_block(&br.block); reftable_buf_release(&want); - reftable_buf_release(&buf); + reftable_buf_release(&block); for (i = 0; i < N; i++) reftable_record_release(&recs[i]); } @@ -113,7 +114,7 @@ static void t_log_block_read_write(void) struct reftable_record recs[30]; const size_t N = ARRAY_SIZE(recs); const size_t block_size = 2048; - struct reftable_block block = { 0 }; + struct reftable_block_source source = { 0 }; struct block_writer bw = { .last_key = REFTABLE_BUF_INIT, }; @@ -124,13 +125,14 @@ static void t_log_block_read_write(void) int ret; struct block_reader br = { 0 }; struct block_iter it = BLOCK_ITER_INIT; - struct reftable_buf want = REFTABLE_BUF_INIT, buf = REFTABLE_BUF_INIT; + struct reftable_buf want = REFTABLE_BUF_INIT; + struct reftable_buf block = REFTABLE_BUF_INIT; - REFTABLE_CALLOC_ARRAY(block.data, block_size); - check(block.data != NULL); + REFTABLE_CALLOC_ARRAY(block.buf, block_size); + check(block.buf != NULL); block.len = block_size; - block_source_from_buf(&block.source ,&buf); - ret = block_writer_init(&bw, BLOCK_TYPE_LOG, block.data, block_size, + + ret = block_writer_init(&bw, BLOCK_TYPE_LOG, (uint8_t *) block.buf, block_size, header_off, hash_size(REFTABLE_HASH_SHA1)); check(!ret); @@ -151,7 +153,8 @@ static void t_log_block_read_write(void) block_writer_release(&bw); - block_reader_init(&br, &block, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); + block_source_from_buf(&source, &block); + block_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); block_iter_seek_start(&it, &br); @@ -190,9 +193,8 @@ static void t_log_block_read_write(void) block_reader_release(&br); block_iter_close(&it); reftable_record_release(&rec); - block_source_return_block(&br.block); reftable_buf_release(&want); - reftable_buf_release(&buf); + reftable_buf_release(&block); for (i = 0; i < N; i++) reftable_record_release(&recs[i]); } @@ -203,7 +205,7 @@ static void t_obj_block_read_write(void) struct reftable_record recs[30]; const size_t N = ARRAY_SIZE(recs); const size_t block_size = 1024; - struct reftable_block block = { 0 }; + struct reftable_block_source source = { 0 }; struct block_writer bw = { .last_key = REFTABLE_BUF_INIT, }; @@ -214,13 +216,14 @@ static void t_obj_block_read_write(void) int ret; struct block_reader br = { 0 }; struct block_iter it = BLOCK_ITER_INIT; - struct reftable_buf want = REFTABLE_BUF_INIT, buf = REFTABLE_BUF_INIT; + struct reftable_buf want = REFTABLE_BUF_INIT; + struct reftable_buf block = REFTABLE_BUF_INIT; - REFTABLE_CALLOC_ARRAY(block.data, block_size); - check(block.data != NULL); + REFTABLE_CALLOC_ARRAY(block.buf, block_size); + check(block.buf != NULL); block.len = block_size; - block_source_from_buf(&block.source, &buf); - ret = block_writer_init(&bw, BLOCK_TYPE_OBJ, block.data, block_size, + + ret = block_writer_init(&bw, BLOCK_TYPE_OBJ, (uint8_t *) block.buf, block_size, header_off, hash_size(REFTABLE_HASH_SHA1)); check(!ret); @@ -243,7 +246,8 @@ static void t_obj_block_read_write(void) block_writer_release(&bw); - block_reader_init(&br, &block, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); + block_source_from_buf(&source, &block); + block_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); block_iter_seek_start(&it, &br); @@ -273,9 +277,8 @@ static void t_obj_block_read_write(void) block_reader_release(&br); block_iter_close(&it); reftable_record_release(&rec); - block_source_return_block(&br.block); reftable_buf_release(&want); - reftable_buf_release(&buf); + reftable_buf_release(&block); for (i = 0; i < N; i++) reftable_record_release(&recs[i]); } @@ -286,7 +289,7 @@ static void t_index_block_read_write(void) struct reftable_record recs[30]; const size_t N = ARRAY_SIZE(recs); const size_t block_size = 1024; - struct reftable_block block = { 0 }; + struct reftable_block_source source = { 0 }; struct block_writer bw = { .last_key = REFTABLE_BUF_INIT, }; @@ -298,13 +301,14 @@ static void t_index_block_read_write(void) int ret; struct block_reader br = { 0 }; struct block_iter it = BLOCK_ITER_INIT; - struct reftable_buf want = REFTABLE_BUF_INIT, buf = REFTABLE_BUF_INIT; + struct reftable_buf want = REFTABLE_BUF_INIT; + struct reftable_buf block = REFTABLE_BUF_INIT; - REFTABLE_CALLOC_ARRAY(block.data, block_size); - check(block.data != NULL); + REFTABLE_CALLOC_ARRAY(block.buf, block_size); + check(block.buf != NULL); block.len = block_size; - block_source_from_buf(&block.source, &buf); - ret = block_writer_init(&bw, BLOCK_TYPE_INDEX, block.data, block_size, + + ret = block_writer_init(&bw, BLOCK_TYPE_INDEX, (uint8_t *) block.buf, block_size, header_off, hash_size(REFTABLE_HASH_SHA1)); check(!ret); @@ -327,7 +331,8 @@ static void t_index_block_read_write(void) block_writer_release(&bw); - block_reader_init(&br, &block, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); + block_source_from_buf(&source, &block); + block_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); block_iter_seek_start(&it, &br); @@ -365,9 +370,8 @@ static void t_index_block_read_write(void) block_reader_release(&br); block_iter_close(&it); reftable_record_release(&rec); - block_source_return_block(&br.block); reftable_buf_release(&want); - reftable_buf_release(&buf); + reftable_buf_release(&block); for (i = 0; i < N; i++) reftable_record_release(&recs[i]); } From patchwork Mon Apr 7 13:16:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040502 Received: from fout-a1-smtp.messagingengine.com (fout-a1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95360253F15 for ; Mon, 7 Apr 2025 13:16:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031790; cv=none; b=dKmQjbiNexB79p48pRlmBtOzOEpCOFVBQoKEoCZfvP1gsypN+cUOzYivnMJT/IxsD7Iw0ULWWJdc9kY10Vx0ynG6jR1Mgc+lXZD5R4ewTED1dqZSRutrTM3j5ayHgDN3vjFQcc2sAvhIKVSGiFFkgNLvIEjhVWMua90b6lYofvQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031790; c=relaxed/simple; bh=rgOGgvWFYy69jPgXLtyh00EttxvyvT9oQbFrByOa5as=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=t3D6wJ5nw39BpQlDVdpFoI6kXVbGWxEi9jklaaJOse0mOSsMQf3s81fNBjDgON207uYcR+nKDYFRDXgAAnwaHoOBiVedmdNls/ZAuFy1DxjXOr+Nwta3axt+R7JCa0DaMYObuC7He2pYfOpUHXHZA0YOvBJ2PBhDaO13qraSrsA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=OJ3Fg0bz; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=kcYfTurD; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="OJ3Fg0bz"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="kcYfTurD" Received: from phl-compute-07.internal (phl-compute-07.phl.internal [10.202.2.47]) by mailfout.phl.internal (Postfix) with ESMTP id B38AA1380357; Mon, 7 Apr 2025 09:16:27 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-07.internal (MEProxy); Mon, 07 Apr 2025 09:16:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031787; x=1744118187; bh=7US3qxaqfJEPhsEx422DMdfZ29dQb2BsQZxDFy+1z6Q=; b= OJ3Fg0bzibocU3JXFrVCj3+AzOueT3XFt5Qoi+xVRfL29hUI3gB5FWgvWZUrBydU ScGO0H42bLpr+knQ9t98gtZre6okgIoFQ3TzPNNuyOXb3rGngItU3FVdL53ajKDj 8XrCOr4HMuDNraSKu8vM1zPD4/CaLXkCzArIoEdZv45KUGXhdX84LKX7M7H/Pt27 7DB81dxkQqLTcI+td7/Ks1TSuuMs8SbYTYW8QHtZXyYIfscDIl5DiN5KKJFT1gdh NiO5MRsifyGcrFy2b63IzOAbnbxnUPa596Q12x8N8ZX2vuugNVlICQraiddz8wrY EqhbPy5QcbE+e3D6we6/pg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031787; x= 1744118187; bh=7US3qxaqfJEPhsEx422DMdfZ29dQb2BsQZxDFy+1z6Q=; b=k cYfTurDfSc3ty0QTddCEdDo9eMMgzzJm+dCqxD9FWjRRdAbZ5tAnhUnaGwmU30r+ xdkEd1I12h48N2PyXK/aTxUTvFMzOkV8hgZTzbnKHp7Q0sXviqGAEdCxhzB96eXB yWKnSPjewEQ1+gZU5bXGAHzh2OeN6e87UWgyVt45bb9C3Jm66h1yel7+S6Qog5y8 tabh0LENAD8ljSn/fUeq+oot9zsfcEhEOgJHdBNra2c+hYRRknmxow8dYG5n0Nf/ osS7fXjfL6/YrBoDWtnBZGjCN8hNa5xGHz7T2QB0FjmK568wJXmx7aeOAxd8sOeI IL+P4UbQid+wi0mH5P9gQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdekucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteek udehjeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehjlhhtohgslhgvrhesghhmrghilhdrtghomh dprhgtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhmpdhrtghpthhtohepghhi thesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehkrghrthhhihhkrdduke eksehgmhgrihhlrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:26 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 3a69f448 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:26 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:18 +0200 Subject: [PATCH v2 06/16] reftable/block: rename `block` to `block_data` Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-6-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 The `reftable_block` structure associates a byte slice with a block source. As such it only holds the data of a reftable block without actually encoding any of the details for how to access that data. Rename the structure to instead be called `reftable_block_data`. Besides clarifying that this really only holds data, it also allows us to rename the `reftable_block_reader` to `reftable_block` in the next commit, as this is the structure that actually encapsulates access to the reftable blocks. Rename the `struct reftable_block_reader::block` member accordingly. Signed-off-by: Patrick Steinhardt --- reftable/block.c | 46 ++++++++++++++++++------------------- reftable/block.h | 2 +- reftable/blocksource.c | 44 +++++++++++++++++------------------ reftable/blocksource.h | 10 ++++---- reftable/iter.c | 4 ++-- reftable/reftable-blocksource.h | 14 +++++------ reftable/table.c | 16 ++++++------- t/unit-tests/t-reftable-readwrite.c | 10 ++++---- 8 files changed, 73 insertions(+), 73 deletions(-) diff --git a/reftable/block.c b/reftable/block.c index 2517108b8ef..dca30b8527f 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -210,16 +210,16 @@ int block_writer_finish(struct block_writer *w) } static int read_block(struct reftable_block_source *source, - struct reftable_block *dest, uint64_t off, + struct reftable_block_data *dest, uint64_t off, uint32_t sz) { size_t size = block_source_size(source); - block_source_return_block(dest); + block_source_release_data(dest); if (off >= size) return 0; if (off + sz > size) sz = size - off; - return block_source_read_block(source, dest, off, sz); + return block_source_read_data(source, dest, off, sz); } int block_reader_init(struct block_reader *br, @@ -236,19 +236,19 @@ int block_reader_init(struct block_reader *br, uint8_t block_type; int err; - err = read_block(source, &br->block, offset, guess_block_size); + err = read_block(source, &br->block_data, offset, guess_block_size); if (err < 0) goto done; - block_type = br->block.data[header_size]; + block_type = br->block_data.data[header_size]; if (!reftable_is_block_type(block_type)) { err = REFTABLE_FORMAT_ERROR; goto done; } - block_size = reftable_get_be24(br->block.data + header_size + 1); + block_size = reftable_get_be24(br->block_data.data + header_size + 1); if (block_size > guess_block_size) { - err = read_block(source, &br->block, offset, block_size); + err = read_block(source, &br->block_data, offset, block_size); if (err < 0) goto done; } @@ -256,7 +256,7 @@ int block_reader_init(struct block_reader *br, if (block_type == BLOCK_TYPE_LOG) { uint32_t block_header_skip = 4 + header_size; uLong dst_len = block_size - block_header_skip; - uLong src_len = br->block.len - block_header_skip; + uLong src_len = br->block_data.len - block_header_skip; /* Log blocks specify the *uncompressed* size in their header. */ REFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, block_size, @@ -267,7 +267,7 @@ int block_reader_init(struct block_reader *br, } /* Copy over the block header verbatim. It's not compressed. */ - memcpy(br->uncompressed_data, br->block.data, block_header_skip); + memcpy(br->uncompressed_data, br->block_data.data, block_header_skip); if (!br->zstream) { REFTABLE_CALLOC_ARRAY(br->zstream, 1); @@ -285,7 +285,7 @@ int block_reader_init(struct block_reader *br, goto done; } - br->zstream->next_in = br->block.data + block_header_skip; + br->zstream->next_in = br->block_data.data + block_header_skip; br->zstream->avail_in = src_len; br->zstream->next_out = br->uncompressed_data + block_header_skip; br->zstream->avail_out = dst_len; @@ -310,21 +310,21 @@ int block_reader_init(struct block_reader *br, } /* We're done with the input data. */ - block_source_return_block(&br->block); - br->block.data = br->uncompressed_data; - br->block.len = block_size; + block_source_release_data(&br->block_data); + br->block_data.data = br->uncompressed_data; + br->block_data.len = block_size; full_block_size = src_len + block_header_skip - br->zstream->avail_in; } else if (full_block_size == 0) { full_block_size = block_size; - } else if (block_size < full_block_size && block_size < br->block.len && - br->block.data[block_size] != 0) { + } else if (block_size < full_block_size && block_size < br->block_data.len && + br->block_data.data[block_size] != 0) { /* If the block is smaller than the full block size, it is padded (data followed by '\0') or the next block is unaligned. */ full_block_size = block_size; } - restart_count = reftable_get_be16(br->block.data + block_size - 2); + restart_count = reftable_get_be16(br->block_data.data + block_size - 2); restart_off = block_size - 2 - 3 * restart_count; br->block_type = block_type; @@ -347,20 +347,20 @@ void block_reader_release(struct block_reader *br) inflateEnd(br->zstream); reftable_free(br->zstream); reftable_free(br->uncompressed_data); - block_source_return_block(&br->block); + block_source_release_data(&br->block_data); memset(br, 0, sizeof(*br)); } uint8_t block_reader_type(const struct block_reader *r) { - return r->block.data[r->header_off]; + return r->block_data.data[r->header_off]; } int block_reader_first_key(const struct block_reader *br, struct reftable_buf *key) { int off = br->header_off + 4, n; struct string_view in = { - .buf = br->block.data + off, + .buf = br->block_data.data + off, .len = br->restart_off - off, }; uint8_t extra = 0; @@ -378,12 +378,12 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k static uint32_t block_reader_restart_offset(const struct block_reader *br, size_t idx) { - return reftable_get_be24(br->block.data + br->restart_off + 3 * idx); + return reftable_get_be24(br->block_data.data + br->restart_off + 3 * idx); } void block_iter_seek_start(struct block_iter *it, const struct block_reader *br) { - it->block = br->block.data; + it->block = br->block_data.data; it->block_len = br->restart_off; it->hash_size = br->hash_size; reftable_buf_reset(&it->last_key); @@ -401,7 +401,7 @@ static int restart_needle_less(size_t idx, void *_args) struct restart_needle_less_args *args = _args; uint32_t off = block_reader_restart_offset(args->reader, idx); struct string_view in = { - .buf = args->reader->block.data + off, + .buf = args->reader->block_data.data + off, .len = args->reader->restart_off - off, }; uint64_t prefix_len, suffix_len; @@ -528,7 +528,7 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br, it->next_off = block_reader_restart_offset(br, i - 1); else it->next_off = br->header_off + 4; - it->block = br->block.data; + it->block = br->block_data.data; it->block_len = br->restart_off; it->hash_size = br->hash_size; diff --git a/reftable/block.h b/reftable/block.h index 6afb1b2952a..948b79e4614 100644 --- a/reftable/block.h +++ b/reftable/block.h @@ -71,7 +71,7 @@ struct block_reader { uint32_t header_off; /* the memory block */ - struct reftable_block block; + struct reftable_block_data block_data; uint32_t hash_size; /* Uncompressed data for log entries. */ diff --git a/reftable/blocksource.c b/reftable/blocksource.c index bc785506fb1..573c81287fe 100644 --- a/reftable/blocksource.c +++ b/reftable/blocksource.c @@ -13,15 +13,15 @@ #include "reftable-blocksource.h" #include "reftable-error.h" -void block_source_return_block(struct reftable_block *block) +void block_source_release_data(struct reftable_block_data *data) { - struct reftable_block_source source = block->source; - if (block && source.ops) - source.ops->return_block(source.arg, block); - block->data = NULL; - block->len = 0; - block->source.ops = NULL; - block->source.arg = NULL; + struct reftable_block_source source = data->source; + if (data && source.ops) + source.ops->release_data(source.arg, data); + data->data = NULL; + data->len = 0; + data->source.ops = NULL; + data->source.arg = NULL; } void block_source_close(struct reftable_block_source *source) @@ -34,11 +34,11 @@ void block_source_close(struct reftable_block_source *source) source->ops = NULL; } -ssize_t block_source_read_block(struct reftable_block_source *source, - struct reftable_block *dest, uint64_t off, - uint32_t size) +ssize_t block_source_read_data(struct reftable_block_source *source, + struct reftable_block_data *dest, uint64_t off, + uint32_t size) { - ssize_t result = source->ops->read_block(source->arg, dest, off, size); + ssize_t result = source->ops->read_data(source->arg, dest, off, size); dest->source = *source; return result; } @@ -48,7 +48,7 @@ uint64_t block_source_size(struct reftable_block_source *source) return source->ops->size(source->arg); } -static void reftable_buf_return_block(void *b REFTABLE_UNUSED, struct reftable_block *dest) +static void reftable_buf_release_data(void *b REFTABLE_UNUSED, struct reftable_block_data *dest) { if (dest->len) memset(dest->data, 0xff, dest->len); @@ -59,8 +59,8 @@ static void reftable_buf_close(void *b REFTABLE_UNUSED) { } -static ssize_t reftable_buf_read_block(void *v, struct reftable_block *dest, - uint64_t off, uint32_t size) +static ssize_t reftable_buf_read_data(void *v, struct reftable_block_data *dest, + uint64_t off, uint32_t size) { struct reftable_buf *b = v; assert(off + size <= b->len); @@ -79,8 +79,8 @@ static uint64_t reftable_buf_size(void *b) static struct reftable_block_source_vtable reftable_buf_vtable = { .size = &reftable_buf_size, - .read_block = &reftable_buf_read_block, - .return_block = &reftable_buf_return_block, + .read_data = &reftable_buf_read_data, + .release_data = &reftable_buf_release_data, .close = &reftable_buf_close, }; @@ -102,7 +102,7 @@ static uint64_t file_size(void *b) return ((struct file_block_source *)b)->size; } -static void file_return_block(void *b REFTABLE_UNUSED, struct reftable_block *dest REFTABLE_UNUSED) +static void file_release_data(void *b REFTABLE_UNUSED, struct reftable_block_data *dest REFTABLE_UNUSED) { } @@ -113,8 +113,8 @@ static void file_close(void *v) reftable_free(b); } -static ssize_t file_read_block(void *v, struct reftable_block *dest, uint64_t off, - uint32_t size) +static ssize_t file_read_data(void *v, struct reftable_block_data *dest, uint64_t off, + uint32_t size) { struct file_block_source *b = v; assert(off + size <= b->size); @@ -125,8 +125,8 @@ static ssize_t file_read_block(void *v, struct reftable_block *dest, uint64_t of static struct reftable_block_source_vtable file_vtable = { .size = &file_size, - .read_block = &file_read_block, - .return_block = &file_return_block, + .read_data = &file_read_data, + .release_data = &file_release_data, .close = &file_close, }; diff --git a/reftable/blocksource.h b/reftable/blocksource.h index 639b9a1a3c5..a110e059580 100644 --- a/reftable/blocksource.h +++ b/reftable/blocksource.h @@ -12,7 +12,7 @@ #include "system.h" struct reftable_block_source; -struct reftable_block; +struct reftable_block_data; struct reftable_buf; /* @@ -24,9 +24,9 @@ void block_source_close(struct reftable_block_source *source); /* * Read a block of length `size` from the source at the given `off`. */ -ssize_t block_source_read_block(struct reftable_block_source *source, - struct reftable_block *dest, uint64_t off, - uint32_t size); +ssize_t block_source_read_data(struct reftable_block_source *source, + struct reftable_block_data *dest, uint64_t off, + uint32_t size); /* * Return the total length of the underlying resource. @@ -37,7 +37,7 @@ uint64_t block_source_size(struct reftable_block_source *source); * Return a block to its original source, releasing any resources associated * with it. */ -void block_source_return_block(struct reftable_block *block); +void block_source_release_data(struct reftable_block_data *data); /* Create an in-memory block source for reading reftables. */ void block_source_from_buf(struct reftable_block_source *bs, diff --git a/reftable/iter.c b/reftable/iter.c index 6af6eb49396..c00ffc88577 100644 --- a/reftable/iter.c +++ b/reftable/iter.c @@ -114,7 +114,7 @@ static void indexed_table_ref_iter_close(void *p) { struct indexed_table_ref_iter *it = p; block_iter_close(&it->cur); - block_source_return_block(&it->block_reader.block); + block_source_release_data(&it->block_reader.block_data); reftable_free(it->offsets); reftable_buf_release(&it->oid); } @@ -128,7 +128,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it) return 1; } - block_source_return_block(&it->block_reader.block); + block_source_release_data(&it->block_reader.block_data); off = it->offsets[it->offset_idx++]; err = table_init_block_reader(it->table, &it->block_reader, off, diff --git a/reftable/reftable-blocksource.h b/reftable/reftable-blocksource.h index 96430b629e4..f5ba867bd60 100644 --- a/reftable/reftable-blocksource.h +++ b/reftable/reftable-blocksource.h @@ -21,7 +21,7 @@ struct reftable_block_source { /* a contiguous segment of bytes. It keeps track of its generating block_source * so it can return itself into the pool. */ -struct reftable_block { +struct reftable_block_data { uint8_t *data; size_t len; struct reftable_block_source source; @@ -29,20 +29,20 @@ struct reftable_block { /* block_source_vtable are the operations that make up block_source */ struct reftable_block_source_vtable { - /* returns the size of a block source */ + /* Returns the size of a block source. */ uint64_t (*size)(void *source); /* * Reads a segment from the block source. It is an error to read beyond * the end of the block. */ - ssize_t (*read_block)(void *source, struct reftable_block *dest, - uint64_t off, uint32_t size); + ssize_t (*read_data)(void *source, struct reftable_block_data *dest, + uint64_t off, uint32_t size); - /* mark the block as read; may return the data back to malloc */ - void (*return_block)(void *source, struct reftable_block *blockp); + /* Mark the block as read; may release the data. */ + void (*release_data)(void *source, struct reftable_block_data *data); - /* release all resources associated with the block source */ + /* Release all resources associated with the block source. */ void (*close)(void *source); }; diff --git a/reftable/table.c b/reftable/table.c index 7c0f1c9e6e3..c86b1d4c649 100644 --- a/reftable/table.c +++ b/reftable/table.c @@ -320,7 +320,7 @@ static int table_iter_seek_linear(struct table_iter *ti, * as we have more than three blocks we would have an index, so * we would not do a linear search there anymore. */ - memset(&next.br.block, 0, sizeof(next.br.block)); + memset(&next.br.block_data, 0, sizeof(next.br.block_data)); next.br.zstream = NULL; next.br.uncompressed_data = NULL; next.br.uncompressed_cap = 0; @@ -526,8 +526,8 @@ int reftable_table_init_log_iterator(struct reftable_table *t, int reftable_table_new(struct reftable_table **out, struct reftable_block_source *source, char const *name) { - struct reftable_block footer = { 0 }; - struct reftable_block header = { 0 }; + struct reftable_block_data footer = { 0 }; + struct reftable_block_data header = { 0 }; struct reftable_table *t; uint64_t file_size = block_source_size(source); uint32_t read_size; @@ -550,7 +550,7 @@ int reftable_table_new(struct reftable_table **out, goto done; } - bytes_read = block_source_read_block(source, &header, 0, read_size); + bytes_read = block_source_read_data(source, &header, 0, read_size); if (bytes_read < 0 || (size_t)bytes_read != read_size) { err = REFTABLE_IO_ERROR; goto done; @@ -576,8 +576,8 @@ int reftable_table_new(struct reftable_table **out, t->hash_id = 0; t->refcount = 1; - bytes_read = block_source_read_block(source, &footer, t->size, - footer_size(t->version)); + bytes_read = block_source_read_data(source, &footer, t->size, + footer_size(t->version)); if (bytes_read < 0 || (size_t)bytes_read != footer_size(t->version)) { err = REFTABLE_IO_ERROR; goto done; @@ -590,8 +590,8 @@ int reftable_table_new(struct reftable_table **out, *out = t; done: - block_source_return_block(&footer); - block_source_return_block(&header); + block_source_release_data(&footer); + block_source_release_data(&header); if (err) { if (t) reftable_free(t->name); diff --git a/t/unit-tests/t-reftable-readwrite.c b/t/unit-tests/t-reftable-readwrite.c index 3fba888cdaa..4c49129439e 100644 --- a/t/unit-tests/t-reftable-readwrite.c +++ b/t/unit-tests/t-reftable-readwrite.c @@ -23,22 +23,22 @@ static void t_buffer(void) { struct reftable_buf buf = REFTABLE_BUF_INIT; struct reftable_block_source source = { 0 }; - struct reftable_block out = { 0 }; + struct reftable_block_data out = { 0 }; int n; uint8_t in[] = "hello"; check(!reftable_buf_add(&buf, in, sizeof(in))); block_source_from_buf(&source, &buf); check_int(block_source_size(&source), ==, 6); - n = block_source_read_block(&source, &out, 0, sizeof(in)); + n = block_source_read_data(&source, &out, 0, sizeof(in)); check_int(n, ==, sizeof(in)); check(!memcmp(in, out.data, n)); - block_source_return_block(&out); + block_source_release_data(&out); - n = block_source_read_block(&source, &out, 1, 2); + n = block_source_read_data(&source, &out, 1, 2); check_int(n, ==, 2); check(!memcmp(out.data, "el", 2)); - block_source_return_block(&out); + block_source_release_data(&out); block_source_close(&source); reftable_buf_release(&buf); } From patchwork Mon Apr 7 13:16:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040506 Received: from fout-a1-smtp.messagingengine.com (fout-a1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9AE08253F35 for ; Mon, 7 Apr 2025 13:16:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031792; cv=none; b=GnbNGRzMhYZ1jcFw68QvW3ymeuV8ku/cGbNbWV+OMVHe/MVmycO2FB6NL01QCiZgxrL3n2vCn/yenB6VS/TNDhRpuUERnlPJNeyTIkYz5CQTSqPzVdAifD8IxOGgh81FFiWTzmXTGVmLRXvhq43XrxfMoBNOVYL5VlJCUCJWZ0Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031792; c=relaxed/simple; bh=CcmhgODBIe4wvYGMs84xTIC8Si2zeuB7pcKy/yggzcI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=f8uyyWk2Q2ibUyWoCKscHbAfGo/D0qG7kL7h6apaE0/ZdgY3mS8xJsanlhJN+JwC0KCD8pIhzrc2pVZBp92j7u55YT5RM/lay3h38Th5yoHpGWqfgGJBBvsHURjpMHz6gMjDEoXbfzh+ny+7Eljuf3/z0MyyDBZRmQcWlPmLSoU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=oCNkAqeU; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=NHh0yn2s; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="oCNkAqeU"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="NHh0yn2s" Received: from phl-compute-11.internal (phl-compute-11.phl.internal [10.202.2.51]) by mailfout.phl.internal (Postfix) with ESMTP id B45121380228; Mon, 7 Apr 2025 09:16:28 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-11.internal (MEProxy); Mon, 07 Apr 2025 09:16:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031788; x=1744118188; bh=ILCfUbydOmeaODO47s+MHhpLAbhJRXBGoWr6DpnHIhM=; b= oCNkAqeUNa/fj6K2B1gR+1MCRRKO62Gci/dtYN+omAHoQD4R1ir4ynzPNu5HMuqm 4yCI51us2JCDIM/FlM5XWSBhNbo6EWa6ILfQfzIjRYNvvFDswKOAHIWlAzlKOgI0 JZZmEeGM0jTMR2Iny96Iwa7eJiVjUh4G17z8jk6tJO8WV/CWYZvxHLv2p8X3xt7w 4EoG1PX0gvdgra6c5pNOC8PteU41IoT33+x/bWEIpTt75QXAAWjGSdZP+KCovsUE uANgWyvpg5njtLV+VeDFIXKR2g5oYwd8TwHOYCh5a6a6l7DUCwLcVSeoYGnPRqHN M3MuxWramGY3+Y1GmjFf9Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031788; x= 1744118188; bh=ILCfUbydOmeaODO47s+MHhpLAbhJRXBGoWr6DpnHIhM=; b=N Hh0yn2s7wk3L3/2iIdJfh9+ZK5L8UdzErZjAO2neY25ZARVvVCmT1y8ncMKNoqbh jvwUttMFVCWLvOtm+WAJ9zcOc4lOcYAjaIqmPz77DwDtaJcbzzdk67No0P50/+3v EsFCBVI93pTrdXzk4gDAn9etkRPgdz3bWpP6Djs9PjWMjFL4w9ZswEppmZCER40J yjm+BRs4MLAldbIHKNw8A8i1QKOXXFmK03YMg2gGBbjFYf4f75lLZTELLBriJy1a 53fpGPFlC8m/udl0hqpVfB1Kj7Jj+fyCD3pnfPo3UpOZtLWqzO64IfU24Qf2RL8L 45/SMdzYN6XSlQ6BSzLlg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdekucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteek udehjeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhmpd hrtghpthhtohepjhhlthhosghlvghrsehgmhgrihhlrdgtohhmpdhrtghpthhtohepghhi thesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehkrghrthhhihhkrdduke eksehgmhgrihhlrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:27 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id b37429bc (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:27 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:19 +0200 Subject: [PATCH v2 07/16] reftable/block: rename `block_reader` to `reftable_block` Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-7-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 The `block_reader` structure is used to access parsed data of a reftable block. The structure is currently treated as an internal implementation detail and not exposed via our public interfaces. The functionality provided by the structure is useful to external users of the reftable library though, for example when implementing consistency checks that need to scan through the blocks manually. Rename the structure to `reftable_block` now that the name has been made available in the preceding commit. This name is in line with the naming schema used for other data structures like `reftable_table` in that it describes the underlying entity that it provides access to. The new data structure isn't yet exposed via the public interface, which is left for a subsequent commit. Signed-off-by: Patrick Steinhardt --- reftable/block.c | 142 ++++++++++++++++++++-------------------- reftable/block.h | 29 ++++---- reftable/iter.c | 9 ++- reftable/iter.h | 2 +- reftable/table.c | 46 ++++++------- reftable/table.h | 8 ++- t/unit-tests/t-reftable-block.c | 102 ++++++++++++++--------------- 7 files changed, 172 insertions(+), 166 deletions(-) diff --git a/reftable/block.c b/reftable/block.c index dca30b8527f..844da397833 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -222,10 +222,10 @@ static int read_block(struct reftable_block_source *source, return block_source_read_data(source, dest, off, sz); } -int block_reader_init(struct block_reader *br, - struct reftable_block_source *source, - uint32_t offset, uint32_t header_size, - uint32_t table_block_size, uint32_t hash_size) +int reftable_block_init(struct reftable_block *block, + struct reftable_block_source *source, + uint32_t offset, uint32_t header_size, + uint32_t table_block_size, uint32_t hash_size) { uint32_t guess_block_size = table_block_size ? table_block_size : DEFAULT_BLOCK_SIZE; @@ -236,19 +236,19 @@ int block_reader_init(struct block_reader *br, uint8_t block_type; int err; - err = read_block(source, &br->block_data, offset, guess_block_size); + err = read_block(source, &block->block_data, offset, guess_block_size); if (err < 0) goto done; - block_type = br->block_data.data[header_size]; + block_type = block->block_data.data[header_size]; if (!reftable_is_block_type(block_type)) { err = REFTABLE_FORMAT_ERROR; goto done; } - block_size = reftable_get_be24(br->block_data.data + header_size + 1); + block_size = reftable_get_be24(block->block_data.data + header_size + 1); if (block_size > guess_block_size) { - err = read_block(source, &br->block_data, offset, block_size); + err = read_block(source, &block->block_data, offset, block_size); if (err < 0) goto done; } @@ -256,39 +256,39 @@ int block_reader_init(struct block_reader *br, if (block_type == BLOCK_TYPE_LOG) { uint32_t block_header_skip = 4 + header_size; uLong dst_len = block_size - block_header_skip; - uLong src_len = br->block_data.len - block_header_skip; + uLong src_len = block->block_data.len - block_header_skip; /* Log blocks specify the *uncompressed* size in their header. */ - REFTABLE_ALLOC_GROW_OR_NULL(br->uncompressed_data, block_size, - br->uncompressed_cap); - if (!br->uncompressed_data) { + REFTABLE_ALLOC_GROW_OR_NULL(block->uncompressed_data, block_size, + block->uncompressed_cap); + if (!block->uncompressed_data) { err = REFTABLE_OUT_OF_MEMORY_ERROR; goto done; } /* Copy over the block header verbatim. It's not compressed. */ - memcpy(br->uncompressed_data, br->block_data.data, block_header_skip); + memcpy(block->uncompressed_data, block->block_data.data, block_header_skip); - if (!br->zstream) { - REFTABLE_CALLOC_ARRAY(br->zstream, 1); - if (!br->zstream) { + if (!block->zstream) { + REFTABLE_CALLOC_ARRAY(block->zstream, 1); + if (!block->zstream) { err = REFTABLE_OUT_OF_MEMORY_ERROR; goto done; } - err = inflateInit(br->zstream); + err = inflateInit(block->zstream); } else { - err = inflateReset(br->zstream); + err = inflateReset(block->zstream); } if (err != Z_OK) { err = REFTABLE_ZLIB_ERROR; goto done; } - br->zstream->next_in = br->block_data.data + block_header_skip; - br->zstream->avail_in = src_len; - br->zstream->next_out = br->uncompressed_data + block_header_skip; - br->zstream->avail_out = dst_len; + block->zstream->next_in = block->block_data.data + block_header_skip; + block->zstream->avail_in = src_len; + block->zstream->next_out = block->uncompressed_data + block_header_skip; + block->zstream->avail_out = dst_len; /* * We know both input as well as output size, and we know that @@ -297,71 +297,71 @@ int block_reader_init(struct block_reader *br, * here to instruct zlib to inflate the data in one go, which * is more efficient than using `Z_NO_FLUSH`. */ - err = inflate(br->zstream, Z_FINISH); + err = inflate(block->zstream, Z_FINISH); if (err != Z_STREAM_END) { err = REFTABLE_ZLIB_ERROR; goto done; } err = 0; - if (br->zstream->total_out + block_header_skip != block_size) { + if (block->zstream->total_out + block_header_skip != block_size) { err = REFTABLE_FORMAT_ERROR; goto done; } /* We're done with the input data. */ - block_source_release_data(&br->block_data); - br->block_data.data = br->uncompressed_data; - br->block_data.len = block_size; - full_block_size = src_len + block_header_skip - br->zstream->avail_in; + block_source_release_data(&block->block_data); + block->block_data.data = block->uncompressed_data; + block->block_data.len = block_size; + full_block_size = src_len + block_header_skip - block->zstream->avail_in; } else if (full_block_size == 0) { full_block_size = block_size; - } else if (block_size < full_block_size && block_size < br->block_data.len && - br->block_data.data[block_size] != 0) { + } else if (block_size < full_block_size && block_size < block->block_data.len && + block->block_data.data[block_size] != 0) { /* If the block is smaller than the full block size, it is padded (data followed by '\0') or the next block is unaligned. */ full_block_size = block_size; } - restart_count = reftable_get_be16(br->block_data.data + block_size - 2); + restart_count = reftable_get_be16(block->block_data.data + block_size - 2); restart_off = block_size - 2 - 3 * restart_count; - br->block_type = block_type; - br->hash_size = hash_size; - br->restart_off = restart_off; - br->full_block_size = full_block_size; - br->header_off = header_size; - br->restart_count = restart_count; + block->block_type = block_type; + block->hash_size = hash_size; + block->restart_off = restart_off; + block->full_block_size = full_block_size; + block->header_off = header_size; + block->restart_count = restart_count; err = 0; done: if (err < 0) - block_reader_release(br); + reftable_block_release(block); return err; } -void block_reader_release(struct block_reader *br) +void reftable_block_release(struct reftable_block *block) { - inflateEnd(br->zstream); - reftable_free(br->zstream); - reftable_free(br->uncompressed_data); - block_source_release_data(&br->block_data); - memset(br, 0, sizeof(*br)); + inflateEnd(block->zstream); + reftable_free(block->zstream); + reftable_free(block->uncompressed_data); + block_source_release_data(&block->block_data); + memset(block, 0, sizeof(*block)); } -uint8_t block_reader_type(const struct block_reader *r) +uint8_t reftable_block_type(const struct reftable_block *b) { - return r->block_data.data[r->header_off]; + return b->block_data.data[b->header_off]; } -int block_reader_first_key(const struct block_reader *br, struct reftable_buf *key) +int reftable_block_first_key(const struct reftable_block *block, struct reftable_buf *key) { - int off = br->header_off + 4, n; + int off = block->header_off + 4, n; struct string_view in = { - .buf = br->block_data.data + off, - .len = br->restart_off - off, + .buf = block->block_data.data + off, + .len = block->restart_off - off, }; uint8_t extra = 0; @@ -376,33 +376,33 @@ int block_reader_first_key(const struct block_reader *br, struct reftable_buf *k return 0; } -static uint32_t block_reader_restart_offset(const struct block_reader *br, size_t idx) +static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx) { - return reftable_get_be24(br->block_data.data + br->restart_off + 3 * idx); + return reftable_get_be24(b->block_data.data + b->restart_off + 3 * idx); } -void block_iter_seek_start(struct block_iter *it, const struct block_reader *br) +void block_iter_seek_start(struct block_iter *it, const struct reftable_block *b) { - it->block = br->block_data.data; - it->block_len = br->restart_off; - it->hash_size = br->hash_size; + it->block = b->block_data.data; + it->block_len = b->restart_off; + it->hash_size = b->hash_size; reftable_buf_reset(&it->last_key); - it->next_off = br->header_off + 4; + it->next_off = b->header_off + 4; } struct restart_needle_less_args { int error; struct reftable_buf needle; - const struct block_reader *reader; + const struct reftable_block *block; }; static int restart_needle_less(size_t idx, void *_args) { struct restart_needle_less_args *args = _args; - uint32_t off = block_reader_restart_offset(args->reader, idx); + uint32_t off = block_restart_offset(args->block, idx); struct string_view in = { - .buf = args->reader->block_data.data + off, - .len = args->reader->restart_off - off, + .buf = args->block->block_data.data + off, + .len = args->block->restart_off - off, }; uint64_t prefix_len, suffix_len; uint8_t extra; @@ -477,12 +477,12 @@ void block_iter_close(struct block_iter *it) reftable_buf_release(&it->scratch); } -int block_iter_seek_key(struct block_iter *it, const struct block_reader *br, +int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block, struct reftable_buf *want) { struct restart_needle_less_args args = { .needle = *want, - .reader = br, + .block = block, }; struct reftable_record rec; int err = 0; @@ -500,7 +500,7 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br, * restart point. While that works alright, we would end up scanning * too many record. */ - i = binsearch(br->restart_count, &restart_needle_less, &args); + i = binsearch(block->restart_count, &restart_needle_less, &args); if (args.error) { err = REFTABLE_FORMAT_ERROR; goto done; @@ -525,21 +525,21 @@ int block_iter_seek_key(struct block_iter *it, const struct block_reader *br, * starting from the preceding restart point. */ if (i > 0) - it->next_off = block_reader_restart_offset(br, i - 1); + it->next_off = block_restart_offset(block, i - 1); else - it->next_off = br->header_off + 4; - it->block = br->block_data.data; - it->block_len = br->restart_off; - it->hash_size = br->hash_size; + it->next_off = block->header_off + 4; + it->block = block->block_data.data; + it->block_len = block->restart_off; + it->hash_size = block->hash_size; - err = reftable_record_init(&rec, block_reader_type(br)); + err = reftable_record_init(&rec, reftable_block_type(block)); if (err < 0) goto done; /* * We're looking for the last entry less than the wanted key so that * the next call to `block_reader_next()` would yield the wanted - * record. We thus don't want to position our reader at the sought + * record. We thus don't want to position our iterator at the sought * after record, but one before. To do so, we have to go one entry too * far and then back up. */ diff --git a/reftable/block.h b/reftable/block.h index 948b79e4614..3957aee4293 100644 --- a/reftable/block.h +++ b/reftable/block.h @@ -64,8 +64,11 @@ void block_writer_release(struct block_writer *bw); struct z_stream; -/* Read a block. */ -struct block_reader { +/* + * A block part of a reftable. Contains records as well as some metadata + * describing them. + */ +struct reftable_block { /* offset of the block header; nonzero for the first block in a * reftable. */ uint32_t header_off; @@ -92,19 +95,21 @@ struct block_reader { uint8_t block_type; }; -/* initializes a block reader. */ -int block_reader_init(struct block_reader *br, - struct reftable_block_source *source, - uint32_t offset, uint32_t header_size, - uint32_t table_block_size, uint32_t hash_size); +/* + * Initialize a reftable block from the given block source. + */ +int reftable_block_init(struct reftable_block *b, + struct reftable_block_source *source, + uint32_t offset, uint32_t header_size, + uint32_t table_block_size, uint32_t hash_size); -void block_reader_release(struct block_reader *br); +void reftable_block_release(struct reftable_block *b); /* Returns the block type (eg. 'r' for refs) */ -uint8_t block_reader_type(const struct block_reader *r); +uint8_t reftable_block_type(const struct reftable_block *b); /* Decodes the first key in the block */ -int block_reader_first_key(const struct block_reader *br, struct reftable_buf *key); +int reftable_block_first_key(const struct reftable_block *b, struct reftable_buf *key); /* Iterate over entries in a block */ struct block_iter { @@ -125,10 +130,10 @@ struct block_iter { } /* Position `it` at start of the block */ -void block_iter_seek_start(struct block_iter *it, const struct block_reader *br); +void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block); /* Position `it` to the `want` key in the block */ -int block_iter_seek_key(struct block_iter *it, const struct block_reader *br, +int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block, struct reftable_buf *want); /* return < 0 for error, 0 for OK, > 0 for EOF. */ diff --git a/reftable/iter.c b/reftable/iter.c index c00ffc88577..9ce675510c9 100644 --- a/reftable/iter.c +++ b/reftable/iter.c @@ -114,7 +114,7 @@ static void indexed_table_ref_iter_close(void *p) { struct indexed_table_ref_iter *it = p; block_iter_close(&it->cur); - block_source_release_data(&it->block_reader.block_data); + block_source_release_data(&it->block.block_data); reftable_free(it->offsets); reftable_buf_release(&it->oid); } @@ -128,11 +128,10 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it) return 1; } - block_source_release_data(&it->block_reader.block_data); + block_source_release_data(&it->block.block_data); off = it->offsets[it->offset_idx++]; - err = table_init_block_reader(it->table, &it->block_reader, off, - BLOCK_TYPE_REF); + err = table_init_block(it->table, &it->block, off, BLOCK_TYPE_REF); if (err < 0) { return err; } @@ -140,7 +139,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it) /* indexed block does not exist. */ return REFTABLE_FORMAT_ERROR; } - block_iter_seek_start(&it->cur, &it->block_reader); + block_iter_seek_start(&it->cur, &it->block); return 0; } diff --git a/reftable/iter.h b/reftable/iter.h index add5c9472fc..cc920970a57 100644 --- a/reftable/iter.h +++ b/reftable/iter.h @@ -68,7 +68,7 @@ struct indexed_table_ref_iter { /* Points to the next offset to read. */ int offset_idx; int offset_len; - struct block_reader block_reader; + struct reftable_block block; struct block_iter cur; int is_finished; }; diff --git a/reftable/table.c b/reftable/table.c index c86b1d4c649..ef1f33c92fc 100644 --- a/reftable/table.c +++ b/reftable/table.c @@ -131,7 +131,7 @@ struct table_iter { struct reftable_table *table; uint8_t typ; uint64_t block_off; - struct block_reader br; + struct reftable_block block; struct block_iter bi; int is_finished; }; @@ -159,12 +159,12 @@ static int table_iter_next_in_block(struct table_iter *ti, static void table_iter_block_done(struct table_iter *ti) { - block_reader_release(&ti->br); + reftable_block_release(&ti->block); block_iter_reset(&ti->bi); } -int table_init_block_reader(struct reftable_table *t, struct block_reader *br, - uint64_t next_off, uint8_t want_typ) +int table_init_block(struct reftable_table *t, struct reftable_block *block, + uint64_t next_off, uint8_t want_typ) { uint32_t header_off = next_off ? 0 : header_size(t->version); int err; @@ -172,19 +172,19 @@ int table_init_block_reader(struct reftable_table *t, struct block_reader *br, if (next_off >= t->size) return 1; - err = block_reader_init(br, &t->source, next_off, header_off, - t->block_size, hash_size(t->hash_id)); + err = reftable_block_init(block, &t->source, next_off, header_off, + t->block_size, hash_size(t->hash_id)); if (err < 0) goto done; - if (want_typ != BLOCK_TYPE_ANY && br->block_type != want_typ) { + if (want_typ != BLOCK_TYPE_ANY && block->block_type != want_typ) { err = 1; goto done; } done: if (err) - block_reader_release(br); + reftable_block_release(block); return err; } @@ -197,10 +197,10 @@ static void table_iter_close(struct table_iter *ti) static int table_iter_next_block(struct table_iter *ti) { - uint64_t next_block_off = ti->block_off + ti->br.full_block_size; + uint64_t next_block_off = ti->block_off + ti->block.full_block_size; int err; - err = table_init_block_reader(ti->table, &ti->br, next_block_off, ti->typ); + err = table_init_block(ti->table, &ti->block, next_block_off, ti->typ); if (err > 0) ti->is_finished = 1; if (err) @@ -208,7 +208,7 @@ static int table_iter_next_block(struct table_iter *ti) ti->block_off = next_block_off; ti->is_finished = 0; - block_iter_seek_start(&ti->bi, &ti->br); + block_iter_seek_start(&ti->bi, &ti->block); return 0; } @@ -250,13 +250,13 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ) { int err; - err = table_init_block_reader(ti->table, &ti->br, off, typ); + err = table_init_block(ti->table, &ti->block, off, typ); if (err != 0) return err; - ti->typ = block_reader_type(&ti->br); + ti->typ = reftable_block_type(&ti->block); ti->block_off = off; - block_iter_seek_start(&ti->bi, &ti->br); + block_iter_seek_start(&ti->bi, &ti->block); ti->is_finished = 0; return 0; } @@ -320,10 +320,10 @@ static int table_iter_seek_linear(struct table_iter *ti, * as we have more than three blocks we would have an index, so * we would not do a linear search there anymore. */ - memset(&next.br.block_data, 0, sizeof(next.br.block_data)); - next.br.zstream = NULL; - next.br.uncompressed_data = NULL; - next.br.uncompressed_cap = 0; + memset(&next.block.block_data, 0, sizeof(next.block.block_data)); + next.block.zstream = NULL; + next.block.uncompressed_data = NULL; + next.block.uncompressed_cap = 0; err = table_iter_next_block(&next); if (err < 0) @@ -331,7 +331,7 @@ static int table_iter_seek_linear(struct table_iter *ti, if (err > 0) break; - err = block_reader_first_key(&next.br, &got_key); + err = reftable_block_first_key(&next.block, &got_key); if (err < 0) goto done; @@ -349,7 +349,7 @@ static int table_iter_seek_linear(struct table_iter *ti, * the wanted key inside of it. If the block does not contain our key * we know that the corresponding record does not exist. */ - err = block_iter_seek_key(&ti->bi, &ti->br, &want_key); + err = block_iter_seek_key(&ti->bi, &ti->block, &want_key); if (err < 0) goto done; err = 0; @@ -417,7 +417,7 @@ static int table_iter_seek_indexed(struct table_iter *ti, if (err != 0) goto done; - err = block_iter_seek_key(&ti->bi, &ti->br, &want_index.u.idx.last_key); + err = block_iter_seek_key(&ti->bi, &ti->block, &want_index.u.idx.last_key); if (err < 0) goto done; @@ -785,8 +785,8 @@ int reftable_table_print_blocks(const char *tablename) printf("%s:\n", sections[i].name); while (1) { - printf(" - length: %u\n", ti.br.restart_off); - printf(" restarts: %u\n", ti.br.restart_count); + printf(" - length: %u\n", ti.block.restart_off); + printf(" restarts: %u\n", ti.block.restart_count); err = table_iter_next_block(&ti); if (err < 0) diff --git a/reftable/table.h b/reftable/table.h index 8d8dd2b413d..e15d58d8e9e 100644 --- a/reftable/table.h +++ b/reftable/table.h @@ -53,8 +53,10 @@ int table_init_iter(struct reftable_table *t, struct reftable_iterator *it, uint8_t typ); -/* initialize a block reader to read from `t` */ -int table_init_block_reader(struct reftable_table *t, struct block_reader *br, - uint64_t next_off, uint8_t want_typ); +/* + * Initialize a block by reading from the given table and offset. + */ +int table_init_block(struct reftable_table *t, struct reftable_block *block, + uint64_t next_off, uint8_t want_typ); #endif diff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c index dbec6a5d0d0..e36ed7ac576 100644 --- a/t/unit-tests/t-reftable-block.c +++ b/t/unit-tests/t-reftable-block.c @@ -28,16 +28,16 @@ static void t_ref_block_read_write(void) }; size_t i = 0; int ret; - struct block_reader br = { 0 }; + struct reftable_block block = { 0 }; struct block_iter it = BLOCK_ITER_INIT; struct reftable_buf want = REFTABLE_BUF_INIT; - struct reftable_buf block = REFTABLE_BUF_INIT; + struct reftable_buf block_data = REFTABLE_BUF_INIT; - REFTABLE_CALLOC_ARRAY(block.buf, block_size); - check(block.buf != NULL); - block.len = block_size; + REFTABLE_CALLOC_ARRAY(block_data.buf, block_size); + check(block_data.buf != NULL); + block_data.len = block_size; - ret = block_writer_init(&bw, BLOCK_TYPE_REF, (uint8_t *) block.buf, block_size, + ret = block_writer_init(&bw, BLOCK_TYPE_REF, (uint8_t *) block_data.buf, block_size, header_off, hash_size(REFTABLE_HASH_SHA1)); check(!ret); @@ -63,10 +63,10 @@ static void t_ref_block_read_write(void) block_writer_release(&bw); - block_source_from_buf(&source ,&block); - block_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); + block_source_from_buf(&source ,&block_data); + reftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); - block_iter_seek_start(&it, &br); + block_iter_seek_start(&it, &block); for (i = 0; ; i++) { ret = block_iter_next(&it, &rec); @@ -82,7 +82,7 @@ static void t_ref_block_read_write(void) block_iter_reset(&it); reftable_record_key(&recs[i], &want); - ret = block_iter_seek_key(&it, &br, &want); + ret = block_iter_seek_key(&it, &block, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -91,7 +91,7 @@ static void t_ref_block_read_write(void) check(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1)); want.len--; - ret = block_iter_seek_key(&it, &br, &want); + ret = block_iter_seek_key(&it, &block, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -99,11 +99,11 @@ static void t_ref_block_read_write(void) check(reftable_record_equal(&recs[10 * (i / 10)], &rec, REFTABLE_HASH_SIZE_SHA1)); } - block_reader_release(&br); + reftable_block_release(&block); block_iter_close(&it); reftable_record_release(&rec); reftable_buf_release(&want); - reftable_buf_release(&block); + reftable_buf_release(&block_data); for (i = 0; i < N; i++) reftable_record_release(&recs[i]); } @@ -123,16 +123,16 @@ static void t_log_block_read_write(void) }; size_t i = 0; int ret; - struct block_reader br = { 0 }; + struct reftable_block block = { 0 }; struct block_iter it = BLOCK_ITER_INIT; struct reftable_buf want = REFTABLE_BUF_INIT; - struct reftable_buf block = REFTABLE_BUF_INIT; + struct reftable_buf block_data = REFTABLE_BUF_INIT; - REFTABLE_CALLOC_ARRAY(block.buf, block_size); - check(block.buf != NULL); - block.len = block_size; + REFTABLE_CALLOC_ARRAY(block_data.buf, block_size); + check(block_data.buf != NULL); + block_data.len = block_size; - ret = block_writer_init(&bw, BLOCK_TYPE_LOG, (uint8_t *) block.buf, block_size, + ret = block_writer_init(&bw, BLOCK_TYPE_LOG, (uint8_t *) block_data.buf, block_size, header_off, hash_size(REFTABLE_HASH_SHA1)); check(!ret); @@ -153,10 +153,10 @@ static void t_log_block_read_write(void) block_writer_release(&bw); - block_source_from_buf(&source, &block); - block_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); + block_source_from_buf(&source, &block_data); + reftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); - block_iter_seek_start(&it, &br); + block_iter_seek_start(&it, &block); for (i = 0; ; i++) { ret = block_iter_next(&it, &rec); @@ -173,7 +173,7 @@ static void t_log_block_read_write(void) reftable_buf_reset(&want); check(!reftable_buf_addstr(&want, recs[i].u.log.refname)); - ret = block_iter_seek_key(&it, &br, &want); + ret = block_iter_seek_key(&it, &block, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -182,7 +182,7 @@ static void t_log_block_read_write(void) check(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1)); want.len--; - ret = block_iter_seek_key(&it, &br, &want); + ret = block_iter_seek_key(&it, &block, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -190,11 +190,11 @@ static void t_log_block_read_write(void) check(reftable_record_equal(&recs[10 * (i / 10)], &rec, REFTABLE_HASH_SIZE_SHA1)); } - block_reader_release(&br); + reftable_block_release(&block); block_iter_close(&it); reftable_record_release(&rec); reftable_buf_release(&want); - reftable_buf_release(&block); + reftable_buf_release(&block_data); for (i = 0; i < N; i++) reftable_record_release(&recs[i]); } @@ -214,16 +214,16 @@ static void t_obj_block_read_write(void) }; size_t i = 0; int ret; - struct block_reader br = { 0 }; + struct reftable_block block = { 0 }; struct block_iter it = BLOCK_ITER_INIT; struct reftable_buf want = REFTABLE_BUF_INIT; - struct reftable_buf block = REFTABLE_BUF_INIT; + struct reftable_buf block_data = REFTABLE_BUF_INIT; - REFTABLE_CALLOC_ARRAY(block.buf, block_size); - check(block.buf != NULL); - block.len = block_size; + REFTABLE_CALLOC_ARRAY(block_data.buf, block_size); + check(block_data.buf != NULL); + block_data.len = block_size; - ret = block_writer_init(&bw, BLOCK_TYPE_OBJ, (uint8_t *) block.buf, block_size, + ret = block_writer_init(&bw, BLOCK_TYPE_OBJ, (uint8_t *) block_data.buf, block_size, header_off, hash_size(REFTABLE_HASH_SHA1)); check(!ret); @@ -246,10 +246,10 @@ static void t_obj_block_read_write(void) block_writer_release(&bw); - block_source_from_buf(&source, &block); - block_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); + block_source_from_buf(&source, &block_data); + reftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); - block_iter_seek_start(&it, &br); + block_iter_seek_start(&it, &block); for (i = 0; ; i++) { ret = block_iter_next(&it, &rec); @@ -265,7 +265,7 @@ static void t_obj_block_read_write(void) block_iter_reset(&it); reftable_record_key(&recs[i], &want); - ret = block_iter_seek_key(&it, &br, &want); + ret = block_iter_seek_key(&it, &block, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -274,11 +274,11 @@ static void t_obj_block_read_write(void) check(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1)); } - block_reader_release(&br); + reftable_block_release(&block); block_iter_close(&it); reftable_record_release(&rec); reftable_buf_release(&want); - reftable_buf_release(&block); + reftable_buf_release(&block_data); for (i = 0; i < N; i++) reftable_record_release(&recs[i]); } @@ -299,16 +299,16 @@ static void t_index_block_read_write(void) }; size_t i = 0; int ret; - struct block_reader br = { 0 }; + struct reftable_block block = { 0 }; struct block_iter it = BLOCK_ITER_INIT; struct reftable_buf want = REFTABLE_BUF_INIT; - struct reftable_buf block = REFTABLE_BUF_INIT; + struct reftable_buf block_data = REFTABLE_BUF_INIT; - REFTABLE_CALLOC_ARRAY(block.buf, block_size); - check(block.buf != NULL); - block.len = block_size; + REFTABLE_CALLOC_ARRAY(block_data.buf, block_size); + check(block_data.buf != NULL); + block_data.len = block_size; - ret = block_writer_init(&bw, BLOCK_TYPE_INDEX, (uint8_t *) block.buf, block_size, + ret = block_writer_init(&bw, BLOCK_TYPE_INDEX, (uint8_t *) block_data.buf, block_size, header_off, hash_size(REFTABLE_HASH_SHA1)); check(!ret); @@ -331,10 +331,10 @@ static void t_index_block_read_write(void) block_writer_release(&bw); - block_source_from_buf(&source, &block); - block_reader_init(&br, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); + block_source_from_buf(&source, &block_data); + reftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); - block_iter_seek_start(&it, &br); + block_iter_seek_start(&it, &block); for (i = 0; ; i++) { ret = block_iter_next(&it, &rec); @@ -350,7 +350,7 @@ static void t_index_block_read_write(void) block_iter_reset(&it); reftable_record_key(&recs[i], &want); - ret = block_iter_seek_key(&it, &br, &want); + ret = block_iter_seek_key(&it, &block, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -359,7 +359,7 @@ static void t_index_block_read_write(void) check(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1)); want.len--; - ret = block_iter_seek_key(&it, &br, &want); + ret = block_iter_seek_key(&it, &block, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -367,11 +367,11 @@ static void t_index_block_read_write(void) check(reftable_record_equal(&recs[10 * (i / 10)], &rec, REFTABLE_HASH_SIZE_SHA1)); } - block_reader_release(&br); + reftable_block_release(&block); block_iter_close(&it); reftable_record_release(&rec); reftable_buf_release(&want); - reftable_buf_release(&block); + reftable_buf_release(&block_data); for (i = 0; i < N; i++) reftable_record_release(&recs[i]); } From patchwork Mon Apr 7 13:16:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040507 Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA2C7254B06 for ; Mon, 7 Apr 2025 13:16:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031792; cv=none; b=iHh/BXPGN1BHqTEXVD59mzoCpoGblPJxmzA1GBS97dj8H86GJTmrSMhRP3qye30WYouV9xApUiiclq8pPSEHMJEJ2fQGbkWgAkRrKuhdhvpk6UUwd2biFPpD1Hklhx8t49uwrac3ToDXgH6YDd+t521df53tITeZnR/Fu/8tS30= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031792; c=relaxed/simple; bh=FNzaxPpnPc1UmJ54YaeF5tdlhCzepg0sEe9Qtc2ik5o=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=UbpG5MXXch7ywNR9ogqCjTB2Nzt8RYqGo8qE146l65KH7rY6m/k/Ur0jYZghhmCiOoIvvKYY6mt/xcehxgRwbnyMP+epp9pkAcqsaKqIxyT45AGK2a69OKB4BW7bWPW8rV3rZI+lg4bi1PVn1Sq0qGk1Ss7GHBmUoXoQEQoJ8H8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=A1nlAVoh; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=s1xfl9LW; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="A1nlAVoh"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="s1xfl9LW" Received: from phl-compute-09.internal (phl-compute-09.phl.internal [10.202.2.49]) by mailfhigh.phl.internal (Postfix) with ESMTP id EB5C8114021E; Mon, 7 Apr 2025 09:16:29 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-09.internal (MEProxy); Mon, 07 Apr 2025 09:16:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031789; x=1744118189; bh=GfI3GjqovTr+NO4OovG9dDXowrPS3ukWkfvxem7+gF4=; b= A1nlAVohY5nfVOMwJkGv1VexzyhBQtPpwhKJuNi1sU63jgC3OmoqYl3z2Jd0QFcK rU39hNu6Npud0kTF0ZcyzwEWw5rY0w311EEC1pFlR0GSU+1WgJIiVs16R9senvaQ Ea/CqTydxm5pRkd71Eyn8R8D4MpdBFkNm0W7lYE0fCaZB6ST62GVVy67BZpz+v/M PVFQheyKRPv8eecqoNjk4BVrXWBcKM6r+6RrGeCR6mTYzZuiZKlGUpGdCiVeN7f3 YaSIol8Q5lKge8OJo3W4bjoOdWt5ePmHHBZdBVG+YQYCU2Te1r91VYXq2vH8yll4 LSnhqK5eqNdvzK19nvHk+w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031789; x= 1744118189; bh=GfI3GjqovTr+NO4OovG9dDXowrPS3ukWkfvxem7+gF4=; b=s 1xfl9LWH/fu6bscy1xVYlQ1aCE5y3K20zJuTqwZ5qmGnpQLL4ogRFZmmS3Az0gL1 DnVCv42zMr51rVWPOlM9nI9f5L7OHD38vaLXo1aaUBlK2tXg2vNbm5VqwezXpUeT 1e/6ijH1XwEr0KxhMT6Yxq/zLJlIJEyuNABH9ol7SEoe3Tfa0LPLIpoKp5sxSwRg Cal6xSa6KkDg9a8TdjyExT3wSHZeIMA815va9K3mn3geFen1/Sq/AtoOUqRMyC9e ntu+S9TZCuj5vzjiZOT7RWUSI1BYQo/q/atg2Wd6ePHbgnf2WdSIwo9Czti/KOGw sqaykVvlVs3L8A6/CbEdQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdelucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteek udehjeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmh epmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehjlhhtohgslhgvrhesghhmrghilhdrtghomh dprhgtphhtthhopehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohep ghhithhsthgvrhesphhosghogidrtghomhdprhgtphhtthhopehkrghrthhhihhkrdduke eksehgmhgrihhlrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:28 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id f21eff2b (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:28 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:20 +0200 Subject: [PATCH v2 08/16] git-zlib: use `struct z_stream_s` instead of typedef Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-8-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 Throughout the Git codebase we're using the typedeffed version of `z_stream`, which maps to `struct z_stream_s`. By using a typedef instead of the struct it becomes somewhat harder to predeclare the symbol so that headers depending on the struct can do so without having to pull in "zlib-compat.h". We don't yet have users that would really care about this: the only users that declare `z_stream` as a pointer are in "reftable/block.h", which is a header that is internal to the reftable library. But in the next step we're going to expose the `struct reftable_block` publicly, and that struct does contain a pointer to `z_stream`. And as the public header shouldn't depend on "reftable/system.h", which is an internal implementation detail, we won't have the typedef for `z_stream` readily available. Prepare for this change by using `struct z_stream_s` throughout our code base. In case zlib-ng is used we use a define to map from `z_stream_s` to `zng_stream_s`. Drop the pre-declaration of `struct z_stream` while at it. This struct does not exist in the first place, and the declaration wasn't needed because "reftable/block.h" already includes "reftable/basics.h" which transitively includes "reftable/system.h" and thus "git-zlib.h". Signed-off-by: Patrick Steinhardt --- Documentation/howto/recover-corrupted-object-harder.adoc | 4 ++-- compat/zlib-compat.h | 4 ++-- git-zlib.h | 2 +- reftable/block.h | 6 ++---- 4 files changed, 7 insertions(+), 9 deletions(-) diff --git a/Documentation/howto/recover-corrupted-object-harder.adoc b/Documentation/howto/recover-corrupted-object-harder.adoc index 5efb4fe81ff..86a1ba75cf9 100644 --- a/Documentation/howto/recover-corrupted-object-harder.adoc +++ b/Documentation/howto/recover-corrupted-object-harder.adoc @@ -125,7 +125,7 @@ static int try_zlib(unsigned char *buf, int len) { /* make this absurdly large so we don't have to loop */ static unsigned char out[1024*1024]; - z_stream z; + struct z_stream_s z; int ret; memset(&z, 0, sizeof(z)); @@ -278,7 +278,7 @@ int main(int argc, char **argv) static unsigned char buf[25 * 1024 * 1024]; static unsigned char out[25 * 1024 * 1024]; int len; - z_stream z; + struct z_stream_s z; int ret; len = read(0, buf, sizeof(buf)); diff --git a/compat/zlib-compat.h b/compat/zlib-compat.h index 0c60e3af331..ac082766229 100644 --- a/compat/zlib-compat.h +++ b/compat/zlib-compat.h @@ -4,8 +4,8 @@ #ifdef HAVE_ZLIB_NG # include -# define z_stream zng_stream -#define gz_header_s zng_gz_header_s +# define z_stream_s zng_stream_s +# define gz_header_s zng_gz_header_s # define crc32(crc, buf, len) zng_crc32(crc, buf, len) diff --git a/git-zlib.h b/git-zlib.h index 1e8d9aabcb4..0e66fefa8c9 100644 --- a/git-zlib.h +++ b/git-zlib.h @@ -4,7 +4,7 @@ #include "compat/zlib-compat.h" typedef struct git_zstream { - z_stream z; + struct z_stream_s z; unsigned long avail_in; unsigned long avail_out; unsigned long total_in; diff --git a/reftable/block.h b/reftable/block.h index 3957aee4293..422e2f872c5 100644 --- a/reftable/block.h +++ b/reftable/block.h @@ -18,7 +18,7 @@ * allocation overhead. */ struct block_writer { - z_stream *zstream; + struct z_stream_s *zstream; unsigned char *compressed; size_t compressed_cap; @@ -62,8 +62,6 @@ int block_writer_finish(struct block_writer *w); /* clears out internally allocated block_writer members. */ void block_writer_release(struct block_writer *bw); -struct z_stream; - /* * A block part of a reftable. Contains records as well as some metadata * describing them. @@ -78,7 +76,7 @@ struct reftable_block { uint32_t hash_size; /* Uncompressed data for log entries. */ - z_stream *zstream; + struct z_stream_s *zstream; unsigned char *uncompressed_data; size_t uncompressed_cap; From patchwork Mon Apr 7 13:16:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040508 Received: from fout-a1-smtp.messagingengine.com (fout-a1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E63A2550A5 for ; Mon, 7 Apr 2025 13:16:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031794; cv=none; b=HhPJ99UzlIZG0PLZ3wY4LfBHSWHISOyfZpw60DCgIJf2iBnElYc3Mj74tppk0V+FOLcnpKydd1WZ6idj5/6XjLwWxOH1SEn97vTvVsUHHJ7+aOjz0uSexHuyAqsLBpLsaybjVZGK0JVyuKhaQUr4ssO77hJC+9lxqCY77/kG92c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031794; c=relaxed/simple; bh=d76KOJKqgcun7pPgDAsx8hFXmOYxGPQdTS5kx0tTClE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Al0vaxSKw7I6JFK46s2ETMDp9Dz6/9jDE6Yp1l5gnmQKvh1K5lG5OxHfBJNLmFGtDQvFMOqrg+2lJKnQ/UK0qhRZu+hR3Q3jmZm53Yf8ezOrXGujaT72CI9WPPJeMpyGFNSJnwWFPNNEtpOVNVlMJXm5uVnpygaPtiTGvekkRUQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=OgJ/lOeD; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=m+YXJqSb; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="OgJ/lOeD"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="m+YXJqSb" Received: from phl-compute-10.internal (phl-compute-10.phl.internal [10.202.2.50]) by mailfout.phl.internal (Postfix) with ESMTP id 297151380333; Mon, 7 Apr 2025 09:16:31 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-10.internal (MEProxy); Mon, 07 Apr 2025 09:16:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031791; x=1744118191; bh=pwS9BwrKMZMUD6+4b7qHWoSaZJ3fTSjJMOA8n41VTus=; b= OgJ/lOeDufguvaegwxyri7BRNBWIpSa3wWDZXqPeQWWTQ2TBQd1UeWc8eO/zIQe2 SqPMnDsV0exRVgmsoVEmPCo3XVLMcWqO1nS1dnTgwJxnqwk19ziuAZNfKCEMz738 6W/AzO37NClyq5ck1g/ffnskGEBz6ipsXcKRMGPfC+XGVrtWoGyD9gUANxaaSszl RtwEtehigODnBjINL5JfJ2ZKQ4Xulq5bB3yiAPnKqnEKL3CUwO3mRlMWCEWYgH13 vB0fTMgTjoCZcppJbyisce5GpoOgzAQ2YK40q7FQI4anUYSyfGzRuGHBaNQBJH8m ImLft97YlgldzgOfbs8UwQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031791; x= 1744118191; bh=pwS9BwrKMZMUD6+4b7qHWoSaZJ3fTSjJMOA8n41VTus=; b=m +YXJqSbH4BsbhUWnCSodajcQM5P+ljv55kpmvqSdBv9kavQWi1J4c9yj5KGhPVlR QbBy0kKWhWTX2hctkwoP7gRprJMS+/IzfCPzXBiRhjA5q88VYQLO+Hv1tilEIyQc 5lmh1R1VjyQv6qRtgN2a7jRfQ1UtDQUGhKRc4CTCKlYAr5F8sUiwzVbNDulAVvUW Xfr3juWxZtrMas1NIFHVOfT7BOON061ImHbmEmDb+GLnE1hS680MtHSQYvzwUGGU js52APtz6co8QQ4nadGB0VD+MXLnHMcKu+BklODseSMoOufb02xs4HGJANTceDmw ohN+/8yrzxDkRAmoGmoZw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdelucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpedthfegfeejuefgieffvefhvedvfedtffdtgffh heevgfdtjedukefgkefgfeevtdenucffohhmrghinhepghhoohhglhgvrdgtohhmnecuve hluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhs rdhimhdpnhgspghrtghpthhtohepgedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtoh epghhithesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehkrghrthhhihhk rddukeeksehgmhgrihhlrdgtohhmpdhrtghpthhtohepjhhlthhosghlvghrsehgmhgrih hlrdgtohhmpdhrtghpthhtohepghhithhsthgvrhesphhosghogidrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:30 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id ee5a1226 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:29 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:21 +0200 Subject: [PATCH v2 09/16] reftable/block: create public interface for reading blocks Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-9-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 While users of the reftable library wouldn't generally require access to individual blocks in a reftable table, there are valid usecases where one may require low-level access to them. One such upcoming usecase in the Git codebase is to implement consistency checks for the reftable library where we want to verify each block individually. Create a public interface for reading blocks. The interface isn't yet complete and lacks e.g. a way to read individual records from a block. Such missing functionality will be backfilled in subsequent commits. Note that this change also requires us to expose `reftable_buf`, which is used by the `reftable_block_first_key()` function. Signed-off-by: Patrick Steinhardt --- reftable/basics.h | 7 ----- reftable/block.h | 48 +------------------------------- reftable/reftable-basics.h | 8 ++++++ reftable/reftable-block.h | 69 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 78 insertions(+), 54 deletions(-) diff --git a/reftable/basics.h b/reftable/basics.h index 96a2f0d3829..d8888c12629 100644 --- a/reftable/basics.h +++ b/reftable/basics.h @@ -18,13 +18,6 @@ #define REFTABLE_UNUSED __attribute__((__unused__)) -struct reftable_buf { - size_t alloc; - size_t len; - char *buf; -}; -#define REFTABLE_BUF_INIT { 0 } - /* * Initialize the buffer such that it is ready for use. This is equivalent to * using REFTABLE_BUF_INIT for stack-allocated variables. diff --git a/reftable/block.h b/reftable/block.h index 422e2f872c5..4f7f29028c4 100644 --- a/reftable/block.h +++ b/reftable/block.h @@ -11,6 +11,7 @@ #include "basics.h" #include "record.h" +#include "reftable-block.h" #include "reftable-blocksource.h" /* @@ -62,53 +63,6 @@ int block_writer_finish(struct block_writer *w); /* clears out internally allocated block_writer members. */ void block_writer_release(struct block_writer *bw); -/* - * A block part of a reftable. Contains records as well as some metadata - * describing them. - */ -struct reftable_block { - /* offset of the block header; nonzero for the first block in a - * reftable. */ - uint32_t header_off; - - /* the memory block */ - struct reftable_block_data block_data; - uint32_t hash_size; - - /* Uncompressed data for log entries. */ - struct z_stream_s *zstream; - unsigned char *uncompressed_data; - size_t uncompressed_cap; - - /* - * Restart point data. Restart points are located after the block's - * record data. - */ - uint16_t restart_count; - uint32_t restart_off; - - /* size of the data in the file. For log blocks, this is the compressed - * size. */ - uint32_t full_block_size; - uint8_t block_type; -}; - -/* - * Initialize a reftable block from the given block source. - */ -int reftable_block_init(struct reftable_block *b, - struct reftable_block_source *source, - uint32_t offset, uint32_t header_size, - uint32_t table_block_size, uint32_t hash_size); - -void reftable_block_release(struct reftable_block *b); - -/* Returns the block type (eg. 'r' for refs) */ -uint8_t reftable_block_type(const struct reftable_block *b); - -/* Decodes the first key in the block */ -int reftable_block_first_key(const struct reftable_block *b, struct reftable_buf *key); - /* Iterate over entries in a block */ struct block_iter { /* offset within the block of the next entry to read. */ diff --git a/reftable/reftable-basics.h b/reftable/reftable-basics.h index ed7c7c9ac2c..6d73f19c85b 100644 --- a/reftable/reftable-basics.h +++ b/reftable/reftable-basics.h @@ -11,6 +11,14 @@ #include +/* A buffer that contains arbitrary byte slices. */ +struct reftable_buf { + size_t alloc; + size_t len; + char *buf; +}; +#define REFTABLE_BUF_INIT { 0 } + /* * Hash functions understood by the reftable library. Note that the values are * arbitrary and somewhat random such that we can easily detect cases where the diff --git a/reftable/reftable-block.h b/reftable/reftable-block.h new file mode 100644 index 00000000000..13bd68be8c5 --- /dev/null +++ b/reftable/reftable-block.h @@ -0,0 +1,69 @@ +/* + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ + +#ifndef REFTABLE_BLOCK_H +#define REFTABLE_BLOCK_H + +#include + +#include "reftable-basics.h" +#include "reftable-blocksource.h" + +struct z_stream_s; + +/* + * A block part of a reftable. Contains records as well as some metadata + * describing them. + */ +struct reftable_block { + /* + * Offset of the block header; nonzero for the first block in a + * reftable. + */ + uint32_t header_off; + + /* The memory block. */ + struct reftable_block_data block_data; + uint32_t hash_size; + + /* Uncompressed data for log entries. */ + struct z_stream_s *zstream; + unsigned char *uncompressed_data; + size_t uncompressed_cap; + + /* + * Restart point data. Restart points are located after the block's + * record data. + */ + uint16_t restart_count; + uint32_t restart_off; + + /* + * Size of the data in the file. For log blocks, this is the compressed + * size. + */ + uint32_t full_block_size; + uint8_t block_type; +}; + +/* Initialize a reftable block from the given block source. */ +int reftable_block_init(struct reftable_block *b, + struct reftable_block_source *source, + uint32_t offset, uint32_t header_size, + uint32_t table_block_size, uint32_t hash_size); + +/* Release resources allocated by the block. */ +void reftable_block_release(struct reftable_block *b); + +/* Returns the block type (eg. 'r' for refs). */ +uint8_t reftable_block_type(const struct reftable_block *b); + +/* Decodes the first key in the block. */ +int reftable_block_first_key(const struct reftable_block *b, struct reftable_buf *key); + +#endif /* REFTABLE_BLOCK_H */ From patchwork Mon Apr 7 13:16:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040509 Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40DD32550D1 for ; Mon, 7 Apr 2025 13:16:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031795; cv=none; b=UbvDgrJqTfDIj/ns5/oP3zaHatuXKpHJicmx1ySyVWdcwsacn5O/YjAmprawMIIhvKoLtGVnDUPKxSpziWTVleo5oeKxJmtRtByHv5QHgE0aZxshA67qvtPKufGihz3pzyrFZdW4UMrFLl1ZdKyYj2U9n5joAkvKlQKIQ9O8aoo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031795; c=relaxed/simple; bh=hdehLccJ6yPcjUXqOdKxfW98H7Fl3vp7olioXiAMyu0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=RR7KJG1YRtKAL3KOcGalu5Y1xKlkSwb6ijbHhWTlIhKwl3y1bX+juTry2Ornx/QKcjM8XTBVaJGIp0IR53gt2A/Avppn5XAVYtLkmvAjtIfnv/ZqqSgP2wvDmDKNAcRjxFpyWrUY9CUjXchEzBx9ATTtW5u9hpzVumhZ9vc9PXw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=KJqvRC90; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=pgfsh4Qt; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="KJqvRC90"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="pgfsh4Qt" Received: from phl-compute-07.internal (phl-compute-07.phl.internal [10.202.2.47]) by mailfhigh.phl.internal (Postfix) with ESMTP id 6F7DD1140211; Mon, 7 Apr 2025 09:16:32 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-07.internal (MEProxy); Mon, 07 Apr 2025 09:16:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031792; x=1744118192; bh=/wx0Xz/k2EfS3cATP92HlBX/k5L8YerMY0JUQETcpW0=; b= KJqvRC901ff9Ua4IgBl6ibH8USgDSSIoRSEqkaVWIVf8OnbmpwqwfpJtpo6tC8Je ocjPZ1tmHLP2UZaIaCSJ5BZR3RT7KpNDs4bBFUHguz8q0S7TN8bYN1LvuFJlrmEp LdP1ryKNFGW1KPbdR5ObcfJTqYDDvQjizjnjtFxUWnoGj9W/Sf3J0CLdJOZu35eP vu0aq8BA8yYzGbcAvkklKW6r24YrrxO+Y1ibCXspggCxzrspVtA61hM6jy5OTEfc YXOzGGU3JwTpczq3/ohG4Ckyz9N9P9dof/ajIcjzV68cuE8teR02QzunUmfo5A3B vSSp3KtXepnzjIIwceaykw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031792; x= 1744118192; bh=/wx0Xz/k2EfS3cATP92HlBX/k5L8YerMY0JUQETcpW0=; b=p gfsh4QtLO1GVwIj0I6NttScA7bplL0FbGnjuGpWpWoypL/RdPsU2CjbRivPOn/KV P5ovUPXkE5aU1j+SXsIJH/zKWqlMmsJiXxgM+Jw2OiBIGDCLgNSfPDLduSKD0Mc9 qTOLVsrJ63g/sCYg/dNmUPQ4FaxE2lhG/ZWiA/ha61aVR2Z/Gai1oMgk8KmxsSMw 1u9iiFyhYr1dJaj3JXwOmmdMxSpmUSodZ6zcnR+x7l+A1MsgRzG2Z21na1NVRfEV OYiWG9GVb/HPegC4StMA+19CTjj8vwwG9kJ3TNuWXxW75ujWwq5c6LBMgNG+TYw/ luvKITDMSQ27h3Dzy21Rw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdekucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteek udehjeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmh epmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhmpd hrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehj lhhtohgslhgvrhesghhmrghilhdrtghomhdprhgtphhtthhopehkrghrthhhihhkrdduke eksehgmhgrihhlrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:31 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id e85ac9be (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:30 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:22 +0200 Subject: [PATCH v2 10/16] reftable/block: store block pointer in the block iterator Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-10-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 The block iterator requires access to a bunch of data from the underlying `reftable_block` that it is iterating over. This data is stored by copying over relevant data into a separate set of variables. This has multiple downsides: - We require more storage space than necessary. This is more of a theoretical issue as we shouldn't ever have many blocks. - We have to perform more bookkeeping, and the variable names are inconsistent across the two data structures. This can lead to some confusion. - The lifetime of the block iterator is tied to the block anyway, but we hide that a bit by only storing pointers pointing into the block. There isn't really any good reason why we rip out parts of the block instead of storing a pointer to the block itself. Refactor the code to do so. Despite being simpler, it also allows us to decouple the lifetime of the block iterator from seeking in a subsequent commit. Signed-off-by: Patrick Steinhardt --- reftable/block.c | 22 ++++++++-------------- reftable/block.h | 4 +--- 2 files changed, 9 insertions(+), 17 deletions(-) diff --git a/reftable/block.c b/reftable/block.c index 844da397833..a425dd7abf3 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -381,13 +381,11 @@ static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx) return reftable_get_be24(b->block_data.data + b->restart_off + 3 * idx); } -void block_iter_seek_start(struct block_iter *it, const struct reftable_block *b) +void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block) { - it->block = b->block_data.data; - it->block_len = b->restart_off; - it->hash_size = b->hash_size; + it->block = block; reftable_buf_reset(&it->last_key); - it->next_off = b->header_off + 4; + it->next_off = block->header_off + 4; } struct restart_needle_less_args { @@ -435,14 +433,14 @@ static int restart_needle_less(size_t idx, void *_args) int block_iter_next(struct block_iter *it, struct reftable_record *rec) { struct string_view in = { - .buf = (unsigned char *) it->block + it->next_off, - .len = it->block_len - it->next_off, + .buf = (unsigned char *) it->block->block_data.data + it->next_off, + .len = it->block->restart_off - it->next_off, }; struct string_view start = in; uint8_t extra = 0; int n = 0; - if (it->next_off >= it->block_len) + if (it->next_off >= it->block->restart_off) return 1; n = reftable_decode_key(&it->last_key, &extra, in); @@ -452,7 +450,7 @@ int block_iter_next(struct block_iter *it, struct reftable_record *rec) return REFTABLE_FORMAT_ERROR; string_view_consume(&in, n); - n = reftable_record_decode(rec, it->last_key, extra, in, it->hash_size, + n = reftable_record_decode(rec, it->last_key, extra, in, it->block->hash_size, &it->scratch); if (n < 0) return -1; @@ -467,8 +465,6 @@ void block_iter_reset(struct block_iter *it) reftable_buf_reset(&it->last_key); it->next_off = 0; it->block = NULL; - it->block_len = 0; - it->hash_size = 0; } void block_iter_close(struct block_iter *it) @@ -528,9 +524,7 @@ int block_iter_seek_key(struct block_iter *it, const struct reftable_block *bloc it->next_off = block_restart_offset(block, i - 1); else it->next_off = block->header_off + 4; - it->block = block->block_data.data; - it->block_len = block->restart_off; - it->hash_size = block->hash_size; + it->block = block; err = reftable_record_init(&rec, reftable_block_type(block)); if (err < 0) diff --git a/reftable/block.h b/reftable/block.h index 4f7f29028c4..268d5a1e005 100644 --- a/reftable/block.h +++ b/reftable/block.h @@ -67,9 +67,7 @@ void block_writer_release(struct block_writer *bw); struct block_iter { /* offset within the block of the next entry to read. */ uint32_t next_off; - const unsigned char *block; - size_t block_len; - uint32_t hash_size; + const struct reftable_block *block; /* key for last entry we read. */ struct reftable_buf last_key; From patchwork Mon Apr 7 13:16:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040510 Received: from fout-a1-smtp.messagingengine.com (fout-a1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AEB6255232 for ; Mon, 7 Apr 2025 13:16:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031796; cv=none; b=O4ioGMCHtqTZnKj6XS476ZApRZYS9eRRA0lWmGLn0d/XNGU/+YmSDUav6LdB6rrZBtUSHfb3pBmDDyUB0APm0+SucHssq1gz+LZePlvc/cek8/xvYnkmneu+l6K7BUepnY5tFW6m3IGPPxTscMuY7yvU2B5TXrtNnVPkU1rRZJ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031796; c=relaxed/simple; bh=yyQH6SpjWFYD+V8dY3VsF1BIHOEX9Q4Mg6SE5HSa7ME=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=MkwGFQIaEn6bjmQzzaoG74bI0l9JW1DrY93+heXSnPWuy9s6U0qcYnQ6qO9nqFv7cFUa0uB0pSs0irixYvyE72PhA4rBFOFEBYi+Bw3+PjEWET71SximjOL46RPWBacXfMYZQXqnY9JOszlVIRRxpqQ6ElZ5guT5et+zvybDwtc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=c5igDEiY; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=F5T/S05J; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="c5igDEiY"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="F5T/S05J" Received: from phl-compute-07.internal (phl-compute-07.phl.internal [10.202.2.47]) by mailfout.phl.internal (Postfix) with ESMTP id A8953138032D; Mon, 7 Apr 2025 09:16:33 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-07.internal (MEProxy); Mon, 07 Apr 2025 09:16:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031793; x=1744118193; bh=DntH2Rgb9d9yQflW03bjRqU0/1TbH36xZ6b3TcB5Lv8=; b= c5igDEiY1+XQIpRfWk41X2pL05gh07Zp7rfPqMbPLBLonI1uiGJWCTEwsIhgHZAb xoNF6B7V4L7E1DxoSS4y63H64RMgwXeUQxVMP6jk0+WS4SpDGd5nQeCMtzFYYFV7 R1Hc2G2nDJqBDJ4shLrWzTscsxJh0R7tt+m5JjRLEoYhVWd+F6fOKTvy4aPibAaw NPmXJWT/d2cUHjo5JbRWFcqzy21iT6evrMWM7pWm7LL5CI+cnqGJuenPTWRJuAvK UKVzlHZ9Nfj0PZhsEsBN+dOGBZAh9jglCp8Ev852i2eyh8Dj1x8WQXkEAwB2IwxE QDGNqFWdWmyufIpVwkpR1A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031793; x= 1744118193; bh=DntH2Rgb9d9yQflW03bjRqU0/1TbH36xZ6b3TcB5Lv8=; b=F 5T/S05J34goCNu64kGiOikWpuHrD0ydNRaGH5UzEWSZHuPIFXcGodIC/U7KpZcLH BYyRmj6PDxZadiW7OW32IZB1rAKOcqUUdbbVYPzgooypxSEv2hCQ/ZMiurT31VSE xe2nB95YUbK2eEV084MVD0AjZcQNFvwf+goNs7KI2qaovED+BSJwy7AeCGwKgZuJ cdaBFpMWcWHKEVdezYdiOSzyysVdeXll+H0KYjjM3Ha7BDekFtxjr9CKfjZ5av92 ORhAE7TiNEhBj2uOXugeY7DsqlrDyQoCueNrzDYf5QHNgy9CHHW37ZOt3I/BtMFG 4Kpp0AkVYnnvkwYxTOArA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdekucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteek udehjeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmh epmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorh hgpdhrtghpthhtohepkhgrrhhthhhikhdrudekkeesghhmrghilhdrtghomhdprhgtphht thhopehgihhtshhtvghrsehpohgsohigrdgtohhmpdhrtghpthhtohepjhhlthhosghlvg hrsehgmhgrihhlrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:32 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id cedf12af (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:30 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:23 +0200 Subject: [PATCH v2 11/16] reftable/block: make block iterators reseekable Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-11-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 Refactor the block iterators so that initialization and seeking are different from one another. This makes the iterator trivially reseekable by storing the pointer to the block at initialization time, which we can then reuse on every seek. This refactoring prepares the code for exposing a `reftable_iterator` interface for blocks in a subsequent commit. Callsites are adjusted accordingly. Signed-off-by: Patrick Steinhardt --- reftable/block.c | 23 +++++++++++++---------- reftable/block.h | 21 ++++++++++++++++----- reftable/iter.c | 2 +- reftable/table.c | 11 +++++++---- t/unit-tests/t-reftable-block.c | 26 +++++++++++--------------- 5 files changed, 48 insertions(+), 35 deletions(-) diff --git a/reftable/block.c b/reftable/block.c index a425dd7abf3..20712be7eee 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -381,11 +381,16 @@ static uint32_t block_restart_offset(const struct reftable_block *b, size_t idx) return reftable_get_be24(b->block_data.data + b->restart_off + 3 * idx); } -void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block) +void block_iter_init(struct block_iter *it, const struct reftable_block *block) { it->block = block; + block_iter_seek_start(it); +} + +void block_iter_seek_start(struct block_iter *it) +{ reftable_buf_reset(&it->last_key); - it->next_off = block->header_off + 4; + it->next_off = it->block->header_off + 4; } struct restart_needle_less_args { @@ -473,12 +478,11 @@ void block_iter_close(struct block_iter *it) reftable_buf_release(&it->scratch); } -int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block, - struct reftable_buf *want) +int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want) { struct restart_needle_less_args args = { .needle = *want, - .block = block, + .block = it->block, }; struct reftable_record rec; int err = 0; @@ -496,7 +500,7 @@ int block_iter_seek_key(struct block_iter *it, const struct reftable_block *bloc * restart point. While that works alright, we would end up scanning * too many record. */ - i = binsearch(block->restart_count, &restart_needle_less, &args); + i = binsearch(it->block->restart_count, &restart_needle_less, &args); if (args.error) { err = REFTABLE_FORMAT_ERROR; goto done; @@ -521,12 +525,11 @@ int block_iter_seek_key(struct block_iter *it, const struct reftable_block *bloc * starting from the preceding restart point. */ if (i > 0) - it->next_off = block_restart_offset(block, i - 1); + it->next_off = block_restart_offset(it->block, i - 1); else - it->next_off = block->header_off + 4; - it->block = block; + it->next_off = it->block->header_off + 4; - err = reftable_record_init(&rec, reftable_block_type(block)); + err = reftable_record_init(&rec, reftable_block_type(it->block)); if (err < 0) goto done; diff --git a/reftable/block.h b/reftable/block.h index 268d5a1e005..1bfd44f56aa 100644 --- a/reftable/block.h +++ b/reftable/block.h @@ -79,12 +79,23 @@ struct block_iter { .scratch = REFTABLE_BUF_INIT, \ } -/* Position `it` at start of the block */ -void block_iter_seek_start(struct block_iter *it, const struct reftable_block *block); +/* + * Initialize the block iterator with the given block. The iterator will be + * positioned at the first record contained in the block. The block must remain + * valid until the end of the iterator's lifetime. It is valid to re-initialize + * iterators multiple times. + */ +void block_iter_init(struct block_iter *it, const struct reftable_block *block); + +/* Position the initialized iterator at the first record of its block. */ +void block_iter_seek_start(struct block_iter *it); -/* Position `it` to the `want` key in the block */ -int block_iter_seek_key(struct block_iter *it, const struct reftable_block *block, - struct reftable_buf *want); +/* + * Position the initialized iterator at the desired record key. It is not an + * error in case the record cannot be found. If so, a subsequent call to + * `block_iter_next()` will indicate that the iterator is exhausted. + */ +int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want); /* return < 0 for error, 0 for OK, > 0 for EOF. */ int block_iter_next(struct block_iter *it, struct reftable_record *rec); diff --git a/reftable/iter.c b/reftable/iter.c index 9ce675510c9..a79c90d9ba6 100644 --- a/reftable/iter.c +++ b/reftable/iter.c @@ -139,7 +139,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it) /* indexed block does not exist. */ return REFTABLE_FORMAT_ERROR; } - block_iter_seek_start(&it->cur, &it->block); + block_iter_init(&it->cur, &it->block); return 0; } diff --git a/reftable/table.c b/reftable/table.c index ef1f33c92fc..50ffad7edcb 100644 --- a/reftable/table.c +++ b/reftable/table.c @@ -208,7 +208,7 @@ static int table_iter_next_block(struct table_iter *ti) ti->block_off = next_block_off; ti->is_finished = 0; - block_iter_seek_start(&ti->bi, &ti->block); + block_iter_init(&ti->bi, &ti->block); return 0; } @@ -256,7 +256,7 @@ static int table_iter_seek_to(struct table_iter *ti, uint64_t off, uint8_t typ) ti->typ = reftable_block_type(&ti->block); ti->block_off = off; - block_iter_seek_start(&ti->bi, &ti->block); + block_iter_init(&ti->bi, &ti->block); ti->is_finished = 0; return 0; } @@ -349,7 +349,8 @@ static int table_iter_seek_linear(struct table_iter *ti, * the wanted key inside of it. If the block does not contain our key * we know that the corresponding record does not exist. */ - err = block_iter_seek_key(&ti->bi, &ti->block, &want_key); + block_iter_init(&ti->bi, &ti->block); + err = block_iter_seek_key(&ti->bi, &want_key); if (err < 0) goto done; err = 0; @@ -417,7 +418,9 @@ static int table_iter_seek_indexed(struct table_iter *ti, if (err != 0) goto done; - err = block_iter_seek_key(&ti->bi, &ti->block, &want_index.u.idx.last_key); + block_iter_init(&ti->bi, &ti->block); + + err = block_iter_seek_key(&ti->bi, &want_index.u.idx.last_key); if (err < 0) goto done; diff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c index e36ed7ac576..9f91d697f6d 100644 --- a/t/unit-tests/t-reftable-block.c +++ b/t/unit-tests/t-reftable-block.c @@ -66,7 +66,7 @@ static void t_ref_block_read_write(void) block_source_from_buf(&source ,&block_data); reftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); - block_iter_seek_start(&it, &block); + block_iter_init(&it, &block); for (i = 0; ; i++) { ret = block_iter_next(&it, &rec); @@ -79,10 +79,9 @@ static void t_ref_block_read_write(void) } for (i = 0; i < N; i++) { - block_iter_reset(&it); reftable_record_key(&recs[i], &want); - ret = block_iter_seek_key(&it, &block, &want); + ret = block_iter_seek_key(&it, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -91,7 +90,7 @@ static void t_ref_block_read_write(void) check(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1)); want.len--; - ret = block_iter_seek_key(&it, &block, &want); + ret = block_iter_seek_key(&it, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -156,7 +155,7 @@ static void t_log_block_read_write(void) block_source_from_buf(&source, &block_data); reftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); - block_iter_seek_start(&it, &block); + block_iter_init(&it, &block); for (i = 0; ; i++) { ret = block_iter_next(&it, &rec); @@ -169,11 +168,10 @@ static void t_log_block_read_write(void) } for (i = 0; i < N; i++) { - block_iter_reset(&it); reftable_buf_reset(&want); check(!reftable_buf_addstr(&want, recs[i].u.log.refname)); - ret = block_iter_seek_key(&it, &block, &want); + ret = block_iter_seek_key(&it, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -182,7 +180,7 @@ static void t_log_block_read_write(void) check(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1)); want.len--; - ret = block_iter_seek_key(&it, &block, &want); + ret = block_iter_seek_key(&it, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -249,7 +247,7 @@ static void t_obj_block_read_write(void) block_source_from_buf(&source, &block_data); reftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); - block_iter_seek_start(&it, &block); + block_iter_init(&it, &block); for (i = 0; ; i++) { ret = block_iter_next(&it, &rec); @@ -262,10 +260,9 @@ static void t_obj_block_read_write(void) } for (i = 0; i < N; i++) { - block_iter_reset(&it); reftable_record_key(&recs[i], &want); - ret = block_iter_seek_key(&it, &block, &want); + ret = block_iter_seek_key(&it, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -334,7 +331,7 @@ static void t_index_block_read_write(void) block_source_from_buf(&source, &block_data); reftable_block_init(&block, &source, 0, header_off, block_size, REFTABLE_HASH_SIZE_SHA1); - block_iter_seek_start(&it, &block); + block_iter_init(&it, &block); for (i = 0; ; i++) { ret = block_iter_next(&it, &rec); @@ -347,10 +344,9 @@ static void t_index_block_read_write(void) } for (i = 0; i < N; i++) { - block_iter_reset(&it); reftable_record_key(&recs[i], &want); - ret = block_iter_seek_key(&it, &block, &want); + ret = block_iter_seek_key(&it, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); @@ -359,7 +355,7 @@ static void t_index_block_read_write(void) check(reftable_record_equal(&recs[i], &rec, REFTABLE_HASH_SIZE_SHA1)); want.len--; - ret = block_iter_seek_key(&it, &block, &want); + ret = block_iter_seek_key(&it, &want); check_int(ret, ==, 0); ret = block_iter_next(&it, &rec); From patchwork Mon Apr 7 13:16:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040511 Received: from fout-a1-smtp.messagingengine.com (fout-a1-smtp.messagingengine.com [103.168.172.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA5B8255244 for ; Mon, 7 Apr 2025 13:16:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031797; cv=none; b=CcYfcUPO78XOpxqbtAZndy2+93geReUzOuVmsxTAncXeVmaowzF5ebO96qnHtKHFOivf69SLEuzwTMCLIpOrQSSlM8TB2OgkYiPgXIJROVxYv8SOyzPbuz235ESHcVKgUIhZdfUhVGM8eDxGHjiuZk3RnS/lksgiD27vCWymTE8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031797; c=relaxed/simple; bh=ka5i/d5p92Ar1syh3xWWJ548/SYGUxiOQytW1kxNRgM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=WmaeIEcmJ6IftkR3tCBWvqpuBTaR1NQAMJ+0JXwY5lt+T8MKy9leb1CHau84RQkLYN6pDqaTEEi07eObrGOEc6A23RhaCeju8Wf0fcFN0TWc0xjZnQkQNqKRzgHLUf83XhztXzkIIpW1v0/Do4kYuwTJ9tJ++uDTtd+6ORCrUUg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=nqWT1G8j; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=lQKDdGZP; arc=none smtp.client-ip=103.168.172.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="nqWT1G8j"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="lQKDdGZP" Received: from phl-compute-10.internal (phl-compute-10.phl.internal [10.202.2.50]) by mailfout.phl.internal (Postfix) with ESMTP id D0A7A1380228; Mon, 7 Apr 2025 09:16:34 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-10.internal (MEProxy); Mon, 07 Apr 2025 09:16:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031794; x=1744118194; bh=1rKZfBUcFZB2jL4FEPdoHeykQsUsGCiu+J7bJrelkTE=; b= nqWT1G8jkwstk8xcvSvJpOcRQNNaDN/5ZMUc0o6ymF9a2gx5LnB4nl3DvzstoGRD NdU3M2lDvlK5XH00zc6EDexbrFbM4o0z180+xrTXdFKYtxpfCBSrqt8zABDsGnXv ur+9NQZpQ5iNf5pj+Vlb3l0WAxj1x3X7kRhVONVnH/CIrnXwAL70nb+qlLKy5Ppn zuszNcd13tcmJ4sLYMB/N7xspJyMv/PUmB6tIqywtByDKBcbHpTtdnRVpVFedF6n rJYukfYxUQGekaEiuq3bLd8t/audzPnjhDYxfH17pzRBJ+HP1uLaPFjJnkGBS+7m 0F+5+gMCfKwIlzYmlhdq1g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031794; x= 1744118194; bh=1rKZfBUcFZB2jL4FEPdoHeykQsUsGCiu+J7bJrelkTE=; b=l QKDdGZPYA07PBjb4LwzjD/7ED+Wkz42jGEpKrSh2dKPsSTJ0Sdb7XOAmbonxH/JM GxmJj887kJQqHvqT2XI+OPCPrXm5xNdxm3mr38/14rg3U9QoPoZBlfUmqZZNFjez ZsJJSnAh3IY8envqSwsmH9/ppwIbzRpzZbxBYdzUkS/b/XIlJWds3KCOPWqRzJSP 4dzjzSRJeU8HFZZXsQ8cah0TYEIPnWXlZa4oAMBusO78LK934O/80poKR3n5Wikh +q7+6GshQR6YtgK+IFotiPZxZUMBZ9Wf0HAG3ZFqq9w3R7/m8fmhYjACfhLQ88eD 2C+R7uoTckeIyPgv883VA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdelucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteek udehjeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmh epmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorh hgpdhrtghpthhtohepkhgrrhhthhhikhdrudekkeesghhmrghilhdrtghomhdprhgtphht thhopehgihhtshhtvghrsehpohgsohigrdgtohhmpdhrtghpthhtohepjhhlthhosghlvg hrsehgmhgrihhlrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:33 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 5f0fdae0 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:31 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:24 +0200 Subject: [PATCH v2 12/16] reftable/block: expose a generic iterator over reftable records Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-12-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 Expose a generic iterator over reftable records and expose it via the public interface. Together with an upcoming iterator for reftable blocks contained in a table this will allow users to trivially iterate through blocks and their respective records individually. This functionality will be used to implement consistency checks for the reftable backend, which requires more fine-grained control over how we read data. Signed-off-by: Patrick Steinhardt --- reftable/block.c | 56 +++++++++++++++++++++++++++++ reftable/block.h | 2 +- reftable/reftable-block.h | 5 +++ t/unit-tests/t-reftable-block.c | 78 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 140 insertions(+), 1 deletion(-) diff --git a/reftable/block.c b/reftable/block.c index 20712be7eee..08e22170d56 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -10,6 +10,7 @@ #include "blocksource.h" #include "constants.h" +#include "iter.h" #include "record.h" #include "reftable-error.h" #include "system.h" @@ -581,6 +582,61 @@ int block_iter_seek_key(struct block_iter *it, struct reftable_buf *want) return err; } +static int block_iter_seek_void(void *it, struct reftable_record *want) +{ + struct reftable_buf buf = REFTABLE_BUF_INIT; + struct block_iter *bi = it; + int err; + + if (bi->block->block_type != want->type) + return REFTABLE_API_ERROR; + + err = reftable_record_key(want, &buf); + if (err < 0) + goto out; + + err = block_iter_seek_key(it, &buf); + if (err < 0) + goto out; + + err = 0; + +out: + reftable_buf_release(&buf); + return err; +} + +static int block_iter_next_void(void *it, struct reftable_record *rec) +{ + return block_iter_next(it, rec); +} + +static void block_iter_close_void(void *it) +{ + block_iter_close(it); +} + +static struct reftable_iterator_vtable block_iter_vtable = { + .seek = &block_iter_seek_void, + .next = &block_iter_next_void, + .close = &block_iter_close_void, +}; + +int reftable_block_init_iterator(const struct reftable_block *b, + struct reftable_iterator *it) +{ + struct block_iter *bi; + + REFTABLE_CALLOC_ARRAY(bi, 1); + block_iter_init(bi, b); + + assert(!it->ops); + it->iter_arg = bi; + it->ops = &block_iter_vtable; + + return 0; +} + void block_writer_release(struct block_writer *bw) { deflateEnd(bw->zstream); diff --git a/reftable/block.h b/reftable/block.h index 1bfd44f56aa..3d069393922 100644 --- a/reftable/block.h +++ b/reftable/block.h @@ -63,7 +63,7 @@ int block_writer_finish(struct block_writer *w); /* clears out internally allocated block_writer members. */ void block_writer_release(struct block_writer *bw); -/* Iterate over entries in a block */ +/* Iterator for records contained in a single block. */ struct block_iter { /* offset within the block of the next entry to read. */ uint32_t next_off; diff --git a/reftable/reftable-block.h b/reftable/reftable-block.h index 13bd68be8c5..04c3b518c87 100644 --- a/reftable/reftable-block.h +++ b/reftable/reftable-block.h @@ -13,6 +13,7 @@ #include "reftable-basics.h" #include "reftable-blocksource.h" +#include "reftable-iterator.h" struct z_stream_s; @@ -60,6 +61,10 @@ int reftable_block_init(struct reftable_block *b, /* Release resources allocated by the block. */ void reftable_block_release(struct reftable_block *b); +/* Initialize a generic record iterator from the given block. */ +int reftable_block_init_iterator(const struct reftable_block *b, + struct reftable_iterator *it); + /* Returns the block type (eg. 'r' for refs). */ uint8_t reftable_block_type(const struct reftable_block *b); diff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c index 9f91d697f6d..5577a5769ed 100644 --- a/t/unit-tests/t-reftable-block.c +++ b/t/unit-tests/t-reftable-block.c @@ -372,12 +372,90 @@ static void t_index_block_read_write(void) reftable_record_release(&recs[i]); } +static void t_block_iterator(void) +{ + struct reftable_block_source source = { 0 }; + struct block_writer writer = { + .last_key = REFTABLE_BUF_INIT, + }; + struct reftable_record expected_refs[20]; + struct reftable_ref_record ref = { 0 }; + struct reftable_iterator it = { 0 }; + struct reftable_block block = { 0 }; + struct reftable_buf data; + int err; + + data.len = 1024; + REFTABLE_CALLOC_ARRAY(data.buf, data.len); + check(data.buf != NULL); + + err = block_writer_init(&writer, BLOCK_TYPE_REF, (uint8_t *) data.buf, data.len, + 0, hash_size(REFTABLE_HASH_SHA1)); + check(!err); + + for (size_t i = 0; i < ARRAY_SIZE(expected_refs); i++) { + expected_refs[i] = (struct reftable_record) { + .type = BLOCK_TYPE_REF, + .u.ref = { + .value_type = REFTABLE_REF_VAL1, + .refname = xstrfmt("refs/heads/branch-%02"PRIuMAX, (uintmax_t)i), + }, + }; + memset(expected_refs[i].u.ref.value.val1, i, REFTABLE_HASH_SIZE_SHA1); + + err = block_writer_add(&writer, &expected_refs[i]); + check_int(err, ==, 0); + } + + err = block_writer_finish(&writer); + check_int(err, >, 0); + + block_source_from_buf(&source, &data); + reftable_block_init(&block, &source, 0, 0, data.len, REFTABLE_HASH_SIZE_SHA1); + + err = reftable_block_init_iterator(&block, &it); + check_int(err, ==, 0); + + for (size_t i = 0; ; i++) { + err = reftable_iterator_next_ref(&it, &ref); + if (err > 0) { + check_int(i, ==, ARRAY_SIZE(expected_refs)); + break; + } + check_int(err, ==, 0); + + check(reftable_ref_record_equal(&ref, &expected_refs[i].u.ref, + REFTABLE_HASH_SIZE_SHA1)); + } + + err = reftable_iterator_seek_ref(&it, "refs/heads/does-not-exist"); + check_int(err, ==, 0); + err = reftable_iterator_next_ref(&it, &ref); + check_int(err, ==, 1); + + err = reftable_iterator_seek_ref(&it, "refs/heads/branch-13"); + check_int(err, ==, 0); + err = reftable_iterator_next_ref(&it, &ref); + check_int(err, ==, 0); + check(reftable_ref_record_equal(&ref, &expected_refs[13].u.ref, + REFTABLE_HASH_SIZE_SHA1)); + + for (size_t i = 0; i < ARRAY_SIZE(expected_refs); i++) + reftable_free(expected_refs[i].u.ref.refname); + reftable_ref_record_release(&ref); + reftable_iterator_destroy(&it); + reftable_block_release(&block); + block_writer_release(&writer); + reftable_buf_release(&data); +} + int cmd_main(int argc UNUSED, const char *argv[] UNUSED) { TEST(t_index_block_read_write(), "read-write operations on index blocks work"); TEST(t_log_block_read_write(), "read-write operations on log blocks work"); TEST(t_obj_block_read_write(), "read-write operations on obj blocks work"); TEST(t_ref_block_read_write(), "read-write operations on ref blocks work"); + TEST(t_block_iterator(), "block iterator works"); return test_done(); } From patchwork Mon Apr 7 13:16:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040512 Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91B36255248 for ; Mon, 7 Apr 2025 13:16:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031798; cv=none; b=cw1BxIhnN/OOJk9C/LJyPmfoIHaU9gxNHNwt+QegwsfLNVqDJWH+yDmqnaD+1OHTj/rgO/u4JQb9YHS/vrBKWnZ4WioGNSEkZ4IavgNck0R+xKxWnuQgKYz+Gizjyykmx3OYsQUZwuaPzMK6KL4EnfisbuuIO0rcJiQtB02xpGw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031798; c=relaxed/simple; bh=l6WJnQBcNmLkTNNSjhjsA7B104zL3JpMeTI7hOFanUk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=gfDTA+UyqEyPvVUz4JxCX+XxLqsJYk2RMR7eNgk0hpuCbd3G2eqMLCGtK8348UeLETC+bPD6bh+C21Q0SfWLVQiwvysnISdShVNhFrCU8U2ONCI5lBuEIv1Z7ocpQlCWr7/O8ocVDgT4U7HoTfU6TNcywluOCC8uI2sEQVAngbk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=AFi3R9bE; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=UlgWB7XP; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="AFi3R9bE"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="UlgWB7XP" Received: from phl-compute-12.internal (phl-compute-12.phl.internal [10.202.2.52]) by mailfhigh.phl.internal (Postfix) with ESMTP id 047C211401D2; Mon, 7 Apr 2025 09:16:36 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-12.internal (MEProxy); Mon, 07 Apr 2025 09:16:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031796; x=1744118196; bh=d5gUcCMqkjk2qFZG6aNfmROI9AnThXvIZvc0vtu75ZI=; b= AFi3R9bEVQ6HNQ/b35MzDEolcFKurokSvuaS48Qy5ZAbUObN07YNDWAvjsoM5duJ Dh/GyUpRZdbsTTAQiNEt+YkYMh/MgDpo63bqg0aFI95DtU3UA32hFyc/8RvUh9Tz rMUokbAXaVbhGi0P7RY2hAu0VlhlUN00NIZ1BUVT7eYUxukrCjHc+yLRX2jU/s7d hT9tDstm2die+P1ncNVePhW6UYxP7QI0L4R+tAI16m2YpQxKKcf9Gj9vCefkrRHP 5zbG8jhatPKdjJ5e8FYyjUJom+j5p9Ai0IPSKidXCCi3zZj2zZ2vUYnSc7kHHwv4 oCoHZtuP+q2M0hsQeunZ+A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031796; x= 1744118196; bh=d5gUcCMqkjk2qFZG6aNfmROI9AnThXvIZvc0vtu75ZI=; b=U lgWB7XPPNRR7cKqH+yOxISA7wWj/QE0cf1NxCw2SQ+t6ToONDMzNW2EMZH9wUBlN 0iT7TeZrDT58JbIVdek7dtVRBokKnfq7k7lDJkoNPzK9HxlWdZZHWt8ztPhtubnz /o/4vYWLszJXTnZM5sWamJ96jwSEm2eF6hJnGuHcStHBS4St4rxYZo6n1HdgngTi d1b5COqrcgaDYfaBvJghTe5Lm9OMgoJIxaQFRp9aCMla/VtgkwUAzBU/uKYkHWmz xwq5pXJMfSA/KNeEf+7l7CohAK55Nn/DxodV5jYVo0b+3A5LD1dagVNgpytJjqOb 2tL3yMVXP8mNonX8wmqBA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdekucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteek udehjeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmh epmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhmpd hrtghpthhtohepkhgrrhhthhhikhdrudekkeesghhmrghilhdrtghomhdprhgtphhtthho pehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepjhhlthhosghlvg hrsehgmhgrihhlrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:34 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id a3fa9e5f (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:32 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:25 +0200 Subject: [PATCH v2 13/16] reftable/table: add `reftable_table` to the public interface Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-13-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 The `reftable_table` interface is an internal implementation detail that callers have no access to. Having direct access to this structure is important though for a subsequent patch series that will implement consistency checks for the reftable backend. Move the structure into "reftable-table.h" so that it part of the public interface. Signed-off-by: Patrick Steinhardt --- reftable/reftable-table.h | 32 +++++++++++++++++++++++++++++++- reftable/table.h | 33 --------------------------------- 2 files changed, 31 insertions(+), 34 deletions(-) diff --git a/reftable/reftable-table.h b/reftable/reftable-table.h index 94379026727..a78db9eea7e 100644 --- a/reftable/reftable-table.h +++ b/reftable/reftable-table.h @@ -20,8 +20,38 @@ * reftable_merged_table and struct reftable_stack. */ +/* Metadata for a block type. */ +struct reftable_table_offsets { + int is_present; + uint64_t offset; + uint64_t index_offset; +}; + /* The table struct is a handle to an open reftable file. */ -struct reftable_table; +struct reftable_table { + /* for convenience, associate a name with the instance. */ + char *name; + struct reftable_block_source source; + + /* Size of the file, excluding the footer. */ + uint64_t size; + + /* The hash function used for ref records. */ + enum reftable_hash hash_id; + + uint32_t block_size; + uint64_t min_update_index; + uint64_t max_update_index; + /* Length of the OID keys in the 'o' section */ + int object_id_len; + int version; + + struct reftable_table_offsets ref_offsets; + struct reftable_table_offsets obj_offsets; + struct reftable_table_offsets log_offsets; + + uint64_t refcount; +}; /* reftable_table_new opens a reftable for reading. If successful, * returns 0 code and sets pp. The name is used for creating a diff --git a/reftable/table.h b/reftable/table.h index e15d58d8e9e..c54703e6210 100644 --- a/reftable/table.h +++ b/reftable/table.h @@ -14,39 +14,6 @@ #include "reftable-iterator.h" #include "reftable-table.h" -/* metadata for a block type */ -struct reftable_table_offsets { - int is_present; - uint64_t offset; - uint64_t index_offset; -}; - -/* The state for reading a reftable file. */ -struct reftable_table { - /* for convenience, associate a name with the instance. */ - char *name; - struct reftable_block_source source; - - /* Size of the file, excluding the footer. */ - uint64_t size; - - /* The hash function used for ref records. */ - enum reftable_hash hash_id; - - uint32_t block_size; - uint64_t min_update_index; - uint64_t max_update_index; - /* Length of the OID keys in the 'o' section */ - int object_id_len; - int version; - - struct reftable_table_offsets ref_offsets; - struct reftable_table_offsets obj_offsets; - struct reftable_table_offsets log_offsets; - - uint64_t refcount; -}; - const char *reftable_table_name(struct reftable_table *t); int table_init_iter(struct reftable_table *t, From patchwork Mon Apr 7 13:16:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040513 Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0A10255259 for ; Mon, 7 Apr 2025 13:16:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031799; cv=none; b=QwgpbZD12JDkZRsoG2EJpdD8FCRh3ntAqOtCjwESPOINm6OtQlubxv917Yll2i/2dV92Rc7NLrXnT88p4x8+LEK87CZCrwOlLKQb4Wy5HG0dLys8lSzE7pCiRGuW36XDaOQHOW8XeQ9GOAjjgilszUe5ATde6Dsx+03VwJwNHRM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031799; c=relaxed/simple; bh=IOIqBBW4tLu63BUvggvZJe8cQW9+9RnwA8bzlcFeW9E=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=sMxvEg602zTI2ryvj6dMRe9ValAWuIHDBU0pqZ5OeM13BiHBua6xf0TeAG55DvenR3QbbnZ7PE+UHJwFP21tjRKgk87fz+38zVweHA1C9+RDenhS4/QMr9WTwnbXb9RIjPFD2pGwgQ97+WxSvJ6/xp3UXUFE8xWpqBQFXX2fVAQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=mxhhMlkm; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=ePKhUnqW; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="mxhhMlkm"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="ePKhUnqW" Received: from phl-compute-11.internal (phl-compute-11.phl.internal [10.202.2.51]) by mailfhigh.phl.internal (Postfix) with ESMTP id 35C99114021F; Mon, 7 Apr 2025 09:16:37 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-11.internal (MEProxy); Mon, 07 Apr 2025 09:16:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031797; x=1744118197; bh=xIXToqpfHdZqMhx9+OhI5WhlGTDm87psX3u4TMU4iSk=; b= mxhhMlkm8eWaB30B7dhMqlPJLEXUgOoZfZ6SeAlxEUyVXWjGA9RNGVq8igB9Z+c6 cKEya0KEqtWznmm7MZP269cJB3vY56o9eM5UrkJAtZaEfsicJ8S1hlELTJ7U5wdW Ny3lOvURY8dMYq18JFz+9Cj6zmadmcnz1dg9pKu7W1t6tiZ576/aXGgfhPSkJMES uRNqt83uwU9xppy14sQsX2LEcL392V+KnjIpuSGM2m9ebjqul0z6xNrWle9KzY1o RFZDFEgsYvmaJHFRvPgLZViuWYoO0uaikj2e/a5RcN6sK8wow9N1ewEdMD9CJt+H 0IN66qYcy1OhuHVZBDz30g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031797; x= 1744118197; bh=xIXToqpfHdZqMhx9+OhI5WhlGTDm87psX3u4TMU4iSk=; b=e PKhUnqWLXgOFfEdFZImgB7YGzsETv6rbQKTP6ht3+DV+cy3bG7n17tU2zlTX2j1M 2UYjp6S3OB267cF6clFxa4m4vILSjPcUo3NcgzEmvDw6fdF6jNqi0IvQXaNXT5g+ HQgVvYol+25HViW8QNjVo/UgxM8SRRZz4w5VMDQjPQQ3EMKMz2mjCZBkexcfUkEl blvF2bFbJuN12DPixNdi1hOdhKsUEwpsLjXd6HNYGXgaTPSDvqI57XqPI7sl4Zqm LNzrs8HrFP8EH73uaTObX4eRKA6goLETgn7lsmHfi6F0682d9aBoGuH5/TKXF8oD G4U84qW9bLQ1fPpb6XZFw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdekucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteek udehjeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmh epmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorh hgpdhrtghpthhtohepkhgrrhhthhhikhdrudekkeesghhmrghilhdrtghomhdprhgtphht thhopehjlhhtohgslhgvrhesghhmrghilhdrtghomhdprhgtphhtthhopehgihhtshhtvg hrsehpohgsohigrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:36 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 12f7b3eb (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:33 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:26 +0200 Subject: [PATCH v2 14/16] reftable/table: introduce iterator for table blocks Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-14-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 Introduce a new iterator that allows the caller to iterate through all blocks contained in a table. This gives users more fine-grained control over how exactly those blocks are being read and exposes information to callers that was previously inaccessible. This iterator will be required by a future patch series that adds consistency checks for the reftable backend. In addition to that though we will also reimplement `reftable_table_print_blocks()` on top of this new iterator in a subsequent commit. Signed-off-by: Patrick Steinhardt --- reftable/reftable-table.h | 16 ++++++ reftable/table.c | 47 +++++++++++++++++ t/unit-tests/t-reftable-table.c | 110 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 173 insertions(+) diff --git a/reftable/reftable-table.h b/reftable/reftable-table.h index a78db9eea7e..4d61ce6cf77 100644 --- a/reftable/reftable-table.h +++ b/reftable/reftable-table.h @@ -10,6 +10,7 @@ #define REFTABLE_TABLE_H #include "reftable-iterator.h" +#include "reftable-block.h" #include "reftable-blocksource.h" /* @@ -99,4 +100,19 @@ uint64_t reftable_table_min_update_index(struct reftable_table *t); /* print blocks onto stdout for debugging. */ int reftable_table_print_blocks(const char *tablename); +/* + * An iterator that iterates through the blocks contained in a given table. + */ +struct reftable_table_iterator { + void *iter_arg; +}; + +int reftable_table_iterator_init(struct reftable_table_iterator *it, + struct reftable_table *t); + +void reftable_table_iterator_release(struct reftable_table_iterator *it); + +int reftable_table_iterator_next(struct reftable_table_iterator *it, + const struct reftable_block **out); + #endif diff --git a/reftable/table.c b/reftable/table.c index 50ffad7edcb..a2a0c7a1d2f 100644 --- a/reftable/table.c +++ b/reftable/table.c @@ -804,3 +804,50 @@ int reftable_table_print_blocks(const char *tablename) table_iter_close(&ti); return err; } + +int reftable_table_iterator_init(struct reftable_table_iterator *it, + struct reftable_table *t) +{ + struct table_iter *ti; + int err; + + REFTABLE_ALLOC_ARRAY(ti, 1); + if (!ti) + return REFTABLE_OUT_OF_MEMORY_ERROR; + + err = table_iter_init(ti, t); + if (err < 0) + goto out; + + it->iter_arg = ti; + err = 0; + +out: + if (err < 0) + reftable_free(ti); + return err; +} + +void reftable_table_iterator_release(struct reftable_table_iterator *it) +{ + if (!it->iter_arg) + return; + table_iter_close(it->iter_arg); + reftable_free(it->iter_arg); + it->iter_arg = NULL; +} + +int reftable_table_iterator_next(struct reftable_table_iterator *it, + const struct reftable_block **out) +{ + struct table_iter *ti = it->iter_arg; + int err; + + err = table_iter_next_block(ti); + if (err) + return err; + + *out = &ti->block; + + return 0; +} diff --git a/t/unit-tests/t-reftable-table.c b/t/unit-tests/t-reftable-table.c index 77c59dbf46d..ba39cdf9a76 100644 --- a/t/unit-tests/t-reftable-table.c +++ b/t/unit-tests/t-reftable-table.c @@ -1,7 +1,10 @@ #include "test-lib.h" #include "lib-reftable.h" #include "reftable/blocksource.h" +#include "reftable/constants.h" +#include "reftable/iter.h" #include "reftable/table.h" +#include "strbuf.h" static int t_table_seek_once(void) { @@ -88,9 +91,116 @@ static int t_table_reseek(void) return 0; } +static int t_table_block_iterator(void) +{ + struct reftable_block_source source = { 0 }; + struct reftable_table_iterator it = { 0 }; + struct reftable_ref_record *records; + const struct reftable_block *block; + struct reftable_table *table; + struct reftable_buf buf = REFTABLE_BUF_INIT; + struct { + uint8_t block_type; + uint16_t header_off; + uint16_t restart_count; + uint16_t record_count; + } expected_blocks[] = { + { + .block_type = BLOCK_TYPE_REF, + .header_off = 24, + .restart_count = 10, + .record_count = 158, + }, + { + .block_type = BLOCK_TYPE_REF, + .restart_count = 10, + .record_count = 159, + }, + { + .block_type = BLOCK_TYPE_REF, + .restart_count = 10, + .record_count = 159, + }, + { + .block_type = BLOCK_TYPE_REF, + .restart_count = 2, + .record_count = 24, + }, + { + .block_type = BLOCK_TYPE_INDEX, + .restart_count = 1, + .record_count = 4, + }, + { + .block_type = BLOCK_TYPE_OBJ, + .restart_count = 1, + .record_count = 1, + }, + }; + const size_t nrecords = 500; + int ret; + + REFTABLE_CALLOC_ARRAY(records, nrecords); + for (size_t i = 0; i < nrecords; i++) { + records[i].value_type = REFTABLE_REF_VAL1; + records[i].refname = xstrfmt("refs/heads/branch-%03"PRIuMAX, + (uintmax_t) i); + } + + t_reftable_write_to_buf(&buf, records, nrecords, NULL, 0, NULL); + block_source_from_buf(&source, &buf); + + ret = reftable_table_new(&table, &source, "name"); + check(!ret); + + ret = reftable_table_iterator_init(&it, table); + check(!ret); + + for (size_t i = 0; i < ARRAY_SIZE(expected_blocks); i++) { + struct reftable_iterator record_it = { 0 }; + struct reftable_record record = { + .type = expected_blocks[i].block_type, + }; + + ret = reftable_table_iterator_next(&it, &block); + check(!ret); + + check_int(block->block_type, ==, expected_blocks[i].block_type); + check_int(block->header_off, ==, expected_blocks[i].header_off); + check_int(block->restart_count, ==, expected_blocks[i].restart_count); + + ret = reftable_block_init_iterator(block, &record_it); + check(!ret); + + for (size_t j = 0; ; j++) { + ret = iterator_next(&record_it, &record); + if (ret > 0) { + check_int(j, ==, expected_blocks[i].record_count); + break; + } + check(!ret); + } + + reftable_iterator_destroy(&record_it); + reftable_record_release(&record); + } + + ret = reftable_table_iterator_next(&it, &block); + check_int(ret, ==, 1); + + for (size_t i = 0; i < nrecords; i++) + reftable_free(records[i].refname); + reftable_table_iterator_release(&it); + reftable_table_decref(table); + reftable_buf_release(&buf); + reftable_free(records); + return 0; +} + int cmd_main(int argc UNUSED, const char *argv[] UNUSED) { TEST(t_table_seek_once(), "table can seek once"); TEST(t_table_reseek(), "table can reseek multiple times"); + TEST(t_table_block_iterator(), "table can iterate through blocks"); return test_done(); } From patchwork Mon Apr 7 13:16:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040514 Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6969F255E2B for ; Mon, 7 Apr 2025 13:16:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031802; cv=none; b=RkzktpQ6gEH7zPwqcJjT8L04JnbnOxhM9G1iq5pfO6wKRngSpBNsXfKXu/eG4F2Z1o15cDHMzNqfHjhbevZDAlPb2It0tSxtNaW1e5DdSI/kVo3C7y0R7ZU9dSJZC2gs+459bWzJIVWtz3MXCamCwZ8iDQ3M8DUUIsI5Jp3IWAc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031802; c=relaxed/simple; bh=flbtjHC9Ql6egAB5eAwjPCaCWTIFmyCTqp857THY7dA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=esgSWFqqazlXZTdA9y7hupGfFeQw9UwAsctCjyg3ZCnP8OAcFA4SwU23ZUfOVgyV8QJmb/m94VaxmkS8Idb8NbZeNvl2QlVDOeesU9wSLpPqcWIwSkOXbb5Q1aBK2AheKGc25G7Yvdch2YQ6mdZYM+toPCnE+usY6CHuj/EiZ44= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=ejoVIUQe; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=FPnNfTGn; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="ejoVIUQe"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="FPnNfTGn" Received: from phl-compute-08.internal (phl-compute-08.phl.internal [10.202.2.48]) by mailfhigh.phl.internal (Postfix) with ESMTP id 76D3C11401D2; Mon, 7 Apr 2025 09:16:38 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-08.internal (MEProxy); Mon, 07 Apr 2025 09:16:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031798; x=1744118198; bh=aqmRiLFExgPp7/Dh1zy0pQkua+d6hzjKihhk41K2vlU=; b= ejoVIUQeh7fr/UZd0ZPXmxbV0jfghDzwNWYpWFkz/tnPkI4/VWrzbfLOrgUxFWvT oymgyePNYImmFX0h5QiAWDi4iOGxsyTQN7jOHfV3izyDVTGm90RSsjJcUygGgdZv yfeFH3eZxWzzXR6om7/BNrn12JtKjB8M8tYollte/riXu4SpU3qkI6SoXl4gQs2Y r0yO60TD2FutRJ6Mlb3LZ85WmfSHT1xrgWFd5tBhrDuMKtoc7L91ovg255mQZpvh Y1YTwuPK1JjOvCSFBQ+Qj/ZG+8IQHkKgk3AMj/GTZm4TIGlBL1XD97Vb/qvIgtWa ctjC79etaLnz7eNuM/hQjw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031798; x= 1744118198; bh=aqmRiLFExgPp7/Dh1zy0pQkua+d6hzjKihhk41K2vlU=; b=F PnNfTGntkuPPSdDGYEhuu2z2oDxUOilMkVyRBC1GWRIGr9Tx+LfBCc9gi/3zNuvb MzBcsQeO70agBx7kWlYV4cJwTmaSG4du+j6otvdOneaYLgwunZXxLwu5Yn/jhAVP n9iUha1oiSRjkg4/PB6AjXRdnD9JHyXPjs+Ara1OpKlzFhxufRvZ/IZ2Q+1M4Mmw dHKRMza/zhxOa6LbZLxKyyph4yOCsFum6BDFTnuSvLcJzo64qhTrg1JEHWdwWJ6b 5BxeN5Dooj6grWNE4dZA7b9RykWT+K0HdXPXQyT7BWhokEdW3ZB5HElnFoSeZPxc P9R1Rpyevec97MaviN2XA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdekucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpedthfegfeejuefgieffvefhvedvfedtffdtgffh heevgfdtjedukefgkefgfeevtdenucffohhmrghinhepghhoohhglhgvrdgtohhmnecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhs rdhimhdpnhgspghrtghpthhtohepgedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtoh epkhgrrhhthhhikhdrudekkeesghhmrghilhdrtghomhdprhgtphhtthhopehgihhtshht vghrsehpohgsohigrdgtohhmpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlh drohhrghdprhgtphhtthhopehjlhhtohgslhgvrhesghhmrghilhdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:37 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 273603ad (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:34 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:27 +0200 Subject: [PATCH v2 15/16] reftable/constants: make block types part of the public interface Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-15-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 Now that reftable blocks can be read individually via the public interface it becomes necessary for callers to be able to distinguish the different types of blocks. Expose the relevant constants. Signed-off-by: Patrick Steinhardt --- reftable/block.c | 4 ++-- reftable/constants.h | 6 +----- reftable/iter.c | 10 +++++----- reftable/merged.c | 4 ++-- reftable/record.c | 40 ++++++++++++++++++++-------------------- reftable/reftable-constants.h | 18 ++++++++++++++++++ reftable/stack.c | 8 ++++---- reftable/table.c | 40 ++++++++++++++++++++-------------------- reftable/writer.c | 22 +++++++++++----------- t/unit-tests/t-reftable-block.c | 22 +++++++++++----------- t/unit-tests/t-reftable-merged.c | 12 ++++++------ t/unit-tests/t-reftable-pq.c | 10 +++++----- t/unit-tests/t-reftable-record.c | 40 ++++++++++++++++++++-------------------- t/unit-tests/t-reftable-table.c | 12 ++++++------ 14 files changed, 131 insertions(+), 117 deletions(-) diff --git a/reftable/block.c b/reftable/block.c index 08e22170d56..795815b4762 100644 --- a/reftable/block.c +++ b/reftable/block.c @@ -160,7 +160,7 @@ int block_writer_finish(struct block_writer *w) * Log records are stored zlib-compressed. Note that the compression * also spans over the restart points we have just written. */ - if (block_writer_type(w) == BLOCK_TYPE_LOG) { + if (block_writer_type(w) == REFTABLE_BLOCK_TYPE_LOG) { int block_header_skip = 4 + w->header_off; uLongf src_len = w->next - block_header_skip, compressed_len; int ret; @@ -254,7 +254,7 @@ int reftable_block_init(struct reftable_block *block, goto done; } - if (block_type == BLOCK_TYPE_LOG) { + if (block_type == REFTABLE_BLOCK_TYPE_LOG) { uint32_t block_header_skip = 4 + header_size; uLong dst_len = block_size - block_header_skip; uLong src_len = block->block_data.len - block_header_skip; diff --git a/reftable/constants.h b/reftable/constants.h index 091728cf033..e3b1aaa5164 100644 --- a/reftable/constants.h +++ b/reftable/constants.h @@ -9,11 +9,7 @@ #ifndef CONSTANTS_H #define CONSTANTS_H -#define BLOCK_TYPE_LOG 'g' -#define BLOCK_TYPE_INDEX 'i' -#define BLOCK_TYPE_REF 'r' -#define BLOCK_TYPE_OBJ 'o' -#define BLOCK_TYPE_ANY 0 +#include "reftable-constants.h" #define MAX_RESTARTS ((1 << 16) - 1) #define DEFAULT_BLOCK_SIZE 4096 diff --git a/reftable/iter.c b/reftable/iter.c index a79c90d9ba6..2ecc52b3366 100644 --- a/reftable/iter.c +++ b/reftable/iter.c @@ -131,7 +131,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it) block_source_release_data(&it->block.block_data); off = it->offsets[it->offset_idx++]; - err = table_init_block(it->table, &it->block, off, BLOCK_TYPE_REF); + err = table_init_block(it->table, &it->block, off, REFTABLE_BLOCK_TYPE_REF); if (err < 0) { return err; } @@ -246,7 +246,7 @@ int reftable_iterator_seek_ref(struct reftable_iterator *it, const char *name) { struct reftable_record want = { - .type = BLOCK_TYPE_REF, + .type = REFTABLE_BLOCK_TYPE_REF, .u.ref = { .refname = (char *)name, }, @@ -258,7 +258,7 @@ int reftable_iterator_next_ref(struct reftable_iterator *it, struct reftable_ref_record *ref) { struct reftable_record rec = { - .type = BLOCK_TYPE_REF, + .type = REFTABLE_BLOCK_TYPE_REF, .u = { .ref = *ref }, @@ -272,7 +272,7 @@ int reftable_iterator_seek_log_at(struct reftable_iterator *it, const char *name, uint64_t update_index) { struct reftable_record want = { - .type = BLOCK_TYPE_LOG, + .type = REFTABLE_BLOCK_TYPE_LOG, .u.log = { .refname = (char *)name, .update_index = update_index, @@ -291,7 +291,7 @@ int reftable_iterator_next_log(struct reftable_iterator *it, struct reftable_log_record *log) { struct reftable_record rec = { - .type = BLOCK_TYPE_LOG, + .type = REFTABLE_BLOCK_TYPE_LOG, .u = { .log = *log, }, diff --git a/reftable/merged.c b/reftable/merged.c index d5b974d660e..733de07454d 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -301,13 +301,13 @@ int merged_table_init_iter(struct reftable_merged_table *mt, int reftable_merged_table_init_ref_iterator(struct reftable_merged_table *mt, struct reftable_iterator *it) { - return merged_table_init_iter(mt, it, BLOCK_TYPE_REF); + return merged_table_init_iter(mt, it, REFTABLE_BLOCK_TYPE_REF); } int reftable_merged_table_init_log_iterator(struct reftable_merged_table *mt, struct reftable_iterator *it) { - return merged_table_init_iter(mt, it, BLOCK_TYPE_LOG); + return merged_table_init_iter(mt, it, REFTABLE_BLOCK_TYPE_LOG); } enum reftable_hash reftable_merged_table_hash_id(struct reftable_merged_table *mt) diff --git a/reftable/record.c b/reftable/record.c index 26cd834d405..ed00a724417 100644 --- a/reftable/record.c +++ b/reftable/record.c @@ -69,10 +69,10 @@ int put_var_int(struct string_view *dest, uint64_t value) int reftable_is_block_type(uint8_t typ) { switch (typ) { - case BLOCK_TYPE_REF: - case BLOCK_TYPE_LOG: - case BLOCK_TYPE_OBJ: - case BLOCK_TYPE_INDEX: + case REFTABLE_BLOCK_TYPE_REF: + case REFTABLE_BLOCK_TYPE_LOG: + case REFTABLE_BLOCK_TYPE_OBJ: + case REFTABLE_BLOCK_TYPE_INDEX: return 1; } return 0; @@ -462,7 +462,7 @@ static int reftable_ref_record_cmp_void(const void *_a, const void *_b) static struct reftable_record_vtable reftable_ref_record_vtable = { .key = &reftable_ref_record_key, - .type = BLOCK_TYPE_REF, + .type = REFTABLE_BLOCK_TYPE_REF, .copy_from = &reftable_ref_record_copy_from, .val_type = &reftable_ref_record_val_type, .encode = &reftable_ref_record_encode, @@ -664,7 +664,7 @@ static int reftable_obj_record_cmp_void(const void *_a, const void *_b) static struct reftable_record_vtable reftable_obj_record_vtable = { .key = &reftable_obj_record_key, - .type = BLOCK_TYPE_OBJ, + .type = REFTABLE_BLOCK_TYPE_OBJ, .copy_from = &reftable_obj_record_copy_from, .val_type = &reftable_obj_record_val_type, .encode = &reftable_obj_record_encode, @@ -1035,7 +1035,7 @@ static int reftable_log_record_is_deletion_void(const void *p) static struct reftable_record_vtable reftable_log_record_vtable = { .key = &reftable_log_record_key, - .type = BLOCK_TYPE_LOG, + .type = REFTABLE_BLOCK_TYPE_LOG, .copy_from = &reftable_log_record_copy_from, .val_type = &reftable_log_record_val_type, .encode = &reftable_log_record_encode, @@ -1137,7 +1137,7 @@ static int reftable_index_record_cmp(const void *_a, const void *_b) static struct reftable_record_vtable reftable_index_record_vtable = { .key = &reftable_index_record_key, - .type = BLOCK_TYPE_INDEX, + .type = REFTABLE_BLOCK_TYPE_INDEX, .copy_from = &reftable_index_record_copy_from, .val_type = &reftable_index_record_val_type, .encode = &reftable_index_record_encode, @@ -1280,13 +1280,13 @@ int reftable_log_record_is_deletion(const struct reftable_log_record *log) static void *reftable_record_data(struct reftable_record *rec) { switch (rec->type) { - case BLOCK_TYPE_REF: + case REFTABLE_BLOCK_TYPE_REF: return &rec->u.ref; - case BLOCK_TYPE_LOG: + case REFTABLE_BLOCK_TYPE_LOG: return &rec->u.log; - case BLOCK_TYPE_INDEX: + case REFTABLE_BLOCK_TYPE_INDEX: return &rec->u.idx; - case BLOCK_TYPE_OBJ: + case REFTABLE_BLOCK_TYPE_OBJ: return &rec->u.obj; } abort(); @@ -1296,13 +1296,13 @@ static struct reftable_record_vtable * reftable_record_vtable(struct reftable_record *rec) { switch (rec->type) { - case BLOCK_TYPE_REF: + case REFTABLE_BLOCK_TYPE_REF: return &reftable_ref_record_vtable; - case BLOCK_TYPE_LOG: + case REFTABLE_BLOCK_TYPE_LOG: return &reftable_log_record_vtable; - case BLOCK_TYPE_INDEX: + case REFTABLE_BLOCK_TYPE_INDEX: return &reftable_index_record_vtable; - case BLOCK_TYPE_OBJ: + case REFTABLE_BLOCK_TYPE_OBJ: return &reftable_obj_record_vtable; } abort(); @@ -1314,11 +1314,11 @@ int reftable_record_init(struct reftable_record *rec, uint8_t typ) rec->type = typ; switch (typ) { - case BLOCK_TYPE_REF: - case BLOCK_TYPE_LOG: - case BLOCK_TYPE_OBJ: + case REFTABLE_BLOCK_TYPE_REF: + case REFTABLE_BLOCK_TYPE_LOG: + case REFTABLE_BLOCK_TYPE_OBJ: return 0; - case BLOCK_TYPE_INDEX: + case REFTABLE_BLOCK_TYPE_INDEX: reftable_buf_init(&rec->u.idx.last_key); return 0; default: diff --git a/reftable/reftable-constants.h b/reftable/reftable-constants.h new file mode 100644 index 00000000000..4ae9ba4bacc --- /dev/null +++ b/reftable/reftable-constants.h @@ -0,0 +1,18 @@ +/* + * Copyright 2020 Google LLC + * + * Use of this source code is governed by a BSD-style + * license that can be found in the LICENSE file or at + * https://developers.google.com/open-source/licenses/bsd + */ + +#ifndef REFTABLE_CONSTANTS_H +#define REFTABLE_CONSTANTS_H + +#define REFTABLE_BLOCK_TYPE_LOG 'g' +#define REFTABLE_BLOCK_TYPE_INDEX 'i' +#define REFTABLE_BLOCK_TYPE_REF 'r' +#define REFTABLE_BLOCK_TYPE_OBJ 'o' +#define REFTABLE_BLOCK_TYPE_ANY 0 + +#endif /* REFTABLE_CONSTANTS_H */ diff --git a/reftable/stack.c b/reftable/stack.c index cc48e725b14..4caf96aa1d6 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -203,14 +203,14 @@ int reftable_stack_init_ref_iterator(struct reftable_stack *st, struct reftable_iterator *it) { return merged_table_init_iter(reftable_stack_merged_table(st), - it, BLOCK_TYPE_REF); + it, REFTABLE_BLOCK_TYPE_REF); } int reftable_stack_init_log_iterator(struct reftable_stack *st, struct reftable_iterator *it) { return merged_table_init_iter(reftable_stack_merged_table(st), - it, BLOCK_TYPE_LOG); + it, REFTABLE_BLOCK_TYPE_LOG); } struct reftable_merged_table * @@ -1098,7 +1098,7 @@ static int stack_write_compact(struct reftable_stack *st, if (err < 0) goto done; - err = merged_table_init_iter(mt, &it, BLOCK_TYPE_REF); + err = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF); if (err < 0) goto done; @@ -1126,7 +1126,7 @@ static int stack_write_compact(struct reftable_stack *st, } reftable_iterator_destroy(&it); - err = merged_table_init_iter(mt, &it, BLOCK_TYPE_LOG); + err = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_LOG); if (err < 0) goto done; diff --git a/reftable/table.c b/reftable/table.c index a2a0c7a1d2f..1af39423224 100644 --- a/reftable/table.c +++ b/reftable/table.c @@ -20,11 +20,11 @@ static struct reftable_table_offsets * table_offsets_for(struct reftable_table *t, uint8_t typ) { switch (typ) { - case BLOCK_TYPE_REF: + case REFTABLE_BLOCK_TYPE_REF: return &t->ref_offsets; - case BLOCK_TYPE_LOG: + case REFTABLE_BLOCK_TYPE_LOG: return &t->log_offsets; - case BLOCK_TYPE_OBJ: + case REFTABLE_BLOCK_TYPE_OBJ: return &t->obj_offsets; } abort(); @@ -112,9 +112,9 @@ static int parse_footer(struct reftable_table *t, uint8_t *footer, } first_block_typ = header[header_size(t->version)]; - t->ref_offsets.is_present = (first_block_typ == BLOCK_TYPE_REF); + t->ref_offsets.is_present = (first_block_typ == REFTABLE_BLOCK_TYPE_REF); t->ref_offsets.offset = 0; - t->log_offsets.is_present = (first_block_typ == BLOCK_TYPE_LOG || + t->log_offsets.is_present = (first_block_typ == REFTABLE_BLOCK_TYPE_LOG || t->log_offsets.offset > 0); t->obj_offsets.is_present = t->obj_offsets.offset > 0; if (t->obj_offsets.is_present && !t->object_id_len) { @@ -150,7 +150,7 @@ static int table_iter_next_in_block(struct table_iter *ti, struct reftable_record *rec) { int res = block_iter_next(&ti->bi, rec); - if (res == 0 && reftable_record_type(rec) == BLOCK_TYPE_REF) { + if (res == 0 && reftable_record_type(rec) == REFTABLE_BLOCK_TYPE_REF) { rec->u.ref.update_index += ti->table->min_update_index; } @@ -177,7 +177,7 @@ int table_init_block(struct reftable_table *t, struct reftable_block *block, if (err < 0) goto done; - if (want_typ != BLOCK_TYPE_ANY && block->block_type != want_typ) { + if (want_typ != REFTABLE_BLOCK_TYPE_ANY && block->block_type != want_typ) { err = 1; goto done; } @@ -270,7 +270,7 @@ static int table_iter_seek_start(struct table_iter *ti, uint8_t typ, int index) if (off == 0) { return 1; } - typ = BLOCK_TYPE_INDEX; + typ = REFTABLE_BLOCK_TYPE_INDEX; } return table_iter_seek_to(ti, off, typ); @@ -366,10 +366,10 @@ static int table_iter_seek_indexed(struct table_iter *ti, struct reftable_record *rec) { struct reftable_record want_index = { - .type = BLOCK_TYPE_INDEX, .u.idx = { .last_key = REFTABLE_BUF_INIT } + .type = REFTABLE_BLOCK_TYPE_INDEX, .u.idx = { .last_key = REFTABLE_BUF_INIT } }; struct reftable_record index_result = { - .type = BLOCK_TYPE_INDEX, + .type = REFTABLE_BLOCK_TYPE_INDEX, .u.idx = { .last_key = REFTABLE_BUF_INIT }, }; int err; @@ -429,7 +429,7 @@ static int table_iter_seek_indexed(struct table_iter *ti, break; } - if (ti->typ != BLOCK_TYPE_INDEX) { + if (ti->typ != REFTABLE_BLOCK_TYPE_INDEX) { err = REFTABLE_FORMAT_ERROR; goto done; } @@ -517,13 +517,13 @@ int table_init_iter(struct reftable_table *t, int reftable_table_init_ref_iterator(struct reftable_table *t, struct reftable_iterator *it) { - return table_init_iter(t, it, BLOCK_TYPE_REF); + return table_init_iter(t, it, REFTABLE_BLOCK_TYPE_REF); } int reftable_table_init_log_iterator(struct reftable_table *t, struct reftable_iterator *it) { - return table_init_iter(t, it, BLOCK_TYPE_LOG); + return table_init_iter(t, it, REFTABLE_BLOCK_TYPE_LOG); } int reftable_table_new(struct reftable_table **out, @@ -625,7 +625,7 @@ static int reftable_table_refs_for_indexed(struct reftable_table *t, uint8_t *oid) { struct reftable_record want = { - .type = BLOCK_TYPE_OBJ, + .type = REFTABLE_BLOCK_TYPE_OBJ, .u.obj = { .hash_prefix = oid, .hash_prefix_len = t->object_id_len, @@ -633,14 +633,14 @@ static int reftable_table_refs_for_indexed(struct reftable_table *t, }; struct reftable_iterator oit = { NULL }; struct reftable_record got = { - .type = BLOCK_TYPE_OBJ, + .type = REFTABLE_BLOCK_TYPE_OBJ, .u.obj = { 0 }, }; int err = 0; struct indexed_table_ref_iter *itr = NULL; /* Look through the reverse index. */ - err = table_init_iter(t, &oit, BLOCK_TYPE_OBJ); + err = table_init_iter(t, &oit, REFTABLE_BLOCK_TYPE_OBJ); if (err < 0) goto done; @@ -692,7 +692,7 @@ static int reftable_table_refs_for_unindexed(struct reftable_table *t, } table_iter_init(ti, t); - err = table_iter_seek_start(ti, BLOCK_TYPE_REF, 0); + err = table_iter_seek_start(ti, REFTABLE_BLOCK_TYPE_REF, 0); if (err < 0) goto out; @@ -748,15 +748,15 @@ int reftable_table_print_blocks(const char *tablename) } sections[] = { { .name = "ref", - .type = BLOCK_TYPE_REF, + .type = REFTABLE_BLOCK_TYPE_REF, }, { .name = "obj", - .type = BLOCK_TYPE_OBJ, + .type = REFTABLE_BLOCK_TYPE_OBJ, }, { .name = "log", - .type = BLOCK_TYPE_LOG, + .type = REFTABLE_BLOCK_TYPE_LOG, }, }; struct reftable_block_source src = { 0 }; diff --git a/reftable/writer.c b/reftable/writer.c index f0accfd0c32..0954c29bcf2 100644 --- a/reftable/writer.c +++ b/reftable/writer.c @@ -172,7 +172,7 @@ int reftable_writer_new(struct reftable_writer **out, wp->write_arg = writer_arg; wp->opts = opts; wp->flush = flush_func; - writer_reinit_block_writer(wp, BLOCK_TYPE_REF); + writer_reinit_block_writer(wp, REFTABLE_BLOCK_TYPE_REF); *out = wp; @@ -347,7 +347,7 @@ int reftable_writer_add_ref(struct reftable_writer *w, struct reftable_ref_record *ref) { struct reftable_record rec = { - .type = BLOCK_TYPE_REF, + .type = REFTABLE_BLOCK_TYPE_REF, .u = { .ref = *ref }, @@ -411,13 +411,13 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w, struct reftable_log_record *log) { struct reftable_record rec = { - .type = BLOCK_TYPE_LOG, + .type = REFTABLE_BLOCK_TYPE_LOG, .u = { .log = *log, }, }; if (w->block_writer && - block_writer_type(w->block_writer) == BLOCK_TYPE_REF) { + block_writer_type(w->block_writer) == REFTABLE_BLOCK_TYPE_REF) { int err = writer_finish_public_section(w); if (err < 0) return err; @@ -537,7 +537,7 @@ static int writer_finish_section(struct reftable_writer *w) max_level++; index_start = w->next; - err = writer_reinit_block_writer(w, BLOCK_TYPE_INDEX); + err = writer_reinit_block_writer(w, REFTABLE_BLOCK_TYPE_INDEX); if (err < 0) return err; @@ -549,7 +549,7 @@ static int writer_finish_section(struct reftable_writer *w) w->index_cap = 0; for (i = 0; i < idx_len; i++) { struct reftable_record rec = { - .type = BLOCK_TYPE_INDEX, + .type = REFTABLE_BLOCK_TYPE_INDEX, .u = { .idx = idx[i], }, @@ -614,7 +614,7 @@ static void write_object_record(void *void_arg, void *key) struct write_record_arg *arg = void_arg; struct obj_index_tree_node *entry = key; struct reftable_record - rec = { .type = BLOCK_TYPE_OBJ, + rec = { .type = REFTABLE_BLOCK_TYPE_OBJ, .u.obj = { .hash_prefix = (uint8_t *)entry->hash.buf, .hash_prefix_len = arg->w->stats.object_id_len, @@ -632,7 +632,7 @@ static void write_object_record(void *void_arg, void *key) if (arg->err < 0) goto done; - arg->err = writer_reinit_block_writer(arg->w, BLOCK_TYPE_OBJ); + arg->err = writer_reinit_block_writer(arg->w, REFTABLE_BLOCK_TYPE_OBJ); if (arg->err < 0) goto done; @@ -670,7 +670,7 @@ static int writer_dump_object_index(struct reftable_writer *w) infix_walk(w->obj_index_tree, &update_common, &common); w->stats.object_id_len = common.max + 1; - err = writer_reinit_block_writer(w, BLOCK_TYPE_OBJ); + err = writer_reinit_block_writer(w, REFTABLE_BLOCK_TYPE_OBJ); if (err < 0) return err; @@ -694,7 +694,7 @@ static int writer_finish_public_section(struct reftable_writer *w) err = writer_finish_section(w); if (err < 0) return err; - if (typ == BLOCK_TYPE_REF && !w->opts.skip_index_objects && + if (typ == REFTABLE_BLOCK_TYPE_REF && !w->opts.skip_index_objects && w->stats.ref_stats.index_blocks > 0) { err = writer_dump_object_index(w); if (err < 0) @@ -799,7 +799,7 @@ static int writer_flush_nonempty_block(struct reftable_writer *w) * By default, all records except for log records are padded to the * block size. */ - if (!w->opts.unpadded && typ != BLOCK_TYPE_LOG) + if (!w->opts.unpadded && typ != REFTABLE_BLOCK_TYPE_LOG) padding = w->opts.block_size - raw_bytes; bstats = writer_reftable_block_stats(w, typ); diff --git a/t/unit-tests/t-reftable-block.c b/t/unit-tests/t-reftable-block.c index 5577a5769ed..7dbd93601c7 100644 --- a/t/unit-tests/t-reftable-block.c +++ b/t/unit-tests/t-reftable-block.c @@ -24,7 +24,7 @@ static void t_ref_block_read_write(void) .last_key = REFTABLE_BUF_INIT, }; struct reftable_record rec = { - .type = BLOCK_TYPE_REF, + .type = REFTABLE_BLOCK_TYPE_REF, }; size_t i = 0; int ret; @@ -37,7 +37,7 @@ static void t_ref_block_read_write(void) check(block_data.buf != NULL); block_data.len = block_size; - ret = block_writer_init(&bw, BLOCK_TYPE_REF, (uint8_t *) block_data.buf, block_size, + ret = block_writer_init(&bw, REFTABLE_BLOCK_TYPE_REF, (uint8_t *) block_data.buf, block_size, header_off, hash_size(REFTABLE_HASH_SHA1)); check(!ret); @@ -118,7 +118,7 @@ static void t_log_block_read_write(void) .last_key = REFTABLE_BUF_INIT, }; struct reftable_record rec = { - .type = BLOCK_TYPE_LOG, + .type = REFTABLE_BLOCK_TYPE_LOG, }; size_t i = 0; int ret; @@ -131,7 +131,7 @@ static void t_log_block_read_write(void) check(block_data.buf != NULL); block_data.len = block_size; - ret = block_writer_init(&bw, BLOCK_TYPE_LOG, (uint8_t *) block_data.buf, block_size, + ret = block_writer_init(&bw, REFTABLE_BLOCK_TYPE_LOG, (uint8_t *) block_data.buf, block_size, header_off, hash_size(REFTABLE_HASH_SHA1)); check(!ret); @@ -208,7 +208,7 @@ static void t_obj_block_read_write(void) .last_key = REFTABLE_BUF_INIT, }; struct reftable_record rec = { - .type = BLOCK_TYPE_OBJ, + .type = REFTABLE_BLOCK_TYPE_OBJ, }; size_t i = 0; int ret; @@ -221,7 +221,7 @@ static void t_obj_block_read_write(void) check(block_data.buf != NULL); block_data.len = block_size; - ret = block_writer_init(&bw, BLOCK_TYPE_OBJ, (uint8_t *) block_data.buf, block_size, + ret = block_writer_init(&bw, REFTABLE_BLOCK_TYPE_OBJ, (uint8_t *) block_data.buf, block_size, header_off, hash_size(REFTABLE_HASH_SHA1)); check(!ret); @@ -291,7 +291,7 @@ static void t_index_block_read_write(void) .last_key = REFTABLE_BUF_INIT, }; struct reftable_record rec = { - .type = BLOCK_TYPE_INDEX, + .type = REFTABLE_BLOCK_TYPE_INDEX, .u.idx.last_key = REFTABLE_BUF_INIT, }; size_t i = 0; @@ -305,7 +305,7 @@ static void t_index_block_read_write(void) check(block_data.buf != NULL); block_data.len = block_size; - ret = block_writer_init(&bw, BLOCK_TYPE_INDEX, (uint8_t *) block_data.buf, block_size, + ret = block_writer_init(&bw, REFTABLE_BLOCK_TYPE_INDEX, (uint8_t *) block_data.buf, block_size, header_off, hash_size(REFTABLE_HASH_SHA1)); check(!ret); @@ -315,7 +315,7 @@ static void t_index_block_read_write(void) snprintf(buf, sizeof(buf), "branch%02"PRIuMAX, (uintmax_t)i); reftable_buf_init(&recs[i].u.idx.last_key); - recs[i].type = BLOCK_TYPE_INDEX; + recs[i].type = REFTABLE_BLOCK_TYPE_INDEX; check(!reftable_buf_addstr(&recs[i].u.idx.last_key, buf)); recs[i].u.idx.offset = i; @@ -389,13 +389,13 @@ static void t_block_iterator(void) REFTABLE_CALLOC_ARRAY(data.buf, data.len); check(data.buf != NULL); - err = block_writer_init(&writer, BLOCK_TYPE_REF, (uint8_t *) data.buf, data.len, + err = block_writer_init(&writer, REFTABLE_BLOCK_TYPE_REF, (uint8_t *) data.buf, data.len, 0, hash_size(REFTABLE_HASH_SHA1)); check(!err); for (size_t i = 0; i < ARRAY_SIZE(expected_refs); i++) { expected_refs[i] = (struct reftable_record) { - .type = BLOCK_TYPE_REF, + .type = REFTABLE_BLOCK_TYPE_REF, .u.ref = { .value_type = REFTABLE_REF_VAL1, .refname = xstrfmt("refs/heads/branch-%02"PRIuMAX, (uintmax_t)i), diff --git a/t/unit-tests/t-reftable-merged.c b/t/unit-tests/t-reftable-merged.c index fed6beb85c0..18c3251a56a 100644 --- a/t/unit-tests/t-reftable-merged.c +++ b/t/unit-tests/t-reftable-merged.c @@ -84,7 +84,7 @@ static void t_merged_single_record(void) struct reftable_iterator it = { 0 }; int err; - err = merged_table_init_iter(mt, &it, BLOCK_TYPE_REF); + err = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF); check(!err); err = reftable_iterator_seek_ref(&it, "a"); check(!err); @@ -164,7 +164,7 @@ static void t_merged_refs(void) size_t cap = 0; size_t i; - err = merged_table_init_iter(mt, &it, BLOCK_TYPE_REF); + err = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF); check(!err); err = reftable_iterator_seek_ref(&it, "a"); check(!err); @@ -244,7 +244,7 @@ static void t_merged_seek_multiple_times(void) struct reftable_merged_table *mt; mt = merged_table_from_records(refs, &sources, &tables, sizes, bufs, 2); - merged_table_init_iter(mt, &it, BLOCK_TYPE_REF); + merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF); for (size_t i = 0; i < 5; i++) { int err = reftable_iterator_seek_ref(&it, "c"); @@ -320,7 +320,7 @@ static void t_merged_seek_multiple_times_without_draining(void) int err; mt = merged_table_from_records(refs, &sources, &tables, sizes, bufs, 2); - merged_table_init_iter(mt, &it, BLOCK_TYPE_REF); + merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_REF); err = reftable_iterator_seek_ref(&it, "b"); check(!err); @@ -445,7 +445,7 @@ static void t_merged_logs(void) size_t cap = 0; size_t i; - err = merged_table_init_iter(mt, &it, BLOCK_TYPE_LOG); + err = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_LOG); check(!err); err = reftable_iterator_seek_log(&it, "a"); check(!err); @@ -469,7 +469,7 @@ static void t_merged_logs(void) check(reftable_log_record_equal(want[i], &out[i], REFTABLE_HASH_SIZE_SHA1)); - err = merged_table_init_iter(mt, &it, BLOCK_TYPE_LOG); + err = merged_table_init_iter(mt, &it, REFTABLE_BLOCK_TYPE_LOG); check(!err); err = reftable_iterator_seek_log_at(&it, "a", 2); check(!err); diff --git a/t/unit-tests/t-reftable-pq.c b/t/unit-tests/t-reftable-pq.c index c128fe8616a..fb5a4eb1877 100644 --- a/t/unit-tests/t-reftable-pq.c +++ b/t/unit-tests/t-reftable-pq.c @@ -34,7 +34,7 @@ static void t_pq_record(void) char *last = NULL; for (i = 0; i < N; i++) { - check(!reftable_record_init(&recs[i], BLOCK_TYPE_REF)); + check(!reftable_record_init(&recs[i], REFTABLE_BLOCK_TYPE_REF)); recs[i].u.ref.refname = xstrfmt("%02"PRIuMAX, (uintmax_t)i); } @@ -57,7 +57,7 @@ static void t_pq_record(void) merged_iter_pqueue_check(&pq); check(pq_entry_equal(&top, &e)); - check(reftable_record_type(e.rec) == BLOCK_TYPE_REF); + check(reftable_record_type(e.rec) == REFTABLE_BLOCK_TYPE_REF); if (last) check_int(strcmp(last, e.rec->u.ref.refname), <, 0); last = e.rec->u.ref.refname; @@ -76,7 +76,7 @@ static void t_pq_index(void) size_t N = ARRAY_SIZE(recs), i; for (i = 0; i < N; i++) { - check(!reftable_record_init(&recs[i], BLOCK_TYPE_REF)); + check(!reftable_record_init(&recs[i], REFTABLE_BLOCK_TYPE_REF)); recs[i].u.ref.refname = (char *) "refs/heads/master"; } @@ -100,7 +100,7 @@ static void t_pq_index(void) merged_iter_pqueue_check(&pq); check(pq_entry_equal(&top, &e)); - check(reftable_record_type(e.rec) == BLOCK_TYPE_REF); + check(reftable_record_type(e.rec) == REFTABLE_BLOCK_TYPE_REF); check_int(e.index, ==, i); if (last) check_str(last, e.rec->u.ref.refname); @@ -117,7 +117,7 @@ static void t_merged_iter_pqueue_top(void) size_t N = ARRAY_SIZE(recs), i; for (i = 0; i < N; i++) { - check(!reftable_record_init(&recs[i], BLOCK_TYPE_REF)); + check(!reftable_record_init(&recs[i], REFTABLE_BLOCK_TYPE_REF)); recs[i].u.ref.refname = (char *) "refs/heads/master"; } diff --git a/t/unit-tests/t-reftable-record.c b/t/unit-tests/t-reftable-record.c index 59549663736..553a0076647 100644 --- a/t/unit-tests/t-reftable-record.c +++ b/t/unit-tests/t-reftable-record.c @@ -84,17 +84,17 @@ static void t_reftable_ref_record_comparison(void) { struct reftable_record in[3] = { { - .type = BLOCK_TYPE_REF, + .type = REFTABLE_BLOCK_TYPE_REF, .u.ref.refname = (char *) "refs/heads/master", .u.ref.value_type = REFTABLE_REF_VAL1, }, { - .type = BLOCK_TYPE_REF, + .type = REFTABLE_BLOCK_TYPE_REF, .u.ref.refname = (char *) "refs/heads/master", .u.ref.value_type = REFTABLE_REF_DELETION, }, { - .type = BLOCK_TYPE_REF, + .type = REFTABLE_BLOCK_TYPE_REF, .u.ref.refname = (char *) "HEAD", .u.ref.value_type = REFTABLE_REF_SYMREF, .u.ref.value.symref = (char *) "refs/heads/master", @@ -141,10 +141,10 @@ static void t_reftable_ref_record_roundtrip(void) for (int i = REFTABLE_REF_DELETION; i < REFTABLE_NR_REF_VALUETYPES; i++) { struct reftable_record in = { - .type = BLOCK_TYPE_REF, + .type = REFTABLE_BLOCK_TYPE_REF, .u.ref.value_type = i, }; - struct reftable_record out = { .type = BLOCK_TYPE_REF }; + struct reftable_record out = { .type = REFTABLE_BLOCK_TYPE_REF }; struct reftable_buf key = REFTABLE_BUF_INIT; uint8_t buffer[1024] = { 0 }; struct string_view dest = { @@ -198,17 +198,17 @@ static void t_reftable_log_record_comparison(void) { struct reftable_record in[3] = { { - .type = BLOCK_TYPE_LOG, + .type = REFTABLE_BLOCK_TYPE_LOG, .u.log.refname = (char *) "refs/heads/master", .u.log.update_index = 42, }, { - .type = BLOCK_TYPE_LOG, + .type = REFTABLE_BLOCK_TYPE_LOG, .u.log.refname = (char *) "refs/heads/master", .u.log.update_index = 22, }, { - .type = BLOCK_TYPE_LOG, + .type = REFTABLE_BLOCK_TYPE_LOG, .u.log.refname = (char *) "refs/heads/main", .u.log.update_index = 22, }, @@ -297,7 +297,7 @@ static void t_reftable_log_record_roundtrip(void) check(!reftable_log_record_is_deletion(&in[2])); for (size_t i = 0; i < ARRAY_SIZE(in); i++) { - struct reftable_record rec = { .type = BLOCK_TYPE_LOG }; + struct reftable_record rec = { .type = REFTABLE_BLOCK_TYPE_LOG }; struct reftable_buf key = REFTABLE_BUF_INIT; uint8_t buffer[1024] = { 0 }; struct string_view dest = { @@ -306,7 +306,7 @@ static void t_reftable_log_record_roundtrip(void) }; /* populate out, to check for leaks. */ struct reftable_record out = { - .type = BLOCK_TYPE_LOG, + .type = REFTABLE_BLOCK_TYPE_LOG, .u.log = { .refname = xstrdup("old name"), .value_type = REFTABLE_LOG_UPDATE, @@ -384,21 +384,21 @@ static void t_reftable_obj_record_comparison(void) uint64_t offsets[] = { 0, 16, 32, 48, 64, 80, 96, 112}; struct reftable_record in[3] = { { - .type = BLOCK_TYPE_OBJ, + .type = REFTABLE_BLOCK_TYPE_OBJ, .u.obj.hash_prefix = id_bytes, .u.obj.hash_prefix_len = 7, .u.obj.offsets = offsets, .u.obj.offset_len = 8, }, { - .type = BLOCK_TYPE_OBJ, + .type = REFTABLE_BLOCK_TYPE_OBJ, .u.obj.hash_prefix = id_bytes, .u.obj.hash_prefix_len = 7, .u.obj.offsets = offsets, .u.obj.offset_len = 5, }, { - .type = BLOCK_TYPE_OBJ, + .type = REFTABLE_BLOCK_TYPE_OBJ, .u.obj.hash_prefix = id_bytes, .u.obj.hash_prefix_len = 5, }, @@ -450,13 +450,13 @@ static void t_reftable_obj_record_roundtrip(void) .len = sizeof(buffer), }; struct reftable_record in = { - .type = BLOCK_TYPE_OBJ, + .type = REFTABLE_BLOCK_TYPE_OBJ, .u = { .obj = recs[i], }, }; struct reftable_buf key = REFTABLE_BUF_INIT; - struct reftable_record out = { .type = BLOCK_TYPE_OBJ }; + struct reftable_record out = { .type = REFTABLE_BLOCK_TYPE_OBJ }; int n, m; uint8_t extra; @@ -482,17 +482,17 @@ static void t_reftable_index_record_comparison(void) { struct reftable_record in[3] = { { - .type = BLOCK_TYPE_INDEX, + .type = REFTABLE_BLOCK_TYPE_INDEX, .u.idx.offset = 22, .u.idx.last_key = REFTABLE_BUF_INIT, }, { - .type = BLOCK_TYPE_INDEX, + .type = REFTABLE_BLOCK_TYPE_INDEX, .u.idx.offset = 32, .u.idx.last_key = REFTABLE_BUF_INIT, }, { - .type = BLOCK_TYPE_INDEX, + .type = REFTABLE_BLOCK_TYPE_INDEX, .u.idx.offset = 32, .u.idx.last_key = REFTABLE_BUF_INIT, }, @@ -523,7 +523,7 @@ static void t_reftable_index_record_comparison(void) static void t_reftable_index_record_roundtrip(void) { struct reftable_record in = { - .type = BLOCK_TYPE_INDEX, + .type = REFTABLE_BLOCK_TYPE_INDEX, .u.idx = { .offset = 42, .last_key = REFTABLE_BUF_INIT, @@ -537,7 +537,7 @@ static void t_reftable_index_record_roundtrip(void) struct reftable_buf scratch = REFTABLE_BUF_INIT; struct reftable_buf key = REFTABLE_BUF_INIT; struct reftable_record out = { - .type = BLOCK_TYPE_INDEX, + .type = REFTABLE_BLOCK_TYPE_INDEX, .u.idx = { .last_key = REFTABLE_BUF_INIT }, }; int n, m; diff --git a/t/unit-tests/t-reftable-table.c b/t/unit-tests/t-reftable-table.c index ba39cdf9a76..7e1eb533d0b 100644 --- a/t/unit-tests/t-reftable-table.c +++ b/t/unit-tests/t-reftable-table.c @@ -106,33 +106,33 @@ static int t_table_block_iterator(void) uint16_t record_count; } expected_blocks[] = { { - .block_type = BLOCK_TYPE_REF, + .block_type = REFTABLE_BLOCK_TYPE_REF, .header_off = 24, .restart_count = 10, .record_count = 158, }, { - .block_type = BLOCK_TYPE_REF, + .block_type = REFTABLE_BLOCK_TYPE_REF, .restart_count = 10, .record_count = 159, }, { - .block_type = BLOCK_TYPE_REF, + .block_type = REFTABLE_BLOCK_TYPE_REF, .restart_count = 10, .record_count = 159, }, { - .block_type = BLOCK_TYPE_REF, + .block_type = REFTABLE_BLOCK_TYPE_REF, .restart_count = 2, .record_count = 24, }, { - .block_type = BLOCK_TYPE_INDEX, + .block_type = REFTABLE_BLOCK_TYPE_INDEX, .restart_count = 1, .record_count = 4, }, { - .block_type = BLOCK_TYPE_OBJ, + .block_type = REFTABLE_BLOCK_TYPE_OBJ, .restart_count = 1, .record_count = 1, }, From patchwork Mon Apr 7 13:16:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14040515 Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A982925484D for ; Mon, 7 Apr 2025 13:16:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.155 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031802; cv=none; b=i2ioSsEhrpO02G4+hcE29UycukrCD7NJwUzC1kZ+qUzATvvp/+R/J/tBTL7GgLDYhH346DXq16J4wVs4eV6DJPqHvwJXBBad0LVZjWq1MfASIqGr3nID4zvw2DU6xHB2jToHk7lrk+n2x93i0scWCWRTOTK0NV8VscH444lM9R8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744031802; c=relaxed/simple; bh=bzCWlFd2s/LUmF7lBtVwOJGAJJCMnFYDfj4EAZkvquA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=i2s9Ab4ymGyoJC66JEWn9onQ2SFStiHnrJeenbWz/Xb6piSrF59QCINj7wM8U/0FVvOJk92Z5zAyEDuRaud7knx8FJPf/Bzv/JutnK+77Rnx8JqmtZsVAKaA/dL9nutyD0yfegE4Nq92b6C6iak4WTPxwWfM7pCNda9CoZnif38= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=KysmMecE; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=eHgD7ug+; arc=none smtp.client-ip=103.168.172.155 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="KysmMecE"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="eHgD7ug+" Received: from phl-compute-10.internal (phl-compute-10.phl.internal [10.202.2.50]) by mailfhigh.phl.internal (Postfix) with ESMTP id AA5F71140211; Mon, 7 Apr 2025 09:16:39 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-10.internal (MEProxy); Mon, 07 Apr 2025 09:16:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744031799; x=1744118199; bh=4n/cd+DqreR51D681/y80p4utDzYHkLdUoq2LwrHbQs=; b= KysmMecExUTKyXQW6Sb/B3ySIaVVv+W9oqO2+M6HDofsYM831UtRicmR0yoPcscG qhpS1/q3ninTY6A8jtl5/YCskV9FzsTG9kpcW67JBC7L9dJRYQzc/4kwEilBGsYA WD0jn5yWsTmpGbdOhUQ9gZWwuPmfotlR5COzF0YtM620VjjXLEXyIZzcbGzoLSNu miymL0cA35pbUMO70gS86PZ/QoybP1gLPpa3I+OXxHE9v3FJF3WOwzlnO4J128yY L5YpkxrxZSWOze1loihl32DHOFLoV3NKIdbwU9M46HDEHtyXI4KfcfOf2oTTSMY1 qf+DwuJUwh/AqYiSDIgHww== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744031799; x= 1744118199; bh=4n/cd+DqreR51D681/y80p4utDzYHkLdUoq2LwrHbQs=; b=e HgD7ug+oyRT4g1ra5qYgiVdMHscYfkB4T7rQhXpPfBE0NZ8dY/riYbSr3uj2OYTZ XH4LRe0i4D+/6tetpkSq5CtMmQNlwK5+ALSK0LVSmQwdotIUtPy2eICojZ3/69jl YbT/ujc8DNsuF0hIRgyp8EnukVpVk4Z2TquwGdkJExhkXnesz/bdL1Ty3WvKaLCZ En5/e0B1mPwhJzJErrSxpRJY4QIygIzgyAfcUEKNI/isKcTXGdfmG6Xps/LrBKKN VIsCr3i4mGkjqMk95wOvCwJFC3jbPzKETlIeAyvUeTOH750bPZJYeZ2a3YYI3m3B om2uBqJ51YKYALi7LW3Dg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddtvdelucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertder tdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhksh drihhmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteek udehjeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmh epmhgrihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhmpd hrtghpthhtohepkhgrrhhthhhikhdrudekkeesghhmrghilhdrtghomhdprhgtphhtthho pehjlhhtohgslhgvrhesghhmrghilhdrtghomhdprhgtphhtthhopehgihhtsehvghgvrh drkhgvrhhnvghlrdhorhhg X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Apr 2025 09:16:38 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id b7e6cf39 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Apr 2025 13:16:35 +0000 (UTC) From: Patrick Steinhardt Date: Mon, 07 Apr 2025 15:16:28 +0200 Subject: [PATCH v2 16/16] reftable/table: move printing logic into test helper Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250407-pks-reftable-polishing-v2-16-316c4ff10571@pks.im> References: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> In-Reply-To: <20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im> To: git@vger.kernel.org Cc: Justin Tobler , Karthik Nayak , Junio C Hamano X-Mailer: b4 0.14.2 The logic to print individual blocks in a table is hosted in the reftable library. This is only the case due to historical reasons though because users of the library had no interfaces to read blocks one by one. Otherwise, printing individual blocks has no place in the reftable library given that the format will not be generic in the first place. We have now grown a public interface to iterate through blocks contained in a table, and thus we can finally move the logic to print them into the test helper. Move over the logic and refactor it accordingly. Note that the iterator also trivially allows us to access index sections, which we previously didn't print at all. This omission wasn't intentional though, so start dumping those sections as well so that we can assert that indices are written as expected. Signed-off-by: Patrick Steinhardt --- reftable/reftable-table.h | 3 -- reftable/table.c | 65 ------------------------------------ t/helper/test-reftable.c | 69 ++++++++++++++++++++++++++++++++++++++- t/t0613-reftable-write-options.sh | 9 +++++ 4 files changed, 77 insertions(+), 69 deletions(-) diff --git a/reftable/reftable-table.h b/reftable/reftable-table.h index 4d61ce6cf77..5f935d02e3b 100644 --- a/reftable/reftable-table.h +++ b/reftable/reftable-table.h @@ -97,9 +97,6 @@ uint64_t reftable_table_max_update_index(struct reftable_table *t); /* return the min_update_index for a table */ uint64_t reftable_table_min_update_index(struct reftable_table *t); -/* print blocks onto stdout for debugging. */ -int reftable_table_print_blocks(const char *tablename); - /* * An iterator that iterates through the blocks contained in a given table. */ diff --git a/reftable/table.c b/reftable/table.c index 1af39423224..ee831276158 100644 --- a/reftable/table.c +++ b/reftable/table.c @@ -740,71 +740,6 @@ uint64_t reftable_table_min_update_index(struct reftable_table *t) return t->min_update_index; } -int reftable_table_print_blocks(const char *tablename) -{ - struct { - const char *name; - int type; - } sections[] = { - { - .name = "ref", - .type = REFTABLE_BLOCK_TYPE_REF, - }, - { - .name = "obj", - .type = REFTABLE_BLOCK_TYPE_OBJ, - }, - { - .name = "log", - .type = REFTABLE_BLOCK_TYPE_LOG, - }, - }; - struct reftable_block_source src = { 0 }; - struct reftable_table *table = NULL; - struct table_iter ti = { 0 }; - size_t i; - int err; - - err = reftable_block_source_from_file(&src, tablename); - if (err < 0) - goto done; - - err = reftable_table_new(&table, &src, tablename); - if (err < 0) - goto done; - - table_iter_init(&ti, table); - - printf("header:\n"); - printf(" block_size: %d\n", table->block_size); - - for (i = 0; i < sizeof(sections) / sizeof(*sections); i++) { - err = table_iter_seek_start(&ti, sections[i].type, 0); - if (err < 0) - goto done; - if (err > 0) - continue; - - printf("%s:\n", sections[i].name); - - while (1) { - printf(" - length: %u\n", ti.block.restart_off); - printf(" restarts: %u\n", ti.block.restart_count); - - err = table_iter_next_block(&ti); - if (err < 0) - goto done; - if (err > 0) - break; - } - } - -done: - reftable_table_decref(table); - table_iter_close(&ti); - return err; -} - int reftable_table_iterator_init(struct reftable_table_iterator *it, struct reftable_table *t) { diff --git a/t/helper/test-reftable.c b/t/helper/test-reftable.c index f8f1956f4f3..b16c0722c84 100644 --- a/t/helper/test-reftable.c +++ b/t/helper/test-reftable.c @@ -2,6 +2,7 @@ #include "hash.h" #include "hex.h" #include "reftable/system.h" +#include "reftable/reftable-constants.h" #include "reftable/reftable-error.h" #include "reftable/reftable-merged.h" #include "reftable/reftable-stack.h" @@ -20,6 +21,72 @@ static void print_help(void) "\n"); } +static int dump_blocks(const char *tablename) +{ + struct reftable_table_iterator ti = { 0 }; + struct reftable_block_source src = { 0 }; + struct reftable_table *table = NULL; + uint8_t section_type = 0; + int err; + + err = reftable_block_source_from_file(&src, tablename); + if (err < 0) + goto done; + + err = reftable_table_new(&table, &src, tablename); + if (err < 0) + goto done; + + err = reftable_table_iterator_init(&ti, table); + if (err < 0) + goto done; + + printf("header:\n"); + printf(" block_size: %d\n", table->block_size); + + while (1) { + const struct reftable_block *block; + + err = reftable_table_iterator_next(&ti, &block); + if (err < 0) + goto done; + if (err > 0) + break; + + if (block->block_type != section_type) { + const char *section; + switch (block->block_type) { + case REFTABLE_BLOCK_TYPE_LOG: + section = "log"; + break; + case REFTABLE_BLOCK_TYPE_REF: + section = "ref"; + break; + case REFTABLE_BLOCK_TYPE_OBJ: + section = "obj"; + break; + case REFTABLE_BLOCK_TYPE_INDEX: + section = "idx"; + break; + default: + err = -1; + goto done; + } + + section_type = block->block_type; + printf("%s:\n", section); + } + + printf(" - length: %u\n", block->restart_off); + printf(" restarts: %u\n", block->restart_count); + } + +done: + reftable_table_iterator_release(&ti); + reftable_table_decref(table); + return err; +} + static int dump_table(struct reftable_merged_table *mt) { struct reftable_iterator it = { NULL }; @@ -184,7 +251,7 @@ int cmd__dump_reftable(int argc, const char **argv) arg = argv[1]; if (opt_dump_blocks) { - err = reftable_table_print_blocks(arg); + err = dump_blocks(arg); } else if (opt_dump_table) { err = dump_reftable(arg); } else if (opt_dump_stack) { diff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh index e2708e11d5b..e4c7461ce9e 100755 --- a/t/t0613-reftable-write-options.sh +++ b/t/t0613-reftable-write-options.sh @@ -93,6 +93,9 @@ test_expect_success 'many refs results in multiple blocks' ' restarts: 3 - length: 3289 restarts: 3 + idx: + - length: 103 + restarts: 1 EOF test-tool dump-reftable -b .git/reftable/*.ref >actual && test_cmp expect actual @@ -241,6 +244,9 @@ test_expect_success 'object index gets written by default with ref index' ' restarts: 1 - length: 80 restarts: 1 + idx: + - length: 55 + restarts: 2 obj: - length: 11 restarts: 1 @@ -277,6 +283,9 @@ test_expect_success 'object index can be disabled' ' restarts: 1 - length: 80 restarts: 1 + idx: + - length: 55 + restarts: 2 EOF test-tool dump-reftable -b .git/reftable/*.ref >actual && test_cmp expect actual