From patchwork Tue Mar 5 12:11:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13582297 Received: from fhigh5-smtp.messagingengine.com (fhigh5-smtp.messagingengine.com [103.168.172.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F47E5A0FA for ; Tue, 5 Mar 2024 12:11:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.156 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709640684; cv=none; b=keiEC9rVKdtiy0/m18W/JuGVU/X8wtUHgH16lXpNiIeE89iBqOc8RVF72Hj+eylWMjs3W8iWq5XwPEWV/544Une54Le/ACMyubpLf/VLTyzjVGzr/ZeqwkYu/lzX8Jj2I2Bl5D33VFKjeOAp2qCNjAHpLqTw9LQfRwo+YuXzL8U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709640684; c=relaxed/simple; bh=3R5mi9XLorxR5mWhqOYVbbcNUODKZrsp1PSF9/lWst4=; h=Date:From:To:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=c7Aw5Fc1126QbDXKK3toHbT8/vtaEbSIyaAjn4epopiIr2J4ubvc5GmxioXzFTmxgeRyUj+rTKFrBX52WesbSpyLYzUnt28wzNpnu0ZwPXCv5cd0GMA1Anen8ZVthMIicIW+TEEjk3QekLzgy+Ko/LpAWZ0MHMRBKn90sAl5X8Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=JcNqW7fL; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=W2KvjH8d; arc=none smtp.client-ip=103.168.172.156 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="JcNqW7fL"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="W2KvjH8d" Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 5C9C411401B6 for ; Tue, 5 Mar 2024 07:11:22 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Tue, 05 Mar 2024 07:11:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1709640682; x=1709727082; bh=99O3wkPsbU QJoW9zhCqu0vAWyYRoTNOssRZw29eailc=; b=JcNqW7fLxCyCsUJcreHzAS3oxN qxYHaT0wetO8zUZzDOZx7RQRC/8QPczjsfa3/zbszGR4JHYDPkfm6hf3myJFEPzf BDYjMjtzq/f6ceG3UZvMz79G+DQpdFOsNazglTqziikmlu0k9vBDVIF8UUJDSzB/ L7fB4Xn6v+zgFzrefPH6kMeg3pyMJYPvkNuABnPUESIsVzMZjOc2uUef9i9o/w/e qi+mKCNBcW8bJRpqVUd8wUnUwuCDVweuYRwy8XNKn0I430VpNxOl228iM12Y83Cp Iw6qtM9Tr/gAngdPvPd/BZ4WhzOAPmBfhdeR9hN76B5oPbxs48A/Bnx7hXZA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1709640682; x=1709727082; bh=99O3wkPsbUQJoW9zhCqu0vAWyYRo TNOssRZw29eailc=; b=W2KvjH8d1XIuEdFRzp8uqNZ58yHYYB5soRO2RkUHd5pS 8tep2xNs9VI8IONO+l2kBYDkITBCnzOshsTbv1OwdOYofX8rIFAKPUcRttUxwqSi oPjNP4Az5e9Te+sUYNhlgA6BZXxnkYQOqVc/N39E+tam2bBOeigUreUMtURHjHg8 7eOUHM61kgOVRfNvYBQLbz3DuJBT92GG67tOylLx0EeXdrttjGBGSex0qy5cleuM 3CkgA1GU9n+RO7lxMbSHi5mkAwNOwoRYxGV1sBaXOo/eaR2gD2GUi9pc4nqkB4LF fUmLjYF80vHNOQX7pOJKpT/mGrLqvw0sIRc5sladHg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrheelgdefgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtderre dttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhk shdrihhmqeenucggtffrrghtthgvrhhnpeehgefhtdefueffheekgfffudelffejtdfhvd ejkedthfehvdelgfetgfdvtedthfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr mhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 5 Mar 2024 07:11:21 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 8d231cf1 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Tue, 5 Mar 2024 12:06:55 +0000 (UTC) Date: Tue, 5 Mar 2024 13:11:20 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 7/7] refs/reftable: track last log record name via strbuf Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The reflog iterator enumerates all reflogs known to a ref backend. In the "reftable" backend there is no way to list all existing reflogs directly. Instead, we have to iterate through all reflog entries and discard all those redundant entries for which we have already returned a reflog entry. This logic is implemented by tracking the last reflog name that we have emitted to the iterator's user. If the next log record has the same name we simply skip it until we find another record with a different refname. This last reflog name is stored in a simple C string, which requires us to free and reallocate it whenever we need to update the reflog name. Convert it to use a `struct strbuf` instead, which reduces the number of allocations. Before: HEAP SUMMARY: in use at exit: 13,473 bytes in 122 blocks total heap usage: 1,068,485 allocs, 1,068,363 frees, 281,122,886 bytes allocated After: HEAP SUMMARY: in use at exit: 13,473 bytes in 122 blocks total heap usage: 68,485 allocs, 68,363 frees, 256,234,072 bytes allocated Note that even after this change we still allocate quite a lot of data, even though the number of allocations does not scale with the number of log records anymore. This remainder comes mostly from decompressing the log blocks, where we decompress each block into newly allocated memory. This will be addressed at a later point in time. Signed-off-by: Patrick Steinhardt --- refs/reftable-backend.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 2de57c047a..12960d93ff 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1575,7 +1575,7 @@ struct reftable_reflog_iterator { struct reftable_ref_store *refs; struct reftable_iterator iter; struct reftable_log_record log; - char *last_name; + struct strbuf last_name; int err; }; @@ -1594,15 +1594,15 @@ static int reftable_reflog_iterator_advance(struct ref_iterator *ref_iterator) * we've already produced this name. This could be faster by * seeking directly to reflog@update_index==0. */ - if (iter->last_name && !strcmp(iter->log.refname, iter->last_name)) + if (!strcmp(iter->log.refname, iter->last_name.buf)) continue; if (check_refname_format(iter->log.refname, REFNAME_ALLOW_ONELEVEL)) continue; - free(iter->last_name); - iter->last_name = xstrdup(iter->log.refname); + strbuf_reset(&iter->last_name); + strbuf_addstr(&iter->last_name, iter->log.refname); iter->base.refname = iter->log.refname; break; @@ -1635,7 +1635,7 @@ static int reftable_reflog_iterator_abort(struct ref_iterator *ref_iterator) (struct reftable_reflog_iterator *)ref_iterator; reftable_log_record_release(&iter->log); reftable_iterator_destroy(&iter->iter); - free(iter->last_name); + strbuf_release(&iter->last_name); free(iter); return ITER_DONE; } @@ -1655,6 +1655,7 @@ static struct reftable_reflog_iterator *reflog_iterator_for_stack(struct reftabl iter = xcalloc(1, sizeof(*iter)); base_ref_iterator_init(&iter->base, &reftable_reflog_iterator_vtable); + strbuf_init(&iter->last_name, 0); iter->refs = refs; ret = refs->err;