From patchwork Wed Jan 3 06:22:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13509683 Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5BCF918623 for ; Wed, 3 Jan 2024 06:22:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="UEZYtAw2"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="KigAHhbm" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 728DB5C01E3; Wed, 3 Jan 2024 01:22:37 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Wed, 03 Jan 2024 01:22:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1704262957; x=1704349357; bh=LUEzIz2MvW GXS0GMwIs0PepWxY49DadEnt4HKy0NAbE=; b=UEZYtAw2b4580caghQSiG5ILHM 8ymDWh2vL7mgSyVmPS4jiOCTUWjVtePgOnrNPPt4pM+Kb8Er3DVDwrkxqB+14SBf Y1sEOL8bPTq6KA+wT6zRNZIaNXa3P9Lf6i/uUnb89VWScFbf/92XTyJsnIE9fule B104EmFkTufq5UmOQ89WTvJFunejvRBz9Ng/NOsTtunCbpFo1y0q/EkoktBgZOjW UAugTOXJOcwU09cRQWn5WmGFv1CdEEaCTlMoXMXleJViA7ftVStcaggPEB0TRSuZ ZJIXNStkvrDJ9al4hD3k+ZjXldV4LSToTYcegKSx5aSGgs5/aDXF3UT6/nig== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1704262957; x=1704349357; bh=LUEzIz2MvWGXS0GMwIs0PepWxY49 DadEnt4HKy0NAbE=; b=KigAHhbmGPxIgPkf396hM07WQZu++/xiqvEub2p2cSQU eXmFmiTV0w4orddm82T6z8aeot892QNctumN7jbLT73U2d/gRh47lVbbw9CmBwNV 6G/D3WQt9YAwmdmrdLG+XLdlBqNDGF2yyQOhzTL5ww4wA6EZuHrr7f1FlVfK0dpB ZpMmz1QAA78SwMVBt3aF6adcMVCFmDRaFdJZLTAM0r2xsfyidVL71TTPMWSPBZ1o u+TC7RmJGrOJ7G+8MY1SHLQZ1D88LVD/1KLYZcAsupaOXK1d9/GbutKn4iKicEi5 Rwmhlf+2xrSnMFct+vnQfFdtGSRzAX+xoxJj4Vb8pA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvdeggedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvvefukfhfgggtuggjsehgtd erredttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshes phhkshdrihhmqeenucggtffrrghtthgvrhhnpeehjeffudefkeejheekfefguedvtdehte fgiefgveelgefgheekhfdvjeefvdekudenucffohhmrghinhepvhgrlhdvrdhtrghrghgv thenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 3 Jan 2024 01:22:36 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id e172148d (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Wed, 3 Jan 2024 06:20:11 +0000 (UTC) Date: Wed, 3 Jan 2024 07:22:34 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Han-Wen Nienhuys , Junio C Hamano Subject: [PATCH v3 6/8] reftable/record: store "val2" hashes as static arrays Message-ID: <6ec02d6775c0f3e3274d57ef4a7dfeefac8308a9.1704262787.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Similar to the preceding commit, convert ref records of type "val2" to store their object IDs in static arrays instead of allocating them for every single record. We're using the same benchmark as in the preceding commit, with `git show-ref --quiet` in a repository with ~350k refs. This time around though the effects aren't this huge. Before: HEAP SUMMARY: in use at exit: 21,163 bytes in 193 blocks total heap usage: 1,419,040 allocs, 1,418,847 frees, 62,153,868 bytes allocated After: HEAP SUMMARY: in use at exit: 21,163 bytes in 193 blocks total heap usage: 1,410,148 allocs, 1,409,955 frees, 61,976,068 bytes allocated This is because "val2"-type records are typically only stored for peeled tags, and the number of annotated tags in the benchmark repository is rather low. Still, it can be seen that this change leads to a reduction of allocations overall, even if only a small one. Signed-off-by: Patrick Steinhardt --- reftable/readwrite_test.c | 12 ++++-------- reftable/record.c | 6 ------ reftable/record_test.c | 4 ---- reftable/reftable-record.h | 4 ++-- 4 files changed, 6 insertions(+), 20 deletions(-) diff --git a/reftable/readwrite_test.c b/reftable/readwrite_test.c index 87b238105c..178766dfa8 100644 --- a/reftable/readwrite_test.c +++ b/reftable/readwrite_test.c @@ -547,8 +547,6 @@ static void test_table_refs_for(int indexed) uint8_t hash[GIT_SHA1_RAWSZ]; char fill[51] = { 0 }; char name[100]; - uint8_t hash1[GIT_SHA1_RAWSZ]; - uint8_t hash2[GIT_SHA1_RAWSZ]; struct reftable_ref_record ref = { NULL }; memset(hash, i, sizeof(hash)); @@ -558,11 +556,9 @@ static void test_table_refs_for(int indexed) name[40] = 0; ref.refname = name; - set_test_hash(hash1, i / 4); - set_test_hash(hash2, 3 + i / 4); ref.value_type = REFTABLE_REF_VAL2; - ref.value.val2.value = hash1; - ref.value.val2.target_value = hash2; + set_test_hash(ref.value.val2.value, i / 4); + set_test_hash(ref.value.val2.target_value, 3 + i / 4); /* 80 bytes / entry, so 3 entries per block. Yields 17 */ @@ -570,8 +566,8 @@ static void test_table_refs_for(int indexed) n = reftable_writer_add_ref(w, &ref); EXPECT(n == 0); - if (!memcmp(hash1, want_hash, GIT_SHA1_RAWSZ) || - !memcmp(hash2, want_hash, GIT_SHA1_RAWSZ)) { + if (!memcmp(ref.value.val2.value, want_hash, GIT_SHA1_RAWSZ) || + !memcmp(ref.value.val2.target_value, want_hash, GIT_SHA1_RAWSZ)) { want_names[want_names_len++] = xstrdup(name); } } diff --git a/reftable/record.c b/reftable/record.c index a67a6b4d8a..5c3fbb7b2a 100644 --- a/reftable/record.c +++ b/reftable/record.c @@ -222,9 +222,7 @@ static void reftable_ref_record_copy_from(void *rec, const void *src_rec, memcpy(ref->value.val1, src->value.val1, hash_size); break; case REFTABLE_REF_VAL2: - ref->value.val2.value = reftable_malloc(hash_size); memcpy(ref->value.val2.value, src->value.val2.value, hash_size); - ref->value.val2.target_value = reftable_malloc(hash_size); memcpy(ref->value.val2.target_value, src->value.val2.target_value, hash_size); break; @@ -298,8 +296,6 @@ void reftable_ref_record_release(struct reftable_ref_record *ref) reftable_free(ref->value.symref); break; case REFTABLE_REF_VAL2: - reftable_free(ref->value.val2.target_value); - reftable_free(ref->value.val2.value); break; case REFTABLE_REF_VAL1: break; @@ -401,11 +397,9 @@ static int reftable_ref_record_decode(void *rec, struct strbuf key, return -1; } - r->value.val2.value = reftable_malloc(hash_size); memcpy(r->value.val2.value, in.buf, hash_size); string_view_consume(&in, hash_size); - r->value.val2.target_value = reftable_malloc(hash_size); memcpy(r->value.val2.target_value, in.buf, hash_size); string_view_consume(&in, hash_size); break; diff --git a/reftable/record_test.c b/reftable/record_test.c index 5c94d26e35..2876db7d27 100644 --- a/reftable/record_test.c +++ b/reftable/record_test.c @@ -122,11 +122,7 @@ static void test_reftable_ref_record_roundtrip(void) set_hash(in.u.ref.value.val1, 1); break; case REFTABLE_REF_VAL2: - in.u.ref.value.val2.value = - reftable_malloc(GIT_SHA1_RAWSZ); set_hash(in.u.ref.value.val2.value, 1); - in.u.ref.value.val2.target_value = - reftable_malloc(GIT_SHA1_RAWSZ); set_hash(in.u.ref.value.val2.target_value, 2); break; case REFTABLE_REF_SYMREF: diff --git a/reftable/reftable-record.h b/reftable/reftable-record.h index 7f3a0df635..bb6e99acd3 100644 --- a/reftable/reftable-record.h +++ b/reftable/reftable-record.h @@ -41,8 +41,8 @@ struct reftable_ref_record { union { unsigned char val1[GIT_MAX_RAWSZ]; struct { - uint8_t *value; /* first value, malloced hash */ - uint8_t *target_value; /* second value, malloced hash */ + unsigned char value[GIT_MAX_RAWSZ]; /* first hash */ + unsigned char target_value[GIT_MAX_RAWSZ]; /* second hash */ } val2; char *symref; /* referent, malloced 0-terminated string */ } value;