From patchwork Wed Oct 12 22:01:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13005486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95DAAC433FE for ; Wed, 12 Oct 2022 22:05:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229815AbiJLWFJ (ORCPT ); Wed, 12 Oct 2022 18:05:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230129AbiJLWE1 (ORCPT ); Wed, 12 Oct 2022 18:04:27 -0400 Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6DEC13956B for ; Wed, 12 Oct 2022 15:01:51 -0700 (PDT) Received: by mail-io1-xd34.google.com with SMTP id p186so14585142iof.5 for ; Wed, 12 Oct 2022 15:01:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20210112.gappssmtp.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=OUXQftnY6+nqpLxksDLiwUZR+iNRuUwx4tvVipJLAr0=; b=F74qKWlqpTdmCtdyYzS/QuQRpHLA/UECKMIJMDdcXhbWkIktn6k/JLzENtAEHbkNO9 AQ21oPzUrC04WU7vPnGKibKJcKQMBPXkD0bbPNVctgnuDEdrLkT0p/3QHONY7BOHyLTe g8etjq3F7u3gAAPuZn1cxM20IN3IiZHKsz8ahIW2Vd7hOhkVOJL904sreGlU55t20wkP guKDDVZiAxels9aDB/SHlaHSXPllBeWVZPpDsn8CbrKZnnMovP+N4tWEKpZQx/AfOvmV 0HNDDSFF6hh74wxiFv0izBvxOw788lDEqpQP9sM/Zkxq8iSwS4YCkUhxJ5O2L0o0BeZw 7vPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=OUXQftnY6+nqpLxksDLiwUZR+iNRuUwx4tvVipJLAr0=; b=1nsN2aJIQ126tKvwX6GT6/zbDpMsGNl4dwuc9b8QlVNHDIV5g52Oc9s+UyBUua9caZ 1eMFU2cnbWT+06roVf/O5dDrofiDWJySVpEu6Mjd6Syg9Wgr4YQSm8H5k5HApVmpEmvy kb1Gr8DRcIm9ODB4Pz0fe5tvCVTwBnZphoIs0WQI1F7RmHOi9XdFwYwQnZbhh74Ehk62 +OwiMt+dVfidlBMdBmEXDQEhZA8kJM1OMa0YCRPU40zvt5mXrloUNHroqWe0v+ZdUxFE 3DkMPLOcxbfza0Bw1OQ/U6PHUSeLG6GTlPsi2XlJTBiakL80BJCW7ye5BC5UMuaEQIlt 9EMg== X-Gm-Message-State: ACrzQf19fxW/6PUUgkISz2J0PgEd5DVfQBbD6Rftmn8cLXY6j1doEtKW hs0DBiHKnX50XJAsn50X4yZuAfuNxzpP/oUx X-Google-Smtp-Source: AMsMyM61WRj+O+5TVdj3iGZFKkZCViShXYNt4LeyoQ+xTqaQBnPpT6M3Dkj9ZzuaV/KlhUpn9fV9pg== X-Received: by 2002:a05:6638:25cb:b0:363:edfe:3e3f with SMTP id u11-20020a05663825cb00b00363edfe3e3fmr1020018jat.270.1665612110455; Wed, 12 Oct 2022 15:01:50 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id c6-20020a92cf46000000b002faf7965948sm3779596ilr.55.2022.10.12.15.01.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 15:01:49 -0700 (PDT) Date: Wed, 12 Oct 2022 18:01:48 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, derrickstolee@github.com, vdye@github.com, gitster@pobox.com Subject: [PATCH 1/4] midx.c: fix whitespace typo Message-ID: <6e92b7707dbe6ec993cf4a36180f5c118e6011d7.1665612094.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org This was unintentionally introduced via 893b563505 (midx: inline nth_midxed_pack_entry(), 2021-09-11) where "struct repository *r" became "struct repository * r". The latter does not adhere to our usual style conventions, so fix that up to look more like our usual declarations. Signed-off-by: Taylor Blau --- midx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/midx.c b/midx.c index 3a8dcfe98e..a9df753220 100644 --- a/midx.c +++ b/midx.c @@ -278,7 +278,7 @@ uint32_t nth_midxed_pack_int_id(struct multi_pack_index *m, uint32_t pos) (off_t)pos * MIDX_CHUNK_OFFSET_WIDTH); } -int fill_midx_entry(struct repository * r, +int fill_midx_entry(struct repository *r, const struct object_id *oid, struct pack_entry *e, struct multi_pack_index *m) From patchwork Wed Oct 12 22:01:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13005487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8ED6C433FE for ; Wed, 12 Oct 2022 22:05:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230137AbiJLWFL (ORCPT ); Wed, 12 Oct 2022 18:05:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230134AbiJLWE1 (ORCPT ); Wed, 12 Oct 2022 18:04:27 -0400 Received: from mail-il1-x132.google.com (mail-il1-x132.google.com [IPv6:2607:f8b0:4864:20::132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7C88459BF for ; Wed, 12 Oct 2022 15:01:54 -0700 (PDT) Received: by mail-il1-x132.google.com with SMTP id d14so85892ilf.2 for ; Wed, 12 Oct 2022 15:01:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20210112.gappssmtp.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=CGw1o+/lUVje47aRWmFO+gWu/rY1GqHjq2CWRyTfyWU=; b=DsDInTBpDBmsTPbUo8r4R8QJq9pnv5qiUFOK7fyFZh8c+u5CSnoaVUQf0lOTo7LhZq cPnFJ4JbaVQrZYMDVHdG3Nw5vo08QnsCtjzph58zAoUCVCrmc+GcOlZZDKlwvGr9sAFu j7/IwfrICzIur/j3reSA5cSqOHTIXBOvw5rFEBnmy2k/wCi4S8DNu2fJ6ponwIDlUIk+ r36QHYoM2qnoFV7HhWk/oremrllfORfC/gGtlOMAQCC/9pUacbFy8Uyki6sQw773To17 rsi5ZoUPzZ51GfDlqZXLoI2gcKWSmagBauQ4T3FyfDjsy1zuaxM64ZOGpTXJMbpFl1VU WWkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=CGw1o+/lUVje47aRWmFO+gWu/rY1GqHjq2CWRyTfyWU=; b=wfN1L0mKua/QGbNCsIcOJz3dRtEW8bV//AGSsD6NNeSWU1nrdapSHtN4if+djQ4xyx p0h9ne2QM78WGGQNFMXz2RqzbsnDLs1qoyKZzaF/pAn1izX2feYkDmjD8xnMoNb86Lp5 2fjFmqQV6p67yT3nDwHcKI6Y6M6xEDewCl21yfTid+6LLYg+1KBJgnfl87xK2HckCnvU kEaL83H06I0XctBd7t/Mp9TQuySyQchzpWMOURIkDiBq8mUyy1cFwJbYvSmmeQkX37e6 u//gCztohFK1cDzagIuN7AQvrcyNQkfK5LCzo5QLau07YxNouKRNGSuQHgmvdgqfXObv pg0Q== X-Gm-Message-State: ACrzQf14lVJzli4+sj+E0FmUtFoElw7BT0tMMX6i828Liwz/MUwMBOAJ Y4YQkasAbfvp+Jgda8cov1qVbdJMIKmHwahR X-Google-Smtp-Source: AMsMyM6AH/5LKO157YELt2wRdo7u07rdR/Dks9WpqyjTUdoQY4tgablHWME5BlELR7oGCybm3zibNg== X-Received: by 2002:a05:6e02:1563:b0:2fb:6601:80d9 with SMTP id k3-20020a056e02156300b002fb660180d9mr11776974ilu.71.1665612113664; Wed, 12 Oct 2022 15:01:53 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id r17-20020a92d991000000b002e8ea827aafsm6213729iln.74.2022.10.12.15.01.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 15:01:53 -0700 (PDT) Date: Wed, 12 Oct 2022 18:01:52 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, derrickstolee@github.com, vdye@github.com, gitster@pobox.com Subject: [PATCH 2/4] midx.c: consider annotated tags during bitmap selection Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When generating a multi-pack bitmap without a `--refs-snapshot` (e.g., by running `git multi-pack-index write --bitmap` directly), we determine the set of bitmap-able commits by enumerating each reference, and adding the referrent as the tip of a reachability traversal when it appears somewhere in the MIDX. (Any commit we encounter during the reachability traversal then becomes a candidate for bitmap selection). But we incorrectly avoid peeling the object at the tip of each reference. So if we see some reference that points at an annotated tag (which in turn points through zero or more additional annotated tags at a commit), that we will not add it as a tip for the reachability traversal. This means that if some commit C is only referenced through one or more annotated tag(s), then C won't become a bitmap candidate. Correct this by peeling the reference tips as we enumerate them to ensure that we consider commits which are the targets of annotated tags, in addition to commits which are referenced directly. Signed-off-by: Taylor Blau --- midx.c | 4 ++++ t/t5326-multi-pack-bitmaps.sh | 24 ++++++++++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/midx.c b/midx.c index a9df753220..d41dfd8174 100644 --- a/midx.c +++ b/midx.c @@ -980,6 +980,7 @@ static int add_ref_to_pending(const char *refname, int flag, void *cb_data) { struct rev_info *revs = (struct rev_info*)cb_data; + struct object_id peeled; struct object *object; if ((flag & REF_ISSYMREF) && (flag & REF_ISBROKEN)) { @@ -987,6 +988,9 @@ static int add_ref_to_pending(const char *refname, return 0; } + if (!peel_iterated_oid(oid, &peeled)) + oid = &peeled; + object = parse_object_or_die(oid, refname); if (object->type != OBJ_COMMIT) return 0; diff --git a/t/t5326-multi-pack-bitmaps.sh b/t/t5326-multi-pack-bitmaps.sh index ad6eea5fa0..0882cbb6e4 100755 --- a/t/t5326-multi-pack-bitmaps.sh +++ b/t/t5326-multi-pack-bitmaps.sh @@ -410,4 +410,28 @@ test_expect_success 'preferred pack change with existing MIDX bitmap' ' ) ' +test_expect_success 'tagged commits are selected for bitmapping' ' + rm -fr repo && + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit --annotate base && + git repack -d && + + # Remove refs/heads/main which points at the commit directly, + # leaving only a reference to the annotated tag. + git branch -M main && + git checkout base && + git branch -d main && + + git multi-pack-index write --bitmap && + + git rev-parse HEAD >want && + test-tool bitmap list-commits >actual && + grep $(cat want) actual + ) +' + test_done From patchwork Wed Oct 12 22:01:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13005488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 077FEC433FE for ; Wed, 12 Oct 2022 22:05:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230001AbiJLWFU (ORCPT ); Wed, 12 Oct 2022 18:05:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229561AbiJLWE3 (ORCPT ); Wed, 12 Oct 2022 18:04:29 -0400 Received: from mail-io1-xd2a.google.com (mail-io1-xd2a.google.com [IPv6:2607:f8b0:4864:20::d2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9BE5F12C893 for ; Wed, 12 Oct 2022 15:01:57 -0700 (PDT) Received: by mail-io1-xd2a.google.com with SMTP id r142so10434355iod.11 for ; Wed, 12 Oct 2022 15:01:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20210112.gappssmtp.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=1Pq8CdmarZyBLqUJFowBNihbqqz1CcoqXvVHo5rt8o8=; b=szxjUpLosjIMLGyheL88lgU3KsZB/ZmJDW4EYRhG94DzMMGBmimVc2lFMpk62975AX cmh6a2gd/d2e/EOlftg9C4dAcXyB2066UqM+1CiPLeW0fuCOntB9KvMpHvUXH6xL6was gcDKHZ5QW1OK5TFH9Eo2icIzBn3GtTGtE7+S/KcnV4qNLOs695Z4v0e4DSAgVf9CkaXE ecVAS7SvvxPfVsQJ/9ssganXCJABo45FoC+jpG89P+wQ+oBEkyHnzn7k853ZjZ5h+4Au HL7PgQo4JtuHSAvt6ZNz5aVTR6BftZp1oyCThrYwm0mIcp6F/bJNsxPfjYFkotdAOxaY MSwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=1Pq8CdmarZyBLqUJFowBNihbqqz1CcoqXvVHo5rt8o8=; b=whfi3t2hDJpk2TlQg8C9GHatzkh42ovkE2MH3FV83rOxAUVYcxZYf7fXB8jYF/Gb1i VZ4+2JbaoLChauvAdjwrVbkfzstcLUPK+MNmMtv0BpLFf/bETGAxOK/JLBxGIfpQxYR3 Gp/7x+9+9s44eepOjqASZmawjTfS01KDifWPnq3GczO+wizv0msCdN174CDaHY5dPmis ukTwCFBPo+u4xe5sfjxTc0Hvm16SNSZ05Ay0CybCLcZm4ChbIAiJrK5AHNuB2i4R8ftQ +sPWUNfe56QGoDdTeM6g3B57GCycTslhe4IFuUcyFZqWHD1t/8LFUo39Qw/mHxteHRDz iLyA== X-Gm-Message-State: ACrzQf2HUPiCegLUNQNy/H/qEhyPdh/Om7QpMJfntZyyEglKrFQaIZho biIjkTNITCzzQzJlPQ9H0G5HbvblMfhQLbMK X-Google-Smtp-Source: AMsMyM6HzzDcNBVivwo65f0FnrVTrHfNRzPzhWo3Yf8oU8MGLZu8//5YvRaibhiGIqB9pCWvZTldiQ== X-Received: by 2002:a05:6602:1409:b0:6a4:cef3:ea7f with SMTP id t9-20020a056602140900b006a4cef3ea7fmr14882317iov.41.1665612116817; Wed, 12 Oct 2022 15:01:56 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id i132-20020a6bb88a000000b006bbf0466587sm5361296iof.49.2022.10.12.15.01.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 15:01:56 -0700 (PDT) Date: Wed, 12 Oct 2022 18:01:55 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, derrickstolee@github.com, vdye@github.com, gitster@pobox.com Subject: [PATCH 3/4] midx.c: instrument MIDX and bitmap generation with trace2 regions Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When debugging MIDX and MIDX-bitmap related issues, it is useful to figure out where Git is spending its time. GitHub has been using the below trace2 regions to instrument various components of generating a MIDX itself, as well time spent preparing to build a MIDX bitmap. These are limited to instrumenting the following functions: - midx.c::find_commits_for_midx_bitmap() - midx.c::midx_pack_order() - midx.c::prepare_midx_packing_data() - midx.c::write_midx_bitmap() - midx.c::write_midx_internal() - midx.c::write_midx_reverse_index() to start and end with a trace2_region_enter() and trace2_region_leave(), respectively. The category for all of these is "midx", which matches the existing convention. The region description matches the name of the function. Signed-off-by: Taylor Blau --- midx.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/midx.c b/midx.c index d41dfd8174..7cfad04a24 100644 --- a/midx.c +++ b/midx.c @@ -913,6 +913,8 @@ static uint32_t *midx_pack_order(struct write_midx_context *ctx) uint32_t *pack_order; uint32_t i; + trace2_region_enter("midx", "midx_pack_order", the_repository); + ALLOC_ARRAY(data, ctx->entries_nr); for (i = 0; i < ctx->entries_nr; i++) { struct pack_midx_entry *e = &ctx->entries[i]; @@ -930,6 +932,8 @@ static uint32_t *midx_pack_order(struct write_midx_context *ctx) pack_order[i] = data[i].nr; free(data); + trace2_region_leave("midx", "midx_pack_order", the_repository); + return pack_order; } @@ -939,6 +943,8 @@ static void write_midx_reverse_index(char *midx_name, unsigned char *midx_hash, struct strbuf buf = STRBUF_INIT; const char *tmp_file; + trace2_region_enter("midx", "write_midx_reverse_index", the_repository); + strbuf_addf(&buf, "%s-%s.rev", midx_name, hash_to_hex(midx_hash)); tmp_file = write_rev_file_order(NULL, ctx->pack_order, ctx->entries_nr, @@ -948,6 +954,8 @@ static void write_midx_reverse_index(char *midx_name, unsigned char *midx_hash, die(_("cannot store reverse index file")); strbuf_release(&buf); + + trace2_region_leave("midx", "write_midx_reverse_index", the_repository); } static void clear_midx_files_ext(const char *object_dir, const char *ext, @@ -963,6 +971,8 @@ static void prepare_midx_packing_data(struct packing_data *pdata, { uint32_t i; + trace2_region_enter("midx", "prepare_midx_packing_data", the_repository); + memset(pdata, 0, sizeof(struct packing_data)); prepare_packing_data(the_repository, pdata); @@ -973,6 +983,8 @@ static void prepare_midx_packing_data(struct packing_data *pdata, oe_set_in_pack(pdata, to, ctx->info[ctx->pack_perm[from->pack_int_id]].p); } + + trace2_region_leave("midx", "prepare_midx_packing_data", the_repository); } static int add_ref_to_pending(const char *refname, @@ -1070,6 +1082,9 @@ static struct commit **find_commits_for_midx_bitmap(uint32_t *indexed_commits_nr struct rev_info revs; struct bitmap_commit_cb cb = {0}; + trace2_region_enter("midx", "find_commits_for_midx_bitmap", + the_repository); + cb.ctx = ctx; repo_init_revisions(the_repository, &revs, NULL); @@ -1103,6 +1118,10 @@ static struct commit **find_commits_for_midx_bitmap(uint32_t *indexed_commits_nr *indexed_commits_nr_p = cb.commits_nr; release_revisions(&revs); + + trace2_region_leave("midx", "find_commits_for_midx_bitmap", + the_repository); + return cb.commits; } @@ -1120,6 +1139,8 @@ static int write_midx_bitmap(const char *midx_name, char *bitmap_name = xstrfmt("%s-%s.bitmap", midx_name, hash_to_hex(midx_hash)); + trace2_region_enter("midx", "write_midx_bitmap", the_repository); + if (flags & MIDX_WRITE_BITMAP_HASH_CACHE) options |= BITMAP_OPT_HASH_CACHE; @@ -1165,6 +1186,9 @@ static int write_midx_bitmap(const char *midx_name, cleanup: free(index); free(bitmap_name); + + trace2_region_leave("midx", "write_midx_bitmap", the_repository); + return ret; } @@ -1211,6 +1235,8 @@ static int write_midx_internal(const char *object_dir, int result = 0; struct chunkfile *cf; + trace2_region_enter("midx", "write_midx_internal", the_repository); + get_midx_filename(&midx_name, object_dir); if (safe_create_leading_directories(midx_name.buf)) die_errno(_("unable to create leading directories of %s"), @@ -1552,6 +1578,8 @@ static int write_midx_internal(const char *object_dir, free(ctx.pack_order); strbuf_release(&midx_name); + trace2_region_leave("midx", "write_midx_internal", the_repository); + return result; } From patchwork Wed Oct 12 22:01:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13005489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB959C4332F for ; Wed, 12 Oct 2022 22:05:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229704AbiJLWFW (ORCPT ); Wed, 12 Oct 2022 18:05:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42294 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229591AbiJLWEa (ORCPT ); Wed, 12 Oct 2022 18:04:30 -0400 Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CA3F139568 for ; Wed, 12 Oct 2022 15:02:00 -0700 (PDT) Received: by mail-io1-xd2d.google.com with SMTP id r142so10434436iod.11 for ; Wed, 12 Oct 2022 15:02:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20210112.gappssmtp.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=rf4GiHqvJ7NfeILaEa63RnOcL/50pM6Dth49LXa/i8s=; b=Mgrhhlg1hMHHU/6U9O/aw0q5iCxNAQIUMsNDgiXJOyIVSJA9L0NW7CQm7JyMeJlIWt PP2Lt/OGVe9gVcIKrmV/pCAMvUS0S8PJUn7a6KW1FCrHzr7Jy/mc+0palsx1ZqeuQaqU tQy7W3FVehPgaf4XD+vsAA4Cdglebo2Y3bg2f1rvVkAroGa1ufWZ7OuA1KiOuZBmavcz 5z9LeE/U52xI5BjAU1zDY6pUTqYBxrVlN9q+QHyoywgHORWGfLJT+TJii5ppnJ4Njbi3 +YhoqkGMeHo4rnDkWiZkMS9SaWs8gInzwBodEf9XE5d+5QhessfV+EzP2ae6J2auFnmL 9arQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=rf4GiHqvJ7NfeILaEa63RnOcL/50pM6Dth49LXa/i8s=; b=yCX+vCFpb/u37YuX6SzwCrlRT65KWp0MbbhnO4FqyBhgcX6uxrG6+ECLZU0oUXWGcM NFi6jBLuz0vQbiZOGZkBk7IpId9NeDXzZG1yfxB6edjM5o3eF1jepCALiUiX2KNTsLnE bc/MzlU/TO8b6wnyWs1IXaJDAl/YYK5I2WVt0+RjWGQqDPRhYJqN3vOKMzXH5cKNb6SF GLWSOXwnSBQPlkgtf7GsuiJXQQxn8hyXz5EwO2fZKkD/nVdVZJKFLJeUhlFkbn4REWSm 5QYfHppyQ31sJ9gVd0AaNYp+jr3//47UanXHnqEqBHFszkX1FsqxI0zH9apM9+BELRt+ Dyfw== X-Gm-Message-State: ACrzQf1zeCy+k9dVIkF1TrqrVi6bOowyzo9XCgPeMmE4+0SDdRZ7DwZg hYWBf+w/P/PKBppvZmzeFI+Y99hI6PesGXcD X-Google-Smtp-Source: AMsMyM4YxtaOfSl3vUr7supNbSazbVbjpmICS9fMC/FFVdgxraqQurLX/eqvWaPR+eb/INBh1JgbfQ== X-Received: by 2002:a05:6602:2e08:b0:6a1:30d1:7ca7 with SMTP id o8-20020a0566022e0800b006a130d17ca7mr15338237iow.45.1665612119353; Wed, 12 Oct 2022 15:01:59 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id z11-20020a05663803ab00b0036335a126d6sm5883442jap.105.2022.10.12.15.01.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 15:01:58 -0700 (PDT) Date: Wed, 12 Oct 2022 18:01:57 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: peff@peff.net, derrickstolee@github.com, vdye@github.com, gitster@pobox.com Subject: [PATCH 4/4] pack-bitmap-write.c: instrument number of reused bitmaps Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When debugging bitmap generation performance, it is useful to know how many bitmaps were generated from scratch, and how many were the result of permuting the bit-order of an existing bitmap. Keep track of the latter, and emit the count as a trace2_data line to aid in debugging. Signed-off-by: Taylor Blau --- pack-bitmap-write.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/pack-bitmap-write.c b/pack-bitmap-write.c index a213f5eddc..cfa67a510f 100644 --- a/pack-bitmap-write.c +++ b/pack-bitmap-write.c @@ -384,6 +384,8 @@ static int fill_bitmap_tree(struct bitmap *bitmap, return 0; } +static int reused_bitmaps_nr; + static int fill_bitmap_commit(struct bb_commit *ent, struct commit *commit, struct prio_queue *queue, @@ -409,8 +411,10 @@ static int fill_bitmap_commit(struct bb_commit *ent, * bitmap and add its bits to this one. No need to walk * parents or the tree for this commit. */ - if (old && !rebuild_bitmap(mapping, old, ent->bitmap)) + if (old && !rebuild_bitmap(mapping, old, ent->bitmap)) { + reused_bitmaps_nr++; continue; + } } /* @@ -526,6 +530,8 @@ int bitmap_writer_build(struct packing_data *to_pack) trace2_region_leave("pack-bitmap-write", "building_bitmaps_total", the_repository); + trace2_data_intmax("pack-bitmap-write", the_repository, + "building_bitmaps_reused", reused_bitmaps_nr); stop_progress(&writer.progress);