From patchwork Tue Nov 20 15:04:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10690473 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A61ED1923 for ; Tue, 20 Nov 2018 15:04:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 96060201A4 for ; Tue, 20 Nov 2018 15:04:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 80AA42892C; Tue, 20 Nov 2018 15:04:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 28FEE201A4 for ; Tue, 20 Nov 2018 15:04:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727810AbeKUBeb (ORCPT ); Tue, 20 Nov 2018 20:34:31 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:39757 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726224AbeKUBeb (ORCPT ); Tue, 20 Nov 2018 20:34:31 -0500 Received: by mail-wr1-f65.google.com with SMTP id b13so2330206wrx.6 for ; Tue, 20 Nov 2018 07:04:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=U7/012vgfZU58ex+5EZrccnyYPYhhrrQXZinDbn9r2w=; b=GXh1wW/uXLniCGa7dW9uDxW8PgGufPEztw7yhT+E8041RgkTlJPlHSrDeamZsxIso2 agNC7iA8OqJ+suyVkdm3PNnfSZK/RQVxfFe7dz2KyDgGVoOEsWTDICOGZHALR/Yh5boV BYk/A9UzYg/tydmjgjdP39sgSw0V3Dfjgt8u2KCSgNUPot9gwQRKk+ccvJ1YZJKQdp1Q wGT333k68+l4q8nfr0nvgf+H/8fGA3lr7EXF16zAGroIhFsab7q+UZ30TFNWz7v0RrDd 6pmvdMESRqbNSYLD1BUt38Kk0qnBsMszbgBhX4fCgugo3NG4be5xpnMT4ZUWIv4iR821 LAmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=U7/012vgfZU58ex+5EZrccnyYPYhhrrQXZinDbn9r2w=; b=mqs8YaDPCqMdbvNgo5v8F6gq/fM1nOCJEPuqVYhTwq/7QdUxjavdROlvAF6vcMux4G GwXQAHLjM2Q6KxjrA4aEicv/2BkLfsA+Q4lwCr0kPI7cUjS7kpZs2a+InsMqlW5utihG XZRVwWflRl21qenWcK9eGbH7u9CCKF7YXlEGUE6zj6vlfbQm1IqwWK3bztS65mW+Ynxo 0qKi0xCBpjwlyosMWrUi9LPj5Ho/8GTu9l0wiitFV/TOhLT9MFuhpzEdt4hYOqBEElOE SrWgWzZmzfJfOrBtGPAc2+bH5Xmp/P9yJSGMmYQYrppqQFD697/cyH4a8V5P8oLfwxw8 m5MQ== X-Gm-Message-State: AA+aEWYDtTlpNAX3JO+63t0CZhBkpl8YkD3AQnH+ll6LZdDn4i+/TqFR 30faF53eOfbbfpCfwVEmyHbBaV6fMrU= X-Google-Smtp-Source: AFSGD/VFgBgphq5Um9W5z27eUrNh9NZO277+UEzvwchkv6uD3tJRUB/x7BQf0WhNUUh709EhSyTaNw== X-Received: by 2002:adf:83a7:: with SMTP id 36mr2317759wre.13.1542726294594; Tue, 20 Nov 2018 07:04:54 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id j13sm26378460wrx.5.2018.11.20.07.04.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 07:04:53 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH 1/6] commit-graph write: rephrase confusing progress output Date: Tue, 20 Nov 2018 15:04:38 +0000 Message-Id: <20181120150443.12997-2-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181119225720.GV30222@szeder.dev> References: <20181119225720.GV30222@szeder.dev> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Rephrase the title shown for the progress output emitted by close_reachable(). The message I added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17) gave the impression that it would count up to the number of commit objects. But that's not what the number means. It just represents the work we're doing in several for-loops to do various work before the graph is written out. So let's just say "Annotating commit graph", that title makes no such promises, and we can add other loops here in the future and still consistently show progress output. See [1] for the initial bug report & subsequent discussion about other approaching to solving this. 1. https://public-inbox.org/git/20181015165447.GH19800@szeder.dev/ Reported-by: SZEDER Gábor Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index 40c855f185..e6d0d7722b 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -648,7 +648,7 @@ static void close_reachable(struct packed_oid_list *oids, int report_progress) if (report_progress) progress = start_delayed_progress( - _("Annotating commits in commit graph"), 0); + _("Annotating commit graph"), 0); for (i = 0; i < oids->nr; i++) { display_progress(progress, ++j); commit = lookup_commit(the_repository, &oids->list[i]); From patchwork Tue Nov 20 15:04:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10690477 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E33CC14BD for ; Tue, 20 Nov 2018 15:05:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D17B2201A4 for ; Tue, 20 Nov 2018 15:05:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C3B362892C; Tue, 20 Nov 2018 15:05:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2DF1C201A4 for ; Tue, 20 Nov 2018 15:05:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727860AbeKUBed (ORCPT ); Tue, 20 Nov 2018 20:34:33 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:45623 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726224AbeKUBed (ORCPT ); Tue, 20 Nov 2018 20:34:33 -0500 Received: by mail-wr1-f67.google.com with SMTP id v6so2311738wrr.12 for ; Tue, 20 Nov 2018 07:04:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9RBjbS9UWyBG7EL+W1rQzpOPsz3LTYjLwKIl9Nh6EVI=; b=erLmhyWpvCQnH4a8bZYwocQaNzHi/W5imoRf218PTmex9K8kvYzO3RoSj6NrudY4VK POzDB2/5/rWn1iwdkSyBzg7+SfAbVt0GMFPHc9AwpKPpxck8ZdCdeiqtuQQx9VCjFc/n T6Mg2vbR6iO7huK2R0ivHqzV+Ah2PgFsVYmTCKweqxf55IFBmvV42W6H8DgRkggcLzAt kCmHmZJb9eJ064AmsNHIC2/ZUrSyuRYv2veTvllesZfj0iAXtm2lPV3qPvc+smPerha5 KewBysYMrQS6GbyNv7zl527MX4teZy9JlETzIZWSXf9tnJnXLniuy/4Tu21wa1zD/RvX MXzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9RBjbS9UWyBG7EL+W1rQzpOPsz3LTYjLwKIl9Nh6EVI=; b=cqtQIAwRXneYIAT1NXR+wHvzE3oslqW044oM03DnpHKEu+c6LJ22FByKFcd99L20jr oqZoQQ3yo/KBA+yFMiE7vBJsHHGiqNklHd2RyRlz3V0BSgfCBzoAncyTh5i7Wwf02CDg WZfAj1ahOZOOaD9X0PVR7vmvtj1IG8VT7BR8fY+9WrbzN5j+2Nywr98HdUp3vzEX/moX JTIQBieJFcSJqxop7QKhpTDUmwxy4LRmhjA0RfBbsH3tmzyGMO6YHGqJRo4Peg7NhBEn b+ccMu8Oichr6W1tgVoAdCyG1puVpJnjjah7MDT+3q5aO7gw2/ZkUj027ouGycuuIktc Zf5A== X-Gm-Message-State: AA+aEWa6p3c+4THazdGvYFQROtWebb7UVgRu0ocTWNU4GCmeOUWG02z8 Lq6k6IClUbS8teaD774Yk/y/FoJouUU= X-Google-Smtp-Source: AFSGD/WECGpXBJUTZTEFHs8+/G79czTSSvoWMFMWKvyeBADYCCIUG8OwMQFixbRu9eMFE/kX5SaRCw== X-Received: by 2002:adf:bb94:: with SMTP id q20mr2273806wrg.183.1542726296258; Tue, 20 Nov 2018 07:04:56 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id j13sm26378460wrx.5.2018.11.20.07.04.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 07:04:55 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH 2/6] commit-graph write: add more progress output Date: Tue, 20 Nov 2018 15:04:39 +0000 Message-Id: <20181120150443.12997-3-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181119225720.GV30222@szeder.dev> References: <20181119225720.GV30222@szeder.dev> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add more progress output to the output already added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17). As noted in that commit most of the progress output isn't displayed on small repositories, but before this change we'd noticeably hang for 2-3 seconds at the end on medium sized repositories such as linux.git. Now we'll instead show output like this, and have no human-observable point at which we're not producing progress output: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph: 6418991, done. Computing commit graph generation numbers: 100% (797205/797205), done. Writing out commit graph chunks: 2399861, done. This "graph chunks" number is not meant to be meaningful to the user, but just to show that we're doing work and the command isn't hanging. On a much larger in-house repository I have we'll show (note how we also say "Annotating[...]"): $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph: 48271163, done. Annotating commit graph: 21424536, done. Computing commit graph generation numbers: 100% (7141512/7141512), done. Writing out commit graph chunks: 21424913, done. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 47 ++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 38 insertions(+), 9 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index e6d0d7722b..afce20dd4d 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -433,7 +433,9 @@ struct tree *get_commit_tree_in_graph(struct repository *r, const struct commit static void write_graph_chunk_fanout(struct hashfile *f, struct commit **commits, - int nr_commits) + int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { int i, count = 0; struct commit **list = commits; @@ -445,6 +447,8 @@ static void write_graph_chunk_fanout(struct hashfile *f, */ for (i = 0; i < 256; i++) { while (count < nr_commits) { + if (progress) + display_progress(progress, ++*progress_cnt); if ((*list)->object.oid.hash[0] != i) break; count++; @@ -456,12 +460,17 @@ static void write_graph_chunk_fanout(struct hashfile *f, } static void write_graph_chunk_oids(struct hashfile *f, int hash_len, - struct commit **commits, int nr_commits) + struct commit **commits, int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; int count; - for (count = 0; count < nr_commits; count++, list++) + for (count = 0; count < nr_commits; count++, list++) { + if (progress) + display_progress(progress, ++*progress_cnt); hashwrite(f, (*list)->object.oid.hash, (int)hash_len); + } } static const unsigned char *commit_to_sha1(size_t index, void *table) @@ -471,7 +480,9 @@ static const unsigned char *commit_to_sha1(size_t index, void *table) } static void write_graph_chunk_data(struct hashfile *f, int hash_len, - struct commit **commits, int nr_commits) + struct commit **commits, int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; struct commit **last = commits + nr_commits; @@ -482,6 +493,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, int edge_value; uint32_t packedDate[2]; + if (progress) + display_progress(progress, ++*progress_cnt); + parse_commit(*list); hashwrite(f, get_commit_tree_oid(*list)->hash, hash_len); @@ -542,7 +556,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, static void write_graph_chunk_large_edges(struct hashfile *f, struct commit **commits, - int nr_commits) + int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; struct commit **last = commits + nr_commits; @@ -566,6 +582,9 @@ static void write_graph_chunk_large_edges(struct hashfile *f, nr_commits, commit_to_sha1); + if (progress) + display_progress(progress, ++*progress_cnt); + if (edge_value < 0) edge_value = GRAPH_PARENT_MISSING; else if (!parent->next) @@ -764,6 +783,7 @@ void write_commit_graph(const char *obj_dir, int num_extra_edges; struct commit_list *parent; struct progress *progress = NULL; + uint64_t progress_cnt; if (!commit_graph_compatible(the_repository)) return; @@ -937,10 +957,19 @@ void write_commit_graph(const char *obj_dir, hashwrite(f, chunk_write, 12); } - write_graph_chunk_fanout(f, commits.list, commits.nr); - write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr); - write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr); - write_graph_chunk_large_edges(f, commits.list, commits.nr); + if (report_progress) + progress = start_delayed_progress( + _("Writing out commit graph chunks"), + progress_cnt = 0); + write_graph_chunk_fanout(f, commits.list, commits.nr, progress, + &progress_cnt); + write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr, + progress, &progress_cnt); + write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr, + progress, &progress_cnt); + write_graph_chunk_large_edges(f, commits.list, commits.nr, progress, + &progress_cnt); + stop_progress(&progress); close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); From patchwork Tue Nov 20 15:04:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10690479 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C1CFC1709 for ; Tue, 20 Nov 2018 15:05:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B2F4D201A4 for ; Tue, 20 Nov 2018 15:05:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A78E52892C; Tue, 20 Nov 2018 15:05:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 37A1A201A4 for ; Tue, 20 Nov 2018 15:05:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727865AbeKUBee (ORCPT ); Tue, 20 Nov 2018 20:34:34 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:34826 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727836AbeKUBee (ORCPT ); Tue, 20 Nov 2018 20:34:34 -0500 Received: by mail-wm1-f68.google.com with SMTP id c126so2590595wmh.0 for ; Tue, 20 Nov 2018 07:04:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wXOeROURnrObUX+SnooD1hB2aI9MdA4lzTWJB+Iaq2c=; b=RVtt1NeOCKGke7uHk+7XG1jxR0mNf0Q+75XHZ59y/+oOn4E6fSE2i4uP8KDawzQAIq lMF3jJ9dfMKAfyUBez/8UJSmCGknHari3kanzKbpYWg5IQXdvw1LJf7yXWJf/CDeRg25 sF99uvdeAaDdxZhiyC2qzz5Q7vxWSsJr5HVTCQBsYKEIneu5Pz9kY4v0IhfsyaqJ4ZuP sDfpSWz8j5rHeaTepXGgFYbNSZJ6VQt68m9hn25Gu6T3MuMii+WU8ay4TJiYp7g/Bs3p U0nvn9KGYrcwyYioMj5iBeTEZtx7dyOvz/nk7eMsXYxeJY+BU4aAg4Vsc9UNmUF5y7u7 kaVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wXOeROURnrObUX+SnooD1hB2aI9MdA4lzTWJB+Iaq2c=; b=oRpMejkH4N8zlekVMeBPZO2UO7flI1YPYAvo7ol7az5HaEE5z7YxXlPPV0ZbaGr6xg HaTiQNiYgaoockyx6v406srwF8Xjaga6Vk3uS0PK1gQS4xLqrTHIiN2HwvK/PC/eHywr st5ROVQr4RcMhe/915N1C4if/p+R1dv2hPS4qdH/kbOKd7lHh6wgI1BVzxUEkGT5QFA7 DCyvs1U+7Vy7pT1MsTNx9SwUBeDbOcK0Vteq6SOAcBISLSPjhhRyQT6WNd2/Yqf6bH7W qpSmaVcWJPhris0Z8bVzcKthHHXm9BQitQ6MAK7A5YtSYKZA+Q34KPb+EvC2PN+cWM6y jJvQ== X-Gm-Message-State: AA+aEWZizEbgHnNqxUkZOJJSjp62LRpoATgYdxEqWQYMjxVwedsTlm9e VktWDy7cYvz1vvlPXz5udNyOJYmVCiM= X-Google-Smtp-Source: AJdET5cJTPlz6txpRJVQls4QiT7dHzLSC1006su9kMJdoGlk9j11oYIajdLrnjDiopba+ez88LFaZQ== X-Received: by 2002:a1c:ab07:: with SMTP id u7-v6mr2479470wme.16.1542726297757; Tue, 20 Nov 2018 07:04:57 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id j13sm26378460wrx.5.2018.11.20.07.04.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 07:04:57 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH 3/6] commit-graph write: show progress for object search Date: Tue, 20 Nov 2018 15:04:40 +0000 Message-Id: <20181120150443.12997-4-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181119225720.GV30222@szeder.dev> References: <20181119225720.GV30222@szeder.dev> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Show the percentage progress for the "Finding commits for commit graph" phase for the common case where we're operating on all packs in the repository, as "commit-graph write" or "gc" will do. Before we'd emit on e.g. linux.git with "commit-graph write": Finding commits for commit graph: 6418991, done. [...] And now: Finding commits for commit graph: 100% (6418991/6418991), done. [...] Since the commit graph only includes those commits that are packed (via for_each_packed_object(...)) the approximate_object_count() returns the actual number of objects we're going to process. Still, it is possible due to a race with "gc" or another process maintaining packs that the number of objects we're going to process is lower than what approximate_object_count() reported. In that case we don't want to stop the progress bar short of 100%. So let's make sure it snaps to 100% at the end. The inverse case is also possible and more likely. I.e. that a new pack has been added between approximate_object_count() and for_each_packed_object(). In that case the percentage will go beyond 100%, and we'll do nothing to snap it back to 100% at the end. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index afce20dd4d..4d03f8aa7f 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -784,12 +784,14 @@ void write_commit_graph(const char *obj_dir, struct commit_list *parent; struct progress *progress = NULL; uint64_t progress_cnt; + unsigned long approx_nr_objects; if (!commit_graph_compatible(the_repository)) return; oids.nr = 0; - oids.alloc = approximate_object_count() / 32; + approx_nr_objects = approximate_object_count(); + oids.alloc = approx_nr_objects / 32; oids.progress = NULL; oids.progress_done = 0; @@ -869,8 +871,11 @@ void write_commit_graph(const char *obj_dir, if (!pack_indexes && !commit_hex) { if (report_progress) oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), 0); + _("Finding commits for commit graph"), + approx_nr_objects); for_each_packed_object(add_packed_commits, &oids, 0); + if (oids.progress_done < approx_nr_objects) + display_progress(oids.progress, approx_nr_objects); stop_progress(&oids.progress); } From patchwork Tue Nov 20 15:04:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10690481 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BA8181709 for ; Tue, 20 Nov 2018 15:05:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AA666201A4 for ; Tue, 20 Nov 2018 15:05:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9CA9C2892C; Tue, 20 Nov 2018 15:05:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 300FB201A4 for ; Tue, 20 Nov 2018 15:05:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727998AbeKUBeh (ORCPT ); Tue, 20 Nov 2018 20:34:37 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:39910 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727836AbeKUBeh (ORCPT ); Tue, 20 Nov 2018 20:34:37 -0500 Received: by mail-wm1-f68.google.com with SMTP id u13-v6so2591795wmc.4 for ; Tue, 20 Nov 2018 07:05:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=x3k4WGKA2Pzr1aqumWsHUcz4rMhV5DYZ29HRIxuo3RE=; b=dCwnd1ZfUSx6395FPx+y2CmjjN8+XqBBdGKXRF6acGNEkPILMsfDJwz+LRFMijx57Y 6nC84oqG8pGqf+T2yEkXjU4HGJkPCo3aWZR8B8QvnfXW1xhyOvd41YUNfDGwgWXXjcs2 Q3ySsIlHufzzL0j9zNuGhn8FPA14CiYJGFZLY5Y6LBHTiwnwdRTDrKFXZ8az6cH9Xd4B uBxwPjuJTmeHOwiYX11Qy9HD2BgJIlCen39OUrG/Khe76U8KbNJgMELaeNvqnTmpL9sd 6mmjixBIbz4NYPJ+0yZExeYuo98XUKvYBCSHl0dVF8BeLmB23XIrGa4RlE3wY1pqJgDA iLDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=x3k4WGKA2Pzr1aqumWsHUcz4rMhV5DYZ29HRIxuo3RE=; b=hnVNDcg+4aWXwRXuCuoJMbA6q0xnBYDFbtNn8UZIWcoALXNMiVaIOKUiNHSfBmbOKy Wsw4iLwfa35oRG1pX/7/l8tEiL/qgZfN/t2Akcc1Z6YJXPmT28M4qsfX0w4qWByohBWU KqrI5IGdN5JgV64YrPIGh6NaPAVT9CnRRiZc/H68i3E1k78fSJKItPd4wuAV1M6JJLjM WNul6ZL3QiwimgVMXE6txYFjLPcTD+Gcq/EOzKYCT11W1QT9kaOT5RAbdiAgVwq3O2U6 ibnXAJups9ZaqHBiMRnewSTwb1K1Fne88qaFwInmgisOmb+1KVErCkyokY6//2hyer6n Twfg== X-Gm-Message-State: AA+aEWZZr5fJhovfehVKSgFnezBcom8Gcpe25nxYYFJBVDp2kRfiIuag piDPUr70rAqkmf15DrzUbi7cffWoE8Q= X-Google-Smtp-Source: AFSGD/XzbAvShggaSjm8KuXn6iVEhDeanTCZFQMwWaVvh+XxZwkXW+LvyVBgVnO1Ke3no7WG+Cc3xA== X-Received: by 2002:a1c:e58c:: with SMTP id c134mr2414490wmh.124.1542726299383; Tue, 20 Nov 2018 07:04:59 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id j13sm26378460wrx.5.2018.11.20.07.04.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 07:04:58 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH 4/6] commit-graph write: add more describing progress output Date: Tue, 20 Nov 2018 15:04:41 +0000 Message-Id: <20181120150443.12997-5-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181119225720.GV30222@szeder.dev> References: <20181119225720.GV30222@szeder.dev> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make the progress output shown when we're searching for commits to include in the graph more descriptive. This amends code I added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17). Now, on linux.git, we'll emit this sort of output in the various modes we support: $ git commit-graph write Finding commits for commit graph among packed objects: 100% (6418991/6418991), done. [...] $ git for-each-ref --format='%(objectname)' | git commit-graph write --stdin-commits Finding commits for commit graph from 584 ref tips: 100% (584/584), done. [...] $ (cd .git/objects/pack/ && ls *idx) | git commit-graph write --stdin-pack Finding commits for commit graph in 4 packs: 6418991, done. [...] The middle on of those is going to be the output users will most commonly see, since it'll be emitted when they get the commit graph via gc.writeCommitGraph=true. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 28 +++++++++++++++++++++------- 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 4d03f8aa7f..a0aea850f1 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -785,6 +785,7 @@ void write_commit_graph(const char *obj_dir, struct progress *progress = NULL; uint64_t progress_cnt; unsigned long approx_nr_objects; + struct strbuf progress_title = STRBUF_INIT; if (!commit_graph_compatible(the_repository)) return; @@ -821,8 +822,12 @@ void write_commit_graph(const char *obj_dir, strbuf_addf(&packname, "%s/pack/", obj_dir); dirlen = packname.len; if (report_progress) { - oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), 0); + strbuf_addf(&progress_title, + Q_("Finding commits for commit graph in %d pack", + "Finding commits for commit graph in %d packs", + pack_indexes->nr), + pack_indexes->nr); + oids.progress = start_delayed_progress(progress_title.buf, 0); oids.progress_done = 0; } for (i = 0; i < pack_indexes->nr; i++) { @@ -839,14 +844,20 @@ void write_commit_graph(const char *obj_dir, free(p); } stop_progress(&oids.progress); + strbuf_reset(&progress_title); strbuf_release(&packname); } if (commit_hex) { - if (report_progress) - progress = start_delayed_progress( - _("Finding commits for commit graph"), - commit_hex->nr); + if (report_progress) { + strbuf_addf(&progress_title, + Q_("Finding commits for commit graph from %d ref tip", + "Finding commits for commit graph from %d ref tips", + commit_hex->nr), + commit_hex->nr); + progress = start_delayed_progress(progress_title.buf, + commit_hex->nr); + } for (i = 0; i < commit_hex->nr; i++) { const char *end; struct object_id oid; @@ -866,12 +877,13 @@ void write_commit_graph(const char *obj_dir, } } stop_progress(&progress); + strbuf_reset(&progress_title); } if (!pack_indexes && !commit_hex) { if (report_progress) oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), + _("Finding commits for commit graph among packed objects"), approx_nr_objects); for_each_packed_object(add_packed_commits, &oids, 0); if (oids.progress_done < approx_nr_objects) @@ -976,6 +988,8 @@ void write_commit_graph(const char *obj_dir, &progress_cnt); stop_progress(&progress); + strbuf_release(&progress_title); + close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); commit_lock_file(&lk); From patchwork Tue Nov 20 15:04:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10690485 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 06A2C1709 for ; Tue, 20 Nov 2018 15:05:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EAE27201A4 for ; Tue, 20 Nov 2018 15:05:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DEFD32892C; Tue, 20 Nov 2018 15:05:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0BF5D201A4 for ; Tue, 20 Nov 2018 15:05:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727956AbeKUBeh (ORCPT ); Tue, 20 Nov 2018 20:34:37 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:40992 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727866AbeKUBeh (ORCPT ); Tue, 20 Nov 2018 20:34:37 -0500 Received: by mail-wr1-f65.google.com with SMTP id x10so2319208wrs.8 for ; Tue, 20 Nov 2018 07:05:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=inQboS/wXMH4ylbBFHEnE5fsSdQ2qCoBc8ODwtFDtN0=; b=IdoD8nCcFeCfHSwcZCr24iFOv+j87e2p54T7YYSc/gMQ3vAxEqbZxwkwxtC3UrU9I0 Gfm+Ek8sUsKqWk9z04+QiCEMGT+y3sfwQaWi/490RZNGJ8k3etb0KHruEyLkcyEwKpcL gUnxVmJvKJiQrcXm5pccUN+jWDLtPCJkHcN2d3Vta8GTjhf7iQhN6ljerLrPbZEK6Hf/ iU7/AGno66JiWPBNTugbOFs4kt1LBJg3VS7+Oa3WvnTe7WrVkALVXoqzKKsSl1hD4Qoj Fz9+5N10hU+meENd7S4A0XDTc5FiVYNO6yAiw7hq4w//z9+ZJnw3LFT+HJFb6S0EHHNS bShA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=inQboS/wXMH4ylbBFHEnE5fsSdQ2qCoBc8ODwtFDtN0=; b=KsNSkS+qK3Xee65Cly4suyuKx6t1tQBuia0/GvNgUfZWQr+HwRdDu1yVUhomhgpE+V 6VfdxM5OtHubNxbEtg/Y9lM+ETk+laCmKGkZe6I0M/qu+NGA7GNZv+xCHPOBdmzEY9Ud DGX1O/Dv+WYazqFeKiV2LZ37TkTNXvZi3jBayWFRXl7oQzNNTEkZlEYMzu4wa5M9QJQt HqYFe0dhjEb5q0GGYsLpAgWcOWManRqoRjxEehCBAoFpoMt2w/h2vAbeNkYiBb0taaPx Ga4sq5JQx9pIeaYoYRxnvUwuYpJQ52UxFhxWe23YlaQGi5mZIvKLt/4MZhOlp9sEdo3a QT5Q== X-Gm-Message-State: AA+aEWYoeopnafL2t7t16OhLgFoGtpbP1HS00OJLNNDase8TTKtDfiFL 2Cw8unqM9OAiSThjozTclPzPMe5k0/A= X-Google-Smtp-Source: AFSGD/XPCGpgFY10RR+m+Jwtd11Pt5txRZCy23JzHIvTMZrYdNyZjpu9DXNOsBnD2F8PRmnIDIXnTA== X-Received: by 2002:adf:c44a:: with SMTP id a10mr2182765wrg.145.1542726301037; Tue, 20 Nov 2018 07:05:01 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id j13sm26378460wrx.5.2018.11.20.07.04.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 07:05:00 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH 5/6] commit-graph write: remove empty line for readability Date: Tue, 20 Nov 2018 15:04:42 +0000 Message-Id: <20181120150443.12997-6-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181119225720.GV30222@szeder.dev> References: <20181119225720.GV30222@szeder.dev> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Remove the empty line between a QSORT(...) and the subsequent oideq() for-loop. This makes it clearer that the QSORT(...) is being done so that we can run the oideq() loop on adjacent OIDs. Amends code added in 08fd81c9b6 ("commit-graph: implement write_commit_graph()", 2018-04-02). Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 1 - 1 file changed, 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index a0aea850f1..d0961e89df 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -894,7 +894,6 @@ void write_commit_graph(const char *obj_dir, close_reachable(&oids, report_progress); QSORT(oids.list, oids.nr, commit_compare); - count_distinct = 1; for (i = 1; i < oids.nr; i++) { if (!oideq(&oids.list[i - 1], &oids.list[i])) From patchwork Tue Nov 20 15:04:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10690483 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F4C314BD for ; Tue, 20 Nov 2018 15:05:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0F4B328390 for ; Tue, 20 Nov 2018 15:05:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 031D429014; Tue, 20 Nov 2018 15:05:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E0EBA28390 for ; Tue, 20 Nov 2018 15:05:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728009AbeKUBek (ORCPT ); Tue, 20 Nov 2018 20:34:40 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:40792 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727836AbeKUBek (ORCPT ); Tue, 20 Nov 2018 20:34:40 -0500 Received: by mail-wr1-f65.google.com with SMTP id p4so2324686wrt.7 for ; Tue, 20 Nov 2018 07:05:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eYf935D3Ojg/RBEynRDtmTVxulEr7XF9D2DlULrMO+8=; b=Lffs47fpaG/q8XL3aM3YCbkyDQN0ps9CnGIubavkwKwGY/hnOUQJRKYbrRPFIKDUKC S91bERly+Od3nGN0LmDwn48d1gDf/HS2z0qpIMdThO+6YQ0C2toZXNdEst1FnzCSP0uu joVZGW+djjtga30t6g4zztlB9r/GaXmVmOCb2yZhIG0i5lYBtt9mGX5ELnw4jLXZFU/J tkoZhMQ96+CZey9u+OKkO/W3M3R5pcRvI0bQLPSLfh1RThLQuhnYEiNyL8HZN+L5vxAh y1VWUbqJVKFUUp72xxIfPC0OD5OQ0nOhFwELmeO0BaH6IwhQb6O6uanjKIHrHBfjNo3Y HRJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eYf935D3Ojg/RBEynRDtmTVxulEr7XF9D2DlULrMO+8=; b=Ll68+bBN8i7jF7aqfjWLJkka5AvVQLJnhmpvjpThrDAuIBOALwgbDAh4uKyQz9BkuI DY2iOUTz484LdMKSM5AiShKtodslP/boM4D1lzVEDFJOJP7ayWZ9+exlr6DTyGtFf+fY YJGOXOT8G6WRdwiJ3UNIbZFsBlyogFvoelXptkqQr9K4OLc/kZdPja6fOSlhXUiKc40f QOqhJjh6vs8a084is+yE7UfhyVwC0gVp9sLvBQ0Cn+kRdZ7++SmjL77H7RiTqCF1iKI+ 86DL8R3yx+vCdFYgoAkZK2yTBLxam2I6eFe3f91AdN2xsvSp/NUH45hI14OmSTFzJCjK B9lg== X-Gm-Message-State: AA+aEWYFkiiR67gcFkcVmx5Xh6QY3Goq5+0zDOJWfWmJT5QEKxhOK27p etTUpObGWbByhaidt8NtQNurBj87dYs= X-Google-Smtp-Source: AFSGD/Whurfgd9/kR706REKkzxLEOb9ZVj/NCQuE5AJlZG/L0Yy2On6geZFkCghJxux16AxulP89/Q== X-Received: by 2002:a5d:4586:: with SMTP id p6mr2226208wrq.69.1542726303178; Tue, 20 Nov 2018 07:05:03 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id j13sm26378460wrx.5.2018.11.20.07.05.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 07:05:02 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH 6/6] commit-graph write: add even more progress output Date: Tue, 20 Nov 2018 15:04:43 +0000 Message-Id: <20181120150443.12997-7-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181119225720.GV30222@szeder.dev> References: <20181119225720.GV30222@szeder.dev> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add more progress output to sections of code that can collectively take 5-10 seconds on a large enough repository. On a test repository with 7141512 commits (see earlier patches for details) we'll now emit: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (50009986/50009986), done. Annotating commit graph: 21564240, done. Counting distinct commits in commit graph: 100% (7188080/7188080), done. Finding extra edges in commit graph: 100% (7188080/7188080), done. Computing commit graph generation numbers: 100% (7143635/7143635), done. Writing out commit graph chunks: 21431282, done. Whereas on a medium-sized repository such as linux.git we'll still emit output like: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (6365328/6365328), done. Annotating commit graph: 2391621, done. Computing commit graph generation numbers: 100% (797207/797207), done. Writing out commit graph chunks: 2399867, done. The "Counting distinct commits in commit graph" phase will spend most of its time paused at "0/*" as we QSORT(...) the list. That's not optimal, but at least we don't seem to be stalling anymore. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/commit-graph.c b/commit-graph.c index d0961e89df..2e2eaa24ca 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -893,12 +893,19 @@ void write_commit_graph(const char *obj_dir, close_reachable(&oids, report_progress); + if (report_progress) + progress = start_delayed_progress( + _("Counting distinct commits in commit graph"), + oids.nr); + display_progress(progress, 0); /* TODO: Measure QSORT() progress */ QSORT(oids.list, oids.nr, commit_compare); count_distinct = 1; for (i = 1; i < oids.nr; i++) { + display_progress(progress, i + 1); if (!oideq(&oids.list[i - 1], &oids.list[i])) count_distinct++; } + stop_progress(&progress); if (count_distinct >= GRAPH_PARENT_MISSING) die(_("the commit graph format cannot write %d commits"), count_distinct); @@ -908,8 +915,13 @@ void write_commit_graph(const char *obj_dir, ALLOC_ARRAY(commits.list, commits.alloc); num_extra_edges = 0; + if (report_progress) + progress = start_delayed_progress( + _("Finding extra edges in commit graph"), + oids.nr); for (i = 0; i < oids.nr; i++) { int num_parents = 0; + display_progress(progress, i + 1); if (i > 0 && oideq(&oids.list[i - 1], &oids.list[i])) continue; @@ -926,6 +938,7 @@ void write_commit_graph(const char *obj_dir, commits.nr++; } num_chunks = num_extra_edges ? 4 : 3; + stop_progress(&progress); if (commits.nr >= GRAPH_PARENT_MISSING) die(_("too many commits to write graph"));