From patchwork Tue Nov 20 19:50:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10691163 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6FA61923 for ; Tue, 20 Nov 2018 19:50:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 978EB2AB03 for ; Tue, 20 Nov 2018 19:50:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8BB7E2A31C; Tue, 20 Nov 2018 19:50:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 16EE42AA66 for ; Tue, 20 Nov 2018 19:50:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727354AbeKUGVb (ORCPT ); Wed, 21 Nov 2018 01:21:31 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:34459 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727342AbeKUGVa (ORCPT ); Wed, 21 Nov 2018 01:21:30 -0500 Received: by mail-wr1-f65.google.com with SMTP id j2so3279077wrw.1 for ; Tue, 20 Nov 2018 11:50:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=U7/012vgfZU58ex+5EZrccnyYPYhhrrQXZinDbn9r2w=; b=p2Pcw7OfZarDmeNH8wLjHCqNsalf2nk3QEt8+WbwMWL5sDdtJLbcTiGpt7VQAB5ToT SNW5Qi4trb6zQQgWX8bVxXPbKJvBP6J4eJN/LB6dNY/B4kI4YfwVw5X6e7WHY6bd1fZV yBXS6q8F+UJ4Sg22/7VT1IFLo6t087Ij6hf3KbH6VmS94pY+PBzM0UfRzeSWJJY44ozP 3aOEVPVcCbxolNI6yHVl7KXrnqAf7Pp0pqkUvwUjzLLQN1edsynL48KzgzJFOB6F7Mo6 nj8NAMQlzSNw06fhyOlutJgjDF67NcLMoTrxuK8pFyCvyByrMXoDiJsNfE7bhWSrynH2 8ssg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=U7/012vgfZU58ex+5EZrccnyYPYhhrrQXZinDbn9r2w=; b=MsTOtZKAz6HJ3LJIn3kiKOcKiXYs7RsA9byvHLuE6646F/aWt1P4jnpCPGnlCc+Uax xrBvwzJHKBQq+oyz6rrSXGUVYroHFZLN59NwgyaPfe8TQEH12AQvzG5kUMqmOLSQYsb8 GB90/7nonlT3vmTcup0nimAy2Vi5CMZJ5qxhPtbcjfXsijDvWqsoMkf5s6hrejYwweTR jfM/+gjIAQG3bOgD4XqfVU7rXNQBa6QHMNyMV6xe1sr0rmfUF5+qQSSocwxRepE1f9et KWXno4+gcijZ7VGt+/CqnRs/BAyobUakGgQFRWi/8NKiHcSOQxkFAxLyvtuJFYFaQEx7 ZZRQ== X-Gm-Message-State: AA+aEWYSwENU/r/yMrfFGtzp0k3eFG4rpPxcTrixwevzikJY7/AjGadk AN4zHWMOl8FPvDVSUYTVLC410pnp/KE= X-Google-Smtp-Source: AFSGD/UhE8EbLoOelD00xlLR5pXIvtzXrpkupCO7dh/2BB4Vv/O2He7Ut0IBeDvsOtNNItlV55t1Ag== X-Received: by 2002:a5d:524b:: with SMTP id p11mr3096577wrv.147.1542743438881; Tue, 20 Nov 2018 11:50:38 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id p14sm12589415wrt.37.2018.11.20.11.50.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 11:50:38 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v2 1/6] commit-graph write: rephrase confusing progress output Date: Tue, 20 Nov 2018 19:50:22 +0000 Message-Id: <20181120195027.3260-2-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181120165800.GB30222@szeder.dev> References: <20181120165800.GB30222@szeder.dev> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Rephrase the title shown for the progress output emitted by close_reachable(). The message I added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17) gave the impression that it would count up to the number of commit objects. But that's not what the number means. It just represents the work we're doing in several for-loops to do various work before the graph is written out. So let's just say "Annotating commit graph", that title makes no such promises, and we can add other loops here in the future and still consistently show progress output. See [1] for the initial bug report & subsequent discussion about other approaching to solving this. 1. https://public-inbox.org/git/20181015165447.GH19800@szeder.dev/ Reported-by: SZEDER Gábor Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index 40c855f185..e6d0d7722b 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -648,7 +648,7 @@ static void close_reachable(struct packed_oid_list *oids, int report_progress) if (report_progress) progress = start_delayed_progress( - _("Annotating commits in commit graph"), 0); + _("Annotating commit graph"), 0); for (i = 0; i < oids->nr; i++) { display_progress(progress, ++j); commit = lookup_commit(the_repository, &oids->list[i]); From patchwork Tue Nov 20 19:50:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10691167 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0AAE313BB for ; Tue, 20 Nov 2018 19:50:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ED00E2A31C for ; Tue, 20 Nov 2018 19:50:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E17582AB03; Tue, 20 Nov 2018 19:50:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F87E2AAE4 for ; Tue, 20 Nov 2018 19:50:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727368AbeKUGVc (ORCPT ); Wed, 21 Nov 2018 01:21:32 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:54175 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727355AbeKUGVc (ORCPT ); Wed, 21 Nov 2018 01:21:32 -0500 Received: by mail-wm1-f65.google.com with SMTP id y1so67001wmi.3 for ; Tue, 20 Nov 2018 11:50:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CGe4mOB8QOEXA7LyBopPmkkIkbTMRdkwIEWvtatKrQc=; b=dR/waV4ApDmIT1MwePXeNHWeYA1P5gB4eFjCH5Puc7Asnyx7juix0SK6IaYuL/zxqm vLTvIcPgQbWt+/goXAbpxiMFufHvPtxo/cn4qwcQoSqNWJvnttKX44wIGlAm/qB6+IpG DGK4e1cm7qg2HGBmGOmwE8osf29Oh9RJZAYPl3NXgQIpv2gV5LB7rZYFdzWUYeWgJ2hk qqcHDnLEGQOQYJ7mOGVmBmIfn52cQF47ILAJvuUF4FJ4L8ACmLSzPJOlKKXIvsIVOo0G xg1bv5+PA4/Bn5yzQC7vqNTt1crB7PHWyywQb++TTdWKf4DpQkKGFx6zXBCthQOTS0rO ynuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CGe4mOB8QOEXA7LyBopPmkkIkbTMRdkwIEWvtatKrQc=; b=DoXxD8Q7IoD3m/Ybhe34qSNml31NmpT/vIkrYxZa6A7SpMITccuOl/8IFNXbH5J4Ni V3VximYtkOn+Rkwv2m9vjRzRKqV8vxncmW3GJVGYjIW0sAK2IHCk+N7DTrbkF8hDAV2h rknwro6/xdGhGGuctxqGbKIf/GEsz0iLjzKlSRi2RT+TnctqN9nrzzuDhgQ610dGPSfG L1xF7ak3SF6Plm7Y+5L6WaiQnQBy5hAdpAI3E229y/9tauN++7k/xKrC09xfnJPaAWnK h7BL4+6SXlbfyyaWXWThVHXUSKWMY6itw1kG2cewReObOA5Jzu6vk761K6+tF46hwLO6 6Ukg== X-Gm-Message-State: AGRZ1gIg3nHHz9U6ExmeKXxp6PheMEi7Bz0k2CKpfwVSqplWhmUOVFbM hZInmtOf6FFe+r+luGG4iv0dAaq8l80= X-Google-Smtp-Source: AJdET5eBH8INu+mmVUuO/GsXk8zxGuOj0MdDx02PYJo+UPzBU4hlo87X51cEs4t+VrSM5XD8OgpWZQ== X-Received: by 2002:a1c:81d3:: with SMTP id c202mr3053561wmd.133.1542743440494; Tue, 20 Nov 2018 11:50:40 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id p14sm12589415wrt.37.2018.11.20.11.50.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 11:50:39 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v2 2/6] commit-graph write: add more progress output Date: Tue, 20 Nov 2018 19:50:23 +0000 Message-Id: <20181120195027.3260-3-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181120165800.GB30222@szeder.dev> References: <20181120165800.GB30222@szeder.dev> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add more progress output to the output already added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17). As noted in that commit most of the progress output isn't displayed on small repositories, but before this change we'd noticeably hang for 2-3 seconds at the end on medium sized repositories such as linux.git. Now we'll instead show output like this, and have no human-observable point at which we're not producing progress output: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph: 6365492, done. Computing commit graph generation numbers: 100% (797222/797222), done. Writing out commit graph: 2399912, done. This "writing out" number is not meant to be meaningful to the user, but just to show that we're doing work and the command isn't hanging. In the current implementation it's approximately 4x the number of commits. As noted in on-list discussion[1] we could add the loops up and show percentage progress here, but I don't think it's worth it. It would make the implementation more complex and harder to maintain for very little gain. On a much larger in-house repository I have we'll show (note how we also say "Annotating[...]"): $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph: 50026015, done. Annotating commit graph: 21567407, done. Computing commit graph generation numbers: 100% (7144680/7144680), done. Writing out commit graph: 21434417, done. 1. https://public-inbox.org/git/20181120165800.GB30222@szeder.dev/ Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 41 ++++++++++++++++++++++++++++++++--------- 1 file changed, 32 insertions(+), 9 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index e6d0d7722b..6f6409b292 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -433,7 +433,9 @@ struct tree *get_commit_tree_in_graph(struct repository *r, const struct commit static void write_graph_chunk_fanout(struct hashfile *f, struct commit **commits, - int nr_commits) + int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { int i, count = 0; struct commit **list = commits; @@ -445,6 +447,7 @@ static void write_graph_chunk_fanout(struct hashfile *f, */ for (i = 0; i < 256; i++) { while (count < nr_commits) { + display_progress(progress, ++*progress_cnt); if ((*list)->object.oid.hash[0] != i) break; count++; @@ -456,12 +459,16 @@ static void write_graph_chunk_fanout(struct hashfile *f, } static void write_graph_chunk_oids(struct hashfile *f, int hash_len, - struct commit **commits, int nr_commits) + struct commit **commits, int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; int count; - for (count = 0; count < nr_commits; count++, list++) + for (count = 0; count < nr_commits; count++, list++) { + display_progress(progress, ++*progress_cnt); hashwrite(f, (*list)->object.oid.hash, (int)hash_len); + } } static const unsigned char *commit_to_sha1(size_t index, void *table) @@ -471,7 +478,9 @@ static const unsigned char *commit_to_sha1(size_t index, void *table) } static void write_graph_chunk_data(struct hashfile *f, int hash_len, - struct commit **commits, int nr_commits) + struct commit **commits, int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; struct commit **last = commits + nr_commits; @@ -481,6 +490,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, struct commit_list *parent; int edge_value; uint32_t packedDate[2]; + display_progress(progress, ++*progress_cnt); parse_commit(*list); hashwrite(f, get_commit_tree_oid(*list)->hash, hash_len); @@ -542,7 +552,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, static void write_graph_chunk_large_edges(struct hashfile *f, struct commit **commits, - int nr_commits) + int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; struct commit **last = commits + nr_commits; @@ -565,6 +577,7 @@ static void write_graph_chunk_large_edges(struct hashfile *f, commits, nr_commits, commit_to_sha1); + display_progress(progress, ++*progress_cnt); if (edge_value < 0) edge_value = GRAPH_PARENT_MISSING; @@ -764,6 +777,7 @@ void write_commit_graph(const char *obj_dir, int num_extra_edges; struct commit_list *parent; struct progress *progress = NULL; + uint64_t progress_cnt = 0; if (!commit_graph_compatible(the_repository)) return; @@ -937,10 +951,19 @@ void write_commit_graph(const char *obj_dir, hashwrite(f, chunk_write, 12); } - write_graph_chunk_fanout(f, commits.list, commits.nr); - write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr); - write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr); - write_graph_chunk_large_edges(f, commits.list, commits.nr); + if (report_progress) + progress = start_delayed_progress( + _("Writing out commit graph"), + 0); + write_graph_chunk_fanout(f, commits.list, commits.nr, progress, + &progress_cnt); + write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr, + progress, &progress_cnt); + write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr, + progress, &progress_cnt); + write_graph_chunk_large_edges(f, commits.list, commits.nr, progress, + &progress_cnt); + stop_progress(&progress); close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); From patchwork Tue Nov 20 19:50:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10691169 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9532117FE for ; Tue, 20 Nov 2018 19:50:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 843512A023 for ; Tue, 20 Nov 2018 19:50:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 78D892A886; Tue, 20 Nov 2018 19:50:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C5CBD2AAEC for ; Tue, 20 Nov 2018 19:50:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727382AbeKUGVd (ORCPT ); Wed, 21 Nov 2018 01:21:33 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:54179 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727342AbeKUGVd (ORCPT ); Wed, 21 Nov 2018 01:21:33 -0500 Received: by mail-wm1-f66.google.com with SMTP id y1so67069wmi.3 for ; Tue, 20 Nov 2018 11:50:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=k0pOw61XeH3TwdSCCELAuqFQL7BRaBHhJP9x+Pydiyc=; b=S84feLwiWxe2k2Nch/0Trbm8MQw/2kxPfFxmgK6E6z8E4KSPxgAufZBSa/xkZZTkDk 2jhTUF/cJ+FrQMNeZ0l7+5Zpuduo2Md7YW3li9x4Kg03Y9y+ZRVkyE9/FrK15Pyni+XY XoymELUyAH5YeaXP8blcPAVP74fVm4hm8QQsq1yvGHnkAnDsOeBFQpm5CUQHtLFdIfJ5 Dv/jTIVi9k9R37g3SyfloRZ+ao1Un95lynEvXJ4Y/yC1epDjcLXmiNfI3BgudhXzcayi +7DPaJjXcFRHaaXiRdv0RROxnHLYfHOLWpNphw8nte1OL/H6BPXtycXui/triwJ6dBqC XvPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=k0pOw61XeH3TwdSCCELAuqFQL7BRaBHhJP9x+Pydiyc=; b=YJFV9KuCkBsCEc3Fsq5d3BLJrGUmBmheWdZWsY+VF2/bZxXJ02lWsOcbS9b4FYnlY/ R5Ps12Vss2fZjLQ/5ydNdjDmkcBRxrJReMpGf+VIBLZKve/3F0OHWnzX9xMsZ1d1oejJ B4AAWxfHUovJCmcvEP+AVHQjiibxW8A3Wct49pNEzrldXyixq7PwrY+LZTICm7Ix1xdr 0i0SekAmCBKc5Q0q7h5r0o7Fh3CSfWIfdB1fMtRRSSRPZtR9iiwnWmfXG30imlhqlODW oxCcKC+Hs7m6edv3gZaO+bYGaDZEBocX7SWMY+50Gw7aQJDAwWzkyb87Rm7VrYeyIeQt V4RQ== X-Gm-Message-State: AGRZ1gLiDt8+4JL9f6XH97Iv75IvnBUh9gXSZh5CyS3qYcH5kmcR1pSQ Xzbj2R+qTWODW151/8PSJCfS4+dw0Mk= X-Google-Smtp-Source: AJdET5fZKzJmnn4qma/1kOIaGBnlh0o8kRMwd82S+X5mxZ5yIYE6HqieySM4M3Y0fDFPcvT65GtWpQ== X-Received: by 2002:a7b:c003:: with SMTP id c3-v6mr3077410wmb.133.1542743442021; Tue, 20 Nov 2018 11:50:42 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id p14sm12589415wrt.37.2018.11.20.11.50.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 11:50:41 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v2 3/6] commit-graph write: show progress for object search Date: Tue, 20 Nov 2018 19:50:24 +0000 Message-Id: <20181120195027.3260-4-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181120165800.GB30222@szeder.dev> References: <20181120165800.GB30222@szeder.dev> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Show the percentage progress for the "Finding commits for commit graph" phase for the common case where we're operating on all packs in the repository, as "commit-graph write" or "gc" will do. Before we'd emit on e.g. linux.git with "commit-graph write": Finding commits for commit graph: 6365492, done. [...] And now: Finding commits for commit graph: 100% (6365492/6365492), done. [...] Since the commit graph only includes those commits that are packed (via for_each_packed_object(...)) the approximate_object_count() returns the actual number of objects we're going to process. Still, it is possible due to a race with "gc" or another process maintaining packs that the number of objects we're going to process is lower than what approximate_object_count() reported. In that case we don't want to stop the progress bar short of 100%. So let's make sure it snaps to 100% at the end. The inverse case is also possible and more likely. I.e. that a new pack has been added between approximate_object_count() and for_each_packed_object(). In that case the percentage will go beyond 100%, and we'll do nothing to snap it back to 100% at the end. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 6f6409b292..7c1afa4704 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -778,12 +778,14 @@ void write_commit_graph(const char *obj_dir, struct commit_list *parent; struct progress *progress = NULL; uint64_t progress_cnt = 0; + unsigned long approx_nr_objects; if (!commit_graph_compatible(the_repository)) return; oids.nr = 0; - oids.alloc = approximate_object_count() / 32; + approx_nr_objects = approximate_object_count(); + oids.alloc = approx_nr_objects / 32; oids.progress = NULL; oids.progress_done = 0; @@ -863,8 +865,11 @@ void write_commit_graph(const char *obj_dir, if (!pack_indexes && !commit_hex) { if (report_progress) oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), 0); + _("Finding commits for commit graph"), + approx_nr_objects); for_each_packed_object(add_packed_commits, &oids, 0); + if (oids.progress_done < approx_nr_objects) + display_progress(oids.progress, approx_nr_objects); stop_progress(&oids.progress); } From patchwork Tue Nov 20 19:50:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10691171 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 046FE13BB for ; Tue, 20 Nov 2018 19:50:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E882B2A896 for ; Tue, 20 Nov 2018 19:50:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E6C032AA26; Tue, 20 Nov 2018 19:50:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 634FA2AC46 for ; Tue, 20 Nov 2018 19:50:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727398AbeKUGVg (ORCPT ); Wed, 21 Nov 2018 01:21:36 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:33140 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727342AbeKUGVf (ORCPT ); Wed, 21 Nov 2018 01:21:35 -0500 Received: by mail-wm1-f65.google.com with SMTP id 79so5743031wmo.0 for ; Tue, 20 Nov 2018 11:50:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rDs38u4VhCwrI+TeHjDJMhrRuItYxAXspp+KKYj0bQE=; b=NxSu80MqinHEHaDEoYnb8da9V7VZswUt0+/5DXvJWd85klIO8OAXeHgm2EuoIGSOvP V3Hxag+RLmbUcdh7q5QeAVtAZsOsGranbpLRpf/BLTSAPjzWDOOBdos7HxEfqncoqT/V NgyMj5sjybydb3PmUEKFzIK0hhKeJ/IE3h4x4xX0fZdwnSZuYzz38xwklbs8kx96qdTi MXCIujraggYgc3DsjgemcZf6H0GMtJudTELX0wIAXjgmJ0tzJc8ajaPQ1LaD1mh2QLxn 9jP2eGiPMNZ/wRA/uYBNgYTwJpuTNcStzviwlDJhLZDXcjH4GKKUla+5MI4VS+FzKTer PnTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rDs38u4VhCwrI+TeHjDJMhrRuItYxAXspp+KKYj0bQE=; b=s8w5W9fJ+6wEc5ArQkGh51Odo9G8vf2+UIln1cU4gI8viwO14E9Vp0Cs2K8Kcn96CL f46H82MlEy/8t4of8AvoMGDQ6a2oh8LV7N/ZR5bEsq2pTqThBPN+K0yXjU1zotpiK//F XvYl+74J9loJ5bKzG4shiOKOLu5NnNt7z4D9x9Ypj5hKkyaldFA/0bYmQWNRvl2u4Buq i3Fhf1q42iSrPyKi2Xs3r9P68CpFS+NVUQysKTrg3LaBowDDHsIxWZZOY23oj4K11uXu 0/2vBsaXfFUele+a6J326WKv2T8S4ChI4cTpULMTD8dj8BklrhI+hjtoLR6BQjeCRID8 9rXA== X-Gm-Message-State: AGRZ1gJScBNuSmbmgjKadoKZKTGF5X55R4tCM9jJ51Q/+HQjZL8hFm7h G2XZVtYMN+Y8iVbvi3f20h750vqD2kc= X-Google-Smtp-Source: AJdET5d+lMxuEWFa+S6SJR+7lj1Vjn8jfTECQ/OCEBUJB279QkTUla9//DBYpF+CxHlWGxvjK1Gspg== X-Received: by 2002:a1c:13c3:: with SMTP id 186-v6mr3224443wmt.125.1542743443632; Tue, 20 Nov 2018 11:50:43 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id p14sm12589415wrt.37.2018.11.20.11.50.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 11:50:42 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v2 4/6] commit-graph write: add more describing progress output Date: Tue, 20 Nov 2018 19:50:25 +0000 Message-Id: <20181120195027.3260-5-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181120165800.GB30222@szeder.dev> References: <20181120165800.GB30222@szeder.dev> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make the progress output shown when we're searching for commits to include in the graph more descriptive. This amends code I added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17). Now, on linux.git, we'll emit this sort of output in the various modes we support: $ git commit-graph write Finding commits for commit graph among packed objects: 100% (6365492/6365492), done. [...] # Actually we don't emit this since this takes almost no time at # all. But if we did (s/_delayed//) we'd show: $ git for-each-ref --format='%(objectname)' | git commit-graph write --stdin-commits Finding commits for commit graph from 584 refs: 100% (584/584), done. [...] $ (cd .git/objects/pack/ && ls *idx) | git commit-graph write --stdin-pack Finding commits for commit graph in 3 packs: 6365492, done. [...] The middle on of those is going to be the output users might see in practice, since it'll be emitted when they get the commit graph via gc.writeCommitGraph=true. But as noted above you need a really large number of refs for this message to show. It'll show up on a test repository I have with ~165k refs: Finding commits for commit graph from 165203 refs: 100% (165203/165203), done. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 28 +++++++++++++++++++++------- 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 7c1afa4704..fd1fd61750 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -779,6 +779,7 @@ void write_commit_graph(const char *obj_dir, struct progress *progress = NULL; uint64_t progress_cnt = 0; unsigned long approx_nr_objects; + struct strbuf progress_title = STRBUF_INIT; if (!commit_graph_compatible(the_repository)) return; @@ -815,8 +816,12 @@ void write_commit_graph(const char *obj_dir, strbuf_addf(&packname, "%s/pack/", obj_dir); dirlen = packname.len; if (report_progress) { - oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), 0); + strbuf_addf(&progress_title, + Q_("Finding commits for commit graph in %d pack", + "Finding commits for commit graph in %d packs", + pack_indexes->nr), + pack_indexes->nr); + oids.progress = start_delayed_progress(progress_title.buf, 0); oids.progress_done = 0; } for (i = 0; i < pack_indexes->nr; i++) { @@ -833,14 +838,20 @@ void write_commit_graph(const char *obj_dir, free(p); } stop_progress(&oids.progress); + strbuf_reset(&progress_title); strbuf_release(&packname); } if (commit_hex) { - if (report_progress) - progress = start_delayed_progress( - _("Finding commits for commit graph"), - commit_hex->nr); + if (report_progress) { + strbuf_addf(&progress_title, + Q_("Finding commits for commit graph from %d ref", + "Finding commits for commit graph from %d refs", + commit_hex->nr), + commit_hex->nr); + progress = start_delayed_progress(progress_title.buf, + commit_hex->nr); + } for (i = 0; i < commit_hex->nr; i++) { const char *end; struct object_id oid; @@ -860,12 +871,13 @@ void write_commit_graph(const char *obj_dir, } } stop_progress(&progress); + strbuf_reset(&progress_title); } if (!pack_indexes && !commit_hex) { if (report_progress) oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), + _("Finding commits for commit graph among packed objects"), approx_nr_objects); for_each_packed_object(add_packed_commits, &oids, 0); if (oids.progress_done < approx_nr_objects) @@ -970,6 +982,8 @@ void write_commit_graph(const char *obj_dir, &progress_cnt); stop_progress(&progress); + strbuf_release(&progress_title); + close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); commit_lock_file(&lk); From patchwork Tue Nov 20 19:50:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10691173 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 542CE1923 for ; Tue, 20 Nov 2018 19:50:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 43B782A896 for ; Tue, 20 Nov 2018 19:50:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4205B2AA26; Tue, 20 Nov 2018 19:50:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DECF32AC5D for ; Tue, 20 Nov 2018 19:50:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727404AbeKUGVg (ORCPT ); Wed, 21 Nov 2018 01:21:36 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:40461 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727384AbeKUGVg (ORCPT ); Wed, 21 Nov 2018 01:21:36 -0500 Received: by mail-wm1-f67.google.com with SMTP id q26so3474940wmf.5 for ; Tue, 20 Nov 2018 11:50:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=E2lrytxya4V7Ko+zCsu0ChFlej5XANHpv+2dI2aMPT0=; b=pFEvt+ZlZd8WXPMote0RR2VPURl99NS7QtRggGR9lehv9wJE20Nz8elVjujQTmy9GW OHr4kyZjZVGaHlHNg6J16fv4jIGEChfjCh8nlZg46RWcSO5aDS9m/DieR0HSAKe59kSL fxUYnn84izOQQw7umxLlX+MgzWlAAl82RGEGvY4xulrgZlHsIJmSgD7nHFl5NFkkKUPD kqjedq5o3rwiGmcKV8OGV6Nr/CTI/pQODepqo2YymDLgUWs3J0fVmS2CGuAAQlVw+qbD vHzzkmMWjT0syJDBF0T/hvmnaW9D5V6Z9hXDkV7/diw3IUXkuGbhz/ZiYYQizRlAzGT9 hvSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=E2lrytxya4V7Ko+zCsu0ChFlej5XANHpv+2dI2aMPT0=; b=MjGgHvxxt57FK5d3Dno4+/cARkIosx1zV4W10kvFjLxBSikbuguegHTW90werf49wy kTBeP5eU7Atp3jjlMu9I/paWaO4ysBxiyh99a3lqhu5BInID1zXlHXEV2Vq/YdilHmb+ 5iZZEpJyTtKXEPqjeEkNfSZ95fTktRArUDrIFTWX0vY+Jd9M26ENZWyShtMiD9Cv7qoS PuDs4I1Sgmx+8tfUDNFKn+hNazmfmdXXBsgW7bGVtiPNU1dz888H3XtSRQDb8USU7NuP 2uOdTOByWMSPWTNiR3xNhpsPFyRFbp6KP6eAwh78rxSGqe5F8t+HWggHVflI+m9VnDYh FqFw== X-Gm-Message-State: AA+aEWYvl908oHTxTRR7RDWLsJJDQ4YcMHnaaWLGaKdWzEuz95QVJLG4 cdMRBtqIcql0DmvjH0vlvct5IN9fUns= X-Google-Smtp-Source: AFSGD/XlZH2HZEzWC7Vm08nr+pwBJ/A73YtgnB9K1K/YRpjbUaq0kRo92lFUeJhta3PdW6hRV5CNHg== X-Received: by 2002:a1c:6382:: with SMTP id x124-v6mr3531705wmb.145.1542743445184; Tue, 20 Nov 2018 11:50:45 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id p14sm12589415wrt.37.2018.11.20.11.50.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 11:50:44 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v2 5/6] commit-graph write: remove empty line for readability Date: Tue, 20 Nov 2018 19:50:26 +0000 Message-Id: <20181120195027.3260-6-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181120165800.GB30222@szeder.dev> References: <20181120165800.GB30222@szeder.dev> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Remove the empty line between a QSORT(...) and the subsequent oideq() for-loop. This makes it clearer that the QSORT(...) is being done so that we can run the oideq() loop on adjacent OIDs. Amends code added in 08fd81c9b6 ("commit-graph: implement write_commit_graph()", 2018-04-02). Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 1 - 1 file changed, 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index fd1fd61750..0e98679bce 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -888,7 +888,6 @@ void write_commit_graph(const char *obj_dir, close_reachable(&oids, report_progress); QSORT(oids.list, oids.nr, commit_compare); - count_distinct = 1; for (i = 1; i < oids.nr; i++) { if (!oideq(&oids.list[i - 1], &oids.list[i])) From patchwork Tue Nov 20 19:50:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10691175 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C23813BB for ; Tue, 20 Nov 2018 19:50:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 696F72A886 for ; Tue, 20 Nov 2018 19:50:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 67F692A896; Tue, 20 Nov 2018 19:50:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F40D22AC42 for ; Tue, 20 Nov 2018 19:50:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727418AbeKUGVi (ORCPT ); Wed, 21 Nov 2018 01:21:38 -0500 Received: from mail-wr1-f66.google.com ([209.85.221.66]:37909 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727384AbeKUGVi (ORCPT ); Wed, 21 Nov 2018 01:21:38 -0500 Received: by mail-wr1-f66.google.com with SMTP id e3-v6so3263071wrs.5 for ; Tue, 20 Nov 2018 11:50:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yqwp0+d5r9LumO5f6VXtKaxV22S+qleUWqiRsciHBcQ=; b=WcYGGVLzqpYEUstQ0pUgN1DCMTwNnCmuxxtimtIqz082OtRMVAgmFm7ewnF5H/TWYS CqAyNdZd9DQedJ8RXKkWmG3wEslaY5/f5FenYfXRIYYMSGq+koXT5z4UddC1R9tydf9g CQ2AD4jSK+V2mAi4amQ7IKEqDWx0K6xxNpETYRvOm55++dY0JzkYeVM1MWaUmSh+mvG7 dPSRW2WBnpdeIHti539u3H+Yd/b2F9AAuI77tgo/Q53miDnI6DB4ipvpMidYdGLCkzvv 1+gd1+0jsY+UyAiZpvM/Crpeb9jo5e5Lu3l4LX8LVzJibVe8hiWp6qBi9m0fp5i9H0hW QTyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yqwp0+d5r9LumO5f6VXtKaxV22S+qleUWqiRsciHBcQ=; b=kKPmn0+IBt2DmXk1Pn8kGnWklNOPMtS5M9gf8WMwEUGn9ZhDrNeq0jR7WqCd6QKuR1 8jVvxsrLIowZ46466XW+VGV4Wp2uuEMEBpGZEjcjkQL1EyBz3WXtD+FZNuhhSsznDrq2 VLEA+pDMLNbQ3DyDiMUf04n4KX8A0n5amPSS3UXfaAsPVYDmHi/1AgnmrJsQcvldylJO Ahr/lM86mQKZn4kpGlZlMntR7tz5YoDGNnsjcVKKLgkify+JaMHeyShgmmmpIwVhfx/g NxdIPyp5gO493PCn9KY87ZjNZ+Bu/460GYvXkRZdUMmh94LzyzBI/ORyUn1Kmzrq+xWa lPHA== X-Gm-Message-State: AA+aEWbI3SzN4iCTCztJy73hukGYeDDrUilppcxN9vl9ydLQf3sP7sXM peLhE+vmNmeSo/ddEOkTVB44E5GCBC4= X-Google-Smtp-Source: AFSGD/XsIDSh3xm2CTpXHDkyIJdjtmzvX3QOGHhOGDRVemq4zeJc/lgLSwpG+SJqgTI/30lmu5sIkw== X-Received: by 2002:a5d:488a:: with SMTP id g10-v6mr3366500wrq.122.1542743446958; Tue, 20 Nov 2018 11:50:46 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id p14sm12589415wrt.37.2018.11.20.11.50.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 11:50:46 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v2 6/6] commit-graph write: add even more progress output Date: Tue, 20 Nov 2018 19:50:27 +0000 Message-Id: <20181120195027.3260-7-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181120165800.GB30222@szeder.dev> References: <20181120165800.GB30222@szeder.dev> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add more progress output to sections of code that can collectively take 5-10 seconds on a large enough repository. On a test repository with I have with ~7 million commits and ~50 million objects we'll now emit: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (50026015/50026015), done. Annotating commit graph: 21567407, done. Counting distinct commits in commit graph: 100% (7189147/7189147), done. Finding extra edges in commit graph: 100% (7189147/7189147), done. Computing commit graph generation numbers: 100% (7144680/7144680), done. Writing out commit graph: 21434417, done. Whereas on a medium-sized repository such as linux.git these new progress bars won't have time to kick in and as before and we'll still emit output like: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (6365492/6365492), done. Annotating commit graph: 2391666, done. Computing commit graph generation numbers: 100% (797222/797222), done. Writing out commit graph: 2399912, done. The "Counting distinct commits in commit graph" phase will spend most of its time paused at "0/*" as we QSORT(...) the list. That's not optimal, but at least we don't seem to be stalling anymore. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/commit-graph.c b/commit-graph.c index 0e98679bce..1ad9000060 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -887,12 +887,19 @@ void write_commit_graph(const char *obj_dir, close_reachable(&oids, report_progress); + if (report_progress) + progress = start_delayed_progress( + _("Counting distinct commits in commit graph"), + oids.nr); + display_progress(progress, 0); /* TODO: Measure QSORT() progress */ QSORT(oids.list, oids.nr, commit_compare); count_distinct = 1; for (i = 1; i < oids.nr; i++) { + display_progress(progress, i + 1); if (!oideq(&oids.list[i - 1], &oids.list[i])) count_distinct++; } + stop_progress(&progress); if (count_distinct >= GRAPH_PARENT_MISSING) die(_("the commit graph format cannot write %d commits"), count_distinct); @@ -902,8 +909,13 @@ void write_commit_graph(const char *obj_dir, ALLOC_ARRAY(commits.list, commits.alloc); num_extra_edges = 0; + if (report_progress) + progress = start_delayed_progress( + _("Finding extra edges in commit graph"), + oids.nr); for (i = 0; i < oids.nr; i++) { int num_parents = 0; + display_progress(progress, i + 1); if (i > 0 && oideq(&oids.list[i - 1], &oids.list[i])) continue; @@ -920,6 +932,7 @@ void write_commit_graph(const char *obj_dir, commits.nr++; } num_chunks = num_extra_edges ? 4 : 3; + stop_progress(&progress); if (commits.nr >= GRAPH_PARENT_MISSING) die(_("too many commits to write graph"));