From patchwork Sat Jan 19 20:21:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10772137 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8A39E186E for ; Sat, 19 Jan 2019 20:21:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7AD4A2BEBA for ; Sat, 19 Jan 2019 20:21:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6EEC12BEC4; Sat, 19 Jan 2019 20:21:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 139F02BEBA for ; Sat, 19 Jan 2019 20:21:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729124AbfASUVk (ORCPT ); Sat, 19 Jan 2019 15:21:40 -0500 Received: from mail-wr1-f68.google.com ([209.85.221.68]:45021 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728993AbfASUVj (ORCPT ); Sat, 19 Jan 2019 15:21:39 -0500 Received: by mail-wr1-f68.google.com with SMTP id z5so18941231wrt.11 for ; Sat, 19 Jan 2019 12:21:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rBhPv90zJG2W11C+QstOxuqhCF+LWqjiQipLdUAnzOc=; b=Wqpe+F1dr4IvYJHhxQ+94u6VpHStgwRvJGOAUlzW0lBCTRe3KOCND3AG56jUvXO1tZ Be1IooqjbQmv69xMcTMpNLAmflU1fXd/jetTuOyOTI9/YNETIJ8R6h11vpZTJtWmA/yQ p3YSt1/0HorXX0LCdL7EBiTqq0VAbUIAQU9CN0JQtr/YE0pNuDCxPwKzOHJeI9gYKalk T+qB1U6Etorpc1B9FtMY7eSO12qt2Zpf8OPfYlf95M72+5oZntfnxvI8EO3Sdz/ud0LS yMkPoPGOZE5YJqKt9lDEaJO41HgWLEMBOnpX/n8Wn/uIFzGqmuLAdZU4oprRWKkrwxcs NhYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rBhPv90zJG2W11C+QstOxuqhCF+LWqjiQipLdUAnzOc=; b=KCSQrO36vWVPCvolX1xfN/w98CFShLCsu72Ideu2DhuTyfF9yjvZejkZKWBrNlvqvM AO3JtF/knhRUSdb9woNAUrOXyLVJHRcDFIpIY1wAM6qTQrnhEeVrfdO3sdf9KRt+7SxV +iw5/gnCzNNA/gZmemB+sV9HkAM3Kk2E/RLdAiHxLI+SpWfZjkLLqX4DUw/sWZp+eyvl IkrCDhCykgClqucCekHoQA9Qd1oqgmNmbiclYjOuO945f/PfB4CULmRSfUxsPpT+IvDM HhU/htPhlwbcFzb/k+whd5U/XucyI1p26E86hqlK/4LZlfikZv0qXimyBX9NLSdt0NOy tyXw== X-Gm-Message-State: AJcUukc6LQJT/i3YvIqbGhFPSwk+iK1bYeZaWDlRDW4B9O9IdV3/+vbu EGzuFpLHILR4j0ZUjyMCwAmHjrV4WaE= X-Google-Smtp-Source: ALg8bN5DRuY1iYq8JWUFZ6lFkLahWVIfKBdVvVtwNiESMGWknijM0btos6POQEYbMhqx/YlPUna17w== X-Received: by 2002:a5d:488f:: with SMTP id g15mr21700841wrq.15.1547929297907; Sat, 19 Jan 2019 12:21:37 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id g188sm56196229wmf.32.2019.01.19.12.21.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Jan 2019 12:21:37 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 01/10] commit-graph write: use pack order when finding commits Date: Sat, 19 Jan 2019 21:21:12 +0100 Message-Id: <20190119202121.3590-2-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20190116132947.26025-1-avarab@gmail.com> References: <20190116132947.26025-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Slightly optimize the "commit-graph write" step by using FOR_EACH_OBJECT_PACK_ORDER with for_each_object_in_pack(). See commit [1] and [2] for the facility and a similar optimization for "cat-file". On Linux it is around 5% slower to run: echo 1 >/proc/sys/vm/drop_caches && cat .git/objects/pack/* >/dev/null && git cat-file --batch-all-objects --batch-check --unordered Than the same thing with the "cat" omitted. This is as expected, since we're iterating in pack order and the "cat" is extra work. Before this change the opposite was true of "commit-graph write". We were 6% faster if we first ran "cat" to efficiently populate the FS cache for our sole big pack on linux.git, than if we had populated it via for_each_object_in_pack(). Now we're 3% faster without the "cat" instead. My tests were done on an unloaded Linux 3.10 system with 10 runs for each. Derrick Stolee did his own tests on Windows[3] showing a 2% improvement with a high degree of accuracy. 1. 736eb88fdc ("for_each_packed_object: support iterating in pack-order", 2018-08-10) 2. 0750bb5b51 ("cat-file: support "unordered" output for --batch-all-objects", 2018-08-10) 3. https://public-inbox.org/git/f71fa868-25e8-a9c9-46a6-611b987f1a8f@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 5c8fb4b134..981faf0465 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -816,7 +816,8 @@ void write_commit_graph(const char *obj_dir, die(_("error adding pack %s"), packname.buf); if (open_pack_index(p)) die(_("error opening index for %s"), packname.buf); - for_each_object_in_pack(p, add_packed_commits, &oids, 0); + for_each_object_in_pack(p, add_packed_commits, &oids, + FOR_EACH_OBJECT_PACK_ORDER); close_pack(p); free(p); } @@ -854,7 +855,8 @@ void write_commit_graph(const char *obj_dir, if (report_progress) oids.progress = start_delayed_progress( _("Finding commits for commit graph"), 0); - for_each_packed_object(add_packed_commits, &oids, 0); + for_each_packed_object(add_packed_commits, &oids, + FOR_EACH_OBJECT_PACK_ORDER); stop_progress(&oids.progress); } From patchwork Sat Jan 19 20:21:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10772139 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3C31B14E5 for ; Sat, 19 Jan 2019 20:21:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2AC362BEBA for ; Sat, 19 Jan 2019 20:21:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1CD112BEC4; Sat, 19 Jan 2019 20:21:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4860A2BEBA for ; Sat, 19 Jan 2019 20:21:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729175AbfASUVo (ORCPT ); Sat, 19 Jan 2019 15:21:44 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:52358 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728993AbfASUVo (ORCPT ); Sat, 19 Jan 2019 15:21:44 -0500 Received: by mail-wm1-f65.google.com with SMTP id m1so7475558wml.2 for ; Sat, 19 Jan 2019 12:21:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=J7LgJrmMN7G4Av+nl7feznJJLcGjH+tmFBRIEL4mAiE=; b=LyZPDtCcNzNTdF5DhiuhmNTiJQPxXm4CBfekylxdBWYrHQEQ/SRaET79bzgr8ZyD1C qI3FulnNGN9S8QhOX3Md/Idjf3YFqc7wCIcP2sj8FG405NLWgHWnQlX24lEKsnU4iQ1n tfFEz5rl9lEpMMMyE3OS2yb/DUHPzP9zf+H/3mKDHI0AoNh2YR6AuBt/pZDmdSAfUEuF 3bjGIaShAEBnWvR13+pzLE3V0WteNkpGH87iUHuaFZ+7wfuy4WwMsbKgmgZ7AmTtY225 z8me0qPX5gOGj04b4Nz21LylSybsVGkiuV4SXo+oZuF82NeRr/ot2ZPqUqX+/u38VU4D Duow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J7LgJrmMN7G4Av+nl7feznJJLcGjH+tmFBRIEL4mAiE=; b=B6NeLIbujp/7r5bxy1nfyfVr9hKfvn7Q1SScauK803Ld/5S/S8A40wg/ofiL2xzFlR BIGFK+W0qL3z3jx4ABFMotwPa3kveX8cNoPkoyllrgPKcTSW2T0M9+PYoagHVRrhvkMo VeUlYS0iZ8WHVMg67xQqTNxsd3787CJ5IEc4JsfqKxVuBBny2E0rmwjdDotGPlgltp99 6v2xxPB6b8Zd8i7y5amLFZI1dQTAbIdiCdi4pUmw//fYJPfhMdeMNf45nDhFdkTQgqBO 959HYK9C+SvZJxZ1UQCMycx7oxQmGJNelAMRGMGDaN/EpEYcmaAsAJN96gYYry2VzJcq NIxg== X-Gm-Message-State: AJcUukd4D6ERkdGnFTECPVdjkk7SdhyT6ZWlWp3DaMtWEPjbrZ03Zzmt e/HIH8ZQTi4UUMcaRyAe1uq22m2OxsY= X-Google-Smtp-Source: ALg8bN7YPHGTMh6bBArncpQilwIUFqEPJmGQSccXzw3LKATCMD/hZsA9ljISTLsSm161Kga+DkkQrg== X-Received: by 2002:a1c:8089:: with SMTP id b131mr19914931wmd.141.1547929300430; Sat, 19 Jan 2019 12:21:40 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id g188sm56196229wmf.32.2019.01.19.12.21.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Jan 2019 12:21:39 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee Subject: [PATCH v6 02/10] commit-graph: rename "large edges" to "extra edges" Date: Sat, 19 Jan 2019 21:21:13 +0100 Message-Id: <20190119202121.3590-3-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20190116132947.26025-1-avarab@gmail.com> References: <20190116132947.26025-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: SZEDER Gábor The optional 'Large Edge List' chunk of the commit graph file stores parent information for commits with more than two parents, and the names of most of the macros, variables, struct fields, and functions related to this chunk contain the term "large edges", e.g. write_graph_chunk_large_edges(). However, it's not a really great term, as the edges to the second and subsequent parents stored in this chunk are not any larger than the edges to the first and second parents stored in the "main" 'Commit Data' chunk. It's the number of edges, IOW number of parents, that is larger compared to non-merge and "regular" two-parent merge commits. And indeed, two functions in 'commit-graph.c' have a local variable called 'num_extra_edges' that refer to the same thing, and this "extra edges" term is much better at describing these edges. So let's rename all these references to "large edges" in macro, variable, function, etc. names to "extra edges". There is a GRAPH_OCTOPUS_EDGES_NEEDED macro as well; for the sake of consistency rename it to GRAPH_EXTRA_EDGES_NEEDED. We can do so safely without causing any incompatibility issues, because the term "large edges" doesn't come up in the file format itself in any form (the chunk's magic is {'E', 'D', 'G', 'E'}, there is no 'L' in there), but only in the specification text. The string "large edges", however, does come up in the output of 'git commit-graph read' and in tests looking at its input, but that command is explicitly documented as debugging aid, so we can change its output and the affected tests safely. Signed-off-by: SZEDER Gábor --- .../technical/commit-graph-format.txt | 4 ++-- builtin/commit-graph.c | 4 ++-- commit-graph.c | 24 +++++++++---------- commit-graph.h | 2 +- t/t5318-commit-graph.sh | 14 +++++------ 5 files changed, 24 insertions(+), 24 deletions(-) diff --git a/Documentation/technical/commit-graph-format.txt b/Documentation/technical/commit-graph-format.txt index cc0474ba3e..16452a0504 100644 --- a/Documentation/technical/commit-graph-format.txt +++ b/Documentation/technical/commit-graph-format.txt @@ -76,7 +76,7 @@ CHUNK DATA: of the ith commit. Stores value 0x7000000 if no parent in that position. If there are more than two parents, the second value has its most-significant bit on and the other bits store an array - position into the Large Edge List chunk. + position into the Extra Edge List chunk. * The next 8 bytes store the generation number of the commit and the commit time in seconds since EPOCH. The generation number uses the higher 30 bits of the first 4 bytes, while the commit @@ -84,7 +84,7 @@ CHUNK DATA: 2 bits of the lowest byte, storing the 33rd and 34th bit of the commit time. - Large Edge List (ID: {'E', 'D', 'G', 'E'}) [Optional] + Extra Edge List (ID: {'E', 'D', 'G', 'E'}) [Optional] This list of 4-byte values store the second through nth parents for all octopus merges. The second parent value in the commit data stores an array position within this list along with the most-significant bit diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index c02a3f1221..4ae502754c 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -110,8 +110,8 @@ static int graph_read(int argc, const char **argv) printf(" oid_lookup"); if (graph->chunk_commit_data) printf(" commit_metadata"); - if (graph->chunk_large_edges) - printf(" large_edges"); + if (graph->chunk_extra_edges) + printf(" extra_edges"); printf("\n"); UNLEAK(graph); diff --git a/commit-graph.c b/commit-graph.c index 981faf0465..c5c6ab5367 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -21,7 +21,7 @@ #define GRAPH_CHUNKID_OIDFANOUT 0x4f494446 /* "OIDF" */ #define GRAPH_CHUNKID_OIDLOOKUP 0x4f49444c /* "OIDL" */ #define GRAPH_CHUNKID_DATA 0x43444154 /* "CDAT" */ -#define GRAPH_CHUNKID_LARGEEDGES 0x45444745 /* "EDGE" */ +#define GRAPH_CHUNKID_EXTRAEDGES 0x45444745 /* "EDGE" */ #define GRAPH_DATA_WIDTH 36 @@ -33,7 +33,7 @@ #define GRAPH_OID_VERSION GRAPH_OID_VERSION_SHA1 #define GRAPH_OID_LEN GRAPH_OID_LEN_SHA1 -#define GRAPH_OCTOPUS_EDGES_NEEDED 0x80000000 +#define GRAPH_EXTRA_EDGES_NEEDED 0x80000000 #define GRAPH_PARENT_MISSING 0x7fffffff #define GRAPH_EDGE_LAST_MASK 0x7fffffff #define GRAPH_PARENT_NONE 0x70000000 @@ -177,11 +177,11 @@ struct commit_graph *load_commit_graph_one(const char *graph_file) graph->chunk_commit_data = data + chunk_offset; break; - case GRAPH_CHUNKID_LARGEEDGES: - if (graph->chunk_large_edges) + case GRAPH_CHUNKID_EXTRAEDGES: + if (graph->chunk_extra_edges) chunk_repeated = 1; else - graph->chunk_large_edges = data + chunk_offset; + graph->chunk_extra_edges = data + chunk_offset; break; } @@ -343,12 +343,12 @@ static int fill_commit_in_graph(struct commit *item, struct commit_graph *g, uin edge_value = get_be32(commit_data + g->hash_len + 4); if (edge_value == GRAPH_PARENT_NONE) return 1; - if (!(edge_value & GRAPH_OCTOPUS_EDGES_NEEDED)) { + if (!(edge_value & GRAPH_EXTRA_EDGES_NEEDED)) { pptr = insert_parent_or_die(g, edge_value, pptr); return 1; } - parent_data_ptr = (uint32_t*)(g->chunk_large_edges + + parent_data_ptr = (uint32_t*)(g->chunk_extra_edges + 4 * (uint64_t)(edge_value & GRAPH_EDGE_LAST_MASK)); do { edge_value = get_be32(parent_data_ptr); @@ -504,7 +504,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, if (!parent) edge_value = GRAPH_PARENT_NONE; else if (parent->next) - edge_value = GRAPH_OCTOPUS_EDGES_NEEDED | num_extra_edges; + edge_value = GRAPH_EXTRA_EDGES_NEEDED | num_extra_edges; else { edge_value = sha1_pos(parent->item->object.oid.hash, commits, @@ -516,7 +516,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, hashwrite_be32(f, edge_value); - if (edge_value & GRAPH_OCTOPUS_EDGES_NEEDED) { + if (edge_value & GRAPH_EXTRA_EDGES_NEEDED) { do { num_extra_edges++; parent = parent->next; @@ -537,7 +537,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, } } -static void write_graph_chunk_large_edges(struct hashfile *f, +static void write_graph_chunk_extra_edges(struct hashfile *f, struct commit **commits, int nr_commits) { @@ -923,7 +923,7 @@ void write_commit_graph(const char *obj_dir, chunk_ids[1] = GRAPH_CHUNKID_OIDLOOKUP; chunk_ids[2] = GRAPH_CHUNKID_DATA; if (num_extra_edges) - chunk_ids[3] = GRAPH_CHUNKID_LARGEEDGES; + chunk_ids[3] = GRAPH_CHUNKID_EXTRAEDGES; else chunk_ids[3] = 0; chunk_ids[4] = 0; @@ -946,7 +946,7 @@ void write_commit_graph(const char *obj_dir, write_graph_chunk_fanout(f, commits.list, commits.nr); write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr); write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr); - write_graph_chunk_large_edges(f, commits.list, commits.nr); + write_graph_chunk_extra_edges(f, commits.list, commits.nr); close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); diff --git a/commit-graph.h b/commit-graph.h index 9db40b4d3a..e6aff2c2e1 100644 --- a/commit-graph.h +++ b/commit-graph.h @@ -49,7 +49,7 @@ struct commit_graph { const uint32_t *chunk_oid_fanout; const unsigned char *chunk_oid_lookup; const unsigned char *chunk_commit_data; - const unsigned char *chunk_large_edges; + const unsigned char *chunk_extra_edges; }; struct commit_graph *load_commit_graph_one(const char *graph_file); diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 5fe21db99f..f4deb13b1d 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -122,7 +122,7 @@ test_expect_success 'write graph with merges' ' cd "$TRASH_DIRECTORY/full" && git commit-graph write && test_path_is_file $objdir/info/commit-graph && - graph_read_expect "10" "large_edges" + graph_read_expect "10" "extra_edges" ' graph_git_behavior 'merge 1 vs 2' full merge/1 merge/2 @@ -157,7 +157,7 @@ test_expect_success 'write graph with new commit' ' cd "$TRASH_DIRECTORY/full" && git commit-graph write && test_path_is_file $objdir/info/commit-graph && - graph_read_expect "11" "large_edges" + graph_read_expect "11" "extra_edges" ' graph_git_behavior 'full graph, commit 8 vs merge 1' full commits/8 merge/1 @@ -167,7 +167,7 @@ test_expect_success 'write graph with nothing new' ' cd "$TRASH_DIRECTORY/full" && git commit-graph write && test_path_is_file $objdir/info/commit-graph && - graph_read_expect "11" "large_edges" + graph_read_expect "11" "extra_edges" ' graph_git_behavior 'cleared graph, commit 8 vs merge 1' full commits/8 merge/1 @@ -177,7 +177,7 @@ test_expect_success 'build graph from latest pack with closure' ' cd "$TRASH_DIRECTORY/full" && cat new-idx | git commit-graph write --stdin-packs && test_path_is_file $objdir/info/commit-graph && - graph_read_expect "9" "large_edges" + graph_read_expect "9" "extra_edges" ' graph_git_behavior 'graph from pack, commit 8 vs merge 1' full commits/8 merge/1 @@ -200,7 +200,7 @@ test_expect_success 'build graph from commits with append' ' cd "$TRASH_DIRECTORY/full" && git rev-parse merge/3 | git commit-graph write --stdin-commits --append && test_path_is_file $objdir/info/commit-graph && - graph_read_expect "10" "large_edges" + graph_read_expect "10" "extra_edges" ' graph_git_behavior 'append graph, commit 8 vs merge 1' full commits/8 merge/1 @@ -210,7 +210,7 @@ test_expect_success 'build graph using --reachable' ' cd "$TRASH_DIRECTORY/full" && git commit-graph write --reachable && test_path_is_file $objdir/info/commit-graph && - graph_read_expect "11" "large_edges" + graph_read_expect "11" "extra_edges" ' graph_git_behavior 'append graph, commit 8 vs merge 1' full commits/8 merge/1 @@ -231,7 +231,7 @@ test_expect_success 'write graph in bare repo' ' cd "$TRASH_DIRECTORY/bare" && git commit-graph write && test_path_is_file $baredir/info/commit-graph && - graph_read_expect "11" "large_edges" + graph_read_expect "11" "extra_edges" ' graph_git_behavior 'bare repo with graph, commit 8 vs merge 1' bare commits/8 merge/1 From patchwork Sat Jan 19 20:21:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10772141 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3838314E5 for ; Sat, 19 Jan 2019 20:21:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2906D2BEBA for ; Sat, 19 Jan 2019 20:21:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1DA0C2BEC4; Sat, 19 Jan 2019 20:21:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A8D112BEBA for ; Sat, 19 Jan 2019 20:21:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729212AbfASUVq (ORCPT ); Sat, 19 Jan 2019 15:21:46 -0500 Received: from mail-wr1-f68.google.com ([209.85.221.68]:37748 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728993AbfASUVq (ORCPT ); Sat, 19 Jan 2019 15:21:46 -0500 Received: by mail-wr1-f68.google.com with SMTP id s12so18996951wrt.4 for ; Sat, 19 Jan 2019 12:21:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UsNJrMfSJ2qH96rKs6jpLOozL3dh50Gq/oVHrSTAsxY=; b=jqeXFOyFKLn1SKeCJx9ueM23SmB1HmYuv43Hlmh61rQfeTX9/PtMGyGBF35Tc2MLGQ Mf2Uzxmn5v4mylixg/5baV/XM5DAwErfADjB2J6aGbg31g9KwH/VYBEdjVxi2UDJ2VkG e65LPATnpY7nCR+9pIKGrS6UG0D2iWl8GL7TNQrltLuOwhGEsKYcTnLurVAKolhRY7R9 tOlfbulvdrG/R85aEvCxS4MW7zgC3DO1p85lIKsgeyFLp3lRNXmTc1pXTGLxclRERE0i zv8cnFAkIV+2Lfdbo5l1eTNNvkdbuQxnt64VeGQhRjenZPW82DGMOuVqNXVxjIBovZug oB2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UsNJrMfSJ2qH96rKs6jpLOozL3dh50Gq/oVHrSTAsxY=; b=Vb6JXcb4d7LPtTGFXt+Hgh54IcgIkoAO2oj7Q565IogZoCq+NPNfej0MJhVl01kKza CQjDXsm+j+gnhsL/IRLRMQpQzeSj/OMEcTG+hPEgi1hdIa/8BWUeWv7qLuef2GxF56sb Sqd7+fpLPUDlnrE2yJgmiiBqIDZS2cfLZ/NfQbHF8xwuHmISscYh41mf2UpNdN5ptwQX BuwYSJAgbyHv8BPMzSEmrXjcOXI+IyBexciPtQceOdOiOHuf2KDpLFQDS+nRLsCsmLnh AoR7O+kT8AI43J41fDxGvYOTxTa1PQvIctyfvLch14MqwLlkfNjDk0Rju7E+VkyPTCMS 7rTw== X-Gm-Message-State: AJcUukdJHhyyx0qJPv6MRW+IDogz6IfZqKss6nifzkONwkRCcsLgTuTs CNf7aNKu/lxJp5MT/nfFLjhTvHsvpXo= X-Google-Smtp-Source: ALg8bN4B38JuRWRqOekzdq+eT5wneftFbcMmN/MPGFaEnsTr2CwRHSirgLpWVxQdrmkX5kNj9tBIMA== X-Received: by 2002:a5d:4b8b:: with SMTP id b11mr21704745wrt.180.1547929304209; Sat, 19 Jan 2019 12:21:44 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id g188sm56196229wmf.32.2019.01.19.12.21.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Jan 2019 12:21:43 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee Subject: [PATCH v6 03/10] commit-graph: don't call write_graph_chunk_large_edges() unnecessarily Date: Sat, 19 Jan 2019 21:21:14 +0100 Message-Id: <20190119202121.3590-4-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20190116132947.26025-1-avarab@gmail.com> References: <20190116132947.26025-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: SZEDER Gábor The optional 'Large Edge List' chunk of the commit graph file stores parent information for commits with more than two parents. Since the chunk is optional, write_commit_graph() looks through all commits to find those with more than two parents, and then writes the commit graph file header accordingly, i.e. if there are no such commits, then there won't be a 'Large Edge List' chunk written, only the three mandatory chunks. However, when it later comes to writing actual chunk data, write_commit_graph() unconditionally invokes write_graph_chunk_large_edges(), even when it was decided earlier that that chunk won't be written. Strictly speaking there is no bug here, because write_graph_chunk_large_edges() won't write anything if it doesn't find any commits with more than two parents, but then it unnecessarily and in vain looks through all commits once again in search for such commits. Don't call write_graph_chunk_large_edges() when that chunk won't be written to spare an unnecessary iteration over all commits. Signed-off-by: SZEDER Gábor --- commit-graph.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index c5c6ab5367..e733ba180a 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -946,7 +946,8 @@ void write_commit_graph(const char *obj_dir, write_graph_chunk_fanout(f, commits.list, commits.nr); write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr); write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr); - write_graph_chunk_extra_edges(f, commits.list, commits.nr); + if (num_extra_edges) + write_graph_chunk_extra_edges(f, commits.list, commits.nr); close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); From patchwork Sat Jan 19 20:21:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10772143 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F1AC61390 for ; Sat, 19 Jan 2019 20:21:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DFF2F2BEBA for ; Sat, 19 Jan 2019 20:21:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D43D52BEC4; Sat, 19 Jan 2019 20:21:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4BE5E2BEBA for ; Sat, 19 Jan 2019 20:21:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729288AbfASUVt (ORCPT ); Sat, 19 Jan 2019 15:21:49 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:33800 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728993AbfASUVs (ORCPT ); Sat, 19 Jan 2019 15:21:48 -0500 Received: by mail-wm1-f68.google.com with SMTP id y185so3361320wmd.1 for ; Sat, 19 Jan 2019 12:21:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=30+zpf+826Gpa9hdpefbBJ/1ds1oDDMk6l4d3mD0IRk=; b=uqCCGB7AzbuPHVaH1jyIyof45sVOApsHRuoaPwcgLk2I1BAdnI+rzc0bO4KnQwc7qN V93xpkXDpIeQ1kEswKTv7oAv8CvSoCxtGoN+SmwFn0H5kvsJGdlOQt/mHQo0aNB0zAU5 BFvCVYpcz1BlWkXZaCJLvO5ROZ7J7Xr3gPjekLbXo0GfPP7u3Nl/N2MHhAPschtnMKP8 5EVcbAiiQnczqv4aUctuCrWhv9KdXpW/hCWD0L7xN4BASeztuAT3tbd5s248BZe4xL8n JrBKTqVHPKlB+3pYmGub0z/15HF5Ns+u2gg8SWqg+zNPT5Vw7Bp/A2otk+vOvLQnG2co Jjtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=30+zpf+826Gpa9hdpefbBJ/1ds1oDDMk6l4d3mD0IRk=; b=gbfJqZckBmW2hH/NT0n+dkdt0fGleLx+Utc5H0htrXQlJ8V6oJN0SjRWvQKaa+uxAV Y5C8XUM46pOIPCs0oFETONY5z9JNRS919htMMqNANqGpx4rOcI7LjKhZ4XXeN1glRlxD ZQbHFWJ47adqAHjknPz318Be04p8Kq4VmxU3yz247nms3dLu58/200OfhxTxUsMYrYIM qApflwQIwwc6wByixKhvBlc1np2jVRDDyyHPGsV/uYwNQd6KqqMmLM0nUQNj/iEF3kd7 dSPQzIfsldW5V/awA15Ir7JF7ri5285WO4c1OdI1AigqgwOTC9RsptsV8Y4VFTAmtCGf tuUw== X-Gm-Message-State: AJcUukc/lV8FUedUbdL9HYBSpyZ422TGYP4zGPm/oIoTKY8K5oFUVmg3 kjWilRJ81jnQ2KqsAWqHRVCmaKKPLBc= X-Google-Smtp-Source: ALg8bN7YjWoHe4n7+sB1yzQ3YO8wELtSGmvJKtzqUZ4lT5JHbjfkQwR2mksqJJcvvXEEvc8ozSX27w== X-Received: by 2002:a1c:4e08:: with SMTP id g8mr19231728wmh.46.1547929305609; Sat, 19 Jan 2019 12:21:45 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id g188sm56196229wmf.32.2019.01.19.12.21.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Jan 2019 12:21:45 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 04/10] commit-graph write: add "Writing out" progress output Date: Sat, 19 Jan 2019 21:21:15 +0100 Message-Id: <20190119202121.3590-5-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20190116132947.26025-1-avarab@gmail.com> References: <20190116132947.26025-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add progress output to be shown when we're writing out the commit-graph, this adds to the output already added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17). As noted in that commit most of the progress output isn't displayed on small repositories, but before this change we'd noticeably hang for 2-3 seconds at the end on medium sized repositories such as linux.git. Now we'll instead show output like this, and reduce the human-observable times at which we're not producing progress output: $ ~/g/git/git --exec-path=$HOME/g/git -C ~/g/2015-04-03-1M-git commit-graph write Finding commits for commit graph: 13064614, done. Expanding reachable commits in commit graph: 1000447, done. Computing commit graph generation numbers: 100% (1000447/1000447), done. Writing out commit graph: 100% (3001341/3001341), done. This "Writing out" number is 3x or 4x the number of commits, depending on the graph we're processing. A later change will make this explicit to the user. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 39 ++++++++++++++++++++++++++++++--------- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index e733ba180a..a407d5bff4 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -430,7 +430,9 @@ struct tree *get_commit_tree_in_graph(struct repository *r, const struct commit static void write_graph_chunk_fanout(struct hashfile *f, struct commit **commits, - int nr_commits) + int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { int i, count = 0; struct commit **list = commits; @@ -444,6 +446,7 @@ static void write_graph_chunk_fanout(struct hashfile *f, while (count < nr_commits) { if ((*list)->object.oid.hash[0] != i) break; + display_progress(progress, ++*progress_cnt); count++; list++; } @@ -453,12 +456,16 @@ static void write_graph_chunk_fanout(struct hashfile *f, } static void write_graph_chunk_oids(struct hashfile *f, int hash_len, - struct commit **commits, int nr_commits) + struct commit **commits, int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; int count; - for (count = 0; count < nr_commits; count++, list++) + for (count = 0; count < nr_commits; count++, list++) { + display_progress(progress, ++*progress_cnt); hashwrite(f, (*list)->object.oid.hash, (int)hash_len); + } } static const unsigned char *commit_to_sha1(size_t index, void *table) @@ -468,7 +475,9 @@ static const unsigned char *commit_to_sha1(size_t index, void *table) } static void write_graph_chunk_data(struct hashfile *f, int hash_len, - struct commit **commits, int nr_commits) + struct commit **commits, int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; struct commit **last = commits + nr_commits; @@ -478,6 +487,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, struct commit_list *parent; int edge_value; uint32_t packedDate[2]; + display_progress(progress, ++*progress_cnt); parse_commit(*list); hashwrite(f, get_commit_tree_oid(*list)->hash, hash_len); @@ -539,7 +549,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, static void write_graph_chunk_extra_edges(struct hashfile *f, struct commit **commits, - int nr_commits) + int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; struct commit **last = commits + nr_commits; @@ -547,6 +559,9 @@ static void write_graph_chunk_extra_edges(struct hashfile *f, while (list < last) { int num_parents = 0; + + display_progress(progress, ++*progress_cnt); + for (parent = (*list)->parents; num_parents < 3 && parent; parent = parent->next) num_parents++; @@ -768,6 +783,7 @@ void write_commit_graph(const char *obj_dir, int num_extra_edges; struct commit_list *parent; struct progress *progress = NULL; + uint64_t progress_cnt = 0; if (!commit_graph_compatible(the_repository)) return; @@ -943,11 +959,16 @@ void write_commit_graph(const char *obj_dir, hashwrite(f, chunk_write, 12); } - write_graph_chunk_fanout(f, commits.list, commits.nr); - write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr); - write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr); + if (report_progress) + progress = start_delayed_progress( + _("Writing out commit graph"), + num_chunks * commits.nr); + write_graph_chunk_fanout(f, commits.list, commits.nr, progress, &progress_cnt); + write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr, progress, &progress_cnt); + write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr, progress, &progress_cnt); if (num_extra_edges) - write_graph_chunk_extra_edges(f, commits.list, commits.nr); + write_graph_chunk_extra_edges(f, commits.list, commits.nr, progress, &progress_cnt); + stop_progress(&progress); close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); From patchwork Sat Jan 19 20:21:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10772145 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8B5B014E5 for ; Sat, 19 Jan 2019 20:21:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7B0B32BEBA for ; Sat, 19 Jan 2019 20:21:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6F9312BEC4; Sat, 19 Jan 2019 20:21:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 125C12BEBA for ; Sat, 19 Jan 2019 20:21:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729315AbfASUVt (ORCPT ); Sat, 19 Jan 2019 15:21:49 -0500 Received: from mail-wr1-f68.google.com ([209.85.221.68]:37750 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729242AbfASUVt (ORCPT ); Sat, 19 Jan 2019 15:21:49 -0500 Received: by mail-wr1-f68.google.com with SMTP id s12so18997010wrt.4 for ; Sat, 19 Jan 2019 12:21:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pgVxHNCxtKCaVqyy5QtmUnUqBhaKqiPOUl7lcgQMAIk=; b=TuADVFrKg5v3/3GoMD5rQVnbbbcRO0qwLs1Og9zM9KPm05WC9hhrMcJFIOUw6/fXKz BggoQrnUP2h3xvPpKusptKVclHuWFm4ZS3CiSGj5tFDtI5h14SnltR4kOhZ0/zT3yCLa +sOAdqcUyrmULsmH7A67y2cNRQk61uf8KX9QPdMSNVtW+nn4PND+IEyBmW8FbOcdMpx1 vV3hHdBVoRuzMRDyMdQY2de59/HKZiqdJbaZOiYrjpiwnnmRHhKlINSxwPkndH+f3TiM j9mOC+7r7j4yD8JA3Otf9f0O1UXBPQYPWEvbdfsFZJZycnLxJWbKi4GkF9T7xe43uLJ2 PNTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pgVxHNCxtKCaVqyy5QtmUnUqBhaKqiPOUl7lcgQMAIk=; b=MLLwhCKI2jOu7KFLC5OFssxj3VewIrEV4HWXEwh0husPspjwMzu2RsQnxLOx3/0+4t kPRGzwBRnbTNUyDSt6BiBGibO0NHX+WBlGUbDtayQP81xQHvuY69DNEZNHq5S8elnxt9 MkHKVap1ch/uTuxGGjPjznX8lQaTlA4pvYknxtVMYaDJJpYcGbRIxd0if2HqqCSjbyxU sMxdQVTHfDrgWvGUglvajkqMm74Gu3wvTUZtLtQkzsi91btgdmmXSlpowZjn9QQeBgPr 7dDuSStLswgMJHZa43lnPbXjpzbNpjYUl5xuFcOa6KXGwqXvUfA31BLD2tkOstB7QDug 9n0Q== X-Gm-Message-State: AJcUukcODchFKyjv/M2S45NNmMQXq7dMlYxt8oN4sWGEjbpAiTcCzNz7 6/eqzRFJlxPnL1bOVt6zSkzvOtfDpe0= X-Google-Smtp-Source: ALg8bN6sCDhizRfNkke0+XNuknzyGNtQhyL2ni87GbGuYJxKTv3ECmbwKGnpGAdn39XnjAuJQ92/VQ== X-Received: by 2002:adf:9123:: with SMTP id j32mr22666638wrj.122.1547929306879; Sat, 19 Jan 2019 12:21:46 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id g188sm56196229wmf.32.2019.01.19.12.21.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Jan 2019 12:21:46 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 05/10] commit-graph write: more descriptive "writing out" output Date: Sat, 19 Jan 2019 21:21:16 +0100 Message-Id: <20190119202121.3590-6-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20190116132947.26025-1-avarab@gmail.com> References: <20190116132947.26025-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make the "Writing out" part of the progress output more descriptive. Depending on the shape of the graph we either make 3 or 4 passes over it. Let's present this information to the user in case they're wondering what this number, which is much larger than their number of commits, has to do with writing out the commit graph. Now e.g. on linux.git we emit: $ ~/g/git/git --exec-path=$HOME/g/git -C ~/g/linux commit-graph write Finding commits for commit graph: 6529159, done. Expanding reachable commits in commit graph: 815990, done. Computing commit graph generation numbers: 100% (815983/815983), done. Writing out commit graph in 4 passes: 100% (3263932/3263932), done. A note on i18n: Why are we using the Q_() function and passing a number & English text for a singular which'll never be used? Because the plural rules of translated languages may not match those of English, and to use the plural function we need to use this format. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index a407d5bff4..7c639c6a65 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -784,6 +784,7 @@ void write_commit_graph(const char *obj_dir, struct commit_list *parent; struct progress *progress = NULL; uint64_t progress_cnt = 0; + struct strbuf progress_title = STRBUF_INIT; if (!commit_graph_compatible(the_repository)) return; @@ -959,16 +960,23 @@ void write_commit_graph(const char *obj_dir, hashwrite(f, chunk_write, 12); } - if (report_progress) + if (report_progress) { + strbuf_addf(&progress_title, + Q_("Writing out commit graph in %d pass", + "Writing out commit graph in %d passes", + num_chunks), + num_chunks); progress = start_delayed_progress( - _("Writing out commit graph"), + progress_title.buf, num_chunks * commits.nr); + } write_graph_chunk_fanout(f, commits.list, commits.nr, progress, &progress_cnt); write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr, progress, &progress_cnt); write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr, progress, &progress_cnt); if (num_extra_edges) write_graph_chunk_extra_edges(f, commits.list, commits.nr, progress, &progress_cnt); stop_progress(&progress); + strbuf_release(&progress_title); close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); From patchwork Sat Jan 19 20:21:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10772147 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E4A741390 for ; Sat, 19 Jan 2019 20:21:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D59EC2BEBA for ; Sat, 19 Jan 2019 20:21:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C9BA02BEC4; Sat, 19 Jan 2019 20:21:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 52A9D2BEBA for ; Sat, 19 Jan 2019 20:21:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729328AbfASUVw (ORCPT ); Sat, 19 Jan 2019 15:21:52 -0500 Received: from mail-wr1-f68.google.com ([209.85.221.68]:45031 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729312AbfASUVu (ORCPT ); Sat, 19 Jan 2019 15:21:50 -0500 Received: by mail-wr1-f68.google.com with SMTP id z5so18941510wrt.11 for ; Sat, 19 Jan 2019 12:21:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/yGvz4PfCVxMpFmK3i8eeRubVvP4JHN04hM8h8FInwE=; b=QZaFXp0hSxv6SeHd+rvwc48tZ5Lk1WCclJVG/LtlIpRyf7nQM42ZZB/EIft8CLyaKl CncZZa/3yVIQ3KKpFJuCxkxStihdnob3GinbikhTOBVj0NfisbMM9TN/L336xleRboyb kItV5cXYOUIiGRm9lth0H59VB/kP2R+3NN6B2/0u4qLxd+V3jnr9ps/MywhaetmDs22F 4bLuooo0tsFIkkRnCgYUtXn5tHVQDNcwITxSCn/ZCFw7d5YvYVNjWsFmvfy9I3Ct+j25 jVDGmmQnBfp7cbmE7utsRPacYoNoubOQUnje0/NKxsnl+cFErec88dzWmSMPOY/nMW3V svcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/yGvz4PfCVxMpFmK3i8eeRubVvP4JHN04hM8h8FInwE=; b=pF4HTKMASNDn7kC6nN2oNA3Mp8PbucAcxLyfQ/LIIkVMMf5qTu6EOm3s+y4oIHNEC0 HgmJ/0pypp+g47IYaZsr634ok2HbNo4ozF+OumniUuLDAYPF39Ra3TdMJOO0OjYRTReX rVnAidegNlO4lS9Edv6D0Acrf/sUrFe3He6+bTjLzkvlqEZGSNrQKy1UZO+fiy/upcD4 03uEWKSKI8WDQJdYaC/6xTMdaMP6zA457YhfLmFovJPXcd0dJCZCpFUXsFC6DlXVhAQB eyRiBX9t/n33tlj6WXia+0tcD/Vpua+GqGlWYLnaUT5IqNSGVGWer+vID/82YkLsHP4U I4xw== X-Gm-Message-State: AJcUukeMtEruSkG1xoYJx2vC5FUiVhrvJAP8fWVtmtJYLIW71MZDdeD1 SUZ1Ap9lfNiZo6nfYpvZrUKMFo8JmY0= X-Google-Smtp-Source: ALg8bN5rh0qbGXnXdv1kSxua/wiO9GMpCu+11P1h9PSNE2thJJd4FVMxHT9fJJHTjeP/Xleim8QDTw== X-Received: by 2002:adf:b243:: with SMTP id y3mr22321251wra.184.1547929308189; Sat, 19 Jan 2019 12:21:48 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id g188sm56196229wmf.32.2019.01.19.12.21.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Jan 2019 12:21:47 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 06/10] commit-graph write: show progress for object search Date: Sat, 19 Jan 2019 21:21:17 +0100 Message-Id: <20190119202121.3590-7-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20190116132947.26025-1-avarab@gmail.com> References: <20190116132947.26025-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Show the percentage progress for the "Finding commits for commit graph" phase for the common case where we're operating on all packs in the repository, as "commit-graph write" or "gc" will do. Before we'd emit on e.g. linux.git with "commit-graph write": Finding commits for commit graph: 6529159, done. [...] And now: Finding commits for commit graph: 100% (6529159/6529159), done. [...] Since the commit graph only includes those commits that are packed (via for_each_packed_object(...)) the approximate_object_count() returns the actual number of objects we're going to process. Still, it is possible due to a race with "gc" or another process maintaining packs that the number of objects we're going to process is lower than what approximate_object_count() reported. In that case we don't want to stop the progress bar short of 100%. So let's make sure it snaps to 100% at the end. The inverse case is also possible and more likely. I.e. that a new pack has been added between approximate_object_count() and for_each_packed_object(). In that case the percentage will go beyond 100%, and we'll do nothing to snap it back to 100% at the end. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 7c639c6a65..3124dfae13 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -785,12 +785,14 @@ void write_commit_graph(const char *obj_dir, struct progress *progress = NULL; uint64_t progress_cnt = 0; struct strbuf progress_title = STRBUF_INIT; + unsigned long approx_nr_objects; if (!commit_graph_compatible(the_repository)) return; oids.nr = 0; - oids.alloc = approximate_object_count() / 32; + approx_nr_objects = approximate_object_count(); + oids.alloc = approx_nr_objects / 32; oids.progress = NULL; oids.progress_done = 0; @@ -871,9 +873,12 @@ void write_commit_graph(const char *obj_dir, if (!pack_indexes && !commit_hex) { if (report_progress) oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), 0); + _("Finding commits for commit graph"), + approx_nr_objects); for_each_packed_object(add_packed_commits, &oids, FOR_EACH_OBJECT_PACK_ORDER); + if (oids.progress_done < approx_nr_objects) + display_progress(oids.progress, approx_nr_objects); stop_progress(&oids.progress); } From patchwork Sat Jan 19 20:21:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10772151 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 54D6514E5 for ; Sat, 19 Jan 2019 20:21:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 41B342BEBA for ; Sat, 19 Jan 2019 20:21:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 341872BEC4; Sat, 19 Jan 2019 20:21:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C2E7F2BEBA for ; Sat, 19 Jan 2019 20:21:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729365AbfASUVy (ORCPT ); Sat, 19 Jan 2019 15:21:54 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:35491 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729242AbfASUVv (ORCPT ); Sat, 19 Jan 2019 15:21:51 -0500 Received: by mail-wm1-f65.google.com with SMTP id t200so7511329wmt.0 for ; Sat, 19 Jan 2019 12:21:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e54Zak99FFaDrZFsHZQS8t0t3aOw3lHqNbIw9RsP3mk=; b=ZVLiWucNp9Oe+bdDyZnYmoG6VvAWk2BsU7nVDhjrEzfBtekEe2JJKQ305jCLzygy3S cLSjRwXkWK24l9JmqGtsTRKG1jiySr8JSEnbYQ2QWoEsR4vaSdfLFkq4vT14Mo3wzXlH eEgD7mQW+5gAiX//Hj/OALfyyuJFN5EdcD6Fs+eWjoAh41ObC4lWWRl5xGrRPEX0MHfw PeiGTlz3yZoHovI+9RbAr7UbChKf71temMT96MVqo/oN4QFTfNsIERjHhcuBjNjljDVr aIwrTqJa+QFrUvxVGGC8k31CGNZ5zqFiUIvaIjDNv2jfMtIY53ZZOmmuPYuSp6DJEV/h jQQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e54Zak99FFaDrZFsHZQS8t0t3aOw3lHqNbIw9RsP3mk=; b=antk3TW3jpGh3lTgJ+BCluG8AmH+SQJ2548nO3VQ2Ry/2rmwZ2d7jim8FjQVUFmE52 lawuc/wKdD1Ae3m3rq5XVcnFbNK8+0pgn0mjiQUzQlLItZqWAsMwpV56fKxk9023jzki /1ovXWlyNZGkkaswy18e4dLt57w2C7yhy/2qHdw+1nwbvddVFMYADDdNMf3Jb0d+M0wN cowRfaVjv/Slz+OuPV/KqQkMSY8NKLEBnRh2pOnTiElLhyC46e12v95vWfovdflQyxii OtOTWVkEth7BUOEV2f5/k+CTaGSuQWbuK1uOF4YDraU99ECaOCiTwCTpNWgN9bJiiZak 4HlA== X-Gm-Message-State: AJcUukfhfmZUQRXsmM602Af87zEH2t/oHn27t6X+l12FO6e7STf/oPCo LbhYk4vY8p9kvmrqp6/n14qx1LVYiXU= X-Google-Smtp-Source: ALg8bN4iGODKhQ/Kpfl23jq7GrNRE8/JwYAYG5knZqB6DTCPLJQFmRgxzTEE8ZE2ZWllZssNNdnQ7w== X-Received: by 2002:a1c:b53:: with SMTP id 80mr18494107wml.121.1547929309361; Sat, 19 Jan 2019 12:21:49 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id g188sm56196229wmf.32.2019.01.19.12.21.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Jan 2019 12:21:48 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 07/10] commit-graph write: add more descriptive progress output Date: Sat, 19 Jan 2019 21:21:18 +0100 Message-Id: <20190119202121.3590-8-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20190116132947.26025-1-avarab@gmail.com> References: <20190116132947.26025-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make the progress output shown when we're searching for commits to include in the graph more descriptive. This amends code I added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17). Now, on linux.git, we'll emit this sort of output in the various modes we support: $ git commit-graph write Finding commits for commit graph among packed objects: 100% (6529159/6529159), done. [...] # Actually we don't emit this since this takes almost no time at # all. But if we did (s/_delayed//) we'd show: $ git for-each-ref --format='%(objectname)' | git commit-graph write --stdin-commits Finding commits for commit graph from 630 refs: 100% (630/630), done. [...] $ (cd .git/objects/pack/ && ls *idx) | git commit-graph write --stdin-pack Finding commits for commit graph in 3 packs: 6529159, done. [...] The middle on of those is going to be the output users might see in practice, since it'll be emitted when they get the commit graph via gc.writeCommitGraph=true. But as noted above you need a really large number of refs for this message to show. It'll show up on a test repository I have with ~165k refs: Finding commits for commit graph from 165203 refs: 100% (165203/165203), done. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 3124dfae13..936119b1f9 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -822,8 +822,12 @@ void write_commit_graph(const char *obj_dir, strbuf_addf(&packname, "%s/pack/", obj_dir); dirlen = packname.len; if (report_progress) { - oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), 0); + strbuf_addf(&progress_title, + Q_("Finding commits for commit graph in %d pack", + "Finding commits for commit graph in %d packs", + pack_indexes->nr), + pack_indexes->nr); + oids.progress = start_delayed_progress(progress_title.buf, 0); oids.progress_done = 0; } for (i = 0; i < pack_indexes->nr; i++) { @@ -841,14 +845,20 @@ void write_commit_graph(const char *obj_dir, free(p); } stop_progress(&oids.progress); + strbuf_reset(&progress_title); strbuf_release(&packname); } if (commit_hex) { - if (report_progress) - progress = start_delayed_progress( - _("Finding commits for commit graph"), - commit_hex->nr); + if (report_progress) { + strbuf_addf(&progress_title, + Q_("Finding commits for commit graph from %d ref", + "Finding commits for commit graph from %d refs", + commit_hex->nr), + commit_hex->nr); + progress = start_delayed_progress(progress_title.buf, + commit_hex->nr); + } for (i = 0; i < commit_hex->nr; i++) { const char *end; struct object_id oid; @@ -868,12 +878,13 @@ void write_commit_graph(const char *obj_dir, } } stop_progress(&progress); + strbuf_reset(&progress_title); } if (!pack_indexes && !commit_hex) { if (report_progress) oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), + _("Finding commits for commit graph among packed objects"), approx_nr_objects); for_each_packed_object(add_packed_commits, &oids, FOR_EACH_OBJECT_PACK_ORDER); From patchwork Sat Jan 19 20:21:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10772149 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7DC391390 for ; Sat, 19 Jan 2019 20:21:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6E1DC2BEBA for ; Sat, 19 Jan 2019 20:21:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6269E2BEC4; Sat, 19 Jan 2019 20:21:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 19CE72BEBA for ; Sat, 19 Jan 2019 20:21:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729375AbfASUVz (ORCPT ); Sat, 19 Jan 2019 15:21:55 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:36649 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729318AbfASUVw (ORCPT ); Sat, 19 Jan 2019 15:21:52 -0500 Received: by mail-wr1-f65.google.com with SMTP id u4so18991363wrp.3 for ; Sat, 19 Jan 2019 12:21:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hyjqvubp003W3u5uYdAHZrnISz65SFC/0PmlICtX8aw=; b=rDTo+NoTPnW0hZBoRqJFHDkHgGsluFDjw/UqfreJEKuodUIZUzZ/lNQVssxsxV5KKC L99+xjf7Lu6kZKP5CjGn5cpNpHCCQoZLLocN8Rf0HEAMv3AzVmtUAqaJu2F0/f5bwc3g yGsD0HvRdfzQEerDANJM+v9dkDx2tglqgZX5kkl+ckFwfp6W/xeaCkZV9KqaQYfMWNhL 6azyrHRcvS1GNJhMwLResq/kNk5eVoK9rkTCI00jRFMQpD71y1y9B1RfBfkDVY2FgJjc s7vw63ERrPhm0/QAgehdms0F1cUXDGmZVM5Mstes2nqatVhivKOO9PdOUN0/HXvLw2TZ 8eqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hyjqvubp003W3u5uYdAHZrnISz65SFC/0PmlICtX8aw=; b=rseER47xfrNG0GfpZmBr4dEJZKpuHNgO7zGZxCwaXbgi1kLzZnoBjoOxlNdJnZ8ZDy pxukjGBK/HWx2lhRFD5t1yoHYMLXQvPhSH+xtuNi4xHwT0OQnkjALe0HItzMk/sqQ5xs ZfEFcVxXjIOmd939PFLpHjfRseNNi5ox+uc4NGgwqEsBkVrGkMvj59LKZzKbzdqDq88x LW2ibPHIiWWD8/ZUlGKrUXSMLUJQT0qf7wGPLvT2k3sERFObVFtU+nFBJIP5agW2rTvA LZ9347Qp6leScO5VGq9dw9mAqbDRSRjFZe8wD3tQbXfsf04+fJNIbfVj714Twp4v1+f/ o8BQ== X-Gm-Message-State: AJcUuke5AVAFfRRm/g1BJup+VBblUaJzTY6HFgNfaU47o13iEHCecP0d jmKQFkSS4Yea5lwgID0KlCG+TTiWC5o= X-Google-Smtp-Source: ALg8bN7IU48qU7+WEoH8GRRc4gDrd8LiN9medn9jDQWCVscz8v8i2AYmxYecUnm4cGXSAik97B80KA== X-Received: by 2002:adf:d243:: with SMTP id o3mr22349322wri.66.1547929310643; Sat, 19 Jan 2019 12:21:50 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id g188sm56196229wmf.32.2019.01.19.12.21.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Jan 2019 12:21:50 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 08/10] commit-graph write: remove empty line for readability Date: Sat, 19 Jan 2019 21:21:19 +0100 Message-Id: <20190119202121.3590-9-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20190116132947.26025-1-avarab@gmail.com> References: <20190116132947.26025-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Remove the empty line between a QSORT(...) and the subsequent oideq() for-loop. This makes it clearer that the QSORT(...) is being done so that we can run the oideq() loop on adjacent OIDs. Amends code added in 08fd81c9b6 ("commit-graph: implement write_commit_graph()", 2018-04-02). Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 1 - 1 file changed, 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index 936119b1f9..d4a7280fa1 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -896,7 +896,6 @@ void write_commit_graph(const char *obj_dir, close_reachable(&oids, report_progress); QSORT(oids.list, oids.nr, commit_compare); - count_distinct = 1; for (i = 1; i < oids.nr; i++) { if (!oideq(&oids.list[i - 1], &oids.list[i])) From patchwork Sat Jan 19 20:21:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10772153 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22EBC14E5 for ; Sat, 19 Jan 2019 20:21:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 124D32BEBA for ; Sat, 19 Jan 2019 20:21:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 063182BEC4; Sat, 19 Jan 2019 20:21:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 908312BEBA for ; Sat, 19 Jan 2019 20:21:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729396AbfASUV5 (ORCPT ); Sat, 19 Jan 2019 15:21:57 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:39517 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729332AbfASUVy (ORCPT ); Sat, 19 Jan 2019 15:21:54 -0500 Received: by mail-wm1-f67.google.com with SMTP id y8so7434930wmi.4 for ; Sat, 19 Jan 2019 12:21:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RZFJBP3YOkKMxljZXP6hYMQ1ADwUWWww3gQINewC3QE=; b=kx0gIF8scuURoPpGp7x+ZOG4FkaOZEBH8qcq1EX/19fZ2vtmAjYwnLEdHxJm5uSyJR UfBg3IGWnlmLBg9BziG6RMWB0RzmswW7UplBych34c2QYkNSTdnFrBiOaS4Ig1vHoxVM 62RmXQeeo5pdJOlDATmn9FGhm9OI5POYWEUZUOVEzJxCrP+UUwHrOS7Qx3GeishGG/tj ZAaABrXHN3yYm+t0z/FkT8wwkuDyGFpmAIBtg+7kZj+m033bzhWm7ScAI60RaEJAiYTz V1HUzeyaM4rwmWvbZJDOEp2UTxFreOu/JtqrDlEx3qffCTp8b9XOX6q6vVWxJKjAL+hd zhSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RZFJBP3YOkKMxljZXP6hYMQ1ADwUWWww3gQINewC3QE=; b=D8lWNv/7Z4rrCUg4FgJO6qGR8srOtYh/1edoFRQ8PiMwVHIpv5Ud/NcY7ZgrU3e0WF YLTaCY184EggNlLfWqApXmV84XQ3zpx5fU/R0OWlVpdZv4/OisjTxleksnYypOPtSkHq nBu5um7gdKm4eQnk8nQQyv/mgIB1mELr7AFZJQ+eM+2VzwGzOiWkMZt8MVSxQhlyUdk9 Rw26kluabMWpkF43PvrwlNwMC4WelftWoEB8EQpiAmSEm2M3v1OexEPahjj9ScaTUnub GLih7cLiX+1qZriYrLEzHW/G8eRpIUqVYZg4RV/45gJAsKygexcYvMTMQz3OPk9VT8+i P9lw== X-Gm-Message-State: AJcUukdBFJLpFBUjaICWuzRtQA4YLzYNL7+vMxe2QMuKIn0mRHuQX2Lg QeY1EUFtnCVXApxR3z8Cdtl/s3iWUQ4= X-Google-Smtp-Source: ALg8bN7Xfz1Ko3CNAtmGfacQm8n86iZFv/Yguqz6ZUUfOUS4Tn8FS1rIYLIBKM6slK0dhvI+XG5ing== X-Received: by 2002:a1c:9a0d:: with SMTP id c13mr20240119wme.41.1547929312463; Sat, 19 Jan 2019 12:21:52 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id g188sm56196229wmf.32.2019.01.19.12.21.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Jan 2019 12:21:51 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 09/10] commit-graph write: add itermediate progress Date: Sat, 19 Jan 2019 21:21:20 +0100 Message-Id: <20190119202121.3590-10-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20190116132947.26025-1-avarab@gmail.com> References: <20190116132947.26025-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add progress output to sections of code between "Annotating[...]" and "Computing[...]generation numbers". This can collectively take 5-10 seconds on a large enough repository. On a test repository with I have with ~7 million commits and ~50 million objects we'll now emit: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (124763727/124763727), done. Loading known commits in commit graph: 100% (18989461/18989461), done. Expanding reachable commits in commit graph: 100% (18989507/18989461), done. Clearing commit marks in commit graph: 100% (18989507/18989507), done. Counting distinct commits in commit graph: 100% (18989507/18989507), done. Finding extra edges in commit graph: 100% (18989507/18989507), done. Computing commit graph generation numbers: 100% (7250302/7250302), done. Writing out commit graph in 4 passes: 100% (29001208/29001208), done. Whereas on a medium-sized repository such as linux.git these new progress bars won't have time to kick in and as before and we'll still emit output like: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (6529159/6529159), done. Expanding reachable commits in commit graph: 815990, done. Computing commit graph generation numbers: 100% (815983/815983), done. Writing out commit graph in 4 passes: 100% (3263932/3263932), done. The "Counting distinct commits in commit graph" phase will spend most of its time paused at "0/*" as we QSORT(...) the list. That's not optimal, but at least we don't seem to be stalling anymore most of the time. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/commit-graph.c b/commit-graph.c index d4a7280fa1..889cdefc49 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -895,12 +895,19 @@ void write_commit_graph(const char *obj_dir, close_reachable(&oids, report_progress); + if (report_progress) + progress = start_delayed_progress( + _("Counting distinct commits in commit graph"), + oids.nr); + display_progress(progress, 0); /* TODO: Measure QSORT() progress */ QSORT(oids.list, oids.nr, commit_compare); count_distinct = 1; for (i = 1; i < oids.nr; i++) { + display_progress(progress, i + 1); if (!oideq(&oids.list[i - 1], &oids.list[i])) count_distinct++; } + stop_progress(&progress); if (count_distinct >= GRAPH_PARENT_MISSING) die(_("the commit graph format cannot write %d commits"), count_distinct); @@ -910,8 +917,13 @@ void write_commit_graph(const char *obj_dir, ALLOC_ARRAY(commits.list, commits.alloc); num_extra_edges = 0; + if (report_progress) + progress = start_delayed_progress( + _("Finding extra edges in commit graph"), + oids.nr); for (i = 0; i < oids.nr; i++) { int num_parents = 0; + display_progress(progress, i + 1); if (i > 0 && oideq(&oids.list[i - 1], &oids.list[i])) continue; @@ -928,6 +940,7 @@ void write_commit_graph(const char *obj_dir, commits.nr++; } num_chunks = num_extra_edges ? 4 : 3; + stop_progress(&progress); if (commits.nr >= GRAPH_PARENT_MISSING) die(_("too many commits to write graph")); From patchwork Sat Jan 19 20:21:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10772155 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0D25E1390 for ; Sat, 19 Jan 2019 20:22:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F13B02BEBA for ; Sat, 19 Jan 2019 20:21:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E58AD2BEC4; Sat, 19 Jan 2019 20:21:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 77AE82BEBA for ; Sat, 19 Jan 2019 20:21:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729380AbfASUV5 (ORCPT ); Sat, 19 Jan 2019 15:21:57 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:33002 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729374AbfASUV4 (ORCPT ); Sat, 19 Jan 2019 15:21:56 -0500 Received: by mail-wm1-f66.google.com with SMTP id r24so3362602wmh.0 for ; Sat, 19 Jan 2019 12:21:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rm9+woAdDa4oRH3tO6XwiozqZ01ZHYXmDWRkXBwcn2k=; b=EtYLvhvODOuhVVNmh1NiKDiiQl+reVB1cIRD9OiMTRZ9Fr8cjHqONCc15uZehS7FNO Sj6h8vpOxb7Xb4lrM27bQJiLeJ/xRFMlWtSXYXP1PSKcKpCph7WyiOKgVsJz64xftiB/ G0NzdEtMYCf2f3uE1DX9Pjoj6GOPR3IIEjU3x3sDUMAXtlmx6pBxBCG4kb9IFqHmTAzF Ha+O+kZq7zUdxWFznpT1OKzb5TTqznDDIjchqw9aiTn/VPpToJTDsYNg+n0pVmh9FNDo 3JvrQtibuiSeIEcRLHKL7gdou38VZ6IPiBvI+XP72Y/tNLWbRVMWACU0H7kjcrH4U/MN VSxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rm9+woAdDa4oRH3tO6XwiozqZ01ZHYXmDWRkXBwcn2k=; b=DqhSPZBVTX5YynPmJcaTuWR+5sKSHkk+EOIq9hJsL/6CdSxeMUlqwYwZemsCp6LZgG +IEmTyWpRHYepxjrkw+7+exuRXUyDxuCgJvyfRDLtrQwzapkB7Bio+RxIWNdqzQakqvI 4Dp01fRuttvcvxox+Ah1VjWe+bYsbfDijCUmx5cx1cPAOoVp1if05/aal4Ze+8Nbtu1t LOjvh6iKW8kgkQg0+B8CHJc0Ja9DAniLZx6s7WhLnc7+40MqqTsJzJbExxgFtfe/alA9 utz8W0jXdAqRbcNOIwYFe5xX2Ra0abMdXjJ0ap4THPPa2ZsdWgluKMVatJOZSJpEeYF/ 3KsA== X-Gm-Message-State: AJcUukfvcB3QFx+Bg+YUoDspX90jKqIJZA9fJPOaX7g15JuFHhEBqH74 0PLSFiyGFXZmBTGvxr/RXj5FkffEIng= X-Google-Smtp-Source: ALg8bN6FL5TYkU2joKDqDkhWH+Rrx2FojOFiFjEiYXeIcRx7v37B2z11T4kWU9nl2nWl0FjOniJZAQ== X-Received: by 2002:a1c:de57:: with SMTP id v84mr19476448wmg.55.1547929313803; Sat, 19 Jan 2019 12:21:53 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id g188sm56196229wmf.32.2019.01.19.12.21.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Jan 2019 12:21:53 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v6 10/10] commit-graph write: emit a percentage for all progress Date: Sat, 19 Jan 2019 21:21:21 +0100 Message-Id: <20190119202121.3590-11-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20190116132947.26025-1-avarab@gmail.com> References: <20190116132947.26025-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Follow-up 01ca387774 ("commit-graph: split up close_reachable() progress output", 2018-11-19) by making the progress bars in close_reachable() report a completion percentage. This fixes the last occurrence where in the commit graph writing where we didn't report that. The change in 01ca387774 split up the 1x progress bar in close_reachable() into 3x, but left them as dumb counters without a percentage completion. Fixing that is easy, and the only reason it wasn't done already is because that commit was rushed in during the v2.20.0 RC period to fix the unrelated issue of over-reporting commit numbers. See [1] and follow-ups for ML activity at the time and [2] for an alternative approach where the progress bars weren't split up. Now for e.g. linux.git we'll emit: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (6529159/6529159), done. Expanding reachable commits in commit graph: 100% (815990/815980), done. Computing commit graph generation numbers: 100% (815983/815983), done. Writing out commit graph in 4 passes: 100% (3263932/3263932), done. 1. https://public-inbox.org/git/20181119202300.18670-1-avarab@gmail.com/ 2. https://public-inbox.org/git/20181122153922.16912-11-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 889cdefc49..017225ccea 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -653,15 +653,15 @@ static void add_missing_parents(struct packed_oid_list *oids, struct commit *com static void close_reachable(struct packed_oid_list *oids, int report_progress) { - int i, j; + int i; struct commit *commit; struct progress *progress = NULL; if (report_progress) progress = start_delayed_progress( - _("Loading known commits in commit graph"), j = 0); + _("Loading known commits in commit graph"), oids->nr); for (i = 0; i < oids->nr; i++) { - display_progress(progress, ++j); + display_progress(progress, i + 1); commit = lookup_commit(the_repository, &oids->list[i]); if (commit) commit->object.flags |= UNINTERESTING; @@ -675,9 +675,9 @@ static void close_reachable(struct packed_oid_list *oids, int report_progress) */ if (report_progress) progress = start_delayed_progress( - _("Expanding reachable commits in commit graph"), j = 0); + _("Expanding reachable commits in commit graph"), oids->nr); for (i = 0; i < oids->nr; i++) { - display_progress(progress, ++j); + display_progress(progress, i + 1); commit = lookup_commit(the_repository, &oids->list[i]); if (commit && !parse_commit(commit)) @@ -687,9 +687,9 @@ static void close_reachable(struct packed_oid_list *oids, int report_progress) if (report_progress) progress = start_delayed_progress( - _("Clearing commit marks in commit graph"), j = 0); + _("Clearing commit marks in commit graph"), oids->nr); for (i = 0; i < oids->nr; i++) { - display_progress(progress, ++j); + display_progress(progress, i + 1); commit = lookup_commit(the_repository, &oids->list[i]); if (commit)