From patchwork Thu Nov 22 15:39:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10694419 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EBCE114DE for ; Thu, 22 Nov 2018 15:39:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB4642CEBB for ; Thu, 22 Nov 2018 15:39:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CF47D2CECA; Thu, 22 Nov 2018 15:39:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 640432CEBB for ; Thu, 22 Nov 2018 15:39:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437781AbeKWCTa (ORCPT ); Thu, 22 Nov 2018 21:19:30 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:38017 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391765AbeKWCT3 (ORCPT ); Thu, 22 Nov 2018 21:19:29 -0500 Received: by mail-wm1-f67.google.com with SMTP id k198so9560958wmd.3 for ; Thu, 22 Nov 2018 07:39:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AMsBOknSbMIu7WHP3pCkMe9SwoVnD2lqpFACmHRvDWc=; b=jLczhgzU84LBwYWE2px8TlsCf54y8aPD9c7xMwTv0VRLuzOkTJinpCataXYx88UqpO TBqoAtemnyPCDJA70lQo2kuwCCOVvucIk65dWH+gUWX0N9diyxSF/90fIUJIZDjGhLKN QZ4xjvYxhSGosE9s2qxnqwh7I3sI4MM5lnElm3d+0ngu4irD8vDU0yrNnPVBYzTdQNtz 1b+eZ5rdIC7T0fruY0290dUH37ukdx+cUxBWg0DHHQzvpROpoftq72v6UcavP+eudsM4 vIpVa5Xb7EOX8w+zZknIFpKuNlazO2GCK4BAZPUERwqDtA+OvxQK4H7klRCoIouaiLrk RXUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AMsBOknSbMIu7WHP3pCkMe9SwoVnD2lqpFACmHRvDWc=; b=ktq58HkbD6ozTnDEmxjrHxC/xdzpttBZthW9sonp8V9FVbFXBIG3lSBCaQbNFaqlwe xBDokg/HE/ZyY8Ez0C71ekRqkYWaiCLGfstxLfuSQxKPuvNk1iIayK5z2VO6CcnNVfQl xAQ9GUj8DRiynVcFKx7ELORvZro2vZx77cDhKf/W9HM6pERbSwcLhw7Am4S8AGxgzTdD 53xIm0izLo1MrXEg5OIgUzRAgZq3228mXZawo20VDvf7EdiqyrTfmco1w+w2aXFqy8GG t3FoFU6XCwAjVkw+i1IukrDzERk3w4gPTdvFkWm1YBhJYPxEyxvUpoWBuWI6f6M2Jgua YvLA== X-Gm-Message-State: AA+aEWZtLUz0RevamvCNuwlJE2rstrCw76kMz64TomsuUYRflaHf85uU fLdisaBY9DKqRJEb4vNSmXTtGrXE35w= X-Google-Smtp-Source: AJdET5dgB8Ng1bSsSxPda1DqATPcvDnYIqw61CfYws7tf88A3RNVVTvDPFDbwxEhS7MLdCUxeNzhmQ== X-Received: by 2002:a1c:7e8e:: with SMTP id z136mr10830246wmc.140.1542901175076; Thu, 22 Nov 2018 07:39:35 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id a17sm7329389wrs.58.2018.11.22.07.39.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 07:39:34 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v4 01/10] commit-graph: rename 'num_extra_edges' variable to 'num_large_edges' Date: Thu, 22 Nov 2018 15:39:13 +0000 Message-Id: <20181122153922.16912-2-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181122132823.9883-1-avarab@gmail.com> References: <20181122132823.9883-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: SZEDER Gábor The commit graph file format describes an optional 'Large Edge List' chunk, and the function writing out this chunk is called write_graph_chunk_large_edges(). Then there are two functions in 'commit-graph.c', namely write_graph_chunk_data() and write_commit_graph(), which have a local variable called 'num_extra_edges'. It can be confusing on first sight whether large edges and extra edges refer to the same thing or not, but they do, so let's rename those variables to 'num_large_edges'. Signed-off-by: SZEDER Gábor Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 40c855f185..7b4e3a02cf 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -475,7 +475,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, { struct commit **list = commits; struct commit **last = commits + nr_commits; - uint32_t num_extra_edges = 0; + uint32_t num_large_edges = 0; while (list < last) { struct commit_list *parent; @@ -507,7 +507,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, if (!parent) edge_value = GRAPH_PARENT_NONE; else if (parent->next) - edge_value = GRAPH_OCTOPUS_EDGES_NEEDED | num_extra_edges; + edge_value = GRAPH_OCTOPUS_EDGES_NEEDED | num_large_edges; else { edge_value = sha1_pos(parent->item->object.oid.hash, commits, @@ -521,7 +521,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, if (edge_value & GRAPH_OCTOPUS_EDGES_NEEDED) { do { - num_extra_edges++; + num_large_edges++; parent = parent->next; } while (parent); } @@ -761,7 +761,7 @@ void write_commit_graph(const char *obj_dir, uint32_t chunk_ids[5]; uint64_t chunk_offsets[5]; int num_chunks; - int num_extra_edges; + int num_large_edges; struct commit_list *parent; struct progress *progress = NULL; @@ -871,7 +871,7 @@ void write_commit_graph(const char *obj_dir, commits.alloc = count_distinct; ALLOC_ARRAY(commits.list, commits.alloc); - num_extra_edges = 0; + num_large_edges = 0; for (i = 0; i < oids.nr; i++) { int num_parents = 0; if (i > 0 && oideq(&oids.list[i - 1], &oids.list[i])) @@ -885,11 +885,11 @@ void write_commit_graph(const char *obj_dir, num_parents++; if (num_parents > 2) - num_extra_edges += num_parents - 1; + num_large_edges += num_parents - 1; commits.nr++; } - num_chunks = num_extra_edges ? 4 : 3; + num_chunks = num_large_edges ? 4 : 3; if (commits.nr >= GRAPH_PARENT_MISSING) die(_("too many commits to write graph")); @@ -916,7 +916,7 @@ void write_commit_graph(const char *obj_dir, chunk_ids[0] = GRAPH_CHUNKID_OIDFANOUT; chunk_ids[1] = GRAPH_CHUNKID_OIDLOOKUP; chunk_ids[2] = GRAPH_CHUNKID_DATA; - if (num_extra_edges) + if (num_large_edges) chunk_ids[3] = GRAPH_CHUNKID_LARGEEDGES; else chunk_ids[3] = 0; @@ -926,7 +926,7 @@ void write_commit_graph(const char *obj_dir, chunk_offsets[1] = chunk_offsets[0] + GRAPH_FANOUT_SIZE; chunk_offsets[2] = chunk_offsets[1] + GRAPH_OID_LEN * commits.nr; chunk_offsets[3] = chunk_offsets[2] + (GRAPH_OID_LEN + 16) * commits.nr; - chunk_offsets[4] = chunk_offsets[3] + 4 * num_extra_edges; + chunk_offsets[4] = chunk_offsets[3] + 4 * num_large_edges; for (i = 0; i <= num_chunks; i++) { uint32_t chunk_write[3]; From patchwork Thu Nov 22 15:39:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10694421 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 995EE1734 for ; Thu, 22 Nov 2018 15:39:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 878242CEBB for ; Thu, 22 Nov 2018 15:39:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7BEE72CECA; Thu, 22 Nov 2018 15:39:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1F4F12CEBB for ; Thu, 22 Nov 2018 15:39:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437784AbeKWCTc (ORCPT ); Thu, 22 Nov 2018 21:19:32 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:35254 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731999AbeKWCTb (ORCPT ); Thu, 22 Nov 2018 21:19:31 -0500 Received: by mail-wr1-f65.google.com with SMTP id 96so9704922wrb.2 for ; Thu, 22 Nov 2018 07:39:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HZMbiccRM6UYVJfNDn+I7hL2sDEeAx0TWmuHfPrTITo=; b=Mj4Va2wUPBn72vzBg7kGFh+AxoVYpRhnqAKCKO/7Sf+/nyre1QoYXojSNw2j5DK19V TZ1fA7NH+MgQAiNhQC9zR0xJJo5Gub2k1X4HnrMsdVu641lZpZ9CgxE7uPtek0P0vzUv /F8AQLPl1Olmhj5ZRxUebDVZPbC92SaboNc32dS/C2gIVucFRRwKEj1IWSqY1DRrTVbd qcxC6e7Mls5C3B+/CvcTSRUXSyiDpI/0iT7jM1ZBzwCJybIWyO1pJuV3yIPWdBit3oVY ef4tbGoRGhvydFD+Yjzp9aS10dsn67o3OEEmU2qDeHoiRDTZkgqw/4b6n4eXVaf42+i0 yiVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HZMbiccRM6UYVJfNDn+I7hL2sDEeAx0TWmuHfPrTITo=; b=HZ6xJa7N9EumNtPaHgWbQrT2ixurqRfMAoOIMVrPODpP+ase+Jh51YfpDKh4W+0R+D 81XJMLVU1kcXm6MKNw4MtqFo/JI0gW48KZHb6Yz3O0JXnBCT4H8jMHcC72cF+5Yf/LSJ RO1uirPMUXxUpoNS3nnOsDLqV+8Cr+2NTEwEEFW3+ShoAeMeB0WJdw5/fHK2uN/AMSCh hZunHx8WyFcuoD/hAenYz6weAVaO2QEphrHeZl55aLl+XiR86LZKnyTJUYXxf+t23/oF rxK6z2o1D9vWbRjelVwk9qRFtCupFXLrjP/+lwcXKIdiSm+YTUij6qR7RUPIKqNLCVxt s43w== X-Gm-Message-State: AA+aEWYrU6A5OTguRVOcTwtv+TybMOoC3iRq4V64n87nzodklNOZRnW/ advkqhniQhCNropWITBFTQBg2ofNrOQ= X-Google-Smtp-Source: AFSGD/WQal1FTb+jsfOui529BfT8neuzkagz7cms1w0LWz9/RYTYIe8cBsykVJy9Cjpl3w//VLqWuQ== X-Received: by 2002:adf:e9d1:: with SMTP id l17mr9645837wrn.73.1542901176776; Thu, 22 Nov 2018 07:39:36 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id a17sm7329389wrs.58.2018.11.22.07.39.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 07:39:36 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v4 02/10] commit-graph: don't call write_graph_chunk_large_edges() unnecessarily Date: Thu, 22 Nov 2018 15:39:14 +0000 Message-Id: <20181122153922.16912-3-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181122132823.9883-1-avarab@gmail.com> References: <20181122132823.9883-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: SZEDER Gábor The optional 'Large Edge List' chunk of the commit graph file stores parent information for commits with more than two parents. Since the chunk is optional, write_commit_graph() looks through all commits to find those with more than two parents, and then writes the commit graph file header accordingly, i.e. if there are no such commits, then there won't be a 'Large Edge List' chunk written, only the three mandatory chunks. However, when it comes to writing chunk data, write_commit_graph() unconditionally invokes write_graph_chunk_large_edges(), even when it was decided earlier that that chunk won't be written. Strictly speaking there is no bug here, because write_graph_chunk_large_edges() won't write anything because it won't find any commits with more than two parents, but then it unnecessarily and in vain looks through all commits once again in search for such commits. Don't call write_graph_chunk_large_edges() when that chunk won't be written to spare an unnecessary iteration over all commits. Signed-off-by: SZEDER Gábor Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index 7b4e3a02cf..965eb23a7b 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -940,7 +940,8 @@ void write_commit_graph(const char *obj_dir, write_graph_chunk_fanout(f, commits.list, commits.nr); write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr); write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr); - write_graph_chunk_large_edges(f, commits.list, commits.nr); + if (num_large_edges) + write_graph_chunk_large_edges(f, commits.list, commits.nr); close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); From patchwork Thu Nov 22 15:39:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10694423 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 047A314DE for ; Thu, 22 Nov 2018 15:39:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7F5F2CEBB for ; Thu, 22 Nov 2018 15:39:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DC4092CECA; Thu, 22 Nov 2018 15:39:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 81ED22CEBB for ; Thu, 22 Nov 2018 15:39:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437787AbeKWCTd (ORCPT ); Thu, 22 Nov 2018 21:19:33 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:42425 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391765AbeKWCTc (ORCPT ); Thu, 22 Nov 2018 21:19:32 -0500 Received: by mail-wr1-f67.google.com with SMTP id q18so9666873wrx.9 for ; Thu, 22 Nov 2018 07:39:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sW0q4dxvjWqTu5oModp82GaBkhs4Vx9uSJ36sGQO/bc=; b=KblFC5hsPqt+EuLSopCQWHpcZDTSrJ5mQVnC/RCiwJFW/aDe3AwWtKjacH00WKmGqX /10dpKYlVaL/FXoIO1AgjVRG3cLngHpRtxDKOW4/Sn6hfNyGForuJFzBK8sNGlXFiBSJ Pji2bIQdcU93BebVKYVWou4R8XzXfLpqAdK3Zk/EKdfYQwEgwAX0KKE/mmZZtYN98Km7 2dFx2nl29Six0O83CxyWKWbp/UO8ETXNxcudkbHTKlRo+8LlGq5DPuQz96Z4XpOkfO8C Kvf2uA6tDuxkP6hNVS/nb5xspRsz6uk2g9IJo4XIdp1S+L7vctgQ1yY8UcX5XJ00vLe6 PC8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sW0q4dxvjWqTu5oModp82GaBkhs4Vx9uSJ36sGQO/bc=; b=gf+wArJccgvSQBz80/efSss0u/+EP3L7RCFKgDS+jsVC2CEnNSZeEBmJ4NHuPYhJ0g TxVd2o4emsRQumKWdocu6Rzp9Dlry5IRfVbjSCNh5S34R4PBHo1fhL9M/zLcqYItco9C F1hHyI0bCNf4YAsTXkuUNqV3LhMCtHeKAm/3qKN7IyH/USAN1/yR89DKtGWPI311wbhA 3gXuewPM3BOa3a7N4k0C805+jxfxE56k7fx09JzNlTd1sz0o3khwwm/n5SptokIHjQMR 2CF9QJbMDg33/0RpgRMJH2D6AKIfa4/bSLGM0MucnLR4fVCFAgK1kGseIdfRyzOelggp 5Mow== X-Gm-Message-State: AA+aEWZGTQw+rtiXPbn452NeDYaeYbQyO0nPeOhKDQ5NucfQSVtCCk92 rjHO6D4aAyfPEPYcZIj1ubNZVGeSaBA= X-Google-Smtp-Source: AFSGD/XXyBaMPKdvnci+8WzUpDiB3xkjwRkI20Q34ogc5w7/aJktjDy1qBjW5BiuPjwcaVQc6pPddw== X-Received: by 2002:adf:9b11:: with SMTP id b17mr9886264wrc.168.1542901178368; Thu, 22 Nov 2018 07:39:38 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id a17sm7329389wrs.58.2018.11.22.07.39.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 07:39:37 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v4 03/10] commit-graph write: rephrase confusing progress output Date: Thu, 22 Nov 2018 15:39:15 +0000 Message-Id: <20181122153922.16912-4-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181122132823.9883-1-avarab@gmail.com> References: <20181122132823.9883-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Rephrase the title shown for the progress output emitted by close_reachable(). The message I added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17) gave the impression that it would count up to the number of commit objects. But that's not what the number means. It just represents the work we're doing in several for-loops to do various work before the graph is written out. So let's just say "Annotating commit graph", that title makes no such promises, and we can add other loops here in the future and still consistently show progress output. See [1] for the initial bug report & subsequent discussion about other approaching to solving this. 1. https://public-inbox.org/git/20181015165447.GH19800@szeder.dev/ Reported-by: SZEDER Gábor Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index 965eb23a7b..d11370a2b3 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -648,7 +648,7 @@ static void close_reachable(struct packed_oid_list *oids, int report_progress) if (report_progress) progress = start_delayed_progress( - _("Annotating commits in commit graph"), 0); + _("Annotating commit graph"), 0); for (i = 0; i < oids->nr; i++) { display_progress(progress, ++j); commit = lookup_commit(the_repository, &oids->list[i]); From patchwork Thu Nov 22 15:39:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10694427 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3B2331734 for ; Thu, 22 Nov 2018 15:39:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 29E9F2CEBB for ; Thu, 22 Nov 2018 15:39:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1E02E2CED4; Thu, 22 Nov 2018 15:39:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8A6F52CEBB for ; Thu, 22 Nov 2018 15:39:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437791AbeKWCTg (ORCPT ); Thu, 22 Nov 2018 21:19:36 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:33694 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2437785AbeKWCTe (ORCPT ); Thu, 22 Nov 2018 21:19:34 -0500 Received: by mail-wr1-f65.google.com with SMTP id c14so4053403wrr.0 for ; Thu, 22 Nov 2018 07:39:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MsPsI8bHqBNYFMY/FR3ifeWBR4iK6UnAFxDMi3VsCfU=; b=qpGcJOtepDuy35zXvoFtjLPYrTqdvJV7Rm3L5ozqmc7OnBRGtnqqcnluG+4sCWA9W4 KN9K49q8WbG2sN7tHnpQsiY/XHjF8rjePbwlj3NOtLpwsBQ+B3hH1H0TvIimIbIr6xs1 5IJ7smZy2JWMV6zwrrvo5VWnxiuVjrk2P7Dib43P2KXTT1rFcq9WfS1Sx4fc3wA0JSKd L8f5EjJFdn9/Z4bj0C8zd65b+jM9NFm+7rfVZVY8wFPkYFd/NNZ99Xdp63+o4B481Tj5 Xe+kLI+kLL54EqmmQ0dKkt842+3rZtb1sFX2UvI8NuaXnM+j6/K5SwQLjKXCBBMwovX+ 5JTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MsPsI8bHqBNYFMY/FR3ifeWBR4iK6UnAFxDMi3VsCfU=; b=k8QLAqPduUY6x32kD5QnySqsSP55NCFXVJ2/XRMowhCNpvMgQuTYvpI78Qrj3nWvDH UV4FLcxGxG+EKjCK/J0yuY0N3/LfFm8I7D0ha6ZWpSdJxegfj9waT7k3NWzjpGJIElIH CzhQtDK7SHJ3OGuRWaLXmmIVuPT+XiD4OU4GYs5RHUgT2IARywvBFSHZtSALDXk6DUa4 rrY20xP7YWkVY/LfZU7obUSO2jmEassl8JlDSnJ4l/ivu+eWJUT/5E2vrMptKbTPIxgq gyXNBubhyp5IF8BCr9UxxyjSDHFfYLK11WeXkP8f97bxlYPaHqj74wvv1T2KgaIHdsUd bxZw== X-Gm-Message-State: AA+aEWbq9R+CQJ3AWoH0FM9NXOMz9kOkP2TNK0JYE6zIHvKEf1y9D05P reNJ9sMisbjIigNzn8ihDfT27twmk8s= X-Google-Smtp-Source: AFSGD/XjGI19Op3JlkTTbcIL5zUsiFJqjLCve083G9r5UjleI07uxF3wO3n7I4KE1sQ9DwZDVG1uJw== X-Received: by 2002:a5d:4046:: with SMTP id w6mr10830208wrp.92.1542901179984; Thu, 22 Nov 2018 07:39:39 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id a17sm7329389wrs.58.2018.11.22.07.39.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 07:39:39 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v4 04/10] commit-graph write: add "Writing out" progress output Date: Thu, 22 Nov 2018 15:39:16 +0000 Message-Id: <20181122153922.16912-5-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181122132823.9883-1-avarab@gmail.com> References: <20181122132823.9883-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add progress output to be shown when we're writing out the commit-graph, this adds to the output already added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17). As noted in that commit most of the progress output isn't displayed on small repositories, but before this change we'd noticeably hang for 2-3 seconds at the end on medium sized repositories such as linux.git. Now we'll instead show output like this, and have no human-observable point at which we're not producing progress output: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph: 6365442, done. Annotating commit graph: 2391666, done. Computing commit graph generation numbers: 100% (797222/797222), done. Writing out commit graph: 100% (3188888/3188888), done. This "Writing out" number is 3x or 4x the number of commits, depending on the graph we're processing. A later change will make this explicit to the user. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 48 +++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 39 insertions(+), 9 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index d11370a2b3..dc57b8fedc 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -433,7 +433,9 @@ struct tree *get_commit_tree_in_graph(struct repository *r, const struct commit static void write_graph_chunk_fanout(struct hashfile *f, struct commit **commits, - int nr_commits) + int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { int i, count = 0; struct commit **list = commits; @@ -447,6 +449,7 @@ static void write_graph_chunk_fanout(struct hashfile *f, while (count < nr_commits) { if ((*list)->object.oid.hash[0] != i) break; + display_progress(progress, ++*progress_cnt); count++; list++; } @@ -456,12 +459,16 @@ static void write_graph_chunk_fanout(struct hashfile *f, } static void write_graph_chunk_oids(struct hashfile *f, int hash_len, - struct commit **commits, int nr_commits) + struct commit **commits, int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; int count; - for (count = 0; count < nr_commits; count++, list++) + for (count = 0; count < nr_commits; count++, list++) { + display_progress(progress, ++*progress_cnt); hashwrite(f, (*list)->object.oid.hash, (int)hash_len); + } } static const unsigned char *commit_to_sha1(size_t index, void *table) @@ -471,7 +478,9 @@ static const unsigned char *commit_to_sha1(size_t index, void *table) } static void write_graph_chunk_data(struct hashfile *f, int hash_len, - struct commit **commits, int nr_commits) + struct commit **commits, int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; struct commit **last = commits + nr_commits; @@ -481,6 +490,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, struct commit_list *parent; int edge_value; uint32_t packedDate[2]; + display_progress(progress, ++*progress_cnt); parse_commit(*list); hashwrite(f, get_commit_tree_oid(*list)->hash, hash_len); @@ -542,7 +552,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, static void write_graph_chunk_large_edges(struct hashfile *f, struct commit **commits, - int nr_commits) + int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; struct commit **last = commits + nr_commits; @@ -550,6 +562,9 @@ static void write_graph_chunk_large_edges(struct hashfile *f, while (list < last) { int num_parents = 0; + + display_progress(progress, ++*progress_cnt); + for (parent = (*list)->parents; num_parents < 3 && parent; parent = parent->next) num_parents++; @@ -764,6 +779,7 @@ void write_commit_graph(const char *obj_dir, int num_large_edges; struct commit_list *parent; struct progress *progress = NULL; + uint64_t progress_cnt = 0; if (!commit_graph_compatible(the_repository)) return; @@ -937,11 +953,25 @@ void write_commit_graph(const char *obj_dir, hashwrite(f, chunk_write, 12); } - write_graph_chunk_fanout(f, commits.list, commits.nr); - write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr); - write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr); + if (report_progress) { + /* + * Each of the write_graph_chunk_*() functions just + * below loops over our N commits. This number must be + * kept in sync with the number of passes we're doing. + */ + int graph_passes = 3; + if (num_large_edges) + graph_passes++; + progress = start_delayed_progress( + _("Writing out commit graph"), + graph_passes * commits.nr); + } + write_graph_chunk_fanout(f, commits.list, commits.nr, progress, &progress_cnt); + write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr, progress, &progress_cnt); + write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr, progress, &progress_cnt); if (num_large_edges) - write_graph_chunk_large_edges(f, commits.list, commits.nr); + write_graph_chunk_large_edges(f, commits.list, commits.nr, progress, &progress_cnt); + stop_progress(&progress); close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); From patchwork Thu Nov 22 15:39:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10694425 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC42014DE for ; Thu, 22 Nov 2018 15:39:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA1482CEBB for ; Thu, 22 Nov 2018 15:39:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CC5F52CECD; Thu, 22 Nov 2018 15:39:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 56B6B2CEBB for ; Thu, 22 Nov 2018 15:39:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437793AbeKWCTg (ORCPT ); Thu, 22 Nov 2018 21:19:36 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:55834 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391765AbeKWCTg (ORCPT ); Thu, 22 Nov 2018 21:19:36 -0500 Received: by mail-wm1-f68.google.com with SMTP id y139so9325013wmc.5 for ; Thu, 22 Nov 2018 07:39:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fZf8tHIfR2VnbHjVyP8iS3nGAFiLxSZPye6DQu92yEE=; b=aOXXfTcm5Pcy1xwR7b9LpPM5BqC2jas/eLNirQ9WIwfCGpPtNu6uQVi54/TpdyJehR B9Vvx5pDtIg550dKpU7yqUm/q5jAatcQLNnEghsRIqeBchqGaX9qkIo8kqo4/rrdcl1W 5vAypIXg449S8CSPfWF7fyBklWDQAhFQ13oYVl+Ypo1lch+Dxm40WZTRiuekx55gGPR2 A3lSb5UMv3SoQN2hBfCORcxgJNigi6jXzVdK7hVq0dtR6rPBN4noHPNEVTZZrXuOogPS wObT1ARTxrhM1MZjaqwU1Krkwi5Nr84NUdkGiSJG77IHOYbq0JdfXFZIFtdwXTHQl2Ak HaHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fZf8tHIfR2VnbHjVyP8iS3nGAFiLxSZPye6DQu92yEE=; b=HVTVr9UEzJZUbCXeUo2JYSdWiaNPh5m2OHaK0IoIqq+pwRzEHP6MtwL5sTzR01zGFR r725WSGVSK0y0VHPGphp3asB93R7zETuwtYIxeT17xHg6JcfUlCiH0bEOYn48LHg8HCU ejYDi41NWfNkIRtYZOxdijRWV2h5ikBYR/TkmU5w9zCjwbF1ojeuQpEJfAj5SzrDhsth fjkd3gHLj411AvxufT5BgM53oSidsDilvt/h/Ped9zHYuW9Hldmm2wm2UX0ACTZCXckW qOmPiQ5/TktUfHge0odbsy4Kp0cbZPfk0a84fTellD+uXsRzxJ1HRRXeErWKkjvnr08n YQsw== X-Gm-Message-State: AA+aEWYPB3xHQuNmAeqKt6AcU2djTKIRAeTIvAuC92I+ahRiDHx1nOU8 SW22F1jS3dFcJ7sNQJcwe/KHNRY02YE= X-Google-Smtp-Source: AFSGD/UixAbK2LhqNHxg3N6nlYfmjkOM9rB9GWKsvHrnlmZmlr+Ha+brKvfA7A2ECy6T0PntDSVpXA== X-Received: by 2002:a1c:adc9:: with SMTP id w192-v6mr10412224wme.103.1542901181531; Thu, 22 Nov 2018 07:39:41 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id a17sm7329389wrs.58.2018.11.22.07.39.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 07:39:40 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v4 05/10] commit-graph write: more descriptive "writing out" output Date: Thu, 22 Nov 2018 15:39:17 +0000 Message-Id: <20181122153922.16912-6-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181122132823.9883-1-avarab@gmail.com> References: <20181122132823.9883-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make the "Writing out" part of the progress output more descriptive. Depending on the shape of the graph we either make 3 or 4 passes over it. Let's present this information to the user in case they're wondering what this number, which is much larger than their number of commits, has to do with writing out the commit graph. Now e.g. on linux.git we emit: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph: 6365442, done. Annotating commit graph: 2391666, done. Computing commit graph generation numbers: 100% (797222/797222), done. Writing out commit graph in 4 passes: 100% (3188888/3188888), done. A note on i18n: Why are we using the Q_() function and passing a number & English text for a singular which'll never be used? Because the plural rules of translated languages may not match those of English, and to use the plural function we need to use this format. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index dc57b8fedc..3de65bc2e9 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -780,6 +780,7 @@ void write_commit_graph(const char *obj_dir, struct commit_list *parent; struct progress *progress = NULL; uint64_t progress_cnt = 0; + struct strbuf progress_title = STRBUF_INIT; if (!commit_graph_compatible(the_repository)) return; @@ -962,8 +963,13 @@ void write_commit_graph(const char *obj_dir, int graph_passes = 3; if (num_large_edges) graph_passes++; + strbuf_addf(&progress_title, + Q_("Writing out commit graph in %d pass", + "Writing out commit graph in %d passes", + graph_passes), + graph_passes); progress = start_delayed_progress( - _("Writing out commit graph"), + progress_title.buf, graph_passes * commits.nr); } write_graph_chunk_fanout(f, commits.list, commits.nr, progress, &progress_cnt); @@ -973,6 +979,8 @@ void write_commit_graph(const char *obj_dir, write_graph_chunk_large_edges(f, commits.list, commits.nr, progress, &progress_cnt); stop_progress(&progress); + strbuf_release(&progress_title); + close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); commit_lock_file(&lk); From patchwork Thu Nov 22 15:39:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10694429 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 84C251926 for ; Thu, 22 Nov 2018 15:39:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 72C172CEBB for ; Thu, 22 Nov 2018 15:39:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 66ED22CED2; Thu, 22 Nov 2018 15:39:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CD0302CECD for ; Thu, 22 Nov 2018 15:39:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437801AbeKWCTi (ORCPT ); Thu, 22 Nov 2018 21:19:38 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:33698 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2437788AbeKWCTh (ORCPT ); Thu, 22 Nov 2018 21:19:37 -0500 Received: by mail-wr1-f65.google.com with SMTP id c14so4053534wrr.0 for ; Thu, 22 Nov 2018 07:39:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Fd3KrzfrUOAKSqPW9KqzCSydlac5ROxV0toEQxp/SmA=; b=q6qiHWZ2K896pyNbmML3CCLB1nrcR65SaOwKurJxGZGd+GwJe1dLD5RXHL40olCWPm WEFNoyiiLCsjK3H+wsVom0cIg61XRu5dtgSM/uek2jBC2UJmEsi21AnxwVvEQ/4OkHID SgvulDUzL4fsGny351ydaaV7WDeUfDA0dZ3/wlW+cMbcG8E7L/p42FE17l9e0qAZmI+z ku75cPK6B8tv3R/DwwNK4wmDVVyFW4JgzBmY8VAWHfrLNNKI1XEI1OiiAEklqPom+7B8 pgNGrIxifk2dXCjUNJxdA+MS6aduP+KBMLBWTO+tyDvHweQSB9eSyclDAkk/1uR3P1Vf CyqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Fd3KrzfrUOAKSqPW9KqzCSydlac5ROxV0toEQxp/SmA=; b=rioOYs+LVm0HPFz04gNb4FQX7uJWHoEcxNKLtY9nOP1ZSFlOKt5YQrLrccSrAV3DYF +6Hcx1n5LeuCarom66ugFe4DCMwDOg96Ue67w1kysn7CkUzRTxCVJc8vpCjBg8UxqY1Z Bd72cucKJSHua/+2/al30dGbjhTc14l8AUoOxKXIwWpYmg1Oc+9+QUnfzZKntmcKClWP IOarZjfLBITXmOMJ6nS0vITUfBxzO//Qp6AFa9/GCVCecIg9vyizEmK8iBSMGraYkP4G ud/XuSAI4q6r35X1YkV11cXv1Sqzr9jWsCOS7Mn/17FxxHdBmLcrMK0VeHn0WHV44T6T lRVQ== X-Gm-Message-State: AA+aEWbWaVoEG/dtvWsQInoOZ0KTgsWf2yU1MTHpyesxrubQvfkUG699 l/3+9lxxTC0PE5AVxR5CbBE1d4EMH7w= X-Google-Smtp-Source: AFSGD/Wq7mkuk247VJ/ZOVPR2Aj/3AX6P/17BaUy+LjAzJORegCfXevWnCh0PAkF4g48h7qo/UT8Eg== X-Received: by 2002:adf:f550:: with SMTP id j16mr10003869wrp.258.1542901183014; Thu, 22 Nov 2018 07:39:43 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id a17sm7329389wrs.58.2018.11.22.07.39.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 07:39:42 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v4 06/10] commit-graph write: show progress for object search Date: Thu, 22 Nov 2018 15:39:18 +0000 Message-Id: <20181122153922.16912-7-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181122132823.9883-1-avarab@gmail.com> References: <20181122132823.9883-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Show the percentage progress for the "Finding commits for commit graph" phase for the common case where we're operating on all packs in the repository, as "commit-graph write" or "gc" will do. Before we'd emit on e.g. linux.git with "commit-graph write": Finding commits for commit graph: 6365442, done. [...] And now: Finding commits for commit graph: 100% (6365442/6365442), done. [...] Since the commit graph only includes those commits that are packed (via for_each_packed_object(...)) the approximate_object_count() returns the actual number of objects we're going to process. Still, it is possible due to a race with "gc" or another process maintaining packs that the number of objects we're going to process is lower than what approximate_object_count() reported. In that case we don't want to stop the progress bar short of 100%. So let's make sure it snaps to 100% at the end. The inverse case is also possible and more likely. I.e. that a new pack has been added between approximate_object_count() and for_each_packed_object(). In that case the percentage will go beyond 100%, and we'll do nothing to snap it back to 100% at the end. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 3de65bc2e9..42d8365f0d 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -781,12 +781,14 @@ void write_commit_graph(const char *obj_dir, struct progress *progress = NULL; uint64_t progress_cnt = 0; struct strbuf progress_title = STRBUF_INIT; + unsigned long approx_nr_objects; if (!commit_graph_compatible(the_repository)) return; oids.nr = 0; - oids.alloc = approximate_object_count() / 32; + approx_nr_objects = approximate_object_count(); + oids.alloc = approx_nr_objects / 32; oids.progress = NULL; oids.progress_done = 0; @@ -866,8 +868,11 @@ void write_commit_graph(const char *obj_dir, if (!pack_indexes && !commit_hex) { if (report_progress) oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), 0); + _("Finding commits for commit graph"), + approx_nr_objects); for_each_packed_object(add_packed_commits, &oids, 0); + if (oids.progress_done < approx_nr_objects) + display_progress(oids.progress, approx_nr_objects); stop_progress(&oids.progress); } From patchwork Thu Nov 22 15:39:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10694431 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D83BC14DE for ; Thu, 22 Nov 2018 15:39:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C772F2CEBB for ; Thu, 22 Nov 2018 15:39:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BC3C72CED4; Thu, 22 Nov 2018 15:39:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 54BF22CEBB for ; Thu, 22 Nov 2018 15:39:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437804AbeKWCTk (ORCPT ); Thu, 22 Nov 2018 21:19:40 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:38336 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391765AbeKWCTj (ORCPT ); Thu, 22 Nov 2018 21:19:39 -0500 Received: by mail-wr1-f65.google.com with SMTP id v13so6211859wrw.5 for ; Thu, 22 Nov 2018 07:39:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZYMxWgScE7YnkHE220fRPaP6PpCZGeCHVYzI+5d38D4=; b=qzb4TajYgU8zwDMduqs1EVzlI0Au7HsMyHyvw7mIUjqAeRI+9OQ5nWb0mv/i4VjPPd JgH/tOt5H1iKuOzK+PCmeokvyyYwqXWiSF74F3o+/jLaqRC4fVkDfhs2TyjrUiJv0HvC 9mbOwF0xyWK/V6+XNoOC5/CKBiFUBd3qesVDbMZwTo0bXhIzeKndpwC3d8G0RBo7RmSc f3t66ns1zncI6nIROAnkMbjphcLABMd5R7QFiYPGiNbbXHs7rtAKtZewbrvpPyLb2Jkt QDd/Y7ZUcS19c80mWp4+kON/Wj6OEPMOSyhQgbfmfbsZGwr0wt8GyeFA1SH/kTLLAvHn w59w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZYMxWgScE7YnkHE220fRPaP6PpCZGeCHVYzI+5d38D4=; b=ukCFE2Enay8kNDVNg+ceYzOP9Dl6kDViMBhgVVeMaDeB/Vo6XPj8MFGjJw1EfjQPLO qKOXaS9xZISGkA2ylYxDcP7xP+v1djYkVXVynNmGWZ4zSyH8qHO6jrjxTQVKM6F6+wu5 IOUchQl9glIg8Jq+P5/4ZjrDUUY/GG5kz0qm/F872LRir7wWiG+zmex16P/loUOnXXgW 6myNr5dhCqcBdsAqjQUvRL0dnObu1NP8Y+KHrkr6qgH8sdb8JLowD4DfDF6feszNiLhd QdwjVJ8EY8r4Fgneq1ISDWthd656moe2FGR16n3rZ7UNjdvsbW92jtk83P/l0VsAwcOr 47UA== X-Gm-Message-State: AA+aEWarz6HmUl383STaF4Yk59exACtaYIr9Xpnpf+EAATPTcyJJBhRn sECAiNhiLasqSK66oRhmbWpDm1vV4yc= X-Google-Smtp-Source: AFSGD/WiwA6VbsDYlUCWp2vXRv7R6J2W1DGp1vf+lVGhqCNJ8529LR9QCutgt4TBQ/U+d5VEuCMjTQ== X-Received: by 2002:adf:9071:: with SMTP id h104-v6mr10074656wrh.65.1542901184738; Thu, 22 Nov 2018 07:39:44 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id a17sm7329389wrs.58.2018.11.22.07.39.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 07:39:43 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v4 07/10] commit-graph write: add more descriptive progress output Date: Thu, 22 Nov 2018 15:39:19 +0000 Message-Id: <20181122153922.16912-8-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181122132823.9883-1-avarab@gmail.com> References: <20181122132823.9883-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make the progress output shown when we're searching for commits to include in the graph more descriptive. This amends code I added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17). Now, on linux.git, we'll emit this sort of output in the various modes we support: $ git commit-graph write Finding commits for commit graph among packed objects: 100% (6365442/6365442), done. [...] # Actually we don't emit this since this takes almost no time at # all. But if we did (s/_delayed//) we'd show: $ git for-each-ref --format='%(objectname)' | git commit-graph write --stdin-commits Finding commits for commit graph from 584 refs: 100% (584/584), done. [...] $ (cd .git/objects/pack/ && ls *idx) | git commit-graph write --stdin-pack Finding commits for commit graph in 2 packs: 6365442, done. [...] The middle on of those is going to be the output users might see in practice, since it'll be emitted when they get the commit graph via gc.writeCommitGraph=true. But as noted above you need a really large number of refs for this message to show. It'll show up on a test repository I have with ~165k refs: Finding commits for commit graph from 165203 refs: 100% (165203/165203), done. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 42d8365f0d..43b15785f6 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -818,8 +818,12 @@ void write_commit_graph(const char *obj_dir, strbuf_addf(&packname, "%s/pack/", obj_dir); dirlen = packname.len; if (report_progress) { - oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), 0); + strbuf_addf(&progress_title, + Q_("Finding commits for commit graph in %d pack", + "Finding commits for commit graph in %d packs", + pack_indexes->nr), + pack_indexes->nr); + oids.progress = start_delayed_progress(progress_title.buf, 0); oids.progress_done = 0; } for (i = 0; i < pack_indexes->nr; i++) { @@ -836,14 +840,20 @@ void write_commit_graph(const char *obj_dir, free(p); } stop_progress(&oids.progress); + strbuf_reset(&progress_title); strbuf_release(&packname); } if (commit_hex) { - if (report_progress) - progress = start_delayed_progress( - _("Finding commits for commit graph"), - commit_hex->nr); + if (report_progress) { + strbuf_addf(&progress_title, + Q_("Finding commits for commit graph from %d ref", + "Finding commits for commit graph from %d refs", + commit_hex->nr), + commit_hex->nr); + progress = start_delayed_progress(progress_title.buf, + commit_hex->nr); + } for (i = 0; i < commit_hex->nr; i++) { const char *end; struct object_id oid; @@ -863,12 +873,13 @@ void write_commit_graph(const char *obj_dir, } } stop_progress(&progress); + strbuf_reset(&progress_title); } if (!pack_indexes && !commit_hex) { if (report_progress) oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), + _("Finding commits for commit graph among packed objects"), approx_nr_objects); for_each_packed_object(add_packed_commits, &oids, 0); if (oids.progress_done < approx_nr_objects) From patchwork Thu Nov 22 15:39:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10694433 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0DD8C1926 for ; Thu, 22 Nov 2018 15:39:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EFBDE2CEBB for ; Thu, 22 Nov 2018 15:39:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E3A732CED2; Thu, 22 Nov 2018 15:39:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 879882CECD for ; Thu, 22 Nov 2018 15:39:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437807AbeKWCTk (ORCPT ); Thu, 22 Nov 2018 21:19:40 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:34856 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2437788AbeKWCTk (ORCPT ); Thu, 22 Nov 2018 21:19:40 -0500 Received: by mail-wm1-f67.google.com with SMTP id c126so9503449wmh.0 for ; Thu, 22 Nov 2018 07:39:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2WLVC23+Lckx1MtsV6l2XoNybaleRo3b4ttuktAeA+M=; b=V50rdkVz1DN1UR/A1qMJ57+NY7f45603VE3/76VYeY9iZw9ZJ5uenfoI0H/GA/LgOi pFLSCP8YYPdRt5zpeojQwJz/MCA9sG291lPljjpJN6zeSSG6+rEqjBDrPd28ESQfzStU kMG3+sjc+B0HUaORYYlXIYBthdpf7QGJrEiYN6fW6TY0ynVpzcReboLd3csj6D2KVtTN aKkqp6Nrq5CGrcUwDCvZ1DtsqGN8OajEli7R2l6/tia1ySziNFSsUrOC5F0YRlTmgFaS umPJO+RSNwueDljy5s2ukyJ+nfpx0LXv/M7QaOq3J7SCedjOvSZk2houlLGDjEXrtPum n2tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2WLVC23+Lckx1MtsV6l2XoNybaleRo3b4ttuktAeA+M=; b=BTE3OYVf/8myMbfbBAO0eEb5+2TC45aQysrUGmfuF9rY1ZlXhObVb05vnjm/T47qWh ophTssT8Ly3IQiYRLXYOSPb/UY5UxML+wFKqnEWnBJtLoBxFbozncmZpQlZLVA01Li48 lOS3ocszbthltWCyq77k+sYvIyyJ0BwXpibO9tJxc811gcrteYjOSk0Nso5qTTOzYC27 oBsV4WpVBy/ZNlZB5QFI9wnFltXqho7bZKhYMgTK8JeLDdIGNTyg/twNNStpfC/Ef3rv Ycbdj750vSQPqzk5It6YVKdNr/sNtdEUAqSA0dkDDp+OyycTwIGJaNBn9aPjgbd0rvfa 39zQ== X-Gm-Message-State: AGRZ1gKuxpOLLYN4u+VVHM0wTC9yZhUAKBo9JP5VlSy2og7V/+qT1X6o D6wb2auO4YnHjW+WuOWFqDroLtAjLQE= X-Google-Smtp-Source: AFSGD/V1OIJwT4Vq2k3AglHIZwKSfvbMTtfUE4dwbbTB6egByiulhLdluCwcl+8n/3rhcRjVq0d5sw== X-Received: by 2002:a1c:a8cf:: with SMTP id r198mr9566838wme.95.1542901186177; Thu, 22 Nov 2018 07:39:46 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id a17sm7329389wrs.58.2018.11.22.07.39.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 07:39:45 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v4 08/10] commit-graph write: remove empty line for readability Date: Thu, 22 Nov 2018 15:39:20 +0000 Message-Id: <20181122153922.16912-9-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181122132823.9883-1-avarab@gmail.com> References: <20181122132823.9883-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Remove the empty line between a QSORT(...) and the subsequent oideq() for-loop. This makes it clearer that the QSORT(...) is being done so that we can run the oideq() loop on adjacent OIDs. Amends code added in 08fd81c9b6 ("commit-graph: implement write_commit_graph()", 2018-04-02). Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 1 - 1 file changed, 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index 43b15785f6..199155bd68 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -890,7 +890,6 @@ void write_commit_graph(const char *obj_dir, close_reachable(&oids, report_progress); QSORT(oids.list, oids.nr, commit_compare); - count_distinct = 1; for (i = 1; i < oids.nr; i++) { if (!oideq(&oids.list[i - 1], &oids.list[i])) From patchwork Thu Nov 22 15:39:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10694435 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 17D291734 for ; Thu, 22 Nov 2018 15:39:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 05DBA2CEBB for ; Thu, 22 Nov 2018 15:39:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EDE932CED2; Thu, 22 Nov 2018 15:39:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 842482CEBB for ; Thu, 22 Nov 2018 15:39:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437810AbeKWCTm (ORCPT ); Thu, 22 Nov 2018 21:19:42 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:44893 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2437805AbeKWCTl (ORCPT ); Thu, 22 Nov 2018 21:19:41 -0500 Received: by mail-wr1-f65.google.com with SMTP id z5so5384048wrt.11 for ; Thu, 22 Nov 2018 07:39:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZRsu+hhayZOTgX5M/9t+10UQVp0RfgvqwjzWEveLhJg=; b=n7PpXEZOzH8BlLzqhRLzo9Ti8Q2p9aIloTLCGDoRRndQ26xzqRr848tfPwpuSC2i20 M4XIMv8i9svRDPsKlhAdB27u298iZjkoelwbxzmSqCaBbquHvhsxt4XbNJLO/gXdEqDu aaEvhJdNcY4a1fg2M1qTTxWE39odGjglQ3L8U5jECDmrucq2pqtg8OY9zgvlo6fbTaV7 0gHRIj+DTkHzMYmr9QtPKGPo01nwLGKtABun2ExlGtlftPeoYxYbfWbp0vL9zazcW3UV /7xKud959x/keBWGWZ6FIs82bvHJlDUxLvNjOeDVCRxOWD6zanyHh8+S24vgIIezTITl QAhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZRsu+hhayZOTgX5M/9t+10UQVp0RfgvqwjzWEveLhJg=; b=W3q9+W0L47R6vbU/LXfgPuNeiSP9J7SXi9vQB1dW+2OYAwiPbxDHxSyR2LoTnEPdXC ln5AvJXB6TSNM5dIg3PfnWDQGjXiIyFdFx+B7E4Rw3NuDi+wr76s6N9WFGzPwrS3SNJa oSn8yVTlBMCizNPKHGQXXQo+mSNNXjKBQ00vdBJkRHkU5K08kO1OqZU4vymIfhPx7/Yp 1P5okdLZOefh3W+UkuxDhmsH/UHvpPSNAQSc2Ot+C4s2TJkAZpxPa5JGAdvs4/oobynM +wunJmphqNt1fiZRMiuTnyjEIQk+h+TWHPY1MsM7p4VvqUUZ2S3jtV/BYqgxr5NkGRiX QSXw== X-Gm-Message-State: AA+aEWYfirV2RJAOniy0ilZo0GGsomFIskY0RNPvX4453A+Tb+CQHVDo VCk9/j6H8k6eNRzJ8BWdul37ogBH3aY= X-Google-Smtp-Source: AFSGD/WYM1zPXIzYd7B3cm3wevocLRBP2h5hkm6CrS6gdIeOLcbwY5cxSMwzSfsIOzIVYL4+UJ2Fhw== X-Received: by 2002:a5d:43d0:: with SMTP id v16mr10805888wrr.67.1542901187588; Thu, 22 Nov 2018 07:39:47 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id a17sm7329389wrs.58.2018.11.22.07.39.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 07:39:46 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v4 09/10] commit-graph write: add itermediate progress Date: Thu, 22 Nov 2018 15:39:21 +0000 Message-Id: <20181122153922.16912-10-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181122132823.9883-1-avarab@gmail.com> References: <20181122132823.9883-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add progress output to sections of code between "Annotating[...]" and "Computing[...]generation numbers". This can collectively take 5-10 seconds on a large enough repository. On a test repository with I have with ~7 million commits and ~50 million objects we'll now emit: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (48333911/48333911), done. Annotating commit graph: 21435984, done. Counting distinct commits in commit graph: 100% (7145328/7145328), done. Finding extra edges in commit graph: 100% (7145328/7145328), done. Computing commit graph generation numbers: 100% (7145328/7145328), done. Writing out commit graph in 4 passes: 100% (28581312/28581312), done. Whereas on a medium-sized repository such as linux.git these new progress bars won't have time to kick in and as before and we'll still emit output like: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (6365442/6365442), done. Annotating commit graph: 2391666, done. Computing commit graph generation numbers: 100% (797222/797222), done. Writing out commit graph in 4 passes: 100% (3188888/3188888), done. The "Counting distinct commits in commit graph" phase will spend most of its time paused at "0/*" as we QSORT(...) the list. That's not optimal, but at least we don't seem to be stalling anymore. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/commit-graph.c b/commit-graph.c index 199155bd68..80f201adf4 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -889,12 +889,19 @@ void write_commit_graph(const char *obj_dir, close_reachable(&oids, report_progress); + if (report_progress) + progress = start_delayed_progress( + _("Counting distinct commits in commit graph"), + oids.nr); + display_progress(progress, 0); /* TODO: Measure QSORT() progress */ QSORT(oids.list, oids.nr, commit_compare); count_distinct = 1; for (i = 1; i < oids.nr; i++) { + display_progress(progress, i + 1); if (!oideq(&oids.list[i - 1], &oids.list[i])) count_distinct++; } + stop_progress(&progress); if (count_distinct >= GRAPH_PARENT_MISSING) die(_("the commit graph format cannot write %d commits"), count_distinct); @@ -904,8 +911,13 @@ void write_commit_graph(const char *obj_dir, ALLOC_ARRAY(commits.list, commits.alloc); num_large_edges = 0; + if (report_progress) + progress = start_delayed_progress( + _("Finding extra edges in commit graph"), + oids.nr); for (i = 0; i < oids.nr; i++) { int num_parents = 0; + display_progress(progress, i + 1); if (i > 0 && oideq(&oids.list[i - 1], &oids.list[i])) continue; @@ -922,6 +934,7 @@ void write_commit_graph(const char *obj_dir, commits.nr++; } num_chunks = num_large_edges ? 4 : 3; + stop_progress(&progress); if (commits.nr >= GRAPH_PARENT_MISSING) die(_("too many commits to write graph")); From patchwork Thu Nov 22 15:39:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10694437 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6504D1734 for ; Thu, 22 Nov 2018 15:39:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 51F452CEC2 for ; Thu, 22 Nov 2018 15:39:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4676C2CED2; Thu, 22 Nov 2018 15:39:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E63562CEC2 for ; Thu, 22 Nov 2018 15:39:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437812AbeKWCTo (ORCPT ); Thu, 22 Nov 2018 21:19:44 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:55847 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2437788AbeKWCTn (ORCPT ); Thu, 22 Nov 2018 21:19:43 -0500 Received: by mail-wm1-f68.google.com with SMTP id y139so9325427wmc.5 for ; Thu, 22 Nov 2018 07:39:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9ZHWkDuRV6sy7Dv4W8CK4OCkYRrSM3Zy5BdK3o811TM=; b=KyFpBM7JD4IBdD5pe8T+AAWlpog8impdcX4QKf9Q3+7Gpd0QyRSg4ZAN6dK942O6FQ RiBLY5y+XbzacOJEmdDrRnRUTVMxUq1cvRkon+QvqyHm0cILFa9NPdg9M2LVp94cITKK K4tbJKTMjNtt4UOYhi9rrOz5dvXTVzFavXoibHTD2ZcbORCCjPpfLSA56v4DeOipF1z9 OIbLfuk7SCyx/MFDlkJSHtSY6kndKFLu9j/NoZlSzDEwe2C6BFtpxCw+4I/4nt0InGmC YWNqh6TuImEJ6u0mzV/QO1r0EHfMOBAom622IdBxYHEZcxHAPg8Zy9oJ7hFB60S4XZlt xf3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9ZHWkDuRV6sy7Dv4W8CK4OCkYRrSM3Zy5BdK3o811TM=; b=cK3YrxsKABGrpp0O503P7OKzIrMknWDcOBx4Z62mDoYHmV7R+0RelyWjp1Rka23FKZ ZH4BgDav7jsUuFAXNH4x6fdjg1D2134zATxroi2oDI1bsvDXoJk51qPEku8xtskyKtfu hd0xac52UpHb9xfgL8dtp1wSFvVUYCwEweRPM+Tgz5dipX0KGbN72AFcLoNVe1pU5s8l arIOxau8sTKstmWwJO7VQF9kU8IsdEasg4gmSBgcfquSxVLjr1x1afup1M5JdzOG01Hw 6ppTuuBXEnjOzSsxhByefEGx7UVR5ftH62xPiIhXgFg17mzt8t+ZlYliDKHkoRMiFepA WdLA== X-Gm-Message-State: AGRZ1gJ3o/E4L1x52Lnkj9BQ40Su5YPngMMAHqm0uQ/J4FWR6QPrj7r6 ugrvdjQ8xoRR4V/xXKr5rX7ZnG0KK+4= X-Google-Smtp-Source: AFSGD/Xev7m7XFdlAYzs05QEQvrqGSy7uTa+86qV8VKp6fUflzzSHq4GVkT9UGF/uDuPxOu3gl4mFg== X-Received: by 2002:a1c:a017:: with SMTP id j23mr10657594wme.40.1542901189076; Thu, 22 Nov 2018 07:39:49 -0800 (PST) Received: from u.nix.is ([2a01:4f8:190:5095::2]) by smtp.gmail.com with ESMTPSA id a17sm7329389wrs.58.2018.11.22.07.39.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 07:39:48 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v4 10/10] commit-graph write: emit a percentage for all progress Date: Thu, 22 Nov 2018 15:39:22 +0000 Message-Id: <20181122153922.16912-11-avarab@gmail.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c In-Reply-To: <20181122132823.9883-1-avarab@gmail.com> References: <20181122132823.9883-1-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Change the "Annotating commit graph" progress output to show a completion percentage. I added this in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17) and evidently didn't notice how easy it was to add a completion percentage. Now for e.g. linux.git we'll emit: ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (6365442/6365442), done. Annotating commit graph: 100% (2391666/2391666), done. Computing commit graph generation numbers: 100% (797222/797222), done. Writing out commit graph in 4 passes: 100% (3188888/3188888), done. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index 80f201adf4..6c6edc679b 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -660,10 +660,17 @@ static void close_reachable(struct packed_oid_list *oids, int report_progress) struct commit *commit; struct progress *progress = NULL; int j = 0; + /* + * We loop over the OIDs N times to close the graph + * below. This number must be kept in sync with the number of + * passes. + */ + const int oid_passes = 3; if (report_progress) progress = start_delayed_progress( - _("Annotating commit graph"), 0); + _("Annotating commit graph"), + oid_passes * oids->nr); for (i = 0; i < oids->nr; i++) { display_progress(progress, ++j); commit = lookup_commit(the_repository, &oids->list[i]);