From patchwork Wed Jan 16 13:29:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10765911 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A205391E for ; Wed, 16 Jan 2019 13:30:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 919452CF3F for ; Wed, 16 Jan 2019 13:30:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 85A312CF80; Wed, 16 Jan 2019 13:30:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0A5F529459 for ; Wed, 16 Jan 2019 13:30:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393162AbfAPNaJ (ORCPT ); Wed, 16 Jan 2019 08:30:09 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:34727 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732885AbfAPNaI (ORCPT ); Wed, 16 Jan 2019 08:30:08 -0500 Received: by mail-wr1-f65.google.com with SMTP id j2so6979005wrw.1 for ; Wed, 16 Jan 2019 05:30:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WKwjG0pedZGSwCGACvJObsPQltN3dQKsxguVFTra7s4=; b=JlKT0VganDGSCobSEwf4vzXBrkalQq2uyuBdevTEXGBEB/ejZ5KO2fsyMxFKOIkOGl qRZrh25ylXSQVA3wyUZ4ihScOx/0Mz7FYftkysPFCIxLZl8O14G7w7OUYmoJE7OM+3WM lKmrnKfU9Mc01qu/4nrjiSIIPJEDSozLU0rN9t+ZB5d8O/1MgoUbL3SZChVaSw9WlL+C YNZQZNE32E67wD9aP3lchbC+2Du0YcupCK6oh0hI3UFssspCJT/RTkb7Q1RwJJmH+dH5 j4stbt1ygKdXW+jvtb0opsXzUI4GLmtO+03hx81bZPs/VTn+zl3fhjnx9fJi23U1X8YO 1Xeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WKwjG0pedZGSwCGACvJObsPQltN3dQKsxguVFTra7s4=; b=gHWpgimzsTKAJxzo+yuMz2mGhMhWn+6cEkPHxygKFb0bX+gR2EAXo0UGxBHDnQS6/b kmRp1CXtvoVcq8B3x/RBVGGKJDq5qLoGQCZ4NGUH/CoYT53hBVwKWnSnNsejOx+Ac7y0 EEntPrBAK1B4/m4c8IN5FIoOEa9peepUbHfDMTjXMlucZrQslwAyJhqY/n899ONlQwVk AwaefQGXyehMO+rLcCDHkPMQZWuL8ax5g/BcAyYLgpDvrzHonvIPURuNY0M5w6jZaQGB G9XiL/4cjEp4+ykGu9lDC/vcbytHb1RKPNSB6jtNIfF0hnVjYEY7YNDvyfOsqWlkGuYI Y4Fw== X-Gm-Message-State: AJcUukfvEO0LZoHSMnzVc7PKUz96rBjbNevthK+r/Z8Bro42XH/UFzBQ hMoekZ4R009Qb4iGRVGHafbozLDB X-Google-Smtp-Source: ALg8bN5HW+1L1SiYzHNWhVB14dQxdUWsxKfUtqXvU1/eZRMnuNJPqwaBM6XWPU3x95oNwdOqGnmdFQ== X-Received: by 2002:adf:f052:: with SMTP id t18mr8001928wro.112.1547645406011; Wed, 16 Jan 2019 05:30:06 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id 10sm44283964wmy.40.2019.01.16.05.30.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:30:05 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v5 1/9] commit-graph: rename 'num_extra_edges' variable to 'num_large_edges' Date: Wed, 16 Jan 2019 14:29:39 +0100 Message-Id: <20190116132947.26025-2-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20181122153922.16912-2-avarab@gmail.com> References: <20181122153922.16912-2-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: SZEDER Gábor The commit graph file format describes an optional 'Large Edge List' chunk, and the function writing out this chunk is called write_graph_chunk_large_edges(). Then there are two functions in 'commit-graph.c', namely write_graph_chunk_data() and write_commit_graph(), which have a local variable called 'num_extra_edges'. It can be confusing on first sight whether large edges and extra edges refer to the same thing or not, but they do, so let's rename those variables to 'num_large_edges'. Signed-off-by: SZEDER Gábor Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 5c8fb4b134..ab292e5d01 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -472,7 +472,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, { struct commit **list = commits; struct commit **last = commits + nr_commits; - uint32_t num_extra_edges = 0; + uint32_t num_large_edges = 0; while (list < last) { struct commit_list *parent; @@ -504,7 +504,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, if (!parent) edge_value = GRAPH_PARENT_NONE; else if (parent->next) - edge_value = GRAPH_OCTOPUS_EDGES_NEEDED | num_extra_edges; + edge_value = GRAPH_OCTOPUS_EDGES_NEEDED | num_large_edges; else { edge_value = sha1_pos(parent->item->object.oid.hash, commits, @@ -518,7 +518,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, if (edge_value & GRAPH_OCTOPUS_EDGES_NEEDED) { do { - num_extra_edges++; + num_large_edges++; parent = parent->next; } while (parent); } @@ -765,7 +765,7 @@ void write_commit_graph(const char *obj_dir, uint32_t chunk_ids[5]; uint64_t chunk_offsets[5]; int num_chunks; - int num_extra_edges; + int num_large_edges; struct commit_list *parent; struct progress *progress = NULL; @@ -875,7 +875,7 @@ void write_commit_graph(const char *obj_dir, commits.alloc = count_distinct; ALLOC_ARRAY(commits.list, commits.alloc); - num_extra_edges = 0; + num_large_edges = 0; for (i = 0; i < oids.nr; i++) { int num_parents = 0; if (i > 0 && oideq(&oids.list[i - 1], &oids.list[i])) @@ -889,11 +889,11 @@ void write_commit_graph(const char *obj_dir, num_parents++; if (num_parents > 2) - num_extra_edges += num_parents - 1; + num_large_edges += num_parents - 1; commits.nr++; } - num_chunks = num_extra_edges ? 4 : 3; + num_chunks = num_large_edges ? 4 : 3; if (commits.nr >= GRAPH_PARENT_MISSING) die(_("too many commits to write graph")); @@ -920,7 +920,7 @@ void write_commit_graph(const char *obj_dir, chunk_ids[0] = GRAPH_CHUNKID_OIDFANOUT; chunk_ids[1] = GRAPH_CHUNKID_OIDLOOKUP; chunk_ids[2] = GRAPH_CHUNKID_DATA; - if (num_extra_edges) + if (num_large_edges) chunk_ids[3] = GRAPH_CHUNKID_LARGEEDGES; else chunk_ids[3] = 0; @@ -930,7 +930,7 @@ void write_commit_graph(const char *obj_dir, chunk_offsets[1] = chunk_offsets[0] + GRAPH_FANOUT_SIZE; chunk_offsets[2] = chunk_offsets[1] + GRAPH_OID_LEN * commits.nr; chunk_offsets[3] = chunk_offsets[2] + (GRAPH_OID_LEN + 16) * commits.nr; - chunk_offsets[4] = chunk_offsets[3] + 4 * num_extra_edges; + chunk_offsets[4] = chunk_offsets[3] + 4 * num_large_edges; for (i = 0; i <= num_chunks; i++) { uint32_t chunk_write[3]; From patchwork Wed Jan 16 13:29:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10765915 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0EF1F1390 for ; Wed, 16 Jan 2019 13:30:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F18F72CF80 for ; Wed, 16 Jan 2019 13:30:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E64B42CEFC; Wed, 16 Jan 2019 13:30:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8E8AB2CEFC for ; Wed, 16 Jan 2019 13:30:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393166AbfAPNaL (ORCPT ); Wed, 16 Jan 2019 08:30:11 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:50230 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732885AbfAPNaK (ORCPT ); Wed, 16 Jan 2019 08:30:10 -0500 Received: by mail-wm1-f68.google.com with SMTP id n190so1994762wmd.0 for ; Wed, 16 Jan 2019 05:30:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=C4AHvfXjVpz6hSJF3GoDRx1XH41Ax8rfPJUlEQHS2Dw=; b=G3fl2do7c0wuoPdxRQ9nwmgB+xVqmILqLdQP+m5zt6E0cOCUl1a1tvusrWb05RrLgf laTuRFIxqJ+QNO9wiSIDmfCwmZKKhOStlVbQuZ1yWHT3zyHgpb/bps7MSyaPxi28flAF GNBYWD4Pf8A9299uNUoF/JgHU0VUqOepqg7L2c+rX3tSZhUkO647B0IFZfiDBf3FdDff +xnBg4WMOkPTpB8rpvEeRt/4NcFQJ+491HdeErAnOmo1OL8uXju1/VcF+bsqCi1xqoAp HEojY8LR8aVVevnCSXabnoqVz9GzEdp4fEMDVAr751oKm4OdfdvS63nNy9CylGKrmWRN tjdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=C4AHvfXjVpz6hSJF3GoDRx1XH41Ax8rfPJUlEQHS2Dw=; b=DSjqZYMyYRWWEyoKDO/uapvzKUXow1aOBQ2Rj3Nl0PcXCSbD41m6a1rt6NJ86o0Qux XCBCkdJ/aJXdCHdavEh7KTuCtUUjrg+qWbGM2XpJd1c3M+/N6dNYWjhKDxlohtG7Pqw4 aeeIHKkofm3hwzdwHvxczSy9beW8fOF8EqFkXuvhAX92MX7rdJRMmWljRFVyW+YbZgB7 17vRiTSlwihUdXW7coXcIeMxqkcTaNkSYdUtE4GQPsIlGuMKeZztEeW4vgnYDXzPUxnM qP+plZ8601RMebkwNVWB+UZ0ZmfPFH9FPHONAGXHldrvAMW+VKErENTxCZmX1ya1+znS tNWQ== X-Gm-Message-State: AJcUukcSNe/UA6QKkCcFcPhRIYPqgl12T/rN5ESZFuj6u5n4bwNNmLV8 JifQ8gWVjiyXwWo7XaK8PEvGQzY9 X-Google-Smtp-Source: ALg8bN68h5YFZD72ueQHK4o8rwRtRaRDHOXB0iqStU3F/w9LILd4+WOgexRTHZH+d8heUgMSVZyRoQ== X-Received: by 2002:a1c:9692:: with SMTP id y140mr7820073wmd.67.1547645407618; Wed, 16 Jan 2019 05:30:07 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id 10sm44283964wmy.40.2019.01.16.05.30.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:30:06 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v5 2/9] commit-graph: don't call write_graph_chunk_large_edges() unnecessarily Date: Wed, 16 Jan 2019 14:29:40 +0100 Message-Id: <20190116132947.26025-3-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20181122153922.16912-2-avarab@gmail.com> References: <20181122153922.16912-2-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: SZEDER Gábor The optional 'Large Edge List' chunk of the commit graph file stores parent information for commits with more than two parents. Since the chunk is optional, write_commit_graph() looks through all commits to find those with more than two parents, and then writes the commit graph file header accordingly, i.e. if there are no such commits, then there won't be a 'Large Edge List' chunk written, only the three mandatory chunks. However, when it comes to writing chunk data, write_commit_graph() unconditionally invokes write_graph_chunk_large_edges(), even when it was decided earlier that that chunk won't be written. Strictly speaking there is no bug here, because write_graph_chunk_large_edges() won't write anything because it won't find any commits with more than two parents, but then it unnecessarily and in vain looks through all commits once again in search for such commits. Don't call write_graph_chunk_large_edges() when that chunk won't be written to spare an unnecessary iteration over all commits. Signed-off-by: SZEDER Gábor Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index ab292e5d01..38b62051c2 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -944,7 +944,8 @@ void write_commit_graph(const char *obj_dir, write_graph_chunk_fanout(f, commits.list, commits.nr); write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr); write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr); - write_graph_chunk_large_edges(f, commits.list, commits.nr); + if (num_large_edges) + write_graph_chunk_large_edges(f, commits.list, commits.nr); close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); From patchwork Wed Jan 16 13:29:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10765917 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 465621390 for ; Wed, 16 Jan 2019 13:30:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 36D282CDF7 for ; Wed, 16 Jan 2019 13:30:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2AB592D024; Wed, 16 Jan 2019 13:30:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9A9CC2CF3D for ; Wed, 16 Jan 2019 13:30:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393168AbfAPNaN (ORCPT ); Wed, 16 Jan 2019 08:30:13 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:34028 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393163AbfAPNaM (ORCPT ); Wed, 16 Jan 2019 08:30:12 -0500 Received: by mail-wm1-f67.google.com with SMTP id y185so1056292wmd.1 for ; Wed, 16 Jan 2019 05:30:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2F8GwpdTG04fhJY3muUz/i65P/oT1SE+CLTsfyKjXic=; b=CrT4F5QgJCSVv62PA0WsSHUJAlOedHaBtWLNct895yMoCPs8pWtEPe56sDvrll14Uo jhMhy8ZTcmYGD/eo3E08PxFV0IkgyLCC8FOjWxkOu0Z3jZfu7s/PkS82knXzAfSUEag5 eGNjSjj4VDy7tH5HfmlythUoRbc4cV+wctK0/mBoYAi2Q1iSTYJ4mGDSEooIWHrblnmK y23EGo+yP6n/n8U9j1hA3Fp6lRvIGrp0wGkwVvUFQeBksC7G8cg/Pi6/6JJpKJSGMFru HnTMffJ7S/6htO93KEovCDwOZLlq44idmvbu76DpM2sapVt6y6c3nUlptItPhF2pZeWB 8Rrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2F8GwpdTG04fhJY3muUz/i65P/oT1SE+CLTsfyKjXic=; b=jhLAIiaiAf+rGH3BWf7KsnlknNKxjK9IAKU+PHFU3AmrZthjLGGnMi9WIyzDRfEaYx xbY0t2sTtPWrALxpAis0P1Pu/9KneuFqAy7bw4dyp41QDbho6Y72voCsuJFHG3a1WJgV fglT4wJ6NAlTbuZa+mPHFsPzImZlyyz8clULlbknho+bTlcS94LOC4rC10CL2nOuv13z 1/flljouV+s2h0o0GNrtIiPzagia6n0klMqVp5xks4DZC4WovtXiwUqClphsvbsIwQaK caiYCMXf83gMrjJOa/W19bsSwU2HKfAT47Sk/5MbC9j412odGIHgHye7N3dkjEHmVbrT NBSQ== X-Gm-Message-State: AJcUukf0lgJ+kVrQHOKbxQt10dw6iRmJ2NfhBxWGbS8eQRjA2O2VCZeD sK3gubTtfO+8O3ZsdXM57k2EQaov X-Google-Smtp-Source: ALg8bN5EBz/tEkE/ZHhCZiKZuhqBRwZkZT/EzNKCIrHdDJRuBR7qslnk68ddNXS0Rs94VffLW771SQ== X-Received: by 2002:a1c:5a42:: with SMTP id o63mr7494311wmb.88.1547645409328; Wed, 16 Jan 2019 05:30:09 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id 10sm44283964wmy.40.2019.01.16.05.30.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:30:08 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v5 3/9] commit-graph write: add "Writing out" progress output Date: Wed, 16 Jan 2019 14:29:41 +0100 Message-Id: <20190116132947.26025-4-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20181122153922.16912-2-avarab@gmail.com> References: <20181122153922.16912-2-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add progress output to be shown when we're writing out the commit-graph, this adds to the output already added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17). As noted in that commit most of the progress output isn't displayed on small repositories, but before this change we'd noticeably hang for 2-3 seconds at the end on medium sized repositories such as linux.git. Now we'll instead show output like this, and reduce the human-observable times at which we're not producing progress output: $ ~/g/git/git --exec-path=$HOME/g/git -C ~/g/2015-04-03-1M-git commit-graph write Finding commits for commit graph: 13064614, done. Expanding reachable commits in commit graph: 1000447, done. Computing commit graph generation numbers: 100% (1000447/1000447), done. Writing out commit graph: 100% (3001341/3001341), done. This "Writing out" number is 3x or 4x the number of commits, depending on the graph we're processing. A later change will make this explicit to the user. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 48 +++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 39 insertions(+), 9 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 38b62051c2..0c3af28399 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -430,7 +430,9 @@ struct tree *get_commit_tree_in_graph(struct repository *r, const struct commit static void write_graph_chunk_fanout(struct hashfile *f, struct commit **commits, - int nr_commits) + int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { int i, count = 0; struct commit **list = commits; @@ -444,6 +446,7 @@ static void write_graph_chunk_fanout(struct hashfile *f, while (count < nr_commits) { if ((*list)->object.oid.hash[0] != i) break; + display_progress(progress, ++*progress_cnt); count++; list++; } @@ -453,12 +456,16 @@ static void write_graph_chunk_fanout(struct hashfile *f, } static void write_graph_chunk_oids(struct hashfile *f, int hash_len, - struct commit **commits, int nr_commits) + struct commit **commits, int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; int count; - for (count = 0; count < nr_commits; count++, list++) + for (count = 0; count < nr_commits; count++, list++) { + display_progress(progress, ++*progress_cnt); hashwrite(f, (*list)->object.oid.hash, (int)hash_len); + } } static const unsigned char *commit_to_sha1(size_t index, void *table) @@ -468,7 +475,9 @@ static const unsigned char *commit_to_sha1(size_t index, void *table) } static void write_graph_chunk_data(struct hashfile *f, int hash_len, - struct commit **commits, int nr_commits) + struct commit **commits, int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; struct commit **last = commits + nr_commits; @@ -478,6 +487,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, struct commit_list *parent; int edge_value; uint32_t packedDate[2]; + display_progress(progress, ++*progress_cnt); parse_commit(*list); hashwrite(f, get_commit_tree_oid(*list)->hash, hash_len); @@ -539,7 +549,9 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, static void write_graph_chunk_large_edges(struct hashfile *f, struct commit **commits, - int nr_commits) + int nr_commits, + struct progress *progress, + uint64_t *progress_cnt) { struct commit **list = commits; struct commit **last = commits + nr_commits; @@ -547,6 +559,9 @@ static void write_graph_chunk_large_edges(struct hashfile *f, while (list < last) { int num_parents = 0; + + display_progress(progress, ++*progress_cnt); + for (parent = (*list)->parents; num_parents < 3 && parent; parent = parent->next) num_parents++; @@ -768,6 +783,7 @@ void write_commit_graph(const char *obj_dir, int num_large_edges; struct commit_list *parent; struct progress *progress = NULL; + uint64_t progress_cnt = 0; if (!commit_graph_compatible(the_repository)) return; @@ -941,11 +957,25 @@ void write_commit_graph(const char *obj_dir, hashwrite(f, chunk_write, 12); } - write_graph_chunk_fanout(f, commits.list, commits.nr); - write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr); - write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr); + if (report_progress) { + /* + * Each of the write_graph_chunk_*() functions just + * below loops over our N commits. This number must be + * kept in sync with the number of passes we're doing. + */ + int graph_passes = 3; + if (num_large_edges) + graph_passes++; + progress = start_delayed_progress( + _("Writing out commit graph"), + graph_passes * commits.nr); + } + write_graph_chunk_fanout(f, commits.list, commits.nr, progress, &progress_cnt); + write_graph_chunk_oids(f, GRAPH_OID_LEN, commits.list, commits.nr, progress, &progress_cnt); + write_graph_chunk_data(f, GRAPH_OID_LEN, commits.list, commits.nr, progress, &progress_cnt); if (num_large_edges) - write_graph_chunk_large_edges(f, commits.list, commits.nr); + write_graph_chunk_large_edges(f, commits.list, commits.nr, progress, &progress_cnt); + stop_progress(&progress); close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); From patchwork Wed Jan 16 13:29:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10765919 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 83D5D91E for ; Wed, 16 Jan 2019 13:30:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 735BA2CEE7 for ; Wed, 16 Jan 2019 13:30:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 66DB52D024; Wed, 16 Jan 2019 13:30:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 09A592CD5A for ; Wed, 16 Jan 2019 13:30:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393170AbfAPNaP (ORCPT ); Wed, 16 Jan 2019 08:30:15 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:40172 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393158AbfAPNaN (ORCPT ); Wed, 16 Jan 2019 08:30:13 -0500 Received: by mail-wm1-f66.google.com with SMTP id f188so1978315wmf.5 for ; Wed, 16 Jan 2019 05:30:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sC2qNZzmj+8JLaFcC1APTX7BuXGegsoDQvpreSQl+KM=; b=HXQCqPFb9y1RzDUM9zEUfvalO09NBfGq1nRNnowMnsS0+V4hHHC7T+o6v+owcJbdjd 8WFo3JOGpsQuWk9Xhfa+wI/UaltbECKzzVl9dYPv6nUqjBMqbmBdeAPx/eMWOX8NQPTG xpz5qlm+4QoW6GECEancDw/mJSNHQIACrlv7wbcKYQj+RaQUU6xUEdRdJGtGOhVVpDah WNxHQqKmbye50DgY+0OcXj76qftXXbAZYnrd8ra0tC39WxplI/de8Z867CO8gAYYJ7rE rp5dCXPcRdxNLWLdz48dgilael3946V3L8ITSYT/9eUPdJgHFY8/8Tl2gCHic9U3JKOZ oQSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sC2qNZzmj+8JLaFcC1APTX7BuXGegsoDQvpreSQl+KM=; b=mEGnvxKdMeaj6ewAKxhSVEglPuAs8AkjHF8IF5N2bl/AivSEUK4Fy5JMQ/5GDtn1EQ cx0js4OX33LN7ReZrV0yRdqcyVnNCHgiokxPS18UIApKK/b7gykGlpOM/T/vR4svX6+n j6DZZzG4AJG+DUqPU683LlFFCtoUELmriJy2q+DxM8JVHMG6sX3JEvdHBPuuIi1PkZ9G SYIMZ6/CjLgyT3NpzIkwC6MYoAR3Dk8Ks7JIEQtMMmLgEF3Ic5n4EXv2j948FQ2hXefS k7slnhH4cqSyt4Qmeeql2lo3Q3xPOkGFL7YLGFCvfaXgxka+7R+27K08ymP4zj7iUXUh Rxxw== X-Gm-Message-State: AJcUukegBD+0U8t8PWYlWGJuHe+hnlp/K8oOugrm6hYMBuulvnmykoPk PMemwdzKTwBmx0haegK2oQ+0/zqKvwI= X-Google-Smtp-Source: ALg8bN731EEZKp81qFsZHu0djTF/7VZD+x2gnsJnbQ4V77XAbGBHg984O5UxCF0I/kDMvnk2vNSo8Q== X-Received: by 2002:a7b:cf0f:: with SMTP id l15mr8104532wmg.30.1547645411895; Wed, 16 Jan 2019 05:30:11 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id 10sm44283964wmy.40.2019.01.16.05.30.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:30:10 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v5 4/9] commit-graph write: more descriptive "writing out" output Date: Wed, 16 Jan 2019 14:29:42 +0100 Message-Id: <20190116132947.26025-5-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20181122153922.16912-2-avarab@gmail.com> References: <20181122153922.16912-2-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make the "Writing out" part of the progress output more descriptive. Depending on the shape of the graph we either make 3 or 4 passes over it. Let's present this information to the user in case they're wondering what this number, which is much larger than their number of commits, has to do with writing out the commit graph. Now e.g. on linux.git we emit: $ ~/g/git/git --exec-path=$HOME/g/git -C ~/g/linux commit-graph write Finding commits for commit graph: 6529159, done. Expanding reachable commits in commit graph: 815990, done. Computing commit graph generation numbers: 100% (815983/815983), done. Writing out commit graph in 4 passes: 100% (3263932/3263932), done. A note on i18n: Why are we using the Q_() function and passing a number & English text for a singular which'll never be used? Because the plural rules of translated languages may not match those of English, and to use the plural function we need to use this format. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index 0c3af28399..7653c4e9d6 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -784,6 +784,7 @@ void write_commit_graph(const char *obj_dir, struct commit_list *parent; struct progress *progress = NULL; uint64_t progress_cnt = 0; + struct strbuf progress_title = STRBUF_INIT; if (!commit_graph_compatible(the_repository)) return; @@ -966,8 +967,13 @@ void write_commit_graph(const char *obj_dir, int graph_passes = 3; if (num_large_edges) graph_passes++; + strbuf_addf(&progress_title, + Q_("Writing out commit graph in %d pass", + "Writing out commit graph in %d passes", + graph_passes), + graph_passes); progress = start_delayed_progress( - _("Writing out commit graph"), + progress_title.buf, graph_passes * commits.nr); } write_graph_chunk_fanout(f, commits.list, commits.nr, progress, &progress_cnt); @@ -976,6 +982,7 @@ void write_commit_graph(const char *obj_dir, if (num_large_edges) write_graph_chunk_large_edges(f, commits.list, commits.nr, progress, &progress_cnt); stop_progress(&progress); + strbuf_release(&progress_title); close_commit_graph(the_repository); finalize_hashfile(f, NULL, CSUM_HASH_IN_STREAM | CSUM_FSYNC); From patchwork Wed Jan 16 13:29:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10765921 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4FD2591E for ; Wed, 16 Jan 2019 13:30:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3B99E2D542 for ; Wed, 16 Jan 2019 13:30:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3007B2D830; Wed, 16 Jan 2019 13:30:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 40FAE2D758 for ; Wed, 16 Jan 2019 13:30:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393173AbfAPNaT (ORCPT ); Wed, 16 Jan 2019 08:30:19 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:38119 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393158AbfAPNaS (ORCPT ); Wed, 16 Jan 2019 08:30:18 -0500 Received: by mail-wm1-f67.google.com with SMTP id m22so2010897wml.3 for ; Wed, 16 Jan 2019 05:30:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3xAVNYoiuQrBWAz3Wo+k/zvFJLUkV5jlSsvAv3GlPnc=; b=H4iFLBOBmvZOalCRr8R7v4JzBf6rfzynjv6AbstEFZ9H8tberYaE57dgg9WZohsaW4 HHk19KJwoCJvalS0yc7wHve31Ax0Nbk3//8Y4ncYmOjqPyoSwOOaEvqKa13dYlWdGVci ydsD5poTzRl7AMbZnkN2EhjNDT9+Tn+hsgnlwDne7NPUM/8w1FxORMcRrWzXq5b5QS6A h60fWFgvdcEV2b8DRLtqf5CuQ9lymtr2YJiVYJuM2CTR+gRLExeR1wT+W01P6PMS7d3U TbttXLhIxV/eOPRM8v9MX1djCtdKFaaBHjlTaGZIXD9giWYTsyeZRNJ/3U2zYW7R8mf9 8YkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3xAVNYoiuQrBWAz3Wo+k/zvFJLUkV5jlSsvAv3GlPnc=; b=VidZtrkGIJLpuxcBZ4pbL8YwSjUTfW3eHP3e0L8BOl2mBEFdNK7rik6tdQzHQLnJiE 2yhp6Us53Rv1YE9tmLCpZ881AlBahMSpcdImG6RFkeaKG6+CGffUJaR5tRS6+T65ZrUE D4bpZ/2wl2NAH8b/E+NS6UazeRZv2O+84r7q/Xj1azG0t1IMoqeeEt/gRznW873+sYEM 9QT+DNgErqBsKYegN7HnmtUmLz4REWR3PMsIHgziNWvhcyVDczXo9zNolLjGY9ItaaR7 mGHtHLrAZCTk0kBOk2s+TMfdofccC99UXwGIWaiq6ZlgcRYCRtsMTS+nVud3vVSLAyjE IbyQ== X-Gm-Message-State: AJcUukf2m6ztq0ZQeNmkDEtdwHX0T8hAu3hb9DODQfzb7hMREPuJBuFM 1HlRXy3FDZOJdh46EcmlfoHqEmMdtzE= X-Google-Smtp-Source: ALg8bN7DKws5AahnRHtYZFAgTjtIkMa5BIvaeVl1+Eky2bAM0GuXwq3d2wisenFuucbxe8aZ9cMRsA== X-Received: by 2002:a1c:8d53:: with SMTP id p80mr8023368wmd.68.1547645415753; Wed, 16 Jan 2019 05:30:15 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id 10sm44283964wmy.40.2019.01.16.05.30.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:30:12 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v5 5/9] commit-graph write: show progress for object search Date: Wed, 16 Jan 2019 14:29:43 +0100 Message-Id: <20190116132947.26025-6-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20181122153922.16912-2-avarab@gmail.com> References: <20181122153922.16912-2-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Show the percentage progress for the "Finding commits for commit graph" phase for the common case where we're operating on all packs in the repository, as "commit-graph write" or "gc" will do. Before we'd emit on e.g. linux.git with "commit-graph write": Finding commits for commit graph: 6529159, done. [...] And now: Finding commits for commit graph: 100% (6529159/6529159), done. [...] Since the commit graph only includes those commits that are packed (via for_each_packed_object(...)) the approximate_object_count() returns the actual number of objects we're going to process. Still, it is possible due to a race with "gc" or another process maintaining packs that the number of objects we're going to process is lower than what approximate_object_count() reported. In that case we don't want to stop the progress bar short of 100%. So let's make sure it snaps to 100% at the end. The inverse case is also possible and more likely. I.e. that a new pack has been added between approximate_object_count() and for_each_packed_object(). In that case the percentage will go beyond 100%, and we'll do nothing to snap it back to 100% at the end. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 7653c4e9d6..f0bcb3c690 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -785,12 +785,14 @@ void write_commit_graph(const char *obj_dir, struct progress *progress = NULL; uint64_t progress_cnt = 0; struct strbuf progress_title = STRBUF_INIT; + unsigned long approx_nr_objects; if (!commit_graph_compatible(the_repository)) return; oids.nr = 0; - oids.alloc = approximate_object_count() / 32; + approx_nr_objects = approximate_object_count(); + oids.alloc = approx_nr_objects / 32; oids.progress = NULL; oids.progress_done = 0; @@ -870,8 +872,11 @@ void write_commit_graph(const char *obj_dir, if (!pack_indexes && !commit_hex) { if (report_progress) oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), 0); + _("Finding commits for commit graph"), + approx_nr_objects); for_each_packed_object(add_packed_commits, &oids, 0); + if (oids.progress_done < approx_nr_objects) + display_progress(oids.progress, approx_nr_objects); stop_progress(&oids.progress); } From patchwork Wed Jan 16 13:29:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10765923 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 96B0091E for ; Wed, 16 Jan 2019 13:30:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 869552D7CD for ; Wed, 16 Jan 2019 13:30:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7B1142D758; Wed, 16 Jan 2019 13:30:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 654482D8ED for ; Wed, 16 Jan 2019 13:30:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393178AbfAPNaV (ORCPT ); Wed, 16 Jan 2019 08:30:21 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:50245 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393158AbfAPNaU (ORCPT ); Wed, 16 Jan 2019 08:30:20 -0500 Received: by mail-wm1-f65.google.com with SMTP id n190so1995416wmd.0 for ; Wed, 16 Jan 2019 05:30:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Cxa4SSIuR7pbrMARIgZVDIxmMdjMX0GROeVkZ+dhHF8=; b=i4+8WFjL+MLjQk2iUNQ8YhB6JjjvgotItVH9GkOp8YYmY6KMK25wzINgsFyKufDUFV wTiLSXsbalD3lF9bIQGElXWyOkMWxpR3k3AU3N10YQVGQx7J5psKkp+Zu2K1qTC/NlXC sl/dUxLaCtqkxRwg+2gAt568uPcHk7Zar8umo5c3f/uKSmfjzhNDtD/qxwe5bvWb4uKg QedFWyTeRjI3pTLgP8QhhTREK/YPEkJGS8ScPtUpkaUVbLmd17X9gnnYKRf6Qc1X1Cyt z3t7l0ZDBqS4RDeWWbr6YHMJq9ylRL7giroX8tqw3XLGPBvYFPzX0R0tcQzYwST6XC2O TvUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Cxa4SSIuR7pbrMARIgZVDIxmMdjMX0GROeVkZ+dhHF8=; b=V06Tdiu/Fowi2dGNy9RG3mMesRaDSAtMCXjLY3QoW1dZ7RroZPO5vD7kY8onHWtnMO TJ9eRi1usCEXTOeI39puNZZ9XFtFjOInamm2qpFaA8mo3W6B+otHnkMCxdHgMaHHMcI5 6nBYEsdMSbbh95UXmLpC6oaLX3f8axvMLPwHpIukgTLmrtQAuGbPh40OxWKEsYyln39B kiDUigp+iR7kFUAk5Z/s+MIOSi6ry4pbA1JtjSrv58xOOftZV9IFPCQrknDPoEsPDUCb U0weCcUuaKZ1OcFWtahET3Zjy24qKu8Th5G1h5Dv5s0UIVHOzuAptosmFQ43NQvdVV2M V17w== X-Gm-Message-State: AJcUukeKiE2fUHPr7mn2Hn3jiv/QHJYmwqgvn4n2STpdQ9aocsYVh5fY 8RtCbXrdMeG9gpxFe498eGed7QFP/zE= X-Google-Smtp-Source: ALg8bN70aYf417IXFdFEyCUMNCE73H2DdfDVIqPkyaachsSXoMGmlMK0mKev+9LO3499Z74EL7X9vQ== X-Received: by 2002:a7b:c04e:: with SMTP id u14mr6989422wmc.133.1547645418508; Wed, 16 Jan 2019 05:30:18 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id 10sm44283964wmy.40.2019.01.16.05.30.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:30:16 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v5 6/9] commit-graph write: add more descriptive progress output Date: Wed, 16 Jan 2019 14:29:44 +0100 Message-Id: <20190116132947.26025-7-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20181122153922.16912-2-avarab@gmail.com> References: <20181122153922.16912-2-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make the progress output shown when we're searching for commits to include in the graph more descriptive. This amends code I added in 7b0f229222 ("commit-graph write: add progress output", 2018-09-17). Now, on linux.git, we'll emit this sort of output in the various modes we support: $ git commit-graph write Finding commits for commit graph among packed objects: 100% (6529159/6529159), done. [...] # Actually we don't emit this since this takes almost no time at # all. But if we did (s/_delayed//) we'd show: $ git for-each-ref --format='%(objectname)' | git commit-graph write --stdin-commits Finding commits for commit graph from 630 refs: 100% (630/630), done. [...] $ (cd .git/objects/pack/ && ls *idx) | git commit-graph write --stdin-pack Finding commits for commit graph in 3 packs: 6529159, done. [...] The middle on of those is going to be the output users might see in practice, since it'll be emitted when they get the commit graph via gc.writeCommitGraph=true. But as noted above you need a really large number of refs for this message to show. It'll show up on a test repository I have with ~165k refs: Finding commits for commit graph from 165203 refs: 100% (165203/165203), done. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index f0bcb3c690..c6f8e73980 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -822,8 +822,12 @@ void write_commit_graph(const char *obj_dir, strbuf_addf(&packname, "%s/pack/", obj_dir); dirlen = packname.len; if (report_progress) { - oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), 0); + strbuf_addf(&progress_title, + Q_("Finding commits for commit graph in %d pack", + "Finding commits for commit graph in %d packs", + pack_indexes->nr), + pack_indexes->nr); + oids.progress = start_delayed_progress(progress_title.buf, 0); oids.progress_done = 0; } for (i = 0; i < pack_indexes->nr; i++) { @@ -840,14 +844,20 @@ void write_commit_graph(const char *obj_dir, free(p); } stop_progress(&oids.progress); + strbuf_reset(&progress_title); strbuf_release(&packname); } if (commit_hex) { - if (report_progress) - progress = start_delayed_progress( - _("Finding commits for commit graph"), - commit_hex->nr); + if (report_progress) { + strbuf_addf(&progress_title, + Q_("Finding commits for commit graph from %d ref", + "Finding commits for commit graph from %d refs", + commit_hex->nr), + commit_hex->nr); + progress = start_delayed_progress(progress_title.buf, + commit_hex->nr); + } for (i = 0; i < commit_hex->nr; i++) { const char *end; struct object_id oid; @@ -867,12 +877,13 @@ void write_commit_graph(const char *obj_dir, } } stop_progress(&progress); + strbuf_reset(&progress_title); } if (!pack_indexes && !commit_hex) { if (report_progress) oids.progress = start_delayed_progress( - _("Finding commits for commit graph"), + _("Finding commits for commit graph among packed objects"), approx_nr_objects); for_each_packed_object(add_packed_commits, &oids, 0); if (oids.progress_done < approx_nr_objects) From patchwork Wed Jan 16 13:29:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10765925 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D6C4D91E for ; Wed, 16 Jan 2019 13:30:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C446A2CD5A for ; Wed, 16 Jan 2019 13:30:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B8AFB2D758; Wed, 16 Jan 2019 13:30:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 64F142CD5A for ; Wed, 16 Jan 2019 13:30:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393181AbfAPNaY (ORCPT ); Wed, 16 Jan 2019 08:30:24 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:45369 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393158AbfAPNaX (ORCPT ); Wed, 16 Jan 2019 08:30:23 -0500 Received: by mail-wr1-f67.google.com with SMTP id t6so6877049wrr.12 for ; Wed, 16 Jan 2019 05:30:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sSbp6ax1SErZl6HkIEM5tIrxmcblhU9P25q521QZwCA=; b=teGQBxdxoWdfaORipg7Lca8/HCapb81CSxMAF07NtD0TFPmbyIc0xi5icpgkJiW0Fw PfB/HD1SJVBmEp9L3NZicnv7GNwQ/j0gNkpNM/M4luUH1LQitIJpIFmUWuHPlBOO8UQw AOCclzadlsf5tFjSOlv0klAt40oy5iC04heKQ3X15f2/cdWDHPD81h8gMPPngGNkCftR pr29X2c1enWUyNZPqtTfW8VGG49k4JTnnpJLCrM94w7yEwfo3jLGxJ9Y791dWvl6ZZ11 c9FRnWUqbyk3Mn4qBpBSDR8cVHqt39YdXENhVdfdqTOUfE+Z8uHOSdaHdkwcRI4sNjQt 9Y8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sSbp6ax1SErZl6HkIEM5tIrxmcblhU9P25q521QZwCA=; b=uZzhdHwnHFPCQPd6NBBNfwBR6KAEHzwjAtA0aJXhItAajqgFHy2+THNDQ0s06Tvs8o ngrrPRm4Ml8pZL2OuyiuzKF65nOyqtNJhb7J+sn1cCJKXwO4PRRGixyptPIqd0ptOL8r ePXVfIkdMGPdM1ZF/9yyr5CoEww4JLrONDKL/f+YkvvmeJ64oB6Yz2hzCu/ABAnxYh1i tJrBlm0dfNz/RIOl3rOtMwZ6qGuAgyc72hxuetIZJeD1bKLPuYiioVqWN2MV3NL6WdR+ eg0mVvJfgbjY7b1HgsrUBb8JYBA+B6u+2N3Qb1yasbyhYmVSm/LZPpKAquPRFN7jOghf vm1g== X-Gm-Message-State: AJcUukfgAwOr+o4x+fZwvRNro3a51ds7ZIAC9xufAXhQgfEgya+pS242 gz07pWl+j4m/PxEz90Pkly+CrZqx0s4= X-Google-Smtp-Source: ALg8bN5Kdnoi1wX84fS9WTgPDiPuAiRwK58wejDV+n8Bp/1BDZzUx9fOhFDUiSgkuqyTXudueft55Q== X-Received: by 2002:adf:fc51:: with SMTP id e17mr7485919wrs.268.1547645421226; Wed, 16 Jan 2019 05:30:21 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id 10sm44283964wmy.40.2019.01.16.05.30.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:30:19 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v5 7/9] commit-graph write: remove empty line for readability Date: Wed, 16 Jan 2019 14:29:45 +0100 Message-Id: <20190116132947.26025-8-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20181122153922.16912-2-avarab@gmail.com> References: <20181122153922.16912-2-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Remove the empty line between a QSORT(...) and the subsequent oideq() for-loop. This makes it clearer that the QSORT(...) is being done so that we can run the oideq() loop on adjacent OIDs. Amends code added in 08fd81c9b6 ("commit-graph: implement write_commit_graph()", 2018-04-02). Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 1 - 1 file changed, 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index c6f8e73980..7138bbd02f 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -894,7 +894,6 @@ void write_commit_graph(const char *obj_dir, close_reachable(&oids, report_progress); QSORT(oids.list, oids.nr, commit_compare); - count_distinct = 1; for (i = 1; i < oids.nr; i++) { if (!oideq(&oids.list[i - 1], &oids.list[i])) From patchwork Wed Jan 16 13:29:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10765927 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6ECB991E for ; Wed, 16 Jan 2019 13:30:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5E9E82D154 for ; Wed, 16 Jan 2019 13:30:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5359B2CE76; Wed, 16 Jan 2019 13:30:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C89EB2D75B for ; Wed, 16 Jan 2019 13:30:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404295AbfAPNa1 (ORCPT ); Wed, 16 Jan 2019 08:30:27 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:39380 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393158AbfAPNa0 (ORCPT ); Wed, 16 Jan 2019 08:30:26 -0500 Received: by mail-wm1-f67.google.com with SMTP id y8so1983708wmi.4 for ; Wed, 16 Jan 2019 05:30:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KUNZq2p4g1oEJp/fK3u2S6boTXD082ZPe8FfGc0PDqs=; b=ObmsebVZZ8ETdce5+vwD64xgOeLRWcyWkAsCH2Hh6KUE651+ibX4v3ExnASMPh0yqp fGtUZ6IxvcE7HyrMqOeBayCEXOBrE6uxx2i4lRbZLEl1agFwN45aR1U5fO5IEA551NsU 0EMo7CGoymAgkEIi0cXdqk040+C+Qhiu6bPO6c4pIKVN7s4kb9tMlyrnfNpcWZQybxbd s9vyekf5tQx9pTx5U/OfmTcGOVuVqYiG5hLZPK36MIsg23ln/H0ZCNjf2B53UOHxfhom PH3yMRsstvy3CItomVuyREMcdf6RrktuZWGEgoMiNePoQNScr+YdxjN5Q7qHUCQHdGvH 3ldg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KUNZq2p4g1oEJp/fK3u2S6boTXD082ZPe8FfGc0PDqs=; b=hYrD0zGYkxJ7wFsMWt86/Ejv++xWdavu0GEMwdt+8hWfueR1JUgS2Ej6ixgzPAZows aNF2iMh9xvFu4u6rjc7L3Vc1hTeySWVtXvnEyyD5rH1n/gIegUZwYrN5QmXMHrkTBdSL a8a8kAiZ+nTLkdFPaSDge79uM6wR6TxDv3GO69WuNLxf/cJlm563EwhTV+E6Yb8wZW3u um34e/yKakEzPA7SZPwsPnbOxQ9xdhBAbUlapY0/8CQSfzpqXiqfNgI8CVfvynYR8QJx UU8Zo33BIckKX7xcyCiyUh8qIh6Yq+oDWzv0N3GzvILjC2e99JhM+5zgi9hCTDomJKRC LzQQ== X-Gm-Message-State: AJcUukdyeEnDiqpvomXEfwVwt20z2T3e43F+wrXx57N7ATp5OWuZyPTQ xXffVKDWnhkdrdgubOHCQkJoZigXSd0= X-Google-Smtp-Source: ALg8bN4sQgGySF9fMlWa/BYO6qThokhJv6+HUNsplj++y08Vjh/PsijQHOl/qVHg5beu1WM4BYvRtg== X-Received: by 2002:a1c:cbc7:: with SMTP id b190mr8112989wmg.13.1547645424350; Wed, 16 Jan 2019 05:30:24 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id 10sm44283964wmy.40.2019.01.16.05.30.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:30:22 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v5 8/9] commit-graph write: add itermediate progress Date: Wed, 16 Jan 2019 14:29:46 +0100 Message-Id: <20190116132947.26025-9-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20181122153922.16912-2-avarab@gmail.com> References: <20181122153922.16912-2-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add progress output to sections of code between "Annotating[...]" and "Computing[...]generation numbers". This can collectively take 5-10 seconds on a large enough repository. On a test repository with I have with ~7 million commits and ~50 million objects we'll now emit: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (124763727/124763727), done. Loading known commits in commit graph: 100% (18989461/18989461), done. Expanding reachable commits in commit graph: 100% (18989507/18989461), done. Clearing commit marks in commit graph: 100% (18989507/18989507), done. Counting distinct commits in commit graph: 100% (18989507/18989507), done. Finding extra edges in commit graph: 100% (18989507/18989507), done. Computing commit graph generation numbers: 100% (7250302/7250302), done. Writing out commit graph in 4 passes: 100% (29001208/29001208), done. Whereas on a medium-sized repository such as linux.git these new progress bars won't have time to kick in and as before and we'll still emit output like: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (6529159/6529159), done. Expanding reachable commits in commit graph: 815990, done. Computing commit graph generation numbers: 100% (815983/815983), done. Writing out commit graph in 4 passes: 100% (3263932/3263932), done. The "Counting distinct commits in commit graph" phase will spend most of its time paused at "0/*" as we QSORT(...) the list. That's not optimal, but at least we don't seem to be stalling anymore most of the time. Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/commit-graph.c b/commit-graph.c index 7138bbd02f..60ec7c1b22 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -893,12 +893,19 @@ void write_commit_graph(const char *obj_dir, close_reachable(&oids, report_progress); + if (report_progress) + progress = start_delayed_progress( + _("Counting distinct commits in commit graph"), + oids.nr); + display_progress(progress, 0); /* TODO: Measure QSORT() progress */ QSORT(oids.list, oids.nr, commit_compare); count_distinct = 1; for (i = 1; i < oids.nr; i++) { + display_progress(progress, i + 1); if (!oideq(&oids.list[i - 1], &oids.list[i])) count_distinct++; } + stop_progress(&progress); if (count_distinct >= GRAPH_PARENT_MISSING) die(_("the commit graph format cannot write %d commits"), count_distinct); @@ -908,8 +915,13 @@ void write_commit_graph(const char *obj_dir, ALLOC_ARRAY(commits.list, commits.alloc); num_large_edges = 0; + if (report_progress) + progress = start_delayed_progress( + _("Finding extra edges in commit graph"), + oids.nr); for (i = 0; i < oids.nr; i++) { int num_parents = 0; + display_progress(progress, i + 1); if (i > 0 && oideq(&oids.list[i - 1], &oids.list[i])) continue; @@ -926,6 +938,7 @@ void write_commit_graph(const char *obj_dir, commits.nr++; } num_chunks = num_large_edges ? 4 : 3; + stop_progress(&progress); if (commits.nr >= GRAPH_PARENT_MISSING) die(_("too many commits to write graph")); From patchwork Wed Jan 16 13:29:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10765929 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 09A8A1390 for ; Wed, 16 Jan 2019 13:30:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EEA892D699 for ; Wed, 16 Jan 2019 13:30:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E21BF2DA0E; Wed, 16 Jan 2019 13:30:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 719C12D699 for ; Wed, 16 Jan 2019 13:30:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404314AbfAPNaa (ORCPT ); Wed, 16 Jan 2019 08:30:30 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:39384 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404304AbfAPNa2 (ORCPT ); Wed, 16 Jan 2019 08:30:28 -0500 Received: by mail-wm1-f68.google.com with SMTP id y8so1983831wmi.4 for ; Wed, 16 Jan 2019 05:30:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RZWNfRTgp+m/ej482CEXCeyP8ot1hSOx4QEB+D6QxsA=; b=H526F/b2BkNQGpKCud7jqr45qmuYE22I3zwKF2pzIGKiOVpjToOBWxJM/qJjm/rp7B BOqamg6nWXyjRzuoM824dLG5IizQwm617F31SF9TMA59AZsGbdSbNouk9f34en71bpwa 8y9Y8/0DLt++0Qea8Fojsy/pxiS5KHR3mfF0s6A+Fsg+wDCDKCs8hz77xd9hvMsm/ztg l6xgJeNrlWbklSXbT7V/IbP36pie4OsQltYt2QfBa2W3seZPrWCy4mFSaaMfbC23kWHv S+I7o55pVom+54X5l/5Zw0Wt4Ys5AjNt3mTdDo5owv1KsSGazm7g4rW8lzyduG5uwDv7 oFaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RZWNfRTgp+m/ej482CEXCeyP8ot1hSOx4QEB+D6QxsA=; b=Yu2ACp9S4H3haqnjKaLsRmgpyTkNXhWdKOlfWZXjkjO6W4Bf5BGs2IgNbPSrLlXQ2m LwTzXdPa1a70T2F7gkGbHZE25oWQm+UJFhNFJBbD5xkIfoVan9tp/mWZe77/2xVmAu1N aLsYwnbXotxLAnk9wOOc9jLvlXl1ZHIjUkYh3/dfr4xXsVnvk3BXEdZ3Q3c+T1+lrgW8 fER3WstZcj4c9XAiSZHdB/1+45hTX33iViaEZiqevLYThSWjjBQq/KL4xwmAb7uO4HfB oMOYifBZ0vI/x3Dl76NtxzuOiWzBGLeVVfUpI4VOVPpvMkmd0rvFKt5h1dnXoHLWiUhq L+GQ== X-Gm-Message-State: AJcUukfMHwl9r43eVfyHc4O+S6Rxpj7kPHVR4WE/VtFfJx3GS0YhWdsi UOe3vIZcBu91bhvfgDPTA8pDF5qFmhc= X-Google-Smtp-Source: ALg8bN5jBAbse1K9vjnvieaLsDkssOPbI3mfLMSQGVOGir/GJzfFUvn/ecD9xPGcWZ38IovWbLxUYg== X-Received: by 2002:a1c:8d53:: with SMTP id p80mr8024260wmd.68.1547645426336; Wed, 16 Jan 2019 05:30:26 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id 10sm44283964wmy.40.2019.01.16.05.30.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:30:25 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= , =?utf-8?q?SZEDER_G=C3=A1bor?= , Eric Sunshine , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v5 9/9] commit-graph write: emit a percentage for all progress Date: Wed, 16 Jan 2019 14:29:47 +0100 Message-Id: <20190116132947.26025-10-avarab@gmail.com> X-Mailer: git-send-email 2.20.1.153.gd81d796ee0 In-Reply-To: <20181122153922.16912-2-avarab@gmail.com> References: <20181122153922.16912-2-avarab@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Follow-up 01ca387774 ("commit-graph: split up close_reachable() progress output", 2018-11-19) by making the progress bars in close_reachable() report a completion percentage. This fixes the last occurrence where in the commit graph writing where we didn't report that. The change in 01ca387774 split up the 1x progress bar in close_reachable() into 3x, but left them as dumb counters without a percentage completion. Fixing that is easy, and the only reason it wasn't done already is because that commit was rushed in during the v2.20.0 RC period to fix the unrelated issue of over-reporting commit numbers. See [1] and follow-ups for ML activity at the time and [2] for an alternative approach where the progress bars weren't split up. Now for e.g. linux.git we'll emit: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (6529159/6529159), done. Expanding reachable commits in commit graph: 100% (815990/815980), done. Computing commit graph generation numbers: 100% (815983/815983), done. Writing out commit graph in 4 passes: 100% (3263932/3263932), done. 1. https://public-inbox.org/git/20181119202300.18670-1-avarab@gmail.com/ 2. https://public-inbox.org/git/20181122153922.16912-11-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- commit-graph.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 60ec7c1b22..b6a074c80d 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -653,15 +653,15 @@ static void add_missing_parents(struct packed_oid_list *oids, struct commit *com static void close_reachable(struct packed_oid_list *oids, int report_progress) { - int i, j; + int i; struct commit *commit; struct progress *progress = NULL; if (report_progress) progress = start_delayed_progress( - _("Loading known commits in commit graph"), j = 0); + _("Loading known commits in commit graph"), oids->nr); for (i = 0; i < oids->nr; i++) { - display_progress(progress, ++j); + display_progress(progress, i + 1); commit = lookup_commit(the_repository, &oids->list[i]); if (commit) commit->object.flags |= UNINTERESTING; @@ -675,9 +675,9 @@ static void close_reachable(struct packed_oid_list *oids, int report_progress) */ if (report_progress) progress = start_delayed_progress( - _("Expanding reachable commits in commit graph"), j = 0); + _("Expanding reachable commits in commit graph"), oids->nr); for (i = 0; i < oids->nr; i++) { - display_progress(progress, ++j); + display_progress(progress, i + 1); commit = lookup_commit(the_repository, &oids->list[i]); if (commit && !parse_commit(commit)) @@ -687,9 +687,9 @@ static void close_reachable(struct packed_oid_list *oids, int report_progress) if (report_progress) progress = start_delayed_progress( - _("Clearing commit marks in commit graph"), j = 0); + _("Clearing commit marks in commit graph"), oids->nr); for (i = 0; i < oids->nr; i++) { - display_progress(progress, ++j); + display_progress(progress, i + 1); commit = lookup_commit(the_repository, &oids->list[i]); if (commit)