From patchwork Thu Nov 10 01:57:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13038240 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1920DC433FE for ; Thu, 10 Nov 2022 01:58:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232489AbiKJB6U (ORCPT ); Wed, 9 Nov 2022 20:58:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232442AbiKJB5u (ORCPT ); Wed, 9 Nov 2022 20:57:50 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7DB7F2DAA4 for ; Wed, 9 Nov 2022 17:57:22 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id a14so320876wru.5 for ; Wed, 09 Nov 2022 17:57:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=VAYMusBKLV09FtvzEd0S4SeLiMFksBaCYaRVdFfaGk4=; b=L5He0z0UFvp2Wyj84gO0/3vW2+6DZFVoXcJPNmpxm3EdKecvlaLi1zltpiDWKFnH97 D/WFAAfgtlCjn4Fq0rN56oQTmW/yiZ10dSX4uPK8DCFCpJk9c1tWtv24CCaSJ5cGp/eV 22fcs8mc5YtJIEkY7grZrKmnwI4LcvABijFHEgIJF6RpFClyNFbMzlXfEJM8E6idh54V 8h6c73uoyrd7FWY/3zuZYSFQRYR7ooXloMAJnBIXv4VWt82rmKwO9HI3e2rOfKv7XpNj 3UexgD5PlA1JfHx7cuqsjO9dHFRS9Xz+ZAzQ3jvJfnaTiUAXE5JHkJQOGFjlnVJibBAP an4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VAYMusBKLV09FtvzEd0S4SeLiMFksBaCYaRVdFfaGk4=; b=BLcaIoPpKqKGRG7DMujPp8Qr897de/UzJ4Psa5DDLG9K9F8SzXk0dKR90n+zjSRp0s x+Q0qCOl34kLiXiPpYTPxdVAojcr8OLL1leweIFtBCVQXr1mAuG9+f3NgQxwqiINKCs/ tpuPCgrHQbSUdfdrIzmpp3qIeQyqRe6PK9TRFQrHyzyMDpNCMg09CsoStQCOxlRhOx96 69ryWsu7QSbQUUj72bo9LS+WzwiPnm0VQxXdEtpr+1ID+Ne/A1mEOV2aLZK55DuXNL9P CxciRs4xlP53AAyNGqTPp/zNxSApJDgvEVOPEsbp7RPkrNsMCXWTtik1QdozsK4d+Q4O MotA== X-Gm-Message-State: ACrzQf3rRkzBOZAbRt86AFED/CPXBbZfMh8PSVn/tZg/kgCwMP+gW0ti TcZ6nn7wKs6giD2Bmjqt13f4VH9++04= X-Google-Smtp-Source: AMsMyM6GPJ7NK0h+tBzkDWo8wOEKglh1GEBEaMp8Ai8dhX5f7SsNYaZ96POc4O1vP+2WSP8PkkygFQ== X-Received: by 2002:a05:6000:22e:b0:236:6e38:4d9b with SMTP id l14-20020a056000022e00b002366e384d9bmr41486211wrz.4.1668045440785; Wed, 09 Nov 2022 17:57:20 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s12-20020adfeb0c000000b0023657e1b980sm14528390wrn.53.2022.11.09.17.57.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Nov 2022 17:57:20 -0800 (PST) Message-Id: <833519d87c843eb8147a45b1ec2c1fd3f3c21905.1668045438.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 10 Nov 2022 01:57:13 +0000 Subject: [PATCH v2 1/5] cache-tree: add perf test comparing update and prime Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, phillip.wood123@gmail.com, derrickstolee@github.com, jonathantanmy@google.com, Taylor Blau , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Add a performance test comparing the execution times of 'prime_cache_tree()' and 'cache_tree_update(_, WRITE_TREE_SILENT | WRITE_TREE_REPAIR)'. The goal of comparing these two is to identify which is the faster method for rebuilding an invalid cache tree, ultimately to remove one when both are (reundantly) called in immediate succession. Both methods are fast, so the new tests in 'p0090-cache-tree.sh' must call each tested function multiple times to ensure the reported times (to 0.01s resolution) convey the differences between them. The tests compare the timing of a 'test-tool cache-tree' run as a no-op (to capture a baseline for the overhead associated with running the tool), 'cache_tree_update()', and 'prime_cache_tree()' on four scenarios: - A completely valid cache tree - A cache tree with 2 invalid paths - A cache tree with 50 invalid paths - A completely empty cache tree Example results: Test this tree ----------------------------------------------------------- 0090.2: no-op, clean 1.27(0.48+0.52) 0090.3: prime_cache_tree, clean 2.02(0.83+0.85) 0090.4: cache_tree_update, clean 1.30(0.49+0.54) 0090.5: no-op, invalidate 2 1.29(0.48+0.54) 0090.6: prime_cache_tree, invalidate 2 1.98(0.81+0.83) 0090.7: cache_tree_update, invalidate 2 2.12(0.94+0.86) 0090.8: no-op, invalidate 50 1.32(0.50+0.55) 0090.9: prime_cache_tree, invalidate 50 2.10(0.86+0.89) 0090.10: cache_tree_update, invalidate 50 2.35(1.14+0.90) 0090.11: no-op, empty 1.33(0.50+0.54) 0090.12: prime_cache_tree, empty 2.04(0.84+0.87) 0090.13: cache_tree_update, empty 2.51(1.27+0.92) These timings show that, while 'cache_tree_update()' is faster when the cache tree is completely valid, it is equal to or slower than 'prime_cache_tree()' when there are any invalid paths. Since the redundant calls are mostly in scenarios where the cache tree will be at least partially invalid (e.g., 'git reset --hard'), 'prime_cache_tree()' will likely perform better than 'cache_tree_update()' in typical cases. Signed-off-by: Victoria Dye --- Makefile | 1 + t/helper/test-cache-tree.c | 64 ++++++++++++++++++++++++++++++++++++++ t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/perf/p0090-cache-tree.sh | 36 +++++++++++++++++++++ 5 files changed, 103 insertions(+) create mode 100644 t/helper/test-cache-tree.c create mode 100755 t/perf/p0090-cache-tree.sh diff --git a/Makefile b/Makefile index 4927379184c..3639c7c2a94 100644 --- a/Makefile +++ b/Makefile @@ -723,6 +723,7 @@ TEST_BUILTINS_OBJS += test-advise.o TEST_BUILTINS_OBJS += test-bitmap.o TEST_BUILTINS_OBJS += test-bloom.o TEST_BUILTINS_OBJS += test-bundle-uri.o +TEST_BUILTINS_OBJS += test-cache-tree.o TEST_BUILTINS_OBJS += test-chmtime.o TEST_BUILTINS_OBJS += test-config.o TEST_BUILTINS_OBJS += test-crontab.o diff --git a/t/helper/test-cache-tree.c b/t/helper/test-cache-tree.c new file mode 100644 index 00000000000..8d06039fb5c --- /dev/null +++ b/t/helper/test-cache-tree.c @@ -0,0 +1,64 @@ +#include "test-tool.h" +#include "cache.h" +#include "tree.h" +#include "cache-tree.h" +#include "parse-options.h" + +static char const * const test_cache_tree_usage[] = { + N_("test-tool cache-tree (control|prime|update)"), + NULL +}; + +int cmd__cache_tree(int argc, const char **argv) +{ + struct object_id oid; + struct tree *tree; + int empty = 0; + int invalidate_qty = 0; + int i; + + struct option options[] = { + OPT_BOOL(0, "empty", &empty, + N_("clear the cache tree before each iteration")), + OPT_INTEGER_F(0, "invalidate", &invalidate_qty, + N_("number of entries in the cache tree to invalidate (default 0)"), + PARSE_OPT_NONEG), + OPT_END() + }; + + setup_git_directory(); + + parse_options(argc, argv, NULL, options, test_cache_tree_usage, 0); + + if (read_cache() < 0) + die("unable to read index file"); + + oidcpy(&oid, &the_index.cache_tree->oid); + tree = parse_tree_indirect(&oid); + if (!tree) + die(_("not a tree object: %s"), oid_to_hex(&oid)); + + if (empty) { + /* clear the cache tree & allocate a new one */ + cache_tree_free(&the_index.cache_tree); + the_index.cache_tree = cache_tree(); + } else if (invalidate_qty) { + /* invalidate the specified number of unique paths */ + float f_interval = (float)the_index.cache_nr / invalidate_qty; + int interval = f_interval < 1.0 ? 1 : (int)f_interval; + for (i = 0; i < invalidate_qty && i * interval < the_index.cache_nr; i++) + cache_tree_invalidate_path(&the_index, the_index.cache[i * interval]->name); + } + + if (!argc) + die("Must specify subcommand"); + else if (!strcmp(argv[0], "prime")) + prime_cache_tree(the_repository, &the_index, tree); + else if (!strcmp(argv[0], "update")) + cache_tree_update(&the_index, WRITE_TREE_SILENT | WRITE_TREE_REPAIR); + /* use "control" subcommand to specify no-op */ + else if (!!strcmp(argv[0], "control")) + die("Unknown command %s", argv[0]); + + return 0; +} diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index 01cda9358df..547a3be1c8b 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -14,6 +14,7 @@ static struct test_cmd cmds[] = { { "bitmap", cmd__bitmap }, { "bloom", cmd__bloom }, { "bundle-uri", cmd__bundle_uri }, + { "cache-tree", cmd__cache_tree }, { "chmtime", cmd__chmtime }, { "config", cmd__config }, { "crontab", cmd__crontab }, diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h index ca2948066fd..e44e1d896d3 100644 --- a/t/helper/test-tool.h +++ b/t/helper/test-tool.h @@ -8,6 +8,7 @@ int cmd__advise_if_enabled(int argc, const char **argv); int cmd__bitmap(int argc, const char **argv); int cmd__bloom(int argc, const char **argv); int cmd__bundle_uri(int argc, const char **argv); +int cmd__cache_tree(int argc, const char **argv); int cmd__chmtime(int argc, const char **argv); int cmd__config(int argc, const char **argv); int cmd__crontab(int argc, const char **argv); diff --git a/t/perf/p0090-cache-tree.sh b/t/perf/p0090-cache-tree.sh new file mode 100755 index 00000000000..a8eabca2c4d --- /dev/null +++ b/t/perf/p0090-cache-tree.sh @@ -0,0 +1,36 @@ +#!/bin/sh + +test_description="Tests performance of cache tree update operations" + +. ./perf-lib.sh + +test_perf_large_repo +test_checkout_worktree + +count=100 + +test_expect_success 'setup cache tree' ' + git write-tree +' + +test_cache_tree () { + test_perf "$1, $3" " + for i in \$(test_seq $count) + do + test-tool cache-tree $4 $2 + done + " +} + +test_cache_tree_update_functions () { + test_cache_tree 'no-op' 'control' "$1" "$2" + test_cache_tree 'prime_cache_tree' 'prime' "$1" "$2" + test_cache_tree 'cache_tree_update' 'update' "$1" "$2" +} + +test_cache_tree_update_functions "clean" "" +test_cache_tree_update_functions "invalidate 2" "--invalidate 2" +test_cache_tree_update_functions "invalidate 50" "--invalidate 50" +test_cache_tree_update_functions "empty" "--empty" + +test_done From patchwork Thu Nov 10 01:57:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13038239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A126C433FE for ; Thu, 10 Nov 2022 01:58:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232408AbiKJB6P (ORCPT ); Wed, 9 Nov 2022 20:58:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51560 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232186AbiKJB5u (ORCPT ); Wed, 9 Nov 2022 20:57:50 -0500 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 506892E68F for ; Wed, 9 Nov 2022 17:57:23 -0800 (PST) Received: by mail-wr1-x435.google.com with SMTP id cl5so301311wrb.9 for ; Wed, 09 Nov 2022 17:57:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=y/zicVtu+r7gO5FwNJ/+MUMzuPbt221qJMX9gz8tkU0=; b=Mlf/LMV9cGa8eOzsIL1+z5TB+5bU8GWETmSvyp7RqBi4ryl4S7ZtgXtf4eMUltEA3G Sgct+IxQ7Mvl2jLCmMZ7mcvddOkbYCM+iOmiQhwRSyuMHxB0idxqAvZOQ02N7sr4DvhP xgmJqN5fPCcWsKw4RaBAJHO5a57FGylp/LaJNR160BN2khXDXjyBTJBaCQYMVFz/ZyS3 ZScuQRN+DR46BU9zKy/UPc+SMCJWppNEg04ElwIwyJFNHuC/KdvuNwBk43O6Y/FtKuO5 6s042rQ3c9gNjm3GVNAb6QIeBw0teLTN8eG9kO5lHSYQdfd/lSsP1HLYgVzgVwn0By7Y 2YoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y/zicVtu+r7gO5FwNJ/+MUMzuPbt221qJMX9gz8tkU0=; b=ATEqEFu+wp0nFuTJghXnS6kbDA6VtRVjZNJyteCyQ6OV4edIyJImtfcItuK19qG1MG 4knI4kZ82AV8+CwB3wrtmyyyv1/eM1MsYGX+QUK9HZ8oyRrvip4VZbSIYfpJBDNSFXxc SPo64ogBu+H6AkHLVrfmQpka1UKwaebv3+OKxK72x4ZKSbJBuP/VW++V7ROF1HwrGzQN y1b0DRO5zPquHpWtHINu/UmJnkDoJ6rjQrY9zkaQ1vyH8h44qcqKJT9sAWXBTVgpYk6W UIX5F8pz418SMh08PB83I+lO9NjgLWtL1eamOHjBrVrX+erkYNMnNVzSQWklEvZm563A sfuA== X-Gm-Message-State: ACrzQf3f6r42MRT78UFMl7Bvb+MYmGEcXpGch+4XwiX+MVbyvA1VXZIy VwwjXZAx+kt86mB0HzEwraNE3WTZNn4= X-Google-Smtp-Source: AMsMyM4nX14oiPEEvx7/IBSmmVT7cX6MKyEU3JVXTRVVwB2w3V8J6p7+WsbYoJxtlRamnY11rB7cHA== X-Received: by 2002:a5d:64cd:0:b0:236:6d1c:c1a2 with SMTP id f13-20020a5d64cd000000b002366d1cc1a2mr41676652wri.360.1668045441641; Wed, 09 Nov 2022 17:57:21 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f17-20020a056000129100b002368a6deaf8sm14584531wrx.57.2022.11.09.17.57.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Nov 2022 17:57:21 -0800 (PST) Message-Id: In-Reply-To: References: Date: Thu, 10 Nov 2022 01:57:14 +0000 Subject: [PATCH v2 2/5] unpack-trees: add 'skip_cache_tree_update' option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, phillip.wood123@gmail.com, derrickstolee@github.com, jonathantanmy@google.com, Taylor Blau , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Add (disabled by default) option to skip the 'cache_tree_update()' at the end of 'unpack_trees()'. In many cases, this cache tree update is redundant because the caller of 'unpack_trees()' immediately follows it with 'prime_cache_tree()', rebuilding the entire cache tree from scratch. While these operations aren't the most expensive part of operations like 'git reset', the duplicate calls still create a minor unnecessary slowdown. Introduce an option for callers to skip the 'cache_tree_update()' in 'unpack_trees()' if it is redundant (that is, if 'prime_cache_tree()' is called afterwards). At the moment, no 'unpack_trees()' callers use the new option; they will be updated in subsequent patches. Signed-off-by: Victoria Dye --- unpack-trees.c | 3 ++- unpack-trees.h | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/unpack-trees.c b/unpack-trees.c index bae812156c4..8a762aa0772 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -2043,7 +2043,8 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options if (!ret) { if (git_env_bool("GIT_TEST_CHECK_CACHE_TREE", 0)) cache_tree_verify(the_repository, &o->result); - if (!cache_tree_fully_valid(o->result.cache_tree)) + if (!o->skip_cache_tree_update && + !cache_tree_fully_valid(o->result.cache_tree)) cache_tree_update(&o->result, WRITE_TREE_SILENT | WRITE_TREE_REPAIR); diff --git a/unpack-trees.h b/unpack-trees.h index efb9edfbb27..6ab0d74c84d 100644 --- a/unpack-trees.h +++ b/unpack-trees.h @@ -71,7 +71,8 @@ struct unpack_trees_options { quiet, exiting_early, show_all_errors, - dry_run; + dry_run, + skip_cache_tree_update; enum unpack_trees_reset_type reset; const char *prefix; int cache_bottom; From patchwork Thu Nov 10 01:57:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13038238 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35730C43219 for ; Thu, 10 Nov 2022 01:58:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232356AbiKJB6N (ORCPT ); Wed, 9 Nov 2022 20:58:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232193AbiKJB5u (ORCPT ); Wed, 9 Nov 2022 20:57:50 -0500 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FBF02A976 for ; Wed, 9 Nov 2022 17:57:24 -0800 (PST) Received: by mail-wm1-x32b.google.com with SMTP id fn7-20020a05600c688700b003b4fb113b86so260732wmb.0 for ; Wed, 09 Nov 2022 17:57:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=Jtq3iPEAnMV1G7hru853ch69TOxtztyX7mt4g8SPMWI=; b=qJMzFY+Ianc8yNMNfnZydV6W155n0L4BjuEg6cbv5XiRkaJ7h+1Df624RiwdPMhEcF prIaMgP5N69qYJZcchBzk5Wee2fKDGaQizDP2N64WO4ADrcWCMKwV1tUv12rgiWu6z1c jfGgi1iH1nnp8Uifekx3u4JOZMToa7Zk2NumQCBrXe4iGTw6j3mf+nJxKApY043NOU9k Fma5XQOAHt8RHpJTxoOgNsjFdAJyGqU06MqNuci5+AqlKXFRy+l5C+q2LPXSpZfnBuzv U5+WBs8ZI0v1rlRDxM1ILKb0WSMXvzDXIQh28lQbc8qf4zODbAoiJ+ec5f60emOY5i4H Ra4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jtq3iPEAnMV1G7hru853ch69TOxtztyX7mt4g8SPMWI=; b=p/hrabtPw4y2ATKlPScY+7s8AOpv2EGdOo7E8QBeqytR59zSseXI+qj+BFqnL0bRj/ cikSiyg41Cj+n27vglABEngDVHcRE9applSh9I5PpIeLgJBDFiL0TAAwCzLOmnZrwUvD ma3FPFJG2kpLpATdI/bNyq/SJqErraKAeDVthJ9pjtIdlH7q7Y1mSrzIIEHKvmfSBS5Q co30Kn28aXgOYV9zkB2OA7E9ms/t2fJneFt3jbjbNMC1rjX46os6dZZM2n2uHdGfWkA4 aU0QC0vG9AspZsvlIQbmVg5iilrvLp/OfAnP1Z1tv28ldqn5uPZwrNFpVYjYyw0Q6pYh 3ojw== X-Gm-Message-State: ACrzQf0msv/Rtd/OlE4JI/jAk3SfxnMApNFX5rQ+3QzLDOOWIU08tupC pEd5UWblH/4hflga7BjR2S5idcwVa7A= X-Google-Smtp-Source: AMsMyM44ORO3OCPrSou4dRB7tHeHnMHGBUJw26WkvRjpeX7fR8v9aaNe7O6GGYKlMGmnWORrjv6a/A== X-Received: by 2002:a05:600c:19d1:b0:3c7:b52:3aa4 with SMTP id u17-20020a05600c19d100b003c70b523aa4mr52856990wmq.56.1668045442538; Wed, 09 Nov 2022 17:57:22 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 26-20020a05600c029a00b003cf5ec79bf9sm3321333wmk.40.2022.11.09.17.57.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Nov 2022 17:57:22 -0800 (PST) Message-Id: <4f6039971b87d84b1f44d8da4cd106777cc1c38f.1668045438.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 10 Nov 2022 01:57:15 +0000 Subject: [PATCH v2 3/5] reset: use 'skip_cache_tree_update' option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, phillip.wood123@gmail.com, derrickstolee@github.com, jonathantanmy@google.com, Taylor Blau , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Enable the 'skip_cache_tree_update' option in the variants that call 'prime_cache_tree()' after 'unpack_trees()' (specifically, 'git reset --mixed' and 'git reset --hard'). This avoids redundantly rebuilding the cache tree in both 'cache_tree_update()' at the end of 'unpack_trees()' and in 'prime_cache_tree()', resulting in a small (but consistent) performance improvement. From the newly-added 'p7102-reset.sh' test: Test before after -------------------------------------------------------------------- 7102.1: reset --hard (...) 2.11(0.40+1.54) 1.97(0.38+1.47) -6.6% Signed-off-by: Victoria Dye --- builtin/reset.c | 2 ++ t/perf/p7102-reset.sh | 21 +++++++++++++++++++++ 2 files changed, 23 insertions(+) create mode 100755 t/perf/p7102-reset.sh diff --git a/builtin/reset.c b/builtin/reset.c index fdce6f8c856..ab027774824 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -73,9 +73,11 @@ static int reset_index(const char *ref, const struct object_id *oid, int reset_t case HARD: opts.update = 1; opts.reset = UNPACK_RESET_OVERWRITE_UNTRACKED; + opts.skip_cache_tree_update = 1; break; case MIXED: opts.reset = UNPACK_RESET_PROTECT_UNTRACKED; + opts.skip_cache_tree_update = 1; /* but opts.update=0, so working tree not updated */ break; default: diff --git a/t/perf/p7102-reset.sh b/t/perf/p7102-reset.sh new file mode 100755 index 00000000000..9b039e8691f --- /dev/null +++ b/t/perf/p7102-reset.sh @@ -0,0 +1,21 @@ +#!/bin/sh + +test_description='performance of reset' +. ./perf-lib.sh + +test_perf_default_repo +test_checkout_worktree + +test_perf 'reset --hard with change in tree' ' + base=$(git rev-parse HEAD) && + test_commit --no-tag A && + new=$(git rev-parse HEAD) && + + for i in $(test_seq 10) + do + git reset --hard $new && + git reset --hard $base || return $? + done +' + +test_done From patchwork Thu Nov 10 01:57:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13038242 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90DF3C43217 for ; Thu, 10 Nov 2022 01:58:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232507AbiKJB6X (ORCPT ); Wed, 9 Nov 2022 20:58:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50264 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232448AbiKJB5y (ORCPT ); Wed, 9 Nov 2022 20:57:54 -0500 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EDA652E697 for ; Wed, 9 Nov 2022 17:57:24 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id o4so317042wrq.6 for ; Wed, 09 Nov 2022 17:57:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=Y9D6pDzcfKa+C9JuV33FlqQspXTfBgY5IZLMOQAlznk=; b=FkAb2pCEAjiGtRjg44dJxCWmMZY5eliUBZ3HC/7dGkKwM7CaZMXYQknRIAl7A0Plty 6b0wZhwar05NKTZn9ujLhRdp41vWDaQaZImk4XdHnA2zIXSjPx/eim0hCpeMIm3Axe7Y AUs90wymtdMI5R3eyTW5mKHyccmjn1jwSlVXiGsQmsTqV4b0B6eqOHVXbsBIod+k/XQW YguhlSV20FpjNgvTO4Rs2CgPD/BB4a9GUddAEj14WnLPySbVhF6yf9aEZwaTRn9Z5+ex +wydSmNIjutDqBEZ0T5A3Z1diCbbERjmV9qonVC5uD84vuInTrTlKnfLy/t2XYsRx0Zm qyOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Y9D6pDzcfKa+C9JuV33FlqQspXTfBgY5IZLMOQAlznk=; b=U6NdBhPHxMY+gjkftNnNKNWUJb8rZH9EGz0whTdsGcXoPtLlsKzBdtAnNCPoE/Hj3j VsRh4HrhA2lyvI+LR+0N9w9jXRblnKWM/WmU5aSiRrfc32ob3Y+Zd4DncafpQBRPURd4 +py3i5tUeDbtEGr6QY51vuXanvknB6bHj3Rh/0h1YVAlc44bYmktGOgqIpKJV/11LppM yzek7M2MJQ2kHzBgx0f7pHLN1+ZDcXFS78lWFndDNCPUZjMitg2jQFqt8nm9NCX9bLhx jnIbslgTHZ7yFf3JTEWTsDaWhVVRJGh3rfa/cEMYxdtiya4H/NQlEfoqsydZySGSzMeG NVew== X-Gm-Message-State: ACrzQf1EkZyJcd9YEkEJKgCj+7BXV9YECM3BUe+cv5NclE/2mTb1gjmT qEH5Vd0cPxvaCn3qVQl4YNlqs/55COo= X-Google-Smtp-Source: AMsMyM4nlvLCXJtp84lG9CbcruN3B/BSYWe0XjVPqFmZAyl3VPUygEO2f2jqe1644OljSfKQofjlUw== X-Received: by 2002:a5d:4692:0:b0:235:b188:3ec with SMTP id u18-20020a5d4692000000b00235b18803ecmr40282958wrq.210.1668045443339; Wed, 09 Nov 2022 17:57:23 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g9-20020a05600c310900b003a2f2bb72d5sm4137248wmo.45.2022.11.09.17.57.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Nov 2022 17:57:22 -0800 (PST) Message-Id: <5a646bc47c911bb6b58e00574ac30afab1eec00b.1668045438.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 10 Nov 2022 01:57:16 +0000 Subject: [PATCH v2 4/5] read-tree: use 'skip_cache_tree_update' option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, phillip.wood123@gmail.com, derrickstolee@github.com, jonathantanmy@google.com, Taylor Blau , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye When running 'read-tree' with a single tree and no prefix, 'prime_cache_tree()' is called after the tree is unpacked. In that situation, skip a redundant call to 'cache_tree_update()' in 'unpack_trees()' by enabling the 'skip_cache_tree_update' unpack option. Removing the redundant cache tree update provides a substantial performance improvement to 'git read-tree ', as shown by a test added to 'p0006-read-tree-checkout.sh': Test before after ---------------------------------------------------------------------- read-tree br_ballast_plus_1 3.94(1.80+1.57) 3.00(1.14+1.28) -23.9% Note that the 'read-tree' in 't1022-read-tree-partial-clone.sh' is updated to read two trees, rather than one. The test was first introduced in d3da223f221 (cache-tree: prefetch in partial clone read-tree, 2021-07-23) to exercise the 'cache_tree_update()' code path, as used in 'git merge'. Since this patch drops the call to 'cache_tree_update()' in single-tree 'git read-tree', change the test to use the two-tree variant so that 'cache_tree_update()' is called as intended. Signed-off-by: Victoria Dye --- builtin/read-tree.c | 4 ++++ t/perf/p0006-read-tree-checkout.sh | 8 ++++++++ t/t1022-read-tree-partial-clone.sh | 2 +- 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/builtin/read-tree.c b/builtin/read-tree.c index f4cbe460b97..45c6652444b 100644 --- a/builtin/read-tree.c +++ b/builtin/read-tree.c @@ -249,6 +249,10 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix) if (opts.debug_unpack) opts.fn = debug_merge; + /* If we're going to prime_cache_tree later, skip cache tree update */ + if (nr_trees == 1 && !opts.prefix) + opts.skip_cache_tree_update = 1; + cache_tree_free(&active_cache_tree); for (i = 0; i < nr_trees; i++) { struct tree *tree = trees[i]; diff --git a/t/perf/p0006-read-tree-checkout.sh b/t/perf/p0006-read-tree-checkout.sh index c481c012d2f..325566e18eb 100755 --- a/t/perf/p0006-read-tree-checkout.sh +++ b/t/perf/p0006-read-tree-checkout.sh @@ -49,6 +49,14 @@ test_perf "read-tree br_base br_ballast ($nr_files)" ' git read-tree -n -m br_base br_ballast ' +test_perf "read-tree br_ballast_plus_1 ($nr_files)" ' + # Run read-tree 100 times for clearer performance results & comparisons + for i in $(test_seq 100) + do + git read-tree -n -m br_ballast_plus_1 || return 1 + done +' + test_perf "switch between br_base br_ballast ($nr_files)" ' git checkout -q br_base && git checkout -q br_ballast diff --git a/t/t1022-read-tree-partial-clone.sh b/t/t1022-read-tree-partial-clone.sh index a9953b6a71c..da539716359 100755 --- a/t/t1022-read-tree-partial-clone.sh +++ b/t/t1022-read-tree-partial-clone.sh @@ -19,7 +19,7 @@ test_expect_success 'read-tree in partial clone prefetches in one batch' ' git -C server config uploadpack.allowfilter 1 && git -C server config uploadpack.allowanysha1inwant 1 && git clone --bare --filter=blob:none "file://$(pwd)/server" client && - GIT_TRACE_PACKET="$(pwd)/trace" git -C client read-tree $TREE && + GIT_TRACE_PACKET="$(pwd)/trace" git -C client read-tree $TREE $TREE && # "done" marks the end of negotiation (once per fetch). Expect that # only one fetch occurs. From patchwork Thu Nov 10 01:57:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13038241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AFFAC4332F for ; Thu, 10 Nov 2022 01:58:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232499AbiKJB6W (ORCPT ); Wed, 9 Nov 2022 20:58:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232447AbiKJB5v (ORCPT ); Wed, 9 Nov 2022 20:57:51 -0500 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8FCE2C65C for ; Wed, 9 Nov 2022 17:57:24 -0800 (PST) Received: by mail-wr1-x42d.google.com with SMTP id v1so292605wrt.11 for ; Wed, 09 Nov 2022 17:57:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=O/qBtdZ69IG5ihaw98dYPOr1oYuv5CWv1pqA2wxwrrQ=; b=TYZJL10xXQBWbr7sUaZ67YGVGyljG0cU/1Mbnd5XSnHUiozqbPrk7cypEeU9vyXKxh nVxOT2kyXvVjwF+/Cj4HbTcuy1wsz3w8RLLzFfPIAyHj8k0LWMDjHVYVR/H0Eig4ol5D Zf8MJWCYeeEH7u5hbrOfbb2AIEWSLSze+nRFWBOM4W5oXoCNC9qSegcZxSGb+DhiUs+K kI8MjtRMNmggFooRYGSB7L/nzWj3ySwgtKznJkrZq+lWhuoe1LjaDPksC3sg2wmOcF3I W0jMu5Gup2ckTmg8UJRw3S7wngJReOK1gP5stk9ND6UGkkigIWZ3V3b3vnZlwbOxjFUx BXwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O/qBtdZ69IG5ihaw98dYPOr1oYuv5CWv1pqA2wxwrrQ=; b=r9Dwpvfq8inZLm7ubnHmPtny8k6jVl2VwH9NfPIX+YCY5tmnrK3PP/c1Sp/E+kRv2f s5dJ0n628uHPyIh+y+K870h6a7S5IKVGj6zECDU2LwiQQ6KUgbURJSnRJ54J4CUogugZ wHX+3SJRm7iZZ8V3GH0IQNpVAsaaae/oIqtcwRxooTBuOKIbdMDjCzoNmdVTsCZBKsg7 j6Tg2F7widA1158z2pKv6OPItyXgmuEd5dolmksWJPQoDfjHjKAnP1luyOgjhxavEARn eIChxHR0x9b2HKnZuwWYYdKG9ptkwdQx+Qhtz7dj7/4yCbAk8z5AF/v7ry8fE6IAVEO5 Q1iw== X-Gm-Message-State: ACrzQf0gSrW2DmrU8eveW4W2IJljMNf+MrARku5ECTWkbyCBghRanCYf qh/sDqiKZuYc8XgYTE7nXSI0lrCKo3Y= X-Google-Smtp-Source: AMsMyM7j321hc07qraHP1L1KZ7qeMf/eGZ7Ms+DdVdn+KweapK+2U3DIEwmWu/FNHfCctSRvjm5B1g== X-Received: by 2002:a5d:5710:0:b0:236:cdd4:4cf4 with SMTP id a16-20020a5d5710000000b00236cdd44cf4mr35880548wrv.376.1668045444349; Wed, 09 Nov 2022 17:57:24 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m16-20020adfdc50000000b00236695ff94fsm14542578wrj.34.2022.11.09.17.57.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Nov 2022 17:57:23 -0800 (PST) Message-Id: In-Reply-To: References: Date: Thu, 10 Nov 2022 01:57:17 +0000 Subject: [PATCH v2 5/5] rebase: use 'skip_cache_tree_update' option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, phillip.wood123@gmail.com, derrickstolee@github.com, jonathantanmy@google.com, Taylor Blau , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Enable the 'skip_cache_tree_update' option in both 'do_reset()' ('sequencer.c') and 'reset_head()' ('reset.c'). Both of these callers invoke 'prime_cache_tree()' after 'unpack_trees()', so we can remove an unnecessary cache tree rebuild by skipping 'cache_tree_update()'. When testing with 'p3400-rebase.sh' and 'p3404-rebase-interactive.sh', the performance change of this update was negligible, likely due to the operation being dominated by more expensive operations (like checking out trees). However, since the change doesn't harm performance, it's worth keeping this 'unpack_trees()' usage consistent with others that subsequently invoke 'prime_cache_tree()'. Signed-off-by: Victoria Dye --- reset.c | 1 + sequencer.c | 1 + 2 files changed, 2 insertions(+) diff --git a/reset.c b/reset.c index e3383a93343..5ded23611f3 100644 --- a/reset.c +++ b/reset.c @@ -128,6 +128,7 @@ int reset_head(struct repository *r, const struct reset_head_opts *opts) unpack_tree_opts.update = 1; unpack_tree_opts.merge = 1; unpack_tree_opts.preserve_ignored = 0; /* FIXME: !overwrite_ignore */ + unpack_tree_opts.skip_cache_tree_update = 1; init_checkout_metadata(&unpack_tree_opts.meta, switch_to_branch, oid, NULL); if (reset_hard) unpack_tree_opts.reset = UNPACK_RESET_PROTECT_UNTRACKED; diff --git a/sequencer.c b/sequencer.c index e658df7e8ff..3f7a73ce4e1 100644 --- a/sequencer.c +++ b/sequencer.c @@ -3750,6 +3750,7 @@ static int do_reset(struct repository *r, unpack_tree_opts.merge = 1; unpack_tree_opts.update = 1; unpack_tree_opts.preserve_ignored = 0; /* FIXME: !overwrite_ignore */ + unpack_tree_opts.skip_cache_tree_update = 1; init_checkout_metadata(&unpack_tree_opts.meta, name, &oid, NULL); if (repo_read_index_unmerged(r)) {