From patchwork Tue Nov 8 22:44:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13036912 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F989C4332F for ; Tue, 8 Nov 2022 22:44:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229722AbiKHWoc (ORCPT ); Tue, 8 Nov 2022 17:44:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229695AbiKHWoa (ORCPT ); Tue, 8 Nov 2022 17:44:30 -0500 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1F07209A9 for ; Tue, 8 Nov 2022 14:44:28 -0800 (PST) Received: by mail-wr1-x432.google.com with SMTP id z14so23201244wrn.7 for ; Tue, 08 Nov 2022 14:44:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=tCdjUgTQDws1aUxV2xH9IPd/S2yz2kRQO7o60vBAWEM=; b=nwn7UsxeafoahtkKm6Qx1vbDtpUJvXrKXt2eVq2s3pu8/szXAPxmX789PcHZYYHbD2 fnO2O6Q76XlTOcBTMyJZONfxJ63WuoByshqxso53mavymhpHa5ISFlfS1PKh+MxrP6dB rhYgQ9rhhcY36vG+2O1+PdyqjMMO69H+8Yd94UsNPC95zeDuhtvL9rHLiI0CKoe0QT3y cLNA9WjVWeH/j6xZFqONDh675GbbjjEB60D3N+LG+iyhNWlBOsQz+D7luinLeH0JHyUu 2qrHrunSykn6uSa2kt6t70nOWlax0Nn/L4u2BbGKEESAYHjNQhaQX3f7HSiIR9xwftl/ RMsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tCdjUgTQDws1aUxV2xH9IPd/S2yz2kRQO7o60vBAWEM=; b=iGDOIJZMxkJ9kK42DBuralP3RivfwW7RQ2MUiWi3xfbSFb9hYUwB6p9RofXGMZQEKg 2K2Bxy3W17FQc5LkGzo7SZq83FC6duXDZvqdjBUVHmyz6xeuRonZ6BvoMdNEOA7wwN6u /5eLObUPJjx8v5oKWf01+H6Oea0MXlpD6sPV52yhHmSA3mdylobz3lnlR+Xqa2mY7v6a K/NdDtNm2EIywVeuw+Yp9zxaJteLyOyvKHLHrlRq2+x3zNPhkMuo4Ad1uoZmkKp6np79 npxU8ZTE6orCD0Mo+JyfO8xF/pYtzS7SONRCqESkQIaHxOO1YSPX3Yzkgn+zHqRWzfZa s5rA== X-Gm-Message-State: ACrzQf3QrwfPTxPtdOU6XJ/LnixXXyKf6GJXs8kGchqaH5zuGqZkirTF 82rNTXVdv9REJAdaunXxTqWY+7b2IOw= X-Google-Smtp-Source: AMsMyM7snmCQbST7m7KX///bCc99DHE7y/hoF5W7QdoAe/UYNbWe7YLA9GU2bPFgBOLCoCu75/FT8A== X-Received: by 2002:adf:f781:0:b0:236:5559:215b with SMTP id q1-20020adff781000000b002365559215bmr37577104wrp.16.1667947467346; Tue, 08 Nov 2022 14:44:27 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m3-20020a05600c4f4300b003b47b80cec3sm17139930wmq.42.2022.11.08.14.44.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Nov 2022 14:44:26 -0800 (PST) Message-Id: <45c198c629da1627eabf0e63539f50aaa50381bf.1667947465.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 08 Nov 2022 22:44:21 +0000 Subject: [PATCH 1/5] cache-tree: add perf test comparing update and prime Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, phillip.wood123@gmail.com, derrickstolee@github.com, jonathantanmy@google.com, Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Add a performance test comparing the execution times of 'prime_cache_tree()' and 'cache_tree_update(_, WRITE_TREE_SILENT | WRITE_TREE_REPAIR)'. The goal of comparing these two is to identify which is the faster method for rebuilding an invalid cache tree, ultimately to remove one when both are (reundantly) called in immediate succession. Both methods are incredibly fast, so the new tests in 'p0090-cache-tree.sh' must call the tested method many times in succession to get non-negligible timing results. Results show a substantial difference in execution time between the two, with 'prime_cache_tree()' appearing to be the overall faster method: Test this tree ---------------------------------------------------- 0090.1: prime_cache_tree, clean 0.07(0.05+0.01) 0090.2: cache_tree_update, clean 0.11(0.05+0.06) 0090.3: prime_cache_tree, invalid 0.06(0.05+0.01) 0090.4: cache_tree_update, invalid 0.50(0.41+0.07) Signed-off-by: Victoria Dye --- Makefile | 1 + t/helper/test-cache-tree.c | 52 ++++++++++++++++++++++++++++++++++++++ t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/perf/p0090-cache-tree.sh | 27 ++++++++++++++++++++ 5 files changed, 82 insertions(+) create mode 100644 t/helper/test-cache-tree.c create mode 100755 t/perf/p0090-cache-tree.sh diff --git a/Makefile b/Makefile index 4927379184c..3639c7c2a94 100644 --- a/Makefile +++ b/Makefile @@ -723,6 +723,7 @@ TEST_BUILTINS_OBJS += test-advise.o TEST_BUILTINS_OBJS += test-bitmap.o TEST_BUILTINS_OBJS += test-bloom.o TEST_BUILTINS_OBJS += test-bundle-uri.o +TEST_BUILTINS_OBJS += test-cache-tree.o TEST_BUILTINS_OBJS += test-chmtime.o TEST_BUILTINS_OBJS += test-config.o TEST_BUILTINS_OBJS += test-crontab.o diff --git a/t/helper/test-cache-tree.c b/t/helper/test-cache-tree.c new file mode 100644 index 00000000000..2fad6d06d30 --- /dev/null +++ b/t/helper/test-cache-tree.c @@ -0,0 +1,52 @@ +#include "test-tool.h" +#include "cache.h" +#include "tree.h" +#include "cache-tree.h" +#include "parse-options.h" + +static char const * const test_cache_tree_usage[] = { + N_("test-tool cache-tree (prime|repair)"), + NULL +}; + +int cmd__cache_tree(int argc, const char **argv) +{ + struct object_id oid; + struct tree *tree; + int fresh = 0; + int count = 1; + int i; + + struct option options[] = { + OPT_BOOL(0, "fresh", &fresh, + N_("clear the cache tree before each repetition")), + OPT_INTEGER_F(0, "count", &count, N_("number of times to repeat the operation"), + PARSE_OPT_NONEG), + OPT_END() + }; + + setup_git_directory(); + + parse_options(argc, argv, NULL, options, test_cache_tree_usage, 0); + + if (read_cache() < 0) + die("unable to read index file"); + + get_oid("HEAD", &oid); + tree = parse_tree_indirect(&oid); + for (i = 0; i < count; i++) { + if (fresh) + cache_tree_free(&the_index.cache_tree); + + if (!argc) + die("Must specify subcommand"); + else if (!strcmp(argv[0], "prime")) + prime_cache_tree(the_repository, &the_index, tree); + else if (!strcmp(argv[0], "update")) + cache_tree_update(&the_index, WRITE_TREE_SILENT | WRITE_TREE_REPAIR); + else + die("Unknown command %s", argv[0]); + } + + return 0; +} diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index 01cda9358df..547a3be1c8b 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -14,6 +14,7 @@ static struct test_cmd cmds[] = { { "bitmap", cmd__bitmap }, { "bloom", cmd__bloom }, { "bundle-uri", cmd__bundle_uri }, + { "cache-tree", cmd__cache_tree }, { "chmtime", cmd__chmtime }, { "config", cmd__config }, { "crontab", cmd__crontab }, diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h index ca2948066fd..e44e1d896d3 100644 --- a/t/helper/test-tool.h +++ b/t/helper/test-tool.h @@ -8,6 +8,7 @@ int cmd__advise_if_enabled(int argc, const char **argv); int cmd__bitmap(int argc, const char **argv); int cmd__bloom(int argc, const char **argv); int cmd__bundle_uri(int argc, const char **argv); +int cmd__cache_tree(int argc, const char **argv); int cmd__chmtime(int argc, const char **argv); int cmd__config(int argc, const char **argv); int cmd__crontab(int argc, const char **argv); diff --git a/t/perf/p0090-cache-tree.sh b/t/perf/p0090-cache-tree.sh new file mode 100755 index 00000000000..91c13e28a27 --- /dev/null +++ b/t/perf/p0090-cache-tree.sh @@ -0,0 +1,27 @@ +#!/bin/sh + +test_description="Tests performance of cache tree operations" + +. ./perf-lib.sh + +test_perf_large_repo +test_checkout_worktree + +count=200 +test_perf "prime_cache_tree, clean" " + test-tool cache-tree --count $count prime +" + +test_perf "cache_tree_update, clean" " + test-tool cache-tree --count $count update +" + +test_perf "prime_cache_tree, invalid" " + test-tool cache-tree --count $count --fresh prime +" + +test_perf "cache_tree_update, invalid" " + test-tool cache-tree --count $count --fresh update +" + +test_done From patchwork Tue Nov 8 22:44:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13036914 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E344C4332F for ; Tue, 8 Nov 2022 22:44:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229695AbiKHWoo (ORCPT ); Tue, 8 Nov 2022 17:44:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38544 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229709AbiKHWob (ORCPT ); Tue, 8 Nov 2022 17:44:31 -0500 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E383D209A8 for ; Tue, 8 Nov 2022 14:44:29 -0800 (PST) Received: by mail-wr1-x433.google.com with SMTP id y16so23140032wrt.12 for ; Tue, 08 Nov 2022 14:44:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=y/zicVtu+r7gO5FwNJ/+MUMzuPbt221qJMX9gz8tkU0=; b=NkdBXINqsm6+EGtYp69ZZyFC1t7EMn4VW+zD+4vb/Tr1Lryd69txVYlivG0nimWNJV yVr5f2JvdQV9ec9U/LilLy/ugrXnK/pz6wCEJnWquku9B76gmvAK7YE7GscHeztyK1xY 7LVVgyXkmS/+cPNYqbJv3AWSgns6lGT8+4CqvDwkTXcsZwxMTOAghJjMxgMVbeOFsBF7 /nqghC4pd1YPL5Qp9eiZ3d82PsHtHKvLW5FvuY7EBthSJiKrGfNJiPQsU0Ycm6A8cq8e JDnZxXABpESHaU9DXP4u067Ojc3YK+RckUxMPGunfZFBLhl0ZM82ZWvAYJEsUVGxvFd3 q33Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y/zicVtu+r7gO5FwNJ/+MUMzuPbt221qJMX9gz8tkU0=; b=agvHefkGr1q7CqPmMZaOKtmX+nC7ZTi/HnAD/L5MnmTQR/bCATGANouqve8E6LwYOY u2FD1ueRdYwth/vwk2tz1Q/jBR71URdpRlykO3qz4Bo6RGPRE2IxQvGMhcoBoKGXWZae d8j8dNzIUoK7J/m9mmgQS6WXSu0Aou4vjElMg/a6swJfje3wOAqzuqHaRi2mN3D1invr U9d0RDoawcYyAp5Er8rlPV3re2b3LO2ErvuUpVr3oIh+hXR8QPmZusjicjw2zkgiIcCe o+dGsaFkPIAH0mnSjcPwAkzvoLQ2OUVFp48F1FGMEpxNTWYWBvsOplknA7rMTTF+6tCQ w65Q== X-Gm-Message-State: ACrzQf1zjRgmquWtnZq0plZGNbHHndHGViLUmM4I2L/rp0KbtNxfZR3a C0Cd0gFVqRYrbGdaJIZu4jE8tlwTc/M= X-Google-Smtp-Source: AMsMyM7zv1EjaVsr5+S4l7IWCJYV4cZahk1CwyfWJvZvv7eznoXWeKYxb6qoYe4qBKPDqgN4u6Qeew== X-Received: by 2002:adf:dd4d:0:b0:236:6e72:be17 with SMTP id u13-20020adfdd4d000000b002366e72be17mr35268804wrm.460.1667947468261; Tue, 08 Nov 2022 14:44:28 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t1-20020a1c7701000000b003b3307fb98fsm12137459wmi.24.2022.11.08.14.44.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Nov 2022 14:44:27 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 08 Nov 2022 22:44:22 +0000 Subject: [PATCH 2/5] unpack-trees: add 'skip_cache_tree_update' option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, phillip.wood123@gmail.com, derrickstolee@github.com, jonathantanmy@google.com, Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Add (disabled by default) option to skip the 'cache_tree_update()' at the end of 'unpack_trees()'. In many cases, this cache tree update is redundant because the caller of 'unpack_trees()' immediately follows it with 'prime_cache_tree()', rebuilding the entire cache tree from scratch. While these operations aren't the most expensive part of operations like 'git reset', the duplicate calls still create a minor unnecessary slowdown. Introduce an option for callers to skip the 'cache_tree_update()' in 'unpack_trees()' if it is redundant (that is, if 'prime_cache_tree()' is called afterwards). At the moment, no 'unpack_trees()' callers use the new option; they will be updated in subsequent patches. Signed-off-by: Victoria Dye --- unpack-trees.c | 3 ++- unpack-trees.h | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/unpack-trees.c b/unpack-trees.c index bae812156c4..8a762aa0772 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -2043,7 +2043,8 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options if (!ret) { if (git_env_bool("GIT_TEST_CHECK_CACHE_TREE", 0)) cache_tree_verify(the_repository, &o->result); - if (!cache_tree_fully_valid(o->result.cache_tree)) + if (!o->skip_cache_tree_update && + !cache_tree_fully_valid(o->result.cache_tree)) cache_tree_update(&o->result, WRITE_TREE_SILENT | WRITE_TREE_REPAIR); diff --git a/unpack-trees.h b/unpack-trees.h index efb9edfbb27..6ab0d74c84d 100644 --- a/unpack-trees.h +++ b/unpack-trees.h @@ -71,7 +71,8 @@ struct unpack_trees_options { quiet, exiting_early, show_all_errors, - dry_run; + dry_run, + skip_cache_tree_update; enum unpack_trees_reset_type reset; const char *prefix; int cache_bottom; From patchwork Tue Nov 8 22:44:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13036913 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3303AC4332F for ; Tue, 8 Nov 2022 22:44:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229713AbiKHWol (ORCPT ); Tue, 8 Nov 2022 17:44:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229702AbiKHWob (ORCPT ); Tue, 8 Nov 2022 17:44:31 -0500 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C97A6209A9 for ; Tue, 8 Nov 2022 14:44:30 -0800 (PST) Received: by mail-wr1-x42a.google.com with SMTP id h9so23257884wrt.0 for ; Tue, 08 Nov 2022 14:44:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=Jtq3iPEAnMV1G7hru853ch69TOxtztyX7mt4g8SPMWI=; b=U8WEFKkYJMKYQUZBLQAA711nB2T+sLqYdBuGPKvm1d/racSVcWop30zJS55OeVQJB6 7XPqYdAG6nGLsBdSAJu8elMlEFDnqWvBVKAPvDJCDln+7iGeXzfg5SxzfqpM6eKGyu5o YcjA42rG/OjIsdSQf1kX0xJks0tcZK/7jCoCP9gGAYzCxU8oxfs7FP2+1FSazuPw2o+4 B1tJ7WgeVhAMAuhEVZE8soBnrqq0+qk+B63bPq+dvAVlBPiFhDIPpE+TFoOOu8od2qU1 A+Vs6SMNw8dYoBHiMdAGHRYRnE3NVXhLDD4BpX+XvByML8y8qqjCzW8itF4FizZ/ZrVB POCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jtq3iPEAnMV1G7hru853ch69TOxtztyX7mt4g8SPMWI=; b=fQ5n3NCi0oidwQcWJLcIeyn97C3DLPWlvaGAMzrbiqkSHnHX76pdmKASI6EFZt3e2P B2izAgi/IerGbHW4DVLiMgWjD9AvPgT9uy1U669Msl2CdGMMXEZUzrsVVmCDVi5oAvHA Khm8ym+hDubkapT+jNUqfqSyFh6ukP9itzJp5Sv2r7rR+bXrf/wjM4tgW1fDvyDPe/W3 fMDDt7md4XV47zQ6cGbd01f4gvbUzQDb8qfuL6OBssbVCXEuLolvjl6SHumXvEGzihXw gr0p9GfgZ53q+Xkxd5Uwvrj/LPwySk2ZnykBIihoY3PhoGmbm1RVuFZHiwzocEBP8WuT O26Q== X-Gm-Message-State: ACrzQf0twT3G9jfPyKZ1FCQCSXB54y6d0WN4PPQizexxnkhsEDC17Iud 0ccSmz2Yb2LK4O/s1+plCkVQeY5SZ5c= X-Google-Smtp-Source: AMsMyM5ZiTXC0EkQziEAhrwTtVwCal91EHSlqvjijfziVXQR4FZA321WvkXGOyDhmfEAWipwJze89A== X-Received: by 2002:adf:f0c9:0:b0:236:26a2:34e3 with SMTP id x9-20020adff0c9000000b0023626a234e3mr35765640wro.541.1667947469076; Tue, 08 Nov 2022 14:44:29 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i19-20020a05600c355300b003b49ab8ff53sm13759305wmq.8.2022.11.08.14.44.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Nov 2022 14:44:28 -0800 (PST) Message-Id: <908fe764670af2572a6171c163abf36f4ed8e41a.1667947465.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 08 Nov 2022 22:44:23 +0000 Subject: [PATCH 3/5] reset: use 'skip_cache_tree_update' option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, phillip.wood123@gmail.com, derrickstolee@github.com, jonathantanmy@google.com, Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Enable the 'skip_cache_tree_update' option in the variants that call 'prime_cache_tree()' after 'unpack_trees()' (specifically, 'git reset --mixed' and 'git reset --hard'). This avoids redundantly rebuilding the cache tree in both 'cache_tree_update()' at the end of 'unpack_trees()' and in 'prime_cache_tree()', resulting in a small (but consistent) performance improvement. From the newly-added 'p7102-reset.sh' test: Test before after -------------------------------------------------------------------- 7102.1: reset --hard (...) 2.11(0.40+1.54) 1.97(0.38+1.47) -6.6% Signed-off-by: Victoria Dye --- builtin/reset.c | 2 ++ t/perf/p7102-reset.sh | 21 +++++++++++++++++++++ 2 files changed, 23 insertions(+) create mode 100755 t/perf/p7102-reset.sh diff --git a/builtin/reset.c b/builtin/reset.c index fdce6f8c856..ab027774824 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -73,9 +73,11 @@ static int reset_index(const char *ref, const struct object_id *oid, int reset_t case HARD: opts.update = 1; opts.reset = UNPACK_RESET_OVERWRITE_UNTRACKED; + opts.skip_cache_tree_update = 1; break; case MIXED: opts.reset = UNPACK_RESET_PROTECT_UNTRACKED; + opts.skip_cache_tree_update = 1; /* but opts.update=0, so working tree not updated */ break; default: diff --git a/t/perf/p7102-reset.sh b/t/perf/p7102-reset.sh new file mode 100755 index 00000000000..9b039e8691f --- /dev/null +++ b/t/perf/p7102-reset.sh @@ -0,0 +1,21 @@ +#!/bin/sh + +test_description='performance of reset' +. ./perf-lib.sh + +test_perf_default_repo +test_checkout_worktree + +test_perf 'reset --hard with change in tree' ' + base=$(git rev-parse HEAD) && + test_commit --no-tag A && + new=$(git rev-parse HEAD) && + + for i in $(test_seq 10) + do + git reset --hard $new && + git reset --hard $base || return $? + done +' + +test_done From patchwork Tue Nov 8 22:44:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13036916 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1B07C4332F for ; Tue, 8 Nov 2022 22:44:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229869AbiKHWor (ORCPT ); Tue, 8 Nov 2022 17:44:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229716AbiKHWoc (ORCPT ); Tue, 8 Nov 2022 17:44:32 -0500 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A042A21273 for ; Tue, 8 Nov 2022 14:44:31 -0800 (PST) Received: by mail-wm1-x336.google.com with SMTP id t4so9689677wmj.5 for ; Tue, 08 Nov 2022 14:44:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=Y9D6pDzcfKa+C9JuV33FlqQspXTfBgY5IZLMOQAlznk=; b=pl+ZJ1xUX0L3auR2rOWWYCIemq05oO/sP1vkcLedTXtq1LkbgrzHhFCXJlVS0kNEnM bnylWV/nQJhUSeDf2SwCSgRisranSGr5OfBIosuwpZFI7fr5zLUjK2QaeNxDWK/U54yU fiEJUKC1ZhdqRXcvkKLFSJ5ZQyk199iaEoPWZgYJtbSgbdcnlZ+PffXAefXAHOZUuvsh JJ58qZr8/9F7E6heM3+vpSkqhF8cbiBr/x/37/sbWIskGZ2I7Btm7lOtN8x2QCLzPLs7 0ULUur5Q0JMAw3Utrfb2JBknLcRXv3IVU0yXgOIVJruy7Xkdky7iOTNUo5kqoM7lABVh aQ2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Y9D6pDzcfKa+C9JuV33FlqQspXTfBgY5IZLMOQAlznk=; b=XU4o+zVS5XAWXDh+VOZ+lnCryUR34UjtTFuGD553M+TqDeoENB4//FoxR6xE1dQDR4 /Xaf6tKrixSVGuogxswQ+vcIfFE4/JfGMU9YFNlI/cNGnYn48SxSANAmvCoAe3wnfNwp JrUkm9wMM66LhJoxqRXYiQcu5QOGjiRDVHfBIY9awYWI1pjxsTX0W3D1uDpTl7ExR4Gv MaC6ZP/ntAo/k8OlPboV+RMMoY6Etv5eWyljyyVbX68ezQb7y7U/4+3caZhguK08fO+M mt8P/7YZVTUoWKSDw1Os2eLxDuUI3+O759R9HaZkJJSN0sYkyAhFpYV1phtpsoIENu6T JpTA== X-Gm-Message-State: ANoB5pmWU8EGQvOtUirjT4bd4invyw3OvtM+R619FcwwVtDvez8VClWs YOQtjHO2J/3tetOoG1c+B6lIugY6li0= X-Google-Smtp-Source: AA0mqf5JVT6/EGwVQYdrrS4WjSUdAwMbZvSMr7qyrnreIBIXfo/dPr2WR36/xX3mPMKWJMhPdZPg4A== X-Received: by 2002:a05:600c:4a11:b0:3cf:b128:39ad with SMTP id c17-20020a05600c4a1100b003cfb12839admr7022676wmp.127.1667947470039; Tue, 08 Nov 2022 14:44:30 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l11-20020a05600c1d0b00b003cf878c4468sm17473583wms.5.2022.11.08.14.44.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Nov 2022 14:44:29 -0800 (PST) Message-Id: <319f1d71b2e63455f319f1b460515fe0c4af33f6.1667947465.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 08 Nov 2022 22:44:24 +0000 Subject: [PATCH 4/5] read-tree: use 'skip_cache_tree_update' option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, phillip.wood123@gmail.com, derrickstolee@github.com, jonathantanmy@google.com, Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye When running 'read-tree' with a single tree and no prefix, 'prime_cache_tree()' is called after the tree is unpacked. In that situation, skip a redundant call to 'cache_tree_update()' in 'unpack_trees()' by enabling the 'skip_cache_tree_update' unpack option. Removing the redundant cache tree update provides a substantial performance improvement to 'git read-tree ', as shown by a test added to 'p0006-read-tree-checkout.sh': Test before after ---------------------------------------------------------------------- read-tree br_ballast_plus_1 3.94(1.80+1.57) 3.00(1.14+1.28) -23.9% Note that the 'read-tree' in 't1022-read-tree-partial-clone.sh' is updated to read two trees, rather than one. The test was first introduced in d3da223f221 (cache-tree: prefetch in partial clone read-tree, 2021-07-23) to exercise the 'cache_tree_update()' code path, as used in 'git merge'. Since this patch drops the call to 'cache_tree_update()' in single-tree 'git read-tree', change the test to use the two-tree variant so that 'cache_tree_update()' is called as intended. Signed-off-by: Victoria Dye --- builtin/read-tree.c | 4 ++++ t/perf/p0006-read-tree-checkout.sh | 8 ++++++++ t/t1022-read-tree-partial-clone.sh | 2 +- 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/builtin/read-tree.c b/builtin/read-tree.c index f4cbe460b97..45c6652444b 100644 --- a/builtin/read-tree.c +++ b/builtin/read-tree.c @@ -249,6 +249,10 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix) if (opts.debug_unpack) opts.fn = debug_merge; + /* If we're going to prime_cache_tree later, skip cache tree update */ + if (nr_trees == 1 && !opts.prefix) + opts.skip_cache_tree_update = 1; + cache_tree_free(&active_cache_tree); for (i = 0; i < nr_trees; i++) { struct tree *tree = trees[i]; diff --git a/t/perf/p0006-read-tree-checkout.sh b/t/perf/p0006-read-tree-checkout.sh index c481c012d2f..325566e18eb 100755 --- a/t/perf/p0006-read-tree-checkout.sh +++ b/t/perf/p0006-read-tree-checkout.sh @@ -49,6 +49,14 @@ test_perf "read-tree br_base br_ballast ($nr_files)" ' git read-tree -n -m br_base br_ballast ' +test_perf "read-tree br_ballast_plus_1 ($nr_files)" ' + # Run read-tree 100 times for clearer performance results & comparisons + for i in $(test_seq 100) + do + git read-tree -n -m br_ballast_plus_1 || return 1 + done +' + test_perf "switch between br_base br_ballast ($nr_files)" ' git checkout -q br_base && git checkout -q br_ballast diff --git a/t/t1022-read-tree-partial-clone.sh b/t/t1022-read-tree-partial-clone.sh index a9953b6a71c..da539716359 100755 --- a/t/t1022-read-tree-partial-clone.sh +++ b/t/t1022-read-tree-partial-clone.sh @@ -19,7 +19,7 @@ test_expect_success 'read-tree in partial clone prefetches in one batch' ' git -C server config uploadpack.allowfilter 1 && git -C server config uploadpack.allowanysha1inwant 1 && git clone --bare --filter=blob:none "file://$(pwd)/server" client && - GIT_TRACE_PACKET="$(pwd)/trace" git -C client read-tree $TREE && + GIT_TRACE_PACKET="$(pwd)/trace" git -C client read-tree $TREE $TREE && # "done" marks the end of negotiation (once per fetch). Expect that # only one fetch occurs. From patchwork Tue Nov 8 22:44:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13036915 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1065C433FE for ; Tue, 8 Nov 2022 22:44:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229747AbiKHWoq (ORCPT ); Tue, 8 Nov 2022 17:44:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229734AbiKHWoc (ORCPT ); Tue, 8 Nov 2022 17:44:32 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 634BA21241 for ; Tue, 8 Nov 2022 14:44:31 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id p16so9705377wmc.3 for ; Tue, 08 Nov 2022 14:44:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=O/qBtdZ69IG5ihaw98dYPOr1oYuv5CWv1pqA2wxwrrQ=; b=oXK/93sI3u1gfnCspoklK2nymcSt/iW1Vy9ClpoNzFyY+WRkxHbNuEck7Vniq0WvLS q8Yc2d2upcDjiCE+SM7OsLgdpiZqIY0rC5nACobU2jcp3k91iA3YkeePhfyqZUb8e1jO NoSdyLxy1vWdlRlKFgFSBaUOFTypuKweo5/gbGKFFzwR6eValC29LbiKrmOYVDF/agYC hcBohb92cBGDWTxzkpDF26lB6ENoeXpzqraBB5fnas0RBw1Zl/QWuqUWZvBFigvKVJyW JT5Iw6+BGp3K98JUB0R3a8juAm7Q/B6h1dLhACFXXE6B+g1Yo3MRf793NAiPtskIMKtB ix3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O/qBtdZ69IG5ihaw98dYPOr1oYuv5CWv1pqA2wxwrrQ=; b=NR2LkmX1QjHpF59fOjkwW5my8unfWLVngoiLfYUiKLzle1IC8oGGsxSCSjj9rhsh7L dyZlKgDan7cuT5an9oHYtj0wrtKLRBC5PyStE3KyFSy6jcNp/XVuQ25EXSw9ohZndNqQ R3GZuMKBZveAO1IX31anNOwSxOcg7auLe6mtqQna5LmDzSjDvWbTwnz8OT1sNHGOy4CM CvzQiOOGjw8DSIzpaHrug1z7q7GG+E1XWeDAmOfEkDzCW5ZX2SJ8yEjfPV5BRrvdkAWq VQivMqTJveBEhxgs3gxBAYp7Glejliqc/NSLSefMtOh1SNIUUjXOFyB2Bctup+fAguZE biCQ== X-Gm-Message-State: ACrzQf2ghjZO9WZxUyrnh2JHi5Lgyx6oyeaBtcOjedkdHTFMQ6rSalJ1 a6HWW32Wz8iAD1aD+tlYFjQStySpMLs= X-Google-Smtp-Source: AMsMyM7KB3iq4IedjXDlTRHYYkWaLmL8P7ZxJQ3q9QHz2lKvi/4Ehi5VdgwCB9nebn5zdSS61poCzA== X-Received: by 2002:a05:600c:2e07:b0:3cf:92cc:4aa8 with SMTP id o7-20020a05600c2e0700b003cf92cc4aa8mr20136233wmf.68.1667947470769; Tue, 08 Nov 2022 14:44:30 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n17-20020a05600c4f9100b003c701c12a17sm16922358wmq.12.2022.11.08.14.44.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Nov 2022 14:44:30 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 08 Nov 2022 22:44:25 +0000 Subject: [PATCH 5/5] rebase: use 'skip_cache_tree_update' option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, phillip.wood123@gmail.com, derrickstolee@github.com, jonathantanmy@google.com, Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Enable the 'skip_cache_tree_update' option in both 'do_reset()' ('sequencer.c') and 'reset_head()' ('reset.c'). Both of these callers invoke 'prime_cache_tree()' after 'unpack_trees()', so we can remove an unnecessary cache tree rebuild by skipping 'cache_tree_update()'. When testing with 'p3400-rebase.sh' and 'p3404-rebase-interactive.sh', the performance change of this update was negligible, likely due to the operation being dominated by more expensive operations (like checking out trees). However, since the change doesn't harm performance, it's worth keeping this 'unpack_trees()' usage consistent with others that subsequently invoke 'prime_cache_tree()'. Signed-off-by: Victoria Dye --- reset.c | 1 + sequencer.c | 1 + 2 files changed, 2 insertions(+) diff --git a/reset.c b/reset.c index e3383a93343..5ded23611f3 100644 --- a/reset.c +++ b/reset.c @@ -128,6 +128,7 @@ int reset_head(struct repository *r, const struct reset_head_opts *opts) unpack_tree_opts.update = 1; unpack_tree_opts.merge = 1; unpack_tree_opts.preserve_ignored = 0; /* FIXME: !overwrite_ignore */ + unpack_tree_opts.skip_cache_tree_update = 1; init_checkout_metadata(&unpack_tree_opts.meta, switch_to_branch, oid, NULL); if (reset_hard) unpack_tree_opts.reset = UNPACK_RESET_PROTECT_UNTRACKED; diff --git a/sequencer.c b/sequencer.c index e658df7e8ff..3f7a73ce4e1 100644 --- a/sequencer.c +++ b/sequencer.c @@ -3750,6 +3750,7 @@ static int do_reset(struct repository *r, unpack_tree_opts.merge = 1; unpack_tree_opts.update = 1; unpack_tree_opts.preserve_ignored = 0; /* FIXME: !overwrite_ignore */ + unpack_tree_opts.skip_cache_tree_update = 1; init_checkout_metadata(&unpack_tree_opts.meta, name, &oid, NULL); if (repo_read_index_unmerged(r)) {