From patchwork Mon Nov 22 22:42:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lessley Dennington X-Patchwork-Id: 12633031 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9DC9C433EF for ; Mon, 22 Nov 2021 22:42:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234626AbhKVWpw (ORCPT ); Mon, 22 Nov 2021 17:45:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234390AbhKVWpu (ORCPT ); Mon, 22 Nov 2021 17:45:50 -0500 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2031CC061574 for ; Mon, 22 Nov 2021 14:42:43 -0800 (PST) Received: by mail-wr1-x42c.google.com with SMTP id u18so35488313wrg.5 for ; Mon, 22 Nov 2021 14:42:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=zrat1o9DQDxUklWklG5AjpqXcgx5ylwZ/LFbnaN2Yug=; b=mtnwbpwAFXPzR+HlW8yrIv+U9xmh8PMAaC1Lnes/vzqwTIYM7zUTIr5jbiz1iRSkyv cGDJQiVNoeOhYodqYiw6rh+xIP9ub2NbFJ7rjIL3mpwFkeQ9yHXVYWLqg+Z4VaiXJhDZ 4dHGZanQ+62sPFNfrHIA4e9bnKVlvKXJ/eSasfY0njP/lkkte9V9dqtBb24CRUUYvLB3 9C+uhiTeUX48virEkPSJoPlni2RlFsfr+erpLFfeXedzGGb4ZZOlvINkz6LWScHCZuz1 qbLeKsKEibS7YfhtHjn74RGXaLMJPqQtSlClJeek5klBa10Ek2mRgAp28gxDPkka2fZm 4Hgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=zrat1o9DQDxUklWklG5AjpqXcgx5ylwZ/LFbnaN2Yug=; b=plhUBU4r0iBqEfvzkCkwriFWaL3A1upOBBT9SBFLIMIaQarBgRgWn2oNPAF+dTmbNV NYTeTk51Ks2vGwCx4FLhCnN+dZyWe+EkNvHhSDyT+cY5LaK005LlPvGfe2pMOK/PlWJU huuZ9uiE+//2Jtn/VTnH9IST5MUtDe8SuzdKiuKW/xDz5OP/Hm9bxUfGLQnofCshakVN UDTLrdYYXZ9qn7+bWOyM2PhbGW2PIPJ3UH5xfCBGlxIy0KZ7VD9rTLSbKyudXiY/Ucge loqxkdee6GeL51uAJhZ1xPmT6xdcrmf9tu92zrvULKPe8hw/YfNsZ+6pt0yiSDlSaqWT KCVA== X-Gm-Message-State: AOAM532T2u4xb4WUGXxYuGbpQuDaSIYF3W9lIO8BzRxTt4IhYAu5lMZE QAbqHZjc/PhIQa+qLlkAY2Xjb4Ga4HA= X-Google-Smtp-Source: ABdhPJxKnKEA5gu7phSaFpNEzttK06VaqF3otTK1bJ7tJlsRyM/mpXELj8EMdjQg6vPtWzwej8hrng== X-Received: by 2002:a05:6000:181:: with SMTP id p1mr1086790wrx.292.1637620961609; Mon, 22 Nov 2021 14:42:41 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f15sm12895941wmg.30.2021.11.22.14.42.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Nov 2021 14:42:41 -0800 (PST) Message-Id: <81e208cf454b61c761fa66e4f43a464ed439ba59.1637620958.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 22 Nov 2021 22:42:35 +0000 Subject: [PATCH v4 1/4] sparse index: enable only for git repos Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Lessley Dennington , Lessley Dennington Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Lessley Dennington From: Lessley Dennington Check whether git dir exists before adding any repo settings. If it does not exist, BUG with the message that one cannot add settings for an uninitialized repository. If it does exist, proceed with adding repo settings. Additionally, ensure the above BUG is not triggered when users pass the -h flag by adding a check for the repository to the checkout and pack-objects builtins. Finally, ensure the above BUG is not triggered for commit-graph by returning early if the git directory does not exist. Signed-off-by: Lessley Dennington --- builtin/checkout.c | 6 ++++-- builtin/pack-objects.c | 9 ++++++--- commit-graph.c | 5 ++++- repo-settings.c | 3 +++ 4 files changed, 17 insertions(+), 6 deletions(-) diff --git a/builtin/checkout.c b/builtin/checkout.c index 8c69dcdf72a..31632b07888 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -1588,8 +1588,10 @@ static int checkout_main(int argc, const char **argv, const char *prefix, git_config(git_checkout_config, opts); - prepare_repo_settings(the_repository); - the_repository->settings.command_requires_full_index = 0; + if (startup_info->have_repository) { + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + } opts->track = BRANCH_TRACK_UNSPECIFIED; diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 1a3dd445f83..45dc2258dc7 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3976,9 +3976,12 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) read_replace_refs = 0; sparse = git_env_bool("GIT_TEST_PACK_SPARSE", -1); - prepare_repo_settings(the_repository); - if (sparse < 0) - sparse = the_repository->settings.pack_use_sparse; + + if (startup_info->have_repository) { + prepare_repo_settings(the_repository); + if (sparse < 0) + sparse = the_repository->settings.pack_use_sparse; + } reset_pack_idx_option(&pack_idx_opts); git_config(git_pack_config, NULL); diff --git a/commit-graph.c b/commit-graph.c index 2706683acfe..265c010122e 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -632,10 +632,13 @@ static int prepare_commit_graph(struct repository *r) struct object_directory *odb; /* + * Early return if there is no git dir or if the commit graph is + * disabled. + * * This must come before the "already attempted?" check below, because * we want to disable even an already-loaded graph file. */ - if (r->commit_graph_disabled) + if (!r->gitdir || r->commit_graph_disabled) return 0; if (r->objects->commit_graph_attempted) diff --git a/repo-settings.c b/repo-settings.c index b93e91a212e..00ca5571a1a 100644 --- a/repo-settings.c +++ b/repo-settings.c @@ -17,6 +17,9 @@ void prepare_repo_settings(struct repository *r) char *strval; int manyfiles; + if (!r->gitdir) + BUG("Cannot add settings for uninitialized repository"); + if (r->settings.initialized++) return; From patchwork Mon Nov 22 22:42:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lessley Dennington X-Patchwork-Id: 12633033 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14722C433F5 for ; Mon, 22 Nov 2021 22:42:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235199AbhKVWpz (ORCPT ); Mon, 22 Nov 2021 17:45:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234237AbhKVWpv (ORCPT ); Mon, 22 Nov 2021 17:45:51 -0500 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA716C061714 for ; Mon, 22 Nov 2021 14:42:43 -0800 (PST) Received: by mail-wm1-x332.google.com with SMTP id 77-20020a1c0450000000b0033123de3425so520399wme.0 for ; Mon, 22 Nov 2021 14:42:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=w6vpqglH5lwDg1E4b+OHGlkiNda+85Fj50ySebHz61U=; b=WH5296vPKyl8VMARfaFndlc/wY9w+Q+YQbPN+yM6kwbwNxHixlyzvTHV9Ypmrm+WmD 6yjI7FdixmXh77VM1I47YOv9nfRLZK7vPGGevNENzL8WKGoRsMroRz7B+unuS20SnrRw RxsFQ/Pu+sKcHjjuVorFvrEnQAjUGuxDZUc8FQ09N+l517peystX2UsE0NVCC7yXM1vy 0MpaO+eWzDGCPthBbxlHDdYJoz8dYvkVhBuVM216dszE1wF2UZX6nwo5BTb+/9rGSR5e RdW6QB/ibJ83QNktketDBKambtCPVepca6/2i2BRB1OM/z+iLa/U+h6957e+naaABk2n m1cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=w6vpqglH5lwDg1E4b+OHGlkiNda+85Fj50ySebHz61U=; b=aA2iEvnFbXPNEbzoDyR4v8Og+hgBGkUIxrrfXg8ao6hI0HL7VD1zHUTLVrFe73y+qa O168Z1qSL49/IJd6bMW6V1Rn9tdiwEL0xIwYZZHIG/PFS2LdyXusMRvhEfJP3O7+uOuF 3dPJ3X8QApEfS0nCOWPPR4Xs3/BDHm5tTIvk1zQFZkkFLcfdYIPeZiPcHxs+twWk2D18 lZ5JUVkqz52c3h9bI7SBMG3a3Bki/K+KL6hKdNh5vMcq8xH/rT0/SPdqmScetA2x/Bj4 wSm8DxfguLdH6JtFjtTctCyFeA0+s8H0qca/wKvWTfqKJMi6G72ya5tRMHo6Z1lr7uPI EZKg== X-Gm-Message-State: AOAM532hOqOUuMB6H3UHJRpvJ/lpYzXG9TDGdaHx4efbCIPBow4zLaX2 DaoWVp7djiQNdGoJINJQGXAHrNXuEoc= X-Google-Smtp-Source: ABdhPJwcmFzQ/I8h93LGNuiCqbdE21RJKQ7+CX35TCXOFD8wix71xCYxWAB6jNWqggPVdh2eJq/4Yg== X-Received: by 2002:a7b:cd96:: with SMTP id y22mr33566242wmj.121.1637620962204; Mon, 22 Nov 2021 14:42:42 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n32sm10972236wms.42.2021.11.22.14.42.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Nov 2021 14:42:41 -0800 (PST) Message-Id: <5bc5e8465ab5fe871965e6c6d578efc51e55b505.1637620958.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 22 Nov 2021 22:42:36 +0000 Subject: [PATCH v4 2/4] test-read-cache: set up repo after git directory Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Lessley Dennington , Lessley Dennington Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Lessley Dennington From: Lessley Dennington Move repo setup to occur after git directory is set up. This will ensure enabling the sparse index for `diff` (and guarding against the nongit scenario) will not cause tests to start failing, since that change will include adding a check to prepare_repo_settings() with the new BUG. Signed-off-by: Lessley Dennington --- t/helper/test-read-cache.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/t/helper/test-read-cache.c b/t/helper/test-read-cache.c index b52c174acc7..0d9f08931a1 100644 --- a/t/helper/test-read-cache.c +++ b/t/helper/test-read-cache.c @@ -39,8 +39,6 @@ int cmd__read_cache(int argc, const char **argv) int table = 0, expand = 0; initialize_the_repository(); - prepare_repo_settings(r); - r->settings.command_requires_full_index = 0; for (++argv, --argc; *argv && starts_with(*argv, "--"); ++argv, --argc) { if (skip_prefix(*argv, "--print-and-refresh=", &name)) @@ -56,6 +54,9 @@ int cmd__read_cache(int argc, const char **argv) setup_git_directory(); git_config(git_default_config, NULL); + prepare_repo_settings(r); + r->settings.command_requires_full_index = 0; + for (i = 0; i < cnt; i++) { repo_read_index(r); From patchwork Mon Nov 22 22:42:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lessley Dennington X-Patchwork-Id: 12633035 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9133C433F5 for ; Mon, 22 Nov 2021 22:42:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235054AbhKVWp5 (ORCPT ); Mon, 22 Nov 2021 17:45:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234577AbhKVWpv (ORCPT ); Mon, 22 Nov 2021 17:45:51 -0500 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6715FC061574 for ; Mon, 22 Nov 2021 14:42:44 -0800 (PST) Received: by mail-wr1-x430.google.com with SMTP id u18so35488393wrg.5 for ; Mon, 22 Nov 2021 14:42:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=5jghPv6mZTLzD8nplFijSKzie4N/s+bO9wMTV4qDxlw=; b=ZuN2r1lBJRc04pOsJiUNvrQV9c7IR4wyTtjKBQeL5i4texrlk5OHsazViMOFt+4oTm 3mnUfLZxROBhovTHerIcZKMjzk9PaeKaN7Php9FQ6RTcISqRPVhPu03s7+2q51g5l4lt m7gF80Out+ihGFOXsyoqByvEqOhHCrRC08pWPGYTTdly96oWyEZOidSj3lNy2TGl1nEq QdxivQMlGbosOh4659gZDs/JoyoBySZm/Q1J62//+oRQJqHjJnlMeYBfEk3BcyplPrep ok3NXFkR24OIzeQwDF7ESHGErgUR8PFhLZMnPtJvVkMrCPgyUogALriY2KySJUSre88O CM8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=5jghPv6mZTLzD8nplFijSKzie4N/s+bO9wMTV4qDxlw=; b=Jst9th0qqcbFylLPfgwd7i+pbOmO6h25eBMXRh6esMVkuG3HaKJuNBofkW7ArcUDmC lOR1poEFJG+39GhStL+dRjrgesWxfIcPBp36B4++48OhrbIaTa5H9vy+lCKPToHhxHWX uqr3fkHzuP3WJViLes84MQBCtwoZZAo6KGFZmRLzaPgpEjFa4Fmbx/HN2QJ6PtWzqLkk Ji+tKearyaZJ9MdAHl1TVKao8LATmtgidhGvDe9QMFIicq/jz8QCw0E++gAfxpnECitu 026Tie0S2+s0SCAToIFnpTbkW6x4nA5GzLuQpkWssmbKCc+0v6lL4uo1ZfvRpLWmmZnV 4GzQ== X-Gm-Message-State: AOAM533I3or7O8wNnFNtQN+xobcFdGwwj5U9YIaeuIVVmLdFIiJDf52l x9GIgBwktXVnN/thwZKvRsygyu1bzfY= X-Google-Smtp-Source: ABdhPJymmIsnPhy/MLLS2bFl6GEP2pn0kTVrFla8fo2BLaY4yvbTNDSahVx09JQ4bSkzodhuH2eC/Q== X-Received: by 2002:a05:6000:1201:: with SMTP id e1mr1057080wrx.298.1637620962872; Mon, 22 Nov 2021 14:42:42 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k8sm9893171wrn.91.2021.11.22.14.42.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Nov 2021 14:42:42 -0800 (PST) Message-Id: <273ee16b74ed4b36ffba5762fa892410317ea02b.1637620958.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 22 Nov 2021 22:42:37 +0000 Subject: [PATCH v4 3/4] diff: enable and test the sparse index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Lessley Dennington , Lessley Dennington Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Lessley Dennington From: Lessley Dennington Enable the sparse index within the 'git diff' command. Its implementation already safely integrates with the sparse index because it shares code with the 'git status' and 'git checkout' commands that were already integrated. For more details see: d76723e (status: use sparse-index throughout, 2021-07-14) 1ba5f45 (checkout: stop expanding sparse indexes, 2021-06-29) The most interesting thing to do is to add tests that verify that 'git diff' behaves correctly when the sparse index is enabled. These cases are: 1. The index is not expanded for 'diff' and 'diff --staged' 2. 'diff' and 'diff --staged' behave the same in full checkout, sparse checkout, and sparse index repositories in the following partially-staged scenarios (i.e. the index, HEAD, and working directory differ at a given path): 1. Path is within sparse-checkout cone 2. Path is outside sparse-checkout cone 3. A merge conflict exists for paths outside sparse-checkout cone The `p2000` tests demonstrate a ~44% execution time reduction for 'git diff' and a ~86% execution time reduction for 'git diff --staged' using a sparse index: Test before after ------------------------------------------------------------- 2000.30: git diff (full-v3) 0.33 0.34 +3.0% 2000.31: git diff (full-v4) 0.33 0.35 +6.1% 2000.32: git diff (sparse-v3) 0.53 0.31 -41.5% 2000.33: git diff (sparse-v4) 0.54 0.29 -46.3% 2000.34: git diff --cached (full-v3) 0.07 0.07 +0.0% 2000.35: git diff --cached (full-v4) 0.07 0.08 +14.3% 2000.36: git diff --cached (sparse-v3) 0.28 0.04 -85.7% 2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0% Co-authored-by: Derrick Stolee Signed-off-by: Derrick Stolee Signed-off-by: Lessley Dennington --- builtin/diff.c | 5 +++ t/perf/p2000-sparse-operations.sh | 2 ++ t/t1092-sparse-checkout-compatibility.sh | 46 ++++++++++++++++++++++++ 3 files changed, 53 insertions(+) diff --git a/builtin/diff.c b/builtin/diff.c index dd8ce688ba7..fa4683377eb 100644 --- a/builtin/diff.c +++ b/builtin/diff.c @@ -437,6 +437,11 @@ int cmd_diff(int argc, const char **argv, const char *prefix) prefix = setup_git_directory_gently(&nongit); + if (!nongit) { + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + } + if (!no_index) { /* * Treat git diff with at least one path outside of the diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index bfd332120c8..5cf94627383 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -113,5 +113,7 @@ test_perf_on_all git checkout -f - test_perf_on_all git reset test_perf_on_all git reset --hard test_perf_on_all git reset -- does-not-exist +test_perf_on_all git diff +test_perf_on_all git diff --cached test_done diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 44d5e11c762..53524660759 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -832,6 +832,52 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' ' ) ' +test_expect_success 'sparse index is not expanded: diff' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + # Add file within cone + test_sparse_match git sparse-checkout set deep && + run_on_all ../edit-contents deep/testfile && + test_all_match git add deep/testfile && + run_on_all ../edit-contents deep/testfile && + + test_all_match git diff && + test_all_match git diff --staged && + ensure_not_expanded diff && + ensure_not_expanded diff --staged && + + # Add file outside cone + test_all_match git reset --hard && + run_on_all mkdir newdirectory && + run_on_all ../edit-contents newdirectory/testfile && + test_sparse_match git sparse-checkout set newdirectory && + test_all_match git add newdirectory/testfile && + run_on_all ../edit-contents newdirectory/testfile && + test_sparse_match git sparse-checkout set && + + test_all_match git diff && + test_all_match git diff --staged && + ensure_not_expanded diff && + ensure_not_expanded diff --staged && + + # Merge conflict outside cone + # The sparse checkout will report a warning that is not in the + # full checkout, so we use `run_on_all` instead of + # `test_all_match` + run_on_all git reset --hard && + test_all_match git checkout merge-left && + test_all_match test_must_fail git merge merge-right && + + test_all_match git diff && + test_all_match git diff --staged && + ensure_not_expanded diff && + ensure_not_expanded diff --staged +' + # NEEDSWORK: a sparse-checkout behaves differently from a full checkout # in this scenario, but it shouldn't. test_expect_success 'reset mixed and checkout orphan' ' From patchwork Mon Nov 22 22:42:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lessley Dennington X-Patchwork-Id: 12633037 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08801C433FE for ; Mon, 22 Nov 2021 22:42:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234897AbhKVWp6 (ORCPT ); Mon, 22 Nov 2021 17:45:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46016 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234856AbhKVWpw (ORCPT ); Mon, 22 Nov 2021 17:45:52 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2BE8AC061714 for ; Mon, 22 Nov 2021 14:42:45 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id i8-20020a7bc948000000b0030db7b70b6bso494749wml.1 for ; Mon, 22 Nov 2021 14:42:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=gjvBVKiO7PicYvpuOn7hXg1b3+7BxCjEUKA8KQhgMZE=; b=T3s6dHVQomtkfakHWD9EYWYeohatiz2HUueXJfnAXDzclZRz2qJOkStylmC+S+/nmN UhJ8B7wfktng8zDJkM6L1Vbtm4zaW0eD6eU4yIYXF+ujfs2La9okvVlt/TrtuVYdBFBw wYC8sPIZmAqjDo9HX9c1tS1umXbleIUJzDtJc8+oBdIR/VYHgKkxYswMcDJFdDpfKvUM 17EI8kNHzl9gP+eYXPi1yPXqps3ddab1gpy5uCIT15Gaarh6DDEZR8A03wm9QbjM8HI8 VUikoOwhOJEp3hWKL46RP10Zi3UUETksTJca7Eod1WbHZh6jQPQA+V2uP41VvESelPwj IMqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=gjvBVKiO7PicYvpuOn7hXg1b3+7BxCjEUKA8KQhgMZE=; b=zht8TUTqHRGlfeTPpn+Gp1O16W/6sj+xEOh3pdSBbVhugvcPsxsjgMmn+6+1ePiFwT icmRVtM8LUZ4fkwCdveAf8HOuMG26/k2bLva2aUC9hFzrwiIkGCGD8lHz557xfIWWn25 sno4LrjMr3nN2UGGzDnQnYu5vWy1Zzl+kCCI4utEiURU7SOmFNFdjnQcCTqq1FgSjTU8 fHu4VcPYD5vCM43zukgitgXKsxjAYr1FlbDFiTUJ8UDI++aTH5F+MdWJ0HgYtegJV67d FaspvhtP9PYXau5YfD53mHHzB+gceq9B5Eb9nT0Cn9Zf7yzdz6kkA5iJ6CmlrrSwSQzG 2cog== X-Gm-Message-State: AOAM533fy0d9uYKmAl7hOCymEg7wfqBlovBHopyiDWpR2zogYlsWpZeC 1YLWuE/NzE1XrH7iHRf/L35TyMHrB+4= X-Google-Smtp-Source: ABdhPJxSBL55YoNMaQcG5riHcqvJhTJHYft+sGMAwh6UgH/ObvyvDETFzeGpR75TFZkM0OQuQFRWQw== X-Received: by 2002:a05:600c:34d6:: with SMTP id d22mr35339819wmq.111.1637620963568; Mon, 22 Nov 2021 14:42:43 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v7sm9809217wrq.25.2021.11.22.14.42.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Nov 2021 14:42:43 -0800 (PST) Message-Id: <7acf5118bf5602fbafca2d42c4f363b5adbcbd54.1637620958.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 22 Nov 2021 22:42:38 +0000 Subject: [PATCH v4 4/4] blame: enable and test the sparse index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Lessley Dennington , Lessley Dennington Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Lessley Dennington From: Lessley Dennington Enable the sparse index for the 'git blame' command. The index was already not expanded with this command, so the most interesting thing to do is to add tests that verify that 'git blame' behaves correctly when the sparse index is enabled and that its performance improves. More specifically, these cases are: 1. The index is not expanded for 'blame' when given paths in the sparse checkout cone at multiple levels. 2. Performance measurably improves for 'blame' with sparse index when given paths in the sparse checkout cone at multiple levels. The `p2000` tests demonstrate a ~60% execution time reduction when running 'blame' for a file two levels deep and and a ~30% execution time reduction for a file three levels deep. Test before after ---------------------------------------------------------------- 2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2% 2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9% 2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2% 2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6% 2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4% 2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8% 2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7% 2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5% We do not include paths outside the sparse checkout cone because blame currently does not support blaming files outside of the sparse definition. Attempting to do so fails with the following error: fatal: no such path '' in HEAD Signed-off-by: Lessley Dennington --- builtin/blame.c | 5 +++ t/perf/p2000-sparse-operations.sh | 4 +- t/t1092-sparse-checkout-compatibility.sh | 49 ++++++++++++++++++------ 3 files changed, 46 insertions(+), 12 deletions(-) diff --git a/builtin/blame.c b/builtin/blame.c index 641523ff9af..247b9eaf88f 100644 --- a/builtin/blame.c +++ b/builtin/blame.c @@ -902,6 +902,11 @@ int cmd_blame(int argc, const char **argv, const char *prefix) long anchor; const int hexsz = the_hash_algo->hexsz; + if (startup_info->have_repository) { + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + } + setup_default_color_by_age(); git_config(git_blame_config, &output_option); repo_init_revisions(the_repository, &revs, NULL); diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index 5cf94627383..9ac76a049b8 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -114,6 +114,8 @@ test_perf_on_all git reset test_perf_on_all git reset --hard test_perf_on_all git reset -- does-not-exist test_perf_on_all git diff -test_perf_on_all git diff --cached +test_perf_on_all git diff --staged +test_perf_on_all git blame $SPARSE_CONE/a +test_perf_on_all git blame $SPARSE_CONE/f3/a test_done diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 53524660759..d680dbd8867 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -442,21 +442,36 @@ test_expect_success 'log with pathspec outside sparse definition' ' test_expect_success 'blame with pathspec inside sparse definition' ' init_repos && - test_all_match git blame a && - test_all_match git blame deep/a && - test_all_match git blame deep/deeper1/a && - test_all_match git blame deep/deeper1/deepest/a + for file in a \ + deep/a \ + deep/deeper1/a \ + deep/deeper1/deepest/a + do + test_all_match git blame $file + done ' -# TODO: blame currently does not support blaming files outside of the -# sparse definition. It complains that the file doesn't exist locally. -test_expect_failure 'blame with pathspec outside sparse definition' ' +# NEEDSWORK: This test documents the current behavior, but this could +# change in the future if we decide to support blaming files outside +# the sparse definition. +test_expect_success 'blame with pathspec outside sparse definition' ' init_repos && + test_sparse_match git sparse-checkout set && - test_all_match git blame folder1/a && - test_all_match git blame folder2/a && - test_all_match git blame deep/deeper2/a && - test_all_match git blame deep/deeper2/deepest/a + for file in a \ + deep/a \ + deep/deeper1/a \ + deep/deeper1/deepest/a + do + test_sparse_match test_must_fail git blame $file && + cat >expect <<-EOF && + fatal: Cannot lstat '"'"'$file'"'"': No such file or directory + EOF + # We compare sparse-checkout-err and sparse-index-err in + # `test_sparse_match`. Given we know they are the same, we + # only check the content of sparse-index-err here. + test_cmp expect sparse-index-err + done ' test_expect_success 'checkout and reset (mixed)' ' @@ -878,6 +893,18 @@ test_expect_success 'sparse index is not expanded: diff' ' ensure_not_expanded diff --staged ' +test_expect_success 'sparse index is not expanded: blame' ' + init_repos && + + for file in a \ + deep/a \ + deep/deeper1/a \ + deep/deeper1/deepest/a + do + ensure_not_expanded blame $file + done +' + # NEEDSWORK: a sparse-checkout behaves differently from a full checkout # in this scenario, but it shouldn't. test_expect_success 'reset mixed and checkout orphan' '