From patchwork Mon Nov 29 15:52:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12645537 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46E6DC433F5 for ; Mon, 29 Nov 2021 19:35:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349986AbhK2TiX (ORCPT ); Mon, 29 Nov 2021 14:38:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48810 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350026AbhK2TgS (ORCPT ); Mon, 29 Nov 2021 14:36:18 -0500 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20D94C0617A2 for ; Mon, 29 Nov 2021 07:52:47 -0800 (PST) Received: by mail-wr1-x432.google.com with SMTP id a9so37839274wrr.8 for ; Mon, 29 Nov 2021 07:52:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ldyVwS09nc2OLJia/8NlZArMqqWxVI8EkShpKdO52GA=; b=ZchwIldkQ2DVpcBPG0872Q1OO2nbclkO+FMReM4G6UZVOQz5+zxuDDWau/cg2vAGdc o+sa3eqjZ7SqG1/7JTpSrVMeqjruShid3jxNeEV3cNder5LnBHpBxPwch3mKZCAqqIAk JrELSpcuS+5UCtKZTSM0uqSBp/qKXV8d1+T5J55khA1rccgdfwJkngBD5CgQFeH4YLNG /VpqnnVTsAlKE1ZlUz38aimfjWlr4aJSEtatdl5JUlw5MF8xkuVe2BG8uSqaM9pTs/73 Dv7hydM0u/u2IGzqcZXvPr/pB8zLIERODVNG07FbIDofVZTAMNQvT4dF+CgBDnp3pJGY PQew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ldyVwS09nc2OLJia/8NlZArMqqWxVI8EkShpKdO52GA=; b=fozqcsP3owFXIOEGqEkj68t51ikd/xaXF41VGsXNFLXgzsa2mGMOh05M5ogVaqY6pq 6frgO9hPHzi/N3240YFqt9JS14lketHQzx3h5fuALho/ZuC/qlB1hUX+FPJOVgi3cXC4 I/8yJvaY194sL5LLTJeMrPZa1xILLekMXgcV80rdkt0pU1yU86ASnaUCAcv3sWc7nV0f hE/bA/3XPlSwWKNBHAFuqEh0jr8nGlpUpSPr37fid6BHc6Hzb1pQGefKbKiaN+iv53Nr oBx3nBLFXni2qd5SYVOb+1TqvbJty0E2LtKzTHdm/w7R2Awufas+ZBiSDMW5c2+fAAm7 u7Rg== X-Gm-Message-State: AOAM530twatd2PTKhYzZh8ereRfZJzcNEQqIpNNqXrUlGfBxxR/Vtu4F Ix6yUHhB1uGbxJWPx2pR5P2zJ8iDMiY= X-Google-Smtp-Source: ABdhPJw7h+ZPygZ1TYPmmxGQhQxuwt0AFi/PVo0CiHKlyZYjdqyzcfSzcFbGZVFQppKktFWJy1KtIQ== X-Received: by 2002:adf:e842:: with SMTP id d2mr35004039wrn.399.1638201165601; Mon, 29 Nov 2021 07:52:45 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z15sm13917625wrr.65.2021.11.29.07.52.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Nov 2021 07:52:45 -0800 (PST) Message-Id: In-Reply-To: References: Date: Mon, 29 Nov 2021 15:52:36 +0000 Subject: [PATCH v6 1/8] reset: rename is_missing to !is_in_reset_tree Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Rename and invert value of `is_missing` to `is_in_reset_tree` to make the variable more descriptive of what it represents. Signed-off-by: Victoria Dye --- builtin/reset.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index 51c9e2f43ff..d3695ce43c4 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -131,10 +131,10 @@ static void update_index_from_diff(struct diff_queue_struct *q, for (i = 0; i < q->nr; i++) { struct diff_filespec *one = q->queue[i]->one; - int is_missing = !(one->mode && !is_null_oid(&one->oid)); + int is_in_reset_tree = one->mode && !is_null_oid(&one->oid); struct cache_entry *ce; - if (is_missing && !intent_to_add) { + if (!is_in_reset_tree && !intent_to_add) { remove_file_from_cache(one->path); continue; } @@ -144,7 +144,7 @@ static void update_index_from_diff(struct diff_queue_struct *q, if (!ce) die(_("make_cache_entry failed for path '%s'"), one->path); - if (is_missing) { + if (!is_in_reset_tree) { ce->ce_flags |= CE_INTENT_TO_ADD; set_object_name_for_intent_to_add_entry(ce); } From patchwork Mon Nov 29 15:52:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12645539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F1C7C433EF for ; Mon, 29 Nov 2021 19:35:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352130AbhK2Ti6 (ORCPT ); Mon, 29 Nov 2021 14:38:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48488 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238675AbhK2Tg5 (ORCPT ); Mon, 29 Nov 2021 14:36:57 -0500 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA83FC0617A5 for ; Mon, 29 Nov 2021 07:52:47 -0800 (PST) Received: by mail-wm1-x335.google.com with SMTP id i8-20020a7bc948000000b0030db7b70b6bso17200377wml.1 for ; Mon, 29 Nov 2021 07:52:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=BOiihwKfyp8h1ZPOwc/Ncb03jmtRWNTQQAMmoxR9+V4=; b=QuNdrwxrrie9yYqutw2LaD/Pc+vqFO7D53yCw/WUpnq9PgXMDFRzc/54Eanb8jLtpe GHZMTZhrY3KrNjcXG1GoFT6qrd0qEjqKb4esSA+Cx9YpkqmKZULVR5rckFeQQHSYjWSX wG7iVmZb+Vg/5Xjln1EtrC6L3pDc71W37N7DM2IANa+7dgF/C3uyAJeZ3U4uHULaOj/R 1lpylzbbevWRSstWMbE2Fodk2yYMZMUo0ifNDgcsZ+Fititt8ZP6PX4DyI5ENY/GZPro Am2MsIK/4fx+mxeJmadHY3WzJZeBm3umL2YKJsXpaf9nivnS2wwKBy+BpKbr+tO2qQoc SgCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=BOiihwKfyp8h1ZPOwc/Ncb03jmtRWNTQQAMmoxR9+V4=; b=EPcIDxHbsiupfj/izJM14emTXfPJGFw8L9u4Ol82Fmx4e3nbRZ7/3UKKYOanSDEgKr npnD2YoEdPv3QCI6yxFnX8SUmmnY3OabNq7Nh8h+v3Gw/biWp3YXHm1uRsKmfpgm0elR b60cAugteYjfe3g8mQrO5qowZNZmtWj0CEbxFrCmj1oANjwiymKAySo7G4CM9oHmorxW a2/q4ZFUHZkyNnkQevHU360vv00LRTMw0dKta1SZj75CHNOhXbaIAMgM2EsNgRHB03+F PPNS1NFAADnFl6YW6sjQqk70ZeVoBGiv6AUm5HKzV8ylGPGHgHebPbVPN//BJ1EQ7WM6 w8dA== X-Gm-Message-State: AOAM5330Zgd8xjcIn6Sb6/GJlB/TJYZQLxe+FGC2FWhTCI3Q3T8MzIj0 +648zmRuzel/bRiMzl/LUfNEUo6IeNE= X-Google-Smtp-Source: ABdhPJyIbs+ryPeAN62aPbfejtmxOHSzQO1i7Xr4X2t407Fqoi2wopZ/6NWWwdSg3SMqcMw+Jfy6wg== X-Received: by 2002:a1c:9851:: with SMTP id a78mr37060500wme.116.1638201166268; Mon, 29 Nov 2021 07:52:46 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s63sm16185979wme.22.2021.11.29.07.52.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Nov 2021 07:52:45 -0800 (PST) Message-Id: In-Reply-To: References: Date: Mon, 29 Nov 2021 15:52:37 +0000 Subject: [PATCH v6 2/8] reset: preserve skip-worktree bit in mixed reset Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Change `update_index_from_diff` to set `skip-worktree` when applicable for new index entries. When `git reset --mixed ` is run, entries in the index with differences between the pre-reset HEAD and reset are identified and handled with `update_index_from_diff`. For each file, a new cache entry in inserted into the index, created from the side of the reset (without changing the working tree). However, the newly-created entry must have `skip-worktree` explicitly set in either of the following scenarios: 1. the file is in the current index and has `skip-worktree` set 2. the file is not in the current index but is outside of a defined sparse checkout definition Not setting the `skip-worktree` bit leads to likely-undesirable results for a user. It causes `skip-worktree` settings to disappear on the "diff"-containing files (but *only* the diff-containing files), leading to those files now showing modifications in `git status`. For example, when running `git reset --mixed` in a sparse checkout, some file entries outside of sparse checkout could show up as deleted, despite the user never deleting anything (and not wanting them on-disk anyway). Additionally, add a test to `t7102` to ensure `skip-worktree` is preserved in a basic `git reset --mixed` scenario and update a failure-documenting test from 19a0acc (t1092: test interesting sparse-checkout scenarios, 2021-01-23) with new expected behavior. Helped-by: Junio C Hamano Signed-off-by: Victoria Dye --- builtin/reset.c | 14 ++++++++++++++ t/t1092-sparse-checkout-compatibility.sh | 22 ++++++++-------------- t/t7102-reset.sh | 17 +++++++++++++++++ 3 files changed, 39 insertions(+), 14 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index d3695ce43c4..e441b6601b9 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -25,6 +25,7 @@ #include "cache-tree.h" #include "submodule.h" #include "submodule-config.h" +#include "dir.h" #define REFRESH_INDEX_DELAY_WARNING_IN_MS (2 * 1000) @@ -130,6 +131,7 @@ static void update_index_from_diff(struct diff_queue_struct *q, int intent_to_add = *(int *)data; for (i = 0; i < q->nr; i++) { + int pos; struct diff_filespec *one = q->queue[i]->one; int is_in_reset_tree = one->mode && !is_null_oid(&one->oid); struct cache_entry *ce; @@ -141,6 +143,18 @@ static void update_index_from_diff(struct diff_queue_struct *q, ce = make_cache_entry(&the_index, one->mode, &one->oid, one->path, 0, 0); + + /* + * If the file 1) corresponds to an existing index entry with + * skip-worktree set, or 2) does not exist in the index but is + * outside the sparse checkout definition, add a skip-worktree bit + * to the new index entry. + */ + pos = cache_name_pos(one->path, strlen(one->path)); + if ((pos >= 0 && ce_skip_worktree(active_cache[pos])) || + (pos < 0 && !path_in_sparse_checkout(one->path, &the_index))) + ce->ce_flags |= CE_SKIP_WORKTREE; + if (!ce) die(_("make_cache_entry failed for path '%s'"), one->path); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 886e78715fe..c7449afe965 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -459,26 +459,20 @@ test_expect_failure 'blame with pathspec outside sparse definition' ' test_all_match git blame deep/deeper2/deepest/a ' -# NEEDSWORK: a sparse-checkout behaves differently from a full checkout -# in this scenario, but it shouldn't. -test_expect_failure 'checkout and reset (mixed)' ' +test_expect_success 'checkout and reset (mixed)' ' init_repos && test_all_match git checkout -b reset-test update-deep && test_all_match git reset deepest && - test_all_match git reset update-folder1 && - test_all_match git reset update-folder2 -' - -# NEEDSWORK: a sparse-checkout behaves differently from a full checkout -# in this scenario, but it shouldn't. -test_expect_success 'checkout and reset (mixed) [sparse]' ' - init_repos && - test_sparse_match git checkout -b reset-test update-deep && - test_sparse_match git reset deepest && + # Because skip-worktree is preserved, resetting to update-folder1 + # will show worktree changes for folder1/a in full-checkout, but not + # in sparse-checkout or sparse-index. + git -C full-checkout reset update-folder1 >full-checkout-out && test_sparse_match git reset update-folder1 && - test_sparse_match git reset update-folder2 + grep "M folder1/a" full-checkout-out && + ! grep "M folder1/a" sparse-checkout-out && + run_on_sparse test_path_is_missing folder1 ' test_expect_success 'merge, cherry-pick, and rebase' ' diff --git a/t/t7102-reset.sh b/t/t7102-reset.sh index 601b2bf97f0..d05426062ec 100755 --- a/t/t7102-reset.sh +++ b/t/t7102-reset.sh @@ -472,6 +472,23 @@ test_expect_success '--mixed refreshes the index' ' test_cmp expect output ' +test_expect_success '--mixed preserves skip-worktree' ' + echo 123 >>file2 && + git add file2 && + git update-index --skip-worktree file2 && + git reset --mixed HEAD >output && + test_must_be_empty output && + + cat >expect <<-\EOF && + Unstaged changes after reset: + M file2 + EOF + git update-index --no-skip-worktree file2 && + git add file2 && + git reset --mixed HEAD >output && + test_cmp expect output +' + test_expect_success 'resetting specific path that is unmerged' ' git rm --cached file2 && F1=$(git rev-parse HEAD:file1) && From patchwork Mon Nov 29 15:52:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12645541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50674C433F5 for ; Mon, 29 Nov 2021 19:35:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352182AbhK2Ti7 (ORCPT ); Mon, 29 Nov 2021 14:38:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238805AbhK2Tg5 (ORCPT ); Mon, 29 Nov 2021 14:36:57 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7806FC061A26 for ; Mon, 29 Nov 2021 07:52:48 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id a18so37859969wrn.6 for ; Mon, 29 Nov 2021 07:52:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=rWdQm5Zkkxjg4tRvPMLEws5NQlPaszk/3jdQlcrd+H8=; b=MMBPAdPO443VYdA8XQaDv85gncWMjuwd+71rtrdbUKTG/P/414X1jgikVmWKE4tl56 CB3Ds3FyoWkb4zOR+eEk6y04NGaEZWWxHKZGHOLXVZCJq23zYTyrxs/6t/Xb8W155YgJ O5nR8Mf99Yr+9piikbveXZrW5VTL+lePDtb8bGLfrRHUz6SqD7HEk1Ywm4J8Dd17/NzU GffaVRbRZ8Jj1axOIGHPgook6VWT+8MF/vZq53eq/34Kerb0ZNnV9wzl2HkYub5RRgCA FT0JaVH1eSp0wzLDXUbiZ68LYQXUSTUqvKOzazB90OL1+W9KRFxQXusV3BlNxRbfMI9o fQsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=rWdQm5Zkkxjg4tRvPMLEws5NQlPaszk/3jdQlcrd+H8=; b=JogAOpFhGHneGoKUPsMt3GUboeC7Jw79AOGthHRAhGpqOA2JEHkS7KeFGN+lfPg1Sq catkbb8S9I4U+879NlcUqml7RdCLDGEY6omqduVIbbxNXsguaR62KCQyyInTleLVVHG3 PDPuwaz9Z90vYHQikMhuH+m7M9ZJau3qqzbdNhp2QswcNGs6+9s2PM0CPGky4EmYki3S vlJ39ZQPZav4U+b4k1HaUTzpy8H7rCFgN4NDlgdsw+ILdw1Xf76BujkXLdi9wwNdr+O+ QITRJSoPLCcRsV3pvYB4JBBmU6//+6QzKXrdiysh0JvuYqFpIlptb1o7Am2kJHumtOOL Hiag== X-Gm-Message-State: AOAM533pNJUdlI+C1ZjLZ7NrEOSnI6h4QKZUyI6bi3OczMduZouPOV7g lCheoUTgOHk2qC1yewGzL+Lm9mkWlI8= X-Google-Smtp-Source: ABdhPJy6mhwrvB6WoMmt9E0rHAbdU4/gmS+XkUDceu4Uu36Kb04fM9Dqwsk3aeF4xQHC3zLLRlmdPg== X-Received: by 2002:a5d:4a0b:: with SMTP id m11mr34292040wrq.120.1638201166993; Mon, 29 Nov 2021 07:52:46 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c6sm26092774wmq.46.2021.11.29.07.52.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Nov 2021 07:52:46 -0800 (PST) Message-Id: <1bb2ca92c60be453d7b151e0a1fb5d471a2e5090.1638201164.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 29 Nov 2021 15:52:38 +0000 Subject: [PATCH v6 3/8] sparse-index: update command for expand/collapse test Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye In anticipation of `git reset --hard` being able to use the sparse index without expanding it, replace the command in `sparse-index is expanded and converted back` with `git reset -- folder1/a`. This command will need to expand the index to work properly, even after integrating the rest of `reset` with sparse index. Helped-by: Derrick Stolee Signed-off-by: Victoria Dye --- t/t1092-sparse-checkout-compatibility.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index c7449afe965..cab6340a9d0 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -634,11 +634,15 @@ test_expect_success 'submodule handling' ' grep "160000 commit $(git -C initial-repo rev-parse HEAD) modules/sub" cache ' +# When working with a sparse index, some commands will need to expand the +# index to operate properly. If those commands also write the index back +# to disk, they need to convert the index to sparse before writing. +# This test verifies that both of these events are logged in trace2 logs. test_expect_success 'sparse-index is expanded and converted back' ' init_repos && GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ - git -C sparse-index -c core.fsmonitor="" reset --hard && + git -C sparse-index reset -- folder1/a && test_region index convert_to_sparse trace2.txt && test_region index ensure_full_index trace2.txt ' From patchwork Mon Nov 29 15:52:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12645543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8783C433F5 for ; Mon, 29 Nov 2021 19:35:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352388AbhK2TjC (ORCPT ); Mon, 29 Nov 2021 14:39:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48500 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238953AbhK2Tg5 (ORCPT ); Mon, 29 Nov 2021 14:36:57 -0500 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2371BC061A27 for ; Mon, 29 Nov 2021 07:52:49 -0800 (PST) Received: by mail-wm1-x32b.google.com with SMTP id 137so15073957wma.1 for ; Mon, 29 Nov 2021 07:52:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=RoWCfn6JRRfStiyXqXlPgVJh3RC3yln4UjwHY17j5yg=; b=ZmwGISjoMGZ/P2JnZg2u6NxGGXfY/C11mS2HxTxmrDahQ2fWp28G2wKyKrwZYiZmyQ DjR4/q45q7729VxfLZShvZbGAa5l61BW6rzYRZlGjqYiejBY//geEYvcBszR9nC8GnVd DUQPQNOoxtsJYC6vbZ+ATl8ZuJRzcP/Ap17ntrxE4VMKXZ1/HqPIiJ1Z/FAtq0Y6bkkj oRvKlvTLS/oQezm2QqFb6bNyvV7lty5Ws660+7kSihSZD9jYhlOlHxxctTnQa/qYjQxt owPTm7F7mokG0VS06thfB54OK375gcjRp5+xealzU0smP7akm28ONeU2ITXBHxh6aNi2 n0eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=RoWCfn6JRRfStiyXqXlPgVJh3RC3yln4UjwHY17j5yg=; b=prDOzyUsu8W1/HWqVnmIPLSjoqmXLJmaRaoQwHyOmEl4ZUUgrxgvEihcmjfTahwIIM rEsbUF312qQ7vPvMzUFWqTfiydXjPOSbe81MtYR899Mn6iI/i7RD42CJeO4mIgVbUGnS +u9iTRZ5UzwkXtJKrssRskmGBQUscXLcObHi38NSvAzEBS1Jm5p2/RKOYb57y19OZ4t5 idQwNCmzwXgMDYIE42H6d+Ze+AgLHU/O2nkOWEgHFIfZ5wdWMUx7GaOm0YFjadjPsc/C uF+cCGouUScaPMM1/pSoTC8WqwXhuKpFM5XLr2rz591b0PEOAHY/vIhgnQb4J62bmZcG A3vA== X-Gm-Message-State: AOAM531aCDenWVyKQyySp+PvtUiSGUtIz3yHafm8Aml7urKQD1maVj5V rPaxVan5ckjQojAVQXtXOZXh1ngRPXc= X-Google-Smtp-Source: ABdhPJymAyS3M/EAVfZpd4e6xKbxkDsr5NohDbpaebnRo9UCmCohvQW40i4qD6n6W2oq+/EJ48xing== X-Received: by 2002:a05:600c:1548:: with SMTP id f8mr37739364wmg.67.1638201167609; Mon, 29 Nov 2021 07:52:47 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d9sm13940578wre.52.2021.11.29.07.52.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Nov 2021 07:52:47 -0800 (PST) Message-Id: <741a2c9ffaa8f3ea5e7fc7d3d44cf0ed3654cff2.1638201164.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 29 Nov 2021 15:52:39 +0000 Subject: [PATCH v6 4/8] reset: expand test coverage for sparse checkouts Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Add new tests for `--merge` and `--keep` modes, as well as mixed reset with pathspecs. New performance test cases exercise various execution paths for `reset`. Co-authored-by: Derrick Stolee Signed-off-by: Derrick Stolee Signed-off-by: Victoria Dye --- t/perf/p2000-sparse-operations.sh | 3 + t/t1092-sparse-checkout-compatibility.sh | 98 ++++++++++++++++++++++++ 2 files changed, 101 insertions(+) diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index 597626276fb..bfd332120c8 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -110,5 +110,8 @@ test_perf_on_all git add -A test_perf_on_all git add . test_perf_on_all git commit -a -m A test_perf_on_all git checkout -f - +test_perf_on_all git reset +test_perf_on_all git reset --hard +test_perf_on_all git reset -- does-not-exist test_done diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index cab6340a9d0..4125525ab86 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -475,6 +475,104 @@ test_expect_success 'checkout and reset (mixed)' ' run_on_sparse test_path_is_missing folder1 ' +test_expect_success 'checkout and reset (merge)' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents a && + test_all_match git reset --merge deepest && + test_all_match git status --porcelain=v2 && + + test_all_match git reset --hard update-deep && + run_on_all ../edit-contents deep/a && + test_all_match test_must_fail git reset --merge deepest +' + +test_expect_success 'checkout and reset (keep)' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents a && + test_all_match git reset --keep deepest && + test_all_match git status --porcelain=v2 && + + test_all_match git reset --hard update-deep && + run_on_all ../edit-contents deep/a && + test_all_match test_must_fail git reset --keep deepest +' + +test_expect_success 'reset with pathspecs inside sparse definition' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents deep/a && + + test_all_match git reset base -- deep/a && + test_all_match git status --porcelain=v2 && + + test_all_match git reset base -- nonexistent-file && + test_all_match git status --porcelain=v2 && + + test_all_match git reset deepest -- deep && + test_all_match git status --porcelain=v2 +' + +# Although the working tree differs between full and sparse checkouts after +# reset, the state of the index is the same. +test_expect_success 'reset with pathspecs outside sparse definition' ' + init_repos && + test_all_match git checkout -b reset-test base && + + test_sparse_match git reset update-folder1 -- folder1 && + git -C full-checkout reset update-folder1 -- folder1 && + test_sparse_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder1 && + + test_sparse_match git reset update-folder2 -- folder2/a && + git -C full-checkout reset update-folder2 -- folder2/a && + test_sparse_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder2/a +' + +test_expect_success 'reset with wildcard pathspec' ' + init_repos && + + test_all_match git reset update-deep -- deep\* && + test_all_match git ls-files -s -- deep && + + test_all_match git reset deepest -- deep\*\*\* && + test_all_match git ls-files -s -- deep && + + # The following `git reset`s result in updating the index on files with + # `skip-worktree` enabled. To avoid failing due to discrepencies in reported + # "modified" files, `test_sparse_match` reset is performed separately from + # "full-checkout" reset, then the index contents of all repos are verified. + + test_sparse_match git reset update-folder1 -- \*/a && + git -C full-checkout reset update-folder1 -- \*/a && + test_all_match git ls-files -s -- deep/a folder1/a && + + test_sparse_match git reset update-folder2 -- folder\* && + git -C full-checkout reset update-folder2 -- folder\* && + test_all_match git ls-files -s -- folder10 folder1 folder2 && + + test_sparse_match git reset base -- folder1/\* && + git -C full-checkout reset base -- folder1/\* && + test_all_match git ls-files -s -- folder1 +' + test_expect_success 'merge, cherry-pick, and rebase' ' init_repos && From patchwork Mon Nov 29 15:52:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12645545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAA04C433EF for ; Mon, 29 Nov 2021 19:35:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233628AbhK2TjF (ORCPT ); Mon, 29 Nov 2021 14:39:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238838AbhK2Tg5 (ORCPT ); Mon, 29 Nov 2021 14:36:57 -0500 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC46DC09B13F for ; Mon, 29 Nov 2021 07:52:49 -0800 (PST) Received: by mail-wr1-x436.google.com with SMTP id u1so37779153wru.13 for ; Mon, 29 Nov 2021 07:52:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+dch2aTLAv85TXXA5lgJ4m8qNH4dps5WqL8q9u/O5kk=; b=Fg71+/Bz75h/TbYRloiE0H2jhF106Pu0WGL1EeYXQB7EY7griarSlZSq2zY8GYaTc+ +J/W+nQHdeihcPgc24yGLOh26tuGEH1+kiT4H7KMYI5Htu5AfLAjD/Yj9LyG/NenzZdf TCfxO5ZVSYWdN/0fkyNMOCOrfsq1J1g+IJxCO0jgriAVO+5iy2VOQlx3S3rVvxNEVulj EzhzZFnB/h3GBv1KnyXO2vmvBTOWy2XpGw3q2wU+AA/Lt1EPvivqZUeAmdZOUz7oHCOi xu9L0PvdCAYCc7aA3yZTj89yu5GrVP4dqF3/pyUaiVBbYBk7OWJ+oKjrwUPprHFTyoUQ Tg+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+dch2aTLAv85TXXA5lgJ4m8qNH4dps5WqL8q9u/O5kk=; b=uf2/oWVdM3+xHgrpMKR5leX1/yYlJ6wZq7BNkP/tFJouvqz+EaoJP5Fw9OpNOKmoeO Hvlb8C0hBT0Qmy/oOxyIX5sMuxjfiKP5O7sUnAArAjzKTdknYbykxRKlINnWeT4l2IKy Watjbujq/KW2JEvSq6PceKw+i76pDM03O2YqIHjFGBbpDbuw1u5BSHfoukk84aJcYShr Kps8R8L/E6QMNiL7JXiHSpVv3oBWk0mni1k6hV0Za1E+rc2lv0EQQWd+ILnPUKV23QmO 8Ard9GrRgKjo6UsmNle8coqST29rh0wTNn+KhS3AVxKEVGwzHGOHSIwZstA7Ccop03k5 rCXg== X-Gm-Message-State: AOAM53101kKuA6bv9vaZjCEk5QBmQ/R4bTaDUAxZnW3aqRI6XXMuUyO3 1NwD2AbR3y3lctF+JPtv65g4fXJiTGM= X-Google-Smtp-Source: ABdhPJzf//UVq5fjn6nTNkSX2Kq/OFE8kwhR2yhHxU+QXtaxv2mcPmIkOQC+B4Ton5Y/FBvw4YSDYg== X-Received: by 2002:a05:6000:1043:: with SMTP id c3mr34737374wrx.64.1638201168299; Mon, 29 Nov 2021 07:52:48 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b197sm14977525wmb.24.2021.11.29.07.52.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Nov 2021 07:52:47 -0800 (PST) Message-Id: <65b0eafd27c76a8b4ced38a5521413c1c23fc2bf.1638201164.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 29 Nov 2021 15:52:40 +0000 Subject: [PATCH v6 5/8] reset: integrate with sparse index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Disable `command_requires_full_index` repo setting and add `ensure_full_index` guards around code paths that cannot yet use sparse directory index entries. `reset --soft` does not modify the index, so no compatibility changes are needed for it to function without expanding the index. For all other reset modes (`--mixed`, `--hard`, `--keep`, `--merge`), the full index is expanded to prevent cache tree corruption and invalid variable accesses. Additionally, the `read_cache()` check verifying an uncorrupted index is moved after argument parsing and preparing the repo settings. The index is not used by the preceding argument handling, but `read_cache()` must be run *after* enabling sparse index for the command (so that the index is not expanded unnecessarily) and *before* using the index for reset (so that it is verified as uncorrupted). Signed-off-by: Victoria Dye --- builtin/reset.c | 10 +++++++--- cache-tree.c | 1 + 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index e441b6601b9..0ac0de7dc97 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -180,6 +180,7 @@ static int read_from_tree(const struct pathspec *pathspec, opt.flags.override_submodule_config = 1; opt.repo = the_repository; + ensure_full_index(&the_index); if (do_diff_cache(tree_oid, &opt)) return 1; diffcore_std(&opt); @@ -257,9 +258,6 @@ static void parse_args(struct pathspec *pathspec, } *rev_ret = rev; - if (read_cache() < 0) - die(_("index file corrupt")); - parse_pathspec(pathspec, 0, PATHSPEC_PREFER_FULL | (patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0), @@ -405,6 +403,12 @@ int cmd_reset(int argc, const char **argv, const char *prefix) if (intent_to_add && reset_type != MIXED) die(_("-N can only be used with --mixed")); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + + if (read_cache() < 0) + die(_("index file corrupt")); + /* Soft reset does not touch the index file nor the working tree * at all, but requires them in a good order. Other resets reset * the index file to the tree object we are switching to. */ diff --git a/cache-tree.c b/cache-tree.c index 90919f9e345..9be19c85b66 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -776,6 +776,7 @@ void prime_cache_tree(struct repository *r, cache_tree_free(&istate->cache_tree); istate->cache_tree = cache_tree(); + ensure_full_index(istate); prime_cache_tree_rec(r, istate->cache_tree, tree); istate->cache_changed |= CACHE_TREE_CHANGED; trace2_region_leave("cache-tree", "prime_cache_tree", the_repository); From patchwork Mon Nov 29 15:52:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12645547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46D6AC433F5 for ; Mon, 29 Nov 2021 19:35:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352492AbhK2TjG (ORCPT ); Mon, 29 Nov 2021 14:39:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48496 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229478AbhK2Tg6 (ORCPT ); Mon, 29 Nov 2021 14:36:58 -0500 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7C15C09B18C for ; Mon, 29 Nov 2021 07:52:50 -0800 (PST) Received: by mail-wr1-x430.google.com with SMTP id s13so37903073wrb.3 for ; Mon, 29 Nov 2021 07:52:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=3s/qAAEmURIchvWtfDhw3vOEGGtlnQ4I11yNkVLL8xc=; b=Mpjulo+gdBX4SA5XIx5c+5GoZWjK/iB4DBVTMaiFZJmjBsXRqTPUZygF3W6zXRI+sa 1A5wBW07meYJ++QdPDP3YiqTLXF1Ktn59xpwA2Tl2VYZ4drW5dStmY+/L6hJlzQEK9A7 owgEQ1EHa7soqZhRAec1jQOb875IESFAMmiDBMRaKEIzUltxL0FYBLhfUUEADd0RHAe0 u1QAR6z2lSrvdg3IaJjpjrfrIo0JttP2Q9Nd5jHCurvL98KdIiJ/RdOMp9+8Ka//Wsay pjI03PN39C0TPrVx0/tygnIXjSSqBbj1ZQKWMlXMDb0hbV8ZCq0ezwCsIxIzIV65e/Tp 3Yig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=3s/qAAEmURIchvWtfDhw3vOEGGtlnQ4I11yNkVLL8xc=; b=z6QMN0fTGo6qB03jSS+UAyppYHySqvapd/Ly4HLqMcGyVSrAw+uPalktQdeMid5m93 6W73YHhfr/BD9ZVy/BIF1rWajGaVmgEefu+3KsHmRxR1wQ3iHL8nZ/Ac3w+hXfLEGtmq wmpIPkcJSWk7j+7A2CgxWnl5c5tqPuA7mNntOrpwbuL5EL7J/HHjQ+i0W5qCPN3dqYr+ JaNE3bbB/PbgRZTa72z0vmuxgr3doF4IcKCfMljNZuw7UFdtu2L5PmFBLi9mp9IlEq2t +lKRvFN1FPJ+8X59MngLJjYCYZqnw9/WIXVNT+DZIApotne+ry9HT5dGvBdYfJdpsSpm so2g== X-Gm-Message-State: AOAM531NgrHr5HnWJVtUdc7zJbjrdOfOVY7JwUbkOTu1cXn+QyM+aSZV Zf3ApOQX2sTI+RgaXJ4PIyvDFFzhdsU= X-Google-Smtp-Source: ABdhPJzd425oQ32il0cbRo3T0GZ4/isdJmTzbYFj04pqdwheTbmWyKmLTFbWxTkaSdKhFBKPylGC0g== X-Received: by 2002:a5d:59af:: with SMTP id p15mr36574194wrr.314.1638201169010; Mon, 29 Nov 2021 07:52:49 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u2sm16586808wrs.17.2021.11.29.07.52.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Nov 2021 07:52:48 -0800 (PST) Message-Id: <908c84005b9b0961a0a5b3b3c34b841a884c0ada.1638201164.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 29 Nov 2021 15:52:41 +0000 Subject: [PATCH v6 6/8] reset: make sparse-aware (except --mixed) Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Remove `ensure_full_index` guard on `prime_cache_tree` and update `prime_cache_tree_rec` to correctly reconstruct sparse directory entries in the cache tree. While processing a tree's entries, `prime_cache_tree_rec` must determine whether a directory entry is sparse or not by searching for it in the index (*without* expanding the index). If a matching sparse directory index entry is found, no subtrees are added to the cache tree entry and the entry count is set to 1 (representing the sparse directory itself). Otherwise, the tree is assumed to not be sparse and its subtrees are recursively added to the cache tree. Helped-by: Elijah Newren Signed-off-by: Victoria Dye --- cache-tree.c | 47 ++++++++++++++++++++++-- cache.h | 10 +++++ read-cache.c | 27 ++++++++++---- t/t1092-sparse-checkout-compatibility.sh | 15 +++++++- 4 files changed, 86 insertions(+), 13 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index 9be19c85b66..2866101052c 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -740,15 +740,26 @@ out: return ret; } +static void prime_cache_tree_sparse_dir(struct cache_tree *it, + struct tree *tree) +{ + + oidcpy(&it->oid, &tree->object.oid); + it->entry_count = 1; +} + static void prime_cache_tree_rec(struct repository *r, struct cache_tree *it, - struct tree *tree) + struct tree *tree, + struct strbuf *tree_path) { struct tree_desc desc; struct name_entry entry; int cnt; + int base_path_len = tree_path->len; oidcpy(&it->oid, &tree->object.oid); + init_tree_desc(&desc, tree->buffer, tree->size); cnt = 0; while (tree_entry(&desc, &entry)) { @@ -757,14 +768,40 @@ static void prime_cache_tree_rec(struct repository *r, else { struct cache_tree_sub *sub; struct tree *subtree = lookup_tree(r, &entry.oid); + if (!subtree->object.parsed) parse_tree(subtree); sub = cache_tree_sub(it, entry.path); sub->cache_tree = cache_tree(); - prime_cache_tree_rec(r, sub->cache_tree, subtree); + + /* + * Recursively-constructed subtree path is only needed when working + * in a sparse index (where it's used to determine whether the + * subtree is a sparse directory in the index). + */ + if (r->index->sparse_index) { + strbuf_setlen(tree_path, base_path_len); + strbuf_grow(tree_path, base_path_len + entry.pathlen + 1); + strbuf_add(tree_path, entry.path, entry.pathlen); + strbuf_addch(tree_path, '/'); + } + + /* + * If a sparse index is in use, the directory being processed may be + * sparse. To confirm that, we can check whether an entry with that + * exact name exists in the index. If it does, the created subtree + * should be sparse. Otherwise, cache tree expansion should continue + * as normal. + */ + if (r->index->sparse_index && + index_entry_exists(r->index, tree_path->buf, tree_path->len)) + prime_cache_tree_sparse_dir(sub->cache_tree, subtree); + else + prime_cache_tree_rec(r, sub->cache_tree, subtree, tree_path); cnt += sub->cache_tree->entry_count; } } + it->entry_count = cnt; } @@ -772,12 +809,14 @@ void prime_cache_tree(struct repository *r, struct index_state *istate, struct tree *tree) { + struct strbuf tree_path = STRBUF_INIT; + trace2_region_enter("cache-tree", "prime_cache_tree", the_repository); cache_tree_free(&istate->cache_tree); istate->cache_tree = cache_tree(); - ensure_full_index(istate); - prime_cache_tree_rec(r, istate->cache_tree, tree); + prime_cache_tree_rec(r, istate->cache_tree, tree, &tree_path); + strbuf_release(&tree_path); istate->cache_changed |= CACHE_TREE_CHANGED; trace2_region_leave("cache-tree", "prime_cache_tree", the_repository); } diff --git a/cache.h b/cache.h index f6295f3b048..1d3e4665562 100644 --- a/cache.h +++ b/cache.h @@ -816,6 +816,16 @@ struct cache_entry *index_file_exists(struct index_state *istate, const char *na */ int index_name_pos(struct index_state *, const char *name, int namelen); +/* + * Determines whether an entry with the given name exists within the + * given index. The return value is 1 if an exact match is found, otherwise + * it is 0. Note that, unlike index_name_pos, this function does not expand + * the index if it is sparse. If an item exists within the full index but it + * is contained within a sparse directory (and not in the sparse index), 0 is + * returned. + */ +int index_entry_exists(struct index_state *, const char *name, int namelen); + /* * Some functions return the negative complement of an insert position when a * precise match was not found but a position was found where the entry would diff --git a/read-cache.c b/read-cache.c index f5d4385c408..c079ece981a 100644 --- a/read-cache.c +++ b/read-cache.c @@ -68,6 +68,11 @@ */ #define CACHE_ENTRY_PATH_LENGTH 80 +enum index_search_mode { + NO_EXPAND_SPARSE = 0, + EXPAND_SPARSE = 1 +}; + static inline struct cache_entry *mem_pool__ce_alloc(struct mem_pool *mem_pool, size_t len) { struct cache_entry *ce; @@ -551,7 +556,10 @@ int cache_name_stage_compare(const char *name1, int len1, int stage1, const char return 0; } -static int index_name_stage_pos(struct index_state *istate, const char *name, int namelen, int stage) +static int index_name_stage_pos(struct index_state *istate, + const char *name, int namelen, + int stage, + enum index_search_mode search_mode) { int first, last; @@ -570,7 +578,7 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in first = next+1; } - if (istate->sparse_index && + if (search_mode == EXPAND_SPARSE && istate->sparse_index && first > 0) { /* Note: first <= istate->cache_nr */ struct cache_entry *ce = istate->cache[first - 1]; @@ -586,7 +594,7 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in ce_namelen(ce) < namelen && !strncmp(name, ce->name, ce_namelen(ce))) { ensure_full_index(istate); - return index_name_stage_pos(istate, name, namelen, stage); + return index_name_stage_pos(istate, name, namelen, stage, search_mode); } } @@ -595,7 +603,12 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in int index_name_pos(struct index_state *istate, const char *name, int namelen) { - return index_name_stage_pos(istate, name, namelen, 0); + return index_name_stage_pos(istate, name, namelen, 0, EXPAND_SPARSE); +} + +int index_entry_exists(struct index_state *istate, const char *name, int namelen) +{ + return index_name_stage_pos(istate, name, namelen, 0, NO_EXPAND_SPARSE) >= 0; } int remove_index_entry_at(struct index_state *istate, int pos) @@ -1222,7 +1235,7 @@ static int has_dir_name(struct index_state *istate, */ } - pos = index_name_stage_pos(istate, name, len, stage); + pos = index_name_stage_pos(istate, name, len, stage, EXPAND_SPARSE); if (pos >= 0) { /* * Found one, but not so fast. This could @@ -1322,7 +1335,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e strcmp(ce->name, istate->cache[istate->cache_nr - 1]->name) > 0) pos = index_pos_to_insert_pos(istate->cache_nr); else - pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce)); + pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), EXPAND_SPARSE); /* existing match? Just replace it. */ if (pos >= 0) { @@ -1357,7 +1370,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e if (!ok_to_replace) return error(_("'%s' appears as both a file and as a directory"), ce->name); - pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce)); + pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), EXPAND_SPARSE); pos = -pos-1; } return pos + 1; diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 4125525ab86..871cc3fcb8d 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -777,9 +777,9 @@ test_expect_success 'sparse-index is not expanded' ' ensure_not_expanded checkout - && ensure_not_expanded switch rename-out-to-out && ensure_not_expanded switch - && - git -C sparse-index reset --hard && + ensure_not_expanded reset --hard && ensure_not_expanded checkout rename-out-to-out -- deep/deeper1 && - git -C sparse-index reset --hard && + ensure_not_expanded reset --hard && ensure_not_expanded restore -s rename-out-to-out -- deep/deeper1 && echo >>sparse-index/README.md && @@ -789,6 +789,17 @@ test_expect_success 'sparse-index is not expanded' ' echo >>sparse-index/untracked.txt && ensure_not_expanded add . && + for ref in update-deep update-folder1 update-folder2 update-deep + do + echo >>sparse-index/README.md && + ensure_not_expanded reset --hard $ref || return 1 + done && + + ensure_not_expanded reset --hard update-deep && + ensure_not_expanded reset --keep base && + ensure_not_expanded reset --merge update-deep && + ensure_not_expanded reset --hard && + ensure_not_expanded checkout -f update-deep && test_config -C sparse-index pull.twohead ort && ( From patchwork Mon Nov 29 15:52:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12645549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF124C433F5 for ; Mon, 29 Nov 2021 19:35:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352735AbhK2TjJ (ORCPT ); Mon, 29 Nov 2021 14:39:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48512 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242094AbhK2ThB (ORCPT ); Mon, 29 Nov 2021 14:37:01 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58DACC09B18F for ; Mon, 29 Nov 2021 07:52:51 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id d72-20020a1c1d4b000000b00331140f3dc8so12740768wmd.1 for ; Mon, 29 Nov 2021 07:52:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=bCfMr+Jv2pEDwIaUjIBgrO8Hy49HM0OPtV6mD7B78o4=; b=CoIH4m1Mt5xTtRCFqpG3d39sPfz9fRibVgEJZeBC95zzoSegeb+cJQWnteAuvupmTu F9/n1pqNRFbRQ4hky2KsTtZvQqGQaMvb8C/zztMRkMp7Xonpw/3aJnhLD2R/vtI06f2D aVZBhnKLHshBZeeO+1LoGEKrzy59bMP3omQzH6qbcOYJMG1XmgW9QcCzbZSDUk6UyYL2 DVThhiafULbOe1gynD2HLG0dIzHZtwkX6jAmN+2VCHeizewMoQBRJWgJTYlgElCM25Vu ynnHPfGup852ZuM5sCM7XE/fI5P/UaOaqMv1GSM6KuLmILessziOvpO3XB8pexCbLarS MtOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=bCfMr+Jv2pEDwIaUjIBgrO8Hy49HM0OPtV6mD7B78o4=; b=cjmve4rhZhReOGmYOqWcBATdm0l03XRTOgehhYp4xjpgrHqdnX0QgiuTEdrhVzKywE MQcUwYEVaBTvnNZWXgFMYcCS+WBIG0QBmWUJCPDHLhuFuzLtRYU+c64cBgFalJLUQZCJ GTWKVjl3tjcF4lZu4C+GN9LLFNioMKofTsN+sm1fpogCFWe7T7KTaCyUN63h9YwUuPaF H6noJ49IUw0cD9aX+iDxZD8hIGrQaoI0N58NRT/i6YXew64bxvyBY3PX5tq/yoFUeS8s 3QpS8VqpRzkGyxNlcFCV/6/cGJ8yzQJekJnnHAeqa8I/yfmNqnUiV9L5SFuGgAMl/8M8 HPUQ== X-Gm-Message-State: AOAM532Xg4h+sGGhM9eWV8+WO58AEsgWQLrRHmrg9B+Bf7J0NQbkR5Mh 1tGNZaaqQsaW1mrPOP2q4wr3QpkYi04= X-Google-Smtp-Source: ABdhPJx3P11C2Iw3ruBK72tJUOzCuyPYYm6Eehd8Up8PpPa/osqD+4jTALeWVyAHY1A0WzG4aIioaQ== X-Received: by 2002:a05:600c:2118:: with SMTP id u24mr37739048wml.0.1638201169740; Mon, 29 Nov 2021 07:52:49 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j40sm17271478wms.16.2021.11.29.07.52.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Nov 2021 07:52:49 -0800 (PST) Message-Id: <822d7344587f698e73abba1ca726c3a905f7b403.1638201164.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 29 Nov 2021 15:52:42 +0000 Subject: [PATCH v6 7/8] reset: make --mixed sparse-aware Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Remove the `ensure_full_index` guard on `read_from_tree` and update `git reset --mixed` to ensure it can use sparse directory index entries wherever possible. Sparse directory entries are reset using `diff_tree_oid`, which requires `change` and `add_remove` functions to process the internal contents of the sparse directory. The `recursive` diff option handles cases in which `reset --mixed` must diff/merge files that are nested multiple levels deep in a sparse directory. The use of pathspecs with `git reset --mixed` introduces scenarios in which internal contents of sparse directories may be matched by the pathspec. In order to reset *all* files in the repo that may match the pathspec, the following conditions on the pathspec require index expansion before performing the reset: * "magic" pathspecs * wildcard pathspecs that do not match only in-cone files or entire sparse directories * literal pathspecs matching something outside the sparse checkout definition Helped-by: Elijah Newren Signed-off-by: Victoria Dye --- builtin/reset.c | 87 +++++++++++++++++++++++- t/t1092-sparse-checkout-compatibility.sh | 17 +++++ 2 files changed, 102 insertions(+), 2 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index 0ac0de7dc97..dcb79fb43a3 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -148,7 +148,9 @@ static void update_index_from_diff(struct diff_queue_struct *q, * If the file 1) corresponds to an existing index entry with * skip-worktree set, or 2) does not exist in the index but is * outside the sparse checkout definition, add a skip-worktree bit - * to the new index entry. + * to the new index entry. Note that a sparse index will be expanded + * if this entry is outside the sparse cone - this is necessary + * to properly construct the reset sparse directory. */ pos = cache_name_pos(one->path, strlen(one->path)); if ((pos >= 0 && ce_skip_worktree(active_cache[pos])) || @@ -166,6 +168,82 @@ static void update_index_from_diff(struct diff_queue_struct *q, } } +static int pathspec_needs_expanded_index(const struct pathspec *pathspec) +{ + unsigned int i, pos; + int res = 0; + char *skip_worktree_seen = NULL; + + /* + * When using a magic pathspec, assume for the sake of simplicity that + * the index needs to be expanded to match all matchable files. + */ + if (pathspec->magic) + return 1; + + for (i = 0; i < pathspec->nr; i++) { + struct pathspec_item item = pathspec->items[i]; + + /* + * If the pathspec item has a wildcard, the index should be expanded + * if the pathspec has the possibility of matching a subset of entries inside + * of a sparse directory (but not the entire directory). + * + * If the pathspec item is a literal path, the index only needs to be expanded + * if a) the pathspec isn't in the sparse checkout cone (to make sure we don't + * expand for in-cone files) and b) it doesn't match any sparse directories + * (since we can reset whole sparse directories without expanding them). + */ + if (item.nowildcard_len < item.len) { + /* + * Special case: if the pattern is a path inside the cone + * followed by only wildcards, the pattern cannot match + * partial sparse directories, so we don't expand the index. + */ + if (path_in_cone_mode_sparse_checkout(item.original, &the_index) && + strspn(item.original + item.nowildcard_len, "*") == item.len - item.nowildcard_len) + continue; + + for (pos = 0; pos < active_nr; pos++) { + struct cache_entry *ce = active_cache[pos]; + + if (!S_ISSPARSEDIR(ce->ce_mode)) + continue; + + /* + * If the pre-wildcard length is longer than the sparse + * directory name and the sparse directory is the first + * component of the pathspec, need to expand the index. + */ + if (item.nowildcard_len > ce_namelen(ce) && + !strncmp(item.original, ce->name, ce_namelen(ce))) { + res = 1; + break; + } + + /* + * If the pre-wildcard length is shorter than the sparse + * directory and the pathspec does not match the whole + * directory, need to expand the index. + */ + if (!strncmp(item.original, ce->name, item.nowildcard_len) && + wildmatch(item.original, ce->name, 0)) { + res = 1; + break; + } + } + } else if (!path_in_cone_mode_sparse_checkout(item.original, &the_index) && + !matches_skip_worktree(pathspec, i, &skip_worktree_seen)) + res = 1; + + if (res > 0) + break; + } + + free(skip_worktree_seen); + return res; +} + static int read_from_tree(const struct pathspec *pathspec, struct object_id *tree_oid, int intent_to_add) @@ -178,9 +256,14 @@ static int read_from_tree(const struct pathspec *pathspec, opt.format_callback = update_index_from_diff; opt.format_callback_data = &intent_to_add; opt.flags.override_submodule_config = 1; + opt.flags.recursive = 1; opt.repo = the_repository; + opt.change = diff_change; + opt.add_remove = diff_addremove; + + if (pathspec->nr && the_index.sparse_index && pathspec_needs_expanded_index(pathspec)) + ensure_full_index(&the_index); - ensure_full_index(&the_index); if (do_diff_cache(tree_oid, &opt)) return 1; diffcore_std(&opt); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 871cc3fcb8d..77e302a0ef3 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -795,11 +795,28 @@ test_expect_success 'sparse-index is not expanded' ' ensure_not_expanded reset --hard $ref || return 1 done && + ensure_not_expanded reset --mixed base && ensure_not_expanded reset --hard update-deep && ensure_not_expanded reset --keep base && ensure_not_expanded reset --merge update-deep && ensure_not_expanded reset --hard && + ensure_not_expanded reset base -- deep/a && + ensure_not_expanded reset base -- nonexistent-file && + ensure_not_expanded reset deepest -- deep && + + # Although folder1 is outside the sparse definition, it exists as a + # directory entry in the index, so the pathspec will not force the + # index to be expanded. + ensure_not_expanded reset deepest -- folder1 && + ensure_not_expanded reset deepest -- folder1/ && + + # Wildcard identifies only in-cone files, no index expansion + ensure_not_expanded reset deepest -- deep/\* && + + # Wildcard identifies only full sparse directories, no index expansion + ensure_not_expanded reset deepest -- folder\* && + ensure_not_expanded checkout -f update-deep && test_config -C sparse-index pull.twohead ort && ( From patchwork Mon Nov 29 15:52:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12645551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7B44C433FE for ; Mon, 29 Nov 2021 19:35:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243669AbhK2TjK (ORCPT ); Mon, 29 Nov 2021 14:39:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242534AbhK2ThC (ORCPT ); Mon, 29 Nov 2021 14:37:02 -0500 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08BE2C09B191 for ; Mon, 29 Nov 2021 07:52:52 -0800 (PST) Received: by mail-wm1-x332.google.com with SMTP id az34-20020a05600c602200b0033bf8662572so12763392wmb.0 for ; Mon, 29 Nov 2021 07:52:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=nEih7G58FeXZR7VfCiUcLv0bmQevEM54qXiHHW/Vpvo=; b=airA1qfQCqHzXo0I3eMn3m2p6JMZhcbcCtlIZRL5SL/0B3BUbHa91fpEAD+uVhqxZS cB3v+h8/XveaoXAnNeN2pqvAYEeN+pQ8vgQQIa1HRRrHbJ0U2DDyb6bqqa1JEy0EJ5kz 43zCaZw0/W2cyal2/Aux6Ymi78/4+va5oC0hsEwDEubDQeKayOaNYTkqTxRV7elByi2a AslHjTc/Y/8eJq7TXYTzSYLRt+ddSbRLhw8HTcXjiRIfHgwR31HuuSkbY4XN3h90dJmH H8Non8K/Ed2lGoLX8RzItCHGAmoc9YCjdEz4QZY7tqexX553R1MmyAgIZ1LmepMIiAl6 BmMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=nEih7G58FeXZR7VfCiUcLv0bmQevEM54qXiHHW/Vpvo=; b=EGhftzKjZzQYddHB4WEZjgVbmdk4DgcnCKLdDBrZfQ6M1gPGpisBVicnJBTPAsRDPb IGG1jZLPXk0yXYwJRyb1HKyxB3QcJno7Xxr9mHoe7KiCCPDXyWr1ioDvjYPnzyX0sS6V 40xgFwIU5s8s3fljPYG1rECYWn9YzbJC42jc0fbq/mXMgqYufYgevTVztF96pDSu3g3x WoTWXqnIIvz/19IfZgxAaO0BRjuSzFXx/veqdRLjgrhYsH9bUNWMkjsExh5NhK3RSJwr RqXeLRUZoep4bItZr6EFYnD4sVmT7sidu0i/wywQmJY63DvifvkHiENrXWTQncSq/Xrt heIQ== X-Gm-Message-State: AOAM530l6De31k2zWCzEE4znDVq4WYj6qoA2lsqy5B7peVXpwma9PIm2 uiWc8UNwldhc8vgvanlP2itfWskgPLk= X-Google-Smtp-Source: ABdhPJzkFL909civf7qPEZ4re/sRPgOYyiHaowP7yOcIOBRnWVp4ITYL2RnDxkd8Q2Yq7miz7KcCNQ== X-Received: by 2002:a05:600c:294c:: with SMTP id n12mr37487102wmd.71.1638201170497; Mon, 29 Nov 2021 07:52:50 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x13sm14180972wrr.47.2021.11.29.07.52.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Nov 2021 07:52:50 -0800 (PST) Message-Id: In-Reply-To: References: Date: Mon, 29 Nov 2021 15:52:43 +0000 Subject: [PATCH v6 8/8] unpack-trees: improve performance of next_cache_entry Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye To find the first non-unpacked cache entry, `next_cache_entry` iterates through index, starting at `cache_bottom`. The performance of this in full indexes is helped by `cache_bottom` advancing with each invocation of `mark_ce_used` (called by `unpack_index_entry`). However, the presence of sparse directories can prevent the `cache_bottom` from advancing in a sparse index case, effectively forcing `next_cache_entry` to search from the beginning of the index each time it is called. The `cache_bottom` must be preserved for the sparse index (see 17a1bb570b (unpack-trees: preserve cache_bottom, 2021-07-14)). Therefore, to retain the benefit `cache_bottom` provides in non-sparse index cases, a separate `hint` position indicates the first position `next_cache_entry` should search, updated each execution with a new position. Signed-off-by: Victoria Dye --- unpack-trees.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/unpack-trees.c b/unpack-trees.c index 8ea0a542da8..b94733de6be 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -645,17 +645,24 @@ static void mark_ce_used_same_name(struct cache_entry *ce, } } -static struct cache_entry *next_cache_entry(struct unpack_trees_options *o) +static struct cache_entry *next_cache_entry(struct unpack_trees_options *o, int *hint) { const struct index_state *index = o->src_index; int pos = o->cache_bottom; + if (*hint > pos) + pos = *hint; + while (pos < index->cache_nr) { struct cache_entry *ce = index->cache[pos]; - if (!(ce->ce_flags & CE_UNPACKED)) + if (!(ce->ce_flags & CE_UNPACKED)) { + *hint = pos + 1; return ce; + } pos++; } + + *hint = pos; return NULL; } @@ -1365,12 +1372,13 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str /* Are we supposed to look at the index too? */ if (o->merge) { + int hint = -1; while (1) { int cmp; struct cache_entry *ce; if (o->diff_index_cached) - ce = next_cache_entry(o); + ce = next_cache_entry(o, &hint); else ce = find_cache_entry(info, p); @@ -1690,7 +1698,7 @@ static int verify_absent(const struct cache_entry *, int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options *o) { struct repository *repo = the_repository; - int i, ret; + int i, hint, ret; static struct cache_entry *dfc; struct pattern_list pl; int free_pattern_list = 0; @@ -1763,13 +1771,15 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options info.pathspec = o->pathspec; if (o->prefix) { + hint = -1; + /* * Unpack existing index entries that sort before the * prefix the tree is spliced into. Note that o->merge * is always true in this case. */ while (1) { - struct cache_entry *ce = next_cache_entry(o); + struct cache_entry *ce = next_cache_entry(o, &hint); if (!ce) break; if (ce_in_traverse_path(ce, &info)) @@ -1790,8 +1800,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options /* Any left-over entries in the index? */ if (o->merge) { + hint = -1; while (1) { - struct cache_entry *ce = next_cache_entry(o); + struct cache_entry *ce = next_cache_entry(o, &hint); if (!ce) break; if (unpack_index_entry(ce, o) < 0)