From patchwork Mon Oct 11 20:30:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12551003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFB4DC433EF for ; Mon, 11 Oct 2021 20:30:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B2CE860F21 for ; Mon, 11 Oct 2021 20:30:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234957AbhJKUc1 (ORCPT ); Mon, 11 Oct 2021 16:32:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42778 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234946AbhJKUc0 (ORCPT ); Mon, 11 Oct 2021 16:32:26 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D6A6C06161C for ; Mon, 11 Oct 2021 13:30:26 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id e3so26293718wrc.11 for ; Mon, 11 Oct 2021 13:30:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ldyVwS09nc2OLJia/8NlZArMqqWxVI8EkShpKdO52GA=; b=OlXK9T0ach0KytFLX22aieHJPrTj+qJk93CXmgdk3suueaeSwlthgZKZOuuzn6yhl4 LA1bGAeKVLHhHNtd4ecyhoLlxr58Urq29LWd38O8hBB/NFLaQEBEv35xSEAcZfyfqnYA 6NKS20NXpr6qTAT4oaAqvh1djjiYnm8lImRkd4u+bFHkmYzqN2xxzjQlD7ASw7SzsfVt Oos718wdLB7w6zrO92+eTTjRZutCKG6JKIupu7bNyt2zT744wsRZ7VG/RMn3X3YeUlnS me83pi9WmLCFJN/9i3/GVffdfmdVu4WuH3MOyi/VuijxiP0uu4iBodbKmC/aIK6JCf4E YnuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ldyVwS09nc2OLJia/8NlZArMqqWxVI8EkShpKdO52GA=; b=4dqb5g08M8hlrWDeuInXzg+oOeToQiOSgJAj1J/gExdF5pzEmPMnjQekm2VJC70Qhn lP+BmBpYO1hMwDKopEH9xX4GjzvtdTzBrgOINyhX2utpOp3c6XQPCSyTXoChWHhWwyYq vCY1rteGwybh6W/tYHcvxd/y+NeoEOneVUgwk9DpBFcf7E3+zhT9Oli2o6plQWWmqXgf 00inBYnPXp0QZKMc5jvcq1QdjHNPuewi0Z3Y7TEKPp1m8GjQF+jykB5dMoUT8XXCiMGa xzpgGyU0YTv6HHSbEzlYlq4qFuEkPs+yoh20lpywNz6FBVjh6qzxHY9hu9+Vyve4VaCK CYuw== X-Gm-Message-State: AOAM532j3nBTx/Goo+5eYOZYw7qsaw0SJwk3dRc3y57fz6y4UzlzYUcQ aft5R5/ph1TiBHD0XpQCJJRGxHVzG5o= X-Google-Smtp-Source: ABdhPJzbgnoDSvV63yoZnmyRlZTeM5uIG4JMQ0TAUWM8W4bxxlyjJ7pYJXftv7wsUBaoIyE8GGKiAw== X-Received: by 2002:adf:a496:: with SMTP id g22mr28191934wrb.13.1633984224808; Mon, 11 Oct 2021 13:30:24 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u2sm8515718wrr.35.2021.10.11.13.30.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Oct 2021 13:30:24 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 11 Oct 2021 20:30:15 +0000 Subject: [PATCH v4 1/8] reset: rename is_missing to !is_in_reset_tree Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Rename and invert value of `is_missing` to `is_in_reset_tree` to make the variable more descriptive of what it represents. Signed-off-by: Victoria Dye --- builtin/reset.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index 51c9e2f43ff..d3695ce43c4 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -131,10 +131,10 @@ static void update_index_from_diff(struct diff_queue_struct *q, for (i = 0; i < q->nr; i++) { struct diff_filespec *one = q->queue[i]->one; - int is_missing = !(one->mode && !is_null_oid(&one->oid)); + int is_in_reset_tree = one->mode && !is_null_oid(&one->oid); struct cache_entry *ce; - if (is_missing && !intent_to_add) { + if (!is_in_reset_tree && !intent_to_add) { remove_file_from_cache(one->path); continue; } @@ -144,7 +144,7 @@ static void update_index_from_diff(struct diff_queue_struct *q, if (!ce) die(_("make_cache_entry failed for path '%s'"), one->path); - if (is_missing) { + if (!is_in_reset_tree) { ce->ce_flags |= CE_INTENT_TO_ADD; set_object_name_for_intent_to_add_entry(ce); } From patchwork Mon Oct 11 20:30:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12551005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B606C433F5 for ; Mon, 11 Oct 2021 20:30:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E59E060F21 for ; Mon, 11 Oct 2021 20:30:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234970AbhJKUc3 (ORCPT ); Mon, 11 Oct 2021 16:32:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42784 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234961AbhJKUc1 (ORCPT ); Mon, 11 Oct 2021 16:32:27 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B9DBC061570 for ; Mon, 11 Oct 2021 13:30:27 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id o20so59695276wro.3 for ; Mon, 11 Oct 2021 13:30:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=GeJg5j0KjhoiYZfkVfGbXTqvOiOqWSt2Hq4eYQJJWig=; b=eaNM2sHBcCkrbKvui3aK0IfFvIUsIpRbDecU2XlLp9wARx+MFVXqM5EXRJmiZ5L3te dFC68WdTnBmmRPbjOMGBMTc/gq1SRQuntwh6zS6p664F1WrV7vx1UzB4+RHtKPN36tJb YnJx3Y4HTby+lT5nn9NfyA5bV06KZM6JMEjfRXEsnu/LgVkGpbfEESPBIrp4rKbp5ZFl 5a0dBPXkX22LiXToP9p3JLyS/ILw0oIAmms4Ck2B+/wZBcsq1G5e9IuKWF/wsOBSPD7p Fd4L3Iz3oTO1rXghxwkxCSb1OHlVKAIP4NprW4VTj+coY3Gs/KVz3dwbQh3Kc3fmZXWD gL6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=GeJg5j0KjhoiYZfkVfGbXTqvOiOqWSt2Hq4eYQJJWig=; b=iEpeWJpiIGoA4LHJyBQhThS6RufG+bjbNw9BWuPkcNgOPu1ozN+wOk1H0Re1X7sgzv ePk9WA9xI38qTkTcJB9McCOnJsRg1i9osHi8cVJzpxiyZ8CQZfQ1h05EjY9m1DVqr8rb VBBFw9KB1HmPVQpVo8z3ilQMkygOlZpFMXLEM/rfqIHTHiC83/VEGOIEeI66aQsnfETP dFlbKvc2EC/fr9aSftJLqG0qHZ3sxAlxCwhD9rBjaOzeg1+gQ4f1FjlxQcpFPpVnf8J5 H180ZtxO3xijqH6WsCVddgWE9Gg8yO5tq/KUUYd1313CU7M9TIuv1h9QDtmW3r0ZoCmC yF0Q== X-Gm-Message-State: AOAM530kNLyKNZ64fHD7wyuramtBKRdiGDAZWQwe6x3WSCpIWDc0ZuHa Vw2I2Lz+ExZNbJQ4OYUa/rCAev66law= X-Google-Smtp-Source: ABdhPJxHZPNB8NqNCftvHFI+86vU518X7k8FjlTZVxZDKu7WyKRINm3jqih5CE08dgIzlfw3uFZNZQ== X-Received: by 2002:a5d:6d02:: with SMTP id e2mr13063390wrq.427.1633984225434; Mon, 11 Oct 2021 13:30:25 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f18sm9094591wrg.3.2021.10.11.13.30.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Oct 2021 13:30:25 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 11 Oct 2021 20:30:16 +0000 Subject: [PATCH v4 2/8] reset: preserve skip-worktree bit in mixed reset Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Change `update_index_from_diff` to set `skip-worktree` when applicable for new index entries. When `git reset --mixed ` is run, entries in the index with differences between the pre-reset HEAD and reset are identified and handled with `update_index_from_diff`. For each file, a new cache entry in inserted into the index, created from the side of the reset (without changing the working tree). However, the newly-created entry must have `skip-worktree` explicitly set in either of the following scenarios: 1. the file is in the current index and has `skip-worktree` set 2. the file is not in the current index but is outside of a defined sparse checkout definition Not setting the `skip-worktree` bit leads to likely-undesirable results for a user. It causes `skip-worktree` settings to disappear on the "diff"-containing files (but *only* the diff-containing files), leading to those files now showing modifications in `git status`. For example, when running `git reset --mixed` in a sparse checkout, some file entries outside of sparse checkout could show up as deleted, despite the user never deleting anything (and not wanting them on-disk anyway). Additionally, add a test to `t7102` to ensure `skip-worktree` is preserved in a basic `git reset --mixed` scenario and update a failure-documenting test from 19a0acc (t1092: test interesting sparse-checkout scenarios, 2021-01-23) with new expected behavior. Helped-by: Junio C Hamano Signed-off-by: Victoria Dye --- builtin/reset.c | 14 ++++++++++++++ t/t1092-sparse-checkout-compatibility.sh | 19 +++++-------------- t/t7102-reset.sh | 17 +++++++++++++++++ 3 files changed, 36 insertions(+), 14 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index d3695ce43c4..e441b6601b9 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -25,6 +25,7 @@ #include "cache-tree.h" #include "submodule.h" #include "submodule-config.h" +#include "dir.h" #define REFRESH_INDEX_DELAY_WARNING_IN_MS (2 * 1000) @@ -130,6 +131,7 @@ static void update_index_from_diff(struct diff_queue_struct *q, int intent_to_add = *(int *)data; for (i = 0; i < q->nr; i++) { + int pos; struct diff_filespec *one = q->queue[i]->one; int is_in_reset_tree = one->mode && !is_null_oid(&one->oid); struct cache_entry *ce; @@ -141,6 +143,18 @@ static void update_index_from_diff(struct diff_queue_struct *q, ce = make_cache_entry(&the_index, one->mode, &one->oid, one->path, 0, 0); + + /* + * If the file 1) corresponds to an existing index entry with + * skip-worktree set, or 2) does not exist in the index but is + * outside the sparse checkout definition, add a skip-worktree bit + * to the new index entry. + */ + pos = cache_name_pos(one->path, strlen(one->path)); + if ((pos >= 0 && ce_skip_worktree(active_cache[pos])) || + (pos < 0 && !path_in_sparse_checkout(one->path, &the_index))) + ce->ce_flags |= CE_SKIP_WORKTREE; + if (!ce) die(_("make_cache_entry failed for path '%s'"), one->path); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 886e78715fe..889079f55b8 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -459,26 +459,17 @@ test_expect_failure 'blame with pathspec outside sparse definition' ' test_all_match git blame deep/deeper2/deepest/a ' -# NEEDSWORK: a sparse-checkout behaves differently from a full checkout -# in this scenario, but it shouldn't. -test_expect_failure 'checkout and reset (mixed)' ' +test_expect_success 'checkout and reset (mixed)' ' init_repos && test_all_match git checkout -b reset-test update-deep && test_all_match git reset deepest && - test_all_match git reset update-folder1 && - test_all_match git reset update-folder2 -' - -# NEEDSWORK: a sparse-checkout behaves differently from a full checkout -# in this scenario, but it shouldn't. -test_expect_success 'checkout and reset (mixed) [sparse]' ' - init_repos && - test_sparse_match git checkout -b reset-test update-deep && - test_sparse_match git reset deepest && + # Because skip-worktree is preserved, resetting to update-folder1 + # will show worktree changes for full-checkout that are not present + # in sparse-checkout or sparse-index. test_sparse_match git reset update-folder1 && - test_sparse_match git reset update-folder2 + run_on_sparse test_path_is_missing folder1 ' test_expect_success 'merge, cherry-pick, and rebase' ' diff --git a/t/t7102-reset.sh b/t/t7102-reset.sh index 601b2bf97f0..d05426062ec 100755 --- a/t/t7102-reset.sh +++ b/t/t7102-reset.sh @@ -472,6 +472,23 @@ test_expect_success '--mixed refreshes the index' ' test_cmp expect output ' +test_expect_success '--mixed preserves skip-worktree' ' + echo 123 >>file2 && + git add file2 && + git update-index --skip-worktree file2 && + git reset --mixed HEAD >output && + test_must_be_empty output && + + cat >expect <<-\EOF && + Unstaged changes after reset: + M file2 + EOF + git update-index --no-skip-worktree file2 && + git add file2 && + git reset --mixed HEAD >output && + test_cmp expect output +' + test_expect_success 'resetting specific path that is unmerged' ' git rm --cached file2 && F1=$(git rev-parse HEAD:file1) && From patchwork Mon Oct 11 20:30:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12551007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E44EEC433EF for ; Mon, 11 Oct 2021 20:30:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CB08260F14 for ; Mon, 11 Oct 2021 20:30:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232004AbhJKUca (ORCPT ); Mon, 11 Oct 2021 16:32:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234946AbhJKUc2 (ORCPT ); Mon, 11 Oct 2021 16:32:28 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58168C06161C for ; Mon, 11 Oct 2021 13:30:27 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id e3so26293895wrc.11 for ; Mon, 11 Oct 2021 13:30:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=BfoqpXwFV4NtR5yoX+ONX0rNlMcxUly/tVDZ9BHmZXk=; b=Ehz/1BOmkfYbJ9Cwf7R017VwGOVAleUz2UD+NFWanQTMq+CcyLkEwjt+QLw3/aXXrU ZnPUl/j4mHeMOyYDY4SGnVe/k/Ull/5d8MKIS6omCTaKBFj/H8ypyMwXzdhaiyLZSREo nJncZhDgvZk+2bf7VR0hfJm0FvNZJO+xlfta77pYUUirrabWJG1KN/enlfLjFGp4zJmL PewehfbEIRPooWBQc37NVsJe6+SkuXZ+tx6Z42HvhiFzy9QqQ6Wdi4ijlj8/7hQV5gK7 CxjiVJXTZ8mu6bKtqN3CRUARAp4xspKVnyllryYnUTP4hBOsXniu677xuAybjzZ8FmaQ mijg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=BfoqpXwFV4NtR5yoX+ONX0rNlMcxUly/tVDZ9BHmZXk=; b=oiumiYPnlUumNuwiXz1cCNRdriFAsvRNYzvvq1SetPdWSesqKTqcgDBUMCZFbR17QO vXZipo4jMh8mQb30Ax6+TohjkgdqZ7sHf596dmBrJnscYU51AvCxkmEaAt4HuWAp5oRm 4wsuDF3ZXeCgyHz5hIQZzbE2STFP+Cz0Nu8qVSNHPRaAtznc5uH0ers7eZ5xApsMChew SoNcR+6PLErWJR8GvPEIDn2ao9Nbl3kLb4dDYm7JP6TXMzkemCf8rbFIH+Zpb20uWtNu yNi5bOZk9mrcEZylDcoBEGz0f5bEEoAZFbq2aVeL0Q/sykCpKh7opQ3hH+CtUB7I1lyF mHjA== X-Gm-Message-State: AOAM530c3VaPOiYtJgpPXUINA3dHxrQbtzu+HCL/AAW/Y++IJHJvnb8+ 7a2na+Adr/LnYrmDs97lPUH6kI7cxzc= X-Google-Smtp-Source: ABdhPJyk7aLEdn2sN0O06Osv9fXnomZhVLg4fKfnz5uByllgw1zjCFXkfx/M5tG7SZ1rWlr645L1Ag== X-Received: by 2002:a1c:7e10:: with SMTP id z16mr1212942wmc.141.1633984225994; Mon, 11 Oct 2021 13:30:25 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o3sm8912883wra.52.2021.10.11.13.30.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Oct 2021 13:30:25 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 11 Oct 2021 20:30:17 +0000 Subject: [PATCH v4 3/8] sparse-index: update command for expand/collapse test Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye In anticipation of `git reset --hard` being able to use the sparse index without expanding it, replace the command in `sparse-index is expanded and converted back` with `git reset -- folder1/a`. This command will need to expand the index to work properly, even after integrating the rest of `reset` with sparse index. Helped-by: Derrick Stolee Signed-off-by: Victoria Dye --- t/t1092-sparse-checkout-compatibility.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 889079f55b8..e1422797013 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -631,11 +631,15 @@ test_expect_success 'submodule handling' ' grep "160000 commit $(git -C initial-repo rev-parse HEAD) modules/sub" cache ' +# When working with a sparse index, some commands will need to expand the +# index to operate properly. If those commands also write the index back +# to disk, they need to convert the index to sparse before writing. +# This test verifies that both of these events are logged in trace2 logs. test_expect_success 'sparse-index is expanded and converted back' ' init_repos && GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ - git -C sparse-index -c core.fsmonitor="" reset --hard && + git -C sparse-index reset -- folder1/a && test_region index convert_to_sparse trace2.txt && test_region index ensure_full_index trace2.txt ' From patchwork Mon Oct 11 20:30:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12551009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91BD3C433EF for ; Mon, 11 Oct 2021 20:30:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7894C60F14 for ; Mon, 11 Oct 2021 20:30:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229632AbhJKUcb (ORCPT ); Mon, 11 Oct 2021 16:32:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234963AbhJKUc2 (ORCPT ); Mon, 11 Oct 2021 16:32:28 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD087C061745 for ; Mon, 11 Oct 2021 13:30:27 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id o20so59695370wro.3 for ; Mon, 11 Oct 2021 13:30:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=jlukojs1Ds21JUu9v6m+mSRfHzD2bmC8njuDTMJkZac=; b=ZGNlbMV0zw2dT+xPJifO1nxvJB0R84DeIgyatWCepbCPzXLys3XNMIiqqTXHppXmdn 2vId31pfm4VjD5cSUfXKptj0zswAR8uCoO0/kukEyMD06tysgyCevvp/TSTafYdkIxjo slYqECZDNepDoy26PuPdc0oKJpTf/+eEHX4TlT5JSsOYz1JYGwinFE/ZtmlgZE+bd1jB 1J4aW3DuNeNXOlLP7/MAk5umUGlxpmqADaAndjqIPX+aYw4yC878mtqz9tEa9wbc8bfN y2ISKvleq1F9PVZmLYITfftbMRAjaG/0cqu+YZyroWXj+t4sYHQBzcTEakjMPQRhWzMh BOCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=jlukojs1Ds21JUu9v6m+mSRfHzD2bmC8njuDTMJkZac=; b=ko9uV5a61A+c2ZHf7TiYWwvvRkTiSD/iIKnalDn/5WjxETavmcqkKy+d9dtCTG/kJj 3RRFysQbVydLjqFDDsI/jdZ3LDfdHb/wdOfVIbprXfQnn2lI1sb/qyAHKTyoKSgAgGKA VwJ2P3oMt30yQkEPYGb0BEIhG63n97RsvF+njFyyd5N4zrG2bxEtVJgxL8mmVWAzoHvM MiwllpiQ+4A2x1GMnFmrlvCXdHomiFKXE48Qs4iMSUf7aR5B7NxMaRjOmqj0lxpez0LJ Dh56K2399NPEgSWVT/1eODZ9ptypaQr6nS/Dqv9GujogxH3iC312PPe6ehqcb2T8cqJl ZRbA== X-Gm-Message-State: AOAM533nnM03fVQ2nC1S5wVCxwGJ+NUVGTDpwgUszrurQ3C24slYSZav lNCaI6c/NZgMB/vWUGjOxNqyJCswb04= X-Google-Smtp-Source: ABdhPJxwpo4+WqaB4apgawi8RTFQ+TTQc4Z4HI+vCV90/YwIvkXCqvKQzyjMXckh1LQSqlI33kc9Zw== X-Received: by 2002:a5d:4f8d:: with SMTP id d13mr26278077wru.239.1633984226525; Mon, 11 Oct 2021 13:30:26 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x24sm531614wmk.31.2021.10.11.13.30.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Oct 2021 13:30:26 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 11 Oct 2021 20:30:18 +0000 Subject: [PATCH v4 4/8] reset: expand test coverage for sparse checkouts Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Add new tests for `--merge` and `--keep` modes, as well as mixed reset with pathspecs. New performance test cases exercise various execution paths for `reset`. Co-authored-by: Derrick Stolee Signed-off-by: Derrick Stolee Signed-off-by: Victoria Dye --- t/perf/p2000-sparse-operations.sh | 3 + t/t1092-sparse-checkout-compatibility.sh | 84 ++++++++++++++++++++++++ 2 files changed, 87 insertions(+) diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index 597626276fb..bfd332120c8 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -110,5 +110,8 @@ test_perf_on_all git add -A test_perf_on_all git add . test_perf_on_all git commit -a -m A test_perf_on_all git checkout -f - +test_perf_on_all git reset +test_perf_on_all git reset --hard +test_perf_on_all git reset -- does-not-exist test_done diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index e1422797013..535686a2954 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -472,6 +472,90 @@ test_expect_success 'checkout and reset (mixed)' ' run_on_sparse test_path_is_missing folder1 ' +test_expect_success 'checkout and reset (merge)' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents a && + test_all_match git reset --merge deepest && + test_all_match git status --porcelain=v2 && + + test_all_match git reset --hard update-deep && + run_on_all ../edit-contents deep/a && + test_all_match test_must_fail git reset --merge deepest +' + +test_expect_success 'checkout and reset (keep)' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents a && + test_all_match git reset --keep deepest && + test_all_match git status --porcelain=v2 && + + test_all_match git reset --hard update-deep && + run_on_all ../edit-contents deep/a && + test_all_match test_must_fail git reset --keep deepest +' + +test_expect_success 'reset with pathspecs inside sparse definition' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents deep/a && + + test_all_match git reset base -- deep/a && + test_all_match git status --porcelain=v2 && + + test_all_match git reset base -- nonexistent-file && + test_all_match git status --porcelain=v2 && + + test_all_match git reset deepest -- deep && + test_all_match git status --porcelain=v2 +' + +# Although the working tree differs between full and sparse checkouts after +# reset, the state of the index is the same. +test_expect_success 'reset with pathspecs outside sparse definition' ' + init_repos && + test_all_match git checkout -b reset-test base && + + test_sparse_match git reset update-folder1 -- folder1 && + git -C full-checkout reset update-folder1 -- folder1 && + test_sparse_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder1 && + + test_sparse_match git reset update-folder2 -- folder2/a && + git -C full-checkout reset update-folder2 -- folder2/a && + test_sparse_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder2/a +' + +test_expect_success 'reset with wildcard pathspec' ' + init_repos && + + test_all_match git checkout -b reset-test update-deep && + test_all_match git reset base -- \*/a && + test_all_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder1/a && + + test_all_match git reset base -- folder\* && + test_all_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder2 +' + test_expect_success 'merge, cherry-pick, and rebase' ' init_repos && From patchwork Mon Oct 11 20:30:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12551011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E32F8C433F5 for ; Mon, 11 Oct 2021 20:30:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CBF5560F21 for ; Mon, 11 Oct 2021 20:30:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235000AbhJKUce (ORCPT ); Mon, 11 Oct 2021 16:32:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234966AbhJKUc3 (ORCPT ); Mon, 11 Oct 2021 16:32:29 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 792E3C061749 for ; Mon, 11 Oct 2021 13:30:28 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id r18so59983373wrg.6 for ; Mon, 11 Oct 2021 13:30:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+dch2aTLAv85TXXA5lgJ4m8qNH4dps5WqL8q9u/O5kk=; b=J9SA2mkxmgHUu2WWyukME+Czy1HiZvCkLyEg61TLRDoAux5vodZRRofa2+BQiWKyzO UciiFX9TSzIIXMwVvrVxzUdFlh/Pxd+0fiz9UBeStAjxTtNM5RWlzOMovOvP1xB27/VW n9SW8m9+f/0MSYlJt35YATikOcUYugKvMJGuG1oEw4vcwnK1gPpD0q/c+Z6Qhd4BfCrR tmZJDm4x5hbsLSE2R2Wvd3EAqrGMw3PHm19oIiqVyPXvWLMw4MmtL7dIJEDo1fwjYvWJ O03Vooi+2ARu2i6OBZp3hZVU4riRMNxpS+X2OoFsCdD5mxW9NFR/3S2CLTXd9GTmUbXc I/Lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+dch2aTLAv85TXXA5lgJ4m8qNH4dps5WqL8q9u/O5kk=; b=GFXRRc7mJgX6U3lkxtyu4Z35wjR2AF2jDBW4O5hVnshpOCwBn7R0L2PDOBLX2VvVdL lEs+FCGIBMa9YNWSJHt05RgYQKbUNBcmMRMxtBmYWSnAq41UX2N5bkLS4/zcxBgl+kyY 4YWSrY5KV5bb7WW/BFMkgjRHlllNs92I2ExEkd+OtSKGoy1qgbjzKUtS3f3Sh1z8TgKz u/Kws4zTlQHAczi/TOzfEKKq9UfoGFNe780HNgeVN8uqZ9CxKkTyIyemBBkDtXOWbOSY XXEAOhihDyspagG26hthcQBLpcTf0AyqP3r4y3x7I9NDI9n7UW/oJbbJQqvL3U1UD0Wv MlBA== X-Gm-Message-State: AOAM533xCvS8GwJVL331qG4izDvlj9bL4m5IZRFB6NZbs0zlCO1ylDV5 7W4DugL2CJyo0/1rB576tvSFkoBn61c= X-Google-Smtp-Source: ABdhPJxxfKqcBNgIFdZRWNbyMMkF57Iz5iYGXnlxuz+ub5ZhlNOvBkIoRfYs+KX5p75lCix1Ktcdlg== X-Received: by 2002:adf:b350:: with SMTP id k16mr26740077wrd.368.1633984227065; Mon, 11 Oct 2021 13:30:27 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v4sm490517wmh.23.2021.10.11.13.30.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Oct 2021 13:30:26 -0700 (PDT) Message-Id: <62fdbf2ad261e5820d42590100d38972c6d5e490.1633984222.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 11 Oct 2021 20:30:19 +0000 Subject: [PATCH v4 5/8] reset: integrate with sparse index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Disable `command_requires_full_index` repo setting and add `ensure_full_index` guards around code paths that cannot yet use sparse directory index entries. `reset --soft` does not modify the index, so no compatibility changes are needed for it to function without expanding the index. For all other reset modes (`--mixed`, `--hard`, `--keep`, `--merge`), the full index is expanded to prevent cache tree corruption and invalid variable accesses. Additionally, the `read_cache()` check verifying an uncorrupted index is moved after argument parsing and preparing the repo settings. The index is not used by the preceding argument handling, but `read_cache()` must be run *after* enabling sparse index for the command (so that the index is not expanded unnecessarily) and *before* using the index for reset (so that it is verified as uncorrupted). Signed-off-by: Victoria Dye --- builtin/reset.c | 10 +++++++--- cache-tree.c | 1 + 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index e441b6601b9..0ac0de7dc97 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -180,6 +180,7 @@ static int read_from_tree(const struct pathspec *pathspec, opt.flags.override_submodule_config = 1; opt.repo = the_repository; + ensure_full_index(&the_index); if (do_diff_cache(tree_oid, &opt)) return 1; diffcore_std(&opt); @@ -257,9 +258,6 @@ static void parse_args(struct pathspec *pathspec, } *rev_ret = rev; - if (read_cache() < 0) - die(_("index file corrupt")); - parse_pathspec(pathspec, 0, PATHSPEC_PREFER_FULL | (patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0), @@ -405,6 +403,12 @@ int cmd_reset(int argc, const char **argv, const char *prefix) if (intent_to_add && reset_type != MIXED) die(_("-N can only be used with --mixed")); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + + if (read_cache() < 0) + die(_("index file corrupt")); + /* Soft reset does not touch the index file nor the working tree * at all, but requires them in a good order. Other resets reset * the index file to the tree object we are switching to. */ diff --git a/cache-tree.c b/cache-tree.c index 90919f9e345..9be19c85b66 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -776,6 +776,7 @@ void prime_cache_tree(struct repository *r, cache_tree_free(&istate->cache_tree); istate->cache_tree = cache_tree(); + ensure_full_index(istate); prime_cache_tree_rec(r, istate->cache_tree, tree); istate->cache_changed |= CACHE_TREE_CHANGED; trace2_region_leave("cache-tree", "prime_cache_tree", the_repository); From patchwork Mon Oct 11 20:30:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12551015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C34FFC433F5 for ; Mon, 11 Oct 2021 20:30:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0964D60F21 for ; Mon, 11 Oct 2021 20:30:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234961AbhJKUcf (ORCPT ); Mon, 11 Oct 2021 16:32:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42800 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234971AbhJKUc3 (ORCPT ); Mon, 11 Oct 2021 16:32:29 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0158BC061570 for ; Mon, 11 Oct 2021 13:30:29 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id t2so59908770wrb.8 for ; Mon, 11 Oct 2021 13:30:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=3eu00jf4e+X22klEq0CYPgJ/CXgNIwvgJzeD6NoWL6E=; b=Gqrcy6rRgqmg4PVg5f74/OBvuAcq3quTWiRXLY6sXds7/CGQMU3vShCgjdFK+UXq3j WkataTCKISzK5TahAe62Hh6e/FyyqEKWekHLgVD5gegPWpXeO/XhX+3hPhZg1Q2IfN1D aTc1wXRvMRi3+DzMkpR2PQeV92cdKkUMI9SzzM6CGBomJdtnQfdITu9NeKDC+LMOSCgR 4gF8O10yCdt1y8HQkTKhctribioVBY27vI6AXOUDkvA1FPL+jTiW4KIk2jkDOh1Fp9Kc Q7f4tPawne8IDKVyagIahRR9jrHkoktIthuyL3f3GPTsy+3t3eSekQGNNYkxhFMKcGLg QqOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=3eu00jf4e+X22klEq0CYPgJ/CXgNIwvgJzeD6NoWL6E=; b=U4hGO0xF5nDHz+EpgFixtyxtIDkeEv16MCLbkSCIBlwXrbm39x+OhQ5TzxPXa8O+lN eE36DIm/NaoZ7DkcbLITdDnuCy+jhJ+yslo+Pk7WnDJMX/kJeffyIkA4o6tI+JUo26rm QITLY2m+YX4dgQp06SbKlNJosiEd112V79xO+Zl3Gg7jHLPZ+gY0a1IJAr19Plqj8WyI m+4SMGv+OxX+m2dAqdp1ad1MZWEIPl/SvVfwYLUnLUoq+EVvrZyY1SWG89v0/WMH24B5 X+7d49eJRJwZODSgmUDIP3YUP/BKLN8lF5ECofRrAUJ0NstY8w9dvFpm1OQxmXZKuQTC VTyw== X-Gm-Message-State: AOAM531/qFPOFqxPZ+nhpybqhznde0WjweCXtwhv85+0HFKQscTgO9a4 dcFOPUsROrBYP3s//cKrv1Im5ITMbPA= X-Google-Smtp-Source: ABdhPJwRmLeVrxsuAXNHsW89AaWvOYR0fKc1+hd6h8KR41aC8do8iahnrbq1TEMtaOJRT+4wG4pqpw== X-Received: by 2002:a1c:9ac4:: with SMTP id c187mr1285918wme.18.1633984227621; Mon, 11 Oct 2021 13:30:27 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c14sm8677935wrd.50.2021.10.11.13.30.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Oct 2021 13:30:27 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 11 Oct 2021 20:30:20 +0000 Subject: [PATCH v4 6/8] reset: make sparse-aware (except --mixed) Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Remove `ensure_full_index` guard on `prime_cache_tree` and update `prime_cache_tree_rec` to correctly reconstruct sparse directory entries in the cache tree. While processing a tree's entries, `prime_cache_tree_rec` must determine whether a directory entry is sparse or not by searching for it in the index (*without* expanding the index). If a matching sparse directory index entry is found, no subtrees are added to the cache tree entry and the entry count is set to 1 (representing the sparse directory itself). Otherwise, the tree is assumed to not be sparse and its subtrees are recursively added to the cache tree. Helped-by: Elijah Newren Signed-off-by: Victoria Dye --- cache-tree.c | 47 ++++++++++++++++++++++-- cache.h | 10 +++++ read-cache.c | 27 ++++++++++---- t/t1092-sparse-checkout-compatibility.sh | 15 +++++++- 4 files changed, 86 insertions(+), 13 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index 9be19c85b66..2866101052c 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -740,15 +740,26 @@ out: return ret; } +static void prime_cache_tree_sparse_dir(struct cache_tree *it, + struct tree *tree) +{ + + oidcpy(&it->oid, &tree->object.oid); + it->entry_count = 1; +} + static void prime_cache_tree_rec(struct repository *r, struct cache_tree *it, - struct tree *tree) + struct tree *tree, + struct strbuf *tree_path) { struct tree_desc desc; struct name_entry entry; int cnt; + int base_path_len = tree_path->len; oidcpy(&it->oid, &tree->object.oid); + init_tree_desc(&desc, tree->buffer, tree->size); cnt = 0; while (tree_entry(&desc, &entry)) { @@ -757,14 +768,40 @@ static void prime_cache_tree_rec(struct repository *r, else { struct cache_tree_sub *sub; struct tree *subtree = lookup_tree(r, &entry.oid); + if (!subtree->object.parsed) parse_tree(subtree); sub = cache_tree_sub(it, entry.path); sub->cache_tree = cache_tree(); - prime_cache_tree_rec(r, sub->cache_tree, subtree); + + /* + * Recursively-constructed subtree path is only needed when working + * in a sparse index (where it's used to determine whether the + * subtree is a sparse directory in the index). + */ + if (r->index->sparse_index) { + strbuf_setlen(tree_path, base_path_len); + strbuf_grow(tree_path, base_path_len + entry.pathlen + 1); + strbuf_add(tree_path, entry.path, entry.pathlen); + strbuf_addch(tree_path, '/'); + } + + /* + * If a sparse index is in use, the directory being processed may be + * sparse. To confirm that, we can check whether an entry with that + * exact name exists in the index. If it does, the created subtree + * should be sparse. Otherwise, cache tree expansion should continue + * as normal. + */ + if (r->index->sparse_index && + index_entry_exists(r->index, tree_path->buf, tree_path->len)) + prime_cache_tree_sparse_dir(sub->cache_tree, subtree); + else + prime_cache_tree_rec(r, sub->cache_tree, subtree, tree_path); cnt += sub->cache_tree->entry_count; } } + it->entry_count = cnt; } @@ -772,12 +809,14 @@ void prime_cache_tree(struct repository *r, struct index_state *istate, struct tree *tree) { + struct strbuf tree_path = STRBUF_INIT; + trace2_region_enter("cache-tree", "prime_cache_tree", the_repository); cache_tree_free(&istate->cache_tree); istate->cache_tree = cache_tree(); - ensure_full_index(istate); - prime_cache_tree_rec(r, istate->cache_tree, tree); + prime_cache_tree_rec(r, istate->cache_tree, tree, &tree_path); + strbuf_release(&tree_path); istate->cache_changed |= CACHE_TREE_CHANGED; trace2_region_leave("cache-tree", "prime_cache_tree", the_repository); } diff --git a/cache.h b/cache.h index f6295f3b048..1d3e4665562 100644 --- a/cache.h +++ b/cache.h @@ -816,6 +816,16 @@ struct cache_entry *index_file_exists(struct index_state *istate, const char *na */ int index_name_pos(struct index_state *, const char *name, int namelen); +/* + * Determines whether an entry with the given name exists within the + * given index. The return value is 1 if an exact match is found, otherwise + * it is 0. Note that, unlike index_name_pos, this function does not expand + * the index if it is sparse. If an item exists within the full index but it + * is contained within a sparse directory (and not in the sparse index), 0 is + * returned. + */ +int index_entry_exists(struct index_state *, const char *name, int namelen); + /* * Some functions return the negative complement of an insert position when a * precise match was not found but a position was found where the entry would diff --git a/read-cache.c b/read-cache.c index f5d4385c408..c079ece981a 100644 --- a/read-cache.c +++ b/read-cache.c @@ -68,6 +68,11 @@ */ #define CACHE_ENTRY_PATH_LENGTH 80 +enum index_search_mode { + NO_EXPAND_SPARSE = 0, + EXPAND_SPARSE = 1 +}; + static inline struct cache_entry *mem_pool__ce_alloc(struct mem_pool *mem_pool, size_t len) { struct cache_entry *ce; @@ -551,7 +556,10 @@ int cache_name_stage_compare(const char *name1, int len1, int stage1, const char return 0; } -static int index_name_stage_pos(struct index_state *istate, const char *name, int namelen, int stage) +static int index_name_stage_pos(struct index_state *istate, + const char *name, int namelen, + int stage, + enum index_search_mode search_mode) { int first, last; @@ -570,7 +578,7 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in first = next+1; } - if (istate->sparse_index && + if (search_mode == EXPAND_SPARSE && istate->sparse_index && first > 0) { /* Note: first <= istate->cache_nr */ struct cache_entry *ce = istate->cache[first - 1]; @@ -586,7 +594,7 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in ce_namelen(ce) < namelen && !strncmp(name, ce->name, ce_namelen(ce))) { ensure_full_index(istate); - return index_name_stage_pos(istate, name, namelen, stage); + return index_name_stage_pos(istate, name, namelen, stage, search_mode); } } @@ -595,7 +603,12 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in int index_name_pos(struct index_state *istate, const char *name, int namelen) { - return index_name_stage_pos(istate, name, namelen, 0); + return index_name_stage_pos(istate, name, namelen, 0, EXPAND_SPARSE); +} + +int index_entry_exists(struct index_state *istate, const char *name, int namelen) +{ + return index_name_stage_pos(istate, name, namelen, 0, NO_EXPAND_SPARSE) >= 0; } int remove_index_entry_at(struct index_state *istate, int pos) @@ -1222,7 +1235,7 @@ static int has_dir_name(struct index_state *istate, */ } - pos = index_name_stage_pos(istate, name, len, stage); + pos = index_name_stage_pos(istate, name, len, stage, EXPAND_SPARSE); if (pos >= 0) { /* * Found one, but not so fast. This could @@ -1322,7 +1335,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e strcmp(ce->name, istate->cache[istate->cache_nr - 1]->name) > 0) pos = index_pos_to_insert_pos(istate->cache_nr); else - pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce)); + pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), EXPAND_SPARSE); /* existing match? Just replace it. */ if (pos >= 0) { @@ -1357,7 +1370,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e if (!ok_to_replace) return error(_("'%s' appears as both a file and as a directory"), ce->name); - pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce)); + pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), EXPAND_SPARSE); pos = -pos-1; } return pos + 1; diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 535686a2954..78476de18ea 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -760,9 +760,9 @@ test_expect_success 'sparse-index is not expanded' ' ensure_not_expanded checkout - && ensure_not_expanded switch rename-out-to-out && ensure_not_expanded switch - && - git -C sparse-index reset --hard && + ensure_not_expanded reset --hard && ensure_not_expanded checkout rename-out-to-out -- deep/deeper1 && - git -C sparse-index reset --hard && + ensure_not_expanded reset --hard && ensure_not_expanded restore -s rename-out-to-out -- deep/deeper1 && echo >>sparse-index/README.md && @@ -772,6 +772,17 @@ test_expect_success 'sparse-index is not expanded' ' echo >>sparse-index/untracked.txt && ensure_not_expanded add . && + for ref in update-deep update-folder1 update-folder2 update-deep + do + echo >>sparse-index/README.md && + ensure_not_expanded reset --hard $ref || return 1 + done && + + ensure_not_expanded reset --hard update-deep && + ensure_not_expanded reset --keep base && + ensure_not_expanded reset --merge update-deep && + ensure_not_expanded reset --hard && + ensure_not_expanded checkout -f update-deep && test_config -C sparse-index pull.twohead ort && ( From patchwork Mon Oct 11 20:30:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12551013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C68D0C433EF for ; Mon, 11 Oct 2021 20:30:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AA78860F3A for ; Mon, 11 Oct 2021 20:30:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234986AbhJKUcg (ORCPT ); Mon, 11 Oct 2021 16:32:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42804 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231341AbhJKUca (ORCPT ); Mon, 11 Oct 2021 16:32:30 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A309DC06161C for ; Mon, 11 Oct 2021 13:30:29 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id e12so59916082wra.4 for ; Mon, 11 Oct 2021 13:30:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=xa3qONcj6w8N7PT2ZmUQK53RJ1vufRoiX8kHUeTGKu8=; b=VY6amkXo480iAK5s+GFzJJ3NvMxoQi+kPr1NvP3tukVRthT6Ja7EYMNc7QPEs5aZuJ toS0XQz72qDngIKDNvQ7hqhjfnvZajrpLvnr7gCiBBEvVKlPjoSz4TVK0VdTQnlbgloM YCuOteDivPcsUVCLAfLNSlAa1gs1qpqkerJOJhZP84GkNl3NAh79IG2gYWsbNSnleAVI lvq8tB9RoTWc9O2/XvMIyZFsI5ITWJGUktM5RsSb6SUaZVNi6FyhLYUTgVPayAMBEgO2 oHObbeGJMI7RxtHo4J5kPkwojIsVlmENynkWajsM+hVJPY2NeueQxbC94JZxU8aJdHyz dGPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=xa3qONcj6w8N7PT2ZmUQK53RJ1vufRoiX8kHUeTGKu8=; b=qoYGN861kU5Dfhf6i+wP9KGr+OVVvPWnKqY7uxRms1mTvGq30VadQ7hsBsFN3ilSfK ZZNDBTeD7lahw9nW6XZyo0a3zDPU+lC9UhIeBsNArkc5LdHUU/dcXrMBqwNGqWBX1VEe Y16XscO/MjjUF9/mHVPNnNbBfInnF4rMfUbkLUmvTxmIDUCI8MDQGSU3VgZxBpWOwkD8 CQ8FradgNcgvUQpjvxJDrpXW8fR9olXJXE0FQmwWXa9X7ol1j1TeT3c68TaYsmO9VwDA Pp9Htca93xNR7+QLjBAxbZ5q/rdPSOmLci6L9aGjSuRwpejQ7gq+ENQTYuPW1bleXAR+ ex/A== X-Gm-Message-State: AOAM533ZgQJK1NzSR5/J4lDDevNS30Y9ZCnixIUVuSsB2kWqDpoVl8cI 9xs85W0gCnw2Pt03EH5f8K2CoZGKTw0= X-Google-Smtp-Source: ABdhPJwtUhHZ3xefuzXkarWH2hgL3Ea1IP9qvbPRQsCAdvUOMXZn3FVCgqsummKSWiXbKmYOk6BY0w== X-Received: by 2002:a05:6000:184c:: with SMTP id c12mr27132248wri.127.1633984228255; Mon, 11 Oct 2021 13:30:28 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g144sm508332wmg.5.2021.10.11.13.30.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Oct 2021 13:30:27 -0700 (PDT) Message-Id: <00d14fb60bd952ac541627e0a1c56370036e3ca0.1633984223.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 11 Oct 2021 20:30:21 +0000 Subject: [PATCH v4 7/8] reset: make --mixed sparse-aware Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Remove the `ensure_full_index` guard on `read_from_tree` and update `git reset --mixed` to ensure it can use sparse directory index entries wherever possible. Sparse directory entries are reset use `diff_tree_oid`, which requires `change` and `add_remove` functions to process the internal contents of the sparse directory. The `recursive` diff option handles cases in which `reset --mixed` must diff/merge files that are nested multiple levels deep in a sparse directory. The use of pathspecs with `git reset --mixed` introduces scenarios in which internal contents of sparse directories may be matched by the pathspec. In order to reset *all* files in the repo that may match the pathspec, the following conditions on the pathspec require index expansion before performing the reset: * "magic" pathspecs * wildcard pathspecs that do not match only in-cone files or entire sparse directories * literal pathspecs matching something outside the sparse checkout definition Helped-by: Elijah Newren Signed-off-by: Victoria Dye --- builtin/reset.c | 78 +++++++++++++++++++++++- t/t1092-sparse-checkout-compatibility.sh | 17 ++++++ 2 files changed, 93 insertions(+), 2 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index 0ac0de7dc97..60517e7e1d6 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -148,7 +148,9 @@ static void update_index_from_diff(struct diff_queue_struct *q, * If the file 1) corresponds to an existing index entry with * skip-worktree set, or 2) does not exist in the index but is * outside the sparse checkout definition, add a skip-worktree bit - * to the new index entry. + * to the new index entry. Note that a sparse index will be expanded + * if this entry is outside the sparse cone - this is necessary + * to properly construct the reset sparse directory. */ pos = cache_name_pos(one->path, strlen(one->path)); if ((pos >= 0 && ce_skip_worktree(active_cache[pos])) || @@ -166,6 +168,73 @@ static void update_index_from_diff(struct diff_queue_struct *q, } } +static int pathspec_needs_expanded_index(const struct pathspec *pathspec) +{ + unsigned int i, pos; + int res = 0; + char *skip_worktree_seen = NULL; + + /* + * When using a magic pathspec, assume for the sake of simplicity that + * the index needs to be expanded to match all matchable files. + */ + if (pathspec->magic) + return 1; + + for (i = 0; i < pathspec->nr; i++) { + struct pathspec_item item = pathspec->items[i]; + + /* + * If the pathspec item has a wildcard, the index should be expanded + * if the pathspec has the possibility of matching a subset of entries inside + * of a sparse directory (but not the entire directory). + * + * If the pathspec item is a literal path, the index only needs to be expanded + * if a) the pathspec isn't in the sparse checkout cone (to make sure we don't + * expand for in-cone files) and b) it doesn't match any sparse directories + * (since we can reset whole sparse directories without expanding them). + */ + if (item.nowildcard_len < item.len) { + for (pos = 0; pos < active_nr; pos++) { + struct cache_entry *ce = active_cache[pos]; + + if (!S_ISSPARSEDIR(ce->ce_mode)) + continue; + + /* + * If the pre-wildcard length is longer than the sparse + * directory name and the sparse directory is the first + * component of the pathspec, need to expand the index. + */ + if (item.nowildcard_len > ce_namelen(ce) && + !strncmp(item.original, ce->name, ce_namelen(ce))) { + res = 1; + break; + } + + /* + * If the pre-wildcard length is shorter than the sparse + * directory and the pathspec does not match the whole + * directory, need to expand the index. + */ + if (!strncmp(item.original, ce->name, item.nowildcard_len) && + wildmatch(item.original, ce->name, 0)) { + res = 1; + break; + } + } + } else if (!path_in_cone_mode_sparse_checkout(item.original, &the_index) && + !matches_skip_worktree(pathspec, i, &skip_worktree_seen)) + res = 1; + + if (res > 0) + break; + } + + free(skip_worktree_seen); + return res; +} + static int read_from_tree(const struct pathspec *pathspec, struct object_id *tree_oid, int intent_to_add) @@ -178,9 +247,14 @@ static int read_from_tree(const struct pathspec *pathspec, opt.format_callback = update_index_from_diff; opt.format_callback_data = &intent_to_add; opt.flags.override_submodule_config = 1; + opt.flags.recursive = 1; opt.repo = the_repository; + opt.change = diff_change; + opt.add_remove = diff_addremove; + + if (pathspec->nr && the_index.sparse_index && pathspec_needs_expanded_index(pathspec)) + ensure_full_index(&the_index); - ensure_full_index(&the_index); if (do_diff_cache(tree_oid, &opt)) return 1; diffcore_std(&opt); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 78476de18ea..f19c1b3e2eb 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -778,11 +778,28 @@ test_expect_success 'sparse-index is not expanded' ' ensure_not_expanded reset --hard $ref || return 1 done && + ensure_not_expanded reset --mixed base && ensure_not_expanded reset --hard update-deep && ensure_not_expanded reset --keep base && ensure_not_expanded reset --merge update-deep && ensure_not_expanded reset --hard && + ensure_not_expanded reset base -- deep/a && + ensure_not_expanded reset base -- nonexistent-file && + ensure_not_expanded reset deepest -- deep && + + # Although folder1 is outside the sparse definition, it exists as a + # directory entry in the index, so the pathspec will not force the + # index to be expanded. + ensure_not_expanded reset deepest -- folder1 && + ensure_not_expanded reset deepest -- folder1/ && + + # Wildcard identifies only in-cone files, no index expansion + ensure_not_expanded reset deepest -- deep/\* && + + # Wildcard identifies only full sparse directories, no index expansion + ensure_not_expanded reset deepest -- folder\* && + ensure_not_expanded checkout -f update-deep && test_config -C sparse-index pull.twohead ort && ( From patchwork Mon Oct 11 20:30:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12551017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76159C433FE for ; Mon, 11 Oct 2021 20:30:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5FE9360C4C for ; Mon, 11 Oct 2021 20:30:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235001AbhJKUch (ORCPT ); Mon, 11 Oct 2021 16:32:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234985AbhJKUcd (ORCPT ); Mon, 11 Oct 2021 16:32:33 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43D70C061762 for ; Mon, 11 Oct 2021 13:30:30 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id t2so59908923wrb.8 for ; Mon, 11 Oct 2021 13:30:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=nEih7G58FeXZR7VfCiUcLv0bmQevEM54qXiHHW/Vpvo=; b=f9jD/6bVwIGH4TKaTR4C/F2tgShYRvzaEG32CY5XT+9as571+S2sFswKlNPcERxuV/ XfXED3GsYWO8Y35WpA3XatS5E9pIlEgJxkkAdzRNWNYjwUV+NCcXsES2yYwQaCcSsDaF vber7ZvMhWsQhy1x66ypLY4y/H1ZEivEnRXgVvwubSecBY+zgGhpx0geheyzPu0/kBXe 4V7b6Py6ADbmYc6aEPAqlWSwBDXqNcIXJkStWSDH8qiZvdUh4h6x2JcObv8gICisjIkV LebQmHKxK05hsUbdL+rW28avWvC8fKEHU+LAy27g8Q0B3bUDp9jm2RqScUn1cfK9nDtw +j9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=nEih7G58FeXZR7VfCiUcLv0bmQevEM54qXiHHW/Vpvo=; b=r7D+zdj3dYx0hHBZS8ScDN7mhOM17598KOrtuXKgm9UKQ68TeDJwjuc1xuYFGo1dAh 38SiEQXVgGL9156URb8K0740DqbPN8iaU5kPC/SXtJFt3bZMtabrylpV54CZLPUON2kt p7ajD1b6IEBB08bdD6m6yIsRMK7r4LLM5bQTSOpkyg/5upF5gifJYYbe//C0h3qx0Bpj iVCXpGCdwFo+TCKVazm1qzscj7wRmda/+O6U7ribUuvFtSsmYr7Vg0N39Qr/Hayth7Zu F/yv4o1W/WvjEHnBUWXQB2ydlo4jSDgvM79JZfJJ+4mBgOC1hdRgg2k7XQqdR4wDeqhq I2NA== X-Gm-Message-State: AOAM530bjVTYfNq+y8a8XqVvU5JY6h9u7PRkEPsgLpiFkUgXHEB2hMZ7 oZeQoWj44jHFB/T2MWdtSZP8D1By9r0= X-Google-Smtp-Source: ABdhPJyPSZKKYX3D2t2kTy+epVv3JPdhGQib9a9BrWwuHKX5sCsZ5TauHid0rdSfIAl5jiXFH5AHXA== X-Received: by 2002:a1c:988c:: with SMTP id a134mr1231708wme.116.1633984228847; Mon, 11 Oct 2021 13:30:28 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s14sm8495712wro.76.2021.10.11.13.30.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Oct 2021 13:30:28 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 11 Oct 2021 20:30:22 +0000 Subject: [PATCH v4 8/8] unpack-trees: improve performance of next_cache_entry Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye To find the first non-unpacked cache entry, `next_cache_entry` iterates through index, starting at `cache_bottom`. The performance of this in full indexes is helped by `cache_bottom` advancing with each invocation of `mark_ce_used` (called by `unpack_index_entry`). However, the presence of sparse directories can prevent the `cache_bottom` from advancing in a sparse index case, effectively forcing `next_cache_entry` to search from the beginning of the index each time it is called. The `cache_bottom` must be preserved for the sparse index (see 17a1bb570b (unpack-trees: preserve cache_bottom, 2021-07-14)). Therefore, to retain the benefit `cache_bottom` provides in non-sparse index cases, a separate `hint` position indicates the first position `next_cache_entry` should search, updated each execution with a new position. Signed-off-by: Victoria Dye --- unpack-trees.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/unpack-trees.c b/unpack-trees.c index 8ea0a542da8..b94733de6be 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -645,17 +645,24 @@ static void mark_ce_used_same_name(struct cache_entry *ce, } } -static struct cache_entry *next_cache_entry(struct unpack_trees_options *o) +static struct cache_entry *next_cache_entry(struct unpack_trees_options *o, int *hint) { const struct index_state *index = o->src_index; int pos = o->cache_bottom; + if (*hint > pos) + pos = *hint; + while (pos < index->cache_nr) { struct cache_entry *ce = index->cache[pos]; - if (!(ce->ce_flags & CE_UNPACKED)) + if (!(ce->ce_flags & CE_UNPACKED)) { + *hint = pos + 1; return ce; + } pos++; } + + *hint = pos; return NULL; } @@ -1365,12 +1372,13 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str /* Are we supposed to look at the index too? */ if (o->merge) { + int hint = -1; while (1) { int cmp; struct cache_entry *ce; if (o->diff_index_cached) - ce = next_cache_entry(o); + ce = next_cache_entry(o, &hint); else ce = find_cache_entry(info, p); @@ -1690,7 +1698,7 @@ static int verify_absent(const struct cache_entry *, int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options *o) { struct repository *repo = the_repository; - int i, ret; + int i, hint, ret; static struct cache_entry *dfc; struct pattern_list pl; int free_pattern_list = 0; @@ -1763,13 +1771,15 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options info.pathspec = o->pathspec; if (o->prefix) { + hint = -1; + /* * Unpack existing index entries that sort before the * prefix the tree is spliced into. Note that o->merge * is always true in this case. */ while (1) { - struct cache_entry *ce = next_cache_entry(o); + struct cache_entry *ce = next_cache_entry(o, &hint); if (!ce) break; if (ce_in_traverse_path(ce, &info)) @@ -1790,8 +1800,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options /* Any left-over entries in the index? */ if (o->merge) { + hint = -1; while (1) { - struct cache_entry *ce = next_cache_entry(o); + struct cache_entry *ce = next_cache_entry(o, &hint); if (!ce) break; if (unpack_index_entry(ce, o) < 0)