From patchwork Thu Oct 7 21:15:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12543421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64440C433FE for ; Thu, 7 Oct 2021 21:15:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 475E2610A5 for ; Thu, 7 Oct 2021 21:15:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233495AbhJGVRi (ORCPT ); Thu, 7 Oct 2021 17:17:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39430 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233283AbhJGVRg (ORCPT ); Thu, 7 Oct 2021 17:17:36 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE527C061755 for ; Thu, 7 Oct 2021 14:15:42 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id r7so23067271wrc.10 for ; Thu, 07 Oct 2021 14:15:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ldyVwS09nc2OLJia/8NlZArMqqWxVI8EkShpKdO52GA=; b=RF81/+dYDiUUltsdKX6Amm90KelFSEfQp+aqh29kF4zyQW/sJsOuox9xnzN4NoKdKu cjI/XSKGNYTG7dttxhtZXmoeq8JvBGmZ9NIncOs+W3yaViz7x7Wfou6C++IYbWiQXYKP Kkwbs8XUReQpJEfQixxPbkQ6SONIrkQ0C98Vjrs+1zSLzgbdkwFngZbL5xfS0cYdwjQ7 KPKxVbf8bocZYlYO1Mi1873gkIE3JCGRJgStovXLAr/ITxTZo9C+FLfR7yiSqAh55XXU fKi8gkgednkiGvQ4nuGN6wAyRiqFpcThBYm8UpScVnLrfgeAnvxd9xS4jjTaAqnjMk4F Blig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ldyVwS09nc2OLJia/8NlZArMqqWxVI8EkShpKdO52GA=; b=JbUjZ6YtIBOQEx/sU4y5HNaKQobJ++/OQitwAL1RnhqPdIqouXIfDrke3npXxX6ZzR LQLuUro8QkDCwQF3qtiFcl0rgZs9qwDqPUUfgNPmes+jyWWHv9cVtZIQIo/KdfFMu3nu A/gYWHRgNHFnKyAnAai4RX6QjVz4GlA8zQD0zbywDd3kcpJS2HgrKjgpoaLAGP0k8VFI rQI/WsApKwLbeznlCIAKFL4bP5Vd+x74ESlB4odhN9otdgJHm7AsOE6jvvHlwtHA5b9u WZkvA7BrY87HJn2cs6d04q+kwnHjFa7FUG0SF/XZeYS2k7Jdkvo3POlcsah+6EscaMB/ PA8Q== X-Gm-Message-State: AOAM533i2uKMZB24ZJ+TGA1R7VYpjX12UtHoy3zJ1mvHksNKgCa/5Sre yGXy9cbmd0kIum4hQymjcD7n8ZUDMFo= X-Google-Smtp-Source: ABdhPJxR8u0ti1tpCc51jBu8plghcI+pmiFij/jR9KqVoyUwTHFEZ8IPGxzqj3L5o1wofdmhqlTktQ== X-Received: by 2002:a05:600c:2193:: with SMTP id e19mr7150075wme.38.1633641341357; Thu, 07 Oct 2021 14:15:41 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z79sm501749wmc.17.2021.10.07.14.15.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Oct 2021 14:15:41 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 07 Oct 2021 21:15:31 +0000 Subject: [PATCH v3 1/8] reset: rename is_missing to !is_in_reset_tree Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Rename and invert value of `is_missing` to `is_in_reset_tree` to make the variable more descriptive of what it represents. Signed-off-by: Victoria Dye --- builtin/reset.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index 51c9e2f43ff..d3695ce43c4 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -131,10 +131,10 @@ static void update_index_from_diff(struct diff_queue_struct *q, for (i = 0; i < q->nr; i++) { struct diff_filespec *one = q->queue[i]->one; - int is_missing = !(one->mode && !is_null_oid(&one->oid)); + int is_in_reset_tree = one->mode && !is_null_oid(&one->oid); struct cache_entry *ce; - if (is_missing && !intent_to_add) { + if (!is_in_reset_tree && !intent_to_add) { remove_file_from_cache(one->path); continue; } @@ -144,7 +144,7 @@ static void update_index_from_diff(struct diff_queue_struct *q, if (!ce) die(_("make_cache_entry failed for path '%s'"), one->path); - if (is_missing) { + if (!is_in_reset_tree) { ce->ce_flags |= CE_INTENT_TO_ADD; set_object_name_for_intent_to_add_entry(ce); } From patchwork Thu Oct 7 21:15:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Willford X-Patchwork-Id: 12543425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CEFBC4332F for ; Thu, 7 Oct 2021 21:15:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7DB4B610A5 for ; Thu, 7 Oct 2021 21:15:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233709AbhJGVRj (ORCPT ); Thu, 7 Oct 2021 17:17:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232200AbhJGVRh (ORCPT ); Thu, 7 Oct 2021 17:17:37 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A93BC061570 for ; Thu, 7 Oct 2021 14:15:43 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id t2so23079176wrb.8 for ; Thu, 07 Oct 2021 14:15:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ebqIh5AHU/zofXoI8jc57lQ9o+gXe7fI807819BxbPM=; b=iyMLAzksOlCfayw/yavdo8CzmBYzV2GmqjfQ5ATKHOumwgiaEh0hYDhMQAMeeWX7f5 nE04I4Zx+UOD0FoM6KqYwImmwiMDBABPHLVtEY1tIQ6qkBjMbGo45We/gDXP/aNuyM+8 qsgT28o+h9GnaQ6uZaN0jCpBE2yafmEP1/to1peQCS6F+nwO1oIE/C2bei15cTWQdXyS GXw/hO9yz9ynprh+2Dd33lw2j8fifIBspG0EEt+WIiv2pqP0VAKh4Q004JyBJQkRF3I6 /BAu14Pm7x0gp21Ng72cVkdt9YQu2htlpxmQ237MLohL4Q/mnoBWf+gG4kC4WNN+uY4F 3yPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ebqIh5AHU/zofXoI8jc57lQ9o+gXe7fI807819BxbPM=; b=X4iMBlhWEXN2vtCNYmZi3s9R9sfnWpi88bgU4oqdWuVvfpv736R/5Df4yJpYYEmr5L E1zWxTJ7wZKu+Sh7dni6R1BowCNR0hybbqdMuniaX38+Jg4qjVWHVXTyV7BMYMdxyH33 6HRJmHIa/1NJAPHP38JfNA9FHa72jt0txU+55bso0+BUARtYlPyni/f7MNMlceMYfbkh Aj06mzAhR6VnoIZU9Lm4zQb2exOPgp99bp8/CNcZ/2nJFSvagxwnnsqLEADYb9MdZbED tBKxiTzX0IBpWRs947iJa/HPktbF+Zn8YkznaIh0zw7V1h/1v2jUigozAc//6pjwikRG S4nQ== X-Gm-Message-State: AOAM5307TiceNAJDJ/K5fG6L++rj3kueH7+4kvcba2ixSJaN6Quy2JVi OOcl3ujwqI3xhNIl19FxnTZSbGWo6v4= X-Google-Smtp-Source: ABdhPJwgks+0YRqjObb26bhR4zi0GNyYLGp4YJnfP80SmFcjxMwnKf6sQzprel4SMDzFfEp8Rb+nKw== X-Received: by 2002:a5d:6d8a:: with SMTP id l10mr8302896wrs.121.1633641342043; Thu, 07 Oct 2021 14:15:42 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j11sm464265wrp.27.2021.10.07.14.15.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Oct 2021 14:15:41 -0700 (PDT) Message-Id: <1f6da84830b00b7d33ac8ae4c6a46a90606c901f.1633641339.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 07 Oct 2021 21:15:32 +0000 Subject: [PATCH v3 2/8] reset: preserve skip-worktree bit in mixed reset Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Kevin Willford Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Kevin Willford From: Kevin Willford Change `update_index_from_diff` to set `skip-worktree` when applicable for new index entries. When `git reset --mixed ` is run, entries in the index with differences between the pre-reset HEAD and reset are identified and handled with `update_index_from_diff`. For each file, a new cache entry in inserted into the index, created from the side of the reset (without changing the working tree). However, the newly-created entry must have `skip-worktree` explicitly set in either of the following scenarios: 1. the file is in the current index and has `skip-worktree` set 2. the file is not in the current index but is outside of a defined sparse checkout definition Not setting the `skip-worktree` bit leads to likely-undesirable results for a user. It causes `skip-worktree` settings to disappear on the "diff"-containing files (but *only* the diff-containing files), leading to those files now showing modifications in `git status`. For example, when running `git reset --mixed` in a sparse checkout, some file entries outside of sparse checkout could show up as deleted, despite the user never deleting anything (and not wanting them on-disk anyway). Additionally, add a test to `t7102` to ensure `skip-worktree` is preserved in a basic `git reset --mixed` scenario and update a failure-documenting test from 19a0acc (t1092: test interesting sparse-checkout scenarios, 2021-01-23) with new expected behavior. Helped-by: Junio C Hamano Signed-off-by: Victoria Dye --- builtin/reset.c | 14 ++++++++++++++ t/t1092-sparse-checkout-compatibility.sh | 19 +++++-------------- t/t7102-reset.sh | 17 +++++++++++++++++ 3 files changed, 36 insertions(+), 14 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index d3695ce43c4..e441b6601b9 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -25,6 +25,7 @@ #include "cache-tree.h" #include "submodule.h" #include "submodule-config.h" +#include "dir.h" #define REFRESH_INDEX_DELAY_WARNING_IN_MS (2 * 1000) @@ -130,6 +131,7 @@ static void update_index_from_diff(struct diff_queue_struct *q, int intent_to_add = *(int *)data; for (i = 0; i < q->nr; i++) { + int pos; struct diff_filespec *one = q->queue[i]->one; int is_in_reset_tree = one->mode && !is_null_oid(&one->oid); struct cache_entry *ce; @@ -141,6 +143,18 @@ static void update_index_from_diff(struct diff_queue_struct *q, ce = make_cache_entry(&the_index, one->mode, &one->oid, one->path, 0, 0); + + /* + * If the file 1) corresponds to an existing index entry with + * skip-worktree set, or 2) does not exist in the index but is + * outside the sparse checkout definition, add a skip-worktree bit + * to the new index entry. + */ + pos = cache_name_pos(one->path, strlen(one->path)); + if ((pos >= 0 && ce_skip_worktree(active_cache[pos])) || + (pos < 0 && !path_in_sparse_checkout(one->path, &the_index))) + ce->ce_flags |= CE_SKIP_WORKTREE; + if (!ce) die(_("make_cache_entry failed for path '%s'"), one->path); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 886e78715fe..889079f55b8 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -459,26 +459,17 @@ test_expect_failure 'blame with pathspec outside sparse definition' ' test_all_match git blame deep/deeper2/deepest/a ' -# NEEDSWORK: a sparse-checkout behaves differently from a full checkout -# in this scenario, but it shouldn't. -test_expect_failure 'checkout and reset (mixed)' ' +test_expect_success 'checkout and reset (mixed)' ' init_repos && test_all_match git checkout -b reset-test update-deep && test_all_match git reset deepest && - test_all_match git reset update-folder1 && - test_all_match git reset update-folder2 -' - -# NEEDSWORK: a sparse-checkout behaves differently from a full checkout -# in this scenario, but it shouldn't. -test_expect_success 'checkout and reset (mixed) [sparse]' ' - init_repos && - test_sparse_match git checkout -b reset-test update-deep && - test_sparse_match git reset deepest && + # Because skip-worktree is preserved, resetting to update-folder1 + # will show worktree changes for full-checkout that are not present + # in sparse-checkout or sparse-index. test_sparse_match git reset update-folder1 && - test_sparse_match git reset update-folder2 + run_on_sparse test_path_is_missing folder1 ' test_expect_success 'merge, cherry-pick, and rebase' ' diff --git a/t/t7102-reset.sh b/t/t7102-reset.sh index 601b2bf97f0..d05426062ec 100755 --- a/t/t7102-reset.sh +++ b/t/t7102-reset.sh @@ -472,6 +472,23 @@ test_expect_success '--mixed refreshes the index' ' test_cmp expect output ' +test_expect_success '--mixed preserves skip-worktree' ' + echo 123 >>file2 && + git add file2 && + git update-index --skip-worktree file2 && + git reset --mixed HEAD >output && + test_must_be_empty output && + + cat >expect <<-\EOF && + Unstaged changes after reset: + M file2 + EOF + git update-index --no-skip-worktree file2 && + git add file2 && + git reset --mixed HEAD >output && + test_cmp expect output +' + test_expect_success 'resetting specific path that is unmerged' ' git rm --cached file2 && F1=$(git rev-parse HEAD:file1) && From patchwork Thu Oct 7 21:15:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12543427 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91C0CC433EF for ; Thu, 7 Oct 2021 21:15:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7BAAB610A5 for ; Thu, 7 Oct 2021 21:15:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233859AbhJGVRl (ORCPT ); Thu, 7 Oct 2021 17:17:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233590AbhJGVRi (ORCPT ); Thu, 7 Oct 2021 17:17:38 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 166D9C061755 for ; Thu, 7 Oct 2021 14:15:44 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id t8so23257907wri.1 for ; Thu, 07 Oct 2021 14:15:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=49LzXIoxEJ8FYGTP5PwYzcjg+epu0jj2WCce6da3r9c=; b=SN6+U3CsyDe+rzqZpCOo8H7Q/fM0YtzIUDDUd+B0+shAHwSPWwp3ckJkvXtY3B3VHX KS6UJmMxo2F6+Y9vHwXY5cqRdSVxfqQ9vytH3EkhW3lqweOdcK5rHi2PXxTX2chQHonP 13kYxXueNmklbfXNZfw7X076PxfnYkiqGG+KW4N8JHBdNLdaMwSE8SF7Qv71aURoKs82 dx8L1g+4VhIFfmwnpeL41NdRJpxzzd5Y/nLLuoKKFKY0zGCRbMguE8o0iBkxm0prYRlQ bW6IstoBg89yg5Hdbu6oSraYJNuTHR0NIVbYsDwn0Bb0zItU2VrLtAgztQbqvZZz/Uwl birA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=49LzXIoxEJ8FYGTP5PwYzcjg+epu0jj2WCce6da3r9c=; b=Dd+L0IAYIJQQ/Mgc6yFIKGPysQ5dSV0fmWbvvyL3tE/Xda5KjCqaz3fB3/lxCmE6Tf +u6bIkH/EOnHr3VwvBZHfyY8iPYc7mw6l9CFyEgRZLiP8nzp9h9iCdDeHHzIdz8uB3/J YVrGTRoMvzvAZki6xWX1W2Jup6CYBNaNvWmwIu7uhlXWY6uDVhBKMCKB3hcPL7SerfDZ t6jRXmwnPofMlMc1PWJxPCy1m5d6e+Mne2LFmStw6fPmjCTr7bdyMX/UUptNBb4ihQB1 YODlqwxxoQybf7t1FvT5zIOObPxoh4EwACFT5VhQmrAVs1yZlfzKN5dZq5xLFOXVvWHB VoTg== X-Gm-Message-State: AOAM530SGYHd/XLheodZ1MHMUUCa/SGlMWVtax85t3lt7EKGDJIoIJ5S o85QrL585AQnfb4Qh6c0SC3PYndoOd0= X-Google-Smtp-Source: ABdhPJy7XM31ruM3ZRNP7FnR7yAdMqBqYX/rZ7KO1akCEH0FhLoITEbtwNhhiBr2UcOP2FHkO2OzFg== X-Received: by 2002:adf:a390:: with SMTP id l16mr8163218wrb.72.1633641342721; Thu, 07 Oct 2021 14:15:42 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i92sm436285wri.28.2021.10.07.14.15.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Oct 2021 14:15:42 -0700 (PDT) Message-Id: <014a408ea5d9894197c60f8d712749ea3cc39c9d.1633641339.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 07 Oct 2021 21:15:33 +0000 Subject: [PATCH v3 3/8] update-index: add --force-full-index option for expand/collapse test Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Add a new `--force-full-index` option to `git update-index`, which skips explicitly setting `command_requires_full_index`. This option, intended for use in internal testing purposes only, lets `git update-index` run as a command without sparse index compatibility implemented, even after it receives updates to otherwise use the sparse index. The specific test `--force-full-index` is intended for - `t1092 - sparse-index is expanded and converted back` - verifies index compatibility in commands that do not change the default (enabled) `command_requires_full_index` repo setting. In the past, the test used `git reset`. However, as `reset` and other commands are integrated with the sparse index, the command used in the test would need to keep changing. Conversely, the `--force-full-index` option makes `git update-index` behave like a not-yet-sparse-aware command, and can be used in the test indefinitely without interfering with future sparse index integrations. Helped-by: Junio C Hamano Signed-off-by: Victoria Dye Reviewed-by: Bagas Sanjaya --- Documentation/git-update-index.txt | 5 +++++ builtin/update-index.c | 11 +++++++++++ t/t1092-sparse-checkout-compatibility.sh | 2 +- 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt index 2853f168d97..06255e321a3 100644 --- a/Documentation/git-update-index.txt +++ b/Documentation/git-update-index.txt @@ -24,6 +24,7 @@ SYNOPSIS [--[no-]fsmonitor] [--really-refresh] [--unresolve] [--again | -g] [--info-only] [--index-info] + [--force-full-index] [-z] [--stdin] [--index-version ] [--verbose] [--] [...] @@ -170,6 +171,10 @@ time. Version 4 is relatively young (first released in 1.8.0 in October 2012). Other Git implementations such as JGit and libgit2 may not support it yet. +--force-full-index:: + Force the command to operate on a full index, expanding a sparse + index if necessary. + -z:: Only meaningful with `--stdin` or `--index-info`; paths are separated with NUL character instead of LF. diff --git a/builtin/update-index.c b/builtin/update-index.c index 187203e8bb5..32ada3ead77 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -964,6 +964,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) int split_index = -1; int force_write = 0; int fsmonitor = -1; + int use_default_full_index = 0; struct lock_file lock_file = LOCK_INIT; struct parse_opt_ctx_t ctx; strbuf_getline_fn getline_fn; @@ -1069,6 +1070,8 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) {OPTION_SET_INT, 0, "no-fsmonitor-valid", &mark_fsmonitor_only, NULL, N_("clear fsmonitor valid bit"), PARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL, UNMARK_FLAG}, + OPT_SET_INT(0, "force-full-index", &use_default_full_index, + N_("run with full index explicitly required"), 1), OPT_END() }; @@ -1082,6 +1085,14 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) if (newfd < 0) lock_error = errno; + /* + * If --force-full-index is set, the command should skip manually + * setting `command_requires_full_index`. + */ + prepare_repo_settings(r); + if (!use_default_full_index) + r->settings.command_requires_full_index = 1; + entries = read_cache(); if (entries < 0) die("cache corrupted"); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 889079f55b8..4aa4fef7b4f 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -635,7 +635,7 @@ test_expect_success 'sparse-index is expanded and converted back' ' init_repos && GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ - git -C sparse-index -c core.fsmonitor="" reset --hard && + git -C sparse-index -c core.fsmonitor="" update-index --force-full-index && test_region index convert_to_sparse trace2.txt && test_region index ensure_full_index trace2.txt ' From patchwork Thu Oct 7 21:15:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12543429 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF76DC433F5 for ; Thu, 7 Oct 2021 21:15:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9783E61058 for ; Thu, 7 Oct 2021 21:15:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233980AbhJGVRm (ORCPT ); Thu, 7 Oct 2021 17:17:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233614AbhJGVRi (ORCPT ); Thu, 7 Oct 2021 17:17:38 -0400 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5C21C061570 for ; Thu, 7 Oct 2021 14:15:44 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id i12so10489890wrb.7 for ; Thu, 07 Oct 2021 14:15:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=sv612smGvd4dq4LynrPLQSKIFJ3a45Vznz69O66lrI4=; b=lhAk9RFblt4jFDWg0yO+Vf6mltnx1yxGpKAqlY3+TwmtLQTXWzQBQyTIjnOZAsFFqO mIuAy1jExOcsCZ8m5UOkW0mMGRA1OlJyVtL9NlmB4pm67y6ulIu1/dViXOT1nhw7BZBn reG1et/C1+pwDGTTZ/cE8zA7Q36izmNJvASCXs8lIOnRfHj2NXwtXTxw8n+OmGP9ZM0p hmluJgfeNZRRlm+yeFiO7dWaK57pGlbRxdVmOY222s4yOr271xL7a/5IY0t1/wk4zqbY VSS+loNFqUd9oXFr6ppfuxKsMdB2NqbBbdRaJCX1EeRpKUhEqSRTO+Z1a/qWRRjuaahV 6OpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=sv612smGvd4dq4LynrPLQSKIFJ3a45Vznz69O66lrI4=; b=DmiPdU2kzQG9UpS90IqRgJD1fbtW2HNoWcq41aHkE67V0833qPX4/CEb88YsI3vFS5 rH9LCSOROC+M9NultHDTmsE3pjd9Jf1DgjpexyuJ1uMmdgN3a9XGYxTZyjj1LX3uD/jd KTf7aWNG6U6yv6cKTrhRZVbvZ1ZK7k1rRaoMr+69AXma6oeV6s2/5EqjRBC+K2ve19Y8 fkbeiepXgPEA5TiYHuA6OWS7sOw7jH5wonklaybJCw9Hh7B8LsabeqywFPa/tCnpCFPR M673n8bcIvkUeIgzex1mk5XkjSiLTuW7V4NN+MrtG+IXb3mpK0MHPPtQwU5jHqKuzTtA GAbg== X-Gm-Message-State: AOAM530WupVqw/mZgCJkjfs/wkW2MJByLm4qBbrTJNZenq9x2B+N+OEp NTd5XaDRSrg595jnjTLV/hk8qHTX5EA= X-Google-Smtp-Source: ABdhPJwBzIVrApOuBP4qF4IKJ0P/74m8t0VhysIZdJtqftsXgQA/Cqii1JYinbgAVateyMYUj9MH8A== X-Received: by 2002:adf:a347:: with SMTP id d7mr8465986wrb.139.1633641343341; Thu, 07 Oct 2021 14:15:43 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i92sm436305wri.28.2021.10.07.14.15.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Oct 2021 14:15:43 -0700 (PDT) Message-Id: <7f21cf53e9d71de8c6351ae0f21ed531b7d13c42.1633641339.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 07 Oct 2021 21:15:34 +0000 Subject: [PATCH v3 4/8] reset: expand test coverage for sparse checkouts Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Add new tests for `--merge` and `--keep` modes, as well as mixed reset with pathspecs. New performance test cases exercise various execution paths for `reset`. Co-authored-by: Derrick Stolee Signed-off-by: Derrick Stolee Signed-off-by: Victoria Dye --- t/perf/p2000-sparse-operations.sh | 3 + t/t1092-sparse-checkout-compatibility.sh | 84 ++++++++++++++++++++++++ 2 files changed, 87 insertions(+) diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index 597626276fb..bfd332120c8 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -110,5 +110,8 @@ test_perf_on_all git add -A test_perf_on_all git add . test_perf_on_all git commit -a -m A test_perf_on_all git checkout -f - +test_perf_on_all git reset +test_perf_on_all git reset --hard +test_perf_on_all git reset -- does-not-exist test_done diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 4aa4fef7b4f..875cdcb0495 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -472,6 +472,90 @@ test_expect_success 'checkout and reset (mixed)' ' run_on_sparse test_path_is_missing folder1 ' +test_expect_success 'checkout and reset (merge)' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents a && + test_all_match git reset --merge deepest && + test_all_match git status --porcelain=v2 && + + test_all_match git reset --hard update-deep && + run_on_all ../edit-contents deep/a && + test_all_match test_must_fail git reset --merge deepest +' + +test_expect_success 'checkout and reset (keep)' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents a && + test_all_match git reset --keep deepest && + test_all_match git status --porcelain=v2 && + + test_all_match git reset --hard update-deep && + run_on_all ../edit-contents deep/a && + test_all_match test_must_fail git reset --keep deepest +' + +test_expect_success 'reset with pathspecs inside sparse definition' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents deep/a && + + test_all_match git reset base -- deep/a && + test_all_match git status --porcelain=v2 && + + test_all_match git reset base -- nonexistent-file && + test_all_match git status --porcelain=v2 && + + test_all_match git reset deepest -- deep && + test_all_match git status --porcelain=v2 +' + +# Although the working tree differs between full and sparse checkouts after +# reset, the state of the index is the same. +test_expect_success 'reset with pathspecs outside sparse definition' ' + init_repos && + test_all_match git checkout -b reset-test base && + + test_sparse_match git reset update-folder1 -- folder1 && + git -C full-checkout reset update-folder1 -- folder1 && + test_sparse_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder1 && + + test_sparse_match git reset update-folder2 -- folder2/a && + git -C full-checkout reset update-folder2 -- folder2/a && + test_sparse_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder2/a +' + +test_expect_success 'reset with wildcard pathspec' ' + init_repos && + + test_all_match git checkout -b reset-test update-deep && + test_all_match git reset base -- \*/a && + test_all_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder1/a && + + test_all_match git reset base -- folder\* && + test_all_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder2 +' + test_expect_success 'merge, cherry-pick, and rebase' ' init_repos && From patchwork Thu Oct 7 21:15:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12543431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 686C6C433FE for ; Thu, 7 Oct 2021 21:15:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 50C8C61058 for ; Thu, 7 Oct 2021 21:15:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234007AbhJGVRn (ORCPT ); Thu, 7 Oct 2021 17:17:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232200AbhJGVRj (ORCPT ); Thu, 7 Oct 2021 17:17:39 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 602C7C061570 for ; Thu, 7 Oct 2021 14:15:45 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id r10so23112827wra.12 for ; Thu, 07 Oct 2021 14:15:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+dch2aTLAv85TXXA5lgJ4m8qNH4dps5WqL8q9u/O5kk=; b=GiqxvrLcKhOyzq6Zj1xNKHxmueqhGbaKAghBQaunqzSDWpcGFfYe46AgQZhjYo1Fgo 3Pv5sBWQ1ZgC0Iunc1DXM6N5TC3Mw9x7JDeqrKlZQZjyXMwPV9TeTocIjX+qRYfWJaXI Dnw9cjMNgZZS7EbgbQcutGuUGZDFZtXHv6DSZ71NH/hrDpqw/CEpRlYqYM9B6n5rBXet GBwzD/AuJV5xLGInpvdJB56qS1hyHeRWAkmZbcbLC3quAqB/imFO4YnZsy7D5vqnKGlQ VnO/2fb/fT1Pk76q6Xb9EoDVSKj6iHtYa9kz8qb0x4RSRjTBTMmuTa1AcGgnGCyCO9U/ H92w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+dch2aTLAv85TXXA5lgJ4m8qNH4dps5WqL8q9u/O5kk=; b=L5mRYbi402d4dL/h45YyRopQAt2K0Q5KKIFyzH2F/nYcUqSY8iQyJXsNzgVIv4cQqW IjVKkzYIF4HG9jc8JxQ52HZg0VuVfHxJB0b+TKWScysSTjPIK2uWN2kBaFilFCNWHo1A tWL5kYO+tzJ4atr2hojA9vY9nnTmg3WyZow+T9Aa4ob4rvqGAVSqwk5O+/5Ybzy/C4L6 aZk6QMXdy77R+nouMSgIpV1uvS+ZmTg1XiLKcoyQ2zHyf3y38IOj8EQ4unAqJH2Bi4qD +dxt2PSiWuxw8pJOLaNPG4KzFAbvSqEyefaWlR+q1wBkk1ICMAXkcxWEHaLBAPf5rXyQ IhTA== X-Gm-Message-State: AOAM533mjyq1DSYKuKiy7Iz+ys5hUkcev6oKXPMvcBwW9H3I5Kqa56zd uG/60WvM8ThWjxDP1MWvPYp3wXzs0jI= X-Google-Smtp-Source: ABdhPJywXTB3D+DTwcxxfnJccBSbnJjlDthoo+AZZEKDTtcaQpk/NNSHlxxkb52ABKTzkFFwoWtBdA== X-Received: by 2002:a7b:cd0f:: with SMTP id f15mr6978913wmj.173.1633641343996; Thu, 07 Oct 2021 14:15:43 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d7sm482799wrh.13.2021.10.07.14.15.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Oct 2021 14:15:43 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 07 Oct 2021 21:15:35 +0000 Subject: [PATCH v3 5/8] reset: integrate with sparse index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Disable `command_requires_full_index` repo setting and add `ensure_full_index` guards around code paths that cannot yet use sparse directory index entries. `reset --soft` does not modify the index, so no compatibility changes are needed for it to function without expanding the index. For all other reset modes (`--mixed`, `--hard`, `--keep`, `--merge`), the full index is expanded to prevent cache tree corruption and invalid variable accesses. Additionally, the `read_cache()` check verifying an uncorrupted index is moved after argument parsing and preparing the repo settings. The index is not used by the preceding argument handling, but `read_cache()` must be run *after* enabling sparse index for the command (so that the index is not expanded unnecessarily) and *before* using the index for reset (so that it is verified as uncorrupted). Signed-off-by: Victoria Dye --- builtin/reset.c | 10 +++++++--- cache-tree.c | 1 + 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index e441b6601b9..0ac0de7dc97 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -180,6 +180,7 @@ static int read_from_tree(const struct pathspec *pathspec, opt.flags.override_submodule_config = 1; opt.repo = the_repository; + ensure_full_index(&the_index); if (do_diff_cache(tree_oid, &opt)) return 1; diffcore_std(&opt); @@ -257,9 +258,6 @@ static void parse_args(struct pathspec *pathspec, } *rev_ret = rev; - if (read_cache() < 0) - die(_("index file corrupt")); - parse_pathspec(pathspec, 0, PATHSPEC_PREFER_FULL | (patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0), @@ -405,6 +403,12 @@ int cmd_reset(int argc, const char **argv, const char *prefix) if (intent_to_add && reset_type != MIXED) die(_("-N can only be used with --mixed")); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + + if (read_cache() < 0) + die(_("index file corrupt")); + /* Soft reset does not touch the index file nor the working tree * at all, but requires them in a good order. Other resets reset * the index file to the tree object we are switching to. */ diff --git a/cache-tree.c b/cache-tree.c index 90919f9e345..9be19c85b66 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -776,6 +776,7 @@ void prime_cache_tree(struct repository *r, cache_tree_free(&istate->cache_tree); istate->cache_tree = cache_tree(); + ensure_full_index(istate); prime_cache_tree_rec(r, istate->cache_tree, tree); istate->cache_changed |= CACHE_TREE_CHANGED; trace2_region_leave("cache-tree", "prime_cache_tree", the_repository); From patchwork Thu Oct 7 21:15:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12543433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58CBEC4332F for ; Thu, 7 Oct 2021 21:15:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4465761058 for ; Thu, 7 Oct 2021 21:15:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233748AbhJGVRo (ORCPT ); Thu, 7 Oct 2021 17:17:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233796AbhJGVRk (ORCPT ); Thu, 7 Oct 2021 17:17:40 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D017C061755 for ; Thu, 7 Oct 2021 14:15:46 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id m22so23220330wrb.0 for ; Thu, 07 Oct 2021 14:15:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=B++aLT8ZW8Km0k3aalgC3sb5QzcJgU4kIuS6VGfQE5A=; b=Gagk1qY5LiwUFbBpb0MRVqDuHo1h63fhiaOlzV/447E+aFjGDv5WCvS9JjfjgZ5NXK Nnkegci/UkMJ/i6wBXz0h63tanHLlHltNRI+Aspnu/Ub53tNeA8J477gg/rsgIeg4EF9 8kEk2xAcsejrh9wEI6h24smpE3OyLYhEuKRW5/rGNp9BplGuI/LaJRJQnnoRc645g5N1 GeQ/hKhtuh6TJMBdWtHrJFK/9SX2E1/1KfsQii6zef7Ewzh3Zs7vzOcidicREww4/KcJ /ZpWm3UxACnZxLlfFfNMCPEr9kpLv/fOQyNtAs5kDLpnMO0CjPEVV1JkIaMuvMJLZyFX //eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=B++aLT8ZW8Km0k3aalgC3sb5QzcJgU4kIuS6VGfQE5A=; b=m8AIjP55QA7qjRO0RImb1+0FsW3hI3D3ZzzFsxB5RupXd/bvUR2hY8jjYytfU0nQM7 9lGTcMl25E0/nR2N5d3gWJinDIcSZZHBneXFVqBavtOx5QxFbuHmwk6HV1yV0YxhAoW9 k20RO6gKkviKt/pmeQ6OJI81LUcVbWgM+JZ9t84PbOLaYvIBz4tIpslFSyMxWb9Eoajq A+9cdaiLzv52dR6RDz23fC3B6MqGcyIx96R0TYgB5Nl8osGaEVb8YMyt9zgB1woTK9NW UZDN/EJKAoZSq4XrZpNZh/bOY1vUnISnKvJrzE4UxleleztAWmqkOV+W5+HLHnnuCHYC eIbA== X-Gm-Message-State: AOAM532WJOi5ARQizMlDX9kj5T6H+vTGFwN0bSkBF4aISv+jQ4vEhQq8 pUMPOUf+G7SULspxGa4L0PiPulg4mXM= X-Google-Smtp-Source: ABdhPJzv/Po+nv+nrU+CBF5MX7Sn/035PUoNBuYS+gYk5Dil9K0JjY0ZDRyl0wjMNu6AjR9J25M0Rw== X-Received: by 2002:adf:a443:: with SMTP id e3mr8279218wra.115.1633641344747; Thu, 07 Oct 2021 14:15:44 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x17sm444456wrc.51.2021.10.07.14.15.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Oct 2021 14:15:44 -0700 (PDT) Message-Id: <330e0c0977480d0506801854fcaa6c9f2b014569.1633641339.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 07 Oct 2021 21:15:36 +0000 Subject: [PATCH v3 6/8] reset: make sparse-aware (except --mixed) Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Remove `ensure_full_index` guard on `prime_cache_tree` and update `prime_cache_tree_rec` to correctly reconstruct sparse directory entries in the cache tree. While processing a tree's entries, `prime_cache_tree_rec` must determine whether a directory entry is sparse or not by searching for it in the index (*without* expanding the index). If a matching sparse directory index entry is found, no subtrees are added to the cache tree entry and the entry count is set to 1 (representing the sparse directory itself). Otherwise, the tree is assumed to not be sparse and its subtrees are recursively added to the cache tree. Helped-by: Elijah Newren Signed-off-by: Victoria Dye --- cache-tree.c | 47 ++++++++++++++++++++++-- cache.h | 10 +++++ read-cache.c | 27 ++++++++++---- t/t1092-sparse-checkout-compatibility.sh | 15 +++++++- 4 files changed, 86 insertions(+), 13 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index 9be19c85b66..2866101052c 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -740,15 +740,26 @@ out: return ret; } +static void prime_cache_tree_sparse_dir(struct cache_tree *it, + struct tree *tree) +{ + + oidcpy(&it->oid, &tree->object.oid); + it->entry_count = 1; +} + static void prime_cache_tree_rec(struct repository *r, struct cache_tree *it, - struct tree *tree) + struct tree *tree, + struct strbuf *tree_path) { struct tree_desc desc; struct name_entry entry; int cnt; + int base_path_len = tree_path->len; oidcpy(&it->oid, &tree->object.oid); + init_tree_desc(&desc, tree->buffer, tree->size); cnt = 0; while (tree_entry(&desc, &entry)) { @@ -757,14 +768,40 @@ static void prime_cache_tree_rec(struct repository *r, else { struct cache_tree_sub *sub; struct tree *subtree = lookup_tree(r, &entry.oid); + if (!subtree->object.parsed) parse_tree(subtree); sub = cache_tree_sub(it, entry.path); sub->cache_tree = cache_tree(); - prime_cache_tree_rec(r, sub->cache_tree, subtree); + + /* + * Recursively-constructed subtree path is only needed when working + * in a sparse index (where it's used to determine whether the + * subtree is a sparse directory in the index). + */ + if (r->index->sparse_index) { + strbuf_setlen(tree_path, base_path_len); + strbuf_grow(tree_path, base_path_len + entry.pathlen + 1); + strbuf_add(tree_path, entry.path, entry.pathlen); + strbuf_addch(tree_path, '/'); + } + + /* + * If a sparse index is in use, the directory being processed may be + * sparse. To confirm that, we can check whether an entry with that + * exact name exists in the index. If it does, the created subtree + * should be sparse. Otherwise, cache tree expansion should continue + * as normal. + */ + if (r->index->sparse_index && + index_entry_exists(r->index, tree_path->buf, tree_path->len)) + prime_cache_tree_sparse_dir(sub->cache_tree, subtree); + else + prime_cache_tree_rec(r, sub->cache_tree, subtree, tree_path); cnt += sub->cache_tree->entry_count; } } + it->entry_count = cnt; } @@ -772,12 +809,14 @@ void prime_cache_tree(struct repository *r, struct index_state *istate, struct tree *tree) { + struct strbuf tree_path = STRBUF_INIT; + trace2_region_enter("cache-tree", "prime_cache_tree", the_repository); cache_tree_free(&istate->cache_tree); istate->cache_tree = cache_tree(); - ensure_full_index(istate); - prime_cache_tree_rec(r, istate->cache_tree, tree); + prime_cache_tree_rec(r, istate->cache_tree, tree, &tree_path); + strbuf_release(&tree_path); istate->cache_changed |= CACHE_TREE_CHANGED; trace2_region_leave("cache-tree", "prime_cache_tree", the_repository); } diff --git a/cache.h b/cache.h index f6295f3b048..1d3e4665562 100644 --- a/cache.h +++ b/cache.h @@ -816,6 +816,16 @@ struct cache_entry *index_file_exists(struct index_state *istate, const char *na */ int index_name_pos(struct index_state *, const char *name, int namelen); +/* + * Determines whether an entry with the given name exists within the + * given index. The return value is 1 if an exact match is found, otherwise + * it is 0. Note that, unlike index_name_pos, this function does not expand + * the index if it is sparse. If an item exists within the full index but it + * is contained within a sparse directory (and not in the sparse index), 0 is + * returned. + */ +int index_entry_exists(struct index_state *, const char *name, int namelen); + /* * Some functions return the negative complement of an insert position when a * precise match was not found but a position was found where the entry would diff --git a/read-cache.c b/read-cache.c index f5d4385c408..c079ece981a 100644 --- a/read-cache.c +++ b/read-cache.c @@ -68,6 +68,11 @@ */ #define CACHE_ENTRY_PATH_LENGTH 80 +enum index_search_mode { + NO_EXPAND_SPARSE = 0, + EXPAND_SPARSE = 1 +}; + static inline struct cache_entry *mem_pool__ce_alloc(struct mem_pool *mem_pool, size_t len) { struct cache_entry *ce; @@ -551,7 +556,10 @@ int cache_name_stage_compare(const char *name1, int len1, int stage1, const char return 0; } -static int index_name_stage_pos(struct index_state *istate, const char *name, int namelen, int stage) +static int index_name_stage_pos(struct index_state *istate, + const char *name, int namelen, + int stage, + enum index_search_mode search_mode) { int first, last; @@ -570,7 +578,7 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in first = next+1; } - if (istate->sparse_index && + if (search_mode == EXPAND_SPARSE && istate->sparse_index && first > 0) { /* Note: first <= istate->cache_nr */ struct cache_entry *ce = istate->cache[first - 1]; @@ -586,7 +594,7 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in ce_namelen(ce) < namelen && !strncmp(name, ce->name, ce_namelen(ce))) { ensure_full_index(istate); - return index_name_stage_pos(istate, name, namelen, stage); + return index_name_stage_pos(istate, name, namelen, stage, search_mode); } } @@ -595,7 +603,12 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in int index_name_pos(struct index_state *istate, const char *name, int namelen) { - return index_name_stage_pos(istate, name, namelen, 0); + return index_name_stage_pos(istate, name, namelen, 0, EXPAND_SPARSE); +} + +int index_entry_exists(struct index_state *istate, const char *name, int namelen) +{ + return index_name_stage_pos(istate, name, namelen, 0, NO_EXPAND_SPARSE) >= 0; } int remove_index_entry_at(struct index_state *istate, int pos) @@ -1222,7 +1235,7 @@ static int has_dir_name(struct index_state *istate, */ } - pos = index_name_stage_pos(istate, name, len, stage); + pos = index_name_stage_pos(istate, name, len, stage, EXPAND_SPARSE); if (pos >= 0) { /* * Found one, but not so fast. This could @@ -1322,7 +1335,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e strcmp(ce->name, istate->cache[istate->cache_nr - 1]->name) > 0) pos = index_pos_to_insert_pos(istate->cache_nr); else - pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce)); + pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), EXPAND_SPARSE); /* existing match? Just replace it. */ if (pos >= 0) { @@ -1357,7 +1370,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e if (!ok_to_replace) return error(_("'%s' appears as both a file and as a directory"), ce->name); - pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce)); + pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), EXPAND_SPARSE); pos = -pos-1; } return pos + 1; diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 875cdcb0495..4ac93874cb2 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -756,9 +756,9 @@ test_expect_success 'sparse-index is not expanded' ' ensure_not_expanded checkout - && ensure_not_expanded switch rename-out-to-out && ensure_not_expanded switch - && - git -C sparse-index reset --hard && + ensure_not_expanded reset --hard && ensure_not_expanded checkout rename-out-to-out -- deep/deeper1 && - git -C sparse-index reset --hard && + ensure_not_expanded reset --hard && ensure_not_expanded restore -s rename-out-to-out -- deep/deeper1 && echo >>sparse-index/README.md && @@ -768,6 +768,17 @@ test_expect_success 'sparse-index is not expanded' ' echo >>sparse-index/untracked.txt && ensure_not_expanded add . && + for ref in update-deep update-folder1 update-folder2 update-deep + do + echo >>sparse-index/README.md && + ensure_not_expanded reset --hard $ref || return 1 + done && + + ensure_not_expanded reset --hard update-deep && + ensure_not_expanded reset --keep base && + ensure_not_expanded reset --merge update-deep && + ensure_not_expanded reset --hard && + ensure_not_expanded checkout -f update-deep && test_config -C sparse-index pull.twohead ort && ( From patchwork Thu Oct 7 21:15:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12543435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3B42C433EF for ; Thu, 7 Oct 2021 21:15:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8470661058 for ; Thu, 7 Oct 2021 21:15:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234031AbhJGVRp (ORCPT ); Thu, 7 Oct 2021 17:17:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233882AbhJGVRl (ORCPT ); Thu, 7 Oct 2021 17:17:41 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1ADBC061570 for ; Thu, 7 Oct 2021 14:15:46 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id v25so23045029wra.2 for ; Thu, 07 Oct 2021 14:15:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ym5RYjvNvaThyBB6K1qa1gYSUUke0ke+OZpefrfVgCU=; b=a5NiIjMAHzkIhBtC4PStpUcAJFeBgncp+lIu58pv/FGNnJezXSaGD+E5mOJlgLmxKY yL6dVFINCsKM1186sax8e1U1LF1A4WDmFltiSruo2aG+QEclF7nijM10QFtxNwClwArp jvqZcUkFaJXMMTSzbkvD6wu+0lVXTHmFHJJnEKJeP/E+4ol2730/RMzNnsKREdvzCKOH baxfXjPA8xmy7COeQLrrtNQF2/ZaqNrQy2r6bchTItgtjkf5Ntla2ARGZObmmJXnWX61 jsDI/Nbm5Oy0qHOk75XEq8FgRKHgPeMT6f5IbCD6wQ5uUGf/eVDrLgj4re1rzCA0Uvb2 nvCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ym5RYjvNvaThyBB6K1qa1gYSUUke0ke+OZpefrfVgCU=; b=Aw/OUI7HlUlkua1P7nJ9Am2dS+kl9D1YeYL8Ec3NGygDzquPBBFXuQmpUC2EylHJTE 53OpCJuhEG4H0LMKn6WHecsPFQN9m6FLyEc05dehg8tmlIvA7sXeY+InlG5rWSmgn3kQ DcECoT2Wkt29AaIimqJ2GP1Y35mwTL725ThiiEszSkw47rXzj4Mc24/QXo8AQ8OTckg/ HJqtqTZMO3DL91/YHceXKX/EcchtyNc7Jv78dglLdCoLgx5zTPhrNsqotjOj7PH7+2T/ 4bzSmHUFRefM/2YQG5tiLHUq71kHD+zYPnaFJvWFHCLEUeRhG++noYGuz57BkHMCfHex Xw2g== X-Gm-Message-State: AOAM531HzyDKm5oDZc6G0n2LeO/Kf4kHBrWq786rAV5/qVFRPTpqWNLk wvFc98QPc7GG3Dn+p9L85wHdhCKIaVI= X-Google-Smtp-Source: ABdhPJypIqt2WN50HK3VCURUGaKth6UeWtVZlZ/vnVHDwnijgJo7Rje5wwz8cffZkBXQxVPSnOLzgQ== X-Received: by 2002:adf:bbd1:: with SMTP id z17mr8413762wrg.61.1633641345507; Thu, 07 Oct 2021 14:15:45 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y11sm518776wrg.18.2021.10.07.14.15.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Oct 2021 14:15:45 -0700 (PDT) Message-Id: <6ef8e4e31d39386e3a6fa1e6c24acd2c005fbd54.1633641339.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 07 Oct 2021 21:15:37 +0000 Subject: [PATCH v3 7/8] reset: make --mixed sparse-aware Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Remove the `ensure_full_index` guard on `read_from_tree` and update `git reset --mixed` to ensure it can use sparse directory index entries wherever possible. Sparse directory entries are reset use `diff_tree_oid`, which requires `change` and `add_remove` functions to process the internal contents of the sparse directory. The `recursive` diff option handles cases in which `reset --mixed` must diff/merge files that are nested multiple levels deep in a sparse directory. The use of pathspecs with `git reset --mixed` introduces scenarios in which internal contents of sparse directories may be matched by the pathspec. In order to reset *all* files in the repo that may match the pathspec, the following conditions on the pathspec require index expansion before performing the reset: * "magic" pathspecs * wildcard pathspecs that do not match only in-cone files or entire sparse directories * literal pathspecs matching something outside the sparse checkout definition Helped-by: Elijah Newren Signed-off-by: Victoria Dye --- builtin/reset.c | 78 +++++++++++++++++++++++- t/t1092-sparse-checkout-compatibility.sh | 17 ++++++ 2 files changed, 93 insertions(+), 2 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index 0ac0de7dc97..60517e7e1d6 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -148,7 +148,9 @@ static void update_index_from_diff(struct diff_queue_struct *q, * If the file 1) corresponds to an existing index entry with * skip-worktree set, or 2) does not exist in the index but is * outside the sparse checkout definition, add a skip-worktree bit - * to the new index entry. + * to the new index entry. Note that a sparse index will be expanded + * if this entry is outside the sparse cone - this is necessary + * to properly construct the reset sparse directory. */ pos = cache_name_pos(one->path, strlen(one->path)); if ((pos >= 0 && ce_skip_worktree(active_cache[pos])) || @@ -166,6 +168,73 @@ static void update_index_from_diff(struct diff_queue_struct *q, } } +static int pathspec_needs_expanded_index(const struct pathspec *pathspec) +{ + unsigned int i, pos; + int res = 0; + char *skip_worktree_seen = NULL; + + /* + * When using a magic pathspec, assume for the sake of simplicity that + * the index needs to be expanded to match all matchable files. + */ + if (pathspec->magic) + return 1; + + for (i = 0; i < pathspec->nr; i++) { + struct pathspec_item item = pathspec->items[i]; + + /* + * If the pathspec item has a wildcard, the index should be expanded + * if the pathspec has the possibility of matching a subset of entries inside + * of a sparse directory (but not the entire directory). + * + * If the pathspec item is a literal path, the index only needs to be expanded + * if a) the pathspec isn't in the sparse checkout cone (to make sure we don't + * expand for in-cone files) and b) it doesn't match any sparse directories + * (since we can reset whole sparse directories without expanding them). + */ + if (item.nowildcard_len < item.len) { + for (pos = 0; pos < active_nr; pos++) { + struct cache_entry *ce = active_cache[pos]; + + if (!S_ISSPARSEDIR(ce->ce_mode)) + continue; + + /* + * If the pre-wildcard length is longer than the sparse + * directory name and the sparse directory is the first + * component of the pathspec, need to expand the index. + */ + if (item.nowildcard_len > ce_namelen(ce) && + !strncmp(item.original, ce->name, ce_namelen(ce))) { + res = 1; + break; + } + + /* + * If the pre-wildcard length is shorter than the sparse + * directory and the pathspec does not match the whole + * directory, need to expand the index. + */ + if (!strncmp(item.original, ce->name, item.nowildcard_len) && + wildmatch(item.original, ce->name, 0)) { + res = 1; + break; + } + } + } else if (!path_in_cone_mode_sparse_checkout(item.original, &the_index) && + !matches_skip_worktree(pathspec, i, &skip_worktree_seen)) + res = 1; + + if (res > 0) + break; + } + + free(skip_worktree_seen); + return res; +} + static int read_from_tree(const struct pathspec *pathspec, struct object_id *tree_oid, int intent_to_add) @@ -178,9 +247,14 @@ static int read_from_tree(const struct pathspec *pathspec, opt.format_callback = update_index_from_diff; opt.format_callback_data = &intent_to_add; opt.flags.override_submodule_config = 1; + opt.flags.recursive = 1; opt.repo = the_repository; + opt.change = diff_change; + opt.add_remove = diff_addremove; + + if (pathspec->nr && the_index.sparse_index && pathspec_needs_expanded_index(pathspec)) + ensure_full_index(&the_index); - ensure_full_index(&the_index); if (do_diff_cache(tree_oid, &opt)) return 1; diffcore_std(&opt); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 4ac93874cb2..c9343ff5b9c 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -774,11 +774,28 @@ test_expect_success 'sparse-index is not expanded' ' ensure_not_expanded reset --hard $ref || return 1 done && + ensure_not_expanded reset --mixed base && ensure_not_expanded reset --hard update-deep && ensure_not_expanded reset --keep base && ensure_not_expanded reset --merge update-deep && ensure_not_expanded reset --hard && + ensure_not_expanded reset base -- deep/a && + ensure_not_expanded reset base -- nonexistent-file && + ensure_not_expanded reset deepest -- deep && + + # Although folder1 is outside the sparse definition, it exists as a + # directory entry in the index, so the pathspec will not force the + # index to be expanded. + ensure_not_expanded reset deepest -- folder1 && + ensure_not_expanded reset deepest -- folder1/ && + + # Wildcard identifies only in-cone files, no index expansion + ensure_not_expanded reset deepest -- deep/\* && + + # Wildcard identifies only full sparse directories, no index expansion + ensure_not_expanded reset deepest -- folder\* && + ensure_not_expanded checkout -f update-deep && test_config -C sparse-index pull.twohead ort && ( From patchwork Thu Oct 7 21:15:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12543437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 121DDC433F5 for ; Thu, 7 Oct 2021 21:15:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EAD9361058 for ; Thu, 7 Oct 2021 21:15:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235284AbhJGVRr (ORCPT ); Thu, 7 Oct 2021 17:17:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233929AbhJGVRl (ORCPT ); Thu, 7 Oct 2021 17:17:41 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B61D9C061764 for ; Thu, 7 Oct 2021 14:15:47 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id v25so23045141wra.2 for ; Thu, 07 Oct 2021 14:15:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=nEih7G58FeXZR7VfCiUcLv0bmQevEM54qXiHHW/Vpvo=; b=ozBHfKP3Xxaby73C2wHvd6XMm7+bzgzsfxgR1swh7E2WoSodQBjaM/U2WtnWLdcShV sqDwB/AAVpE4TmIlzeH++/bUiB39AI+WgG9irsLZdn5mIN3rgwFcaDAJL0TV7ZMQIo5P F+25ydHNJn3wZKKQB2geXPKG2j4viUrEXECEnZDOISLo9KBzt/QbetfSYJc4EqqZVkCm pDT96iuEU2UOx04YkxmSV8sJZFyT3p4GDCSuchQWj3JAqhPqE5Ph5iZVsRPQHH6DTzEi ZZHjvydUmS53prZTqdMpB5zOPFR04SQ5ILxeJZa8X68Qdkn/5JOvj1OnoahGGV0upmXn pRXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=nEih7G58FeXZR7VfCiUcLv0bmQevEM54qXiHHW/Vpvo=; b=IAY5N9bbhQECm8i+xduUbDNO/Hgkfw5xa66v9QWAAdhAXvsaWWDj+kpOn4ZVPH4CdB T7KSx/vyOLEoSVntkY39L8UWee7kY7xeKKz23KfhP/0ioC6Jg8wAhJF910fBQq9GfKQs Ws673ll5hhy1CGx7Tmawq93JFA6HVpP7edWRQmhhFRXcQAOyOQJPaKl8aw9J44ywxVsx 32ohyl1KnF0/Vg788M+E6LdakBr7P/eMfkd7nI/+WVQMFP6TZmw4JpjCJAlxMAKfeUtS Bp8DZ2RNTdCsMmgblMIqTI09K5QiRHrfs5bV8wqPa72WXhNmMIjWxic723z0nHRie/k7 4OEg== X-Gm-Message-State: AOAM533a8KZbPIIDCEE4RQQpDYqi4u3W6JfwBTA89kbzsiOqtkGBNc34 5bvKlhkFTZJAt35UBV9gEvKzl9Q9YYU= X-Google-Smtp-Source: ABdhPJxvizPGvkd9Wh9g7rN12oAy8t4MjBjwIDs0EyXa6yKXXGXp3LBe9GSykBmHAOmJx+q/gxOq5A== X-Received: by 2002:a1c:7e04:: with SMTP id z4mr19054227wmc.95.1633641346204; Thu, 07 Oct 2021 14:15:46 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y6sm450462wro.63.2021.10.07.14.15.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Oct 2021 14:15:45 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 07 Oct 2021 21:15:38 +0000 Subject: [PATCH v3 8/8] unpack-trees: improve performance of next_cache_entry Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye To find the first non-unpacked cache entry, `next_cache_entry` iterates through index, starting at `cache_bottom`. The performance of this in full indexes is helped by `cache_bottom` advancing with each invocation of `mark_ce_used` (called by `unpack_index_entry`). However, the presence of sparse directories can prevent the `cache_bottom` from advancing in a sparse index case, effectively forcing `next_cache_entry` to search from the beginning of the index each time it is called. The `cache_bottom` must be preserved for the sparse index (see 17a1bb570b (unpack-trees: preserve cache_bottom, 2021-07-14)). Therefore, to retain the benefit `cache_bottom` provides in non-sparse index cases, a separate `hint` position indicates the first position `next_cache_entry` should search, updated each execution with a new position. Signed-off-by: Victoria Dye --- unpack-trees.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/unpack-trees.c b/unpack-trees.c index 8ea0a542da8..b94733de6be 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -645,17 +645,24 @@ static void mark_ce_used_same_name(struct cache_entry *ce, } } -static struct cache_entry *next_cache_entry(struct unpack_trees_options *o) +static struct cache_entry *next_cache_entry(struct unpack_trees_options *o, int *hint) { const struct index_state *index = o->src_index; int pos = o->cache_bottom; + if (*hint > pos) + pos = *hint; + while (pos < index->cache_nr) { struct cache_entry *ce = index->cache[pos]; - if (!(ce->ce_flags & CE_UNPACKED)) + if (!(ce->ce_flags & CE_UNPACKED)) { + *hint = pos + 1; return ce; + } pos++; } + + *hint = pos; return NULL; } @@ -1365,12 +1372,13 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str /* Are we supposed to look at the index too? */ if (o->merge) { + int hint = -1; while (1) { int cmp; struct cache_entry *ce; if (o->diff_index_cached) - ce = next_cache_entry(o); + ce = next_cache_entry(o, &hint); else ce = find_cache_entry(info, p); @@ -1690,7 +1698,7 @@ static int verify_absent(const struct cache_entry *, int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options *o) { struct repository *repo = the_repository; - int i, ret; + int i, hint, ret; static struct cache_entry *dfc; struct pattern_list pl; int free_pattern_list = 0; @@ -1763,13 +1771,15 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options info.pathspec = o->pathspec; if (o->prefix) { + hint = -1; + /* * Unpack existing index entries that sort before the * prefix the tree is spliced into. Note that o->merge * is always true in this case. */ while (1) { - struct cache_entry *ce = next_cache_entry(o); + struct cache_entry *ce = next_cache_entry(o, &hint); if (!ce) break; if (ce_in_traverse_path(ce, &info)) @@ -1790,8 +1800,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options /* Any left-over entries in the index? */ if (o->merge) { + hint = -1; while (1) { - struct cache_entry *ce = next_cache_entry(o); + struct cache_entry *ce = next_cache_entry(o, &hint); if (!ce) break; if (unpack_index_entry(ce, o) < 0)