From patchwork Wed Oct 27 14:39:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12587469 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B667C4332F for ; Wed, 27 Oct 2021 14:39:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E564660720 for ; Wed, 27 Oct 2021 14:39:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238869AbhJ0Olx (ORCPT ); Wed, 27 Oct 2021 10:41:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46430 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237806AbhJ0Olv (ORCPT ); Wed, 27 Oct 2021 10:41:51 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 720FBC061745 for ; Wed, 27 Oct 2021 07:39:26 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id 192so1307062wme.3 for ; Wed, 27 Oct 2021 07:39:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ldyVwS09nc2OLJia/8NlZArMqqWxVI8EkShpKdO52GA=; b=WE11Y4Xm89Drt5ObByCAKBzNVGwg2Kndfg1InPxOHLbhm8+V5FygVGNLwIGU4/Wp04 b40rv0izcjvnRALsE6oM/9c/MKvKJUMYJDhYVpG+hgVOrU+vkjx2umBatC1mrS4nECAu T+pwYjquHgMW2vlcBR2D2mCl31CcGgM9egqRCWrzYyYzlOzPe8b83ZBg66uiUvZ+zFgJ gQDPxZdOersp20VVEnGV1Ukhz9jUxrmvkmmr59wBdYMm6uhQZzb529I8BDfBRtiWxKTz hc+LpAfBL2d2mRFWSMdVM8Uw7cK+u3SMsq7T+wIK2bBU/+PY6R4DZ968jxVayq9w/WCh oekg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ldyVwS09nc2OLJia/8NlZArMqqWxVI8EkShpKdO52GA=; b=uR8+SV1vGt+otMKNQDyWUniPEPw2C59cT5YIvLCeBNXqaOKOsU4+55lXcLIo31ZHmG envYg45vC8FCjj1BtHM1RfkE+CZ3XdeURPc907Zt7B3V5xZOFSUayCcsS6PibuuGGDfh d5l3KW5TdgBBjBI3weQeUdZ7ulrXCfG4DyxGAiERu1ZlQPAPbEh2lL709PLL4sk+4920 cojFZb43GWOlCH7JXwK/9qLQgzSs9I2aMnbtG9M0fLqD8PVJ3ea2s8vqZMfp7wIs2m9A ee0FfpOTNUsZ1Y+fUDY4DpRk2BCFfIOGCY42U2lHiwiFY5MWHvwUfVwcWtwvr5at64X/ XM3w== X-Gm-Message-State: AOAM5310Y2YTaUKkH9E+/sUwvuqiqI+3JXzKqWkWd5tY6f9GCAU/Ok+3 c3LxV3GjG7SFFcJrS8a8YmxJaYmRLII= X-Google-Smtp-Source: ABdhPJxx8aqVKvIrPJKrJG15DnERYYD8QgNRF5r/JNX/wY52L/hJRqUWhl2uEzj9GgosGrcbaIAQXg== X-Received: by 2002:a1c:2507:: with SMTP id l7mr6009571wml.186.1635345565031; Wed, 27 Oct 2021 07:39:25 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t6sm45249wrw.78.2021.10.27.07.39.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Oct 2021 07:39:24 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 27 Oct 2021 14:39:16 +0000 Subject: [PATCH v5 1/8] reset: rename is_missing to !is_in_reset_tree Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Rename and invert value of `is_missing` to `is_in_reset_tree` to make the variable more descriptive of what it represents. Signed-off-by: Victoria Dye --- builtin/reset.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index 51c9e2f43ff..d3695ce43c4 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -131,10 +131,10 @@ static void update_index_from_diff(struct diff_queue_struct *q, for (i = 0; i < q->nr; i++) { struct diff_filespec *one = q->queue[i]->one; - int is_missing = !(one->mode && !is_null_oid(&one->oid)); + int is_in_reset_tree = one->mode && !is_null_oid(&one->oid); struct cache_entry *ce; - if (is_missing && !intent_to_add) { + if (!is_in_reset_tree && !intent_to_add) { remove_file_from_cache(one->path); continue; } @@ -144,7 +144,7 @@ static void update_index_from_diff(struct diff_queue_struct *q, if (!ce) die(_("make_cache_entry failed for path '%s'"), one->path); - if (is_missing) { + if (!is_in_reset_tree) { ce->ce_flags |= CE_INTENT_TO_ADD; set_object_name_for_intent_to_add_entry(ce); } From patchwork Wed Oct 27 14:39:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12587471 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9449EC433F5 for ; Wed, 27 Oct 2021 14:39:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7F94B60F6F for ; Wed, 27 Oct 2021 14:39:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238956AbhJ0Oly (ORCPT ); Wed, 27 Oct 2021 10:41:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231462AbhJ0Olw (ORCPT ); Wed, 27 Oct 2021 10:41:52 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E51AC061570 for ; Wed, 27 Oct 2021 07:39:27 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id e4so4557258wrc.7 for ; Wed, 27 Oct 2021 07:39:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=BOiihwKfyp8h1ZPOwc/Ncb03jmtRWNTQQAMmoxR9+V4=; b=A0eN2o0INh1UKElEyN9dtMOED4Dg0fw2opXpVRWZRnwHS3kR/VzkiWiKqsn66qKlxw wtTYmEe+ELgbKyMJHz3c4k2SgW5ohib33opwwEXSxjnyLl0pMXJzOw82oev6JKzjLTtY gcwey34fZ0ldvewMnUfUOb8jYJ6ouTcSC7F6VRPAoj2+FYp/5dLppq+Siglxl9umsrUz cWqJ8RcDXmYBpInUv7DZNd6q9lm3V775agXNwzR7TyImWJhDHB3Q3mARthjoC/0+FOkv 2VkcQvDxVtEzwol32W1Imfquxlpa6IsDVILmLgcXIkaHcnVz8p1nmpkmtk4V9kmKuAlB ijDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=BOiihwKfyp8h1ZPOwc/Ncb03jmtRWNTQQAMmoxR9+V4=; b=QrPN+btworVwZnK/FzlFoMd0JrNDteQSIzpD1xmxyGI9bQ7O0L0iOZuvxqoTet6Z65 UR3Xqro1CBK9RoYXiVzE9Ya16olNvg35jJ1OT0RoYyMVmhQsDU785zMRHHTSNfWVkQM7 y126SRN8Mx+To1qZvIcM/DaS2VHjNeA1+/YzRqR7MNcZ4bIj9iCXUDurDUcuvIeQnj6n Yvt8iYIrZVraxvvnMGcmqLZ47AUq9uJC/lk5cOmb1SG59Wi7sSOGb5CG0b/A667X/CuH 3B0d3W0yrqk8ep9Xb95CdX0IoXXh8ZQFQLV62JIsLFp3eSF0BaulQt9JAHAovEXUGIeM gVKg== X-Gm-Message-State: AOAM530iprBviM5/dNi4BErxsLHzYRF7qhEAMbU31irB1PWLeVSkngnQ KJDXXSADrp+5bcEvByCLYjFTYwFBliY= X-Google-Smtp-Source: ABdhPJw4V/8MTlvMn6ltsSefOSFvduBUyoNOahMf/qSjwBLj5LSjau5OgSIkfTy6eKYCPtBCVVVHiQ== X-Received: by 2002:a5d:528b:: with SMTP id c11mr40731845wrv.35.1635345565592; Wed, 27 Oct 2021 07:39:25 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g2sm76461wrb.20.2021.10.27.07.39.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Oct 2021 07:39:25 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 27 Oct 2021 14:39:17 +0000 Subject: [PATCH v5 2/8] reset: preserve skip-worktree bit in mixed reset Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Change `update_index_from_diff` to set `skip-worktree` when applicable for new index entries. When `git reset --mixed ` is run, entries in the index with differences between the pre-reset HEAD and reset are identified and handled with `update_index_from_diff`. For each file, a new cache entry in inserted into the index, created from the side of the reset (without changing the working tree). However, the newly-created entry must have `skip-worktree` explicitly set in either of the following scenarios: 1. the file is in the current index and has `skip-worktree` set 2. the file is not in the current index but is outside of a defined sparse checkout definition Not setting the `skip-worktree` bit leads to likely-undesirable results for a user. It causes `skip-worktree` settings to disappear on the "diff"-containing files (but *only* the diff-containing files), leading to those files now showing modifications in `git status`. For example, when running `git reset --mixed` in a sparse checkout, some file entries outside of sparse checkout could show up as deleted, despite the user never deleting anything (and not wanting them on-disk anyway). Additionally, add a test to `t7102` to ensure `skip-worktree` is preserved in a basic `git reset --mixed` scenario and update a failure-documenting test from 19a0acc (t1092: test interesting sparse-checkout scenarios, 2021-01-23) with new expected behavior. Helped-by: Junio C Hamano Signed-off-by: Victoria Dye --- builtin/reset.c | 14 ++++++++++++++ t/t1092-sparse-checkout-compatibility.sh | 22 ++++++++-------------- t/t7102-reset.sh | 17 +++++++++++++++++ 3 files changed, 39 insertions(+), 14 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index d3695ce43c4..e441b6601b9 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -25,6 +25,7 @@ #include "cache-tree.h" #include "submodule.h" #include "submodule-config.h" +#include "dir.h" #define REFRESH_INDEX_DELAY_WARNING_IN_MS (2 * 1000) @@ -130,6 +131,7 @@ static void update_index_from_diff(struct diff_queue_struct *q, int intent_to_add = *(int *)data; for (i = 0; i < q->nr; i++) { + int pos; struct diff_filespec *one = q->queue[i]->one; int is_in_reset_tree = one->mode && !is_null_oid(&one->oid); struct cache_entry *ce; @@ -141,6 +143,18 @@ static void update_index_from_diff(struct diff_queue_struct *q, ce = make_cache_entry(&the_index, one->mode, &one->oid, one->path, 0, 0); + + /* + * If the file 1) corresponds to an existing index entry with + * skip-worktree set, or 2) does not exist in the index but is + * outside the sparse checkout definition, add a skip-worktree bit + * to the new index entry. + */ + pos = cache_name_pos(one->path, strlen(one->path)); + if ((pos >= 0 && ce_skip_worktree(active_cache[pos])) || + (pos < 0 && !path_in_sparse_checkout(one->path, &the_index))) + ce->ce_flags |= CE_SKIP_WORKTREE; + if (!ce) die(_("make_cache_entry failed for path '%s'"), one->path); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 886e78715fe..c7449afe965 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -459,26 +459,20 @@ test_expect_failure 'blame with pathspec outside sparse definition' ' test_all_match git blame deep/deeper2/deepest/a ' -# NEEDSWORK: a sparse-checkout behaves differently from a full checkout -# in this scenario, but it shouldn't. -test_expect_failure 'checkout and reset (mixed)' ' +test_expect_success 'checkout and reset (mixed)' ' init_repos && test_all_match git checkout -b reset-test update-deep && test_all_match git reset deepest && - test_all_match git reset update-folder1 && - test_all_match git reset update-folder2 -' - -# NEEDSWORK: a sparse-checkout behaves differently from a full checkout -# in this scenario, but it shouldn't. -test_expect_success 'checkout and reset (mixed) [sparse]' ' - init_repos && - test_sparse_match git checkout -b reset-test update-deep && - test_sparse_match git reset deepest && + # Because skip-worktree is preserved, resetting to update-folder1 + # will show worktree changes for folder1/a in full-checkout, but not + # in sparse-checkout or sparse-index. + git -C full-checkout reset update-folder1 >full-checkout-out && test_sparse_match git reset update-folder1 && - test_sparse_match git reset update-folder2 + grep "M folder1/a" full-checkout-out && + ! grep "M folder1/a" sparse-checkout-out && + run_on_sparse test_path_is_missing folder1 ' test_expect_success 'merge, cherry-pick, and rebase' ' diff --git a/t/t7102-reset.sh b/t/t7102-reset.sh index 601b2bf97f0..d05426062ec 100755 --- a/t/t7102-reset.sh +++ b/t/t7102-reset.sh @@ -472,6 +472,23 @@ test_expect_success '--mixed refreshes the index' ' test_cmp expect output ' +test_expect_success '--mixed preserves skip-worktree' ' + echo 123 >>file2 && + git add file2 && + git update-index --skip-worktree file2 && + git reset --mixed HEAD >output && + test_must_be_empty output && + + cat >expect <<-\EOF && + Unstaged changes after reset: + M file2 + EOF + git update-index --no-skip-worktree file2 && + git add file2 && + git reset --mixed HEAD >output && + test_cmp expect output +' + test_expect_success 'resetting specific path that is unmerged' ' git rm --cached file2 && F1=$(git rev-parse HEAD:file1) && From patchwork Wed Oct 27 14:39:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12587473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 893D9C433F5 for ; Wed, 27 Oct 2021 14:39:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 70A8060F56 for ; Wed, 27 Oct 2021 14:39:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242625AbhJ0Ol6 (ORCPT ); Wed, 27 Oct 2021 10:41:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237877AbhJ0Olx (ORCPT ); Wed, 27 Oct 2021 10:41:53 -0400 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A29B9C061570 for ; Wed, 27 Oct 2021 07:39:27 -0700 (PDT) Received: by mail-wm1-x333.google.com with SMTP id 82-20020a1c0055000000b0032ccc3ad5c1so4537173wma.2 for ; Wed, 27 Oct 2021 07:39:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=rWdQm5Zkkxjg4tRvPMLEws5NQlPaszk/3jdQlcrd+H8=; b=dVkfkgw/Jx/ggUgfL1ZNQxbJGATyN4s+juJ87G7jLKnTecpxOYsG3zV2cYvb1sPbc5 cZet03C6wgl26ZIwVSWySvZBHm+OaaJYzW6uGqR8oW+25eQ78SuwJvy3oDIMBih3cYbu mkJbDyzG6XwcI5uyizNJcJ+PSfnh8BNkrgJwvMIP0QnjDx2xwyUcvBh14IN5K6D4XwQY 8gF0imq4kp1/bOS6SnHf5n/BxFXOd+cPFImJeHeq54GhqHGHvi1CGgH1yuz2zQzZF+Xw tKhtdxU3Jrsl14GOnieOFkvVndF6brUoKgE5TDm22ZzEwp2MyZQMpjQ8hRKTi35EQp58 fXVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=rWdQm5Zkkxjg4tRvPMLEws5NQlPaszk/3jdQlcrd+H8=; b=lOiCiGhcmJMH/QBOIj92op+lXlc0z9263Xm+sQfzwpnxRm5HtgzoH2AFeiQyYzDlTb ZgF7WcwmS+3eDLxXIw3f1qpC+69G1nUawzCoIGjwz638GQoORrX17irZ216TVcIGfGTj g55w5lFI+9+u9+OozNZc3/Yq3uNIEMwQf+4aw/B/++N5fTk7hyCRZl19ZjN1kZpn/xku 7d9vP2Nmu825L53wuklgW6H78ppogb2d98q4SHMvcw5LYfVW+MLEKaJx/qATudq2t41M kkr74orfUyt5NUXblaDJlpV6v7pCSRR/F8c9iexk+/dnYD6TfAQwbrhm6Vxerddqe85Y ir/w== X-Gm-Message-State: AOAM531vLlyw0jxNSILGDhJ8nYLobzGjHsUWgJI4KZPq5+kU1KX8Jayv eAbjxRMKsD4PGVHif8TUjW0NOoyqAPA= X-Google-Smtp-Source: ABdhPJyeHD9by2pBG7iVEBnMPmEeoy3Bl0ym9mxKg9Slv0LB1CfhSebzGtFqVGBb69wipOdTCubcPQ== X-Received: by 2002:a05:600c:19cd:: with SMTP id u13mr6030571wmq.148.1635345566252; Wed, 27 Oct 2021 07:39:26 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k10sm3637873wmr.32.2021.10.27.07.39.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Oct 2021 07:39:25 -0700 (PDT) Message-Id: <1bb2ca92c60be453d7b151e0a1fb5d471a2e5090.1635345563.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 27 Oct 2021 14:39:18 +0000 Subject: [PATCH v5 3/8] sparse-index: update command for expand/collapse test Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye In anticipation of `git reset --hard` being able to use the sparse index without expanding it, replace the command in `sparse-index is expanded and converted back` with `git reset -- folder1/a`. This command will need to expand the index to work properly, even after integrating the rest of `reset` with sparse index. Helped-by: Derrick Stolee Signed-off-by: Victoria Dye --- t/t1092-sparse-checkout-compatibility.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index c7449afe965..cab6340a9d0 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -634,11 +634,15 @@ test_expect_success 'submodule handling' ' grep "160000 commit $(git -C initial-repo rev-parse HEAD) modules/sub" cache ' +# When working with a sparse index, some commands will need to expand the +# index to operate properly. If those commands also write the index back +# to disk, they need to convert the index to sparse before writing. +# This test verifies that both of these events are logged in trace2 logs. test_expect_success 'sparse-index is expanded and converted back' ' init_repos && GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ - git -C sparse-index -c core.fsmonitor="" reset --hard && + git -C sparse-index reset -- folder1/a && test_region index convert_to_sparse trace2.txt && test_region index ensure_full_index trace2.txt ' From patchwork Wed Oct 27 14:39:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12587475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDDF9C433EF for ; Wed, 27 Oct 2021 14:39:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A6C2460720 for ; Wed, 27 Oct 2021 14:39:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242628AbhJ0OmA (ORCPT ); Wed, 27 Oct 2021 10:42:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46446 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238902AbhJ0Oly (ORCPT ); Wed, 27 Oct 2021 10:41:54 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C64AC061570 for ; Wed, 27 Oct 2021 07:39:28 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id u18so4577976wrg.5 for ; Wed, 27 Oct 2021 07:39:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=kb73tIJKEniDWqTeYxI92yj2S3EyQHnwcWkfm6tEo94=; b=nyt4U4dJ8eXw4UwPi3pgct9lwrAbgWnhnywKM9to+WJavlFa52j7UjwrYJW0dMj9yY MgVz4/uP/VYvZL+L4CQxmCxk3VTxjXKIJCVFomcZrrftG2Qz0MFzWm/xYhom2QWhYkPn kjGSv+JUgo8bl8bFoUmLyfkW4h6eux3BtCltuJVAeyrv4Hcjuu3Q5IfvU5rU/hjR7mTu bdmagkROej0nyjzfK1bfnifqJcnlh37w7V3HnDANnIIM3e066T7MZCseQbFV0DThnmyd DjxVUe+ACc4FSoYZ9ZzWQKPsgfkaPaqjYfIbAAageQs1g0QXdJSaTqAt+ZfFwsfXbaZB 2PkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=kb73tIJKEniDWqTeYxI92yj2S3EyQHnwcWkfm6tEo94=; b=JH71OaSZkR566RuvB5qoRysi92cGh1vr+UEXu0Mf8Cvh0UfDw4JnFF7t3syQ7LLxJW 4zAC2EKyIdhDbVHWRQNwSYieQ3kYBbRtwdD+dBG8d9UzNq9GrqRtDeWUVNRk6y7m+Jhg mMcE7E/JL1aCD/xI3xr7pzNqLJCa11ZWjzBnANbtVFO+dZt+m0zS42yh7jpCxvb912P8 OzAOOoUjP6DD2tQJ2bpGeKAX/srisUJEeRPxpF2/cZMJ02m4URxDBbSRXUIDFhwMHMNo EdVGgXaND1G2ceNru3tmwCuDp8sZTabPBPfe0i/RwLe5BztNz/B1yqAufO6fu4uV2u8V rTqQ== X-Gm-Message-State: AOAM531yrb1PowK8SVBb7jWN+vJwGyZpHOnYB9FT0ZLy5DaguujD3oXa 6xcgs5ZYOPWiqafq/sm8uJZh/G2nARs= X-Google-Smtp-Source: ABdhPJye7g4g0PtC6vcVTpGhEIWpD81q5OAVDQS9zTezgahRo7RAbFNU8Kw4V0YWXTVqnE/8jIZfKw== X-Received: by 2002:a5d:5092:: with SMTP id a18mr6025685wrt.421.1635345567013; Wed, 27 Oct 2021 07:39:27 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d8sm49351wrv.80.2021.10.27.07.39.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Oct 2021 07:39:26 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 27 Oct 2021 14:39:19 +0000 Subject: [PATCH v5 4/8] reset: expand test coverage for sparse checkouts Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Add new tests for `--merge` and `--keep` modes, as well as mixed reset with pathspecs. New performance test cases exercise various execution paths for `reset`. Co-authored-by: Derrick Stolee Signed-off-by: Derrick Stolee Signed-off-by: Victoria Dye --- t/perf/p2000-sparse-operations.sh | 3 + t/t1092-sparse-checkout-compatibility.sh | 84 ++++++++++++++++++++++++ 2 files changed, 87 insertions(+) diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index 597626276fb..bfd332120c8 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -110,5 +110,8 @@ test_perf_on_all git add -A test_perf_on_all git add . test_perf_on_all git commit -a -m A test_perf_on_all git checkout -f - +test_perf_on_all git reset +test_perf_on_all git reset --hard +test_perf_on_all git reset -- does-not-exist test_done diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index cab6340a9d0..a8583030b38 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -475,6 +475,90 @@ test_expect_success 'checkout and reset (mixed)' ' run_on_sparse test_path_is_missing folder1 ' +test_expect_success 'checkout and reset (merge)' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents a && + test_all_match git reset --merge deepest && + test_all_match git status --porcelain=v2 && + + test_all_match git reset --hard update-deep && + run_on_all ../edit-contents deep/a && + test_all_match test_must_fail git reset --merge deepest +' + +test_expect_success 'checkout and reset (keep)' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents a && + test_all_match git reset --keep deepest && + test_all_match git status --porcelain=v2 && + + test_all_match git reset --hard update-deep && + run_on_all ../edit-contents deep/a && + test_all_match test_must_fail git reset --keep deepest +' + +test_expect_success 'reset with pathspecs inside sparse definition' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents deep/a && + + test_all_match git reset base -- deep/a && + test_all_match git status --porcelain=v2 && + + test_all_match git reset base -- nonexistent-file && + test_all_match git status --porcelain=v2 && + + test_all_match git reset deepest -- deep && + test_all_match git status --porcelain=v2 +' + +# Although the working tree differs between full and sparse checkouts after +# reset, the state of the index is the same. +test_expect_success 'reset with pathspecs outside sparse definition' ' + init_repos && + test_all_match git checkout -b reset-test base && + + test_sparse_match git reset update-folder1 -- folder1 && + git -C full-checkout reset update-folder1 -- folder1 && + test_sparse_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder1 && + + test_sparse_match git reset update-folder2 -- folder2/a && + git -C full-checkout reset update-folder2 -- folder2/a && + test_sparse_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder2/a +' + +test_expect_success 'reset with wildcard pathspec' ' + init_repos && + + test_all_match git checkout -b reset-test update-deep && + test_all_match git reset base -- \*/a && + test_all_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder1/a && + + test_all_match git reset base -- folder\* && + test_all_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD:folder2 +' + test_expect_success 'merge, cherry-pick, and rebase' ' init_repos && From patchwork Wed Oct 27 14:39:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12587477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 506CCC433EF for ; Wed, 27 Oct 2021 14:39:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 36BC460F56 for ; Wed, 27 Oct 2021 14:39:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242637AbhJ0OmB (ORCPT ); Wed, 27 Oct 2021 10:42:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238962AbhJ0Olz (ORCPT ); Wed, 27 Oct 2021 10:41:55 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14BB8C061767 for ; Wed, 27 Oct 2021 07:39:29 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id y205-20020a1c7dd6000000b0032cc8c2800fso5582869wmc.4 for ; Wed, 27 Oct 2021 07:39:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+dch2aTLAv85TXXA5lgJ4m8qNH4dps5WqL8q9u/O5kk=; b=bII53/sIC1NfTxjRcrbZJCY9Z2EUfYZpA4UyFtuBmOTn/tFHLtHEF437rmdKG/QrQ0 fxPJH8Dg/9p0uc8onsLJFZCGMTbTliGwmsQsIHr2XLEveU+VxuRxCxTT7+BB8H68TVl5 r/lS6wubgw4sU2yekqr+XBIVrsbi2RWaXWpMtw84kFeeGNd539ME+6xBsrCpmqQTQwyL pWD1OMg/DP6KLMiFdDMdYHMPSl1839nhnqDJLitjGmqafNY338BXHfwDa2RCXVDawEEk k053FodA6GQj1WMeP6oXZDtC3OYencTexfMUb2qjpatH7c5540gbqWmYoqlePVVFMdJG a6aA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+dch2aTLAv85TXXA5lgJ4m8qNH4dps5WqL8q9u/O5kk=; b=FnkzsOeJ5puwM3h3iSCSyc+W9ErNsRJ102KlFp/EtRvawYMdCkJsHcHo3X9/iG+Cum URdST+9PPmxDRerMKTfGW7uA8q1955R5+zadqjXnKo+f10FvghvrspsEyhVQNfVMvAcp 0Jrs2+JPo2xzGZx5Ye2e7n9sYXNqzFBwRyqIqeo2cLhpqp0GpzQ/WhoknBSi6RjGa8dR R4HvzLngUI45bdnAYmHGqcMNyFJvQWZ3mJyNdQ+73qaM3xL/F7847Gc0jJ3wn/qemRX1 NXmA+cN+oQ1NjvwUp45hSpHGDa8jYS+9ei+1rqnljYbiy/q9xe8viYG6BCMjPitAsm48 ISMQ== X-Gm-Message-State: AOAM532JdASlEJS63gYb9sE09Ud639SMo7UvarkWOeIOXyXi8DmD+r08 YdjyqJ5aTYKjXkZ4A280VxDMKTwGjj0= X-Google-Smtp-Source: ABdhPJwGDuazfhjwHma73IE6kKyPfisW7FuWVnRZJvvsf7b7qaN/lJUvz5WshWqY2d5G5BHZziLmUg== X-Received: by 2002:a7b:cc8c:: with SMTP id p12mr2477701wma.158.1635345567599; Wed, 27 Oct 2021 07:39:27 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o2sm85031wrg.1.2021.10.27.07.39.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Oct 2021 07:39:27 -0700 (PDT) Message-Id: <217ae44541878b712b45db9525a6de5b854f27bb.1635345563.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 27 Oct 2021 14:39:20 +0000 Subject: [PATCH v5 5/8] reset: integrate with sparse index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Disable `command_requires_full_index` repo setting and add `ensure_full_index` guards around code paths that cannot yet use sparse directory index entries. `reset --soft` does not modify the index, so no compatibility changes are needed for it to function without expanding the index. For all other reset modes (`--mixed`, `--hard`, `--keep`, `--merge`), the full index is expanded to prevent cache tree corruption and invalid variable accesses. Additionally, the `read_cache()` check verifying an uncorrupted index is moved after argument parsing and preparing the repo settings. The index is not used by the preceding argument handling, but `read_cache()` must be run *after* enabling sparse index for the command (so that the index is not expanded unnecessarily) and *before* using the index for reset (so that it is verified as uncorrupted). Signed-off-by: Victoria Dye --- builtin/reset.c | 10 +++++++--- cache-tree.c | 1 + 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index e441b6601b9..0ac0de7dc97 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -180,6 +180,7 @@ static int read_from_tree(const struct pathspec *pathspec, opt.flags.override_submodule_config = 1; opt.repo = the_repository; + ensure_full_index(&the_index); if (do_diff_cache(tree_oid, &opt)) return 1; diffcore_std(&opt); @@ -257,9 +258,6 @@ static void parse_args(struct pathspec *pathspec, } *rev_ret = rev; - if (read_cache() < 0) - die(_("index file corrupt")); - parse_pathspec(pathspec, 0, PATHSPEC_PREFER_FULL | (patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0), @@ -405,6 +403,12 @@ int cmd_reset(int argc, const char **argv, const char *prefix) if (intent_to_add && reset_type != MIXED) die(_("-N can only be used with --mixed")); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + + if (read_cache() < 0) + die(_("index file corrupt")); + /* Soft reset does not touch the index file nor the working tree * at all, but requires them in a good order. Other resets reset * the index file to the tree object we are switching to. */ diff --git a/cache-tree.c b/cache-tree.c index 90919f9e345..9be19c85b66 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -776,6 +776,7 @@ void prime_cache_tree(struct repository *r, cache_tree_free(&istate->cache_tree); istate->cache_tree = cache_tree(); + ensure_full_index(istate); prime_cache_tree_rec(r, istate->cache_tree, tree); istate->cache_changed |= CACHE_TREE_CHANGED; trace2_region_leave("cache-tree", "prime_cache_tree", the_repository); From patchwork Wed Oct 27 14:39:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12587479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15A43C433FE for ; Wed, 27 Oct 2021 14:39:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 00F7B60F6F for ; Wed, 27 Oct 2021 14:39:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242616AbhJ0OmD (ORCPT ); Wed, 27 Oct 2021 10:42:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231462AbhJ0Olz (ORCPT ); Wed, 27 Oct 2021 10:41:55 -0400 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B33E2C0613B9 for ; Wed, 27 Oct 2021 07:39:29 -0700 (PDT) Received: by mail-wm1-x333.google.com with SMTP id j2-20020a1c2302000000b0032ca9b0a057so2541860wmj.3 for ; Wed, 27 Oct 2021 07:39:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=uAzG6uoZEE9KFkpoAiZpv05CNx239mJ72+2x7m8hZ0w=; b=fCwjW0CfHbvlWpt9kjlqUCR5wNdstVs1rtgj+eMW4OJjlXFPUiZtOChvanAAh4VSvD uZenZdZ+Tani2X8LEogBffjqjfAo2YXtdtNeck1Pgsf/q4piMwzPqRMYK4gkQ090WXq7 iExlvyaU77B2YQmnbA8VROn+O17QXosVBGBKyNwAHcPjRJkGTbav1raPnnV/jlZTejD1 GAckhUMwSly9D+SFoWRDBOA/Wx/2UEMJ/pch8F3JuIeERTLr9IvOPPPEOBgV0wyjvZgn 1BpZadFUtypcW+2R4NtPC6khmPHPC6KWR5s0uLKg6iMF6vV1H4pYCDooT/B6z5TDIvag 8skg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=uAzG6uoZEE9KFkpoAiZpv05CNx239mJ72+2x7m8hZ0w=; b=ztSYyjO03HMWD+Ai7fJuFTzerEnz3KbaUkjOknJk6A2dfAmn8pcr1pKmjyl+Sp/aNM eO3Ei1YVFgzQy0mo/96WGfMUCY+ceMqftMHFYTLEt8NV6i+3EGCyhSexf44DBMG/qJYB /WOUlGi/LN4OA1aB+Fu9NreeR/S0rOdEb7PlnjU1bjEkMy+O4ltEOnMqf7xO9C8rU0pn Jnbz097Ha5euKnwUw8Wc0SJMX9/PAslLyaKacKSxqufMVze/cyyv161VkHnNyjSGSyoC LR5VaWXOoUeM/cRALt2z6siKAHhYpw7AzxhfQx+6iVXxazIYhGIjwq1Nt8d4VFoPmSZu NuwA== X-Gm-Message-State: AOAM5327NaVUn55f0d5/0qHdB5fvwMKLk7/XLudlW2ipdCfP2d8ToIEA nfkntRYLeuFGd8B7LQMcW9Bf4RuOa1I= X-Google-Smtp-Source: ABdhPJyoi8egN4V8cHYh/lHqXoTdDPf3WtNqa6voKXTrjoRxJ+QPzVCRaanHZYlbUG5ukcoTkCO+6w== X-Received: by 2002:a1c:540c:: with SMTP id i12mr6035369wmb.6.1635345568277; Wed, 27 Oct 2021 07:39:28 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l6sm25875wmq.17.2021.10.27.07.39.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Oct 2021 07:39:28 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 27 Oct 2021 14:39:21 +0000 Subject: [PATCH v5 6/8] reset: make sparse-aware (except --mixed) Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Remove `ensure_full_index` guard on `prime_cache_tree` and update `prime_cache_tree_rec` to correctly reconstruct sparse directory entries in the cache tree. While processing a tree's entries, `prime_cache_tree_rec` must determine whether a directory entry is sparse or not by searching for it in the index (*without* expanding the index). If a matching sparse directory index entry is found, no subtrees are added to the cache tree entry and the entry count is set to 1 (representing the sparse directory itself). Otherwise, the tree is assumed to not be sparse and its subtrees are recursively added to the cache tree. Helped-by: Elijah Newren Signed-off-by: Victoria Dye --- cache-tree.c | 47 ++++++++++++++++++++++-- cache.h | 10 +++++ read-cache.c | 27 ++++++++++---- t/t1092-sparse-checkout-compatibility.sh | 15 +++++++- 4 files changed, 86 insertions(+), 13 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index 9be19c85b66..2866101052c 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -740,15 +740,26 @@ out: return ret; } +static void prime_cache_tree_sparse_dir(struct cache_tree *it, + struct tree *tree) +{ + + oidcpy(&it->oid, &tree->object.oid); + it->entry_count = 1; +} + static void prime_cache_tree_rec(struct repository *r, struct cache_tree *it, - struct tree *tree) + struct tree *tree, + struct strbuf *tree_path) { struct tree_desc desc; struct name_entry entry; int cnt; + int base_path_len = tree_path->len; oidcpy(&it->oid, &tree->object.oid); + init_tree_desc(&desc, tree->buffer, tree->size); cnt = 0; while (tree_entry(&desc, &entry)) { @@ -757,14 +768,40 @@ static void prime_cache_tree_rec(struct repository *r, else { struct cache_tree_sub *sub; struct tree *subtree = lookup_tree(r, &entry.oid); + if (!subtree->object.parsed) parse_tree(subtree); sub = cache_tree_sub(it, entry.path); sub->cache_tree = cache_tree(); - prime_cache_tree_rec(r, sub->cache_tree, subtree); + + /* + * Recursively-constructed subtree path is only needed when working + * in a sparse index (where it's used to determine whether the + * subtree is a sparse directory in the index). + */ + if (r->index->sparse_index) { + strbuf_setlen(tree_path, base_path_len); + strbuf_grow(tree_path, base_path_len + entry.pathlen + 1); + strbuf_add(tree_path, entry.path, entry.pathlen); + strbuf_addch(tree_path, '/'); + } + + /* + * If a sparse index is in use, the directory being processed may be + * sparse. To confirm that, we can check whether an entry with that + * exact name exists in the index. If it does, the created subtree + * should be sparse. Otherwise, cache tree expansion should continue + * as normal. + */ + if (r->index->sparse_index && + index_entry_exists(r->index, tree_path->buf, tree_path->len)) + prime_cache_tree_sparse_dir(sub->cache_tree, subtree); + else + prime_cache_tree_rec(r, sub->cache_tree, subtree, tree_path); cnt += sub->cache_tree->entry_count; } } + it->entry_count = cnt; } @@ -772,12 +809,14 @@ void prime_cache_tree(struct repository *r, struct index_state *istate, struct tree *tree) { + struct strbuf tree_path = STRBUF_INIT; + trace2_region_enter("cache-tree", "prime_cache_tree", the_repository); cache_tree_free(&istate->cache_tree); istate->cache_tree = cache_tree(); - ensure_full_index(istate); - prime_cache_tree_rec(r, istate->cache_tree, tree); + prime_cache_tree_rec(r, istate->cache_tree, tree, &tree_path); + strbuf_release(&tree_path); istate->cache_changed |= CACHE_TREE_CHANGED; trace2_region_leave("cache-tree", "prime_cache_tree", the_repository); } diff --git a/cache.h b/cache.h index f6295f3b048..1d3e4665562 100644 --- a/cache.h +++ b/cache.h @@ -816,6 +816,16 @@ struct cache_entry *index_file_exists(struct index_state *istate, const char *na */ int index_name_pos(struct index_state *, const char *name, int namelen); +/* + * Determines whether an entry with the given name exists within the + * given index. The return value is 1 if an exact match is found, otherwise + * it is 0. Note that, unlike index_name_pos, this function does not expand + * the index if it is sparse. If an item exists within the full index but it + * is contained within a sparse directory (and not in the sparse index), 0 is + * returned. + */ +int index_entry_exists(struct index_state *, const char *name, int namelen); + /* * Some functions return the negative complement of an insert position when a * precise match was not found but a position was found where the entry would diff --git a/read-cache.c b/read-cache.c index f5d4385c408..c079ece981a 100644 --- a/read-cache.c +++ b/read-cache.c @@ -68,6 +68,11 @@ */ #define CACHE_ENTRY_PATH_LENGTH 80 +enum index_search_mode { + NO_EXPAND_SPARSE = 0, + EXPAND_SPARSE = 1 +}; + static inline struct cache_entry *mem_pool__ce_alloc(struct mem_pool *mem_pool, size_t len) { struct cache_entry *ce; @@ -551,7 +556,10 @@ int cache_name_stage_compare(const char *name1, int len1, int stage1, const char return 0; } -static int index_name_stage_pos(struct index_state *istate, const char *name, int namelen, int stage) +static int index_name_stage_pos(struct index_state *istate, + const char *name, int namelen, + int stage, + enum index_search_mode search_mode) { int first, last; @@ -570,7 +578,7 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in first = next+1; } - if (istate->sparse_index && + if (search_mode == EXPAND_SPARSE && istate->sparse_index && first > 0) { /* Note: first <= istate->cache_nr */ struct cache_entry *ce = istate->cache[first - 1]; @@ -586,7 +594,7 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in ce_namelen(ce) < namelen && !strncmp(name, ce->name, ce_namelen(ce))) { ensure_full_index(istate); - return index_name_stage_pos(istate, name, namelen, stage); + return index_name_stage_pos(istate, name, namelen, stage, search_mode); } } @@ -595,7 +603,12 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in int index_name_pos(struct index_state *istate, const char *name, int namelen) { - return index_name_stage_pos(istate, name, namelen, 0); + return index_name_stage_pos(istate, name, namelen, 0, EXPAND_SPARSE); +} + +int index_entry_exists(struct index_state *istate, const char *name, int namelen) +{ + return index_name_stage_pos(istate, name, namelen, 0, NO_EXPAND_SPARSE) >= 0; } int remove_index_entry_at(struct index_state *istate, int pos) @@ -1222,7 +1235,7 @@ static int has_dir_name(struct index_state *istate, */ } - pos = index_name_stage_pos(istate, name, len, stage); + pos = index_name_stage_pos(istate, name, len, stage, EXPAND_SPARSE); if (pos >= 0) { /* * Found one, but not so fast. This could @@ -1322,7 +1335,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e strcmp(ce->name, istate->cache[istate->cache_nr - 1]->name) > 0) pos = index_pos_to_insert_pos(istate->cache_nr); else - pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce)); + pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), EXPAND_SPARSE); /* existing match? Just replace it. */ if (pos >= 0) { @@ -1357,7 +1370,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e if (!ok_to_replace) return error(_("'%s' appears as both a file and as a directory"), ce->name); - pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce)); + pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), EXPAND_SPARSE); pos = -pos-1; } return pos + 1; diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index a8583030b38..5664ff8f039 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -763,9 +763,9 @@ test_expect_success 'sparse-index is not expanded' ' ensure_not_expanded checkout - && ensure_not_expanded switch rename-out-to-out && ensure_not_expanded switch - && - git -C sparse-index reset --hard && + ensure_not_expanded reset --hard && ensure_not_expanded checkout rename-out-to-out -- deep/deeper1 && - git -C sparse-index reset --hard && + ensure_not_expanded reset --hard && ensure_not_expanded restore -s rename-out-to-out -- deep/deeper1 && echo >>sparse-index/README.md && @@ -775,6 +775,17 @@ test_expect_success 'sparse-index is not expanded' ' echo >>sparse-index/untracked.txt && ensure_not_expanded add . && + for ref in update-deep update-folder1 update-folder2 update-deep + do + echo >>sparse-index/README.md && + ensure_not_expanded reset --hard $ref || return 1 + done && + + ensure_not_expanded reset --hard update-deep && + ensure_not_expanded reset --keep base && + ensure_not_expanded reset --merge update-deep && + ensure_not_expanded reset --hard && + ensure_not_expanded checkout -f update-deep && test_config -C sparse-index pull.twohead ort && ( From patchwork Wed Oct 27 14:39:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12587481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D302C433F5 for ; Wed, 27 Oct 2021 14:39:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3110D60F56 for ; Wed, 27 Oct 2021 14:39:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242649AbhJ0OmE (ORCPT ); Wed, 27 Oct 2021 10:42:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239046AbhJ0Olz (ORCPT ); Wed, 27 Oct 2021 10:41:55 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B92EC061348 for ; Wed, 27 Oct 2021 07:39:30 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id g7so3495580wrb.2 for ; Wed, 27 Oct 2021 07:39:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=J5gycgtHpviMm7267zW7Aw7f2/Uvmyd+O1rDc19cPtc=; b=hrDEWFKve1dsIAkqtQUCgxyXiSPQKuqj9DoA4vgpViroeWvf0lpLBrtjtLfJCsrLFS JGiqfkiBumZ1DeBofwdjN0kMI63tryTxu4NTU/vbUfgo1p4HvTtlGoi2JFqVW51IBryb BPBmDC4bP5lRFQEwZ2eIBJUhcOnZszkDDZNt9wNa8NJsM0hZrfqKAlWwVYxBBNi5iVAA AbHLqmy27Vi8Ihe6teGBjleaBqxuPfnDjidfUWPmPAM3eClBK3oQygKu1FVrP8+IVmNQ CZKMcR0EznKcaYhY1kesx65Kao3bnlE3R3bac2QBHO/pM343eX3RO1Tqv/sUMcex7PJa gB9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=J5gycgtHpviMm7267zW7Aw7f2/Uvmyd+O1rDc19cPtc=; b=fYABBHwUe0CoMz4r+rDmeKXclpy0AT0mso3HYsVZrmaSC6efxt+zX6sphDYjiBfe0d X0+Nhzgt1C8I33K9gp5jIb0xmN+VMF1Ws/4bIwgjwwmOWFJBpuKzEPvJGILXopQBsLQI enBd1kYVYUqsJ3N+x7+MRa/XRXwneiSQTl4gkLiQfmUG5AQHE8Sb8n0IFJWW3rFiduzB beHo8qmU5LxNWj7opUvEbjkrrA5wmqMl4xsf5EP6+p+LuJF75z0BTAdjSKF54jOnDjR8 zfotlcNJDWWZClzBTTsyeCDTBOoI89Ug4Y6wzr3Rf+85CcaIw1cCL4ysqFw+TPy59gfi QAQg== X-Gm-Message-State: AOAM533ixoEa6045HtdwoC4Wi21ZjpN/6d7S9WZ9XIm5mA5+RKTdvLaA S4F8RHMgvhDl8jR4nO6N53IC7N5ZhAw= X-Google-Smtp-Source: ABdhPJziAWBEvS8EhBoVkZxiZTYfD6ifdfYC2uRpiI2zxSdjaHw2s69+LgwZp22e5kwke3LPVI60Ag== X-Received: by 2002:adf:f902:: with SMTP id b2mr41287053wrr.265.1635345568924; Wed, 27 Oct 2021 07:39:28 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d8sm49423wrv.80.2021.10.27.07.39.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Oct 2021 07:39:28 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 27 Oct 2021 14:39:22 +0000 Subject: [PATCH v5 7/8] reset: make --mixed sparse-aware Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Remove the `ensure_full_index` guard on `read_from_tree` and update `git reset --mixed` to ensure it can use sparse directory index entries wherever possible. Sparse directory entries are reset use `diff_tree_oid`, which requires `change` and `add_remove` functions to process the internal contents of the sparse directory. The `recursive` diff option handles cases in which `reset --mixed` must diff/merge files that are nested multiple levels deep in a sparse directory. The use of pathspecs with `git reset --mixed` introduces scenarios in which internal contents of sparse directories may be matched by the pathspec. In order to reset *all* files in the repo that may match the pathspec, the following conditions on the pathspec require index expansion before performing the reset: * "magic" pathspecs * wildcard pathspecs that do not match only in-cone files or entire sparse directories * literal pathspecs matching something outside the sparse checkout definition Helped-by: Elijah Newren Signed-off-by: Victoria Dye --- builtin/reset.c | 78 +++++++++++++++++++++++- t/t1092-sparse-checkout-compatibility.sh | 17 ++++++ 2 files changed, 93 insertions(+), 2 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index 0ac0de7dc97..60517e7e1d6 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -148,7 +148,9 @@ static void update_index_from_diff(struct diff_queue_struct *q, * If the file 1) corresponds to an existing index entry with * skip-worktree set, or 2) does not exist in the index but is * outside the sparse checkout definition, add a skip-worktree bit - * to the new index entry. + * to the new index entry. Note that a sparse index will be expanded + * if this entry is outside the sparse cone - this is necessary + * to properly construct the reset sparse directory. */ pos = cache_name_pos(one->path, strlen(one->path)); if ((pos >= 0 && ce_skip_worktree(active_cache[pos])) || @@ -166,6 +168,73 @@ static void update_index_from_diff(struct diff_queue_struct *q, } } +static int pathspec_needs_expanded_index(const struct pathspec *pathspec) +{ + unsigned int i, pos; + int res = 0; + char *skip_worktree_seen = NULL; + + /* + * When using a magic pathspec, assume for the sake of simplicity that + * the index needs to be expanded to match all matchable files. + */ + if (pathspec->magic) + return 1; + + for (i = 0; i < pathspec->nr; i++) { + struct pathspec_item item = pathspec->items[i]; + + /* + * If the pathspec item has a wildcard, the index should be expanded + * if the pathspec has the possibility of matching a subset of entries inside + * of a sparse directory (but not the entire directory). + * + * If the pathspec item is a literal path, the index only needs to be expanded + * if a) the pathspec isn't in the sparse checkout cone (to make sure we don't + * expand for in-cone files) and b) it doesn't match any sparse directories + * (since we can reset whole sparse directories without expanding them). + */ + if (item.nowildcard_len < item.len) { + for (pos = 0; pos < active_nr; pos++) { + struct cache_entry *ce = active_cache[pos]; + + if (!S_ISSPARSEDIR(ce->ce_mode)) + continue; + + /* + * If the pre-wildcard length is longer than the sparse + * directory name and the sparse directory is the first + * component of the pathspec, need to expand the index. + */ + if (item.nowildcard_len > ce_namelen(ce) && + !strncmp(item.original, ce->name, ce_namelen(ce))) { + res = 1; + break; + } + + /* + * If the pre-wildcard length is shorter than the sparse + * directory and the pathspec does not match the whole + * directory, need to expand the index. + */ + if (!strncmp(item.original, ce->name, item.nowildcard_len) && + wildmatch(item.original, ce->name, 0)) { + res = 1; + break; + } + } + } else if (!path_in_cone_mode_sparse_checkout(item.original, &the_index) && + !matches_skip_worktree(pathspec, i, &skip_worktree_seen)) + res = 1; + + if (res > 0) + break; + } + + free(skip_worktree_seen); + return res; +} + static int read_from_tree(const struct pathspec *pathspec, struct object_id *tree_oid, int intent_to_add) @@ -178,9 +247,14 @@ static int read_from_tree(const struct pathspec *pathspec, opt.format_callback = update_index_from_diff; opt.format_callback_data = &intent_to_add; opt.flags.override_submodule_config = 1; + opt.flags.recursive = 1; opt.repo = the_repository; + opt.change = diff_change; + opt.add_remove = diff_addremove; + + if (pathspec->nr && the_index.sparse_index && pathspec_needs_expanded_index(pathspec)) + ensure_full_index(&the_index); - ensure_full_index(&the_index); if (do_diff_cache(tree_oid, &opt)) return 1; diffcore_std(&opt); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 5664ff8f039..44d5e11c762 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -781,11 +781,28 @@ test_expect_success 'sparse-index is not expanded' ' ensure_not_expanded reset --hard $ref || return 1 done && + ensure_not_expanded reset --mixed base && ensure_not_expanded reset --hard update-deep && ensure_not_expanded reset --keep base && ensure_not_expanded reset --merge update-deep && ensure_not_expanded reset --hard && + ensure_not_expanded reset base -- deep/a && + ensure_not_expanded reset base -- nonexistent-file && + ensure_not_expanded reset deepest -- deep && + + # Although folder1 is outside the sparse definition, it exists as a + # directory entry in the index, so the pathspec will not force the + # index to be expanded. + ensure_not_expanded reset deepest -- folder1 && + ensure_not_expanded reset deepest -- folder1/ && + + # Wildcard identifies only in-cone files, no index expansion + ensure_not_expanded reset deepest -- deep/\* && + + # Wildcard identifies only full sparse directories, no index expansion + ensure_not_expanded reset deepest -- folder\* && + ensure_not_expanded checkout -f update-deep && test_config -C sparse-index pull.twohead ort && ( From patchwork Wed Oct 27 14:39:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12587483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF100C433EF for ; Wed, 27 Oct 2021 14:39:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B91C860FC0 for ; Wed, 27 Oct 2021 14:39:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242652AbhJ0OmF (ORCPT ); Wed, 27 Oct 2021 10:42:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238841AbhJ0Ol7 (ORCPT ); Wed, 27 Oct 2021 10:41:59 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E35CAC061220 for ; Wed, 27 Oct 2021 07:39:30 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id m42so2856404wms.2 for ; Wed, 27 Oct 2021 07:39:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=nEih7G58FeXZR7VfCiUcLv0bmQevEM54qXiHHW/Vpvo=; b=mS5GBY3gCI4NiNMCM7r3FYnAdSjzATfxo9FQPnDGHJjamKo8Z2uxQbF6E2c+9TnE+w hEexrgRGFs8xULrKaFl9yXfxGzREi3W/gSZFirHORen+ChecdhF90yhmEfnzuowpWt3l pW1QKJa1J/gU0dnSVGKfTGLP60HDPTnEjqlKBdanLTZ8pK74BdmKPdjQitcms/+lwsgw pJfdm//19jRCWN3Zu5q+vVTI/8CGaKPoYVbfxeRFpAB7/dEepRpPI/ssg8MuAjKJ43fY DtOi36D3VzWiSVE+cU7si9gjENOoaICh2G5lC6wVjgDtX4KgfsXPD72qDYRAr2QHTjcn JP1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=nEih7G58FeXZR7VfCiUcLv0bmQevEM54qXiHHW/Vpvo=; b=zxV8Eq7qASu1RlnUm8tHLVuU30OR+xO+2Co3qhD/FJO0ZBU9RSgOcgpmgLrgWHmc72 Ph+HRvIWCo9TLDNfRDvjgwNBDnh1pJjfxMaaNtW0hc463ZCjl45VbbXNPpXLVtcb2BbW 1FCQArIYnob/ALZ1xJWDTsVcBsPPfjuRxRaLRknW0FKgUMGgkROElPXQin5gji+prcLJ zfTfBYeOHGdYkW29flWvzY2DigsfRegtg4n9NEN6N/nNH0gTN7AyZT6acXOFL5oJtnFL OsUWnZpDIXlMddrB5SpXXlRMNLPQYzg2Qi+Ls2aOmYznUYDh3x8KJqOwGfzgGP8Dbbva qqvw== X-Gm-Message-State: AOAM531oyEyKuTO4NP3zpt8GX4awzoPxne+TGaDfBoQEK3FqqaO9Zn+f u64OMUNthIU436KF9doirYBp9rjoIwQ= X-Google-Smtp-Source: ABdhPJz5R4k6rc5pjaDR7yOcelK670LD1iWtREfu0i0SA+emW9KEiA0CU2IFpWT9NxYedIcBa7qQWQ== X-Received: by 2002:a1c:7e41:: with SMTP id z62mr6163079wmc.9.1635345569492; Wed, 27 Oct 2021 07:39:29 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g2sm76616wrb.20.2021.10.27.07.39.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Oct 2021 07:39:29 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 27 Oct 2021 14:39:23 +0000 Subject: [PATCH v5 8/8] unpack-trees: improve performance of next_cache_entry Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Taylor Blau , Bagas Sanjaya , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Phillip Wood , Emily Shaffer , Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye To find the first non-unpacked cache entry, `next_cache_entry` iterates through index, starting at `cache_bottom`. The performance of this in full indexes is helped by `cache_bottom` advancing with each invocation of `mark_ce_used` (called by `unpack_index_entry`). However, the presence of sparse directories can prevent the `cache_bottom` from advancing in a sparse index case, effectively forcing `next_cache_entry` to search from the beginning of the index each time it is called. The `cache_bottom` must be preserved for the sparse index (see 17a1bb570b (unpack-trees: preserve cache_bottom, 2021-07-14)). Therefore, to retain the benefit `cache_bottom` provides in non-sparse index cases, a separate `hint` position indicates the first position `next_cache_entry` should search, updated each execution with a new position. Signed-off-by: Victoria Dye --- unpack-trees.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/unpack-trees.c b/unpack-trees.c index 8ea0a542da8..b94733de6be 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -645,17 +645,24 @@ static void mark_ce_used_same_name(struct cache_entry *ce, } } -static struct cache_entry *next_cache_entry(struct unpack_trees_options *o) +static struct cache_entry *next_cache_entry(struct unpack_trees_options *o, int *hint) { const struct index_state *index = o->src_index; int pos = o->cache_bottom; + if (*hint > pos) + pos = *hint; + while (pos < index->cache_nr) { struct cache_entry *ce = index->cache[pos]; - if (!(ce->ce_flags & CE_UNPACKED)) + if (!(ce->ce_flags & CE_UNPACKED)) { + *hint = pos + 1; return ce; + } pos++; } + + *hint = pos; return NULL; } @@ -1365,12 +1372,13 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str /* Are we supposed to look at the index too? */ if (o->merge) { + int hint = -1; while (1) { int cmp; struct cache_entry *ce; if (o->diff_index_cached) - ce = next_cache_entry(o); + ce = next_cache_entry(o, &hint); else ce = find_cache_entry(info, p); @@ -1690,7 +1698,7 @@ static int verify_absent(const struct cache_entry *, int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options *o) { struct repository *repo = the_repository; - int i, ret; + int i, hint, ret; static struct cache_entry *dfc; struct pattern_list pl; int free_pattern_list = 0; @@ -1763,13 +1771,15 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options info.pathspec = o->pathspec; if (o->prefix) { + hint = -1; + /* * Unpack existing index entries that sort before the * prefix the tree is spliced into. Note that o->merge * is always true in this case. */ while (1) { - struct cache_entry *ce = next_cache_entry(o); + struct cache_entry *ce = next_cache_entry(o, &hint); if (!ce) break; if (ce_in_traverse_path(ce, &info)) @@ -1790,8 +1800,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options /* Any left-over entries in the index? */ if (o->merge) { + hint = -1; while (1) { - struct cache_entry *ce = next_cache_entry(o); + struct cache_entry *ce = next_cache_entry(o, &hint); if (!ce) break; if (unpack_index_entry(ce, o) < 0)