From patchwork Thu Sep 30 14:50:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Willford X-Patchwork-Id: 12528597 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5570C433FE for ; Thu, 30 Sep 2021 14:51:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 99E1E61A53 for ; Thu, 30 Sep 2021 14:51:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351919AbhI3Ows (ORCPT ); Thu, 30 Sep 2021 10:52:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349797AbhI3Owr (ORCPT ); Thu, 30 Sep 2021 10:52:47 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2D41C06176A for ; Thu, 30 Sep 2021 07:51:04 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id q2so664173wrc.4 for ; Thu, 30 Sep 2021 07:51:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=gGCfyG6tsBdfVw7efmwpQlI96WeOhFsi+r0liCqT06s=; b=ca2XUPJ5FZtgYCvvlZQ38seqbWWCr2Cr6p/+IaHFiYh26Q6yZLBWY8Wf4ECFesr+ib eXmlAwU4I5gewKjrILEPtXmnZbSdoLJaGkm6CPczvxYhrTf4EYzeTFDcKsY7lzdg2mUQ uTUBTB6sfjjDvbk28zmqBtVnYJlMX2oeQUKN/+xowpVaCPw5wcO5HIgBHjPSuWaDB/ap Hcm11VoaddiL0IdHnisIJ1N5/og/82WbmkKMcgmRVCJfr23kxKvbMcrfYXA8e011cn9q QSLn8lk8zg3OYBliZz+jBJlepV97rEF0Szl+EGTImrx36U9vyyGdmIog3UNz45pSKW5F mOWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=gGCfyG6tsBdfVw7efmwpQlI96WeOhFsi+r0liCqT06s=; b=0Vlr/+Xi4qQBq8quhZ+JsnaIBN5jc86Mj8huG5y/4hbswkGvCEIme7rhgh6pHtDaf1 eFgV4gdPYu5sVLzzsPLFTEDdF7A1mJK6nEphM1+0FD5BdX64JhWT+QKFmyXSJj9FobVw 7N8szLN+4ubb6laYR7lx+oubVe4F6pPdt9RoGgQr43Eh8UhLj14Xb3ZXIPU9tB0sUJMi JRcxx6hYOvw4Spjgul6Gcnmb9EhQ8DOj3CPIFmre/fSMh/VSMaINJQCUTbva7KG9lm7X hHm3gflqFAk5GTlQRGvzmQXZKzuSa5ekgi0OkZPgMxpuJfylzqkoRZqnYrXQiiM4SrlT H/Hg== X-Gm-Message-State: AOAM531cMqJ1omqoKMKNVWCNJlbs3BK1QtVtm2xQTIAAkBzJsePnJZRH jc6gDx5wvIaxUHgxR3jqqlVUeyzRNR8= X-Google-Smtp-Source: ABdhPJyfHDTDWOZF0YPfJfBQzM76OMrOU0FmMnmC9yCxePGVcr+ICigX4rvBLMKlBnQG4QIIp5lkMQ== X-Received: by 2002:adf:e643:: with SMTP id b3mr6887644wrn.67.1633013463395; Thu, 30 Sep 2021 07:51:03 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d2sm3197952wrc.32.2021.09.30.07.51.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:51:03 -0700 (PDT) Message-Id: <65905bf4e001118e8b9ced95c1bcecbacb6334ac.1633013461.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 30 Sep 2021 14:50:55 +0000 Subject: [PATCH 1/7] reset: behave correctly with sparse-checkout Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Victoria Dye , Kevin Willford Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Kevin Willford From: Kevin Willford When using the sparse checkout feature, 'git reset' will add entries to the index that will have the skip-worktree bit off but will leave the working directory empty. File data is lost because the index version of the files has been changed but there is nothing that is in the working directory. This will cause the next 'git status' call to show either deleted for files modified or deleting or nothing for files added. The added files should be shown as untracked and modified files should be shown as modified. To fix this when the reset is running if there is not a file in the working directory and if it will be missing with the new index entry or was not missing in the previous version, we create the previous index version of the file in the working directory so that status will report correctly and the files will be availble for the user to deal with. This fixes a documented failure from t1092 that was created in 19a0acc (t1092: test interesting sparse-checkout scenarios, 2021-01-23). Signed-off-by: Kevin Willford Signed-off-by: Derrick Stolee Signed-off-by: Victoria Dye --- builtin/reset.c | 39 +++++++++++++++ t/t1092-sparse-checkout-compatibility.sh | 4 +- t/t7114-reset-sparse-checkout.sh | 61 ++++++++++++++++++++++++ 3 files changed, 101 insertions(+), 3 deletions(-) create mode 100755 t/t7114-reset-sparse-checkout.sh diff --git a/builtin/reset.c b/builtin/reset.c index 51c9e2f43ff..8ffcd713720 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -25,6 +25,8 @@ #include "cache-tree.h" #include "submodule.h" #include "submodule-config.h" +#include "dir.h" +#include "entry.h" #define REFRESH_INDEX_DELAY_WARNING_IN_MS (2 * 1000) @@ -127,12 +129,49 @@ static void update_index_from_diff(struct diff_queue_struct *q, struct diff_options *opt, void *data) { int i; + int pos; int intent_to_add = *(int *)data; for (i = 0; i < q->nr; i++) { struct diff_filespec *one = q->queue[i]->one; + struct diff_filespec *two = q->queue[i]->two; int is_missing = !(one->mode && !is_null_oid(&one->oid)); + int was_missing = !two->mode && is_null_oid(&two->oid); struct cache_entry *ce; + struct cache_entry *ce_before; + struct checkout state = CHECKOUT_INIT; + + /* + * When using the sparse-checkout feature the cache entries + * that are added here will not have the skip-worktree bit + * set. Without this code there is data that is lost because + * the files that would normally be in the working directory + * are not there and show as deleted for the next status. + * In the case of added files, they just disappear. + * + * We need to create the previous version of the files in + * the working directory so that they will have the right + * content and the next status call will show modified or + * untracked files correctly. + */ + if (core_apply_sparse_checkout && !file_exists(two->path)) { + pos = cache_name_pos(two->path, strlen(two->path)); + if ((pos >= 0 && ce_skip_worktree(active_cache[pos])) && + (is_missing || !was_missing)) { + state.force = 1; + state.refresh_cache = 1; + state.istate = &the_index; + + ce_before = make_cache_entry(&the_index, two->mode, + &two->oid, two->path, + 0, 0); + if (!ce_before) + die(_("make_cache_entry failed for path '%s'"), + two->path); + + checkout_entry(ce_before, &state, NULL, NULL); + } + } if (is_missing && !intent_to_add) { remove_file_from_cache(one->path); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 886e78715fe..c5977152661 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -459,9 +459,7 @@ test_expect_failure 'blame with pathspec outside sparse definition' ' test_all_match git blame deep/deeper2/deepest/a ' -# NEEDSWORK: a sparse-checkout behaves differently from a full checkout -# in this scenario, but it shouldn't. -test_expect_failure 'checkout and reset (mixed)' ' +test_expect_success 'checkout and reset (mixed)' ' init_repos && test_all_match git checkout -b reset-test update-deep && diff --git a/t/t7114-reset-sparse-checkout.sh b/t/t7114-reset-sparse-checkout.sh new file mode 100755 index 00000000000..a8029707fb1 --- /dev/null +++ b/t/t7114-reset-sparse-checkout.sh @@ -0,0 +1,61 @@ +#!/bin/sh + +test_description='reset when using a sparse-checkout' + +. ./test-lib.sh + +test_expect_success 'setup' ' + test_tick && + echo "checkout file" >c && + echo "modify file" >m && + echo "delete file" >d && + git add . && + git commit -m "initial commit" && + echo "added file" >a && + echo "modification of a file" >m && + git rm d && + git add . && + git commit -m "second commit" && + git checkout -b endCommit +' + +test_expect_success 'reset when there is a sparse-checkout' ' + echo "/c" >.git/info/sparse-checkout && + test_config core.sparsecheckout true && + git checkout -B resetBranch && + test_path_is_missing m && + test_path_is_missing a && + test_path_is_missing d && + git reset HEAD~1 && + echo "checkout file" >expect && + test_cmp expect c && + echo "added file" >expect && + test_cmp expect a && + echo "modification of a file" >expect && + test_cmp expect m && + test_path_is_missing d +' + +test_expect_success 'reset after deleting file without skip-worktree bit' ' + git checkout -f endCommit && + git clean -xdf && + cat >.git/info/sparse-checkout <<-\EOF && + /c + /m + EOF + test_config core.sparsecheckout true && + git checkout -B resetAfterDelete && + test_path_is_file m && + test_path_is_missing a && + test_path_is_missing d && + rm -f m && + git reset HEAD~1 && + echo "checkout file" >expect && + test_cmp expect c && + echo "added file" >expect && + test_cmp expect a && + test_path_is_missing m && + test_path_is_missing d +' + +test_done From patchwork Thu Sep 30 14:50:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12528595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82D63C4332F for ; Thu, 30 Sep 2021 14:51:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6896E61A55 for ; Thu, 30 Sep 2021 14:51:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351926AbhI3Owt (ORCPT ); Thu, 30 Sep 2021 10:52:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351915AbhI3Ows (ORCPT ); Thu, 30 Sep 2021 10:52:48 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7473DC06176A for ; Thu, 30 Sep 2021 07:51:05 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id m22so5086221wrb.0 for ; Thu, 30 Sep 2021 07:51:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=kWQPjrk76/fCKSTx7JTIfz6mPHkIGUtZo1+CF8cCSwM=; b=UlvV1jAHkkaTlOPkDGbT6NdT/wJ2jL5xpNKInkZSHM7RRnApDmuKA5CY/5cQRj6FB9 qSC9i+AaqUSXxEJyFNlM7bAO3z9pPNrdTCJmW4h2i4N/kq9fSyB1pCcYU5S8goVEihOk AmjRfcRZ+K6AepFaIZlngof3KrdiMqx5P1RszOUCWiJ6yDxSsCVeCO3oDZh4qbQxHMRy kbz32DvKzl335aYZz5X/42egVbnMwQq0djd01kEgiXwSn8q4VymD7a7eLLNrH1HNRWtQ g7G/tkmJ4DCGv6KIwJYazLXM6tQaJ3fOXFlfR+gXsH00XW+Wu6rckLKQ3fjvXFk9Bf0a 7U3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=kWQPjrk76/fCKSTx7JTIfz6mPHkIGUtZo1+CF8cCSwM=; b=XrVNH0WLJa4R1w+wtCP7L+TMYwPmoE5CK/B4SGqd09cfosoDyh/txMRBYjPwUuW0ZK 7+rQg2ai8A0JTWevhnkN0sPCpelrlJK4FQs9ads/q8chm7dfSh9ALmqULMvjTgDqjjxA 5BDQDJPlHGKwRbdsP1ZMp9fcN5pbg1YGAFwdfeTEdYTAjLANQ2f9udaAUkRTcQe+qtXb doE6axAgETY+NZ6pLvwKAsOkI4uwpN6OMNFj2NSNmR9ZY4sx2kk8eWK/U2kcQxIFp/MJ 74RmWm3QA94dLkYxxIyLHC6+Vd+JJ0Uh+3y7tvB9NPvZobJP9B1hUbdUfNI6O6vwuNOh wlKg== X-Gm-Message-State: AOAM531v4Dzrf8LnGdh6Ooe5x4ZOaMsYiU/ov7twRMjuRi2Ozw7GcdVd mqD6uXQ4Y5s5GRYL7T83Dxx+CgKY4fg= X-Google-Smtp-Source: ABdhPJzG/pR1APraOVqqMd/p0ppchvxH0YV43q0bwBOybMqjdpyUgVKBpvUZpmuR0lTp0d32/P/AhQ== X-Received: by 2002:adf:b7c1:: with SMTP id t1mr6622593wre.387.1633013464117; Thu, 30 Sep 2021 07:51:04 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c17sm4843637wmr.15.2021.09.30.07.51.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:51:03 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 30 Sep 2021 14:50:56 +0000 Subject: [PATCH 2/7] sparse-index: update command for expand/collapse test Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye In anticipation of multiple commands being fully integrated with sparse index, update the test for index expand and collapse for non-sparse index integrated commands to use `mv`. Signed-off-by: Victoria Dye --- t/t1092-sparse-checkout-compatibility.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index c5977152661..aed8683e629 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -642,7 +642,7 @@ test_expect_success 'sparse-index is expanded and converted back' ' init_repos && GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ - git -C sparse-index -c core.fsmonitor="" reset --hard && + git -C sparse-index -c core.fsmonitor="" mv a b && test_region index convert_to_sparse trace2.txt && test_region index ensure_full_index trace2.txt ' From patchwork Thu Sep 30 14:50:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12528599 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3613C433EF for ; Thu, 30 Sep 2021 14:51:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 98DB961A56 for ; Thu, 30 Sep 2021 14:51:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351938AbhI3Owt (ORCPT ); Thu, 30 Sep 2021 10:52:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351923AbhI3Owt (ORCPT ); Thu, 30 Sep 2021 10:52:49 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20C87C06176A for ; Thu, 30 Sep 2021 07:51:06 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id d21so10497486wra.12 for ; Thu, 30 Sep 2021 07:51:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=pq38z/Ts2Q1i/3RuHIH4JqscSdd1ZyLNOkr2P0NxmCM=; b=X7lPXlRuqHhzinvfx4sdNwU5PLtVEB6GPBFPNcviY/sIZTVMGtVMiubqr+XL1UV+95 wAyMmCTPPB8CG7Vp7c1okS7lNNmY40tASyHllkutoN4ZT66cUTrJcHRuc/e5OT9aYOja MHYS5urn6Faeggd8ISGD0gCN/sU+nAstUzcbJS7HDvDOTGILQf5aTkoif8yI7pYXcQfd VCSuA+yqUp47odsFjiahepQo4VLxtMNT+9MGbA3aW1qNkX7rNAu5EHoA9kluRdA924De CBkb6pAZHqda7AIKvqvnLMHwZ40iymRXkTW70MuaiUFpa5WMAxhWCjG1BwLAhe6dtsYj j3Pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=pq38z/Ts2Q1i/3RuHIH4JqscSdd1ZyLNOkr2P0NxmCM=; b=dT2DeTVk1B0J8AUNuP+2EHXqe2UEsVnL23SQdKw7rmDo6XJQkIVmShL4N3YzzccpDb cNDm9SyNLkWkllVcT5BOKxP4iJdLp0fTzf4yB/niJs8XgWa2FrDVf6+IibLWZE5psZVJ h5wmCA0X8YxnLMdm/H4ObCeDS1p6W7JEpyBzKDiqI8U82fc8zKfJjQ+2tVBxUKDFrT/s hhlDhVlDvSRWAxK5thMRiHTZVHWMNnBSY9Vm2TOMJRjfihra7kqbVcdiJBF8jJ4fkmWY 2zQAcy7Dk13z5vczIE5COWD3hdpWYefAxiP3IHTpP+R4SG+kxBH2OB+XvBJgABPU/Q45 z7uw== X-Gm-Message-State: AOAM530wVhjlskBexak8Gpmke+BwGpkbxTTx0Std2GY8omUx6sat8vD8 MjGliPFn0bf8xR9mi22JjydL8cQuozM= X-Google-Smtp-Source: ABdhPJyEQ56Y5YTS4D8f0xyqajlne6iVj2LQxaII9DPB21Rjo84MaHFiwLt0KN3y3w+IlUs1C/gv3Q== X-Received: by 2002:a5d:50cf:: with SMTP id f15mr6913238wrt.237.1633013464771; Thu, 30 Sep 2021 07:51:04 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x17sm3225346wrc.51.2021.09.30.07.51.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:51:04 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 30 Sep 2021 14:50:57 +0000 Subject: [PATCH 3/7] reset: expand test coverage for sparse checkouts Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Add new tests for `--merge` and `--keep` modes, as well as mixed reset with pathspecs both inside and outside of the sparse checkout definition. New performance test cases exercise various execution paths for `reset`. Co-authored-by: Derrick Stolee Signed-off-by: Derrick Stolee Signed-off-by: Victoria Dye --- t/perf/p2000-sparse-operations.sh | 3 + t/t1092-sparse-checkout-compatibility.sh | 107 +++++++++++++++++++++++ 2 files changed, 110 insertions(+) diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index 597626276fb..bfd332120c8 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -110,5 +110,8 @@ test_perf_on_all git add -A test_perf_on_all git add . test_perf_on_all git commit -a -m A test_perf_on_all git checkout -f - +test_perf_on_all git reset +test_perf_on_all git reset --hard +test_perf_on_all git reset -- does-not-exist test_done diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index aed8683e629..e36fb18098d 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -479,6 +479,113 @@ test_expect_success 'checkout and reset (mixed) [sparse]' ' test_sparse_match git reset update-folder2 ' +# NEEDSWORK: with mixed reset, files with differences between HEAD and +# will be added to the work tree even if outside the sparse checkout +# definition, and even if the file is modified to a state of having no local +# changes. The file is "re-ignored" if a hard reset is executed. We may want to +# change this behavior in the future and enforce that files are not written +# outside of the sparse checkout definition. +test_expect_success 'checkout and mixed reset file tracking [sparse]' ' + init_repos && + + test_all_match git checkout -b reset-test update-deep && + test_all_match git reset update-folder1 && + test_all_match git reset update-deep && + + # At this point, there are no changes in the working tree. However, + # folder1/a now exists locally (even though it is outside of the sparse + # paths). + run_on_sparse test_path_exists folder1 && + + run_on_all rm folder1/a && + test_all_match git status --porcelain=v2 && + + test_all_match git reset --hard update-deep && + run_on_sparse test_path_is_missing folder1 && + test_path_exists full-checkout/folder1 +' + +test_expect_success 'checkout and reset (merge)' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents a && + test_all_match git reset --merge deepest && + test_all_match git status --porcelain=v2 && + + test_all_match git reset --hard update-deep && + run_on_all ../edit-contents deep/a && + test_all_match test_must_fail git reset --merge deepest +' + +test_expect_success 'checkout and reset (keep)' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents a && + test_all_match git reset --keep deepest && + test_all_match git status --porcelain=v2 && + + test_all_match git reset --hard update-deep && + run_on_all ../edit-contents deep/a && + test_all_match test_must_fail git reset --keep deepest +' + +test_expect_success 'reset with pathspecs inside sparse definition' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + + test_all_match git checkout -b reset-test update-deep && + run_on_all ../edit-contents deep/a && + + test_all_match git reset base -- deep/a && + test_all_match git status --porcelain=v2 && + + test_all_match git reset base -- nonexistent-file && + test_all_match git status --porcelain=v2 && + + test_all_match git reset deepest -- deep && + test_all_match git status --porcelain=v2 +' + +test_expect_success 'reset with sparse directory pathspec outside definition' ' + init_repos && + + test_all_match git checkout -b reset-test update-deep && + test_all_match git reset --hard update-folder1 && + test_all_match git reset base -- folder1 && + test_all_match git status --porcelain=v2 +' + +test_expect_success 'reset with pathspec match in sparse directory' ' + init_repos && + + test_all_match git checkout -b reset-test update-deep && + test_all_match git reset --hard update-folder1 && + test_all_match git reset base -- folder1/a && + test_all_match git status --porcelain=v2 +' + +test_expect_success 'reset with wildcard pathspec' ' + init_repos && + + test_all_match git checkout -b reset-test update-deep && + test_all_match git reset --hard update-folder1 && + test_all_match git reset base -- \*/a && + test_all_match git status --porcelain=v2 +' + test_expect_success 'merge, cherry-pick, and rebase' ' init_repos && From patchwork Thu Sep 30 14:50:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12528601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70119C433FE for ; Thu, 30 Sep 2021 14:51:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 51B0361A53 for ; Thu, 30 Sep 2021 14:51:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351966AbhI3Ow5 (ORCPT ); Thu, 30 Sep 2021 10:52:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41140 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351935AbhI3Owt (ORCPT ); Thu, 30 Sep 2021 10:52:49 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFB1CC06176C for ; Thu, 30 Sep 2021 07:51:06 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id t8so10591913wri.1 for ; Thu, 30 Sep 2021 07:51:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=w79fXmbOvGKw0WLlbAEiKlsjPfzf+UKHarCUeckfFeI=; b=H6tQ5CUgF7XW3aM9Inw+RcOYdeTKEEjMlvBmOWNuSIgwe08pEDzGFU9Iha90I6i3mn 0++yvbny3wtnaoIc4rEuxiW9K1xPPMXIjoE5FVe1Mcf73UMJf93l0Rd20OokIPk6d18J asaxxTSj3QX+6PHvgQ0EipvY9/00d8HCJMvmu0g5BgLtI69DjlsQbC0r0HUe0p84ZFQb tnd2GhJMZynvSfbDG/8/05kdzTemFPF/jPuIYZBv/4ckYgODfYYjO0MACjaVDe8n7Yl9 cpuYS2jubCOBfn8MrNNSH0QR+jwGHMIEJS7v+EbTgS8WfynSPg5iLTefa/6h4KGKFKan 1blQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=w79fXmbOvGKw0WLlbAEiKlsjPfzf+UKHarCUeckfFeI=; b=m41TDRBLQkYClMpe5TLmO0CFdsYW2w/Byq+ceAvADJod95MuBnkccN2NbR+N99h/8g AxPmjSq4q1a+BBIdUY7Ag6BhyuV9zX3kH9VBK+5uovWoj/x3YbPUJp5pJTH4ox6ugc8K gxW9C+OtULQsfDJg/N8Qcrzw0A4YPMBFqFCL9XC+rUjxTvA1/C8jY2EzZbz2yzgg5hut j7vseAuKMJfUWemvgoGO/UdkKwRUaKHoZcaPfhg5xUuuCDo/BZ/GqVEqq6wz6s2Acplp ilYQRLgaUVixmxRtj/PFlqFADye+Z0OxOYgrLpJ6RjnIuQiG7T25PJvedq36gyETeEOg m5Yw== X-Gm-Message-State: AOAM530ivY5HzmSb8ZsLGdB0/xTl9Q2Iv5V+D1QiRxZ5nGO+RsqGRopt glGnrc+OamxofatyJYBx+Zwve4OxGyg= X-Google-Smtp-Source: ABdhPJw3gh4DA562UPyqbhJEWo4elbD2ljk2TFQZGaHCLvxAIvIiIzcKciiQlKKd13H+qjU6NfjSMA== X-Received: by 2002:adf:b304:: with SMTP id j4mr6864854wrd.160.1633013465331; Thu, 30 Sep 2021 07:51:05 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b130sm5525123wmd.44.2021.09.30.07.51.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:51:05 -0700 (PDT) Message-Id: <2d63a250637d62f8966b8b78b4f57b7c6784c967.1633013461.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 30 Sep 2021 14:50:58 +0000 Subject: [PATCH 4/7] reset: integrate with sparse index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye `reset --soft` does not modify the index, so no compatibility changes are needed for it to function without expanding the index. For all other reset modes (`--mixed`, `--hard`, `--keep`, `--merge`), the full index is explicitly expanded with `ensure_full_index` to maintain current behavior. Additionally, the `read_cache()` check verifying an uncorrupted index is moved after argument parsing and preparing the repo settings. The index is not used by the preceding argument handling, but `read_cache()` does need to be run after enabling sparse index for the command and before resetting. Signed-off-by: Victoria Dye --- builtin/reset.c | 10 +++++++--- cache-tree.c | 1 + 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index 8ffcd713720..92b9a3815c7 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -205,6 +205,7 @@ static int read_from_tree(const struct pathspec *pathspec, opt.flags.override_submodule_config = 1; opt.repo = the_repository; + ensure_full_index(&the_index); if (do_diff_cache(tree_oid, &opt)) return 1; diffcore_std(&opt); @@ -282,9 +283,6 @@ static void parse_args(struct pathspec *pathspec, } *rev_ret = rev; - if (read_cache() < 0) - die(_("index file corrupt")); - parse_pathspec(pathspec, 0, PATHSPEC_PREFER_FULL | (patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0), @@ -430,6 +428,12 @@ int cmd_reset(int argc, const char **argv, const char *prefix) if (intent_to_add && reset_type != MIXED) die(_("-N can only be used with --mixed")); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + + if (read_cache() < 0) + die(_("index file corrupt")); + /* Soft reset does not touch the index file nor the working tree * at all, but requires them in a good order. Other resets reset * the index file to the tree object we are switching to. */ diff --git a/cache-tree.c b/cache-tree.c index 90919f9e345..9be19c85b66 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -776,6 +776,7 @@ void prime_cache_tree(struct repository *r, cache_tree_free(&istate->cache_tree); istate->cache_tree = cache_tree(); + ensure_full_index(istate); prime_cache_tree_rec(r, istate->cache_tree, tree); istate->cache_changed |= CACHE_TREE_CHANGED; trace2_region_leave("cache-tree", "prime_cache_tree", the_repository); From patchwork Thu Sep 30 14:50:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12528603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26AC9C433EF for ; Thu, 30 Sep 2021 14:51:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0BA2761A60 for ; Thu, 30 Sep 2021 14:51:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351973AbhI3Ow6 (ORCPT ); Thu, 30 Sep 2021 10:52:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351915AbhI3Owu (ORCPT ); Thu, 30 Sep 2021 10:52:50 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E9DEC06176C for ; Thu, 30 Sep 2021 07:51:07 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id w29so10532472wra.8 for ; Thu, 30 Sep 2021 07:51:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=xk1RNAVfiBZq1Y54Z/enWBmL6YQ8PMyjPngQ4ycl5tE=; b=YaiuyXRFWyk4miWnH6NP4K4utGQitjpUsri7l2VItAqHuMHhkQYwiSbVIr+fY9LF9/ w5J1gpxTQ+5F8sWV5lwmkJ08xL8WaD+LegHt5vEvU4doIjGqIW4hGk4O4nq8OtS+DrKc PFg4pnz8trfV6sCDGWXT5a2dIt4iBPCIg7aTuR8tMgcJMG5/73xscLwWW6ql6q1o9Vbl Ow42CR1wbDyErROjcZ4U3SQNDzKkK9s5tDGWNz9npuA1/ZXUH5A7e7jN0+TzS+aZsftp 22+s7kLIK2w5QiU8t1IjWxUzNu/0/i3GCjbI/5gxbC17lyLf/pMCKOSMFJNkYH7aLrdK zE3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=xk1RNAVfiBZq1Y54Z/enWBmL6YQ8PMyjPngQ4ycl5tE=; b=3z1wsKK4Yj6sNvpW2pkanS8ppF55JNEyWo/DPZjc8yQLmhf26QEy26Qlr+eSd7UP9o Kf9lGJpqED9/P2OUWqtkC7pNVc0lx+mDmeRdl7+BxbWnIEerrWBm0z6Cu+EvSsISD28p 0hTdpeNQ9xWmkyM22RCfypO9uM3MloEJtLHpUUnltCVK5JYNE3e4ZpcuL1pV+n8o/XRM qgZap1axcCBZhVsC0A6CwBQ5Mme0tQBqK0xRm6QqtutnQ80iYPrJAieTnDmoj8VdeLlR oi/zPDU+mXiRWGRxI0jDlufEiJhZ6cElZ6jlWolmK11g6VM8qQ9QU2T7gyq9JoqJ2eIc qQHw== X-Gm-Message-State: AOAM531jQB3cBHxWJkpCzjklf1nCu+krKNr9rM7S42Yglz05UPSxsj0f CZhWP9M8m7KJ0xKdKqL88cJxclHT4Q0= X-Google-Smtp-Source: ABdhPJx8QYkDg3jNSFRmjnpGFCbk5Oqw3iuTpxakmn2f2F/fukx+I6rIecSYbu9+DW1E+MbWehCmEw== X-Received: by 2002:adf:e583:: with SMTP id l3mr6766346wrm.390.1633013466000; Thu, 30 Sep 2021 07:51:06 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c8sm3309866wru.30.2021.09.30.07.51.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:51:05 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 30 Sep 2021 14:50:59 +0000 Subject: [PATCH 5/7] reset: make sparse-aware (except --mixed) Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye In order to accurately reconstruct the cache tree in `prime_cache_tree_rec`, the function must determine whether the currently-processing directory in the tree is sparse or not. If it is not sparse, the tree is parsed and subtree recursively constructed. If it is sparse, no subtrees are added to the tree and the entry count is set to 1 (representing the sparse directory itself). Signed-off-by: Victoria Dye --- cache-tree.c | 44 +++++++++++++++++++++--- cache.h | 10 ++++++ read-cache.c | 22 ++++++++---- t/t1092-sparse-checkout-compatibility.sh | 15 ++++++-- 4 files changed, 78 insertions(+), 13 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index 9be19c85b66..9021669d682 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -740,15 +740,29 @@ out: return ret; } +static void prime_cache_tree_sparse_dir(struct repository *r, + struct cache_tree *it, + struct tree *tree, + struct strbuf *tree_path) +{ + + oidcpy(&it->oid, &tree->object.oid); + it->entry_count = 1; + return; +} + static void prime_cache_tree_rec(struct repository *r, struct cache_tree *it, - struct tree *tree) + struct tree *tree, + struct strbuf *tree_path) { + struct strbuf subtree_path = STRBUF_INIT; struct tree_desc desc; struct name_entry entry; int cnt; oidcpy(&it->oid, &tree->object.oid); + init_tree_desc(&desc, tree->buffer, tree->size); cnt = 0; while (tree_entry(&desc, &entry)) { @@ -757,27 +771,49 @@ static void prime_cache_tree_rec(struct repository *r, else { struct cache_tree_sub *sub; struct tree *subtree = lookup_tree(r, &entry.oid); + if (!subtree->object.parsed) parse_tree(subtree); sub = cache_tree_sub(it, entry.path); sub->cache_tree = cache_tree(); - prime_cache_tree_rec(r, sub->cache_tree, subtree); + strbuf_reset(&subtree_path); + strbuf_grow(&subtree_path, tree_path->len + entry.pathlen + 1); + strbuf_addbuf(&subtree_path, tree_path); + strbuf_add(&subtree_path, entry.path, entry.pathlen); + strbuf_addch(&subtree_path, '/'); + + /* + * If a sparse index is in use, the directory being processed may be + * sparse. To confirm that, we can check whether an entry with that + * exact name exists in the index. If it does, the created subtree + * should be sparse. Otherwise, cache tree expansion should continue + * as normal. + */ + if (r->index->sparse_index && + index_entry_exists(r->index, subtree_path.buf, subtree_path.len)) + prime_cache_tree_sparse_dir(r, sub->cache_tree, subtree, &subtree_path); + else + prime_cache_tree_rec(r, sub->cache_tree, subtree, &subtree_path); cnt += sub->cache_tree->entry_count; } } it->entry_count = cnt; + + strbuf_release(&subtree_path); } void prime_cache_tree(struct repository *r, struct index_state *istate, struct tree *tree) { + struct strbuf tree_path = STRBUF_INIT; + trace2_region_enter("cache-tree", "prime_cache_tree", the_repository); cache_tree_free(&istate->cache_tree); istate->cache_tree = cache_tree(); - ensure_full_index(istate); - prime_cache_tree_rec(r, istate->cache_tree, tree); + prime_cache_tree_rec(r, istate->cache_tree, tree, &tree_path); + strbuf_release(&tree_path); istate->cache_changed |= CACHE_TREE_CHANGED; trace2_region_leave("cache-tree", "prime_cache_tree", the_repository); } diff --git a/cache.h b/cache.h index f6295f3b048..1d3e4665562 100644 --- a/cache.h +++ b/cache.h @@ -816,6 +816,16 @@ struct cache_entry *index_file_exists(struct index_state *istate, const char *na */ int index_name_pos(struct index_state *, const char *name, int namelen); +/* + * Determines whether an entry with the given name exists within the + * given index. The return value is 1 if an exact match is found, otherwise + * it is 0. Note that, unlike index_name_pos, this function does not expand + * the index if it is sparse. If an item exists within the full index but it + * is contained within a sparse directory (and not in the sparse index), 0 is + * returned. + */ +int index_entry_exists(struct index_state *, const char *name, int namelen); + /* * Some functions return the negative complement of an insert position when a * precise match was not found but a position was found where the entry would diff --git a/read-cache.c b/read-cache.c index f5d4385c408..ea1166895f8 100644 --- a/read-cache.c +++ b/read-cache.c @@ -551,7 +551,10 @@ int cache_name_stage_compare(const char *name1, int len1, int stage1, const char return 0; } -static int index_name_stage_pos(struct index_state *istate, const char *name, int namelen, int stage) +static int index_name_stage_pos(struct index_state *istate, + const char *name, int namelen, + int stage, + int search_sparse) { int first, last; @@ -570,7 +573,7 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in first = next+1; } - if (istate->sparse_index && + if (search_sparse && istate->sparse_index && first > 0) { /* Note: first <= istate->cache_nr */ struct cache_entry *ce = istate->cache[first - 1]; @@ -586,7 +589,7 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in ce_namelen(ce) < namelen && !strncmp(name, ce->name, ce_namelen(ce))) { ensure_full_index(istate); - return index_name_stage_pos(istate, name, namelen, stage); + return index_name_stage_pos(istate, name, namelen, stage, search_sparse); } } @@ -595,7 +598,12 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in int index_name_pos(struct index_state *istate, const char *name, int namelen) { - return index_name_stage_pos(istate, name, namelen, 0); + return index_name_stage_pos(istate, name, namelen, 0, 1); +} + +int index_entry_exists(struct index_state *istate, const char *name, int namelen) +{ + return index_name_stage_pos(istate, name, namelen, 0, 0) >= 0; } int remove_index_entry_at(struct index_state *istate, int pos) @@ -1222,7 +1230,7 @@ static int has_dir_name(struct index_state *istate, */ } - pos = index_name_stage_pos(istate, name, len, stage); + pos = index_name_stage_pos(istate, name, len, stage, 1); if (pos >= 0) { /* * Found one, but not so fast. This could @@ -1322,7 +1330,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e strcmp(ce->name, istate->cache[istate->cache_nr - 1]->name) > 0) pos = index_pos_to_insert_pos(istate->cache_nr); else - pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce)); + pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), 1); /* existing match? Just replace it. */ if (pos >= 0) { @@ -1357,7 +1365,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e if (!ok_to_replace) return error(_("'%s' appears as both a file and as a directory"), ce->name); - pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce)); + pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), 1); pos = -pos-1; } return pos + 1; diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index e36fb18098d..0b6ff0de17d 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -786,9 +786,9 @@ test_expect_success 'sparse-index is not expanded' ' ensure_not_expanded checkout - && ensure_not_expanded switch rename-out-to-out && ensure_not_expanded switch - && - git -C sparse-index reset --hard && + ensure_not_expanded reset --hard && ensure_not_expanded checkout rename-out-to-out -- deep/deeper1 && - git -C sparse-index reset --hard && + ensure_not_expanded reset --hard && ensure_not_expanded restore -s rename-out-to-out -- deep/deeper1 && echo >>sparse-index/README.md && @@ -798,6 +798,17 @@ test_expect_success 'sparse-index is not expanded' ' echo >>sparse-index/untracked.txt && ensure_not_expanded add . && + for ref in update-deep update-folder1 update-folder2 update-deep + do + echo >>sparse-index/README.md && + ensure_not_expanded reset --hard $ref || return 1 + done && + + ensure_not_expanded reset --hard update-deep && + ensure_not_expanded reset --keep base && + ensure_not_expanded reset --merge update-deep && + ensure_not_expanded reset --hard && + ensure_not_expanded checkout -f update-deep && test_config -C sparse-index pull.twohead ort && ( From patchwork Thu Sep 30 14:51:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12528605 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03A47C433F5 for ; Thu, 30 Sep 2021 14:51:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D7E8F61A55 for ; Thu, 30 Sep 2021 14:51:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351945AbhI3OxC (ORCPT ); Thu, 30 Sep 2021 10:53:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351940AbhI3Owu (ORCPT ); Thu, 30 Sep 2021 10:52:50 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10A42C06176A for ; Thu, 30 Sep 2021 07:51:08 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id u18so10551141wrg.5 for ; Thu, 30 Sep 2021 07:51:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=qAk86aBmKlCy4z8JyEWL95ajU/tKM5J+bszwzD+1GQM=; b=XmCzfMXu41eGGjl1mfGJK8I0UXCHmvSyUAlMxgfF3UH1cu28cD2mqk7ALhlP7WlILZ hn0rDsgeg2VbcHpEgplNPMTMk2uWZ+aUqN4arBQUqw22VFwsNgo8KJypzgwlvfKruQtt kJvlGqFZEc91G7UqcUb1e4+pK1XvZXK4U9DRosTIUlUrP8ZlgfvJ1gu+UZMKsLILa/iA /tO3ig6defc3iKeryqc92nBfLv48OkH4Y+txvTkg0kH6WAE7Mqf87IYsRucy935wbUms WFX3/1H8MmbjpgfdIccja/6XUTq1wuq/pRCuwbGNvhx8OzMN42D4IiP+B4x6XBGdq1F5 fDag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=qAk86aBmKlCy4z8JyEWL95ajU/tKM5J+bszwzD+1GQM=; b=gSK2j6wVH/G50f+yxf6EY0Z1wvf7NTWs5AF3K5zv0tyXUAz/bIwt5LiXktoTJ/NBzS rYLje/2shiM1o0ITOmcSThQUSbFZ6kq4Qnec0aLhg5Ume8R9jbowVWtFCTq3yRgghp8w 5ZQF/41CDDOW4TNkfDHR7SuzFpbeIfN7wNz04P9LJUTYkHtH7cOv045QEtwccadqcrfg DOFcAomEeXrz7WGBCt73yGCUxoVm55AxgqHzfRHrvtopD/TkLcg5ARXwvro3MU3W5aoA qgk5zZ+Vxq3Nj/MFDDguheKZZFx63qWDCapvPqxtPi3/5lUDKli/fkp28xbc/KNUWS4n Pnqw== X-Gm-Message-State: AOAM533RCON6EzMTw+2htzSjupMpcaEZs1/hOoogb2uVmAlDFOElNMoq LcVHb45EPq7rCksyjvGrw8dclzxAr6M= X-Google-Smtp-Source: ABdhPJxqniqvKmHMNqrqzzQuP4zVoJ5nZcRo1XULFPw2O28mP0/+dVMqurVJApdyDYiGHhvKM4hXhw== X-Received: by 2002:adf:9bd3:: with SMTP id e19mr772480wrc.136.1633013466614; Thu, 30 Sep 2021 07:51:06 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c132sm5120291wma.22.2021.09.30.07.51.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:51:06 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 30 Sep 2021 14:51:00 +0000 Subject: [PATCH 6/7] reset: make --mixed sparse-aware Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye Sparse directory entries are "diffed" as trees in `diff_cache` (used internally by `reset --mixed`), following a code path separate from individual file handling. The use of `diff_tree_oid` there requires setting explicit `change` and `add_remove` functions to process the internal contents of a sparse directory. Additionally, the `recursive` diff option handles cases in which `reset --mixed` must diff/merge files that are nested multiple levels deep in a sparse directory. Signed-off-by: Victoria Dye --- builtin/reset.c | 30 +++++++++++++++++++++++- t/t1092-sparse-checkout-compatibility.sh | 13 +++++++++- 2 files changed, 41 insertions(+), 2 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index 92b9a3815c7..2d95ce76f20 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -196,6 +196,8 @@ static int read_from_tree(const struct pathspec *pathspec, int intent_to_add) { struct diff_options opt; + unsigned int i; + char *skip_worktree_seen = NULL; memset(&opt, 0, sizeof(opt)); copy_pathspec(&opt.pathspec, pathspec); @@ -203,9 +205,35 @@ static int read_from_tree(const struct pathspec *pathspec, opt.format_callback = update_index_from_diff; opt.format_callback_data = &intent_to_add; opt.flags.override_submodule_config = 1; + opt.flags.recursive = 1; opt.repo = the_repository; + opt.change = diff_change; + opt.add_remove = diff_addremove; + + /* + * When pathspec is given for resetting a cone-mode sparse checkout, it may + * identify entries that are nested in sparse directories, in which case the + * index should be expanded. For the sake of efficiency, this check is + * overly-cautious: anything with a wildcard or a magic prefix requires + * expansion, as well as literal paths that aren't in the sparse checkout + * definition AND don't match any directory in the index. + */ + if (pathspec->nr && the_index.sparse_index) { + if (pathspec->magic || pathspec->has_wildcard) { + ensure_full_index(&the_index); + } else { + for (i = 0; i < pathspec->nr; i++) { + if (!path_in_cone_mode_sparse_checkout(pathspec->items[i].original, &the_index) && + !matches_skip_worktree(pathspec, i, &skip_worktree_seen)) { + ensure_full_index(&the_index); + break; + } + } + } + } + + free(skip_worktree_seen); - ensure_full_index(&the_index); if (do_diff_cache(tree_oid, &opt)) return 1; diffcore_std(&opt); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 0b6ff0de17d..c9b9ef4992c 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -801,14 +801,25 @@ test_expect_success 'sparse-index is not expanded' ' for ref in update-deep update-folder1 update-folder2 update-deep do echo >>sparse-index/README.md && + ensure_not_expanded reset --mixed $ref ensure_not_expanded reset --hard $ref || return 1 done && ensure_not_expanded reset --hard update-deep && ensure_not_expanded reset --keep base && ensure_not_expanded reset --merge update-deep && - ensure_not_expanded reset --hard && + ensure_not_expanded reset base -- deep/a && + ensure_not_expanded reset base -- nonexistent-file && + ensure_not_expanded reset deepest -- deep && + + # Although folder1 is outside the sparse definition, it exists as a + # directory entry in the index, so it will be reset without needing to + # expand the full index. + ensure_not_expanded reset --hard update-folder1 && + ensure_not_expanded reset base -- folder1 && + + ensure_not_expanded reset --hard update-deep && ensure_not_expanded checkout -f update-deep && test_config -C sparse-index pull.twohead ort && ( From patchwork Thu Sep 30 14:51:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 12528607 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBE39C433EF for ; Thu, 30 Sep 2021 14:51:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C374B61A55 for ; Thu, 30 Sep 2021 14:51:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351957AbhI3OxD (ORCPT ); Thu, 30 Sep 2021 10:53:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351588AbhI3Owv (ORCPT ); Thu, 30 Sep 2021 10:52:51 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E35BC06176D for ; Thu, 30 Sep 2021 07:51:09 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id 205-20020a1c01d6000000b0030cd17ffcf8so8670825wmb.3 for ; Thu, 30 Sep 2021 07:51:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=2HRlxeP3tz/uNY5+unoDO9/v7e3CHEEqIbEgAsKvEsY=; b=TJdDPhNdD2GGzb+ll2IqaLMvZWfxOf6OtyRjlMgLyi0UvAhBlqV6nIEAITZS8o+u7s XW0/+pctnztXaEXGW9ldtaCKbb0sn450Sq4K2qUxKjW8T51bvqzFdKiF35sD4itamjef 9RYWF+odCfkkINGF4HF9glsvk6S6L/rLYg8xV6BMDaDYK+tCVjv/orh/ZA9mvYBzIFnJ QIbWkisKBMJjHQagHR8kS1VdrrjoCft6ImB5PQs26cGzb0mhHAabZX6wLRXiqWWn2Dxd Rn1w7ayHxYnb7lCyQkpKwV4tGGwyD2wIcAp7HFVy1ZYvNp6gKivdHPTGyeayFrBa7NAE syQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=2HRlxeP3tz/uNY5+unoDO9/v7e3CHEEqIbEgAsKvEsY=; b=f7JfqazWVL5eBBHjM6v/zGi/aGGpXWhms8BQ8qsS4YHPKekLCWwy84pW2l+mwvuhuj 8zF1D4LEhh16prJ3JHgV5UChduzgiAJhZj/AWT1HNYsBAKEyXkaM5L97ubBCi2DNpuYa V4okjLPWo7xp5PzQKHTrO3PmOsz5/kmLs3x84rgN86s7+3myDl084R9uJhcBoTtVkmD1 h/ZafbqyNpUBFpGzZFB1PDVk8LFKZRC5qpv3rd6j4tAzHl0hSpB/bwTM7QrIlnLFnVTa je4XQZKhKCzwTatnU7n893LY73lpr+phKPd9NUEtzOHv85JdJLTR5u50kEkCeQGqxBct Sj7A== X-Gm-Message-State: AOAM531Grxj9uoCxIGSo+GUk4BYRD1YkQ7j2qVkBijzTbdP1ewprpYpW hrGQ8d9urG78iT9lTsG2qrz1cBhhDq8= X-Google-Smtp-Source: ABdhPJytIMA/tirUJ/BTtebDz6c7MXieobae8KCc0YQ0u3AF1u39IOayRV47Bm1dtIUxQecfC6up6g== X-Received: by 2002:a7b:c842:: with SMTP id c2mr16270216wml.93.1633013467213; Thu, 30 Sep 2021 07:51:07 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j27sm3792093wms.6.2021.09.30.07.51.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:51:06 -0700 (PDT) Message-Id: <8637ec1660ebb1b19f1f73c38debb7b098187e6f.1633013461.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 30 Sep 2021 14:51:01 +0000 Subject: [PATCH 7/7] unpack-trees: improve performance of next_cache_entry Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com, Victoria Dye , Victoria Dye Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Victoria Dye From: Victoria Dye To find the first non-unpacked cache entry, `next_cache_entry` iterates through index, starting at `cache_bottom`. The performance of this in full indexes is helped by `cache_bottom` advancing with each invocation of `mark_ce_used` (called by `unpack_index_entry`). However, the presence of sparse directories can prevent the `cache_bottom` from advancing in a sparse index case, effectively forcing `next_cache_entry` to search from the beginning of the index each time it is called. The `cache_bottom` must be preserved for the sparse index (see 17a1bb570b (unpack-trees: preserve cache_bottom, 2021-07-14)). Therefore, to retain the benefit `cache_bottom` provides in non-sparse index cases, a separate `hint` position indicates the first position `next_cache_entry` should search, updated each execution with a new position. The performance of `git reset -- does-not-exist` (testing the "worst case" in which all entries in the index are unpacked with `next_cache_entry`) is significantly improved for the sparse index case: Test before after ------------------------------------------------------ (full-v3) 0.79(0.38+0.30) 0.91(0.43+0.34) +15.2% (full-v4) 0.80(0.38+0.29) 0.85(0.40+0.35) +6.2% (sparse-v3) 0.76(0.43+0.69) 0.44(0.08+0.67) -42.1% (sparse-v4) 0.71(0.40+0.65) 0.41(0.09+0.65) -42.3% Signed-off-by: Victoria Dye --- unpack-trees.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/unpack-trees.c b/unpack-trees.c index 8ea0a542da8..b94733de6be 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -645,17 +645,24 @@ static void mark_ce_used_same_name(struct cache_entry *ce, } } -static struct cache_entry *next_cache_entry(struct unpack_trees_options *o) +static struct cache_entry *next_cache_entry(struct unpack_trees_options *o, int *hint) { const struct index_state *index = o->src_index; int pos = o->cache_bottom; + if (*hint > pos) + pos = *hint; + while (pos < index->cache_nr) { struct cache_entry *ce = index->cache[pos]; - if (!(ce->ce_flags & CE_UNPACKED)) + if (!(ce->ce_flags & CE_UNPACKED)) { + *hint = pos + 1; return ce; + } pos++; } + + *hint = pos; return NULL; } @@ -1365,12 +1372,13 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str /* Are we supposed to look at the index too? */ if (o->merge) { + int hint = -1; while (1) { int cmp; struct cache_entry *ce; if (o->diff_index_cached) - ce = next_cache_entry(o); + ce = next_cache_entry(o, &hint); else ce = find_cache_entry(info, p); @@ -1690,7 +1698,7 @@ static int verify_absent(const struct cache_entry *, int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options *o) { struct repository *repo = the_repository; - int i, ret; + int i, hint, ret; static struct cache_entry *dfc; struct pattern_list pl; int free_pattern_list = 0; @@ -1763,13 +1771,15 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options info.pathspec = o->pathspec; if (o->prefix) { + hint = -1; + /* * Unpack existing index entries that sort before the * prefix the tree is spliced into. Note that o->merge * is always true in this case. */ while (1) { - struct cache_entry *ce = next_cache_entry(o); + struct cache_entry *ce = next_cache_entry(o, &hint); if (!ce) break; if (ce_in_traverse_path(ce, &info)) @@ -1790,8 +1800,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options /* Any left-over entries in the index? */ if (o->merge) { + hint = -1; while (1) { - struct cache_entry *ce = next_cache_entry(o); + struct cache_entry *ce = next_cache_entry(o, &hint); if (!ce) break; if (unpack_index_entry(ce, o) < 0)