From patchwork Mon Jul 26 15:18:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12399903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1167AC4338F for ; Mon, 26 Jul 2021 15:18:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E8B7F60F51 for ; Mon, 26 Jul 2021 15:18:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234925AbhGZOiY (ORCPT ); Mon, 26 Jul 2021 10:38:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234355AbhGZOiW (ORCPT ); Mon, 26 Jul 2021 10:38:22 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3077C061764 for ; Mon, 26 Jul 2021 08:18:50 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id m20-20020a05600c4f54b029024e75a15716so233760wmq.2 for ; Mon, 26 Jul 2021 08:18:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ZuvaER8Jpu1eePGIIP6OxC71SlC/J0HLvbS0opCYx5U=; b=l00/pF834aP1Jv9ph62yUbDag/GyXek/43rDKsXTSuVjnc5AB9t6FuGbm4VHMF48gA +tH+XjpzNrVpy0czg11ezVeNgrtb26IHS3+bYMqYCQP8Qfu1ZMAmqcqD+F42p9gjC7T7 XOOdN56vyZXlW0Hi4ItJz+IkMxsjLbe051d4Qbwb7lpOceIm33Cc+HXuLK98bo2ugXrG 9KfspZwpQK2IgCbkiYnTbQpcWEertaCUcG6VxM2Nzszi8+rch+58kgXkXYnQmdFoSUBD tRQb/JJ7qEexW3efg/o/jhAL3Q9tujv1qb/cGsU7GuLkAjudHbFcAICnZxIXi4v1eQS2 lZTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ZuvaER8Jpu1eePGIIP6OxC71SlC/J0HLvbS0opCYx5U=; b=Y0kdd0GZ+FnUVPpz1qGo6ROSWtT03BrFpapq02PCvYduXluOQOz8XQZJ6RO3nFkQW/ hrD1EU0XkjDZuA9PXp/J7J1boX7eWPL3AnWqRQSzEzDe8DDIlzjreE8MRgg68FD49zAN JMOVVR82eEx+iO5Fm599aXnjBSyj8FLlVLMQ83M7ngYn/rXzrvc+cG9Kz6IvJ5s7yb1q U4ZoSA3JZIti5iCsH2Me12Hrt3IKuiE2Eo3o1TH1NWinUy9P5UEY6CtwR2j1UhRSugx9 3JQd9D9FhrNluRW/MdXhBZgr7b1fYTFQfZPBcOuYH9s+wszp4QgrdXCOt+RpFiFKV0KM WisQ== X-Gm-Message-State: AOAM531Hh307foFETYEyjovTk7uu/SehD3VntZJv/idQ4igTOotCEEBE ju1etlvRJgiT0HQPtrk9Vd6sKGhk3JY= X-Google-Smtp-Source: ABdhPJwLKOw5KKgk8C4LZRruu+iYw2iR7oroVzVEIuo555S9VISos759+Px6IvxODESRpRHTShE6gA== X-Received: by 2002:a05:600c:a05:: with SMTP id z5mr14164257wmp.73.1627312729597; Mon, 26 Jul 2021 08:18:49 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f15sm80245wre.66.2021.07.26.08.18.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jul 2021 08:18:49 -0700 (PDT) Message-Id: <8f2fd9370fe078923256381ddd6ff4534d736ec7.1627312727.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 26 Jul 2021 15:18:43 +0000 Subject: [PATCH v2 1/5] t1092: test merge conflicts outside cone Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Eric Sunshine , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Signed-off-by: Derrick Stolee --- t/t1092-sparse-checkout-compatibility.sh | 38 ++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 91e30d6ec22..47f8e5e54e3 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -114,6 +114,16 @@ test_expect_success 'setup' ' git add . && git commit -m "file to dir" && + for side in left right + do + git checkout -b merge-$side base && + echo $side >>deep/deeper2/a && + echo $side >>folder1/a && + echo $side >>folder2/a && + git add . && + git commit -m "$side" || return 1 + done && + git checkout -b deepest base && echo "updated deepest" >deep/deeper1/deepest/a && git commit -a -m "update deepest" && @@ -482,6 +492,34 @@ test_expect_success 'merge' ' test_all_match git rev-parse HEAD^{tree} ' +test_expect_success 'merge with conflict outside cone' ' + init_repos && + + test_all_match git checkout -b merge-tip merge-left && + test_all_match git status --porcelain=v2 && + test_all_match test_must_fail git merge -m merge merge-right && + test_all_match git status --porcelain=v2 && + + # Resolve the conflict in different ways: + # 1. Revert to the base + test_all_match git checkout base -- deep/deeper2/a && + test_all_match git status --porcelain=v2 && + + # 2. Add the file with conflict markers + test_all_match git add folder1/a && + test_all_match git status --porcelain=v2 && + + # 3. Rename the file to another sparse filename and + # accept conflict markers as resolved content. + run_on_all mv folder2/a folder2/z && + test_all_match git add folder2 && + test_all_match git status --porcelain=v2 && + + test_all_match git merge --continue && + test_all_match git status --porcelain=v2 && + test_all_match git rev-parse HEAD^{tree} +' + test_expect_success 'merge with outside renames' ' init_repos && From patchwork Mon Jul 26 15:18:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12399907 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24BF4C432BE for ; Mon, 26 Jul 2021 15:19:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 040AA60F51 for ; Mon, 26 Jul 2021 15:19:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235056AbhGZOil (ORCPT ); Mon, 26 Jul 2021 10:38:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234778AbhGZOiX (ORCPT ); Mon, 26 Jul 2021 10:38:23 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C26BEC061760 for ; Mon, 26 Jul 2021 08:18:51 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id l4so11542465wrs.4 for ; Mon, 26 Jul 2021 08:18:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=CbrIICt1ZseenszcmvctOJ+a8mIFfyJkW43dB77sJ+w=; b=cyM7k/pX5lxB5KYAkbSeqxp+LyXJWm42MjKPeKEGJxnYwILBbhT9tNY8Rpyj8MsENl Li5JBGgmq9N4XzL5IoVViXhpFGXZa51I9y6zG16JwOq0mBZAA4agiI4IZBw3OsZ8b/gC k+1kyhQX+nReCKJ53yxxxfzo3y0mbqwb/j/5kCa2xsvNhYUo4uaoXWeKZkAsA2Bni2wJ 5oHyxfnqU7XicNpZozDjV9DetMfcZWYnhiQY2o4PqT2fcjCgj15trc5UGZryF3GI2EF/ QWrYLmqD3LM1aSQah+WvSATbWH1CSVkRAHoRGjI65TykwrrEB/u3Gyzz3jB+yPOp4eRA /gpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=CbrIICt1ZseenszcmvctOJ+a8mIFfyJkW43dB77sJ+w=; b=dnvkUm5CetmDcEIWjRCZ/FR5Gd8BHOQB+oDffihiQwdB7KqeDnUZ7NK+Q1yn+WEKhu 6XvBy7z7RY5ioYDN7DBAWTEHdBefT7rouoT00HF8oMtBP7nCrnhtfT2XPqrIEqyvr8Ml pzaMVq5ilLkza4nupzpSIIwD37VHK0ReYJ1hOMUW5CvfqhOk/dt6paQolAkY5kgvutm3 YRbOMF8+7kZkBgIOFLWOdJalI24aEHYlaE+G1buEc/FLXkCoSvJAgs1ssAzE5GWFeIQy z+A44DJss/Eg9CUyf5ikhnjr5Nn7eFGWfvHGAWD8GC9DPth+0uZCTMF5IaQjWisZ7FXl xVpg== X-Gm-Message-State: AOAM53298RD+GHJVjLlVNTST4+RySlCO5YKLcG6Y2QmK5dRO3zPYA8lx IjEaylkiVQiFOYkZnzJyrwTxFVSCKDw= X-Google-Smtp-Source: ABdhPJzBPPjJ1rUf8zykw5hmRhfTnPyGHBVWETdl8JDxwKFb9LukRQBnRMmPa0pUPjFJmNYD62lG4Q== X-Received: by 2002:a5d:6481:: with SMTP id o1mr20505732wri.164.1627312730426; Mon, 26 Jul 2021 08:18:50 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s14sm78658wmc.25.2021.07.26.08.18.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jul 2021 08:18:49 -0700 (PDT) Message-Id: <6e43f118fa0b7a6ba2e841f75a90406d847d9f09.1627312727.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 26 Jul 2021 15:18:44 +0000 Subject: [PATCH v2 2/5] add: allow operating on a sparse-only index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Eric Sunshine , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Disable command_requires_full_index for 'git add'. This does not require any additional removals of ensure_full_index(). The main reason is that 'git add' discovers changes based on the pathspec and the worktree itself. These are then inserted into the index directly, and calls to index_name_pos() or index_file_exists() already call expand_to_path() at the appropriate time to support a sparse-index. Add a test to check that 'git add -A' and 'git add ' does not expand the index at all, as long as is not within a sparse directory. This does not help the global 'git add .' case. We can measure the improvement using p2000-sparse-operations.sh with these results: Test HEAD~1 HEAD ------------------------------------------------------------------------------ 2000.6: git add -A (full-index-v3) 0.35(0.30+0.05) 0.37(0.29+0.06) +5.7% 2000.7: git add -A (full-index-v4) 0.31(0.26+0.06) 0.33(0.27+0.06) +6.5% 2000.8: git add -A (sparse-index-v3) 0.57(0.53+0.07) 0.05(0.04+0.08) -91.2% 2000.9: git add -A (sparse-index-v4) 0.58(0.55+0.06) 0.05(0.05+0.06) -91.4% While the 91% improvement seems impressive, it's important to recognize that previously we had significant overhead for expanding the sparse-index. Comparing to the full index case, 'git add -A' goes from 0.37s to 0.05s, which is "only" an 86% improvement. This modification to 'git add' creates some behavior change depending on the use of a sparse index. We modify a test in t1092 to demonstrate these changes which will be remedied in future changes. Signed-off-by: Derrick Stolee --- builtin/add.c | 3 +++ t/t1092-sparse-checkout-compatibility.sh | 25 +++++++++++++++++------- 2 files changed, 21 insertions(+), 7 deletions(-) diff --git a/builtin/add.c b/builtin/add.c index b773b5a4993..c76e6ddd359 100644 --- a/builtin/add.c +++ b/builtin/add.c @@ -528,6 +528,9 @@ int cmd_add(int argc, const char **argv, const char *prefix) add_new_files = !take_worktree_changes && !refresh_only && !add_renormalize; require_pathspec = !(take_worktree_changes || (0 < addremove_explicit)); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + hold_locked_index(&lock_file, LOCK_DIE_ON_ERROR); /* diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 47f8e5e54e3..19d38f18ed6 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -340,21 +340,27 @@ test_expect_success 'status/add: outside sparse cone' ' test_sparse_match git status --porcelain=v2 && - # This "git add folder1/a" fails with a warning - # in the sparse repos, differing from the full - # repo. This is intentional. + # Adding the path outside of the sparse-checkout cone should fail. test_sparse_match test_must_fail git add folder1/a && - test_sparse_match test_must_fail git add --refresh folder1/a && - test_all_match git status --porcelain=v2 && + + test_must_fail git -C sparse-checkout add --refresh folder1/a 2>sparse-checkout-err && + test_must_fail git -C sparse-index add --refresh folder1/a 2>sparse-index-err && + # NEEDSWORK: A sparse index changes the error message. + ! test_cmp sparse-checkout-err sparse-index-err && + + # NEEDSWORK: Adding a newly-tracked file outside the cone succeeds + test_sparse_match git add folder1/new && test_all_match git add . && test_all_match git status --porcelain=v2 && test_all_match git commit -m folder1/new && + test_all_match git rev-parse HEAD^{tree} && run_on_all ../edit-contents folder1/newer && test_all_match git add folder1/ && test_all_match git status --porcelain=v2 && - test_all_match git commit -m folder1/newer + test_all_match git commit -m folder1/newer && + test_all_match git rev-parse HEAD^{tree} ' test_expect_success 'checkout and reset --hard' ' @@ -636,7 +642,12 @@ test_expect_success 'sparse-index is not expanded' ' git -C sparse-index reset --hard && ensure_not_expanded checkout rename-out-to-out -- deep/deeper1 && git -C sparse-index reset --hard && - ensure_not_expanded restore -s rename-out-to-out -- deep/deeper1 + ensure_not_expanded restore -s rename-out-to-out -- deep/deeper1 && + + echo >>sparse-index/README.md && + ensure_not_expanded add -A && + echo >>sparse-index/extra.txt && + ensure_not_expanded add extra.txt ' # NEEDSWORK: a sparse-checkout behaves differently from a full checkout From patchwork Mon Jul 26 15:18:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12399911 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49F11C4320E for ; Mon, 26 Jul 2021 15:19:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3212E60F55 for ; Mon, 26 Jul 2021 15:19:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235081AbhGZOin (ORCPT ); Mon, 26 Jul 2021 10:38:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234941AbhGZOiZ (ORCPT ); Mon, 26 Jul 2021 10:38:25 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B0F7C061757 for ; Mon, 26 Jul 2021 08:18:53 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id d131-20020a1c1d890000b02902516717f562so230886wmd.3 for ; Mon, 26 Jul 2021 08:18:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=aU4ldpjK0Tx5Ez2WDHqvYUwk/EaPbybiuDakOe7VSrs=; b=nQYrgKcq1LmqnjCgTmeJIJI1gFjQVO6r0v+26iknW/M6ZIa4VNRIIlCJZ+NMHVYeOF MSg7FqVM6nRH8fXqGcxscUbdEgbUKp7KKIZjsPN2g6HF/p+l89pMT/wRDLM5Q95Z8wqR QAQemZvf8ft3W+QNVEgUzOeG1a9Q6y6ItCLts2KW/ondoFpnoVjxfPPxkj+qzwukcQjO fHVojA7GEnJ+k8I0a1w4m8kQDAK5XbKotXwU3yujlynhuOEKkuqgchHsn3gyq8U2kVc2 dse0xEzsGPaj8EGCbY3W71jsCi24wcE1nXJ3s1y1uzV0dYw5bjx/ZKJ77YVuLuYCzxYU i5WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=aU4ldpjK0Tx5Ez2WDHqvYUwk/EaPbybiuDakOe7VSrs=; b=i5oQIAk+G2OA7i4jA7KaTMaHj+Nd0fBkqtDzo80Gzg8vWf0QbqIQR/lLkxnQKLUVZS SWVGGkpJQ+vTM32HmTNJ1rADDArcRUCrYYPhR1rv/6Y6eJlT8/o9/rMMTDAWe6NeP7sy 2c5b4Wy+tUx0vqSWBYwJHM9Voo7XNIPG1ZtgEKI2GtXW9TbOvdRDXo3psfwl9hR77lbn R5vi+k72XzDDovzpYLSfgnR4hCYEHFFscSotFpkUgUg7A1ydFS1U6v6fBPNWfwQxYT/z BpZIJUuf6zGRQeMdZTvYPTnsv+uwaWGX+6rkzlxjGjpbw9J+IN3j/ni0ROE/iyqsBPfG lXlw== X-Gm-Message-State: AOAM530PXDo+EsZV/S9x2QuuzvZeEBgpCPF7ASu7s1HrkZqeXZ4PU09r +lPQID8X7kS3FIvHg67n6rua0AFXrS0= X-Google-Smtp-Source: ABdhPJwHUrTUVB2ZJ2WgfRDsbu62e4M0DacPjd+MO3MhiNoDrNjd6WpuXXo4REg+sZlMvhmpIupnTw== X-Received: by 2002:a1c:a510:: with SMTP id o16mr14824010wme.176.1627312731711; Mon, 26 Jul 2021 08:18:51 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 125sm12939419wmb.12.2021.07.26.08.18.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jul 2021 08:18:51 -0700 (PDT) Message-Id: <2ae91e0af29bcc5ba118cba4ba051deefd3b9764.1627312727.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 26 Jul 2021 15:18:45 +0000 Subject: [PATCH v2 3/5] pathspec: stop calling ensure_full_index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Eric Sunshine , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The add_pathspec_matches_against_index() focuses on matching a pathspec to file entries in the index. This already works correctly for its only use: checking if untracked files exist in the index. The compatibility checks in t1092 already test that 'git add ' works for a directory outside of the sparse cone. That provides coverage for removing this guard. This finalizes our ability to run 'git add .' without expanding a sparse index to a full one. This is evidenced by an update to t1092 and by these performance numbers for p2000-sparse-operations.sh: Test HEAD~1 HEAD -------------------------------------------------------------------------------- 2000.10: git add . (full-index-v3) 0.37(0.28+0.07) 0.36(0.27+0.06) -2.7% 2000.11: git add . (full-index-v4) 0.33(0.26+0.06) 0.32(0.28+0.05) -3.0% 2000.12: git add . (sparse-index-v3) 0.57(0.53+0.07) 0.06(0.06+0.07) -89.5% 2000.13: git add . (sparse-index-v4) 0.57(0.53+0.07) 0.05(0.03+0.09) -91.2% While the ~90% improvement is shown by the test results, it is worth noting that expanding the sparse index was adding overhead in previous commits. Comparing to the full index case, we see the performance go from 0.33s to 0.05s, an 85% improvement. Signed-off-by: Derrick Stolee --- pathspec.c | 2 -- t/t1092-sparse-checkout-compatibility.sh | 7 +++---- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/pathspec.c b/pathspec.c index 08f8d3eedc3..44306fdaca2 100644 --- a/pathspec.c +++ b/pathspec.c @@ -37,8 +37,6 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec, num_unmatched++; if (!num_unmatched) return; - /* TODO: audit for interaction with sparse-index. */ - ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { const struct cache_entry *ce = istate->cache[i]; if (sw_action == PS_IGNORE_SKIP_WORKTREE && ce_skip_worktree(ce)) diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 19d38f18ed6..50dce0e0f99 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -322,9 +322,6 @@ test_expect_success 'commit including unstaged changes' ' test_expect_success 'status/add: outside sparse cone' ' init_repos && - # adding a "missing" file outside the cone should fail - test_sparse_match test_must_fail git add folder1/a && - # folder1 is at HEAD, but outside the sparse cone run_on_sparse mkdir folder1 && cp initial-repo/folder1/a sparse-checkout/folder1/a && @@ -647,7 +644,9 @@ test_expect_success 'sparse-index is not expanded' ' echo >>sparse-index/README.md && ensure_not_expanded add -A && echo >>sparse-index/extra.txt && - ensure_not_expanded add extra.txt + ensure_not_expanded add extra.txt && + echo >>sparse-index/untracked.txt && + ensure_not_expanded add . ' # NEEDSWORK: a sparse-checkout behaves differently from a full checkout From patchwork Mon Jul 26 15:18:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12399909 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74416C4338F for ; Mon, 26 Jul 2021 15:19:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4FA2660D07 for ; Mon, 26 Jul 2021 15:19:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235073AbhGZOim (ORCPT ); Mon, 26 Jul 2021 10:38:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40694 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234921AbhGZOiZ (ORCPT ); Mon, 26 Jul 2021 10:38:25 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CECB8C061760 for ; Mon, 26 Jul 2021 08:18:53 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id q3so11616224wrx.0 for ; Mon, 26 Jul 2021 08:18:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=gqAHQC4rWZ2nY5de6/Y5DYkEUm2gFg2s3OsnrG2b+/s=; b=h9jnvq++mTnZ4g3xkOfR4778EtN/xCJ2PIm3UB6vBs3lBu/E0ADtuPx28hcNB+pZxG VAxphcIt20jXZA+YCXFk8u2S3Q+flBPVXUC3Ew5maM+AGaPlrDG18cDHErotkOsLJi+b qba1MLk1a/xb208ufsMwCGayF8hpaqrzOTaCmVZBLOrccbM8Iv6Ra78QQMKHxSUzksbn 1Fr5FvzBVecYcrSoX2dYnI356tT7arjDR6ts6PR9j64gpnTBHF6655p5NH5/guFFEKfr T2wcH5R5FW6t6ffx3V13oSFJ7ixavm3T0PEnk+zxcQbXmPgyQ1U7udu8KEfVKdqXlJ6d juTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=gqAHQC4rWZ2nY5de6/Y5DYkEUm2gFg2s3OsnrG2b+/s=; b=mzvtmpPpB5dSInBWdyyHYryDOt8EeQAjU+crizKztezXgSXDae7qbZ5ZBnh+p3XtW3 7mc/q5gh7+vcjjMfEvTS9OAIqU3gbBdn4XGuYUjPhFWjSZHEtvAzmyTuXd2PrRvkKboX ORHp8Pw7IjIfWhoyjN2NrwGIz7VRHpCk8iPHzDKzpHJpwzFdMPJzfTkRU+8ZoTyNjkBf YZXENmjSJgjZd8wZJw3k0es5eepo3tTtOKCUEmjXzpatEdkOtA/HZgsVvKgNrfwmLwZb DkQrZxPqi6AHh5TjmGmjKzC4gl9oaUxTu6cJmOjGI2Ap7kndJSYRAzte+Xs8I9fm6HLx k6Cg== X-Gm-Message-State: AOAM532f1LT2Su4Fc5CYM//yDEFEeRPyeBNXLKQO50OyzQrqKPJuUVRV nGMC2Fl363uiELNCVJEdFlas6+tXDY0= X-Google-Smtp-Source: ABdhPJyuIWQX84o4zfB5pQkJG77up+k4zNdyEB2NA+9tzBALx/m1y8IL+aEDale96YMj/amHGYMq9g== X-Received: by 2002:adf:fc0c:: with SMTP id i12mr9936272wrr.373.1627312732492; Mon, 26 Jul 2021 08:18:52 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c15sm85301wmr.28.2021.07.26.08.18.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jul 2021 08:18:52 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 26 Jul 2021 15:18:46 +0000 Subject: [PATCH v2 4/5] add: ignore outside the sparse-checkout in refresh() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Eric Sunshine , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Since b243012 (refresh_index(): add flag to ignore SKIP_WORKTREE entries, 2021-04-08), 'git add --refresh ' will output a warning message when the path is outside the sparse-checkout definition. The implementation of this warning happened in parallel with the sparse-index work to add ensure_full_index() calls throughout the codebase. Update this loop to have the proper logic that checks to see if the pathspec is outside the sparse-checkout definition. This avoids the need to expand the sparse directory entry and determine if the path is tracked, untracked, or ignored. We simply avoid updating the stat() information because there isn't even an entry that matches the path! Signed-off-by: Derrick Stolee --- builtin/add.c | 10 +++++++++- t/t1092-sparse-checkout-compatibility.sh | 6 +----- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/builtin/add.c b/builtin/add.c index c76e6ddd359..d512ece655b 100644 --- a/builtin/add.c +++ b/builtin/add.c @@ -192,13 +192,21 @@ static int refresh(int verbose, const struct pathspec *pathspec) struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP; int flags = REFRESH_IGNORE_SKIP_WORKTREE | (verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET); + struct pattern_list pl = { 0 }; + int sparse_checkout_enabled = !get_sparse_checkout_patterns(&pl); seen = xcalloc(pathspec->nr, 1); refresh_index(&the_index, flags, pathspec, seen, _("Unstaged changes after refreshing the index:")); for (i = 0; i < pathspec->nr; i++) { if (!seen[i]) { - if (matches_skip_worktree(pathspec, i, &skip_worktree_seen)) { + const char *path = pathspec->items[i].original; + int dtype = DT_REG; + + if (matches_skip_worktree(pathspec, i, &skip_worktree_seen) || + (sparse_checkout_enabled && + !path_matches_pattern_list(path, strlen(path), NULL, + &dtype, &pl, &the_index))) { string_list_append(&only_match_skip_worktree, pathspec->items[i].original); } else { diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 50dce0e0f99..3e201546577 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -339,11 +339,7 @@ test_expect_success 'status/add: outside sparse cone' ' # Adding the path outside of the sparse-checkout cone should fail. test_sparse_match test_must_fail git add folder1/a && - - test_must_fail git -C sparse-checkout add --refresh folder1/a 2>sparse-checkout-err && - test_must_fail git -C sparse-index add --refresh folder1/a 2>sparse-index-err && - # NEEDSWORK: A sparse index changes the error message. - ! test_cmp sparse-checkout-err sparse-index-err && + test_sparse_match test_must_fail git add --refresh folder1/a && # NEEDSWORK: Adding a newly-tracked file outside the cone succeeds test_sparse_match git add folder1/new && From patchwork Mon Jul 26 15:18:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12399913 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B783C4338F for ; Mon, 26 Jul 2021 15:19:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6177060D07 for ; Mon, 26 Jul 2021 15:19:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235096AbhGZOir (ORCPT ); Mon, 26 Jul 2021 10:38:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234990AbhGZOi0 (ORCPT ); Mon, 26 Jul 2021 10:38:26 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94247C061764 for ; Mon, 26 Jul 2021 08:18:54 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id q3so11616277wrx.0 for ; Mon, 26 Jul 2021 08:18:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=sVu7pFKkbkvWdqnDCNoAaVOoPWAajLXV8osQWTlKXmA=; b=t+9s0BFot+OEMzWyb75ixedPlHnTg/SxMlWaYE6+9q/0EYUcX3bEARZYMeAWJAf9w7 qud+V5Lt1D1MGj0ayKwtNuItOGL8gphjuRhWs/2wmW8+LfCG9do++2Zlz1k2yz5WNfC/ NXtJUZ2aw6qnlC2Mrz5OAwV5yPouBgiYO2pqpjluCZX2tXqCq0aYDmwePcA2FEU0bc28 LdOvpzg2I08kOtv4UcECLNSPs3p4JwJtcqWO9evvnjwFSiZpgk23dqV17UmtbvXm1m65 KeatYN+hXlvYk2qvB2+AnmkVj5pRH42fH65lqKcn+zEF+zDBOR/YD2SAcVfU2QNnKcGa 4deg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=sVu7pFKkbkvWdqnDCNoAaVOoPWAajLXV8osQWTlKXmA=; b=lxoHofNBx5Xx4nYnXX8utIUx2An9qWqWXMitXWpXvZy5AiYRzAagXKFjr3LKBZ8XTe AyIZmPJsnF3Hm1X19UVzCMtxcJ4jXhf5hnXDFtrzYbOFRdOxwKbj+C8kSdN4nisPlz6n Q+VPPJzlzySKdN/+gxeeH/PvlYGcdouTOh3l0+iVtw6e0VpSl03Puyxc9cnYtp2CtbzO sl4HAvWWYZ5d4/CAGI9iYJTi87XzYfqgJVY45tBkyULbQxPQ7J6yaoyzpsVpFOXcuAsM drUlH1ahpPM5ELflUnjeevxa2qUAh2apnZwQ18sSFSIRNLeQrsBSyCQ6N5zUYWD6Gl0s x5xA== X-Gm-Message-State: AOAM53170hOKaP+cocLHs2DRA5lDLm/vk8VrE7SxTs/iGRW0JN6b4A6D RZ2FpLR7U+yrlED4gZJ3nGvs4TKz8mM= X-Google-Smtp-Source: ABdhPJx4LJUau5CX/zb0IKEZRRo0ukHPEhr47GcyKL/wyDT/tf+qbo+wEKjCLdFdg0xBUctpxOVm9A== X-Received: by 2002:a5d:4912:: with SMTP id x18mr20599775wrq.310.1627312733251; Mon, 26 Jul 2021 08:18:53 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k9sm127984wrc.6.2021.07.26.08.18.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jul 2021 08:18:52 -0700 (PDT) Message-Id: <1543550a4e8cc4dda495c08c924034291001386a.1627312727.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 26 Jul 2021 15:18:47 +0000 Subject: [PATCH v2 5/5] add: remove ensure_full_index() with --renormalize Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Eric Sunshine , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The --renormalize option updates the EOL conversions for the tracked files. However, the loop already ignores files marked with the SKIP_WORKTREE bit, so it will continue to do so with a sparse index because the sparse directory entries also have this bit set. Signed-off-by: Derrick Stolee --- builtin/add.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/builtin/add.c b/builtin/add.c index d512ece655b..c49e179abc3 100644 --- a/builtin/add.c +++ b/builtin/add.c @@ -144,8 +144,6 @@ static int renormalize_tracked_files(const struct pathspec *pathspec, int flags) { int i, retval = 0; - /* TODO: audit for interaction with sparse-index. */ - ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) { struct cache_entry *ce = active_cache[i];