From patchwork Wed Dec 22 14:20:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12691686 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 033AEC433EF for ; Wed, 22 Dec 2021 14:21:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245730AbhLVOVD (ORCPT ); Wed, 22 Dec 2021 09:21:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245711AbhLVOVA (ORCPT ); Wed, 22 Dec 2021 09:21:00 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 71A82C06173F for ; Wed, 22 Dec 2021 06:21:00 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id q16so5244420wrg.7 for ; Wed, 22 Dec 2021 06:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=vmoaaXJ+MYXx7Ti9375WCn8U2vmzd0IxNp++fHCascg=; b=nWsLh1XloFuZtLOkfAFj27aM8VDuZSISpbqnGTFaBJzfOZKiuDjsHGXYgRh7XyFsms zXM5CpFWuHz/6k5bKJ0Eaq43an0eGvpBWXtAC0MXWID+3t8uLqGuC+YIbB7ca/LPLYrT CGwwAQVEMQqFR2CKppBmg7hfc6swyIo893UYzf7G2nZxiCoEb5mONBfKCnGR4TvYiZ5p 4La87mvCgiMd5kqHljaAbOJyXbm6fm7n1FdKBsfGjwbLJvprynw0XDBUJTOKKEZefDZ8 7nftmkaJ80Gm+6wi6tPXMLSYw+QwRTxPSO4RPCFYzNQFTUsdH1UKJfwrxW/pPsLl/alW agBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=vmoaaXJ+MYXx7Ti9375WCn8U2vmzd0IxNp++fHCascg=; b=BuzJWwb4hDN0NkFak1M+3GURniEfswLlegAYFqfdgnz7GTISDIMKu6nS8aHJfsAadg mNFta1pZfr+sbxGLusNWwLxdfLpuBYsyOPB9P9tI+rNgo5zZ4hdgYTgqVXzSLxPFaV1g HXZ+diltpDWfFOwja836ojwVX0kQEVwgldsbJCDcBGFuc0FE5xUms1Sy6eKCKE+U/GoE uEz/WwEpecuATeV5p6tewRwSoImU9RfyzKn+VbU/tjNUPjbo7VQGFBxoZU+h7MrUduPx 3ko+tGPbtudhVycZaDnny59jRtEHyEHgwt/osA8YE7av/5Z55uiCPdPxnGA4+Z+pLiq0 vO4Q== X-Gm-Message-State: AOAM533Ntyz4gTNVqJxpFE8dLGokV4OpyZjNkNJXuS3gx2YYmwPPxoqQ y84tP/5V7LXoaNwv2eA36GDfjfOo498= X-Google-Smtp-Source: ABdhPJztgHBp7Ha6zqDqFknN2WvFSrFjMdZUzmJ/eSQQ7Kk82hgMXaOCmJLq4I4UBl0sxmesbuGT7w== X-Received: by 2002:a5d:55c5:: with SMTP id i5mr2191034wrw.445.1640182858921; Wed, 22 Dec 2021 06:20:58 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a198sm1907645wmd.42.2021.12.22.06.20.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Dec 2021 06:20:58 -0800 (PST) Message-Id: <1043a1927d220345c3a2fdba518708869f71b542.1640182856.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 22 Dec 2021 14:20:52 +0000 Subject: [PATCH v4 1/5] fetch/pull: use the sparse index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The 'git fetch' and 'git pull' commands parse the index in order to determine if submodules exist. Without command_requires_full_index=0, this will expand a sparse index, causing slow performance even when there is no new data to fetch. The .gitmodules file will never be inside a sparse directory entry, and even if it was, the index_name_pos() method would expand the sparse index if needed as we search for the path by name. These commands do not iterate over the index, which is the typical thing we are careful about when integrating with the sparse index. Signed-off-by: Derrick Stolee --- builtin/fetch.c | 2 ++ builtin/pull.c | 2 ++ t/t1092-sparse-checkout-compatibility.sh | 10 ++++++++++ 3 files changed, 14 insertions(+) diff --git a/builtin/fetch.c b/builtin/fetch.c index e45185cf9cf..a0757dd0158 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1999,6 +1999,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) } git_config(git_fetch_config, NULL); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; argc = parse_options(argc, argv, prefix, builtin_fetch_options, builtin_fetch_usage, 0); diff --git a/builtin/pull.c b/builtin/pull.c index c8457619d86..100cbf9fb85 100644 --- a/builtin/pull.c +++ b/builtin/pull.c @@ -994,6 +994,8 @@ int cmd_pull(int argc, const char **argv, const char *prefix) set_reflog_message(argc, argv); git_config(git_pull_config, NULL); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; argc = parse_options(argc, argv, prefix, pull_options, pull_usage, 0); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 49f70a65692..8d3c21fc84c 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -1009,6 +1009,16 @@ test_expect_success 'sparse index is not expanded: blame' ' done ' +test_expect_success 'sparse index is not expanded: fetch/pull' ' + init_repos && + + git -C sparse-index remote add full "file://$(pwd)/full-checkout" && + ensure_not_expanded fetch full && + git -C full-checkout commit --allow-empty -m "for pull merge" && + git -C sparse-index commit --allow-empty -m "for pull merge" && + ensure_not_expanded pull full base +' + # NEEDSWORK: a sparse-checkout behaves differently from a full checkout # in this scenario, but it shouldn't. test_expect_success 'reset mixed and checkout orphan' ' From patchwork Wed Dec 22 14:20:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12691687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87516C4332F for ; Wed, 22 Dec 2021 14:21:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245754AbhLVOVG (ORCPT ); Wed, 22 Dec 2021 09:21:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245661AbhLVOVB (ORCPT ); Wed, 22 Dec 2021 09:21:01 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5551FC061574 for ; Wed, 22 Dec 2021 06:21:01 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id e5so5256399wrc.5 for ; Wed, 22 Dec 2021 06:21:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=D8biI8g5hUBagVH35EigGLK5NbB9QTcBEuE3+vURGtQ=; b=mWPbC2oOcX9LJXQ5qX6rw2xK2EcnKLtEfcyySZF0cBwk+3w75vscK61h4dzqn8Wfqa 8YYG6dMBben1dnsj7mUDk9/KbUwccdqPR8gII9zo5Tr8XpPVjaLTyqLWjESrPVNomUGa GtWqMUzrBIngHsHfOCDvcIgLKkrkL6vxRbsgz0VFNSsg/snhAQ6HGOQ7JYbAGF7n8UY8 47n4B6P+Ho1lidHgNCAGxOILJRgiI4Kv82Mq/Byt+ylW70vGbxJf5tdpBJL/2ANrHJkN BCMjV0hb7FXJX0MOgd6leV+mmGmmJsrlJbFIMOiBqPzVvfeNwuzY6x855EspHPS0URlk aPDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=D8biI8g5hUBagVH35EigGLK5NbB9QTcBEuE3+vURGtQ=; b=fynDzFDN8DxnNnne/36sVqcK2c/VuFJooJqyL34yHFeGeaYbaLSEvQDhN/bekwntqd ApjTJoLGRshrWpo/vZJGKvkmePjk6q3mSPG8LEzwof/Exo0eowyD6+vPJ7qdvpq+Lr8N FNHRA8D+eVZ3YnokZE0xZg6qJSy4Ho7iVw+OvumaK1O0I5mXnNbSzLCTVKB/aPCR8AU+ Kq3esP6KbhfNFHu08ZgH+NiXL21wuWSuzh7hJsF2xQkWmJEfdQIHXp7xoJq5kiebSB3E G+X935M74whVJ8l+yjoiH04YiG8gvmlUR5Q0s7I+RBkdSzha/C5IEG6KhKz2N91cAYaJ wpTg== X-Gm-Message-State: AOAM532KCgHmtTpYT/ukTUxY9V+YGNhu2bnJ66UbXKlAS7sFQpL+hzet OIAF6ZuXpBCgcbW0h/BJmckLpEW7UV4= X-Google-Smtp-Source: ABdhPJxelhgG41QGBfS93A9eyjqaRngZEjZpcrBPlgNrQuItgmuX1gYfYfO7Q9f7Tq3KcqANktXyhg== X-Received: by 2002:adf:e683:: with SMTP id r3mr2207363wrm.480.1640182859656; Wed, 22 Dec 2021 06:20:59 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e4sm2149062wrh.56.2021.12.22.06.20.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Dec 2021 06:20:59 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 22 Dec 2021 14:20:53 +0000 Subject: [PATCH v4 2/5] ls-files: add --sparse option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Existing callers to 'git ls-files' are expecting file names, not directories. It is best to expand a sparse index to show all of the contained files in this case. However, expert users may want to inspect the contents of the index itself including which directories are sparse. Add a --sparse option to allow users to request this information. During testing, I noticed that options such as --modified did not affect the output when the files in question were outside the sparse-checkout definition. Tests are added to document this preexisting behavior and how it remains unchanged with the sparse index and the --sparse option. Signed-off-by: Derrick Stolee --- Documentation/git-ls-files.txt | 5 ++ builtin/ls-files.c | 12 +++- t/t1092-sparse-checkout-compatibility.sh | 91 ++++++++++++++++++++++++ 3 files changed, 106 insertions(+), 2 deletions(-) diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt index 2e3d695fa21..48cc7c0b6f4 100644 --- a/Documentation/git-ls-files.txt +++ b/Documentation/git-ls-files.txt @@ -187,6 +187,11 @@ Both the in the index ("i/") and in the working tree ("w/") are shown for regular files, followed by the ("attr/"). +--sparse:: + If the index is sparse, show the sparse directories without expanding + to the contained files. Sparse directories will be shown with a + trailing slash, such as "x/" for a sparse directory "x". + \--:: Do not interpret any more arguments as options. diff --git a/builtin/ls-files.c b/builtin/ls-files.c index 031fef1bcaa..c151eb1fb77 100644 --- a/builtin/ls-files.c +++ b/builtin/ls-files.c @@ -37,6 +37,7 @@ static int debug_mode; static int show_eol; static int recurse_submodules; static int skipping_duplicates; +static int show_sparse_dirs; static const char *prefix; static int max_prefix_len; @@ -315,8 +316,10 @@ static void show_files(struct repository *repo, struct dir_struct *dir) if (!(show_cached || show_stage || show_deleted || show_modified)) return; - /* TODO: audit for interaction with sparse-index. */ - ensure_full_index(repo->index); + + if (!show_sparse_dirs) + ensure_full_index(repo->index); + for (i = 0; i < repo->index->cache_nr; i++) { const struct cache_entry *ce = repo->index->cache[i]; struct stat st; @@ -670,6 +673,8 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) OPT_BOOL(0, "debug", &debug_mode, N_("show debugging data")), OPT_BOOL(0, "deduplicate", &skipping_duplicates, N_("suppress duplicate entries")), + OPT_BOOL(0, "sparse", &show_sparse_dirs, + N_("show sparse directories in the presence of a sparse index")), OPT_END() }; int ret = 0; @@ -677,6 +682,9 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) if (argc == 2 && !strcmp(argv[1], "-h")) usage_with_options(ls_files_usage, builtin_ls_files_options); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + prefix = cmd_prefix; if (prefix) prefix_len = strlen(prefix); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 8d3c21fc84c..0c51e9bd3b7 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -816,6 +816,12 @@ test_expect_success 'sparse-index is expanded and converted back' ' GIT_TRACE2_EVENT="$(pwd)/trace2.txt" \ git -C sparse-index reset -- folder1/a && test_region index convert_to_sparse trace2.txt && + test_region index ensure_full_index trace2.txt && + + # ls-files expands on read, but does not write. + rm trace2.txt && + GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ + git -C sparse-index ls-files && test_region index ensure_full_index trace2.txt ' @@ -871,6 +877,7 @@ test_expect_success 'sparse-index is not expanded' ' init_repos && ensure_not_expanded status && + ensure_not_expanded ls-files --sparse && ensure_not_expanded commit --allow-empty -m empty && echo >>sparse-index/a && ensure_not_expanded commit -a -m a && @@ -1019,6 +1026,90 @@ test_expect_success 'sparse index is not expanded: fetch/pull' ' ensure_not_expanded pull full base ' +test_expect_success 'ls-files' ' + init_repos && + + # Use a smaller sparse-checkout for reduced output + test_sparse_match git sparse-checkout set && + + # Behavior agrees by default. Sparse index is expanded. + test_all_match git ls-files && + + # With --sparse, the sparse index data changes behavior. + git -C sparse-index ls-files --sparse >actual && + + cat >expect <<-\EOF && + a + deep/ + e + folder1- + folder1.x + folder1/ + folder10 + folder2/ + g + x/ + z + EOF + + test_cmp expect actual && + + # With --sparse and no sparse index, nothing changes. + git -C sparse-checkout ls-files >dense && + git -C sparse-checkout ls-files --sparse >sparse && + test_cmp dense sparse && + + # Set up a strange condition of having a file edit + # outside of the sparse-checkout cone. This is just + # to verify that sparse-checkout and sparse-index + # behave the same in this case. + write_script edit-content <<-\EOF && + mkdir folder1 && + echo content >>folder1/a + EOF + run_on_sparse ../edit-content && + + # ls-files does not currently notice modified files whose + # cache entries are marked SKIP_WORKTREE. This may change + # in the future, but here we test that sparse index does + # not accidentally create a change of behavior. + test_sparse_match git ls-files --modified && + test_must_be_empty sparse-checkout-out && + test_must_be_empty sparse-index-out && + + git -C sparse-index ls-files --sparse --modified >sparse-index-out && + test_must_be_empty sparse-index-out && + + # Add folder1 to the sparse-checkout cone and + # check that ls-files shows the expanded files. + test_sparse_match git sparse-checkout add folder1 && + test_sparse_match git ls-files --modified && + + test_all_match git ls-files && + git -C sparse-index ls-files --sparse >actual && + + cat >expect <<-\EOF && + a + deep/ + e + folder1- + folder1.x + folder1/0/0/0 + folder1/0/1 + folder1/a + folder10 + folder2/ + g + x/ + z + EOF + + test_cmp expect actual && + + # Double-check index expansion is avoided + ensure_not_expanded ls-files --sparse +' + # NEEDSWORK: a sparse-checkout behaves differently from a full checkout # in this scenario, but it shouldn't. test_expect_success 'reset mixed and checkout orphan' ' From patchwork Wed Dec 22 14:20:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12691688 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47FE6C433F5 for ; Wed, 22 Dec 2021 14:21:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343538AbhLVOVO (ORCPT ); Wed, 22 Dec 2021 09:21:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245720AbhLVOVC (ORCPT ); Wed, 22 Dec 2021 09:21:02 -0500 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49AB7C061574 for ; Wed, 22 Dec 2021 06:21:02 -0800 (PST) Received: by mail-wm1-x335.google.com with SMTP id g7-20020a7bc4c7000000b00345c4bb365aso1178318wmk.4 for ; Wed, 22 Dec 2021 06:21:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=LptVVk401A/kcVr4wa7dIRYaI4RiLqOsY/gC10856C0=; b=P1hi40b0d4u5XPlroftdrGSWmSEIoPJYa8APFbxQ1+yM3IX2WsBwBbmz77BEosW+5Q KFltbsx+2nlZC++rR5+N4HRhzfyr9P7hWyr1VinpOdSCMPUsKzG9YfVmnNtfOX9bijq0 vZcKj32Y2FNUjvv6wKB5xFjjNTRvieeeh7LBQZsl4XTZOgPYClPdbwuY1ycVAcfJv9Si Po/HwIKQRAZKOhpPMLkgFJ3qNGcb0jLJnl+e2of7xlmZQhe6EP/NTVxieGZ6LXQrji/E nfrLwLV/C6lAr1/BNJEFzBprK5LeupoUATaSuCDVO8eJD384Xpg1l67BEOkmjm9t/NIz RIng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=LptVVk401A/kcVr4wa7dIRYaI4RiLqOsY/gC10856C0=; b=YgIRYrSCL7c1hCWv4h8LmRej7DU9JWJjyrIHbwOCZEvwZPBaHaAD9TUHifaSBT3fd+ ZRB6yGjF8gfnCWqHUDQr8d8VJX43y+dm9/35+RI78i7qqV/ATwVLDi1Hxd70/fRtGTZz lE0kBnGpwBMbIh+2oC25KriAW+RhDGOr1wg1mq8Pgql4iCrRxDEkb067in408UXZu86V pSdgdvxdfBHTPJ1mq4cHyzjIq4+iwkCuHAq2jerZGm8dihdi6XJWkTxKKpSf82xZ3Sh4 1mX8po0QLvJBf+k5mP8tu21R9ahfxgUcTzW58jJWcp4lyhWnZOiyyBNlHMxX4CVWaOKp P0Bw== X-Gm-Message-State: AOAM530UIo3GqqqS0Pno9FgHg4/Ghecy0kYGqUE9qiq+Lw9XjgiNfvhq 4xUgbWWjBLTH06Fu/ELwoIbKaEWFp4Q= X-Google-Smtp-Source: ABdhPJyNVMGh6Nanehl9uPhhfIvqs0MRmjiN4Gfyc9s0OqsXYIENyXdQ69xLXmVvTL2KdjZ86u66JA== X-Received: by 2002:a05:600c:4f55:: with SMTP id m21mr1095736wmq.49.1640182860695; Wed, 22 Dec 2021 06:21:00 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u19sm1958974wmq.30.2021.12.22.06.20.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Dec 2021 06:21:00 -0800 (PST) Message-Id: <0c53bd09ba4a4c16e2be4b92c4386e22dbe98b50.1640182856.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 22 Dec 2021 14:20:54 +0000 Subject: [PATCH v4 3/5] t1092: replace 'read-cache --table' with 'ls-files --sparse' Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Now that 'git ls-files --sparse' exists, we can use it to verify the state of a sparse index instead of 'test-tool read-cache table'. Replace these usages within t1092-sparse-checkout-compatibility.sh. The important changes are due to the different output format. We need to use the '--stage' output to get a file mode and OID, but it also includes a stage value and drops the object type. This leads to some differences in how we handle looking for specific entries. Some places where we previously looked for no 'tree' entries, we can instead directly compare the output across the repository with a sparse index and the one without. Signed-off-by: Derrick Stolee --- t/t1092-sparse-checkout-compatibility.sh | 31 +++++++++++------------- 1 file changed, 14 insertions(+), 17 deletions(-) diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 0c51e9bd3b7..4ba16177528 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -206,45 +206,42 @@ test_sparse_unstaged () { test_expect_success 'sparse-index contents' ' init_repos && - test-tool -C sparse-index read-cache --table >cache && + git -C sparse-index ls-files --sparse --stage >cache && for dir in folder1 folder2 x do TREE=$(git -C sparse-index rev-parse HEAD:$dir) && - grep "040000 tree $TREE $dir/" cache \ + grep "040000 $TREE 0 $dir/" cache \ || return 1 done && git -C sparse-index sparse-checkout set folder1 && - test-tool -C sparse-index read-cache --table >cache && + git -C sparse-index ls-files --sparse --stage >cache && for dir in deep folder2 x do TREE=$(git -C sparse-index rev-parse HEAD:$dir) && - grep "040000 tree $TREE $dir/" cache \ + grep "040000 $TREE 0 $dir/" cache \ || return 1 done && git -C sparse-index sparse-checkout set deep/deeper1 && - test-tool -C sparse-index read-cache --table >cache && + git -C sparse-index ls-files --sparse --stage >cache && for dir in deep/deeper2 folder1 folder2 x do TREE=$(git -C sparse-index rev-parse HEAD:$dir) && - grep "040000 tree $TREE $dir/" cache \ + grep "040000 $TREE 0 $dir/" cache \ || return 1 done && - # Disabling the sparse-index removes tree entries with full ones + # Disabling the sparse-index replaces tree entries with full ones git -C sparse-index sparse-checkout init --no-sparse-index && - - test-tool -C sparse-index read-cache --table >cache && - ! grep "040000 tree" cache && - test_sparse_match test-tool read-cache --table + test_sparse_match git ls-files --stage --sparse ' test_expect_success 'expanded in-memory index matches full index' ' init_repos && - test_sparse_match test-tool read-cache --expand --table + test_sparse_match git ls-files --stage ' test_expect_success 'status with options' ' @@ -801,9 +798,9 @@ test_expect_success 'submodule handling' ' # having a submodule prevents "modules" from collapse test_sparse_match git sparse-checkout set deep/deeper1 && - test-tool -C sparse-index read-cache --table >cache && - grep "100644 blob .* modules/a" cache && - grep "160000 commit $(git -C initial-repo rev-parse HEAD) modules/sub" cache + git -C sparse-index ls-files --sparse --stage >cache && + grep "100644 .* modules/a" cache && + grep "160000 $(git -C initial-repo rev-parse HEAD) 0 modules/sub" cache ' # When working with a sparse index, some commands will need to expand the @@ -1125,13 +1122,13 @@ test_expect_success 'reset mixed and checkout orphan' ' # the sparse checkouts skip "adding" the other side of # the conflict. test_sparse_match git reset --mixed HEAD~1 && - test_sparse_match test-tool read-cache --table --expand && + test_sparse_match git ls-files --stage && test_sparse_match git status --porcelain=v2 && # At this point, sparse-checkouts behave differently # from the full-checkout. test_sparse_match git checkout --orphan new-branch && - test_sparse_match test-tool read-cache --table --expand && + test_sparse_match git ls-files --stage && test_sparse_match git status --porcelain=v2 ' From patchwork Wed Dec 22 14:20:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12691689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB647C433EF for ; Wed, 22 Dec 2021 14:21:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235851AbhLVOVP (ORCPT ); Wed, 22 Dec 2021 09:21:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60424 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245725AbhLVOVD (ORCPT ); Wed, 22 Dec 2021 09:21:03 -0500 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20C90C061747 for ; Wed, 22 Dec 2021 06:21:03 -0800 (PST) Received: by mail-wm1-x336.google.com with SMTP id c66so1689740wma.5 for ; Wed, 22 Dec 2021 06:21:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=z1lsY1IELUoUSjAMa97erruqhiY9BJgxhgMt/g/gEqo=; b=XWyfrZpAfofFbgBhbrSXLVOXKfecLh3s3AEPCrDQpgmdEQ9PC8S2FMVag3rNAsMY+u RXjqdgzx4kUIA9OQwSQ9hjTcQKJgvmt3MqF5ynjpoS6on/UGeAKRAgCDWffVvc8o15kY ieuH93HOBVQXxJo0aPBBBBgObefRtyhCgTid5K0SEb4bDgbJc5ZoMVuU3M1aOM1ZHH2n /XqdjLMnWweW7ZNlwFlm34UAfi04ZM/iqbUGmzRpx39xzW3SONWDMCyGt+wJkDnrx2+O /R5aVb30ZiIUT8/M94Fc+62t3uDW0g+EJb2cAERZYeBFjhdEH/16JX/hGwlHfh16E3kf 3big== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=z1lsY1IELUoUSjAMa97erruqhiY9BJgxhgMt/g/gEqo=; b=QA1k97M9phvc8atFksauM6i1rBt0mgmLidW/wbYp3QsSJa44nK46j9Fvhkm+5+xiLv i35U/MoUzCTjxIBEq5lX6r16rKaXUnfL1htUoe5IZD0Vfm70VlZcKGvYQb064shLxhPC FW0J4TzaoLRojkLbP7wlw9W4y2EpHqSCLHcieIi4Tj2Oud1sE4PdITcTR/U7wRWgjSCM 1k0fKPO36ghNviNwy9IySKzcKR+bl75g/rLUPzAIrHivw+zfDUmTn8XepUeoWMRY8BkS M7ubduV7IK6GURICUGebAoG0Nilpk8QFsE3iV9+Ss/0U2Wlg2ZCx0oIBcltVsBZYQVj7 vSMg== X-Gm-Message-State: AOAM531q7Y2LGd7gcB7rAV1wU4P7FAj4/v6kThE8AFC8lmWbpvDuQNYn Ir52xOYB/Hwb2m5wV8kpj+vwWq6LtG4= X-Google-Smtp-Source: ABdhPJxM2qYhi8fTh+LgDP3Hgjtr3H8rW9V+U/HK5Ig9ZGvZVFb9FpUZwKBiwwAjknk7kudWOrJ2iw== X-Received: by 2002:a1c:730a:: with SMTP id d10mr1133275wmb.108.1640182861578; Wed, 22 Dec 2021 06:21:01 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h27sm5687749wmc.43.2021.12.22.06.21.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Dec 2021 06:21:01 -0800 (PST) Message-Id: <4952c9e724b82356ca2debfe4fd115b63198bc74.1640182856.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 22 Dec 2021 14:20:55 +0000 Subject: [PATCH v4 4/5] t1091/t3705: remove 'test-tool read-cache --table' Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Now that 'git ls-files --sparse' exists, we can use it to verify the state of a sparse index instead of 'test-tool read-cache table'. Replace these usages within t1091-sparse-checkout-builtin.sh and t3705-add-sparse-checkout.sh. The important changes are due to the different output format. In t3705, we need to use the '--stage' output to get a file mode and OID, but it also includes a stage value and drops the object type. This leads to some differences in how we handle looking for specific entries. In t1091, the test focuses on enabling the sparse index, so we do not need the --stage flag to demonstrate how the index changes, and instead can use a diff. Signed-off-by: Derrick Stolee --- t/t1091-sparse-checkout-builtin.sh | 25 ++++++++++++++++++++----- t/t3705-add-sparse-checkout.sh | 8 ++++---- 2 files changed, 24 insertions(+), 9 deletions(-) diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 272ba1b566b..680e0043c36 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -212,12 +212,27 @@ test_expect_success 'sparse-index enabled and disabled' ' git -C repo sparse-checkout init --cone --sparse-index && test_cmp_config -C repo true index.sparse && - test-tool -C repo read-cache --table >cache && - grep " tree " cache && - + git -C repo ls-files --sparse >sparse && git -C repo sparse-checkout disable && - test-tool -C repo read-cache --table >cache && - ! grep " tree " cache && + git -C repo ls-files --sparse >full && + + cat >expect <<-\EOF && + @@ -1,4 +1,7 @@ + a + -deep/ + -folder1/ + -folder2/ + +deep/a + +deep/deeper1/a + +deep/deeper1/deepest/a + +deep/deeper2/a + +folder1/a + +folder2/a + EOF + + diff -u sparse full | tail -n +3 >actual && + test_cmp expect actual && + git -C repo config --list >config && ! grep index.sparse config ) diff --git a/t/t3705-add-sparse-checkout.sh b/t/t3705-add-sparse-checkout.sh index f3143c92908..81f3384eeed 100755 --- a/t/t3705-add-sparse-checkout.sh +++ b/t/t3705-add-sparse-checkout.sh @@ -181,13 +181,13 @@ test_expect_success 'git add fails outside of sparse-checkout definition' ' # Avoid munging CRLFs to avoid an error message git -c core.autocrlf=input add --sparse sparse_entry 2>stderr && test_must_be_empty stderr && - test-tool read-cache --table >actual && - grep "^100644 blob.*sparse_entry\$" actual && + git ls-files --stage >actual && + grep "^100644 .*sparse_entry\$" actual && git add --sparse --chmod=+x sparse_entry 2>stderr && test_must_be_empty stderr && - test-tool read-cache --table >actual && - grep "^100755 blob.*sparse_entry\$" actual && + git ls-files --stage >actual && + grep "^100755 .*sparse_entry\$" actual && git reset && From patchwork Wed Dec 22 14:20:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12691690 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 128B1C433EF for ; Wed, 22 Dec 2021 14:21:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245737AbhLVOVR (ORCPT ); Wed, 22 Dec 2021 09:21:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60458 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245713AbhLVOVN (ORCPT ); Wed, 22 Dec 2021 09:21:13 -0500 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CBF60C061748 for ; Wed, 22 Dec 2021 06:21:03 -0800 (PST) Received: by mail-wm1-x32c.google.com with SMTP id c66so1689776wma.5 for ; Wed, 22 Dec 2021 06:21:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=a6pLplMAAQeTlcrzPDl3fCgT4av+qlYwa5dP/iJQKp0=; b=B+cb3fuXmybX3IoSp7ZJP8jI/LFAHyx659vo+UrWi5DrwowmiOQZMpskfi5dDA3Z+x VweALsceeGLGbxA+ufbclPEEVBw+VqO+Xq6ro5JBSByuJBdvhmYIFaa3ag7/UoKboTLJ 1uZaBZLIq9yvNh3AS4gnCdiQoqoEWMjMNPtPyvskmFxT/NMOAR3QA0yC1dSxdMw49jbG EkcfzRmXScmopCyAJNm9EpOuDKqwcQbmA5l6itVMeiS3MKOOvzxfkC2eXzxYI+N7s0tD e1JY7u343r9n7TLp2YF9Q+oAZ64000XmNzxO7XdhYbT+t1cc0mhOFzQXrC8jeyO6+KS2 KUOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=a6pLplMAAQeTlcrzPDl3fCgT4av+qlYwa5dP/iJQKp0=; b=Y2p/SMRCRq/SjsnmBQwyV9hOwwsVkY9fNnWUvbwMJ5132+C683pTw+6zjwLgLm0zGZ qbTxh5hMeNX2tBfY4N4uuXQxJInrXY7HUfAFakf0PQfhN7S19pRsrmrZEVUiRvhiABrL d3DwS9T2bHlddEqAXil6lB/Fas1v/RcyFy46/LjGvMUC52k/wK5FUD/QX8f/lhu7j6wM 9ugM6VY7SAW1kWcIebnvcX+WIjOr6VIcoRGkOAw52NHZCInjH5D/4GjIZRvPNdD5KxvO EM9VDskzt/T6JAts4MTdeDbkeVXHLN/efWVWGCmFOm4KgNLEiwhaRQKW6qwAOPaQGz1N xsXg== X-Gm-Message-State: AOAM532bHHrw1ylMNth4ZsKMem6DUKBg3jo1Ntavf1BDbDLWbqRnLOWx QLeWoYLAHItjhjwiP4HttcJ/Yavwn+A= X-Google-Smtp-Source: ABdhPJyp4lZTv5ttJIqkd94HUg4IZNldwSqvrj3Nelbo3EcTgvxn02gYOE7h/vV7/yNzl79pnJcGkA== X-Received: by 2002:a05:600c:a0e:: with SMTP id z14mr1071628wmp.43.1640182862274; Wed, 22 Dec 2021 06:21:02 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t11sm1967991wrz.97.2021.12.22.06.21.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Dec 2021 06:21:01 -0800 (PST) Message-Id: <3efffad814c7183ebe5538c145c4fd263e5a9bca.1640182856.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 22 Dec 2021 14:20:56 +0000 Subject: [PATCH v4 5/5] test-read-cache: remove --table, --expand options Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee This commit effectively reverts 2782db3 (test-tool: don't force full index, 2021-03-30) and e2df6c3 (test-read-cache: print cache entries with --table, 2021-03-30) to remove the --table and --expand options from 'test-tool read-cache'. The previous changes already removed these options from the test suite in favor of 'git ls-files --sparse'. The initial thought of creating these options was to allow for tests to see additional information with every cache entry. In particular, the object type is still not mirrored in 'git ls-files'. Since sparse directory entries always end with a slash, the object type is not critical to verify the sparse index is enabled. It was thought that it would be helpful to have additional information, such as flags, but that was not needed for the FS Monitor integration and hasn't been needed since. Signed-off-by: Derrick Stolee --- t/helper/test-read-cache.c | 64 ++++++-------------------------------- 1 file changed, 10 insertions(+), 54 deletions(-) diff --git a/t/helper/test-read-cache.c b/t/helper/test-read-cache.c index 0d9f08931a1..b736ef16421 100644 --- a/t/helper/test-read-cache.c +++ b/t/helper/test-read-cache.c @@ -1,83 +1,39 @@ #include "test-tool.h" #include "cache.h" #include "config.h" -#include "blob.h" -#include "commit.h" -#include "tree.h" -#include "sparse-index.h" - -static void print_cache_entry(struct cache_entry *ce) -{ - const char *type; - printf("%06o ", ce->ce_mode & 0177777); - - if (S_ISSPARSEDIR(ce->ce_mode)) - type = tree_type; - else if (S_ISGITLINK(ce->ce_mode)) - type = commit_type; - else - type = blob_type; - - printf("%s %s\t%s\n", - type, - oid_to_hex(&ce->oid), - ce->name); -} - -static void print_cache(struct index_state *istate) -{ - int i; - for (i = 0; i < istate->cache_nr; i++) - print_cache_entry(istate->cache[i]); -} int cmd__read_cache(int argc, const char **argv) { - struct repository *r = the_repository; int i, cnt = 1; const char *name = NULL; - int table = 0, expand = 0; initialize_the_repository(); - for (++argv, --argc; *argv && starts_with(*argv, "--"); ++argv, --argc) { - if (skip_prefix(*argv, "--print-and-refresh=", &name)) - continue; - if (!strcmp(*argv, "--table")) - table = 1; - else if (!strcmp(*argv, "--expand")) - expand = 1; + if (argc > 1 && skip_prefix(argv[1], "--print-and-refresh=", &name)) { + argc--; + argv++; } - if (argc == 1) - cnt = strtol(argv[0], NULL, 0); + if (argc == 2) + cnt = strtol(argv[1], NULL, 0); setup_git_directory(); git_config(git_default_config, NULL); - prepare_repo_settings(r); - r->settings.command_requires_full_index = 0; - for (i = 0; i < cnt; i++) { - repo_read_index(r); - - if (expand) - ensure_full_index(r->index); - + read_cache(); if (name) { int pos; - refresh_index(r->index, REFRESH_QUIET, + refresh_index(&the_index, REFRESH_QUIET, NULL, NULL, NULL); - pos = index_name_pos(r->index, name, strlen(name)); + pos = index_name_pos(&the_index, name, strlen(name)); if (pos < 0) die("%s not in index", name); printf("%s is%s up to date\n", name, - ce_uptodate(r->index->cache[pos]) ? "" : " not"); + ce_uptodate(the_index.cache[pos]) ? "" : " not"); write_file(name, "%d\n", i); } - if (table) - print_cache(r->index); - discard_index(r->index); + discard_cache(); } return 0; }