From patchwork Wed Dec 8 19:39:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12665175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94461C433F5 for ; Wed, 8 Dec 2021 19:40:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237298AbhLHTnc (ORCPT ); Wed, 8 Dec 2021 14:43:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234621AbhLHTnc (ORCPT ); Wed, 8 Dec 2021 14:43:32 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9508C0617A1 for ; Wed, 8 Dec 2021 11:39:59 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id a9so5909370wrr.8 for ; Wed, 08 Dec 2021 11:39:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=P83ymSPA3zYRQA1Q95p87N+uvnV0pSZrI9/9u64n4Y8=; b=RlpJz5+mYk/QDyxA0Po64bulLon4tqZx4EqvXL/DCjQl28kaj4v5pAWWx60Z0vJ5af 5r9pq2NWlQBkIin/ORLD/NCmjoV6uDaITt4Rn8KV0T8lIJmBsYvQ6c6AKf694NpQAeth 3b/68wlAhf9RmHNTBmDi+CiZTXtcPvtJljW1+2M/2QASjADfL0/YMo8NDaWepjvw8X2W hkUYLkQ74fYnEMBko+r4tYjWul0cKdgCG9IAopUVCmEUyw3WyefIxZECN1ucAdYevbHB aCXmE2/ZQpdIZ/7yvLBUdznD0NwAWF8lP/1H930cjRYTXyLK5GqlabCyS1JEFlcZxtpQ HhSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=P83ymSPA3zYRQA1Q95p87N+uvnV0pSZrI9/9u64n4Y8=; b=BoDnAhCL3lEV11AaOXqL83TTxVaKucYbRi9AGpUOGTrfd0vSh267aMmomNIW2X70Nr +XKpKypnSLwX3GbAj0YbCW5aK3QPvfmW64j2kHf5j8lgP3p1rr9SWgw2AaAaQ1LKMsc0 29y1EVaK5gBth6HSqfPrG0oiESuMHWY+IrtvmzxbUdsnBR0hFv8yAAj6Tv2OtJtLE2uk KmrZWeoglut2Gf5YpcqDf4abpfCkmXcvtOpNu1X0Pa1OYGlJ0C8+TuZenBIMaqfOVRzM kSUDeMXxVMApI43bnIns1o/L2QTzKyhcJc5c+2hTGKk7pVlFOKD8IL2bHp5h9Qs5ocIn UvUA== X-Gm-Message-State: AOAM532epJFzPXNUOf2thrntYmClPT7OsCMirSakpROfm9vi6CeAjNPv d5V7fevk1KaADFr3bSQaer224eA3dHI= X-Google-Smtp-Source: ABdhPJwWXyAGafK1ETqQJHh9uSJ/bliXc3MHP74RENNLSjBW/wuiooXgsvwGLIN6HBHF6hLeatSvLQ== X-Received: by 2002:adf:c10e:: with SMTP id r14mr807744wre.558.1638992398106; Wed, 08 Dec 2021 11:39:58 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e12sm4930411wrq.20.2021.12.08.11.39.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Dec 2021 11:39:57 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 08 Dec 2021 19:39:51 +0000 Subject: [PATCH v2 1/5] fetch/pull: use the sparse index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The 'git fetch' and 'git pull' commands parse the index in order to determine if submodules exist. Without command_requires_full_index=0, this will expand a sparse index, causing slow performance even when there is no new data to fetch. The .gitmodules file will never be inside a sparse directory entry, and even if it was, the index_name_pos() method would expand the sparse index if needed as we search for the path by name. These commands do not iterate over the index, which is the typical thing we are careful about when integrating with the sparse index. Signed-off-by: Derrick Stolee --- builtin/fetch.c | 2 ++ builtin/pull.c | 2 ++ t/t1092-sparse-checkout-compatibility.sh | 10 ++++++++++ 3 files changed, 14 insertions(+) diff --git a/builtin/fetch.c b/builtin/fetch.c index f7abbc31ff1..1696b7791d4 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1993,6 +1993,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) } git_config(git_fetch_config, NULL); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; argc = parse_options(argc, argv, prefix, builtin_fetch_options, builtin_fetch_usage, 0); diff --git a/builtin/pull.c b/builtin/pull.c index 1cfaf9f3436..7bce3bd80f9 100644 --- a/builtin/pull.c +++ b/builtin/pull.c @@ -994,6 +994,8 @@ int cmd_pull(int argc, const char **argv, const char *prefix) set_reflog_message(argc, argv); git_config(git_pull_config, NULL); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; argc = parse_options(argc, argv, prefix, pull_options, pull_usage, 0); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 83971174ac3..3ba19ba1c14 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -964,6 +964,16 @@ test_expect_success 'sparse index is not expanded: blame' ' done ' +test_expect_success 'sparse index is not expanded: fetch/pull' ' + init_repos && + + git -C sparse-index remote add full "file://$(pwd)/full-checkout" && + ensure_not_expanded fetch full && + git -C full-checkout commit --allow-empty -m "for pull merge" && + git -C sparse-index commit --allow-empty -m "for pull merge" && + ensure_not_expanded pull full base +' + # NEEDSWORK: a sparse-checkout behaves differently from a full checkout # in this scenario, but it shouldn't. test_expect_success 'reset mixed and checkout orphan' ' From patchwork Wed Dec 8 19:39:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12665177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEC5BC433FE for ; Wed, 8 Dec 2021 19:40:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239703AbhLHTnd (ORCPT ); Wed, 8 Dec 2021 14:43:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237307AbhLHTnd (ORCPT ); Wed, 8 Dec 2021 14:43:33 -0500 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D12AC061746 for ; Wed, 8 Dec 2021 11:40:00 -0800 (PST) Received: by mail-wm1-x329.google.com with SMTP id o29so2580409wms.2 for ; Wed, 08 Dec 2021 11:40:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=jpKzSaa1vLOzLitspYXOY0rB72oZr2VnYo72GMh22u4=; b=A56zCfK/kM1L0zQpwiIMcpMPF2iZftd/Vz/NvGqeXsYZot0aihLkcA7Kil+/81VxoV tG2exSs0TzGjk54G8MNYcucb/MhNHZwg1YOg1WExyFRBObCgTC6zcJlPG2SGfpyf/E2v VxnIoaw97hN4F3iz5b9LzCXWvXEkNl7L//PIKQit7+t/gkpYB6lcmKq8Icuja/hF0AOV KnQ6tdWfAVwJu3GPHR+NvNTRXuEEwz1rqu7nVfBuaShS9HESHGCnaezxUNxqx7yK62v/ miemJKoqSVsVlep3uMXkwW10We5GNQIMpzRBrS4J3UAqSPA/nwbpbhV5aHP92yjazTll QJog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=jpKzSaa1vLOzLitspYXOY0rB72oZr2VnYo72GMh22u4=; b=CXmpoDuTyv+nnXMcIIZlCCe8NTrJiZVeM+c+tYSN5USBe0Pz2/NXqs0S2AamccxBe+ 6RKGKZMY1AnG/ukAN+dhJ9Q+82FLTslRToR5E7i9GrIn/CT90zMc+pxB3imWhPulzyPd jO9KT6WOCKIDCXbO++Apr7dUqCCVs4dge8o+oYy3BPslcbpoVBRQbYAt5k2mAiCj0Xqd DuniZtIgMxliqiRxxVNLYmblK2NfeDjBizyHTi9jkjH8s/Jf2zzOqdD9O9TzzlfS7mta o6rRE/WC6CzEGEeH4QvAda13Hj/rmlZZ4cpfPhdroxShPVDb0TqzS744cFf589PkxCmV 2Zxw== X-Gm-Message-State: AOAM530RnhnzUvsRYL796fQTTbzyc+oGuIUKfdlqegEK2ZrPkH1Xl8Hc +FwViaovJ+NWKeMpH5oa/zu1Ca6LBSY= X-Google-Smtp-Source: ABdhPJxkKwEEiLjELScPkxRLFcV5+ZICrPdzC4lnDUQLykW9Md1POffnVEnmNy2jiyt4xeBbqPLZiA== X-Received: by 2002:a05:600c:4e51:: with SMTP id e17mr781155wmq.127.1638992399020; Wed, 08 Dec 2021 11:39:59 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h2sm3520301wrz.23.2021.12.08.11.39.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Dec 2021 11:39:58 -0800 (PST) Message-Id: <58b5eca483502b0d45601490f6b4255f6419e6bd.1638992396.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 08 Dec 2021 19:39:52 +0000 Subject: [PATCH v2 2/5] ls-files: add --sparse option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Existing callers to 'git ls-files' are expecting file names, not directories. It is best to expand a sparse index to show all of the contained files in this case. However, expert users may want to inspect the contents of the index itself including which directories are sparse. Add a --sparse option to allow users to request this information. During testing, I noticed that options such as --modified did not affect the output when the files in question were outside the sparse-checkout definition. Tests are added to document this preexisting behavior and how it remains unchanged with the sparse index and the --sparse option. Signed-off-by: Derrick Stolee --- Documentation/git-ls-files.txt | 4 ++ builtin/ls-files.c | 12 +++- t/t1092-sparse-checkout-compatibility.sh | 90 ++++++++++++++++++++++++ 3 files changed, 104 insertions(+), 2 deletions(-) diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt index 6d11ab506b7..1c5d5f85ec5 100644 --- a/Documentation/git-ls-files.txt +++ b/Documentation/git-ls-files.txt @@ -187,6 +187,10 @@ Both the in the index ("i/") and in the working tree ("w/") are shown for regular files, followed by the ("attr/"). +--sparse:: + If the index is sparse, show the sparse directories without expanding + to the contained files. + \--:: Do not interpret any more arguments as options. diff --git a/builtin/ls-files.c b/builtin/ls-files.c index 031fef1bcaa..c151eb1fb77 100644 --- a/builtin/ls-files.c +++ b/builtin/ls-files.c @@ -37,6 +37,7 @@ static int debug_mode; static int show_eol; static int recurse_submodules; static int skipping_duplicates; +static int show_sparse_dirs; static const char *prefix; static int max_prefix_len; @@ -315,8 +316,10 @@ static void show_files(struct repository *repo, struct dir_struct *dir) if (!(show_cached || show_stage || show_deleted || show_modified)) return; - /* TODO: audit for interaction with sparse-index. */ - ensure_full_index(repo->index); + + if (!show_sparse_dirs) + ensure_full_index(repo->index); + for (i = 0; i < repo->index->cache_nr; i++) { const struct cache_entry *ce = repo->index->cache[i]; struct stat st; @@ -670,6 +673,8 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) OPT_BOOL(0, "debug", &debug_mode, N_("show debugging data")), OPT_BOOL(0, "deduplicate", &skipping_duplicates, N_("suppress duplicate entries")), + OPT_BOOL(0, "sparse", &show_sparse_dirs, + N_("show sparse directories in the presence of a sparse index")), OPT_END() }; int ret = 0; @@ -677,6 +682,9 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) if (argc == 2 && !strcmp(argv[1], "-h")) usage_with_options(ls_files_usage, builtin_ls_files_options); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + prefix = cmd_prefix; if (prefix) prefix_len = strlen(prefix); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 3ba19ba1c14..bf2c6b169b9 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -802,6 +802,12 @@ test_expect_success 'sparse-index is expanded and converted back' ' GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ git -C sparse-index reset -- folder1/a && test_region index convert_to_sparse trace2.txt && + test_region index ensure_full_index trace2.txt && + + # ls-files expands on read, but does not write. + rm trace2.txt && + GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ + git -C sparse-index ls-files && test_region index ensure_full_index trace2.txt ' @@ -826,6 +832,7 @@ test_expect_success 'sparse-index is not expanded' ' init_repos && ensure_not_expanded status && + ensure_not_expanded ls-files --sparse && ensure_not_expanded commit --allow-empty -m empty && echo >>sparse-index/a && ensure_not_expanded commit -a -m a && @@ -974,6 +981,89 @@ test_expect_success 'sparse index is not expanded: fetch/pull' ' ensure_not_expanded pull full base ' +test_expect_success 'ls-files' ' + init_repos && + + # Behavior agrees by default. Sparse index is expanded. + test_all_match git ls-files && + + # With --sparse, the sparse index data changes behavior. + git -C sparse-index ls-files >dense && + git -C sparse-index ls-files --sparse >sparse && + + cat >expect <<-\EOF && + @@ -13,13 +13,9 @@ + e + folder1- + folder1.x + -folder1/0/0/0 + -folder1/0/1 + -folder1/a + +folder1/ + folder10 + -folder2/0/0/0 + -folder2/0/1 + -folder2/a + +folder2/ + g + -x/a + +x/ + z + EOF + + diff -u dense sparse | tail -n +3 >actual && + test_cmp expect actual && + + # With --sparse and no sparse index, nothing changes. + git -C sparse-checkout ls-files >dense && + git -C sparse-checkout ls-files --sparse >sparse && + test_cmp dense sparse && + + write_script edit-content <<-\EOF && + mkdir folder1 && + echo content >>folder1/a + EOF + run_on_sparse ../edit-content && + + # ls-files does not notice modified files whose + # cache entries are marked SKIP_WORKTREE. + test_sparse_match git ls-files --modified && + test_must_be_empty sparse-checkout-out && + test_must_be_empty sparse-index-out && + + git -C sparse-index ls-files --sparse --modified >sparse-index-out && + test_must_be_empty sparse-index-out && + + # Add folder1 to the sparse-checkout cone and + # check that ls-files shows the expanded files. + test_sparse_match git sparse-checkout add folder1 && + test_sparse_match git ls-files --modified && + + git -C sparse-index ls-files >dense && + git -C sparse-index ls-files --sparse >sparse && + + cat >expect <<-\EOF && + @@ -17,9 +17,7 @@ + folder1/0/1 + folder1/a + folder10 + -folder2/0/0/0 + -folder2/0/1 + -folder2/a + +folder2/ + g + -x/a + +x/ + z + EOF + + diff -u dense sparse | tail -n +3 >actual && + test_cmp expect actual && + + # Double-check index expansion is avoided + ensure_not_expanded ls-files --sparse +' + # NEEDSWORK: a sparse-checkout behaves differently from a full checkout # in this scenario, but it shouldn't. test_expect_success 'reset mixed and checkout orphan' ' From patchwork Wed Dec 8 19:39:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12665179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 213C2C433F5 for ; Wed, 8 Dec 2021 19:40:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239713AbhLHTnf (ORCPT ); Wed, 8 Dec 2021 14:43:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239704AbhLHTnd (ORCPT ); Wed, 8 Dec 2021 14:43:33 -0500 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C22AC061746 for ; Wed, 8 Dec 2021 11:40:01 -0800 (PST) Received: by mail-wm1-x331.google.com with SMTP id n33-20020a05600c502100b0032fb900951eso4954245wmr.4 for ; Wed, 08 Dec 2021 11:40:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=qC7INu0Mmhl0d/PVdY5sWbIdLLIsuB827d1moBqyaAU=; b=nIXTQIxJ9RiPNOietAlR1/oNduWUdObDNiR7tgAGfqktH//5okD73MAkYbVD2dI/bV JGIAl8gdoGAuNSl1irMsv238YeSKd/D+xgIKCACC4iPzX4EompymkFl0tpGdZVsctLeF NnV0SL0zLGGN6jIO1SdZCW9rqafMRdvwIxUeFFb0GRHkaZKQ9n/rpEdtNUKJCJbKcfrQ /sjL2WEqBwYomrrYG+UKTYCny9A2HqklDsteclk/XxRykRUFtm9eo5Vlssmq2XHogrvq VPZnYe3psCnwpaMShpIv0ORT46S+PLS8iU3or0cuRq80lSODV8dD8lMFobNRLZ7zbjWO ieSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=qC7INu0Mmhl0d/PVdY5sWbIdLLIsuB827d1moBqyaAU=; b=nBVtf81qtHinKVUqNAH/sdUK7OZDh2UTJMKjjYiamt5IFap7pLoVHCQi+KRcUTeljL RKHN04zHUC9Phmfc1SMjvGf4rURCPzeLQ/qVcjPEahiHFGcbxqVKHwPck9K4k7wVDbma d9crghP4hHyJHOf8EzFORprz74/SCZqTlA6ulIkmMl0KQ4f7JRuNL5ldG7Tk2NY9MhQK V7t1qCoOywPOHr+dDUFHLV/EwQbs9bO/YxsZ4ZUeCmYPkdMYw7m7hwet3uDB42t3UBeD YyYB5Y7yqRQuarQvSTqRMxrG90MsM47c6iGiGqTBWKrQnsPbrvr1DPoqsC1aHlPgYFD1 jKiw== X-Gm-Message-State: AOAM533p7/GfTwgiWAWrNa4a/jKuYMGTozdsRgU6CCCajC39Ge0gC4bT baz3DF2pPd1o9zu0XVj/pPZUCRZjLvg= X-Google-Smtp-Source: ABdhPJy8YfMOdeVTH2sL23akL8rGgeFW6aVCKtk7gUsCfyfzKK1C/DaImAc6JtWusXgqdm0p/rWcAQ== X-Received: by 2002:a1c:23d2:: with SMTP id j201mr1001965wmj.76.1638992399777; Wed, 08 Dec 2021 11:39:59 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p19sm4060650wmq.4.2021.12.08.11.39.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Dec 2021 11:39:59 -0800 (PST) Message-Id: <5ffae2a03ae3d00a1f128d7cf2a397081055a2fe.1638992396.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 08 Dec 2021 19:39:53 +0000 Subject: [PATCH v2 3/5] t1092: replace 'read-cache --table' with 'ls-files --sparse' Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Now that 'git ls-files --sparse' exists, we can use it to verify the state of a sparse index instead of 'test-tool read-cache table'. Replace these usages within t1092-sparse-checkout-compatibility.sh. The important changes are due to the different output format. We need to use the '--stage' output to get a file mode and OID, but it also includes a stage value and drops the object type. This leads to some differences in how we handle looking for specific entries. Some places where we previously looked for no 'tree' entries, we can instead directly compare the output across the repository with a sparse index and the one without. Signed-off-by: Derrick Stolee --- t/t1092-sparse-checkout-compatibility.sh | 29 +++++++++++------------- 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index bf2c6b169b9..4a6832ea3c5 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -200,45 +200,42 @@ test_sparse_unstaged () { test_expect_success 'sparse-index contents' ' init_repos && - test-tool -C sparse-index read-cache --table >cache && + git -C sparse-index ls-files --sparse --stage >cache && for dir in folder1 folder2 x do TREE=$(git -C sparse-index rev-parse HEAD:$dir) && - grep "040000 tree $TREE $dir/" cache \ + grep "040000 $TREE 0 $dir/" cache \ || return 1 done && git -C sparse-index sparse-checkout set folder1 && - test-tool -C sparse-index read-cache --table >cache && + git -C sparse-index ls-files --sparse --stage >cache && for dir in deep folder2 x do TREE=$(git -C sparse-index rev-parse HEAD:$dir) && - grep "040000 tree $TREE $dir/" cache \ + grep "040000 $TREE 0 $dir/" cache \ || return 1 done && git -C sparse-index sparse-checkout set deep/deeper1 && - test-tool -C sparse-index read-cache --table >cache && + git -C sparse-index ls-files --sparse --stage >cache && for dir in deep/deeper2 folder1 folder2 x do TREE=$(git -C sparse-index rev-parse HEAD:$dir) && - grep "040000 tree $TREE $dir/" cache \ + grep "040000 $TREE 0 $dir/" cache \ || return 1 done && # Disabling the sparse-index removes tree entries with full ones git -C sparse-index sparse-checkout init --no-sparse-index && - - test-tool -C sparse-index read-cache --table >cache && - ! grep "040000 tree" cache && - test_sparse_match test-tool read-cache --table + test_sparse_match git ls-files --stage --sparse ' test_expect_success 'expanded in-memory index matches full index' ' init_repos && - test_sparse_match test-tool read-cache --expand --table + test_sparse_match git ls-files --stage ' test_expect_success 'status with options' ' @@ -787,9 +784,9 @@ test_expect_success 'submodule handling' ' # having a submodule prevents "modules" from collapse test_sparse_match git sparse-checkout set deep/deeper1 && - test-tool -C sparse-index read-cache --table >cache && - grep "100644 blob .* modules/a" cache && - grep "160000 commit $(git -C initial-repo rev-parse HEAD) modules/sub" cache + git -C sparse-index ls-files --sparse --stage >cache && + grep "100644 .* modules/a" cache && + grep "160000 $(git -C initial-repo rev-parse HEAD) 0 modules/sub" cache ' # When working with a sparse index, some commands will need to expand the @@ -1079,13 +1076,13 @@ test_expect_success 'reset mixed and checkout orphan' ' # the sparse checkouts skip "adding" the other side of # the conflict. test_sparse_match git reset --mixed HEAD~1 && - test_sparse_match test-tool read-cache --table --expand && + test_sparse_match git ls-files --stage && test_sparse_match git status --porcelain=v2 && # At this point, sparse-checkouts behave differently # from the full-checkout. test_sparse_match git checkout --orphan new-branch && - test_sparse_match test-tool read-cache --table --expand && + test_sparse_match git ls-files --stage && test_sparse_match git status --porcelain=v2 ' From patchwork Wed Dec 8 19:39:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12665181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5EC15C433EF for ; Wed, 8 Dec 2021 19:40:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239721AbhLHTng (ORCPT ); Wed, 8 Dec 2021 14:43:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58666 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237307AbhLHTne (ORCPT ); Wed, 8 Dec 2021 14:43:34 -0500 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B795C0617A1 for ; Wed, 8 Dec 2021 11:40:02 -0800 (PST) Received: by mail-wr1-x42c.google.com with SMTP id u1so5860815wru.13 for ; Wed, 08 Dec 2021 11:40:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=5DIcA/rK56j1N1MDKr0SkoRx4hSRnncyQmOOHUYySdo=; b=bFVNtt+vx10QF7UDlNKaaw7A5sIapBLkUVeRSMZtqKC5y2mn+/OdRcXc5WVhhaLyVt RuVgekJWh6Abii9tDFiVyerRXn7EtsA9vpjsUzthgivJx3dMlVA/cMDmb7VpMnZWKlHe TxiUMdQVdoGVIQR4MHcmpHK0iMORrtXBJXrg8ZE0QstVHLnv0DBYK3XsUGN6v6hDp6Y0 EPytK/KOzym3Y3/B/xKy/ID1A8Un9V8RPKUB3HceOLTrJfFlBQFuoJPn83Gh6zKzuSzp gW1rKoiSURr0Rj2GD7QNkAMPhySiGi2QvezviIU7bYvd5MlDVAPUb1tXbvY9ja/yGKZf gSAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=5DIcA/rK56j1N1MDKr0SkoRx4hSRnncyQmOOHUYySdo=; b=LRd9HvBrtxcOgB1kIQBYPCNqenh1qtMlqjGVyIEGFYO3zZsNXu8MpwFLnGD2tGScYF xD5+Wen3FxbCsf3ApoZDApFGI4sunq15TlxTJcTLucfWjmMwWUWCnComakJJXz+cq28R x0Qr8GTIxz6Z8wa9LGh3u+jZy+2WIBU8u3vkkm4AVWC0SnGAL6hO8vJCxaZ6P2P1JeJW mH7eiisxKsz+JqrGPWxCpziykMeHmBlpOvCYfMLSZgBqtJGUjD4o8MZtYYWp8FkVYVyj I1gK/cljuRWtA7biMPmcLu1a8RwWnhci5cIvqGNafC2N8dX/upgU5Qy4WANLz23Btb4Y ccNg== X-Gm-Message-State: AOAM5329HkW+C5pL0CfLtKpPv3ngD4uobVT1/NNh7ceyvU8e7+zeKfct 9ju2LFLvN2TwStNGs91caBzuvCNbk4E= X-Google-Smtp-Source: ABdhPJxy+uw1UIZPDFLh0yqUgcu9YcvNr5j3EVD2/jICKrIJ8+HSfLfdcn5x4xrIygWcxxvJGZMHeQ== X-Received: by 2002:adf:d1c1:: with SMTP id b1mr835804wrd.296.1638992400547; Wed, 08 Dec 2021 11:40:00 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l1sm3539490wrn.15.2021.12.08.11.40.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Dec 2021 11:40:00 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 08 Dec 2021 19:39:54 +0000 Subject: [PATCH v2 4/5] t1091/t3705: remove 'test-tool read-cache --table' Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Now that 'git ls-files --sparse' exists, we can use it to verify the state of a sparse index instead of 'test-tool read-cache table'. Replace these usages within t1091-sparse-checkout-builtin.sh and t3705-add-sparse-checkout.sh. The important changes are due to the different output format. In t3705, wWe need to use the '--stage' output to get a file mode and OID, but it also includes a stage value and drops the object type. This leads to some differences in how we handle looking for specific entries. In t1091, the test focuses on enabling the sparse index, so we do not need the --stage flag to demonstrate how the index changes, and instead can use a diff. Signed-off-by: Derrick Stolee --- t/t1091-sparse-checkout-builtin.sh | 25 ++++++++++++++++++++----- t/t3705-add-sparse-checkout.sh | 8 ++++---- 2 files changed, 24 insertions(+), 9 deletions(-) diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 272ba1b566b..680e0043c36 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -212,12 +212,27 @@ test_expect_success 'sparse-index enabled and disabled' ' git -C repo sparse-checkout init --cone --sparse-index && test_cmp_config -C repo true index.sparse && - test-tool -C repo read-cache --table >cache && - grep " tree " cache && - + git -C repo ls-files --sparse >sparse && git -C repo sparse-checkout disable && - test-tool -C repo read-cache --table >cache && - ! grep " tree " cache && + git -C repo ls-files --sparse >full && + + cat >expect <<-\EOF && + @@ -1,4 +1,7 @@ + a + -deep/ + -folder1/ + -folder2/ + +deep/a + +deep/deeper1/a + +deep/deeper1/deepest/a + +deep/deeper2/a + +folder1/a + +folder2/a + EOF + + diff -u sparse full | tail -n +3 >actual && + test_cmp expect actual && + git -C repo config --list >config && ! grep index.sparse config ) diff --git a/t/t3705-add-sparse-checkout.sh b/t/t3705-add-sparse-checkout.sh index f3143c92908..81f3384eeed 100755 --- a/t/t3705-add-sparse-checkout.sh +++ b/t/t3705-add-sparse-checkout.sh @@ -181,13 +181,13 @@ test_expect_success 'git add fails outside of sparse-checkout definition' ' # Avoid munging CRLFs to avoid an error message git -c core.autocrlf=input add --sparse sparse_entry 2>stderr && test_must_be_empty stderr && - test-tool read-cache --table >actual && - grep "^100644 blob.*sparse_entry\$" actual && + git ls-files --stage >actual && + grep "^100644 .*sparse_entry\$" actual && git add --sparse --chmod=+x sparse_entry 2>stderr && test_must_be_empty stderr && - test-tool read-cache --table >actual && - grep "^100755 blob.*sparse_entry\$" actual && + git ls-files --stage >actual && + grep "^100755 .*sparse_entry\$" actual && git reset && From patchwork Wed Dec 8 19:39:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12665183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BB58C433FE for ; Wed, 8 Dec 2021 19:40:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239744AbhLHTni (ORCPT ); Wed, 8 Dec 2021 14:43:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239706AbhLHTnf (ORCPT ); Wed, 8 Dec 2021 14:43:35 -0500 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B0B3FC0617A2 for ; Wed, 8 Dec 2021 11:40:02 -0800 (PST) Received: by mail-wm1-x331.google.com with SMTP id az34-20020a05600c602200b0033bf8662572so2566016wmb.0 for ; Wed, 08 Dec 2021 11:40:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=a6pLplMAAQeTlcrzPDl3fCgT4av+qlYwa5dP/iJQKp0=; b=Cc8KsF1Cq5swjaqILZ6iEdjDUfJq0xtgGGgwlJT0gvIXSmzlJXYvhD9Sb0G0cSVWyF YEIj334WOd1fFOe8t6isGIhcdkbrDGdRivh/oVhA07mxZdsuodwL/WNLnjhl5nBvtM3C Qmestem0IzyK4SSig1wUtIEZSJo17KbXBbncBdJGQIh1wHc8hNrxtLG/Lq8n/cvlUsnt 04eG9jIfXuqtGnF7qo8tkI/VcxtdboAe1WH98dN/lu7F1QSiz4EzaWGtJeXjLotGX940 ptUfooOd0iAMhvg4Z/TcfBhSgS5CkecYqoE1vL+aikpZBHb4yeDqhEnvHHxcB2yzix+c s1BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=a6pLplMAAQeTlcrzPDl3fCgT4av+qlYwa5dP/iJQKp0=; b=R+D+J3ymcLMLw/U+lViI7IG4tDHco1qLCeVUyF5pnyTUEs/nZarQn3cpYyJ1IRKRO3 HdylFtfBTNJ0QruLCZqTAB2M1j1lrQnHWa9sJM0ih02nUtDeB+lXlBGko4ndfZTMo5C0 EZhu+5SRY4f/R4YVjmuXFHibIN+1C9vKThCoU+CiBPYrhJ3880XKKde2d7EVwjUjPoXX MTuQXw3uXMwhG168NT1iEaiBWTf5BJN0rNwY4nKZBDQZYVBn/c7P236KZiZZ1/e95Ipa FUgeoQtI8U/SImRsn7ODnYrNnTB/IcS0uJ3nvoHVZVMWBfu4oPyW/OO7Fde+wzre37o/ Gsbg== X-Gm-Message-State: AOAM530RnDLRYX/3E6ZLinH8NQ9ime4N/TZ7KZmcncTi/PHxTFIVgUB5 OuZ++HRSe1HWEYdH2/HvSXzcpT47lPI= X-Google-Smtp-Source: ABdhPJyjwuQHzXstd3+LpYlmnr4NSNRh4N+7d7U+wRkiVlst0IUYHi2sJ4iPSKvUSTGAA50vF0kGmA== X-Received: by 2002:a1c:7f43:: with SMTP id a64mr801918wmd.133.1638992401184; Wed, 08 Dec 2021 11:40:01 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e18sm3556881wrs.48.2021.12.08.11.40.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Dec 2021 11:40:00 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 08 Dec 2021 19:39:55 +0000 Subject: [PATCH v2 5/5] test-read-cache: remove --table, --expand options Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee This commit effectively reverts 2782db3 (test-tool: don't force full index, 2021-03-30) and e2df6c3 (test-read-cache: print cache entries with --table, 2021-03-30) to remove the --table and --expand options from 'test-tool read-cache'. The previous changes already removed these options from the test suite in favor of 'git ls-files --sparse'. The initial thought of creating these options was to allow for tests to see additional information with every cache entry. In particular, the object type is still not mirrored in 'git ls-files'. Since sparse directory entries always end with a slash, the object type is not critical to verify the sparse index is enabled. It was thought that it would be helpful to have additional information, such as flags, but that was not needed for the FS Monitor integration and hasn't been needed since. Signed-off-by: Derrick Stolee --- t/helper/test-read-cache.c | 64 ++++++-------------------------------- 1 file changed, 10 insertions(+), 54 deletions(-) diff --git a/t/helper/test-read-cache.c b/t/helper/test-read-cache.c index 0d9f08931a1..b736ef16421 100644 --- a/t/helper/test-read-cache.c +++ b/t/helper/test-read-cache.c @@ -1,83 +1,39 @@ #include "test-tool.h" #include "cache.h" #include "config.h" -#include "blob.h" -#include "commit.h" -#include "tree.h" -#include "sparse-index.h" - -static void print_cache_entry(struct cache_entry *ce) -{ - const char *type; - printf("%06o ", ce->ce_mode & 0177777); - - if (S_ISSPARSEDIR(ce->ce_mode)) - type = tree_type; - else if (S_ISGITLINK(ce->ce_mode)) - type = commit_type; - else - type = blob_type; - - printf("%s %s\t%s\n", - type, - oid_to_hex(&ce->oid), - ce->name); -} - -static void print_cache(struct index_state *istate) -{ - int i; - for (i = 0; i < istate->cache_nr; i++) - print_cache_entry(istate->cache[i]); -} int cmd__read_cache(int argc, const char **argv) { - struct repository *r = the_repository; int i, cnt = 1; const char *name = NULL; - int table = 0, expand = 0; initialize_the_repository(); - for (++argv, --argc; *argv && starts_with(*argv, "--"); ++argv, --argc) { - if (skip_prefix(*argv, "--print-and-refresh=", &name)) - continue; - if (!strcmp(*argv, "--table")) - table = 1; - else if (!strcmp(*argv, "--expand")) - expand = 1; + if (argc > 1 && skip_prefix(argv[1], "--print-and-refresh=", &name)) { + argc--; + argv++; } - if (argc == 1) - cnt = strtol(argv[0], NULL, 0); + if (argc == 2) + cnt = strtol(argv[1], NULL, 0); setup_git_directory(); git_config(git_default_config, NULL); - prepare_repo_settings(r); - r->settings.command_requires_full_index = 0; - for (i = 0; i < cnt; i++) { - repo_read_index(r); - - if (expand) - ensure_full_index(r->index); - + read_cache(); if (name) { int pos; - refresh_index(r->index, REFRESH_QUIET, + refresh_index(&the_index, REFRESH_QUIET, NULL, NULL, NULL); - pos = index_name_pos(r->index, name, strlen(name)); + pos = index_name_pos(&the_index, name, strlen(name)); if (pos < 0) die("%s not in index", name); printf("%s is%s up to date\n", name, - ce_uptodate(r->index->cache[pos]) ? "" : " not"); + ce_uptodate(the_index.cache[pos]) ? "" : " not"); write_file(name, "%d\n", i); } - if (table) - print_cache(r->index); - discard_index(r->index); + discard_cache(); } return 0; }