From patchwork Wed Jan 19 17:29:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Schindelin X-Patchwork-Id: 12717686 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0909CC433EF for ; Wed, 19 Jan 2022 17:29:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1356413AbiASR3p (ORCPT ); Wed, 19 Jan 2022 12:29:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58392 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1356412AbiASR3o (ORCPT ); Wed, 19 Jan 2022 12:29:44 -0500 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 70910C061574 for ; Wed, 19 Jan 2022 09:29:43 -0800 (PST) Received: by mail-wm1-x32c.google.com with SMTP id 25-20020a05600c231900b003497473a9c4so15781809wmo.5 for ; Wed, 19 Jan 2022 09:29:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=d177RoXwBN4waEXcE0hYaWy8QTe4DCf5iLYnapbu+cE=; b=JQYCrFdFT0Et6C8CLhPMGDf/CT44BPMrHK1tCnzfnBtCH0WmvC3bcQE+9ImWIqCpes 9Kl7alOMSzhTtxsKOia16iC3SCKsWzpRJ4eae3fmAH71ktM8Omax53sIv0Qo4chdn/Dz nXy5fUBBCy+zowtDE3h/P3p560C7ncYicDB6U8zHEk0eubg4P8dmVziJoocNoqW0zlxl CE2reSxavVIpp7coSVdGy5Usn8qv+CU4TVNEP2SsIE8CXuInOP7GoHVeGoSZpJB6FuQW cKLMIdsaDUPS6MHz276FJtytWBDRPBSD9MJqxsWUKjDN2mo+s68fDZr7jiUAeffwKweM am3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=d177RoXwBN4waEXcE0hYaWy8QTe4DCf5iLYnapbu+cE=; b=PM20QXy6ga/SxFBvrTbDbnXU8k/zFxBBnAOTIM8AvOE2DhFoM86bscoHFbtSe5TuRv jHb6Dhmgb8n2JNTzEhs739KKRT44URbOKRqp8ZbW8UAieaH5vjFNsqU21DswmDyHQ/ev ojzAnAfo/HrLecxzodBbdImjznl29T+vlnPKbc0S1UbFN8gx80myJ8Vg6t5EKcloZ3lT ylh62ZKf85ddrZq0Vw61hQxEogkqQMUoUTEGLMpwavHTW3e+9nYuifS/Vod255L+QDEl uNEsFhrT6chaNZ0YLJnJUnXmmtgIol1td25E2yWOSF3Hi2x8PyFQnqtTe0PbcfFD+P5N M+oA== X-Gm-Message-State: AOAM530qNO4VheWGZRfzlf7FHQdfvrtERH8YMQ4mX5c6fzB1hO04UKqe +dEbQanCwr4+RljyhBpQB75GAyuh848= X-Google-Smtp-Source: ABdhPJxNKcNGHQPEUmGtiwAzyRBGwdLwh305xJmA85bI0l/fCl9VSjlrsrMroWoB4yGLQpaNSSdRaw== X-Received: by 2002:a5d:6848:: with SMTP id o8mr30294117wrw.2.1642613381913; Wed, 19 Jan 2022 09:29:41 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c18sm411720wrr.64.2022.01.19.09.29.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 09:29:41 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 19 Jan 2022 17:29:37 +0000 Subject: [PATCH 1/3] sparse-index: sparse index is disallowed when split index is active Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin From: Johannes Schindelin In 6e773527b6b (sparse-index: convert from full to sparse, 2021-03-30), we introduced initial support for a sparse index, and were careful to avoid converting to a sparse index in the presence of a split index. However, when we _just_ read a freshly-initialized index, it might not contain a split index even if _writing_ it will add one by virtue of being asked for via the `GIT_TEST_SPLIT_INDEX` variable. We did not notice any problems with checking _only_ for `split_index` (and not `GIT_TEST_SPLIT_INDEX`) right until both `vd/sparse-sparsity-fix-on-read` _and_ `vd/sparse-reset` were merged. Those two topics' interplay triggers a bug in conjunction with running t1091.15 when `GIT_TEST_SPLIT_INDEX=true` in the following way: `vd/sparse-sparsity-fix-on-read` ensures that the index is made sparse right after reading, and `vd/sparse-reset` ensures that the index is made non-sparse again unless running in the `--soft` mode. Since the split index feature is incompatible with the sparse index feature, we see a symptom like this: fatal: position for replacement 4 exceeds base index size 4 Let's fix this by avoiding the conversion to a sparse index when `GIT_TEST_SPLIT_INDEX=true`. Signed-off-by: Johannes Schindelin --- sparse-index.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparse-index.c b/sparse-index.c index a1d505d50e9..08f54747bb4 100644 --- a/sparse-index.c +++ b/sparse-index.c @@ -136,7 +136,7 @@ static int is_sparse_index_allowed(struct index_state *istate, int flags) /* * The sparse index is not (yet) integrated with a split index. */ - if (istate->split_index) + if (istate->split_index || git_env_bool("GIT_TEST_SPLIT_INDEX", 0)) return 0; /* * The GIT_TEST_SPARSE_INDEX environment variable triggers the From patchwork Wed Jan 19 17:29:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Schindelin X-Patchwork-Id: 12717687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 585CEC433FE for ; Wed, 19 Jan 2022 17:29:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1356417AbiASR3q (ORCPT ); Wed, 19 Jan 2022 12:29:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347255AbiASR3o (ORCPT ); Wed, 19 Jan 2022 12:29:44 -0500 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3CE82C06161C for ; Wed, 19 Jan 2022 09:29:44 -0800 (PST) Received: by mail-wm1-x32e.google.com with SMTP id l35-20020a05600c1d2300b0034d477271c1so6973514wms.3 for ; Wed, 19 Jan 2022 09:29:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=g/OwK2IGtJnN3AzhDkiQ7pdCzEjK1f95SeANY/cCZnA=; b=RX3uDR+p2ccrgPPkb/22JpXkRss4qe5XDv21Smq5qmojYwXzQbavMDSZHkcztVnnEi DLATXZiwVVQ6jlGy9at5NmrW5hjXX9SuA7MGXk5QTRCihTnJQl+FS9c+C152ii62W2Rh s5oSVWe8oYOjwHplZLqLzLjKy0PJr1Jp4nkoarCC/ULC2PhKvMw4gOyvn9hcf3Gj5zH6 WU9uh1mjPfpOCt4SBaoF/ZaidBcR6J/H2+IbHa70RcmCHrNOHRKXmldqOw7Ul3h5lX1C Z1z5WgiLE2YkHn0aZKueCZwQs1e+toFyUHvLmasUCbx6NkKd4q825z9P0gCb4yA09rKc NMVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=g/OwK2IGtJnN3AzhDkiQ7pdCzEjK1f95SeANY/cCZnA=; b=TpHDfU0zCY+b4fuSfX2B59Fbf04z6SG/SVBQZ/rtsleMt90TaQiyCncT0h4rhW/4vL OFlLgUZP5VccHkgdXCGN7ziKgbhdpnOB6EHFuRVMVZLdG8q3CiHD3Tiwutpb+h0Dv5MO AjlIs/5LWTd/XDSQl2KZByHIVBoXyUidO/fH1go9wRkFRxoEIb88Ls1tIzNBPhN0g5Uy S/TMkWLBuP8tT/jmR3cRoxHNJMWwkg6SVRxJDn2gYeob9zceTvkcufoBRf4AqVnE7e7+ ZxuTuHLeqJbA+k3a/MinIXlNqERLeLV9vPTiHF8lNZuV6kxjCFWDfovjiy0PmDOol079 LiZw== X-Gm-Message-State: AOAM5320pvQ70R1veJ8rQsUNSEJNZKJFWjrNZ8o/aKXPhU4krw5pNbuh yypJNN/fYxnVGyqnhnvRwaqj5W9rxv8= X-Google-Smtp-Source: ABdhPJzcaqIfXmMoMcSqaWBJBRhGakE4nJg3OkuyUyGTcFxVnr0onzVI66IqZThF+CfVnWvjLECz6w== X-Received: by 2002:a1c:f606:: with SMTP id w6mr4707077wmc.76.1642613382697; Wed, 19 Jan 2022 09:29:42 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o12sm673526wrc.51.2022.01.19.09.29.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 09:29:42 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 19 Jan 2022 17:29:38 +0000 Subject: [PATCH 2/3] t1091: disable split index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin From: Johannes Schindelin In 61feddcdf28 (tests: disable GIT_TEST_SPLIT_INDEX for sparse index tests, 2021-08-26), it was already called out that the split index feature is incompatible with the sparse index feature, and its commit message wondered aloud whether more checks would be required to ensure that the split index and sparse index features aren't enabled at the same time. We are about to introduce such additional checks, and indeed, t1091 would utterly fail with them. Therefore, let's preemptively disable the split index for the entirety of t1091. This partially reverts above-mentioned patch because it covered only one test case whereas we want to cover the entire test script. Signed-off-by: Johannes Schindelin --- t/t1091-sparse-checkout-builtin.sh | 54 ++++++++++++++---------------- 1 file changed, 26 insertions(+), 28 deletions(-) diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 42776984fe7..b229a8274d8 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -5,6 +5,9 @@ test_description='sparse checkout builtin tests' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +GIT_TEST_SPLIT_INDEX=false +export GIT_TEST_SPLIT_INDEX + . ./test-lib.sh list_files() { @@ -228,36 +231,31 @@ test_expect_success 'sparse-checkout disable' ' ' test_expect_success 'sparse-index enabled and disabled' ' - ( - sane_unset GIT_TEST_SPLIT_INDEX && - git -C repo update-index --no-split-index && - - git -C repo sparse-checkout init --cone --sparse-index && - test_cmp_config -C repo true index.sparse && - git -C repo ls-files --sparse >sparse && - git -C repo sparse-checkout disable && - git -C repo ls-files --sparse >full && - - cat >expect <<-\EOF && - @@ -1,4 +1,7 @@ - a - -deep/ - -folder1/ - -folder2/ - +deep/a - +deep/deeper1/a - +deep/deeper1/deepest/a - +deep/deeper2/a - +folder1/a - +folder2/a - EOF + git -C repo sparse-checkout init --cone --sparse-index && + test_cmp_config -C repo true index.sparse && + git -C repo ls-files --sparse >sparse && + git -C repo sparse-checkout disable && + git -C repo ls-files --sparse >full && - diff -u sparse full | tail -n +3 >actual && - test_cmp expect actual && + cat >expect <<-\EOF && + @@ -1,4 +1,7 @@ + a + -deep/ + -folder1/ + -folder2/ + +deep/a + +deep/deeper1/a + +deep/deeper1/deepest/a + +deep/deeper2/a + +folder1/a + +folder2/a + EOF + + diff -u sparse full | tail -n +3 >actual && + test_cmp expect actual && - git -C repo config --list >config && - ! grep index.sparse config - ) + git -C repo config --list >config && + ! grep index.sparse config ' test_expect_success 'cone mode: init and set' ' From patchwork Wed Jan 19 17:29:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Schindelin X-Patchwork-Id: 12717688 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95A01C433EF for ; Wed, 19 Jan 2022 17:29:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1356429AbiASR3r (ORCPT ); Wed, 19 Jan 2022 12:29:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1356412AbiASR3p (ORCPT ); Wed, 19 Jan 2022 12:29:45 -0500 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6F7FC061574 for ; Wed, 19 Jan 2022 09:29:44 -0800 (PST) Received: by mail-wm1-x331.google.com with SMTP id f202-20020a1c1fd3000000b0034dd403f4fbso3699369wmf.1 for ; Wed, 19 Jan 2022 09:29:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=u3hSLdYUV9oM5IlLV6aazOYIDhcZRaEMA3A4Vn00mfo=; b=knAHuAesRFfqeHUnStQyC8Ru6AqH2omehYUAEn6t4OFY9b1rlx+5/qGEUWrbIEkQ+3 jUkAZ9RpbHbb1Cp9GbSrblooT5XRYStMeIjdT38nD6b6dROXtJoJUVpIESMA0WKt0jMx rLpGg8fUVy/aZ445WRHck5Cl/7NHRuxPNzOIlR9mwbhY5FNfvA8BNYMfdTdA46DGSVIj CrIO73RG+SvLOeA20KF6AVB/gUcQD7BgicJogY58I6gI93Y+DPGYCLDc5gK9lXAH7wWB E4QnFzh5FQ2pGVNc94Cfg88tskpyHfW6wI9Z+c72CfnPb5xBjQkvFFpi0/7VtPiKdtqe 4Fkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=u3hSLdYUV9oM5IlLV6aazOYIDhcZRaEMA3A4Vn00mfo=; b=W2JD0tkL8x7rjN2fCOjA4k5hOBCH1rPuzyedYwiwEjCc2idCP+D5njCqZK7SrA/p5q SC8Pol6m5PirQvBMF6DjR82PdegsKZ5jLQaON4wAbLJ1IOZb/QPWsW+5Aj4HHM707E6M 9NTH3OH5FPZJLHQXdtDbz4IOs5Lt5JYEVHi1cJIkQxLUeY+eG+Ad76rDNZ6zfi38jkLo 0WM6b2IOEKF4VoQXnvHTA3sbEgyr8JKgI8LEk2ImjnnKTSlKQfFziKbNazslF692vP78 pzNmsJzEHw2pZFZ1SjU4HcWOcqXizUCz4v91FA87jqflJ23u0O96Yo030jY5TRxuuXUM 69WA== X-Gm-Message-State: AOAM530ExauKqnA/TbUi85IAJSBPn9eMDIS2qGX9K/YQ3hVe0/psjn5h 36ilFIXPNVB8tXYFtBTqGsaC2+JVaqk= X-Google-Smtp-Source: ABdhPJzLa3sECFAdZ1CVk4dsnHitJUAdCpFwSD8xLQ4zN6jZrHSUTeb5YGjmdXv5Z4suG7SPdkWmFg== X-Received: by 2002:a05:600c:21c6:: with SMTP id x6mr4596915wmj.135.1642613383451; Wed, 19 Jan 2022 09:29:43 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p17sm230400wrf.111.2022.01.19.09.29.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 09:29:43 -0800 (PST) Message-Id: <1bb57ccd61452124119bb663f5e35e9676748c82.1642613380.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 19 Jan 2022 17:29:39 +0000 Subject: [PATCH 3/3] split-index: it really is incompatible with the sparse index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin From: Johannes Schindelin ... at least for now. So let's error out if we are even trying to initialize the split index when the index is sparse, or when trying to write the split index extension for a sparse index. Signed-off-by: Johannes Schindelin --- read-cache.c | 3 +++ split-index.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/read-cache.c b/read-cache.c index cbe73f14e5e..a932e01fc7a 100644 --- a/read-cache.c +++ b/read-cache.c @@ -3009,6 +3009,9 @@ static int do_write_index(struct index_state *istate, struct tempfile *tempfile, !is_null_oid(&istate->split_index->base_oid)) { struct strbuf sb = STRBUF_INIT; + if (istate->sparse_index) + die(_("cannot write split index for a sparse index")); + err = write_link_extension(&sb, istate) < 0 || write_index_ext_header(f, eoie_c, CACHE_EXT_LINK, sb.len) < 0; diff --git a/split-index.c b/split-index.c index 8e52e891c3b..9d0ccc30d00 100644 --- a/split-index.c +++ b/split-index.c @@ -5,6 +5,9 @@ struct split_index *init_split_index(struct index_state *istate) { if (!istate->split_index) { + if (istate->sparse_index) + die(_("cannot use split index with a sparse index")); + CALLOC_ARRAY(istate->split_index, 1); istate->split_index->refcount = 1; }