From patchwork Thu Apr 1 01:49:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176785 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7F5FC433ED for ; Thu, 1 Apr 2021 01:51:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7158660FE9 for ; Thu, 1 Apr 2021 01:51:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231620AbhDABua (ORCPT ); Wed, 31 Mar 2021 21:50:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51788 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232958AbhDABuH (ORCPT ); Wed, 31 Mar 2021 21:50:07 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9DC2CC061761 for ; Wed, 31 Mar 2021 18:50:06 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id v11so182317wro.7 for ; Wed, 31 Mar 2021 18:50:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=EEiLnZcxBnalSrtS0KgvDP7CGUo9RluvCjn0alofKlM=; b=c9Sl1w7XA5U8s4eBcfSa5OIgRuRr+AOzgoZuqZ72V6W6drRQ887VkVRwBXBiTeqMqr hWNIDTSfHj0xOJXdnEyLS3WgWGBPX/RtXwDtiYeF8xn/l7OxVFeofVQdGAMlvN0X2TI9 6bZOxd+lLfHdf9mtxkfx5J4FBBKOKKKYfZRhGDJv1Pc42P3KIbkUAfNEHtZ1XbMiVh5l aWnUk7pKUYycKJ53ADxZskOCojaIipQsGGNGD2gD8aDyBNGlw3CIM+E4VFMelrgHrSpu aSpqofR786ju8WaxhfpXZLIQPpyr+9F0TFzcClahfebqABD+VtTVE8WVgOmJAyiwknGU o+vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=EEiLnZcxBnalSrtS0KgvDP7CGUo9RluvCjn0alofKlM=; b=Hte92g2Qp9A+3PGXt+crqGRyDTZZAF20dxWr+bhyJRsqmvGYBH/lR7ruoJAjXmUyih duXsiuGtZSPy31FhovMEuv0m1BnkbwXnBKSSHKihZnDy7/768MCRCNV045HJelsUIQJO kHWnzvqEF9MpRI7HJdINv1ozLOFT9wmTfYTAl/PjrqkiKKojY7FL0rALYfeBB5EXLw9S 7SZcrFUJ7rjsvDWknmyK7evlK9Zw+t8yOp1QkhQoN0yvAoXQdxReWNhdo/kaSarHw04j YZy8N6wKyhV24JvoSV5kZOTbEBBniiDGi40SkZDMJ75xAe8CpZwEboZWsscosX5L7+up raGQ== X-Gm-Message-State: AOAM531HGzOnTNC3bApxfkfaZMxdCmiSGpNIoGvswBgo3x1ht14RF+GC zlESfS8xG6UK0R+GFyZcHZB926VO/aw= X-Google-Smtp-Source: ABdhPJzn8jqXZi2mj+VqymEzh0HdCaUqJ36vRoffamDVkH8+GRbuJb/8q1CkDk7GkHkY0MsLkXuV4w== X-Received: by 2002:adf:fb0b:: with SMTP id c11mr6716770wrr.425.1617241805392; Wed, 31 Mar 2021 18:50:05 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p16sm9304963wrt.54.2021.03.31.18.50.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:05 -0700 (PDT) Message-Id: <7484e085e3426fbcc696e8122de512ab3a846c09.1617241802.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:38 +0000 Subject: [PATCH v2 01/25] sparse-index: API protection strategy Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Edit and expand the sparse-index design document with the plan for guarding index operations with ensure_full_index(). Notably, the plan has changed to not have an expand_to_path() method in favor of checking for a sparse-directory hit inside of the index_path_pos() API. The changes that follow this one will incrementally add ensure_full_index() guards to iterations over all cache entries. Some iterations over the cache entries are not protected due to a few categories listed in the document. Since these are not being modified, here is a short list of the files and methods that will not receive these guards: Looking for non-zero stage: * builtin/add.c:chmod_pathspec() * builtin/merge.c:count_unmerged_entries() * merge-ort.c:record_conflicted_index_entries() * read-cache.c:unmerged_index() * rerere.c:check_one_conflict(), find_conflict(), rerere_remaining() * revision.c:prepare_show_merge() * sequencer.c:append_conflicts_hint() * wt-status.c:wt_status_collect_changes_initial() Looking for submodules: * builtin/submodule--helper.c:module_list_compute() * submodule.c: several methods * worktree.c:validate_no_submodules() Part of the index API: * name-hash.c: lazy init methods * preload-index.c:preload_thread(), preload_index() * read-cache.c: file format methods Checking for correct order of cache entries: * read-cache.c:check_ce_order() Ignores SKIP_WORKTREE entries or already aware: * unpack-trees.c:mark_new_skip_worktree() * wt-status.c:wt_status_check_sparse_checkout() Signed-off-by: Derrick Stolee --- Documentation/technical/sparse-index.txt | 37 ++++++++++++++++++++++-- 1 file changed, 35 insertions(+), 2 deletions(-) diff --git a/Documentation/technical/sparse-index.txt b/Documentation/technical/sparse-index.txt index 8d3d80804604..3b24c1a219f8 100644 --- a/Documentation/technical/sparse-index.txt +++ b/Documentation/technical/sparse-index.txt @@ -85,8 +85,41 @@ index, as well. Next, consumers of the index will be guarded against operating on a sparse-index by inserting calls to `ensure_full_index()` or -`expand_index_to_path()`. After these guards are in place, we can begin -leaving sparse-directory entries in the in-memory index structure. +`expand_index_to_path()`. If a specific path is requested, then those will +be protected from within the `index_file_exists()` and `index_name_pos()` +API calls: they will call `ensure_full_index()` if necessary. The +intention here is to preserve existing behavior when interacting with a +sparse-checkout. We don't want a change to happen by accident, without +tests. Many of these locations may not need any change before removing the +guards, but we should not do so without tests to ensure the expected +behavior happens. + +It may be desirable to _change_ the behavior of some commands in the +presence of a sparse index or more generally in any sparse-checkout +scenario. In such cases, these should be carefully communicated and +tested. No such behavior changes are intended during this phase. + +During a scan of the codebase, not every iteration of the cache entries +needs an `ensure_full_index()` check. The basic reasons include: + +1. The loop is scanning for entries with non-zero stage. These entries + are not collapsed into a sparse-directory entry. + +2. The loop is scanning for submodules. These entries are not collapsed + into a sparse-directory entry. + +3. The loop is part of the index API, especially around reading or + writing the format. + +4. The loop is checking for correct order of cache entries and that is + correct if and only if the sparse-directory entries are in the correct + location. + +5. The loop ignores entries with the `SKIP_WORKTREE` bit set, or is + otherwise already aware of sparse directory entries. + +6. The sparse-index is disabled at this point when using the split-index + feature, so no effort is made to protect the split-index API. Even after inserting these guards, we will keep expanding sparse-indexes for most Git commands using the `command_requires_full_index` repository From patchwork Thu Apr 1 01:49:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B3ADC43470 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EED1B6023F for ; Thu, 1 Apr 2021 01:51:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232975AbhDABub (ORCPT ); Wed, 31 Mar 2021 21:50:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232976AbhDABuH (ORCPT ); Wed, 31 Mar 2021 21:50:07 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5566EC061574 for ; Wed, 31 Mar 2021 18:50:07 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id v11so182332wro.7 for ; Wed, 31 Mar 2021 18:50:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=5UGhbG5cF/mdIR/eQy+KY1VVNkBNL/pS5IaECgqGc0E=; b=ifT959NfAgL6Nj5qdqteL4WENuWPzrUcLuk4oHh4i+bYkomRjnvUNtZjApoOqT1Pvk gEb4Qs8V/o1b0c2l8lgFiqZzNO4/JRCDTaEjDS59IW0nSJSdzN2mPmU5X+b1JGamzsIa 3xl6RHtBTtsH8Lq97tcegXtXt8pxWJasaz2ZdYPaq9Q2j3zBTYtQAOl8KLEcQSa4SfbX NNw8CyY57RXJn/4qNToYmEaDJLFN71KoA3Q49cS/jWTz1u/WGoyre6OBA9YlqCqfhwrI bhkUQItcaeJ50bMGWq2skSi8uk++lkh/nVnV67zQ7ooT+akNYDpnY6HPBaT08MOZVi8t cThg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=5UGhbG5cF/mdIR/eQy+KY1VVNkBNL/pS5IaECgqGc0E=; b=bEZ3ghbphmUCT0SW+DbcDnZAWsgtd9NybEkUXrEWPdkOoXEBvJt5YJ4z2GZucnE8jL 5IpJmuM/8mv+zLAj6CBhCrCkAPm4za/EIW6vVS75LyUx4wI5nQ0eL/vyAGUr58eDG00c iOKTZB7dbtM7yguWXKt1TpZndJubNWR6fVACubkb4UZm+q18nSPyev0OjukH8WqF4rXj 4LzR/k3PzaabfcHeaTD8pdzrFW7npOmE+E3hlq6r1WwO13UZ/qz95TgRxTgWnISpWjJa 5KjRWOYL/cksyqVz6yBQuxrkWjvyQmBN8AhAG4M51RM4aD6oDiwoAodrJ4yiWNHGgUHb Pg2g== X-Gm-Message-State: AOAM531n09K3KrG/Yp6NJ1zEAyQTJCRtXyLP4UAxBCwQOdrnvJFkTb5s 4nOebkT1A04bbqIBhVbAYpE8ohPrmnM= X-Google-Smtp-Source: ABdhPJycLG0Yfzrx/X2rb6xWOhdqeKWxyAbo5aPrH2KbxaxHCBiFZK/Ri88yc4GmadHKn6rua2DEmg== X-Received: by 2002:a5d:404d:: with SMTP id w13mr6695636wrp.230.1617241805984; Wed, 31 Mar 2021 18:50:05 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r1sm9596339wrj.63.2021.03.31.18.50.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:05 -0700 (PDT) Message-Id: <098b2c9ef3523d9052cc3834871c0078f9616edf.1617241802.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:39 +0000 Subject: [PATCH v2 02/25] *: remove 'const' qualifier for struct index_state Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Several methods specify that they take a 'struct index_state' pointer with the 'const' qualifier because they intend to only query the data, not change it. However, we will be introducing a step very low in the method stack that might modify a sparse-index to become a full index in the case that our queries venture inside a sparse-directory entry. This change only removes the 'const' qualifiers that are necessary for the following change which will actually modify the implementation of index_name_stage_pos(). Signed-off-by: Derrick Stolee --- attr.c | 14 +++++++------- attr.h | 4 ++-- builtin/ls-files.c | 10 +++++----- cache.h | 6 +++--- convert.c | 26 +++++++++++++------------- convert.h | 20 ++++++++++---------- dir.c | 12 ++++++------ dir.h | 8 ++++---- merge-recursive.c | 2 +- pathspec.c | 6 +++--- pathspec.h | 6 +++--- read-cache.c | 10 +++++----- submodule.c | 6 +++--- submodule.h | 6 +++--- 14 files changed, 68 insertions(+), 68 deletions(-) diff --git a/attr.c b/attr.c index 4ef85d668b54..8de553293ee4 100644 --- a/attr.c +++ b/attr.c @@ -718,7 +718,7 @@ static struct attr_stack *read_attr_from_file(const char *path, int macro_ok) return res; } -static struct attr_stack *read_attr_from_index(const struct index_state *istate, +static struct attr_stack *read_attr_from_index(struct index_state *istate, const char *path, int macro_ok) { @@ -748,7 +748,7 @@ static struct attr_stack *read_attr_from_index(const struct index_state *istate, return res; } -static struct attr_stack *read_attr(const struct index_state *istate, +static struct attr_stack *read_attr(struct index_state *istate, const char *path, int macro_ok) { struct attr_stack *res = NULL; @@ -840,7 +840,7 @@ static void push_stack(struct attr_stack **attr_stack_p, } } -static void bootstrap_attr_stack(const struct index_state *istate, +static void bootstrap_attr_stack(struct index_state *istate, struct attr_stack **stack) { struct attr_stack *e; @@ -878,7 +878,7 @@ static void bootstrap_attr_stack(const struct index_state *istate, push_stack(stack, e, NULL, 0); } -static void prepare_attr_stack(const struct index_state *istate, +static void prepare_attr_stack(struct index_state *istate, const char *path, int dirlen, struct attr_stack **stack) { @@ -1078,7 +1078,7 @@ static void determine_macros(struct all_attrs_item *all_attrs, * If check->check_nr is non-zero, only attributes in check[] are collected. * Otherwise all attributes are collected. */ -static void collect_some_attrs(const struct index_state *istate, +static void collect_some_attrs(struct index_state *istate, const char *path, struct attr_check *check) { @@ -1107,7 +1107,7 @@ static void collect_some_attrs(const struct index_state *istate, fill(path, pathlen, basename_offset, check->stack, check->all_attrs, rem); } -void git_check_attr(const struct index_state *istate, +void git_check_attr(struct index_state *istate, const char *path, struct attr_check *check) { @@ -1124,7 +1124,7 @@ void git_check_attr(const struct index_state *istate, } } -void git_all_attrs(const struct index_state *istate, +void git_all_attrs(struct index_state *istate, const char *path, struct attr_check *check) { int i; diff --git a/attr.h b/attr.h index 404548f028a8..3732505edae8 100644 --- a/attr.h +++ b/attr.h @@ -190,14 +190,14 @@ void attr_check_free(struct attr_check *check); */ const char *git_attr_name(const struct git_attr *); -void git_check_attr(const struct index_state *istate, +void git_check_attr(struct index_state *istate, const char *path, struct attr_check *check); /* * Retrieve all attributes that apply to the specified path. * check holds the attributes and their values. */ -void git_all_attrs(const struct index_state *istate, +void git_all_attrs(struct index_state *istate, const char *path, struct attr_check *check); enum git_attr_direction { diff --git a/builtin/ls-files.c b/builtin/ls-files.c index 60a2913a01e9..4f9ed1fb29b7 100644 --- a/builtin/ls-files.c +++ b/builtin/ls-files.c @@ -57,7 +57,7 @@ static const char *tag_modified = ""; static const char *tag_skip_worktree = ""; static const char *tag_resolve_undo = ""; -static void write_eolinfo(const struct index_state *istate, +static void write_eolinfo(struct index_state *istate, const struct cache_entry *ce, const char *path) { if (show_eol) { @@ -122,7 +122,7 @@ static void print_debug(const struct cache_entry *ce) } } -static void show_dir_entry(const struct index_state *istate, +static void show_dir_entry(struct index_state *istate, const char *tag, struct dir_entry *ent) { int len = max_prefix_len; @@ -139,7 +139,7 @@ static void show_dir_entry(const struct index_state *istate, write_name(ent->name); } -static void show_other_files(const struct index_state *istate, +static void show_other_files(struct index_state *istate, const struct dir_struct *dir) { int i; @@ -152,7 +152,7 @@ static void show_other_files(const struct index_state *istate, } } -static void show_killed_files(const struct index_state *istate, +static void show_killed_files(struct index_state *istate, const struct dir_struct *dir) { int i; @@ -254,7 +254,7 @@ static void show_ce(struct repository *repo, struct dir_struct *dir, } } -static void show_ru_info(const struct index_state *istate) +static void show_ru_info(struct index_state *istate) { struct string_list_item *item; diff --git a/cache.h b/cache.h index 8aede373aeb3..5006278c13ca 100644 --- a/cache.h +++ b/cache.h @@ -800,7 +800,7 @@ struct cache_entry *index_file_exists(struct index_state *istate, const char *na * index_name_pos(&index, "f", 1) -> -3 * index_name_pos(&index, "g", 1) -> -5 */ -int index_name_pos(const struct index_state *, const char *name, int namelen); +int index_name_pos(struct index_state *, const char *name, int namelen); /* * Some functions return the negative complement of an insert position when a @@ -850,8 +850,8 @@ int add_file_to_index(struct index_state *, const char *path, int flags); int chmod_index_entry(struct index_state *, struct cache_entry *ce, char flip); int ce_same_name(const struct cache_entry *a, const struct cache_entry *b); void set_object_name_for_intent_to_add_entry(struct cache_entry *ce); -int index_name_is_other(const struct index_state *, const char *, int); -void *read_blob_data_from_index(const struct index_state *, const char *, unsigned long *); +int index_name_is_other(struct index_state *, const char *, int); +void *read_blob_data_from_index(struct index_state *, const char *, unsigned long *); /* do stat comparison even if CE_VALID is true */ #define CE_MATCH_IGNORE_VALID 01 diff --git a/convert.c b/convert.c index ee360c2f07ce..20b19abffa36 100644 --- a/convert.c +++ b/convert.c @@ -138,7 +138,7 @@ static const char *gather_convert_stats_ascii(const char *data, unsigned long si } } -const char *get_cached_convert_stats_ascii(const struct index_state *istate, +const char *get_cached_convert_stats_ascii(struct index_state *istate, const char *path) { const char *ret; @@ -222,7 +222,7 @@ static void check_global_conv_flags_eol(const char *path, } } -static int has_crlf_in_index(const struct index_state *istate, const char *path) +static int has_crlf_in_index(struct index_state *istate, const char *path) { unsigned long sz; void *data; @@ -496,7 +496,7 @@ static int encode_to_worktree(const char *path, const char *src, size_t src_len, return 1; } -static int crlf_to_git(const struct index_state *istate, +static int crlf_to_git(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *buf, enum crlf_action crlf_action, int conv_flags) @@ -1307,7 +1307,7 @@ struct conv_attrs { static struct attr_check *check; -static void convert_attrs(const struct index_state *istate, +static void convert_attrs(struct index_state *istate, struct conv_attrs *ca, const char *path) { struct attr_check_item *ccheck = NULL; @@ -1369,7 +1369,7 @@ void reset_parsed_attributes(void) user_convert_tail = NULL; } -int would_convert_to_git_filter_fd(const struct index_state *istate, const char *path) +int would_convert_to_git_filter_fd(struct index_state *istate, const char *path) { struct conv_attrs ca; @@ -1388,7 +1388,7 @@ int would_convert_to_git_filter_fd(const struct index_state *istate, const char return apply_filter(path, NULL, 0, -1, NULL, ca.drv, CAP_CLEAN, NULL, NULL); } -const char *get_convert_attr_ascii(const struct index_state *istate, const char *path) +const char *get_convert_attr_ascii(struct index_state *istate, const char *path) { struct conv_attrs ca; @@ -1414,7 +1414,7 @@ const char *get_convert_attr_ascii(const struct index_state *istate, const char return ""; } -int convert_to_git(const struct index_state *istate, +int convert_to_git(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, int conv_flags) { @@ -1448,7 +1448,7 @@ int convert_to_git(const struct index_state *istate, return ret | ident_to_git(src, len, dst, ca.ident); } -void convert_to_git_filter_fd(const struct index_state *istate, +void convert_to_git_filter_fd(struct index_state *istate, const char *path, int fd, struct strbuf *dst, int conv_flags) { @@ -1466,7 +1466,7 @@ void convert_to_git_filter_fd(const struct index_state *istate, ident_to_git(dst->buf, dst->len, dst, ca.ident); } -static int convert_to_working_tree_internal(const struct index_state *istate, +static int convert_to_working_tree_internal(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, int normalizing, @@ -1510,7 +1510,7 @@ static int convert_to_working_tree_internal(const struct index_state *istate, return ret | ret_filter; } -int async_convert_to_working_tree(const struct index_state *istate, +int async_convert_to_working_tree(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, const struct checkout_metadata *meta, @@ -1519,7 +1519,7 @@ int async_convert_to_working_tree(const struct index_state *istate, return convert_to_working_tree_internal(istate, path, src, len, dst, 0, meta, dco); } -int convert_to_working_tree(const struct index_state *istate, +int convert_to_working_tree(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, const struct checkout_metadata *meta) @@ -1527,7 +1527,7 @@ int convert_to_working_tree(const struct index_state *istate, return convert_to_working_tree_internal(istate, path, src, len, dst, 0, meta, NULL); } -int renormalize_buffer(const struct index_state *istate, const char *path, +int renormalize_buffer(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst) { int ret = convert_to_working_tree_internal(istate, path, src, len, dst, 1, NULL, NULL); @@ -1964,7 +1964,7 @@ static struct stream_filter *ident_filter(const struct object_id *oid) * Note that you would be crazy to set CRLF, smudge/clean or ident to a * large binary blob you would want us not to slurp into the memory! */ -struct stream_filter *get_stream_filter(const struct index_state *istate, +struct stream_filter *get_stream_filter(struct index_state *istate, const char *path, const struct object_id *oid) { diff --git a/convert.h b/convert.h index e29d1026a686..0f7c8a1f0446 100644 --- a/convert.h +++ b/convert.h @@ -65,41 +65,41 @@ struct checkout_metadata { extern enum eol core_eol; extern char *check_roundtrip_encoding; -const char *get_cached_convert_stats_ascii(const struct index_state *istate, +const char *get_cached_convert_stats_ascii(struct index_state *istate, const char *path); const char *get_wt_convert_stats_ascii(const char *path); -const char *get_convert_attr_ascii(const struct index_state *istate, +const char *get_convert_attr_ascii(struct index_state *istate, const char *path); /* returns 1 if *dst was used */ -int convert_to_git(const struct index_state *istate, +int convert_to_git(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, int conv_flags); -int convert_to_working_tree(const struct index_state *istate, +int convert_to_working_tree(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, const struct checkout_metadata *meta); -int async_convert_to_working_tree(const struct index_state *istate, +int async_convert_to_working_tree(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, const struct checkout_metadata *meta, void *dco); int async_query_available_blobs(const char *cmd, struct string_list *available_paths); -int renormalize_buffer(const struct index_state *istate, +int renormalize_buffer(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst); -static inline int would_convert_to_git(const struct index_state *istate, +static inline int would_convert_to_git(struct index_state *istate, const char *path) { return convert_to_git(istate, path, NULL, 0, NULL, 0); } /* Precondition: would_convert_to_git_filter_fd(path) == true */ -void convert_to_git_filter_fd(const struct index_state *istate, +void convert_to_git_filter_fd(struct index_state *istate, const char *path, int fd, struct strbuf *dst, int conv_flags); -int would_convert_to_git_filter_fd(const struct index_state *istate, +int would_convert_to_git_filter_fd(struct index_state *istate, const char *path); /* @@ -133,7 +133,7 @@ void reset_parsed_attributes(void); struct stream_filter; /* opaque */ -struct stream_filter *get_stream_filter(const struct index_state *istate, +struct stream_filter *get_stream_filter(struct index_state *istate, const char *path, const struct object_id *); void free_stream_filter(struct stream_filter *); diff --git a/dir.c b/dir.c index fd8aa7c40faa..5b00dfb5b144 100644 --- a/dir.c +++ b/dir.c @@ -306,7 +306,7 @@ static int do_read_blob(const struct object_id *oid, struct oid_stat *oid_stat, * [1] Only if DO_MATCH_DIRECTORY is passed; otherwise, this is NOT a match. * [2] Only if DO_MATCH_LEADING_PATHSPEC is passed; otherwise, not a match. */ -static int match_pathspec_item(const struct index_state *istate, +static int match_pathspec_item(struct index_state *istate, const struct pathspec_item *item, int prefix, const char *name, int namelen, unsigned flags) { @@ -429,7 +429,7 @@ static int match_pathspec_item(const struct index_state *istate, * pathspec did not match any names, which could indicate that the * user mistyped the nth pathspec. */ -static int do_match_pathspec(const struct index_state *istate, +static int do_match_pathspec(struct index_state *istate, const struct pathspec *ps, const char *name, int namelen, int prefix, char *seen, @@ -500,7 +500,7 @@ static int do_match_pathspec(const struct index_state *istate, return retval; } -static int match_pathspec_with_flags(const struct index_state *istate, +static int match_pathspec_with_flags(struct index_state *istate, const struct pathspec *ps, const char *name, int namelen, int prefix, char *seen, unsigned flags) @@ -516,7 +516,7 @@ static int match_pathspec_with_flags(const struct index_state *istate, return negative ? 0 : positive; } -int match_pathspec(const struct index_state *istate, +int match_pathspec(struct index_state *istate, const struct pathspec *ps, const char *name, int namelen, int prefix, char *seen, int is_dir) @@ -529,7 +529,7 @@ int match_pathspec(const struct index_state *istate, /** * Check if a submodule is a superset of the pathspec */ -int submodule_path_match(const struct index_state *istate, +int submodule_path_match(struct index_state *istate, const struct pathspec *ps, const char *submodule_name, char *seen) @@ -892,7 +892,7 @@ void add_pattern(const char *string, const char *base, add_pattern_to_hashsets(pl, pattern); } -static int read_skip_worktree_file_from_index(const struct index_state *istate, +static int read_skip_worktree_file_from_index(struct index_state *istate, const char *path, size_t *size_out, char **data_out, struct oid_stat *oid_stat) diff --git a/dir.h b/dir.h index facfae47402a..51cb0e217247 100644 --- a/dir.h +++ b/dir.h @@ -354,7 +354,7 @@ int count_slashes(const char *s); int simple_length(const char *match); int no_wildcard(const char *string); char *common_prefix(const struct pathspec *pathspec); -int match_pathspec(const struct index_state *istate, +int match_pathspec(struct index_state *istate, const struct pathspec *pathspec, const char *name, int namelen, int prefix, char *seen, int is_dir); @@ -492,12 +492,12 @@ int git_fnmatch(const struct pathspec_item *item, const char *pattern, const char *string, int prefix); -int submodule_path_match(const struct index_state *istate, +int submodule_path_match(struct index_state *istate, const struct pathspec *ps, const char *submodule_name, char *seen); -static inline int ce_path_match(const struct index_state *istate, +static inline int ce_path_match(struct index_state *istate, const struct cache_entry *ce, const struct pathspec *pathspec, char *seen) @@ -506,7 +506,7 @@ static inline int ce_path_match(const struct index_state *istate, S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode)); } -static inline int dir_path_match(const struct index_state *istate, +static inline int dir_path_match(struct index_state *istate, const struct dir_entry *ent, const struct pathspec *pathspec, int prefix, char *seen) diff --git a/merge-recursive.c b/merge-recursive.c index 3d9207455b3a..b8de7a704eae 100644 --- a/merge-recursive.c +++ b/merge-recursive.c @@ -2987,7 +2987,7 @@ static int blob_unchanged(struct merge_options *opt, struct strbuf obuf = STRBUF_INIT; struct strbuf abuf = STRBUF_INIT; int ret = 0; /* assume changed for safety */ - const struct index_state *idx = opt->repo->index; + struct index_state *idx = opt->repo->index; if (a->mode != o->mode) return 0; diff --git a/pathspec.c b/pathspec.c index 7a229d8d22f2..b6e333965cb4 100644 --- a/pathspec.c +++ b/pathspec.c @@ -20,7 +20,7 @@ * to use find_pathspecs_matching_against_index() instead. */ void add_pathspec_matches_against_index(const struct pathspec *pathspec, - const struct index_state *istate, + struct index_state *istate, char *seen) { int num_unmatched = 0, i; @@ -51,7 +51,7 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec, * given pathspecs achieves against all items in the index. */ char *find_pathspecs_matching_against_index(const struct pathspec *pathspec, - const struct index_state *istate) + struct index_state *istate) { char *seen = xcalloc(pathspec->nr, 1); add_pathspec_matches_against_index(pathspec, istate, seen); @@ -702,7 +702,7 @@ void clear_pathspec(struct pathspec *pathspec) pathspec->nr = 0; } -int match_pathspec_attrs(const struct index_state *istate, +int match_pathspec_attrs(struct index_state *istate, const char *name, int namelen, const struct pathspec_item *item) { diff --git a/pathspec.h b/pathspec.h index 454ce364fac7..2ccc8080b6cc 100644 --- a/pathspec.h +++ b/pathspec.h @@ -150,11 +150,11 @@ static inline int ps_strcmp(const struct pathspec_item *item, } void add_pathspec_matches_against_index(const struct pathspec *pathspec, - const struct index_state *istate, + struct index_state *istate, char *seen); char *find_pathspecs_matching_against_index(const struct pathspec *pathspec, - const struct index_state *istate); -int match_pathspec_attrs(const struct index_state *istate, + struct index_state *istate); +int match_pathspec_attrs(struct index_state *istate, const char *name, int namelen, const struct pathspec_item *item); diff --git a/read-cache.c b/read-cache.c index 2410e6e0df13..3ad94578095e 100644 --- a/read-cache.c +++ b/read-cache.c @@ -549,7 +549,7 @@ int cache_name_stage_compare(const char *name1, int len1, int stage1, const char return 0; } -static int index_name_stage_pos(const struct index_state *istate, const char *name, int namelen, int stage) +static int index_name_stage_pos(struct index_state *istate, const char *name, int namelen, int stage) { int first, last; @@ -570,7 +570,7 @@ static int index_name_stage_pos(const struct index_state *istate, const char *na return -first-1; } -int index_name_pos(const struct index_state *istate, const char *name, int namelen) +int index_name_pos(struct index_state *istate, const char *name, int namelen) { return index_name_stage_pos(istate, name, namelen, 0); } @@ -3389,8 +3389,8 @@ int repo_read_index_unmerged(struct repository *repo) * We helpfully remove a trailing "/" from directories so that * the output of read_directory can be used as-is. */ -int index_name_is_other(const struct index_state *istate, const char *name, - int namelen) +int index_name_is_other(struct index_state *istate, const char *name, + int namelen) { int pos; if (namelen && name[namelen - 1] == '/') @@ -3408,7 +3408,7 @@ int index_name_is_other(const struct index_state *istate, const char *name, return 1; } -void *read_blob_data_from_index(const struct index_state *istate, +void *read_blob_data_from_index(struct index_state *istate, const char *path, unsigned long *size) { int pos, len; diff --git a/submodule.c b/submodule.c index 9767ba9893cc..83809a4f7bd2 100644 --- a/submodule.c +++ b/submodule.c @@ -33,7 +33,7 @@ static struct oid_array ref_tips_after_fetch; * will be disabled because we can't guess what might be configured in * .gitmodules unless the user resolves the conflict. */ -int is_gitmodules_unmerged(const struct index_state *istate) +int is_gitmodules_unmerged(struct index_state *istate) { int pos = index_name_pos(istate, GITMODULES_FILE, strlen(GITMODULES_FILE)); if (pos < 0) { /* .gitmodules not found or isn't merged */ @@ -301,7 +301,7 @@ int is_submodule_populated_gently(const char *path, int *return_error_code) /* * Dies if the provided 'prefix' corresponds to an unpopulated submodule */ -void die_in_unpopulated_submodule(const struct index_state *istate, +void die_in_unpopulated_submodule(struct index_state *istate, const char *prefix) { int i, prefixlen; @@ -331,7 +331,7 @@ void die_in_unpopulated_submodule(const struct index_state *istate, /* * Dies if any paths in the provided pathspec descends into a submodule */ -void die_path_inside_submodule(const struct index_state *istate, +void die_path_inside_submodule(struct index_state *istate, const struct pathspec *ps) { int i, j; diff --git a/submodule.h b/submodule.h index 4ac6e31cf1f7..84640c49c114 100644 --- a/submodule.h +++ b/submodule.h @@ -39,7 +39,7 @@ struct submodule_update_strategy { }; #define SUBMODULE_UPDATE_STRATEGY_INIT {SM_UPDATE_UNSPECIFIED, NULL} -int is_gitmodules_unmerged(const struct index_state *istate); +int is_gitmodules_unmerged(struct index_state *istate); int is_writing_gitmodules_ok(void); int is_staging_gitmodules_ok(struct index_state *istate); int update_path_in_gitmodules(const char *oldpath, const char *newpath); @@ -60,9 +60,9 @@ int is_submodule_active(struct repository *repo, const char *path); * Otherwise the return error code is the same as of resolve_gitdir_gently. */ int is_submodule_populated_gently(const char *path, int *return_error_code); -void die_in_unpopulated_submodule(const struct index_state *istate, +void die_in_unpopulated_submodule(struct index_state *istate, const char *prefix); -void die_path_inside_submodule(const struct index_state *istate, +void die_path_inside_submodule(struct index_state *istate, const struct pathspec *ps); enum submodule_update_type parse_submodule_update_type(const char *value); int parse_submodule_update_strategy(const char *value, From patchwork Thu Apr 1 01:49:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7167C433B4 for ; Thu, 1 Apr 2021 01:51:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AAC4D6101E for ; Thu, 1 Apr 2021 01:51:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232944AbhDABub (ORCPT ); Wed, 31 Mar 2021 21:50:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232984AbhDABuI (ORCPT ); Wed, 31 Mar 2021 21:50:08 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B31F4C061761 for ; Wed, 31 Mar 2021 18:50:07 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id j18so198783wra.2 for ; Wed, 31 Mar 2021 18:50:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=gJLBcZut5BdoZ62MD0BUa+X2VnGfeSA3JBLk1DbzHGQ=; b=RiLHpllmzBRpvpSNNHUm1faX0KOVHTK1BCTqiedeqkx4CnGQXC9bE3BZh+XnwzbvaY 4lZqKtmLvHqy8YL2eG4i41VjYLy7k8XZiMYXaYUYP7uZKwNKdch8YVxzK7A7Inrq3J0I FE8thFX2lBiedgEQRsSGb1huPup/q2JfoJQwcVfjQOAkfL4WFMWsCNZlF+IZOv8MslnM eYz/fse/Sva+FRz6XrvlEEt10ozO4q7vELKtujfhplRQTNa1+GNrjbiHs0EfxA9RJpU7 urew5ypk/EEzX5BiEsDj5Lax5WsCHfgpBmACkcBXTVY+Mwk0HY9W8id8OwcVzs3W5XH6 1PRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=gJLBcZut5BdoZ62MD0BUa+X2VnGfeSA3JBLk1DbzHGQ=; b=p8piZQynJow5GnTGfWrBIiFisYkJdMQzZBSjOb+ts+4MeHxRwD6IQDD3C1MmUphv3A U3iKFSbFHV9WVduHecVvgvIC5bjpTrDZxvtDMuDT1jizknhRuBy8r8nXF/oxjuJoKVkd KzUBOXBBZ0zINEB1MUXSMNCwobQty5UKIpOt2nvb0r3H9u4bR0yr+p+0LjEKcIND8rM7 ZZpvdvW/ilftDMDlgv2VCvTyFHAdQibQyU2XL2NX6ZJCR3tK1E6Xo/0wXS4YjfNEaouP x8dcV4VNwJ2YuCrl7xTTG/uciOZXOfpkWJQ6PrXkZ4eE9/oSoJQwXryVV0u3ZgEBWADk 2DxQ== X-Gm-Message-State: AOAM53309L84ImN72AzAfKRVpECPjBhXr2HQGKEM39YyVI5ob2RSqyXq BSvVHRN/7RMFodNgsGsrijSeCaKCCK4= X-Google-Smtp-Source: ABdhPJynMIj3mYWDJaOBxbcIDknr/r9xrY9gVs+pFuB4jw5LsM7ASIjBqSgDF5gGNdN0+M+RYTgwDQ== X-Received: by 2002:a5d:5088:: with SMTP id a8mr6737356wrt.294.1617241806578; Wed, 31 Mar 2021 18:50:06 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h9sm6208369wmb.35.2021.03.31.18.50.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:06 -0700 (PDT) Message-Id: <737d27e18d64d7151447eca90e74bf58451cc9a6.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:40 +0000 Subject: [PATCH v2 03/25] read-cache: expand on query into sparse-directory entry Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Callers to index_name_pos() or index_name_stage_pos() have a specific path in mind. If that happens to be a path with an ancestor being a sparse-directory entry, it can lead to unexpected results. In the case that we did not find the requested path, check to see if the position _before_ the inserted position is a sparse directory entry that matches the initial segment of the input path (including the directory separator at the end of the directory name). If so, then expand the index to be a full index and search again. This expansion will only happen once per index read. Future enhancements could be more careful to expand only the necessary sparse directory entry, but then we would have a special "not fully sparse, but also not fully expanded" mode that could affect writing the index to file. Since this only occurs if a specific file is requested outside of the sparse checkout definition, this is unlikely to be a common situation. Signed-off-by: Derrick Stolee --- read-cache.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/read-cache.c b/read-cache.c index 3ad94578095e..3698bc7bf77d 100644 --- a/read-cache.c +++ b/read-cache.c @@ -567,6 +567,27 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in } first = next+1; } + + if (istate->sparse_index && + first > 0) { + /* Note: first <= istate->cache_nr */ + struct cache_entry *ce = istate->cache[first - 1]; + + /* + * If we are in a sparse-index _and_ the entry before the + * insertion position is a sparse-directory entry that is + * an ancestor of 'name', then we need to expand the index + * and search again. This will only trigger once, because + * thereafter the index is fully expanded. + */ + if (S_ISSPARSEDIR(ce->ce_mode) && + ce_namelen(ce) < namelen && + !strncmp(name, ce->name, ce_namelen(ce))) { + ensure_full_index(istate); + return index_name_stage_pos(istate, name, namelen, stage); + } + } + return -first-1; } From patchwork Thu Apr 1 01:49:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176789 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D405C43461 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D4C946108C for ; Thu, 1 Apr 2021 01:51:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232982AbhDABuc (ORCPT ); Wed, 31 Mar 2021 21:50:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232989AbhDABuI (ORCPT ); Wed, 31 Mar 2021 21:50:08 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 353C6C061574 for ; Wed, 31 Mar 2021 18:50:08 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id k8so195514wrc.3 for ; Wed, 31 Mar 2021 18:50:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=icvbxp/b5Ftybmb6oGJlzeU1p7Pw4+4yErHEotpNNtc=; b=IUWz1fU36WdSnA7SuNbfbSVZLkzbqdv1jGqePNUDzJjwZoXw4F5WzlV/5DGARkuph4 bhewh2cBQt6O1dVEGay/eB011qI1zmBQ1MA7p/ROM6Nsi26MjF+B4zrQIDFvKN0s192L g3M7OcqQpevbj/BxyW+J5VQKRv0rdtTkNXN1XT+bfz3chrDmUh0HkIO2yER27fgZlM6Q l+TenWCTCElk/Jg+B3FQIdta7dCtLkA6dxfHneWfYddBfZRDRbtnyf3/QO8+GM3Uq335 h9ufs/vag+01Q8YdNXmWIIPudVqRWFzp7iYwIgUGS9lU6bZUEcPf2e4E2BrA2CxOPS80 0zCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=icvbxp/b5Ftybmb6oGJlzeU1p7Pw4+4yErHEotpNNtc=; b=Z0oNpV7AsN55byeM5vvGVySTeKG/KfMtgSaYzTLETfMlYPydUz6n3MoeZ7G5S2CzTU jkCX/8Zt34KJR74XVx1nJeqMgmy9qmZHK8a2LwcHf6R8XJGTTn+3NT7YM0SG95UqUUnv rtOME3ApQ2zBZ6cVeNKtYKBBds9WzEjc41FVUapYcZNN39IcqcH2F3Gt6ZZk7DgfH+Sm aJoIsrS60trCCFAH5uxbzxqzH3ieyTVWS5Grp/19TdR8RzBtX5WKosnGovzYyaIUv5ZL AeGQYDkQl+x4/mjllOdeH4LQxuyTV0H/ZL3SMsn0zLCRPeNXSxzkYAjlZj1bGxX/lw4w KYHg== X-Gm-Message-State: AOAM530D84IEghMIFdY/KdWMv8tSwi+O5zmp8mAmC/Yzekb0/joExyeU PYU6eLb5i93MW7Z+wJBCjN8Uaft96Jo= X-Google-Smtp-Source: ABdhPJwgCKZLCSCQRoAtlXZpn3UPAPitPzDKQ1zVcblziclc2D7UTSBwBo3ayIC7ocNnzGFfEvPHSQ== X-Received: by 2002:adf:e412:: with SMTP id g18mr6834167wrm.159.1617241807061; Wed, 31 Mar 2021 18:50:07 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e8sm6210936wme.14.2021.03.31.18.50.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:06 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:41 +0000 Subject: [PATCH v2 04/25] cache: move ensure_full_index() to cache.h Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Soon we will insert ensure_full_index() calls across the codebase. Instead of also adding include statements for sparse-index.h, let's just use the fact that anything that cares about the index already has cache.h in its includes. Signed-off-by: Derrick Stolee --- cache.h | 1 + sparse-index.h | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/cache.h b/cache.h index 5006278c13ca..b7e20e9778db 100644 --- a/cache.h +++ b/cache.h @@ -350,6 +350,7 @@ void add_name_hash(struct index_state *istate, struct cache_entry *ce); void remove_name_hash(struct index_state *istate, struct cache_entry *ce); void free_name_hash(struct index_state *istate); +void ensure_full_index(struct index_state *istate); /* Cache entry creation and cleanup */ diff --git a/sparse-index.h b/sparse-index.h index 39dcc859735e..0268f38753c0 100644 --- a/sparse-index.h +++ b/sparse-index.h @@ -2,7 +2,6 @@ #define SPARSE_INDEX_H__ struct index_state; -void ensure_full_index(struct index_state *istate); int convert_to_sparse(struct index_state *istate); struct repository; From patchwork Thu Apr 1 01:49:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42917C43603 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2B44F61078 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233112AbhDABuc (ORCPT ); Wed, 31 Mar 2021 21:50:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51804 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232994AbhDABuJ (ORCPT ); Wed, 31 Mar 2021 21:50:09 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC898C061761 for ; Wed, 31 Mar 2021 18:50:08 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id j20-20020a05600c1914b029010f31e15a7fso2034644wmq.1 for ; Wed, 31 Mar 2021 18:50:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=dTUCQB3WXCizXGIVy+rJNkDOcat4/a7nb8eHNj32aL4=; b=MDw0DO+UxfNC7ypccxote0qn95kGrLcsEEI8f6TixkLyc5kj9PbnovwUNhCnLdLQ0f X0RBi3IiJ+K8gNsTOA3vrj9+5SkglbD8RvyVxO9C2b8LyjpJdb2lwx4DtFpChtjSwHms buJ8od/NuFG8DHKxHatLxr4MVNGt0QHrRAjKY/3hKCMZpT7GQIPFe49M8GLIWXdB89Dy nKlmaxCntDzjJPShgG3OZ8YR+w/3SYux02tKqVSg7qslnM+6BW5MH2jW0gPbaAAmodBi 5IemE/G2m6elasys23XUxEXNACSaOcUpEQ/2hMWkuPMqUcyR41ppSN+IPTrlWNNhAEvZ IT0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=dTUCQB3WXCizXGIVy+rJNkDOcat4/a7nb8eHNj32aL4=; b=OlituhsR8REzS4O1RDnopT3aZ2ES4vvhGm41I4g4kDsmknwNmGmLRjc1PdjCz3A0HM f9itenfsNUF7o0V+E2G+F+Kw850aHWyEgA6FB7ez+Xtav5VxemFbkpFG9GkB/IODBJUK +ck9Yi9DdWBN/217xdzZGZVC/2zxOWwbBKfK9Gluy/AM8sDcWr9jx6x36WYlcYPsyEpo Z0y+B17vYq+vP+Nqpmzx/ccKm0BWoLr6cTsnr1kovtm6JiF3R/Hx/zQRHsWw56qIsHQJ 3gJG9JV4OoJeNMAcaFgvTmRIpAcAmyO91mCaPvh4xdAFZjNeiWcJ76UKVUpxyWM/eXVP YRZQ== X-Gm-Message-State: AOAM5320pgo4sO7GSL4z+303eACUW7hfKyV5DUeAWkmW0KtVPjgL5hjp TBUreoNpoiBoeXqLwHF/juzFHwWk2W0= X-Google-Smtp-Source: ABdhPJzmcLXvPohfLnRJlR+t0wM7B3QX17IGdyGXer2SwyhzFbGap3RX8TFjZVNnnQ+kwpxiGBitIQ== X-Received: by 2002:a1c:7415:: with SMTP id p21mr5621013wmc.187.1617241807566; Wed, 31 Mar 2021 18:50:07 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j14sm6902866wrw.69.2021.03.31.18.50.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:07 -0700 (PDT) Message-Id: <4a5fc2eb5a9fab5276ae502a7fd6e35309d6f17b.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:42 +0000 Subject: [PATCH v2 05/25] add: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/add.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/add.c b/builtin/add.c index ea762a41e3a2..afccf2fd5543 100644 --- a/builtin/add.c +++ b/builtin/add.c @@ -141,6 +141,8 @@ static int renormalize_tracked_files(const struct pathspec *pathspec, int flags) { int i, retval = 0; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) { struct cache_entry *ce = active_cache[i]; From patchwork Thu Apr 1 01:49:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 897A5C43611 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 642A061078 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233145AbhDABug (ORCPT ); Wed, 31 Mar 2021 21:50:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51812 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233002AbhDABuK (ORCPT ); Wed, 31 Mar 2021 21:50:10 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D741C061574 for ; Wed, 31 Mar 2021 18:50:09 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id j7so201719wrd.1 for ; Wed, 31 Mar 2021 18:50:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=YIfeJaOwJ2PF4AvCIcf7mIqwTKwspa0fWugQUcTQFgU=; b=O2Sdfhz0003IJNqN7DwyqlPld8PYDHg/VjHV7XRol+seGtZ1WiJ8eFQFaphSnFpEOy WIbQcofwxAkRjc0y0jjYQdtmnz17L8j14vJDysmiEIm1Jo96aJ6vlFGK0/VBjcHmY5hd Av3HSO8K09aogQQ9UeXfH2tZamX0ap6/gHEavBzUGuZDgvEvCZ6K225ZY4reTauKyjvX YuUpdtdBMA8ds7T3eqVXj+Pc81S52fdx/aFrrwO8y8OYuLIMvuAyyiiPYgxWacbkwWTV tYZ+6noceNusDBrmHnKahvjpeLcQ4l3vcarRl7PdhhJ4gtZHMqRykpsQ12vWuVXAVnOt 1HcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=YIfeJaOwJ2PF4AvCIcf7mIqwTKwspa0fWugQUcTQFgU=; b=bpgmGACVZ51tJyy4UTpE/blvcVIRJz8tMyCSiu2AzMpSTZEbtdGGQ3B3uNprDj1icw revvFhlJEUOkDLJIrYiWRe8ZatjPfeyPAX3V875vyYpERPzeyr4vWykLLuc0Q4jQkd6w aebtukJwsM5JZKqOX5NWnsHNZk44Q7gZiO27NaIRkOxPjzxD/MDxx3ohKk7o7Ea590Qa F3ZYqdzk3WDWmWnQF/A/3Roq4H5SxSIyQnwfoOycDdQHD2WJNl9+mEWsOL67ksB6fIm8 j9g53TYuooFXteIXvGu1dptX+2UVloI9H73peh4jnakRoBdjGDsmVAHZMDbXxWQ+1w7F 8U0g== X-Gm-Message-State: AOAM532yUZvIT+hyUoo3W0TKNLw+ShxKarD3rLB7DXDamMJNkDhFS81f 8TQELenofnWsRov1Ax1UcRRHRKRRG8k= X-Google-Smtp-Source: ABdhPJw4kIrNvoHxxI/cuEnWowCCxEa+ayOFPDGEa/6FtYmvVUjdloe82bFDvW7+et9UDkAOfRqRdQ== X-Received: by 2002:adf:9d82:: with SMTP id p2mr6686484wre.234.1617241808102; Wed, 31 Mar 2021 18:50:08 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t1sm8682603wry.90.2021.03.31.18.50.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:07 -0700 (PDT) Message-Id: <11c38f7277c5217b13065ee4c53f6f94e1c6a7a9.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:43 +0000 Subject: [PATCH v2 06/25] checkout-index: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before we iterate over all cache entries, ensure that the index is not sparse. This loop in checkout_all() might be safe to iterate over a sparse index, but let's put this protection here until it can be carefully tested. Signed-off-by: Derrick Stolee --- builtin/checkout-index.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/checkout-index.c b/builtin/checkout-index.c index 023e49e271c2..2c2936a9dae0 100644 --- a/builtin/checkout-index.c +++ b/builtin/checkout-index.c @@ -119,6 +119,8 @@ static void checkout_all(const char *prefix, int prefix_length) int i, errs = 0; struct cache_entry *last_ce = NULL; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr ; i++) { struct cache_entry *ce = active_cache[i]; if (ce_stage(ce) != checkout_stage From patchwork Thu Apr 1 01:49:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FCA6C43616 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7964361090 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233139AbhDABuf (ORCPT ); Wed, 31 Mar 2021 21:50:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233015AbhDABuK (ORCPT ); Wed, 31 Mar 2021 21:50:10 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA683C061762 for ; Wed, 31 Mar 2021 18:50:09 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id x13so174883wrs.9 for ; Wed, 31 Mar 2021 18:50:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=rzO06S1ief0qGjiR3z1dlgur8sjPW9z+JkmBA4FJEXI=; b=FkP2dQCFQxNP5RU7x5TH+yakq5+nQRO1K24j1RjBaijghGPUQlkXDrNW5VbxO3Ldza TQV49VhJPo3knJ0Cirfb9TptbP3Siajx1h55TfyeausVUH3TJrkzW8VLokW2YCIQDOcm a526mGubfuzGDAQ/7RqWYgFFlLfr4BqMIPi9n3VXsG7+vlXz2D5mhJU0dwwzsGnaiZSN ZgCCOhVI/akaz9XDDfT1yj+6coW5aM0T99BEbf67gR1WGtK4dQQ5Bce0mkdj0M9CR4Is AqLd+FE4eUE+h+44z+Sg8JG1PrtVLKXnoa6Bkmr9ULWSghlx8gmH40A7RpGyanBrVCmi 4Q5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=rzO06S1ief0qGjiR3z1dlgur8sjPW9z+JkmBA4FJEXI=; b=nMCNqaOAycI0lkAAkjhem/c8pKdc3s3TpX5Q1qIpyXB1HLJXMfsWqG263BV7VEpLRB UTeIEoX74xwXAU4bxttRdlQFRATfy90sCNdaBErii4iQJO82NLTPIsgEARNxgbD/2M0x Tyjc6nnWpeFMRFEDFm5AegPvjdhasm/ZG8gqnbRCPeD8ZmhkQs5rX/gx2ShQdhVcpZMG NLSSswoRlJs3i/HLi+votgJRXlV/wIQkLdKVBD26aE/bk3xyN1FMmmvJpHKLbdSdC3bS qnoe4hi84ki61JOAhnlhkEDXCVv4ERygKhr8f6MA6vi0rDgIFQfq/AraeJKX88lMKhLo A9yQ== X-Gm-Message-State: AOAM531+/rlNVKhR4Hl29Mv+a1xjmlWjOgi8awdNv36vYiQTQ0OBN/eI OKZD9Bbe3lbHAMcCOMjoRiGlx7Xrv38= X-Google-Smtp-Source: ABdhPJw0bbCxbiaefF9NyI0gGJSLJvCM0ItX9DXmcJbjt/56pNE510RaQ9OcJN0WWctBbmMe1e3gdQ== X-Received: by 2002:adf:b355:: with SMTP id k21mr6725711wrd.156.1617241808608; Wed, 31 Mar 2021 18:50:08 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t14sm6783151wru.64.2021.03.31.18.50.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:08 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:44 +0000 Subject: [PATCH v2 07/25] checkout: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries in the checkout builtin, ensure that we have a full index to avoid any unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/checkout.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/builtin/checkout.c b/builtin/checkout.c index 0e6639052001..d0dbe63ea119 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -368,6 +368,9 @@ static int checkout_worktree(const struct checkout_opts *opts, NULL); enable_delayed_checkout(&state); + + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (pos = 0; pos < active_nr; pos++) { struct cache_entry *ce = active_cache[pos]; if (ce->ce_flags & CE_MATCHED) { @@ -512,6 +515,8 @@ static int checkout_paths(const struct checkout_opts *opts, * Make sure all pathspecs participated in locating the paths * to be checked out. */ + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (pos = 0; pos < active_nr; pos++) if (opts->overlay_mode) mark_ce_for_checkout_overlay(active_cache[pos], From patchwork Thu Apr 1 01:49:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61764C4360C for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 52514610A1 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233131AbhDABue (ORCPT ); Wed, 31 Mar 2021 21:50:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51816 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233017AbhDABuK (ORCPT ); Wed, 31 Mar 2021 21:50:10 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 434D0C061761 for ; Wed, 31 Mar 2021 18:50:10 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id j18so198867wra.2 for ; Wed, 31 Mar 2021 18:50:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=zERwEV3I5iAUQNSYAsx1jxOMzyYF1N8NzKg79Kbu40c=; b=XYbWCF5VixSPpdVkZ5uosSKvSbYDJJQSVP87GF1jdPIfDu8CpMwAv4U7LD+ANYUpv4 a951OTzRNNKkrosIwWx/FET9srYsTkucfUDcqQDIg/wk/pzYySSyhGDaB5yJcyRxgala mq/kGuQSAm9LkpYlHHRG8WXvlrypYcVvghvgZ70vkq0J7QDatWgnyHvek7tSlINBXLDj /iLB7KB42Y/noO/6t7g/7cnDYf8e5Gk7B1FyOrHPzFQqFRVDHLlKqPhTWKiBYKGLdZs0 v1jyrnvFSudDod3UfwqzTu4GCjg3O0JgCAm0nL4IufENM/5zuFV9f9EWufg0bby9pmRd vzXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=zERwEV3I5iAUQNSYAsx1jxOMzyYF1N8NzKg79Kbu40c=; b=h03pEQzs8/wjBvmWDAFtOvtQodNFcz/RZlAm2jizrE0Sfj4WNBAfaCSnHICRgtUPvc sYKOKRFOtwHTef9qnv6pAvguIiPqjdhUcw3jnwpMkl+8J19/dSiwBqT4Oy2O7pL7Olzg frAYoKprxu1FZ6i1ACpxnUz0M+0tDfMHDGSphkYQgc1F8Jc+Zngisr6QAMnfcd9fC7fb JsGFo4c2KVVT9OowZeSv2E+qpnjCNdTBj7NnZmuzsIV1S+YO0AnG74INi7J8mlo2sKNg E5oA3FWdUN0dQlW1zpTs/cf1HhxOMbQdogQxpsMrAWfkSV+w2n3vbIJ8cnViQA6hQwK4 CLqg== X-Gm-Message-State: AOAM531J1CKnWYRbmLWLVXIBR4hnt8K/1B8DiTekVL7QIZb2HLyxlmy0 6YUZ87clFMSsrT2hAYAZ9NbLTSOn0ko= X-Google-Smtp-Source: ABdhPJzZD8lXfG8P+xeuaw/YSUO8kf/7gLzCtrikSqdQ5zRBJY4WRdxqCMZKslUPoHN+DNQQC8lnqg== X-Received: by 2002:adf:c752:: with SMTP id b18mr6626282wrh.233.1617241809123; Wed, 31 Mar 2021 18:50:09 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a14sm6743955wrn.5.2021.03.31.18.50.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:08 -0700 (PDT) Message-Id: <65704f39edc971a2f866d6a526d04ea2ed1927ae.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:45 +0000 Subject: [PATCH v2 08/25] commit: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee These two loops iterate over all cache entries, so ensure that a sparse index is expanded to a full index before we do so. Signed-off-by: Derrick Stolee --- builtin/commit.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/builtin/commit.c b/builtin/commit.c index 739110c5a7f6..cf0c36d1dcb2 100644 --- a/builtin/commit.c +++ b/builtin/commit.c @@ -251,6 +251,8 @@ static int list_paths(struct string_list *list, const char *with_tree, free(max_prefix); } + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) { const struct cache_entry *ce = active_cache[i]; struct string_list_item *item; @@ -931,6 +933,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix, if (get_oid(parent, &oid)) { int i, ita_nr = 0; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) if (ce_intent_to_add(active_cache[i])) ita_nr++; From patchwork Thu Apr 1 01:49:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CA36C43600 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3C6466109F for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233128AbhDABue (ORCPT ); Wed, 31 Mar 2021 21:50:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51820 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233020AbhDABuL (ORCPT ); Wed, 31 Mar 2021 21:50:11 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC0B2C061763 for ; Wed, 31 Mar 2021 18:50:10 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id g20so235051wmk.3 for ; Wed, 31 Mar 2021 18:50:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=/kQ/3c8UbEqQOrilneslkKpts4OYd95dOalBYXNQd/M=; b=QuoE3btE/OXFegoi+cHKmImwdgwRDw+fLHEekPg/JSuvSjLMTAYq57z13YfRselOjl 5qDC7MyWyBfQMy+ez8drxCaoF5pZISl/Tts1I+wtEQOFel7dkPtXlKoJ6WtbwSvT6w3k a5886rASMaK+XR5w1NiTGQsG/sDS/nR5DMYhEWgQPffzTnXYaVHkRlFePSNS56At7WfR xIWtyaCfGp+/F5Geo97R2w+34Gbr3M4zkV4jRrf7IeROPZgGNJKbmNBpvo/8ltNSZvnG P+8V9tIntvWmP5Bzypy5dsA4YeA/e18V60sVW1npcuOmZTaGprdJzZCVSik6grnFL4OJ NiLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=/kQ/3c8UbEqQOrilneslkKpts4OYd95dOalBYXNQd/M=; b=OBVxtirOvi43aq7YbNyMYzN43M/kyXUhrT4cXtxBTTff2q76r3KFP8d9H+Q8wEslwf 8Blw61jrTWcgXnOgntFAVOKEIEcRmij8m8wjKovJ7MjokCrUctd4z2EwS2LbwtHUmVi8 +DQmso8021MpcJFHtGvU64YCBZBgcxkemD0dZbLZMROaTvqkdKsPootjMcs70GIcssdO rLVdWhDwRzxcAvMeYytn0Qadv17xgE+qt45Gw1T2REH5RHPd6KbWGJyD6ZA18fabCZSr AtWWThLr/t0lh14mzeFw3kK2IYQNOLwVSTTuFDRiE00mebUd8cbDtYFhSaDXXRnidzpz wL6Q== X-Gm-Message-State: AOAM532vBSEh7CRYm7z0e0YgztAdhPw0mZDjBvi6UBBBj5+hnyrSFj22 43An3djpLPGaPECi0hFlvaQ3N5HNmzs= X-Google-Smtp-Source: ABdhPJzM9YZU8N9Kf39DUtrdBlT6JeF2kG9jJYoT6ODeuLisSOZdAJdbxUmfphNFC3lTbjIDQMglZg== X-Received: by 2002:a7b:c047:: with SMTP id u7mr5649013wmc.98.1617241809683; Wed, 31 Mar 2021 18:50:09 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g9sm7152629wrp.14.2021.03.31.18.50.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:09 -0700 (PDT) Message-Id: <739f3fe9edf276555cfe153e6f165b0af68a72ba.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:46 +0000 Subject: [PATCH v2 09/25] difftool: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index has been expanded to a full one to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/difftool.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/builtin/difftool.c b/builtin/difftool.c index 6e18e623fddf..32c914dde6a0 100644 --- a/builtin/difftool.c +++ b/builtin/difftool.c @@ -584,6 +584,9 @@ static int run_dir_diff(const char *extcmd, int symlinks, const char *prefix, setenv("GIT_DIFFTOOL_DIRDIFF", "true", 1); rc = run_command_v_opt(helper_argv, flags); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&wtindex); + /* * If the diff includes working copy files and those * files were modified during the diff, then the changes From patchwork Thu Apr 1 01:49:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DAD9C43462 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0B6EE6108D for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233119AbhDABud (ORCPT ); Wed, 31 Mar 2021 21:50:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233024AbhDABuL (ORCPT ); Wed, 31 Mar 2021 21:50:11 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 658E3C061764 for ; Wed, 31 Mar 2021 18:50:11 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id j9so166923wrx.12 for ; Wed, 31 Mar 2021 18:50:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=63cgv+d6hF7bJxk9XJMcdXLWzyM+y2aIDW1QZ5VUN1A=; b=A7m9rt6w4Hg8Pe/0RAnlaW4T04G2vGCJtMz+qDFfycG8RmVzJz4n0PVn5HRI/TOqH0 flHy9R6CxbimC7w+vitU9QAQzjajxlSKq13slmau+nyEDPTEYF5qxAlr0191cHU6Wny+ WLEkIF5Mt7zC8LAXt7gI4FkI9f05PKtnribZkItQZ++yiTbaLNJKS2uaQWibKo6grF2g yc4VH5OkW467OSR+ymukNk7Q01Rb659hqHqOpjhGHZeWmi5WRxwfHscm3v65r8seuKFY rpUU5+h66liIdX08plYszx1pYLSUwau8QV/nvsYaD7UilVk0NgpVa59yqYBs49kg25I6 454Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=63cgv+d6hF7bJxk9XJMcdXLWzyM+y2aIDW1QZ5VUN1A=; b=QFi90t1N777mdNflzlMWReXb9eyXY3wTiStbrcKcOwPRm+iYL2h4GBVIYVycXNWIJK Kk1JBBora3zw2qimgL2O7rvMXb4QNjIzIcs2pMNSwbfFibsaj102FH04Gx/yr+t76MUt kW5pO6JC0WuR5TM/Ho0XzLIZxGG4Js+7XMa1vHhwrDt0fOYE4VNH0DCl0xFkJwJ+dAHg MBsiE10M4O4ki+k3JMGl6xj+Yz1sjoWulC9wAQSLP+7SEhgte+uLF4oRwigjyKup+VeB X2h9JvphdBICiZ7AZYhpkZKaUf7cMbPFd+uYzBExVFjzeh69e8xGRaLIpBBQ16Yrymom doxQ== X-Gm-Message-State: AOAM530aRYtnZxFM34jpCVet1B95OOJFg5i8O3anNx8JKGEaB+x7axHW eHsADoufxmhuV9sqmiBajoX/KKgwrTY= X-Google-Smtp-Source: ABdhPJzVCgnJ1cuMPfvzkMmeOZyFWDiacepj6qPOd2kFsQRX7OtmBpHvUDFAx42M6fG8HCxPltq/fw== X-Received: by 2002:adf:e4c7:: with SMTP id v7mr6701076wrm.245.1617241810254; Wed, 31 Mar 2021 18:50:10 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a6sm7783968wmm.0.2021.03.31.18.50.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:09 -0700 (PDT) Message-Id: <779a86ad1ec46f53ae38b17cb53df347b01576fe.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:47 +0000 Subject: [PATCH v2 10/25] fsck: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee When verifying all blobs reachable from the index, ensure that a sparse index has been expanded to a full one to avoid missing some blobs. Signed-off-by: Derrick Stolee --- builtin/fsck.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/fsck.c b/builtin/fsck.c index 821e7798c706..4d7f5c63ce0d 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -883,6 +883,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix) verify_index_checksum = 1; verify_ce_order = 1; read_cache(); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) { unsigned int mode; struct blob *blob; From patchwork Thu Apr 1 01:49:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9659C43618 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 99E9E6023F for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233153AbhDABug (ORCPT ); Wed, 31 Mar 2021 21:50:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233026AbhDABuM (ORCPT ); Wed, 31 Mar 2021 21:50:12 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E71BDC061574 for ; Wed, 31 Mar 2021 18:50:11 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id z2so189830wrl.5 for ; Wed, 31 Mar 2021 18:50:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=AqzgF3ydQhx2SGKGBO9a/JEpTg8x4sKJCQWGxp4c3hY=; b=fN88q7UJGQffTF18jUED8/zfp2PAl6MCduHfnLKQbBZWRLr4SLa82m85UHfXWfH0SF OULuBmQFJQustz1ouY08t7SsDwyu9A6Qc62xcqQFA7Io57hRjmNB+1DVV1XRImwfLBh/ Y6dNLOV2LkVe3yfcCsJmYObHYsx0U8I4M4uR3lYoi7RCiBxmN3z6COAg9QuujZYUZxZQ hsLIfTzTrMVNLDKOD450RMBuOGh6vyKipjncLN8N+uzjozpcVTkQXHx2D3Xf1g9mOyvi /8I83dMK1pNPgZ62tcThYmyk04TDqHy6W83O25gnykuznPdNdN+5eyITYcJ9A2FiC4if 7WQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=AqzgF3ydQhx2SGKGBO9a/JEpTg8x4sKJCQWGxp4c3hY=; b=pEm5g0PuVJ5hABhrUQUV/MwnP3d/LHLvWOZec8IsHvGwZPPa0PTnryNVr9VAvW1DJK T7LXSjSmGSKZRh9c9zq/WAtqr5LuPQEFEqlc9o4FJlGFzyocdIgPb47XL5N89krws3TN zfwLkj0aZCrOyWbU6XUyJFrSfU6WjepEmVK5rqh2jTRTjIaz5FFC76fRkoo6iK1Y5HH1 HWZj5xasXY1L0enAGy4dhDISXV+MDgI9fl4BswQab2ioaQMntiNN2pSmaBB1pkCL8tKZ 17BxbsvkbpRV24UYAUuP2aIlhMCddZn4DWf+XltEUJ7NoCWTqktgUBdEEe1MGZowZEqu oZPw== X-Gm-Message-State: AOAM532Xmkz5WK9LlMRYiIFNM291dwvcGlCD11LqeNSDEvK3zuWz9hGX 3U+68Raj/wNLQlLWprVwGhjq7cWPbPs= X-Google-Smtp-Source: ABdhPJxFNRHbzB6qtwp9fo9zTg3sAROqaWTERr7kEWB48WGShcRcK1zqXfrvZOIEWkXu2ydAowdzgA== X-Received: by 2002:a5d:410b:: with SMTP id l11mr6727379wrp.16.1617241810791; Wed, 31 Mar 2021 18:50:10 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n6sm7362554wrt.1.2021.03.31.18.50.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:10 -0700 (PDT) Message-Id: <8c0d377054fa126f6b7426f8d87cd9bd57f5328c.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:48 +0000 Subject: [PATCH v2 11/25] grep: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full one so we do not miss blobs to scan. Later, this can integrate more carefully with sparse indexes with proper testing. Signed-off-by: Derrick Stolee --- builtin/grep.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/grep.c b/builtin/grep.c index 4e91a253ac3b..c2d40414e975 100644 --- a/builtin/grep.c +++ b/builtin/grep.c @@ -504,6 +504,8 @@ static int grep_cache(struct grep_opt *opt, if (repo_read_index(repo) < 0) die(_("index file corrupt")); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(repo->index); for (nr = 0; nr < repo->index->cache_nr; nr++) { const struct cache_entry *ce = repo->index->cache[nr]; From patchwork Thu Apr 1 01:49:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2F67C43617 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8C08261077 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233155AbhDABuh (ORCPT ); Wed, 31 Mar 2021 21:50:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51816 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232951AbhDABuM (ORCPT ); Wed, 31 Mar 2021 21:50:12 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7908BC061762 for ; Wed, 31 Mar 2021 18:50:12 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id x13so174970wrs.9 for ; Wed, 31 Mar 2021 18:50:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=DOjJ2jNd1O2RVKGUUk1xSCeZyh/PdDSjmF8DqHmFsvk=; b=o6SWvgVcqKO5Wt06pA3w16hxHxPwZXMujbmXKWUrRpBXa3+M9142ULayexMaRtrCre T8fgMd3c9D3mYHuQZkU6sHPDJ2iJcYhprGLyNN+phl5xK5eL18wDLnD0VaR+w5AGrmE8 9M+RdQyuw4xP7NzgWGT+O9gZTu35d4x+vmj1bhLS9JvCYKr8oS4EWb3czINe2deu5KpU 79lWhQFqqw0gDpD+w3kXhhmpv077IImsdAy9EbFY0DeMhK7BacuFOiyE3FbJGOGoCUeD w6dE+Va6NYdhYHYkG2arqtEVVsyp+dV0TUFazgRu73jmK9cqQ8eV0WzgsLkq42ry/TIp +PQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=DOjJ2jNd1O2RVKGUUk1xSCeZyh/PdDSjmF8DqHmFsvk=; b=cjZcMZdnCuapvGCqwmKCvFJ8XMM+0VSk+ZfWNuxG819im9xN3tA4hmAMjfdwpE85Ev tXvesxqavtnNjG5eHZ6neBgdAmJmjPSBFuDq9wNm483HzFwL32SmrEn+QR+Ydxprav9F i6mmrsZRha0B7g5lf7ZZ4xOWBumH+RiPUPas1LoLtUGtynX7oCciEPf3ieB6oV9tOSOz cljDkMf3kSLSUhxCVGz49esBtPm8D8bKmBSvDzWeIzhEsztbRTGbkqttvdaEa7WCoZEC MAjfUm5CfPN5gaOHC199+hLa78tT56CpxGN2Lkm6/zmTaR+07GqpcT5T5SBRGgC11QbN pmaQ== X-Gm-Message-State: AOAM5338I+Ens8U0EGjQX+8BbgOcfye/PhVXGnAKMpCDXqza0urOf+BK QtQ6HM4RLpu7xBHzh4ng54Zfqw374fA= X-Google-Smtp-Source: ABdhPJw4Bm4NCUPG8wqGv5A5bflNl8Cz/Nsaf7WwKJS0+TsnGvsfBuun7AW7EGNhFKBylXTDa5DUQQ== X-Received: by 2002:adf:f4c1:: with SMTP id h1mr6694783wrp.71.1617241811332; Wed, 31 Mar 2021 18:50:11 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j30sm8018952wrj.62.2021.03.31.18.50.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:11 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:49 +0000 Subject: [PATCH v2 12/25] ls-files: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full one to avoid missing files. Signed-off-by: Derrick Stolee --- builtin/ls-files.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/builtin/ls-files.c b/builtin/ls-files.c index 4f9ed1fb29b7..a0b4e54d1149 100644 --- a/builtin/ls-files.c +++ b/builtin/ls-files.c @@ -317,6 +317,8 @@ static void show_files(struct repository *repo, struct dir_struct *dir) if (!(show_cached || show_stage || show_deleted || show_modified)) return; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(repo->index); for (i = 0; i < repo->index->cache_nr; i++) { const struct cache_entry *ce = repo->index->cache[i]; struct stat st; @@ -494,6 +496,8 @@ void overlay_tree_on_index(struct index_state *istate, die("bad tree-ish %s", tree_name); /* Hoist the unmerged entries up to stage #3 to make room */ + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { struct cache_entry *ce = istate->cache[i]; if (!ce_stage(ce)) From patchwork Thu Apr 1 01:49:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176821 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61C43C001B3 for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 504FD610A2 for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233244AbhDABup (ORCPT ); Wed, 31 Mar 2021 21:50:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233080AbhDABuS (ORCPT ); Wed, 31 Mar 2021 21:50:18 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C5BBC061761 for ; Wed, 31 Mar 2021 18:50:18 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id r10-20020a05600c35cab029010c946c95easo75257wmq.4 for ; Wed, 31 Mar 2021 18:50:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=CJjXReyOnuxiS+rkr7b3ZvpKZ1VCS6UDhvoKhleGnNw=; b=ecLhslrSl3bH/hqP8Iowj17+AkMdI77L5YAVO+jAD/6FjjFgn0jmKvbCH28yUufgmV HVaJdYzjkNQCyuYiZN1KxIIMWKrWp1Q2w/cGU2GeLr69fzOnNw4uO1vufghF21ccFr0t 07EWjM+TAnm69/fcW8E0AcgembMd6fLi9B37+T36ofRYXLHdJjHcQ/+MM76V30bFJSCg oS2B06rJHYBxPT+/AoHbGTt0JiiIB91t2GYD3k99GIgPLTfnbuJ8M7NtfXCdtOTRDH+v PTvWOOYrVII+9PoPaT3+0i70Zl4YSGRoJ8JLM2/Bzlbm/uYRyDjv7TKvHYh9jP2pWemb 8zIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=CJjXReyOnuxiS+rkr7b3ZvpKZ1VCS6UDhvoKhleGnNw=; b=kmH+2MxqYy9BHqDg1R5rkKkWe9daRCV5Ud/IlY08q2FhpwhGDre3XZjkhVyCTmInPc t0Si1hcQqNdSp/5u5pKuaznnwwEwwAVQODfTspJMvbyTBjgGJfQI9amWDJpmRPaC08Ht hBsTJCxXxTGcAvHZyc1GmXknoj+YkyM9E5D5lO99tKW63ZSMdBHZGwWE93UU5a2lO/01 VNraSqGkqCdvUy28RJ7UjbGYyqpMzKo6QjDZXA7857vRExlVeyvHuo/vMEARntkNwfqu s5RgyHWcnKVxFWPywo4+eYX5WeZzVjQBCwIADpffKoL7jrqxM7a9cu6Lb0Tpf6iz3X/1 fVTw== X-Gm-Message-State: AOAM530ZHl3fQ2RauZuWOffv4HfdSqdotn/AgF004EKGi8CX8iZ1vJ3P pt6kGn2M8eWHSg4Iipq5aAlp/QnCxCs= X-Google-Smtp-Source: ABdhPJx6894ZIDqn/smOfu7uSGp8oCTH206akrRxwGPB/buwlvdGdTr7LUP3oes5hlZZ2ullkWjlFw== X-Received: by 2002:a1c:e041:: with SMTP id x62mr5438656wmg.95.1617241811859; Wed, 31 Mar 2021 18:50:11 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u2sm8644859wmm.5.2021.03.31.18.50.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:11 -0700 (PDT) Message-Id: <73684141fcff4d9c0df45e4bbc8d37729bd14ffb.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:50 +0000 Subject: [PATCH v2 13/25] merge-index: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full one to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/merge-index.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/builtin/merge-index.c b/builtin/merge-index.c index 38ea6ad6ca25..c0383fe9df9a 100644 --- a/builtin/merge-index.c +++ b/builtin/merge-index.c @@ -58,6 +58,8 @@ static void merge_one_path(const char *path) static void merge_all(void) { int i; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) { const struct cache_entry *ce = active_cache[i]; if (!ce_stage(ce)) @@ -80,6 +82,9 @@ int cmd_merge_index(int argc, const char **argv, const char *prefix) read_cache(); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); + i = 1; if (!strcmp(argv[i], "-o")) { one_shot = 1; From patchwork Thu Apr 1 01:49:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E20E8C4361B for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CFCBB610A2 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233174AbhDABui (ORCPT ); Wed, 31 Mar 2021 21:50:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233028AbhDABuO (ORCPT ); Wed, 31 Mar 2021 21:50:14 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8A2B9C061574 for ; Wed, 31 Mar 2021 18:50:13 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id x16so191934wrn.4 for ; Wed, 31 Mar 2021 18:50:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=bdB5os8ysADiA9ihl7DyMq1Tj0074xySPTMD9uYQI04=; b=M/FqdlHbw8nUMck5A6jUqhBTUq7h/YU4BGZ7EGklRsnU31ZXKVhQMrBsfbne5ZqMvF TGFR5Lg9bapvzC4Bh8Bgbbp59P76LI0oTA2nGnMn4Pc+eI1TqGc4gUDlnEFbJNeaPMcr gpZQ1uvnPeqxpUTRYrXtGCTSStC54h0BaUIatQ1kKUt/92yLdR+eRFeGbQstzu/lRPs8 NgJiERtB5j+KLm8d5UXil9+MKFmqh+L8cSgsBa/os05kGyU2PIuD6105O6dekya4iiYk ghuEWL+l5rmISWXlCJEvuroonQquLw4ZiZZeULqpojQKRk7dNDHR0lCaLJuUdu9c9f85 wF6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=bdB5os8ysADiA9ihl7DyMq1Tj0074xySPTMD9uYQI04=; b=j2U2gjhVRoM7sZxTE9diDh6fsD3PO1CrGf1wbxpo7vQRH0uMAYOfKtlXD7H/Bv+Y3k 7b4wdxjIGRLnxqLmpYJULpWzvTJh0BudMXgA8qsTrGXQrOMhQVlP7/7U5f70nvWHtuEi c6yHW7HA3YJvTvtG9pB546UdmBlKzMgbsz4UqaHsRbFmow6oWmHTs73M3T6hhn03aFC9 BesMJKT7NPz72rkEFsutgH3KvFXpFLB+M7LNua0m3uhlEzlI//9Hz9lKCuhKOooJcOz8 VZToDLZgnoIFs67LZLLFhcG51xi94ZZcL5EyWcnOClGniWgRVK4uhuHnU6AyzgImNLo4 hJ6Q== X-Gm-Message-State: AOAM532/wW7g6Z2nqrMbOe+o4Ts816HzYsr18OfX6OPaTffJVaB/7OJO C2jmtBUhre63alf2zBjelelvfnwtXQU= X-Google-Smtp-Source: ABdhPJwtimp+9KpoKqaTvVW7VAqtkFQU/k9zj3dIjFPicHr3sURUKKiZoIeOq4UK38fxCyQO/2K4rA== X-Received: by 2002:adf:e481:: with SMTP id i1mr6903284wrm.63.1617241812355; Wed, 31 Mar 2021 18:50:12 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 64sm6046656wmz.7.2021.03.31.18.50.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:12 -0700 (PDT) Message-Id: <6ea81a49c6b51e98f223b59df6e8e73eac894bbe.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:51 +0000 Subject: [PATCH v2 14/25] rm: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/rm.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/rm.c b/builtin/rm.c index 4858631e0f02..5559a0b453a3 100644 --- a/builtin/rm.c +++ b/builtin/rm.c @@ -293,6 +293,8 @@ int cmd_rm(int argc, const char **argv, const char *prefix) seen = xcalloc(pathspec.nr, 1); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) { const struct cache_entry *ce = active_cache[i]; if (!ce_path_match(&the_index, ce, &pathspec, seen)) From patchwork Thu Apr 1 01:49:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176813 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDCECC4361A for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A89D1610A6 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233170AbhDABuh (ORCPT ); Wed, 31 Mar 2021 21:50:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51840 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233030AbhDABuO (ORCPT ); Wed, 31 Mar 2021 21:50:14 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13485C061761 for ; Wed, 31 Mar 2021 18:50:14 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id e18so185464wrt.6 for ; Wed, 31 Mar 2021 18:50:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=y3kBuuE1kqkzlxpCDekvhONE9z9UoTHKBwBc33s5cD4=; b=LH6xeNoYQBh3O51dRTxElDIlcxulpEi78vEq2FrXi78qytlBv9vzHpbLfxQpZ8ChS3 P2YC384rYHf0DcK/c7F90KboBZgkevO1p+38W1mEkORmaZqHSFnRV08Xkfo2++0I5146 CR81bSZBdVJE1U+w+zafR6Niu3e9pTV4LR196Rh/fc2+gC19NvegqgT+SH8sDaPNgKMI TVYnVlnLfeaOhTHVEAp8lWB3XNq+ugN/3dMhVWhxCrGfPhbHZqT3JR3v0HuQ6bd/xc9v OVyJBVnjN/vxnYjukEP0OBiLH1sGOmvhmwUXMpQyF4/szIFfiVg9J6alsoHZIGSZt1St L5sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=y3kBuuE1kqkzlxpCDekvhONE9z9UoTHKBwBc33s5cD4=; b=Plfgtoi+QNV6opcCwzC2UAQMqE+GLrzfowZIB0kn2KUGCc4bzsncgKWLIC1Em1F/Jp s/NOTU/tT3vkbivtyCSiWyXBscKLmhI3kmPoV0CodbB6+34INJcFm69z6TK5wTvENtmK baU/MqOtUCMUyqB3mdb4qGoXH9EUxXmL8FLcu3ns/sU8GJhqdue/HyoB//iGc6lFDynT 7kYBW1dp9beOLAZnKBAx4rW8Uwu/U1zp5xfXNHIsoJffz6KoeRDDeotrhiGTNQrLLKHz 2begwsejMUQi79JDPLJiD1dlYzJHMVVJ/GATHN8Se7Q7RkOOeWK/enX/UGP9sjS0bkVv 4AuA== X-Gm-Message-State: AOAM5320WeGuzFK+5YurAavPt+gmn3W8LpWCibMnwysBagubIDBZrfh0 NA+arQxIkFUlX+R8kw7BJpRBxd8+fFI= X-Google-Smtp-Source: ABdhPJwskrQBaI4ZfYEZLmjPvyvh/yJuVRr2zo+98Ss7WRM2s+Brax0jcwEQtY5PxgS18KHBfJ1SRw== X-Received: by 2002:a5d:658d:: with SMTP id q13mr6687620wru.388.1617241812933; Wed, 31 Mar 2021 18:50:12 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 135sm6739970wma.44.2021.03.31.18.50.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:12 -0700 (PDT) Message-Id: <49ca5ed05c8d0e9333494ebb08ed65cb167dc96e.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:52 +0000 Subject: [PATCH v2 15/25] stash: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/stash.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/stash.c b/builtin/stash.c index ba774cce674f..6fb7178ef2fa 100644 --- a/builtin/stash.c +++ b/builtin/stash.c @@ -1350,6 +1350,8 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q int i; char *ps_matched = xcalloc(ps->nr, 1); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) ce_path_match(&the_index, active_cache[i], ps, ps_matched); From patchwork Thu Apr 1 01:49:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 002D0C4363E for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E231B61078 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233201AbhDABuk (ORCPT ); Wed, 31 Mar 2021 21:50:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233036AbhDABuO (ORCPT ); Wed, 31 Mar 2021 21:50:14 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8220CC061574 for ; Wed, 31 Mar 2021 18:50:14 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id x7so172515wrw.10 for ; Wed, 31 Mar 2021 18:50:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=P9rJd368hm+Jl6xbpIBIuWVC2ZyfP3m2Ah742l9tbUc=; b=Q/EqsvSsnlXuwRai6R10KSwGYW525vJRpCfL3gc/ytAF9Pv6uVJ26cM8nS4y6tdrx1 eAs/vloywkMhRZ2BqsJ4T1nm5gxoLyUzKKLfaLwmxoLS+CZcHwrfum3nXlkaZd3nzlqL YlkK6Nbqypr0H4dDY3+QJu1r5hiwyuS/fgCrt7O2gX5qhISYMV2CF7eJMaA+1uFjUFq8 +eV5PcTQuprVEJV1g5LhBkaEMQUcd1dd64ECNOGA82jHbNsERN0EcEywQ3swOMECcabd UcQSiINl//8SDX8AlXVpGJlgH+N4LsDclWZ7799Y/b1OMl1545sniHUPRPKd5VHov+Yn ESLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=P9rJd368hm+Jl6xbpIBIuWVC2ZyfP3m2Ah742l9tbUc=; b=LPnAgqn1oegGJeTD3bsGDTLAk+kUeZX1t/Kl/FeZfOJBKelMwWjAs6G+O3GL1Oitgt kG9QKzdVi56tDVd7KTirzwFVSnT2rqM9dGhdkLPIZxaiAr8pdpa4CT8Yixd20euq2FQa /IyLKcojaXttsRYZnhZ4sTRrx683CGeQ22L/RpvEsorH/4/FjEUnjTCtBYzSN+n+3gjf TqftXVRQi2SzyMnkb3yUJTZUPFe2leoLCouf52JlOHxMpHKokcM4jTlciXqnbbQxTlMK lW/hKvgzzduqU80smair/zZhtGlSSVjvHWrGe57y+Ft4aOg+S/FeKmFREYgfHYImuUQb 6cmg== X-Gm-Message-State: AOAM533rCQ/liNYuIrWLrPf/IVLmY9PTdCsMLVv6/P0ZetNPijlpEDmc 8VjeJgM58U2ALRMPck7GGoqdREhPyHU= X-Google-Smtp-Source: ABdhPJwl/xMLPZCG7Mmz75zlQnH+AcWBHpbHNUNdqCX8Ktg/9dIae3OqF/dk4B7Wxd3VY61UkcEbGA== X-Received: by 2002:a5d:44c5:: with SMTP id z5mr6343023wrr.319.1617241813391; Wed, 31 Mar 2021 18:50:13 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b131sm6129013wmb.34.2021.03.31.18.50.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:13 -0700 (PDT) Message-Id: <9c4bb187c15da527f2a1692cafce7ec844a94ae1.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:53 +0000 Subject: [PATCH v2 16/25] update-index: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/update-index.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/update-index.c b/builtin/update-index.c index 79087bccea4b..f1f16f2de526 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -745,6 +745,8 @@ static int do_reupdate(int ac, const char **av, */ has_head = 0; redo: + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (pos = 0; pos < active_nr; pos++) { const struct cache_entry *ce = active_cache[pos]; struct cache_entry *old = NULL; From patchwork Thu Apr 1 01:49:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF415C43619 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BA8D86023F for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233185AbhDABuj (ORCPT ); Wed, 31 Mar 2021 21:50:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51844 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233048AbhDABuP (ORCPT ); Wed, 31 Mar 2021 21:50:15 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A003C061574 for ; Wed, 31 Mar 2021 18:50:15 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id v11so182551wro.7 for ; Wed, 31 Mar 2021 18:50:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=7qKugyqjTL3Qic3WcAPItv5DSYivY5GEtPPJAkAOQqc=; b=cUky643FgEqggIT8LwTyUSDrFvDFXdcugNdzwG9Ni+6x/p/4+K4SSk/iVAhY/8raio xeoLKIFXmGR1q+4jL3TgqLnr0eRd4cBWP2R9/VtPvqyPDi17mLP8h51XX3RPpWk+N75A lyDTXLTAnJebH/HTaDKrAIvN2CpKRNdMtwwwWw7PpLvlL6/l59uvbN5FWV/Ut1Sb76sK 91cf7NMrju96jkkKbAyzYjWJatpYQXfDIzNAnDeq2JFNINbG1TyALskLsRKGSqxjDHys CGEMO/1gSzLES7zdPx7uiEeodFoPacT5ZS2b9PWfoz6xba8dQAqjZ1bI/YGYIjPb/D88 rctQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=7qKugyqjTL3Qic3WcAPItv5DSYivY5GEtPPJAkAOQqc=; b=spqn2CJSPaj2yYe3UK3MyDirmczU3B/uD1o2HGguxlemqCGIxovu5u5EoHFvSAwqL+ M58mDz+u1xw7SCCmv6pFTEToSjKFMoRN8wn4PT+PgvX2dz1LKe8lnPZnVPInabtYD/lj VpsGYweGueUJ1qNq2fehnpTf447hBdwFM6hNT1lGBTJpWpFa5OiEwqEw6D6p7BCG4xYD BSoL+jBFVMKkXP3XzBuUeS7TA4h39IaZC83+oUZFryTR1IOWfIau7dECZSgFTDjDh514 1djsLtYU4w9GDms1R+/zKz6nazghEasa2gMA4ap0LqSmTYc/ePn3J8ybmBQ3CC9HEhAh ISlQ== X-Gm-Message-State: AOAM531dGb93J6cZtQ8ObhSRZr9Bkw58PfW8EPRhAk2seJjOr0egh2DK QX3O+xiTtYmxGSSbW0kf9OFg8dUw6VY= X-Google-Smtp-Source: ABdhPJz7W1xmErRoW1QQLhvkU+NnEXCLMgKfBIMRJ+892/8VsGYVHwFUEANp3A5gaEFHVr1fzJc6hw== X-Received: by 2002:adf:df10:: with SMTP id y16mr6654107wrl.372.1617241813934; Wed, 31 Mar 2021 18:50:13 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z15sm7236206wrw.50.2021.03.31.18.50.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:13 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:54 +0000 Subject: [PATCH v2 17/25] dir: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- dir.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/dir.c b/dir.c index 5b00dfb5b144..166238e79f52 100644 --- a/dir.c +++ b/dir.c @@ -3533,6 +3533,8 @@ static void connect_wt_gitdir_in_nested(const char *sub_worktree, if (repo_read_index(&subrepo) < 0) die(_("index file corrupt in repo %s"), subrepo.gitdir); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(subrepo.index); for (i = 0; i < subrepo.index->cache_nr; i++) { const struct cache_entry *ce = subrepo.index->cache[i]; From patchwork Thu Apr 1 01:49:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176819 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DB58C43140 for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 10F67610A4 for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233223AbhDABum (ORCPT ); Wed, 31 Mar 2021 21:50:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233049AbhDABuP (ORCPT ); Wed, 31 Mar 2021 21:50:15 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93115C061574 for ; Wed, 31 Mar 2021 18:50:15 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id j9so167015wrx.12 for ; Wed, 31 Mar 2021 18:50:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=DN0bJRJEdqZFG9ILPnNiBkEoj0NZzXfi9k5Yav8/FLY=; b=BB1NuVv6QsOMAxK0gOhFggd6jXhYYCy2tCDSy1EXKxCIBInaNzxZlOTtoShPVtdlNa tD95NM2pouqachIbrMg7oSRH1ndDy3smprNVehKmxGsUlq6zPc/69IkMrH/9/GxnrKam V/v8n+RAHQy6cqFaAFNyXmV5bSGNvQc7LzJj1ZqkKhZnTrPULaGqB1uuZPaCTzGn4mzd aR+H69EI9spIHAT7XruW8yCKBOLT+DBLBBHQTPlEUdugXk8MCBZo2MXJxMXh8baBaY7f 2ThkGfarRi7Uta+60RLdiVeGmCGzul4OBN+xXVasH16VqF6CCJYXWkhRXLS1dItQZYKS 1LdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=DN0bJRJEdqZFG9ILPnNiBkEoj0NZzXfi9k5Yav8/FLY=; b=TbrWlra+IFKXkbGJR2aDfq4PRwaehKP/Lr5a7hrIfq6TUXW7LX+WPWR6SLLZ22/W4g cUuR4Re3+73cbewslVSoKBz2k9yArvjb1O619jDCUyRxAH7aDUpwm8GpUQRWYaxydCXm ginqI2XAKxMX9oijpH/Rd4XXuQvtMObuwnoOMr3UQI9Ap23W441HUTzfPfmoOdqL50vc kuF8TcvYhYR5II1pMNg4yJUFfKHjsVGxFqCFplEAYo3ECUzvQ1kRNQjh+RfO4ZVBKxzt y5bKZCksHfApeQ6B/z5OtR+MD9nQAgFL5hX6UwxFITo2TTYi+Lfmf5LCA8nlri+wE/W1 LXUg== X-Gm-Message-State: AOAM533za2Zaq9g91Go0LDwzHNkhs1UPvcCrCxSOZCC0TRmJKSInEFq0 8Mj8gMAQ/hrkiG3oU7KbDBg0hA+i6jg= X-Google-Smtp-Source: ABdhPJz8oRms8/JHmIVaUQrWCjpCrJdD7BPHnp/TimsdepRvZwDtopa/D/KxVDOCKd/mq+OeDijPZg== X-Received: by 2002:adf:f54c:: with SMTP id j12mr6684173wrp.264.1617241814457; Wed, 31 Mar 2021 18:50:14 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m11sm7061629wri.44.2021.03.31.18.50.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:14 -0700 (PDT) Message-Id: <2b9180ee77d30c797f58f7e50ff9174e70797069.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:55 +0000 Subject: [PATCH v2 18/25] entry: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- entry.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/entry.c b/entry.c index 7b9f43716f76..891e4ba2b45a 100644 --- a/entry.c +++ b/entry.c @@ -412,6 +412,8 @@ static void mark_colliding_entries(const struct checkout *state, ce->ce_flags |= CE_MATCHED; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(state->istate); for (i = 0; i < state->istate->cache_nr; i++) { struct cache_entry *dup = state->istate->cache[i]; From patchwork Thu Apr 1 01:49:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10E30C4363C for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 01752610A2 for ; Thu, 1 Apr 2021 01:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233205AbhDABul (ORCPT ); Wed, 31 Mar 2021 21:50:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233050AbhDABuQ (ORCPT ); Wed, 31 Mar 2021 21:50:16 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A5F2C061574 for ; Wed, 31 Mar 2021 18:50:16 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id p19so247886wmq.1 for ; Wed, 31 Mar 2021 18:50:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Cwy0/DFLfwxmnrNn1jvtNHYmLs4yYZjayHJKnh4xlfU=; b=pyQoLIEImAwiQ68TTjrWuXg7jTNmGjElZol7PnopIjJ6oUr4MaHsvQl3FP/aW97tNl p2rcXk9vZidXN8DxJBRk0FuRgRvYTmERX6HCaZgi0eHPASjDZK9P0t3EozF+fw7b5zqj ZKLyE88EIpwzPD1ItvlRjf02PWybQeryUfypSwpPzHv/LvgzN225VvSwB6LZIZmS6/gQ EX0azO4egfUgfr9oaPxoxpz7PMiJv1tWEdvjL0XDwthqFzLCjkLLiN4xLGz8BCOHY3q7 bgMECkGVXumx8l6E6imyR3Wz+ug1UhQK91mEH9ZcuzEYSbeKNP2wx5qi1ncce3iaXc1b wNAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Cwy0/DFLfwxmnrNn1jvtNHYmLs4yYZjayHJKnh4xlfU=; b=X1HVE28w8q0g/re35j8nMmhD6efKw6sgSeFKFlNKsn90rlKkVC8azJZ7oNQ21LtqEa BH/UAVr2qnT4aoFD2Z6fXn3tK6J9mUxc3fVxU/dTvvCc7v1nb7MiNj5wURLAwDnMhfnq d1T0QGyE6ICKGgW0O+JqMzicGQOVSDZWXArV0W1CtCJ0aAbcFvSEyjhTCthkKoFW+V6u +T+DERu9JtIgJ7OenPP/+Wr3SI5YC6eGGJV9CL2K12BPXUKnvXNquT16mY9FsrOkwsDk D2nPeSxyK63Jwx4EOsHNlAEyPJsDERjfQoX++vBB2ArXfMHdWut4eGJikDjnxauuSb71 PlcQ== X-Gm-Message-State: AOAM5328ZgTcBAUTgBv9FcnkHmcIQmev8xtUe9Ixt3fopOZVdI1I0Y0e wousyR/O9VihUs5Fx1PJOeKbN5JE2Vk= X-Google-Smtp-Source: ABdhPJxKt7nyVRLopyYpgL+GkqPZY1yq/5UD1F4qGUeCfsSukB+XeNnSw1QOclvFIVex2538ALjXtQ== X-Received: by 2002:a05:600c:9:: with SMTP id g9mr5671303wmc.134.1617241815057; Wed, 31 Mar 2021 18:50:15 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y1sm5846202wmq.29.2021.03.31.18.50.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:14 -0700 (PDT) Message-Id: <1e3f6085a405aa39692f3100790a596cc7b6fbfd.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:56 +0000 Subject: [PATCH v2 19/25] merge-recursive: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- merge-recursive.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/merge-recursive.c b/merge-recursive.c index b8de7a704eae..91d8597728c1 100644 --- a/merge-recursive.c +++ b/merge-recursive.c @@ -522,6 +522,8 @@ static struct string_list *get_unmerged(struct index_state *istate) unmerged->strdup_strings = 1; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { struct string_list_item *item; struct stage_data *e; From patchwork Thu Apr 1 01:49:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176817 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5269FC41602 for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2E779610A6 for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233235AbhDABuo (ORCPT ); Wed, 31 Mar 2021 21:50:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51858 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233070AbhDABuR (ORCPT ); Wed, 31 Mar 2021 21:50:17 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B2A23C061761 for ; Wed, 31 Mar 2021 18:50:16 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id j20-20020a05600c1914b029010f31e15a7fso2034724wmq.1 for ; Wed, 31 Mar 2021 18:50:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=xOPUNFGXkgQBSC36MohDBVpe4Y/fE1xOn5Zg727kSss=; b=Tp3bjBNEdl4AFXDb+nzhbi4PDdC99yvaD8ZC70RsviWBMBQeu9N1jQg1IPFEf2vneU 1HUTkCiM/HXhCmtKs2VCjsPomzF9IMIOtXTKulpzG0GqWzQKxfV60Eo/aTrS05TCZH65 So3eavzkB6s56to7aBAdB+JPqJIdXKN1WDzmMgrf9ByJfTdgV0m3AU7rXF+7kEAOH5qY qY5MZ74KfHkKICIaYGC+das7ggmIT4CnvbZ1qGB/BHnqA7SI/kxKI9TGGEDRVhncFj1w LLEFS8QVTB3FLW0lvZYHjdaTLOtBTtXWBgyiiFV2uHwR0hkt+h/FldSH0Jm/N5W3f4NB n47A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=xOPUNFGXkgQBSC36MohDBVpe4Y/fE1xOn5Zg727kSss=; b=rODgpc37IelcAd+U2X8Rdl6sEs/UdGfP82PzAqQjRF6ekjD72D1+KmmHRnTGRKrqcj yqXBJnhf/mhM493ll5qChcIEDfzuMbrd/RG44QGmE7Cmim7qZLU1cH3AI0GqPij5WmVu /80/NZIHbwzyDYx/N3gOip7tNPDfgcAX+90zcQK30XYUik4Gm6bkfLq/vDjgSTCRa3Ap RnCvQzHlU+SyuZkOKMdHyDzRkQpEh6/M2Oq5h4HUN2mpuMbOUoqvYRV1n/JUzs6SYgxW 3llYvvbEjmgLf7J1VbhLhziWyqCBYCfnsJX6FyYHlbCOuxV2NTzxcL2DC+gb4HqbXTBq eVAg== X-Gm-Message-State: AOAM530d9pg1VLS0vg+IK7z0u/07nkWOVO2A07ox2j2M4hkXrgqm7X3U 0HPcve786esRCgZX2AngbeKQod1YZis= X-Google-Smtp-Source: ABdhPJwRBrVVfQf2VFeBGMCbq9AQv51iVTpzPUH7WZ63obdzQHozRTvmmT6Swr94xbM77yeTnxFfow== X-Received: by 2002:a1c:7e16:: with SMTP id z22mr5542475wmc.74.1617241815577; Wed, 31 Mar 2021 18:50:15 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v7sm6439081wme.47.2021.03.31.18.50.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:15 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:57 +0000 Subject: [PATCH v2 20/25] pathspec: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- pathspec.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/pathspec.c b/pathspec.c index b6e333965cb4..d67688bab74b 100644 --- a/pathspec.c +++ b/pathspec.c @@ -36,6 +36,8 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec, num_unmatched++; if (!num_unmatched) return; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { const struct cache_entry *ce = istate->cache[i]; ce_path_match(istate, ce, pathspec, seen); From patchwork Thu Apr 1 01:49:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52486C4363F for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3DF42610CB for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233227AbhDABun (ORCPT ); Wed, 31 Mar 2021 21:50:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233076AbhDABuR (ORCPT ); Wed, 31 Mar 2021 21:50:17 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B032C061574 for ; Wed, 31 Mar 2021 18:50:17 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id u5-20020a7bcb050000b029010e9316b9d5so84256wmj.2 for ; Wed, 31 Mar 2021 18:50:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=gXHTyk/0rr+wbe8rzhDt1uZ1A0ElBNbmaZcI4hkvDxk=; b=SVviZ/TDvWySctTT/qISNXo45/+Drbc20zla6Zl/lEC8NK2gudLs6MxVjxqrQdDQD7 UGDocQ7cVn59tGh+gQZB5TnKEYg27GX41ZyQa8nykW0RRZLkn9E0fUEuYpBLrNkAEwc5 FeqgrKxeWhQWBu5gp0gw81iesPDo1aDJvoAfnC/sYdKWcYp/Ijfd6u5Ea/6GsFEq4YAI TEEfenhOqMVospcraDoEXqppagVYlHoCUcL2MySgdGuR2DGOFiD+Wfx9y+jY6TBicmZM /+D5r+KekuNA/L1Agqq4RV4g7nccqrTHr5L4VX4kkaSdQa07Mtjq0td0KHL4LJe24IE9 9Blg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=gXHTyk/0rr+wbe8rzhDt1uZ1A0ElBNbmaZcI4hkvDxk=; b=uhlQsVbsU6gpLBGsxkS6ciuA0LMGgPb2X1BBpv0UPXazVcIV+F9e2LpHRgoD9m5VBm mPdw51OIDb7nnZEhHlNVwzCbf/VKaNTAiLTG/vMRoX8ylYVch0if/Kukj1CX8H1g9TI9 2CIFNwofYKc5gR13QkOMQS+2hyv9FExwuOIHJPXMi1eOygrwJ88RqmxTKrTrZ3tKNdhG y9qO8cIFOn1j2gP5p1r27R56yRldrrHo4N2PDJioF9WcTgQPkllncRSGfnvjHBC27mwi B4+SaXNs2D220n/gWR25BdPv+KEZ+NavOCf7T8RMcG5H4IK0qTnPIL+erPL4GQEuc+RY ipyQ== X-Gm-Message-State: AOAM533+DEmH+zDtOzcL9yjQH294WLGEqMSvZvXbrzOommZM0n2sN764 HL9kkcZ/lQL6F2nJ/5NrgzKMKxdAakc= X-Google-Smtp-Source: ABdhPJzkgSo8XuUgsubdBFn+0mJxvIOCs8qdCa1IWXufqWP+4eJhsWcFEabCq+4brBMoob/eOU3jIw== X-Received: by 2002:a05:600c:3647:: with SMTP id y7mr5604917wmq.17.1617241816130; Wed, 31 Mar 2021 18:50:16 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x14sm7046382wrw.13.2021.03.31.18.50.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:15 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:58 +0000 Subject: [PATCH v2 21/25] read-cache: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- read-cache.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/read-cache.c b/read-cache.c index 3698bc7bf77d..a9dcf0ab4f78 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1577,6 +1577,8 @@ int refresh_index(struct index_state *istate, unsigned int flags, */ preload_index(istate, pathspec, 0); trace2_region_enter("index", "refresh", NULL); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { struct cache_entry *ce, *new_entry; int cache_errno = 0; @@ -2498,6 +2500,8 @@ int repo_index_has_changes(struct repository *repo, diff_flush(&opt); return opt.flags.has_changes != 0; } else { + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; sb && i < istate->cache_nr; i++) { if (i) strbuf_addch(sb, ' '); From patchwork Thu Apr 1 01:49:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 899D8C001B4 for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 71F55610A6 for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233252AbhDABup (ORCPT ); Wed, 31 Mar 2021 21:50:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51868 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233077AbhDABuS (ORCPT ); Wed, 31 Mar 2021 21:50:18 -0400 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D3815C061574 for ; Wed, 31 Mar 2021 18:50:17 -0700 (PDT) Received: by mail-wm1-x332.google.com with SMTP id a132-20020a1c668a0000b029010f141fe7c2so2062548wmc.0 for ; Wed, 31 Mar 2021 18:50:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=5ujY+w6SXlDeI30iu//cxteaQB/3nSsRybSiz0GS0fM=; b=UnO8PiA2R1gz2JwafpCz7GzCwiOVmy5jVmN/2i59cCVuR6/GZFnWxgpv7hVdPh6a8U IfnBpfdp4RnzI79jy4jJghs7qVMnmF+EffKtg3xBL7SK6QItuH4oS6yQEH/JT+3KpU26 kXZJ9IPdhbLEBHbNz73Ghi6/NMwkGELLTqqBjOpt7QaLcSDxwPja5gXX1pG3Jc5BR9Vo IBSokR1FJ/mPSU2r47oYZPbMQPNaMuWsUsYKXZdwVhtUaDB8dQC5Dxwd0I9XzXLCm5uz W46jRUGFCxFpzGHNUU7tNkb6C6WpqrUne1UirQTXUzxCIvw7YomNdKAYz1SUvk33iChk XZIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=5ujY+w6SXlDeI30iu//cxteaQB/3nSsRybSiz0GS0fM=; b=PV7uYatHdPAy4TPO74/z6+uhpXM7eQ2ZeieAd8TDXTZM8lPCSivslrkXrgN/MOI8bZ kifBGfoBOcvwTLwbyZh8/de6GKuKkr1/BrlQ928fXemKUiVbnpL/aVu37Jwv19WjXj1K oBsTrcNipJ++LVulk1cLOSb+CZpbe6A6PuR6g1JQtmRipUxXKmwBoyXQzZBqZiLsXNNH V7TtR7t3oFVTVD4eqruaqcRY3d3JjFZO2eWrHZW5vUIOam+CWkUrWbioQJBYSrUSt9QB P5vrSETZJy4lrGSWV+A4Z3fO0Z7AMVUsD/dUmsxc/1bYaLAv3jDpTkqSzK07rBqucv4h PbSg== X-Gm-Message-State: AOAM533N6iTvXPFIsQa3hDFGotm+mXJOyOK7T4aHzMdRqKU5AR5twzYQ 7HjM3xTIzTFbl7ClfI0SwyNKfRKIEB0= X-Google-Smtp-Source: ABdhPJy+7BYeVz1wBovKiDcgPqNuSt2kasMJu/USEBypdQMCvsK+8T/M+GfQ3rbxAPtJund1I3rVNg== X-Received: by 2002:a1c:c1:: with SMTP id 184mr5596845wma.143.1617241816647; Wed, 31 Mar 2021 18:50:16 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d204sm6540344wmc.17.2021.03.31.18.50.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:16 -0700 (PDT) Message-Id: <495b07a87973f32ad81e513b2be04a1b1b92bdc8.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:49:59 +0000 Subject: [PATCH v2 22/25] resolve-undo: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- resolve-undo.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/resolve-undo.c b/resolve-undo.c index 236320f179cb..ee36ca58b135 100644 --- a/resolve-undo.c +++ b/resolve-undo.c @@ -172,6 +172,8 @@ void unmerge_marked_index(struct index_state *istate) if (!istate->resolve_undo) return; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { const struct cache_entry *ce = istate->cache[i]; if (ce->ce_flags & CE_MATCHED) @@ -186,6 +188,8 @@ void unmerge_index(struct index_state *istate, const struct pathspec *pathspec) if (!istate->resolve_undo) return; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { const struct cache_entry *ce = istate->cache[i]; if (!ce_path_match(istate, ce, pathspec, NULL)) From patchwork Thu Apr 1 01:50:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7474AC433ED for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5F955610CD for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233258AbhDABuq (ORCPT ); Wed, 31 Mar 2021 21:50:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233090AbhDABuS (ORCPT ); Wed, 31 Mar 2021 21:50:18 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58C79C061762 for ; Wed, 31 Mar 2021 18:50:18 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id d8-20020a1c1d080000b029010f15546281so2019061wmd.4 for ; Wed, 31 Mar 2021 18:50:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=hhLYYq4GDu3XvoSbcr7sJoPjWfM0oLCRJFTPz+pdmsw=; b=QpV5QV25g58QUP/MAYM0mY0gTg4s9x9928neRIwYNhc0KGC1a2NoLCAQaNF3mFBhDZ +whNEq4AuyAP9yADrXeovr9hA1ccc+AhexbRVIgQXWy0DA086slXP50w6OIgurt2Jmo3 nl2MYqb0XsB4zP1E/rvLOfqPV1nuivjzKGuTkaqBensFVjcXbPYxOh738J59q0BZ6mGD gDu8H+ZbdICUzwVyYBUQLDWxLH38NQf2N1iHZp83ImYqEIqSvyFY0s6f9QuibiAPbOGh CAfYlw5s//9AOBb7Jv2xYYQaC1bxZPPwnEIvIbmI4VVjhH6T8s7isGAR4+KEg6p5IXrs bU3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=hhLYYq4GDu3XvoSbcr7sJoPjWfM0oLCRJFTPz+pdmsw=; b=gCscttpS6kzb6DKq+49MieHNYGRwUwwlbI1qEpeUI0hLmypLYaMdkMBjtzgs1JhpF4 dneo8V854dsN2PE/eDpNu4oUynb6ZRNlDd4nSO3I3kGuIy63OYE9jNQHGxDHq6iAkz0n 5KvcYkQoT6ZJrHL0uRS98TBdTjoFapub82DAfBjMkxFNacWzhHwt5KYXRAZkGs6kntX5 vrStXcYijA1PRTzpVR++kSRJCzZa9mMPihlOpbc2WE52Hk6k83TzVVxyc0CO1MVDauoF vTmB0thwVrC1g8UVAJoHyljOs6njTDK4g3QQHZHxzj3Yjf41QMmKEbHHxSl6WxCBXMsc njdQ== X-Gm-Message-State: AOAM5334uiC50F23qZnNSic3XCAKi8FHjRkwR0RNXi2Zd4BTMMApzi1d bpocvFIAMRBlI9G5VOSI7LJ5juVju2Y= X-Google-Smtp-Source: ABdhPJz4099SZwDfJe1eiJE0xRHXZDGbvISvOr/Qma19i0mE/fEgcGOO35hSDYA4Jtpl+uo7U+1gkw== X-Received: by 2002:a05:600c:2312:: with SMTP id 18mr5526393wmo.8.1617241817188; Wed, 31 Mar 2021 18:50:17 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v18sm7854442wrf.41.2021.03.31.18.50.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:16 -0700 (PDT) Message-Id: <3144114d1a75a8515fc35ea9625a7d088652b178.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:50:00 +0000 Subject: [PATCH v2 23/25] revision: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all index entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. This case could be integrated later by ensuring that we walk the tree in the sparse-directory entry, but the current behavior is only expecting blobs. Save this integration for later when it can be properly tested. Signed-off-by: Derrick Stolee --- revision.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/revision.c b/revision.c index b78733f5089b..b72e0ac1bdca 100644 --- a/revision.c +++ b/revision.c @@ -1680,6 +1680,8 @@ static void do_add_index_objects_to_pending(struct rev_info *revs, { int i; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { struct cache_entry *ce = istate->cache[i]; struct blob *blob; From patchwork Thu Apr 1 01:50:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94C81C4161D for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 84D51610CD for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233273AbhDABur (ORCPT ); Wed, 31 Mar 2021 21:50:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51874 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233092AbhDABuT (ORCPT ); Wed, 31 Mar 2021 21:50:19 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03C19C061574 for ; Wed, 31 Mar 2021 18:50:19 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id x16so192060wrn.4 for ; Wed, 31 Mar 2021 18:50:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=WLojLww17wXcQpERxx79/BMroTeok/fkDDxZFf6UytM=; b=bBTx3K6ToSZUdWjn2tbrMC+/OAAipovoAsRPPRwyvytd29DmoHUBMa/cMom3FVidN3 wFm0E8PA0Fw/ur7WrnmFevHLuIMTG5506MIcHGaB4uyjjli7+Ch0B3daSARl0MGQ9XSd ZSYsp/xHLVtOROM4a9XYtUCwzx/7ddHkqOueNOGMoGmWjpKQZoyiRSteOuqApWIAhdMs d0Lz2Kmg5B20+Z+CaKTCDRZueFdMvMghh9ePB2UAkylc+lcdYW23HbWvsY66ol+e5km4 F0FTODul0ZCYCrxYqOdwSlaqsjEKr1ZIInN66tTSSVF3Bv7zthsgQ6kWm6apxUjQX+8v hIYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=WLojLww17wXcQpERxx79/BMroTeok/fkDDxZFf6UytM=; b=UnaqR2OnU55GKSkt6CDyr9KSZOuNB6tcZjKePzCIFOV4jaCTtbqp0J46xbhOcuPg5u QJjBkqqNb1HltaQkytnugOFvqpmmjkF6x5yOxZt4oVL+CbyRH4yfX3bNkzVgyXi/h+l2 RtLhErdtxNXC3E2guwYCit8fefPmWtl+BeoWkr+q1H/zMVgn2RE0O1DWwVeNyAilGCJX SJSn7kd+A4g05Mct098USehYMcmEF+hia+4EgCz/dMZek8ULbR7bfpiQ0aJQ0FCKblfJ ER6sTXm9SdSyTVygpenA5heKaFMndaPkN6mF0V6L3x9Q6j5bMFmAgw62+ATL4Xp4R9IK t4HQ== X-Gm-Message-State: AOAM531qHlMKnj/QtKcVxPSKB9vcYAv3/n67dUfF6Ap1MsHWG6NuDdA0 ABJRn7XaC1h7Dcc74kvHEi3PqBbEISM= X-Google-Smtp-Source: ABdhPJyWrdtQoUfg5nUvOiIjaN983MpzSXrMYaoKlIh39CbRnRHnDapEDcgKDnjhiS6nGEBQR086Vg== X-Received: by 2002:a05:6000:2c8:: with SMTP id o8mr6639219wry.407.1617241817761; Wed, 31 Mar 2021 18:50:17 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u4sm7170658wrm.24.2021.03.31.18.50.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:17 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Thu, 01 Apr 2021 01:50:01 +0000 Subject: [PATCH v2 24/25] sparse-index: expand_to_path() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Some users of the index API have a specific path they are looking for, but choose to use index_file_exists() to rely on the name-hash hashtable instead of doing binary search with index_name_pos(). These users only need to know a yes/no answer, not a position within the cache array. When the index is sparse, the name-hash hash table does not contain the full list of paths within sparse directories. It _does_ contain the directory names for the sparse-directory entries. Create a helper function, expand_to_path(), for intended use with the name-hash hashtable functions. The integration with name-hash.c will follow in a later change. The solution here is to use ensure_full_index() when we determine that the requested path is within a sparse directory entry. This will populate the name-hash hashtable as the index is recomputed from scratch. There may be cases where the caller is trying to find an untracked path that is not in the index but also is not within a sparse directory entry. We want to minimize the overhead for these requests. If we used index_name_pos() to find the insertion order of the path, then we could determine from that position if a sparse-directory exists. (In fact, just calling index_name_pos() in that case would lead to expanding the index to a full index.) However, this takes O(log N) time where N is the number of cache entries. To keep the performance of this call based mostly on the input string, use index_file_exists() to look for the ancestors of the path. Using the heuristic that a sparse directory is likely to have a small number of parent directories, we start from the bottom and build up. Use a string buffer to allow mutating the path name to terminate after each slash for each hashset test. Signed-off-by: Derrick Stolee --- sparse-index.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++ sparse-index.h | 13 +++++++++ 2 files changed, 85 insertions(+) diff --git a/sparse-index.c b/sparse-index.c index 95ea17174da3..8a1223041296 100644 --- a/sparse-index.c +++ b/sparse-index.c @@ -283,3 +283,75 @@ void ensure_full_index(struct index_state *istate) trace2_region_leave("index", "ensure_full_index", istate->repo); } + +/* + * This static global helps avoid infinite recursion between + * expand_to_path() and index_file_exists(). + */ +static int in_expand_to_path = 0; + +void expand_to_path(struct index_state *istate, + const char *path, size_t pathlen, int icase) +{ + struct strbuf path_mutable = STRBUF_INIT; + size_t substr_len; + + /* prevent extra recursion */ + if (in_expand_to_path) + return; + + if (!istate || !istate->sparse_index) + return; + + if (!istate->repo) + istate->repo = the_repository; + + in_expand_to_path = 1; + + /* + * We only need to actually expand a region if the + * following are both true: + * + * 1. 'path' is not already in the index. + * 2. Some parent directory of 'path' is a sparse directory. + */ + + if (index_file_exists(istate, path, pathlen, icase)) + goto cleanup; + + strbuf_add(&path_mutable, path, pathlen); + strbuf_addch(&path_mutable, '/'); + + /* Check the name hash for all parent directories */ + substr_len = 0; + while (substr_len < pathlen) { + char temp; + char *replace = strchr(path_mutable.buf + substr_len, '/'); + + if (!replace) + break; + + /* replace the character _after_ the slash */ + replace++; + temp = *replace; + *replace = '\0'; + if (index_file_exists(istate, path_mutable.buf, + path_mutable.len, icase)) { + /* + * We found a parent directory in the name-hash + * hashtable, which means that this entry could + * exist within a sparse-directory entry. Expand + * accordingly. + */ + ensure_full_index(istate); + break; + } + + *replace = temp; + substr_len = replace - path_mutable.buf; + } + +cleanup: + strbuf_release(&path_mutable); + in_expand_to_path = 0; +} diff --git a/sparse-index.h b/sparse-index.h index 0268f38753c0..1115a0d7dd98 100644 --- a/sparse-index.h +++ b/sparse-index.h @@ -4,6 +4,19 @@ struct index_state; int convert_to_sparse(struct index_state *istate); +/* + * Some places in the codebase expect to search for a specific path. + * This path might be outside of the sparse-checkout definition, in + * which case a sparse-index may not contain a path for that index. + * + * Given an index and a path, check to see if a leading directory for + * 'path' exists in the index as a sparse directory. In that case, + * expand that sparse directory to a full range of cache entries and + * populate the index accordingly. + */ +void expand_to_path(struct index_state *istate, + const char *path, size_t pathlen, int icase); + struct repository; int set_sparse_index_config(struct repository *repo, int enable); From patchwork Thu Apr 1 01:50:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12176833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C349DC001B6 for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 943A761078 for ; Thu, 1 Apr 2021 01:51:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233270AbhDABuq (ORCPT ); Wed, 31 Mar 2021 21:50:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233102AbhDABuU (ORCPT ); Wed, 31 Mar 2021 21:50:20 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E364C061761 for ; Wed, 31 Mar 2021 18:50:19 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id j7so201978wrd.1 for ; Wed, 31 Mar 2021 18:50:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=bFOrvSw80zbNP+4EnPIiQiVg9EM0jRAPgsEPYaxvRlQ=; b=FP+Mco7594b0830RaXxZ1dqFnuwCahjLdeohrN6gehTBsqj6IBCIgauzywtmXdcAZN MH/FhMLoHbqTnjTQdmteq5lm+mOrbN73DwwkjRSUynuzlCZOL67uv5RmYd2MoAe+wfLD ECeYlLWeYmETA41n5+xacaN5q2Orlf5N7m/6gGvef/MZgAoPlb3rksjJ9e4SEMVb+b9q Dbim86OcoYWG9dJ+0w6pKnoUK8EZilhKYXfrjPeypuvKnQBO2ygAju3ici4JNQDErNzl akoeYxdDTOPw8lTnRHpDeuOmNAprg49FwCHdCDBSgmmZ8MFq9rCOOFnk6fpaHldolQ4N WBQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=bFOrvSw80zbNP+4EnPIiQiVg9EM0jRAPgsEPYaxvRlQ=; b=iPR77zyvJLKvYudJdfA1kmTTFcupl/O7F6H+tFslWNmOydVAI42JDB1cS4ipDo1OI/ vegL2nZBWpJY5GHRK9brO2UfelC6ipwGT9X42WKA9eM9LjgI54DbOCSnhxuhJYNw/X95 R/U6gwSS7iSTn6YSDl/0WmiC8UQIaG8zXjyNAjs04uw/0Nhyx86ck2ECjOBXD6QFmqeT 4P/7OJM8kCt9xykd+kL2Hg6utzYgNfjzlNyNkMQ03Xzd+HQt79vH/ctyBQr9Pr6qKBbE C2eSAEBCJLjPSoMtUGVu8uHGbo7N8hlVXwNzz6gWoBC7eTWGBczFHbxbJtC9XQ9dCbje nW/Q== X-Gm-Message-State: AOAM531CM7DIwLdWKuWWmNi6J18r0rQOJmtImn+yPL0hlX8fcUuJIenX wAHGW4A9VTQfgiVll/IhY+eZV6GeXfQ= X-Google-Smtp-Source: ABdhPJz4I5xTeM8r3FjKk2P8UR0MNY0c/KEQgcu+AaMPtBVMwAtIC7fjJ8yvXf6tjyXfZOsDGWKEMA== X-Received: by 2002:adf:f150:: with SMTP id y16mr6629576wro.414.1617241818313; Wed, 31 Mar 2021 18:50:18 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u3sm7090652wrt.82.2021.03.31.18.50.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Mar 2021 18:50:18 -0700 (PDT) Message-Id: <7e2d3fae9a2a0574d2f881d0bcd4931a2cad6b45.1617241803.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Thu, 01 Apr 2021 01:50:02 +0000 Subject: [PATCH v2 25/25] name-hash: use expand_to_path() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee A sparse-index loads the name-hash data for its entries, including the sparse-directory entries. If a caller asks for a path that is contained within a sparse-directory entry, we need to expand to a full index and recalculate the name hash table before returning the result. Insert calls to expand_to_path() to protect against this case. Signed-off-by: Derrick Stolee --- name-hash.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/name-hash.c b/name-hash.c index 4e03fac9bb12..75c159e06eeb 100644 --- a/name-hash.c +++ b/name-hash.c @@ -8,6 +8,7 @@ #include "cache.h" #include "thread-utils.h" #include "trace2.h" +#include "sparse-index.h" struct dir_entry { struct hashmap_entry ent; @@ -109,6 +110,12 @@ static void hash_index_entry(struct index_state *istate, struct cache_entry *ce) if (ce->ce_flags & CE_HASHED) return; ce->ce_flags |= CE_HASHED; + + if (S_ISSPARSEDIR(ce->ce_mode)) { + add_dir_entry(istate, ce); + return; + } + hashmap_entry_init(&ce->ent, memihash(ce->name, ce_namelen(ce))); hashmap_add(&istate->name_hash, &ce->ent); @@ -680,6 +687,7 @@ int index_dir_exists(struct index_state *istate, const char *name, int namelen) struct dir_entry *dir; lazy_init_name_hash(istate); + expand_to_path(istate, name, namelen, 0); dir = find_dir_entry(istate, name, namelen); return dir && dir->nr; } @@ -690,6 +698,7 @@ void adjust_dirname_case(struct index_state *istate, char *name) const char *ptr = startPtr; lazy_init_name_hash(istate); + expand_to_path(istate, name, strlen(name), 0); while (*ptr) { while (*ptr && *ptr != '/') ptr++; @@ -713,6 +722,7 @@ struct cache_entry *index_file_exists(struct index_state *istate, const char *na unsigned int hash = memihash(name, namelen); lazy_init_name_hash(istate); + expand_to_path(istate, name, namelen, icase); ce = hashmap_get_entry_from_hash(&istate->name_hash, hash, NULL, struct cache_entry, ent);