config: don't BUG when both kvi and source are set

Commit Message

Glen Choo June 26, 2023, 5:41 p.m. UTC

From: Glen Choo <[e-mail]>

When iterating through config, we read config source metadata from
global values - either a "struct config_source + enum config_scope"
or a "struct key_value_info", using the current_config* functions. Prior
to the series starting from 0c60285147 (config.c: create config_reader
and the_reader, 2023-03-28), we weren't very picky about which values we
should read in which situation; we did note that both groups of values
generally shouldn't be set together, but if both were set,
current_config* preferentially reads key_value_info. When that series
added more structure, we enforced that either the former (when parsing a
config source) can be set, or the latter (when iterating a config set),
but *never* both at the same time. See 9828453ff0 (config.c: remove
current_config_kvi, 2023-03-28) and 5cdf18e7cd (config.c: remove
current_parsing_scope, 2023-03-28).

That was a good simplifying constraint that helped us reason about the
global state, but it turns out that there is at least one situation
where we need both to be set at the same time: in a blobless partial
clone where .gitmodules is missing. "git fetch" in such a repo will
start a config parse over .gitmodules (setting the config_source), and
Git will attempt to lazy-fetch it from the promisor remote. However,
when we try to read the promisor configuration, we start iterating a
config set (setting the key_value_info), and we BUG() out because that's
not allowed any more.

Teaching config_reader to gracefully handle this is somewhat
complicated, but fortunately, there are proposed changes to the config.c
machinery to get rid of this global state, and make the BUG() obsolete
[1]. We should rely on that as the eventual solution, and avoid doing
yet another refactor in the meantime.

Therefore, fix the bug by removing the BUG() check. We're reverting to
an older, less safe state, but that's generally okay since
key_value_info is always preferentially read, so we'd always read the
correct values when we iterate a config set in the middle of a config
parse (like we are here). The reverse would be wrong, but extremely
unlikely to happen since very few callers parse config without going
through a config set.

[1] https://lore.kernel.org/git/[e-mail]

Signed-off-by: Glen Choo <[e-mail]>
---
    config: don't BUG when both kvi and source are set
    
    Here's a quick fix for the bug reported at [1]. As noted in the commit
    message and that thread, I think the real fix to take [2], which
    simplifies the config.c state and makes this a non-issue, so this is
    just a band-aid while we wait for that.
    
    [1]
    https://lore.kernel.org/git/CAJSLrw6qhHj8Kxrqhp7xN=[e-mail]/
    [2]
    https://lore.kernel.org/git/[e-mail]

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1535%2Fchooglen%2Fpush-ppuusrxwqpkt-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1535/chooglen/push-ppuusrxwqpkt-v1
Pull-Request: https://github.com/git/git/pull/1535

 config.c                 |  6 ------
 t/t5616-partial-clone.sh | 10 ++++++++--
 2 files changed, 8 insertions(+), 8 deletions(-)


base-commit: 6640c2d06d112675426cf436f0594f0e8c614848

Comments

Junio C Hamano June 26, 2023, 7:06 p.m. UTC | #1

"Glen Choo via GitGitGadget" <[e-mail]> writes:

> Therefore, fix the bug by removing the BUG() check. We're reverting to
> an older, less safe state, but that's generally okay since
> key_value_info is always preferentially read, so we'd always read the
> correct values when we iterate a config set in the middle of a config
> parse (like we are here).

I wonder if the source being pushed and config_kvi value at this
point have some particular relationship (like "if kvi exists, the
source must match kvi's source" or something) that we can cheaply
use to avoid "reverting to an older less safe state"?

I would agree that, as long as we know by the end of this summer a
real fix would come to rescue us ;-), it is sensible not to add too
much code to work it around for the short-term.

> The reverse would be wrong, but extremely
> unlikely to happen since very few callers parse config without going
> through a config set.

Sorry, but I do not quite get this comment.

>     Here's a quick fix for the bug reported at [1]. As noted in the commit
>     message and that thread, I think the real fix to take [2], which
>     simplifies the config.c state and makes this a non-issue, so this is
>     just a band-aid while we wait for that.

Thanks for a quick fix.  Will queue.

> diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh
> index f519d2a87a7..8759fc28533 100755
> --- a/t/t5616-partial-clone.sh
> +++ b/t/t5616-partial-clone.sh
> @@ -257,8 +257,8 @@ test_expect_success 'partial clone with transfer.fsckobjects=1 works with submod
>  	test_commit -C submodule mycommit &&
>  
>  	test_create_repo src_with_sub &&
> -	test_config -C src_with_sub uploadpack.allowfilter 1 &&
> -	test_config -C src_with_sub uploadpack.allowanysha1inwant 1 &&
> +	git -C src_with_sub config uploadpack.allowfilter 1 &&
> +	git -C src_with_sub config uploadpack.allowanysha1inwant 1 &&

We only tentatively configured uploadpack in src_with_sub in the
original because this single test piece was the only place where
src_with_sub repository was used, but now we use a more permanent
configuration because ...

> @@ -270,6 +270,12 @@ test_expect_success 'partial clone with transfer.fsckobjects=1 works with submod
>  	test_when_finished rm -rf dst
>  '
>  
> +test_expect_success 'lazily fetched .gitmodules works' '
> +	git clone --filter="blob:none" --no-checkout "file://$(pwd)/src_with_sub" dst &&
> +	git -C dst fetch &&
> +	test_when_finished rm -rf dst
> +'

... we run another "git clone" from the repository now.

OK.

Thanks.

Glen Choo June 26, 2023, 10:56 p.m. UTC | #2

Junio C Hamano <[e-mail]> writes:

>> Therefore, fix the bug by removing the BUG() check. We're reverting to
>> an older, less safe state, but that's generally okay since
>> key_value_info is always preferentially read, so we'd always read the
>> correct values when we iterate a config set in the middle of a config
>> parse (like we are here).
>
> I wonder if the source being pushed and config_kvi value at this
> point have some particular relationship (like "if kvi exists, the
> source must match kvi's source" or something) that we can cheaply
> use to avoid "reverting to an older less safe state"?

Not at all. In this case, the source should reflect .gitmodules, but the
config_kvi should reflect the promisor config (aka the full repo
config). config_source implements stack semantics, so we could co-opt it
by e.g. converting config_kvi into a fake config_source and pushing it
onto the stack (at which point, we could just get rid of config_kvi
altogether too), but that's really way too much work for something that
will _hopefully_ go away soon.

>> The reverse would be wrong, but extremely
>> unlikely to happen since very few callers parse config without going
>> through a config set.
>
> Sorry, but I do not quite get this comment.

Ah, I meant that this bug occurred because most users of config use
git_config()/repo_config() (a wrapper around config sets), so it's very
easy to accidentally read repo config, e.g. in the middle of parsing
config (config file -> config set). I'd imagine it might also be quite
easy to read repo config while reading repo config (config set -> config
set), which would make current_config_* return the wrong thing, but at
least it doesn't BUG().

The "reverse" case (config set -> config file) is very _unlikely_
because very few places need to know about config files, so it's
unlikely that we'd have an explicit call to parse a config file,
especially in the middle of reading repo config.

Junio C Hamano June 26, 2023, 11:05 p.m. UTC | #3

Glen Choo <[e-mail]> writes:

> Ah, I meant that this bug occurred because most users of config use
> git_config()/repo_config() (a wrapper around config sets), so it's very
> easy to accidentally read repo config, e.g. in the middle of parsing
> config (config file -> config set). I'd imagine it might also be quite
> easy to read repo config while reading repo config (config set -> config
> set), which would make current_config_* return the wrong thing, but at
> least it doesn't BUG().

I think BUG() is better than silently computing a wrong result, but
it would probably be much rare than the problem at hand, and with
the getting rid of global dependencies, it won't be an issue anymore,
hopefull?  So it is good.

> The "reverse" case (config set -> config file) is very _unlikely_
> because very few places need to know about config files, so it's
> unlikely that we'd have an explicit call to parse a config file,
> especially in the middle of reading repo config.

As long as existing codepaths do not do that, it would be OK ;-)

Thanks.

Patch

diff --git a/config.c b/config.c
index f5bdac0aeed..3edb9d72dd3 100644
--- a/config.c
+++ b/config.c
@@ -106,8 +106,6 @@  static struct config_reader the_reader;
 static inline void config_reader_push_source(struct config_reader *reader,
 					     struct config_source *top)
 {
-	if (reader->config_kvi)
-		BUG("source should not be set while iterating a config set");
 	top->prev = reader->source;
 	reader->source = top;
 }
@@ -125,16 +123,12 @@  static inline struct config_source *config_reader_pop_source(struct config_reade
 static inline void config_reader_set_kvi(struct config_reader *reader,
 					 struct key_value_info *kvi)
 {
-	if (kvi && (reader->source || reader->parsing_scope))
-		BUG("kvi should not be set while parsing a config source");
 	reader->config_kvi = kvi;
 }
 
 static inline void config_reader_set_scope(struct config_reader *reader,
 					   enum config_scope scope)
 {
-	if (scope && reader->config_kvi)
-		BUG("scope should only be set when iterating through a config source");
 	reader->parsing_scope = scope;
 }
 
diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh
index f519d2a87a7..8759fc28533 100755
--- a/t/t5616-partial-clone.sh
+++ b/t/t5616-partial-clone.sh
@@ -257,8 +257,8 @@  test_expect_success 'partial clone with transfer.fsckobjects=1 works with submod
 	test_commit -C submodule mycommit &&
 
 	test_create_repo src_with_sub &&
-	test_config -C src_with_sub uploadpack.allowfilter 1 &&
-	test_config -C src_with_sub uploadpack.allowanysha1inwant 1 &&
+	git -C src_with_sub config uploadpack.allowfilter 1 &&
+	git -C src_with_sub config uploadpack.allowanysha1inwant 1 &&
 
 	test_config_global protocol.file.allow always &&
 
@@ -270,6 +270,12 @@  test_expect_success 'partial clone with transfer.fsckobjects=1 works with submod
 	test_when_finished rm -rf dst
 '
 
+test_expect_success 'lazily fetched .gitmodules works' '
+	git clone --filter="blob:none" --no-checkout "file://$(pwd)/src_with_sub" dst &&
+	git -C dst fetch &&
+	test_when_finished rm -rf dst
+'
+
 test_expect_success 'partial clone with transfer.fsckobjects=1 uses index-pack --fsck-objects' '
 	git init src &&
 	test_commit -C src x &&