| Message ID | patch-v9-9.9-445467e87f7-20220127T115058Z-avarab@gmail.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | grep: simplify & delete "init" & "config" code | expand |
Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > In a preceding commit we changed grep_config() to be called after > grep_init(), which means that much of the complexity here can go > away. > > As before both "grep.patternType" and "grep.extendedRegexp" are > last-one-wins variable, with "grep.extendedRegexp" yielding to > "grep.patternType", except when "grep.patternType=default". > > Note that this applies as we parse the config, i.e. a sequence of: > > -c grep.patternType=perl > -c grep.extendedRegexp=true \ > -c grep.patternType=default > > should select ERE due to "grep.extendedRegexp=true and > grep.patternType=default". Correct. The same reasoning would apply to: -c grep.extendedRegexp=true \ -c grep.patternType=perl \ -c grep.patternType=default which should also select ERE due to "grep.extendedRegexp=true and grep.patternType=default". As we re-check grep.extendedRegexp every time grep.patternType changes, and we keep extendedRegexp as a separate copy without getting lost like earlier rounds of this series, this should work. Would this also work? -c grep.extendedRegexp=false \ -c grep.patternType=default \ -c grep.extendedRegexp=true We do keep extendedRegexp, but as soon as we read .patternType that is default, adjust_pattern_type() overwrites the pattern_type_option member with BRE, and the fact that .patternType was specified as "do whatever the .extendedRegexp says" is lost when we read the third one. So, no, I am not sure this is correct. > diff --git a/grep.c b/grep.c > index 60228a95a4f..f07a21ff1aa 100644 > --- a/grep.c > +++ b/grep.c > @@ -48,6 +48,12 @@ static int parse_pattern_type_arg(const char *opt, const char *arg) > > define_list_config_array_extra(color_grep_slots, {"match"}); > > +static void adjust_pattern_type(enum grep_pattern_type *pto, const int ero) > +{ > + if (*pto == GREP_PATTERN_TYPE_UNSPECIFIED) > + *pto = ero ? GREP_PATTERN_TYPE_ERE : GREP_PATTERN_TYPE_BRE; > +} OK. Earlier rounds just replaced the UNSPECIFIED with hardcoded value "0", which was more or less pointless. I think this is easier to follow. But as I said, "committing" ERE vs BRE in this manner is probably way too early and produce an incorrect result. Instead ... > @@ -490,9 +446,9 @@ static void compile_regexp(struct grep_pat *p, struct grep_opt *opt) ... this is the right place to do the "see if pattern_type_option is 'default' and if so use 'extended_regexp_option' to commit to either BRE or ERE". I guess that is what I have been repeating during the review of the past few rounds. Am I overlooking some other cases where that simpler-to-explain approach does not work? > p->word_regexp = opt->word_regexp; > p->ignore_case = opt->ignore_case; > - p->fixed = opt->fixed; > + p->fixed = opt->pattern_type_option == GREP_PATTERN_TYPE_FIXED; > > - if (!opt->pcre2 && > + if (opt->pattern_type_option != GREP_PATTERN_TYPE_PCRE && > memchr(p->pattern, 0, p->patternlen)) > die(_("given pattern contains NULL byte (via -f <file>). This is only supported with -P under PCRE v2")); > > @@ -544,14 +500,14 @@ static void compile_regexp(struct grep_pat *p, struct grep_opt *opt) > return; > } > > - if (opt->pcre2) { > + if (opt->pattern_type_option == GREP_PATTERN_TYPE_PCRE) { > compile_pcre2_pattern(p, opt); > return; > } > > if (p->ignore_case) > regflags |= REG_ICASE; > - if (opt->extended_regexp_option) > + if (opt->pattern_type_option == GREP_PATTERN_TYPE_ERE) > regflags |= REG_EXTENDED; > err = regcomp(&p->regexp, p->pattern, regflags); > if (err) { > diff --git a/grep.h b/grep.h > index 89a2ce51130..f89324e9aa9 100644 > --- a/grep.h > +++ b/grep.h > @@ -143,7 +143,6 @@ struct grep_opt { > int unmatch_name_only; > int count; > int word_regexp; > - int fixed; > int all_match; > int no_body_match; > int body_hit; > @@ -154,7 +153,6 @@ struct grep_opt { > int allow_textconv; > int extended; > int use_reflog_filter; > - int pcre2; > int relative; > int pathname; > int null_following_name; > @@ -183,6 +181,7 @@ struct grep_opt { > .relative = 1, \ > .pathname = 1, \ > .max_depth = -1, \ > + .extended_regexp_option = -1, \ I do not think you meant this. Uninitialized grep.extendedRegexp defaults to 0 (BRE), I think. > .pattern_type_option = GREP_PATTERN_TYPE_UNSPECIFIED, \ > .colors = { \ > [GREP_COLOR_CONTEXT] = "", \ > @@ -202,7 +201,6 @@ struct grep_opt { > > int grep_config(const char *var, const char *value, void *); > void grep_init(struct grep_opt *, struct repository *repo); > -void grep_commit_pattern_type(enum grep_pattern_type, struct grep_opt *opt); > > void append_grep_pat(struct grep_opt *opt, const char *pat, size_t patlen, const char *origin, int no, enum grep_pat_token t); > void append_grep_pattern(struct grep_opt *opt, const char *pat, const char *origin, int no, enum grep_pat_token t); > diff --git a/revision.c b/revision.c > index d6e0e2b23b5..dd301e30147 100644 > --- a/revision.c > +++ b/revision.c > @@ -2860,8 +2860,6 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s > > diff_setup_done(&revs->diffopt); > > - grep_commit_pattern_type(GREP_PATTERN_TYPE_UNSPECIFIED, > - &revs->grep_filter); > if (!is_encoding_utf8(get_log_output_encoding())) > revs->grep_filter.ignore_locale = 1; > compile_grep_patterns(&revs->grep_filter);
Junio C Hamano <gitster@pobox.com> writes: > Would this also work? > > -c grep.extendedRegexp=false \ > -c grep.patternType=default \ > -c grep.extendedRegexp=true > > We do keep extendedRegexp, but as soon as we read .patternType that > is default, adjust_pattern_type() overwrites the pattern_type_option > member with BRE, and the fact that .patternType was specified as "do > whatever the .extendedRegexp says" is lost when we read the third > one. > > So, no, I am not sure this is correct. > ... > But as I said, "committing" ERE vs BRE in this manner is probably > way too early and produce an incorrect result. Instead ... > >> @@ -490,9 +446,9 @@ static void compile_regexp(struct grep_pat *p, struct grep_opt *opt) > > ... this is the right place to do the "see if pattern_type_option is > 'default' and if so use 'extended_regexp_option' to commit to either > BRE or ERE". > > I guess that is what I have been repeating during the review of the > past few rounds. Am I overlooking some other cases where that > simpler-to-explain approach does not work? > ... >> .max_depth = -1, \ >> + .extended_regexp_option = -1, \ > > I do not think you meant this. Uninitialized grep.extendedRegexp > defaults to 0 (BRE), I think. Taking all together, here is a squashable fix with an additional test. In addition to squashing the following in, we must update the proposed log message. Given that, even after taking _this long_ (I think I have been saying this since the review of v5 iteration), this series is still making the same mistake again, the fact that the code needs to read all the configuration variables before it can correctly decide what type the user really means deserves to be stressed in the log message. Despite what the proposed log message for this round (and many other previous iterations) claimed, it fundamentally cannot be done inside the callback, simply because the callback will not know how many more times it will be called with what value for grep.patternType and grep.extendedRegexp. It can be done anywhere after the option parser has finished reading all the options and knows there will not be any more grep.patternType and grep.extendedRegexp that would affect the computation. One of the most natural such place is at the beginning of compile_regexp(), I would think. Other than that, all the previous steps looked good, so did the parts of this commit that the attached fix-up does not touch. It is great that we do not have to carry "fixed", "pcre", etc. around as separate members. Thanks. grep.c | 17 +++++------------ grep.h | 2 +- t/t7810-grep.sh | 10 ++++++++++ 3 files changed, 16 insertions(+), 13 deletions(-) diff --git c/grep.c w/grep.c index f07a21ff1a..a8f503f55c 100644 --- c/grep.c +++ w/grep.c @@ -48,12 +48,6 @@ static int parse_pattern_type_arg(const char *opt, const char *arg) define_list_config_array_extra(color_grep_slots, {"match"}); -static void adjust_pattern_type(enum grep_pattern_type *pto, const int ero) -{ - if (*pto == GREP_PATTERN_TYPE_UNSPECIFIED) - *pto = ero ? GREP_PATTERN_TYPE_ERE : GREP_PATTERN_TYPE_BRE; -} - /* * Read the configuration file once and store it in * the grep_defaults template. @@ -68,17 +62,11 @@ int grep_config(const char *var, const char *value, void *cb) if (!strcmp(var, "grep.extendedregexp")) { opt->extended_regexp_option = git_config_bool(var, value); - adjust_pattern_type(&opt->pattern_type_option, - opt->extended_regexp_option); return 0; } if (!strcmp(var, "grep.patterntype")) { opt->pattern_type_option = parse_pattern_type_arg(var, value); - if (opt->extended_regexp_option == -1) - return 0; - adjust_pattern_type(&opt->pattern_type_option, - opt->extended_regexp_option); return 0; } @@ -444,6 +432,11 @@ static void compile_regexp(struct grep_pat *p, struct grep_opt *opt) int err; int regflags = REG_NEWLINE; + if (opt->pattern_type_option == GREP_PATTERN_TYPE_UNSPECIFIED) + opt->pattern_type_option = (opt->extended_regexp_option + ? GREP_PATTERN_TYPE_ERE + : GREP_PATTERN_TYPE_BRE); + p->word_regexp = opt->word_regexp; p->ignore_case = opt->ignore_case; p->fixed = opt->pattern_type_option == GREP_PATTERN_TYPE_FIXED; diff --git c/grep.h w/grep.h index f89324e9aa..bdc6765482 100644 --- c/grep.h +++ w/grep.h @@ -181,7 +181,7 @@ struct grep_opt { .relative = 1, \ .pathname = 1, \ .max_depth = -1, \ - .extended_regexp_option = -1, \ + .extended_regexp_option = 0, \ .pattern_type_option = GREP_PATTERN_TYPE_UNSPECIFIED, \ .colors = { \ [GREP_COLOR_CONTEXT] = "", \ diff --git c/t/t7810-grep.sh w/t/t7810-grep.sh index 34d8f69c1d..b818e656ad 100755 --- c/t/t7810-grep.sh +++ w/t/t7810-grep.sh @@ -491,6 +491,16 @@ do test_cmp expected actual ' + test_expect_success "grep $L with grep.extendedRegexp and grep.patternType are both last-one-wins independently (ERE)" ' + echo "${HC}ab:abc" >expected && + git \ + -c grep.extendedRegexp=false \ + -c grep.patternType=default \ + -c grep.extendedRegexp=true \ + grep "a+b*c" $H ab >actual && + test_cmp expected actual + ' + test_expect_success "grep $L with grep.patternType=extended and grep.patternType=default" ' echo "${HC}ab:a+bc" >expected && git \
Junio C Hamano <gitster@pobox.com> writes: > In addition to squashing the following in, we must update the > proposed log message. Given that, even after taking _this long_ (I > think I have been saying this since the review of v5 iteration), I found a reference: https://lore.kernel.org/git/xmqqv8zf6j86.fsf@gitster.g/ it indeed was [v5 7/7]. It is interesting that the breaking example I gave in the message is exactly the one that I wrote (without going back to the archive) in the "squashable fix" patch in the message I am responding to.
diff --git a/builtin/grep.c b/builtin/grep.c index 0ea124321b6..942c4b25077 100644 --- a/builtin/grep.c +++ b/builtin/grep.c @@ -845,7 +845,6 @@ int cmd_grep(int argc, const char **argv, const char *prefix) int i; int dummy; int use_index = 1; - int pattern_type_arg = GREP_PATTERN_TYPE_UNSPECIFIED; int allow_revs; struct option options[] = { @@ -879,16 +878,16 @@ int cmd_grep(int argc, const char **argv, const char *prefix) N_("descend at most <depth> levels"), PARSE_OPT_NONEG, NULL, 1 }, OPT_GROUP(""), - OPT_SET_INT('E', "extended-regexp", &pattern_type_arg, + OPT_SET_INT('E', "extended-regexp", &opt.pattern_type_option, N_("use extended POSIX regular expressions"), GREP_PATTERN_TYPE_ERE), - OPT_SET_INT('G', "basic-regexp", &pattern_type_arg, + OPT_SET_INT('G', "basic-regexp", &opt.pattern_type_option, N_("use basic POSIX regular expressions (default)"), GREP_PATTERN_TYPE_BRE), - OPT_SET_INT('F', "fixed-strings", &pattern_type_arg, + OPT_SET_INT('F', "fixed-strings", &opt.pattern_type_option, N_("interpret patterns as fixed strings"), GREP_PATTERN_TYPE_FIXED), - OPT_SET_INT('P', "perl-regexp", &pattern_type_arg, + OPT_SET_INT('P', "perl-regexp", &opt.pattern_type_option, N_("use Perl-compatible regular expressions"), GREP_PATTERN_TYPE_PCRE), OPT_GROUP(""), @@ -982,7 +981,6 @@ int cmd_grep(int argc, const char **argv, const char *prefix) argc = parse_options(argc, argv, prefix, options, grep_usage, PARSE_OPT_KEEP_DASHDASH | PARSE_OPT_STOP_AT_NON_OPTION); - grep_commit_pattern_type(pattern_type_arg, &opt); if (use_index && !startup_info->have_repository) { int fallback = 0; diff --git a/grep.c b/grep.c index 60228a95a4f..f07a21ff1aa 100644 --- a/grep.c +++ b/grep.c @@ -48,6 +48,12 @@ static int parse_pattern_type_arg(const char *opt, const char *arg) define_list_config_array_extra(color_grep_slots, {"match"}); +static void adjust_pattern_type(enum grep_pattern_type *pto, const int ero) +{ + if (*pto == GREP_PATTERN_TYPE_UNSPECIFIED) + *pto = ero ? GREP_PATTERN_TYPE_ERE : GREP_PATTERN_TYPE_BRE; +} + /* * Read the configuration file once and store it in * the grep_defaults template. @@ -62,11 +68,17 @@ int grep_config(const char *var, const char *value, void *cb) if (!strcmp(var, "grep.extendedregexp")) { opt->extended_regexp_option = git_config_bool(var, value); + adjust_pattern_type(&opt->pattern_type_option, + opt->extended_regexp_option); return 0; } if (!strcmp(var, "grep.patterntype")) { opt->pattern_type_option = parse_pattern_type_arg(var, value); + if (opt->extended_regexp_option == -1) + return 0; + adjust_pattern_type(&opt->pattern_type_option, + opt->extended_regexp_option); return 0; } @@ -115,62 +127,6 @@ void grep_init(struct grep_opt *opt, struct repository *repo) opt->header_tail = &opt->header_list; } -static void grep_set_pattern_type_option(enum grep_pattern_type pattern_type, struct grep_opt *opt) -{ - /* - * When committing to the pattern type by setting the relevant - * fields in grep_opt it's generally not necessary to zero out - * the fields we're not choosing, since they won't have been - * set by anything. The extended_regexp_option field is the - * only exception to this. - * - * This is because in the process of parsing grep.patternType - * & grep.extendedRegexp we set opt->pattern_type_option and - * opt->extended_regexp_option, respectively. We then - * internally use opt->extended_regexp_option to see if we're - * compiling an ERE. It must be unset if that's not actually - * the case. - */ - if (pattern_type != GREP_PATTERN_TYPE_ERE && - opt->extended_regexp_option) - opt->extended_regexp_option = 0; - - switch (pattern_type) { - case GREP_PATTERN_TYPE_UNSPECIFIED: - /* fall through */ - - case GREP_PATTERN_TYPE_BRE: - break; - - case GREP_PATTERN_TYPE_ERE: - opt->extended_regexp_option = 1; - break; - - case GREP_PATTERN_TYPE_FIXED: - opt->fixed = 1; - break; - - case GREP_PATTERN_TYPE_PCRE: - opt->pcre2 = 1; - break; - } -} - -void grep_commit_pattern_type(enum grep_pattern_type pattern_type, struct grep_opt *opt) -{ - if (pattern_type != GREP_PATTERN_TYPE_UNSPECIFIED) - grep_set_pattern_type_option(pattern_type, opt); - else if (opt->pattern_type_option != GREP_PATTERN_TYPE_UNSPECIFIED) - grep_set_pattern_type_option(opt->pattern_type_option, opt); - else if (opt->extended_regexp_option) - /* - * This branch *must* happen after setting from the - * opt->pattern_type_option above, we don't want - * grep.extendedRegexp to override grep.patternType! - */ - grep_set_pattern_type_option(GREP_PATTERN_TYPE_ERE, opt); -} - static struct grep_pat *create_grep_pat(const char *pat, size_t patlen, const char *origin, int no, enum grep_pat_token t, @@ -490,9 +446,9 @@ static void compile_regexp(struct grep_pat *p, struct grep_opt *opt) p->word_regexp = opt->word_regexp; p->ignore_case = opt->ignore_case; - p->fixed = opt->fixed; + p->fixed = opt->pattern_type_option == GREP_PATTERN_TYPE_FIXED; - if (!opt->pcre2 && + if (opt->pattern_type_option != GREP_PATTERN_TYPE_PCRE && memchr(p->pattern, 0, p->patternlen)) die(_("given pattern contains NULL byte (via -f <file>). This is only supported with -P under PCRE v2")); @@ -544,14 +500,14 @@ static void compile_regexp(struct grep_pat *p, struct grep_opt *opt) return; } - if (opt->pcre2) { + if (opt->pattern_type_option == GREP_PATTERN_TYPE_PCRE) { compile_pcre2_pattern(p, opt); return; } if (p->ignore_case) regflags |= REG_ICASE; - if (opt->extended_regexp_option) + if (opt->pattern_type_option == GREP_PATTERN_TYPE_ERE) regflags |= REG_EXTENDED; err = regcomp(&p->regexp, p->pattern, regflags); if (err) { diff --git a/grep.h b/grep.h index 89a2ce51130..f89324e9aa9 100644 --- a/grep.h +++ b/grep.h @@ -143,7 +143,6 @@ struct grep_opt { int unmatch_name_only; int count; int word_regexp; - int fixed; int all_match; int no_body_match; int body_hit; @@ -154,7 +153,6 @@ struct grep_opt { int allow_textconv; int extended; int use_reflog_filter; - int pcre2; int relative; int pathname; int null_following_name; @@ -183,6 +181,7 @@ struct grep_opt { .relative = 1, \ .pathname = 1, \ .max_depth = -1, \ + .extended_regexp_option = -1, \ .pattern_type_option = GREP_PATTERN_TYPE_UNSPECIFIED, \ .colors = { \ [GREP_COLOR_CONTEXT] = "", \ @@ -202,7 +201,6 @@ struct grep_opt { int grep_config(const char *var, const char *value, void *); void grep_init(struct grep_opt *, struct repository *repo); -void grep_commit_pattern_type(enum grep_pattern_type, struct grep_opt *opt); void append_grep_pat(struct grep_opt *opt, const char *pat, size_t patlen, const char *origin, int no, enum grep_pat_token t); void append_grep_pattern(struct grep_opt *opt, const char *pat, const char *origin, int no, enum grep_pat_token t); diff --git a/revision.c b/revision.c index d6e0e2b23b5..dd301e30147 100644 --- a/revision.c +++ b/revision.c @@ -2860,8 +2860,6 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s diff_setup_done(&revs->diffopt); - grep_commit_pattern_type(GREP_PATTERN_TYPE_UNSPECIFIED, - &revs->grep_filter); if (!is_encoding_utf8(get_log_output_encoding())) revs->grep_filter.ignore_locale = 1; compile_grep_patterns(&revs->grep_filter);
Simplify the parsing of "grep.patternType" and "grep.extendedRegexp". This changes no behavior, but gets rid of complex parsing logic that isn't needed anymore. When "grep.patternType" was introduced in 84befcd0a4a (grep: add a grep.patternType configuration setting, 2012-08-03) we promised that: 1. You can set "grep.patternType", and "[setting it to] 'default' will return to the default matching behavior". In that context "the default" meant whatever the configuration system specified before that change, i.e. via grep.extendedRegexp. 2. We'd support the existing "grep.extendedRegexp" option, but ignore it when the new "grep.patternType" option is set. We said we'd only ignore the older "grep.extendedRegexp" option "when the `grep.patternType` option is set to a value other than 'default'". In a preceding commit we changed grep_config() to be called after grep_init(), which means that much of the complexity here can go away. As before both "grep.patternType" and "grep.extendedRegexp" are last-one-wins variable, with "grep.extendedRegexp" yielding to "grep.patternType", except when "grep.patternType=default". Note that this applies as we parse the config, i.e. a sequence of: -c grep.patternType=perl -c grep.extendedRegexp=true \ -c grep.patternType=default should select ERE due to "grep.extendedRegexp=true and grep.patternType=default". We can determine this as we parse the config, because: * If we see "grep.extendedRegexp" we set "extended_regexp_option" to its boolean value. * If we see "grep.extendedRegexp" but "grep.patternType=[default|<unset>]" is in effect we *don't* set the internal "pattern_type_option" to update the pattern type. * If we see "grep.patternType!=default" we can set our internal "pattern_type_option" directly, it doesn't matter what the state of "extended_regexp_option" is, but we don't forget what it was, in case we see a "grep.patternType=default" again. * If we see a "grep.patternType=default" we can set the pattern to ERE or BRE depending on whether we last saw a "grep.extendedRegexp=true" or "grep.extendedRegexp=[false|<unset>]". With this change the "extended_regexp_option" member is only used within grep_config(), and in the current codebase we could equally track it as a "static" variable within that function, see [1] for a version for this patch that did that. We're keeping it a struct member to make that function reentrant, in case it ends up mattering in the future. The command-line parsing in cmd_grep() can then completely ignore "grep.extendedRegexp". Whatever effect it had before that step won't matter if we see -G, -E, -P etc. See my 07a3d411739 (grep: remove regflags from the public grep_opt API, 2017-06-29) for addition of the two comments being removed here, i.e. the complexity noted in that commit is now going away. 1. https://lore.kernel.org/git/patch-v8-09.10-c211bb0c69d-20220118T155211Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- builtin/grep.c | 10 +++---- grep.c | 76 +++++++++++--------------------------------------- grep.h | 4 +-- revision.c | 2 -- 4 files changed, 21 insertions(+), 71 deletions(-)