expand: Always quote caret when using fnmatch

Commit Message

Herbert Xu Jan. 18, 2022, 6:13 a.m. UTC

Harald van Dijk <[e-mail]> wrote:
> 
> On 12/01/2022 16:25, Christoph Anton Mitterer wrote:
>> The results for the run-circumflex seem pretty odd.
>> Apparently, the ^ is taken literally, but the other two are negated.
> 
> The ^ is not taken literally. The ^ in the pattern is wrongly taken as 
> the negation operator, and the ^ in the argument is then reported as a 
> match because it is neither . nor a.
> 
> This bug (you're right that it's a bug) is specific to builds that use 
> fnmatch(). In dash itself, ^ is always assumed as a literal. In builds 
> with --disable-fnmatch you get correct results. In builds with 
> --enable-fnmatch, because dash assumes ^ is assumed as a literal, dash 
> fails to escape it before passing it on to fnmatch(), and the system 
> fnmatch() may choose differently from dash on how to deal with unquoted 
> ^s. What dash should do to get whatever behaviour the system fnmatch() 
> chooses is leave unquoted ^s unquoted, and leave quoted ^s quoted. This 
> can be achieved by
> 
> --- a/src/mksyntax.c
> +++ b/src/mksyntax.c
> @@ -178,14 +178,14 @@ main(int argc, char **argv)
>         add("$", "CVAR");
>         add("}", "CENDVAR");
>         /* ':/' for tilde expansion, '-' for [a\-x] pattern ranges */
> -       add("!*?[=~:/-]", "CCTL");
> +       add("!*?[^=~:/-]", "CCTL");
>         print("dqsyntax");
>         init();
>         fputs("\n/* syntax table used when in single quotes */\n", cfile);
>         add("\n", "CNL");
>         add("'", "CENDQUOTE");
>         /* ':/' for tilde expansion, '-' for [a\-x] pattern ranges */
> -       add("!*?[=~:/-]\\", "CCTL");
> +       add("!*?[^=~:/-]\\", "CCTL");
>         print("sqsyntax");
>         init();
>         fputs("\n/* syntax table used when in arithmetic */\n", cfile);
> 
> However, whether this is the correct approach is a matter of opinion: 
> dash could alternatively choose to always take ^ as a literal and always 
> escape it before passing it on to fnmatch(), overriding whatever 
> decision the libc people had taken.

Yes, this would produce the most consistent result.

This patch forces ^ to be a literal when we use fnmatch.

Fixes: 7638476c18f2 ("shell: Enable fnmatch/glob by default")
Reported-by: Christoph Anton Mitterer <[e-mail]>
Suggested-by: Harald van Dijk <[e-mail]>
Signed-off-by: Herbert Xu <[e-mail]>

Comments

Harald van Dijk Jan. 18, 2022, 8:44 a.m. UTC | #1

On 18/01/2022 06:13, Herbert Xu wrote:
> This patch forces ^ to be a literal when we use fnmatch.
> 
> Fixes: 7638476c18f2 ("shell: Enable fnmatch/glob by default")
> Reported-by: Christoph Anton Mitterer <[e-mail]>
> Suggested-by: Harald van Dijk <[e-mail]>
> Signed-off-by: Herbert Xu <[e-mail]>
> 
> diff --git a/src/expand.c b/src/expand.c
> index aea5cc4..04bf8fb 100644
> --- a/src/expand.c
> +++ b/src/expand.c
> @@ -47,6 +47,9 @@
>   #include <string.h>
>   #ifdef HAVE_FNMATCH
>   #include <fnmatch.h>
> +#define FNMATCH_IS_ENABLED 1
> +#else
> +#define FNMATCH_IS_ENABLED 0
>   #endif
>   #ifdef HAVE_GLOB
>   #include <glob.h>
> @@ -1693,8 +1696,11 @@ _rmescapes(char *str, int flag)
>   			notescaped = 0;
>   			goto copy;
>   		}
> +		if (FNMATCH_IS_ENABLED && *p == '^')
> +			goto add_escape;
>   		if (*p == (char)CTLESC) {
>   			p++;
> +add_escape:
>   			if (notescaped)
>   				*q++ = '\\';
>   		}
> 

The loop that is modified by this patch is only taken after any qchars 
are seen, so for e.g.

   var=abc
   echo ${var#[^a]}

it has no effect. More importantly though, _rmescapes is used to modify 
strings in place. This patch causes _rmescapes to try and grow strings, 
which cannot ever work. A test case for this is

   case aa in \a[^a]) echo match ;; esac

which fails with a segfault after this patch is applied.

Cheers,
Harald van Dijk

Christoph Anton Mitterer Jan. 18, 2022, 2:29 p.m. UTC | #2

Hey.

Just for confirmation:

Would that also be an issue in fnmatch() (i.e. should it be forwarded?)
or really just something in dash?

Cheers,
Chris.

Chet Ramey Jan. 18, 2022, 2:54 p.m. UTC | #3

On 1/18/22 9:29 AM, Christoph Anton Mitterer wrote:
> Hey.
> 
> Just for confirmation:
> 
> Would that also be an issue in fnmatch() (i.e. should it be forwarded?)
> or really just something in dash?

The behavior of an unquoted carat as the first character in a bracket
expression is unspecified. Dash can't count on any particular behavior.

Herbert Xu Jan. 18, 2022, 10:33 p.m. UTC | #4

Christoph Anton Mitterer <[e-mail]> wrote:
> Hey.
> 
> Just for confirmation:
> 
> Would that also be an issue in fnmatch() (i.e. should it be forwarded?)
> or really just something in dash?

No I don't think this is an fnmatch issue.  It's dash's fault
for not quoting the caret before passing it to glibc.

Thanks,

Patch

diff --git a/src/expand.c b/src/expand.c
index aea5cc4..04bf8fb 100644
--- a/src/expand.c
+++ b/src/expand.c
@@ -47,6 +47,9 @@ 
 #include <string.h>
 #ifdef HAVE_FNMATCH
 #include <fnmatch.h>
+#define FNMATCH_IS_ENABLED 1
+#else
+#define FNMATCH_IS_ENABLED 0
 #endif
 #ifdef HAVE_GLOB
 #include <glob.h>
@@ -1693,8 +1696,11 @@  _rmescapes(char *str, int flag)
 			notescaped = 0;
 			goto copy;
 		}
+		if (FNMATCH_IS_ENABLED && *p == '^')
+			goto add_escape;
 		if (*p == (char)CTLESC) {
 			p++;
+add_escape:
 			if (notescaped)
 				*q++ = '\\';
 		}