run-command: document use_shell option

Commit Message

Jeff King Jan. 22, 2021, 9:03 p.m. UTC

On Fri, Jan 22, 2021 at 03:32:04PM -0500, Jeff King wrote:

> Yeah, that is exactly right. "use_shell" just means that the command is
> (possibly) run with a shell. Quoting for any extra arguments is handled
> automatically.
> 
> I think you're correct that this was broken from the start in 10ac85c785
> (upload-pack: add object filtering for partial clone, 2017-12-08).
> That's even before the use_shell was added, and then later it was pushed
> into that conditional by 0b6069fe0a (fetch-pack: test support excluding
> large blobs, 2017-12-08). Presumably because the non-hook path would not
> have worked at all, and that was the first time any of it was actually
> tested. ;)
> 
> (I've cc'd authors of those commits as an FYI; I think both were
> relatively new to the project at the time so misunderstanding this
> subtlety of run-command is not too surprising).

While we're thinking about it, let's beef up the documentation a bit.

-- >8 --
Subject: [PATCH] run-command: document use_shell option

It's unclear how run-command's use_shell option should impact the
arguments fed to a command. Plausibly it could mean that we glue all of
the arguments together into a string to pass to the shell, in which case
that opens the question of whether the caller needs to quote them.

But in fact we don't implement it that way (and even if we did, we'd
probably auto-quote the arguments as part of the glue step). And we must
not receive quoted arguments, because we might actually optimize out the
shell entirely (i.e., the caller does not even know if a shell will be
involved in the end or not).

Since this ambiguity may have been the cause of a recent bug, let's
document the option a bit.

Signed-off-by: Jeff King <[e-mail]>
---
 run-command.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

Comments

Taylor Blau Jan. 22, 2021, 9:32 p.m. UTC | #1

On Fri, Jan 22, 2021 at 04:03:33PM -0500, Jeff King wrote:
> Subject: [PATCH] run-command: document use_shell option

Nice. This does indeed make things a little clearer (and as we've seen
it was certainly not as clear before), so I think this is worth doing.

  Reviewed-by: Taylor Blau <[e-mail]>

Thanks,
Taylor

Junio C Hamano Jan. 22, 2021, 10:21 p.m. UTC | #2

Jeff King <[e-mail]> writes:

> diff --git a/run-command.h b/run-command.h
> index 6472b38bde..d08414a92e 100644
> --- a/run-command.h
> +++ b/run-command.h
> @@ -126,8 +126,15 @@ struct child_process {
>  	 */
>  	unsigned silent_exec_failure:1;
>  
> -	unsigned stdout_to_stderr:1;
> +	/**
> +	 * Run the command from argv[0] using a shell (but note that we may
> +	 * still optimize out the shell call if the command contains no
> +	 * metacharacters). Note that further arguments to the command in
> +	 * argv[1], etc, do not need to be shell-quoted.
> +	 */
>  	unsigned use_shell:1;
> +
> +	unsigned stdout_to_stderr:1;

Reads well.  Thanks.

It is curious why "diff" chose to move stdout_to_stderr line around,
though.

>  	unsigned clean_on_exit:1;
>  	unsigned wait_after_clean:1;
>  	void (*clean_on_exit_handler)(struct child_process *process);

Jeff King Jan. 23, 2021, 12:04 a.m. UTC | #3

On Fri, Jan 22, 2021 at 02:21:15PM -0800, Junio C Hamano wrote:

> > -	unsigned stdout_to_stderr:1;
> > +	/**
> > +	 * Run the command from argv[0] using a shell (but note that we may
> > +	 * still optimize out the shell call if the command contains no
> > +	 * metacharacters). Note that further arguments to the command in
> > +	 * argv[1], etc, do not need to be shell-quoted.
> > +	 */
> >  	unsigned use_shell:1;
> > +
> > +	unsigned stdout_to_stderr:1;
> 
> Reads well.  Thanks.
> 
> It is curious why "diff" chose to move stdout_to_stderr line around,
> though.

Heh, I also noticed that and tried a few options to no avail (both
patience/histogram, but also diff.indentHeuristic). It is one deletion
and 8 insertions even if it is done the other way:

  + /*
  + ... 6 more lines ...
  + */
  + unsigned use_shell:1;
  +
    unsigned stdout_to_stderr:1;
  - unsigned use_shell:1;

so it may just come down to the order of xdiff parsing the lines.

-Peff

Patch

diff --git a/run-command.h b/run-command.h
index 6472b38bde..d08414a92e 100644
--- a/run-command.h
+++ b/run-command.h
@@ -126,8 +126,15 @@  struct child_process {
 	 */
 	unsigned silent_exec_failure:1;
 
-	unsigned stdout_to_stderr:1;
+	/**
+	 * Run the command from argv[0] using a shell (but note that we may
+	 * still optimize out the shell call if the command contains no
+	 * metacharacters). Note that further arguments to the command in
+	 * argv[1], etc, do not need to be shell-quoted.
+	 */
 	unsigned use_shell:1;
+
+	unsigned stdout_to_stderr:1;
 	unsigned clean_on_exit:1;
 	unsigned wait_after_clean:1;
 	void (*clean_on_exit_handler)(struct child_process *process);