apply: canonicalize modes read from patches

Commit Message

Jeff King Aug. 5, 2024, 6 a.m. UTC

On Fri, Aug 02, 2024 at 07:57:54AM -0700, Junio C Hamano wrote:

> Makes sense.
> 
> The above is consistent with what we do for the permission bits;
> only the execute bit matters, and the patch recording 100664 should
> mean the same thing to us as permission bits 100644---we should warn
> if the on-disk file is executable while applying such a patch, and
> we should not warn otherwise.

OK, here it is with tests and a commit message. I dug around to make
sure there were no cases where the unusual mode would cause other
behavior changes, but there aren't any. We are careful to use the
canonical mode whenever we create a file.

So the tests here may be overkill (since except for the warning message,
they'd pass already), but I thought it worth demonstrating the complete
set of expected behavior. Likewise the commit message is long because I
laid out all of the things I poked at.

I didn't add tests confirming that we complain when the executable bit
is not as expected. Earlier tests in t4129 already cover that.

-- >8 --
Subject: apply: canonicalize modes read from patches

Git stores only canonical modes for blobs. So for a regular file, we
care about only "100644" or "100755" (depending only on the executable
bit), but never modes where the group or other permissions are more
exotic. So never "100664", "100700", etc. When a file in the working
tree has such a mode, we quietly turn it into one of the two canonical
modes, and that's what is stored both in the index and in tree objects.

However, we don't canonicalize modes we read from incoming patches in
git-apply. These may appear in a few lines:

  - "old mode" / "new mode" lines for mode changes

  - "new file mode" lines for newly created files

  - "deleted file mode" for removing files

For "new mode" and for "new file mode", this is harmless. The patch is
asking the result to have a certain mode, but:

  - when we add an index entry (for --index or --cached), it is
    canonicalized as we create the entry, via create_ce_mode().

  - for a working tree file, try_create_file() passes either 0777 or
    0666 to open(), so what you get depends only on your umask, not any
    other bits (aside from the executable bit) in the original mode.

However, for "old mode" and "deleted file mode", there is a minor
annoyance. We compare the patch's expected preimage mode with the
current state. But that current state is always going to be a canonical
mode itself:

  - updating an index entry via --cached will have the canonical mode in
    the index

  - for updating a working tree file, check_preimage() runs the mode
    through ce_mode_from_stat(), which does the usual canonicalization

So if the patch feeds a non-canonical mode, it's impossible for it to
match, and we will always complain with something like:

  file has type 100644, expected 100664

Since this is just a warning, the operation proceeds, but it's
confusing and annoying.

These cases should be pretty rare in practice. Git would never produce a
patch with non-canonical modes itself (since it doesn't store them).
And while we do accept patches from other programs, all of those lines
were invented by Git. So you'd need a program trying to be Git
compatible, but not handling canonicalization the same way. Reportedly
"quilt" is such a program.

We should canonicalize the modes as we read them so that the user never
sees the useless warning.

A few notes on the tests:

  - I've covered instances of all lines for completeness, even though
    the "new mode" / "new file mode" ones behave OK currently.

  - the tests apply patches to both the index and working tree, and
    check the result of both. Again, we know that all of these paths
    canonicalize anyway, but it's giving us extra coverage (although we
    are even less likely to have such a bug now since we canonicalize up
    front).

  - the test patches are missing "index" lines, which is also something
    Git would never produce. But they don't matter for the test, they do
    match the case from quilt we saw in the wild, and they avoid some
    sha1/sha256 complexity.

Reported-by: Andrew Morton <[e-mail]>
Signed-off-by: Jeff King <[e-mail]>
---
 apply.c                   |  1 +
 t/t4129-apply-samemode.sh | 62 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 63 insertions(+)

Comments

Junio C Hamano Aug. 15, 2024, 2:52 p.m. UTC | #1

Jeff King <[e-mail]> writes:

> +test_expect_success POSIXPERM 'patch mode for deleted file is canonicalized' '

This test seems to fail under "--stress" and I need to borrow a
brain better clued than mine.  It appears to be fooled by mtime that
is not updated immediately and failing match_stat check, but since
the index file is written on the other side of the second resolution
boundary, racy-git double-checking code does not trigger, or
something like that.

Here is how it fails:

expecting success of 4129.13 'patch mode for deleted file is canonicalized':
        test_when_finished "git reset --hard" &&
        echo content >non-canon &&
        git add non-canon &&
        chmod 666 non-canon &&

        cat >patch <<-\EOF &&
        diff --git a/non-canon b/non-canon
        deleted file mode 100660
        --- a/non-canon
        +++ /dev/null
        @@ -1 +0,0 @@
        -content
        EOF
        git apply --index patch 2>err &&
        test_must_be_empty err &&
        git ls-files -- non-canon >staged &&
        test_must_be_empty staged &&
        test_path_is_missing non-canon

++ test_when_finished 'git reset --hard'
++ test 0 = 0
++ test_cleanup='{ git reset --hard
                } && (exit "$eval_ret"); eval_ret=$?; :'
++ echo content
++ git add non-canon
++ chmod 666 non-canon
++ cat
++ git apply --index patch
error: last command exited with $?=1
not ok 13 - patch mode for deleted file is canonicalized
#
#               test_when_finished "git reset --hard" &&
#               echo content >non-canon &&
#               git add non-canon &&
#               chmod 666 non-canon &&
#
#               cat >patch <<-\EOF &&
#               diff --git a/non-canon b/non-canon
#               deleted file mode 100660
#               --- a/non-canon
#               +++ /dev/null
#               @@ -1 +0,0 @@
#               -content
#               EOF
#               git apply --index patch 2>err &&
#               test_must_be_empty err &&
#               git ls-files -- non-canon >staged &&
#               test_must_be_empty staged &&
#               test_path_is_missing non-canon
#
1..13
$ echo $?
1
$ git -C trash\ directory.t4129-apply-samemode.stress-failed/.git ls-files --debug non-canon
non-canon
  ctime: 1723701364:980719772
  mtime: 1723701364:980719772
  dev: 65024    ino: 1980747
  uid: 110493   gid: 89939
  size: 8       flags: 0
: git t/master; stat trash\ directory.t4129-apply-samemode.stress-failed/non-canon
  File: trash directory.t4129-apply-samemode.stress-failed/non-canon
  Size: 8               Blocks: 8          IO Block: 4096   regular file
Device: 254,0   Inode: 1980747     Links: 1
Access: (0666/-rw-rw-rw-)  Uid: (110493/     jch)   Gid: (89939/primarygroup)
Access: 2024-08-15 06:54:43.808293635 -0700
Modify: 2024-08-14 22:56:04.980719772 -0700
Change: 2024-08-14 22:56:05.020719706 -0700
 Birth: 2024-08-14 22:56:04.980719772 -0700
$ stat trash\ directory.t4129-apply-samemode.stress-failed/.git/index
  File: trash directory.t4129-apply-samemode.stress-failed/.git/index
  Size: 432             Blocks: 8          IO Block: 4096   regular file
Device: 254,0   Inode: 1980724     Links: 1
Access: (0600/-rw-------)  Uid: (110493/     jch)   Gid: (89939/primarygroup)
Access: 2024-08-14 22:56:05.044719667 -0700
Modify: 2024-08-14 22:56:05.008719726 -0700
Change: 2024-08-14 22:56:05.016719713 -0700
 Birth: 2024-08-14 22:56:04.996719746 -0700

Patch

diff --git a/apply.c b/apply.c
index 0f2f5dabe3..6e1060a952 100644
--- a/apply.c
+++ b/apply.c
@@ -995,6 +995,7 @@  static int parse_mode_line(const char *line, int linenr, unsigned int *mode)
 	*mode = strtoul(line, &end, 8);
 	if (end == line || !isspace(*end))
 		return error(_("invalid mode on line %d: %s"), linenr, line);
+	*mode = canon_mode(*mode);
 	return 0;
 }
 
diff --git a/t/t4129-apply-samemode.sh b/t/t4129-apply-samemode.sh
index 4eb8444029..d9a1084b5e 100755
--- a/t/t4129-apply-samemode.sh
+++ b/t/t4129-apply-samemode.sh
@@ -130,4 +130,66 @@  test_expect_success 'git apply respects core.fileMode' '
 	test_grep ! "has type 100644, expected 100755" err
 '
 
+test_expect_success POSIXPERM 'patch mode for new file is canonicalized' '
+	cat >patch <<-\EOF &&
+	diff --git a/non-canon b/non-canon
+	new file mode 100660
+	--- /dev/null
+	+++ b/non-canon
+	+content
+	EOF
+	test_when_finished "git reset --hard" &&
+	(
+		umask 0 &&
+		git apply --index patch 2>err
+	) &&
+	test_must_be_empty err &&
+	git ls-files -s -- non-canon >staged &&
+	test_grep "^100644" staged &&
+	ls -l non-canon >worktree &&
+	test_grep "^-rw-rw-rw" worktree
+'
+
+test_expect_success POSIXPERM 'patch mode for deleted file is canonicalized' '
+	test_when_finished "git reset --hard" &&
+	echo content >non-canon &&
+	git add non-canon &&
+	chmod 666 non-canon &&
+
+	cat >patch <<-\EOF &&
+	diff --git a/non-canon b/non-canon
+	deleted file mode 100660
+	--- a/non-canon
+	+++ /dev/null
+	@@ -1 +0,0 @@
+	-content
+	EOF
+	git apply --index patch 2>err &&
+	test_must_be_empty err &&
+	git ls-files -- non-canon >staged &&
+	test_must_be_empty staged &&
+	test_path_is_missing non-canon
+'
+
+test_expect_success POSIXPERM 'patch mode for mode change is canonicalized' '
+	test_when_finished "git reset --hard" &&
+	echo content >non-canon &&
+	git add non-canon &&
+
+	cat >patch <<-\EOF &&
+	diff --git a/non-canon b/non-canon
+	old mode 100660
+	new mode 100770
+	EOF
+	(
+		umask 0 &&
+		git apply --index patch 2>err
+	) &&
+	test_must_be_empty err &&
+	git ls-files -s -- non-canon >staged &&
+	test_grep "^100755" staged &&
+	ls -l non-canon >worktree &&
+	test_grep "^-rwxrwxrwx" worktree
+'
+
 test_done