[v2,0/3] mailinfo: disallow and complains about NUL character

Message

Đoàn Trần Công Danh April 19, 2020, 11 a.m. UTC

On 2020-04-18 22:32:30-0700, Junio C Hamano <[e-mail]> wrote:
> Martin Ågren <[e-mail]> writes:
>
> > My knee-jerk reaction to Junio's question was along the same line:
> > surely if we could have a NUL in there, the current `strlen()` would use
> > it as an excuse to silently truncate the string, either before
> > processing or afterwards. Thanks for looking into that more.
>
> Exactly.
>
> The change introduces a behaviour change, and we should describe it
> in the log message to help future developers know that we did this
> change, knowingly that we are changing the behaviour, and believing
> that the new behaviour is a better one.

Well, I intended to do something else first (because of Thunderbird started
with HTML email and replace space with non-breaking-space).

I made this change without thinking that much :( And I weren't sure if this
change is well received.

I've rephased the commit message, added some test, and disallow another NUL
characters mailinfo.

-------------------8<------------------------

As of current state, Git disallow NUL character in commit message.

This indirectly disallow NUL in the body of commit message that sent by email
in UTF-8 encoding.

In those below cases:

* NUL character in header field (be it Subject, Author, Email, etc...)
* NUL in the body of commit message that originally sent in other than UTF-8
  encoding

Git silently accepts them, albeit, truncate at the NUL character.

Those email is clearly not generated by Git, they must be crafted.

Complains loudly about those NUL characters.

Đoàn Trần Công Danh (3):
  t4254: merge 2 steps of a single test
  mailinfo.c::convert_to_utf8: reuse strlen info
  mailinfo: disallow NUL character in mail's header

 mailinfo.c            | 11 +++++++--
 t/t4254-am-corrupt.sh | 52 +++++++++++++++++++++++++++++++++++++++----
 2 files changed, 57 insertions(+), 6 deletions(-)

Range-diff against v1:
-:  ---------- > 1:  d1bc31e692 t4254: merge 2 steps of a single test
1:  9ce4b7c350 ! 2:  e3e542d388 mailinfo.c::convert_to_utf8: reuse strlen info
    @@ Commit message
         We're passing buffer from strbuf to reencode_string,
         which will call strlen(3) on that buffer,
         and discard the length of newly created buffer.
    -
         Then, we compute the length of the return buffer to attach to strbuf.
     
    +    During this process, we introduce a discrimination between mail
    +    originally written in utf-8 and other encoding.
    +
    +    * if the email was written in utf-8, we leave it as is. If there is
    +      a NUL character in that line, we complains loudly:
    +
    +            error: a NUL byte in commit log message not allowed.
    +
    +    * if the email was written in other encoding, we truncate the data as
    +      the NUL character in that line, then we used the truncated line for
    +      the metadata.
    +
         We can do better by reusing all the available information,
         and call the underlying lower level function that will be called
    -    indirectly by reencode_string.
    +    indirectly by reencode_string. By doing this, we will also postpone
    +    the NUL character processing to the commit step, which will
    +    complains about the faulty metadata.
     
         Signed-off-by: Đoàn Trần Công Danh <[e-mail]>
     
    @@ mailinfo.c: static int convert_to_utf8(struct mailinfo *mi,
      	return 0;
      }
      
    +
    + ## t/t4254-am-corrupt.sh ##
    +@@
    + test_description='git am with corrupt input'
    + . ./test-lib.sh
    + 
    ++write_nul_patch() {
    ++	space=' '
    ++	qNUL=
    ++	case "$1" in
    ++		subject) qNUL='=00' ;;
    ++	esac
    ++	cat <<-EOF
    ++	From ec7364544f690c560304f5a5de9428ea3b978b26 Mon Sep 17 00:00:00 2001
    ++	From: A U Thor <[e-mail]>
    ++	Date: Sun, 19 Apr 2020 13:42:07 +0700
    ++	Subject: [PATCH] =?ISO-8859-1?q?=C4=CB${qNUL}=D1=CF=D6?=
    ++	MIME-Version: 1.0
    ++	Content-Type: text/plain; charset=ISO-8859-1
    ++	Content-Transfer-Encoding: 8bit
    ++
    ++	EOF
    ++	if test "$1" = body
    ++	then
    ++		printf "%s\0%s\n" abc def
    ++	fi
    ++	cat <<-\EOF
    ++	---
    ++	diff --git a/afile b/afile
    ++	new file mode 100644
    ++	index 0000000000..e69de29bb2
    ++	--$space
    ++	2.26.1
    ++	EOF
    ++}
    ++
    + test_expect_success setup '
    + 	# Note the missing "+++" line:
    + 	cat >bad-patch.diff <<-\EOF &&
    +@@ t/t4254-am-corrupt.sh: test_expect_success 'try to apply corrupted patch' '
    + 	test_i18ncmp expected actual
    + '
    + 
    ++test_expect_success "NUL in commit message's body" '
    ++	test_when_finished "git am --abort" &&
    ++	write_nul_patch body >body.patch &&
    ++	test_must_fail git am body.patch 2>err &&
    ++	grep "a NUL byte in commit log message not allowed" err
    ++'
    ++
    ++test_expect_failure "NUL in commit message's header" '
    ++	test_when_finished "git am --abort" &&
    ++	write_nul_patch subject >subject.patch &&
    ++	test_must_fail git am subject.patch 2>err &&
    ++	grep "a NUL byte in Subject is not allowed" err
    ++'
    ++
    + test_done
-:  ---------- > 3:  cb96947b36 mailinfo: disallow NUL character in mail's header