diff mbox series

[1/2] t5534: new test case for atomic signed push

Message ID 20200915095827.52047-1-hanxin.hx@alibaba-inc.com
State New
Headers show
Series [1/2] t5534: new test case for atomic signed push | expand

Commit Message

Han Xin Sept. 15, 2020, 9:58 a.m. UTC
In order to test signed atomic push, add a new test case.

Reviewed-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Han Xin <hanxin.hx@alibaba-inc.com>
---
 t/t5534-push-signed.sh | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

Comments

Junio C Hamano Sept. 15, 2020, 8:31 p.m. UTC | #1
Han Xin <chiyutianyi@gmail.com> writes:

> In order to test signed atomic push, add a new test case.
>
> Reviewed-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
> Signed-off-by: Han Xin <hanxin.hx@alibaba-inc.com>
> ---

Thanks, but nowhere in the above it does not say what is being
tested.  By looking at 2/2 (by the way, these should be a single
atomic patch, not a "failure turns into success", as it is not even
a bug fix), readers may be able to guess that you want to enforce
that with even broken implementation of GPG, an immediate failure to
push one of the refs will be noticed by looking at their refs, but
it is unclear why that is even desirable---if you combine the two
patches, you may have a better place to argue why it is a good idea,
but a test-only patch makes it even less clear why the new behavior
expected by this test is desirable.

>  t/t5534-push-signed.sh | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
>
> diff --git a/t/t5534-push-signed.sh b/t/t5534-push-signed.sh
> index 030331f1c5..d0fcdc900e 100755
> --- a/t/t5534-push-signed.sh
> +++ b/t/t5534-push-signed.sh
> @@ -273,4 +273,21 @@ test_expect_success GPGSM 'fail without key and heed user.signingkey x509' '
>  	test_cmp expect dst/push-cert-status
>  '
>  
> +test_expect_failure GPG 'check atomic push before running GPG' '
> +	prepare_dst &&
> +	git -C dst config receive.certnonceseed sekrit &&
> +	write_script gpg <<-EOF &&
> +	echo >&2 "Fake gpg is called."
> +	exit 1
> +	EOF
> +	test_must_fail env PATH="$TRASH_DIRECTORY:$PATH" git push --signed --atomic \
> +			dst noop ff noff >out 2>&1 &&
> +	grep "^error:" out >actual &&
> +	cat >expect <<-EOF &&
> +	error: atomic push failed for ref refs/heads/noff. status: 2
> +	error: failed to push some refs to '"'"'dst'"'"'
> +	EOF
> +	test_i18ncmp expect actual
> +'
> +
>  test_done
Junio C Hamano Sept. 15, 2020, 8:34 p.m. UTC | #2
Han Xin <chiyutianyi@gmail.com> writes:

> +	cat >expect <<-EOF &&
> +	error: atomic push failed for ref refs/heads/noff. status: 2
> +	error: failed to push some refs to '"'"'dst'"'"'
> +	EOF
> +	test_i18ncmp expect actual

Another thing I forgot to say.  This expects the exact phrasing of
error message to stay the same, which is not really desirable.  We
might want to start quoting `refs/heads/noff` in the message like
other messages often do, for example, and this test will have to
match.

If you expect that the failure is not due to GPG (i.e. the updated
code in 2/2 wants to fail before asking GPG to do anything), why
not grep for what your "Fake gpg" says in the error output and make
sure that message does not appear?  That would make this test a lot
more robust, I suspect.



> +'
> +
>  test_done
brian m. carlson Sept. 16, 2020, 12:34 a.m. UTC | #3
On 2020-09-15 at 20:31:38, Junio C Hamano wrote:
> Han Xin <chiyutianyi@gmail.com> writes:
> 
> > In order to test signed atomic push, add a new test case.
> >
> > Reviewed-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
> > Signed-off-by: Han Xin <hanxin.hx@alibaba-inc.com>
> > ---
> 
> Thanks, but nowhere in the above it does not say what is being
> tested.  By looking at 2/2 (by the way, these should be a single
> atomic patch, not a "failure turns into success", as it is not even
> a bug fix), readers may be able to guess that you want to enforce
> that with even broken implementation of GPG, an immediate failure to
> push one of the refs will be noticed by looking at their refs, but
> it is unclear why that is even desirable---if you combine the two
> patches, you may have a better place to argue why it is a good idea,
> but a test-only patch makes it even less clear why the new behavior
> expected by this test is desirable.

Yeah, I find myself a little confused by this, and I think maybe a more
verbose commit message could be valuable in clearing that up.  I think
what this series is trying to do is check that if we can tell on the
client side that the push will be rejected, then not to invoke GnuPG to
generate the push certificate.

If so, that would be a nice change; after all, the user's key may
involve a smartcard or a passphrase and avoiding needless hassle for the
user would be desirable.  But even after reading the series, it's not
clear to me that that _is_ what the goal is here or that this is
necessarily the best way of going about it.  Telling us more about the
reason for the patch would help us understand the change and why it's
valuable better.
diff mbox series

Patch

diff --git a/t/t5534-push-signed.sh b/t/t5534-push-signed.sh
index 030331f1c5..d0fcdc900e 100755
--- a/t/t5534-push-signed.sh
+++ b/t/t5534-push-signed.sh
@@ -273,4 +273,21 @@  test_expect_success GPGSM 'fail without key and heed user.signingkey x509' '
 	test_cmp expect dst/push-cert-status
 '
 
+test_expect_failure GPG 'check atomic push before running GPG' '
+	prepare_dst &&
+	git -C dst config receive.certnonceseed sekrit &&
+	write_script gpg <<-EOF &&
+	echo >&2 "Fake gpg is called."
+	exit 1
+	EOF
+	test_must_fail env PATH="$TRASH_DIRECTORY:$PATH" git push --signed --atomic \
+			dst noop ff noff >out 2>&1 &&
+	grep "^error:" out >actual &&
+	cat >expect <<-EOF &&
+	error: atomic push failed for ref refs/heads/noff. status: 2
+	error: failed to push some refs to '"'"'dst'"'"'
+	EOF
+	test_i18ncmp expect actual
+'
+
 test_done