| Message ID | 20250407111824.46518-2-anthonywang03@icloud.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | t9811: Improve test coverage and clarity | expand |
On Mon, Apr 7, 2025 at 7:18 AM Anthony Wang <anthonywang513@gmail.com> wrote: > The exit code of the upstream in a pipe is suppressed > thus we lose any exit codes of git commands that are piped. In order to > ensure we pick up the exit code, we can write the output of the git command > to a file, testing the exit codes of both the commands. > --- Missing sign-off. > diff --git a/t/t9811-git-p4-label-import.sh b/t/t9811-git-p4-label-import.sh > @@ -95,9 +95,10 @@ test_expect_success 'two labels on the same changelist' ' > - git tag | grep TAG_F1 && > - git tag | grep -q TAG_F1_1 && > - git tag | grep -q TAG_F1_2 && > + git tag >output && > + grep TAG_F1 output && > + grep -q TAG_F1_1 output && > + grep -q TAG_F1_2 output && Since process creation is so expensive on Microsoft Windows, folks on that platform should also appreciate that this eliminates two git-tag invocations. Nice.
Anthony Wang <anthonywang513@gmail.com> writes: > The exit code of the upstream in a pipe is suppressed > thus we lose any exit codes of git commands that are piped. In order to > ensure we pick up the exit code, we can write the output of the git command > to a file, testing the exit codes of both the commands. Sort of correct, but ... > --- > t/t9811-git-p4-label-import.sh | 10 ++++++---- > 1 file changed, 6 insertions(+), 4 deletions(-) Missing sign-off. > diff --git a/t/t9811-git-p4-label-import.sh b/t/t9811-git-p4-label-import.sh > index 5ac5383fb7..5abac938d0 100755 > --- a/t/t9811-git-p4-label-import.sh > +++ b/t/t9811-git-p4-label-import.sh > @@ -95,9 +95,10 @@ test_expect_success 'two labels on the same changelist' ' > cd "$git" && > git p4 sync --import-labels && > > - git tag | grep TAG_F1 && > - git tag | grep -q TAG_F1_1 && > - git tag | grep -q TAG_F1_2 && > + git tag >output && > + grep TAG_F1 output && > + grep -q TAG_F1_1 output && > + grep -q TAG_F1_2 output && Think what these tests are trying to do. After a "git p4 sync" operation, they want to ensure that tags TAG_F1_1 and TAG_F1_2 exist? Does the test want to see a tag "TAG_F1", or is it only that the test is written in a so sloppy way that grepping for TAG_F1 will be happy when any one of TAG_F1_1, TAG_F1_2 and TAG_F1_ONLY exists, making its purpose of verifying that the tags are in the expected state pretty much useless, and that is the reason why it needs to be followed up with the two extra tests? What is the desired state you want to ensure after "git p4 sync" operation above? I do not do "git p4", but you may know better than I do, as you are the one who is patching this test file ;-) I am guessing that you want TAG_F1_1 and TAG_F1_2 to exist and you do not want TAG_F1_ONLY to exist? If so, instead of grepping around, we should be testing that in a more direct way, perhaps with something like git show-ref --verify refs/tags/TAG_F1_1 && git show-ref --verify refs/tags/TAG_F1_2 && test_must_fail git show-ref --verify refs/tags/TAG_F1_ONLY && no? > cd main && > > @@ -208,7 +209,8 @@ test_expect_success 'use git config to enable import/export of tags' ' > git p4 rebase --verbose && > git p4 submit --verbose && > git tag && > - git tag | grep TAG_F1_1 > + git tag >output && > + grep TAG_F1_1 output > ) && > ( > cd "$cli" &&
From: Anthony Wang <anthonywang513@gmail.com> On Mon, Apr 7, 2025 at 7:20 PM Junio C Hamano <gitster@pobox.com> wrote: > Anthony Wang <anthonywang513@gmail.com> writes: > > > The exit code of the upstream in a pipe is suppressed > > thus we lose any exit codes of git commands that are piped. In order to > > ensure we pick up the exit code, we can write the output of the git command > > to a file, testing the exit codes of both the commands. > > Sort of correct, but ... Would it be more correct to say that the shell only returns the exit code of the last command in the pipeline? Then, by writing out the output of the git command to a file, we can test the exit codes of both the commands? If not, I am not quite sure what the additional nuance is in this case. > > > --- > > t/t9811-git-p4-label-import.sh | 10 ++++++---- > > 1 file changed, 6 insertions(+), 4 deletions(-) > > Missing sign-off. > > > diff --git a/t/t9811-git-p4-label-import.sh b/t/t9811-git-p4-label-import.sh > > index 5ac5383fb7..5abac938d0 100755 > > --- a/t/t9811-git-p4-label-import.sh > > +++ b/t/t9811-git-p4-label-import.sh > > @@ -95,9 +95,10 @@ test_expect_success 'two labels on the same changelist' ' > > cd "$git" && > > git p4 sync --import-labels && > > > > - git tag | grep TAG_F1 && > > - git tag | grep -q TAG_F1_1 && > > - git tag | grep -q TAG_F1_2 && > > + git tag >output && > > + grep TAG_F1 output && > > + grep -q TAG_F1_1 output && > > + grep -q TAG_F1_2 output && > > Think what these tests are trying to do. After a "git p4 sync" > operation, they want to ensure that tags TAG_F1_1 and TAG_F1_2 > exist? Does the test want to see a tag "TAG_F1", or is it only that > the test is written in a so sloppy way that grepping for TAG_F1 will > be happy when any one of TAG_F1_1, TAG_F1_2 and TAG_F1_ONLY exists, > making its purpose of verifying that the tags are in the expected > state pretty much useless, and that is the reason why it needs to be > followed up with the two extra tests? I see, it seems I didn't fully consider the goals of the test. It does indeed seem like the test has some redundancies. If I am understanding correctly, the test grepping for TAG_F1 is not even necessary, and it seems like the tests for TAG_F1_1, TAG_F1_2 were added afterwards to ensure that grep TAG_F1 was not passing when TAG_F1_ONLY existed. If this is the case, would it be better to just remove the grep TAG_F1 test and just keep the two tests for TAG_F1_1 and TAG_F1_2? This would simplify the test and make it clearer that we are only interested in the existence of those two tags. > > What is the desired state you want to ensure after "git p4 sync" > operation above? I do not do "git p4", but you may know better than > I do, as you are the one who is patching this test file ;-) I am > guessing that you want TAG_F1_1 and TAG_F1_2 to exist and you do not > want TAG_F1_ONLY to exist? > > If so, instead of grepping around, we should be testing that in a > more direct way, perhaps with something like > > git show-ref --verify refs/tags/TAG_F1_1 && > git show-ref --verify refs/tags/TAG_F1_2 && > test_must_fail git show-ref --verify refs/tags/TAG_F1_ONLY && > > no? > Possibly, but I believe adding the test_must_fail check would be modifying the original intent of the test, as it would pass even with the existence of TAG_F1_ONLY. However, if we are only performing actions to cause TAG_F1_1 and TAG_F1_2 to exist, then it would be an issue if TAG_F1_ONLY existed. My original plan was not to change any test coverage, and to only make small changes following the goals of the microproject, but if you think it would be beneficial to change the test to check for the absence of TAG_F1_ONLY, then I will proceed. > > cd main && > > > > @@ -208,7 +209,8 @@ test_expect_success 'use git config to enable import/export of tags' ' > > git p4 rebase --verbose && > > git p4 submit --verbose && > > git tag && > > - git tag | grep TAG_F1_1 > > + git tag >output && > > + grep TAG_F1_1 output > > ) && > > ( > > cd "$cli" &&
Anthony Wang <anthonywang513@gmail.com> writes: >> If so, instead of grepping around, we should be testing that in a >> more direct way, perhaps with something like >> >> git show-ref --verify refs/tags/TAG_F1_1 && >> git show-ref --verify refs/tags/TAG_F1_2 && >> test_must_fail git show-ref --verify refs/tags/TAG_F1_ONLY && >> >> no? >> > > Possibly, but I believe adding the test_must_fail check would be modifying > the original intent of the test, as it would pass even with the existence of > TAG_F1_ONLY. However, if we are only performing actions to cause TAG_F1_1 > and TAG_F1_2 to exist, then it would be an issue if TAG_F1_ONLY existed. I view it a bit differently. Use of "grep" over the output of "git tag" is simply a sloppy programming. If the test wanted to verify "TAG_F1_1 exists", it shouldn't have grepped for TAG_F1_1, because another tag T_TAG_F1_1 would produce a false positive hit if the earlier test gets updated. Similarly, not verifying what should not exist is being sloppy. People who come up with a new feature (in this case, "git p4 sync" involving tags) tend to test positive effects to show how their shiny new toy does things, and forgets to test lack of effects to ensure that their shiny new toy does *not* do what they should not do. If the original test were written solidly and use of pipe hiding exit code were the only problem it had, I would agree that making minimum change should be preferrable, but the original test seems to be so sloppy in this case. Thanks.
diff --git a/t/t9811-git-p4-label-import.sh b/t/t9811-git-p4-label-import.sh index 5ac5383fb7..5abac938d0 100755 --- a/t/t9811-git-p4-label-import.sh +++ b/t/t9811-git-p4-label-import.sh @@ -95,9 +95,10 @@ test_expect_success 'two labels on the same changelist' ' cd "$git" && git p4 sync --import-labels && - git tag | grep TAG_F1 && - git tag | grep -q TAG_F1_1 && - git tag | grep -q TAG_F1_2 && + git tag >output && + grep TAG_F1 output && + grep -q TAG_F1_1 output && + grep -q TAG_F1_2 output && cd main && @@ -208,7 +209,8 @@ test_expect_success 'use git config to enable import/export of tags' ' git p4 rebase --verbose && git p4 submit --verbose && git tag && - git tag | grep TAG_F1_1 + git tag >output && + grep TAG_F1_1 output ) && ( cd "$cli" &&