ci(github): restore "print test failures" step name

Commit Message

Phillip Wood Jan. 3, 2023, 10:27 a.m. UTC

From: Phillip Wood <[e-mail]>

As well as removing the explicit shell setting d8b21a0fe2 (CI: don't
explicitly pick "bash" shell outside of Windows, fix regression,
2022-12-07) also reverted the name of the print test failures step
introduced by 5aeb145780f (ci(github): bring back the 'print test
failures' step, 2022-06-08). This is unfortunate as 5aeb145780f added a
message to direct contributors to the "print test failures" step when a
test fails and that step is no-longer known by that name on the
non-windows ci jobs.

In principle we could update the message to print the correct name for
the step but then we'd have to deal with having two different names for
the same step on different jobs. It is simpler for the implementation
and contributors to use the same name for this step on all jobs.

Signed-off-by: Phillip Wood <[e-mail]>
---
    ci(github): restore "print test failures" step name
    
    Ævar seems to think the name change in 5aeb145780f was unintentional [1]
    but looking at the original commit I don't think that's the case.
    
    [1]
    https://lore.kernel.org/git/[e-mail]/

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1453%2Fphillipwood%2Fci-print-test-failures-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1453/phillipwood/ci-print-test-failures-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1453

 .github/workflows/main.yml | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)


base-commit: 8a4e8f6a67e7fc97048d4666eec38399b88e0e3b

Comments

Ævar Arnfjörð Bjarmason Jan. 4, 2023, 9:41 p.m. UTC | #1

On Tue, Jan 03 2023, Phillip Wood via GitGitGadget wrote:

> From: Phillip Wood <[e-mail]>
>
> As well as removing the explicit shell setting d8b21a0fe2 (CI: don't
> explicitly pick "bash" shell outside of Windows, fix regression,
> 2022-12-07) also reverted the name of the print test failures step
> introduced by 5aeb145780f (ci(github): bring back the 'print test
> failures' step, 2022-06-08). This is unfortunate as 5aeb145780f added a
> message to direct contributors to the "print test failures" step when a
> test fails and that step is no-longer known by that name on the
> non-windows ci jobs.
>
> In principle we could update the message to print the correct name for
> the step but then we'd have to deal with having two different names for
> the same step on different jobs. It is simpler for the implementation
> and contributors to use the same name for this step on all jobs.
>
> Signed-off-by: Phillip Wood <[e-mail]>
> ---
>     ci(github): restore "print test failures" step name
>     
>     Ævar seems to think the name change in 5aeb145780f was unintentional [1]
>     but looking at the original commit I don't think that's the case.
>     
>     [1]
>     https://lore.kernel.org/git/[e-mail]/

Reading it again I think you're right, i.e. that the migration of
everything to "bash" was unintentional & a result of copy/pasting, but
the initial goal was to mention the "print test failures" step....

> diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
> index e67847a682c..126eac8239d 100644
> --- a/.github/workflows/main.yml
> @@ -265,8 +265,9 @@ jobs:
>      - uses: actions/checkout@v3
>      - run: ci/install-dependencies.sh
>      - run: ci/run-build-and-tests.sh
> -    - run: ci/print-test-failures.sh
> +    - name: print test failures
>        if: failure() && env.FAILED_TEST_ARTIFACTS != ''
> +      run: ci/print-test-failures.sh
>      - name: Upload failed tests' directories
>        if: failure() && env.FAILED_TEST_ARTIFACTS != ''
>        uses: actions/upload-artifact@v3
> @@ -298,8 +299,9 @@ jobs:
>        if: matrix.vector.jobname == 'linux32'
>      - run: ci/install-docker-dependencies.sh
>      - run: ci/run-build-and-tests.sh
> -    - run: ci/print-test-failures.sh
> +    - name: print test failures
>        if: failure() && env.FAILED_TEST_ARTIFACTS != ''
> +      run: ci/print-test-failures.sh
>      - name: Upload failed tests' directories
>        if: failure() && env.FAILED_TEST_ARTIFACTS != '' && matrix.vector.jobname != 'linux32'
>        uses: actions/upload-artifact@v3
>
> base-commit: 8a4e8f6a67e7fc97048d4666eec38399b88e0e3b

...but as far as moving forward, why make every other job be
inconsistent in naming this one step, rather than just making the
Windows ones consistent with rest?

I.e. why not:

	diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
	index e67847a682c..2bfb841206e 100644
	--- a/.github/workflows/main.yml
	+++ b/.github/workflows/main.yml
	@@ -119,8 +119,7 @@ jobs:
	     - name: test
	       shell: bash
	       run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10
	-    - name: print test failures
	-      if: failure() && env.FAILED_TEST_ARTIFACTS != ''
	+    - if: failure() && env.FAILED_TEST_ARTIFACTS != ''
	       shell: bash
	       run: ci/print-test-failures.sh
	     - name: Upload failed tests' directories
	diff --git a/ci/lib.sh b/ci/lib.sh
	index db7105e8a8d..d6450fd4957 100755
	--- a/ci/lib.sh
	+++ b/ci/lib.sh
	@@ -183,7 +183,7 @@ then
	 			test_name="${test_exit%.exit}"
	 			test_name="${test_name##*/}"
	 			printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
	-			echo "The full logs are in the 'print test failures' step below."
	+			echo "The full logs are in the 'ci/print-test-failures.sh' step below."
	 			echo "See also the 'failed-tests-*' artifacts attached to this run."
	 			cat "t/test-results/$test_name.markup"
	 

The implicit argument being made here & in the original 5aeb145780f is
that it's a good thing to give explicit "names"'s to all of these steps.

For some of the steps I agree, e.g. "test" is probably better than
". /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10" or whatever,
and e.g. "generate Visual Studio solution" is definitely better than
some multi-line shellscript (I'm not sure how the UX would even turn
that into a collapsible title).

But in the case of "ci/print-test-failures.sh",
"ci/run-build-and-tests.sh", "ci/install-dependencies.sh" etc. the
script name is already self-descriptive, so giving it a name just serves
to obscure the connection between the step & the script implementing it,
which is otherwise immediately apparent.

So I think it's better to just remove the "name" label in those cases
where the Windows jobs can use such a script, which we're already not
naming in the *nix jobs.

It's a minor point either way, and your change would also be an
improvement. I just think it's better to aim for e.g. this snippet (from
the current main.yml):

    - run: ci/install-docker-dependencies.sh
    - run: ci/run-build-and-tests.sh
    - run: ci/print-test-failures.sh

Rather than providing "name" fields for them all, and doubling the line
count for no apparent (at least to me) reason.

Patch

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index e67847a682c..126eac8239d 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -265,8 +265,9 @@  jobs:
     - uses: actions/checkout@v3
     - run: ci/install-dependencies.sh
     - run: ci/run-build-and-tests.sh
-    - run: ci/print-test-failures.sh
+    - name: print test failures
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
+      run: ci/print-test-failures.sh
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
       uses: actions/upload-artifact@v3
@@ -298,8 +299,9 @@  jobs:
       if: matrix.vector.jobname == 'linux32'
     - run: ci/install-docker-dependencies.sh
     - run: ci/run-build-and-tests.sh
-    - run: ci/print-test-failures.sh
+    - name: print test failures
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
+      run: ci/print-test-failures.sh
     - name: Upload failed tests' directories
       if: failure() && env.FAILED_TEST_ARTIFACTS != '' && matrix.vector.jobname != 'linux32'
       uses: actions/upload-artifact@v3