[v2,0/2] test-lib: improve missing prereq handling

Message

Fabian Stelzer Nov. 17, 2021, 9:04 a.m. UTC

The ssh signing feature was breaking tests when the broken openssh-8.7
was used. We have now fixed that by checking for this exact case in the
GPGSSH prereq and I will improve that check further in a future patch.
However we are now in a situation where a broken openssh in the future
will result in successfull tests but not a working git build afterwards
(either not compiling in the expected feature or like in the ssh case
runtime failures) resulting in a false sense of security in the tests.
This patches try to improve this situation by showing which prereqs
failed in the test summary and by adding an environment variable to
enforce certain prereqs to succeed or abort the test otherwise.

See also:
https://public-inbox.org/git/xmqqv916wh7t.fsf@gitster.g/

changes since v1:
 - use \012 instead of \n for possible portability reasons
 - fix typo in commit msg

Fabian Stelzer (2):
  test-lib: show missing prereq summary
  test-lib: introduce required prereq for test runs

 t/README                |  6 ++++++
 t/aggregate-results.sh  | 17 +++++++++++++++++
 t/test-lib-functions.sh | 11 +++++++++++
 t/test-lib.sh           | 11 +++++++++++
 4 files changed, 45 insertions(+)

Range-diff against v1:
1:  69e77cd854 ! 1:  775c0e5ef0 test-lib: show missing prereq summary
    @@ t/aggregate-results.sh: do
     +then
     +	unique_missing_prereq=$(
     +		echo $missing_prereq |
    -+		tr -s "," "\n" |
    ++		tr -s "," "\012" |
     +		grep -v '^$' |
     +		sort -u |
     +		paste -s -d ',')
2:  12bd18c5ce ! 2:  eb1bbb8d01 test-lib: introduce required prereq for test runs
    @@ Commit message
         test-lib: introduce required prereq for test runs
     
         In certain environments or for specific test scenarios we might expect a
    -    specific prerequisite check to be succeed. Therefore we would like to
    +    specific prerequisite check to succeed. Therefore we would like to
         trigger an error when running our tests if this is not the case.
     
         To remedy this we add the environment variable GIT_TEST_REQUIRE_PREREQ