diff mbox

[i-g-t,2/5] lib/tests: don't use hard error status to indicate test failure

Message ID 1415014265-858-2-git-send-email-thomas.wood@intel.com (mailing list archive)
State New, archived
Headers show

Commit Message

Thomas Wood Nov. 3, 2014, 11:31 a.m. UTC
Signed-off-by: Thomas Wood <thomas.wood@intel.com>
---
 lib/tests/igt_command_line.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

Comments

Daniel Vetter Nov. 3, 2014, 3:02 p.m. UTC | #1
On Mon, Nov 03, 2014 at 11:31:02AM +0000, Thomas Wood wrote:
> Signed-off-by: Thomas Wood <thomas.wood@intel.com>
> ---
>  lib/tests/igt_command_line.sh | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/lib/tests/igt_command_line.sh b/lib/tests/igt_command_line.sh
> index 7e6ca67..5cf2584 100755
> --- a/lib/tests/igt_command_line.sh
> +++ b/lib/tests/igt_command_line.sh
> @@ -48,20 +48,20 @@ for test in $TESTLIST; do
>  
>  	# check invalid option handling
>  	echo "  Checking invalid option handling..."
> -	./$test --invalid-option 2> /dev/null && exit 99
> +	./$test --invalid-option 2> /dev/null && exit 1

Just a curious question: What's better with hardcoding 1 than hardcoding
99? Otherwise series lgtm.
-Daniel
Thomas Wood Nov. 3, 2014, 3:42 p.m. UTC | #2
On 3 November 2014 15:02, Daniel Vetter <daniel@ffwll.ch> wrote:
> On Mon, Nov 03, 2014 at 11:31:02AM +0000, Thomas Wood wrote:
>> Signed-off-by: Thomas Wood <thomas.wood@intel.com>
>> ---
>>  lib/tests/igt_command_line.sh | 8 ++++----
>>  1 file changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/lib/tests/igt_command_line.sh b/lib/tests/igt_command_line.sh
>> index 7e6ca67..5cf2584 100755
>> --- a/lib/tests/igt_command_line.sh
>> +++ b/lib/tests/igt_command_line.sh
>> @@ -48,20 +48,20 @@ for test in $TESTLIST; do
>>
>>       # check invalid option handling
>>       echo "  Checking invalid option handling..."
>> -     ./$test --invalid-option 2> /dev/null && exit 99
>> +     ./$test --invalid-option 2> /dev/null && exit 1
>
> Just a curious question: What's better with hardcoding 1 than hardcoding
> 99? Otherwise series lgtm.

From the automake manual:

"When no test protocol is in use, an exit status of 0 from a test
script will denote a success, an exit status of 77 a skipped test, an
exit status of 99 an hard error, and any other exit status will denote
a failure."

So, an exit status of 99 is reported separately in the summary as an
error, rather than as a test failure.


> -Daniel
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> +41 (0) 79 365 57 48 - http://blog.ffwll.ch
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Daniel Vetter Nov. 3, 2014, 4:44 p.m. UTC | #3
On Mon, Nov 03, 2014 at 03:42:06PM +0000, Thomas Wood wrote:
> On 3 November 2014 15:02, Daniel Vetter <daniel@ffwll.ch> wrote:
> > On Mon, Nov 03, 2014 at 11:31:02AM +0000, Thomas Wood wrote:
> >> Signed-off-by: Thomas Wood <thomas.wood@intel.com>
> >> ---
> >>  lib/tests/igt_command_line.sh | 8 ++++----
> >>  1 file changed, 4 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/lib/tests/igt_command_line.sh b/lib/tests/igt_command_line.sh
> >> index 7e6ca67..5cf2584 100755
> >> --- a/lib/tests/igt_command_line.sh
> >> +++ b/lib/tests/igt_command_line.sh
> >> @@ -48,20 +48,20 @@ for test in $TESTLIST; do
> >>
> >>       # check invalid option handling
> >>       echo "  Checking invalid option handling..."
> >> -     ./$test --invalid-option 2> /dev/null && exit 99
> >> +     ./$test --invalid-option 2> /dev/null && exit 1
> >
> > Just a curious question: What's better with hardcoding 1 than hardcoding
> > 99? Otherwise series lgtm.
> 
> From the automake manual:
> 
> "When no test protocol is in use, an exit status of 0 from a test
> script will denote a success, an exit status of 77 a skipped test, an
> exit status of 99 an hard error, and any other exit status will denote
> a failure."
> 
> So, an exit status of 99 is reported separately in the summary as an
> error, rather than as a test failure.

Ah, makes sense. Can you please add this bit of important information to
the commit message before pushing?

Thanks, Daniel
diff mbox

Patch

diff --git a/lib/tests/igt_command_line.sh b/lib/tests/igt_command_line.sh
index 7e6ca67..5cf2584 100755
--- a/lib/tests/igt_command_line.sh
+++ b/lib/tests/igt_command_line.sh
@@ -48,20 +48,20 @@  for test in $TESTLIST; do
 
 	# check invalid option handling
 	echo "  Checking invalid option handling..."
-	./$test --invalid-option 2> /dev/null && exit 99
+	./$test --invalid-option 2> /dev/null && exit 1
 
 	# check valid options succeed
 	echo "  Checking valid option handling..."
-	./$test --help > /dev/null || exit 99
+	./$test --help > /dev/null || exit 1
 
 	# check --list-subtests works correctly
 	echo "  Checking subtest enumeration..."
 	./$test --list-subtests > /dev/null
 	if [ $? -ne 0 -a $? -ne 79 ]; then
-		exit 99
+		exit 1
 	fi
 
 	# check invalid subtest handling
 	echo "  Checking invalid subtest handling..."
-	./$test --run-subtest invalid-subtest > /dev/null 2>&1 && exit 99
+	./$test --run-subtest invalid-subtest > /dev/null 2>&1 && exit 1
 done