| Message ID | 20161110144520.24965-2-luc.vanoostenryck@gmail.com (mailing list archive) |
|---|---|
| State | Mainlined, archived |
| Headers | show |
Applied. Chris On Thu, Nov 10, 2016 at 10:45 PM, Luc Van Oostenryck <luc.vanoostenryck@gmail.com> wrote: > Currently the test suite always check the exit value and the output > of the command used for the test. This is fine and allow use to catch > the most common situations: > - failure or crash (via the exit value) > - (un)expected output (like when testing the result of the preprocessor) > - (un)expected errors & warnings (like when testing sparse's warnings) > > But sometimes, we're not interested in the output or the output (as is) > is simply not meaningful for the test or can't be compared textually > to some reference. > > This patch add two new tags (check-output-ignore & check-error-ignore), > telling to test suite to ignore the content of stdout or stderr when > testing this file. > > Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com> > --- > Documentation/test-suite | 6 ++++++ > validation/test-suite | 2 ++ > 2 files changed, 8 insertions(+) > > diff --git a/Documentation/test-suite b/Documentation/test-suite > index 6c4f24f6..6936feeb 100644 > --- a/Documentation/test-suite > +++ b/Documentation/test-suite > @@ -29,6 +29,12 @@ check-output-start / check-output-end (optional) > The expected output (stdout and stderr) of check-command lies between > those two tags. It defaults to no output. > > +check-output-ignore / check-error-ignore (optional) > + Don't check the expected output (stdout or stderr) of check-command > + (usefull when this output is not comparable or if you're only interested > + in the exit value). > + By default this check is done. > + > check-known-to-fail (optional) > Mark the test as being known to fail. > > diff --git a/validation/test-suite b/validation/test-suite > index df5a7c60..0d874e07 100755 > --- a/validation/test-suite > +++ b/validation/test-suite > @@ -146,6 +146,8 @@ do_test() > actual_exit_value=$? > > for stream in output error; do > + grep -s -q "check-$stream-ignore" $file && continue > + > diff -u "$file".$stream.expected "$file".$stream.got > "$file".$stream.diff > if [ "$?" -ne "0" ]; then > error "actual $stream text does not match expected $stream text." > -- > 2.10.1 > -- To unsubscribe from this list: send the line "unsubscribe linux-sparse" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/Documentation/test-suite b/Documentation/test-suite index 6c4f24f6..6936feeb 100644 --- a/Documentation/test-suite +++ b/Documentation/test-suite @@ -29,6 +29,12 @@ check-output-start / check-output-end (optional) The expected output (stdout and stderr) of check-command lies between those two tags. It defaults to no output. +check-output-ignore / check-error-ignore (optional) + Don't check the expected output (stdout or stderr) of check-command + (usefull when this output is not comparable or if you're only interested + in the exit value). + By default this check is done. + check-known-to-fail (optional) Mark the test as being known to fail. diff --git a/validation/test-suite b/validation/test-suite index df5a7c60..0d874e07 100755 --- a/validation/test-suite +++ b/validation/test-suite @@ -146,6 +146,8 @@ do_test() actual_exit_value=$? for stream in output error; do + grep -s -q "check-$stream-ignore" $file && continue + diff -u "$file".$stream.expected "$file".$stream.got > "$file".$stream.diff if [ "$?" -ne "0" ]; then error "actual $stream text does not match expected $stream text."
Currently the test suite always check the exit value and the output of the command used for the test. This is fine and allow use to catch the most common situations: - failure or crash (via the exit value) - (un)expected output (like when testing the result of the preprocessor) - (un)expected errors & warnings (like when testing sparse's warnings) But sometimes, we're not interested in the output or the output (as is) is simply not meaningful for the test or can't be compared textually to some reference. This patch add two new tags (check-output-ignore & check-error-ignore), telling to test suite to ignore the content of stdout or stderr when testing this file. Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com> --- Documentation/test-suite | 6 ++++++ validation/test-suite | 2 ++ 2 files changed, 8 insertions(+)