| Message ID | 20190225200549.41654-1-raspl@linux.ibm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [kvm-unit-tests,v1] tap13: make testcase names unique | expand |
On 25/02/19 21:05, Stefan Raspl wrote: > Previously, we were using the full output of each test for the name, e.g. > > ok 104 - intercept: stpx: Program interrupt: expected(4) == received(4) > > But that's not a stable testcase name: The "received(4)" might change in case of > an error. Therefore, we truncate everything after the "expected()". > Helps in CI environments when comparing with previous output. > > Signed-off-by: Stefan Raspl <raspl@linux.ibm.com> Hmm, I'd rather fix the testcases... Paolo
On 26.02.19 13:50, Paolo Bonzini wrote: > On 25/02/19 21:05, Stefan Raspl wrote: >> Previously, we were using the full output of each test for the name, e.g. >> >> ok 104 - intercept: stpx: Program interrupt: expected(4) == received(4) >> >> But that's not a stable testcase name: The "received(4)" might change in case of >> an error. Therefore, we truncate everything after the "expected()". >> Helps in CI environments when comparing with previous output. >> >> Signed-off-by: Stefan Raspl <raspl@linux.ibm.com> > > Hmm, I'd rather fix the testcases... No doubt about that. But to coordinate that in an efficient manner, being able to identify testcases in a unique manner helps, especially when considering automation! Ciao, Stefan
diff --git a/run_tests.sh b/run_tests.sh index 01e36dc..baf8c70 100755 --- a/run_tests.sh +++ b/run_tests.sh @@ -83,7 +83,7 @@ else process_test_output() { CR=$'\r' while read -r line; do - line="${line%$CR}" + line="${line% ==*}" case "${line:0:4}" in PASS) echo "ok TEST_NUMBER - ${line#??????}" >&3
Previously, we were using the full output of each test for the name, e.g. ok 104 - intercept: stpx: Program interrupt: expected(4) == received(4) But that's not a stable testcase name: The "received(4)" might change in case of an error. Therefore, we truncate everything after the "expected()". Helps in CI environments when comparing with previous output. Signed-off-by: Stefan Raspl <raspl@linux.ibm.com> --- run_tests.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)