[RFC,v2,0/6] check: add option to rerun failed tests

Message

David Disseldorp June 27, 2022, 10:22 p.m. UTC

This RFC patchset adds support to loop on failed tests, as proposed by
Ted Ts'o in https://lwn.net/Articles/897061/:
  add a mode that will immediately rerun a failed test 25 or 100 times
  to establish a failure percentage.

Changes since previous version (v1):
- rebased atop upstream v2022.06.26
- added a few extra cleanup commits
- append details for every rerun to xunit output
  + provide aggregate stats via <testcase status=X>
- extend _stash_test_status to call report hook, as well as save failure
  artifacts with a .rerun# suffix

Still flagged as RFC as I'd like to do a bit more testing and work out
whether the regular / xunit reporting output makes sense. Feedback
appreciated.

Diffstat:
 check         | 146 ++++++++++++++++++++++++++++++++++++++++++--------
 common/report |  96 ++++++++++++++++-----------------
 2 files changed, 170 insertions(+), 72 deletions(-)

Cheers, David