| Message ID | 20220708085142.20991-1-ddiss@suse.de (mailing list archive) |
|---|---|
| Headers | show |
| Series | check: add option to rerun failed tests | expand |
On Fri, Jul 08, 2022 at 10:51:37AM +0200, David Disseldorp wrote: > This patchset adds support to loop on failed tests, as proposed by > Ted Ts'o in https://lwn.net/Articles/897061/: > add a mode that will immediately rerun a failed test 25 or 100 times > to establish a failure percentage. This patchset is good to me. if no more objections, I think it's time to merge this feature. I'll give it a testing, then try to push it this weekend. Reviewed-by: Zorro Lang <zlang@redhat.com> Thanks, Zorro > > Changes since (v3), following Darrick's review: > - extended full/out.bad/etc. retention to cover more paths and remove > stale .rerun# files > > Changes since (v2), following Darrick's review: > - dropped RFC flag > - rebased atop v2022.07.03 > - simplified test iterator > + results stashed at the end of each iteration, rather than start of > next / loop-exit > - dropped aggregate loop stats message from xunit report > - squashed full/dmesg/out.bad file retention patch with rerun patch > > Changes since (v1): > - rebased atop upstream v2022.06.26 > - added a few extra cleanup commits > - append details for every rerun to xunit output > + provide aggregate stats via <testcase status=X> > - extend _stash_test_status to call report hook, as well as save failure > artifacts with a .rerun# suffix > > Diffstat: > check | 137 +++++++++++++++++++++++++++++++++++++------------- > common/report | 90 ++++++++++++++++----------------- > 2 files changed, 144 insertions(+), 83 deletions(-) > >
This patchset adds support to loop on failed tests, as proposed by Ted Ts'o in https://lwn.net/Articles/897061/: add a mode that will immediately rerun a failed test 25 or 100 times to establish a failure percentage. Changes since (v3), following Darrick's review: - extended full/out.bad/etc. retention to cover more paths and remove stale .rerun# files Changes since (v2), following Darrick's review: - dropped RFC flag - rebased atop v2022.07.03 - simplified test iterator + results stashed at the end of each iteration, rather than start of next / loop-exit - dropped aggregate loop stats message from xunit report - squashed full/dmesg/out.bad file retention patch with rerun patch Changes since (v1): - rebased atop upstream v2022.06.26 - added a few extra cleanup commits - append details for every rerun to xunit output + provide aggregate stats via <testcase status=X> - extend _stash_test_status to call report hook, as well as save failure artifacts with a .rerun# suffix Diffstat: check | 137 +++++++++++++++++++++++++++++++++++++------------- common/report | 90 ++++++++++++++++----------------- 2 files changed, 144 insertions(+), 83 deletions(-)