[PATCHSET,v2,0/4] fstests: direct specification of looping test duration

Message

Darrick J. Wong April 26, 2023, 12:14 a.m. UTC

Hi all,

One of the things that I do as a maintainer is to designate a handful of
VMs to run fstests for unusually long periods of time.  This practice I
call long term soak testing.  There are actually three separate fleets
for this -- one runs alongside the nightly builds, one runs alongside
weekly rebases, and the last one runs stable releases.

My interactions with all three fleets is pretty much the same -- load
current builds of software, and try to run the exerciser tests for a
duration of time -- 12 hours, 6.5 days, 30 days, etc.  TIME_FACTOR does
not work well for this usage model, because it is difficult to guess
the correct time factor given that the VMs are hetergeneous and the IO
completion rate is not perfectly predictable.

Worse yet, if you want to run (say) all the recoveryloop tests on one VM
(because recoveryloop is prone to crashing), it's impossible to set a
TIME_FACTOR so that each loop test gets equal runtime.  That can be
hacked around with config sections, but that doesn't solve the first
problem.

This series introduces a new configuration variable, SOAK_DURATION, that
allows test runners to control directly various long soak and looping
recovery tests.  This is intended to be an alternative to TIME_FACTOR,
since that variable usually adjusts operation counts, which are
proportional to runtime but otherwise not a direct measure of time.

With this override in place, I can configure the long soak fleet to run
for exactly as long as I want them to, and they actually hit the time
budget targets.  The recoveryloop fleet now divides looping-test time
equally among the four that are in that group so that they all get ~3
hours of coverage every night.

There are more tests that could use this than I actually modified here,
but I've done enough to show this off as a proof of concept.

v2: document TIME/LOAD_FACTOR, redefine the soak tag, remove extranous
changes to g482

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=soak-duration
---
 README                |   11 +++++++
 check                 |   12 ++++++++
 common/config         |    2 +
 common/fuzzy          |    7 ++++
 common/rc             |   34 +++++++++++++++++++++
 common/report         |    1 +
 doc/group-names.txt   |    3 +-
 ltp/fsstress.c        |   78 +++++++++++++++++++++++++++++++++++++++++++++++--
 ltp/fsx.c             |   50 +++++++++++++++++++++++++++++++
 src/soak_duration.awk |   23 ++++++++++++++
 tests/generic/019     |    1 +
 tests/generic/388     |    2 +
 tests/generic/475     |    2 +
 tests/generic/476     |    7 +++-
 tests/generic/482     |    1 +
 tests/generic/521     |    1 +
 tests/generic/522     |    1 +
 tests/generic/642     |    1 +
 tests/generic/648     |    8 +++--
 19 files changed, 231 insertions(+), 14 deletions(-)
 create mode 100644 src/soak_duration.awk