mbox series

[0/5] : CLI and feature improvements for check-parallel

Message ID 20250115060258.3951185-1-david@fromorbit.com (mailing list archive)
Headers show
Series : CLI and feature improvements for check-parallel | expand

Message

Dave Chinner Jan. 15, 2025, 5:51 a.m. UTC
Hi folks,

This patchset brings some new functionality to check-parallel to
make it easier to run across different machines and filesystems.
The patch seti adds these features:

- auto-adjusts concurrency for the machine it is running on.
- adds CLI support of selection of tests. These CLI parameters are
  identical to the check CLI parameters and the test lists are built
  using the same code.
- adds support for logwrites devices so that all the test that use
  dm-logwrites are now enabled
- adds support for specifying the initial filesystem type to test
  on the CLI. This makes it easy to select xfs, btrfs, ext4, etc as
  the target filesystem type that is to be tested.
- Only block device based filesystems can be used with
  check-parallel, and this is now enforced at FSTYP selection time.

For example, testing the rw group on ext4 is now a simple matter of
adding the "-f ext4" parameter to the command line like so:

$ time sudo ./check-parallel -D /mnt/xfs -f ext4 -g rw -x dump
Runner 12 Failures:  ext4/308
Runner 1 Failures:  generic/095
Runner 21 Failures:  generic/042
Runner 51 Failures:  generic/627
Runner 8 Failures:  generic/032
Runner 13 Failures:  generic/019
Runner 34 Failures:  generic/347
.....

Options like exclude files, groups and lists also work natively in
check-parallel now, same as they do in check...

-Dave.

Comments

Darrick J. Wong Jan. 15, 2025, 6:29 a.m. UTC | #1
On Wed, Jan 15, 2025 at 04:51:11PM +1100, Dave Chinner wrote:
> Hi folks,
> 
> This patchset brings some new functionality to check-parallel to
> make it easier to run across different machines and filesystems.
> The patch seti adds these features:
> 
> - auto-adjusts concurrency for the machine it is running on.
> - adds CLI support of selection of tests. These CLI parameters are
>   identical to the check CLI parameters and the test lists are built
>   using the same code.
> - adds support for logwrites devices so that all the test that use
>   dm-logwrites are now enabled
> - adds support for specifying the initial filesystem type to test
>   on the CLI. This makes it easy to select xfs, btrfs, ext4, etc as
>   the target filesystem type that is to be tested.
> - Only block device based filesystems can be used with
>   check-parallel, and this is now enforced at FSTYP selection time.

Can you please add a --help so that curious users don't have to go find
the calling conventions by reading the bash? :)

--D

> For example, testing the rw group on ext4 is now a simple matter of
> adding the "-f ext4" parameter to the command line like so:
> 
> $ time sudo ./check-parallel -D /mnt/xfs -f ext4 -g rw -x dump
> Runner 12 Failures:  ext4/308
> Runner 1 Failures:  generic/095
> Runner 21 Failures:  generic/042
> Runner 51 Failures:  generic/627
> Runner 8 Failures:  generic/032
> Runner 13 Failures:  generic/019
> Runner 34 Failures:  generic/347
> .....
> 
> Options like exclude files, groups and lists also work natively in
> check-parallel now, same as they do in check...
> 
> -Dave.
>
Darrick J. Wong Jan. 15, 2025, 6:47 a.m. UTC | #2
On Tue, Jan 14, 2025 at 10:29:16PM -0800, Darrick J. Wong wrote:
> On Wed, Jan 15, 2025 at 04:51:11PM +1100, Dave Chinner wrote:
> > Hi folks,
> > 
> > This patchset brings some new functionality to check-parallel to
> > make it easier to run across different machines and filesystems.
> > The patch seti adds these features:
> > 
> > - auto-adjusts concurrency for the machine it is running on.
> > - adds CLI support of selection of tests. These CLI parameters are
> >   identical to the check CLI parameters and the test lists are built
> >   using the same code.
> > - adds support for logwrites devices so that all the test that use
> >   dm-logwrites are now enabled
> > - adds support for specifying the initial filesystem type to test
> >   on the CLI. This makes it easy to select xfs, btrfs, ext4, etc as
> >   the target filesystem type that is to be tested.
> > - Only block device based filesystems can be used with
> >   check-parallel, and this is now enforced at FSTYP selection time.
> 
> Can you please add a --help so that curious users don't have to go find
> the calling conventions by reading the bash? :)

Oh, there is one in patch 3, please disregard this message.

--D

> --D
> 
> > For example, testing the rw group on ext4 is now a simple matter of
> > adding the "-f ext4" parameter to the command line like so:
> > 
> > $ time sudo ./check-parallel -D /mnt/xfs -f ext4 -g rw -x dump
> > Runner 12 Failures:  ext4/308
> > Runner 1 Failures:  generic/095
> > Runner 21 Failures:  generic/042
> > Runner 51 Failures:  generic/627
> > Runner 8 Failures:  generic/032
> > Runner 13 Failures:  generic/019
> > Runner 34 Failures:  generic/347
> > .....
> > 
> > Options like exclude files, groups and lists also work natively in
> > check-parallel now, same as they do in check...
> > 
> > -Dave.
> > 
>
Dave Chinner Jan. 15, 2025, 9:54 a.m. UTC | #3
On Tue, Jan 14, 2025 at 10:29:16PM -0800, Darrick J. Wong wrote:
> On Wed, Jan 15, 2025 at 04:51:11PM +1100, Dave Chinner wrote:
> > Hi folks,
> > 
> > This patchset brings some new functionality to check-parallel to
> > make it easier to run across different machines and filesystems.
> > The patch seti adds these features:
> > 
> > - auto-adjusts concurrency for the machine it is running on.
> > - adds CLI support of selection of tests. These CLI parameters are
> >   identical to the check CLI parameters and the test lists are built
> >   using the same code.
> > - adds support for logwrites devices so that all the test that use
> >   dm-logwrites are now enabled
> > - adds support for specifying the initial filesystem type to test
> >   on the CLI. This makes it easy to select xfs, btrfs, ext4, etc as
> >   the target filesystem type that is to be tested.
> > - Only block device based filesystems can be used with
> >   check-parallel, and this is now enforced at FSTYP selection time.
> 
> Can you please add a --help so that curious users don't have to go find
> the calling conventions by reading the bash? :)

Already implemented:

$ ./check-parallel --help
Usage: ./check-parallel [options] [testlist]

check options
    -D <dir>            Directory to run in
    -n                  Output test list, do not run tests
    -r                  randomize test order
    --exact-order       run tests in the exact order specified
    -s section          run only specified section from config file
    -f <FSTYPE>         specify the filesystem type to test

testlist options
    -g group[,group...] include tests from these groups
    -x group[,group...] exclude tests from these groups
    -X exclude_file     exclude individual tests
    -e testlist         exclude a specific list of tests
    -E external_file    exclude individual tests
    [testlist]          include tests matching names in testlist

testlist argument is a list of tests in the form of <test dir>/<test name>.

<test dir> is a directory under tests that contains a group file,
with a list of the names of the tests in that directory.

<test name> may be either a specific test file name (e.g. xfs/001) or
a test file name match pattern (e.g. xfs/*).

group argument is either a name of a tests group to collect from all
the test dirs (e.g. quick) or a name of a tests group to collect from
a specific tests dir in the form of <test dir>/<group name> (e.g. xfs/quick).
If you want to run all the tests in the test suite, use "-g all" to specify all
groups.

exclude_file argument refers to a name of a file inside each test directory.
for every test dir where this file is found, the listed test names are
excluded from the list of tests to run from that test dir.

external_file argument is a path to a single file containing a list of tests
to exclude in the form of <test dir>/<test name>.

examples:
 check-parallel -f xfs -D /mnt xfs/001
 check-parallel -f ext4 -D /mnt -g quick
 check-parallel -D /mnt -g xfs/quick
 check-parallel -D /mnt -x stress xfs/*
 check-parallel -f btrfs -D /mnt -X .exclude -g auto
 check-parallel -f udf -D /mnt -E ~/.xfstests.exclude
$

-Dave.