| Message ID | 20240328142004.2144568-1-jackmanb@google.com (mailing list archive) |
|---|---|
| State | New |
| Delegated to: | Brendan Higgins |
| Headers | show |
| Series | Documentation: kunit: Clarify test filter format | expand |
On Thu, Mar 28, 2024 at 7:20 AM 'Brendan Jackman' via KUnit Development <kunit-dev@googlegroups.com> wrote: > > It seems obvious once you know, but at first I didn't realise that the > suite name is part of this format. Document it and add example. > > Signed-off-by: Brendan Jackman <jackmanb@google.com> > --- > Documentation/dev-tools/kunit/run_wrapper.rst | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst > index 19ddf5e07013..e75a5fc05814 100644 > --- a/Documentation/dev-tools/kunit/run_wrapper.rst > +++ b/Documentation/dev-tools/kunit/run_wrapper.rst > @@ -156,13 +156,20 @@ Filtering tests > =============== > > By passing a bash style glob filter to the ``exec`` or ``run`` > -commands, we can run a subset of the tests built into a kernel . For > +commands, we can run a subset of the tests built into a kernel, > +identified by a string like ``$suite_name.$test_name``. For Apologies for the overly terse docs, that's my fault :) I'm wondering if we can further improve it while we're here. Note, the format for the glob is: $suite_name[.$test_name]. This current wording and examples (before and after this change) might make the user think otherwise, i.e. that it works like effective_name = suite_name + '.' + test_name return glob_matches(effective_name, filter_glob) E.g. given a test name like `suite.test_name` and glob='suite*name' they might expect it to match, but it does *not*. The logic actually works like: suite_glob, test_glob = split(filter_glob) if not_glob_matches(suite_name, suite_glob): return False if test_glob and not glob_matches(test_name, test_glob): return False return True Perhaps expanding the list of examples to cover more of the edge cases could help get the right intuition? E.g. perhaps these: kunit.py run <suite_name> # runs all tests in a specific suite kunit.py run <suite_name>.<test_name> # run a specific test kunit.py run suite_prefix* # what the current example shows kunit.py run *.*test_suffix # matches all suites, only tests w/ a certain suffix kunit.py run suite_prefix*.*test_suffix # combined version of above Thoughts? Thanks, Daniel
On Thu, 28 Mar 2024 at 19:27, Daniel Latypov <dlatypov@google.com> wrote: > This current wording and examples (before and after this change) might > make the user think otherwise, i.e. that it works like > effective_name = suite_name + '.' + test_name > return glob_matches(effective_name, filter_glob) > > E.g. given a test name like `suite.test_name` and glob='suite*name' > they might expect it to match, but it does *not*. > > The logic actually works like: > suite_glob, test_glob = split(filter_glob) > if not_glob_matches(suite_name, suite_glob): > return False > if test_glob and not glob_matches(test_name, test_glob): > return False > return True > > Perhaps expanding the list of examples to cover more of the edge cases > could help get the right intuition? > > E.g. perhaps these: > kunit.py run <suite_name> # runs all tests in a specific suite > kunit.py run <suite_name>.<test_name> # run a specific test > > kunit.py run suite_prefix* # what the current example shows > kunit.py run *.*test_suffix # matches all suites, only tests w/ a > certain suffix > kunit.py run suite_prefix*.*test_suffix # combined version of above > > Thoughts? Thanks yeah, good point. The result is pretty verbose but it doesn't create much cognitive load for the reader so might as well just be really explicit. v2 incoming if `make htmldocs` ever finishes....
diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst index 19ddf5e07013..e75a5fc05814 100644 --- a/Documentation/dev-tools/kunit/run_wrapper.rst +++ b/Documentation/dev-tools/kunit/run_wrapper.rst @@ -156,13 +156,20 @@ Filtering tests =============== By passing a bash style glob filter to the ``exec`` or ``run`` -commands, we can run a subset of the tests built into a kernel . For +commands, we can run a subset of the tests built into a kernel, +identified by a string like ``$suite_name.$test_name``. For example: if we only want to run KUnit resource tests, use: .. code-block:: ./tools/testing/kunit/kunit.py run 'kunit-resource*' +Or to run just one specific test from that suite: + +.. code-block:: + + ./tools/testing/kunit/kunit.py run 'kunit-resource-test.kunit_resource_test_init_resources' + This uses the standard glob format with wildcard characters. .. _kunit-on-qemu:
It seems obvious once you know, but at first I didn't realise that the suite name is part of this format. Document it and add example. Signed-off-by: Brendan Jackman <jackmanb@google.com> --- Documentation/dev-tools/kunit/run_wrapper.rst | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)