| Message ID | 20240216181859.788521-3-mcgrof@kernel.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | few enhancements | expand |
On Fri, Feb 16, 2024 at 10:18:58AM -0800, Luis Chamberlain wrote: > Since the prior commit adds the ability to list groups but is used > only when we use --start-after, let's add an option which leverages this > to also allow us to easily query which tests are part of the groups > specified. > > This can be used for dynamic test configuration suites such as kdevops > which may want to take advantage of this information to deterministically > determine if a test falls part of a specific group. > Demo: > > root@demo-xfs-reflink /var/lib/xfstests # ./check --list-group-tests -g soak > > generic/019 generic/388 generic/475 generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/648 generic/650 xfs/285 xfs/517 xfs/560 xfs/561 xfs/562 xfs/565 xfs/570 xfs/571 xfs/572 xfs/573 xfs/574 xfs/575 xfs/576 xfs/577 xfs/578 xfs/579 xfs/580 xfs/581 xfs/582 xfs/583 xfs/584 xfs/585 xfs/586 xfs/587 xfs/588 xfs/589 xfs/590 xfs/591 xfs/592 xfs/593 xfs/594 xfs/595 xfs/727 xfs/729 xfs/800 So how is this different to ./check -n -g soak? '-n' is supposed to show you want tests are going to be run without actually running them, so why can't you use that? -Dave.
On Mon, Feb 19, 2024 at 02:38:12PM +1100, Dave Chinner wrote: > On Fri, Feb 16, 2024 at 10:18:58AM -0800, Luis Chamberlain wrote: > > Since the prior commit adds the ability to list groups but is used > > only when we use --start-after, let's add an option which leverages this > > to also allow us to easily query which tests are part of the groups > > specified. > > > > This can be used for dynamic test configuration suites such as kdevops > > which may want to take advantage of this information to deterministically > > determine if a test falls part of a specific group. > > Demo: > > > > root@demo-xfs-reflink /var/lib/xfstests # ./check --list-group-tests -g soak > > > > generic/019 generic/388 generic/475 generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/648 generic/650 xfs/285 xfs/517 xfs/560 xfs/561 xfs/562 xfs/565 xfs/570 xfs/571 xfs/572 xfs/573 xfs/574 xfs/575 xfs/576 xfs/577 xfs/578 xfs/579 xfs/580 xfs/581 xfs/582 xfs/583 xfs/584 xfs/585 xfs/586 xfs/587 xfs/588 xfs/589 xfs/590 xfs/591 xfs/592 xfs/593 xfs/594 xfs/595 xfs/727 xfs/729 xfs/800 > > So how is this different to ./check -n -g soak? > > '-n' is supposed to show you want tests are going to be run > without actually running them, so why can't you use that? '-n' will replicate as if you are running all tests but just skip while --list-group-tests will just look for the tests for the group and bail right away, and it is machine readable. Luis
On Wed, Feb 21, 2024 at 08:45:37AM -0800, Luis Chamberlain wrote: > On Mon, Feb 19, 2024 at 02:38:12PM +1100, Dave Chinner wrote: > > On Fri, Feb 16, 2024 at 10:18:58AM -0800, Luis Chamberlain wrote: > > > Since the prior commit adds the ability to list groups but is used > > > only when we use --start-after, let's add an option which leverages this > > > to also allow us to easily query which tests are part of the groups > > > specified. > > > > > > This can be used for dynamic test configuration suites such as kdevops > > > which may want to take advantage of this information to deterministically > > > determine if a test falls part of a specific group. > > > Demo: > > > > > > root@demo-xfs-reflink /var/lib/xfstests # ./check --list-group-tests -g soak > > > > > > generic/019 generic/388 generic/475 generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/648 generic/650 xfs/285 xfs/517 xfs/560 xfs/561 xfs/562 xfs/565 xfs/570 xfs/571 xfs/572 xfs/573 xfs/574 xfs/575 xfs/576 xfs/577 xfs/578 xfs/579 xfs/580 xfs/581 xfs/582 xfs/583 xfs/584 xfs/585 xfs/586 xfs/587 xfs/588 xfs/589 xfs/590 xfs/591 xfs/592 xfs/593 xfs/594 xfs/595 xfs/727 xfs/729 xfs/800 > > > > So how is this different to ./check -n -g soak? > > > > '-n' is supposed to show you want tests are going to be run > > without actually running them, so why can't you use that? > > '-n' will replicate as if you are running all tests but just skip while > --list-group-tests will just look for the tests for the group and bail right > away, and it is machine readable. What do you mean "replicate as if you are running all tests but just skip"? Sorry I don't understand this explanation 100%, can you show us some examples to explain what kind of job you hope "--list-group-tests" to do, but the "-n" is helpless? Thanks, Zorro > > Luis >
diff --git a/check b/check index f081bf8ce685..523cf024c139 100755 --- a/check +++ b/check @@ -19,6 +19,7 @@ have_test_arg=false randomize=false exact_order=false start_after_test="" +list_group_tests=false export here=`pwd` xfile="" subdir_xfile="" @@ -81,6 +82,7 @@ check options -b brief test summary -R fmt[,fmt] generate report in formats specified. Supported formats: xunit, xunit-quiet --large-fs optimise scratch device for large filesystems + --list-group-tests only list tests part of the groups you specified, do not run the tests --start-after only start testing after the test specified -s section run only specified section from config file -S section exclude the specified section from the config file @@ -276,8 +278,16 @@ _prepare_test_list() done group_all="$group_all $list" done + + group_all=$(echo $group_all | sed -e 's|tests/||g') + + # Keep it simple, allow for easy machine scraping + if $list_group_tests ; then + echo $group_all + exit 0 + fi + if [[ "$start_after_test" != "" && $start_after_found -ne 1 ]]; then - group_all=$(echo $group_all | sed -e 's|tests/||g') echo "Start after test $start_after_test not found in any group specified." echo "Be sure you specify a test present in one of your test run groups if using --start-after." echo @@ -366,6 +376,9 @@ while [ $# -gt 0 ]; do start_after_test="$2" shift ;; + --list-group-tests) + list_group_tests=true + ;; -s) RUN_SECTION="$RUN_SECTION $2"; shift ;; -S) EXCLUDE_SECTION="$EXCLUDE_SECTION $2"; shift ;; -l) diff="diff" ;;
Since the prior commit adds the ability to list groups but is used only when we use --start-after, let's add an option which leverages this to also allow us to easily query which tests are part of the groups specified. This can be used for dynamic test configuration suites such as kdevops which may want to take advantage of this information to deterministically determine if a test falls part of a specific group. Demo: root@demo-xfs-reflink /var/lib/xfstests # ./check --list-group-tests -g soak generic/019 generic/388 generic/475 generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/648 generic/650 xfs/285 xfs/517 xfs/560 xfs/561 xfs/562 xfs/565 xfs/570 xfs/571 xfs/572 xfs/573 xfs/574 xfs/575 xfs/576 xfs/577 xfs/578 xfs/579 xfs/580 xfs/581 xfs/582 xfs/583 xfs/584 xfs/585 xfs/586 xfs/587 xfs/588 xfs/589 xfs/590 xfs/591 xfs/592 xfs/593 xfs/594 xfs/595 xfs/727 xfs/729 xfs/800 Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> --- check | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)