diff mbox series

[2/3] check: add support for --list-group-tests

Message ID 20240216181859.788521-3-mcgrof@kernel.org (mailing list archive)
State New, archived
Headers show
Series few enhancements | expand

Commit Message

Luis Chamberlain Feb. 16, 2024, 6:18 p.m. UTC
Since the prior commit adds the ability to list groups but is used
only when we use --start-after, let's add an option which leverages this
to also allow us to easily query which tests are part of the groups
specified.

This can be used for dynamic test configuration suites such as kdevops
which may want to take advantage of this information to deterministically
determine if a test falls part of a specific group.

Demo:

root@demo-xfs-reflink /var/lib/xfstests # ./check --list-group-tests -g soak

generic/019 generic/388 generic/475 generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/648 generic/650 xfs/285 xfs/517 xfs/560 xfs/561 xfs/562 xfs/565 xfs/570 xfs/571 xfs/572 xfs/573 xfs/574 xfs/575 xfs/576 xfs/577 xfs/578 xfs/579 xfs/580 xfs/581 xfs/582 xfs/583 xfs/584 xfs/585 xfs/586 xfs/587 xfs/588 xfs/589 xfs/590 xfs/591 xfs/592 xfs/593 xfs/594 xfs/595 xfs/727 xfs/729 xfs/800

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 check | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

Comments

Dave Chinner Feb. 19, 2024, 3:38 a.m. UTC | #1
On Fri, Feb 16, 2024 at 10:18:58AM -0800, Luis Chamberlain wrote:
> Since the prior commit adds the ability to list groups but is used
> only when we use --start-after, let's add an option which leverages this
> to also allow us to easily query which tests are part of the groups
> specified.
> 
> This can be used for dynamic test configuration suites such as kdevops
> which may want to take advantage of this information to deterministically
> determine if a test falls part of a specific group.
> Demo:
> 
> root@demo-xfs-reflink /var/lib/xfstests # ./check --list-group-tests -g soak
> 
> generic/019 generic/388 generic/475 generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/648 generic/650 xfs/285 xfs/517 xfs/560 xfs/561 xfs/562 xfs/565 xfs/570 xfs/571 xfs/572 xfs/573 xfs/574 xfs/575 xfs/576 xfs/577 xfs/578 xfs/579 xfs/580 xfs/581 xfs/582 xfs/583 xfs/584 xfs/585 xfs/586 xfs/587 xfs/588 xfs/589 xfs/590 xfs/591 xfs/592 xfs/593 xfs/594 xfs/595 xfs/727 xfs/729 xfs/800

So how is this different to ./check -n -g soak?

'-n' is supposed to show you want tests are going to be run
without actually running them, so why can't you use that?

-Dave.
Luis Chamberlain Feb. 21, 2024, 4:45 p.m. UTC | #2
On Mon, Feb 19, 2024 at 02:38:12PM +1100, Dave Chinner wrote:
> On Fri, Feb 16, 2024 at 10:18:58AM -0800, Luis Chamberlain wrote:
> > Since the prior commit adds the ability to list groups but is used
> > only when we use --start-after, let's add an option which leverages this
> > to also allow us to easily query which tests are part of the groups
> > specified.
> > 
> > This can be used for dynamic test configuration suites such as kdevops
> > which may want to take advantage of this information to deterministically
> > determine if a test falls part of a specific group.
> > Demo:
> > 
> > root@demo-xfs-reflink /var/lib/xfstests # ./check --list-group-tests -g soak
> > 
> > generic/019 generic/388 generic/475 generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/648 generic/650 xfs/285 xfs/517 xfs/560 xfs/561 xfs/562 xfs/565 xfs/570 xfs/571 xfs/572 xfs/573 xfs/574 xfs/575 xfs/576 xfs/577 xfs/578 xfs/579 xfs/580 xfs/581 xfs/582 xfs/583 xfs/584 xfs/585 xfs/586 xfs/587 xfs/588 xfs/589 xfs/590 xfs/591 xfs/592 xfs/593 xfs/594 xfs/595 xfs/727 xfs/729 xfs/800
> 
> So how is this different to ./check -n -g soak?
> 
> '-n' is supposed to show you want tests are going to be run
> without actually running them, so why can't you use that?

'-n' will replicate as if you are running all tests but just skip while
--list-group-tests will just look for the tests for the group and bail right
away, and it is machine readable.

  Luis
Zorro Lang Feb. 25, 2024, 4:08 p.m. UTC | #3
On Wed, Feb 21, 2024 at 08:45:37AM -0800, Luis Chamberlain wrote:
> On Mon, Feb 19, 2024 at 02:38:12PM +1100, Dave Chinner wrote:
> > On Fri, Feb 16, 2024 at 10:18:58AM -0800, Luis Chamberlain wrote:
> > > Since the prior commit adds the ability to list groups but is used
> > > only when we use --start-after, let's add an option which leverages this
> > > to also allow us to easily query which tests are part of the groups
> > > specified.
> > > 
> > > This can be used for dynamic test configuration suites such as kdevops
> > > which may want to take advantage of this information to deterministically
> > > determine if a test falls part of a specific group.
> > > Demo:
> > > 
> > > root@demo-xfs-reflink /var/lib/xfstests # ./check --list-group-tests -g soak
> > > 
> > > generic/019 generic/388 generic/475 generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/648 generic/650 xfs/285 xfs/517 xfs/560 xfs/561 xfs/562 xfs/565 xfs/570 xfs/571 xfs/572 xfs/573 xfs/574 xfs/575 xfs/576 xfs/577 xfs/578 xfs/579 xfs/580 xfs/581 xfs/582 xfs/583 xfs/584 xfs/585 xfs/586 xfs/587 xfs/588 xfs/589 xfs/590 xfs/591 xfs/592 xfs/593 xfs/594 xfs/595 xfs/727 xfs/729 xfs/800
> > 
> > So how is this different to ./check -n -g soak?
> > 
> > '-n' is supposed to show you want tests are going to be run
> > without actually running them, so why can't you use that?
> 
> '-n' will replicate as if you are running all tests but just skip while
> --list-group-tests will just look for the tests for the group and bail right
> away, and it is machine readable.

What do you mean "replicate as if you are running all tests but just skip"?
Sorry I don't understand this explanation 100%, can you show us some examples
to explain what kind of job you hope "--list-group-tests" to do, but the "-n"
is helpless?

Thanks,
Zorro

> 
>   Luis
>
diff mbox series

Patch

diff --git a/check b/check
index f081bf8ce685..523cf024c139 100755
--- a/check
+++ b/check
@@ -19,6 +19,7 @@  have_test_arg=false
 randomize=false
 exact_order=false
 start_after_test=""
+list_group_tests=false
 export here=`pwd`
 xfile=""
 subdir_xfile=""
@@ -81,6 +82,7 @@  check options
     -b			brief test summary
     -R fmt[,fmt]	generate report in formats specified. Supported formats: xunit, xunit-quiet
     --large-fs		optimise scratch device for large filesystems
+    --list-group-tests	only list tests part of the groups you specified, do not run the tests
     --start-after	only start testing after the test specified
     -s section		run only specified section from config file
     -S section		exclude the specified section from the config file
@@ -276,8 +278,16 @@  _prepare_test_list()
 			done
 			group_all="$group_all $list"
 		done
+
+		group_all=$(echo $group_all | sed -e 's|tests/||g')
+
+		# Keep it simple, allow for easy machine scraping
+		if $list_group_tests ; then
+			echo $group_all
+			exit 0
+		fi
+
 		if [[ "$start_after_test" != "" && $start_after_found -ne 1 ]]; then
-			group_all=$(echo $group_all | sed -e 's|tests/||g')
 			echo "Start after test $start_after_test not found in any group specified."
 			echo "Be sure you specify a test present in one of your test run groups if using --start-after."
 			echo
@@ -366,6 +376,9 @@  while [ $# -gt 0 ]; do
 		start_after_test="$2"
 		shift
 		;;
+	--list-group-tests)
+		list_group_tests=true
+		;;
 	-s)	RUN_SECTION="$RUN_SECTION $2"; shift ;;
 	-S)	EXCLUDE_SECTION="$EXCLUDE_SECTION $2"; shift ;;
 	-l)	diff="diff" ;;