| Message ID | 1455188974-17801-3-git-send-email-david.weinehall@linux.intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, Feb 11, 2016 at 01:09:33PM +0200, David Weinehall wrote: > +enum { > + /* The set of tests run if nothing else is specified */ > + SUBTEST_TYPE_NORMAL = 1 << 0, > + /* Basic Acceptance Testing set */ > + SUBTEST_TYPE_BASIC = 1 << 1, > + /* Tests that are very slow */ > + SUBTEST_TYPE_SLOW = 1 << 2, I still feel that slow isn't a useful descriminant for tests. Off the opt of my head, I would like HANG, SWAP, STRESS (this is too also undefined) that should cover the majority of the GEM cases. Looking at kms_flip should also offer a few categories. As for gem_concurrent_blit, I basically expect to have h->subtest_flags | p->subtest_flags | subtest_flags. I also expect QA to run the full gem_concurrent_blit anyway since it is showing up regressions from over a year ago. -Chris
On Thu, Feb 11, 2016 at 01:04:14PM +0000, Chris Wilson wrote: > On Thu, Feb 11, 2016 at 01:09:33PM +0200, David Weinehall wrote: > > +enum { > > + /* The set of tests run if nothing else is specified */ > > + SUBTEST_TYPE_NORMAL = 1 << 0, > > + /* Basic Acceptance Testing set */ > > + SUBTEST_TYPE_BASIC = 1 << 1, > > + /* Tests that are very slow */ > > + SUBTEST_TYPE_SLOW = 1 << 2, > > I still feel that slow isn't a useful descriminant for tests. Off the > opt of my head, I would like > > HANG, > SWAP, > STRESS (this is too also undefined) > > that should cover the majority of the GEM cases. Looking at kms_flip > should also offer a few categories. As I said the first time I posted this patch, I'm open to both alternative names and help with categorising the subtests. > As for gem_concurrent_blit, I basically expect to > have h->subtest_flags | p->subtest_flags | subtest_flags. > > I also expect QA to run the full gem_concurrent_blit anyway since it is > showing up regressions from over a year ago. Yeah, I did a partial run of the full set yesterday and spotted about 150 failed tests within an hour of so. After that I had to abort the test to continue with other work on that machine. But it's fairly obvious that the limited set is a lot better tested. Kind regards, David
On Thu, Feb 11, 2016 at 01:09:33PM +0200, David Weinehall wrote: > Some subtests are not run by default, for various reasons; > be it because they're only for debugging, because they're slow, > or because they are not of high enough quality. > > This patch aims to introduce a common mechanism for categorising > the subtests and introduces a flag (--all) that runs/lists all > subtests instead of just the default set. Is the idea to also add a --test-flags interface? How is this meant to integrate with the current BAT runtime? Also we need to figure out how/whether we need to change the spec for how testrunners are supposed to run igt tests. Plus obviously patch igt.py in piglit. I think I like where this is going, but I guess needs a bit more polish still. Integrating this into how we run/select BAT would be a great showcase of your concept I think. Cheers, Daniel > > Signed-off-by: David Weinehall <david.weinehall@linux.intel.com> > --- > lib/igt_core.c | 43 +++++++-- > lib/igt_core.h | 42 +++++++++ > tests/gem_concurrent_blit.c | 50 +++++------ > tests/kms_frontbuffer_tracking.c | 186 ++++++++++++++++++++++----------------- > 4 files changed, 210 insertions(+), 111 deletions(-) > > diff --git a/lib/igt_core.c b/lib/igt_core.c > index 6b69bb780700..e6e6949ed65a 100644 > --- a/lib/igt_core.c > +++ b/lib/igt_core.c > @@ -216,6 +216,7 @@ const char *igt_interactive_debug; > > /* subtests helpers */ > static bool list_subtests = false; > +static unsigned int subtest_types_mask = SUBTEST_TYPE_NORMAL; > static char *run_single_subtest = NULL; > static bool run_single_subtest_found = false; > static const char *in_subtest = NULL; > @@ -237,12 +238,13 @@ int test_children_sz; > bool test_child; > > enum { > - OPT_LIST_SUBTESTS, > - OPT_RUN_SUBTEST, > - OPT_DESCRIPTION, > - OPT_DEBUG, > - OPT_INTERACTIVE_DEBUG, > - OPT_HELP = 'h' > + OPT_LIST_SUBTESTS, > + OPT_WITH_ALL_SUBTESTS, > + OPT_RUN_SUBTEST, > + OPT_DESCRIPTION, > + OPT_DEBUG, > + OPT_INTERACTIVE_DEBUG, > + OPT_HELP = 'h' > }; > > static int igt_exitcode = IGT_EXIT_SUCCESS; > @@ -516,6 +518,7 @@ static void print_usage(const char *help_str, bool output_on_stderr) > > fprintf(f, "Usage: %s [OPTIONS]\n", command_str); > fprintf(f, " --list-subtests\n" > + " --all\n" > " --run-subtest <pattern>\n" > " --debug[=log-domain]\n" > " --interactive-debug[=domain]\n" > @@ -548,6 +551,7 @@ static int common_init(int *argc, char **argv, > int c, option_index = 0, i, x; > static struct option long_options[] = { > {"list-subtests", 0, 0, OPT_LIST_SUBTESTS}, > + {"all", 0, 0, OPT_WITH_ALL_SUBTESTS}, > {"run-subtest", 1, 0, OPT_RUN_SUBTEST}, > {"help-description", 0, 0, OPT_DESCRIPTION}, > {"debug", optional_argument, 0, OPT_DEBUG}, > @@ -659,6 +663,10 @@ static int common_init(int *argc, char **argv, > if (!run_single_subtest) > list_subtests = true; > break; > + case OPT_WITH_ALL_SUBTESTS: > + if (!run_single_subtest) > + subtest_types_mask = SUBTEST_TYPE_ALL; > + break; > case OPT_RUN_SUBTEST: > if (!list_subtests) > run_single_subtest = strdup(optarg); > @@ -1667,6 +1675,29 @@ void igt_skip_on_simulation(void) > igt_require(!igt_run_in_simulation()); > } > > +/** > + * igt_match_subtest_flags: > + * > + * This function is used to check whether the attributes of the subtest > + * makes it a candidate for inclusion in the test run; this is used to > + * categorise tests, for instance to exclude tests that are purely for > + * debug purposes, tests that are specific to certain environments, > + * or tests that are very slow. > + * > + * Note that a test has to have all its flags met to be run; for instance > + * a subtest with the flags SUBTEST_TYPE_SLOW | SUBTEST_TYPE_DEBUG requires > + * "--subtest-types=slow,debug" or "--all" to be executed > + * > + * @subtest_flags: The subtests to check for > + * > + * Returns: true if the subtest test should be run, > + * false if the subtest should be skipped > + */ > +bool igt_match_subtest_flags(unsigned long subtest_flags) > +{ > + return ((subtest_flags & subtest_types_mask) == subtest_flags); > +} > + > /* structured logging */ > > /** > diff --git a/lib/igt_core.h b/lib/igt_core.h > index 8f297e06a068..bf83de609bfa 100644 > --- a/lib/igt_core.h > +++ b/lib/igt_core.h > @@ -193,6 +193,48 @@ bool __igt_run_subtest(const char *subtest_name); > #define igt_subtest_f(f...) \ > __igt_subtest_f(igt_tokencat(__tmpchar, __LINE__), f) > > +enum { > + /* The set of tests run if nothing else is specified */ > + SUBTEST_TYPE_NORMAL = 1 << 0, > + /* Basic Acceptance Testing set */ > + SUBTEST_TYPE_BASIC = 1 << 1, > + /* Tests that are very slow */ > + SUBTEST_TYPE_SLOW = 1 << 2, > + /* Tests that mainly intended for debugging */ > + SUBTEST_TYPE_DEBUG = 1 << 3, > + SUBTEST_TYPE_ALL = ~0 > +} subtest_types; > + > +bool igt_match_subtest_flags(unsigned long subtest_flags); > + > +/** > + * igt_subtest_flags: > + * @name: name of the subtest > + * @__subtest_flags: the categories the subtest belongs to > + * > + * This is a wrapper around igt_subtest that will only execute the > + * testcase if all of the flags passed to this function match those > + * specified by the list of subtest categories passed from the > + * command line; the default category is SUBTEST_TYPE_NORMAL. > + */ > +#define igt_subtest_flags(name, __subtest_flags) \ > + if (igt_match_subtest_flags(__subtest_flags)) \ > + igt_subtest(name) > + > +/** > + * igt_subtest_flags_f: > + * @...: format string and optional arguments > + * @__subtest_flags: the categories the subtest belongs to > + * > + * This is a wrapper around igt_subtest_f that will only execute the > + * testcase if all of the flags passed to this function match those > + * specified by the list of subtest categories passed from the > + * command line; the default category is SUBTEST_TYPE_NORMAL. > + */ > +#define igt_subtest_flags_f(__subtest_flags, f...) \ > + if (igt_match_subtest_flags(__subtest_flags)) \ > + __igt_subtest_f(igt_tokencat(__tmpchar, __LINE__), f) > + > const char *igt_subtest_name(void); > bool igt_only_list_subtests(void); > > diff --git a/tests/gem_concurrent_blit.c b/tests/gem_concurrent_blit.c > index 9b7ef8700e31..3c23a95c4631 100644 > --- a/tests/gem_concurrent_blit.c > +++ b/tests/gem_concurrent_blit.c > @@ -64,7 +64,6 @@ struct local_i915_gem_userptr { > > int fd, devid, gen; > struct intel_batchbuffer *batch; > -int all; > int pass; > > struct buffers { > @@ -1256,10 +1255,14 @@ run_basic_modes(const char *prefix, > }, *h; > > for (h = hangs; h->suffix; h++) { > - if (!all && *h->suffix) > - continue; > + unsigned int subtest_flags; > > - for (p = all ? pipelines : pskip; p->prefix; p++) { > + if (*h->suffix) > + subtest_flags = SUBTEST_TYPE_SLOW; > + else > + subtest_flags = SUBTEST_TYPE_NORMAL; > + > + for (p = igt_match_subtest_flags(SUBTEST_TYPE_SLOW) ? pipelines : pskip; p->prefix; p++) { > struct buffers buffers; > > igt_fixture > @@ -1267,21 +1270,21 @@ run_basic_modes(const char *prefix, > 512, 512, fd, > run_wrap_func != run_child); > > - igt_subtest_f("%s-%s-%s-sanitycheck0%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-sanitycheck0%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > buffers_create(&buffers, num_buffers); > run_wrap_func(&buffers, do_basic0, > p->copy, h->hang); > } > > - igt_subtest_f("%s-%s-%s-sanitycheck1%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-sanitycheck1%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > buffers_create(&buffers, num_buffers); > run_wrap_func(&buffers, do_basic1, > p->copy, h->hang); > } > > - igt_subtest_f("%s-%s-%s-sanitycheckN%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-sanitycheckN%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > buffers_create(&buffers, num_buffers); > run_wrap_func(&buffers, do_basicN, > @@ -1289,7 +1292,7 @@ run_basic_modes(const char *prefix, > } > > /* try to overwrite the source values */ > - igt_subtest_f("%s-%s-%s-overwrite-source-one%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-overwrite-source-one%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > buffers_create(&buffers, num_buffers); > run_wrap_func(&buffers, > @@ -1297,7 +1300,7 @@ run_basic_modes(const char *prefix, > p->copy, h->hang); > } > > - igt_subtest_f("%s-%s-%s-overwrite-source%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-overwrite-source%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > buffers_create(&buffers, num_buffers); > run_wrap_func(&buffers, > @@ -1305,7 +1308,7 @@ run_basic_modes(const char *prefix, > p->copy, h->hang); > } > > - igt_subtest_f("%s-%s-%s-overwrite-source-read-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-overwrite-source-read-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > buffers_create(&buffers, num_buffers); > run_wrap_func(&buffers, > @@ -1313,7 +1316,7 @@ run_basic_modes(const char *prefix, > p->copy, h->hang); > } > > - igt_subtest_f("%s-%s-%s-overwrite-source-read-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-overwrite-source-read-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > igt_require(rendercopy); > buffers_create(&buffers, num_buffers); > @@ -1322,7 +1325,7 @@ run_basic_modes(const char *prefix, > p->copy, h->hang); > } > > - igt_subtest_f("%s-%s-%s-overwrite-source-rev%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-overwrite-source-rev%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > buffers_create(&buffers, num_buffers); > run_wrap_func(&buffers, > @@ -1331,7 +1334,7 @@ run_basic_modes(const char *prefix, > } > > /* try to intermix copies with GPU copies*/ > - igt_subtest_f("%s-%s-%s-intermix-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-intermix-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > igt_require(rendercopy); > buffers_create(&buffers, num_buffers); > @@ -1339,7 +1342,7 @@ run_basic_modes(const char *prefix, > do_intermix_rcs, > p->copy, h->hang); > } > - igt_subtest_f("%s-%s-%s-intermix-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-intermix-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > igt_require(rendercopy); > buffers_create(&buffers, num_buffers); > @@ -1347,7 +1350,7 @@ run_basic_modes(const char *prefix, > do_intermix_bcs, > p->copy, h->hang); > } > - igt_subtest_f("%s-%s-%s-intermix-both%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-intermix-both%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > igt_require(rendercopy); > buffers_create(&buffers, num_buffers); > @@ -1357,7 +1360,7 @@ run_basic_modes(const char *prefix, > } > > /* try to read the results before the copy completes */ > - igt_subtest_f("%s-%s-%s-early-read%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-early-read%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > buffers_create(&buffers, num_buffers); > run_wrap_func(&buffers, > @@ -1366,14 +1369,14 @@ run_basic_modes(const char *prefix, > } > > /* concurrent reads */ > - igt_subtest_f("%s-%s-%s-read-read-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-read-read-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > buffers_create(&buffers, num_buffers); > run_wrap_func(&buffers, > do_read_read_bcs, > p->copy, h->hang); > } > - igt_subtest_f("%s-%s-%s-read-read-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-read-read-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > igt_require(rendercopy); > buffers_create(&buffers, num_buffers); > @@ -1383,14 +1386,14 @@ run_basic_modes(const char *prefix, > } > > /* split copying between rings */ > - igt_subtest_f("%s-%s-%s-write-read-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-write-read-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > buffers_create(&buffers, num_buffers); > run_wrap_func(&buffers, > do_write_read_bcs, > p->copy, h->hang); > } > - igt_subtest_f("%s-%s-%s-write-read-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-write-read-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > igt_require(rendercopy); > buffers_create(&buffers, num_buffers); > @@ -1399,8 +1402,8 @@ run_basic_modes(const char *prefix, > p->copy, h->hang); > } > > - /* and finally try to trick the kernel into loosing the pending write */ > - igt_subtest_f("%s-%s-%s-gpu-read-after-write%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > + /* and finally try to trick the kernel into losing the pending write */ > + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-gpu-read-after-write%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { > p->require(); > buffers_create(&buffers, num_buffers); > run_wrap_func(&buffers, > @@ -1454,9 +1457,6 @@ igt_main > > igt_skip_on_simulation(); > > - if (strstr(igt_test_name(), "all")) > - all = true; > - > igt_fixture { > fd = drm_open_driver(DRIVER_INTEL); > intel_detect_and_clear_missed_interrupts(fd); > diff --git a/tests/kms_frontbuffer_tracking.c b/tests/kms_frontbuffer_tracking.c > index 64f880c667a3..651303e9d392 100644 > --- a/tests/kms_frontbuffer_tracking.c > +++ b/tests/kms_frontbuffer_tracking.c > @@ -47,8 +47,7 @@ IGT_TEST_DESCRIPTION("Test the Kernel's frontbuffer tracking mechanism and " > * combinations that are somewhat redundant and don't add much value to the > * test. For example, since we already do the offscreen testing with a single > * pipe enabled, there's no much value in doing it again with dual pipes. If you > - * still want to try these redundant tests, you need to use the --show-hidden > - * option. > + * still want to try these redundant tests, you need to use the --all option. > * > * The most important hidden thing is the FEATURE_NONE set of tests. Whenever > * you get a failure on any test, it is important to check whether the same test > @@ -126,6 +125,9 @@ struct test_mode { > } flip; > > enum igt_draw_method method; > + > + /* Specifies the subtest categories this subtest belongs to */ > + unsigned long subtest_flags; > }; > > enum color { > @@ -241,7 +243,6 @@ struct { > bool fbc_check_last_action; > bool no_edp; > bool small_modes; > - bool show_hidden; > int step; > int only_pipes; > int shared_fb_x_offset; > @@ -253,7 +254,6 @@ struct { > .fbc_check_last_action = true, > .no_edp = false, > .small_modes = false, > - .show_hidden= false, > .step = 0, > .only_pipes = PIPE_COUNT, > .shared_fb_x_offset = 500, > @@ -3075,9 +3075,6 @@ static int opt_handler(int option, int option_index, void *data) > case 'm': > opt.small_modes = true; > break; > - case 'i': > - opt.show_hidden = true; > - break; > case 't': > opt.step++; > break; > @@ -3113,7 +3110,6 @@ const char *help_str = > " --no-fbc-action-check Don't check for the FBC last action\n" > " --no-edp Don't use eDP monitors\n" > " --use-small-modes Use smaller resolutions for the modes\n" > -" --show-hidden Show hidden subtests\n" > " --step Stop on each step so you can check the screen\n" > " --shared-fb-x offset Use 'offset' as the X offset for the shared FB\n" > " --shared-fb-y offset Use 'offset' as the Y offset for the shared FB\n" > @@ -3227,18 +3223,19 @@ static const char *flip_str(enum flip_type flip) > for (t.plane = 0; t.plane < PLANE_COUNT; t.plane++) { \ > for (t.fbs = 0; t.fbs < FBS_COUNT; t.fbs++) { \ > for (t.method = 0; t.method < IGT_DRAW_METHOD_COUNT; t.method++) { \ > + t.subtest_flags = SUBTEST_TYPE_NORMAL; \ > if (t.pipes == PIPE_SINGLE && t.screen == SCREEN_SCND) \ > continue; \ > if (t.screen == SCREEN_OFFSCREEN && t.plane != PLANE_PRI) \ > continue; \ > - if (!opt.show_hidden && t.pipes == PIPE_DUAL && \ > + if (t.pipes == PIPE_DUAL && \ > t.screen == SCREEN_OFFSCREEN) \ > - continue; \ > - if (!opt.show_hidden && t.feature == FEATURE_NONE) \ > - continue; \ > - if (!opt.show_hidden && t.fbs == FBS_SHARED && \ > + t.subtest_flags = SUBTEST_TYPE_SLOW; \ > + if (t.feature == FEATURE_NONE) \ > + t.subtest_flags = SUBTEST_TYPE_SLOW; \ > + if (t.fbs == FBS_SHARED && \ > (t.plane == PLANE_CUR || t.plane == PLANE_SPR)) \ > - continue; > + t.subtest_flags = SUBTEST_TYPE_SLOW; > > > #define TEST_MODE_ITER_END } } } } } } > @@ -3253,7 +3250,6 @@ int main(int argc, char *argv[]) > { "no-fbc-action-check", 0, 0, 'a'}, > { "no-edp", 0, 0, 'e'}, > { "use-small-modes", 0, 0, 'm'}, > - { "show-hidden", 0, 0, 'i'}, > { "step", 0, 0, 't'}, > { "shared-fb-x", 1, 0, 'x'}, > { "shared-fb-y", 1, 0, 'y'}, > @@ -3269,8 +3265,9 @@ int main(int argc, char *argv[]) > setup_environment(); > > for (t.feature = 0; t.feature < FEATURE_COUNT; t.feature++) { > - if (!opt.show_hidden && t.feature == FEATURE_NONE) > - continue; > + t.subtest_flags = SUBTEST_TYPE_NORMAL; > + if (t.feature == FEATURE_NONE) > + t.subtest_flags = SUBTEST_TYPE_SLOW; > for (t.pipes = 0; t.pipes < PIPE_COUNT; t.pipes++) { > t.screen = SCREEN_PRIM; > t.plane = PLANE_PRI; > @@ -3280,38 +3277,43 @@ int main(int argc, char *argv[]) > t.flip = -1; > t.method = -1; > > - igt_subtest_f("%s-%s-rte", > - feature_str(t.feature), > - pipes_str(t.pipes)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-%s-rte", > + feature_str(t.feature), > + pipes_str(t.pipes)) > rte_subtest(&t); > } > } > > TEST_MODE_ITER_BEGIN(t) > - igt_subtest_f("%s-%s-%s-%s-%s-draw-%s", > - feature_str(t.feature), > - pipes_str(t.pipes), > - screen_str(t.screen), > - plane_str(t.plane), > - fbs_str(t.fbs), > - igt_draw_get_method_name(t.method)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-%s-%s-%s-%s-draw-%s", > + feature_str(t.feature), > + pipes_str(t.pipes), > + screen_str(t.screen), > + plane_str(t.plane), > + fbs_str(t.fbs), > + igt_draw_get_method_name(t.method)) > draw_subtest(&t); > TEST_MODE_ITER_END > > TEST_MODE_ITER_BEGIN(t) > if (t.plane != PLANE_PRI || > - t.screen == SCREEN_OFFSCREEN || > - (!opt.show_hidden && t.method != IGT_DRAW_BLT)) > + t.screen == SCREEN_OFFSCREEN) > continue; > > + if (t.method != IGT_DRAW_BLT) > + t.subtest_flags = SUBTEST_TYPE_SLOW; > + > for (t.flip = 0; t.flip < FLIP_COUNT; t.flip++) > - igt_subtest_f("%s-%s-%s-%s-%sflip-%s", > - feature_str(t.feature), > - pipes_str(t.pipes), > - screen_str(t.screen), > - fbs_str(t.fbs), > - flip_str(t.flip), > - igt_draw_get_method_name(t.method)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-%s-%s-%s-%sflip-%s", > + feature_str(t.feature), > + pipes_str(t.pipes), > + screen_str(t.screen), > + fbs_str(t.fbs), > + flip_str(t.flip), > + igt_draw_get_method_name(t.method)) > flip_subtest(&t); > TEST_MODE_ITER_END > > @@ -3322,10 +3324,11 @@ int main(int argc, char *argv[]) > (t.feature & FEATURE_FBC) == 0) > continue; > > - igt_subtest_f("%s-%s-%s-fliptrack", > - feature_str(t.feature), > - pipes_str(t.pipes), > - fbs_str(t.fbs)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-%s-%s-fliptrack", > + feature_str(t.feature), > + pipes_str(t.pipes), > + fbs_str(t.fbs)) > fliptrack_subtest(&t, FLIP_PAGEFLIP); > TEST_MODE_ITER_END > > @@ -3335,20 +3338,22 @@ int main(int argc, char *argv[]) > t.plane == PLANE_PRI) > continue; > > - igt_subtest_f("%s-%s-%s-%s-%s-move", > - feature_str(t.feature), > - pipes_str(t.pipes), > - screen_str(t.screen), > - plane_str(t.plane), > - fbs_str(t.fbs)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-%s-%s-%s-%s-move", > + feature_str(t.feature), > + pipes_str(t.pipes), > + screen_str(t.screen), > + plane_str(t.plane), > + fbs_str(t.fbs)) > move_subtest(&t); > > - igt_subtest_f("%s-%s-%s-%s-%s-onoff", > - feature_str(t.feature), > - pipes_str(t.pipes), > - screen_str(t.screen), > - plane_str(t.plane), > - fbs_str(t.fbs)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-%s-%s-%s-%s-onoff", > + feature_str(t.feature), > + pipes_str(t.pipes), > + screen_str(t.screen), > + plane_str(t.plane), > + fbs_str(t.fbs)) > onoff_subtest(&t); > TEST_MODE_ITER_END > > @@ -3358,27 +3363,31 @@ int main(int argc, char *argv[]) > t.plane != PLANE_SPR) > continue; > > - igt_subtest_f("%s-%s-%s-%s-%s-fullscreen", > - feature_str(t.feature), > - pipes_str(t.pipes), > - screen_str(t.screen), > - plane_str(t.plane), > - fbs_str(t.fbs)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-%s-%s-%s-%s-fullscreen", > + feature_str(t.feature), > + pipes_str(t.pipes), > + screen_str(t.screen), > + plane_str(t.plane), > + fbs_str(t.fbs)) > fullscreen_plane_subtest(&t); > TEST_MODE_ITER_END > > TEST_MODE_ITER_BEGIN(t) > if (t.screen != SCREEN_PRIM || > - t.method != IGT_DRAW_BLT || > - (!opt.show_hidden && t.plane != PLANE_PRI) || > - (!opt.show_hidden && t.fbs != FBS_INDIVIDUAL)) > + t.method != IGT_DRAW_BLT) > continue; > > - igt_subtest_f("%s-%s-%s-%s-multidraw", > - feature_str(t.feature), > - pipes_str(t.pipes), > - plane_str(t.plane), > - fbs_str(t.fbs)) > + if (t.plane != PLANE_PRI || > + t.fbs != FBS_INDIVIDUAL) > + t.subtest_flags = SUBTEST_TYPE_SLOW; > + > + igt_subtest_flags_f(t.subtest_flags, > + "%s-%s-%s-%s-multidraw", > + feature_str(t.feature), > + pipes_str(t.pipes), > + plane_str(t.plane), > + fbs_str(t.fbs)) > multidraw_subtest(&t); > TEST_MODE_ITER_END > > @@ -3390,7 +3399,9 @@ int main(int argc, char *argv[]) > t.method != IGT_DRAW_MMAP_GTT) > continue; > > - igt_subtest_f("%s-farfromfence", feature_str(t.feature)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-farfromfence", > + feature_str(t.feature)) > farfromfence_subtest(&t); > TEST_MODE_ITER_END > > @@ -3406,10 +3417,11 @@ int main(int argc, char *argv[]) > if (t.format == FORMAT_DEFAULT) > continue; > > - igt_subtest_f("%s-%s-draw-%s", > - feature_str(t.feature), > - format_str(t.format), > - igt_draw_get_method_name(t.method)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-%s-draw-%s", > + feature_str(t.feature), > + format_str(t.format), > + igt_draw_get_method_name(t.method)) > format_draw_subtest(&t); > } > TEST_MODE_ITER_END > @@ -3420,9 +3432,11 @@ int main(int argc, char *argv[]) > t.plane != PLANE_PRI || > t.method != IGT_DRAW_MMAP_CPU) > continue; > - igt_subtest_f("%s-%s-scaledprimary", > - feature_str(t.feature), > - fbs_str(t.fbs)) > + > + igt_subtest_flags_f(t.subtest_flags, > + "%s-%s-scaledprimary", > + feature_str(t.feature), > + fbs_str(t.fbs)) > scaledprimary_subtest(&t); > TEST_MODE_ITER_END > > @@ -3434,25 +3448,37 @@ int main(int argc, char *argv[]) > t.method != IGT_DRAW_MMAP_CPU) > continue; > > - igt_subtest_f("%s-modesetfrombusy", feature_str(t.feature)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-modesetfrombusy", > + feature_str(t.feature)) > modesetfrombusy_subtest(&t); > > if (t.feature & FEATURE_FBC) { > - igt_subtest_f("%s-badstride", feature_str(t.feature)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-badstride", > + feature_str(t.feature)) > badstride_subtest(&t); > > - igt_subtest_f("%s-stridechange", feature_str(t.feature)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-stridechange", > + feature_str(t.feature)) > stridechange_subtest(&t); > > - igt_subtest_f("%s-tilingchange", feature_str(t.feature)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-tilingchange", > + feature_str(t.feature)) > tilingchange_subtest(&t); > } > > if (t.feature & FEATURE_PSR) > - igt_subtest_f("%s-slowdraw", feature_str(t.feature)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-slowdraw", > + feature_str(t.feature)) > slow_draw_subtest(&t); > > - igt_subtest_f("%s-suspend", feature_str(t.feature)) > + igt_subtest_flags_f(t.subtest_flags, > + "%s-suspend", > + feature_str(t.feature)) > suspend_subtest(&t); > TEST_MODE_ITER_END > > -- > 2.7.0 > > _______________________________________________ > Intel-gfx mailing list > Intel-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/intel-gfx
On Tue, Feb 16, 2016 at 04:45:26PM +0100, Daniel Vetter wrote: > On Thu, Feb 11, 2016 at 01:09:33PM +0200, David Weinehall wrote: > > Some subtests are not run by default, for various reasons; > > be it because they're only for debugging, because they're slow, > > or because they are not of high enough quality. > > > > This patch aims to introduce a common mechanism for categorising > > the subtests and introduces a flag (--all) that runs/lists all > > subtests instead of just the default set. > > Is the idea to also add a --test-flags interface? How is this meant to > integrate with the current BAT runtime? Also we need to figure out > how/whether we need to change the spec for how testrunners are supposed to > run igt tests. Plus obviously patch igt.py in piglit. > > I think I like where this is going, but I guess needs a bit more polish > still. Integrating this into how we run/select BAT would be a great > showcase of your concept I think. The idea is to also add a --test-flags interface, yes, but I deemed it outside the scope of this patchset. The origin of this patch is a suggestion which I believe was from you -- to handle combinatorial tests in one unified way instead of using various different methods (--all, vs different test invocation names). Since the first patch got feedback that not all excluded subtests were because they were slow I introduced the flags concept (the patch still classifies everything non-default as slow though, seeing as I'd rather leave such classification to the test authors). Since neither BAT nor piglit at the moment uses gem_concurrent_blit[1] or kms_frontbuffer_tracking --all, I don't think there's any need to patch either of those at this point. Until there's some agreement on a reasonable subset of gem_concurrent_blit subtests that can be run within at most 30 seconds, there's no possibility to have it included in BAT or piglit -- currently even the default set of subtests (without passing --all) will take several hours to complete. There are a bunch of subtests that won't complete within 30 seconds even when run on their own[2], which obviously complicates things. FWIW, both Skylake and Broadwell hard-hangs when running gem_concurrent_blit --all (and up until that point spews a mighty amount of FAIL). Kind regards, David [1] gem_concurrent_blit is currently not run at all due to it being too slow. At the very least it would be nice to have a dedicated machine that ran it on a daily basis. Having weekly runs with --all should be a longer-term goal, but with the current hangs and massive amount of FAILs that is not feasible yet. [2] Notably the "swap-*" subtests.
diff --git a/lib/igt_core.c b/lib/igt_core.c index 6b69bb780700..e6e6949ed65a 100644 --- a/lib/igt_core.c +++ b/lib/igt_core.c @@ -216,6 +216,7 @@ const char *igt_interactive_debug; /* subtests helpers */ static bool list_subtests = false; +static unsigned int subtest_types_mask = SUBTEST_TYPE_NORMAL; static char *run_single_subtest = NULL; static bool run_single_subtest_found = false; static const char *in_subtest = NULL; @@ -237,12 +238,13 @@ int test_children_sz; bool test_child; enum { - OPT_LIST_SUBTESTS, - OPT_RUN_SUBTEST, - OPT_DESCRIPTION, - OPT_DEBUG, - OPT_INTERACTIVE_DEBUG, - OPT_HELP = 'h' + OPT_LIST_SUBTESTS, + OPT_WITH_ALL_SUBTESTS, + OPT_RUN_SUBTEST, + OPT_DESCRIPTION, + OPT_DEBUG, + OPT_INTERACTIVE_DEBUG, + OPT_HELP = 'h' }; static int igt_exitcode = IGT_EXIT_SUCCESS; @@ -516,6 +518,7 @@ static void print_usage(const char *help_str, bool output_on_stderr) fprintf(f, "Usage: %s [OPTIONS]\n", command_str); fprintf(f, " --list-subtests\n" + " --all\n" " --run-subtest <pattern>\n" " --debug[=log-domain]\n" " --interactive-debug[=domain]\n" @@ -548,6 +551,7 @@ static int common_init(int *argc, char **argv, int c, option_index = 0, i, x; static struct option long_options[] = { {"list-subtests", 0, 0, OPT_LIST_SUBTESTS}, + {"all", 0, 0, OPT_WITH_ALL_SUBTESTS}, {"run-subtest", 1, 0, OPT_RUN_SUBTEST}, {"help-description", 0, 0, OPT_DESCRIPTION}, {"debug", optional_argument, 0, OPT_DEBUG}, @@ -659,6 +663,10 @@ static int common_init(int *argc, char **argv, if (!run_single_subtest) list_subtests = true; break; + case OPT_WITH_ALL_SUBTESTS: + if (!run_single_subtest) + subtest_types_mask = SUBTEST_TYPE_ALL; + break; case OPT_RUN_SUBTEST: if (!list_subtests) run_single_subtest = strdup(optarg); @@ -1667,6 +1675,29 @@ void igt_skip_on_simulation(void) igt_require(!igt_run_in_simulation()); } +/** + * igt_match_subtest_flags: + * + * This function is used to check whether the attributes of the subtest + * makes it a candidate for inclusion in the test run; this is used to + * categorise tests, for instance to exclude tests that are purely for + * debug purposes, tests that are specific to certain environments, + * or tests that are very slow. + * + * Note that a test has to have all its flags met to be run; for instance + * a subtest with the flags SUBTEST_TYPE_SLOW | SUBTEST_TYPE_DEBUG requires + * "--subtest-types=slow,debug" or "--all" to be executed + * + * @subtest_flags: The subtests to check for + * + * Returns: true if the subtest test should be run, + * false if the subtest should be skipped + */ +bool igt_match_subtest_flags(unsigned long subtest_flags) +{ + return ((subtest_flags & subtest_types_mask) == subtest_flags); +} + /* structured logging */ /** diff --git a/lib/igt_core.h b/lib/igt_core.h index 8f297e06a068..bf83de609bfa 100644 --- a/lib/igt_core.h +++ b/lib/igt_core.h @@ -193,6 +193,48 @@ bool __igt_run_subtest(const char *subtest_name); #define igt_subtest_f(f...) \ __igt_subtest_f(igt_tokencat(__tmpchar, __LINE__), f) +enum { + /* The set of tests run if nothing else is specified */ + SUBTEST_TYPE_NORMAL = 1 << 0, + /* Basic Acceptance Testing set */ + SUBTEST_TYPE_BASIC = 1 << 1, + /* Tests that are very slow */ + SUBTEST_TYPE_SLOW = 1 << 2, + /* Tests that mainly intended for debugging */ + SUBTEST_TYPE_DEBUG = 1 << 3, + SUBTEST_TYPE_ALL = ~0 +} subtest_types; + +bool igt_match_subtest_flags(unsigned long subtest_flags); + +/** + * igt_subtest_flags: + * @name: name of the subtest + * @__subtest_flags: the categories the subtest belongs to + * + * This is a wrapper around igt_subtest that will only execute the + * testcase if all of the flags passed to this function match those + * specified by the list of subtest categories passed from the + * command line; the default category is SUBTEST_TYPE_NORMAL. + */ +#define igt_subtest_flags(name, __subtest_flags) \ + if (igt_match_subtest_flags(__subtest_flags)) \ + igt_subtest(name) + +/** + * igt_subtest_flags_f: + * @...: format string and optional arguments + * @__subtest_flags: the categories the subtest belongs to + * + * This is a wrapper around igt_subtest_f that will only execute the + * testcase if all of the flags passed to this function match those + * specified by the list of subtest categories passed from the + * command line; the default category is SUBTEST_TYPE_NORMAL. + */ +#define igt_subtest_flags_f(__subtest_flags, f...) \ + if (igt_match_subtest_flags(__subtest_flags)) \ + __igt_subtest_f(igt_tokencat(__tmpchar, __LINE__), f) + const char *igt_subtest_name(void); bool igt_only_list_subtests(void); diff --git a/tests/gem_concurrent_blit.c b/tests/gem_concurrent_blit.c index 9b7ef8700e31..3c23a95c4631 100644 --- a/tests/gem_concurrent_blit.c +++ b/tests/gem_concurrent_blit.c @@ -64,7 +64,6 @@ struct local_i915_gem_userptr { int fd, devid, gen; struct intel_batchbuffer *batch; -int all; int pass; struct buffers { @@ -1256,10 +1255,14 @@ run_basic_modes(const char *prefix, }, *h; for (h = hangs; h->suffix; h++) { - if (!all && *h->suffix) - continue; + unsigned int subtest_flags; - for (p = all ? pipelines : pskip; p->prefix; p++) { + if (*h->suffix) + subtest_flags = SUBTEST_TYPE_SLOW; + else + subtest_flags = SUBTEST_TYPE_NORMAL; + + for (p = igt_match_subtest_flags(SUBTEST_TYPE_SLOW) ? pipelines : pskip; p->prefix; p++) { struct buffers buffers; igt_fixture @@ -1267,21 +1270,21 @@ run_basic_modes(const char *prefix, 512, 512, fd, run_wrap_func != run_child); - igt_subtest_f("%s-%s-%s-sanitycheck0%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-sanitycheck0%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); buffers_create(&buffers, num_buffers); run_wrap_func(&buffers, do_basic0, p->copy, h->hang); } - igt_subtest_f("%s-%s-%s-sanitycheck1%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-sanitycheck1%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); buffers_create(&buffers, num_buffers); run_wrap_func(&buffers, do_basic1, p->copy, h->hang); } - igt_subtest_f("%s-%s-%s-sanitycheckN%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-sanitycheckN%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); buffers_create(&buffers, num_buffers); run_wrap_func(&buffers, do_basicN, @@ -1289,7 +1292,7 @@ run_basic_modes(const char *prefix, } /* try to overwrite the source values */ - igt_subtest_f("%s-%s-%s-overwrite-source-one%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-overwrite-source-one%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); buffers_create(&buffers, num_buffers); run_wrap_func(&buffers, @@ -1297,7 +1300,7 @@ run_basic_modes(const char *prefix, p->copy, h->hang); } - igt_subtest_f("%s-%s-%s-overwrite-source%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-overwrite-source%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); buffers_create(&buffers, num_buffers); run_wrap_func(&buffers, @@ -1305,7 +1308,7 @@ run_basic_modes(const char *prefix, p->copy, h->hang); } - igt_subtest_f("%s-%s-%s-overwrite-source-read-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-overwrite-source-read-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); buffers_create(&buffers, num_buffers); run_wrap_func(&buffers, @@ -1313,7 +1316,7 @@ run_basic_modes(const char *prefix, p->copy, h->hang); } - igt_subtest_f("%s-%s-%s-overwrite-source-read-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-overwrite-source-read-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); igt_require(rendercopy); buffers_create(&buffers, num_buffers); @@ -1322,7 +1325,7 @@ run_basic_modes(const char *prefix, p->copy, h->hang); } - igt_subtest_f("%s-%s-%s-overwrite-source-rev%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-overwrite-source-rev%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); buffers_create(&buffers, num_buffers); run_wrap_func(&buffers, @@ -1331,7 +1334,7 @@ run_basic_modes(const char *prefix, } /* try to intermix copies with GPU copies*/ - igt_subtest_f("%s-%s-%s-intermix-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-intermix-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); igt_require(rendercopy); buffers_create(&buffers, num_buffers); @@ -1339,7 +1342,7 @@ run_basic_modes(const char *prefix, do_intermix_rcs, p->copy, h->hang); } - igt_subtest_f("%s-%s-%s-intermix-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-intermix-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); igt_require(rendercopy); buffers_create(&buffers, num_buffers); @@ -1347,7 +1350,7 @@ run_basic_modes(const char *prefix, do_intermix_bcs, p->copy, h->hang); } - igt_subtest_f("%s-%s-%s-intermix-both%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-intermix-both%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); igt_require(rendercopy); buffers_create(&buffers, num_buffers); @@ -1357,7 +1360,7 @@ run_basic_modes(const char *prefix, } /* try to read the results before the copy completes */ - igt_subtest_f("%s-%s-%s-early-read%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-early-read%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); buffers_create(&buffers, num_buffers); run_wrap_func(&buffers, @@ -1366,14 +1369,14 @@ run_basic_modes(const char *prefix, } /* concurrent reads */ - igt_subtest_f("%s-%s-%s-read-read-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-read-read-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); buffers_create(&buffers, num_buffers); run_wrap_func(&buffers, do_read_read_bcs, p->copy, h->hang); } - igt_subtest_f("%s-%s-%s-read-read-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-read-read-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); igt_require(rendercopy); buffers_create(&buffers, num_buffers); @@ -1383,14 +1386,14 @@ run_basic_modes(const char *prefix, } /* split copying between rings */ - igt_subtest_f("%s-%s-%s-write-read-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-write-read-bcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); buffers_create(&buffers, num_buffers); run_wrap_func(&buffers, do_write_read_bcs, p->copy, h->hang); } - igt_subtest_f("%s-%s-%s-write-read-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-write-read-rcs%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); igt_require(rendercopy); buffers_create(&buffers, num_buffers); @@ -1399,8 +1402,8 @@ run_basic_modes(const char *prefix, p->copy, h->hang); } - /* and finally try to trick the kernel into loosing the pending write */ - igt_subtest_f("%s-%s-%s-gpu-read-after-write%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { + /* and finally try to trick the kernel into losing the pending write */ + igt_subtest_flags_f(subtest_flags, "%s-%s-%s-gpu-read-after-write%s%s", prefix, mode->name, p->prefix, suffix, h->suffix) { p->require(); buffers_create(&buffers, num_buffers); run_wrap_func(&buffers, @@ -1454,9 +1457,6 @@ igt_main igt_skip_on_simulation(); - if (strstr(igt_test_name(), "all")) - all = true; - igt_fixture { fd = drm_open_driver(DRIVER_INTEL); intel_detect_and_clear_missed_interrupts(fd); diff --git a/tests/kms_frontbuffer_tracking.c b/tests/kms_frontbuffer_tracking.c index 64f880c667a3..651303e9d392 100644 --- a/tests/kms_frontbuffer_tracking.c +++ b/tests/kms_frontbuffer_tracking.c @@ -47,8 +47,7 @@ IGT_TEST_DESCRIPTION("Test the Kernel's frontbuffer tracking mechanism and " * combinations that are somewhat redundant and don't add much value to the * test. For example, since we already do the offscreen testing with a single * pipe enabled, there's no much value in doing it again with dual pipes. If you - * still want to try these redundant tests, you need to use the --show-hidden - * option. + * still want to try these redundant tests, you need to use the --all option. * * The most important hidden thing is the FEATURE_NONE set of tests. Whenever * you get a failure on any test, it is important to check whether the same test @@ -126,6 +125,9 @@ struct test_mode { } flip; enum igt_draw_method method; + + /* Specifies the subtest categories this subtest belongs to */ + unsigned long subtest_flags; }; enum color { @@ -241,7 +243,6 @@ struct { bool fbc_check_last_action; bool no_edp; bool small_modes; - bool show_hidden; int step; int only_pipes; int shared_fb_x_offset; @@ -253,7 +254,6 @@ struct { .fbc_check_last_action = true, .no_edp = false, .small_modes = false, - .show_hidden= false, .step = 0, .only_pipes = PIPE_COUNT, .shared_fb_x_offset = 500, @@ -3075,9 +3075,6 @@ static int opt_handler(int option, int option_index, void *data) case 'm': opt.small_modes = true; break; - case 'i': - opt.show_hidden = true; - break; case 't': opt.step++; break; @@ -3113,7 +3110,6 @@ const char *help_str = " --no-fbc-action-check Don't check for the FBC last action\n" " --no-edp Don't use eDP monitors\n" " --use-small-modes Use smaller resolutions for the modes\n" -" --show-hidden Show hidden subtests\n" " --step Stop on each step so you can check the screen\n" " --shared-fb-x offset Use 'offset' as the X offset for the shared FB\n" " --shared-fb-y offset Use 'offset' as the Y offset for the shared FB\n" @@ -3227,18 +3223,19 @@ static const char *flip_str(enum flip_type flip) for (t.plane = 0; t.plane < PLANE_COUNT; t.plane++) { \ for (t.fbs = 0; t.fbs < FBS_COUNT; t.fbs++) { \ for (t.method = 0; t.method < IGT_DRAW_METHOD_COUNT; t.method++) { \ + t.subtest_flags = SUBTEST_TYPE_NORMAL; \ if (t.pipes == PIPE_SINGLE && t.screen == SCREEN_SCND) \ continue; \ if (t.screen == SCREEN_OFFSCREEN && t.plane != PLANE_PRI) \ continue; \ - if (!opt.show_hidden && t.pipes == PIPE_DUAL && \ + if (t.pipes == PIPE_DUAL && \ t.screen == SCREEN_OFFSCREEN) \ - continue; \ - if (!opt.show_hidden && t.feature == FEATURE_NONE) \ - continue; \ - if (!opt.show_hidden && t.fbs == FBS_SHARED && \ + t.subtest_flags = SUBTEST_TYPE_SLOW; \ + if (t.feature == FEATURE_NONE) \ + t.subtest_flags = SUBTEST_TYPE_SLOW; \ + if (t.fbs == FBS_SHARED && \ (t.plane == PLANE_CUR || t.plane == PLANE_SPR)) \ - continue; + t.subtest_flags = SUBTEST_TYPE_SLOW; #define TEST_MODE_ITER_END } } } } } } @@ -3253,7 +3250,6 @@ int main(int argc, char *argv[]) { "no-fbc-action-check", 0, 0, 'a'}, { "no-edp", 0, 0, 'e'}, { "use-small-modes", 0, 0, 'm'}, - { "show-hidden", 0, 0, 'i'}, { "step", 0, 0, 't'}, { "shared-fb-x", 1, 0, 'x'}, { "shared-fb-y", 1, 0, 'y'}, @@ -3269,8 +3265,9 @@ int main(int argc, char *argv[]) setup_environment(); for (t.feature = 0; t.feature < FEATURE_COUNT; t.feature++) { - if (!opt.show_hidden && t.feature == FEATURE_NONE) - continue; + t.subtest_flags = SUBTEST_TYPE_NORMAL; + if (t.feature == FEATURE_NONE) + t.subtest_flags = SUBTEST_TYPE_SLOW; for (t.pipes = 0; t.pipes < PIPE_COUNT; t.pipes++) { t.screen = SCREEN_PRIM; t.plane = PLANE_PRI; @@ -3280,38 +3277,43 @@ int main(int argc, char *argv[]) t.flip = -1; t.method = -1; - igt_subtest_f("%s-%s-rte", - feature_str(t.feature), - pipes_str(t.pipes)) + igt_subtest_flags_f(t.subtest_flags, + "%s-%s-rte", + feature_str(t.feature), + pipes_str(t.pipes)) rte_subtest(&t); } } TEST_MODE_ITER_BEGIN(t) - igt_subtest_f("%s-%s-%s-%s-%s-draw-%s", - feature_str(t.feature), - pipes_str(t.pipes), - screen_str(t.screen), - plane_str(t.plane), - fbs_str(t.fbs), - igt_draw_get_method_name(t.method)) + igt_subtest_flags_f(t.subtest_flags, + "%s-%s-%s-%s-%s-draw-%s", + feature_str(t.feature), + pipes_str(t.pipes), + screen_str(t.screen), + plane_str(t.plane), + fbs_str(t.fbs), + igt_draw_get_method_name(t.method)) draw_subtest(&t); TEST_MODE_ITER_END TEST_MODE_ITER_BEGIN(t) if (t.plane != PLANE_PRI || - t.screen == SCREEN_OFFSCREEN || - (!opt.show_hidden && t.method != IGT_DRAW_BLT)) + t.screen == SCREEN_OFFSCREEN) continue; + if (t.method != IGT_DRAW_BLT) + t.subtest_flags = SUBTEST_TYPE_SLOW; + for (t.flip = 0; t.flip < FLIP_COUNT; t.flip++) - igt_subtest_f("%s-%s-%s-%s-%sflip-%s", - feature_str(t.feature), - pipes_str(t.pipes), - screen_str(t.screen), - fbs_str(t.fbs), - flip_str(t.flip), - igt_draw_get_method_name(t.method)) + igt_subtest_flags_f(t.subtest_flags, + "%s-%s-%s-%s-%sflip-%s", + feature_str(t.feature), + pipes_str(t.pipes), + screen_str(t.screen), + fbs_str(t.fbs), + flip_str(t.flip), + igt_draw_get_method_name(t.method)) flip_subtest(&t); TEST_MODE_ITER_END @@ -3322,10 +3324,11 @@ int main(int argc, char *argv[]) (t.feature & FEATURE_FBC) == 0) continue; - igt_subtest_f("%s-%s-%s-fliptrack", - feature_str(t.feature), - pipes_str(t.pipes), - fbs_str(t.fbs)) + igt_subtest_flags_f(t.subtest_flags, + "%s-%s-%s-fliptrack", + feature_str(t.feature), + pipes_str(t.pipes), + fbs_str(t.fbs)) fliptrack_subtest(&t, FLIP_PAGEFLIP); TEST_MODE_ITER_END @@ -3335,20 +3338,22 @@ int main(int argc, char *argv[]) t.plane == PLANE_PRI) continue; - igt_subtest_f("%s-%s-%s-%s-%s-move", - feature_str(t.feature), - pipes_str(t.pipes), - screen_str(t.screen), - plane_str(t.plane), - fbs_str(t.fbs)) + igt_subtest_flags_f(t.subtest_flags, + "%s-%s-%s-%s-%s-move", + feature_str(t.feature), + pipes_str(t.pipes), + screen_str(t.screen), + plane_str(t.plane), + fbs_str(t.fbs)) move_subtest(&t); - igt_subtest_f("%s-%s-%s-%s-%s-onoff", - feature_str(t.feature), - pipes_str(t.pipes), - screen_str(t.screen), - plane_str(t.plane), - fbs_str(t.fbs)) + igt_subtest_flags_f(t.subtest_flags, + "%s-%s-%s-%s-%s-onoff", + feature_str(t.feature), + pipes_str(t.pipes), + screen_str(t.screen), + plane_str(t.plane), + fbs_str(t.fbs)) onoff_subtest(&t); TEST_MODE_ITER_END @@ -3358,27 +3363,31 @@ int main(int argc, char *argv[]) t.plane != PLANE_SPR) continue; - igt_subtest_f("%s-%s-%s-%s-%s-fullscreen", - feature_str(t.feature), - pipes_str(t.pipes), - screen_str(t.screen), - plane_str(t.plane), - fbs_str(t.fbs)) + igt_subtest_flags_f(t.subtest_flags, + "%s-%s-%s-%s-%s-fullscreen", + feature_str(t.feature), + pipes_str(t.pipes), + screen_str(t.screen), + plane_str(t.plane), + fbs_str(t.fbs)) fullscreen_plane_subtest(&t); TEST_MODE_ITER_END TEST_MODE_ITER_BEGIN(t) if (t.screen != SCREEN_PRIM || - t.method != IGT_DRAW_BLT || - (!opt.show_hidden && t.plane != PLANE_PRI) || - (!opt.show_hidden && t.fbs != FBS_INDIVIDUAL)) + t.method != IGT_DRAW_BLT) continue; - igt_subtest_f("%s-%s-%s-%s-multidraw", - feature_str(t.feature), - pipes_str(t.pipes), - plane_str(t.plane), - fbs_str(t.fbs)) + if (t.plane != PLANE_PRI || + t.fbs != FBS_INDIVIDUAL) + t.subtest_flags = SUBTEST_TYPE_SLOW; + + igt_subtest_flags_f(t.subtest_flags, + "%s-%s-%s-%s-multidraw", + feature_str(t.feature), + pipes_str(t.pipes), + plane_str(t.plane), + fbs_str(t.fbs)) multidraw_subtest(&t); TEST_MODE_ITER_END @@ -3390,7 +3399,9 @@ int main(int argc, char *argv[]) t.method != IGT_DRAW_MMAP_GTT) continue; - igt_subtest_f("%s-farfromfence", feature_str(t.feature)) + igt_subtest_flags_f(t.subtest_flags, + "%s-farfromfence", + feature_str(t.feature)) farfromfence_subtest(&t); TEST_MODE_ITER_END @@ -3406,10 +3417,11 @@ int main(int argc, char *argv[]) if (t.format == FORMAT_DEFAULT) continue; - igt_subtest_f("%s-%s-draw-%s", - feature_str(t.feature), - format_str(t.format), - igt_draw_get_method_name(t.method)) + igt_subtest_flags_f(t.subtest_flags, + "%s-%s-draw-%s", + feature_str(t.feature), + format_str(t.format), + igt_draw_get_method_name(t.method)) format_draw_subtest(&t); } TEST_MODE_ITER_END @@ -3420,9 +3432,11 @@ int main(int argc, char *argv[]) t.plane != PLANE_PRI || t.method != IGT_DRAW_MMAP_CPU) continue; - igt_subtest_f("%s-%s-scaledprimary", - feature_str(t.feature), - fbs_str(t.fbs)) + + igt_subtest_flags_f(t.subtest_flags, + "%s-%s-scaledprimary", + feature_str(t.feature), + fbs_str(t.fbs)) scaledprimary_subtest(&t); TEST_MODE_ITER_END @@ -3434,25 +3448,37 @@ int main(int argc, char *argv[]) t.method != IGT_DRAW_MMAP_CPU) continue; - igt_subtest_f("%s-modesetfrombusy", feature_str(t.feature)) + igt_subtest_flags_f(t.subtest_flags, + "%s-modesetfrombusy", + feature_str(t.feature)) modesetfrombusy_subtest(&t); if (t.feature & FEATURE_FBC) { - igt_subtest_f("%s-badstride", feature_str(t.feature)) + igt_subtest_flags_f(t.subtest_flags, + "%s-badstride", + feature_str(t.feature)) badstride_subtest(&t); - igt_subtest_f("%s-stridechange", feature_str(t.feature)) + igt_subtest_flags_f(t.subtest_flags, + "%s-stridechange", + feature_str(t.feature)) stridechange_subtest(&t); - igt_subtest_f("%s-tilingchange", feature_str(t.feature)) + igt_subtest_flags_f(t.subtest_flags, + "%s-tilingchange", + feature_str(t.feature)) tilingchange_subtest(&t); } if (t.feature & FEATURE_PSR) - igt_subtest_f("%s-slowdraw", feature_str(t.feature)) + igt_subtest_flags_f(t.subtest_flags, + "%s-slowdraw", + feature_str(t.feature)) slow_draw_subtest(&t); - igt_subtest_f("%s-suspend", feature_str(t.feature)) + igt_subtest_flags_f(t.subtest_flags, + "%s-suspend", + feature_str(t.feature)) suspend_subtest(&t); TEST_MODE_ITER_END
Some subtests are not run by default, for various reasons; be it because they're only for debugging, because they're slow, or because they are not of high enough quality. This patch aims to introduce a common mechanism for categorising the subtests and introduces a flag (--all) that runs/lists all subtests instead of just the default set. Signed-off-by: David Weinehall <david.weinehall@linux.intel.com> --- lib/igt_core.c | 43 +++++++-- lib/igt_core.h | 42 +++++++++ tests/gem_concurrent_blit.c | 50 +++++------ tests/kms_frontbuffer_tracking.c | 186 ++++++++++++++++++++++----------------- 4 files changed, 210 insertions(+), 111 deletions(-)