| Message ID | 20221028210256.3776835-1-dlatypov@google.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 3ffdcf7e3b7dff04b055771c03c9646aa383cc1e |
| Delegated to: | Brendan Higgins |
| Headers | show |
| Series | [1/2] kunit: tool: make unit test not print parsed testdata to stdout | expand |
On Sat, Oct 29, 2022 at 5:03 AM Daniel Latypov <dlatypov@google.com> wrote: > > Currently, if you run > $ ./tools/testing/kunit/kunit_tool_test.py > you'll see a lot of output from the parser as we feed it testdata. > > This makes the output hard to read and fairly confusing, esp. since our > testdata includes example failures, which get printed out in red. > > Silence that output so real failures are easier to see. > > Signed-off-by: Daniel Latypov <dlatypov@google.com> > --- Thanks -- this has been annoying me for ages. That being said, this isn't a perfect fix, the "usage" text and "Reconfiguring .config" still show up for me: --- davidgow@slicestar:~/Development/linux-kselftest$ ./tools/testing/kunit/kunit_tool_test.py ..............................usage: kunit_tool_test.py run [-h] [--build_dir DIR] [--make_options X=Y] [--alltests] [--kunitconfig PATHS] [--kconfig_add CONFIG_X=Y] [--arch ARCH] [--cross_compile PREFIX] [--qemu_config FILE] [--qemu_ar gs] [--jobs N] [--timeout SECONDS] [--kernel_args] [--run_isolated {suite,test}] [--raw_output [{all,kunit}]] [--json [FILE]] [filter_glob] kunit_tool_test.py run: error: argument --raw_output: invalid choice: 'invalid' (choose from 'all', 'kunit') ..............................Generating .config ... .Regenerating .config ... ......... ---------------------------------------------------------------------- Ran 70 tests in 0.232s OK --- That's still a significant improvement on what we had before, though, so: Reviewed-by: David Gow <davidgow@google.com> Cheers, -- David > tools/testing/kunit/kunit_tool_test.py | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py > index e2cd2cc2e98f..a6e53945656e 100755 > --- a/tools/testing/kunit/kunit_tool_test.py > +++ b/tools/testing/kunit/kunit_tool_test.py > @@ -80,6 +80,9 @@ class KconfigTest(unittest.TestCase): > self.assertEqual(actual_kconfig, expected_kconfig) > > class KUnitParserTest(unittest.TestCase): > + def setUp(self): > + self.print_mock = mock.patch('kunit_printer.Printer.print').start() > + self.addCleanup(mock.patch.stopall) > > def assertContains(self, needle: str, haystack: kunit_parser.LineStream): > # Clone the iterator so we can print the contents on failure. > @@ -485,6 +488,9 @@ class LinuxSourceTreeTest(unittest.TestCase): > > > class KUnitJsonTest(unittest.TestCase): > + def setUp(self): > + self.print_mock = mock.patch('kunit_printer.Printer.print').start() > + self.addCleanup(mock.patch.stopall) > > def _json_for(self, log_file): > with open(test_data_path(log_file)) as file: > > base-commit: 8f8b51f7d5c8bd3a89e7ea87aed2cdaa52ca5ba4 > -- > 2.38.1.273.g43a17bfeac-goog >
On Sat, Oct 29, 2022 at 8:31 PM David Gow <davidgow@google.com> wrote: > > On Sat, Oct 29, 2022 at 5:03 AM Daniel Latypov <dlatypov@google.com> wrote: > > > > Currently, if you run > > $ ./tools/testing/kunit/kunit_tool_test.py > > you'll see a lot of output from the parser as we feed it testdata. > > > > This makes the output hard to read and fairly confusing, esp. since our > > testdata includes example failures, which get printed out in red. > > > > Silence that output so real failures are easier to see. > > > > Signed-off-by: Daniel Latypov <dlatypov@google.com> > > --- > > Thanks -- this has been annoying me for ages. > > That being said, this isn't a perfect fix, the "usage" text and > "Reconfiguring .config" still show up for me: > --- > davidgow@slicestar:~/Development/linux-kselftest$ > ./tools/testing/kunit/kunit_tool_test.py > ..............................usage: kunit_tool_test.py run [-h] > [--build_dir DIR] [--make_options X=Y] [--alltests] [--kunitconfig > PATHS] [--kconfig_add CONFIG_X=Y] [--arch ARCH] [--cross_compile > PREFIX] [--qemu_config FILE] [--qemu_ar > gs] [--jobs N] > [--timeout SECONDS] [--kernel_args] > [--run_isolated {suite,test}] [--raw_output [{all,kunit}]] [--json > [FILE]] > [filter_glob] > kunit_tool_test.py run: error: argument --raw_output: invalid choice: > 'invalid' (choose from 'all', 'kunit') > ..............................Generating .config ... > .Regenerating .config ... > ......... > ---------------------------------------------------------------------- > Ran 70 tests in 0.232s > > OK > --- > > That's still a significant improvement on what we had before, though, so: Yeah, I was originally going to make this patch silence all the output. But I figured I would focus this patch on just the code using our kunit_printer abstraction, aka the parser. We could address the other ones, but we'd have to mock out the built-in print function. As someone who is guilty of print(f) debugging, doing that means I no longer have an easy of way of adding debugging code to those test cases :P
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index e2cd2cc2e98f..a6e53945656e 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -80,6 +80,9 @@ class KconfigTest(unittest.TestCase): self.assertEqual(actual_kconfig, expected_kconfig) class KUnitParserTest(unittest.TestCase): + def setUp(self): + self.print_mock = mock.patch('kunit_printer.Printer.print').start() + self.addCleanup(mock.patch.stopall) def assertContains(self, needle: str, haystack: kunit_parser.LineStream): # Clone the iterator so we can print the contents on failure. @@ -485,6 +488,9 @@ class LinuxSourceTreeTest(unittest.TestCase): class KUnitJsonTest(unittest.TestCase): + def setUp(self): + self.print_mock = mock.patch('kunit_printer.Printer.print').start() + self.addCleanup(mock.patch.stopall) def _json_for(self, log_file): with open(test_data_path(log_file)) as file:
Currently, if you run $ ./tools/testing/kunit/kunit_tool_test.py you'll see a lot of output from the parser as we feed it testdata. This makes the output hard to read and fairly confusing, esp. since our testdata includes example failures, which get printed out in red. Silence that output so real failures are easier to see. Signed-off-by: Daniel Latypov <dlatypov@google.com> --- tools/testing/kunit/kunit_tool_test.py | 6 ++++++ 1 file changed, 6 insertions(+) base-commit: 8f8b51f7d5c8bd3a89e7ea87aed2cdaa52ca5ba4