[0/2] Add support for inlined documentation for kunit and kselftests

Message

Mauro Carvalho Chehab Sept. 1, 2023, 6:51 a.m. UTC

This is a follow-up of the discussions taken here:

   https://lore.kernel.org/linux-doc/20230704132812.02ba97ba@maurocar-mobl2/T/#t

I sent a previous version as RFC. This is basically what we had there, with some
improvements at test_list.py.

It adds a new extension that allows documenting tests using the same tool we're
using for DRM unit tests at IGT GPU tools: https://gitlab.freedesktop.org/drm/igt-gpu-tools.

While kernel-doc has provided documentation for in-lined functions/struct comments,
it was not meant to document tests.

Tests need to be grouped by the test functions. It should also be possible to produce
other outputs from the documentation, to integrate it with test suites. For instance, 
Internally at Intel, we use the comments to generate DOT files hierarchically grouped
per feature categories.

This is meant to be an initial series to start documenting kunit.

It comes with a documenation for a kunit suite from drm at drm_buddy_test.c.

My plan is that, once we get this merged, start documenting other unit tests and work
to add extra requirements at the tool needed by Linux Kernel.

On this series, we have:

- patch 1:
  adds test_list.py - which is identical to the file with the same name I developed for
  IGT tree;
- patch 2:
  adds  test documentation from a single DRM kunit file (drm_buddy_test.c).

Mauro Carvalho Chehab (2):
  docs: add support for documenting kUnit and kSelftests
  drm: add documentation for drm_buddy_test kUnit test

 Documentation/conf.py                  |    2 +-
 Documentation/index.rst                |    2 +-
 Documentation/sphinx/test_kdoc.py      |  108 ++
 Documentation/sphinx/test_list.py      | 1314 ++++++++++++++++++++++++
 Documentation/tests/index.rst          |    6 +
 Documentation/tests/kunit.rst          |    5 +
 drivers/gpu/drm/tests/drm_buddy_test.c |   12 +
 7 files changed, 1447 insertions(+), 2 deletions(-)
 create mode 100755 Documentation/sphinx/test_kdoc.py
 create mode 100644 Documentation/sphinx/test_list.py
 create mode 100644 Documentation/tests/index.rst
 create mode 100644 Documentation/tests/kunit.rst

Comments

Jonathan Corbet Oct. 3, 2023, 5 p.m. UTC | #1

Mauro Carvalho Chehab <[e-mail]> writes:

> This is a follow-up of the discussions taken here:
>
>    https://lore.kernel.org/linux-doc/20230704132812.02ba97ba@maurocar-mobl2/T/#t
>
> I sent a previous version as RFC. This is basically what we had there, with some
> improvements at test_list.py.
>
> It adds a new extension that allows documenting tests using the same tool we're
> using for DRM unit tests at IGT GPU tools: https://gitlab.freedesktop.org/drm/igt-gpu-tools.
>
> While kernel-doc has provided documentation for in-lined functions/struct comments,
> it was not meant to document tests.
>
> Tests need to be grouped by the test functions. It should also be possible to produce
> other outputs from the documentation, to integrate it with test suites. For instance, 
> Internally at Intel, we use the comments to generate DOT files hierarchically grouped
> per feature categories.
>
> This is meant to be an initial series to start documenting kunit.

I've played with this a bit...a couple of quick impressions:

- That's quite a chunk of Python code to be adding.  I've not yet had
  the chance to read it through properly, will hopefully be able to do
  so soon.  A bit more commenting would not have gone amiss here...

- I kind of think that this should go under dev-tools rather than being
  a new top-level directory.  Is there a reason not to put it there?

Thanks,

jon

Mauro Carvalho Chehab Oct. 7, 2023, 8:09 a.m. UTC | #2

Em Tue, 03 Oct 2023 11:00:20 -0600
Jonathan Corbet <[e-mail]> escreveu:

> Mauro Carvalho Chehab <[e-mail]> writes:
> 
> > This is a follow-up of the discussions taken here:
> >
> >    https://lore.kernel.org/linux-doc/20230704132812.02ba97ba@maurocar-mobl2/T/#t
> >
> > I sent a previous version as RFC. This is basically what we had there, with some
> > improvements at test_list.py.
> >
> > It adds a new extension that allows documenting tests using the same tool we're
> > using for DRM unit tests at IGT GPU tools: https://gitlab.freedesktop.org/drm/igt-gpu-tools.
> >
> > While kernel-doc has provided documentation for in-lined functions/struct comments,
> > it was not meant to document tests.
> >
> > Tests need to be grouped by the test functions. It should also be possible to produce
> > other outputs from the documentation, to integrate it with test suites. For instance, 
> > Internally at Intel, we use the comments to generate DOT files hierarchically grouped
> > per feature categories.
> >
> > This is meant to be an initial series to start documenting kunit.  
> 
> I've played with this a bit...a couple of quick impressions:
> 
> - That's quite a chunk of Python code to be adding.  I've not yet had
>   the chance to read it through properly, will hopefully be able to do
>   so soon.  A bit more commenting would not have gone amiss here...

I'll try to add more comments when respin this series.

I guess I should also add a documentation similar to the one I wrote
for IGT [1]:
	https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/blob/master/docs/test_documentation.md?ref_type=heads#documenting-tests-via-testplan

[1] this document is specific for the way IGT uses it; I'll write
    something similar to it considering the names we've agreed for
    KUnit.

> 
> - I kind of think that this should go under dev-tools rather than being
>   a new top-level directory.  Is there a reason not to put it there?

No particular reason. I'll change it to be under dev-tools/tests at
the next submission.

Should I wait for you to take a look at patch 1/2 before sending
a new version?

Regards,
Mauro

Jonathan Corbet Oct. 7, 2023, 12:52 p.m. UTC | #3

Mauro Carvalho Chehab <[e-mail]> writes:

> Should I wait for you to take a look at patch 1/2 before sending
> a new version?

Go ahead and resend whenever...I'm still digging out from the last few
weeks.

Thanks,

jon