[00/23] docs: add basic sphinx-domain rST generator to qapidoc

Message

John Snow Dec. 13, 2024, 2:18 a.m. UTC

based-on: https://patchew.org/QEMU/[e-mail]/

Hi!

This series is a very, very barebones implementation for the new QAPI
doc generator. It does not have many features that I presented on at KVM
Forum; the point of this patch set is instead to present a stripped down
basis for ongoing work so we can discuss on-list with full context of
the code available to do so.

The documentation this series generates is *not suitable* for replacing
the current document generator, it has a few glaring omissions - on
purpose - those features have been factored out intentionally so they
can be reviewed with fuller context and more careful review.

What this series does:

- Adds the new "Transmogrifier" rST generator to qapidoc.py, which
  generates an in-memory rST document using qapi-domain directives.
- Adds a test document that showcases this new transmogrifier.

What this series very notably does not do (yet):

- "ifcond" data for anything other than top-level entities is not
  considered or rendered. This means "if" statements for features and
  members are entirely absent.

- The inliner is not present at all. This series renders only
  documentation exactly as it is exists in the source files.

- *branches* are themselves not considered at all; they're skipped
   entirely for now. They will be included alongside the inliner in
   either a subsequent series or a followup to this series.

- Undocumented members and return statements are not autogenerated.

- Pseudofeatures (Things like allow-oob) are not generated as documented
  features.

- Documentation culling: all entities are documented whether or not
  they're relevant to the wire format.

My goal in doing it this way is to save the "fancy" features for later
so we can focus on reviewing and tightening up the core functionality of
the transmogrifier. Once we're on steadier ground, I will re-add the
fanciful features while adjusting the qapi-domain mechanisms. Once
everything looks "roughly right, give or take some minor nits", I will
switch back to the qapi-domain series itself for review before we merge
everything together.

John Snow (23):
  docs/qapidoc: support header-less freeform sections
  qapi/parser: adjust info location for doc body section
  docs/qapidoc: remove example section support
  qapi: expand tags to all doc sections
  qapi/schema: add __repr__ to QAPIDoc.Section
  docs/qapidoc: add transmogrifier stub
  docs/qapidoc: add transmogrifier class stub
  docs/qapidoc: add visit_module() method
  qapi/source: allow multi-line QAPISourceInfo advancing
  docs/qapidoc: add visit_freeform() method
  docs/qapidoc: add preamble() method
  docs/qapidoc: add visit_paragraph() method
  docs/qapidoc: add visit_errors() method
  docs/qapidoc: add format_type() method
  docs/qapidoc: add add_field() and generate_field() helper methods
  docs/qapidoc: add visit_feature() method
  docs/qapidoc: record current documented entity in transmogrifier
  docs/qapidoc: add visit_returns() method
  docs/qapidoc: add visit_member() method
  docs/qapidoc: add visit_sections() method
  docs/qapidoc: add visit_entity()
  docs/qapidoc: implement transmogrify() method
  docs/qapidoc: add transmogrifier test document

 docs/index.rst         |   1 +
 docs/qapi/index.rst    |  53 ++++++
 docs/sphinx/qapidoc.py | 419 ++++++++++++++++++++++++++++++++++++++---
 scripts/qapi/parser.py |  97 +++++++---
 scripts/qapi/source.py |   4 +-
 5 files changed, 524 insertions(+), 50 deletions(-)
 create mode 100644 docs/qapi/index.rst

Comments

Markus Armbruster Dec. 19, 2024, 12:31 p.m. UTC | #1

John Snow <[e-mail]> writes:

> based-on: https://patchew.org/QEMU/[e-mail]/
>
> Hi!
>
> This series is a very, very barebones implementation for the new QAPI
> doc generator. It does not have many features that I presented on at KVM
> Forum; the point of this patch set is instead to present a stripped down
> basis for ongoing work so we can discuss on-list with full context of
> the code available to do so.
>
> The documentation this series generates is *not suitable* for replacing
> the current document generator, it has a few glaring omissions - on
> purpose - those features have been factored out intentionally so they
> can be reviewed with fuller context and more careful review.
>
> What this series does:
>
> - Adds the new "Transmogrifier" rST generator to qapidoc.py, which
>   generates an in-memory rST document using qapi-domain directives.
> - Adds a test document that showcases this new transmogrifier.

Note to other reviewers: transmogrifier output is
docs/manual/qapi/index.html.

> What this series very notably does not do (yet):
>
> - "ifcond" data for anything other than top-level entities is not
>   considered or rendered. This means "if" statements for features and
>   members are entirely absent.
>
> - The inliner is not present at all. This series renders only
>   documentation exactly as it is exists in the source files.

This item is not even a regression.

> - *branches* are themselves not considered at all; they're skipped
>    entirely for now. They will be included alongside the inliner in
>    either a subsequent series or a followup to this series.
>
> - Undocumented members and return statements are not autogenerated.

The current doc generator auto-generates missing member documentation
("Not documented").  It doesn't auto-generate missing returns
documentation.  I explored auto-generating them, but shelved my work to
not interfere with yours.

> - Pseudofeatures (Things like allow-oob) are not generated as documented
>   features.

What exactly are "pseudofeatures"?

> - Documentation culling: all entities are documented whether or not
>   they're relevant to the wire format.

Also not a regression.

> My goal in doing it this way is to save the "fancy" features for later
> so we can focus on reviewing and tightening up the core functionality of
> the transmogrifier. Once we're on steadier ground, I will re-add the
> fanciful features while adjusting the qapi-domain mechanisms. Once
> everything looks "roughly right, give or take some minor nits", I will
> switch back to the qapi-domain series itself for review before we merge
> everything together.

Makes sense to me.

[...]

>  docs/index.rst         |   1 +
>  docs/qapi/index.rst    |  53 ++++++
>  docs/sphinx/qapidoc.py | 419 ++++++++++++++++++++++++++++++++++++++---
>  scripts/qapi/parser.py |  97 +++++++---
>  scripts/qapi/source.py |   4 +-
>  5 files changed, 524 insertions(+), 50 deletions(-)
>  create mode 100644 docs/qapi/index.rst

The changes to the QAPI generator core (scripts/qapi/) are small, and
spread over just four patches:

    qapi/source: allow multi-line QAPISourceInfo advancing
    qapi/schema: add __repr__ to QAPIDoc.Section
    qapi: expand tags to all doc sections
    qapi/parser: adjust info location for doc body section