[00/42] docs: add sphinx-domain rST generator to qapidoc

Message

John Snow Feb. 5, 2025, 11:11 p.m. UTC

based-on: https://patchew.org/QEMU/[e-mail]/

Hiya! This series is based on a rebased version of the above
series. Apply the above patches to origin/master and then apply this
series and you should be good to go. Or just snitch the patches from my
GitLab branch:

https://gitlab.com/jsnow/qemu/-/commits/sphinx-domain-blergh2

(... ignore the branch name. I ran into some problems with stacked git
corrupting my branches ...)

If you're just tuning in, this series adds a new sphinx documentation
generator for QAPI as presented on at KVM Forum; the big win here is
cross-references and indices for QMP commands and events. It depends on
the qapi-domain plugin for sphinx which is posted in the pre-requisite
series; it's not polished for review and should be considered
POC-quality. I felt it was easier to review this series backwards,
because the design of the rST generator informs the design of the domain
plugin. (Which makes sense: the rST is generated first, and then it's
parsed. Our review follows the flow of data through the generator.)

Overview:

Patches 1-24: Mostly the same as in v2; implements the very basics of
    the new qapidoc generator. "type" was changed to "kind" for the doc
    section metadata, and "untagged" changed to "plain". Some small
    phrasing tweaks here and there.

Patches 25-26: Add auto-generated stub docs for undocumented members.

Patches 27-29: Restrict the source QAPIDoc syntax slightly and
    differentiate "plain" sections as either intro or details. Necessary
    for the inliner.

Patch 30: Add the "inliner", the component that squishes "inherited"
    arguments/members into a single reference for commands/events.

Patches 31-32: Add auto-generated documentation for commands that return
    a value that is not documented.

Patches 33-35: Document the "out-of-band" property on QMP commands.

Patches 36-38: Add branch support to the inliner. Ish. See below.

Patches 39-40: Cull unused definitions from the generated QMP docs; cull
    anything that has been inlined and no longer needs to be documented
    separately.

Patches 41-42: Add intermediate representation rST document writing in
    DEBUG mode

Things notably still not perfect:

  (ignoring aesthetics; we care only about the rST generator itself in
  this series.)

- ifcond for anything other than root level entires is still ignored
- branch inliner ignores all sections except members (ifcond, details,
  features)
- intro/details separation enforces no plain paragraphs to appear in the
  "middle" of the documentation section; new markup may be desired if we
  want to add annotations to categories/regions instead of to specific
  members/features.

If you want to give this a whirl yourself, build QEMU with documentation
support enabled and look at docs/manual/qapi/index.html for a sample
generation of the QMP manual using the new system. You probably need
sphinx >= 4.0 for the time being to do so.

John Snow (42):
  docs/qapidoc: support header-less freeform sections
  qapi/parser: adjust info location for doc body section
  docs/qapidoc: remove example section support
  qapi: expand tags to all doc sections
  qapi/schema: add __repr__ to QAPIDoc.Section
  docs/qapidoc: add transmogrifier stub
  docs/qapidoc: add transmogrifier class stub
  docs/qapidoc: add visit_module() method
  qapi/source: allow multi-line QAPISourceInfo advancing
  docs/qapidoc: add visit_freeform() method
  docs/qapidoc: add preamble() method
  docs/qapidoc: add visit_paragraph() method
  docs/qapidoc: add visit_errors() method
  docs/qapidoc: add format_type() method
  docs/qapidoc: add add_field() and generate_field() helper methods
  docs/qapidoc: add visit_feature() method
  docs/qapidoc: prepare to record entity being transmogrified
  docs/qapidoc: add visit_returns() method
  docs/qapidoc: add visit_member() method
  docs/qapidoc: add visit_sections() method
  docs/qapidoc: add visit_entity()
  docs/qapidoc: implement transmogrify() method
  docs: disambiguate cross-references
  docs/qapidoc: add transmogrifier test document
  docs/qapidoc: generate entries for undocumented members
  qapi/parser: add undocumented stub members to all_sections
  qapi: differentiate "intro" and "detail" sections
  qapi/parser: prohibit untagged sections between tagged sections
  qapi: Add "Details:" disambiguation marker
  docs/qapidoc: add minimalistic inliner
  docs/qapidoc: autogenerate undocumented return docs
  docs/qapidoc: Add generated returns documentation to inliner
  docs/qmp: add target to Out-of-band execution section
  docs/qapidoc: document the "out-of-band" pseudofeature
  docs/qapidoc: generate out-of-band pseudofeature sections
  qapi/parser: add "meta" kind to QAPIDoc.Kind
  qapi/schema: add __iter__ method to QAPISchemaVariants
  docs/qapi: add branch support to inliner
  qapi/schema: add doc_visible property to QAPISchemaDefinition
  docs/qapidoc: cull (most) un-named entities from docs
  qapi: resolve filenames in info structures
  docs/qapidoc: add intermediate output debugger

 docs/devel/codebase.rst         |   6 +-
 docs/glossary.rst               |  10 +-
 docs/index.rst                  |   1 +
 docs/interop/qmp-spec.rst       |   2 +
 docs/qapi/index.rst             |  53 +++
 docs/sphinx/qapidoc.py          | 716 ++++++++++++++++++++++++++++++--
 qapi/machine.json               |   2 +
 qapi/migration.json             |   4 +
 qapi/net.json                   |   4 +-
 qapi/qom.json                   |   8 +-
 qapi/yank.json                  |   2 +
 scripts/qapi/introspect.py      |   4 +-
 scripts/qapi/parser.py          | 178 ++++++--
 scripts/qapi/schema.py          |  48 ++-
 scripts/qapi/source.py          |   4 +-
 scripts/qapi/types.py           |   4 +-
 scripts/qapi/visit.py           |   4 +-
 tests/qapi-schema/doc-good.json |   4 +-
 tests/qapi-schema/doc-good.out  |  12 +-
 tests/qapi-schema/doc-good.txt  |   8 +-
 tests/qapi-schema/test-qapi.py  |   4 +-
 21 files changed, 975 insertions(+), 103 deletions(-)
 create mode 100644 docs/qapi/index.rst