From patchwork Wed Feb 5 23:11:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 13962007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 35507C02192 for ; Wed, 5 Feb 2025 23:13:31 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tfoZN-0001FM-G6; Wed, 05 Feb 2025 18:12:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tfoZK-0001Ey-5H for qemu-devel@nongnu.org; Wed, 05 Feb 2025 18:12:30 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tfoZG-0005ae-TD for qemu-devel@nongnu.org; Wed, 05 Feb 2025 18:12:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738797144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yAmmG7l3qu6h6I5cJ/VHSxEbL4AyzQnT22w/Z2JplrM=; b=W3I1YvmrLSAo2RKUNeeJRbza7Fozvu+y1atZ4mgO00iKjHyy5oKAnlhsDt2dPKACRM/HnS HUl6ZV0W1ImW9spUq74ZuJUg4+Wwo6wMBscqQtsDCcCCQ/CF6ReQ7ZPkXsV+Z+d68ubC2j Q7Ru6cxB5jIhFVPILg4rNnjJo5tDW3c= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-47-kiTKx8_fNNS3t-OFZcemmA-1; Wed, 05 Feb 2025 18:12:20 -0500 X-MC-Unique: kiTKx8_fNNS3t-OFZcemmA-1 X-Mimecast-MFC-AGG-ID: kiTKx8_fNNS3t-OFZcemmA Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D769A1956086; Wed, 5 Feb 2025 23:12:15 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.66.104]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E0B671800570; Wed, 5 Feb 2025 23:12:10 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Cc: Peter Maydell , Thomas Huth , Yanan Wang , Fabiano Rosas , Zhao Liu , Lukas Straub , Eduardo Habkost , Michael Roth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Peter Xu , Eric Blake , Marcel Apfelbaum , Markus Armbruster , =?utf-8?q?Alex_Benn=C3=A9e?= , Jason Wang , Paolo Bonzini , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , John Snow Subject: [PATCH 00/42] docs: add sphinx-domain rST generator to qapidoc Date: Wed, 5 Feb 2025 18:11:26 -0500 Message-ID: <20250205231208.1480762-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org based-on: https://patchew.org/QEMU/20241213011307.2942030-1-jsnow@redhat.com/ Hiya! This series is based on a rebased version of the above series. Apply the above patches to origin/master and then apply this series and you should be good to go. Or just snitch the patches from my GitLab branch: https://gitlab.com/jsnow/qemu/-/commits/sphinx-domain-blergh2 (... ignore the branch name. I ran into some problems with stacked git corrupting my branches ...) If you're just tuning in, this series adds a new sphinx documentation generator for QAPI as presented on at KVM Forum; the big win here is cross-references and indices for QMP commands and events. It depends on the qapi-domain plugin for sphinx which is posted in the pre-requisite series; it's not polished for review and should be considered POC-quality. I felt it was easier to review this series backwards, because the design of the rST generator informs the design of the domain plugin. (Which makes sense: the rST is generated first, and then it's parsed. Our review follows the flow of data through the generator.) Overview: Patches 1-24: Mostly the same as in v2; implements the very basics of the new qapidoc generator. "type" was changed to "kind" for the doc section metadata, and "untagged" changed to "plain". Some small phrasing tweaks here and there. Patches 25-26: Add auto-generated stub docs for undocumented members. Patches 27-29: Restrict the source QAPIDoc syntax slightly and differentiate "plain" sections as either intro or details. Necessary for the inliner. Patch 30: Add the "inliner", the component that squishes "inherited" arguments/members into a single reference for commands/events. Patches 31-32: Add auto-generated documentation for commands that return a value that is not documented. Patches 33-35: Document the "out-of-band" property on QMP commands. Patches 36-38: Add branch support to the inliner. Ish. See below. Patches 39-40: Cull unused definitions from the generated QMP docs; cull anything that has been inlined and no longer needs to be documented separately. Patches 41-42: Add intermediate representation rST document writing in DEBUG mode Things notably still not perfect: (ignoring aesthetics; we care only about the rST generator itself in this series.) - ifcond for anything other than root level entires is still ignored - branch inliner ignores all sections except members (ifcond, details, features) - intro/details separation enforces no plain paragraphs to appear in the "middle" of the documentation section; new markup may be desired if we want to add annotations to categories/regions instead of to specific members/features. If you want to give this a whirl yourself, build QEMU with documentation support enabled and look at docs/manual/qapi/index.html for a sample generation of the QMP manual using the new system. You probably need sphinx >= 4.0 for the time being to do so. John Snow (42): docs/qapidoc: support header-less freeform sections qapi/parser: adjust info location for doc body section docs/qapidoc: remove example section support qapi: expand tags to all doc sections qapi/schema: add __repr__ to QAPIDoc.Section docs/qapidoc: add transmogrifier stub docs/qapidoc: add transmogrifier class stub docs/qapidoc: add visit_module() method qapi/source: allow multi-line QAPISourceInfo advancing docs/qapidoc: add visit_freeform() method docs/qapidoc: add preamble() method docs/qapidoc: add visit_paragraph() method docs/qapidoc: add visit_errors() method docs/qapidoc: add format_type() method docs/qapidoc: add add_field() and generate_field() helper methods docs/qapidoc: add visit_feature() method docs/qapidoc: prepare to record entity being transmogrified docs/qapidoc: add visit_returns() method docs/qapidoc: add visit_member() method docs/qapidoc: add visit_sections() method docs/qapidoc: add visit_entity() docs/qapidoc: implement transmogrify() method docs: disambiguate cross-references docs/qapidoc: add transmogrifier test document docs/qapidoc: generate entries for undocumented members qapi/parser: add undocumented stub members to all_sections qapi: differentiate "intro" and "detail" sections qapi/parser: prohibit untagged sections between tagged sections qapi: Add "Details:" disambiguation marker docs/qapidoc: add minimalistic inliner docs/qapidoc: autogenerate undocumented return docs docs/qapidoc: Add generated returns documentation to inliner docs/qmp: add target to Out-of-band execution section docs/qapidoc: document the "out-of-band" pseudofeature docs/qapidoc: generate out-of-band pseudofeature sections qapi/parser: add "meta" kind to QAPIDoc.Kind qapi/schema: add __iter__ method to QAPISchemaVariants docs/qapi: add branch support to inliner qapi/schema: add doc_visible property to QAPISchemaDefinition docs/qapidoc: cull (most) un-named entities from docs qapi: resolve filenames in info structures docs/qapidoc: add intermediate output debugger docs/devel/codebase.rst | 6 +- docs/glossary.rst | 10 +- docs/index.rst | 1 + docs/interop/qmp-spec.rst | 2 + docs/qapi/index.rst | 53 +++ docs/sphinx/qapidoc.py | 716 ++++++++++++++++++++++++++++++-- qapi/machine.json | 2 + qapi/migration.json | 4 + qapi/net.json | 4 +- qapi/qom.json | 8 +- qapi/yank.json | 2 + scripts/qapi/introspect.py | 4 +- scripts/qapi/parser.py | 178 ++++++-- scripts/qapi/schema.py | 48 ++- scripts/qapi/source.py | 4 +- scripts/qapi/types.py | 4 +- scripts/qapi/visit.py | 4 +- tests/qapi-schema/doc-good.json | 4 +- tests/qapi-schema/doc-good.out | 12 +- tests/qapi-schema/doc-good.txt | 8 +- tests/qapi-schema/test-qapi.py | 4 +- 21 files changed, 975 insertions(+), 103 deletions(-) create mode 100644 docs/qapi/index.rst