| Message ID | 20240626222128.406106-6-jsnow@redhat.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | qapi: convert "Note" and "Example" sections to rST | expand |
John Snow <jsnow@redhat.com> writes: > Change get_doc_indented() to preserve indentation on all subsequent text > lines, and create a compatibility dedent() function for qapidoc.py that > removes indentation the same way get_doc_indented() did. > > This is being done for the benefit of a new qapidoc generator which > requires that indentation in argument and features sections are > preserved. > > Prior to this patch, a section like this: > > ``` > @name: lorem ipsum > dolor sit amet > consectetur adipiscing elit > ``` > > would have its body text be parsed into: > > ``` > lorem ipsum > dolor sit amet > consectetur adipiscing elit > ``` > > We want to preserve the indentation for even the first body line so that > the entire block can be parsed directly as rST. This patch would now > parse that segment into: > > ``` > lorem ipsum > dolor sit amet > consectetur adipiscing elit > ``` > > This is helpful for formatting arguments and features as field lists in > rST, where the new generator will format this information as: > > ``` > :arg type name: lorem ipsum > dolor sit amet > consectetur apidiscing elit > ``` > > ...and can be formed by the simple concatenation of the field list > construct and the body text. The indents help preserve the continuation > of a block-level element, and further allow the use of additional rST > block-level constructs such as code blocks, lists, and other such > markup. > > This understandably breaks the existing qapidoc.py; so a new function is > added there to dedent the text for compatibility. Once the new generator > is merged, this function will not be needed any longer and can be > dropped. I'll restore this paragraph if you don't mind: I verified this patch changes absolutely nothing by comparing the md5sums of the QMP ref html pages both before and after the change, so it's certified inert. QAPI test output has been updated to reflect the new strategy of preserving indents for rST. > Signed-off-by: John Snow <jsnow@redhat.com> > [Edited commit message and code comments per review --js] And I'll drop this line. > Reviewed-by: Markus Armbruster <armbru@redhat.com>
On Thu, Jun 27, 2024, 2:25 AM Markus Armbruster <armbru@redhat.com> wrote: > John Snow <jsnow@redhat.com> writes: > > > Change get_doc_indented() to preserve indentation on all subsequent text > > lines, and create a compatibility dedent() function for qapidoc.py that > > removes indentation the same way get_doc_indented() did. > > > > This is being done for the benefit of a new qapidoc generator which > > requires that indentation in argument and features sections are > > preserved. > > > > Prior to this patch, a section like this: > > > > ``` > > @name: lorem ipsum > > dolor sit amet > > consectetur adipiscing elit > > ``` > > > > would have its body text be parsed into: > > > > ``` > > lorem ipsum > > dolor sit amet > > consectetur adipiscing elit > > ``` > > > > We want to preserve the indentation for even the first body line so that > > the entire block can be parsed directly as rST. This patch would now > > parse that segment into: > > > > ``` > > lorem ipsum > > dolor sit amet > > consectetur adipiscing elit > > ``` > > > > This is helpful for formatting arguments and features as field lists in > > rST, where the new generator will format this information as: > > > > ``` > > :arg type name: lorem ipsum > > dolor sit amet > > consectetur apidiscing elit > > ``` > > > > ...and can be formed by the simple concatenation of the field list > > construct and the body text. The indents help preserve the continuation > > of a block-level element, and further allow the use of additional rST > > block-level constructs such as code blocks, lists, and other such > > markup. > > > > This understandably breaks the existing qapidoc.py; so a new function is > > added there to dedent the text for compatibility. Once the new generator > > is merged, this function will not be needed any longer and can be > > dropped. > > I'll restore this paragraph if you don't mind: > > I verified this patch changes absolutely nothing by comparing the > md5sums of the QMP ref html pages both before and after the change, so > it's certified inert. QAPI test output has been updated to reflect the > new strategy of preserving indents for rST. > Nope, misunderstood the instruction. Go right ahead. > > Signed-off-by: John Snow <jsnow@redhat.com> > > [Edited commit message and code comments per review --js] > > And I'll drop this line. > Vestigial remnants of a more optimistic time. > > Reviewed-by: Markus Armbruster <armbru@redhat.com> > >
John Snow <jsnow@redhat.com> writes: > Change get_doc_indented() to preserve indentation on all subsequent text > lines, and create a compatibility dedent() function for qapidoc.py that > removes indentation the same way get_doc_indented() did. > > This is being done for the benefit of a new qapidoc generator which > requires that indentation in argument and features sections are > preserved. > > Prior to this patch, a section like this: > > ``` > @name: lorem ipsum > dolor sit amet > consectetur adipiscing elit > ``` > > would have its body text be parsed into: > > ``` > lorem ipsum > dolor sit amet > consectetur adipiscing elit > ``` > > We want to preserve the indentation for even the first body line so that > the entire block can be parsed directly as rST. This patch would now > parse that segment into: > > ``` > lorem ipsum > dolor sit amet > consectetur adipiscing elit > ``` > > This is helpful for formatting arguments and features as field lists in > rST, where the new generator will format this information as: > > ``` > :arg type name: lorem ipsum > dolor sit amet > consectetur apidiscing elit > ``` > > ...and can be formed by the simple concatenation of the field list > construct and the body text. The indents help preserve the continuation > of a block-level element, and further allow the use of additional rST > block-level constructs such as code blocks, lists, and other such > markup. > > This understandably breaks the existing qapidoc.py; so a new function is > added there to dedent the text for compatibility. Once the new generator > is merged, this function will not be needed any longer and can be > dropped. I'll restore this paragraph if you don't mind: I verified this patch changes absolutely nothing by comparing the md5sums of the QMP ref html pages both before and after the change, so it's certified inert. QAPI test output has been updated to reflect the new strategy of preserving indents for rST. > Signed-off-by: John Snow <jsnow@redhat.com> > [Edited commit message and code comments per review --js] And I'll drop this line. > Reviewed-by: Markus Armbruster <armbru@redhat.com>
Accidental duplicate, please ignore.
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py index 659e507353a..f9683444b14 100644 --- a/docs/sphinx/qapidoc.py +++ b/docs/sphinx/qapidoc.py @@ -26,6 +26,7 @@ import os import re +import textwrap from docutils import nodes from docutils.parsers.rst import Directive, directives @@ -53,6 +54,19 @@ __version__ = "1.0" +def dedent(text: str) -> str: + # Adjust indentation to make description text parse as paragraph. + + lines = text.splitlines(True) + if re.match(r"\s+", lines[0]): + # First line is indented; description started on the line after + # the name. dedent the whole block. + return textwrap.dedent(text) + + # Descr started on same line. Dedent line 2+. + return lines[0] + textwrap.dedent("".join(lines[1:])) + + # Disable black auto-formatter until re-enabled: # fmt: off @@ -164,7 +178,7 @@ def _nodes_for_members(self, doc, what, base=None, branches=None): term = self._nodes_for_one_member(section.member) # TODO drop fallbacks when undocumented members are outlawed if section.text: - defn = section.text + defn = dedent(section.text) else: defn = [nodes.Text('Not documented')] @@ -202,7 +216,7 @@ def _nodes_for_enum_values(self, doc): termtext.extend(self._nodes_for_ifcond(section.member.ifcond)) # TODO drop fallbacks when undocumented members are outlawed if section.text: - defn = section.text + defn = dedent(section.text) else: defn = [nodes.Text('Not documented')] @@ -237,7 +251,7 @@ def _nodes_for_features(self, doc): dlnode = nodes.definition_list() for section in doc.features.values(): dlnode += self._make_dlitem( - [nodes.literal('', section.member.name)], section.text) + [nodes.literal('', section.member.name)], dedent(section.text)) seen_item = True if not seen_item: @@ -260,9 +274,12 @@ def _nodes_for_sections(self, doc): continue snode = self._make_section(section.tag) if section.tag and section.tag.startswith('Example'): - snode += self._nodes_for_example(section.text) + snode += self._nodes_for_example(dedent(section.text)) else: - self._parse_text_into_node(section.text, snode) + self._parse_text_into_node( + dedent(section.text) if section.tag else section.text, + snode, + ) nodelist.append(snode) return nodelist diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 7b13a583ac1..1ef1f85b028 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -448,7 +448,7 @@ def get_doc_indented(self, doc: 'QAPIDoc') -> Optional[str]: indent = must_match(r'\s*', line).end() if not indent: return line - doc.append_line(line[indent:]) + doc.append_line(line) prev_line_blank = False while True: self.accept(False) @@ -465,7 +465,7 @@ def get_doc_indented(self, doc: 'QAPIDoc') -> Optional[str]: self, "unexpected de-indent (expected at least %d spaces)" % indent) - doc.append_line(line[indent:]) + doc.append_line(line) prev_line_blank = True def get_doc_paragraph(self, doc: 'QAPIDoc') -> Optional[str]: diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out index 716a9a41026..435f6e6d768 100644 --- a/tests/qapi-schema/doc-good.out +++ b/tests/qapi-schema/doc-good.out @@ -117,8 +117,8 @@ doc symbol=Base body= arg=base1 -description starts on a new line, -minimally indented + description starts on a new line, + minimally indented doc symbol=Variant1 body= A paragraph @@ -145,8 +145,8 @@ doc symbol=Alternate arg=i description starts on the same line -remainder indented the same -@b is undocumented + remainder indented the same + @b is undocumented arg=b feature=alt-feat @@ -158,11 +158,11 @@ doc symbol=cmd body= arg=arg1 -description starts on a new line, -indented + description starts on a new line, + indented arg=arg2 description starts on the same line -remainder indented differently + remainder indented differently arg=arg3 feature=cmd-feat1 @@ -178,16 +178,16 @@ some section=TODO frobnicate section=Notes -- Lorem ipsum dolor sit amet -- Ut enim ad minim veniam + - Lorem ipsum dolor sit amet + - Ut enim ad minim veniam -Duis aute irure dolor + Duis aute irure dolor section=Example --> in -<- out + -> in + <- out section=Examples -- *verbatim* -- {braces} + - *verbatim* + - {braces} section=Since 2.10 doc symbol=cmd-boxed @@ -198,9 +198,9 @@ a feature feature=cmd-feat2 another feature section=Example --> in + -> in -<- out + <- out doc symbol=EVT_BOXED body=
Change get_doc_indented() to preserve indentation on all subsequent text lines, and create a compatibility dedent() function for qapidoc.py that removes indentation the same way get_doc_indented() did. This is being done for the benefit of a new qapidoc generator which requires that indentation in argument and features sections are preserved. Prior to this patch, a section like this: ``` @name: lorem ipsum dolor sit amet consectetur adipiscing elit ``` would have its body text be parsed into: ``` lorem ipsum dolor sit amet consectetur adipiscing elit ``` We want to preserve the indentation for even the first body line so that the entire block can be parsed directly as rST. This patch would now parse that segment into: ``` lorem ipsum dolor sit amet consectetur adipiscing elit ``` This is helpful for formatting arguments and features as field lists in rST, where the new generator will format this information as: ``` :arg type name: lorem ipsum dolor sit amet consectetur apidiscing elit ``` ...and can be formed by the simple concatenation of the field list construct and the body text. The indents help preserve the continuation of a block-level element, and further allow the use of additional rST block-level constructs such as code blocks, lists, and other such markup. This understandably breaks the existing qapidoc.py; so a new function is added there to dedent the text for compatibility. Once the new generator is merged, this function will not be needed any longer and can be dropped. Signed-off-by: John Snow <jsnow@redhat.com> [Edited commit message and code comments per review --js] Reviewed-by: Markus Armbruster <armbru@redhat.com> --- docs/sphinx/qapidoc.py | 27 ++++++++++++++++++++++----- scripts/qapi/parser.py | 4 ++-- tests/qapi-schema/doc-good.out | 32 ++++++++++++++++---------------- 3 files changed, 40 insertions(+), 23 deletions(-)