From patchwork Fri May 17 14:42:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 10948061 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C9ABF912 for ; Fri, 17 May 2019 14:49:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B882E27F8C for ; Fri, 17 May 2019 14:49:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ABDC127FB1; Fri, 17 May 2019 14:49:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EEE5D27F8C for ; Fri, 17 May 2019 14:49:09 +0000 (UTC) Received: from localhost ([127.0.0.1]:49715 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hReAX-0006mb-7E for patchwork-qemu-devel@patchwork.kernel.org; Fri, 17 May 2019 10:49:09 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45782) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hRe4f-0001n3-2L for qemu-devel@nongnu.org; Fri, 17 May 2019 10:43:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hRe4d-00043S-Kt for qemu-devel@nongnu.org; Fri, 17 May 2019 10:43:05 -0400 Received: from mx1.redhat.com ([209.132.183.28]:17787) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hRe4a-00040E-DH; Fri, 17 May 2019 10:43:00 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AF92E369CC; Fri, 17 May 2019 14:42:54 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-116-213.ams2.redhat.com [10.36.116.213]) by smtp.corp.redhat.com (Postfix) with ESMTP id 493915D719; Fri, 17 May 2019 14:42:52 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Fri, 17 May 2019 16:42:30 +0200 Message-Id: <20190517144232.18965-5-kwolf@redhat.com> In-Reply-To: <20190517144232.18965-1-kwolf@redhat.com> References: <20190517144232.18965-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 17 May 2019 14:42:54 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 4/6] qapi: Disentangle QAPIDoc code X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, pkrempa@redhat.com, armbru@redhat.com, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Documentation comment follow a certain structure: First, we have a text with a general description (called QAPIDoc.body). After this, descriptions of the arguments follow. Finally, we have part that contains various named sections. The code doesn't show this structure but just checks the right side conditions so it happens to do the right set of things in the right phase. This is hard to follow, and adding support for documentation of features would be even harder. This restructures the code so that the three parts are clearly separated. The code becomes a bit longer, but easier to follow. Signed-off-by: Kevin Wolf --- scripts/qapi/common.py | 107 ++++++++++++++++++++++++++++++++--------- 1 file changed, 83 insertions(+), 24 deletions(-) diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py index 71944e2e30..1d0f4847db 100644 --- a/scripts/qapi/common.py +++ b/scripts/qapi/common.py @@ -120,6 +120,27 @@ class QAPIDoc(object): def connect(self, member): self.member = member + class SymbolPart: + """ + Describes which part of the documentation we're parsing right now. + + BODY means that we're ready to process freeform text into self.body. A + symbol name is only allowed if no other text was parsed yet. It is + interpreted as the symbol name that describes the currently documented + object. On getting the second symbol name, we proceed to ARGS. + + ARGS means that we're parsing the arguments section. Any symbol name is + interpreted as an argument and an ArgSection is created for it. + + VARIOUS is the final part where freeform sections may appear. This + includes named sections such as "Return:" as well as unnamed + paragraphs. No symbols are allowed any more in this part. + """ + # Can't make it a subclass of Enum because of Python 2 + BODY = 0 + ARGS = 1 + VARIOUS = 2 + def __init__(self, parser, info): # self._parser is used to report errors with QAPIParseError. The # resulting error position depends on the state of the parser. @@ -135,6 +156,7 @@ class QAPIDoc(object): self.sections = [] # the current section self._section = self.body + self._part = QAPIDoc.SymbolPart.BODY def has_section(self, name): """Return True if we have a section with this name.""" @@ -154,37 +176,84 @@ class QAPIDoc(object): raise QAPIParseError(self._parser, "Missing space after #") line = line[1:] + if self._part == QAPIDoc.SymbolPart.BODY: + self._append_body_line(line) + elif self._part == QAPIDoc.SymbolPart.ARGS: + self._append_args_line(line) + elif self._part == QAPIDoc.SymbolPart.VARIOUS: + self._append_various_line(line) + else: + assert False + + + def end_comment(self): + self._end_section() + + def _check_named_section(self, line, name): + if name in ('Returns:', 'Since:', + # those are often singular or plural + 'Note:', 'Notes:', + 'Example:', 'Examples:', + 'TODO:'): + self._part = QAPIDoc.SymbolPart.VARIOUS + return True + return False + + def _append_body_line(self, line): + name = line.split(' ', 1)[0] # FIXME not nice: things like '# @foo:' and '# @foo: ' aren't # recognized, and get silently treated as ordinary text - if self.symbol: - self._append_symbol_line(line) - elif not self.body.text and line.startswith('@'): + if not self.symbol and not self.body.text and line.startswith('@'): if not line.endswith(':'): raise QAPIParseError(self._parser, "Line should end with :") self.symbol = line[1:-1] # FIXME invalid names other than the empty string aren't flagged if not self.symbol: raise QAPIParseError(self._parser, "Invalid name") + elif self.symbol: + # We already know that we document some symbol + if name.startswith('@') and name.endswith(':'): + self._part = QAPIDoc.SymbolPart.ARGS + self._append_args_line(line) + elif self.symbol and self._check_named_section(line, name): + self._append_various_line(line) + else: + self._append_freeform(line.strip()) else: - self._append_freeform(line) - - def end_comment(self): - self._end_section() + # This is free-form documentation without a symbol + self._append_freeform(line.strip()) - def _append_symbol_line(self, line): + def _append_args_line(self, line): name = line.split(' ', 1)[0] if name.startswith('@') and name.endswith(':'): line = line[len(name)+1:] self._start_args_section(name[1:-1]) - elif name in ('Returns:', 'Since:', - # those are often singular or plural - 'Note:', 'Notes:', - 'Example:', 'Examples:', - 'TODO:'): + elif self._check_named_section(line, name): + return self._append_various_line(line) + elif (self._section.text.endswith('\n\n') + and line and not line[0].isspace()): + self._start_section() + self._part = QAPIDoc.SymbolPart.VARIOUS + return self._append_various_line(line) + + self._append_freeform(line.strip()) + + def _append_various_line(self, line): + name = line.split(' ', 1)[0] + + if name.startswith('@') and name.endswith(':'): + raise QAPIParseError(self._parser, + "'%s' can't follow '%s' section" + % (name, self.sections[0].name)) + elif self._check_named_section(line, name): line = line[len(name)+1:] self._start_section(name[:-1]) + if (not self._section.name or + not self._section.name.startswith('Example')): + line = line.strip() + self._append_freeform(line) def _start_args_section(self, name): @@ -194,10 +263,7 @@ class QAPIDoc(object): if name in self.args: raise QAPIParseError(self._parser, "'%s' parameter name duplicated" % name) - if self.sections: - raise QAPIParseError(self._parser, - "'@%s:' can't follow '%s' section" - % (name, self.sections[0].name)) + assert not self.sections self._end_section() self._section = QAPIDoc.ArgSection(name) self.args[name] = self._section @@ -219,13 +285,6 @@ class QAPIDoc(object): self._section = None def _append_freeform(self, line): - in_arg = isinstance(self._section, QAPIDoc.ArgSection) - if (in_arg and self._section.text.endswith('\n\n') - and line and not line[0].isspace()): - self._start_section() - if (in_arg or not self._section.name - or not self._section.name.startswith('Example')): - line = line.strip() match = re.match(r'(@\S+:)', line) if match: raise QAPIParseError(self._parser,