From patchwork Mon Jan 14 16:25:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Blake X-Patchwork-Id: 10763047 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD905746 for ; Mon, 14 Jan 2019 16:32:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC687295A8 for ; Mon, 14 Jan 2019 16:32:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BF496295C7; Mon, 14 Jan 2019 16:32:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4945B295A8 for ; Mon, 14 Jan 2019 16:32:52 +0000 (UTC) Received: from localhost ([127.0.0.1]:40280 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gj5AR-0000W9-3X for patchwork-qemu-devel@patchwork.kernel.org; Mon, 14 Jan 2019 11:32:51 -0500 Received: from eggs.gnu.org ([209.51.188.92]:57652) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gj54W-0003p8-0l for qemu-devel@nongnu.org; Mon, 14 Jan 2019 11:26:45 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gj54V-0005Og-5d for qemu-devel@nongnu.org; Mon, 14 Jan 2019 11:26:44 -0500 Received: from mx1.redhat.com ([209.132.183.28]:44112) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gj54S-000557-G7; Mon, 14 Jan 2019 11:26:40 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 93D3589AD5; Mon, 14 Jan 2019 16:26:18 +0000 (UTC) Received: from blue.redhat.com (ovpn-117-16.phx2.redhat.com [10.3.117.16]) by smtp.corp.redhat.com (Postfix) with ESMTP id F1B8A6014F; Mon, 14 Jan 2019 16:26:17 +0000 (UTC) From: Eric Blake To: qemu-devel@nongnu.org Date: Mon, 14 Jan 2019 10:25:52 -0600 Message-Id: <20190114162605.5330-8-eblake@redhat.com> In-Reply-To: <20190114162605.5330-1-eblake@redhat.com> References: <20190114162605.5330-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 14 Jan 2019 16:26:18 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 07/20] iotests: add qmp recursive sorting function X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , John Snow , "open list:Block layer core" , Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: John Snow Python before 3.6 does not sort dictionaries (including kwargs). Therefore, printing QMP objects involves sorting the keys to have a predictable ordering in the iotests output. This means that iotests output will sometimes show arguments in an order not specified by the test author. Presently, we accomplish this by using json.dumps' sort_keys argument, where we only serialize the arguments dictionary, but not the command. However, if we want to pretty-print QMP objects being sent to the QEMU process, we need to build the entire command before logging it. Ordinarily, this would then involve "arguments" being sorted above "execute", which would necessitate a rather ugly and harder-to-read change to many iotests outputs. To facilitate pretty-printing AND maintaining predictable output AND having "arguments" sort after "execute", add a custom sort function that takes a dictionary and recursively builds an OrderedDict that maintains the specific key order we wish to see in iotests output. The qmp_log function uses this to build a QMP object that keeps "execute" above "arguments", but sorts all keys and keys in any subdicts in "arguments" lexicographically to maintain consistent iotests output, with no incompatible changes to any current test. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <20181221093529.23855-8-jsnow@redhat.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/iotests.py | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 9595429fea2..565eebb1ab5 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -30,6 +30,7 @@ import signal import logging import atexit import io +from collections import OrderedDict sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'scripts')) import qtest @@ -75,6 +76,16 @@ def qemu_img(*args): sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) return exitcode +def ordered_kwargs(kwargs): + # kwargs prior to 3.6 are not ordered, so: + od = OrderedDict() + for k, v in sorted(kwargs.items()): + if isinstance(v, dict): + od[k] = ordered_kwargs(v) + else: + od[k] = v + return od + def qemu_img_create(*args): args = list(args) @@ -257,8 +268,10 @@ def filter_img_info(output, filename): def log(msg, filters=[]): for flt in filters: msg = flt(msg) - if type(msg) is dict or type(msg) is list: - print(json.dumps(msg, sort_keys=True)) + if isinstance(msg, dict) or isinstance(msg, list): + # Don't sort if it's already sorted + do_sort = not isinstance(msg, OrderedDict) + print(json.dumps(msg, sort_keys=do_sort)) else: print(msg) @@ -448,8 +461,11 @@ class VM(qtest.QEMUQtestMachine): return result def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs): - logmsg = '{"execute": "%s", "arguments": %s}' % \ - (cmd, json.dumps(kwargs, sort_keys=True)) + full_cmd = OrderedDict(( + ("execute", cmd), + ("arguments", ordered_kwargs(kwargs)) + )) + logmsg = json.dumps(full_cmd) log(logmsg, filters) result = self.qmp(cmd, **kwargs) log(json.dumps(result, sort_keys=True), filters)