From patchwork Thu Jan 14 17:02:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12020449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9125FC433E0 for ; Thu, 14 Jan 2021 17:58:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1CF6B23B4B for ; Thu, 14 Jan 2021 17:58:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1CF6B23B4B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:37426 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l06sg-0001Sk-WC for qemu-devel@archiver.kernel.org; Thu, 14 Jan 2021 12:57:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:51932) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l061w-0007J2-RW for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:28 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:39521) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1l061k-0006oe-7p for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610643791; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=11QPy0wHX9mepMvqvaTwI4Is3PznSJANnlkJzhuKDZI=; b=gjXcvzhfczNQU++hxyIGKz9vO66YAnn6Aw4UUntKRg6SccTsfM7+7Xv4a9PRinIEBuzvaB kPGIJWPW+TzgYtzL3g41jeCI5ttqJbD4Vya+wESz5BWwXX3jexiYmbEcLDY9DZfFGX3bRb h+36PNO07bE2E0mmyFjZeOjMRghm8/M= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-70-siUNXCnSMnSPdkazNlDocg-1; Thu, 14 Jan 2021 12:03:10 -0500 X-MC-Unique: siUNXCnSMnSPdkazNlDocg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 093E9CC621; Thu, 14 Jan 2021 17:03:09 +0000 (UTC) Received: from localhost (ovpn-115-31.ams2.redhat.com [10.36.115.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 98B846F439; Thu, 14 Jan 2021 17:03:08 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v3 01/10] iotests.py: Assume a couple of variables as given Date: Thu, 14 Jan 2021 18:02:55 +0100 Message-Id: <20210114170304.87507-2-mreitz@redhat.com> In-Reply-To: <20210114170304.87507-1-mreitz@redhat.com> References: <20210114170304.87507-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" There are a couple of environment variables that we fetch with os.environ.get() without supplying a default. Clearly they are required and expected to be set by the ./check script (as evidenced by execute_setup_common(), which checks for test_dir and qemu_default_machine to be set, and aborts if they are not). Using .get() this way has the disadvantage of returning an Optional[str] type, which mypy will complain about when tests just assume these values to be str. Use [] instead, which raises a KeyError for environment variables that are not set. When this exception is raised, catch it and move the abort code from execute_setup_common() there. Drop the 'assert iotests.sock_dir is not None' from iotest 300, because that sort of thing is precisely what this patch wants to prevent. Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/300 | 1 - tests/qemu-iotests/iotests.py | 26 +++++++++++++------------- 2 files changed, 13 insertions(+), 14 deletions(-) diff --git a/tests/qemu-iotests/300 b/tests/qemu-iotests/300 index 5b75121b84..b864a21d5e 100755 --- a/tests/qemu-iotests/300 +++ b/tests/qemu-iotests/300 @@ -27,7 +27,6 @@ import qemu BlockBitmapMapping = List[Dict[str, Union[str, List[Dict[str, str]]]]] -assert iotests.sock_dir is not None mig_sock = os.path.join(iotests.sock_dir, 'mig_sock') diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index dcdcd0387f..52facb8e04 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -75,12 +75,20 @@ qemu_opts = os.environ.get('QEMU_OPTIONS', '').strip().split(' ') imgfmt = os.environ.get('IMGFMT', 'raw') imgproto = os.environ.get('IMGPROTO', 'file') -test_dir = os.environ.get('TEST_DIR') -sock_dir = os.environ.get('SOCK_DIR') output_dir = os.environ.get('OUTPUT_DIR', '.') -cachemode = os.environ.get('CACHEMODE') -aiomode = os.environ.get('AIOMODE') -qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE') + +try: + test_dir = os.environ['TEST_DIR'] + sock_dir = os.environ['SOCK_DIR'] + cachemode = os.environ['CACHEMODE'] + aiomode = os.environ['AIOMODE'] + qemu_default_machine = os.environ['QEMU_DEFAULT_MACHINE'] +except KeyError: + # We are using these variables as proxies to indicate that we're + # not being run via "check". There may be other things set up by + # "check" that individual test cases rely on. + sys.stderr.write('Please run this test via the "check" script\n') + sys.exit(os.EX_USAGE) socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper') @@ -1294,14 +1302,6 @@ def execute_setup_common(supported_fmts: Sequence[str] = (), """ # Note: Python 3.6 and pylint do not like 'Collection' so use 'Sequence'. - # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to - # indicate that we're not being run via "check". There may be - # other things set up by "check" that individual test cases rely - # on. - if test_dir is None or qemu_default_machine is None: - sys.stderr.write('Please run this test via the "check" script\n') - sys.exit(os.EX_USAGE) - debug = '-d' in sys.argv if debug: sys.argv.remove('-d') From patchwork Thu Jan 14 17:02:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12020413 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D950C433DB for ; Thu, 14 Jan 2021 17:33:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A95FD23B31 for ; Thu, 14 Jan 2021 17:33:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A95FD23B31 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:47416 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l06UY-0005x1-OY for qemu-devel@archiver.kernel.org; Thu, 14 Jan 2021 12:33:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:51920) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l061w-0007I4-DF for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:28 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:49704) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1l061k-0006ox-M6 for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610643795; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nUOH6VRFC3WbRU69fiomKZTa3HERk87bF9xDE/qZ5iw=; b=XPbBGa6rAWRw7BOLEo/UMlXoInw/qCH5ic8vudWOsgTCh1TmKkfVY6lwPbalyOSuXgGTjj Sdm7g3L1CmatFOqVLcgIk8Fn8NyEVcsG7jJYgmoUJ7emJ9aejfygVLoJgNCZJuY0ANxsOZ uYBmgTRMFCW4riob2jhwhMwNiJnP8MQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-593-1UJ-bSGEN_6e1yd6YWEMZg-1; Thu, 14 Jan 2021 12:03:13 -0500 X-MC-Unique: 1UJ-bSGEN_6e1yd6YWEMZg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 50CB485EE86; Thu, 14 Jan 2021 17:03:11 +0000 (UTC) Received: from localhost (ovpn-115-31.ams2.redhat.com [10.36.115.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DD0DE10023B7; Thu, 14 Jan 2021 17:03:10 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v3 02/10] iotests/297: Rewrite in Python and extend reach Date: Thu, 14 Jan 2021 18:02:56 +0100 Message-Id: <20210114170304.87507-3-mreitz@redhat.com> In-Reply-To: <20210114170304.87507-1-mreitz@redhat.com> References: <20210114170304.87507-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Instead of checking iotests.py only, check all Python files in the qemu-iotests/ directory. Of course, most of them do not pass, so there is an extensive skip list for now. (The only files that do pass are 209, 254, 283, and iotests.py.) (Alternatively, we could have the opposite, i.e. an explicit list of files that we do want to check, but I think it is better to check files by default.) Unless started in debug mode (./check -d), the output has no information on which files are tested, so we will not have a problem e.g. with backports, where some files may be missing when compared to upstream. Besides the technical rewrite, some more things are changed: - For the pylint invocation, PYTHONPATH is adjusted. This mirrors setting MYPYPATH for mypy. - Also, MYPYPATH is now derived from PYTHONPATH, so that we include paths set by the environment. Maybe at some point we want to let the check script add '../../python/' to PYTHONPATH so that iotests.py does not need to do that. - Passing --notes=FIXME,XXX to pylint suppresses warnings for TODO comments. TODO is fine, we do not need 297 to complain about such comments. - The "Success" line from mypy's output is suppressed, because (A) it does not add useful information, and (B) it would leak information about the files having been tested to the reference output, which we decidedly do not want. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/297 | 109 +++++++++++++++++++++++++++++-------- tests/qemu-iotests/297.out | 5 +- 2 files changed, 89 insertions(+), 25 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 5c5420712b..bfa26d280b 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -1,4 +1,4 @@ -#!/usr/bin/env bash +#!/usr/bin/env python3 # # Copyright (C) 2020 Red Hat, Inc. # @@ -15,30 +15,95 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see . -seq=$(basename $0) -echo "QA output created by $seq" +import os +import re +import shutil +import subprocess +import sys -status=1 # failure is the default! +import iotests -# get standard environment -. ./common.rc -if ! type -p "pylint-3" > /dev/null; then - _notrun "pylint-3 not found" -fi -if ! type -p "mypy" > /dev/null; then - _notrun "mypy not found" -fi +# TODO: Empty this list! +SKIP_FILES = ( + '030', '040', '041', '044', '045', '055', '056', '057', '065', '093', + '096', '118', '124', '129', '132', '136', '139', '147', '148', '149', + '151', '152', '155', '163', '165', '169', '194', '196', '199', '202', + '203', '205', '206', '207', '208', '210', '211', '212', '213', '216', + '218', '219', '222', '224', '228', '234', '235', '236', '237', '238', + '240', '242', '245', '246', '248', '255', '256', '257', '258', '260', + '262', '264', '266', '274', '277', '280', '281', '295', '296', '298', + '299', '300', '302', '303', '304', '307', + 'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py' +) -pylint-3 --score=n iotests.py -MYPYPATH=../../python/ mypy --warn-unused-configs --disallow-subclassing-any \ - --disallow-any-generics --disallow-incomplete-defs \ - --disallow-untyped-decorators --no-implicit-optional \ - --warn-redundant-casts --warn-unused-ignores \ - --no-implicit-reexport iotests.py +def is_python_file(filename): + if not os.path.isfile(filename): + return False -# success, all done -echo "*** done" -rm -f $seq.full -status=0 + if filename.endswith('.py'): + return True + + with open(filename) as f: + try: + first_line = f.readline() + return re.match('^#!.*python', first_line) is not None + except UnicodeDecodeError: # Ignore binary files + return False + +def run_linters(): + files = [filename for filename in (set(os.listdir('.')) - set(SKIP_FILES)) + if is_python_file(filename)] + + iotests.logger.debug('Files to be checked:') + iotests.logger.debug(', '.join(sorted(files))) + + print('=== pylint ===') + sys.stdout.flush() + + # Todo notes are fine, but fixme's or xxx's should probably just be + # fixed (in tests, at least) + env = os.environ + try: + env['PYTHONPATH'] += ':../../python/' + except KeyError: + env['PYTHONPATH'] = '../../python/' + subprocess.run(('pylint-3', '--score=n', '--notes=FIXME,XXX', *files), + env=env, check=False) + + print('=== mypy ===') + sys.stdout.flush() + + # We have to call mypy separately for each file. Otherwise, it + # will interpret all given files as belonging together (i.e., they + # may not both define the same classes, etc.; most notably, they + # must not both define the __main__ module). + env['MYPYPATH'] = env['PYTHONPATH'] + for filename in files: + p = subprocess.run(('mypy', + '--warn-unused-configs', + '--disallow-subclassing-any', + '--disallow-any-generics', + '--disallow-incomplete-defs', + '--disallow-untyped-decorators', + '--no-implicit-optional', + '--warn-redundant-casts', + '--warn-unused-ignores', + '--no-implicit-reexport', + filename), + env=env, + check=False, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + text=True) + + if p.returncode != 0: + print(p.stdout) + + +for linter in ('pylint-3', 'mypy'): + if shutil.which(linter) is None: + iotests.notrun(f'{linter} not found') + +iotests.script_main(run_linters) diff --git a/tests/qemu-iotests/297.out b/tests/qemu-iotests/297.out index 6acc843649..f2e1314d10 100644 --- a/tests/qemu-iotests/297.out +++ b/tests/qemu-iotests/297.out @@ -1,3 +1,2 @@ -QA output created by 297 -Success: no issues found in 1 source file -*** done +=== pylint === +=== mypy === From patchwork Thu Jan 14 17:02:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12020483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 904D2C433DB for ; Thu, 14 Jan 2021 18:02:14 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0483E23A34 for ; Thu, 14 Jan 2021 18:02:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0483E23A34 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:46252 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l06wm-0005Nx-PF for qemu-devel@archiver.kernel.org; Thu, 14 Jan 2021 13:02:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:51940) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l061x-0007Jo-8E for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:29 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:57390) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1l061m-0006rE-JS for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610643798; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RcJKj6wXMF7lgVQPcnR3ZE7i+jQrH9qvRgCCeOxzi3Y=; b=Jx9NjWhpb0aLdGmxRXCQ1NZTo1Ycxknmdnu+YnBojKqDcNhgKgHsFBPB/MOqhomwTbS5IV juk/M4CpjDXFddZ+fJMGaLS3pzOtBbwG712ABjG0UrNH9hDWPd7CtatmW8XvyP1XJRtN2s qeIRH8+dKHcdMALL7+RM/rMTxMJ4Hv8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-481-b7h50Gl7MRmbE7KOabrgQQ-1; Thu, 14 Jan 2021 12:03:15 -0500 X-MC-Unique: b7h50Gl7MRmbE7KOabrgQQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F03C2100D671; Thu, 14 Jan 2021 17:03:13 +0000 (UTC) Received: from localhost (ovpn-115-31.ams2.redhat.com [10.36.115.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8759612D7E; Thu, 14 Jan 2021 17:03:13 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v3 03/10] iotests: Move try_remove to iotests.py Date: Thu, 14 Jan 2021 18:02:57 +0100 Message-Id: <20210114170304.87507-4-mreitz@redhat.com> In-Reply-To: <20210114170304.87507-1-mreitz@redhat.com> References: <20210114170304.87507-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/124 | 8 +------- tests/qemu-iotests/iotests.py | 11 +++++++---- 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 index 3705cbb6b3..e40eeb50b9 100755 --- a/tests/qemu-iotests/124 +++ b/tests/qemu-iotests/124 @@ -22,6 +22,7 @@ import os import iotests +from iotests import try_remove def io_write_patterns(img, patterns): @@ -29,13 +30,6 @@ def io_write_patterns(img, patterns): iotests.qemu_io('-c', 'write -P%s %s %s' % pattern, img) -def try_remove(img): - try: - os.remove(img) - except OSError: - pass - - def transaction_action(action, **kwargs): return { 'type': action, diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 52facb8e04..a69b4cdc4e 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -523,12 +523,15 @@ class FilePath: return False +def try_remove(img): + try: + os.remove(img) + except OSError: + pass + def file_path_remover(): for path in reversed(file_path_remover.paths): - try: - os.remove(path) - except OSError: - pass + try_remove(path) def file_path(*names, base_dir=test_dir): From patchwork Thu Jan 14 17:02:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12020383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD257C433E0 for ; Thu, 14 Jan 2021 17:28:42 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5223623A5E for ; Thu, 14 Jan 2021 17:28:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5223623A5E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:35984 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l06QL-0000rL-7c for qemu-devel@archiver.kernel.org; Thu, 14 Jan 2021 12:28:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:52032) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0620-0007R9-8G for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:56508) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1l061r-0006tu-1B for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610643802; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LxANx+Rz2agcyu5X1D3Ejr1ptPnBeUWQBvphznPKVvk=; b=LkxClfwyKFFB5PrppLihuQpYjqoPbw+oU0Vg4u1F2DfRjibF7loecW1h5Znn9IJOS1akmk CiXVPkz7ae8tzn18ncYdic26tMAVeKtr8+o9chedQGOUelYLAExzhx7xFEBZMMcgusDUUS d8DYSZ/EKBmYCuB040Y8OPGl9lG2kWA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-468-PH-KRbE5Pt6QOUtF9rHWIQ-1; Thu, 14 Jan 2021 12:03:19 -0500 X-MC-Unique: PH-KRbE5Pt6QOUtF9rHWIQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B5E12107ACF7; Thu, 14 Jan 2021 17:03:18 +0000 (UTC) Received: from localhost (ovpn-115-31.ams2.redhat.com [10.36.115.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D95D86F439; Thu, 14 Jan 2021 17:03:15 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v3 04/10] iotests/129: Remove test images in tearDown() Date: Thu, 14 Jan 2021 18:02:58 +0100 Message-Id: <20210114170304.87507-5-mreitz@redhat.com> In-Reply-To: <20210114170304.87507-1-mreitz@redhat.com> References: <20210114170304.87507-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Willian Rampazzo --- tests/qemu-iotests/129 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 0e13244d85..2fc65ada6a 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -47,6 +47,8 @@ class TestStopWithBlockJob(iotests.QMPTestCase): result = self.vm.qmp("block_set_io_throttle", conv_keys=False, **params) self.vm.shutdown() + for img in (self.test_img, self.target_img, self.base_img): + iotests.try_remove(img) def do_test_stop(self, cmd, **args): """Test 'stop' while block job is running on a throttled drive. From patchwork Thu Jan 14 17:02:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12020485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E3D8C433E0 for ; Thu, 14 Jan 2021 18:05:38 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DA92523741 for ; Thu, 14 Jan 2021 18:05:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA92523741 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:53456 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l0704-0008LZ-R8 for qemu-devel@archiver.kernel.org; Thu, 14 Jan 2021 13:05:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:52046) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0621-0007T6-1v for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:33 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:29866) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1l061v-0006uF-Tg for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610643804; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0sOKI0PkBbpsD1lkPyyE95je+xXaX1EDNteAdaV4lNM=; b=KWVmuu7MrMgIx8JjhzCT0ZaLtzl5FQDhg6BBmhjz1CrcRhN5gRy1UtEHkb3g6tTIWZXEI2 VQ7006nQwMFSqOJuBEKJOqABpDhNme5Si0fR7jCEhYWxwwfH0a32gkZA/S0Jj5xrw9tOBK Kx5Qxf+A810i37mYyylxs6auzCANvTs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-435-Dw580iF5OyqRYYW6wWDeVw-1; Thu, 14 Jan 2021 12:03:22 -0500 X-MC-Unique: Dw580iF5OyqRYYW6wWDeVw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4A016806661; Thu, 14 Jan 2021 17:03:21 +0000 (UTC) Received: from localhost (ovpn-115-31.ams2.redhat.com [10.36.115.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DECE360867; Thu, 14 Jan 2021 17:03:20 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v3 05/10] iotests/129: Do not check @busy Date: Thu, 14 Jan 2021 18:02:59 +0100 Message-Id: <20210114170304.87507-6-mreitz@redhat.com> In-Reply-To: <20210114170304.87507-1-mreitz@redhat.com> References: <20210114170304.87507-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" @busy is false when the job is paused, which happens all the time because that is how jobs yield (e.g. for mirror at least since commit 565ac01f8d3). Back when 129 was added (2015), perhaps there was no better way of checking whether the job was still actually running. Now we have the @status field (as of 58b295ba52c, i.e. 2018), which can give us exactly that information. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 2fc65ada6a..dd23bb2e5a 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -69,7 +69,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): result = self.vm.qmp("stop") self.assert_qmp(result, 'return', {}) result = self.vm.qmp("query-block-jobs") - self.assert_qmp(result, 'return[0]/busy', True) + self.assert_qmp(result, 'return[0]/status', 'running') self.assert_qmp(result, 'return[0]/ready', False) def test_drive_mirror(self): From patchwork Thu Jan 14 17:03:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12020409 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 261F6C433E0 for ; Thu, 14 Jan 2021 17:32:10 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D312723B31 for ; Thu, 14 Jan 2021 17:32:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D312723B31 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:44694 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l06Tg-0004jg-PC for qemu-devel@archiver.kernel.org; Thu, 14 Jan 2021 12:32:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:52102) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0622-0007XS-Pl for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:34 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:39051) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1l061x-0006ux-OM for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610643808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0ZlDK3ti6VD9Wg4mk5fXQ4rn8uIlnrUMVgAf+ekSndY=; b=Y4tEyf6aKOsH/Y+cXAfPTxA5AXMcOktQ07K7GAVzUDx25CMYAMX9l4j7WTdMWADd1TPpgL I4HSRnietpb1OXLQ4fGiiZcjbMm+E+fjjl8RBwbI9G2BSvQmpUaUS0FOtTVZAVoItlJb7y kUYNEqXsMi+asMSj/6qI05TYnxqThws= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-89-383evvt1OaKGC9FuUVqXbA-1; Thu, 14 Jan 2021 12:03:24 -0500 X-MC-Unique: 383evvt1OaKGC9FuUVqXbA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 897ACCC622; Thu, 14 Jan 2021 17:03:23 +0000 (UTC) Received: from localhost (ovpn-115-31.ams2.redhat.com [10.36.115.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 20C945D71D; Thu, 14 Jan 2021 17:03:22 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v3 06/10] iotests/129: Use throttle node Date: Thu, 14 Jan 2021 18:03:00 +0100 Message-Id: <20210114170304.87507-7-mreitz@redhat.com> In-Reply-To: <20210114170304.87507-1-mreitz@redhat.com> References: <20210114170304.87507-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 37 +++++++++++++------------------------ 1 file changed, 13 insertions(+), 24 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index dd23bb2e5a..58536bc6ee 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -32,20 +32,18 @@ class TestStopWithBlockJob(iotests.QMPTestCase): iotests.qemu_img('create', '-f', iotests.imgfmt, self.test_img, "-b", self.base_img, '-F', iotests.imgfmt) iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', self.test_img) - self.vm = iotests.VM().add_drive(self.test_img) + self.vm = iotests.VM() + self.vm.add_object('throttle-group,id=tg0,x-bps-total=1024') + + source_drive = 'driver=throttle,' \ + 'throttle-group=tg0,' \ + f'file.driver={iotests.imgfmt},' \ + f'file.file.filename={self.test_img}' + + self.vm.add_drive(None, source_drive) self.vm.launch() def tearDown(self): - params = {"device": "drive0", - "bps": 0, - "bps_rd": 0, - "bps_wr": 0, - "iops": 0, - "iops_rd": 0, - "iops_wr": 0, - } - result = self.vm.qmp("block_set_io_throttle", conv_keys=False, - **params) self.vm.shutdown() for img in (self.test_img, self.target_img, self.base_img): iotests.try_remove(img) @@ -53,33 +51,24 @@ class TestStopWithBlockJob(iotests.QMPTestCase): def do_test_stop(self, cmd, **args): """Test 'stop' while block job is running on a throttled drive. The 'stop' command shouldn't drain the job""" - params = {"device": "drive0", - "bps": 1024, - "bps_rd": 0, - "bps_wr": 0, - "iops": 0, - "iops_rd": 0, - "iops_wr": 0, - } - result = self.vm.qmp("block_set_io_throttle", conv_keys=False, - **params) - self.assert_qmp(result, 'return', {}) result = self.vm.qmp(cmd, **args) self.assert_qmp(result, 'return', {}) + result = self.vm.qmp("stop") self.assert_qmp(result, 'return', {}) result = self.vm.qmp("query-block-jobs") + self.assert_qmp(result, 'return[0]/status', 'running') self.assert_qmp(result, 'return[0]/ready', False) def test_drive_mirror(self): self.do_test_stop("drive-mirror", device="drive0", - target=self.target_img, + target=self.target_img, format=iotests.imgfmt, sync="full") def test_drive_backup(self): self.do_test_stop("drive-backup", device="drive0", - target=self.target_img, + target=self.target_img, format=iotests.imgfmt, sync="full") def test_block_commit(self): From patchwork Thu Jan 14 17:03:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12020531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74C50C433DB for ; Thu, 14 Jan 2021 18:09:34 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E14A123A04 for ; Thu, 14 Jan 2021 18:09:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E14A123A04 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34370 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l073s-0003ty-N6 for qemu-devel@archiver.kernel.org; Thu, 14 Jan 2021 13:09:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:52090) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0622-0007Wd-E4 for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:34 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:55042) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1l061x-0006un-AH for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610643808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iVnHNMSZS/bcYc9NyCWat/9jmqgK0G82ubZDwoGFKUE=; b=V3mG2eBCer/txvz0opWFXZuBqwZbjSf4hPwckWiS4bCkJwllfSWSw7/1JrbT0TtComhAD5 Ohr5JQDcr+wweh4C/7A2mSAm56lmqGbrnQUTLe0X4ee1lFl8kt0zcGI+m7hNRDwjSKw6XA QxgnBOmF1ovBNIoMCLQluGcl1m3sQtA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-381-13kiV7TqP-mbE8WScyMijA-1; Thu, 14 Jan 2021 12:03:26 -0500 X-MC-Unique: 13kiV7TqP-mbE8WScyMijA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C0DC8800050; Thu, 14 Jan 2021 17:03:25 +0000 (UTC) Received: from localhost (ovpn-115-31.ams2.redhat.com [10.36.115.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 550C36F439; Thu, 14 Jan 2021 17:03:25 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v3 07/10] iotests/129: Actually test a commit job Date: Thu, 14 Jan 2021 18:03:01 +0100 Message-Id: <20210114170304.87507-8-mreitz@redhat.com> In-Reply-To: <20210114170304.87507-1-mreitz@redhat.com> References: <20210114170304.87507-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Before this patch, test_block_commit() performs an active commit, which under the hood is a mirror job. If we want to test various different block jobs, we should perhaps run an actual commit job instead. Doing so requires adding an overlay above the source node before the commit is done (and then specifying the source node as the top node for the commit job). Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 58536bc6ee..7b4b6649f0 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -26,6 +26,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): test_img = os.path.join(iotests.test_dir, 'test.img') target_img = os.path.join(iotests.test_dir, 'target.img') base_img = os.path.join(iotests.test_dir, 'base.img') + overlay_img = os.path.join(iotests.test_dir, 'overlay.img') def setUp(self): iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, "1G") @@ -36,6 +37,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): self.vm.add_object('throttle-group,id=tg0,x-bps-total=1024') source_drive = 'driver=throttle,' \ + 'node-name=source,' \ 'throttle-group=tg0,' \ f'file.driver={iotests.imgfmt},' \ f'file.file.filename={self.test_img}' @@ -45,7 +47,8 @@ class TestStopWithBlockJob(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - for img in (self.test_img, self.target_img, self.base_img): + for img in (self.test_img, self.target_img, self.base_img, + self.overlay_img): iotests.try_remove(img) def do_test_stop(self, cmd, **args): @@ -72,7 +75,27 @@ class TestStopWithBlockJob(iotests.QMPTestCase): sync="full") def test_block_commit(self): - self.do_test_stop("block-commit", device="drive0") + # Add overlay above the source node so that we actually use a + # commit job instead of a mirror job + + iotests.qemu_img('create', '-f', iotests.imgfmt, self.overlay_img, + '1G') + + result = self.vm.qmp('blockdev-add', **{ + 'node-name': 'overlay', + 'driver': iotests.imgfmt, + 'file': { + 'driver': 'file', + 'filename': self.overlay_img + } + }) + self.assert_qmp(result, 'return', {}) + + result = self.vm.qmp('blockdev-snapshot', + node='source', overlay='overlay') + self.assert_qmp(result, 'return', {}) + + self.do_test_stop('block-commit', device='drive0', top_node='source') if __name__ == '__main__': iotests.main(supported_fmts=["qcow2"], From patchwork Thu Jan 14 17:03:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12020415 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EF9EC433E0 for ; Thu, 14 Jan 2021 17:36:03 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 24F2A23B52 for ; Thu, 14 Jan 2021 17:36:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 24F2A23B52 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:53280 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l06XS-0008R4-7A for qemu-devel@archiver.kernel.org; Thu, 14 Jan 2021 12:36:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:52112) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0623-0007YB-2X for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:35 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:49024) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1l061z-0006w5-JK for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610643810; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bU+AGmGeqHFzTcyXBp/no/OR5o6fbi0ENmf3b/ZaRlQ=; b=GmEyPxLEumEfT4naQsBfkR+iwGKoYfil9NbuG2xn/1CuYseWloAuTbP6U1FogURVdMgnLF sN8uVXK/oOfJh5Xdh/6ZlYN6V+vjxcata8nerQogFR9EPW5zxIYXODYVmY1POHD5kPnzpc gzkTi4KkNPUILPa88ER5vEBKVUCVn+I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-587-ufEYqsXAOnaLlvi1RT0ZvA-1; Thu, 14 Jan 2021 12:03:29 -0500 X-MC-Unique: ufEYqsXAOnaLlvi1RT0ZvA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 570FF107ACF7; Thu, 14 Jan 2021 17:03:28 +0000 (UTC) Received: from localhost (ovpn-115-31.ams2.redhat.com [10.36.115.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E308460938; Thu, 14 Jan 2021 17:03:27 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v3 08/10] iotests/129: Limit mirror job's buffer size Date: Thu, 14 Jan 2021 18:03:02 +0100 Message-Id: <20210114170304.87507-9-mreitz@redhat.com> In-Reply-To: <20210114170304.87507-1-mreitz@redhat.com> References: <20210114170304.87507-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Issuing 'stop' on the VM drains all nodes. If the mirror job has many large requests in flight, this may lead to significant I/O that looks a bit like 'stop' would make the job try to complete (which is what 129 should verify not to happen). We can limit the I/O in flight by limiting the buffer size, so mirror will make very little progress during the 'stop' drain. (We do not need to do anything about commit, which has a buffer size of 512 kB by default; or backup, which goes cluster by cluster. Once we have asynchronous requests for backup, that will change, but then we can fine-tune the backup job to only perform a single request on a very small chunk, too.) Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Willian Rampazzo --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 7b4b6649f0..6d21470cd7 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -67,7 +67,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): def test_drive_mirror(self): self.do_test_stop("drive-mirror", device="drive0", target=self.target_img, format=iotests.imgfmt, - sync="full") + sync="full", buf_size=65536) def test_drive_backup(self): self.do_test_stop("drive-backup", device="drive0", From patchwork Thu Jan 14 17:03:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12020423 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D33AC433DB for ; Thu, 14 Jan 2021 17:39:50 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9292B23B40 for ; Thu, 14 Jan 2021 17:39:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9292B23B40 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:33446 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l06b6-0003dR-It for qemu-devel@archiver.kernel.org; Thu, 14 Jan 2021 12:39:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:52138) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0624-0007bQ-Ao for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:36 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:21630) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1l0622-0006wt-6N for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610643813; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=88jW508my9YkqN6iINXbulZF2jhBRgtWFg2K4BkF2YI=; b=euu5fCT38WolIHmuWhIgUmqpqSbsaLU2QoWAQg4z+/InkkDgwKRbNZOhMwvLWUUK4jIMvV JQwqQjL5iVrA7q1rcO7Xs6z/bwe3QN2AkYJ+Tt48nBu9m7Kky7hHRZVwj70TWPcTU5Yu5z wehNNSywZmy/rqeqH2KpYCxhZZFf/88= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-481-ya5ixl4yPIC_g1C76bl75A-1; Thu, 14 Jan 2021 12:03:31 -0500 X-MC-Unique: ya5ixl4yPIC_g1C76bl75A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AE812100D671; Thu, 14 Jan 2021 17:03:30 +0000 (UTC) Received: from localhost (ovpn-115-31.ams2.redhat.com [10.36.115.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 468AB6F439; Thu, 14 Jan 2021 17:03:30 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v3 09/10] iotests/129: Clean up pylint and mypy complaints Date: Thu, 14 Jan 2021 18:03:03 +0100 Message-Id: <20210114170304.87507-10-mreitz@redhat.com> In-Reply-To: <20210114170304.87507-1-mreitz@redhat.com> References: <20210114170304.87507-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Willian Rampazzo Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 4 ++-- tests/qemu-iotests/297 | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 6d21470cd7..201d9e0a0b 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -20,7 +20,6 @@ import os import iotests -import time class TestStopWithBlockJob(iotests.QMPTestCase): test_img = os.path.join(iotests.test_dir, 'test.img') @@ -32,7 +31,8 @@ class TestStopWithBlockJob(iotests.QMPTestCase): iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, "1G") iotests.qemu_img('create', '-f', iotests.imgfmt, self.test_img, "-b", self.base_img, '-F', iotests.imgfmt) - iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', self.test_img) + iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', + self.test_img) self.vm = iotests.VM() self.vm.add_object('throttle-group,id=tg0,x-bps-total=1024') diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index bfa26d280b..1dce1d1b1c 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -27,7 +27,7 @@ import iotests # TODO: Empty this list! SKIP_FILES = ( '030', '040', '041', '044', '045', '055', '056', '057', '065', '093', - '096', '118', '124', '129', '132', '136', '139', '147', '148', '149', + '096', '118', '124', '132', '136', '139', '147', '148', '149', '151', '152', '155', '163', '165', '169', '194', '196', '199', '202', '203', '205', '206', '207', '208', '210', '211', '212', '213', '216', '218', '219', '222', '224', '228', '234', '235', '236', '237', '238', From patchwork Thu Jan 14 17:03:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12020427 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 209C3C433DB for ; Thu, 14 Jan 2021 17:43:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C756E23B40 for ; Thu, 14 Jan 2021 17:43:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C756E23B40 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:41868 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l06f3-0007Ik-Rt for qemu-devel@archiver.kernel.org; Thu, 14 Jan 2021 12:43:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:52202) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l062D-0007et-Tm for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:46 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:47794) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1l062A-0006yd-9D for qemu-devel@nongnu.org; Thu, 14 Jan 2021 12:03:45 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610643816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ml7ShADg32L/X4Ot+87tc9sMP34mb0kfAydIJ0wnwOk=; b=akNUWuBO04M/NuC5HiyE1NtRbrrKWyQPbeygP+Mg+IHyzA5cuCiTUe5X1hVkI7yjv8W9QB wIGI2N3o+RfGbpHXOrYBXjf4sSub3KLKri5zZYLHVsu19WZcQylTsFNxAaZc3/XzZSZVjY J6M/ArPOtF3xqvhTjHLe3Css1iAR6lY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-590-oH4s-2YSOFOCMT1B4sUgUw-1; Thu, 14 Jan 2021 12:03:34 -0500 X-MC-Unique: oH4s-2YSOFOCMT1B4sUgUw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id ED0C71927803; Thu, 14 Jan 2021 17:03:32 +0000 (UTC) Received: from localhost (ovpn-115-31.ams2.redhat.com [10.36.115.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 86D3610023BD; Thu, 14 Jan 2021 17:03:32 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v3 10/10] iotests/300: Clean up pylint and mypy complaints Date: Thu, 14 Jan 2021 18:03:04 +0100 Message-Id: <20210114170304.87507-11-mreitz@redhat.com> In-Reply-To: <20210114170304.87507-1-mreitz@redhat.com> References: <20210114170304.87507-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/297 | 2 +- tests/qemu-iotests/300 | 18 +++++++++++++++--- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 1dce1d1b1c..03d8604538 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -33,7 +33,7 @@ SKIP_FILES = ( '218', '219', '222', '224', '228', '234', '235', '236', '237', '238', '240', '242', '245', '246', '248', '255', '256', '257', '258', '260', '262', '264', '266', '274', '277', '280', '281', '295', '296', '298', - '299', '300', '302', '303', '304', '307', + '299', '302', '303', '304', '307', 'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py' ) diff --git a/tests/qemu-iotests/300 b/tests/qemu-iotests/300 index b864a21d5e..64d388a8fa 100755 --- a/tests/qemu-iotests/300 +++ b/tests/qemu-iotests/300 @@ -22,7 +22,11 @@ import os import random import re from typing import Dict, List, Optional, Union + import iotests + +# Import qemu after iotests.py has amended the PYTHONPATH +# pylint: disable=wrong-import-order import qemu BlockBitmapMapping = List[Dict[str, Union[str, List[Dict[str, str]]]]] @@ -110,10 +114,14 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): If @msg is None, check that there has not been any error. """ self.vm_b.shutdown() + + log = self.vm_b.get_log() + assert log is not None # Loaded after shutdown + if msg is None: - self.assertNotIn('qemu-system-', self.vm_b.get_log()) + self.assertNotIn('qemu-system-', log) else: - self.assertIn(msg, self.vm_b.get_log()) + self.assertIn(msg, log) @staticmethod def mapping(node_name: str, node_alias: str, @@ -445,9 +453,13 @@ class TestBlockBitmapMappingErrors(TestDirtyBitmapMigration): # Check for the error in the source's log self.vm_a.shutdown() + + log = self.vm_a.get_log() + assert log is not None # Loaded after shutdown + self.assertIn(f"Cannot migrate bitmap '{name}' on node " f"'{self.src_node_name}': Name is longer than 255 bytes", - self.vm_a.get_log()) + log) # Expect abnormal shutdown of the destination VM because of # the failed migration From patchwork Fri Jan 15 11:53:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 12022327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56905C433DB for ; Fri, 15 Jan 2021 11:55:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DFBA0233CF for ; Fri, 15 Jan 2021 11:55:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DFBA0233CF Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:56752 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l0NhQ-0001Mn-Ui for qemu-devel@archiver.kernel.org; Fri, 15 Jan 2021 06:55:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46622) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0Ng4-0000bk-DQ; Fri, 15 Jan 2021 06:54:08 -0500 Received: from mail-eopbgr130123.outbound.protection.outlook.com ([40.107.13.123]:59230 helo=EUR01-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0Nfx-0006OH-9F; Fri, 15 Jan 2021 06:54:00 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EAITi5MfedTtrJuhg0jBytpIKyb7sW1BSW34/dGrF/fhAf/QA9ILQayFh4ff/2AE9EATxsdkiFU87YLbb130xRSa+MSOqrEWGClN901urgTnzP4v8bnF6zrzb6U+lAF4r07p4PQUaN7VVpImDlqQJlMhYJdmuuOcaSd7VCM5tACyws5/p00UmezvjLyQl2C/vLeoipEpkMTBaq+lrVZ3C3CutPwQLPgyVXxuivLMOmQ+aSS+FWyKLR212H1FjLIgGJo0IVeceK+6Si+52hqkovpQATay807Sxr8VbNcRyQohn4kBxpffcK42IdZZwdrxjevBwWeggrXXsHcfaNyRWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fSA175Ue4X4Pyd+JjRo3xxI7MAscX1s3Mjvx8lMZygA=; b=FpciHTQb6P+tWrMAaOVDu/lmvhJ7VYtf2xGAqABuy4pZTwHIp9/dxcCk/gbtvcCrof20JzCDt7Nu5T4sp8MPvQZUlwbnbYtcHHBH4hyCdvEnUZu5u6/nfCtWTl6QAqmsm1abmqETEL8QUbPSZ6VbOZ8137k23qqOhzdex+9ZAjb42DcaRS3YlwV9NaB8anje7Y5N4/ewAyhiM0egflDiwoA7GuHzsAYfeaAGQoN4Gtz7sTs0rlnKQht6+xzuE5kE4NetZaGHjfFOBkj8dYXRefcj1FXcjq5gk0XznPt074wKQB/bo9asOtIwF5oQvwQOavitFAa5FvNvoOnxqG3eZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fSA175Ue4X4Pyd+JjRo3xxI7MAscX1s3Mjvx8lMZygA=; b=qKAyKWJDpNIw3Iyosmipn8atYBEYu9zKm5vdsHT+f+D2bL5kPGG7WiAizu8bjpdTlTohlEiVUTfrL4ZxzDh46iZ7F9IJkujDzeouQJtqhdWBTXYbj0w0gbxStMwT2Q6MHvc2TalMmWfrZZIprKg+cd4Yeu+dKrW8Ce8O8tva+hQ= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM6PR08MB4070.eurprd08.prod.outlook.com (2603:10a6:20b:a3::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.11; Fri, 15 Jan 2021 11:53:50 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::214a:3545:368c:7ae8]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::214a:3545:368c:7ae8%9]) with mapi id 15.20.3763.011; Fri, 15 Jan 2021 11:53:49 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v3 11/10] iotests: add flake8 linter Date: Fri, 15 Jan 2021 14:53:16 +0300 Message-Id: <20210115115316.24633-1-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210114170304.87507-1-mreitz@redhat.com> References: <20210114170304.87507-1-mreitz@redhat.com> X-Originating-IP: [185.215.60.75] X-ClientProxiedBy: AM4PR07CA0026.eurprd07.prod.outlook.com (2603:10a6:205:1::39) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from kvm.sw.ru (185.215.60.75) by AM4PR07CA0026.eurprd07.prod.outlook.com (2603:10a6:205:1::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3784.6 via Frontend Transport; Fri, 15 Jan 2021 11:53:49 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9f04a844-bdae-4801-2d32-08d8b94c38e7 X-MS-TrafficTypeDiagnostic: AM6PR08MB4070: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1060; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zXiNa0gSNXIebVgHNS8umdHAPhX3UGhnSj/xi3PUFzEdGZ6RfFJJr9+PopySZepWFslZY+WfBCXRsx7JkBQWVHWtCpxAvzfnSiMrAabbGcEMLhKbBIt2ob8h6OqE4KMrjELp6+6xuVjxdx9JKn5j3+S95tolCGdMbk4eQAg39v23vZQkON4A8qyZ+e7sB8mCWartCDP7N6J5sQG7+eYQ59FjgzlwKGDIUv3OpCsB20nd3GUtIdbJDbKQPffX7jHSpgkekXF75oX6Uu9Cf0lIMOzblSTC0ePEUir8fz+4Peskalp3C6f/Bf4KPVI0+E4qdcHbQnpcgESedlKF507YuvFVcOPVj5OPG9CZ0piussk26R9VAiSZvmM5naS6d7RMGnwz0gi05ImygIYLjs7QkWLcaK5RoQPNvvPtBxF+8MTune9TByfMS5PPzycbZcAJ X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(366004)(346002)(376002)(39840400004)(396003)(956004)(1076003)(6666004)(26005)(16526019)(186003)(6486002)(86362001)(6916009)(6506007)(83380400001)(52116002)(36756003)(8936002)(5660300002)(478600001)(66556008)(107886003)(30864003)(66476007)(4326008)(316002)(2616005)(2906002)(8676002)(66946007)(6512007)(21314003); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: HuZDfds1eERwQOELKChHqQSSDLMvfwwF76PSXA60sV3xAamcrFfTZ4Qa9Z1ePoqZydXqMrq46/1p17FvqL5Y6XtVrSm4EWX+HHoV8R4cSkA1+P2SA07Rd/lMvfGEIUHayuExgx0pMieQnZIXC9PAkIlT3sa54ba4n0ELx4xBeOkYCb1Cy56o2LNbcPkkNtv2djXy1jHcGd2oUaBhO57tLDr7M5svzwKnUUwSsqUkCahMSv6Y0UVKyc6AVIL3R2iOMfBiGqPp5aeZ3uJsuu8V1yLryVLjnCvB+Y4mjQ79FTbrrBMa1f8Rp4mrBDI7raAvINOFL2YolIT/cEiItDWH/6W1+BHBXxvwVyCDLkk4Gd2Xl6rv0PtkOhjnZygq4CfWGyCapKkipKJWOFcYjJBtPA6l5jh4kBC8J/Rfylpp33ouhwMGDvrs1JGqbZ/3E46rEPDN0o8KP++jsT1bX8x+9ZiB5B7C+ixbqY36zI4G6WypAV7jZ+FHr6q+s7LpLML4Vuz2Ya0PmUOcBautdHR+HBKYnMLfvTP14vWigIwv96on+acX4cDraY+r04mgPnu1m4XFT3EaY0gL4dccWNHSNwQhXWHdQZ7qV8h0XnOBtONwdGTDnhQrN4quGQveGffwkH86JP+JkxqRfd/MtKUcmM+7vQW6NV8Gd/sSb458AE+K5kKGXPEGn7dlwpDlYX/nQYBCKAlSvqBTigvZd9RJcXNIE6gQ/jaGKK7GsKhiT/yzYmg8CE2njUBHbUBOq7oRGdffPQVtdz1vVDxePS4w9Iria02EmZefJa1st+hFoMJuV2EXBp0i6px1bs9018c/EJJo/vNMBgP11z1I0vqreP1zlvmOn7VvnoFMlV/BjbRl34NmOlDfBOq0SgLVf7oU+1ua7knGPzjwjdWkMd9b9TTWDuAZU6rcaiOgbhEsO8kZrbwQUzEzDD7c8aepCYo/PKLEmL6U03u8jkv4lJfJxLRqmpy4960FHv8YrbDLl011dB8U1C2onQEC9TdnePrB X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9f04a844-bdae-4801-2d32-08d8b94c38e7 X-MS-Exchange-CrossTenant-AuthSource: AM7PR08MB5494.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jan 2021 11:53:49.8970 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6igB29rHGsYygwIC6benv0I6cBSE9Ik8hVMtTHlYfRUISp5OylyGatAKL9eIh0Ah6HoBUfa6lFID29PvArFlwOhgnSH4nDLrOGzM/fvzavM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4070 Received-SPF: pass client-ip=40.107.13.123; envelope-from=vsementsov@virtuozzo.com; helo=EUR01-HE1-obe.outbound.protection.outlook.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" pylint is good, but doesn't cover the PEP8. Let's add flake8, to be sure that our code sutisfy PEP8. Add new linter and fix some code style in checked files. Signed-off-by: Vladimir Sementsov-Ogievskiy --- Hi! Here is my small addition to Max's series, hope you like it! Note, that this is not the first occurrence of 'flake8' in Qemu: # git grep flake8 python/qemu/.flake8:[flake8] scripts/qapi/.flake8:[flake8] scripts/qapi/.flake8:extend-ignore = E722 # Prefer pylint's bare-except checks to flake8's tests/qemu-iotests/129 | 6 ++- tests/qemu-iotests/254 | 2 +- tests/qemu-iotests/297 | 21 ++++++--- tests/qemu-iotests/297.out | 1 + tests/qemu-iotests/300 | 4 +- tests/qemu-iotests/iotests.py | 88 +++++++++++++++++++++++++++++++++-- 6 files changed, 106 insertions(+), 16 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 201d9e0a0b..28e6666c1d 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -21,6 +21,7 @@ import os import iotests + class TestStopWithBlockJob(iotests.QMPTestCase): test_img = os.path.join(iotests.test_dir, 'test.img') target_img = os.path.join(iotests.test_dir, 'target.img') @@ -39,8 +40,8 @@ class TestStopWithBlockJob(iotests.QMPTestCase): source_drive = 'driver=throttle,' \ 'node-name=source,' \ 'throttle-group=tg0,' \ - f'file.driver={iotests.imgfmt},' \ - f'file.file.filename={self.test_img}' + f'file.driver={iotests.imgfmt},' \ + f'file.file.filename={self.test_img}' self.vm.add_drive(None, source_drive) self.vm.launch() @@ -97,6 +98,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): self.do_test_stop('block-commit', device='drive0', top_node='source') + if __name__ == '__main__': iotests.main(supported_fmts=["qcow2"], supported_protocols=["file"]) diff --git a/tests/qemu-iotests/254 b/tests/qemu-iotests/254 index 150e58be8e..798f316e36 100755 --- a/tests/qemu-iotests/254 +++ b/tests/qemu-iotests/254 @@ -74,7 +74,7 @@ log("query-block: device = {}, node-name = {}, dirty-bitmaps:".format( result['device'], result['inserted']['node-name'])) log(result['dirty-bitmaps'], indent=2) log("\nbitmaps in backing image:") -log(result['inserted']['image']['backing-image']['format-specific'] \ +log(result['inserted']['image']['backing-image']['format-specific'] ['data']['bitmaps'], indent=2) vm.shutdown() diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 03d8604538..b79c341a3c 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -49,9 +49,10 @@ def is_python_file(filename): try: first_line = f.readline() return re.match('^#!.*python', first_line) is not None - except UnicodeDecodeError: # Ignore binary files + except UnicodeDecodeError: # Ignore binary files return False + def run_linters(): files = [filename for filename in (set(os.listdir('.')) - set(SKIP_FILES)) if is_python_file(filename)] @@ -59,16 +60,22 @@ def run_linters(): iotests.logger.debug('Files to be checked:') iotests.logger.debug(', '.join(sorted(files))) - print('=== pylint ===') - sys.stdout.flush() - - # Todo notes are fine, but fixme's or xxx's should probably just be - # fixed (in tests, at least) env = os.environ try: env['PYTHONPATH'] += ':../../python/' except KeyError: env['PYTHONPATH'] = '../../python/' + + print('=== flake8 ===') + sys.stdout.flush() + + subprocess.run(('flake8', *files), env=env, check=False) + + print('=== pylint ===') + sys.stdout.flush() + + # Todo notes are fine, but fixme's or xxx's should probably just be + # fixed (in tests, at least) subprocess.run(('pylint-3', '--score=n', '--notes=FIXME,XXX', *files), env=env, check=False) @@ -102,7 +109,7 @@ def run_linters(): print(p.stdout) -for linter in ('pylint-3', 'mypy'): +for linter in ('flake8', 'pylint-3', 'mypy'): if shutil.which(linter) is None: iotests.notrun(f'{linter} not found') diff --git a/tests/qemu-iotests/297.out b/tests/qemu-iotests/297.out index f2e1314d10..46bf3e665d 100644 --- a/tests/qemu-iotests/297.out +++ b/tests/qemu-iotests/297.out @@ -1,2 +1,3 @@ +=== flake8 === === pylint === === mypy === diff --git a/tests/qemu-iotests/300 b/tests/qemu-iotests/300 index 64d388a8fa..bbd227ff73 100755 --- a/tests/qemu-iotests/300 +++ b/tests/qemu-iotests/300 @@ -116,7 +116,7 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): self.vm_b.shutdown() log = self.vm_b.get_log() - assert log is not None # Loaded after shutdown + assert log is not None # Loaded after shutdown if msg is None: self.assertNotIn('qemu-system-', log) @@ -455,7 +455,7 @@ class TestBlockBitmapMappingErrors(TestDirtyBitmapMigration): self.vm_a.shutdown() log = self.vm_a.get_log() - assert log is not None # Loaded after shutdown + assert log is not None # Loaded after shutdown self.assertIn(f"Cannot migrate bitmap '{name}' on node " f"'{self.src_node_name}': Name is longer than 255 bytes", diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index a69b4cdc4e..e9cf8e6c28 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -37,8 +37,8 @@ from contextlib import contextmanager # pylint: disable=import-error, wrong-import-position sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) -from qemu import qtest -from qemu.qmp import QMPMessage +from qemu import qtest # noqa: E402 +from qemu.qmp import QMPMessage # noqa: E402 # Use this logger for logging messages directly from the iotests module logger = logging.getLogger('qemu.iotests') @@ -114,6 +114,7 @@ def qemu_tool_pipe_and_status(tool: str, args: Sequence[str], ' '.join(qemu_img_args + list(args)))) return (output, subp.returncode) + def qemu_img_pipe_and_status(*args: str) -> Tuple[str, int]: """ Run qemu-img and return both its output and its exit code @@ -121,10 +122,12 @@ def qemu_img_pipe_and_status(*args: str) -> Tuple[str, int]: full_args = qemu_img_args + list(args) return qemu_tool_pipe_and_status('qemu-img', full_args) + def qemu_img(*args: str) -> int: '''Run qemu-img and return the exit code''' return qemu_img_pipe_and_status(*args)[1] + def ordered_qmp(qmsg, conv_keys=True): # Dictionaries are not ordered prior to 3.6, therefore: if isinstance(qmsg, list): @@ -138,6 +141,7 @@ def ordered_qmp(qmsg, conv_keys=True): return od return qmsg + def qemu_img_create(*args): args = list(args) @@ -157,12 +161,15 @@ def qemu_img_create(*args): return qemu_img(*args) + def qemu_img_measure(*args): return json.loads(qemu_img_pipe("measure", "--output", "json", *args)) + def qemu_img_check(*args): return json.loads(qemu_img_pipe("check", "--output", "json", *args)) + def qemu_img_verbose(*args): '''Run qemu-img without suppressing its output and return the exit code''' exitcode = subprocess.call(qemu_img_args + list(args)) @@ -171,15 +178,18 @@ def qemu_img_verbose(*args): % (-exitcode, ' '.join(qemu_img_args + list(args)))) return exitcode + def qemu_img_pipe(*args: str) -> str: '''Run qemu-img and return its output''' return qemu_img_pipe_and_status(*args)[0] + def qemu_img_log(*args): result = qemu_img_pipe(*args) log(result, filters=[filter_testfiles]) return result + def img_info_log(filename, filter_path=None, imgopts=False, extra_args=()): args = ['info'] if imgopts: @@ -194,6 +204,7 @@ def img_info_log(filename, filter_path=None, imgopts=False, extra_args=()): filter_path = filename log(filter_img_info(output, filter_path)) + def qemu_io(*args): '''Run qemu-io and return the stdout data''' args = qemu_io_args + list(args) @@ -206,11 +217,13 @@ def qemu_io(*args): % (-subp.returncode, ' '.join(args))) return output + def qemu_io_log(*args): result = qemu_io(*args) log(result, filters=[filter_testfiles, filter_qemu_io]) return result + def qemu_io_silent(*args): '''Run qemu-io and return the exit code, suppressing stdout''' if '-f' in args or '--image-opts' in args: @@ -225,6 +238,7 @@ def qemu_io_silent(*args): (-exitcode, ' '.join(args))) return exitcode + def qemu_io_silent_check(*args): '''Run qemu-io and return the true if subprocess returned 0''' args = qemu_io_args + list(args) @@ -232,11 +246,13 @@ def qemu_io_silent_check(*args): stderr=subprocess.STDOUT) return exitcode == 0 + def get_virtio_scsi_device(): if qemu_default_machine == 's390-ccw-virtio': return 'virtio-scsi-ccw' return 'virtio-scsi-pci' + class QemuIoInteractive: def __init__(self, *args): self.args = qemu_io_args_no_fmt + list(args) @@ -286,6 +302,7 @@ def qemu_nbd(*args): '''Run qemu-nbd in daemon mode and return the parent's exit code''' return subprocess.call(qemu_nbd_args + ['--fork'] + list(args)) + def qemu_nbd_early_pipe(*args: str) -> Tuple[int, str]: '''Run qemu-nbd in daemon mode and return both the parent's exit code and its output in case of an error''' @@ -294,6 +311,7 @@ def qemu_nbd_early_pipe(*args: str) -> Tuple[int, str]: connect_stderr=False) return returncode, output if returncode else '' + def qemu_nbd_list_log(*args: str) -> str: '''Run qemu-nbd to list remote exports''' full_args = [qemu_nbd_prog, '-L'] + list(args) @@ -301,6 +319,7 @@ def qemu_nbd_list_log(*args: str) -> str: log(output, filters=[filter_testfiles, filter_nbd_exports]) return output + @contextmanager def qemu_nbd_popen(*args): '''Context manager running qemu-nbd within the context''' @@ -326,11 +345,13 @@ def qemu_nbd_popen(*args): p.kill() p.wait() + def compare_images(img1, img2, fmt1=imgfmt, fmt2=imgfmt): '''Return True if two image files are identical''' return qemu_img('compare', '-f', fmt1, '-F', fmt2, img1, img2) == 0 + def create_image(name, size): '''Create a fully-allocated raw image with sector markers''' file = open(name, 'wb') @@ -341,34 +362,49 @@ def create_image(name, size): i = i + 512 file.close() + def image_size(img): '''Return image's virtual size''' r = qemu_img_pipe('info', '--output=json', '-f', imgfmt, img) return json.loads(r)['virtual-size'] + def is_str(val): return isinstance(val, str) + test_dir_re = re.compile(r"%s" % test_dir) + + def filter_test_dir(msg): return test_dir_re.sub("TEST_DIR", msg) + win32_re = re.compile(r"\r") + + def filter_win32(msg): return win32_re.sub("", msg) + qemu_io_re = re.compile(r"[0-9]* ops; [0-9\/:. sec]* " r"\([0-9\/.inf]* [EPTGMKiBbytes]*\/sec " r"and [0-9\/.inf]* ops\/sec\)") + + def filter_qemu_io(msg): msg = filter_win32(msg) return qemu_io_re.sub("X ops; XX:XX:XX.X " "(XXX YYY/sec and XXX ops/sec)", msg) + chown_re = re.compile(r"chown [0-9]+:[0-9]+") + + def filter_chown(msg): return chown_re.sub("chown UID:GID", msg) + def filter_qmp_event(event): '''Filter a QMP event dict''' event = dict(event) @@ -377,6 +413,7 @@ def filter_qmp_event(event): event['timestamp']['microseconds'] = 'USECS' return event + def filter_qmp(qmsg, filter_fn): '''Given a string filter, filter a QMP object's values. filter_fn takes a (key, value) pair.''' @@ -393,11 +430,13 @@ def filter_qmp(qmsg, filter_fn): qmsg[k] = filter_fn(k, v) return qmsg + def filter_testfiles(msg): pref1 = os.path.join(test_dir, "%s-" % (os.getpid())) pref2 = os.path.join(sock_dir, "%s-" % (os.getpid())) return msg.replace(pref1, 'TEST_DIR/PID-').replace(pref2, 'SOCK_DIR/PID-') + def filter_qmp_testfiles(qmsg): def _filter(_key, value): if is_str(value): @@ -405,9 +444,11 @@ def filter_qmp_testfiles(qmsg): return value return filter_qmp(qmsg, _filter) + def filter_virtio_scsi(output: str) -> str: return re.sub(r'(virtio-scsi)-(ccw|pci)', r'\1', output) + def filter_qmp_virtio_scsi(qmsg): def _filter(_key, value): if is_str(value): @@ -415,9 +456,11 @@ def filter_qmp_virtio_scsi(qmsg): return value return filter_qmp(qmsg, _filter) + def filter_generated_node_ids(msg): return re.sub("#block[0-9]+", "NODE_NAME", msg) + def filter_img_info(output, filename): lines = [] for line in output.split('\n'): @@ -434,9 +477,11 @@ def filter_img_info(output, filename): lines.append(line) return '\n'.join(lines) + def filter_imgfmt(msg): return msg.replace(imgfmt, 'IMGFMT') + def filter_qmp_imgfmt(qmsg): def _filter(_key, value): if is_str(value): @@ -444,12 +489,14 @@ def filter_qmp_imgfmt(qmsg): return value return filter_qmp(qmsg, _filter) + def filter_nbd_exports(output: str) -> str: return re.sub(r'((min|opt|max) block): [0-9]+', r'\1: XXX', output) Msg = TypeVar('Msg', Dict[str, Any], List[Any], str) + def log(msg: Msg, filters: Iterable[Callable[[Msg], Msg]] = (), indent: Optional[int] = None) -> None: @@ -466,23 +513,29 @@ def log(msg: Msg, else: test_logger.info(msg) + class Timeout: def __init__(self, seconds, errmsg="Timeout"): self.seconds = seconds self.errmsg = errmsg + def __enter__(self): signal.signal(signal.SIGALRM, self.timeout) signal.setitimer(signal.ITIMER_REAL, self.seconds) return self + def __exit__(self, exc_type, value, traceback): signal.setitimer(signal.ITIMER_REAL, 0) return False + def timeout(self, signum, frame): raise Exception(self.errmsg) + def file_pattern(name): return "{0}-{1}".format(os.getpid(), name) + class FilePath: """ Context manager generating multiple file names. The generated files are @@ -529,6 +582,7 @@ def try_remove(img): except OSError: pass + def file_path_remover(): for path in reversed(file_path_remover.paths): try_remove(path) @@ -556,6 +610,7 @@ def file_path(*names, base_dir=test_dir): return paths[0] if len(paths) == 1 else paths + def remote_filename(path): if imgproto == 'file': return path @@ -564,6 +619,7 @@ def remote_filename(path): else: raise Exception("Protocol %s not supported" % (imgproto)) + class VM(qtest.QEMUQtestMachine): '''A QEMU VM''' @@ -664,7 +720,7 @@ class VM(qtest.QEMUQtestMachine): for key in obj: self.flatten_qmp_object(obj[key], output, basestr + key + '.') else: - output[basestr[:-1]] = obj # Strip trailing '.' + output[basestr[:-1]] = obj # Strip trailing '.' return output def qmp_to_opts(self, obj): @@ -884,8 +940,10 @@ class VM(qtest.QEMUQtestMachine): 'Found node %s under %s (but expected %s)' % \ (node['name'], path, expected_node) + index_re = re.compile(r'([^\[]+)\[([^\]]+)\]') + class QMPTestCase(unittest.TestCase): '''Abstract base class for QMP test cases''' @@ -992,7 +1050,6 @@ class QMPTestCase(unittest.TestCase): elif event['event'] == 'JOB_STATUS_CHANGE': self.assert_qmp(event, 'data/id', drive) - self.assert_no_active_block_jobs() return result @@ -1078,6 +1135,7 @@ def notrun(reason): logger.warning("%s not run: %s", seq, reason) sys.exit(0) + def case_notrun(reason): '''Mark this test case as not having been run (without actually skipping it, that is left to the caller). See @@ -1090,6 +1148,7 @@ def case_notrun(reason): open('%s/%s.casenotrun' % (output_dir, seq), 'a').write( ' [case not run] ' + reason + '\n') + def _verify_image_format(supported_fmts: Sequence[str] = (), unsupported_fmts: Sequence[str] = ()) -> None: if 'generic' in supported_fmts and \ @@ -1106,6 +1165,7 @@ def _verify_image_format(supported_fmts: Sequence[str] = (), if imgfmt == 'luks': verify_working_luks() + def _verify_protocol(supported: Sequence[str] = (), unsupported: Sequence[str] = ()) -> None: assert not (supported and unsupported) @@ -1117,6 +1177,7 @@ def _verify_protocol(supported: Sequence[str] = (), if not_sup or (imgproto in unsupported): notrun('not suitable for this protocol: %s' % imgproto) + def _verify_platform(supported: Sequence[str] = (), unsupported: Sequence[str] = ()) -> None: if any((sys.platform.startswith(x) for x in unsupported)): @@ -1126,27 +1187,33 @@ def _verify_platform(supported: Sequence[str] = (), if not any((sys.platform.startswith(x) for x in supported)): notrun('not suitable for this OS: %s' % sys.platform) + def _verify_cache_mode(supported_cache_modes: Sequence[str] = ()) -> None: if supported_cache_modes and (cachemode not in supported_cache_modes): notrun('not suitable for this cache mode: %s' % cachemode) + def _verify_aio_mode(supported_aio_modes: Sequence[str] = ()) -> None: if supported_aio_modes and (aiomode not in supported_aio_modes): notrun('not suitable for this aio mode: %s' % aiomode) + def _verify_formats(required_formats: Sequence[str] = ()) -> None: usf_list = list(set(required_formats) - set(supported_formats())) if usf_list: notrun(f'formats {usf_list} are not whitelisted') + def supports_quorum(): return 'quorum' in qemu_img_pipe('--help') + def verify_quorum(): '''Skip test suite if quorum support is not available''' if not supports_quorum(): notrun('quorum support missing') + def has_working_luks() -> Tuple[bool, str]: """ Check whether our LUKS driver can actually create images @@ -1178,6 +1245,7 @@ def has_working_luks() -> Tuple[bool, str]: else: return (True, '') + def verify_working_luks(): """ Skip test suite if LUKS does not work @@ -1186,6 +1254,7 @@ def verify_working_luks(): if not working: notrun(reason) + def qemu_pipe(*args: str) -> str: """ Run qemu with an option to print something and exit (e.g. a help option). @@ -1196,6 +1265,7 @@ def qemu_pipe(*args: str) -> str: output, _ = qemu_tool_pipe_and_status('qemu', full_args) return output + def supported_formats(read_only=False): '''Set 'read_only' to True to check ro-whitelist Otherwise, rw-whitelist is checked''' @@ -1211,6 +1281,7 @@ def supported_formats(read_only=False): return supported_formats.formats[read_only] + def skip_if_unsupported(required_formats=(), read_only=False): '''Skip Test Decorator Runs the test if all the required formats are whitelisted''' @@ -1231,6 +1302,7 @@ def skip_if_unsupported(required_formats=(), read_only=False): return func_wrapper return skip_test_decorator + def skip_for_formats(formats: Sequence[str] = ()) \ -> Callable[[Callable[[QMPTestCase, List[Any], Dict[str, Any]], None]], Callable[[QMPTestCase, List[Any], Dict[str, Any]], None]]: @@ -1247,6 +1319,7 @@ def skip_for_formats(formats: Sequence[str] = ()) \ return func_wrapper return skip_test_decorator + def skip_if_user_is_root(func): '''Skip Test Decorator Runs the test only without root permissions''' @@ -1258,6 +1331,7 @@ def skip_if_user_is_root(func): return func(*args, **kwargs) return func_wrapper + def execute_unittest(debug=False): """Executes unittests within the calling module.""" @@ -1290,6 +1364,7 @@ def execute_unittest(debug=False): sys.stderr.write(out) + def execute_setup_common(supported_fmts: Sequence[str] = (), supported_platforms: Sequence[str] = (), supported_cache_modes: Sequence[str] = (), @@ -1319,6 +1394,7 @@ def execute_setup_common(supported_fmts: Sequence[str] = (), return debug + def execute_test(*args, test_function=None, **kwargs): """Run either unittest or script-style tests.""" @@ -1328,6 +1404,7 @@ def execute_test(*args, test_function=None, **kwargs): else: test_function() + def activate_logging(): """Activate iotests.log() output to stdout for script-style tests.""" handler = logging.StreamHandler(stream=sys.stdout) @@ -1337,18 +1414,21 @@ def activate_logging(): test_logger.setLevel(logging.INFO) test_logger.propagate = False + # This is called from script-style iotests without a single point of entry def script_initialize(*args, **kwargs): """Initialize script-style tests without running any tests.""" activate_logging() execute_setup_common(*args, **kwargs) + # This is called from script-style iotests with a single point of entry def script_main(test_function, *args, **kwargs): """Run script-style tests outside of the unittest framework""" activate_logging() execute_test(*args, test_function=test_function, **kwargs) + # This is called from unittest style iotests def main(*args, **kwargs): """Run tests using the unittest framework"""