From patchwork Wed Jan 13 17:57:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12017481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EC13C433DB for ; Wed, 13 Jan 2021 18:04:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AB0BE217BA for ; Wed, 13 Jan 2021 18:04:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB0BE217BA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:57626 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kzkVl-0006nH-I1 for qemu-devel@archiver.kernel.org; Wed, 13 Jan 2021 13:04:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49982) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kzkPG-0001fh-HP for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:06 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:24393) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kzkPC-0000tZ-QT for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610560680; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JF5NaNd9qbTnGxc5ZHwbkkpbYpE9YgCzet4pLF9ZTac=; b=c/mEQhubgA0oGY5cJnb1WQ7semCGmhBllWENyHBUMiTRUutF80Iz68EOqxMgG+qiXM6+I8 HAVkuWwKfYRVg4UMFmPYDVm22C5KL2WH0fT2N+eAIQ5/vWVeXq1/LlbbLjaEXiqm+qYpo7 5b0rOaWkJ1KV00Y3/Va4zt3t9blog3I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-399-52rQalgpMkCUVBcCLkEquA-1; Wed, 13 Jan 2021 12:57:58 -0500 X-MC-Unique: 52rQalgpMkCUVBcCLkEquA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D6C021014E78; Wed, 13 Jan 2021 17:57:56 +0000 (UTC) Received: from localhost (ovpn-113-103.ams2.redhat.com [10.36.113.103]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6AD4E19C47; Wed, 13 Jan 2021 17:57:56 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v2 1/8] iotests/297: Allow checking all Python test files Date: Wed, 13 Jan 2021 18:57:45 +0100 Message-Id: <20210113175752.403022-2-mreitz@redhat.com> In-Reply-To: <20210113175752.403022-1-mreitz@redhat.com> References: <20210113175752.403022-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" I.e., all Python files in the qemu-iotests/ directory. Most files of course do not pass, so there is an extensive skip list for now. (The only files that do pass are 209, 254, 283, and iotests.py.) (Alternatively, we could have the opposite, i.e. an explicit list of files that we do want to check, but I think it is better to check files by default.) I decided to include the list of files checked in the reference output, so we do not accidentally lose coverage of anything. That means adding new Python tests will require a change to 297.out, but that should not be a problem. On the other hand, I decided to hide mypy's "Success" lines from the reference output, because they do not add anything useful. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/297 | 66 ++++++++++++++++++++++++++++++++++---- tests/qemu-iotests/297.out | 6 +++- 2 files changed, 65 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 5c5420712b..b1a7d6d5e8 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -30,13 +30,67 @@ if ! type -p "mypy" > /dev/null; then _notrun "mypy not found" fi -pylint-3 --score=n iotests.py +# TODO: Empty this list! +skip_files=( + 030 040 041 044 045 055 056 057 065 093 096 118 124 129 132 136 139 147 148 + 149 151 152 155 163 165 169 194 196 199 202 203 205 206 207 208 210 211 212 + 213 216 218 219 222 224 228 234 235 236 237 238 240 242 245 246 248 255 256 + 257 258 260 262 264 266 274 277 280 281 295 296 298 299 300 302 303 304 307 + nbd-fault-injector.py qcow2.py qcow2_format.py qed.py +) -MYPYPATH=../../python/ mypy --warn-unused-configs --disallow-subclassing-any \ - --disallow-any-generics --disallow-incomplete-defs \ - --disallow-untyped-decorators --no-implicit-optional \ - --warn-redundant-casts --warn-unused-ignores \ - --no-implicit-reexport iotests.py +file_list=() +for file in *; do + # Check files with a .py extension or a Python shebang + # (Unless they are in the skip_files list) + if [ -f "$file" ] && ((echo "$file" | grep -q '\.py$') || + (head -n 1 "$file" | grep -q '^#!.*python')) + then + skip_file=false + for skip in "${skip_files[@]}"; do + if [ "$skip" = "$file" ]; then + skip_file=true + break + fi + done + + if ! $skip_file; then + file_list+=("$file") + fi + fi +done + +# Emit list of all files that are checked so we will not accidentally +# lose coverage +echo 'Files to be checked:' + +file_list_str='' +for file in "${file_list[@]}"; do + echo " $file" +done | sort + +# We can pass all files to pylint at once... +pylint-3 --score=n "${file_list[@]}" + +# ...but mypy needs to be called once per file. Otherwise, it will +# interpret all given files as belonging together (i.e., they may not +# both define the same classes, etc.; most notably, they must not both +# define the __main__ module). +for file in "${file_list[@]}"; do + mypy_output=$( + MYPYPATH=../../python/ mypy --warn-unused-configs \ + --disallow-subclassing-any \ + --disallow-any-generics --disallow-incomplete-defs \ + --disallow-untyped-decorators --no-implicit-optional \ + --warn-redundant-casts --warn-unused-ignores \ + --no-implicit-reexport "$file" \ + 2>&1 + ) + + if [ $? != 0 ]; then + echo "$mypy_output" + fi +done # success, all done echo "*** done" diff --git a/tests/qemu-iotests/297.out b/tests/qemu-iotests/297.out index 6acc843649..c5ebbf6a17 100644 --- a/tests/qemu-iotests/297.out +++ b/tests/qemu-iotests/297.out @@ -1,3 +1,7 @@ QA output created by 297 -Success: no issues found in 1 source file +Files to be checked: + 209 + 254 + 283 + iotests.py *** done From patchwork Wed Jan 13 17:57:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12017477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97859C433DB for ; Wed, 13 Jan 2021 18:00:02 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0065821534 for ; Wed, 13 Jan 2021 18:00:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0065821534 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50712 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kzkR6-0003Zr-DN for qemu-devel@archiver.kernel.org; Wed, 13 Jan 2021 13:00:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49978) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kzkPG-0001eu-Cr for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:06 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:44127) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kzkPE-0000vA-Bk for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610560683; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rwDLgbif8bpJ7F2UY/z831fx0M5z2L/TjfKn/9WlPTQ=; b=IIPt8qRPJN2Z44Bzbx6AiM066CWGsjbepquyG0Gdi++em5LUYsUBPB0bDxt0ro5+pWyZAH V4gv3j+VI35tUPdct/lA2OewfBDt5eel9mfJvtnRhRQQ8CcNqBg/sr9y0UPOO9A15uMZ00 KbQxDql3Z2cdeZAMqhq6F1Vsb5zPX/0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-75-DTGP0LwhNKi1UDXd93RdfA-1; Wed, 13 Jan 2021 12:58:00 -0500 X-MC-Unique: DTGP0LwhNKi1UDXd93RdfA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3F18D8066E0; Wed, 13 Jan 2021 17:57:59 +0000 (UTC) Received: from localhost (ovpn-113-103.ams2.redhat.com [10.36.113.103]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CC84660BF1; Wed, 13 Jan 2021 17:57:58 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v2 2/8] iotests: Move try_remove to iotests.py Date: Wed, 13 Jan 2021 18:57:46 +0100 Message-Id: <20210113175752.403022-3-mreitz@redhat.com> In-Reply-To: <20210113175752.403022-1-mreitz@redhat.com> References: <20210113175752.403022-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/124 | 8 +------- tests/qemu-iotests/iotests.py | 11 +++++++---- 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 index 3705cbb6b3..e40eeb50b9 100755 --- a/tests/qemu-iotests/124 +++ b/tests/qemu-iotests/124 @@ -22,6 +22,7 @@ import os import iotests +from iotests import try_remove def io_write_patterns(img, patterns): @@ -29,13 +30,6 @@ def io_write_patterns(img, patterns): iotests.qemu_io('-c', 'write -P%s %s %s' % pattern, img) -def try_remove(img): - try: - os.remove(img) - except OSError: - pass - - def transaction_action(action, **kwargs): return { 'type': action, diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index dcdcd0387f..15704ca466 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -515,12 +515,15 @@ class FilePath: return False +def try_remove(img): + try: + os.remove(img) + except OSError: + pass + def file_path_remover(): for path in reversed(file_path_remover.paths): - try: - os.remove(path) - except OSError: - pass + try_remove(path) def file_path(*names, base_dir=test_dir): From patchwork Wed Jan 13 17:57:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12017483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CFB2C433E0 for ; Wed, 13 Jan 2021 18:04:52 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7E4FC230F9 for ; Wed, 13 Jan 2021 18:04:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7E4FC230F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:57702 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kzkVm-0006p5-BN for qemu-devel@archiver.kernel.org; Wed, 13 Jan 2021 13:04:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50008) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kzkPH-0001j8-JP for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:07 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:53705) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kzkPF-0000wM-71 for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610560684; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LxANx+Rz2agcyu5X1D3Ejr1ptPnBeUWQBvphznPKVvk=; b=ejlVgxaFADl2bzlVaiqyvU6F51Q9OdSW+lGt3LgKUGZ4xLP1rxgLP7/JKtI6eeOs+vpzun D0XvVqWz+e/AvlFSmjJIRmbWuNetjsRyRmZH+ORq8iY/yyaPp9PdpJSLt8pEGv84YX4tcK de1IbyNjANdKcpTDlx2HS9T0jxyn6j4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-281-hEdWS9etOj-w0wTPeN3qqQ-1; Wed, 13 Jan 2021 12:58:02 -0500 X-MC-Unique: hEdWS9etOj-w0wTPeN3qqQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B4FD2612AB; Wed, 13 Jan 2021 17:58:01 +0000 (UTC) Received: from localhost (ovpn-113-103.ams2.redhat.com [10.36.113.103]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3C8F15C3E5; Wed, 13 Jan 2021 17:58:01 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v2 3/8] iotests/129: Remove test images in tearDown() Date: Wed, 13 Jan 2021 18:57:47 +0100 Message-Id: <20210113175752.403022-4-mreitz@redhat.com> In-Reply-To: <20210113175752.403022-1-mreitz@redhat.com> References: <20210113175752.403022-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- tests/qemu-iotests/129 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 0e13244d85..2fc65ada6a 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -47,6 +47,8 @@ class TestStopWithBlockJob(iotests.QMPTestCase): result = self.vm.qmp("block_set_io_throttle", conv_keys=False, **params) self.vm.shutdown() + for img in (self.test_img, self.target_img, self.base_img): + iotests.try_remove(img) def do_test_stop(self, cmd, **args): """Test 'stop' while block job is running on a throttled drive. From patchwork Wed Jan 13 17:57:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12017479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C9E2C433DB for ; Wed, 13 Jan 2021 18:00:17 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D98F721534 for ; Wed, 13 Jan 2021 18:00:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D98F721534 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:51744 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kzkRL-00041h-R9 for qemu-devel@archiver.kernel.org; Wed, 13 Jan 2021 13:00:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50032) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kzkPJ-0001oQ-Hp for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:09 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:60408) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kzkPH-0000yQ-Te for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610560687; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0sOKI0PkBbpsD1lkPyyE95je+xXaX1EDNteAdaV4lNM=; b=Cu0BhaAWIU3paQdv+tsYwQ+rinMcZctParuOSF3jQBAm1MaZ7CFZUnu0CCoY+Ga65RnesK A2Bj6Tp3Jl1415R7dV8wAi4r5hFk5y1pXaZvt4P7KqRMBGmrZ16m1HzeUngzPtdUs5cmJj pIe9XS5dgJSe558naPA+2r0DKy4y/g4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-446-SNEesIsgPNuYu_FxQrt9kw-1; Wed, 13 Jan 2021 12:58:05 -0500 X-MC-Unique: SNEesIsgPNuYu_FxQrt9kw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3E326802B45; Wed, 13 Jan 2021 17:58:04 +0000 (UTC) Received: from localhost (ovpn-113-103.ams2.redhat.com [10.36.113.103]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B9AEA60938; Wed, 13 Jan 2021 17:58:03 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v2 4/8] iotests/129: Do not check @busy Date: Wed, 13 Jan 2021 18:57:48 +0100 Message-Id: <20210113175752.403022-5-mreitz@redhat.com> In-Reply-To: <20210113175752.403022-1-mreitz@redhat.com> References: <20210113175752.403022-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" @busy is false when the job is paused, which happens all the time because that is how jobs yield (e.g. for mirror at least since commit 565ac01f8d3). Back when 129 was added (2015), perhaps there was no better way of checking whether the job was still actually running. Now we have the @status field (as of 58b295ba52c, i.e. 2018), which can give us exactly that information. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 2fc65ada6a..dd23bb2e5a 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -69,7 +69,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): result = self.vm.qmp("stop") self.assert_qmp(result, 'return', {}) result = self.vm.qmp("query-block-jobs") - self.assert_qmp(result, 'return[0]/busy', True) + self.assert_qmp(result, 'return[0]/status', 'running') self.assert_qmp(result, 'return[0]/ready', False) def test_drive_mirror(self): From patchwork Wed Jan 13 17:57:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12017493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80B95C433DB for ; Wed, 13 Jan 2021 18:12:53 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CDAB823435 for ; Wed, 13 Jan 2021 18:12:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CDAB823435 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:45088 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kzkdX-0005mu-Nb for qemu-devel@archiver.kernel.org; Wed, 13 Jan 2021 13:12:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50098) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kzkPP-0001yt-Ln for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:16 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:21108) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kzkPN-00010O-Cx for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610560692; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hVw7md4Lj6AOU0w+kotowNbYNWj6T7IA5qZebkNO3ko=; b=QIJEuV2NsazkWbxCmPNjn5vnYsACio/6lVdXKMBrOgvkMoFKUaqC3STWqzsZ0j0qxv4rgs HjdJmFoaUeQThbES/xJzfua+s2oOwiAzvkkH0rB/G9tqvLfw4CktcJmqtxjRf7bIw/DW3C HMMyAPGMxGdmcjZl+a/Tj5/df7NE5L8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-90-YevLG2L3M36xsYvazWwIOA-1; Wed, 13 Jan 2021 12:58:08 -0500 X-MC-Unique: YevLG2L3M36xsYvazWwIOA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D92AC802B40; Wed, 13 Jan 2021 17:58:06 +0000 (UTC) Received: from localhost (ovpn-113-103.ams2.redhat.com [10.36.113.103]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 722475B4BA; Wed, 13 Jan 2021 17:58:06 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v2 5/8] iotests/129: Use throttle node Date: Wed, 13 Jan 2021 18:57:49 +0100 Message-Id: <20210113175752.403022-6-mreitz@redhat.com> In-Reply-To: <20210113175752.403022-1-mreitz@redhat.com> References: <20210113175752.403022-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 37 +++++++++++++------------------------ 1 file changed, 13 insertions(+), 24 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index dd23bb2e5a..58536bc6ee 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -32,20 +32,18 @@ class TestStopWithBlockJob(iotests.QMPTestCase): iotests.qemu_img('create', '-f', iotests.imgfmt, self.test_img, "-b", self.base_img, '-F', iotests.imgfmt) iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', self.test_img) - self.vm = iotests.VM().add_drive(self.test_img) + self.vm = iotests.VM() + self.vm.add_object('throttle-group,id=tg0,x-bps-total=1024') + + source_drive = 'driver=throttle,' \ + 'throttle-group=tg0,' \ + f'file.driver={iotests.imgfmt},' \ + f'file.file.filename={self.test_img}' + + self.vm.add_drive(None, source_drive) self.vm.launch() def tearDown(self): - params = {"device": "drive0", - "bps": 0, - "bps_rd": 0, - "bps_wr": 0, - "iops": 0, - "iops_rd": 0, - "iops_wr": 0, - } - result = self.vm.qmp("block_set_io_throttle", conv_keys=False, - **params) self.vm.shutdown() for img in (self.test_img, self.target_img, self.base_img): iotests.try_remove(img) @@ -53,33 +51,24 @@ class TestStopWithBlockJob(iotests.QMPTestCase): def do_test_stop(self, cmd, **args): """Test 'stop' while block job is running on a throttled drive. The 'stop' command shouldn't drain the job""" - params = {"device": "drive0", - "bps": 1024, - "bps_rd": 0, - "bps_wr": 0, - "iops": 0, - "iops_rd": 0, - "iops_wr": 0, - } - result = self.vm.qmp("block_set_io_throttle", conv_keys=False, - **params) - self.assert_qmp(result, 'return', {}) result = self.vm.qmp(cmd, **args) self.assert_qmp(result, 'return', {}) + result = self.vm.qmp("stop") self.assert_qmp(result, 'return', {}) result = self.vm.qmp("query-block-jobs") + self.assert_qmp(result, 'return[0]/status', 'running') self.assert_qmp(result, 'return[0]/ready', False) def test_drive_mirror(self): self.do_test_stop("drive-mirror", device="drive0", - target=self.target_img, + target=self.target_img, format=iotests.imgfmt, sync="full") def test_drive_backup(self): self.do_test_stop("drive-backup", device="drive0", - target=self.target_img, + target=self.target_img, format=iotests.imgfmt, sync="full") def test_block_commit(self): From patchwork Wed Jan 13 17:57:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12017487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14921C433DB for ; Wed, 13 Jan 2021 18:08:02 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 74125233ED for ; Wed, 13 Jan 2021 18:08:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 74125233ED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36772 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kzkYq-000206-Av for qemu-devel@archiver.kernel.org; Wed, 13 Jan 2021 13:08:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50088) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kzkPP-0001ys-Au for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:16 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:21070) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kzkPM-0000zp-Qa for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610560692; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iVnHNMSZS/bcYc9NyCWat/9jmqgK0G82ubZDwoGFKUE=; b=E3Va5gV5LCIqLTY06o7llTJX8MHTwIF69Ezp3++BmgseTc7jFbrmn0Q2YRxAKWom8gNN1h sE6ArjYd+DVsEsdfGXgbrKon1VV8f2o6QrJteYJmEiOwuW1TFjq9+iw7kxQPH+ZgE3/454 2sCnfB58yxM5qY26Bn16Tw/6SSHKskE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-344-9X7XAjlNOzmk724i0YeIPA-1; Wed, 13 Jan 2021 12:58:10 -0500 X-MC-Unique: 9X7XAjlNOzmk724i0YeIPA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 74CF6802B46; Wed, 13 Jan 2021 17:58:09 +0000 (UTC) Received: from localhost (ovpn-113-103.ams2.redhat.com [10.36.113.103]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EC98B5D9F4; Wed, 13 Jan 2021 17:58:08 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v2 6/8] iotests/129: Actually test a commit job Date: Wed, 13 Jan 2021 18:57:50 +0100 Message-Id: <20210113175752.403022-7-mreitz@redhat.com> In-Reply-To: <20210113175752.403022-1-mreitz@redhat.com> References: <20210113175752.403022-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Before this patch, test_block_commit() performs an active commit, which under the hood is a mirror job. If we want to test various different block jobs, we should perhaps run an actual commit job instead. Doing so requires adding an overlay above the source node before the commit is done (and then specifying the source node as the top node for the commit job). Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 58536bc6ee..7b4b6649f0 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -26,6 +26,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): test_img = os.path.join(iotests.test_dir, 'test.img') target_img = os.path.join(iotests.test_dir, 'target.img') base_img = os.path.join(iotests.test_dir, 'base.img') + overlay_img = os.path.join(iotests.test_dir, 'overlay.img') def setUp(self): iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, "1G") @@ -36,6 +37,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): self.vm.add_object('throttle-group,id=tg0,x-bps-total=1024') source_drive = 'driver=throttle,' \ + 'node-name=source,' \ 'throttle-group=tg0,' \ f'file.driver={iotests.imgfmt},' \ f'file.file.filename={self.test_img}' @@ -45,7 +47,8 @@ class TestStopWithBlockJob(iotests.QMPTestCase): def tearDown(self): self.vm.shutdown() - for img in (self.test_img, self.target_img, self.base_img): + for img in (self.test_img, self.target_img, self.base_img, + self.overlay_img): iotests.try_remove(img) def do_test_stop(self, cmd, **args): @@ -72,7 +75,27 @@ class TestStopWithBlockJob(iotests.QMPTestCase): sync="full") def test_block_commit(self): - self.do_test_stop("block-commit", device="drive0") + # Add overlay above the source node so that we actually use a + # commit job instead of a mirror job + + iotests.qemu_img('create', '-f', iotests.imgfmt, self.overlay_img, + '1G') + + result = self.vm.qmp('blockdev-add', **{ + 'node-name': 'overlay', + 'driver': iotests.imgfmt, + 'file': { + 'driver': 'file', + 'filename': self.overlay_img + } + }) + self.assert_qmp(result, 'return', {}) + + result = self.vm.qmp('blockdev-snapshot', + node='source', overlay='overlay') + self.assert_qmp(result, 'return', {}) + + self.do_test_stop('block-commit', device='drive0', top_node='source') if __name__ == '__main__': iotests.main(supported_fmts=["qcow2"], From patchwork Wed Jan 13 17:57:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12017489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C841C433DB for ; Wed, 13 Jan 2021 18:08:08 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EDD63233ED for ; Wed, 13 Jan 2021 18:08:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EDD63233ED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:37294 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kzkYw-0002E4-V0 for qemu-devel@archiver.kernel.org; Wed, 13 Jan 2021 13:08:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50144) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kzkPR-000206-Nh for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:18 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:37113) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kzkPP-00010w-Ga for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610560694; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bU+AGmGeqHFzTcyXBp/no/OR5o6fbi0ENmf3b/ZaRlQ=; b=FAIR5fiMLInlLJNoezW8ksO+2P3/1LwNaU+zd/hKYhXJK89SaIGlZB+17fbvHV5ckuMx/G Hmvb+85ijomQsA//OqJy8UXfb3wNxv4y3LOKapxIeT24ERZpWe5YgFY8q/tOj5APAOG3Hz 9fC1FPtLSFUFXCRlzXNRCUdtkWt0Bq4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-65-TARGPk3bPqyXJ4EMnkkEOA-1; Wed, 13 Jan 2021 12:58:12 -0500 X-MC-Unique: TARGPk3bPqyXJ4EMnkkEOA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D656018C89C4; Wed, 13 Jan 2021 17:58:11 +0000 (UTC) Received: from localhost (ovpn-113-103.ams2.redhat.com [10.36.113.103]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7216660C0F; Wed, 13 Jan 2021 17:58:11 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v2 7/8] iotests/129: Limit mirror job's buffer size Date: Wed, 13 Jan 2021 18:57:51 +0100 Message-Id: <20210113175752.403022-8-mreitz@redhat.com> In-Reply-To: <20210113175752.403022-1-mreitz@redhat.com> References: <20210113175752.403022-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Issuing 'stop' on the VM drains all nodes. If the mirror job has many large requests in flight, this may lead to significant I/O that looks a bit like 'stop' would make the job try to complete (which is what 129 should verify not to happen). We can limit the I/O in flight by limiting the buffer size, so mirror will make very little progress during the 'stop' drain. (We do not need to do anything about commit, which has a buffer size of 512 kB by default; or backup, which goes cluster by cluster. Once we have asynchronous requests for backup, that will change, but then we can fine-tune the backup job to only perform a single request on a very small chunk, too.) Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 7b4b6649f0..6d21470cd7 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -67,7 +67,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): def test_drive_mirror(self): self.do_test_stop("drive-mirror", device="drive0", target=self.target_img, format=iotests.imgfmt, - sync="full") + sync="full", buf_size=65536) def test_drive_backup(self): self.do_test_stop("drive-backup", device="drive0", From patchwork Wed Jan 13 17:57:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 12017485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1452BC433E0 for ; Wed, 13 Jan 2021 18:05:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 71655233ED for ; Wed, 13 Jan 2021 18:05:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 71655233ED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:59904 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kzkWn-0007sk-Cu for qemu-devel@archiver.kernel.org; Wed, 13 Jan 2021 13:05:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50180) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kzkPU-00021d-3h for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:20 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:58636) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kzkPR-00012D-QO for qemu-devel@nongnu.org; Wed, 13 Jan 2021 12:58:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610560697; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zAr8NGcSB0K1UzQorV+b3CqRa/5TFojC7jgHvo+KKeY=; b=UxBevN+ggYAMcOaoIfboKRT4CujOA5HEn3seRoBsWkPRJGARBG2+WnVP5hlVQh3uKsM/Hh H/KgKYi730af4aMyp+iB1CeURTdRG6C4XukCKtUrv4SchvpmP1Afu9ncM/ggs4DMTBdkJv 4tThi1kTmZi7yBYFkoML6an5j/9y4Ag= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-221-02efRLyENhin50p0EmhsDA-1; Wed, 13 Jan 2021 12:58:15 -0500 X-MC-Unique: 02efRLyENhin50p0EmhsDA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 762F41081B32; Wed, 13 Jan 2021 17:58:14 +0000 (UTC) Received: from localhost (ovpn-113-103.ams2.redhat.com [10.36.113.103]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0D2B75C3E0; Wed, 13 Jan 2021 17:58:13 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH v2 8/8] iotests/129: Clean up pylint and mypy complaints Date: Wed, 13 Jan 2021 18:57:52 +0100 Message-Id: <20210113175752.403022-9-mreitz@redhat.com> In-Reply-To: <20210113175752.403022-1-mreitz@redhat.com> References: <20210113175752.403022-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=mreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 6 ++++-- tests/qemu-iotests/297 | 2 +- tests/qemu-iotests/297.out | 1 + 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 6d21470cd7..64578493c1 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -20,9 +20,10 @@ import os import iotests -import time class TestStopWithBlockJob(iotests.QMPTestCase): + assert iotests.test_dir is not None + test_img = os.path.join(iotests.test_dir, 'test.img') target_img = os.path.join(iotests.test_dir, 'target.img') base_img = os.path.join(iotests.test_dir, 'base.img') @@ -32,7 +33,8 @@ class TestStopWithBlockJob(iotests.QMPTestCase): iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, "1G") iotests.qemu_img('create', '-f', iotests.imgfmt, self.test_img, "-b", self.base_img, '-F', iotests.imgfmt) - iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', self.test_img) + iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', + self.test_img) self.vm = iotests.VM() self.vm.add_object('throttle-group,id=tg0,x-bps-total=1024') diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index b1a7d6d5e8..88f00415c8 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -32,7 +32,7 @@ fi # TODO: Empty this list! skip_files=( - 030 040 041 044 045 055 056 057 065 093 096 118 124 129 132 136 139 147 148 + 030 040 041 044 045 055 056 057 065 093 096 118 124 132 136 139 147 148 149 151 152 155 163 165 169 194 196 199 202 203 205 206 207 208 210 211 212 213 216 218 219 222 224 228 234 235 236 237 238 240 242 245 246 248 255 256 257 258 260 262 264 266 274 277 280 281 295 296 298 299 300 302 303 304 307 diff --git a/tests/qemu-iotests/297.out b/tests/qemu-iotests/297.out index c5ebbf6a17..92cae940c5 100644 --- a/tests/qemu-iotests/297.out +++ b/tests/qemu-iotests/297.out @@ -1,5 +1,6 @@ QA output created by 297 Files to be checked: + 129 209 254 283