From patchwork Mon Oct 28 12:13:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215251 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1F031390 for ; Mon, 28 Oct 2019 12:18:42 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B8B792086D for ; Mon, 28 Oct 2019 12:18:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XQc2qqvY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B8B792086D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53112 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3yr-0005RC-59 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:18:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49707) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vU-000169-7D for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vS-00018X-VY for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:12 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:47099 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vS-00017t-RW for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264910; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WZKGwwrQNIsZCQmXWzekNWbQo7lQgOe1Uzx83/bgdcA=; b=XQc2qqvYSUaxq0UB4r8+0hLVjnF4hN8LviXHwzwKoUA/WSzmno1tTDpRah2Gpn4CI3k3Kp HHTRyhkGOS1NJyUdT/Z1zhD9VdQfzjMJaybWDKXd86YZJhqG43JInfhjrfpqaGfOSH7AI9 0v0yYyMN1B8kOGdjWiEjOcoHt4In1B0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-128-cAv2Zm1INu-ZXoyXqgbtYQ-1; Mon, 28 Oct 2019 08:15:06 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D25F7800FF1; Mon, 28 Oct 2019 12:15:05 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 78793261A7; Mon, 28 Oct 2019 12:15:05 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 01/69] iotests: Prefer null-co over null-aio Date: Mon, 28 Oct 2019 13:13:53 +0100 Message-Id: <20191028121501.15279-2-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: cAv2Zm1INu-ZXoyXqgbtYQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We use null-co basically everywhere in the iotests. Unless we want to test null-aio specifically, we should use it instead (for consistency). Signed-off-by: Max Reitz Reviewed-by: John Snow Reviewed-by: Thomas Huth Message-id: 20190917092004.999-2-mreitz@redhat.com Reviewed-by: Andrey Shinkevich Signed-off-by: Max Reitz --- tests/qemu-iotests/093 | 7 +++---- tests/qemu-iotests/245 | 2 +- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/093 b/tests/qemu-iotests/093 index 3c4f5173ce..50c1e7f2ec 100755 --- a/tests/qemu-iotests/093 +++ b/tests/qemu-iotests/093 @@ -267,13 +267,12 @@ class ThrottleTestCoroutine(ThrottleTestCase): test_img = "null-co://" class ThrottleTestGroupNames(iotests.QMPTestCase): - test_img = "null-aio://" max_drives = 3 def setUp(self): self.vm = iotests.VM() for i in range(0, self.max_drives): - self.vm.add_drive(self.test_img, + self.vm.add_drive("null-co://", "throttling.iops-total=100,file.read-zeroes=on") self.vm.launch() @@ -376,10 +375,10 @@ class ThrottleTestRemovableMedia(iotests.QMPTestCase): def test_removable_media(self): # Add a couple of dummy nodes named cd0 and cd1 - result = self.vm.qmp("blockdev-add", driver="null-aio", + result = self.vm.qmp("blockdev-add", driver="null-co", read_zeroes=True, node_name="cd0") self.assert_qmp(result, 'return', {}) - result = self.vm.qmp("blockdev-add", driver="null-aio", + result = self.vm.qmp("blockdev-add", driver="null-co", read_zeroes=True, node_name="cd1") self.assert_qmp(result, 'return', {}) diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245 index 41218d5f1d..e66a23c5f0 100644 --- a/tests/qemu-iotests/245 +++ b/tests/qemu-iotests/245 @@ -598,7 +598,7 @@ class TestBlockdevReopen(iotests.QMPTestCase): ################## ###### null ###### ################## - opts = {'driver': 'null-aio', 'node-name': 'root', 'size': 1024} + opts = {'driver': 'null-co', 'node-name': 'root', 'size': 1024} result = self.vm.qmp('blockdev-add', conv_keys = False, **opts) self.assert_qmp(result, 'return', {}) From patchwork Mon Oct 28 12:13:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215247 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D01313BD for ; Mon, 28 Oct 2019 12:18:20 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 631B921721 for ; Mon, 28 Oct 2019 12:18:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MundnMn2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 631B921721 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53108 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3yV-0004ti-1Q for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:18:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49724) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vU-00016K-TK for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vT-00019L-JB for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:12 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:50716 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vT-00018c-F9 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264910; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0BJR7+7aH2YQU3yk/3SlY1tFahOTGnVjCwU3veQoNec=; b=MundnMn23qQ4cSmoeJSL/y28wCG5MPGGGVlAIPKF2a//BQYCar950nz/GD5DFkk9wU8eQX 0PtxeXMPlU2RfF6yvXrYv+fS9JxEkRkN6A91BJeIvflSRPY93bZ7fQ/VXxRCyJV67qdwQF XLsdExZwv+Rph5ETjB00nkyvc3jQWWU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-213-YR7kYPSGNJiHZbe_2KEh0Q-1; Mon, 28 Oct 2019 08:15:08 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EC2248017DD; Mon, 28 Oct 2019 12:15:07 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8CEB05C219; Mon, 28 Oct 2019 12:15:07 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 02/69] iotests: Allow skipping test cases Date: Mon, 28 Oct 2019 13:13:54 +0100 Message-Id: <20191028121501.15279-3-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: YR7kYPSGNJiHZbe_2KEh0Q-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" case_notrun() does not actually skip the current test case. It just adds a "notrun" note and then returns to the caller, who manually has to skip the test. Generally, skipping a test case is as simple as returning from the current function, but not always: For example, this model does not allow skipping tests already in the setUp() function. Thus, add a QMPTestCase.case_skip() function that invokes case_notrun() and then self.skipTest(). To make this work, we need to filter the information on how many test cases were skipped from the unittest output. Signed-off-by: Max Reitz Reviewed-by: John Snow Reviewed-by: Andrey Shinkevich Message-id: 20190917092004.999-3-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 709def4d5d..6d5de6e504 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -838,6 +838,11 @@ class QMPTestCase(unittest.TestCase): return self.pause_wait(job_id) return result + def case_skip(self, reason): + '''Skip this test case''' + case_notrun(reason) + self.skipTest(reason) + def notrun(reason): '''Skip this test suite''' @@ -849,7 +854,11 @@ def notrun(reason): sys.exit(0) def case_notrun(reason): - '''Skip this test case''' + '''Mark this test case as not having been run (without actually + skipping it, that is left to the caller). See + QMPTestCase.case_skip() for a variant that actually skips the + current test case.''' + # Each test in qemu-iotests has a number ("seq") seq = os.path.basename(sys.argv[0]) @@ -950,8 +959,15 @@ def execute_unittest(output, verbosity, debug): unittest.main(testRunner=runner) finally: if not debug: - sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s', - r'Ran \1 tests', output.getvalue())) + out = output.getvalue() + out = re.sub(r'Ran (\d+) tests? in [\d.]+s', r'Ran \1 tests', out) + + # Hide skipped tests from the reference output + out = re.sub(r'OK \(skipped=\d+\)', 'OK', out) + out_first_line, out_rest = out.split('\n', 1) + out = out_first_line.replace('s', '.') + '\n' + out_rest + + sys.stderr.write(out) def execute_test(test_function=None, supported_fmts=[], supported_oses=['linux'], From patchwork Mon Oct 28 12:13:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215261 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 606EA1747 for ; Mon, 28 Oct 2019 12:26:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3627520717 for ; Mon, 28 Oct 2019 12:26:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="aqZX4s5N" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3627520717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53176 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP45v-0007Nr-0L for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:25:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49763) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vZ-0001D8-9W for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vY-0001Ch-5S for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:17 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:45943 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vY-0001CQ-1O for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264915; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+sFl7vp6h1iBNHEgQncQy6nm9F7BxXtnEy0wC1s13J0=; b=aqZX4s5NE7EWFZY/3QWUbhLS3GzWnKP+ebuJa66uMMONXD4A4U4sQfQRFS8nT6nskDRo2U joRBxS7UG3l01/HfULkkoS7tak+n5mDX2eFRHB4lXBy/w2NZAQwVJYNpo5aeIRAHXAsha1 V7tdYpWQcsSTgJwcycCvU1BS18smnXU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-48-yb6o2CfcPuqUOdIp_EUw8A-1; Mon, 28 Oct 2019 08:15:11 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 17D6A180496E; Mon, 28 Oct 2019 12:15:10 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A92FA100164D; Mon, 28 Oct 2019 12:15:09 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 03/69] iotests: Use case_skip() in skip_if_unsupported() Date: Mon, 28 Oct 2019 13:13:55 +0100 Message-Id: <20191028121501.15279-4-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: yb6o2CfcPuqUOdIp_EUw8A-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" skip_if_unsupported() should use the stronger variant case_skip(), because this allows it to be used even with setUp() (in a meaningful way). In the process, make it explicit what we expect the first argument of the func_wrapper to be (namely something derived of QMPTestCase). Signed-off-by: Max Reitz Message-id: 20190917092004.999-4-mreitz@redhat.com Reviewed-by: Andrey Shinkevich Reviewed-by: John Snow Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 6d5de6e504..bd867d7e02 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -929,14 +929,14 @@ def skip_if_unsupported(required_formats=[], read_only=False): '''Skip Test Decorator Runs the test if all the required formats are whitelisted''' def skip_test_decorator(func): - def func_wrapper(*args, **kwargs): + def func_wrapper(test_case: QMPTestCase, *args, **kwargs): usf_list = list(set(required_formats) - set(supported_formats(read_only))) if usf_list: - case_notrun('{}: formats {} are not whitelisted'.format( - args[0], usf_list)) + test_case.case_skip('{}: formats {} are not whitelisted'.format( + test_case, usf_list)) else: - return func(*args, **kwargs) + return func(test_case, *args, **kwargs) return func_wrapper return skip_test_decorator From patchwork Mon Oct 28 12:13:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215253 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7251F1390 for ; Mon, 28 Oct 2019 12:22:38 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 48ACB20873 for ; Mon, 28 Oct 2019 12:22:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BLbJjUvN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 48ACB20873 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53142 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP42e-0003IB-Qa for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:22:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49753) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vY-0001BR-Ju for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vX-0001CF-Cb for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:16 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:50637 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vX-0001C0-8x for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264914; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qbC9UZ0xsPhtoSiYs/wfvSdprO+PyukdEqG/LpCyRNM=; b=BLbJjUvN+dETbK1sS7QEGzdYXSnYKzOfFSHXt/iV8N3wNEZdLSHrnepk2Uv/39vYMZF1Bq v+yq49Tuv41eomqObGAZOQhbYtjJa11XwsDd4TlGZ84zn9GUvMZTFtCra2iVO0HNo6mQs/ Ja8SkG8MtdSJ7WMIH+coovBSdt1IBrA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-247-uYABZKqNPJembGVFDraTBw-1; Mon, 28 Oct 2019 08:15:13 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4954080183E; Mon, 28 Oct 2019 12:15:12 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D759C5D6BB; Mon, 28 Oct 2019 12:15:11 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 04/69] iotests: Let skip_if_unsupported accept a function Date: Mon, 28 Oct 2019 13:13:56 +0100 Message-Id: <20191028121501.15279-5-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-MC-Unique: uYABZKqNPJembGVFDraTBw-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" This lets tests use skip_if_unsupported() with a potentially variable list of required formats. Suggested-by: Kevin Wolf Signed-off-by: Max Reitz Message-id: 20190917092004.999-5-mreitz@redhat.com Reviewed-by: Andrey Shinkevich Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index bd867d7e02..936d33df61 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -930,8 +930,12 @@ def skip_if_unsupported(required_formats=[], read_only=False): Runs the test if all the required formats are whitelisted''' def skip_test_decorator(func): def func_wrapper(test_case: QMPTestCase, *args, **kwargs): - usf_list = list(set(required_formats) - - set(supported_formats(read_only))) + if callable(required_formats): + fmts = required_formats(test_case) + else: + fmts = required_formats + + usf_list = list(set(fmts) - set(supported_formats(read_only))) if usf_list: test_case.case_skip('{}: formats {} are not whitelisted'.format( test_case, usf_list)) From patchwork Mon Oct 28 12:13:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215255 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D44961390 for ; Mon, 28 Oct 2019 12:22:56 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AAB6D20717 for ; Mon, 28 Oct 2019 12:22:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ZBI9lVCu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AAB6D20717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53144 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP42w-0003Up-GW for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:22:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49860) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vh-0001K3-T5 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vf-0001GG-VI for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:25 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:41009 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vf-0001FY-Qv for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L6HzeEpFuvx1s/KTUCZtpigth3fiHipwxY43BzIrOrA=; b=ZBI9lVCuphdvWLGeXHR0Ut3w1r1Dy202lg/8Csh2QOxfYSnmb5mkxE6TaA5pznGwjopGgq CEn4xJxOM9A9W3+l4+bItZQr/4Dw+8ewFbFKG5Mv/m2X2PHQ4F3rLcgEO+mSLtxXw6rWRM OQ3NeJPlsb+87J5JshD0yIV8oxO3kos= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-336-a0n2Vi_aN4-Apw-U4dIW4w-1; Mon, 28 Oct 2019 08:15:15 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5790180183E; Mon, 28 Oct 2019 12:15:14 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E601926186; Mon, 28 Oct 2019 12:15:13 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 05/69] iotests: Test driver whitelisting in 093 Date: Mon, 28 Oct 2019 13:13:57 +0100 Message-Id: <20191028121501.15279-6-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: a0n2Vi_aN4-Apw-U4dIW4w-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" null-aio may not be whitelisted. Skip all test cases that require it. (And skip the whole test if null-co is not whitelisted.) Signed-off-by: Max Reitz Message-id: 20190917092004.999-6-mreitz@redhat.com Reviewed-by: Andrey Shinkevich Reviewed-by: John Snow Signed-off-by: Max Reitz --- tests/qemu-iotests/093 | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/093 b/tests/qemu-iotests/093 index 50c1e7f2ec..f03fa24a07 100755 --- a/tests/qemu-iotests/093 +++ b/tests/qemu-iotests/093 @@ -24,7 +24,7 @@ import iotests nsec_per_sec = 1000000000 class ThrottleTestCase(iotests.QMPTestCase): - test_img = "null-aio://" + test_driver = "null-aio" max_drives = 3 def blockstats(self, device): @@ -35,10 +35,14 @@ class ThrottleTestCase(iotests.QMPTestCase): return stat['rd_bytes'], stat['rd_operations'], stat['wr_bytes'], stat['wr_operations'] raise Exception("Device not found for blockstats: %s" % device) + def required_drivers(self): + return [self.test_driver] + + @iotests.skip_if_unsupported(required_drivers) def setUp(self): self.vm = iotests.VM() for i in range(0, self.max_drives): - self.vm.add_drive(self.test_img, "file.read-zeroes=on") + self.vm.add_drive(self.test_driver + "://", "file.read-zeroes=on") self.vm.launch() def tearDown(self): @@ -264,7 +268,7 @@ class ThrottleTestCase(iotests.QMPTestCase): self.assertEqual(self.blockstats('drive1')[0], 4096) class ThrottleTestCoroutine(ThrottleTestCase): - test_img = "null-co://" + test_driver = "null-co" class ThrottleTestGroupNames(iotests.QMPTestCase): max_drives = 3 @@ -425,4 +429,6 @@ class ThrottleTestRemovableMedia(iotests.QMPTestCase): if __name__ == '__main__': + if 'null-co' not in iotests.supported_formats(): + iotests.notrun('null-co driver support missing') iotests.main(supported_fmts=["raw"]) From patchwork Mon Oct 28 12:13:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215249 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 302741390 for ; Mon, 28 Oct 2019 12:18:41 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 073572086D for ; Mon, 28 Oct 2019 12:18:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WeJK+Lgq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 073572086D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53110 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3yp-0005NA-H1 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:18:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49820) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vf-0001JL-VK for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vd-0001FQ-RQ for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:23 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:36878 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vd-0001F0-ON for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264920; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6pjfgNJz2k72QEAD4z+ODEgWsNy3o8hO+FNn9sL+rQI=; b=WeJK+LgqW4XofimM5WRZTZ9DE5zKyboEGMUS0fjNr9/Yubhe0J382sPOCc02AFcnWEvv8W X7kagaqHYA6NR5wEJ7O64qMK5org4gkFVFQOb3JjWkHdwfnViDpGQyrpyfY7/+kCMkicBo c7Ppu4yn/6G29vaVY1bVCB6muJDg8Yg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-346-77lUsRrcMeuFOCK66w99Ig-1; Mon, 28 Oct 2019 08:15:17 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 659CA10052E0; Mon, 28 Oct 2019 12:15:16 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0987C5DA8C; Mon, 28 Oct 2019 12:15:15 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 06/69] iotests: Test driver whitelisting in 136 Date: Mon, 28 Oct 2019 13:13:58 +0100 Message-Id: <20191028121501.15279-7-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: 77lUsRrcMeuFOCK66w99Ig-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" null-aio may not be whitelisted. Skip all test cases that require it. Signed-off-by: Max Reitz Message-id: 20190917092004.999-7-mreitz@redhat.com Reviewed-by: Andrey Shinkevich Reviewed-by: John Snow Signed-off-by: Max Reitz --- tests/qemu-iotests/136 | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/136 b/tests/qemu-iotests/136 index a46a7b7630..012ea111ac 100755 --- a/tests/qemu-iotests/136 +++ b/tests/qemu-iotests/136 @@ -30,7 +30,7 @@ bad_offset = bad_sector * 512 blkdebug_file = os.path.join(iotests.test_dir, 'blkdebug.conf') class BlockDeviceStatsTestCase(iotests.QMPTestCase): - test_img = "null-aio://" + test_driver = "null-aio" total_rd_bytes = 0 total_rd_ops = 0 total_wr_bytes = 0 @@ -67,6 +67,10 @@ sector = "%d" ''' % (bad_sector, bad_sector)) file.close() + def required_drivers(self): + return [self.test_driver] + + @iotests.skip_if_unsupported(required_drivers) def setUp(self): drive_args = [] drive_args.append("stats-intervals.0=%d" % interval_length) @@ -76,8 +80,8 @@ sector = "%d" (self.account_failed and "on" or "off")) drive_args.append("file.image.read-zeroes=on") self.create_blkdebug_file() - self.vm = iotests.VM().add_drive('blkdebug:%s:%s' % - (blkdebug_file, self.test_img), + self.vm = iotests.VM().add_drive('blkdebug:%s:%s://' % + (blkdebug_file, self.test_driver), ','.join(drive_args)) self.vm.launch() # Set an initial value for the clock @@ -337,7 +341,9 @@ class BlockDeviceStatsTestAccountBoth(BlockDeviceStatsTestCase): account_failed = True class BlockDeviceStatsTestCoroutine(BlockDeviceStatsTestCase): - test_img = "null-co://" + test_driver = "null-co" if __name__ == '__main__': + if 'null-co' not in iotests.supported_formats(): + iotests.notrun('null-co driver support missing') iotests.main(supported_fmts=["raw"]) From patchwork Mon Oct 28 12:13:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215257 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8A6E413BD for ; Mon, 28 Oct 2019 12:23:12 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 60CCF20717 for ; Mon, 28 Oct 2019 12:23:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="M/oR6vD0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 60CCF20717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53148 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP43D-0003uW-6O for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:23:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49857) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vh-0001K1-Ta for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vg-0001GX-6V for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:25 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:22466 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vf-0001Fx-Tf for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264923; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Kl79FN+OL3HPkPGAYUaT5f+1b0u1rlEs1rN9DSJ5JBg=; b=M/oR6vD0VTBQ9sgnk41nV5I/Oygn24s+5iVKqV9xopMaRXpd+dlre5ofcOavj74BFexy9n IKZb/NGhmgkqlY4BWXsBJ/Vf1EQujesDwBFxthNMOd+M17PZFA4zqftHLIk7txayvdL2A4 4Bmj3iFaqthennIQIz0lpUGDyCi2Egg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-124-ldqXuYncNYG1xdc0aV4nPQ-1; Mon, 28 Oct 2019 08:15:19 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 728AD10052E1; Mon, 28 Oct 2019 12:15:18 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 13CB4100164D; Mon, 28 Oct 2019 12:15:17 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 07/69] iotests: Cache supported_formats() Date: Mon, 28 Oct 2019 13:13:59 +0100 Message-Id: <20191028121501.15279-8-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: ldqXuYncNYG1xdc0aV4nPQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: John Snow Message-id: 20190917092004.999-8-mreitz@redhat.com Reviewed-by: Andrey Shinkevich Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 936d33df61..a4a2238b42 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -921,9 +921,17 @@ def qemu_pipe(*args): def supported_formats(read_only=False): '''Set 'read_only' to True to check ro-whitelist Otherwise, rw-whitelist is checked''' - format_message = qemu_pipe("-drive", "format=help") - line = 1 if read_only else 0 - return format_message.splitlines()[line].split(":")[1].split() + + if not hasattr(supported_formats, "formats"): + supported_formats.formats = {} + + if read_only not in supported_formats.formats: + format_message = qemu_pipe("-drive", "format=help") + line = 1 if read_only else 0 + supported_formats.formats[read_only] = \ + format_message.splitlines()[line].split(":")[1].split() + + return supported_formats.formats[read_only] def skip_if_unsupported(required_formats=[], read_only=False): '''Skip Test Decorator From patchwork Mon Oct 28 12:14:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215263 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E246413BD for ; Mon, 28 Oct 2019 12:26:33 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B7C5720717 for ; Mon, 28 Oct 2019 12:26:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Py3Jr1yc" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B7C5720717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53178 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP46S-00083q-Lq for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:26:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49892) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vj-0001KY-Cb for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vh-0001Ha-Sv for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:27 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:32941 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vh-0001HD-Px for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264925; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+u3lp8jOpNBLwLVCT1zHP7gb5DU6N9uK+QT6kLBH7ew=; b=Py3Jr1yc3iFmNlOVE7w9HvLLPqIyImwKi1OMXksOA69zAnH1hSpMZnQZjn5fSd2tinG825 byAkoDeWj6wHvjC7vGi0AGEF9dZ773yEQjlcT+F54wb1qgJ9d43tLnPGaCXp4v8W+f1fKr Sjkqf+BBHUMhP37GQW3Kj48WoB90/Ck= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-65-calFd69FMcKkOYYSNPTMAQ-1; Mon, 28 Oct 2019 08:15:21 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 802B5800FF1; Mon, 28 Oct 2019 12:15:20 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1EFF4600C7; Mon, 28 Oct 2019 12:15:19 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 08/69] hbitmap: handle set/reset with zero length Date: Mon, 28 Oct 2019 13:14:00 +0100 Message-Id: <20191028121501.15279-9-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: calFd69FMcKkOYYSNPTMAQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Passing zero length to these functions leads to unpredicted results. Zero-length set/reset may occur in active-mirror, on zero-length write (which is unlikely, but not guaranteed to never happen). Let's just do nothing on zero-length request. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20191011090711.19940-2-vsementsov@virtuozzo.com Reviewed-by: Max Reitz Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz --- util/hbitmap.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/util/hbitmap.c b/util/hbitmap.c index 66db87c6ff..242c6e519c 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -387,6 +387,10 @@ void hbitmap_set(HBitmap *hb, uint64_t start, uint64_t count) uint64_t first, n; uint64_t last = start + count - 1; + if (count == 0) { + return; + } + trace_hbitmap_set(hb, start, count, start >> hb->granularity, last >> hb->granularity); @@ -478,6 +482,10 @@ void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t count) uint64_t last = start + count - 1; uint64_t gran = 1ULL << hb->granularity; + if (count == 0) { + return; + } + assert(QEMU_IS_ALIGNED(start, gran)); assert(QEMU_IS_ALIGNED(count, gran) || (start + count == hb->orig_size)); From patchwork Mon Oct 28 12:14:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215271 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C126D13BD for ; Mon, 28 Oct 2019 12:30:56 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 975C5208C0 for ; Mon, 28 Oct 2019 12:30:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KQKtHJ1e" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 975C5208C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53212 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Ah-0005hT-FD for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:30:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49914) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vl-0001LM-Te for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vj-0001In-Sx for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:29 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:60480 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vj-0001IR-P5 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264927; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f5OQWPv79KD3SNwPjN2Ee0ud2g7tdqc31S9r0UcfX68=; b=KQKtHJ1eDWmG7YMsXZDlssPsRDJsAJqgtbQWFvbcfEp3XVoJT8D/nLMoCEqDANgMzqozUG qgQD5Jkzl0gbnP4RRoL4h0v6hIDZ0WP5p67HT3k97QZgbW+UaleBBwmIwXjg/Zj2Sv7YHw G/ZeX7od6gd0LEvROidKMLsDi20LKks= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-261-QEBbO9jgPAysu4tq7Yks2g-1; Mon, 28 Oct 2019 08:15:23 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9344581A334; Mon, 28 Oct 2019 12:15:22 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3892F60BF7; Mon, 28 Oct 2019 12:15:22 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 09/69] block/mirror: simplify do_sync_target_write Date: Mon, 28 Oct 2019 13:14:01 +0100 Message-Id: <20191028121501.15279-10-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: QEBbO9jgPAysu4tq7Yks2g-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy do_sync_target_write is called from bdrv_mirror_top_do_write after write/discard operation, all inside active_write/active_write_settle protecting us from mirror iteration. So the whole area is dirty for sure, no reason to examine dirty bitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20191011090711.19940-3-vsementsov@virtuozzo.com Signed-off-by: Max Reitz --- block/mirror.c | 95 +++++++++++++++----------------------------------- 1 file changed, 28 insertions(+), 67 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index a6c50caea4..351faf9367 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1181,84 +1181,45 @@ do_sync_target_write(MirrorBlockJob *job, MirrorMethod method, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { - QEMUIOVector target_qiov; - uint64_t dirty_offset = offset; - uint64_t dirty_bytes; - - if (qiov) { - qemu_iovec_init(&target_qiov, qiov->niov); - } - - while (true) { - bool valid_area; - int ret; - - bdrv_dirty_bitmap_lock(job->dirty_bitmap); - dirty_bytes = MIN(offset + bytes - dirty_offset, INT_MAX); - valid_area = bdrv_dirty_bitmap_next_dirty_area(job->dirty_bitmap, - &dirty_offset, - &dirty_bytes); - if (!valid_area) { - bdrv_dirty_bitmap_unlock(job->dirty_bitmap); - break; - } + int ret; - bdrv_reset_dirty_bitmap_locked(job->dirty_bitmap, - dirty_offset, dirty_bytes); - bdrv_dirty_bitmap_unlock(job->dirty_bitmap); + bdrv_reset_dirty_bitmap(job->dirty_bitmap, offset, bytes); - job_progress_increase_remaining(&job->common.job, dirty_bytes); + job_progress_increase_remaining(&job->common.job, bytes); - assert(dirty_offset - offset <= SIZE_MAX); - if (qiov) { - qemu_iovec_reset(&target_qiov); - qemu_iovec_concat(&target_qiov, qiov, - dirty_offset - offset, dirty_bytes); - } - - switch (method) { - case MIRROR_METHOD_COPY: - ret = blk_co_pwritev(job->target, dirty_offset, dirty_bytes, - qiov ? &target_qiov : NULL, flags); - break; + switch (method) { + case MIRROR_METHOD_COPY: + ret = blk_co_pwritev(job->target, offset, bytes, qiov, flags); + break; - case MIRROR_METHOD_ZERO: - assert(!qiov); - ret = blk_co_pwrite_zeroes(job->target, dirty_offset, dirty_bytes, - flags); - break; + case MIRROR_METHOD_ZERO: + assert(!qiov); + ret = blk_co_pwrite_zeroes(job->target, offset, bytes, flags); + break; - case MIRROR_METHOD_DISCARD: - assert(!qiov); - ret = blk_co_pdiscard(job->target, dirty_offset, dirty_bytes); - break; + case MIRROR_METHOD_DISCARD: + assert(!qiov); + ret = blk_co_pdiscard(job->target, offset, bytes); + break; - default: - abort(); - } + default: + abort(); + } - if (ret >= 0) { - job_progress_update(&job->common.job, dirty_bytes); - } else { - BlockErrorAction action; + if (ret >= 0) { + job_progress_update(&job->common.job, bytes); + } else { + BlockErrorAction action; - bdrv_set_dirty_bitmap(job->dirty_bitmap, dirty_offset, dirty_bytes); - job->actively_synced = false; + bdrv_set_dirty_bitmap(job->dirty_bitmap, offset, bytes); + job->actively_synced = false; - action = mirror_error_action(job, false, -ret); - if (action == BLOCK_ERROR_ACTION_REPORT) { - if (!job->ret) { - job->ret = ret; - } - break; + action = mirror_error_action(job, false, -ret); + if (action == BLOCK_ERROR_ACTION_REPORT) { + if (!job->ret) { + job->ret = ret; } } - - dirty_offset += dirty_bytes; - } - - if (qiov) { - qemu_iovec_destroy(&target_qiov); } } From patchwork Mon Oct 28 12:14:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215269 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BEBC91599 for ; Mon, 28 Oct 2019 12:30:14 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 93D3320717 for ; Mon, 28 Oct 2019 12:30:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="W/G9vZdy" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 93D3320717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53210 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4A1-0004UK-84 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:30:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49930) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vo-0001OB-0p for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vm-0001K2-Ku for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:31 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:27848 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vm-0001Jv-HO for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264930; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1u7kdgL2BXgYlqlkUPf9AKtLZyipQBK0M1JmEjfhCWE=; b=W/G9vZdy6m+be1ev23y9cxQhXkzRcAJxlNCdT3fxEfuKBQ+27Kf8GHwTsaFDjQ7907Kcp7 llsdTOrIsfSzQyDxnPhznY8GXxVRIxDLVCce+1LNp+IJkQ+BVxJfOsPiCV7O68qu65c4aJ kxLw72SIW3Hzuf//yr6ZysLPSvTBpIc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-239-ZvY9vhjMM7qS95j5-pW0qw-1; Mon, 28 Oct 2019 08:15:25 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 974A6107AD28; Mon, 28 Oct 2019 12:15:24 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3F04A600C7; Mon, 28 Oct 2019 12:15:24 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 10/69] block/block-backend: add blk_co_pwritev_part Date: Mon, 28 Oct 2019 13:14:02 +0100 Message-Id: <20191028121501.15279-11-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: ZvY9vhjMM7qS95j5-pW0qw-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Add blk write function with qiov_offset parameter. It's needed for the following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20191011090711.19940-4-vsementsov@virtuozzo.com Signed-off-by: Max Reitz --- include/sysemu/block-backend.h | 4 ++++ block/block-backend.c | 17 +++++++++++++---- 2 files changed, 17 insertions(+), 4 deletions(-) diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h index 368d53af77..73f2cef7fe 100644 --- a/include/sysemu/block-backend.h +++ b/include/sysemu/block-backend.h @@ -121,6 +121,10 @@ void blk_set_dev_ops(BlockBackend *blk, const BlockDevOps *ops, void *opaque); int coroutine_fn blk_co_preadv(BlockBackend *blk, int64_t offset, unsigned int bytes, QEMUIOVector *qiov, BdrvRequestFlags flags); +int coroutine_fn blk_co_pwritev_part(BlockBackend *blk, int64_t offset, + unsigned int bytes, + QEMUIOVector *qiov, size_t qiov_offset, + BdrvRequestFlags flags); int coroutine_fn blk_co_pwritev(BlockBackend *blk, int64_t offset, unsigned int bytes, QEMUIOVector *qiov, BdrvRequestFlags flags); diff --git a/block/block-backend.c b/block/block-backend.c index eb22ff306e..912c50678d 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -1178,9 +1178,10 @@ int coroutine_fn blk_co_preadv(BlockBackend *blk, int64_t offset, return ret; } -int coroutine_fn blk_co_pwritev(BlockBackend *blk, int64_t offset, - unsigned int bytes, QEMUIOVector *qiov, - BdrvRequestFlags flags) +int coroutine_fn blk_co_pwritev_part(BlockBackend *blk, int64_t offset, + unsigned int bytes, + QEMUIOVector *qiov, size_t qiov_offset, + BdrvRequestFlags flags) { int ret; BlockDriverState *bs; @@ -1207,11 +1208,19 @@ int coroutine_fn blk_co_pwritev(BlockBackend *blk, int64_t offset, flags |= BDRV_REQ_FUA; } - ret = bdrv_co_pwritev(blk->root, offset, bytes, qiov, flags); + ret = bdrv_co_pwritev_part(blk->root, offset, bytes, qiov, qiov_offset, + flags); bdrv_dec_in_flight(bs); return ret; } +int coroutine_fn blk_co_pwritev(BlockBackend *blk, int64_t offset, + unsigned int bytes, QEMUIOVector *qiov, + BdrvRequestFlags flags) +{ + return blk_co_pwritev_part(blk, offset, bytes, qiov, 0, flags); +} + typedef struct BlkRwCo { BlockBackend *blk; int64_t offset; From patchwork Mon Oct 28 12:14:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215281 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E998112C for ; Mon, 28 Oct 2019 12:36:09 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4476B205ED for ; Mon, 28 Oct 2019 12:36:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Mq42u4Ar" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4476B205ED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53256 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Fj-00082Y-PZ for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:36:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49958) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vq-0001S5-1j for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vo-0001Ki-1j for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:33 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:39961 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vn-0001KP-Th for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264931; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xNVjvSYdT3Fxsm1A4G+GPPJYwmuAaRyR339cHbHqnos=; b=Mq42u4ArnlQlKPNxZu1fz33nXnwdJU2on/BFCFil5TI/dfMg65Kk5qSSbENEt+RkZHUipr WFLdgOA4uZSU+ZVJFNvKByNPH73VtBJO4zDHalL60j4ylAAfofDHI5YsUgGzIFn5a5EMl2 Cprvpi/zFPqfeCPJli7hoOj9r/I3N0U= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-195-p5BY4Aw5PHeQQisYTnykhA-1; Mon, 28 Oct 2019 08:15:27 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 979F8180496E; Mon, 28 Oct 2019 12:15:26 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3E7D4600C7; Mon, 28 Oct 2019 12:15:26 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 11/69] block/mirror: support unaligned write in active mirror Date: Mon, 28 Oct 2019 13:14:03 +0100 Message-Id: <20191028121501.15279-12-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: p5BY4Aw5PHeQQisYTnykhA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Prior 9adc1cb49af8d do_sync_target_write had a bug: it reset aligned-up region in the dirty bitmap, which means that we may not copy some bytes and assume them copied, which actually leads to producing corrupted target. So 9adc1cb49af8d forced dirty bitmap granularity to be request_alignment for mirror-top filter, so we are not working with unaligned requests. However forcing large alignment obviously decreases performance of unaligned requests. This commit provides another solution for the problem: if unaligned padding is already dirty, we can safely ignore it, as 1. It's dirty, it will be copied by mirror_iteration anyway 2. It's dirty, so skipping it now we don't increase dirtiness of the bitmap and therefore don't damage "synchronicity" of the write-blocking mirror. If unaligned padding is not dirty, we just write it, no reason to touch dirty bitmap if we succeed (on failure we'll set the whole region ofcourse, but we loss "synchronicity" on failure anyway). Note: we need to disable dirty_bitmap, otherwise we will not be able to see in do_sync_target_write bitmap state before current operation. We may of course check dirty bitmap before the operation in bdrv_mirror_top_do_write and remember it, but we don't need active dirty bitmap for write-blocking mirror anyway. New code-path is unused until the following commit reverts 9adc1cb49af8d. Suggested-by: Denis V. Lunev Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20191011090711.19940-5-vsementsov@virtuozzo.com Reviewed-by: Max Reitz Signed-off-by: Max Reitz --- block/mirror.c | 71 +++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 68 insertions(+), 3 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 351faf9367..11d4d66f43 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1182,14 +1182,67 @@ do_sync_target_write(MirrorBlockJob *job, MirrorMethod method, QEMUIOVector *qiov, int flags) { int ret; + size_t qiov_offset = 0; + int64_t bitmap_offset, bitmap_end; - bdrv_reset_dirty_bitmap(job->dirty_bitmap, offset, bytes); + if (!QEMU_IS_ALIGNED(offset, job->granularity) && + bdrv_dirty_bitmap_get(job->dirty_bitmap, offset)) + { + /* + * Dirty unaligned padding: ignore it. + * + * Reasoning: + * 1. If we copy it, we can't reset corresponding bit in + * dirty_bitmap as there may be some "dirty" bytes still not + * copied. + * 2. It's already dirty, so skipping it we don't diverge mirror + * progress. + * + * Note, that because of this, guest write may have no contribution + * into mirror converge, but that's not bad, as we have background + * process of mirroring. If under some bad circumstances (high guest + * IO load) background process starve, we will not converge anyway, + * even if each write will contribute, as guest is not guaranteed to + * rewrite the whole disk. + */ + qiov_offset = QEMU_ALIGN_UP(offset, job->granularity) - offset; + if (bytes <= qiov_offset) { + /* nothing to do after shrink */ + return; + } + offset += qiov_offset; + bytes -= qiov_offset; + } + + if (!QEMU_IS_ALIGNED(offset + bytes, job->granularity) && + bdrv_dirty_bitmap_get(job->dirty_bitmap, offset + bytes - 1)) + { + uint64_t tail = (offset + bytes) % job->granularity; + + if (bytes <= tail) { + /* nothing to do after shrink */ + return; + } + bytes -= tail; + } + + /* + * Tails are either clean or shrunk, so for bitmap resetting + * we safely align the range down. + */ + bitmap_offset = QEMU_ALIGN_UP(offset, job->granularity); + bitmap_end = QEMU_ALIGN_DOWN(offset + bytes, job->granularity); + if (bitmap_offset < bitmap_end) { + bdrv_reset_dirty_bitmap(job->dirty_bitmap, bitmap_offset, + bitmap_end - bitmap_offset); + } job_progress_increase_remaining(&job->common.job, bytes); switch (method) { case MIRROR_METHOD_COPY: - ret = blk_co_pwritev(job->target, offset, bytes, qiov, flags); + ret = blk_co_pwritev_part(job->target, offset, bytes, + qiov, qiov_offset, flags); break; case MIRROR_METHOD_ZERO: @@ -1211,7 +1264,16 @@ do_sync_target_write(MirrorBlockJob *job, MirrorMethod method, } else { BlockErrorAction action; - bdrv_set_dirty_bitmap(job->dirty_bitmap, offset, bytes); + /* + * We failed, so we should mark dirty the whole area, aligned up. + * Note that we don't care about shrunk tails if any: they were dirty + * at function start, and they must be still dirty, as we've locked + * the region for in-flight op. + */ + bitmap_offset = QEMU_ALIGN_DOWN(offset, job->granularity); + bitmap_end = QEMU_ALIGN_UP(offset + bytes, job->granularity); + bdrv_set_dirty_bitmap(job->dirty_bitmap, bitmap_offset, + bitmap_end - bitmap_offset); job->actively_synced = false; action = mirror_error_action(job, false, -ret); @@ -1618,6 +1680,9 @@ static BlockJob *mirror_start_job( if (!s->dirty_bitmap) { goto fail; } + if (s->copy_mode == MIRROR_COPY_MODE_WRITE_BLOCKING) { + bdrv_disable_dirty_bitmap(s->dirty_bitmap); + } ret = block_job_add_bdrv(&s->common, "source", bs, 0, BLK_PERM_WRITE_UNCHANGED | BLK_PERM_WRITE | From patchwork Mon Oct 28 12:14:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215265 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 66EEC1390 for ; Mon, 28 Oct 2019 12:26:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3DD2B20717 for ; Mon, 28 Oct 2019 12:26:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="V1KFU3pH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3DD2B20717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53180 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP46m-0008IQ-RB for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:26:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49957) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vp-0001S2-Tj for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vo-0001L2-BS for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:33 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:30363 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vo-0001Kc-7X for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264931; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TE96eOaW6aiFCYl3gxB+X0JZqjsnCz0s2Kz/h/5Cz9A=; b=V1KFU3pH9pHt2tnThGgVgNmhj6FpfBzoZ1uEtw9Bxg9FUfPW9vb96PDARoE1WFA1DYsHNv mvozz8evXfGH0FEWLYIOr4Z+ztj5f3mHtI3IRPPDQrD3yfRRPm3FeRGDZ6LmarQuDS1P7C ZnCKoI9DDpBK4BpxGrQ06lNwOVx5vDM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-34-rQJv7L6eNgiDWFeqWjFTTQ-1; Mon, 28 Oct 2019 08:15:29 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AB8A6107AD29; Mon, 28 Oct 2019 12:15:28 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 523C11001B03; Mon, 28 Oct 2019 12:15:28 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 12/69] Revert "mirror: Only mirror granularity-aligned chunks" Date: Mon, 28 Oct 2019 13:14:04 +0100 Message-Id: <20191028121501.15279-13-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: rQJv7L6eNgiDWFeqWjFTTQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy This reverts commit 9adc1cb49af8d4e54f57980b1eed5c0a4b2dafa6. "mirror: Only mirror granularity-aligned chunks" Since previous commit unaligned chunks are supported by do_sync_target_write. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20191011090711.19940-6-vsementsov@virtuozzo.com Signed-off-by: Max Reitz --- block/mirror.c | 29 ----------------------------- 1 file changed, 29 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 11d4d66f43..454365ce00 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1488,15 +1488,6 @@ static void bdrv_mirror_top_child_perm(BlockDriverState *bs, BdrvChild *c, *nshared = BLK_PERM_ALL; } -static void bdrv_mirror_top_refresh_limits(BlockDriverState *bs, Error **errp) -{ - MirrorBDSOpaque *s = bs->opaque; - - if (s && s->job && s->job->copy_mode == MIRROR_COPY_MODE_WRITE_BLOCKING) { - bs->bl.request_alignment = s->job->granularity; - } -} - /* Dummy node that provides consistent read to its users without requiring it * from its backing file and that allows writes on the backing file chain. */ static BlockDriver bdrv_mirror_top = { @@ -1509,7 +1500,6 @@ static BlockDriver bdrv_mirror_top = { .bdrv_co_block_status = bdrv_co_block_status_from_backing, .bdrv_refresh_filename = bdrv_mirror_top_refresh_filename, .bdrv_child_perm = bdrv_mirror_top_child_perm, - .bdrv_refresh_limits = bdrv_mirror_top_refresh_limits, }; static BlockJob *mirror_start_job( @@ -1657,25 +1647,6 @@ static BlockJob *mirror_start_job( s->should_complete = true; } - /* - * Must be called before we start tracking writes, but after - * - * ((MirrorBlockJob *) - * ((MirrorBDSOpaque *) - * mirror_top_bs->opaque - * )->job - * )->copy_mode - * - * has the correct value. - * (We start tracking writes as of the following - * bdrv_create_dirty_bitmap() call.) - */ - bdrv_refresh_limits(mirror_top_bs, &local_err); - if (local_err) { - error_propagate(errp, local_err); - goto fail; - } - s->dirty_bitmap = bdrv_create_dirty_bitmap(bs, granularity, NULL, errp); if (!s->dirty_bitmap) { goto fail; From patchwork Mon Oct 28 12:14:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215273 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 14DC613BD for ; Mon, 28 Oct 2019 12:31:07 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E03D5208C0 for ; Mon, 28 Oct 2019 12:31:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YnNVEc0X" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E03D5208C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53214 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Ar-0005y8-9i for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:31:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50004) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vs-0001WV-I8 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vr-0001Mh-8z for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:36 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:50516 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vr-0001MW-5F for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264934; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rf2k1CkyiJ3JgpyB5YlM9gSsjM5IJicA78cGlgC8bws=; b=YnNVEc0XnjrGFemZeXRReCgD4ofChVTVWaXO3F3GMPA5D+X+M8NFfVMkf0p4ZPlduvmTyy c5nMoVE6nD9ljQiS1I3vl8pDIfRBX5/EL24pWLGkA0iSPBzc1K+5mDETWtCvs4Q4y0w7Jm fwx77uOStLh5SEM2h9rwTXDdBggx+7A= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-313-yOVfPDdTMr6ukefQFv0WvA-1; Mon, 28 Oct 2019 08:15:31 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B3965476; Mon, 28 Oct 2019 12:15:30 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 551E6600C7; Mon, 28 Oct 2019 12:15:30 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 13/69] iotests: Introduce $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:05 +0100 Message-Id: <20191028121501.15279-14-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: yOVfPDdTMr6ukefQFv0WvA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Unix sockets generally have a maximum path length. Depending on your $TEST_DIR, it may be exceeded and then all tests that create and use Unix sockets there may fail. Circumvent this by adding a new scratch directory specifically for Unix socket files. It defaults to a temporary directory (mktemp -d) that is completely removed after the iotests are done. (By default, mktemp -d creates a /tmp/tmp.XXXXXXXXXX directory, which should be short enough for our use cases.) Use mkdir -p to create the directory (because it seems right), and do the same for $TEST_DIR (because there is no reason for that to be created in any different way). Signed-off-by: Max Reitz Message-id: 20191017133155.5327-2-mreitz@redhat.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- tests/qemu-iotests/check | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 588c453a94..71fe38834e 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -97,6 +97,7 @@ IMGFMT -- $FULL_IMGFMT_DETAILS IMGPROTO -- $IMGPROTO PLATFORM -- $FULL_HOST_DETAILS TEST_DIR -- $TEST_DIR +SOCK_DIR -- $SOCK_DIR SOCKET_SCM_HELPER -- $SOCKET_SCM_HELPER EOF @@ -116,10 +117,14 @@ set_prog_path() if [ -z "$TEST_DIR" ]; then TEST_DIR=$PWD/scratch fi +mkdir -p "$TEST_DIR" || _init_error 'Failed to create TEST_DIR' -if [ ! -e "$TEST_DIR" ]; then - mkdir "$TEST_DIR" +tmp_sock_dir=false +if [ -z "$SOCK_DIR" ]; then + SOCK_DIR=$(mktemp -d) + tmp_sock_dir=true fi +mkdir -p "$SOCK_DIR" || _init_error 'Failed to create SOCK_DIR' diff="diff -u" verbose=false @@ -534,6 +539,7 @@ if [ -z "$SAMPLE_IMG_DIR" ]; then fi export TEST_DIR +export SOCK_DIR export SAMPLE_IMG_DIR if [ -s $tmp.list ] @@ -716,6 +722,11 @@ END { if (NR > 0) { rm -f "${TEST_DIR}"/*.out "${TEST_DIR}"/*.err "${TEST_DIR}"/*.time rm -f "${TEST_DIR}"/check.pid "${TEST_DIR}"/check.sts rm -f $tmp.* + + if $tmp_sock_dir + then + rm -rf "$SOCK_DIR" + fi } trap "_wrapup; exit \$status" 0 1 2 3 15 From patchwork Mon Oct 28 12:14:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215323 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8F3B7112C for ; Mon, 28 Oct 2019 12:39:44 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 64F61208C0 for ; Mon, 28 Oct 2019 12:39:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="d40SJIxX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 64F61208C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53286 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4JD-000653-32 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:39:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50026) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vu-0001ZX-Ao for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vs-0001NZ-SA for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:38 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:24862 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vs-0001NQ-O6 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264936; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s+6HHvmXTwLL5Br/Swin1Mm/2hcp4AacjnAi9Rdl8mo=; b=d40SJIxXzRFK05QcQM3mDPNfILWA4xfPfomRoltMyAxTQZdkqqum5qMX9S1cCerNnOpJOC X1HwJYtV9KOs/+6laKG68libIFD6TAE0aXrUWRzU6Oujm1ivI0IcheilNLho9c5egRTJE1 Zvcdjo5x+uWGwAtTiEWofoQEOHL6Yh8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-304-zdTHcMUaNiSZKMZjylvSoA-1; Mon, 28 Oct 2019 08:15:33 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C322E800FF1; Mon, 28 Oct 2019 12:15:32 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6672D100164D; Mon, 28 Oct 2019 12:15:32 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 14/69] iotests.py: Store socket files in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:06 +0100 Message-Id: <20191028121501.15279-15-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: zdTHcMUaNiSZKMZjylvSoA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" iotests.py itself does not store socket files, but machine.py and qtest.py do. iotests.py needs to pass the respective path to them, and they need to adhere to it. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Thomas Huth Message-id: 20191017133155.5327-3-mreitz@redhat.com Signed-off-by: Max Reitz --- python/qemu/machine.py | 15 ++++++++++++--- python/qemu/qtest.py | 9 ++++++--- tests/qemu-iotests/iotests.py | 4 +++- 3 files changed, 21 insertions(+), 7 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 128a3d1dc2..2024e8b1b1 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -71,7 +71,7 @@ class QEMUMachine(object): def __init__(self, binary, args=None, wrapper=None, name=None, test_dir="/var/tmp", monitor_address=None, - socket_scm_helper=None): + socket_scm_helper=None, sock_dir=None): ''' Initialize a QEMUMachine @@ -90,6 +90,8 @@ class QEMUMachine(object): wrapper = [] if name is None: name = "qemu-%d" % os.getpid() + if sock_dir is None: + sock_dir = test_dir self._name = name self._monitor_address = monitor_address self._vm_monitor = None @@ -106,12 +108,14 @@ class QEMUMachine(object): self._qemu_full_args = None self._test_dir = test_dir self._temp_dir = None + self._sock_dir = sock_dir self._launched = False self._machine = None self._console_set = False self._console_device_type = None self._console_address = None self._console_socket = None + self._remove_files = [] # just in case logging wasn't configured by the main script: logging.basicConfig() @@ -236,8 +240,9 @@ class QEMUMachine(object): if self._machine is not None: args.extend(['-machine', self._machine]) if self._console_set: - self._console_address = os.path.join(self._temp_dir, + self._console_address = os.path.join(self._sock_dir, self._name + "-console.sock") + self._remove_files.append(self._console_address) chardev = ('socket,id=console,path=%s,server,nowait' % self._console_address) args.extend(['-chardev', chardev]) @@ -253,8 +258,9 @@ class QEMUMachine(object): if self._monitor_address is not None: self._vm_monitor = self._monitor_address else: - self._vm_monitor = os.path.join(self._temp_dir, + self._vm_monitor = os.path.join(self._sock_dir, self._name + "-monitor.sock") + self._remove_files.append(self._vm_monitor) self._qemu_log_path = os.path.join(self._temp_dir, self._name + ".log") self._qemu_log_file = open(self._qemu_log_path, 'wb') @@ -279,6 +285,9 @@ class QEMUMachine(object): shutil.rmtree(self._temp_dir) self._temp_dir = None + while len(self._remove_files) > 0: + self._remove_if_exists(self._remove_files.pop()) + def launch(self): """ Launch the VM and make sure we cleanup and expose the diff --git a/python/qemu/qtest.py b/python/qemu/qtest.py index 3f1d2cb325..d24ad04256 100644 --- a/python/qemu/qtest.py +++ b/python/qemu/qtest.py @@ -84,14 +84,17 @@ class QEMUQtestMachine(QEMUMachine): '''A QEMU VM''' def __init__(self, binary, args=None, name=None, test_dir="/var/tmp", - socket_scm_helper=None): + socket_scm_helper=None, sock_dir=None): if name is None: name = "qemu-%d" % os.getpid() + if sock_dir is None: + sock_dir = test_dir super(QEMUQtestMachine, self).__init__(binary, args, name=name, test_dir=test_dir, - socket_scm_helper=socket_scm_helper) + socket_scm_helper=socket_scm_helper, + sock_dir=sock_dir) self._qtest = None - self._qtest_path = os.path.join(test_dir, name + "-qtest.sock") + self._qtest_path = os.path.join(sock_dir, name + "-qtest.sock") def _base_args(self): args = super(QEMUQtestMachine, self)._base_args() diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index a4a2238b42..9bfcef1f14 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -57,6 +57,7 @@ qemu_opts = os.environ.get('QEMU_OPTIONS', '').strip().split(' ') imgfmt = os.environ.get('IMGFMT', 'raw') imgproto = os.environ.get('IMGPROTO', 'file') test_dir = os.environ.get('TEST_DIR') +sock_dir = os.environ.get('SOCK_DIR') output_dir = os.environ.get('OUTPUT_DIR', '.') cachemode = os.environ.get('CACHEMODE') qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE') @@ -456,7 +457,8 @@ class VM(qtest.QEMUQtestMachine): name = "qemu%s-%d" % (path_suffix, os.getpid()) super(VM, self).__init__(qemu_prog, qemu_opts, name=name, test_dir=test_dir, - socket_scm_helper=socket_scm_helper) + socket_scm_helper=socket_scm_helper, + sock_dir=sock_dir) self._num_drives = 0 def add_object(self, opts): From patchwork Mon Oct 28 12:14:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215259 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 90DC013BD for ; Mon, 28 Oct 2019 12:23:20 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 675D120717 for ; Mon, 28 Oct 2019 12:23:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QnThgWja" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 675D120717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53150 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP43K-000462-My for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:23:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50057) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vx-0001dl-Eg for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vw-0001Ow-6Z for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:41 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:38790 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vw-0001Ol-3F for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264939; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qxpTgjhe4LBYsfVnIw1qeVBA6HbE1S152Eb+XejyuR8=; b=QnThgWjaCY0b2a5SNv/ke5ueza8CLpaIC7hz8OTaADTFgv9Ksil2oKS7IVHrCTvgldjDBg DBDdGwja5fBHgJCFzCxVk9h5z+ft8WJmcRURksA+oOG0vsowftFLzu2c00eF7P0zdfhf3W 5Oe1hitaALNEgrX34OvwzvrULPyP+Pc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-1-dCOJTIVOO7We15riUcbxEg-1; Mon, 28 Oct 2019 08:15:35 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D23ED180496E; Mon, 28 Oct 2019 12:15:34 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7433F100194E; Mon, 28 Oct 2019 12:15:34 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 15/69] iotests.py: Add @base_dir to FilePaths etc. Date: Mon, 28 Oct 2019 13:14:07 +0100 Message-Id: <20191028121501.15279-16-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: dCOJTIVOO7We15riUcbxEg-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Specifying this optional parameter allows creating temporary files in other directories than the test_dir; for example in sock_dir. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-4-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 9bfcef1f14..075f4739da 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -386,10 +386,10 @@ class FilePaths(object): qemu_img('create', img_path, '1G') # migration_sock_path is automatically deleted """ - def __init__(self, names): + def __init__(self, names, base_dir=test_dir): self.paths = [] for name in names: - self.paths.append(os.path.join(test_dir, file_pattern(name))) + self.paths.append(os.path.join(base_dir, file_pattern(name))) def __enter__(self): return self.paths @@ -406,8 +406,8 @@ class FilePath(FilePaths): """ FilePath is a specialization of FilePaths that takes a single filename. """ - def __init__(self, name): - super(FilePath, self).__init__([name]) + def __init__(self, name, base_dir=test_dir): + super(FilePath, self).__init__([name], base_dir) def __enter__(self): return self.paths[0] @@ -420,7 +420,7 @@ def file_path_remover(): pass -def file_path(*names): +def file_path(*names, base_dir=test_dir): ''' Another way to get auto-generated filename that cleans itself up. Use is as simple as: @@ -436,7 +436,7 @@ def file_path(*names): paths = [] for name in names: filename = file_pattern(name) - path = os.path.join(test_dir, filename) + path = os.path.join(base_dir, filename) file_path_remover.paths.append(path) paths.append(path) From patchwork Mon Oct 28 12:14:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215267 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E2E9413BD for ; Mon, 28 Oct 2019 12:26:58 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B887120717 for ; Mon, 28 Oct 2019 12:26:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hC/03T75" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B887120717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53182 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP46r-0008On-8j for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:26:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50062) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vx-0001eZ-V1 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vw-0001PL-PC for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:41 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:33383 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vw-0001PD-Li for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264940; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7AegR6O/GB/NSUyImdF/YuOZTD70N9BVgHZLYSLHW7s=; b=hC/03T75ZtwXp3WNQ+UZgWf9Mxgb7fEXB+yCjJFA5wliIePn5O5e1DutB7j2QtL38/sSpa CGbl37EkfpUXf5PThiPgwagfekQWbaetlxBZqzQo82at4/Nc870Tp+FQnmYXRvU7/pa7xO ojF4tAMLa8AeFXxRyhxkXF0xLwRvlO4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-86-zWWSmfdTO-SirtPf3j-0YQ-1; Mon, 28 Oct 2019 08:15:37 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DD79B1804970; Mon, 28 Oct 2019 12:15:36 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 826426084E; Mon, 28 Oct 2019 12:15:36 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 16/69] iotests: Filter $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:08 +0100 Message-Id: <20191028121501.15279-17-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: zWWSmfdTO-SirtPf3j-0YQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-5-mreitz@redhat.com Reviewed-by: Thomas Huth Signed-off-by: Max Reitz --- tests/qemu-iotests/common.filter | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index 9f418b4881..0ee6042524 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -43,7 +43,8 @@ _filter_qom_path() # replace occurrences of the actual TEST_DIR value with TEST_DIR _filter_testdir() { - $SED -e "s#$TEST_DIR/#TEST_DIR/#g" + $SED -e "s#$TEST_DIR/#TEST_DIR/#g" \ + -e "s#$SOCK_DIR/#SOCK_DIR/#g" } # replace occurrences of the actual IMGFMT value with IMGFMT @@ -124,6 +125,7 @@ _filter_img_create() $SED -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \ -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \ -e "s#$TEST_DIR#TEST_DIR#g" \ + -e "s#$SOCK_DIR#SOCK_DIR#g" \ -e "s#$IMGFMT#IMGFMT#g" \ -e 's#nbd:127.0.0.1:10810#TEST_DIR/t.IMGFMT#g' \ -e "s# encryption=off##g" \ @@ -160,6 +162,7 @@ _filter_img_info() $SED -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \ -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \ -e "s#$TEST_DIR#TEST_DIR#g" \ + -e "s#$SOCK_DIR#SOCK_DIR#g" \ -e "s#$IMGFMT#IMGFMT#g" \ -e 's#nbd://127.0.0.1:10810$#TEST_DIR/t.IMGFMT#g' \ -e 's#json.*vdisk-id.*vxhs"}}#TEST_DIR/t.IMGFMT#' \ @@ -219,6 +222,7 @@ _filter_nbd() $SED -e '/nbd\/.*\.c:/d' \ -e 's#127\.0\.0\.1:[0-9]*#127.0.0.1:PORT#g' \ -e "s#?socket=$TEST_DIR#?socket=TEST_DIR#g" \ + -e "s#?socket=$SOCK_DIR#?socket=SOCK_DIR#g" \ -e 's#\(foo\|PORT/\?\|.sock\): Failed to .*$#\1#' } From patchwork Mon Oct 28 12:14:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 40774112C for ; Mon, 28 Oct 2019 12:37:07 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 162CE208C0 for ; Mon, 28 Oct 2019 12:37:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iCCzlX6B" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 162CE208C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53258 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Gf-0002Jp-Ue for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:37:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50094) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3vz-0001hA-Lc for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3vy-0001QT-Ev for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:43 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:32957) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3vy-0001Q0-BG for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264941; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LWAJzrkgp4bSHSVktmBeu7aN2rV4fKsTo365vflS3Og=; b=iCCzlX6BpieScrxW3YenrVfSN0yL8QdXxrEsFKnWrHimULpnXMWiiOxYK4DV77RlBvvDR8 8UquuAM4qjabAP4A/uBSw2mo5Ep0ps25v4uk1465ZsHg0ALE02pULsSgIcb3EWxgN5Wi0P Fh/IAVQ5gdCnuv5jCeWWBISjd9g8Myc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-18-Nr43nzHlM0y_PjLDAX0AIA-1; Mon, 28 Oct 2019 08:15:39 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E1935800FF1; Mon, 28 Oct 2019 12:15:38 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8808C5C207; Mon, 28 Oct 2019 12:15:38 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 17/69] iotests: Let common.nbd create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:09 +0100 Message-Id: <20191028121501.15279-18-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: Nr43nzHlM0y_PjLDAX0AIA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" In addition, drop the nbd_unix_socket assignment in 241 because it does not really do anything. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-6-mreitz@redhat.com Reviewed-by: Thomas Huth Signed-off-by: Max Reitz --- tests/qemu-iotests/241 | 2 -- tests/qemu-iotests/common.nbd | 2 +- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/tests/qemu-iotests/241 b/tests/qemu-iotests/241 index 58b64ebf41..8dae8d39e4 100755 --- a/tests/qemu-iotests/241 +++ b/tests/qemu-iotests/241 @@ -23,8 +23,6 @@ echo "QA output created by $seq" status=1 # failure is the default! -nbd_unix_socket=$TEST_DIR/test_qemu_nbd_socket - _cleanup() { _cleanup_test_img diff --git a/tests/qemu-iotests/common.nbd b/tests/qemu-iotests/common.nbd index 24b01b60aa..a8cae8fe2c 100644 --- a/tests/qemu-iotests/common.nbd +++ b/tests/qemu-iotests/common.nbd @@ -19,7 +19,7 @@ # along with this program. If not, see . # -nbd_unix_socket="${TEST_DIR}/qemu-nbd.sock" +nbd_unix_socket="${SOCK_DIR}/qemu-nbd.sock" nbd_tcp_addr="127.0.0.1" nbd_pid_file="${TEST_DIR}/qemu-nbd.pid" nbd_stderr_fifo="${TEST_DIR}/qemu-nbd.fifo" From patchwork Mon Oct 28 12:14:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D9A6813B1 for ; Mon, 28 Oct 2019 12:37:19 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B0A1D208C0 for ; Mon, 28 Oct 2019 12:37:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="b6ceNBth" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B0A1D208C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53260 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Gs-0002p8-Ic for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:37:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50148) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3w4-0001oK-IY for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3w2-0001T5-Px for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:48 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:42786 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3w2-0001Ss-LL for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264946; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X+T5sUsAMqhzKjQpHaXLv+CzeSkI65I6IV4Nh3mOMPA=; b=b6ceNBthCxyl3DSvmUldGXgdsMjUMPDEkex8Nmr/80vcR/N9QMebF065dK39nb1gj1iCA8 PXd5wWZSrsdkyQwkBUuG6r6rGgAApJZgdphScSWQuo0sKyLMIetAqPVSLLP2dXx/NGP3MY j1sjoBOcG3JOIYQbt6ofoh1sPzEjYjs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-402-7Pwjz-gIMKGGzMA2AFvzoQ-1; Mon, 28 Oct 2019 08:15:41 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EB47310052E0; Mon, 28 Oct 2019 12:15:40 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 91EBF60BF7; Mon, 28 Oct 2019 12:15:40 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 18/69] iotests/083: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:10 +0100 Message-Id: <20191028121501.15279-19-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: 7Pwjz-gIMKGGzMA2AFvzoQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Thomas Huth Message-id: 20191017133155.5327-7-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/083 | 6 +++--- tests/qemu-iotests/083.out | 34 +++++++++++++++++----------------- 2 files changed, 20 insertions(+), 20 deletions(-) diff --git a/tests/qemu-iotests/083 b/tests/qemu-iotests/083 index b270550d3e..10fdfc8ebb 100755 --- a/tests/qemu-iotests/083 +++ b/tests/qemu-iotests/083 @@ -28,7 +28,7 @@ status=1 # failure is the default! _cleanup() { - rm -f nbd.sock + rm -f "$SOCK_DIR/nbd.sock" rm -f nbd-fault-injector.out rm -f nbd-fault-injector.conf } @@ -80,10 +80,10 @@ EOF if [ "$proto" = "tcp" ]; then nbd_addr="127.0.0.1:0" else - nbd_addr="$TEST_DIR/nbd.sock" + nbd_addr="$SOCK_DIR/nbd.sock" fi - rm -f "$TEST_DIR/nbd.sock" + rm -f "$SOCK_DIR/nbd.sock" echo > "$TEST_DIR/nbd-fault-injector.out" $PYTHON nbd-fault-injector.py $extra_args "$nbd_addr" "$TEST_DIR/nbd-fault-injector.conf" >"$TEST_DIR/nbd-fault-injector.out" 2>&1 & diff --git a/tests/qemu-iotests/083.out b/tests/qemu-iotests/083.out index eee6dd1379..2090ee693c 100644 --- a/tests/qemu-iotests/083.out +++ b/tests/qemu-iotests/083.out @@ -110,43 +110,43 @@ read failed: Input/output error === Check disconnect before neg1 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect after neg1 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect 8 neg1 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect 16 neg1 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect before export === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect after export === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect 4 export === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect 12 export === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect 16 export === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect before neg2 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect after neg2 === @@ -154,11 +154,11 @@ read failed: Input/output error === Check disconnect 8 neg2 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect 10 neg2 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect before request === @@ -195,23 +195,23 @@ read 512/512 bytes at offset 0 === Check disconnect before neg-classic === -qemu-io: can't open device nbd+unix:///?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///?socket=SOCK_DIR/nbd.sock === Check disconnect 8 neg-classic === -qemu-io: can't open device nbd+unix:///?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///?socket=SOCK_DIR/nbd.sock === Check disconnect 16 neg-classic === -qemu-io: can't open device nbd+unix:///?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///?socket=SOCK_DIR/nbd.sock === Check disconnect 24 neg-classic === -qemu-io: can't open device nbd+unix:///?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///?socket=SOCK_DIR/nbd.sock === Check disconnect 28 neg-classic === -qemu-io: can't open device nbd+unix:///?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///?socket=SOCK_DIR/nbd.sock === Check disconnect after neg-classic === From patchwork Mon Oct 28 12:14:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215275 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 91D8913BD for ; Mon, 28 Oct 2019 12:31:32 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 682B8214D9 for ; Mon, 28 Oct 2019 12:31:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SoXutIhN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 682B8214D9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53216 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4BG-0006su-Vl for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:31:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50127) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3w3-0001me-BS for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3w2-0001Se-22 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:47 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:46583 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3w1-0001SO-UC for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264945; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wZTsLCacB11zcXpx9AAjf3yjkwEckl/Z+NRGF86/q3s=; b=SoXutIhNxuKHyJaYlFTe9ZfJ4kzF7lx33D2gvo7F5st8L2/Mqn2vUhTl2Qpqhkm0HpL0f+ JE6YHJYUPySEkYqct1VvjE7ddY9TSskqxOPsE6hcK47VaKmiHZcF/i6WL/mrUkew4+SdpA x2CV34ppQzzVEZqsaa+FIJ0R+eAwq4g= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-247-QVBKgQGSPtyuSkhkegl_kQ-1; Mon, 28 Oct 2019 08:15:44 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 25B6C476; Mon, 28 Oct 2019 12:15:43 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B40D35D6BB; Mon, 28 Oct 2019 12:15:42 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 19/69] iotests/140: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:11 +0100 Message-Id: <20191028121501.15279-20-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-MC-Unique: QVBKgQGSPtyuSkhkegl_kQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Thomas Huth Message-id: 20191017133155.5327-8-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/140 | 8 ++++---- tests/qemu-iotests/140.out | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/140 b/tests/qemu-iotests/140 index b965b1dd5d..8d2ce5d9e3 100755 --- a/tests/qemu-iotests/140 +++ b/tests/qemu-iotests/140 @@ -34,7 +34,7 @@ _cleanup() { _cleanup_qemu _cleanup_test_img - rm -f "$TEST_DIR/nbd" + rm -f "$SOCK_DIR/nbd" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -69,7 +69,7 @@ _send_qemu_cmd $QEMU_HANDLE \ _send_qemu_cmd $QEMU_HANDLE \ "{ 'execute': 'nbd-server-start', 'arguments': { 'addr': { 'type': 'unix', - 'data': { 'path': '$TEST_DIR/nbd' }}}}" \ + 'data': { 'path': '$SOCK_DIR/nbd' }}}}" \ 'return' _send_qemu_cmd $QEMU_HANDLE \ @@ -78,7 +78,7 @@ _send_qemu_cmd $QEMU_HANDLE \ 'return' $QEMU_IO_PROG -f raw -r -c 'read -P 42 0 64k' \ - "nbd+unix:///drv?socket=$TEST_DIR/nbd" 2>&1 \ + "nbd+unix:///drv?socket=$SOCK_DIR/nbd" 2>&1 \ | _filter_qemu_io | _filter_nbd _send_qemu_cmd $QEMU_HANDLE \ @@ -87,7 +87,7 @@ _send_qemu_cmd $QEMU_HANDLE \ 'return' $QEMU_IO_PROG -f raw -r -c close \ - "nbd+unix:///drv?socket=$TEST_DIR/nbd" 2>&1 \ + "nbd+unix:///drv?socket=$SOCK_DIR/nbd" 2>&1 \ | _filter_qemu_io | _filter_nbd _send_qemu_cmd $QEMU_HANDLE \ diff --git a/tests/qemu-iotests/140.out b/tests/qemu-iotests/140.out index 67fe44a3e3..2511eb7369 100644 --- a/tests/qemu-iotests/140.out +++ b/tests/qemu-iotests/140.out @@ -8,7 +8,7 @@ wrote 65536/65536 bytes at offset 0 read 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": {}} -qemu-io: can't open device nbd+unix:///drv?socket=TEST_DIR/nbd: Requested export not available +qemu-io: can't open device nbd+unix:///drv?socket=SOCK_DIR/nbd: Requested export not available server reported: export 'drv' not present {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} From patchwork Mon Oct 28 12:14:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215333 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 022F413B1 for ; Mon, 28 Oct 2019 12:42:37 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 89C4720578 for ; Mon, 28 Oct 2019 12:42:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cgcnBVf5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 89C4720578 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53318 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Lz-0002Lc-5U for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:42:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50199) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3w8-0001sX-4m for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3w6-0001VC-Ty for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:52 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:42269 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3w6-0001Uf-A6 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264949; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XTWUKIMC+SzwJJX09S5GxuyIqJgaDZS198wieJIhHR0=; b=cgcnBVf5CTZhGHpPzgeYMpJm7z6cy+rk+mxugYZcRZBihjrkcpGvWiyJNPG3960JdKr4y6 gnhbvATQ1NAq7GYPluKYPYZ7CyHrsdnJx31pEcXDVfbmjkXMjB8/EjuEYZOO9+fGyOadFf gqRYczuyZec+X4vpJHLduF71TWVh1UQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-310-xm5T5vFdMI-RDaS6OFMJLA-1; Mon, 28 Oct 2019 08:15:46 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3170D5E4; Mon, 28 Oct 2019 12:15:45 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CB338100194E; Mon, 28 Oct 2019 12:15:44 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 20/69] iotests/143: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:12 +0100 Message-Id: <20191028121501.15279-21-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: xm5T5vFdMI-RDaS6OFMJLA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Thomas Huth Message-id: 20191017133155.5327-9-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/143 | 6 +++--- tests/qemu-iotests/143.out | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/143 b/tests/qemu-iotests/143 index 92249ac8da..f649b36195 100755 --- a/tests/qemu-iotests/143 +++ b/tests/qemu-iotests/143 @@ -29,7 +29,7 @@ status=1 # failure is the default! _cleanup() { _cleanup_qemu - rm -f "$TEST_DIR/nbd" + rm -f "$SOCK_DIR/nbd" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -51,12 +51,12 @@ _send_qemu_cmd $QEMU_HANDLE \ _send_qemu_cmd $QEMU_HANDLE \ "{ 'execute': 'nbd-server-start', 'arguments': { 'addr': { 'type': 'unix', - 'data': { 'path': '$TEST_DIR/nbd' }}}}" \ + 'data': { 'path': '$SOCK_DIR/nbd' }}}}" \ 'return' # This should just result in a client error, not in the server crashing $QEMU_IO_PROG -f raw -c quit \ - "nbd+unix:///no_such_export?socket=$TEST_DIR/nbd" 2>&1 \ + "nbd+unix:///no_such_export?socket=$SOCK_DIR/nbd" 2>&1 \ | _filter_qemu_io | _filter_nbd _send_qemu_cmd $QEMU_HANDLE \ diff --git a/tests/qemu-iotests/143.out b/tests/qemu-iotests/143.out index ee71b5aa42..037d34a409 100644 --- a/tests/qemu-iotests/143.out +++ b/tests/qemu-iotests/143.out @@ -1,7 +1,7 @@ QA output created by 143 {"return": {}} {"return": {}} -qemu-io: can't open device nbd+unix:///no_such_export?socket=TEST_DIR/nbd: Requested export not available +qemu-io: can't open device nbd+unix:///no_such_export?socket=SOCK_DIR/nbd: Requested export not available server reported: export 'no_such_export' not present {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} From patchwork Mon Oct 28 12:14:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215325 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 863D51747 for ; Mon, 28 Oct 2019 12:40:09 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5CD5A214E0 for ; Mon, 28 Oct 2019 12:40:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LeFewGNa" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5CD5A214E0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53288 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Jc-0006w2-8d for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:40:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50220) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3w9-0001u6-7V for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3w8-0001Vx-4G for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:53 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:44056) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3w8-0001Vd-1H for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264951; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aX7tdlNCSqFCzCFd76kLOahQsl2L1xEKXDIb76FldjM=; b=LeFewGNar+0go3ZpqxobpujkSFSBlj3aL3QH1woBkW7T86GE+YQssSaSlISSR3A7TI+NFg A0zUNuJltmz6r08CMwMLgWAsytD7pY+oHMbzGuiC1IpmmPe+PETbjCgphoGrlwCpN3JbwT 6ddC4sqCJ/7vwpAhs5AyHvTH24CDdE8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-1-t9d1UdFLPQqKMKDigspd7g-1; Mon, 28 Oct 2019 08:15:48 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3DDBD10052E1; Mon, 28 Oct 2019 12:15:47 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D72A226186; Mon, 28 Oct 2019 12:15:46 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 21/69] iotests/147: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:13 +0100 Message-Id: <20191028121501.15279-22-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: t9d1UdFLPQqKMKDigspd7g-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Thomas Huth Message-id: 20191017133155.5327-10-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/147 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147 index ab8480b9a4..03fc2fabcf 100755 --- a/tests/qemu-iotests/147 +++ b/tests/qemu-iotests/147 @@ -32,7 +32,7 @@ NBD_IPV6_PORT_START = NBD_PORT_END NBD_IPV6_PORT_END = NBD_IPV6_PORT_START + 1024 test_img = os.path.join(iotests.test_dir, 'test.img') -unix_socket = os.path.join(iotests.test_dir, 'nbd.socket') +unix_socket = os.path.join(iotests.sock_dir, 'nbd.socket') def flatten_sock_addr(crumpled_address): From patchwork Mon Oct 28 12:14:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215423 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 35253112C for ; Mon, 28 Oct 2019 12:47:11 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0A9B2204FD for ; Mon, 28 Oct 2019 12:47:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Hknui58x" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0A9B2204FD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53372 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4QP-00049b-Q7 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:47:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50248) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wB-0001x0-9s for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wA-0001X7-8I for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:55 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:25332 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wA-0001Wx-4i for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264953; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lPGrGamxeSG5kU8cyjtAGmTiygdK/rXU1SwofdY8sCw=; b=Hknui58xzrzknTHIsCiezad114eBvUgwOpE6fPg4IdNiBeVmMbiK7S5G8BIb2JydGA7i3K +PHppVwb/sJfV9AH6JUu2pjEymluRlanFCMxAYeqzsFzSuf64Pn9zHOUkdE77WtmVmWLBJ eHRvXtJMSF66OVM6dBBpjSNnqcS15VM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-173-ds5LUK8kMWW6_PXjQcyzvg-1; Mon, 28 Oct 2019 08:15:50 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 43D6C476; Mon, 28 Oct 2019 12:15:49 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DED385C1D6; Mon, 28 Oct 2019 12:15:48 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 22/69] iotests/181: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:14 +0100 Message-Id: <20191028121501.15279-23-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: ds5LUK8kMWW6_PXjQcyzvg-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Thomas Huth Message-id: 20191017133155.5327-11-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/181 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/181 b/tests/qemu-iotests/181 index e317e63422..378c2899d1 100755 --- a/tests/qemu-iotests/181 +++ b/tests/qemu-iotests/181 @@ -26,7 +26,7 @@ echo "QA output created by $seq" status=1 # failure is the default! -MIG_SOCKET="${TEST_DIR}/migrate" +MIG_SOCKET="${SOCK_DIR}/migrate" _cleanup() { From patchwork Mon Oct 28 12:14:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215431 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0682E112B for ; Mon, 28 Oct 2019 12:50:56 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D1170214E0 for ; Mon, 28 Oct 2019 12:50:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RRtvfQBC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D1170214E0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53398 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4U2-00012A-DR for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:50:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50250) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wB-0001xR-KV for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wA-0001XD-9p for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:55 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:44209 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wA-0001Wv-5t for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264953; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6GmQnVN7RpWtGc1vbCo4n/Sp17GVmocc+188KfQn4pg=; b=RRtvfQBC1fgFxnGMpmUBHpaYcVE/E2XmA21wMROCRFnXKxFJLmhp6EziftdAMlFFH8UaQl hh87fGJe2SiONM3lucSAmxCMGK6PVTv7jy1bbkLWGD6z8xB+wpHAAtMWF5AQWqJO48VKza k9dcAQ4Ym7uNRgpXUfXq62khPjubfJw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-318-rmmaSws2NaG2TieHwZ-kIA-1; Mon, 28 Oct 2019 08:15:52 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4DA71180496E; Mon, 28 Oct 2019 12:15:51 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E6F18100164D; Mon, 28 Oct 2019 12:15:50 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 23/69] iotests/182: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:15 +0100 Message-Id: <20191028121501.15279-24-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: rmmaSws2NaG2TieHwZ-kIA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Thomas Huth Message-id: 20191017133155.5327-12-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/182 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/182 b/tests/qemu-iotests/182 index 7f494eb9bb..1ccb850055 100755 --- a/tests/qemu-iotests/182 +++ b/tests/qemu-iotests/182 @@ -31,7 +31,7 @@ _cleanup() { _cleanup_test_img rm -f "$TEST_IMG.overlay" - rm -f "$TEST_DIR/nbd.socket" + rm -f "$SOCK_DIR/nbd.socket" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -133,7 +133,7 @@ success_or_failure=y _send_qemu_cmd $QEMU_HANDLE \ 'addr': { 'type': 'unix', 'data': { - 'path': '$TEST_DIR/nbd.socket' + 'path': '$SOCK_DIR/nbd.socket' } } } }" \ 'return' \ 'error' From patchwork Mon Oct 28 12:14:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215449 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E6AE3112C for ; Mon, 28 Oct 2019 12:57:14 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BDA03214D9 for ; Mon, 28 Oct 2019 12:57:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RDJL3Fi0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BDA03214D9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53458 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4a9-0007Om-De for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:57:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50324) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wI-00027F-Uy for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wD-0001YX-19 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:02 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:30097 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wC-0001YJ-Tt for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:15:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264956; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9ClX8Dcj/b9NyXoYstV/5bZOBSuCzt7TPzVHiWSS1jk=; b=RDJL3Fi0ezkEkjkIuhpXE59yj42HSQsDw1TeAoJtGWB7wJ5bMbnrruKxSuMuyZtNOmQs1+ 9Of+AtLxy/mQNbI+DnO+jzj9nnIm5WTl/EcC1BEYn+HhVe9VD0eEvHmtqKFqaDzI/dWLK/ 0le3WaHOShCm9wO3dBhkzDCU7KWszAM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-107-_a-qmJHMOZmtpb700fDpbA-1; Mon, 28 Oct 2019 08:15:54 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5E5EA800FF1; Mon, 28 Oct 2019 12:15:53 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 03AAB600C7; Mon, 28 Oct 2019 12:15:52 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 24/69] iotests/183: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:16 +0100 Message-Id: <20191028121501.15279-25-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: _a-qmJHMOZmtpb700fDpbA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Thomas Huth Message-id: 20191017133155.5327-13-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/183 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/183 b/tests/qemu-iotests/183 index 04fb344d08..bced83fae0 100755 --- a/tests/qemu-iotests/183 +++ b/tests/qemu-iotests/183 @@ -26,7 +26,7 @@ echo "QA output created by $seq" status=1 # failure is the default! -MIG_SOCKET="${TEST_DIR}/migrate" +MIG_SOCKET="${SOCK_DIR}/migrate" _cleanup() { From patchwork Mon Oct 28 12:14:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215439 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A1C9C112B for ; Mon, 28 Oct 2019 12:53:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 77E3120873 for ; Mon, 28 Oct 2019 12:53:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="OweeTwfi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 77E3120873 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53431 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Wd-0006eK-7X for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:53:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50314) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wH-00024w-Bb for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wG-0001a3-2Q for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:01 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:40192 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wF-0001Zs-VG for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264959; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WUZxlFAlgm2HeSYNVOY/MIClvfbL18TqjsSWOMeUolA=; b=OweeTwfivDgjXR8SkhB/Svink2dJIBJdmdvzrKZvHwI5nur135ACFiALStKed1/8azqjl2 pBWT4VQ4tlz7QU6AgH+MHGWrCQeql51DulgOciAJl1zYguMdWnNzZBbPWbJB7s4Z4BZ5Dc IHUyeFILDnE35N5r6cNvSofSLdr9qJs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-203-l_Sj_xUqOoiyovzyAo_nGA-1; Mon, 28 Oct 2019 08:15:56 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 74EAE476; Mon, 28 Oct 2019 12:15:55 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 17EE660C4B; Mon, 28 Oct 2019 12:15:54 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 25/69] iotests/192: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:17 +0100 Message-Id: <20191028121501.15279-26-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: l_Sj_xUqOoiyovzyAo_nGA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-14-mreitz@redhat.com Reviewed-by: Thomas Huth Signed-off-by: Max Reitz --- tests/qemu-iotests/192 | 4 ++-- tests/qemu-iotests/192.out | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/192 b/tests/qemu-iotests/192 index 034432272f..d2ba55dd90 100755 --- a/tests/qemu-iotests/192 +++ b/tests/qemu-iotests/192 @@ -31,7 +31,7 @@ _cleanup() { _cleanup_qemu _cleanup_test_img - rm -f "$TEST_DIR/nbd" + rm -f "$SOCK_DIR/nbd" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -66,7 +66,7 @@ else QEMU_COMM_TIMEOUT=1 fi -_send_qemu_cmd $h "nbd_server_start unix:$TEST_DIR/nbd" "(qemu)" +_send_qemu_cmd $h "nbd_server_start unix:$SOCK_DIR/nbd" "(qemu)" _send_qemu_cmd $h "nbd_server_add -w drive0" "(qemu)" _send_qemu_cmd $h "q" "(qemu)" diff --git a/tests/qemu-iotests/192.out b/tests/qemu-iotests/192.out index 1e0be4c4d7..b9429dbe36 100644 --- a/tests/qemu-iotests/192.out +++ b/tests/qemu-iotests/192.out @@ -1,7 +1,7 @@ QA output created by 192 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 QEMU X.Y.Z monitor - type 'help' for more information -(qemu) nbd_server_start unix:TEST_DIR/nbd +(qemu) nbd_server_start unix:SOCK_DIR/nbd (qemu) nbd_server_add -w drive0 (qemu) q *** done From patchwork Mon Oct 28 12:14:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215339 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A2516112C for ; Mon, 28 Oct 2019 12:43:01 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 78E4F214E0 for ; Mon, 28 Oct 2019 12:43:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YHqEAmEh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 78E4F214E0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53320 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4MN-0003LD-Vr for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:43:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50345) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wJ-000289-Mm for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wI-0001cU-Hq for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:03 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:53174 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wI-0001cH-Ds for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264962; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ePALGZNtYN4yb2Q1Q4Gzvr4QXvNLnWTMvFhNoKXatIo=; b=YHqEAmEhCX78FpjXlC2X2+B+pxCra9g032gL/bvOkJZVPex7YxAKZaZ8+HYvFwMz4EPjlN itcMBdfOtfdAIRST5N93BkUNsMbjiaJNz4/H0+ls6wuwObbcvTGxjEbp+bWnSW0moMGXkU 8j5cnXP00kgL80Yyf00DN3/O7MIbVpk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-167-TBWfczjTOTWzOtP5Wqr1jg-1; Mon, 28 Oct 2019 08:15:58 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 72764180496E; Mon, 28 Oct 2019 12:15:57 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1BA925DA32; Mon, 28 Oct 2019 12:15:56 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 26/69] iotests/194: Create sockets in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:18 +0100 Message-Id: <20191028121501.15279-27-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: TBWfczjTOTWzOtP5Wqr1jg-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-15-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/194 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194 index d746ab1e21..72e47e8833 100755 --- a/tests/qemu-iotests/194 +++ b/tests/qemu-iotests/194 @@ -26,8 +26,8 @@ iotests.verify_platform(['linux']) with iotests.FilePath('source.img') as source_img_path, \ iotests.FilePath('dest.img') as dest_img_path, \ - iotests.FilePath('migration.sock') as migration_sock_path, \ - iotests.FilePath('nbd.sock') as nbd_sock_path, \ + iotests.FilePaths(['migration.sock', 'nbd.sock'], iotests.sock_dir) as \ + [migration_sock_path, nbd_sock_path], \ iotests.VM('source') as source_vm, \ iotests.VM('dest') as dest_vm: From patchwork Mon Oct 28 12:14:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215329 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B806113B1 for ; Mon, 28 Oct 2019 12:40:20 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8E1E4214D9 for ; Mon, 28 Oct 2019 12:40:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JIV4DjGR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E1E4214D9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53292 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Jm-0007KZ-VM for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:40:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50375) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wM-0002BH-0s for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wK-0001do-VT for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:05 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:34732 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wK-0001dd-SQ for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264964; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KdZ18UvxzPckPU2fWdziFcivT7uUX45HOdRdzeCHdV4=; b=JIV4DjGR7nbKLHX7o7MDqhIR0RaTrVtk/788qat9RokNqlyC3VLIahUcd3Zk5IinlJ796h +Y0ir1PnURNp/51a/looGN4emT96mg5b7tpnB7LAqevGVvXvWSxv4BvnCWOSGtyhUz0lcv a3AOmbStps5rpegSyqYsYEhR/wFCBDU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-327-eYyhQfl_OaqZj5YjAp4JfA-1; Mon, 28 Oct 2019 08:16:00 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 781C280183E; Mon, 28 Oct 2019 12:15:59 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1E1C560BF7; Mon, 28 Oct 2019 12:15:58 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 27/69] iotests/201: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:19 +0100 Message-Id: <20191028121501.15279-28-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: eYyhQfl_OaqZj5YjAp4JfA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-16-mreitz@redhat.com Reviewed-by: Thomas Huth Signed-off-by: Max Reitz --- tests/qemu-iotests/201 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/201 b/tests/qemu-iotests/201 index 7abf740fe4..86fa37e714 100755 --- a/tests/qemu-iotests/201 +++ b/tests/qemu-iotests/201 @@ -24,7 +24,7 @@ echo "QA output created by $seq" status=1 # failure is the default! -MIG_SOCKET="${TEST_DIR}/migrate" +MIG_SOCKET="${SOCK_DIR}/migrate" # get standard environment, filters and checks . ./common.rc From patchwork Mon Oct 28 12:14:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215291 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A444213B1 for ; Mon, 28 Oct 2019 12:37:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7AE34208C0 for ; Mon, 28 Oct 2019 12:37:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BFsm/EhB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7AE34208C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53262 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4H3-0003Dc-3h for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:37:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50376) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wM-0002BL-2G for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wL-0001dy-3d for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:05 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:31995 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wK-0001dY-W5 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264964; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ifyOe2MZlefiHmTiAm8gLP6U8cs2dI3DYXMTC8qkDDg=; b=BFsm/EhBjik5jGVA7TSSHZjCuhPdxESGr11HZr2vLNYVqNTa9rPBvLElrOMJMZhz3ibTzG 3s9mNtR5d4TlFeZ3hjVseSodWfxYQ2bwGvUx3AC7gMXaXrPw/FSNGEwGf/BkYn+4sLbvVr 8QDd5OKYDnLVy2eeWMMTeOxpz45/S/M= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-250-rAX136aQP3aqIqTIQ2jENQ-1; Mon, 28 Oct 2019 08:16:02 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7BFD9800FF1; Mon, 28 Oct 2019 12:16:01 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 251905C1D6; Mon, 28 Oct 2019 12:16:00 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 28/69] iotests/205: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:20 +0100 Message-Id: <20191028121501.15279-29-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: rAX136aQP3aqIqTIQ2jENQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-17-mreitz@redhat.com Reviewed-by: Thomas Huth Signed-off-by: Max Reitz --- tests/qemu-iotests/205 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/205 b/tests/qemu-iotests/205 index 76f6c5fa2b..4bb2c21e8b 100755 --- a/tests/qemu-iotests/205 +++ b/tests/qemu-iotests/205 @@ -24,7 +24,7 @@ import iotests import time from iotests import qemu_img_create, qemu_io, filter_qemu_io, QemuIoInteractive -nbd_sock = os.path.join(iotests.test_dir, 'nbd_sock') +nbd_sock = os.path.join(iotests.sock_dir, 'nbd_sock') nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock disk = os.path.join(iotests.test_dir, 'disk') From patchwork Mon Oct 28 12:14:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215359 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DC693112C for ; Mon, 28 Oct 2019 12:44:02 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B1DB021734 for ; Mon, 28 Oct 2019 12:44:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CTvF7mg5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B1DB021734 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53336 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4NN-0006Mi-KA for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:44:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50406) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wS-0002FB-9r for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wQ-0001gH-4K for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:12 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:34105 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wP-0001fe-Am for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aQySpcdMF2qRUs1pIR4JbiwZcLMvjua8tE1iMlsuzl4=; b=CTvF7mg5MUdKEe+gTlIWjbiaQHq68qyZDTS1S61UelmQIR7cykLJEy5s/soynWjoBaiG9e N3pJUZommYfoW2VhoaAi9XPmgwBOzooD5Eso1HKQTDG/5PgH5izHq1BeHEdmupvj+tYyU7 hFg6JvrdKNnBELcfqQuC3aW8KYZTQ3E= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-292-c7-SvNFNNCG3_RNQZyFR1g-1; Mon, 28 Oct 2019 08:16:04 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 807DD80183E; Mon, 28 Oct 2019 12:16:03 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 248705DA32; Mon, 28 Oct 2019 12:16:02 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 29/69] iotests/208: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:21 +0100 Message-Id: <20191028121501.15279-30-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: c7-SvNFNNCG3_RNQZyFR1g-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-18-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/208 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/208 b/tests/qemu-iotests/208 index 1e202388dc..546eb1de3e 100755 --- a/tests/qemu-iotests/208 +++ b/tests/qemu-iotests/208 @@ -26,7 +26,7 @@ iotests.verify_image_format(supported_fmts=['generic']) with iotests.FilePath('disk.img') as disk_img_path, \ iotests.FilePath('disk-snapshot.img') as disk_snapshot_img_path, \ - iotests.FilePath('nbd.sock') as nbd_sock_path, \ + iotests.FilePath('nbd.sock', iotests.sock_dir) as nbd_sock_path, \ iotests.VM() as vm: img_size = '10M' From patchwork Mon Oct 28 12:14:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215327 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 37A6913B1 for ; Mon, 28 Oct 2019 12:40:16 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0B5A7214D9 for ; Mon, 28 Oct 2019 12:40:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="c2Jd4mf8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0B5A7214D9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53291 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Ji-00079i-PE for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:40:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50422) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wU-0002GC-5X for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wS-0001hB-3d for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:13 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:22975 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wQ-0001fi-4N for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0KtF1ttxzQVWMJ0uFhuyIKocv6MjBfAG0v3x4sYo5rw=; b=c2Jd4mf8A7Ymp95YLW8pRJHlG4io37u1kDl4jE6HoMasDpLKRW1YSUx1wLE/VTghEFkR/q lkBqbk0uzKcEExXRfWGSuRGw4SmDj6Bf4Krxvv0tASvA8oDV/MAdQA2tJs7sRQev9QhQ3Z WhIt+NlM5Fqq1aqJ7zYrscHegI0lU/o= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-168-JheJ54MEMNqZOCp8-NO06Q-1; Mon, 28 Oct 2019 08:16:06 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 88D2310052E0; Mon, 28 Oct 2019 12:16:05 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 284D160856; Mon, 28 Oct 2019 12:16:05 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 30/69] iotests/209: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:22 +0100 Message-Id: <20191028121501.15279-31-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: JheJ54MEMNqZOCp8-NO06Q-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-19-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/209 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/209 b/tests/qemu-iotests/209 index 259e991ec6..e0f464bcbe 100755 --- a/tests/qemu-iotests/209 +++ b/tests/qemu-iotests/209 @@ -24,7 +24,8 @@ from iotests import qemu_img_create, qemu_io, qemu_img_verbose, qemu_nbd, \ iotests.verify_image_format(supported_fmts=['qcow2']) -disk, nbd_sock = file_path('disk', 'nbd-sock') +disk = file_path('disk') +nbd_sock = file_path('nbd-sock', base_dir=iotests.sock_dir) nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock qemu_img_create('-f', iotests.imgfmt, disk, '1M') From patchwork Mon Oct 28 12:14:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215469 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A0F313B1 for ; Mon, 28 Oct 2019 13:00:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2E312214D9 for ; Mon, 28 Oct 2019 13:00:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UWpbfxrv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2E312214D9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53502 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4dE-00078c-Fx for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:00:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50471) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wW-0002JN-4u for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wU-0001hy-2R for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:16 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:51976 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wS-0001gP-9C for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264970; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ok0saJFqnZVLEoIQ9B9ZFAcmtjJaXJ+UXcAM646dHUQ=; b=UWpbfxrv5ywIRmmnic/ClIfshIZbka6Zhh0CEmWYK+WYTBE8qS+GUdQA9dbGKFrew3fYAe 3yBI3EjzGqHqvcio7JrD8lgJoUiRspGhasZolb7u8xeGG+qzIJXeWD6cl0gNA9QOIau6yo gaAY55lnDNqduuaIcG8spdpRCkAXvzE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-344-OSRhla0nM525ZfbL8VxOkA-1; Mon, 28 Oct 2019 08:16:08 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 904F1800FF1; Mon, 28 Oct 2019 12:16:07 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 362C7261A7; Mon, 28 Oct 2019 12:16:07 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 31/69] iotests/222: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:23 +0100 Message-Id: <20191028121501.15279-32-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: OSRhla0nM525ZfbL8VxOkA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-20-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/222 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222 index 0ead56d574..3f9f934ad8 100644 --- a/tests/qemu-iotests/222 +++ b/tests/qemu-iotests/222 @@ -48,7 +48,7 @@ remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1] with iotests.FilePath('base.img') as base_img_path, \ iotests.FilePath('fleece.img') as fleece_img_path, \ - iotests.FilePath('nbd.sock') as nbd_sock_path, \ + iotests.FilePath('nbd.sock', iotests.sock_dir) as nbd_sock_path, \ iotests.VM() as vm: log('--- Setting up images ---') From patchwork Mon Oct 28 12:14:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215341 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D100112C for ; Mon, 28 Oct 2019 12:43:07 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4345B214E0 for ; Mon, 28 Oct 2019 12:43:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IT+PTfjH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4345B214E0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53322 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4MT-0003dE-TO for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:43:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50454) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wV-0002IX-KK for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wU-0001iS-8S for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:15 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:22212 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wU-0001ha-3S for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264972; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nzYfsrvbFcvTUAC4fmCCmaulR3gKaiRr0GcI/WZnGqs=; b=IT+PTfjH0+vOP5bOV2x406QYCDWdWsdJYgbxhdFSKiRxLcWIWnB9/DMOY3m8RcvtvthxlD h4ecgDS5piMksI5Tk3aVPccp1k31BBdlQ8CKCKnE0FwZjpoSaqcEVTiYRgiCjKl7BQDkAD 31g51KGeBpXZat4BxP4sWXff4nyPxBQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-351-xz8P0AQ2PDusQ9eD_d9U9g-1; Mon, 28 Oct 2019 08:16:10 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A2649180496E; Mon, 28 Oct 2019 12:16:09 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4774A60850; Mon, 28 Oct 2019 12:16:09 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 32/69] iotests/223: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:24 +0100 Message-Id: <20191028121501.15279-33-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: xz8P0AQ2PDusQ9eD_d9U9g-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-21-mreitz@redhat.com Reviewed-by: Thomas Huth Signed-off-by: Max Reitz --- tests/qemu-iotests/223 | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/223 b/tests/qemu-iotests/223 index 2ba3d8124b..b5a80e50bb 100755 --- a/tests/qemu-iotests/223 +++ b/tests/qemu-iotests/223 @@ -28,7 +28,7 @@ _cleanup() nbd_server_stop _cleanup_test_img _cleanup_qemu - rm -f "$TEST_DIR/nbd" + rm -f "$SOCK_DIR/nbd" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -125,11 +125,11 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add", "arguments":{"device":"n"}}' "error" # Attempt add without server _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-start", "arguments":{"addr":{"type":"unix", - "data":{"path":"'"$TEST_DIR/nbd"'"}}}}' "return" + "data":{"path":"'"$SOCK_DIR/nbd"'"}}}}' "return" _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-start", "arguments":{"addr":{"type":"unix", - "data":{"path":"'"$TEST_DIR/nbd"1'"}}}}' "error" # Attempt second server -$QEMU_NBD_PROG -L -k "$TEST_DIR/nbd" + "data":{"path":"'"$SOCK_DIR/nbd"1'"}}}}' "error" # Attempt second server +$QEMU_NBD_PROG -L -k "$SOCK_DIR/nbd" _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add", "arguments":{"device":"n", "bitmap":"b"}}' "return" _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add", @@ -145,14 +145,14 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add", _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add", "arguments":{"device":"n", "name":"n2", "writable":true, "bitmap":"b2"}}' "return" -$QEMU_NBD_PROG -L -k "$TEST_DIR/nbd" +$QEMU_NBD_PROG -L -k "$SOCK_DIR/nbd" echo echo "=== Contrast normal status to large granularity dirty-bitmap ===" echo QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT -IMG="driver=nbd,export=n,server.type=unix,server.path=$TEST_DIR/nbd" +IMG="driver=nbd,export=n,server.type=unix,server.path=$SOCK_DIR/nbd" $QEMU_IO -r -c 'r -P 0x22 512 512' -c 'r -P 0 512k 512k' -c 'r -P 0x11 1m 1m' \ -c 'r -P 0x33 2m 2m' --image-opts "$IMG" | _filter_qemu_io $QEMU_IMG map --output=json --image-opts \ @@ -164,7 +164,7 @@ echo echo "=== Contrast to small granularity dirty-bitmap ===" echo -IMG="driver=nbd,export=n2,server.type=unix,server.path=$TEST_DIR/nbd" +IMG="driver=nbd,export=n2,server.type=unix,server.path=$SOCK_DIR/nbd" $QEMU_IMG map --output=json --image-opts \ "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b2" | _filter_qemu_img_map From patchwork Mon Oct 28 12:14:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215425 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 34DB8112C for ; Mon, 28 Oct 2019 12:47:28 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0B254204FD for ; Mon, 28 Oct 2019 12:47:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="OarLdnCI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0B254204FD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53376 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Qg-0004cN-HS for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:47:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50500) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wX-0002Lk-QV for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wW-0001jv-Nw for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:17 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:48105 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wW-0001jZ-J4 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264976; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=riNSMiO+14ulx+UiW3yO9j58OeQiJkagmAlO8zmtw1M=; b=OarLdnCIprZ0ivDb3+jlIFNvGmm1TPkVA6W1eUF/K6pHFoF4T5KRNbT6osqCmnZt/OZqUs X4gUKtHyp9LTAFi/EA/XgUav8ClkPuI0r3QCiB8YC+80+A7XJ3BvmmsVtUpozUvtRI0/VQ WRCV6afc2gevOw9FGkygzVG9XK4fJbs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-234-EKzE7KbiN1OxYlem2PRICw-1; Mon, 28 Oct 2019 08:16:12 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AA39A10052E0; Mon, 28 Oct 2019 12:16:11 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4ED53600C7; Mon, 28 Oct 2019 12:16:11 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 33/69] iotests/240: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:25 +0100 Message-Id: <20191028121501.15279-34-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: EKzE7KbiN1OxYlem2PRICw-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-22-mreitz@redhat.com Reviewed-by: Thomas Huth Signed-off-by: Max Reitz --- tests/qemu-iotests/240 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/240 b/tests/qemu-iotests/240 index f73bc07d80..8b4337b58d 100755 --- a/tests/qemu-iotests/240 +++ b/tests/qemu-iotests/240 @@ -29,7 +29,7 @@ status=1 # failure is the default! _cleanup() { - rm -f "$TEST_DIR/nbd" + rm -f "$SOCK_DIR/nbd" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -135,7 +135,7 @@ echo run_qemu < X-Patchwork-Id: 11215433 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 614F2112B for ; Mon, 28 Oct 2019 12:51:15 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 37DA420873 for ; Mon, 28 Oct 2019 12:51:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="h6h40v7e" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 37DA420873 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53410 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4UL-0001j7-UE for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:51:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50526) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wa-0002OZ-03 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wY-0001lV-Op for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:19 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:55531 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wY-0001kf-KY for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264978; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=C+MXzl/3CFa1bQWMYehBdS2keQt5KxH37UFTv13JeXw=; b=h6h40v7e4kFzlGCXEceW7UX/CkMMoKwMRANWmJhSI1EpGy477bMHsancbjYueQ9j3mpVd2 aaw/modQaONIaBAclL+sS28UE1KHDfuFz7e0xWMNWkdkl2yD3QYEqVDkiNfi7il2Qs4b9F 4fXUD+15cKMREbwT6cnOZgN+u9Wf4NA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-202-T-IqAX4LPEmgeE5c1PjXvg-1; Mon, 28 Oct 2019 08:16:14 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AA792107AD28; Mon, 28 Oct 2019 12:16:13 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 53BEE5DA2C; Mon, 28 Oct 2019 12:16:13 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 34/69] iotests/267: Create socket in $SOCK_DIR Date: Mon, 28 Oct 2019 13:14:26 +0100 Message-Id: <20191028121501.15279-35-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: T-IqAX4LPEmgeE5c1PjXvg-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-23-mreitz@redhat.com Reviewed-by: Thomas Huth Signed-off-by: Max Reitz --- tests/qemu-iotests/267 | 4 ++-- tests/qemu-iotests/267.out | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/267 b/tests/qemu-iotests/267 index d37a67c012..170e173c0a 100755 --- a/tests/qemu-iotests/267 +++ b/tests/qemu-iotests/267 @@ -29,7 +29,7 @@ status=1 # failure is the default! _cleanup() { _cleanup_test_img - rm -f "$TEST_DIR/nbd" + rm -f "$SOCK_DIR/nbd" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -143,7 +143,7 @@ echo IMGOPTS="backing_file=$TEST_IMG.base" _make_test_img $size cat < X-Patchwork-Id: 11215441 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DDE57112C for ; Mon, 28 Oct 2019 12:53:49 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B326420873 for ; Mon, 28 Oct 2019 12:53:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="OD3m3/Id" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B326420873 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53436 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Wq-00073F-7p for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:53:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50552) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wc-0002RR-32 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wa-0001nt-TX for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:21 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:43339 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wa-0001nU-Pi for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264980; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+uE33LcIOWs/sS0rsD39zRzH6rBJR6SpdgAzF0EyPWQ=; b=OD3m3/IdtoeBcSwPel+PM7E606Ia0sE7dnKd6OxsvBkrBPEkU99Ce+W5ZLMJxJ52R9AHCy RoxI0j/sL1K68PL2enSUazSIR55H8gcp9qoZ4KR175HO9rundu48Ai0pra+gMjRiEYN27r +78fz1mEuiqWpcMNq10sKIolMVwmlsA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-324-N95CV77UM3-eOq9XMJ87BQ-1; Mon, 28 Oct 2019 08:16:16 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B84BC10052E0; Mon, 28 Oct 2019 12:16:15 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 56391600C7; Mon, 28 Oct 2019 12:16:15 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 35/69] iotests: Drop TEST_DIR filter from _filter_nbd Date: Mon, 28 Oct 2019 13:14:27 +0100 Message-Id: <20191028121501.15279-36-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: N95CV77UM3-eOq9XMJ87BQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Sockets should be placed into $SOCK_DIR instead of $TEST_DIR, so remove the $TEST_DIR filter from _filter_nbd. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191017133155.5327-24-mreitz@redhat.com Reviewed-by: Thomas Huth Signed-off-by: Max Reitz --- tests/qemu-iotests/common.filter | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index 0ee6042524..f870e00e44 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -221,7 +221,6 @@ _filter_nbd() # Filter out the TCP port number since this changes between runs. $SED -e '/nbd\/.*\.c:/d' \ -e 's#127\.0\.0\.1:[0-9]*#127.0.0.1:PORT#g' \ - -e "s#?socket=$TEST_DIR#?socket=TEST_DIR#g" \ -e "s#?socket=$SOCK_DIR#?socket=SOCK_DIR#g" \ -e 's#\(foo\|PORT/\?\|.sock\): Failed to .*$#\1#' } From patchwork Mon Oct 28 12:14:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215475 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F0FCF14E5 for ; Mon, 28 Oct 2019 13:05:33 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BF17C21734 for ; Mon, 28 Oct 2019 13:05:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="R0iG9ZPs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BF17C21734 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53544 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4iC-0007Iy-Cc for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:05:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50556) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wc-0002Rn-Eo for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wb-0001o8-5m for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:22 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:41699 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wa-0001nl-Vm for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264980; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FQH891K9cRbxEVZvL7E4z3vBB3xxoYPbvBEZln4pkVQ=; b=R0iG9ZPstwVyO2iiS6PtuPShr0XfTn/+xgJfCrz55ipggzUE9QjsINFnLHbIm0iSBCU/kp XaoOeVLOPAJvAaxON8VHLqPyJP4pswa8r2lAacCQR1VnalBzjnFu7TmXDQErtqgyR+shRM 8fNMZ3IIFipTRxbZM2RhoQLmqrklS8A= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-391-UUWKxpB8MbG7Yg3hWyY8VQ-1; Mon, 28 Oct 2019 08:16:18 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C0D26800FF1; Mon, 28 Oct 2019 12:16:17 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6298F6084E; Mon, 28 Oct 2019 12:16:17 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 36/69] block/block-copy: allocate buffer in block_copy_with_bounce_buffer Date: Mon, 28 Oct 2019 13:14:28 +0100 Message-Id: <20191028121501.15279-37-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: UUWKxpB8MbG7Yg3hWyY8VQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Move bounce_buffer allocation block_copy_with_bounce_buffer. This commit simplifies further work on implementing copying by larger chunks (of different size) and further asynchronous handling of block_copy iterations (with help of block/aio_task API). Allocation works fast, a lot faster than disk io, so it's not a problem that we now allocate/free bounce_buffer more times. And we anyway will have to allocate several bounce_buffers for parallel execution of loop iterations in future. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20191022111805.3432-2-vsementsov@virtuozzo.com Signed-off-by: Max Reitz --- block/block-copy.c | 21 ++++++++------------- 1 file changed, 8 insertions(+), 13 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 066e3a7274..ecd086010e 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -126,20 +126,17 @@ void block_copy_set_callbacks( static int coroutine_fn block_copy_with_bounce_buffer(BlockCopyState *s, int64_t start, int64_t end, - bool *error_is_read, - void **bounce_buffer) + bool *error_is_read) { int ret; int nbytes; + void *bounce_buffer = qemu_blockalign(s->source->bs, s->cluster_size); assert(QEMU_IS_ALIGNED(start, s->cluster_size)); bdrv_reset_dirty_bitmap(s->copy_bitmap, start, s->cluster_size); nbytes = MIN(s->cluster_size, s->len - start); - if (!*bounce_buffer) { - *bounce_buffer = qemu_blockalign(s->source->bs, s->cluster_size); - } - ret = bdrv_co_pread(s->source, start, nbytes, *bounce_buffer, 0); + ret = bdrv_co_pread(s->source, start, nbytes, bounce_buffer, 0); if (ret < 0) { trace_block_copy_with_bounce_buffer_read_fail(s, start, ret); if (error_is_read) { @@ -148,7 +145,7 @@ static int coroutine_fn block_copy_with_bounce_buffer(BlockCopyState *s, goto fail; } - ret = bdrv_co_pwrite(s->target, start, nbytes, *bounce_buffer, + ret = bdrv_co_pwrite(s->target, start, nbytes, bounce_buffer, s->write_flags); if (ret < 0) { trace_block_copy_with_bounce_buffer_write_fail(s, start, ret); @@ -158,8 +155,11 @@ static int coroutine_fn block_copy_with_bounce_buffer(BlockCopyState *s, goto fail; } + qemu_vfree(bounce_buffer); + return nbytes; fail: + qemu_vfree(bounce_buffer); bdrv_set_dirty_bitmap(s->copy_bitmap, start, s->cluster_size); return ret; @@ -271,7 +271,6 @@ int coroutine_fn block_copy(BlockCopyState *s, { int ret = 0; int64_t end = bytes + start; /* bytes */ - void *bounce_buffer = NULL; int64_t status_bytes; BlockCopyInFlightReq req; @@ -324,7 +323,7 @@ int coroutine_fn block_copy(BlockCopyState *s, } if (!s->use_copy_range) { ret = block_copy_with_bounce_buffer(s, start, dirty_end, - error_is_read, &bounce_buffer); + error_is_read); } if (ret < 0) { break; @@ -335,10 +334,6 @@ int coroutine_fn block_copy(BlockCopyState *s, ret = 0; } - if (bounce_buffer) { - qemu_vfree(bounce_buffer); - } - block_copy_inflight_req_end(&req); return ret; From patchwork Mon Oct 28 12:14:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215429 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56193112B for ; Mon, 28 Oct 2019 12:49:33 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2076D20873 for ; Mon, 28 Oct 2019 12:49:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ZuvtLgZ5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2076D20873 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53386 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Sh-00075n-N3 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:49:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50603) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wg-0002XK-Gv for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wf-0001qQ-9M for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:26 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:21591 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wf-0001q7-4Z for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264984; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZF9YymZ7kRE4bi8nH2i1Dm5sPp21aqCJLNHtMFDrRco=; b=ZuvtLgZ5U9PWtRm7IR/L8e2lPpdK0lVA3+HPdnR5f7GQdLWkUbXyx6gxIoZwAOCYQvQcj9 9ZQEXaHp8C5Vr1lUHq7y/Nhs7KiEhelx0q0O8a2BtLGyC3KMMSsO1mscTIGhyTBn9fdCXf 2lbPB9v829JnjB6RYk5WfYIabLtSjSM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-398-0S_vy_QCMxaxFNhVPt5wmQ-1; Mon, 28 Oct 2019 08:16:20 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C90FA476; Mon, 28 Oct 2019 12:16:19 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6BE396084E; Mon, 28 Oct 2019 12:16:19 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 37/69] block/block-copy: limit copy_range_size to 16 MiB Date: Mon, 28 Oct 2019 13:14:29 +0100 Message-Id: <20191028121501.15279-38-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: 0S_vy_QCMxaxFNhVPt5wmQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Large copy range may imply memory allocation and large io effort, so using 2G copy range request may be bad idea. Let's limit it to 16 MiB. It also helps the following patch to refactor copy-with-offload fallback to copy-with-bounce-buffer. Note, that total memory usage of backup is still not limited, it will be fixed in further commit. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20191022111805.3432-3-vsementsov@virtuozzo.com Signed-off-by: Max Reitz --- block/block-copy.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index ecd086010e..cd45b373a9 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -18,6 +18,9 @@ #include "qapi/error.h" #include "block/block-copy.h" #include "sysemu/block-backend.h" +#include "qemu/units.h" + +#define BLOCK_COPY_MAX_COPY_RANGE (16 * MiB) static void coroutine_fn block_copy_wait_inflight_reqs(BlockCopyState *s, int64_t start, @@ -70,9 +73,12 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, { BlockCopyState *s; BdrvDirtyBitmap *copy_bitmap; + + /* Ignore BLOCK_COPY_MAX_COPY_RANGE if requested cluster_size is larger */ uint32_t max_transfer = - MIN_NON_ZERO(INT_MAX, MIN_NON_ZERO(source->bs->bl.max_transfer, - target->bs->bl.max_transfer)); + MIN_NON_ZERO(MAX(cluster_size, BLOCK_COPY_MAX_COPY_RANGE), + MIN_NON_ZERO(source->bs->bl.max_transfer, + target->bs->bl.max_transfer)); copy_bitmap = bdrv_create_dirty_bitmap(source->bs, cluster_size, NULL, errp); From patchwork Mon Oct 28 12:14:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215453 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AA4341747 for ; Mon, 28 Oct 2019 12:57:23 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 80E9021721 for ; Mon, 28 Oct 2019 12:57:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ObSjtJl2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 80E9021721 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53462 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4aI-0007ij-2O for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:57:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50667) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wl-0002dK-Pz for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wg-0001r4-6Z for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:31 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:20975 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wg-0001qh-1n for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264985; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u2JO2BghQIoNSQXVsfHG4EPUO/umTOL8x7M1bb2GdOs=; b=ObSjtJl2l8GMsWnE6BVftWDzWpAEtMW9AStX3tPGRAyB+vGMcm5153gzbuKhHSB3q5gpzp dbMOaU4fUOcNIiNsBTZs1XxDwhg0e6x9+9l2DbyEQDpHwD1H7/KQsutVvrqYIE8eSKWFKG eAb0cAil8DsmnZdjItS9dWwiK6qn6jw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-10-FGE-2gvoPz27jgK9cvbpew-1; Mon, 28 Oct 2019 08:16:22 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 001FF80183E; Mon, 28 Oct 2019 12:16:21 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6F3315D6BB; Mon, 28 Oct 2019 12:16:21 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 38/69] block/block-copy: refactor copying Date: Mon, 28 Oct 2019 13:14:30 +0100 Message-Id: <20191028121501.15279-39-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-MC-Unique: FGE-2gvoPz27jgK9cvbpew-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Merge copying code into one function block_copy_do_copy, which only calls bdrv_ io functions and don't do any synchronization (like dirty bitmap set/reset). Refactor block_copy() function so that it takes full decision about size of chunk to be copied and does all the synchronization (checking intersecting requests, set/reset dirty bitmaps). It will help: - introduce parallel processing of block_copy iterations: we need to calculate chunk size, start async chunk copying and go to the next iteration - simplify synchronization improvement (like memory limiting in further commit and reducing critical section (now we lock the whole requested range, when actually we need to lock only dirty region which we handle at the moment)) Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20191022111805.3432-4-vsementsov@virtuozzo.com Signed-off-by: Max Reitz --- block/block-copy.c | 118 ++++++++++++++++++++------------------------- block/trace-events | 6 +-- 2 files changed, 54 insertions(+), 70 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index cd45b373a9..845b2423dc 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -126,79 +126,64 @@ void block_copy_set_callbacks( } /* - * Copy range to target with a bounce buffer and return the bytes copied. If - * error occurred, return a negative error number + * block_copy_do_copy + * + * Do copy of cluser-aligned chunk. @end is allowed to exceed s->len only to + * cover last cluster when s->len is not aligned to clusters. + * + * No sync here: nor bitmap neighter intersecting requests handling, only copy. + * + * Returns 0 on success. */ -static int coroutine_fn block_copy_with_bounce_buffer(BlockCopyState *s, - int64_t start, - int64_t end, - bool *error_is_read) +static int coroutine_fn block_copy_do_copy(BlockCopyState *s, + int64_t start, int64_t end, + bool *error_is_read) { int ret; - int nbytes; - void *bounce_buffer = qemu_blockalign(s->source->bs, s->cluster_size); + int nbytes = MIN(end, s->len) - start; + void *bounce_buffer = NULL; assert(QEMU_IS_ALIGNED(start, s->cluster_size)); - bdrv_reset_dirty_bitmap(s->copy_bitmap, start, s->cluster_size); - nbytes = MIN(s->cluster_size, s->len - start); + assert(QEMU_IS_ALIGNED(end, s->cluster_size)); + assert(end < s->len || end == QEMU_ALIGN_UP(s->len, s->cluster_size)); + + if (s->use_copy_range) { + ret = bdrv_co_copy_range(s->source, start, s->target, start, nbytes, + 0, s->write_flags); + if (ret < 0) { + trace_block_copy_copy_range_fail(s, start, ret); + s->use_copy_range = false; + /* Fallback to read+write with allocated buffer */ + } else { + goto out; + } + } + + bounce_buffer = qemu_blockalign(s->source->bs, nbytes); ret = bdrv_co_pread(s->source, start, nbytes, bounce_buffer, 0); if (ret < 0) { - trace_block_copy_with_bounce_buffer_read_fail(s, start, ret); + trace_block_copy_read_fail(s, start, ret); if (error_is_read) { *error_is_read = true; } - goto fail; + goto out; } ret = bdrv_co_pwrite(s->target, start, nbytes, bounce_buffer, s->write_flags); if (ret < 0) { - trace_block_copy_with_bounce_buffer_write_fail(s, start, ret); + trace_block_copy_write_fail(s, start, ret); if (error_is_read) { *error_is_read = false; } - goto fail; + goto out; } +out: qemu_vfree(bounce_buffer); - return nbytes; -fail: - qemu_vfree(bounce_buffer); - bdrv_set_dirty_bitmap(s->copy_bitmap, start, s->cluster_size); return ret; - -} - -/* - * Copy range to target and return the bytes copied. If error occurred, return a - * negative error number. - */ -static int coroutine_fn block_copy_with_offload(BlockCopyState *s, - int64_t start, - int64_t end) -{ - int ret; - int nr_clusters; - int nbytes; - - assert(QEMU_IS_ALIGNED(s->copy_range_size, s->cluster_size)); - assert(QEMU_IS_ALIGNED(start, s->cluster_size)); - nbytes = MIN(s->copy_range_size, MIN(end, s->len) - start); - nr_clusters = DIV_ROUND_UP(nbytes, s->cluster_size); - bdrv_reset_dirty_bitmap(s->copy_bitmap, start, - s->cluster_size * nr_clusters); - ret = bdrv_co_copy_range(s->source, start, s->target, start, nbytes, - 0, s->write_flags); - if (ret < 0) { - trace_block_copy_with_offload_fail(s, start, ret); - bdrv_set_dirty_bitmap(s->copy_bitmap, start, - s->cluster_size * nr_clusters); - return ret; - } - - return nbytes; } /* @@ -294,7 +279,7 @@ int coroutine_fn block_copy(BlockCopyState *s, block_copy_inflight_req_begin(s, &req, start, end); while (start < end) { - int64_t dirty_end; + int64_t next_zero, chunk_end; if (!bdrv_dirty_bitmap_get(s->copy_bitmap, start)) { trace_block_copy_skip(s, start); @@ -302,10 +287,15 @@ int coroutine_fn block_copy(BlockCopyState *s, continue; /* already copied */ } - dirty_end = bdrv_dirty_bitmap_next_zero(s->copy_bitmap, start, - (end - start)); - if (dirty_end < 0) { - dirty_end = end; + chunk_end = MIN(end, start + (s->use_copy_range ? + s->copy_range_size : s->cluster_size)); + + next_zero = bdrv_dirty_bitmap_next_zero(s->copy_bitmap, start, + chunk_end - start); + if (next_zero >= 0) { + assert(next_zero > start); /* start is dirty */ + assert(next_zero < chunk_end); /* no need to do MIN() */ + chunk_end = next_zero; } if (s->skip_unallocated) { @@ -316,27 +306,21 @@ int coroutine_fn block_copy(BlockCopyState *s, continue; } /* Clamp to known allocated region */ - dirty_end = MIN(dirty_end, start + status_bytes); + chunk_end = MIN(chunk_end, start + status_bytes); } trace_block_copy_process(s, start); - if (s->use_copy_range) { - ret = block_copy_with_offload(s, start, dirty_end); - if (ret < 0) { - s->use_copy_range = false; - } - } - if (!s->use_copy_range) { - ret = block_copy_with_bounce_buffer(s, start, dirty_end, - error_is_read); - } + bdrv_reset_dirty_bitmap(s->copy_bitmap, start, chunk_end - start); + + ret = block_copy_do_copy(s, start, chunk_end, error_is_read); if (ret < 0) { + bdrv_set_dirty_bitmap(s->copy_bitmap, start, chunk_end - start); break; } - start += ret; - s->progress_bytes_callback(ret, s->progress_opaque); + s->progress_bytes_callback(chunk_end - start, s->progress_opaque); + start = chunk_end; ret = 0; } diff --git a/block/trace-events b/block/trace-events index b8d70f5242..ccde15a14c 100644 --- a/block/trace-events +++ b/block/trace-events @@ -45,9 +45,9 @@ backup_do_cow_return(void *job, int64_t offset, uint64_t bytes, int ret) "job %p block_copy_skip(void *bcs, int64_t start) "bcs %p start %"PRId64 block_copy_skip_range(void *bcs, int64_t start, uint64_t bytes) "bcs %p start %"PRId64" bytes %"PRId64 block_copy_process(void *bcs, int64_t start) "bcs %p start %"PRId64 -block_copy_with_bounce_buffer_read_fail(void *bcs, int64_t start, int ret) "bcs %p start %"PRId64" ret %d" -block_copy_with_bounce_buffer_write_fail(void *bcs, int64_t start, int ret) "bcs %p start %"PRId64" ret %d" -block_copy_with_offload_fail(void *bcs, int64_t start, int ret) "bcs %p start %"PRId64" ret %d" +block_copy_copy_range_fail(void *bcs, int64_t start, int ret) "bcs %p start %"PRId64" ret %d" +block_copy_read_fail(void *bcs, int64_t start, int ret) "bcs %p start %"PRId64" ret %d" +block_copy_write_fail(void *bcs, int64_t start, int ret) "bcs %p start %"PRId64" ret %d" # ../blockdev.c qmp_block_job_cancel(void *job) "job %p" From patchwork Mon Oct 28 12:14:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215445 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A7CBF112B for ; Mon, 28 Oct 2019 12:55:50 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7E0A520873 for ; Mon, 28 Oct 2019 12:55:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CDMxkRV2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7E0A520873 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53444 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Yn-0003pw-3z for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:55:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50639) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wk-0002b3-4k for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wi-0001s4-Cp for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:29 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:54672 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wi-0001rO-8E for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264986; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NLXLDNRwLKHMK2M0jKKx2XmiojPxKUfmgaG8PxKK2W8=; b=CDMxkRV26mJP8DUcpzxBw2ebxyQQInHYa8RkqWUe5iy1EXx/0PmJHfxRyV+/gsSKG5rz3k PZlZjMvnT1XrovI++dZTqh5E608x97XDq15TXO8u5HvtUb1uECpqWJ9U5hrIotrzsbFmue OYeNZRRMuc1XoXpnKgsOenek3ZY30+k= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-222-vtoB4tZoNAyLtoeVwvAN1g-1; Mon, 28 Oct 2019 08:16:25 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 271D6107AD28; Mon, 28 Oct 2019 12:16:24 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9B35A5C1D6; Mon, 28 Oct 2019 12:16:23 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 39/69] util: introduce SharedResource Date: Mon, 28 Oct 2019 13:14:31 +0100 Message-Id: <20191028121501.15279-40-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: vtoB4tZoNAyLtoeVwvAN1g-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Introduce an API for some shared splittable resource, like memory. It's going to be used by backup. Backup uses both read/write io and copy_range. copy_range may consume memory implictly, so the new API is abstract: it doesn't allocate any real memory but only hands out tickets. The idea is that we have some total amount of something and callers should wait in coroutine queue if there is not enough of the resource at the moment. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20191022111805.3432-5-vsementsov@virtuozzo.com Signed-off-by: Max Reitz --- util/Makefile.objs | 1 + include/qemu/co-shared-resource.h | 71 +++++++++++++++++++++++++++++ util/qemu-co-shared-resource.c | 76 +++++++++++++++++++++++++++++++ 3 files changed, 148 insertions(+) create mode 100644 include/qemu/co-shared-resource.h create mode 100644 util/qemu-co-shared-resource.c diff --git a/util/Makefile.objs b/util/Makefile.objs index 41bf59d127..df124af1c5 100644 --- a/util/Makefile.objs +++ b/util/Makefile.objs @@ -37,6 +37,7 @@ util-obj-y += rcu.o util-obj-$(CONFIG_MEMBARRIER) += sys_membarrier.o util-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o util-obj-y += qemu-coroutine-sleep.o +util-obj-y += qemu-co-shared-resource.o util-obj-y += coroutine-$(CONFIG_COROUTINE_BACKEND).o util-obj-y += buffer.o util-obj-y += timed-average.o diff --git a/include/qemu/co-shared-resource.h b/include/qemu/co-shared-resource.h new file mode 100644 index 0000000000..4e4503004c --- /dev/null +++ b/include/qemu/co-shared-resource.h @@ -0,0 +1,71 @@ +/* + * Helper functionality for distributing a fixed total amount of + * an abstract resource among multiple coroutines. + * + * Copyright (c) 2019 Virtuozzo International GmbH + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#ifndef QEMU_CO_SHARED_RESOURCE_H +#define QEMU_CO_SHARED_RESOURCE_H + + +typedef struct SharedResource SharedResource; + +/* + * Create SharedResource structure + * + * @total: total amount of some resource to be shared between clients + * + * Note: this API is not thread-safe. + */ +SharedResource *shres_create(uint64_t total); + +/* + * Release SharedResource structure + * + * This function may only be called once everything allocated by all + * clients has been deallocated. + */ +void shres_destroy(SharedResource *s); + +/* + * Try to allocate an amount of @n. Return true on success, and false + * if there is too little left of the collective resource to fulfill + * the request. + */ +bool co_try_get_from_shres(SharedResource *s, uint64_t n); + +/* + * Allocate an amount of @n, and, if necessary, yield until + * that becomes possible. + */ +void coroutine_fn co_get_from_shres(SharedResource *s, uint64_t n); + +/* + * Deallocate an amount of @n. The total amount allocated by a caller + * does not need to be deallocated/released with a single call, but may + * be split over several calls. For example, get(4), get(3), and then + * put(5), put(2). + */ +void coroutine_fn co_put_to_shres(SharedResource *s, uint64_t n); + + +#endif /* QEMU_CO_SHARED_RESOURCE_H */ diff --git a/util/qemu-co-shared-resource.c b/util/qemu-co-shared-resource.c new file mode 100644 index 0000000000..1c83cd9d29 --- /dev/null +++ b/util/qemu-co-shared-resource.c @@ -0,0 +1,76 @@ +/* + * Helper functionality for distributing a fixed total amount of + * an abstract resource among multiple coroutines. + * + * Copyright (c) 2019 Virtuozzo International GmbH + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#include "qemu/osdep.h" +#include "qemu/coroutine.h" +#include "qemu/co-shared-resource.h" + +struct SharedResource { + uint64_t total; + uint64_t available; + + CoQueue queue; +}; + +SharedResource *shres_create(uint64_t total) +{ + SharedResource *s = g_new0(SharedResource, 1); + + s->total = s->available = total; + qemu_co_queue_init(&s->queue); + + return s; +} + +void shres_destroy(SharedResource *s) +{ + assert(s->available == s->total); + g_free(s); +} + +bool co_try_get_from_shres(SharedResource *s, uint64_t n) +{ + if (s->available >= n) { + s->available -= n; + return true; + } + + return false; +} + +void coroutine_fn co_get_from_shres(SharedResource *s, uint64_t n) +{ + assert(n <= s->total); + while (!co_try_get_from_shres(s, n)) { + qemu_co_queue_wait(&s->queue, NULL); + } +} + +void coroutine_fn co_put_to_shres(SharedResource *s, uint64_t n) +{ + assert(s->total - s->available >= n); + s->available += n; + qemu_co_queue_restart_all(&s->queue); +} From patchwork Mon Oct 28 12:14:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215437 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8182E112B for ; Mon, 28 Oct 2019 12:52:21 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 57D0420659 for ; Mon, 28 Oct 2019 12:52:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QDjdQk4n" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 57D0420659 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53420 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4VQ-0003lF-1i for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:52:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50650) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wk-0002bl-Nk for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wj-0001sg-GV for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:30 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:57595 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wj-0001sN-Bn for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264988; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UXPmQau9KjGn9ZwtheUphXzz220rJhcKYzxel56lcL0=; b=QDjdQk4nbYcy+h3P5SpB+HJVakrYYBCbSyKpcaPkBiNoztoRiLnCOgNPub63LyUZAf/J+T lJOt+W6/ho5uw//5zFvZLb5zyth5bw5xxonS1YeedSt3q+uOhlkQ2bRo13rMN6c/t8c5u3 y8T/sV6TVNmQk98JpzThK55nIJogKFE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-166-HynnLaVeOwup4IXkt7awfQ-1; Mon, 28 Oct 2019 08:16:27 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 32077180496E; Mon, 28 Oct 2019 12:16:26 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C868660C4B; Mon, 28 Oct 2019 12:16:25 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 40/69] block/block-copy: add memory limit Date: Mon, 28 Oct 2019 13:14:32 +0100 Message-Id: <20191028121501.15279-41-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: HynnLaVeOwup4IXkt7awfQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Currently total allocation for parallel requests to block-copy instance is unlimited. Let's limit it to 128 MiB. For now block-copy is used only in backup, so actually we limit total allocation for backup job. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20191022111805.3432-6-vsementsov@virtuozzo.com Signed-off-by: Max Reitz --- include/block/block-copy.h | 3 +++ block/block-copy.c | 5 +++++ 2 files changed, 8 insertions(+) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index e2e135ff1b..edcdf0072d 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -16,6 +16,7 @@ #define BLOCK_COPY_H #include "block/block.h" +#include "qemu/co-shared-resource.h" typedef struct BlockCopyInFlightReq { int64_t start_byte; @@ -69,6 +70,8 @@ typedef struct BlockCopyState { */ ProgressResetCallbackFunc progress_reset_callback; void *progress_opaque; + + SharedResource *mem; } BlockCopyState; BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, diff --git a/block/block-copy.c b/block/block-copy.c index 845b2423dc..7f0ebb58f8 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -21,6 +21,7 @@ #include "qemu/units.h" #define BLOCK_COPY_MAX_COPY_RANGE (16 * MiB) +#define BLOCK_COPY_MAX_MEM (128 * MiB) static void coroutine_fn block_copy_wait_inflight_reqs(BlockCopyState *s, int64_t start, @@ -64,6 +65,7 @@ void block_copy_state_free(BlockCopyState *s) } bdrv_release_dirty_bitmap(s->copy_bitmap); + shres_destroy(s->mem); g_free(s); } @@ -95,6 +97,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, .cluster_size = cluster_size, .len = bdrv_dirty_bitmap_size(copy_bitmap), .write_flags = write_flags, + .mem = shres_create(BLOCK_COPY_MAX_MEM), }; s->copy_range_size = QEMU_ALIGN_DOWN(max_transfer, cluster_size), @@ -313,7 +316,9 @@ int coroutine_fn block_copy(BlockCopyState *s, bdrv_reset_dirty_bitmap(s->copy_bitmap, start, chunk_end - start); + co_get_from_shres(s->mem, chunk_end - start); ret = block_copy_do_copy(s, start, chunk_end, error_is_read); + co_put_to_shres(s->mem, chunk_end - start); if (ret < 0) { bdrv_set_dirty_bitmap(s->copy_bitmap, start, chunk_end - start); break; From patchwork Mon Oct 28 12:14:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215467 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1D2BE112B for ; Mon, 28 Oct 2019 12:59:56 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E709620873 for ; Mon, 28 Oct 2019 12:59:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="TDzRW9K6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E709620873 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53492 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4ck-0005rK-P2 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:59:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50698) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wo-0002iM-Jc for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wn-0001uW-4A for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:34 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:22129 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wm-0001uG-U6 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264992; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c6OTm5ZJcB6kowqx+WMPInET1NNXiqV4DJc8qPb1V0s=; b=TDzRW9K6a7KV62vyR3tUTlktlsL1o9MTGxR7MkNI0YnKODH9mQDZePvMZx6cUc57M/mn/3 y8cFtDyTa5nNcIAZSQocbD1emIPM6sAPiDi7FvxhYrYiUf5M6vddc7/MzhqtmYWNvhGbhW 5e5ds3w3vzy7tg5k6COXimIXoTqR8tM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-394-V5pyRu0cP1mBu2iZCSaz9A-1; Mon, 28 Oct 2019 08:16:29 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 332C380183E; Mon, 28 Oct 2019 12:16:28 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CEAB726199; Mon, 28 Oct 2019 12:16:27 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 41/69] block/block-copy: increase buffered copy request Date: Mon, 28 Oct 2019 13:14:33 +0100 Message-Id: <20191028121501.15279-42-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: V5pyRu0cP1mBu2iZCSaz9A-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy No reason to limit buffered copy to one cluster. Let's allow up to 1 MiB. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20191022111805.3432-7-vsementsov@virtuozzo.com Signed-off-by: Max Reitz --- include/block/block-copy.h | 2 +- block/block-copy.c | 48 +++++++++++++++++++++++++------------- 2 files changed, 33 insertions(+), 17 deletions(-) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index edcdf0072d..0a161724d7 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -38,7 +38,7 @@ typedef struct BlockCopyState { BdrvDirtyBitmap *copy_bitmap; int64_t cluster_size; bool use_copy_range; - int64_t copy_range_size; + int64_t copy_size; uint64_t len; QLIST_HEAD(, BlockCopyInFlightReq) inflight_reqs; diff --git a/block/block-copy.c b/block/block-copy.c index 7f0ebb58f8..c39cc9cffe 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -21,6 +21,7 @@ #include "qemu/units.h" #define BLOCK_COPY_MAX_COPY_RANGE (16 * MiB) +#define BLOCK_COPY_MAX_BUFFER (1 * MiB) #define BLOCK_COPY_MAX_MEM (128 * MiB) static void coroutine_fn block_copy_wait_inflight_reqs(BlockCopyState *s, @@ -75,10 +76,8 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, { BlockCopyState *s; BdrvDirtyBitmap *copy_bitmap; - - /* Ignore BLOCK_COPY_MAX_COPY_RANGE if requested cluster_size is larger */ uint32_t max_transfer = - MIN_NON_ZERO(MAX(cluster_size, BLOCK_COPY_MAX_COPY_RANGE), + MIN_NON_ZERO(INT_MAX, MIN_NON_ZERO(source->bs->bl.max_transfer, target->bs->bl.max_transfer)); @@ -100,17 +99,28 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, .mem = shres_create(BLOCK_COPY_MAX_MEM), }; - s->copy_range_size = QEMU_ALIGN_DOWN(max_transfer, cluster_size), - /* - * Set use_copy_range, consider the following: - * 1. Compression is not supported for copy_range. - * 2. copy_range does not respect max_transfer (it's a TODO), so we factor - * that in here. If max_transfer is smaller than the job->cluster_size, - * we do not use copy_range (in that case it's zero after aligning down - * above). - */ - s->use_copy_range = - !(write_flags & BDRV_REQ_WRITE_COMPRESSED) && s->copy_range_size > 0; + if (max_transfer < cluster_size) { + /* + * copy_range does not respect max_transfer. We don't want to bother + * with requests smaller than block-copy cluster size, so fallback to + * buffered copying (read and write respect max_transfer on their + * behalf). + */ + s->use_copy_range = false; + s->copy_size = cluster_size; + } else if (write_flags & BDRV_REQ_WRITE_COMPRESSED) { + /* Compression is not supported for copy_range */ + s->use_copy_range = false; + s->copy_size = MAX(cluster_size, BLOCK_COPY_MAX_BUFFER); + } else { + /* + * copy_range does not respect max_transfer (it's a TODO), so we factor + * that in here. + */ + s->use_copy_range = true; + s->copy_size = MIN(MAX(cluster_size, BLOCK_COPY_MAX_COPY_RANGE), + QEMU_ALIGN_DOWN(max_transfer, cluster_size)); + } QLIST_INIT(&s->inflight_reqs); @@ -156,12 +166,19 @@ static int coroutine_fn block_copy_do_copy(BlockCopyState *s, if (ret < 0) { trace_block_copy_copy_range_fail(s, start, ret); s->use_copy_range = false; + s->copy_size = MAX(s->cluster_size, BLOCK_COPY_MAX_BUFFER); /* Fallback to read+write with allocated buffer */ } else { goto out; } } + /* + * In case of failed copy_range request above, we may proceed with buffered + * request larger than BLOCK_COPY_MAX_BUFFER. Still, further requests will + * be properly limited, so don't care too much. + */ + bounce_buffer = qemu_blockalign(s->source->bs, nbytes); ret = bdrv_co_pread(s->source, start, nbytes, bounce_buffer, 0); @@ -290,8 +307,7 @@ int coroutine_fn block_copy(BlockCopyState *s, continue; /* already copied */ } - chunk_end = MIN(end, start + (s->use_copy_range ? - s->copy_range_size : s->cluster_size)); + chunk_end = MIN(end, start + s->copy_size); next_zero = bdrv_dirty_bitmap_next_zero(s->copy_bitmap, start, chunk_end - start); From patchwork Mon Oct 28 12:14:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215463 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A79B112C for ; Mon, 28 Oct 2019 12:58:34 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1036020873 for ; Mon, 28 Oct 2019 12:58:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hxgoSgZd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1036020873 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53470 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4bQ-00027Q-H3 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:58:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50724) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3wt-0002lY-V1 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wq-0001wX-Jj for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:38 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:26956 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wq-0001vn-FR for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264995; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MxBajAlKda7hh+AywwwSzXruhU7iGlOEpLawksUo4Uo=; b=hxgoSgZdLgWxJG5uNFsM5FaAToGPTRmDdtd5FH+EyI9Gf12E2ByQdMfcoQUfEusDox2s+s qtNiXzz08jqtdTlZf3Z3aooU2YOKQNbN6CTNOfXi8vgO5QbG3PsGCkHY82MgSihXbWDDQ/ mWJC/LRMl+Mphv1WfXyQdh5+3VSc0TI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-80-49kPFR0pOOi9lILjD0HoGw-1; Mon, 28 Oct 2019 08:16:31 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 60C00800FF1; Mon, 28 Oct 2019 12:16:30 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CFF8E60BF7; Mon, 28 Oct 2019 12:16:29 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 42/69] block/nvme: add support for write zeros Date: Mon, 28 Oct 2019 13:14:34 +0100 Message-Id: <20191028121501.15279-43-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: 49kPFR0pOOi9lILjD0HoGw-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Maxim Levitsky Signed-off-by: Maxim Levitsky Message-id: 20190913133627.28450-2-mlevitsk@redhat.com Reviewed-by: John Snow Signed-off-by: Max Reitz --- include/block/nvme.h | 19 +++++++++++- block/nvme.c | 72 +++++++++++++++++++++++++++++++++++++++++++- block/trace-events | 1 + 3 files changed, 90 insertions(+), 2 deletions(-) diff --git a/include/block/nvme.h b/include/block/nvme.h index 3ec8efcc43..ab5943b90a 100644 --- a/include/block/nvme.h +++ b/include/block/nvme.h @@ -653,12 +653,29 @@ typedef struct NvmeIdNs { uint8_t mc; uint8_t dpc; uint8_t dps; - uint8_t res30[98]; + + uint8_t nmic; + uint8_t rescap; + uint8_t fpi; + uint8_t dlfeat; + + uint8_t res34[94]; NvmeLBAF lbaf[16]; uint8_t res192[192]; uint8_t vs[3712]; } NvmeIdNs; + +/*Deallocate Logical Block Features*/ +#define NVME_ID_NS_DLFEAT_GUARD_CRC(dlfeat) ((dlfeat) & 0x10) +#define NVME_ID_NS_DLFEAT_WRITE_ZEROES(dlfeat) ((dlfeat) & 0x08) + +#define NVME_ID_NS_DLFEAT_READ_BEHAVIOR(dlfeat) ((dlfeat) & 0x7) +#define NVME_ID_NS_DLFEAT_READ_BEHAVIOR_UNDEFINED 0 +#define NVME_ID_NS_DLFEAT_READ_BEHAVIOR_ZEROES 1 +#define NVME_ID_NS_DLFEAT_READ_BEHAVIOR_ONES 2 + + #define NVME_ID_NS_NSFEAT_THIN(nsfeat) ((nsfeat & 0x1)) #define NVME_ID_NS_FLBAS_EXTENDED(flbas) ((flbas >> 4) & 0x1) #define NVME_ID_NS_FLBAS_INDEX(flbas) ((flbas & 0xf)) diff --git a/block/nvme.c b/block/nvme.c index 910872ec59..0ac38c24e6 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -112,6 +112,8 @@ typedef struct { uint64_t max_transfer; bool plugged; + bool supports_write_zeroes; + CoMutex dma_map_lock; CoQueue dma_flush_queue; @@ -423,6 +425,7 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp) NvmeIdNs *idns; NvmeLBAF *lbaf; uint8_t *resp; + uint16_t oncs; int r; uint64_t iova; NvmeCmd cmd = { @@ -460,6 +463,9 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp) s->max_transfer = MIN_NON_ZERO(s->max_transfer, s->page_size / sizeof(uint64_t) * s->page_size); + oncs = le16_to_cpu(idctrl->oncs); + s->supports_write_zeroes = !!(oncs & NVME_ONCS_WRITE_ZEROS); + memset(resp, 0, 4096); cmd.cdw10 = 0; @@ -472,6 +478,12 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp) s->nsze = le64_to_cpu(idns->nsze); lbaf = &idns->lbaf[NVME_ID_NS_FLBAS_INDEX(idns->flbas)]; + if (NVME_ID_NS_DLFEAT_WRITE_ZEROES(idns->dlfeat) && + NVME_ID_NS_DLFEAT_READ_BEHAVIOR(idns->dlfeat) == + NVME_ID_NS_DLFEAT_READ_BEHAVIOR_ZEROES) { + bs->supported_write_flags |= BDRV_REQ_MAY_UNMAP; + } + if (lbaf->ms) { error_setg(errp, "Namespaces with metadata are not yet supported"); goto out; @@ -766,6 +778,8 @@ static int nvme_file_open(BlockDriverState *bs, QDict *options, int flags, int ret; BDRVNVMeState *s = bs->opaque; + bs->supported_write_flags = BDRV_REQ_FUA; + opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort); qemu_opts_absorb_qdict(opts, options, &error_abort); device = qemu_opt_get(opts, NVME_BLOCK_OPT_DEVICE); @@ -794,7 +808,6 @@ static int nvme_file_open(BlockDriverState *bs, QDict *options, int flags, goto fail; } } - bs->supported_write_flags = BDRV_REQ_FUA; return 0; fail: nvme_close(bs); @@ -1088,6 +1101,60 @@ static coroutine_fn int nvme_co_flush(BlockDriverState *bs) } +static coroutine_fn int nvme_co_pwrite_zeroes(BlockDriverState *bs, + int64_t offset, + int bytes, + BdrvRequestFlags flags) +{ + BDRVNVMeState *s = bs->opaque; + NVMeQueuePair *ioq = s->queues[1]; + NVMeRequest *req; + + uint32_t cdw12 = ((bytes >> s->blkshift) - 1) & 0xFFFF; + + if (!s->supports_write_zeroes) { + return -ENOTSUP; + } + + NvmeCmd cmd = { + .opcode = NVME_CMD_WRITE_ZEROS, + .nsid = cpu_to_le32(s->nsid), + .cdw10 = cpu_to_le32((offset >> s->blkshift) & 0xFFFFFFFF), + .cdw11 = cpu_to_le32(((offset >> s->blkshift) >> 32) & 0xFFFFFFFF), + }; + + NVMeCoData data = { + .ctx = bdrv_get_aio_context(bs), + .ret = -EINPROGRESS, + }; + + if (flags & BDRV_REQ_MAY_UNMAP) { + cdw12 |= (1 << 25); + } + + if (flags & BDRV_REQ_FUA) { + cdw12 |= (1 << 30); + } + + cmd.cdw12 = cpu_to_le32(cdw12); + + trace_nvme_write_zeroes(s, offset, bytes, flags); + assert(s->nr_queues > 1); + req = nvme_get_free_req(ioq); + assert(req); + + nvme_submit_command(s, ioq, req, &cmd, nvme_rw_cb, &data); + + data.co = qemu_coroutine_self(); + while (data.ret == -EINPROGRESS) { + qemu_coroutine_yield(); + } + + trace_nvme_rw_done(s, true, offset, bytes, data.ret); + return data.ret; +} + + static int nvme_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue *queue, Error **errp) { @@ -1192,6 +1259,9 @@ static BlockDriver bdrv_nvme = { .bdrv_co_preadv = nvme_co_preadv, .bdrv_co_pwritev = nvme_co_pwritev, + + .bdrv_co_pwrite_zeroes = nvme_co_pwrite_zeroes, + .bdrv_co_flush_to_disk = nvme_co_flush, .bdrv_reopen_prepare = nvme_reopen_prepare, diff --git a/block/trace-events b/block/trace-events index ccde15a14c..8706eab170 100644 --- a/block/trace-events +++ b/block/trace-events @@ -152,6 +152,7 @@ nvme_submit_command_raw(int c0, int c1, int c2, int c3, int c4, int c5, int c6, nvme_handle_event(void *s) "s %p" nvme_poll_cb(void *s) "s %p" nvme_prw_aligned(void *s, int is_write, uint64_t offset, uint64_t bytes, int flags, int niov) "s %p is_write %d offset %"PRId64" bytes %"PRId64" flags %d niov %d" +nvme_write_zeroes(void *s, uint64_t offset, uint64_t bytes, int flags) "s %p offset %"PRId64" bytes %"PRId64" flags %d" nvme_qiov_unaligned(const void *qiov, int n, void *base, size_t size, int align) "qiov %p n %d base %p size 0x%zx align 0x%x" nvme_prw_buffered(void *s, uint64_t offset, uint64_t bytes, int niov, int is_write) "s %p offset %"PRId64" bytes %"PRId64" niov %d is_write %d" nvme_rw_done(void *s, int is_write, uint64_t offset, uint64_t bytes, int ret) "s %p is_write %d offset %"PRId64" bytes %"PRId64" ret %d" From patchwork Mon Oct 28 12:14:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215493 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 01F231599 for ; Mon, 28 Oct 2019 13:10:05 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C01CE214D9 for ; Mon, 28 Oct 2019 13:10:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Jj/IV3iG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C01CE214D9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53582 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4mZ-0005rV-98 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:10:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50747) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3ww-0002mk-Iy for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wt-0001xR-0k for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:40 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:56555 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wr-0001wj-Lf for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264997; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vEa8sQ3p2Gih5MTY0vLHkOBrDz/zYHejSw4/57eoWCA=; b=Jj/IV3iGtNNCr3kX4BjuXXwU7D3U2VqAzx7Om18RUzLsod4U0z3ho3fauhc05n2+6NA1ng SgNGCr9oQPWxWymqQsqVuwoi81QG7W5j7i1DK2YrB2zCXuZ/9lDvL4zTs0piT/8D++D4qm RRvRba/eWTomLcHiUzlwSPLzor5McW4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-329-VFcexWwiPoaZDROsngkKUg-1; Mon, 28 Oct 2019 08:16:33 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5EF46107AD28; Mon, 28 Oct 2019 12:16:32 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 05B2560BF7; Mon, 28 Oct 2019 12:16:31 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 43/69] block/nvme: add support for discard Date: Mon, 28 Oct 2019 13:14:35 +0100 Message-Id: <20191028121501.15279-44-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: VFcexWwiPoaZDROsngkKUg-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Maxim Levitsky Signed-off-by: Maxim Levitsky Message-id: 20190913133627.28450-3-mlevitsk@redhat.com Reviewed-by: John Snow Signed-off-by: Max Reitz --- block/nvme.c | 83 ++++++++++++++++++++++++++++++++++++++++++++++ block/trace-events | 2 ++ 2 files changed, 85 insertions(+) diff --git a/block/nvme.c b/block/nvme.c index 0ac38c24e6..d41c4bda6e 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -113,6 +113,7 @@ typedef struct { bool plugged; bool supports_write_zeroes; + bool supports_discard; CoMutex dma_map_lock; CoQueue dma_flush_queue; @@ -465,6 +466,7 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp) oncs = le16_to_cpu(idctrl->oncs); s->supports_write_zeroes = !!(oncs & NVME_ONCS_WRITE_ZEROS); + s->supports_discard = !!(oncs & NVME_ONCS_DSM); memset(resp, 0, 4096); @@ -1155,6 +1157,86 @@ static coroutine_fn int nvme_co_pwrite_zeroes(BlockDriverState *bs, } +static int coroutine_fn nvme_co_pdiscard(BlockDriverState *bs, + int64_t offset, + int bytes) +{ + BDRVNVMeState *s = bs->opaque; + NVMeQueuePair *ioq = s->queues[1]; + NVMeRequest *req; + NvmeDsmRange *buf; + QEMUIOVector local_qiov; + int ret; + + NvmeCmd cmd = { + .opcode = NVME_CMD_DSM, + .nsid = cpu_to_le32(s->nsid), + .cdw10 = cpu_to_le32(0), /*number of ranges - 0 based*/ + .cdw11 = cpu_to_le32(1 << 2), /*deallocate bit*/ + }; + + NVMeCoData data = { + .ctx = bdrv_get_aio_context(bs), + .ret = -EINPROGRESS, + }; + + if (!s->supports_discard) { + return -ENOTSUP; + } + + assert(s->nr_queues > 1); + + buf = qemu_try_blockalign0(bs, s->page_size); + if (!buf) { + return -ENOMEM; + } + + buf->nlb = cpu_to_le32(bytes >> s->blkshift); + buf->slba = cpu_to_le64(offset >> s->blkshift); + buf->cattr = 0; + + qemu_iovec_init(&local_qiov, 1); + qemu_iovec_add(&local_qiov, buf, 4096); + + req = nvme_get_free_req(ioq); + assert(req); + + qemu_co_mutex_lock(&s->dma_map_lock); + ret = nvme_cmd_map_qiov(bs, &cmd, req, &local_qiov); + qemu_co_mutex_unlock(&s->dma_map_lock); + + if (ret) { + req->busy = false; + goto out; + } + + trace_nvme_dsm(s, offset, bytes); + + nvme_submit_command(s, ioq, req, &cmd, nvme_rw_cb, &data); + + data.co = qemu_coroutine_self(); + while (data.ret == -EINPROGRESS) { + qemu_coroutine_yield(); + } + + qemu_co_mutex_lock(&s->dma_map_lock); + ret = nvme_cmd_unmap_qiov(bs, &local_qiov); + qemu_co_mutex_unlock(&s->dma_map_lock); + + if (ret) { + goto out; + } + + ret = data.ret; + trace_nvme_dsm_done(s, offset, bytes, ret); +out: + qemu_iovec_destroy(&local_qiov); + qemu_vfree(buf); + return ret; + +} + + static int nvme_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue *queue, Error **errp) { @@ -1261,6 +1343,7 @@ static BlockDriver bdrv_nvme = { .bdrv_co_pwritev = nvme_co_pwritev, .bdrv_co_pwrite_zeroes = nvme_co_pwrite_zeroes, + .bdrv_co_pdiscard = nvme_co_pdiscard, .bdrv_co_flush_to_disk = nvme_co_flush, .bdrv_reopen_prepare = nvme_reopen_prepare, diff --git a/block/trace-events b/block/trace-events index 8706eab170..6ba86decca 100644 --- a/block/trace-events +++ b/block/trace-events @@ -156,6 +156,8 @@ nvme_write_zeroes(void *s, uint64_t offset, uint64_t bytes, int flags) "s %p off nvme_qiov_unaligned(const void *qiov, int n, void *base, size_t size, int align) "qiov %p n %d base %p size 0x%zx align 0x%x" nvme_prw_buffered(void *s, uint64_t offset, uint64_t bytes, int niov, int is_write) "s %p offset %"PRId64" bytes %"PRId64" niov %d is_write %d" nvme_rw_done(void *s, int is_write, uint64_t offset, uint64_t bytes, int ret) "s %p is_write %d offset %"PRId64" bytes %"PRId64" ret %d" +nvme_dsm(void *s, uint64_t offset, uint64_t bytes) "s %p offset %"PRId64" bytes %"PRId64"" +nvme_dsm_done(void *s, uint64_t offset, uint64_t bytes, int ret) "s %p offset %"PRId64" bytes %"PRId64" ret %d" nvme_dma_map_flush(void *s) "s %p" nvme_free_req_queue_wait(void *q) "q %p" nvme_cmd_map_qiov(void *s, void *cmd, void *req, void *qiov, int entries) "s %p cmd %p req %p qiov %p entries %d" From patchwork Mon Oct 28 12:14:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215473 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 012DD112B for ; Mon, 28 Oct 2019 13:03:03 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 87E6920717 for ; Mon, 28 Oct 2019 13:03:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="A8TRoQTf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 87E6920717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53512 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4fi-0003N0-Ic for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:03:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50790) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3x2-0002tI-EB for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wx-0001zi-Hn for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:45 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:21801 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wu-0001xV-6Y for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264999; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rDRhJaImYLPxs5FqXhGCslMsXlMubC4w5oQ2JKHYQkw=; b=A8TRoQTfhgMttdZRwJFY4/9r1mpD4Ed4BHa8ZUo3Kxr+9TSAY8SCbQtID+RdZl2qy3SClD Ae0eFNUyAaw1YRufi3yChjDFj1PMDJzYtU3bGerNdQUzerGmUmF3T6YEXHJLNx8HM6b13e nT9Agzb1Fa8GgeaNXoxupkAv4myZM0w= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-387-G6eGbPJqNc6_PWx_pvxb0A-1; Mon, 28 Oct 2019 08:16:35 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 69D98180496E; Mon, 28 Oct 2019 12:16:34 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0F0EB5D6BB; Mon, 28 Oct 2019 12:16:33 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 44/69] mirror: Do not dereference invalid pointers Date: Mon, 28 Oct 2019 13:14:36 +0100 Message-Id: <20191028121501.15279-45-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-MC-Unique: G6eGbPJqNc6_PWx_pvxb0A-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" mirror_exit_common() may be called twice (if it is called from mirror_prepare() and fails, it will be called from mirror_abort() again). In such a case, many of the pointers in the MirrorBlockJob object will already be freed. This can be seen most reliably for s->target, which is set to NULL (and then dereferenced by blk_bs()). Cc: qemu-stable@nongnu.org Fixes: 737efc1eda23b904fbe0e66b37715fb0e5c3e58b Signed-off-by: Max Reitz Reviewed-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Message-id: 20191014153931.20699-2-mreitz@redhat.com Signed-off-by: Max Reitz --- block/mirror.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 454365ce00..bb17cfce31 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -620,11 +620,11 @@ static int mirror_exit_common(Job *job) { MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job); BlockJob *bjob = &s->common; - MirrorBDSOpaque *bs_opaque = s->mirror_top_bs->opaque; + MirrorBDSOpaque *bs_opaque; AioContext *replace_aio_context = NULL; - BlockDriverState *src = s->mirror_top_bs->backing->bs; - BlockDriverState *target_bs = blk_bs(s->target); - BlockDriverState *mirror_top_bs = s->mirror_top_bs; + BlockDriverState *src; + BlockDriverState *target_bs; + BlockDriverState *mirror_top_bs; Error *local_err = NULL; bool abort = job->ret < 0; int ret = 0; @@ -634,6 +634,11 @@ static int mirror_exit_common(Job *job) } s->prepared = true; + mirror_top_bs = s->mirror_top_bs; + bs_opaque = mirror_top_bs->opaque; + src = mirror_top_bs->backing->bs; + target_bs = blk_bs(s->target); + if (bdrv_chain_contains(src, target_bs)) { bdrv_unfreeze_backing_chain(mirror_top_bs, target_bs); } From patchwork Mon Oct 28 12:14:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215427 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 64778112C for ; Mon, 28 Oct 2019 12:47:43 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3AE42204FD for ; Mon, 28 Oct 2019 12:47:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DHfJZgVO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3AE42204FD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53378 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Qv-000504-Lr for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:47:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50797) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3x4-0002uO-HQ for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3wy-00020D-1n for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:46 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:38235 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wx-0001xa-Hc for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572264999; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+KCmWSdKj/kmBW5vvX21H2fiPUW+Ik3EEjUw39XtQIM=; b=DHfJZgVOwGFHZt1fcDbBs0KA5kmKjIjEfoVAasyCni7gyBeWsnVLinNKNkzzwa2CqtZw4+ 9agRFzcWb05ljmvcLubyqbES60zWoxUtzAfoJNOiQ7+FfHyCZ5JjRVcIPJ7ITfn4wMErBY xRU/1Jfw3lyQd0ZN/vkCV+3FrJNKuoQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-29-OoZQ5jusMN-3m-eNQdhqUQ-1; Mon, 28 Oct 2019 08:16:37 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7D6C2107AD28; Mon, 28 Oct 2019 12:16:36 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2184E1001902; Mon, 28 Oct 2019 12:16:35 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 45/69] include: Move endof() up from hw/virtio/virtio.h Date: Mon, 28 Oct 2019 13:14:37 +0100 Message-Id: <20191028121501.15279-46-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: OoZQ5jusMN-3m-eNQdhqUQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" endof() is a useful macro, we can make use of it outside of virtio. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-2-mreitz@redhat.com Signed-off-by: Max Reitz --- include/hw/virtio/virtio.h | 7 ------- include/qemu/compiler.h | 7 +++++++ hw/block/virtio-blk.c | 4 ++-- hw/net/virtio-net.c | 10 +++++----- 4 files changed, 14 insertions(+), 14 deletions(-) diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index 48e8d04ff6..ef083af550 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -37,13 +37,6 @@ static inline hwaddr vring_align(hwaddr addr, return QEMU_ALIGN_UP(addr, align); } -/* - * Calculate the number of bytes up to and including the given 'field' of - * 'container'. - */ -#define virtio_endof(container, field) \ - (offsetof(container, field) + sizeof_field(container, field)) - typedef struct VirtIOFeature { uint64_t flags; size_t end; diff --git a/include/qemu/compiler.h b/include/qemu/compiler.h index 7b93c73340..85c02c16d3 100644 --- a/include/qemu/compiler.h +++ b/include/qemu/compiler.h @@ -60,6 +60,13 @@ #define sizeof_field(type, field) sizeof(((type *)0)->field) +/* + * Calculate the number of bytes up to and including the given 'field' of + * 'container'. + */ +#define endof(container, field) \ + (offsetof(container, field) + sizeof_field(container, field)) + /* Convert from a base type to a parent type, with compile time checking. */ #ifdef __GNUC__ #define DO_UPCAST(type, field, dev) ( __extension__ ( { \ diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index 14e9f85b8b..156416dfd5 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -42,9 +42,9 @@ */ static VirtIOFeature feature_sizes[] = { {.flags = 1ULL << VIRTIO_BLK_F_DISCARD, - .end = virtio_endof(struct virtio_blk_config, discard_sector_alignment)}, + .end = endof(struct virtio_blk_config, discard_sector_alignment)}, {.flags = 1ULL << VIRTIO_BLK_F_WRITE_ZEROES, - .end = virtio_endof(struct virtio_blk_config, write_zeroes_may_unmap)}, + .end = endof(struct virtio_blk_config, write_zeroes_may_unmap)}, {} }; diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index 9f11422337..2c4909c5f9 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -90,15 +90,15 @@ static inline __virtio16 *virtio_net_rsc_ext_num_dupacks( static VirtIOFeature feature_sizes[] = { {.flags = 1ULL << VIRTIO_NET_F_MAC, - .end = virtio_endof(struct virtio_net_config, mac)}, + .end = endof(struct virtio_net_config, mac)}, {.flags = 1ULL << VIRTIO_NET_F_STATUS, - .end = virtio_endof(struct virtio_net_config, status)}, + .end = endof(struct virtio_net_config, status)}, {.flags = 1ULL << VIRTIO_NET_F_MQ, - .end = virtio_endof(struct virtio_net_config, max_virtqueue_pairs)}, + .end = endof(struct virtio_net_config, max_virtqueue_pairs)}, {.flags = 1ULL << VIRTIO_NET_F_MTU, - .end = virtio_endof(struct virtio_net_config, mtu)}, + .end = endof(struct virtio_net_config, mtu)}, {.flags = 1ULL << VIRTIO_NET_F_SPEED_DUPLEX, - .end = virtio_endof(struct virtio_net_config, duplex)}, + .end = endof(struct virtio_net_config, duplex)}, {} }; From patchwork Mon Oct 28 12:14:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215499 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D3FB41599 for ; Mon, 28 Oct 2019 13:14:44 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AA09120717 for ; Mon, 28 Oct 2019 13:14:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Kwg1/Zze" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA09120717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53628 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4r5-0004JQ-4W for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:14:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50852) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3x9-00039i-Tn for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3x4-00023N-Ea for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:53 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:44039 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3wz-0001zx-BQ for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265003; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cfzDy2KOnVpKUUBJ6sbP5TwiIljc61cRVIU05TJ6Nzo=; b=Kwg1/ZzequqmGweGqhEvyqFezooPf24qToLsysaF3MU2e3XqDHn+2t82qyRTRb07cJNSbK 9YVeS3nSEWdY0MUAkKz1Uq25VcXwgq5yHWwYUXf20WeatjTDTvaQNYbLireAbbfg6CYPX+ FBlDf+j6ugSel8HxaEq/aLyTZzh+Yvc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-197-gzAlgX6tPyiLnt8sFGTySw-1; Mon, 28 Oct 2019 08:16:39 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 86BD3107AD29; Mon, 28 Oct 2019 12:16:38 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2983360850; Mon, 28 Oct 2019 12:16:38 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 46/69] qcow2: Use endof() Date: Mon, 28 Oct 2019 13:14:38 +0100 Message-Id: <20191028121501.15279-47-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: gzAlgX6tPyiLnt8sFGTySw-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-3-mreitz@redhat.com Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index d0e7fa9311..752883e5c3 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -92,11 +92,12 @@ int qcow2_read_snapshots(BlockDriverState *bs) } offset += extra_data_size; - if (extra_data_size >= 8) { + if (extra_data_size >= endof(QCowSnapshotExtraData, + vm_state_size_large)) { sn->vm_state_size = be64_to_cpu(extra.vm_state_size_large); } - if (extra_data_size >= 16) { + if (extra_data_size >= endof(QCowSnapshotExtraData, disk_size)) { sn->disk_size = be64_to_cpu(extra.disk_size); } else { sn->disk_size = bs->total_sectors * BDRV_SECTOR_SIZE; @@ -251,7 +252,7 @@ static int qcow2_write_snapshots(BlockDriverState *bs) } QEMU_BUILD_BUG_ON(offsetof(QCowHeader, snapshots_offset) != - offsetof(QCowHeader, nb_snapshots) + sizeof(header_data.nb_snapshots)); + endof(QCowHeader, nb_snapshots)); header_data.nb_snapshots = cpu_to_be32(s->nb_snapshots); header_data.snapshots_offset = cpu_to_be64(snapshots_offset); From patchwork Mon Oct 28 12:14:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215487 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B6AA814E5 for ; Mon, 28 Oct 2019 13:07:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8CA2B20717 for ; Mon, 28 Oct 2019 13:07:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MDOW+/Ny" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8CA2B20717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53550 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4kA-0001p4-Jh for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:07:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50860) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xA-0003B8-CD for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3x4-00023i-JC for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:53 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:42430 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3x4-00021V-DR for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265006; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/YCjMGfIw2K+Gj+nqzvEHNscCa92sIB53x/RktvvZBI=; b=MDOW+/NyVticeaPMGROMNXgkZPB46V3g9SZaF3DlsKtA19QH79i4L5LzZAp2vuXZT9DiEH OsvsQH2rG0zqHQETHmubOQtr698kOYLmQ7Jqjz8rD0QLc+XpWcJBe1RuQYiPr7TaiiDnaU PvKrjT4CwE+y+QIc5Wf5joqz7ZTWk3Y= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-267-dMhaFpzGO_eHyfkC4KQhqA-1; Mon, 28 Oct 2019 08:16:44 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8DBD980183E; Mon, 28 Oct 2019 12:16:40 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3148360BF7; Mon, 28 Oct 2019 12:16:40 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 47/69] qcow2: Add Error ** to qcow2_read_snapshots() Date: Mon, 28 Oct 2019 13:14:39 +0100 Message-Id: <20191028121501.15279-48-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: dMhaFpzGO_eHyfkC4KQhqA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-4-mreitz@redhat.com Signed-off-by: Max Reitz --- block/qcow2.h | 2 +- block/qcow2-snapshot.c | 7 ++++++- block/qcow2.c | 3 +-- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 5cccd87162..cc0d5eadd0 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -708,7 +708,7 @@ int qcow2_snapshot_load_tmp(BlockDriverState *bs, Error **errp); void qcow2_free_snapshots(BlockDriverState *bs); -int qcow2_read_snapshots(BlockDriverState *bs); +int qcow2_read_snapshots(BlockDriverState *bs, Error **errp); /* qcow2-cache.c functions */ Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables, diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 752883e5c3..80d32e4504 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -43,7 +43,7 @@ void qcow2_free_snapshots(BlockDriverState *bs) s->nb_snapshots = 0; } -int qcow2_read_snapshots(BlockDriverState *bs) +int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) { BDRVQcow2State *s = bs->opaque; QCowSnapshotHeader h; @@ -68,6 +68,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) offset = ROUND_UP(offset, 8); ret = bdrv_pread(bs->file, offset, &h, sizeof(h)); if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to read snapshot table"); goto fail; } @@ -88,6 +89,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) ret = bdrv_pread(bs->file, offset, &extra, MIN(sizeof(extra), extra_data_size)); if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to read snapshot table"); goto fail; } offset += extra_data_size; @@ -107,6 +109,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) sn->id_str = g_malloc(id_str_size + 1); ret = bdrv_pread(bs->file, offset, sn->id_str, id_str_size); if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to read snapshot table"); goto fail; } offset += id_str_size; @@ -116,6 +119,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) sn->name = g_malloc(name_size + 1); ret = bdrv_pread(bs->file, offset, sn->name, name_size); if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to read snapshot table"); goto fail; } offset += name_size; @@ -123,6 +127,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) if (offset - s->snapshots_offset > QCOW_MAX_SNAPSHOTS_SIZE) { ret = -EFBIG; + error_setg(errp, "Snapshot table is too big"); goto fail; } } diff --git a/block/qcow2.c b/block/qcow2.c index 0bc69e6996..60ed084e5c 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1584,9 +1584,8 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, s->snapshots_offset = header.snapshots_offset; s->nb_snapshots = header.nb_snapshots; - ret = qcow2_read_snapshots(bs); + ret = qcow2_read_snapshots(bs, errp); if (ret < 0) { - error_setg_errno(errp, -ret, "Could not read snapshots"); goto fail; } From patchwork Mon Oct 28 12:14:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215495 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1E6514E5 for ; Mon, 28 Oct 2019 13:10:52 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B658320717 for ; Mon, 28 Oct 2019 13:10:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hnytL4m3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B658320717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53588 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4nL-0006rN-RU for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:10:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50859) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xA-0003B7-CK for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3x4-00023s-KC for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:53 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:57076 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3x4-00021h-DQ for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265006; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BvB63R2vBgFG+G5Cw2JrFXw2RumcbXnacdA0qJFni34=; b=hnytL4m31iAaaU1r2t6vQLEuhTfXtRGn4XJFJdbYTze7ITeLiiEnAE7wO+lT0tE2tbOhPP PXAf7+81l2FtlsBTD4KnVKrOECGVsfBFs3yi+8Du1xPzEcxQTrf4FK5RU3S3bmbNtdN1YE TNZ20WhxsdJryjtYhq3GUbx2GkF82Lo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-342-UZlp0lupNoO8J4T6lQeAzQ-1; Mon, 28 Oct 2019 08:16:44 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C1525800FF1; Mon, 28 Oct 2019 12:16:42 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3EEAE5D6BB; Mon, 28 Oct 2019 12:16:42 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 48/69] qcow2: Keep unknown extra snapshot data Date: Mon, 28 Oct 2019 13:14:40 +0100 Message-Id: <20191028121501.15279-49-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-MC-Unique: UZlp0lupNoO8J4T6lQeAzQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" The qcow2 specification says to ignore unknown extra data fields in snapshot table entries. Currently, we discard it whenever we update the image, which is a bit different from "ignore". This patch makes the qcow2 driver keep all unknown extra data fields when updating an image's snapshot table. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-5-mreitz@redhat.com [mreitz: Adjusted comments as proposed by Eric] Signed-off-by: Max Reitz --- block/qcow2.h | 7 +++++ block/qcow2-snapshot.c | 61 +++++++++++++++++++++++++++++++++++------- 2 files changed, 58 insertions(+), 10 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index cc0d5eadd0..363681cfdc 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -61,6 +61,9 @@ * space for snapshot names and IDs */ #define QCOW_MAX_SNAPSHOTS_SIZE (1024 * QCOW_MAX_SNAPSHOTS) +/* Maximum amount of extra data per snapshot table entry to accept */ +#define QCOW_MAX_SNAPSHOT_EXTRA_DATA 1024 + /* Bitmap header extension constraints */ #define QCOW2_MAX_BITMAPS 65535 #define QCOW2_MAX_BITMAP_DIRECTORY_SIZE (1024 * QCOW2_MAX_BITMAPS) @@ -181,6 +184,10 @@ typedef struct QCowSnapshot { uint32_t date_sec; uint32_t date_nsec; uint64_t vm_clock_nsec; + /* Size of all extra data, including QCowSnapshotExtraData if available */ + uint32_t extra_data_size; + /* Data beyond QCowSnapshotExtraData, if any */ + void *unknown_extra_data; } QCowSnapshot; struct Qcow2Cache; diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 80d32e4504..120cb7fa09 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -37,6 +37,7 @@ void qcow2_free_snapshots(BlockDriverState *bs) for(i = 0; i < s->nb_snapshots; i++) { g_free(s->snapshots[i].name); g_free(s->snapshots[i].id_str); + g_free(s->snapshots[i].unknown_extra_data); } g_free(s->snapshots); s->snapshots = NULL; @@ -51,7 +52,6 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) QCowSnapshot *sn; int i, id_str_size, name_size; int64_t offset; - uint32_t extra_data_size; int ret; if (!s->nb_snapshots) { @@ -80,31 +80,53 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) sn->date_sec = be32_to_cpu(h.date_sec); sn->date_nsec = be32_to_cpu(h.date_nsec); sn->vm_clock_nsec = be64_to_cpu(h.vm_clock_nsec); - extra_data_size = be32_to_cpu(h.extra_data_size); + sn->extra_data_size = be32_to_cpu(h.extra_data_size); id_str_size = be16_to_cpu(h.id_str_size); name_size = be16_to_cpu(h.name_size); - /* Read extra data */ + if (sn->extra_data_size > QCOW_MAX_SNAPSHOT_EXTRA_DATA) { + ret = -EFBIG; + error_setg(errp, "Too much extra metadata in snapshot table " + "entry %i", i); + goto fail; + } + + /* Read known extra data */ ret = bdrv_pread(bs->file, offset, &extra, - MIN(sizeof(extra), extra_data_size)); + MIN(sizeof(extra), sn->extra_data_size)); if (ret < 0) { error_setg_errno(errp, -ret, "Failed to read snapshot table"); goto fail; } - offset += extra_data_size; + offset += MIN(sizeof(extra), sn->extra_data_size); - if (extra_data_size >= endof(QCowSnapshotExtraData, - vm_state_size_large)) { + if (sn->extra_data_size >= endof(QCowSnapshotExtraData, + vm_state_size_large)) { sn->vm_state_size = be64_to_cpu(extra.vm_state_size_large); } - if (extra_data_size >= endof(QCowSnapshotExtraData, disk_size)) { + if (sn->extra_data_size >= endof(QCowSnapshotExtraData, disk_size)) { sn->disk_size = be64_to_cpu(extra.disk_size); } else { sn->disk_size = bs->total_sectors * BDRV_SECTOR_SIZE; } + if (sn->extra_data_size > sizeof(extra)) { + /* Store unknown extra data */ + size_t unknown_extra_data_size = + sn->extra_data_size - sizeof(extra); + + sn->unknown_extra_data = g_malloc(unknown_extra_data_size); + ret = bdrv_pread(bs->file, offset, sn->unknown_extra_data, + unknown_extra_data_size); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to read snapshot table"); + goto fail; + } + offset += unknown_extra_data_size; + } + /* Read snapshot ID */ sn->id_str = g_malloc(id_str_size + 1); ret = bdrv_pread(bs->file, offset, sn->id_str, id_str_size); @@ -162,7 +184,7 @@ static int qcow2_write_snapshots(BlockDriverState *bs) sn = s->snapshots + i; offset = ROUND_UP(offset, 8); offset += sizeof(h); - offset += sizeof(extra); + offset += MAX(sizeof(extra), sn->extra_data_size); offset += strlen(sn->id_str); offset += strlen(sn->name); @@ -209,7 +231,8 @@ static int qcow2_write_snapshots(BlockDriverState *bs) h.date_sec = cpu_to_be32(sn->date_sec); h.date_nsec = cpu_to_be32(sn->date_nsec); h.vm_clock_nsec = cpu_to_be64(sn->vm_clock_nsec); - h.extra_data_size = cpu_to_be32(sizeof(extra)); + h.extra_data_size = cpu_to_be32(MAX(sizeof(extra), + sn->extra_data_size)); memset(&extra, 0, sizeof(extra)); extra.vm_state_size_large = cpu_to_be64(sn->vm_state_size); @@ -234,6 +257,22 @@ static int qcow2_write_snapshots(BlockDriverState *bs) } offset += sizeof(extra); + if (sn->extra_data_size > sizeof(extra)) { + size_t unknown_extra_data_size = + sn->extra_data_size - sizeof(extra); + + /* qcow2_read_snapshots() ensures no unbounded allocation */ + assert(unknown_extra_data_size <= BDRV_REQUEST_MAX_BYTES); + assert(sn->unknown_extra_data); + + ret = bdrv_pwrite(bs->file, offset, sn->unknown_extra_data, + unknown_extra_data_size); + if (ret < 0) { + goto fail; + } + offset += unknown_extra_data_size; + } + ret = bdrv_pwrite(bs->file, offset, sn->id_str, id_str_size); if (ret < 0) { goto fail; @@ -376,6 +415,7 @@ int qcow2_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info) sn->date_sec = sn_info->date_sec; sn->date_nsec = sn_info->date_nsec; sn->vm_clock_nsec = sn_info->vm_clock_nsec; + sn->extra_data_size = sizeof(QCowSnapshotExtraData); /* Allocate the L1 table of the snapshot and copy the current one there. */ l1_table_offset = qcow2_alloc_clusters(bs, s->l1_size * sizeof(uint64_t)); @@ -647,6 +687,7 @@ int qcow2_snapshot_delete(BlockDriverState *bs, * The snapshot is now unused, clean up. If we fail after this point, we * won't recover but just leak clusters. */ + g_free(sn.unknown_extra_data); g_free(sn.id_str); g_free(sn.name); From patchwork Mon Oct 28 12:14:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215435 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7FF7D1747 for ; Mon, 28 Oct 2019 12:51:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5604920873 for ; Mon, 28 Oct 2019 12:51:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hT/R9ugS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5604920873 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53412 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Ub-000281-1c for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:51:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50853) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3x9-00039k-Tl for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3x4-00024E-N5 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:53 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:53885 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3x4-000223-Gv for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265007; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bTj7X5bks0tq/xCmZbPfLh5BopxSxIN2X7uEbFLpJCw=; b=hT/R9ugSjBPwL1badnLkJCgGStM/4hZc9wyhyOEwNJQ0W0yanQmnzebdgnPGTWzB40gJMy aFwaStLU/2QRth403PdlNLJ1cw/4UXZgpOnNXUIQg0YjMSWqfDrnfEaRnnMt+kSLMU+15/ TXkHkGnknvPQrNCC4QdK8BNInfX2TCY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-278-pPtka8DyP9OTQdJSnhdTiA-1; Mon, 28 Oct 2019 08:16:45 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D075681A334; Mon, 28 Oct 2019 12:16:44 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 774825DA60; Mon, 28 Oct 2019 12:16:44 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 49/69] qcow2: Make qcow2_write_snapshots() public Date: Mon, 28 Oct 2019 13:14:41 +0100 Message-Id: <20191028121501.15279-50-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: pPtka8DyP9OTQdJSnhdTiA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Updating the snapshot list will be useful when upgrading a v2 image to v3, so we will need to call this function in qcow2.c. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-6-mreitz@redhat.com Signed-off-by: Max Reitz --- block/qcow2.h | 1 + block/qcow2-snapshot.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/block/qcow2.h b/block/qcow2.h index 363681cfdc..6c7a23fdf6 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -716,6 +716,7 @@ int qcow2_snapshot_load_tmp(BlockDriverState *bs, void qcow2_free_snapshots(BlockDriverState *bs); int qcow2_read_snapshots(BlockDriverState *bs, Error **errp); +int qcow2_write_snapshots(BlockDriverState *bs); /* qcow2-cache.c functions */ Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables, diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 120cb7fa09..e3bf4c9776 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -164,7 +164,7 @@ fail: } /* add at the end of the file a new list of snapshots */ -static int qcow2_write_snapshots(BlockDriverState *bs) +int qcow2_write_snapshots(BlockDriverState *bs) { BDRVQcow2State *s = bs->opaque; QCowSnapshot *sn; From patchwork Mon Oct 28 12:14:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215483 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB62D1599 for ; Mon, 28 Oct 2019 13:05:59 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B1AF421734 for ; Mon, 28 Oct 2019 13:05:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bhZXFBGY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B1AF421734 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53546 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4ic-0000Lb-5l for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:05:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50943) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xH-0003IA-S2 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xB-00028w-V4 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:00 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:35096 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xB-00024r-Ok for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265011; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bJM2+9gNPSmHQaqUcxmZtG8O2VsIhH9fUg2cSb1GHbA=; b=bhZXFBGYoHpbmwADeTH9BcoyIYwCPc2LhKIZ8LagTD3R+K11fRYkaGna2JMZNDBW4W+3W3 ISZ637wu1PlJYHbeuwgOnLAJUPLvaGBDn4A5zkGnuzJz35Rqdl5I7bXdCsmz9E6/7K0lvl 53rViySIFsegP8YVwCzmwDkiZgdjp84= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-129-59498fmLPfio9I5MUyqozw-1; Mon, 28 Oct 2019 08:16:47 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DC669476; Mon, 28 Oct 2019 12:16:46 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7ED965D6BB; Mon, 28 Oct 2019 12:16:46 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 50/69] qcow2: Put qcow2_upgrade() into its own function Date: Mon, 28 Oct 2019 13:14:42 +0100 Message-Id: <20191028121501.15279-51-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-MC-Unique: 59498fmLPfio9I5MUyqozw-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" This does not make sense right now, but it will make sense once we need to do more than to just update s->qcow_version. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-7-mreitz@redhat.com Signed-off-by: Max Reitz --- block/qcow2.c | 43 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 38 insertions(+), 5 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 60ed084e5c..af08757055 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -4912,12 +4912,46 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version, return 0; } +/* + * Upgrades an image's version. While newer versions encompass all + * features of older versions, some things may have to be presented + * differently. + */ +static int qcow2_upgrade(BlockDriverState *bs, int target_version, + BlockDriverAmendStatusCB *status_cb, void *cb_opaque, + Error **errp) +{ + BDRVQcow2State *s = bs->opaque; + int current_version = s->qcow_version; + int ret; + + /* This is qcow2_upgrade(), not qcow2_downgrade() */ + assert(target_version > current_version); + + /* There are no other versions (yet) that you can upgrade to */ + assert(target_version == 3); + + status_cb(bs, 0, 1, cb_opaque); + + s->qcow_version = target_version; + ret = qcow2_update_header(bs); + if (ret < 0) { + s->qcow_version = current_version; + error_setg_errno(errp, -ret, "Failed to update the image header"); + return ret; + } + status_cb(bs, 1, 1, cb_opaque); + + return 0; +} + typedef enum Qcow2AmendOperation { /* This is the value Qcow2AmendHelperCBInfo::last_operation will be * statically initialized to so that the helper CB can discern the first * invocation from an operation change */ QCOW2_NO_OPERATION = 0, + QCOW2_UPGRADING, QCOW2_CHANGING_REFCOUNT_ORDER, QCOW2_DOWNGRADING, } Qcow2AmendOperation; @@ -5100,17 +5134,16 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, helper_cb_info = (Qcow2AmendHelperCBInfo){ .original_status_cb = status_cb, .original_cb_opaque = cb_opaque, - .total_operations = (new_version < old_version) + .total_operations = (new_version != old_version) + (s->refcount_bits != refcount_bits) }; /* Upgrade first (some features may require compat=1.1) */ if (new_version > old_version) { - s->qcow_version = new_version; - ret = qcow2_update_header(bs); + helper_cb_info.current_operation = QCOW2_UPGRADING; + ret = qcow2_upgrade(bs, new_version, &qcow2_amend_helper_cb, + &helper_cb_info, errp); if (ret < 0) { - s->qcow_version = old_version; - error_setg_errno(errp, -ret, "Failed to update the image header"); return ret; } } From patchwork Mon Oct 28 12:14:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215443 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F25B0112C for ; Mon, 28 Oct 2019 12:54:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C81D220873 for ; Mon, 28 Oct 2019 12:54:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="evb5n8qI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C81D220873 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53438 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4Xa-00008C-LE for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:54:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50942) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xH-0003I9-SC for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xB-00028T-Qb for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:00 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:29570 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xB-00025q-Jd for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265013; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S2ttQR79Z06v2c1QVZU1sPgiAUsv2sQpamGH9nr+sA4=; b=evb5n8qIPKMx1exji3Ag+u5LSIMikn6ZEDbPwtOtGkldkBF9GP6FvkPdLETyjdYzBks00X 0QkmCJigVa12vr6pIfgXrJ5OxD8hxTiEnr6RxKD65hduB4RtcWRDRxxa3t8iuc5rmDSiTN yHFMAya71ugbT7CaEw9AW0Ov+q3w9Lw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-100-M675tbriNnOgl9qXxLQR-g-1; Mon, 28 Oct 2019 08:16:49 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E0F94180496E; Mon, 28 Oct 2019 12:16:48 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 85ADC4D; Mon, 28 Oct 2019 12:16:48 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 51/69] qcow2: Write v3-compliant snapshot list on upgrade Date: Mon, 28 Oct 2019 13:14:43 +0100 Message-Id: <20191028121501.15279-52-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: M675tbriNnOgl9qXxLQR-g-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" qcow2 v3 requires every snapshot table entry to have two extra data fields: The 64-bit VM state size, and the virtual disk size. Both are optional for v2 images, so they may not be present. qcow2_upgrade() therefore should update the snapshot table to ensure all entries have these extra data fields. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1727347 Reported-by: Eric Blake Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-8-mreitz@redhat.com Signed-off-by: Max Reitz --- block/qcow2.c | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index af08757055..e3a4c44fb7 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -4922,7 +4922,9 @@ static int qcow2_upgrade(BlockDriverState *bs, int target_version, Error **errp) { BDRVQcow2State *s = bs->opaque; + bool need_snapshot_update; int current_version = s->qcow_version; + int i; int ret; /* This is qcow2_upgrade(), not qcow2_downgrade() */ @@ -4931,7 +4933,33 @@ static int qcow2_upgrade(BlockDriverState *bs, int target_version, /* There are no other versions (yet) that you can upgrade to */ assert(target_version == 3); - status_cb(bs, 0, 1, cb_opaque); + status_cb(bs, 0, 2, cb_opaque); + + /* + * In v2, snapshots do not need to have extra data. v3 requires + * the 64-bit VM state size and the virtual disk size to be + * present. + * qcow2_write_snapshots() will always write the list in the + * v3-compliant format. + */ + need_snapshot_update = false; + for (i = 0; i < s->nb_snapshots; i++) { + if (s->snapshots[i].extra_data_size < + sizeof_field(QCowSnapshotExtraData, vm_state_size_large) + + sizeof_field(QCowSnapshotExtraData, disk_size)) + { + need_snapshot_update = true; + break; + } + } + if (need_snapshot_update) { + ret = qcow2_write_snapshots(bs); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to update the snapshot table"); + return ret; + } + } + status_cb(bs, 1, 2, cb_opaque); s->qcow_version = target_version; ret = qcow2_update_header(bs); @@ -4940,7 +4968,7 @@ static int qcow2_upgrade(BlockDriverState *bs, int target_version, error_setg_errno(errp, -ret, "Failed to update the image header"); return ret; } - status_cb(bs, 1, 1, cb_opaque); + status_cb(bs, 2, 2, cb_opaque); return 0; } From patchwork Mon Oct 28 12:14:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215501 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CF1361599 for ; Mon, 28 Oct 2019 13:15:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A4A7920717 for ; Mon, 28 Oct 2019 13:15:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Mksr8Fm0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A4A7920717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53640 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4ru-0005nh-CO for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:15:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50940) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xH-0003I8-ST for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xB-00028X-Qr for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:00 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:27906 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xB-00026q-IU for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:16:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265015; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D2dynU3jJdzXFuHkYeYdBBE1XhWxu9jnbRtx2yuyH20=; b=Mksr8Fm0wedUI0wkd8XftW0+Czlr5kscZS8XwtRnm2xilxoSKPqaXMkAS3CA1UWUX80htc 9xgOXheyku15w7bNUUXzdgxBnfMiMwc6QZ2P9FViTqXbklTaev/jXF4FtaMh/FWUicza2W LIaSF35xycwHH3GWdVMYAHK6giTH9PM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-164-RTdN3phsPfiFnE5vNaRXdw-1; Mon, 28 Oct 2019 08:16:52 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1B21E10052E0; Mon, 28 Oct 2019 12:16:51 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 898315D6BB; Mon, 28 Oct 2019 12:16:50 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 52/69] qcow2: Separate qcow2_check_read_snapshot_table() Date: Mon, 28 Oct 2019 13:14:44 +0100 Message-Id: <20191028121501.15279-53-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-MC-Unique: RTdN3phsPfiFnE5vNaRXdw-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Reading the snapshot table can fail. That is a problem when we want to repair the image. Therefore, stop reading the snapshot table in qcow2_do_open() in check mode. Instead, add a new function qcow2_check_read_snapshot_table() that reads the snapshot table at a later point. In the future, we want to handle errors here and fix them. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-9-mreitz@redhat.com Signed-off-by: Max Reitz --- block/qcow2.h | 4 +++ block/qcow2-snapshot.c | 58 ++++++++++++++++++++++++++++++++ block/qcow2.c | 76 ++++++++++++++++++++++++++++++++---------- 3 files changed, 120 insertions(+), 18 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 6c7a23fdf6..adb5c3bc28 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -718,6 +718,10 @@ void qcow2_free_snapshots(BlockDriverState *bs); int qcow2_read_snapshots(BlockDriverState *bs, Error **errp); int qcow2_write_snapshots(BlockDriverState *bs); +int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, + BdrvCheckResult *result, + BdrvCheckMode fix); + /* qcow2-cache.c functions */ Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables, unsigned table_size); diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index e3bf4c9776..d667bfd935 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -322,6 +322,64 @@ fail: return ret; } +int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, + BdrvCheckResult *result, + BdrvCheckMode fix) +{ + BDRVQcow2State *s = bs->opaque; + Error *local_err = NULL; + int ret; + struct { + uint32_t nb_snapshots; + uint64_t snapshots_offset; + } QEMU_PACKED snapshot_table_pointer; + + /* qcow2_do_open() discards this information in check mode */ + ret = bdrv_pread(bs->file, offsetof(QCowHeader, nb_snapshots), + &snapshot_table_pointer, sizeof(snapshot_table_pointer)); + if (ret < 0) { + result->check_errors++; + fprintf(stderr, "ERROR failed to read the snapshot table pointer from " + "the image header: %s\n", strerror(-ret)); + return ret; + } + + s->snapshots_offset = be64_to_cpu(snapshot_table_pointer.snapshots_offset); + s->nb_snapshots = be32_to_cpu(snapshot_table_pointer.nb_snapshots); + + ret = qcow2_validate_table(bs, s->snapshots_offset, s->nb_snapshots, + sizeof(QCowSnapshotHeader), + sizeof(QCowSnapshotHeader) * QCOW_MAX_SNAPSHOTS, + "snapshot table", &local_err); + if (ret < 0) { + result->check_errors++; + error_reportf_err(local_err, "ERROR "); + + /* We did not read the snapshot table, so invalidate this information */ + s->snapshots_offset = 0; + s->nb_snapshots = 0; + + return ret; + } + + qemu_co_mutex_unlock(&s->lock); + ret = qcow2_read_snapshots(bs, &local_err); + qemu_co_mutex_lock(&s->lock); + if (ret < 0) { + result->check_errors++; + error_reportf_err(local_err, + "ERROR failed to read the snapshot table: "); + + /* We did not read the snapshot table, so invalidate this information */ + s->snapshots_offset = 0; + s->nb_snapshots = 0; + + return ret; + } + + return 0; +} + static void find_new_snapshot_id(BlockDriverState *bs, char *id_str, int id_str_size) { diff --git a/block/qcow2.c b/block/qcow2.c index e3a4c44fb7..b0026e8a9a 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -570,11 +570,40 @@ int qcow2_mark_consistent(BlockDriverState *bs) return 0; } +static void qcow2_add_check_result(BdrvCheckResult *out, + const BdrvCheckResult *src, + bool set_allocation_info) +{ + out->corruptions += src->corruptions; + out->leaks += src->leaks; + out->check_errors += src->check_errors; + out->corruptions_fixed += src->corruptions_fixed; + out->leaks_fixed += src->leaks_fixed; + + if (set_allocation_info) { + out->image_end_offset = src->image_end_offset; + out->bfi = src->bfi; + } +} + static int coroutine_fn qcow2_co_check_locked(BlockDriverState *bs, BdrvCheckResult *result, BdrvCheckMode fix) { - int ret = qcow2_check_refcounts(bs, result, fix); + BdrvCheckResult snapshot_res = {}; + BdrvCheckResult refcount_res = {}; + int ret; + + memset(result, 0, sizeof(*result)); + + ret = qcow2_check_read_snapshot_table(bs, &snapshot_res, fix); + qcow2_add_check_result(result, &snapshot_res, false); + if (ret < 0) { + return ret; + } + + ret = qcow2_check_refcounts(bs, &refcount_res, fix); + qcow2_add_check_result(result, &refcount_res, true); if (ret < 0) { return ret; } @@ -1410,17 +1439,22 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, goto fail; } - /* The total size in bytes of the snapshot table is checked in - * qcow2_read_snapshots() because the size of each snapshot is - * variable and we don't know it yet. - * Here we only check the offset and number of snapshots. */ - ret = qcow2_validate_table(bs, header.snapshots_offset, - header.nb_snapshots, - sizeof(QCowSnapshotHeader), - sizeof(QCowSnapshotHeader) * QCOW_MAX_SNAPSHOTS, - "Snapshot table", errp); - if (ret < 0) { - goto fail; + if (!(flags & BDRV_O_CHECK)) { + /* + * The total size in bytes of the snapshot table is checked in + * qcow2_read_snapshots() because the size of each snapshot is + * variable and we don't know it yet. + * Here we only check the offset and number of snapshots. + */ + ret = qcow2_validate_table(bs, header.snapshots_offset, + header.nb_snapshots, + sizeof(QCowSnapshotHeader), + sizeof(QCowSnapshotHeader) * + QCOW_MAX_SNAPSHOTS, + "Snapshot table", errp); + if (ret < 0) { + goto fail; + } } /* read the level 1 table */ @@ -1580,13 +1614,19 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, s->image_backing_file = g_strdup(bs->auto_backing_file); } - /* Internal snapshots */ - s->snapshots_offset = header.snapshots_offset; - s->nb_snapshots = header.nb_snapshots; + /* + * Internal snapshots; skip reading them in check mode, because + * we do not need them then, and we do not want to abort because + * of a broken table. + */ + if (!(flags & BDRV_O_CHECK)) { + s->snapshots_offset = header.snapshots_offset; + s->nb_snapshots = header.nb_snapshots; - ret = qcow2_read_snapshots(bs, errp); - if (ret < 0) { - goto fail; + ret = qcow2_read_snapshots(bs, errp); + if (ret < 0) { + goto fail; + } } /* Clear unknown autoclear feature bits */ From patchwork Mon Oct 28 12:14:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215497 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8DC8C1599 for ; Mon, 28 Oct 2019 13:13:32 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6433420717 for ; Mon, 28 Oct 2019 13:13:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BzvKwnQB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6433420717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53626 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4pv-0002Rp-4Q for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:13:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51020) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xO-0003YZ-8g for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xI-0002EC-VN for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:07 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:27767 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xE-00029S-CK for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265018; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d8nQl2CsqL3dQqVIsVKROXIrGrVJwou6Y9YYBNm5fVY=; b=BzvKwnQBfJYgsffVox5LrX20g7Zlwq+o8HOw7k9UC+Z1GRTD3mSv46G84ZEnKsjERwY5I6 F7ZWaHBsrympoiktDk/qcTsUYA4fnANUcjimt1TMN9VqPc8xGgr0iukAmAsMtNXNFzN6Nd Y4CM0GBignFOnWOGUO9ivLh5Ef4auMk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-425-79GTOUHrM0CrHt9hsaSspg-1; Mon, 28 Oct 2019 08:16:54 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 203CF476; Mon, 28 Oct 2019 12:16:53 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B92BE60BF7; Mon, 28 Oct 2019 12:16:52 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 53/69] qcow2: Add qcow2_check_fix_snapshot_table() Date: Mon, 28 Oct 2019 13:14:45 +0100 Message-Id: <20191028121501.15279-54-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: 79GTOUHrM0CrHt9hsaSspg-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" qcow2_check_read_snapshot_table() can perform consistency checks, but it cannot fix everything. Specifically, it cannot allocate new clusters, because that should wait until the refcount structures are known to be consistent (i.e., after qcow2_check_refcounts()). Thus, it cannot call qcow2_write_snapshots(). Do that in qcow2_check_fix_snapshot_table(), which is called after qcow2_check_refcounts(). Currently, there is nothing that would set result->corruptions, so this is a no-op. A follow-up patch will change that. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-10-mreitz@redhat.com Signed-off-by: Max Reitz --- block/qcow2.h | 3 +++ block/qcow2-snapshot.c | 25 +++++++++++++++++++++++++ block/qcow2.c | 9 ++++++++- 3 files changed, 36 insertions(+), 1 deletion(-) diff --git a/block/qcow2.h b/block/qcow2.h index adb5c3bc28..601c2e4c82 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -721,6 +721,9 @@ int qcow2_write_snapshots(BlockDriverState *bs); int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, BdrvCheckResult *result, BdrvCheckMode fix); +int coroutine_fn qcow2_check_fix_snapshot_table(BlockDriverState *bs, + BdrvCheckResult *result, + BdrvCheckMode fix); /* qcow2-cache.c functions */ Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables, diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index d667bfd935..b526a8f819 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -380,6 +380,31 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, return 0; } +int coroutine_fn qcow2_check_fix_snapshot_table(BlockDriverState *bs, + BdrvCheckResult *result, + BdrvCheckMode fix) +{ + BDRVQcow2State *s = bs->opaque; + int ret; + + if (result->corruptions && (fix & BDRV_FIX_ERRORS)) { + qemu_co_mutex_unlock(&s->lock); + ret = qcow2_write_snapshots(bs); + qemu_co_mutex_lock(&s->lock); + if (ret < 0) { + result->check_errors++; + fprintf(stderr, "ERROR failed to update snapshot table: %s\n", + strerror(-ret)); + return ret; + } + + result->corruptions_fixed += result->corruptions; + result->corruptions = 0; + } + + return 0; +} + static void find_new_snapshot_id(BlockDriverState *bs, char *id_str, int id_str_size) { diff --git a/block/qcow2.c b/block/qcow2.c index b0026e8a9a..8d4f38ae74 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -597,13 +597,20 @@ static int coroutine_fn qcow2_co_check_locked(BlockDriverState *bs, memset(result, 0, sizeof(*result)); ret = qcow2_check_read_snapshot_table(bs, &snapshot_res, fix); - qcow2_add_check_result(result, &snapshot_res, false); if (ret < 0) { + qcow2_add_check_result(result, &snapshot_res, false); return ret; } ret = qcow2_check_refcounts(bs, &refcount_res, fix); qcow2_add_check_result(result, &refcount_res, true); + if (ret < 0) { + qcow2_add_check_result(result, &snapshot_res, false); + return ret; + } + + ret = qcow2_check_fix_snapshot_table(bs, &snapshot_res, fix); + qcow2_add_check_result(result, &snapshot_res, false); if (ret < 0) { return ret; } From patchwork Mon Oct 28 12:14:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215461 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 34204112B for ; Mon, 28 Oct 2019 12:58:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0627C20873 for ; Mon, 28 Oct 2019 12:58:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Z3X2rnB5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0627C20873 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53466 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4aw-0000wI-Ma for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 08:58:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51030) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xO-0003a1-VP for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xJ-0002FE-F1 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:08 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:52001 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xJ-00029v-9q for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265019; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q63O/eKry3U3/uIP7JVKqCBRcTVea0uKQQCNAQWrrqQ=; b=Z3X2rnB5FdfAw7MMe2xuclzU7fZthjGHFeEsmEz9xyHB19w+q/DsngcdYPdggdXMJGMPGc Q8t9wh9XhUerDmhuxAzF4BXZtnxABPoNvZAGp+0Np08e+7vm11/ZIIlnXkFW9cubIH+WJO BjJv2JiGZf1GqWIR8QLYw0L7//IVki0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-61-gkscDd9oNEOqw3brVKDbFA-1; Mon, 28 Oct 2019 08:16:56 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2A56C10052E0; Mon, 28 Oct 2019 12:16:55 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BF35C26186; Mon, 28 Oct 2019 12:16:54 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 54/69] qcow2: Fix broken snapshot table entries Date: Mon, 28 Oct 2019 13:14:46 +0100 Message-Id: <20191028121501.15279-55-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: gkscDd9oNEOqw3brVKDbFA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" The only case where we currently reject snapshot table entries is when they have too much extra data. Fix them with qemu-img check -r all by counting it as a corruption, reducing their extra_data_size, and then letting qcow2_check_fix_snapshot_table() do the rest. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-11-mreitz@redhat.com Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 67 +++++++++++++++++++++++++++++++++++------- 1 file changed, 56 insertions(+), 11 deletions(-) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index b526a8f819..53dc1635ec 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -44,7 +44,23 @@ void qcow2_free_snapshots(BlockDriverState *bs) s->nb_snapshots = 0; } -int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) +/* + * If @repair is true, try to repair a broken snapshot table instead + * of just returning an error: + * + * - If there were snapshots with too much extra metadata, increment + * *extra_data_dropped for each. + * This requires the caller to eventually rewrite the whole snapshot + * table, which requires cluster allocation. Therefore, this should + * be done only after qcow2_check_refcounts() made sure the refcount + * structures are valid. + * (In the meantime, the image is still valid because + * qcow2_check_refcounts() does not do anything with snapshots' + * extra data.) + */ +static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, + int *extra_data_dropped, + Error **errp) { BDRVQcow2State *s = bs->opaque; QCowSnapshotHeader h; @@ -64,6 +80,8 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) s->snapshots = g_new0(QCowSnapshot, s->nb_snapshots); for(i = 0; i < s->nb_snapshots; i++) { + bool truncate_unknown_extra_data = false; + /* Read statically sized part of the snapshot header */ offset = ROUND_UP(offset, 8); ret = bdrv_pread(bs->file, offset, &h, sizeof(h)); @@ -86,10 +104,21 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) name_size = be16_to_cpu(h.name_size); if (sn->extra_data_size > QCOW_MAX_SNAPSHOT_EXTRA_DATA) { - ret = -EFBIG; - error_setg(errp, "Too much extra metadata in snapshot table " - "entry %i", i); - goto fail; + if (!repair) { + ret = -EFBIG; + error_setg(errp, "Too much extra metadata in snapshot table " + "entry %i", i); + error_append_hint(errp, "You can force-remove this extra " + "metadata with qemu-img check -r all\n"); + goto fail; + } + + fprintf(stderr, "Discarding too much extra metadata in snapshot " + "table entry %i (%" PRIu32 " > %u)\n", + i, sn->extra_data_size, QCOW_MAX_SNAPSHOT_EXTRA_DATA); + + (*extra_data_dropped)++; + truncate_unknown_extra_data = true; } /* Read known extra data */ @@ -113,18 +142,26 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) } if (sn->extra_data_size > sizeof(extra)) { - /* Store unknown extra data */ - size_t unknown_extra_data_size = - sn->extra_data_size - sizeof(extra); + uint64_t extra_data_end; + size_t unknown_extra_data_size; + + extra_data_end = offset + sn->extra_data_size - sizeof(extra); + if (truncate_unknown_extra_data) { + sn->extra_data_size = QCOW_MAX_SNAPSHOT_EXTRA_DATA; + } + + /* Store unknown extra data */ + unknown_extra_data_size = sn->extra_data_size - sizeof(extra); sn->unknown_extra_data = g_malloc(unknown_extra_data_size); ret = bdrv_pread(bs->file, offset, sn->unknown_extra_data, unknown_extra_data_size); if (ret < 0) { - error_setg_errno(errp, -ret, "Failed to read snapshot table"); + error_setg_errno(errp, -ret, + "Failed to read snapshot table"); goto fail; } - offset += unknown_extra_data_size; + offset = extra_data_end; } /* Read snapshot ID */ @@ -163,6 +200,11 @@ fail: return ret; } +int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) +{ + return qcow2_do_read_snapshots(bs, false, NULL, errp); +} + /* add at the end of the file a new list of snapshots */ int qcow2_write_snapshots(BlockDriverState *bs) { @@ -328,6 +370,7 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, { BDRVQcow2State *s = bs->opaque; Error *local_err = NULL; + int extra_data_dropped = 0; int ret; struct { uint32_t nb_snapshots; @@ -363,7 +406,8 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, } qemu_co_mutex_unlock(&s->lock); - ret = qcow2_read_snapshots(bs, &local_err); + ret = qcow2_do_read_snapshots(bs, fix & BDRV_FIX_ERRORS, + &extra_data_dropped, &local_err); qemu_co_mutex_lock(&s->lock); if (ret < 0) { result->check_errors++; @@ -376,6 +420,7 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, return ret; } + result->corruptions += extra_data_dropped; return 0; } From patchwork Mon Oct 28 12:14:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215507 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4918E912 for ; Mon, 28 Oct 2019 13:19:47 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1F97320650 for ; Mon, 28 Oct 2019 13:19:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gjVP2ugH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F97320650 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53684 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4vy-0003nD-2T for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:19:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51027) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xO-0003Zw-Us for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xJ-0002FI-Fd for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:08 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:53979 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xJ-0002A1-7n for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265019; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0kPed5/fsHktSEaX2QTOmKq2J20PAl/gvdH8RRD3K/M=; b=gjVP2ugHunTxvmLbem0uNLDq3S+h8SSa8DJYv70oXHAUWjsZ9EhvexFPEsVdRjnjq0nA4p KXU7+TVqgXPK2QwqsnzYI54G7xXCMlphjpBfQp0XdnGypwwSkVyJzj6SkNkIyWvEjG3tDy vSS6H5zEZ5Xti6WxcHRj/ZeKcYxFlnI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-302-UFj1NjvBOJerjysGg9R4hw-1; Mon, 28 Oct 2019 08:16:58 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 36B8280183E; Mon, 28 Oct 2019 12:16:57 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CD43E100164D; Mon, 28 Oct 2019 12:16:56 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 55/69] qcow2: Keep track of the snapshot table length Date: Mon, 28 Oct 2019 13:14:47 +0100 Message-Id: <20191028121501.15279-56-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: UFj1NjvBOJerjysGg9R4hw-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" When repairing the snapshot table, we truncate entries that have too much extra data. This frees up space that we do not have to count towards the snapshot table size. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-12-mreitz@redhat.com Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 53dc1635ec..582eb3386a 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -68,6 +68,7 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, QCowSnapshot *sn; int i, id_str_size, name_size; int64_t offset; + uint64_t table_length = 0; int ret; if (!s->nb_snapshots) { @@ -82,6 +83,8 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, for(i = 0; i < s->nb_snapshots; i++) { bool truncate_unknown_extra_data = false; + table_length = ROUND_UP(table_length, 8); + /* Read statically sized part of the snapshot header */ offset = ROUND_UP(offset, 8); ret = bdrv_pread(bs->file, offset, &h, sizeof(h)); @@ -184,7 +187,16 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, offset += name_size; sn->name[name_size] = '\0'; - if (offset - s->snapshots_offset > QCOW_MAX_SNAPSHOTS_SIZE) { + /* Note that the extra data may have been truncated */ + table_length += sizeof(h) + sn->extra_data_size + id_str_size + + name_size; + if (!repair) { + assert(table_length == offset - s->snapshots_offset); + } + + if (table_length > QCOW_MAX_SNAPSHOTS_SIZE || + offset - s->snapshots_offset > INT_MAX) + { ret = -EFBIG; error_setg(errp, "Snapshot table is too big"); goto fail; From patchwork Mon Oct 28 12:14:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215513 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33ACB1390 for ; Mon, 28 Oct 2019 13:22:47 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0A1DD20663 for ; Mon, 28 Oct 2019 13:22:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CPcCJ7pS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0A1DD20663 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53706 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4yr-0000JL-RK for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:22:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51066) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xQ-0003d7-8D for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xJ-0002Eo-Bp for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:09 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:57330 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xJ-0002CC-3u for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265021; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AEWfKLHP0Mmcgw2XnEAAUVX8I6NYdFGqjUtq4PtaZKY=; b=CPcCJ7pSp3GTSQYBU6sYsahLK0jxI+jwPcvMTHVY7ftJicPaufb3cBJErI9zWNmlJ1lLoa Q5tODF0CZEXDUH/DiJJ8tYmYLgVxxwTjdaAnX38Sgna2WjoUyr3mB/uLeP3EbZ2pHkhrHX 9FyLoK6Jg+Fk2rpVru+epbevPSTBUw8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-334-uBLxAiTiND6CAl8nxLF83w-1; Mon, 28 Oct 2019 08:17:00 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6D6BE10052E0; Mon, 28 Oct 2019 12:16:59 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DDC005C1D6; Mon, 28 Oct 2019 12:16:58 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 56/69] qcow2: Fix overly long snapshot tables Date: Mon, 28 Oct 2019 13:14:48 +0100 Message-Id: <20191028121501.15279-57-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: uBLxAiTiND6CAl8nxLF83w-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We currently refuse to open qcow2 images with overly long snapshot tables. This patch makes qemu-img check -r all drop all offending entries past what we deem acceptable. The user cannot choose which snapshots are removed. This is fine because we have chosen the maximum snapshot table size to be so large (64 MB) that it cannot be reasonably reached. If the snapshot table exceeds this size, the image has probably been corrupted in some way; in this case, it is most important to just make the image usable such that the user can copy off at least the active layer. (Also note that the snapshots will be removed only with "-r all", so a plain "check" or "check -r leaks" will not delete any data.) Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-13-mreitz@redhat.com Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 88 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 78 insertions(+), 10 deletions(-) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 582eb3386a..366d9f574c 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -29,15 +29,24 @@ #include "qemu/error-report.h" #include "qemu/cutils.h" +static void qcow2_free_single_snapshot(BlockDriverState *bs, int i) +{ + BDRVQcow2State *s = bs->opaque; + + assert(i >= 0 && i < s->nb_snapshots); + g_free(s->snapshots[i].name); + g_free(s->snapshots[i].id_str); + g_free(s->snapshots[i].unknown_extra_data); + memset(&s->snapshots[i], 0, sizeof(s->snapshots[i])); +} + void qcow2_free_snapshots(BlockDriverState *bs) { BDRVQcow2State *s = bs->opaque; int i; for(i = 0; i < s->nb_snapshots; i++) { - g_free(s->snapshots[i].name); - g_free(s->snapshots[i].id_str); - g_free(s->snapshots[i].unknown_extra_data); + qcow2_free_single_snapshot(bs, i); } g_free(s->snapshots); s->snapshots = NULL; @@ -48,6 +57,14 @@ void qcow2_free_snapshots(BlockDriverState *bs) * If @repair is true, try to repair a broken snapshot table instead * of just returning an error: * + * - If the snapshot table was too long, set *nb_clusters_reduced to + * the number of snapshots removed off the end. + * The caller will update the on-disk nb_snapshots accordingly; + * this leaks clusters, but is safe. + * (The on-disk information must be updated before + * qcow2_check_refcounts(), because that function relies on + * s->nb_snapshots to reflect the on-disk value.) + * * - If there were snapshots with too much extra metadata, increment * *extra_data_dropped for each. * This requires the caller to eventually rewrite the whole snapshot @@ -59,6 +76,7 @@ void qcow2_free_snapshots(BlockDriverState *bs) * extra data.) */ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, + int *nb_clusters_reduced, int *extra_data_dropped, Error **errp) { @@ -67,7 +85,7 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, QCowSnapshotExtraData extra; QCowSnapshot *sn; int i, id_str_size, name_size; - int64_t offset; + int64_t offset, pre_sn_offset; uint64_t table_length = 0; int ret; @@ -83,6 +101,7 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, for(i = 0; i < s->nb_snapshots; i++) { bool truncate_unknown_extra_data = false; + pre_sn_offset = offset; table_length = ROUND_UP(table_length, 8); /* Read statically sized part of the snapshot header */ @@ -197,9 +216,31 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, if (table_length > QCOW_MAX_SNAPSHOTS_SIZE || offset - s->snapshots_offset > INT_MAX) { - ret = -EFBIG; - error_setg(errp, "Snapshot table is too big"); - goto fail; + if (!repair) { + ret = -EFBIG; + error_setg(errp, "Snapshot table is too big"); + error_append_hint(errp, "You can force-remove all %u " + "overhanging snapshots with qemu-img check " + "-r all\n", s->nb_snapshots - i); + goto fail; + } + + fprintf(stderr, "Discarding %u overhanging snapshots (snapshot " + "table is too big)\n", s->nb_snapshots - i); + + *nb_clusters_reduced += (s->nb_snapshots - i); + + /* Discard current snapshot also */ + qcow2_free_single_snapshot(bs, i); + + /* + * This leaks all the rest of the snapshot table and the + * snapshots' clusters, but we run in check -r all mode, + * so qcow2_check_refcounts() will take care of it. + */ + s->nb_snapshots = i; + offset = pre_sn_offset; + break; } } @@ -214,7 +255,7 @@ fail: int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) { - return qcow2_do_read_snapshots(bs, false, NULL, errp); + return qcow2_do_read_snapshots(bs, false, NULL, NULL, errp); } /* add at the end of the file a new list of snapshots */ @@ -382,6 +423,7 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, { BDRVQcow2State *s = bs->opaque; Error *local_err = NULL; + int nb_clusters_reduced = 0; int extra_data_dropped = 0; int ret; struct { @@ -419,7 +461,8 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, qemu_co_mutex_unlock(&s->lock); ret = qcow2_do_read_snapshots(bs, fix & BDRV_FIX_ERRORS, - &extra_data_dropped, &local_err); + &nb_clusters_reduced, &extra_data_dropped, + &local_err); qemu_co_mutex_lock(&s->lock); if (ret < 0) { result->check_errors++; @@ -432,7 +475,32 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, return ret; } - result->corruptions += extra_data_dropped; + result->corruptions += nb_clusters_reduced + extra_data_dropped; + + if (nb_clusters_reduced) { + /* + * Update image header now, because: + * (1) qcow2_check_refcounts() relies on s->nb_snapshots to be + * the same as what the image header says, + * (2) this leaks clusters, but qcow2_check_refcounts() will + * fix that. + */ + assert(fix & BDRV_FIX_ERRORS); + + snapshot_table_pointer.nb_snapshots = cpu_to_be32(s->nb_snapshots); + ret = bdrv_pwrite_sync(bs->file, offsetof(QCowHeader, nb_snapshots), + &snapshot_table_pointer.nb_snapshots, + sizeof(snapshot_table_pointer.nb_snapshots)); + if (ret < 0) { + result->check_errors++; + fprintf(stderr, "ERROR failed to update the snapshot count in the " + "image header: %s\n", strerror(-ret)); + return ret; + } + + result->corruptions_fixed += nb_clusters_reduced; + result->corruptions -= nb_clusters_reduced; + } return 0; } From patchwork Mon Oct 28 12:14:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215491 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A469D1599 for ; Mon, 28 Oct 2019 13:09:41 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7A9C520717 for ; Mon, 28 Oct 2019 13:09:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fqYuhQyi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7A9C520717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53580 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4mB-0005RN-Tx for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:09:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51029) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xO-0003a0-VS for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xJ-0002FO-G0 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:08 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:59828 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xJ-0002Dt-9S for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265024; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qlezfbb+DroBt18ovLnjs/YdrlwJEgwyBAG2+VKYQKI=; b=fqYuhQyimCEsfNdDHT3Dsa3XVjOL4ZIVyPTY4QI30kooipoi62HpbKKpQv24Y8cRU4Remb URhhb9vBFB63njHmXIbMeAkWyZZpG37Tv/LDZEQ5gkFQnuUBx7a2Molovd8m5d82UM5fgy j8FKf4BdVBUm9+8XZjPEioQyccoZlOQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-125-E7H7BWO_MZiiSzYJ-S0v1Q-1; Mon, 28 Oct 2019 08:17:02 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7EB46107AD28; Mon, 28 Oct 2019 12:17:01 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1DE39100194E; Mon, 28 Oct 2019 12:17:00 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 57/69] qcow2: Repair snapshot table with too many entries Date: Mon, 28 Oct 2019 13:14:49 +0100 Message-Id: <20191028121501.15279-58-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: E7H7BWO_MZiiSzYJ-S0v1Q-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" The user cannot choose which snapshots are removed. This is fine because we have chosen the maximum snapshot table size to be so large (65536 entries) that it cannot be reasonably reached. If the snapshot table exceeds this size, the image has probably been corrupted in some way; in this case, it is most important to just make the image usable such that the user can copy off at least the active layer. (Also note that the snapshots will be removed only with "-r all", so a plain "check" or "check -r leaks" will not delete any data.) Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-14-mreitz@redhat.com Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index 366d9f574c..dac8a778e4 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -444,6 +444,14 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, s->snapshots_offset = be64_to_cpu(snapshot_table_pointer.snapshots_offset); s->nb_snapshots = be32_to_cpu(snapshot_table_pointer.nb_snapshots); + if (s->nb_snapshots > QCOW_MAX_SNAPSHOTS && (fix & BDRV_FIX_ERRORS)) { + fprintf(stderr, "Discarding %u overhanging snapshots\n", + s->nb_snapshots - QCOW_MAX_SNAPSHOTS); + + nb_clusters_reduced += s->nb_snapshots - QCOW_MAX_SNAPSHOTS; + s->nb_snapshots = QCOW_MAX_SNAPSHOTS; + } + ret = qcow2_validate_table(bs, s->snapshots_offset, s->nb_snapshots, sizeof(QCowSnapshotHeader), sizeof(QCowSnapshotHeader) * QCOW_MAX_SNAPSHOTS, @@ -452,6 +460,12 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, result->check_errors++; error_reportf_err(local_err, "ERROR "); + if (s->nb_snapshots > QCOW_MAX_SNAPSHOTS) { + fprintf(stderr, "You can force-remove all %u overhanging snapshots " + "with qemu-img check -r all\n", + s->nb_snapshots - QCOW_MAX_SNAPSHOTS); + } + /* We did not read the snapshot table, so invalidate this information */ s->snapshots_offset = 0; s->nb_snapshots = 0; From patchwork Mon Oct 28 12:14:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215511 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CAFFD1390 for ; Mon, 28 Oct 2019 13:22:06 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A132320663 for ; Mon, 28 Oct 2019 13:22:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ElGAE28i" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A132320663 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53702 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4yD-0007OK-J3 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:22:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51108) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xS-0003fo-Vv for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xQ-0002JG-Nr for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:14 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:33962 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xQ-0002GT-CO for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265028; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2ks7fqz6OarAeHOWFTI0dVvnz7s3pMBNrIuytG56UVo=; b=ElGAE28iwHNJqTONTWXJsVr3s2nisG184tWZsoKioJSFlptNBHZwc2xdSUunbEHO5/r7K3 W5VXhQe/nooxJZhFr9RnWPxX+y0CxGXclJBeLHQEOfO2H+zwruMrjQufdlship1SDYHWK7 s90tvG4Kcv/vTLdAeB1jKf7/QyNpoG8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-387-Oaz8i3rFPX2tjNANa0ZOjA-1; Mon, 28 Oct 2019 08:17:04 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 823B0800FF1; Mon, 28 Oct 2019 12:17:03 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 26D1926186; Mon, 28 Oct 2019 12:17:02 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 58/69] qcow2: Fix v3 snapshot table entry compliancy Date: Mon, 28 Oct 2019 13:14:50 +0100 Message-Id: <20191028121501.15279-59-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: Oaz8i3rFPX2tjNANa0ZOjA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" qcow2 v3 images require every snapshot table entry to have at least 16 bytes of extra data. If they do not, let qemu-img check -r all fix it. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-15-mreitz@redhat.com Signed-off-by: Max Reitz --- block/qcow2-snapshot.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c index dac8a778e4..5ab64da1ec 100644 --- a/block/qcow2-snapshot.c +++ b/block/qcow2-snapshot.c @@ -516,6 +516,24 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, result->corruptions -= nb_clusters_reduced; } + /* + * All of v3 images' snapshot table entries need to have at least + * 16 bytes of extra data. + */ + if (s->qcow_version >= 3) { + int i; + for (i = 0; i < s->nb_snapshots; i++) { + if (s->snapshots[i].extra_data_size < + sizeof_field(QCowSnapshotExtraData, vm_state_size_large) + + sizeof_field(QCowSnapshotExtraData, disk_size)) + { + result->corruptions++; + fprintf(stderr, "%s snapshot table entry %i is incomplete\n", + fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR", i); + } + } + } + return 0; } From patchwork Mon Oct 28 12:14:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215505 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB98C197C for ; Mon, 28 Oct 2019 13:19:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 926B120650 for ; Mon, 28 Oct 2019 13:19:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VrZmVnhi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 926B120650 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53680 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4vH-0002sW-2X for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:19:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51107) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xS-0003fm-UT for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xQ-0002JC-O4 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:14 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:34756 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xQ-0002HD-Dy for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265030; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UA5gbQSMau0y4MqDITjLNuAOKJfVKqcPdWnvVyrKsos=; b=VrZmVnhiEj5AsSdLh69gU1QEhEd7hF2Ja9JkIka4m7BCnfkO2DTB2MSacIopW8fhGaFWty 9lkwtNgeQ2H2ZxdR/0nT5EZke0hQKSxhThyb+m73SAkxIgiIAIGNLjv0J+R7JhfR9IYPB6 AUmavhisHWyumTuO1JbTGlSOGwhKm6M= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-431-xEeOcmfhOhK8iRaIi_dhLg-1; Mon, 28 Oct 2019 08:17:06 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 81E4B180496E; Mon, 28 Oct 2019 12:17:05 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 291C626186; Mon, 28 Oct 2019 12:17:05 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 59/69] iotests: Add peek_file* functions Date: Mon, 28 Oct 2019 13:14:51 +0100 Message-Id: <20191028121501.15279-60-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: xEeOcmfhOhK8iRaIi_dhLg-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-16-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/common.rc | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 12b4751848..fa7bae2422 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -53,6 +53,26 @@ poke_file() printf "$3" | dd "of=$1" bs=1 "seek=$2" conv=notrunc &>/dev/null } +# peek_file_le 'test.img' 512 2 => 65534 +peek_file_le() +{ + # Wrap in echo $() to strip spaces + echo $(od -j"$2" -N"$3" --endian=little -An -vtu"$3" "$1") +} + +# peek_file_be 'test.img' 512 2 => 65279 +peek_file_be() +{ + # Wrap in echo $() to strip spaces + echo $(od -j"$2" -N"$3" --endian=big -An -vtu"$3" "$1") +} + +# peek_file_raw 'test.img' 512 2 => '\xff\xfe' +peek_file_raw() +{ + dd if="$1" bs=1 skip="$2" count="$3" status=none +} + if ! . ./common.config then From patchwork Mon Oct 28 12:14:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215517 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 13F97912 for ; Mon, 28 Oct 2019 13:26:24 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C269220663 for ; Mon, 28 Oct 2019 13:26:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="avyY+s7W" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C269220663 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53738 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP52M-0005oA-Rg for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:26:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51162) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xX-0003i8-E7 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xS-0002KI-Rl for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:19 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:31657 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xR-0002JL-0Q for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265032; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ASGiUXPMKPMYEWXoJfb208s4zhtmYyg1rLUCrjWaxqQ=; b=avyY+s7Wjk63QIrUcrCYqPcpfP8OqBvytOiMWfwhMuwBSxK5lFCqUMFyCg34vTXpH67n1k Tj/Wt4kUqEViD8+Urn12wTTxWNmIOgM8PUVB8ZCHOlZWwb4ecskP4lW86KYJxKHLxLWfoG 69RRKFTLzl83MNtsYfQqrVvC/rJ02Yc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-59-iWa2fSNSOg6LlGRo6IvJSg-1; Mon, 28 Oct 2019 08:17:10 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id ECD7A10052E0; Mon, 28 Oct 2019 12:17:09 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 316FA261B0; Mon, 28 Oct 2019 12:17:07 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 60/69] iotests: Test qcow2's snapshot table handling Date: Mon, 28 Oct 2019 13:14:52 +0100 Message-Id: <20191028121501.15279-61-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: iWa2fSNSOg6LlGRo6IvJSg-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Add a test how our qcow2 driver handles extra data in snapshot table entries, and how it repairs overly long snapshot tables. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Message-id: 20191011152814.14791-17-mreitz@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/261 | 523 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/261.out | 346 ++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 870 insertions(+) create mode 100755 tests/qemu-iotests/261 create mode 100644 tests/qemu-iotests/261.out diff --git a/tests/qemu-iotests/261 b/tests/qemu-iotests/261 new file mode 100755 index 0000000000..fb96bcfbe2 --- /dev/null +++ b/tests/qemu-iotests/261 @@ -0,0 +1,523 @@ +#!/usr/bin/env bash +# +# Test case for qcow2's handling of extra data in snapshot table entries +# (and more generally, how certain cases of broken snapshot tables are +# handled) +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=mreitz@redhat.com + +seq=$(basename $0) +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img + rm -f "$TEST_IMG".v{2,3}.orig + rm -f "$TEST_DIR"/sn{0,1,2}{,-pre,-extra,-post} +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +# This tests qocw2-specific low-level functionality +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux +# (1) We create a v2 image that supports nothing but refcount_bits=16 +# (2) We do some refcount management on our own which expects +# refcount_bits=16 +_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' + +# Parameters: +# $1: image filename +# $2: snapshot table entry offset in the image +snapshot_table_entry_size() +{ + id_len=$(peek_file_be "$1" $(($2 + 12)) 2) + name_len=$(peek_file_be "$1" $(($2 + 14)) 2) + extra_len=$(peek_file_be "$1" $(($2 + 36)) 4) + + full_len=$((40 + extra_len + id_len + name_len)) + echo $(((full_len + 7) / 8 * 8)) +} + +# Parameter: +# $1: image filename +print_snapshot_table() +{ + nb_entries=$(peek_file_be "$1" 60 4) + offset=$(peek_file_be "$1" 64 8) + + echo "Snapshots in $1:" | _filter_testdir | _filter_imgfmt + + for ((i = 0; i < nb_entries; i++)); do + id_len=$(peek_file_be "$1" $((offset + 12)) 2) + name_len=$(peek_file_be "$1" $((offset + 14)) 2) + extra_len=$(peek_file_be "$1" $((offset + 36)) 4) + + extra_ofs=$((offset + 40)) + id_ofs=$((extra_ofs + extra_len)) + name_ofs=$((id_ofs + id_len)) + + echo " [$i]" + echo " ID: $(peek_file_raw "$1" $id_ofs $id_len)" + echo " Name: $(peek_file_raw "$1" $name_ofs $name_len)" + echo " Extra data size: $extra_len" + if [ $extra_len -ge 8 ]; then + echo " VM state size: $(peek_file_be "$1" $extra_ofs 8)" + fi + if [ $extra_len -ge 16 ]; then + echo " Disk size: $(peek_file_be "$1" $((extra_ofs + 8)) 8)" + fi + if [ $extra_len -gt 16 ]; then + echo ' Unknown extra data:' \ + "$(peek_file_raw "$1" $((extra_ofs + 16)) $((extra_len - 16)) \ + | tr -d '\0')" + fi + + offset=$((offset + $(snapshot_table_entry_size "$1" $offset))) + done +} + +# Mark clusters as allocated; works only in refblock 0 (i.e. before +# cluster #32768). +# Parameters: +# $1: Start offset of what to allocate +# $2: End offset (exclusive) +refblock0_allocate() +{ + reftable_ofs=$(peek_file_be "$TEST_IMG" 48 8) + refblock_ofs=$(peek_file_be "$TEST_IMG" $reftable_ofs 8) + + cluster=$(($1 / 65536)) + ecluster=$((($2 + 65535) / 65536)) + + while [ $cluster -lt $ecluster ]; do + if [ $cluster -ge 32768 ]; then + echo "*** Abort: Cluster $cluster exceeds refblock 0 ***" + exit 1 + fi + poke_file "$TEST_IMG" $((refblock_ofs + cluster * 2)) '\x00\x01' + cluster=$((cluster + 1)) + done +} + + +echo +echo '=== Create v2 template ===' +echo + +# Create v2 image with a snapshot table with three entries: +# [0]: No extra data (valid with v2, not valid with v3) +# [1]: Has extra data unknown to qemu +# [2]: Has the 64-bit VM state size, but not the disk size (again, +# valid with v2, not valid with v3) + +TEST_IMG="$TEST_IMG.v2.orig" IMGOPTS='compat=0.10' _make_test_img 64M +$QEMU_IMG snapshot -c sn0 "$TEST_IMG.v2.orig" +$QEMU_IMG snapshot -c sn1 "$TEST_IMG.v2.orig" +$QEMU_IMG snapshot -c sn2 "$TEST_IMG.v2.orig" + +# Copy out all existing snapshot table entries +sn_table_ofs=$(peek_file_be "$TEST_IMG.v2.orig" 64 8) + +# ofs: Snapshot table entry offset +# eds: Extra data size +# ids: Name + ID size +# len: Total entry length +sn0_ofs=$sn_table_ofs +sn0_eds=$(peek_file_be "$TEST_IMG.v2.orig" $((sn0_ofs + 36)) 4) +sn0_ids=$(($(peek_file_be "$TEST_IMG.v2.orig" $((sn0_ofs + 12)) 2) + + $(peek_file_be "$TEST_IMG.v2.orig" $((sn0_ofs + 14)) 2))) +sn0_len=$(snapshot_table_entry_size "$TEST_IMG.v2.orig" $sn0_ofs) +sn1_ofs=$((sn0_ofs + sn0_len)) +sn1_eds=$(peek_file_be "$TEST_IMG.v2.orig" $((sn1_ofs + 36)) 4) +sn1_ids=$(($(peek_file_be "$TEST_IMG.v2.orig" $((sn1_ofs + 12)) 2) + + $(peek_file_be "$TEST_IMG.v2.orig" $((sn1_ofs + 14)) 2))) +sn1_len=$(snapshot_table_entry_size "$TEST_IMG.v2.orig" $sn1_ofs) +sn2_ofs=$((sn1_ofs + sn1_len)) +sn2_eds=$(peek_file_be "$TEST_IMG.v2.orig" $((sn2_ofs + 36)) 4) +sn2_ids=$(($(peek_file_be "$TEST_IMG.v2.orig" $((sn2_ofs + 12)) 2) + + $(peek_file_be "$TEST_IMG.v2.orig" $((sn2_ofs + 14)) 2))) +sn2_len=$(snapshot_table_entry_size "$TEST_IMG.v2.orig" $sn2_ofs) + +# Data before extra data +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn0-pre" bs=1 skip=$sn0_ofs count=40 \ + &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn1-pre" bs=1 skip=$sn1_ofs count=40 \ + &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn2-pre" bs=1 skip=$sn2_ofs count=40 \ + &> /dev/null + +# Extra data +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn0-extra" bs=1 \ + skip=$((sn0_ofs + 40)) count=$sn0_eds &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn1-extra" bs=1 \ + skip=$((sn1_ofs + 40)) count=$sn1_eds &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn2-extra" bs=1 \ + skip=$((sn2_ofs + 40)) count=$sn2_eds &> /dev/null + +# Data after extra data +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn0-post" bs=1 \ + skip=$((sn0_ofs + 40 + sn0_eds)) count=$sn0_ids \ + &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn1-post" bs=1 \ + skip=$((sn1_ofs + 40 + sn1_eds)) count=$sn1_ids \ + &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn2-post" bs=1 \ + skip=$((sn2_ofs + 40 + sn2_eds)) count=$sn2_ids \ + &> /dev/null + +# Amend them, one by one +# Set sn0's extra data size to 0 +poke_file "$TEST_DIR/sn0-pre" 36 '\x00\x00\x00\x00' +truncate -s 0 "$TEST_DIR/sn0-extra" +# Grow sn0-post to pad +truncate -s $(($(snapshot_table_entry_size "$TEST_DIR/sn0-pre") - 40)) \ + "$TEST_DIR/sn0-post" + +# Set sn1's extra data size to 42 +poke_file "$TEST_DIR/sn1-pre" 36 '\x00\x00\x00\x2a' +truncate -s 42 "$TEST_DIR/sn1-extra" +poke_file "$TEST_DIR/sn1-extra" 16 'very important data' +# Grow sn1-post to pad +truncate -s $(($(snapshot_table_entry_size "$TEST_DIR/sn1-pre") - 82)) \ + "$TEST_DIR/sn1-post" + +# Set sn2's extra data size to 8 +poke_file "$TEST_DIR/sn2-pre" 36 '\x00\x00\x00\x08' +truncate -s 8 "$TEST_DIR/sn2-extra" +# Grow sn2-post to pad +truncate -s $(($(snapshot_table_entry_size "$TEST_DIR/sn2-pre") - 48)) \ + "$TEST_DIR/sn2-post" + +# Construct snapshot table +cat "$TEST_DIR"/sn0-{pre,extra,post} \ + "$TEST_DIR"/sn1-{pre,extra,post} \ + "$TEST_DIR"/sn2-{pre,extra,post} \ + | dd of="$TEST_IMG.v2.orig" bs=1 seek=$sn_table_ofs conv=notrunc \ + &> /dev/null + +# Done! +TEST_IMG="$TEST_IMG.v2.orig" _check_test_img +print_snapshot_table "$TEST_IMG.v2.orig" + +echo +echo '=== Upgrade to v3 ===' +echo + +cp "$TEST_IMG.v2.orig" "$TEST_IMG.v3.orig" +$QEMU_IMG amend -o compat=1.1 "$TEST_IMG.v3.orig" +TEST_IMG="$TEST_IMG.v3.orig" _check_test_img +print_snapshot_table "$TEST_IMG.v3.orig" + +echo +echo '=== Repair botched v3 ===' +echo + +# Force the v2 file to be v3. v3 requires each snapshot table entry +# to have at least 16 bytes of extra data, so it will not comply to +# the qcow2 v3 specification; but we can fix that. +cp "$TEST_IMG.v2.orig" "$TEST_IMG" + +# Set version +poke_file "$TEST_IMG" 4 '\x00\x00\x00\x03' +# Increase header length (necessary for v3) +poke_file "$TEST_IMG" 100 '\x00\x00\x00\x68' +# Set refcount order (necessary for v3) +poke_file "$TEST_IMG" 96 '\x00\x00\x00\x04' + +_check_test_img -r all +print_snapshot_table "$TEST_IMG" + + +# From now on, just test the qcow2 version we are supposed to test. +# (v3 by default, v2 by choice through $IMGOPTS.) +# That works because we always write all known extra data when +# updating the snapshot table, independent of the version. + +if echo "$IMGOPTS" | grep -q 'compat=\(0\.10\|v2\)' 2> /dev/null; then + subver=v2 +else + subver=v3 +fi + +echo +echo '=== Add new snapshot ===' +echo + +cp "$TEST_IMG.$subver.orig" "$TEST_IMG" +$QEMU_IMG snapshot -c sn3 "$TEST_IMG" +_check_test_img +print_snapshot_table "$TEST_IMG" + +echo +echo '=== Remove different snapshots ===' + +for sn in sn0 sn1 sn2; do + echo + echo "--- $sn ---" + + cp "$TEST_IMG.$subver.orig" "$TEST_IMG" + $QEMU_IMG snapshot -d $sn "$TEST_IMG" + _check_test_img + print_snapshot_table "$TEST_IMG" +done + +echo +echo '=== Reject too much unknown extra data ===' +echo + +cp "$TEST_IMG.$subver.orig" "$TEST_IMG" +$QEMU_IMG snapshot -c sn3 "$TEST_IMG" + +sn_table_ofs=$(peek_file_be "$TEST_IMG" 64 8) +sn0_ofs=$sn_table_ofs +sn1_ofs=$((sn0_ofs + $(snapshot_table_entry_size "$TEST_IMG" $sn0_ofs))) +sn2_ofs=$((sn1_ofs + $(snapshot_table_entry_size "$TEST_IMG" $sn1_ofs))) +sn3_ofs=$((sn2_ofs + $(snapshot_table_entry_size "$TEST_IMG" $sn2_ofs))) + +# 64 kB of extra data should be rejected +# (Note that this also induces a refcount error, because it spills +# over to the next cluster. That's a good way to test that we can +# handle simultaneous snapshot table and refcount errors.) +poke_file "$TEST_IMG" $((sn3_ofs + 36)) '\x00\x01\x00\x00' + +# Print error +_img_info +echo +_check_test_img +echo + +# Should be repairable +_check_test_img -r all + +echo +echo '=== Snapshot table too big ===' +echo + +sn_table_ofs=$(peek_file_be "$TEST_IMG.v3.orig" 64 8) + +# Fill a snapshot with 1 kB of extra data, a 65535-char ID, and a +# 65535-char name, and repeat it as many times as necessary to fill +# 64 MB (the maximum supported by qemu) + +touch "$TEST_DIR/sn0" + +# Full size (fixed + extra + ID + name + padding) +sn_size=$((40 + 1024 + 65535 + 65535 + 2)) + +# We only need the fixed part, though. +truncate -s 40 "$TEST_DIR/sn0" + +# 65535-char ID string +poke_file "$TEST_DIR/sn0" 12 '\xff\xff' +# 65535-char name +poke_file "$TEST_DIR/sn0" 14 '\xff\xff' +# 1 kB of extra data +poke_file "$TEST_DIR/sn0" 36 '\x00\x00\x04\x00' + +# Create test image +_make_test_img 64M + +# Hook up snapshot table somewhere safe (at 1 MB) +poke_file "$TEST_IMG" 64 '\x00\x00\x00\x00\x00\x10\x00\x00' + +offset=1048576 +size_written=0 +sn_count=0 +while [ $size_written -le $((64 * 1048576)) ]; do + dd if="$TEST_DIR/sn0" of="$TEST_IMG" bs=1 seek=$offset conv=notrunc \ + &> /dev/null + offset=$((offset + sn_size)) + size_written=$((size_written + sn_size)) + sn_count=$((sn_count + 1)) +done +truncate -s "$offset" "$TEST_IMG" + +# Give the last snapshot (the one to be removed) an L1 table so we can +# see how that is handled when repairing the image +# (Put it two clusters before 1 MB, and one L2 table one cluster +# before 1 MB) +poke_file "$TEST_IMG" $((offset - sn_size + 0)) \ + '\x00\x00\x00\x00\x00\x0e\x00\x00' +poke_file "$TEST_IMG" $((offset - sn_size + 8)) \ + '\x00\x00\x00\x01' + +# Hook up the L2 table +poke_file "$TEST_IMG" $((1048576 - 2 * 65536)) \ + '\x80\x00\x00\x00\x00\x0f\x00\x00' + +# Make sure all of the clusters we just hooked up are allocated: +# - The snapshot table +# - The last snapshot's L1 and L2 table +refblock0_allocate $((1048576 - 2 * 65536)) $offset + +poke_file "$TEST_IMG" 60 \ + "$(printf '%08x' $sn_count | sed -e 's/\(..\)/\\x\1/g')" + +# Print error +_img_info +echo +_check_test_img +echo + +# Should be repairable +_check_test_img -r all + +echo +echo "$((sn_count - 1)) snapshots should remain:" +echo " qemu-img info reports $(_img_info | grep -c '^ \{34\}') snapshots" +echo " Image header reports $(peek_file_be "$TEST_IMG" 60 4) snapshots" + +echo +echo '=== Snapshot table too big with one entry with too much extra data ===' +echo + +# For this test, we reuse the image from the previous case, which has +# a snapshot table that is right at the limit. +# Our layout looks like this: +# - (a number of snapshot table entries) +# - One snapshot with $extra_data_size extra data +# - One normal snapshot that breaks the 64 MB boundary +# - One normal snapshot beyond the 64 MB boundary +# +# $extra_data_size is calculated so that simply by virtue of it +# decreasing to 1 kB, the penultimate snapshot will fit into 64 MB +# limit again. The final snapshot will always be beyond the limit, so +# that we can see that the repair algorithm does still determine the +# limit to be somewhere, even when truncating one snapshot's extra +# data. + +# The last case has removed the last snapshot, so calculate +# $old_offset to get the current image's real length +old_offset=$((offset - sn_size)) + +# The layout from the previous test had one snapshot beyond the 64 MB +# limit; we want the same (after the oversized extra data has been +# truncated to 1 kB), so we drop the last three snapshots and +# construct them from scratch. +offset=$((offset - 3 * sn_size)) +sn_count=$((sn_count - 3)) + +# Assuming we had already written one of the three snapshots +# (necessary so we can calculate $extra_data_size next). +size_written=$((size_written - 2 * sn_size)) + +# Increase the extra data size so we go past the limit +# (The -1024 comes from the 1 kB of extra data we already have) +extra_data_size=$((64 * 1048576 + 8 - sn_size - (size_written - 1024))) + +poke_file "$TEST_IMG" $((offset + 36)) \ + "$(printf '%08x' $extra_data_size | sed -e 's/\(..\)/\\x\1/g')" + +offset=$((offset + sn_size - 1024 + extra_data_size)) +size_written=$((size_written - 1024 + extra_data_size)) +sn_count=$((sn_count + 1)) + +# Write the two normal snapshots +for ((i = 0; i < 2; i++)); do + dd if="$TEST_DIR/sn0" of="$TEST_IMG" bs=1 seek=$offset conv=notrunc \ + &> /dev/null + offset=$((offset + sn_size)) + size_written=$((size_written + sn_size)) + sn_count=$((sn_count + 1)) + + if [ $i = 0 ]; then + # Check that the penultimate snapshot is beyond the 64 MB limit + echo "Snapshot table size should equal $((64 * 1048576 + 8)):" \ + $size_written + echo + fi +done + +truncate -s $offset "$TEST_IMG" +refblock0_allocate $old_offset $offset + +poke_file "$TEST_IMG" 60 \ + "$(printf '%08x' $sn_count | sed -e 's/\(..\)/\\x\1/g')" + +# Print error +_img_info +echo +_check_test_img +echo + +# Just truncating the extra data should be sufficient to shorten the +# snapshot table so only one snapshot exceeds the extra size +_check_test_img -r all + +echo +echo '=== Too many snapshots ===' +echo + +# Create a v2 image, for speeds' sake: All-zero snapshot table entries +# are only valid in v2. +IMGOPTS='compat=0.10' _make_test_img 64M + +# Hook up snapshot table somewhere safe (at 1 MB) +poke_file "$TEST_IMG" 64 '\x00\x00\x00\x00\x00\x10\x00\x00' +# "Create" more than 65536 snapshots (twice that many here) +poke_file "$TEST_IMG" 60 '\x00\x02\x00\x00' + +# 40-byte all-zero snapshot table entries are valid snapshots, but +# only in v2 (v3 needs 16 bytes of extra data, so we would have to +# write 131072x '\x10'). +truncate -s $((1048576 + 40 * 131072)) "$TEST_IMG" + +# But let us give one of the snapshots to be removed an L1 table so +# we can see how that is handled when repairing the image. +# (Put it two clusters before 1 MB, and one L2 table one cluster +# before 1 MB) +poke_file "$TEST_IMG" $((1048576 + 40 * 65536 + 0)) \ + '\x00\x00\x00\x00\x00\x0e\x00\x00' +poke_file "$TEST_IMG" $((1048576 + 40 * 65536 + 8)) \ + '\x00\x00\x00\x01' + +# Hook up the L2 table +poke_file "$TEST_IMG" $((1048576 - 2 * 65536)) \ + '\x80\x00\x00\x00\x00\x0f\x00\x00' + +# Make sure all of the clusters we just hooked up are allocated: +# - The snapshot table +# - The last snapshot's L1 and L2 table +refblock0_allocate $((1048576 - 2 * 65536)) $((1048576 + 40 * 131072)) + +# Print error +_img_info +echo +_check_test_img +echo + +# Should be repairable +_check_test_img -r all + +echo +echo '65536 snapshots should remain:' +echo " qemu-img info reports $(_img_info | grep -c '^ \{34\}') snapshots" +echo " Image header reports $(peek_file_be "$TEST_IMG" 60 4) snapshots" + +# success, all done +echo "*** done" +status=0 diff --git a/tests/qemu-iotests/261.out b/tests/qemu-iotests/261.out new file mode 100644 index 0000000000..2600354566 --- /dev/null +++ b/tests/qemu-iotests/261.out @@ -0,0 +1,346 @@ +QA output created by 261 + +=== Create v2 template === + +Formatting 'TEST_DIR/t.IMGFMT.v2.orig', fmt=IMGFMT size=67108864 +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT.v2.orig: + [0] + ID: 1 + Name: sn0 + Extra data size: 0 + [1] + ID: 2 + Name: sn1 + Extra data size: 42 + VM state size: 0 + Disk size: 67108864 + Unknown extra data: very important data + [2] + ID: 3 + Name: sn2 + Extra data size: 8 + VM state size: 0 + +=== Upgrade to v3 === + +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT.v3.orig: + [0] + ID: 1 + Name: sn0 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + [1] + ID: 2 + Name: sn1 + Extra data size: 42 + VM state size: 0 + Disk size: 67108864 + Unknown extra data: very important data + [2] + ID: 3 + Name: sn2 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + +=== Repair botched v3 === + +Repairing snapshot table entry 0 is incomplete +Repairing snapshot table entry 2 is incomplete +The following inconsistencies were found and repaired: + + 0 leaked clusters + 2 corruptions + +Double checking the fixed image now... +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT: + [0] + ID: 1 + Name: sn0 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + [1] + ID: 2 + Name: sn1 + Extra data size: 42 + VM state size: 0 + Disk size: 67108864 + Unknown extra data: very important data + [2] + ID: 3 + Name: sn2 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + +=== Add new snapshot === + +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT: + [0] + ID: 1 + Name: sn0 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + [1] + ID: 2 + Name: sn1 + Extra data size: 42 + VM state size: 0 + Disk size: 67108864 + Unknown extra data: very important data + [2] + ID: 3 + Name: sn2 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + [3] + ID: 4 + Name: sn3 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + +=== Remove different snapshots === + +--- sn0 --- +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT: + [0] + ID: 2 + Name: sn1 + Extra data size: 42 + VM state size: 0 + Disk size: 67108864 + Unknown extra data: very important data + [1] + ID: 3 + Name: sn2 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + +--- sn1 --- +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT: + [0] + ID: 1 + Name: sn0 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + [1] + ID: 3 + Name: sn2 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + +--- sn2 --- +No errors were found on the image. +Snapshots in TEST_DIR/t.IMGFMT: + [0] + ID: 1 + Name: sn0 + Extra data size: 16 + VM state size: 0 + Disk size: 67108864 + [1] + ID: 2 + Name: sn1 + Extra data size: 42 + VM state size: 0 + Disk size: 67108864 + Unknown extra data: very important data + +=== Reject too much unknown extra data === + +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Too much extra metadata in snapshot table entry 3 +You can force-remove this extra metadata with qemu-img check -r all + +qemu-img: ERROR failed to read the snapshot table: Too much extra metadata in snapshot table entry 3 +You can force-remove this extra metadata with qemu-img check -r all +qemu-img: Check failed: File too large + +Discarding too much extra metadata in snapshot table entry 3 (65536 > 1024) +ERROR cluster 10 refcount=0 reference=1 +Rebuilding refcount structure +Repairing cluster 1 refcount=1 reference=0 +Repairing cluster 2 refcount=1 reference=0 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 2 corruptions + +Double checking the fixed image now... +No errors were found on the image. + +=== Snapshot table too big === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Snapshot table is too big +You can force-remove all 1 overhanging snapshots with qemu-img check -r all + +qemu-img: ERROR failed to read the snapshot table: Snapshot table is too big +You can force-remove all 1 overhanging snapshots with qemu-img check -r all +qemu-img: Check failed: File too large + +Discarding 1 overhanging snapshots (snapshot table is too big) +Leaked cluster 14 refcount=1 reference=0 +Leaked cluster 15 refcount=1 reference=0 +Leaked cluster 1039 refcount=1 reference=0 +Leaked cluster 1040 refcount=1 reference=0 +Repairing cluster 14 refcount=1 reference=0 +Repairing cluster 15 refcount=1 reference=0 +Repairing cluster 1039 refcount=1 reference=0 +Repairing cluster 1040 refcount=1 reference=0 +The following inconsistencies were found and repaired: + + 4 leaked clusters + 1 corruptions + +Double checking the fixed image now... +No errors were found on the image. + +507 snapshots should remain: + qemu-img info reports 507 snapshots + Image header reports 507 snapshots + +=== Snapshot table too big with one entry with too much extra data === + +Snapshot table size should equal 67108872: 67108872 + +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Too much extra metadata in snapshot table entry 505 +You can force-remove this extra metadata with qemu-img check -r all + +qemu-img: ERROR failed to read the snapshot table: Too much extra metadata in snapshot table entry 505 +You can force-remove this extra metadata with qemu-img check -r all +qemu-img: Check failed: File too large + +Discarding too much extra metadata in snapshot table entry 505 (116944 > 1024) +Discarding 1 overhanging snapshots (snapshot table is too big) +Leaked cluster 1041 refcount=1 reference=0 +Leaked cluster 1042 refcount=1 reference=0 +Repairing cluster 1041 refcount=1 reference=0 +Repairing cluster 1042 refcount=1 reference=0 +The following inconsistencies were found and repaired: + + 2 leaked clusters + 2 corruptions + +Double checking the fixed image now... +No errors were found on the image. + +=== Too many snapshots === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Snapshot table too large + +qemu-img: ERROR snapshot table too large +You can force-remove all 65536 overhanging snapshots with qemu-img check -r all +qemu-img: Check failed: File too large + +Discarding 65536 overhanging snapshots +Leaked cluster 14 refcount=1 reference=0 +Leaked cluster 15 refcount=1 reference=0 +Leaked cluster 56 refcount=1 reference=0 +Leaked cluster 57 refcount=1 reference=0 +Leaked cluster 58 refcount=1 reference=0 +Leaked cluster 59 refcount=1 reference=0 +Leaked cluster 60 refcount=1 reference=0 +Leaked cluster 61 refcount=1 reference=0 +Leaked cluster 62 refcount=1 reference=0 +Leaked cluster 63 refcount=1 reference=0 +Leaked cluster 64 refcount=1 reference=0 +Leaked cluster 65 refcount=1 reference=0 +Leaked cluster 66 refcount=1 reference=0 +Leaked cluster 67 refcount=1 reference=0 +Leaked cluster 68 refcount=1 reference=0 +Leaked cluster 69 refcount=1 reference=0 +Leaked cluster 70 refcount=1 reference=0 +Leaked cluster 71 refcount=1 reference=0 +Leaked cluster 72 refcount=1 reference=0 +Leaked cluster 73 refcount=1 reference=0 +Leaked cluster 74 refcount=1 reference=0 +Leaked cluster 75 refcount=1 reference=0 +Leaked cluster 76 refcount=1 reference=0 +Leaked cluster 77 refcount=1 reference=0 +Leaked cluster 78 refcount=1 reference=0 +Leaked cluster 79 refcount=1 reference=0 +Leaked cluster 80 refcount=1 reference=0 +Leaked cluster 81 refcount=1 reference=0 +Leaked cluster 82 refcount=1 reference=0 +Leaked cluster 83 refcount=1 reference=0 +Leaked cluster 84 refcount=1 reference=0 +Leaked cluster 85 refcount=1 reference=0 +Leaked cluster 86 refcount=1 reference=0 +Leaked cluster 87 refcount=1 reference=0 +Leaked cluster 88 refcount=1 reference=0 +Leaked cluster 89 refcount=1 reference=0 +Leaked cluster 90 refcount=1 reference=0 +Leaked cluster 91 refcount=1 reference=0 +Leaked cluster 92 refcount=1 reference=0 +Leaked cluster 93 refcount=1 reference=0 +Leaked cluster 94 refcount=1 reference=0 +Leaked cluster 95 refcount=1 reference=0 +Repairing cluster 14 refcount=1 reference=0 +Repairing cluster 15 refcount=1 reference=0 +Repairing cluster 56 refcount=1 reference=0 +Repairing cluster 57 refcount=1 reference=0 +Repairing cluster 58 refcount=1 reference=0 +Repairing cluster 59 refcount=1 reference=0 +Repairing cluster 60 refcount=1 reference=0 +Repairing cluster 61 refcount=1 reference=0 +Repairing cluster 62 refcount=1 reference=0 +Repairing cluster 63 refcount=1 reference=0 +Repairing cluster 64 refcount=1 reference=0 +Repairing cluster 65 refcount=1 reference=0 +Repairing cluster 66 refcount=1 reference=0 +Repairing cluster 67 refcount=1 reference=0 +Repairing cluster 68 refcount=1 reference=0 +Repairing cluster 69 refcount=1 reference=0 +Repairing cluster 70 refcount=1 reference=0 +Repairing cluster 71 refcount=1 reference=0 +Repairing cluster 72 refcount=1 reference=0 +Repairing cluster 73 refcount=1 reference=0 +Repairing cluster 74 refcount=1 reference=0 +Repairing cluster 75 refcount=1 reference=0 +Repairing cluster 76 refcount=1 reference=0 +Repairing cluster 77 refcount=1 reference=0 +Repairing cluster 78 refcount=1 reference=0 +Repairing cluster 79 refcount=1 reference=0 +Repairing cluster 80 refcount=1 reference=0 +Repairing cluster 81 refcount=1 reference=0 +Repairing cluster 82 refcount=1 reference=0 +Repairing cluster 83 refcount=1 reference=0 +Repairing cluster 84 refcount=1 reference=0 +Repairing cluster 85 refcount=1 reference=0 +Repairing cluster 86 refcount=1 reference=0 +Repairing cluster 87 refcount=1 reference=0 +Repairing cluster 88 refcount=1 reference=0 +Repairing cluster 89 refcount=1 reference=0 +Repairing cluster 90 refcount=1 reference=0 +Repairing cluster 91 refcount=1 reference=0 +Repairing cluster 92 refcount=1 reference=0 +Repairing cluster 93 refcount=1 reference=0 +Repairing cluster 94 refcount=1 reference=0 +Repairing cluster 95 refcount=1 reference=0 +The following inconsistencies were found and repaired: + + 42 leaked clusters + 65536 corruptions + +Double checking the fixed image now... +No errors were found on the image. + +65536 snapshots should remain: + qemu-img info reports 65536 snapshots + Image header reports 65536 snapshots +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index af322af756..28871604cd 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -274,6 +274,7 @@ 257 rw 258 rw quick 260 rw quick +261 rw 262 rw quick migration 263 rw quick 264 rw From patchwork Mon Oct 28 12:14:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215471 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A2EF013B1 for ; Mon, 28 Oct 2019 13:01:29 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 76F2420717 for ; Mon, 28 Oct 2019 13:01:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ejmWN1eI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76F2420717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53504 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4eF-0000cT-TU for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:01:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51195) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xY-0003kf-Rf for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xX-0002NO-BW for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:20 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:43016 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xX-0002Lh-4i for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265036; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lP2YHcXJHDezvwbghFtZ159Ceu5JspkLq15PL35uTvM=; b=ejmWN1eI9hSxY9GomhLvtyvl/hnnfOysvmmLIjKSdjUK1SbLjIzDSbZY5+k4vWoT+Aaxm/ kJJYBnz4bbf24cfMklDnlQiYJtAsjsUTExh7HL8FMIp4aaWvD0/EtsWm2/QZLfrV8cbDLV UjNF4z75VYsd8x9ZsTHCrPhYXaGSbeE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-47-ou7QfFRhO4mMUFxGnrnZ4Q-1; Mon, 28 Oct 2019 08:17:12 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 027CF8017DD; Mon, 28 Oct 2019 12:17:12 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9DDB95C1D6; Mon, 28 Oct 2019 12:17:11 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 61/69] block: Handle filter truncation like native impl. Date: Mon, 28 Oct 2019 13:14:53 +0100 Message-Id: <20191028121501.15279-62-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: ou7QfFRhO4mMUFxGnrnZ4Q-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Make the filter truncation (passing it through to bs->file) a first-class citizen and handle it exactly as if it was the filter driver's native implementation of .bdrv_co_truncate(). I do not see a reason not to, it makes the code a bit shorter, and may be even more correct because this gets us to finish the write_req that we prepared before (may be important to e.g. bring dirty bitmaps to the correct size). Signed-off-by: Max Reitz Message-id: 20190918095144.955-2-mreitz@redhat.com Reviewed-by: Maxim Levitsky Signed-off-by: Max Reitz --- block/io.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/block/io.c b/block/io.c index f0b86c1d19..8ff3b47fb4 100644 --- a/block/io.c +++ b/block/io.c @@ -3347,20 +3347,19 @@ int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, goto out; } - if (!drv->bdrv_co_truncate) { - if (bs->file && drv->is_filter) { - ret = bdrv_co_truncate(bs->file, offset, prealloc, errp); - goto out; - } + if (drv->bdrv_co_truncate) { + ret = drv->bdrv_co_truncate(bs, offset, prealloc, errp); + } else if (bs->file && drv->is_filter) { + ret = bdrv_co_truncate(bs->file, offset, prealloc, errp); + } else { error_setg(errp, "Image format driver does not support resize"); ret = -ENOTSUP; goto out; } - - ret = drv->bdrv_co_truncate(bs, offset, prealloc, errp); if (ret < 0) { goto out; } + ret = refresh_total_sectors(bs, offset >> BDRV_SECTOR_BITS); if (ret < 0) { error_setg_errno(errp, -ret, "Could not refresh total sector count"); From patchwork Mon Oct 28 12:14:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215509 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 81F1E1390 for ; Mon, 28 Oct 2019 13:20:21 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 585BE20650 for ; Mon, 28 Oct 2019 13:20:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hrgeKGLM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 585BE20650 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53688 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4wW-0004Qs-6C for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:20:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51194) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xY-0003kU-Py for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xX-0002NV-DV for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:20 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:52095 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xX-0002MZ-5K for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265038; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cQcWinhkCWGpNGRCGIX778zf+/fAN8/KiI2OgoQdaJc=; b=hrgeKGLMoUhr1DUp2klZNjdsM+G2giu1NhjbC0iBsmfW+R45jhuADSvs2AX1UDfpwZhsYt U8SY7TfpjzbR1U+gKPhJID9wVWPjWX5v2D5a7SwFXHsoSDQ35c6GR9KK1KeJdT6jIBB0z7 vazRE+bIUpiPMJrvKm8rnnJyXf4OTog= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-245-ym_DuMvsM3-H66AlGGUwFg-1; Mon, 28 Oct 2019 08:17:14 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0F2FA180496E; Mon, 28 Oct 2019 12:17:14 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A952D6084E; Mon, 28 Oct 2019 12:17:13 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 62/69] block/cor: Drop cor_co_truncate() Date: Mon, 28 Oct 2019 13:14:54 +0100 Message-Id: <20191028121501.15279-63-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: ym_DuMvsM3-H66AlGGUwFg-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" No other filter driver has a .bdrv_co_truncate() implementation, and there is no need to because the general block layer code can handle it just as well. Signed-off-by: Max Reitz Message-id: 20190918095144.955-3-mreitz@redhat.com Reviewed-by: Maxim Levitsky Signed-off-by: Max Reitz --- block/copy-on-read.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index 6631f30205..e95223d3cb 100644 --- a/block/copy-on-read.c +++ b/block/copy-on-read.c @@ -73,13 +73,6 @@ static int64_t cor_getlength(BlockDriverState *bs) } -static int coroutine_fn cor_co_truncate(BlockDriverState *bs, int64_t offset, - PreallocMode prealloc, Error **errp) -{ - return bdrv_co_truncate(bs->file, offset, prealloc, errp); -} - - static int coroutine_fn cor_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) @@ -139,7 +132,6 @@ static BlockDriver bdrv_copy_on_read = { .bdrv_child_perm = cor_child_perm, .bdrv_getlength = cor_getlength, - .bdrv_co_truncate = cor_co_truncate, .bdrv_co_preadv = cor_co_preadv, .bdrv_co_pwritev = cor_co_pwritev, From patchwork Mon Oct 28 12:14:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215515 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E6B3912 for ; Mon, 28 Oct 2019 13:23:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0029C20663 for ; Mon, 28 Oct 2019 13:23:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fj4JF2ht" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0029C20663 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53712 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4zY-0001OU-D0 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:23:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51226) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xb-0003p1-5x for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xZ-0002Ot-82 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:22 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:39466 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xZ-0002OX-3u for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265040; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6aKuvnMVQFuDU56JNrd/dh0dSuS58g2T5cVoRy4wUis=; b=fj4JF2htImgh3yg94VK7OIt+SpNcjl5ma6VM+cSq7wdOL55b/jDkaZQ1U/ifxg1QDxJ9WT KwFz7LRwFX4y+tpABZR2tcMisUbEL94Duv/HNLS9e09u9A655yNlc5WM1q9Ye95P8mSW93 g+B4YoaJWOmIcBbo/0SFlGxgCLuOaDg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-389-3zlq06D3Or-0OHPcpcfeIw-1; Mon, 28 Oct 2019 08:17:17 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 18C8E800FF1; Mon, 28 Oct 2019 12:17:16 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B49A75DA2C; Mon, 28 Oct 2019 12:17:15 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 63/69] block: Do not truncate file node when formatting Date: Mon, 28 Oct 2019 13:14:55 +0100 Message-Id: <20191028121501.15279-64-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: 3zlq06D3Or-0OHPcpcfeIw-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" There is no reason why the format drivers need to truncate the protocol node when formatting it. When using the old .bdrv_co_create_ops() interface, the file will be created with no size option anyway, which generally gives it a size of 0. (Exceptions are block devices, which cannot be truncated anyway.) When using blockdev-create, the user must have given the file node some size anyway, so there is no reason why we should override that. qed is an exception, it needs the file to start completely empty (as explained by c743849bee7333c7ef256b7e12e34ed6f907064f). Signed-off-by: Max Reitz Message-id: 20190918095144.955-4-mreitz@redhat.com Reviewed-by: Maxim Levitsky Signed-off-by: Max Reitz --- block/parallels.c | 5 ----- block/qcow.c | 5 ----- block/qcow2.c | 6 ------ 3 files changed, 16 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index 7cd2714b69..905cac35c6 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -563,11 +563,6 @@ static int coroutine_fn parallels_co_create(BlockdevCreateOptions* opts, blk_set_allow_write_beyond_eof(blk, true); /* Create image format */ - ret = blk_truncate(blk, 0, PREALLOC_MODE_OFF, errp); - if (ret < 0) { - goto out; - } - bat_entries = DIV_ROUND_UP(total_size, cl_size); bat_sectors = DIV_ROUND_UP(bat_entry_off(bat_entries), cl_size); bat_sectors = (bat_sectors * cl_size) >> BDRV_SECTOR_BITS; diff --git a/block/qcow.c b/block/qcow.c index 5bdf72ba33..17705015ca 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -858,11 +858,6 @@ static int coroutine_fn qcow_co_create(BlockdevCreateOptions *opts, blk_set_allow_write_beyond_eof(qcow_blk, true); /* Create image format */ - ret = blk_truncate(qcow_blk, 0, PREALLOC_MODE_OFF, errp); - if (ret < 0) { - goto exit; - } - memset(&header, 0, sizeof(header)); header.magic = cpu_to_be32(QCOW_MAGIC); header.version = cpu_to_be32(QCOW_VERSION); diff --git a/block/qcow2.c b/block/qcow2.c index 8d4f38ae74..bf29d1c460 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3391,12 +3391,6 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) } blk_set_allow_write_beyond_eof(blk, true); - /* Clear the protocol layer and preallocate it if necessary */ - ret = blk_truncate(blk, 0, PREALLOC_MODE_OFF, errp); - if (ret < 0) { - goto out; - } - /* Write the header */ QEMU_BUILD_BUG_ON((1 << MIN_CLUSTER_BITS) < sizeof(*header)); header = g_malloc0(cluster_size); From patchwork Mon Oct 28 12:14:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215535 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 93BC014E5 for ; Mon, 28 Oct 2019 13:39:10 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 14F17214B2 for ; Mon, 28 Oct 2019 13:39:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XdnfeXDn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 14F17214B2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53858 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP5Ei-00082b-Sr for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:39:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51309) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xh-00042O-6P for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xd-0002R3-93 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:29 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:57487 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xd-0002Qm-4e for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265044; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QYQyIWhY/ug5VN7dxrRGA1VU17Sv0iTxSaaU09XaQyg=; b=XdnfeXDnLlRGmR35jsm9+s2wpHWwsNhjUq/XLAt/wWcPlMHziLic9b1Cjzcq7ttwQXh0F6 Ztg7HyuZ4inuAF9og8xrrSl8YdmVWuKtHu0y4az2NKP9WRIs+W3faxZmQuJ0JW6dee8++G tmojrox7sXsDr+AUiYlpu65jCKeVmVU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-39-_pRApR47M8CZw6xvQvtv5g-1; Mon, 28 Oct 2019 08:17:19 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 77E4D107AD28; Mon, 28 Oct 2019 12:17:18 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B98BA600C3; Mon, 28 Oct 2019 12:17:17 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 64/69] block: Add @exact parameter to bdrv_co_truncate() Date: Mon, 28 Oct 2019 13:14:56 +0100 Message-Id: <20191028121501.15279-65-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: _pRApR47M8CZw6xvQvtv5g-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We have two drivers (iscsi and file-posix) that (in some cases) return success from their .bdrv_co_truncate() implementation if the block device is larger than the requested offset, but cannot be shrunk. Some callers do not want that behavior, so this patch adds a new parameter that they can use to turn off that behavior. This patch just adds the parameter and lets the block/io.c and block/block-backend.c functions pass it around. All other callers always pass false and none of the implementations evaluate it, so that this patch does not change existing behavior. Future patches take care of that. Suggested-by: Maxim Levitsky Signed-off-by: Max Reitz Message-id: 20190918095144.955-5-mreitz@redhat.com Reviewed-by: Maxim Levitsky Signed-off-by: Max Reitz --- include/block/block.h | 6 +++--- include/block/block_int.h | 17 ++++++++++++++++- include/sysemu/block-backend.h | 4 ++-- block/block-backend.c | 6 +++--- block/commit.c | 5 +++-- block/crypto.c | 8 ++++---- block/file-posix.c | 3 ++- block/file-win32.c | 3 ++- block/gluster.c | 1 + block/io.c | 20 +++++++++++++------- block/iscsi.c | 3 ++- block/mirror.c | 4 ++-- block/nfs.c | 2 +- block/parallels.c | 6 +++--- block/qcow.c | 4 ++-- block/qcow2-refcount.c | 2 +- block/qcow2.c | 22 ++++++++++++---------- block/qed.c | 3 ++- block/raw-format.c | 5 +++-- block/rbd.c | 1 + block/sheepdog.c | 5 +++-- block/ssh.c | 3 ++- block/vdi.c | 2 +- block/vhdx-log.c | 4 ++-- block/vhdx.c | 7 ++++--- block/vmdk.c | 8 ++++---- block/vpc.c | 2 +- blockdev.c | 2 +- qemu-img.c | 2 +- qemu-io-cmds.c | 2 +- tests/test-block-iothread.c | 8 ++++---- 31 files changed, 102 insertions(+), 68 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index 89606bd9f8..1df9848e74 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -346,10 +346,10 @@ BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs, const char *backing_file); void bdrv_refresh_filename(BlockDriverState *bs); -int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, +int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, bool exact, PreallocMode prealloc, Error **errp); -int bdrv_truncate(BdrvChild *child, int64_t offset, PreallocMode prealloc, - Error **errp); +int bdrv_truncate(BdrvChild *child, int64_t offset, bool exact, + PreallocMode prealloc, Error **errp); int64_t bdrv_nb_sectors(BlockDriverState *bs); int64_t bdrv_getlength(BlockDriverState *bs); diff --git a/include/block/block_int.h b/include/block/block_int.h index ca4ccac4c1..02dc0034a2 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -334,8 +334,23 @@ struct BlockDriver { * bdrv_parse_filename. */ const char *protocol_name; + + /* + * Truncate @bs to @offset bytes using the given @prealloc mode + * when growing. Modes other than PREALLOC_MODE_OFF should be + * rejected when shrinking @bs. + * + * If @exact is true, @bs must be resized to exactly @offset. + * Otherwise, it is sufficient for @bs (if it is a host block + * device and thus there is no way to resize it) to be at least + * @offset bytes in length. + * + * If @exact is true and this function fails but would succeed + * with @exact = false, it should return -ENOTSUP. + */ int coroutine_fn (*bdrv_co_truncate)(BlockDriverState *bs, int64_t offset, - PreallocMode prealloc, Error **errp); + bool exact, PreallocMode prealloc, + Error **errp); int64_t (*bdrv_getlength)(BlockDriverState *bs); bool has_variable_length; diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h index 73f2cef7fe..b198deca0b 100644 --- a/include/sysemu/block-backend.h +++ b/include/sysemu/block-backend.h @@ -237,8 +237,8 @@ int coroutine_fn blk_co_pwrite_zeroes(BlockBackend *blk, int64_t offset, int bytes, BdrvRequestFlags flags); int blk_pwrite_compressed(BlockBackend *blk, int64_t offset, const void *buf, int bytes); -int blk_truncate(BlockBackend *blk, int64_t offset, PreallocMode prealloc, - Error **errp); +int blk_truncate(BlockBackend *blk, int64_t offset, bool exact, + PreallocMode prealloc, Error **errp); int blk_pdiscard(BlockBackend *blk, int64_t offset, int bytes); int blk_save_vmstate(BlockBackend *blk, const uint8_t *buf, int64_t pos, int size); diff --git a/block/block-backend.c b/block/block-backend.c index 912c50678d..8b8f2a80a0 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -2072,15 +2072,15 @@ int blk_pwrite_compressed(BlockBackend *blk, int64_t offset, const void *buf, BDRV_REQ_WRITE_COMPRESSED); } -int blk_truncate(BlockBackend *blk, int64_t offset, PreallocMode prealloc, - Error **errp) +int blk_truncate(BlockBackend *blk, int64_t offset, bool exact, + PreallocMode prealloc, Error **errp) { if (!blk_is_available(blk)) { error_setg(errp, "No medium inserted"); return -ENOMEDIUM; } - return bdrv_truncate(blk->root, offset, prealloc, errp); + return bdrv_truncate(blk->root, offset, exact, prealloc, errp); } static void blk_pdiscard_entry(void *opaque) diff --git a/block/commit.c b/block/commit.c index bc8454463d..23c90b3b91 100644 --- a/block/commit.c +++ b/block/commit.c @@ -155,7 +155,7 @@ static int coroutine_fn commit_run(Job *job, Error **errp) } if (base_len < len) { - ret = blk_truncate(s->base, len, PREALLOC_MODE_OFF, NULL); + ret = blk_truncate(s->base, len, false, PREALLOC_MODE_OFF, NULL); if (ret) { goto out; } @@ -471,7 +471,8 @@ int bdrv_commit(BlockDriverState *bs) * grow the backing file image if possible. If not possible, * we must return an error */ if (length > backing_length) { - ret = blk_truncate(backing, length, PREALLOC_MODE_OFF, &local_err); + ret = blk_truncate(backing, length, false, PREALLOC_MODE_OFF, + &local_err); if (ret < 0) { error_report_err(local_err); goto ro_cleanup; diff --git a/block/crypto.c b/block/crypto.c index 7eb698774e..e5a1a2cdf3 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -113,8 +113,8 @@ static ssize_t block_crypto_init_func(QCryptoBlock *block, * available to the guest, so we must take account of that * which will be used by the crypto header */ - return blk_truncate(data->blk, data->size + headerlen, data->prealloc, - errp); + return blk_truncate(data->blk, data->size + headerlen, false, + data->prealloc, errp); } @@ -297,7 +297,7 @@ static int block_crypto_co_create_generic(BlockDriverState *bs, } static int coroutine_fn -block_crypto_co_truncate(BlockDriverState *bs, int64_t offset, +block_crypto_co_truncate(BlockDriverState *bs, int64_t offset, bool exact, PreallocMode prealloc, Error **errp) { BlockCrypto *crypto = bs->opaque; @@ -311,7 +311,7 @@ block_crypto_co_truncate(BlockDriverState *bs, int64_t offset, offset += payload_offset; - return bdrv_co_truncate(bs->file, offset, prealloc, errp); + return bdrv_co_truncate(bs->file, offset, false, prealloc, errp); } static void block_crypto_close(BlockDriverState *bs) diff --git a/block/file-posix.c b/block/file-posix.c index 695fcf740d..a3e8a8aa70 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -2019,7 +2019,8 @@ raw_regular_truncate(BlockDriverState *bs, int fd, int64_t offset, } static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, - PreallocMode prealloc, Error **errp) + bool exact, PreallocMode prealloc, + Error **errp) { BDRVRawState *s = bs->opaque; struct stat st; diff --git a/block/file-win32.c b/block/file-win32.c index 41f55dfece..77e8ff7b68 100644 --- a/block/file-win32.c +++ b/block/file-win32.c @@ -468,7 +468,8 @@ static void raw_close(BlockDriverState *bs) } static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, - PreallocMode prealloc, Error **errp) + bool exact, PreallocMode prealloc, + Error **errp) { BDRVRawState *s = bs->opaque; LONG low, high; diff --git a/block/gluster.c b/block/gluster.c index 64028b2cba..4fa4a77a47 100644 --- a/block/gluster.c +++ b/block/gluster.c @@ -1225,6 +1225,7 @@ static coroutine_fn int qemu_gluster_co_rw(BlockDriverState *bs, static coroutine_fn int qemu_gluster_co_truncate(BlockDriverState *bs, int64_t offset, + bool exact, PreallocMode prealloc, Error **errp) { diff --git a/block/io.c b/block/io.c index 8ff3b47fb4..6e2b59802e 100644 --- a/block/io.c +++ b/block/io.c @@ -3291,8 +3291,12 @@ static void bdrv_parent_cb_resize(BlockDriverState *bs) /** * Truncate file to 'offset' bytes (needed only for file protocols) + * + * If 'exact' is true, the file must be resized to exactly the given + * 'offset'. Otherwise, it is sufficient for the node to be at least + * 'offset' bytes in length. */ -int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, +int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, bool exact, PreallocMode prealloc, Error **errp) { BlockDriverState *bs = child->bs; @@ -3348,9 +3352,9 @@ int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, } if (drv->bdrv_co_truncate) { - ret = drv->bdrv_co_truncate(bs, offset, prealloc, errp); + ret = drv->bdrv_co_truncate(bs, offset, exact, prealloc, errp); } else if (bs->file && drv->is_filter) { - ret = bdrv_co_truncate(bs->file, offset, prealloc, errp); + ret = bdrv_co_truncate(bs->file, offset, exact, prealloc, errp); } else { error_setg(errp, "Image format driver does not support resize"); ret = -ENOTSUP; @@ -3381,6 +3385,7 @@ out: typedef struct TruncateCo { BdrvChild *child; int64_t offset; + bool exact; PreallocMode prealloc; Error **errp; int ret; @@ -3389,18 +3394,19 @@ typedef struct TruncateCo { static void coroutine_fn bdrv_truncate_co_entry(void *opaque) { TruncateCo *tco = opaque; - tco->ret = bdrv_co_truncate(tco->child, tco->offset, tco->prealloc, - tco->errp); + tco->ret = bdrv_co_truncate(tco->child, tco->offset, tco->exact, + tco->prealloc, tco->errp); aio_wait_kick(); } -int bdrv_truncate(BdrvChild *child, int64_t offset, PreallocMode prealloc, - Error **errp) +int bdrv_truncate(BdrvChild *child, int64_t offset, bool exact, + PreallocMode prealloc, Error **errp) { Coroutine *co; TruncateCo tco = { .child = child, .offset = offset, + .exact = exact, .prealloc = prealloc, .errp = errp, .ret = NOT_DONE, diff --git a/block/iscsi.c b/block/iscsi.c index 2ced15066a..677946cf09 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -2123,7 +2123,8 @@ static void iscsi_reopen_commit(BDRVReopenState *reopen_state) } static int coroutine_fn iscsi_co_truncate(BlockDriverState *bs, int64_t offset, - PreallocMode prealloc, Error **errp) + bool exact, PreallocMode prealloc, + Error **errp) { IscsiLun *iscsilun = bs->opaque; Error *local_err = NULL; diff --git a/block/mirror.c b/block/mirror.c index bb17cfce31..f0f2d9dff1 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -878,8 +878,8 @@ static int coroutine_fn mirror_run(Job *job, Error **errp) } if (s->bdev_length > base_length) { - ret = blk_truncate(s->target, s->bdev_length, PREALLOC_MODE_OFF, - NULL); + ret = blk_truncate(s->target, s->bdev_length, false, + PREALLOC_MODE_OFF, NULL); if (ret < 0) { goto immediate_exit; } diff --git a/block/nfs.c b/block/nfs.c index 40f23495a0..9a6311e270 100644 --- a/block/nfs.c +++ b/block/nfs.c @@ -752,7 +752,7 @@ static int64_t nfs_get_allocated_file_size(BlockDriverState *bs) } static int coroutine_fn -nfs_file_co_truncate(BlockDriverState *bs, int64_t offset, +nfs_file_co_truncate(BlockDriverState *bs, int64_t offset, bool exact, PreallocMode prealloc, Error **errp) { NFSClient *client = bs->opaque; diff --git a/block/parallels.c b/block/parallels.c index 905cac35c6..a1a92c97a4 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -203,7 +203,7 @@ static int64_t allocate_clusters(BlockDriverState *bs, int64_t sector_num, } else { ret = bdrv_truncate(bs->file, (s->data_end + space) << BDRV_SECTOR_BITS, - PREALLOC_MODE_OFF, NULL); + false, PREALLOC_MODE_OFF, NULL); } if (ret < 0) { return ret; @@ -487,7 +487,7 @@ static int coroutine_fn parallels_co_check(BlockDriverState *bs, res->leaks += count; if (fix & BDRV_FIX_LEAKS) { Error *local_err = NULL; - ret = bdrv_truncate(bs->file, res->image_end_offset, + ret = bdrv_truncate(bs->file, res->image_end_offset, false, PREALLOC_MODE_OFF, &local_err); if (ret < 0) { error_report_err(local_err); @@ -880,7 +880,7 @@ static void parallels_close(BlockDriverState *bs) if ((bs->open_flags & BDRV_O_RDWR) && !(bs->open_flags & BDRV_O_INACTIVE)) { s->header->inuse = 0; parallels_update_header(bs); - bdrv_truncate(bs->file, s->data_end << BDRV_SECTOR_BITS, + bdrv_truncate(bs->file, s->data_end << BDRV_SECTOR_BITS, false, PREALLOC_MODE_OFF, NULL); } diff --git a/block/qcow.c b/block/qcow.c index 17705015ca..fce8989868 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -480,7 +480,7 @@ static int get_cluster_offset(BlockDriverState *bs, return -E2BIG; } ret = bdrv_truncate(bs->file, cluster_offset + s->cluster_size, - PREALLOC_MODE_OFF, NULL); + false, PREALLOC_MODE_OFF, NULL); if (ret < 0) { return ret; } @@ -1033,7 +1033,7 @@ static int qcow_make_empty(BlockDriverState *bs) if (bdrv_pwrite_sync(bs->file, s->l1_table_offset, s->l1_table, l1_length) < 0) return -1; - ret = bdrv_truncate(bs->file, s->l1_table_offset + l1_length, + ret = bdrv_truncate(bs->file, s->l1_table_offset + l1_length, false, PREALLOC_MODE_OFF, NULL); if (ret < 0) return ret; diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 0d64bf5a5e..f67ac6b2d8 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -2016,7 +2016,7 @@ static int check_refblocks(BlockDriverState *bs, BdrvCheckResult *res, goto resize_fail; } - ret = bdrv_truncate(bs->file, offset + s->cluster_size, + ret = bdrv_truncate(bs->file, offset + s->cluster_size, false, PREALLOC_MODE_OFF, &local_err); if (ret < 0) { error_report_err(local_err); diff --git a/block/qcow2.c b/block/qcow2.c index bf29d1c460..de284ad803 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3074,8 +3074,8 @@ static int coroutine_fn preallocate_co(BlockDriverState *bs, uint64_t offset, if (mode == PREALLOC_MODE_METADATA) { mode = PREALLOC_MODE_OFF; } - ret = bdrv_co_truncate(s->data_file, host_offset + cur_bytes, mode, - errp); + ret = bdrv_co_truncate(s->data_file, host_offset + cur_bytes, false, + mode, errp); if (ret < 0) { return ret; } @@ -3489,7 +3489,8 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) } /* Okay, now that we have a valid image, let's give it the right size */ - ret = blk_truncate(blk, qcow2_opts->size, qcow2_opts->preallocation, errp); + ret = blk_truncate(blk, qcow2_opts->size, false, qcow2_opts->preallocation, + errp); if (ret < 0) { error_prepend(errp, "Could not resize image: "); goto out; @@ -3937,7 +3938,8 @@ fail: } static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, - PreallocMode prealloc, Error **errp) + bool exact, PreallocMode prealloc, + Error **errp) { BDRVQcow2State *s = bs->opaque; uint64_t old_length; @@ -4026,7 +4028,7 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, Error *local_err = NULL; bdrv_co_truncate(bs->file, (last_cluster + 1) * s->cluster_size, - PREALLOC_MODE_OFF, &local_err); + false, PREALLOC_MODE_OFF, &local_err); if (local_err) { warn_reportf_err(local_err, "Failed to truncate the tail of the image: "); @@ -4043,7 +4045,7 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, switch (prealloc) { case PREALLOC_MODE_OFF: if (has_data_file(bs)) { - ret = bdrv_co_truncate(s->data_file, offset, prealloc, errp); + ret = bdrv_co_truncate(s->data_file, offset, false, prealloc, errp); if (ret < 0) { goto fail; } @@ -4128,7 +4130,7 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, /* Allocate the data area */ new_file_size = allocation_start + nb_new_data_clusters * s->cluster_size; - ret = bdrv_co_truncate(bs->file, new_file_size, prealloc, errp); + ret = bdrv_co_truncate(bs->file, new_file_size, false, prealloc, errp); if (ret < 0) { error_prepend(errp, "Failed to resize underlying file: "); qcow2_free_clusters(bs, allocation_start, @@ -4231,7 +4233,7 @@ qcow2_co_pwritev_compressed_part(BlockDriverState *bs, if (len < 0) { return len; } - return bdrv_co_truncate(bs->file, len, PREALLOC_MODE_OFF, NULL); + return bdrv_co_truncate(bs->file, len, false, PREALLOC_MODE_OFF, NULL); } if (offset_into_cluster(s, offset)) { @@ -4468,7 +4470,7 @@ static int make_completely_empty(BlockDriverState *bs) goto fail; } - ret = bdrv_truncate(bs->file, (3 + l1_clusters) * s->cluster_size, + ret = bdrv_truncate(bs->file, (3 + l1_clusters) * s->cluster_size, false, PREALLOC_MODE_OFF, &local_err); if (ret < 0) { error_report_err(local_err); @@ -5308,7 +5310,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, return ret; } - ret = blk_truncate(blk, new_size, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, new_size, false, PREALLOC_MODE_OFF, errp); blk_unref(blk); if (ret < 0) { return ret; diff --git a/block/qed.c b/block/qed.c index 0d8fd507aa..7c2a65af40 100644 --- a/block/qed.c +++ b/block/qed.c @@ -674,7 +674,7 @@ static int coroutine_fn bdrv_qed_co_create(BlockdevCreateOptions *opts, l1_size = header.cluster_size * header.table_size; /* File must start empty and grow, check truncate is supported */ - ret = blk_truncate(blk, 0, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, 0, false, PREALLOC_MODE_OFF, errp); if (ret < 0) { goto out; } @@ -1461,6 +1461,7 @@ static int coroutine_fn bdrv_qed_co_pwrite_zeroes(BlockDriverState *bs, static int coroutine_fn bdrv_qed_co_truncate(BlockDriverState *bs, int64_t offset, + bool exact, PreallocMode prealloc, Error **errp) { diff --git a/block/raw-format.c b/block/raw-format.c index 42c28cc29a..57d84d0bae 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -370,7 +370,8 @@ static void raw_refresh_limits(BlockDriverState *bs, Error **errp) } static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, - PreallocMode prealloc, Error **errp) + bool exact, PreallocMode prealloc, + Error **errp) { BDRVRawState *s = bs->opaque; @@ -386,7 +387,7 @@ static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, s->size = offset; offset += s->offset; - return bdrv_co_truncate(bs->file, offset, prealloc, errp); + return bdrv_co_truncate(bs->file, offset, false, prealloc, errp); } static void raw_eject(BlockDriverState *bs, bool eject_flag) diff --git a/block/rbd.c b/block/rbd.c index c71e45d7c3..027cbcc695 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -1087,6 +1087,7 @@ static int64_t qemu_rbd_getlength(BlockDriverState *bs) static int coroutine_fn qemu_rbd_co_truncate(BlockDriverState *bs, int64_t offset, + bool exact, PreallocMode prealloc, Error **errp) { diff --git a/block/sheepdog.c b/block/sheepdog.c index 773dfc6ab1..cfa84338a2 100644 --- a/block/sheepdog.c +++ b/block/sheepdog.c @@ -2285,7 +2285,8 @@ static int64_t sd_getlength(BlockDriverState *bs) } static int coroutine_fn sd_co_truncate(BlockDriverState *bs, int64_t offset, - PreallocMode prealloc, Error **errp) + bool exact, PreallocMode prealloc, + Error **errp) { BDRVSheepdogState *s = bs->opaque; int ret, fd; @@ -2601,7 +2602,7 @@ static coroutine_fn int sd_co_writev(BlockDriverState *bs, int64_t sector_num, assert(!flags); if (offset > s->inode.vdi_size) { - ret = sd_co_truncate(bs, offset, PREALLOC_MODE_OFF, NULL); + ret = sd_co_truncate(bs, offset, false, PREALLOC_MODE_OFF, NULL); if (ret < 0) { return ret; } diff --git a/block/ssh.c b/block/ssh.c index 84d01e892b..b4375cf7d2 100644 --- a/block/ssh.c +++ b/block/ssh.c @@ -1295,7 +1295,8 @@ static int64_t ssh_getlength(BlockDriverState *bs) } static int coroutine_fn ssh_co_truncate(BlockDriverState *bs, int64_t offset, - PreallocMode prealloc, Error **errp) + bool exact, PreallocMode prealloc, + Error **errp) { BDRVSSHState *s = bs->opaque; diff --git a/block/vdi.c b/block/vdi.c index 806ba7f53c..0142da7233 100644 --- a/block/vdi.c +++ b/block/vdi.c @@ -874,7 +874,7 @@ static int coroutine_fn vdi_co_do_create(BlockdevCreateOptions *create_options, } if (image_type == VDI_TYPE_STATIC) { - ret = blk_truncate(blk, offset + blocks * block_size, + ret = blk_truncate(blk, offset + blocks * block_size, false, PREALLOC_MODE_OFF, errp); if (ret < 0) { error_prepend(errp, "Failed to statically allocate file"); diff --git a/block/vhdx-log.c b/block/vhdx-log.c index fdd3a7adc3..13a49c2a33 100644 --- a/block/vhdx-log.c +++ b/block/vhdx-log.c @@ -557,8 +557,8 @@ static int vhdx_log_flush(BlockDriverState *bs, BDRVVHDXState *s, ret = -EINVAL; goto exit; } - ret = bdrv_truncate(bs->file, new_file_size, PREALLOC_MODE_OFF, - NULL); + ret = bdrv_truncate(bs->file, new_file_size, false, + PREALLOC_MODE_OFF, NULL); if (ret < 0) { goto exit; } diff --git a/block/vhdx.c b/block/vhdx.c index 371f226286..f02d2611be 100644 --- a/block/vhdx.c +++ b/block/vhdx.c @@ -1263,7 +1263,7 @@ static int vhdx_allocate_block(BlockDriverState *bs, BDRVVHDXState *s, return -EINVAL; } - return bdrv_truncate(bs->file, *new_offset + s->block_size, + return bdrv_truncate(bs->file, *new_offset + s->block_size, false, PREALLOC_MODE_OFF, NULL); } @@ -1702,12 +1702,13 @@ static int vhdx_create_bat(BlockBackend *blk, BDRVVHDXState *s, if (type == VHDX_TYPE_DYNAMIC) { /* All zeroes, so we can just extend the file - the end of the BAT * is the furthest thing we have written yet */ - ret = blk_truncate(blk, data_file_offset, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, data_file_offset, false, PREALLOC_MODE_OFF, + errp); if (ret < 0) { goto exit; } } else if (type == VHDX_TYPE_FIXED) { - ret = blk_truncate(blk, data_file_offset + image_size, + ret = blk_truncate(blk, data_file_offset + image_size, false, PREALLOC_MODE_OFF, errp); if (ret < 0) { goto exit; diff --git a/block/vmdk.c b/block/vmdk.c index fed3b50c8a..20e909d997 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -2076,7 +2076,7 @@ vmdk_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, return length; } length = QEMU_ALIGN_UP(length, BDRV_SECTOR_SIZE); - ret = bdrv_truncate(s->extents[i].file, length, + ret = bdrv_truncate(s->extents[i].file, length, false, PREALLOC_MODE_OFF, NULL); if (ret < 0) { return ret; @@ -2118,7 +2118,7 @@ static int vmdk_init_extent(BlockBackend *blk, int gd_buf_size; if (flat) { - ret = blk_truncate(blk, filesize, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, filesize, false, PREALLOC_MODE_OFF, errp); goto exit; } magic = cpu_to_be32(VMDK4_MAGIC); @@ -2181,7 +2181,7 @@ static int vmdk_init_extent(BlockBackend *blk, goto exit; } - ret = blk_truncate(blk, le64_to_cpu(header.grain_offset) << 9, + ret = blk_truncate(blk, le64_to_cpu(header.grain_offset) << 9, false, PREALLOC_MODE_OFF, errp); if (ret < 0) { goto exit; @@ -2523,7 +2523,7 @@ static int coroutine_fn vmdk_co_do_create(int64_t size, /* bdrv_pwrite write padding zeros to align to sector, we don't need that * for description file */ if (desc_offset == 0) { - ret = blk_truncate(blk, desc_len, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, desc_len, false, PREALLOC_MODE_OFF, errp); if (ret < 0) { goto exit; } diff --git a/block/vpc.c b/block/vpc.c index 5cd3890780..a65550298e 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -898,7 +898,7 @@ static int create_fixed_disk(BlockBackend *blk, uint8_t *buf, /* Add footer to total size */ total_size += HEADER_SIZE; - ret = blk_truncate(blk, total_size, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, total_size, false, PREALLOC_MODE_OFF, errp); if (ret < 0) { return ret; } diff --git a/blockdev.c b/blockdev.c index ba491e3ef5..8e029e9c01 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3204,7 +3204,7 @@ void qmp_block_resize(bool has_device, const char *device, } bdrv_drained_begin(bs); - ret = blk_truncate(blk, size, PREALLOC_MODE_OFF, errp); + ret = blk_truncate(blk, size, false, PREALLOC_MODE_OFF, errp); bdrv_drained_end(bs); out: diff --git a/qemu-img.c b/qemu-img.c index 8b03ef8171..c738297b04 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -3831,7 +3831,7 @@ static int img_resize(int argc, char **argv) } } - ret = blk_truncate(blk, total_size, prealloc, &err); + ret = blk_truncate(blk, total_size, false, prealloc, &err); if (ret < 0) { error_report_err(err); goto out; diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c index 349256a5fe..5e9017c979 100644 --- a/qemu-io-cmds.c +++ b/qemu-io-cmds.c @@ -1710,7 +1710,7 @@ static int truncate_f(BlockBackend *blk, int argc, char **argv) return offset; } - ret = blk_truncate(blk, offset, PREALLOC_MODE_OFF, &local_err); + ret = blk_truncate(blk, offset, false, PREALLOC_MODE_OFF, &local_err); if (ret < 0) { error_report_err(local_err); return ret; diff --git a/tests/test-block-iothread.c b/tests/test-block-iothread.c index cfe30bab21..0c861809f0 100644 --- a/tests/test-block-iothread.c +++ b/tests/test-block-iothread.c @@ -45,7 +45,7 @@ static int coroutine_fn bdrv_test_co_pdiscard(BlockDriverState *bs, } static int coroutine_fn -bdrv_test_co_truncate(BlockDriverState *bs, int64_t offset, +bdrv_test_co_truncate(BlockDriverState *bs, int64_t offset, bool exact, PreallocMode prealloc, Error **errp) { return 0; @@ -185,18 +185,18 @@ static void test_sync_op_truncate(BdrvChild *c) int ret; /* Normal success path */ - ret = bdrv_truncate(c, 65536, PREALLOC_MODE_OFF, NULL); + ret = bdrv_truncate(c, 65536, false, PREALLOC_MODE_OFF, NULL); g_assert_cmpint(ret, ==, 0); /* Early error: Negative offset */ - ret = bdrv_truncate(c, -2, PREALLOC_MODE_OFF, NULL); + ret = bdrv_truncate(c, -2, false, PREALLOC_MODE_OFF, NULL); g_assert_cmpint(ret, ==, -EINVAL); /* Error: Read-only image */ c->bs->read_only = true; c->bs->open_flags &= ~BDRV_O_RDWR; - ret = bdrv_truncate(c, 65536, PREALLOC_MODE_OFF, NULL); + ret = bdrv_truncate(c, 65536, false, PREALLOC_MODE_OFF, NULL); g_assert_cmpint(ret, ==, -EACCES); c->bs->read_only = false; From patchwork Mon Oct 28 12:14:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215531 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1ED03112B for ; Mon, 28 Oct 2019 13:35:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E7DC8214D9 for ; Mon, 28 Oct 2019 13:35:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="OkWUfyKI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E7DC8214D9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP5BG-0002Z6-Ku for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:35:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51318) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xh-00043E-I7 for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xg-0002Sl-3l for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:29 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:43471) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xf-0002SK-VZ for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265047; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4yoWAkbu15EIciX01QYDu36TS21E+/DI2SOJps+PMEs=; b=OkWUfyKIXIqVXwadBnA4eK6/7mb5qqa2tth23kqohnVKgK4krGqw4jEOgmWQUeLfzeF2ue baEjz7+xflGoSoQpvJ/PeQgnmT1+sWsts3LVi/iUOvEXdv9aNJSQuzqpg4oETjcsPcBgET v3s7ZHTG/sKFpewgqRF+TYk3WtZXEz4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-20-jwUhd48OOceWIe6y3FY-vA-1; Mon, 28 Oct 2019 08:17:21 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7CA7A476; Mon, 28 Oct 2019 12:17:20 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2176C26186; Mon, 28 Oct 2019 12:17:19 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 65/69] block: Evaluate @exact in protocol drivers Date: Mon, 28 Oct 2019 13:14:57 +0100 Message-Id: <20191028121501.15279-66-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: jwUhd48OOceWIe6y3FY-vA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We have two protocol drivers that return success when trying to shrink a block device even though they cannot shrink it. This behavior is now only allowed with exact=false, so they should return an error with exact=true. Signed-off-by: Max Reitz Message-id: 20190918095144.955-6-mreitz@redhat.com Reviewed-by: Maxim Levitsky Signed-off-by: Max Reitz --- block/file-posix.c | 8 +++++++- block/iscsi.c | 7 ++++++- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/block/file-posix.c b/block/file-posix.c index a3e8a8aa70..e0ea1a7446 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -2033,6 +2033,7 @@ static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, } if (S_ISREG(st.st_mode)) { + /* Always resizes to the exact @offset */ return raw_regular_truncate(bs, s->fd, offset, prealloc, errp); } @@ -2043,7 +2044,12 @@ static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, } if (S_ISCHR(st.st_mode) || S_ISBLK(st.st_mode)) { - if (offset > raw_getlength(bs)) { + int64_t cur_length = raw_getlength(bs); + + if (offset != cur_length && exact) { + error_setg(errp, "Cannot resize device files"); + return -ENOTSUP; + } else if (offset > cur_length) { error_setg(errp, "Cannot grow device files"); return -EINVAL; } diff --git a/block/iscsi.c b/block/iscsi.c index 677946cf09..2aea7e3f13 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -2127,6 +2127,7 @@ static int coroutine_fn iscsi_co_truncate(BlockDriverState *bs, int64_t offset, Error **errp) { IscsiLun *iscsilun = bs->opaque; + int64_t cur_length; Error *local_err = NULL; if (prealloc != PREALLOC_MODE_OFF) { @@ -2146,7 +2147,11 @@ static int coroutine_fn iscsi_co_truncate(BlockDriverState *bs, int64_t offset, return -EIO; } - if (offset > iscsi_getlength(bs)) { + cur_length = iscsi_getlength(bs); + if (offset != cur_length && exact) { + error_setg(errp, "Cannot resize iSCSI devices"); + return -ENOTSUP; + } else if (offset > cur_length) { error_setg(errp, "Cannot grow iSCSI devices"); return -EINVAL; } From patchwork Mon Oct 28 12:14:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215525 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DCE1514DB for ; Mon, 28 Oct 2019 13:30:45 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B1AEE20663 for ; Mon, 28 Oct 2019 13:30:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CBctYie2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B1AEE20663 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53776 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP56a-0004EK-GQ for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:30:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51279) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xf-0003zT-Rc for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xe-0002Rh-EF for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:27 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:51987 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xe-0002RT-9M for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265045; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0fkbcTi961hq47nQeUqVstg0lUGdtnHyS6F1r2aTKUI=; b=CBctYie2Ejjt00uJnkHlswFBfeAX38cXxG50aKoFHf5J0LSfAYe45wfy4KYja/y/uEL3du zWkdsyVH2ITkICfVXiua9iGLTKFT8V8eeP/hY9WZJFZxQmy3zmuRlw8I2N+o91uCHtsAgG lFD14vlnV771YAvVIOgettxwylmTrCM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-121-Hh0Z4yNcNwuqy63uq4wlHQ-1; Mon, 28 Oct 2019 08:17:23 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7A97610052E0; Mon, 28 Oct 2019 12:17:22 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1D0916085E; Mon, 28 Oct 2019 12:17:21 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 66/69] block: Let format drivers pass @exact Date: Mon, 28 Oct 2019 13:14:58 +0100 Message-Id: <20191028121501.15279-67-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: Hh0Z4yNcNwuqy63uq4wlHQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" When truncating a format node, the @exact parameter is generally handled simply by virtue of the format storing the new size in the image metadata. Such formats do not need to pass on the parameter to their file nodes. There are exceptions, though: - raw and crypto cannot store the image size, and thus must pass on @exact. - When using qcow2 with an external data file, it just makes sense to keep its size in sync with the qcow2 virtual disk (because the external data file is the virtual disk). Therefore, we should pass @exact when truncating it. Signed-off-by: Max Reitz Message-id: 20190918095144.955-7-mreitz@redhat.com Reviewed-by: Maxim Levitsky Signed-off-by: Max Reitz --- block/crypto.c | 2 +- block/qcow2.c | 15 ++++++++++++++- block/raw-format.c | 2 +- 3 files changed, 16 insertions(+), 3 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index e5a1a2cdf3..24823835c1 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -311,7 +311,7 @@ block_crypto_co_truncate(BlockDriverState *bs, int64_t offset, bool exact, offset += payload_offset; - return bdrv_co_truncate(bs->file, offset, false, prealloc, errp); + return bdrv_co_truncate(bs->file, offset, exact, prealloc, errp); } static void block_crypto_close(BlockDriverState *bs) diff --git a/block/qcow2.c b/block/qcow2.c index de284ad803..9f32dae41f 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -4027,6 +4027,13 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, if ((last_cluster + 1) * s->cluster_size < old_file_size) { Error *local_err = NULL; + /* + * Do not pass @exact here: It will not help the user if + * we get an error here just because they wanted to shrink + * their qcow2 image (on a block device) with qemu-img. + * (And on the qcow2 layer, the @exact requirement is + * always fulfilled, so there is no need to pass it on.) + */ bdrv_co_truncate(bs->file, (last_cluster + 1) * s->cluster_size, false, PREALLOC_MODE_OFF, &local_err); if (local_err) { @@ -4045,7 +4052,12 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, switch (prealloc) { case PREALLOC_MODE_OFF: if (has_data_file(bs)) { - ret = bdrv_co_truncate(s->data_file, offset, false, prealloc, errp); + /* + * If the caller wants an exact resize, the external data + * file should be resized to the exact target size, too, + * so we pass @exact here. + */ + ret = bdrv_co_truncate(s->data_file, offset, exact, prealloc, errp); if (ret < 0) { goto fail; } @@ -4130,6 +4142,7 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, /* Allocate the data area */ new_file_size = allocation_start + nb_new_data_clusters * s->cluster_size; + /* Image file grows, so @exact does not matter */ ret = bdrv_co_truncate(bs->file, new_file_size, false, prealloc, errp); if (ret < 0) { error_prepend(errp, "Failed to resize underlying file: "); diff --git a/block/raw-format.c b/block/raw-format.c index 57d84d0bae..3a76ec7dd2 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -387,7 +387,7 @@ static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, s->size = offset; offset += s->offset; - return bdrv_co_truncate(bs->file, offset, false, prealloc, errp); + return bdrv_co_truncate(bs->file, offset, exact, prealloc, errp); } static void raw_eject(BlockDriverState *bs, bool eject_flag) From patchwork Mon Oct 28 12:14:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215519 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 173AE1390 for ; Mon, 28 Oct 2019 13:27:42 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E25B920663 for ; Mon, 28 Oct 2019 13:27:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="h4ClCV9h" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E25B920663 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53740 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP53c-0008FA-Cj for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:27:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51319) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xh-00043L-Kh for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xf-0002SU-Rn for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:29 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:26204 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xf-0002SD-Na for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265047; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cw8wuCRkorBrlkC2Rr+V+y+jyP67yBOBt4kLF1Ox6ek=; b=h4ClCV9h5SjcBAanaSX9N3RDVH/3oudNf85D66fNXqXeATvAxsj0lK3wk5X0J365gMozJM S+fSUp5jD7TfJYe+yg8KL9FpU+r4AVZcCrIapLyJGvNWCVMpmFhtoVO2KhXHRlkyQP+rc0 kOUGIkyZYHnngQH3ZrYbKrgqrcqttLc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-305-IRCMxcqTNtKQBuUY2_LjcQ-1; Mon, 28 Oct 2019 08:17:25 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 82FC7800FF1; Mon, 28 Oct 2019 12:17:24 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2A4BE5DA2C; Mon, 28 Oct 2019 12:17:24 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 67/69] block: Pass truncate exact=true where reasonable Date: Mon, 28 Oct 2019 13:14:59 +0100 Message-Id: <20191028121501.15279-68-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: IRCMxcqTNtKQBuUY2_LjcQ-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" This is a change in behavior, so all instances need a good justification. The comments added here should explain my reasoning. qed already had a comment that suggests it always expected bdrv_truncate()/blk_truncate() to behave as if exact=true were passed (c743849bee7 came eight months before 55b949c8476), so it was simply broken until now. Signed-off-by: Max Reitz Message-id: 20190918095144.955-8-mreitz@redhat.com Reviewed-by: Maxim Levitsky [mreitz: Changed comment in qed.c to explain why a new QED file must be empty, as requested and suggested by Maxim] Signed-off-by: Max Reitz --- block/parallels.c | 11 +++++++++-- block/qcow2.c | 6 +++++- block/qed.c | 7 +++++-- qemu-img.c | 7 ++++++- qemu-io-cmds.c | 7 ++++++- 5 files changed, 31 insertions(+), 7 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index a1a92c97a4..603211f83c 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -487,7 +487,12 @@ static int coroutine_fn parallels_co_check(BlockDriverState *bs, res->leaks += count; if (fix & BDRV_FIX_LEAKS) { Error *local_err = NULL; - ret = bdrv_truncate(bs->file, res->image_end_offset, false, + + /* + * In order to really repair the image, we must shrink it. + * That means we have to pass exact=true. + */ + ret = bdrv_truncate(bs->file, res->image_end_offset, true, PREALLOC_MODE_OFF, &local_err); if (ret < 0) { error_report_err(local_err); @@ -880,7 +885,9 @@ static void parallels_close(BlockDriverState *bs) if ((bs->open_flags & BDRV_O_RDWR) && !(bs->open_flags & BDRV_O_INACTIVE)) { s->header->inuse = 0; parallels_update_header(bs); - bdrv_truncate(bs->file, s->data_end << BDRV_SECTOR_BITS, false, + + /* errors are ignored, so we might as well pass exact=true */ + bdrv_truncate(bs->file, s->data_end << BDRV_SECTOR_BITS, true, PREALLOC_MODE_OFF, NULL); } diff --git a/block/qcow2.c b/block/qcow2.c index 9f32dae41f..7c18721741 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -5323,7 +5323,11 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, return ret; } - ret = blk_truncate(blk, new_size, false, PREALLOC_MODE_OFF, errp); + /* + * Amending image options should ensure that the image has + * exactly the given new values, so pass exact=true here. + */ + ret = blk_truncate(blk, new_size, true, PREALLOC_MODE_OFF, errp); blk_unref(blk); if (ret < 0) { return ret; diff --git a/block/qed.c b/block/qed.c index 7c2a65af40..d8c4e5fb1e 100644 --- a/block/qed.c +++ b/block/qed.c @@ -673,8 +673,11 @@ static int coroutine_fn bdrv_qed_co_create(BlockdevCreateOptions *opts, l1_size = header.cluster_size * header.table_size; - /* File must start empty and grow, check truncate is supported */ - ret = blk_truncate(blk, 0, false, PREALLOC_MODE_OFF, errp); + /* + * The QED format associates file length with allocation status, + * so a new file (which is empty) must have a length of 0. + */ + ret = blk_truncate(blk, 0, true, PREALLOC_MODE_OFF, errp); if (ret < 0) { goto out; } diff --git a/qemu-img.c b/qemu-img.c index c738297b04..b288c967b5 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -3831,7 +3831,12 @@ static int img_resize(int argc, char **argv) } } - ret = blk_truncate(blk, total_size, false, prealloc, &err); + /* + * The user expects the image to have the desired size after + * resizing, so pass @exact=true. It is of no use to report + * success when the image has not actually been resized. + */ + ret = blk_truncate(blk, total_size, true, prealloc, &err); if (ret < 0) { error_report_err(err); goto out; diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c index 5e9017c979..1b7e700020 100644 --- a/qemu-io-cmds.c +++ b/qemu-io-cmds.c @@ -1710,7 +1710,12 @@ static int truncate_f(BlockBackend *blk, int argc, char **argv) return offset; } - ret = blk_truncate(blk, offset, false, PREALLOC_MODE_OFF, &local_err); + /* + * qemu-io is a debugging tool, so let us be strict here and pass + * exact=true. It is better to err on the "emit more errors" side + * than to be overly permissive. + */ + ret = blk_truncate(blk, offset, true, PREALLOC_MODE_OFF, &local_err); if (ret < 0) { error_report_err(local_err); return ret; From patchwork Mon Oct 28 12:15:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215521 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 42FDF1390 for ; Mon, 28 Oct 2019 13:27:59 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1939220663 for ; Mon, 28 Oct 2019 13:27:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HkX1zh65" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1939220663 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53742 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP53t-0008Uw-Qz for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:27:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51350) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xj-00046O-1d for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xh-0002U1-QB for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:30 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:41120 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xh-0002Tf-MG for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265049; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eR57hYOpETc1WOwroQ3gB2aclK6c6r/Yqpqzc1DOwpg=; b=HkX1zh65tOwQGn1vCnv0MfG56VjkHMH/Lq680O6cRHwjdSJn8BR7Nm1dXIZUqdB5EkRFzg w8/emNAc51JqpQ5zStswwpWoWBcyPlepc+aVULXc5N+6N7o0LXU2IOhFM3Udo0FZUH+153 CHCQvddmsAJ4R9JZAsSsIsFYNS4LJAA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-64-HzHk1RTYO8-eDWEsdo7OxA-1; Mon, 28 Oct 2019 08:17:27 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8DA72107AD28; Mon, 28 Oct 2019 12:17:26 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 301BA1001B03; Mon, 28 Oct 2019 12:17:26 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 68/69] Revert "qemu-img: Check post-truncation size" Date: Mon, 28 Oct 2019 13:15:00 +0100 Message-Id: <20191028121501.15279-69-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: HzHk1RTYO8-eDWEsdo7OxA-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" This reverts commit 5279b30392da7a3248b320c75f20c61e3a95863c. We no longer need this check because exact=true forces the block driver to give the image the exact size requested by the user. Signed-off-by: Max Reitz Message-id: 20190918095144.955-9-mreitz@redhat.com Reviewed-by: Maxim Levitsky Signed-off-by: Max Reitz --- qemu-img.c | 39 ++++----------------------------------- 1 file changed, 4 insertions(+), 35 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index b288c967b5..95a24b9762 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -3656,7 +3656,7 @@ static int img_resize(int argc, char **argv) Error *err = NULL; int c, ret, relative; const char *filename, *fmt, *size; - int64_t n, total_size, current_size, new_size; + int64_t n, total_size, current_size; bool quiet = false; BlockBackend *blk = NULL; PreallocMode prealloc = PREALLOC_MODE_OFF; @@ -3837,42 +3837,11 @@ static int img_resize(int argc, char **argv) * success when the image has not actually been resized. */ ret = blk_truncate(blk, total_size, true, prealloc, &err); - if (ret < 0) { + if (!ret) { + qprintf(quiet, "Image resized.\n"); + } else { error_report_err(err); - goto out; - } - - new_size = blk_getlength(blk); - if (new_size < 0) { - error_report("Failed to verify truncated image length: %s", - strerror(-new_size)); - ret = -1; - goto out; } - - /* Some block drivers implement a truncation method, but only so - * the user can cause qemu to refresh the image's size from disk. - * The idea is that the user resizes the image outside of qemu and - * then invokes block_resize to inform qemu about it. - * (This includes iscsi and file-posix for device files.) - * Of course, that is not the behavior someone invoking - * qemu-img resize would find useful, so we catch that behavior - * here and tell the user. */ - if (new_size != total_size && new_size == current_size) { - error_report("Image was not resized; resizing may not be supported " - "for this image"); - ret = -1; - goto out; - } - - if (new_size != total_size) { - warn_report("Image should have been resized to %" PRIi64 - " bytes, but was resized to %" PRIi64 " bytes", - total_size, new_size); - } - - qprintf(quiet, "Image resized.\n"); - out: blk_unref(blk); if (ret) { From patchwork Mon Oct 28 12:15:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Reitz X-Patchwork-Id: 11215485 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3D3F614E5 for ; Mon, 28 Oct 2019 13:07:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1394D214D9 for ; Mon, 28 Oct 2019 13:07:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bh0XOtht" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1394D214D9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53548 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP4ja-0001I8-Dx for patchwork-qemu-devel@patchwork.kernel.org; Mon, 28 Oct 2019 09:06:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51438) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iP3xs-0004O6-5D for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iP3xm-0002Xr-JZ for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:36 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:55383 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iP3xm-0002XK-8f for qemu-devel@nongnu.org; Mon, 28 Oct 2019 08:17:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572265053; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Rjjz3MsAGUfHfQ9MWUBX1nt5O6CnXPytlrmKQsKSedM=; b=bh0XOthtOMRiztCTm8rYF90DTRE9QlNSBd8E5fDeAvBFBFUIxc4Zv8oWh6weaH1yXv5Plb 1tDZRAOxAd9ay1ZsE+H4iJpOL8K8q+msMxwbU43jaiPK7LOEJ/IVMqLE1y0Rwmt5Usgbsj 5ZGTtkwj/zSX5V4w30OgntILMlLhUOI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-283-qrVjP8ooM2G6QniIJQdXkw-1; Mon, 28 Oct 2019 08:17:29 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 94A88476; Mon, 28 Oct 2019 12:17:28 +0000 (UTC) Received: from localhost (ovpn-117-83.ams2.redhat.com [10.36.117.83]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 38B465D6BB; Mon, 28 Oct 2019 12:17:28 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Subject: [PULL 69/69] qemu-iotests: restrict 264 to qcow2 only Date: Mon, 28 Oct 2019 13:15:01 +0100 Message-Id: <20191028121501.15279-70-mreitz@redhat.com> In-Reply-To: <20191028121501.15279-1-mreitz@redhat.com> References: <20191028121501.15279-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-MC-Unique: qrVjP8ooM2G6QniIJQdXkw-1 X-Mimecast-Spam-Score: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy 264 is unprepared to run with different formats, for example luks needs handling keys, cloop doesn't support image creation, vpc creates image larger than requested (which breaks "Backup completed: 5242880" in test output). The test is here to check nbd-reconnect feature and we actually don't need it for all formats. Let's restrict it to qcow2 only. Reported-by: Max Reitz Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20191025145023.6182-1-vsementsov@virtuozzo.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- tests/qemu-iotests/264 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/264 b/tests/qemu-iotests/264 index c8cd97ae2b..131366422b 100755 --- a/tests/qemu-iotests/264 +++ b/tests/qemu-iotests/264 @@ -24,6 +24,8 @@ import iotests from iotests import qemu_img_create, qemu_io_silent_check, file_path, \ qemu_nbd_popen, log +iotests.verify_image_format(supported_fmts=['qcow2']) + disk_a, disk_b, nbd_sock = file_path('disk_a', 'disk_b', 'nbd-sock') nbd_uri = 'nbd+unix:///?socket=' + nbd_sock size = 5 * 1024 * 1024