From patchwork Thu May 23 17:06:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 10958519 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AA60776 for ; Thu, 23 May 2019 17:08:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 84A6B1FE8B for ; Thu, 23 May 2019 17:08:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 77AE4283EE; Thu, 23 May 2019 17:08:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 17BAE1FE8B for ; Thu, 23 May 2019 17:08:29 +0000 (UTC) Received: from localhost ([127.0.0.1]:40840 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrCe-0004rp-RN for patchwork-qemu-devel@patchwork.kernel.org; Thu, 23 May 2019 13:08:28 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49203) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrBK-0003jv-Ao for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hTrBF-0001fp-4a for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:06 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44776) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hTrB5-0001Xn-9b; Thu, 23 May 2019 13:06:52 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B63E03001A77; Thu, 23 May 2019 17:06:49 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-187.bos.redhat.com [10.18.17.187]) by smtp.corp.redhat.com (Postfix) with ESMTP id C31D210027C5; Thu, 23 May 2019 17:06:48 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 23 May 2019 13:06:39 -0400 Message-Id: <20190523170643.20794-2-jsnow@redhat.com> In-Reply-To: <20190523170643.20794-1-jsnow@redhat.com> References: <20190523170643.20794-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Thu, 23 May 2019 17:06:49 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 1/5] blockdev-backup: don't check aio_context too early X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , aihua liang , qemu-stable@nongnu.org, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP in blockdev_backup_prepare, we check to make sure that the target is associated with a compatible aio context. However, do_blockdev_backup is called later and has some logic to move the target to a compatible aio_context. The transaction version will fail certain commands needlessly early as a result. Allow blockdev_backup_prepare to simply call do_blockdev_backup, which will ultimately decide if the contexts are compatible or not. Note: the transaction version has always disallowed this operation since its initial commit bd8baecd (2014), whereas the version of qmp_blockdev_backup at the time, from commit c29c1dd312f, tried to enforce the aio_context switch instead. It's not clear, and I can't see from the mailing list archives at the time, why the two functions take a different approach. It wasn't until later in efd7556708b (2016) that the standalone version tried to determine if it could set the context or not. Reported-by: aihua liang Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1683498 Signed-off-by: John Snow --- blockdev.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/blockdev.c b/blockdev.c index e856ca4be9..01a48a2a5a 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1872,10 +1872,6 @@ static void blockdev_backup_prepare(BlkActionState *common, Error **errp) } aio_context = bdrv_get_aio_context(bs); - if (aio_context != bdrv_get_aio_context(target)) { - error_setg(errp, "Backup between two IO threads is not implemented"); - return; - } aio_context_acquire(aio_context); state->bs = bs; From patchwork Thu May 23 17:06:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 10958525 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BE436112C for ; Thu, 23 May 2019 17:10:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A4BCF27E01 for ; Thu, 23 May 2019 17:10:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 99325286E6; Thu, 23 May 2019 17:10:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3AD79286DF for ; Thu, 23 May 2019 17:10:56 +0000 (UTC) Received: from localhost ([127.0.0.1]:40905 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrF1-0006aV-II for patchwork-qemu-devel@patchwork.kernel.org; Thu, 23 May 2019 13:10:55 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49198) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrBK-0003js-9K for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hTrBG-0001iC-S9 for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:05 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54824) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hTrB7-0001Zu-Qa; Thu, 23 May 2019 13:06:54 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B7815330260; Thu, 23 May 2019 17:06:50 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-187.bos.redhat.com [10.18.17.187]) by smtp.corp.redhat.com (Postfix) with ESMTP id D800B1001DEF; Thu, 23 May 2019 17:06:49 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 23 May 2019 13:06:40 -0400 Message-Id: <20190523170643.20794-3-jsnow@redhat.com> In-Reply-To: <20190523170643.20794-1-jsnow@redhat.com> References: <20190523170643.20794-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 23 May 2019 17:06:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 2/5] iotests.py: do not use infinite waits X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Cap waits to 60 seconds so that iotests can fail gracefully if something goes wrong. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 6bcddd8870..6e17c040dc 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -521,7 +521,7 @@ class VM(qtest.QEMUQtestMachine): output_list += [key + '=' + obj[key]] return ','.join(output_list) - def get_qmp_events_filtered(self, wait=True): + def get_qmp_events_filtered(self, wait=60.0): result = [] for ev in self.get_qmp_events(wait=wait): result.append(filter_qmp_event(ev)) @@ -539,10 +539,10 @@ class VM(qtest.QEMUQtestMachine): # Returns None on success, and an error string on failure def run_job(self, job, auto_finalize=True, auto_dismiss=False, - pre_finalize=None): + pre_finalize=None, wait=60.0): error = None while True: - for ev in self.get_qmp_events_filtered(wait=True): + for ev in self.get_qmp_events_filtered(wait=wait): if ev['event'] == 'JOB_STATUS_CHANGE': status = ev['data']['status'] if status == 'aborting': @@ -647,7 +647,7 @@ class QMPTestCase(unittest.TestCase): self.assertEqual(self.vm.flatten_qmp_object(json.loads(json_filename[5:])), self.vm.flatten_qmp_object(reference)) - def cancel_and_wait(self, drive='drive0', force=False, resume=False): + def cancel_and_wait(self, drive='drive0', force=False, resume=False, wait=60.0): '''Cancel a block job and wait for it to finish, returning the event''' result = self.vm.qmp('block-job-cancel', device=drive, force=force) self.assert_qmp(result, 'return', {}) @@ -658,7 +658,7 @@ class QMPTestCase(unittest.TestCase): cancelled = False result = None while not cancelled: - for event in self.vm.get_qmp_events(wait=True): + for event in self.vm.get_qmp_events(wait=wait): if event['event'] == 'BLOCK_JOB_COMPLETED' or \ event['event'] == 'BLOCK_JOB_CANCELLED': self.assert_qmp(event, 'data/device', drive) @@ -671,10 +671,10 @@ class QMPTestCase(unittest.TestCase): self.assert_no_active_block_jobs() return result - def wait_until_completed(self, drive='drive0', check_offset=True): + def wait_until_completed(self, drive='drive0', check_offset=True, wait=60.0): '''Wait for a block job to finish, returning the event''' while True: - for event in self.vm.get_qmp_events(wait=True): + for event in self.vm.get_qmp_events(wait=wait): if event['event'] == 'BLOCK_JOB_COMPLETED': self.assert_qmp(event, 'data/device', drive) self.assert_qmp_absent(event, 'data/error') From patchwork Thu May 23 17:06:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 10958527 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5FF3476 for ; Thu, 23 May 2019 17:12:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 45E9D286F9 for ; Thu, 23 May 2019 17:12:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A77128706; Thu, 23 May 2019 17:12:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B56D1286F9 for ; Thu, 23 May 2019 17:12:27 +0000 (UTC) Received: from localhost ([127.0.0.1]:40917 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrGV-0007ZF-0m for patchwork-qemu-devel@patchwork.kernel.org; Thu, 23 May 2019 13:12:27 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49194) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrBK-0003jo-8G for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hTrBF-0001fm-4k for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:04 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33704) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hTrB7-0001Zp-Um; Thu, 23 May 2019 13:06:54 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B50187FDFD; Thu, 23 May 2019 17:06:51 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-187.bos.redhat.com [10.18.17.187]) by smtp.corp.redhat.com (Postfix) with ESMTP id D6C8B10027C5; Thu, 23 May 2019 17:06:50 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 23 May 2019 13:06:41 -0400 Message-Id: <20190523170643.20794-4-jsnow@redhat.com> In-Reply-To: <20190523170643.20794-1-jsnow@redhat.com> References: <20190523170643.20794-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Thu, 23 May 2019 17:06:51 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 3/5] QEMUMachine: add events_wait method X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Instead of event_wait which looks for a single event, add an events_wait which can look for any number of events simultaneously. However, it will still only return one at a time, whichever happens first. Signed-off-by: John Snow Reviewed-by: Max Reitz --- python/qemu/__init__.py | 69 +++++++++++++++++++++++++++++------------ 1 file changed, 49 insertions(+), 20 deletions(-) diff --git a/python/qemu/__init__.py b/python/qemu/__init__.py index 81d9657ec0..98ed8a2e28 100644 --- a/python/qemu/__init__.py +++ b/python/qemu/__init__.py @@ -402,42 +402,71 @@ class QEMUMachine(object): self._qmp.clear_events() return events - def event_wait(self, name, timeout=60.0, match=None): + @staticmethod + def event_match(event, match=None): """ - Wait for specified timeout on named event in QMP; optionally filter - results by match. + Check if an event matches optional match criteria. - The 'match' is checked to be a recursive subset of the 'event'; skips - branch processing on match's value None - {"foo": {"bar": 1}} matches {"foo": None} - {"foo": {"bar": 1}} does not matches {"foo": {"baz": None}} + The match criteria takes the form of a matching subdict. The event is + checked to be a superset of the subdict, recursively, with matching + values whenever those values are not None. + + Examples, with the subdict queries on the left: + - None matches any object. + - {"foo": None} matches {"foo": {"bar": 1}} + - {"foo": {"baz": None}} does not match {"foo": {"bar": 1}} + - {"foo": {"baz": 2}} matches {"foo": {"bar": 1, "baz": 2}} """ - def event_match(event, match=None): - if match is None: - return True + if match is None: + return True - for key in match: - if key in event: - if isinstance(event[key], dict): - if not event_match(event[key], match[key]): - return False - elif event[key] != match[key]: + for key in match: + if key in event: + if isinstance(event[key], dict): + if not QEMUMachine.event_match(event[key], match[key]): return False - else: + elif event[key] != match[key]: return False + else: + return False + return True - return True + def event_wait(self, name, timeout=60.0, match=None): + """ + event_wait waits for and returns a named event from QMP with a timeout. + + name: The event to wait for. + timeout: QEMUMonitorProtocol.pull_event timeout parameter. + match: Optional match criteria. See event_match for details. + """ + return self.events_wait([(name, match)], timeout) + + def events_wait(self, events, timeout=60.0): + """ + events_wait waits for and returns a named event from QMP with a timeout. + + events: a sequence of (name, match_criteria) tuples. + The match criteria are optional and may be None. + See event_match for details. + timeout: QEMUMonitorProtocol.pull_event timeout parameter. + """ + def _match(event): + for name, match in events: + if (event['event'] == name and + self.event_match(event, match)): + return True + return False # Search cached events for event in self._events: - if (event['event'] == name) and event_match(event, match): + if _match(event): self._events.remove(event) return event # Poll for new events while True: event = self._qmp.pull_event(wait=timeout) - if (event['event'] == name) and event_match(event, match): + if _match(event): return event self._events.append(event) From patchwork Thu May 23 17:06:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 10958521 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA93876 for ; Thu, 23 May 2019 17:08:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B1B571FE8B for ; Thu, 23 May 2019 17:08:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A5768283EE; Thu, 23 May 2019 17:08:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4D0B21FE8B for ; Thu, 23 May 2019 17:08:31 +0000 (UTC) Received: from localhost ([127.0.0.1]:40843 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrCg-0004tR-Fg for patchwork-qemu-devel@patchwork.kernel.org; Thu, 23 May 2019 13:08:30 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49193) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrBK-0003jn-89 for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hTrBF-0001fi-4s for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:04 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42606) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hTrB7-0001Zh-JW; Thu, 23 May 2019 13:06:53 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B1F363002F44; Thu, 23 May 2019 17:06:52 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-187.bos.redhat.com [10.18.17.187]) by smtp.corp.redhat.com (Postfix) with ESMTP id D46251001DEF; Thu, 23 May 2019 17:06:51 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 23 May 2019 13:06:42 -0400 Message-Id: <20190523170643.20794-5-jsnow@redhat.com> In-Reply-To: <20190523170643.20794-1-jsnow@redhat.com> References: <20190523170643.20794-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Thu, 23 May 2019 17:06:52 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 4/5] iotests.py: rewrite run_job to be pickier X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Don't pull events out of the queue that don't belong to us; be choosier so that we can use this method to drive jobs that were launched by transactions that may have more jobs. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 48 +++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 19 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 6e17c040dc..dc77d3fba0 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -540,27 +540,37 @@ class VM(qtest.QEMUQtestMachine): # Returns None on success, and an error string on failure def run_job(self, job, auto_finalize=True, auto_dismiss=False, pre_finalize=None, wait=60.0): + match_device = {'data': {'device': job}} + match_id = {'data': {'id': job}} + events = [ + ('BLOCK_JOB_COMPLETED', match_device), + ('BLOCK_JOB_CANCELLED', match_device), + ('BLOCK_JOB_ERROR', match_device), + ('BLOCK_JOB_READY', match_device), + ('BLOCK_JOB_PENDING', match_id), + ('JOB_STATUS_CHANGE', match_id) + ] error = None while True: - for ev in self.get_qmp_events_filtered(wait=wait): - if ev['event'] == 'JOB_STATUS_CHANGE': - status = ev['data']['status'] - if status == 'aborting': - result = self.qmp('query-jobs') - for j in result['return']: - if j['id'] == job: - error = j['error'] - log('Job failed: %s' % (j['error'])) - elif status == 'pending' and not auto_finalize: - if pre_finalize: - pre_finalize() - self.qmp_log('job-finalize', id=job) - elif status == 'concluded' and not auto_dismiss: - self.qmp_log('job-dismiss', id=job) - elif status == 'null': - return error - else: - log(ev) + ev = filter_qmp_event(self.events_wait(events)) + if ev['event'] != 'JOB_STATUS_CHANGE': + log(ev) + continue + status = ev['data']['status'] + if status == 'aborting': + result = self.qmp('query-jobs') + for j in result['return']: + if j['id'] == job: + error = j['error'] + log('Job failed: %s' % (j['error'])) + elif status == 'pending' and not auto_finalize: + if pre_finalize: + pre_finalize() + self.qmp_log('job-finalize', id=job) + elif status == 'concluded' and not auto_dismiss: + self.qmp_log('job-dismiss', id=job) + elif status == 'null': + return error def node_info(self, node_name): nodes = self.qmp('query-named-block-nodes') From patchwork Thu May 23 17:06:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 10958531 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EAC4B112C for ; Thu, 23 May 2019 17:13:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CCA7A28707 for ; Thu, 23 May 2019 17:13:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C12AB2870E; Thu, 23 May 2019 17:13:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 10A5828707 for ; Thu, 23 May 2019 17:13:55 +0000 (UTC) Received: from localhost ([127.0.0.1]:40925 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrHv-0008Vq-35 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 23 May 2019 13:13:55 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49287) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrBO-0003oW-CN for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hTrBM-0001mX-8O for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:10 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35118) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hTrBD-0001aq-6K; Thu, 23 May 2019 13:06:59 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B15703079B76; Thu, 23 May 2019 17:06:53 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-187.bos.redhat.com [10.18.17.187]) by smtp.corp.redhat.com (Postfix) with ESMTP id D4A371017E2E; Thu, 23 May 2019 17:06:52 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 23 May 2019 13:06:43 -0400 Message-Id: <20190523170643.20794-6-jsnow@redhat.com> In-Reply-To: <20190523170643.20794-1-jsnow@redhat.com> References: <20190523170643.20794-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Thu, 23 May 2019 17:06:53 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 5/5] iotests: add iotest 250 for testing blockdev-backup across iothread contexts X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: John Snow --- tests/qemu-iotests/250 | 122 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/250.out | 119 ++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 242 insertions(+) create mode 100755 tests/qemu-iotests/250 create mode 100644 tests/qemu-iotests/250.out diff --git a/tests/qemu-iotests/250 b/tests/qemu-iotests/250 new file mode 100755 index 0000000000..c594a43205 --- /dev/null +++ b/tests/qemu-iotests/250 @@ -0,0 +1,122 @@ +#!/usr/bin/env python +# +# Test incremental/backup across iothread contexts +# +# Copyright (c) 2019 John Snow for Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# owner=jsnow@redhat.com + +import os +import iotests +from iotests import log + +iotests.verify_image_format(supported_fmts=['qcow2']) +size = 64 * 1024 * 1024 + +with iotests.FilePath('img0') as img0_path, \ + iotests.FilePath('img1') as img1_path, \ + iotests.FilePath('img0-full') as img0_full_path, \ + iotests.FilePath('img1-full') as img1_full_path, \ + iotests.FilePath('img0-incr') as img0_incr_path, \ + iotests.FilePath('img1-incr') as img1_incr_path, \ + iotests.VM() as vm: + + def create_target(filepath, name, size): + basename = os.path.basename(filepath) + nodename = "file_{}".format(basename) + log(vm.command('blockdev-create', job_id='job1', + options={ + 'driver': 'file', + 'filename': filepath, + 'size': 0, + })) + vm.run_job('job1') + log(vm.command('blockdev-add', driver='file', + node_name=nodename, filename=filepath)) + log(vm.command('blockdev-create', job_id='job2', + options={ + 'driver': iotests.imgfmt, + 'file': nodename, + 'size': size, + })) + vm.run_job('job2') + log(vm.command('blockdev-add', driver=iotests.imgfmt, + node_name=name, + file=nodename)) + + log('--- Preparing images & VM ---\n') + vm.add_object('iothread,id=iothread0') + vm.add_object('iothread,id=iothread1') + vm.add_device('virtio-scsi-pci,id=scsi0,iothread=iothread0') + vm.add_device('virtio-scsi-pci,id=scsi1,iothread=iothread1') + iotests.qemu_img_create('-f', iotests.imgfmt, img0_path, str(size)) + iotests.qemu_img_create('-f', iotests.imgfmt, img1_path, str(size)) + vm.add_drive(img0_path, interface='none') + vm.add_device('scsi-hd,id=device0,drive=drive0,bus=scsi0.0') + vm.add_drive(img1_path, interface='none') + vm.add_device('scsi-hd,id=device1,drive=drive1,bus=scsi1.0') + + log('--- Starting VM ---\n') + vm.launch() + + log('--- Create Targets & Full Backups ---\n') + create_target(img0_full_path, 'img0-full', size) + create_target(img1_full_path, 'img1-full', size) + ret = vm.qmp_log('transaction', indent=2, actions=[ + { 'type': 'block-dirty-bitmap-add', + 'data': { 'node': 'drive0', 'name': 'bitmap0' }}, + { 'type': 'block-dirty-bitmap-add', + 'data': { 'node': 'drive1', 'name': 'bitmap1' }}, + { 'type': 'blockdev-backup', + 'data': { 'device': 'drive0', + 'target': 'img0-full', + 'sync': 'full', + 'job-id': 'j0' }}, + { 'type': 'blockdev-backup', + 'data': { 'device': 'drive1', + 'target': 'img1-full', + 'sync': 'full', + 'job-id': 'j1' }} + ]) + if "error" in ret: + raise Exception(ret['error']['desc']) + vm.run_job('j0', auto_dismiss=True) + vm.run_job('j1', auto_dismiss=True) + + log('\n--- Create Targets & Incremental Backups ---\n') + create_target(img0_incr_path, 'img0-incr', size) + create_target(img1_incr_path, 'img1-incr', size) + ret = vm.qmp_log('transaction', indent=2, actions=[ + { 'type': 'blockdev-backup', + 'data': { 'device': 'drive0', + 'target': 'img0-incr', + 'sync': 'incremental', + 'bitmap': 'bitmap0', + 'job-id': 'j2' }}, + { 'type': 'blockdev-backup', + 'data': { 'device': 'drive1', + 'target': 'img1-incr', + 'sync': 'incremental', + 'bitmap': 'bitmap1', + 'job-id': 'j3' }} + ]) + if "error" in ret: + raise Exception(ret['error']['desc']) + vm.run_job('j2', auto_dismiss=True) + vm.run_job('j3', auto_dismiss=True) + + log('\n--- Done ---') + vm.shutdown() diff --git a/tests/qemu-iotests/250.out b/tests/qemu-iotests/250.out new file mode 100644 index 0000000000..eec38614ec --- /dev/null +++ b/tests/qemu-iotests/250.out @@ -0,0 +1,119 @@ +--- Preparing images & VM --- + +--- Starting VM --- + +--- Create Targets & Full Backups --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "job1"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job2"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job1"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job2"}} +{"return": {}} +{} +{ + "execute": "transaction", + "arguments": { + "actions": [ + { + "data": { + "name": "bitmap0", + "node": "drive0" + }, + "type": "block-dirty-bitmap-add" + }, + { + "data": { + "name": "bitmap1", + "node": "drive1" + }, + "type": "block-dirty-bitmap-add" + }, + { + "data": { + "device": "drive0", + "job-id": "j0", + "sync": "full", + "target": "img0-full" + }, + "type": "blockdev-backup" + }, + { + "data": { + "device": "drive1", + "job-id": "j1", + "sync": "full", + "target": "img1-full" + }, + "type": "blockdev-backup" + } + ] + } +} +{ + "return": {} +} +{"data": {"device": "j0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "j1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} + +--- Create Targets & Incremental Backups --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "job1"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job2"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job1"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job2"}} +{"return": {}} +{} +{ + "execute": "transaction", + "arguments": { + "actions": [ + { + "data": { + "bitmap": "bitmap0", + "device": "drive0", + "job-id": "j2", + "sync": "incremental", + "target": "img0-incr" + }, + "type": "blockdev-backup" + }, + { + "data": { + "bitmap": "bitmap1", + "device": "drive1", + "job-id": "j3", + "sync": "incremental", + "target": "img1-incr" + }, + "type": "blockdev-backup" + } + ] + } +} +{ + "return": {} +} +{"data": {"device": "j2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "j3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} + +--- Done --- diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 2758f48143..ea52e8660d 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -249,6 +249,7 @@ 247 rw auto quick 248 rw auto quick 249 rw auto quick +250 rw auto quick 252 rw auto backing quick 253 rw auto quick 255 rw auto quick