From patchwork Wed Sep 11 02:35:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 11140217 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E7C9E1395 for ; Wed, 11 Sep 2019 02:38:11 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 85B632171F for ; Wed, 11 Sep 2019 02:38:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 85B632171F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:46132 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i7sWI-0004PS-K0 for patchwork-qemu-devel@patchwork.kernel.org; Tue, 10 Sep 2019 22:38:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33745) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i7sUL-0001my-6F for qemu-devel@nongnu.org; Tue, 10 Sep 2019 22:36:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i7sUJ-0004nZ-FT for qemu-devel@nongnu.org; Tue, 10 Sep 2019 22:36:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43662) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1i7sUJ-0004nM-AA for qemu-devel@nongnu.org; Tue, 10 Sep 2019 22:36:07 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 97FC530BA1B0 for ; Wed, 11 Sep 2019 02:36:06 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-121-171.rdu2.redhat.com [10.10.121.171]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C4B5E60852; Wed, 11 Sep 2019 02:36:05 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org Date: Tue, 10 Sep 2019 22:35:57 -0400 Message-Id: <20190911023558.4880-2-crosa@redhat.com> In-Reply-To: <20190911023558.4880-1-crosa@redhat.com> References: <20190911023558.4880-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Wed, 11 Sep 2019 02:36:06 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 1/2] Python libs: close console sockets before shutting down the VMs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Willian Rampazzo , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= , Eduardo Habkost , Wainer dos Santos Moschetta , Cleber Rosa Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Currently, the console socket on QEMUMachine is closed after the QMP command to gracefully exit QEMU is executed. Because of a possible deadlock (QEMU waiting for the socket to become writable) let's close the console socket earlier. Reference: <20190607034214.GB22416@habkost.net> Reference: https://bugs.launchpad.net/qemu/+bug/1829779 From: Eduardo Habkost Signed-off-by: Cleber Rosa --- python/qemu/machine.py | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 128a3d1dc2..4f533b7881 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -271,10 +271,6 @@ class QEMUMachine(object): self._qemu_log_path = None - if self._console_socket is not None: - self._console_socket.close() - self._console_socket = None - if self._temp_dir is not None: shutil.rmtree(self._temp_dir) self._temp_dir = None @@ -333,6 +329,13 @@ class QEMUMachine(object): """ Terminate the VM and clean up """ + # If we keep the console socket open, we may deadlock waiting + # for QEMU to exit, while QEMU is waiting for the socket to + # become writeable. + if self._console_socket is not None: + self._console_socket.close() + self._console_socket = None + if self.is_running(): try: if not has_quit: From patchwork Wed Sep 11 02:35:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 11140215 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9A9DE1395 for ; Wed, 11 Sep 2019 02:36:56 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7B19F2171F for ; Wed, 11 Sep 2019 02:36:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B19F2171F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:46120 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i7sV5-0002wA-LL for patchwork-qemu-devel@patchwork.kernel.org; Tue, 10 Sep 2019 22:36:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33758) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i7sUL-0001n3-PH for qemu-devel@nongnu.org; Tue, 10 Sep 2019 22:36:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i7sUK-0004nq-Et for qemu-devel@nongnu.org; Tue, 10 Sep 2019 22:36:09 -0400 Received: from mx1.redhat.com ([209.132.183.28]:32948) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1i7sUK-0004ng-7Y for qemu-devel@nongnu.org; Tue, 10 Sep 2019 22:36:08 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 96403300DA3A for ; Wed, 11 Sep 2019 02:36:07 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-121-171.rdu2.redhat.com [10.10.121.171]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C0FC460852; Wed, 11 Sep 2019 02:36:06 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org Date: Tue, 10 Sep 2019 22:35:58 -0400 Message-Id: <20190911023558.4880-3-crosa@redhat.com> In-Reply-To: <20190911023558.4880-1-crosa@redhat.com> References: <20190911023558.4880-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Wed, 11 Sep 2019 02:36:07 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 2/2] Python libs: enable machine type auto selection X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Willian Rampazzo , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= , Eduardo Habkost , Wainer dos Santos Moschetta , Cleber Rosa Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" The arm and aarch64 targets are noteworthy because they don't have a default machine type. This introduces and opt-in option to users of the qemu.machine.QEMUMachine Python class, in which a default machine type will be set when a number of conditions are met. The motivation for this feature is that a number of (acceptance) tests are target agnostic, and should run with all tarets (in theory while it's pending continuous test execution). These tests can be listed with: avocado list --filter-by-tags='-arch' --filter-by-tags-include-empty \ --filter-by-tags-include-empty-key tests/acceptance/ Combined with the "qemu_bin" parameter that acceptance tests take, one can run those tests with arm and aarch64 targets without adding specific logic in each test. To execute those with a aarch64 QEMU binary one can run: avocado run --filter-by-tags='-arch' --filter-by-tags-include-empty \ --filter-by-tags-include-empty-key \ -p qemu_bin=aarch64-softmmu/qemu-system-aarch64 \ tests/acceptance/ When this feature is activated, the default automatic selection of a machine type will be logged (as it will also be done in the QEMU command line that is already logged). Signed-off-by: Cleber Rosa --- python/qemu/machine.py | 81 +++++++++++++++++++++++ tests/acceptance/avocado_qemu/__init__.py | 1 + 2 files changed, 82 insertions(+) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 4f533b7881..350052a30b 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -69,6 +69,15 @@ class QEMUMachine(object): # vm is guaranteed to be shut down here """ + + #: The default machine model, used by :meth:`set_machine_auto_selection` and + #: :meth:`get_machine_auto_selection` + DEFAULT_MACHINE_MODELS = { + 'arm': 'virt', + 'aarch64': 'virt', + } + + def __init__(self, binary, args=None, wrapper=None, name=None, test_dir="/var/tmp", monitor_address=None, socket_scm_helper=None): @@ -108,6 +117,7 @@ class QEMUMachine(object): self._temp_dir = None self._launched = False self._machine = None + self._machine_auto_selection = False self._console_set = False self._console_device_type = None self._console_address = None @@ -286,6 +296,7 @@ class QEMUMachine(object): self._iolog = None self._qemu_full_args = None + self._perform_machine_auto_selection() try: self._launch() self._launched = True @@ -533,3 +544,73 @@ class QEMUMachine(object): socket.SOCK_STREAM) self._console_socket.connect(self._console_address) return self._console_socket + + def set_machine_auto_selection(self, value): + """ + Defines whether QEMUMachine will set a machine model when needed + + The machine model will only be set automatically if: + + 1) No machine model was explicity set with :meth:`set_machine` + 2) Target binary is known to not work properly when a machine + is not specified (usually producing "No machine specified, + and there's no default" on the command line) + 3) A default machine model exists in :data:`DEFAULT_MACHINE_MODELS` + 4) :func:`get_target_from_qemu_binary` is capable of returning + the target for the binary set on this class + + If machine auto selection is enabled, it will be done so by + calling :meth:`set_machine` during the machine launc, that is + on :meth:`launch` so that a manually set machine model is not + discarded. + + @param value: wether to enable or disable the machine auto selection + feature + @type value: bool + """ + self._machine_auto_selection = value + + def get_machine_auto_selection(self): + """ + Returns whether QEMUMachine will set a machine model when needed + + Please refer to :meth:`set_machine_auto_selection` for a more + complete explanation. + """ + return self._machine_auto_selection + + def _perform_machine_auto_selection(self): + if not (self.get_machine_auto_selection() and self._machine is None): + return + + target = self._get_target_from_qemu_binary() + if target is None: + LOG.warn('Machine auto selection was requested, but it was not ' + 'applied as the binary "%s" could not be probed for its ' + 'target architecture', self._binary) + return + + machine_model = self.DEFAULT_MACHINE_MODELS.get(target, None) + if machine_model is None: + LOG.warn('Machine auto selection was requested, but it was not ' + 'applied as there is no default machine model defined ' + 'for target architecture "%s"', target) + return + + LOG.debug('Setting machine model to: %s', machine_model) + self.set_machine(machine_model) + + def _get_target_from_qemu_binary(self): + """ + Returns the target for this machine's QEMU binary + """ + qemu_machine = QEMUMachine(self._binary) + qemu_machine.add_args('-S') + qemu_machine.set_machine('none') + try: + qemu_machine.launch() + target_resp = qemu_machine.command('query-target') + qemu_machine.shutdown() + return target_resp['arch'] + except Exception: + return None diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py index bd41e0443c..025bb6a725 100644 --- a/tests/acceptance/avocado_qemu/__init__.py +++ b/tests/acceptance/avocado_qemu/__init__.py @@ -70,6 +70,7 @@ class Test(avocado.Test): def _new_vm(self, *args): vm = QEMUMachine(self.qemu_bin) + vm.set_machine_auto_selection(True) if args: vm.add_args(*args) return vm