From patchwork Tue Jul 16 11:26:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 13734466 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C62DC3DA49 for ; Tue, 16 Jul 2024 11:28:45 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sTgKO-00006y-Ie; Tue, 16 Jul 2024 07:26:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKM-0008U0-V2 for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKF-0003dP-6t for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721129189; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZNYAGbIPyRsOwVWWlqiOVUa84ONQUbH1ubPK+l47OlA=; b=VKbwolrP5KFst4+0sZL+v7XxzghpH3ly1dQmyvf34xSxCrC1JDKqfWHlUVJr11kqldLBVC IzBqX664pP2QYEU/FugTFDLt7u5jq1dScXSnbzrr3VlhF+GO2MdE1T2CMH8/or9oBxoqCD oHG886kdjTZ8+89HFBAd46R34gkfp/g= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-623-bJa1TgmGNpeG2rPVtJk-XQ-1; Tue, 16 Jul 2024 07:26:26 -0400 X-MC-Unique: bJa1TgmGNpeG2rPVtJk-XQ-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 60D60191042F; Tue, 16 Jul 2024 11:26:24 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.192.154]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A96FB1955F40; Tue, 16 Jul 2024 11:26:20 +0000 (UTC) From: Thomas Huth To: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Cc: Ani Sinha , Richard Henderson , Paolo Bonzini , "Daniel P . Berrange" , John Snow , =?utf-8?q?C=C3=A9dric_Le_Goater?= , Fabiano Rosas Subject: [PATCH 01/11] tests/functional: Add base classes for the upcoming pytest-based tests Date: Tue, 16 Jul 2024 13:26:04 +0200 Message-ID: <20240716112614.1755692-2-thuth@redhat.com> In-Reply-To: <20240716112614.1755692-1-thuth@redhat.com> References: <20240716112614.1755692-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The file is mostly a copy of the tests/avocado/avocado_qemu/__init__.py file with some adjustments to get rid of the Avocado dependencies (i.e. we also have to drop the LinuxSSHMixIn and LinuxTest for now). The emulator binary and build directory are now passed via environment variables that will be set via meson.build later. Signed-off-by: Thomas Huth --- tests/functional/qemu_test/__init__.py | 326 +++++++++++++++++++++++++ 1 file changed, 326 insertions(+) create mode 100644 tests/functional/qemu_test/__init__.py diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py new file mode 100644 index 0000000000..e73705d40a --- /dev/null +++ b/tests/functional/qemu_test/__init__.py @@ -0,0 +1,326 @@ +# Test class and utilities for functional tests +# +# Copyright 2018, 2024 Red Hat, Inc. +# +# Original Author (Avocado-based tests): +# Cleber Rosa +# +# Adaption for standalone version: +# Thomas Huth +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import logging +import os +import pycotap +import shutil +import subprocess +import sys +import tempfile +import time +import uuid +import unittest + +from pathlib import Path +from qemu.machine import QEMUMachine +from qemu.utils import (get_info_usernet_hostfwd_port, kvm_available, + tcg_available) + +def _source_dir(): + # Determine top-level directory of the QEMU sources + return Path(__file__).parent.parent.parent.parent + +def _build_dir(): + root = os.getenv('QEMU_BUILD_ROOT') + if root is not None: + return Path(root) + # Makefile.mtest only exists in build dir, so if it is available, use CWD + if os.path.exists('Makefile.mtest'): + return Path(os.getcwd()) + + root = os.path.join(_source_dir(), 'build') + if os.path.exists(root): + return Path(root) + + raise Exception("Cannot identify build dir, set QEMU_BUILD_ROOT") + +def has_cmd(name, args=None): + """ + This function is for use in a @skipUnless decorator, e.g.: + + @skipUnless(*has_cmd('sudo -n', ('sudo', '-n', 'true'))) + def test_something_that_needs_sudo(self): + ... + """ + + if args is None: + args = ('which', name) + + try: + _, stderr, exitcode = run_cmd(args) + except Exception as e: + exitcode = -1 + stderr = str(e) + + if exitcode != 0: + cmd_line = ' '.join(args) + err = f'{name} required, but "{cmd_line}" failed: {stderr.strip()}' + return (False, err) + else: + return (True, '') + +def has_cmds(*cmds): + """ + This function is for use in a @skipUnless decorator and + allows checking for the availability of multiple commands, e.g.: + + @skipUnless(*has_cmds(('cmd1', ('cmd1', '--some-parameter')), + 'cmd2', 'cmd3')) + def test_something_that_needs_cmd1_and_cmd2(self): + ... + """ + + for cmd in cmds: + if isinstance(cmd, str): + cmd = (cmd,) + + ok, errstr = has_cmd(*cmd) + if not ok: + return (False, errstr) + + return (True, '') + +def run_cmd(args): + subp = subprocess.Popen(args, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + universal_newlines=True) + stdout, stderr = subp.communicate() + ret = subp.returncode + + return (stdout, stderr, ret) + +def is_readable_executable_file(path): + return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK) + +def _console_interaction(test, success_message, failure_message, + send_string, keep_sending=False, vm=None): + assert not keep_sending or send_string + if vm is None: + vm = test.vm + console = vm.console_file + console_logger = logging.getLogger('console') + while True: + if send_string: + vm.console_socket.sendall(send_string.encode()) + if not keep_sending: + send_string = None # send only once + try: + msg = console.readline().decode().strip() + except UnicodeDecodeError: + msg = None + if not msg: + continue + console_logger.debug(msg) + if success_message is None or success_message in msg: + break + if failure_message and failure_message in msg: + console.close() + fail = 'Failure message found in console: "%s". Expected: "%s"' % \ + (failure_message, success_message) + test.fail(fail) + +def interrupt_interactive_console_until_pattern(test, success_message, + failure_message=None, + interrupt_string='\r'): + """ + Keep sending a string to interrupt a console prompt, while logging the + console output. Typical use case is to break a boot loader prompt, such: + + Press a key within 5 seconds to interrupt boot process. + 5 + 4 + 3 + 2 + 1 + Booting default image... + + :param test: a test containing a VM that will have its console + read and probed for a success or failure message + :type test: :class:`qemu_test.QemuSystemTest` + :param success_message: if this message appears, test succeeds + :param failure_message: if this message appears, test fails + :param interrupt_string: a string to send to the console before trying + to read a new line + """ + _console_interaction(test, success_message, failure_message, + interrupt_string, True) + +def wait_for_console_pattern(test, success_message, failure_message=None, + vm=None): + """ + Waits for messages to appear on the console, while logging the content + + :param test: a test containing a VM that will have its console + read and probed for a success or failure message + :type test: :class:`qemu_test.QemuSystemTest` + :param success_message: if this message appears, test succeeds + :param failure_message: if this message appears, test fails + """ + _console_interaction(test, success_message, failure_message, None, vm=vm) + +def exec_command(test, command): + """ + Send a command to a console (appending CRLF characters), while logging + the content. + + :param test: a test containing a VM. + :type test: :class:`qemu_test.QemuSystemTest` + :param command: the command to send + :type command: str + """ + _console_interaction(test, None, None, command + '\r') + +def exec_command_and_wait_for_pattern(test, command, + success_message, failure_message=None): + """ + Send a command to a console (appending CRLF characters), then wait + for success_message to appear on the console, while logging the. + content. Mark the test as failed if failure_message is found instead. + + :param test: a test containing a VM that will have its console + read and probed for a success or failure message + :type test: :class:`qemu_test.QemuSystemTest` + :param command: the command to send + :param success_message: if this message appears, test succeeds + :param failure_message: if this message appears, test fails + """ + _console_interaction(test, success_message, failure_message, command + '\r') + +class QemuBaseTest(unittest.TestCase): + + qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY') + + BUILD_DIR = _build_dir() + + workdir = None + log = logging.getLogger('qemu-test') + + def setUp(self, bin_prefix): + self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set') + + self.workdir = os.path.join(self.BUILD_DIR, 'tests/functional') + self.workdir = os.path.join(self.workdir, self.id()) + if not os.path.exists(self.workdir): + os.makedirs(self.workdir) + + def main(): + tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError, + test_output_log = pycotap.LogMode.LogToError) + unittest.main(testRunner = tr) + + +class QemuSystemTest(QemuBaseTest): + """Facilitates system emulation tests.""" + + cpu = None + machine = None + _machinehelp = None + + def setUp(self): + self._vms = {} + + super().setUp('qemu-system-') + + def set_machine(self, machinename): + # TODO: We should use QMP to get the list of available machines + if not self._machinehelp: + self._machinehelp = run_cmd([self.qemu_bin, '-M', 'help'])[0]; + if self._machinehelp.find(machinename) < 0: + self.skipTest('no support for machine ' + machinename) + self.machine = machinename + + def require_accelerator(self, accelerator): + """ + Requires an accelerator to be available for the test to continue + + It takes into account the currently set qemu binary. + + If the check fails, the test is canceled. If the check itself + for the given accelerator is not available, the test is also + canceled. + + :param accelerator: name of the accelerator, such as "kvm" or "tcg" + :type accelerator: str + """ + checker = {'tcg': tcg_available, + 'kvm': kvm_available}.get(accelerator) + if checker is None: + self.skipTest("Don't know how to check for the presence " + "of accelerator %s" % accelerator) + if not checker(qemu_bin=self.qemu_bin): + self.skipTest("%s accelerator does not seem to be " + "available" % accelerator) + + def require_netdev(self, netdevname): + netdevhelp = run_cmd([self.qemu_bin, + '-M', 'none', '-netdev', 'help'])[0]; + if netdevhelp.find('\n' + netdevname + '\n') < 0: + self.skipTest('no support for " + netdevname + " networking') + + def require_device(self, devicename): + devhelp = run_cmd([self.qemu_bin, + '-M', 'none', '-device', 'help'])[0]; + if devhelp.find(devicename) < 0: + self.skipTest('no support for device ' + devicename) + + def _new_vm(self, name, *args): + vm = QEMUMachine(self.qemu_bin, base_temp_dir=self.workdir) + self.log.debug('QEMUMachine "%s" created', name) + self.log.debug('QEMUMachine "%s" temp_dir: %s', name, vm.temp_dir) + self.log.debug('QEMUMachine "%s" log_dir: %s', name, vm.log_dir) + if args: + vm.add_args(*args) + return vm + + @property + def vm(self): + return self.get_vm(name='default') + + def get_vm(self, *args, name=None): + if not name: + name = str(uuid.uuid4()) + if self._vms.get(name) is None: + self._vms[name] = self._new_vm(name, *args) + if self.cpu is not None: + self._vms[name].add_args('-cpu', self.cpu) + if self.machine is not None: + self._vms[name].set_machine(self.machine) + return self._vms[name] + + def set_vm_arg(self, arg, value): + """ + Set an argument to list of extra arguments to be given to the QEMU + binary. If the argument already exists then its value is replaced. + + :param arg: the QEMU argument, such as "-cpu" in "-cpu host" + :type arg: str + :param value: the argument value, such as "host" in "-cpu host" + :type value: str + """ + if not arg or not value: + return + if arg not in self.vm.args: + self.vm.args.extend([arg, value]) + else: + idx = self.vm.args.index(arg) + 1 + if idx < len(self.vm.args): + self.vm.args[idx] = value + else: + self.vm.args.append(value) + + def tearDown(self): + for vm in self._vms.values(): + vm.shutdown() + super().tearDown() From patchwork Tue Jul 16 11:26:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 13734475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D38A6C41513 for ; Tue, 16 Jul 2024 11:30:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sTgKN-0008V9-H3; Tue, 16 Jul 2024 07:26:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKL-0008TT-S9 for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKH-0003dk-Uh for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721129193; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eOQyPkgbOHdxK2vV5sE/Al50Xpo82g2Pi5D+dxkdVGU=; b=N6g2FuzMaF89eXcS+SHRbSfJqiWfLlkg80wQ/GYHT7AuKwL6w9NxwAh0v9RW5IsxgsDqAc kNh+GWSxfzwujYfXuAmQ8V4m+uVZkNFS+qEPU2ihvA/4VS4+G8ZjE46amQCmEMKlM6SNwD FqGIApXxTyCNu3Zb9KuTdekuToJcJsg= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-673-Citnl_BnM8Oa_WY0VEMiqw-1; Tue, 16 Jul 2024 07:26:30 -0400 X-MC-Unique: Citnl_BnM8Oa_WY0VEMiqw-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D386C1944AB5; Tue, 16 Jul 2024 11:26:28 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.192.154]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CCAE31955F40; Tue, 16 Jul 2024 11:26:24 +0000 (UTC) From: Thomas Huth To: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Cc: Ani Sinha , Richard Henderson , Paolo Bonzini , "Daniel P . Berrange" , John Snow , =?utf-8?q?C=C3=A9dric_Le_Goater?= , Fabiano Rosas Subject: [PATCH 02/11] tests/functional: Convert simple avocado tests into standalone python tests Date: Tue, 16 Jul 2024 13:26:05 +0200 Message-ID: <20240716112614.1755692-3-thuth@redhat.com> In-Reply-To: <20240716112614.1755692-1-thuth@redhat.com> References: <20240716112614.1755692-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org These test are rather simple and don't need any modifications apart from adjusting the "from avocado_qemu" line. To ease debugging, make the files executable and add a shebang line and Python '__main__' handling, too, so that these tests can now be run by executing them directly. Signed-off-by: Thomas Huth --- .../test_cpu_queries.py} | 7 ++- .../test_empty_cpu_model.py} | 7 ++- .../test_mem_addr_space.py} | 53 +++---------------- .../test_pc_cpu_hotplug_props.py} | 11 ++-- .../test_virtio_version.py} | 8 +-- 5 files changed, 29 insertions(+), 57 deletions(-) rename tests/{avocado/cpu_queries.py => functional/test_cpu_queries.py} (89%) mode change 100644 => 100755 rename tests/{avocado/empty_cpu_model.py => functional/test_empty_cpu_model.py} (84%) mode change 100644 => 100755 rename tests/{avocado/mem-addr-space-check.py => functional/test_mem_addr_space.py} (92%) mode change 100644 => 100755 rename tests/{avocado/pc_cpu_hotplug_props.py => functional/test_pc_cpu_hotplug_props.py} (90%) mode change 100644 => 100755 rename tests/{avocado/virtio_version.py => functional/test_virtio_version.py} (98%) mode change 100644 => 100755 diff --git a/tests/avocado/cpu_queries.py b/tests/functional/test_cpu_queries.py old mode 100644 new mode 100755 similarity index 89% rename from tests/avocado/cpu_queries.py rename to tests/functional/test_cpu_queries.py index d3faa14720..97dd600c01 --- a/tests/avocado/cpu_queries.py +++ b/tests/functional/test_cpu_queries.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Sanity check of query-cpu-* results # # Copyright (c) 2019 Red Hat, Inc. @@ -8,7 +10,7 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest class QueryCPUModelExpansion(QemuSystemTest): """ @@ -33,3 +35,6 @@ def test(self): e = self.vm.cmd('query-cpu-model-expansion', model=model, type='full') self.assertEqual(e['model']['name'], c['name']) + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/empty_cpu_model.py b/tests/functional/test_empty_cpu_model.py old mode 100644 new mode 100755 similarity index 84% rename from tests/avocado/empty_cpu_model.py rename to tests/functional/test_empty_cpu_model.py index d906ef3d3c..0081b06d85 --- a/tests/avocado/empty_cpu_model.py +++ b/tests/functional/test_empty_cpu_model.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Check for crash when using empty -cpu option # # Copyright (c) 2019 Red Hat, Inc. @@ -7,7 +9,7 @@ # # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest class EmptyCPUModel(QemuSystemTest): def test(self): @@ -17,3 +19,6 @@ def test(self): self.vm.wait() self.assertEqual(self.vm.exitcode(), 1, "QEMU exit code should be 1") self.assertRegex(self.vm.get_log(), r'-cpu option cannot be empty') + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/mem-addr-space-check.py b/tests/functional/test_mem_addr_space.py old mode 100644 new mode 100755 similarity index 92% rename from tests/avocado/mem-addr-space-check.py rename to tests/functional/test_mem_addr_space.py index 85541ea051..bb0cf062ca --- a/tests/avocado/mem-addr-space-check.py +++ b/tests/functional/test_mem_addr_space.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Check for crash when using memory beyond the available guest processor # address space. # @@ -8,8 +10,7 @@ # # SPDX-License-Identifier: GPL-2.0-or-later -from avocado_qemu import QemuSystemTest -import signal +from qemu_test import QemuSystemTest import time class MemAddrCheck(QemuSystemTest): @@ -23,9 +24,6 @@ class MemAddrCheck(QemuSystemTest): # for all 32-bit cases, pci64_hole_size is 0. def test_phybits_low_pse36(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - With pse36 feature ON, a processor has 36 bits of addressing. So it can access up to a maximum of 64GiB of memory. Memory hotplug region begins at 4 GiB boundary when "above_4g_mem_size" is 0 (this would be true when @@ -53,9 +51,6 @@ def test_phybits_low_pse36(self): def test_phybits_low_pae(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - With pae feature ON, a processor has 36 bits of addressing. So it can access up to a maximum of 64GiB of memory. Rest is the same as the case with pse36 above. @@ -73,9 +68,6 @@ def test_phybits_low_pae(self): def test_phybits_ok_pentium_pse36(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - Setting maxmem to 59.5G and making sure that QEMU can start with the same options as the failing case above with pse36 cpu feature. """ @@ -92,9 +84,6 @@ def test_phybits_ok_pentium_pse36(self): def test_phybits_ok_pentium_pae(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - Test is same as above but now with pae cpu feature turned on. Setting maxmem to 59.5G and making sure that QEMU can start fine with the same options as the case above. @@ -112,9 +101,6 @@ def test_phybits_ok_pentium_pae(self): def test_phybits_ok_pentium2(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - Pentium2 has 36 bits of addressing, so its same as pentium with pse36 ON. """ @@ -131,9 +117,6 @@ def test_phybits_ok_pentium2(self): def test_phybits_low_nonpse36(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - Pentium processor has 32 bits of addressing without pse36 or pae so it can access physical address up to 4 GiB. Setting maxmem to 4 GiB should make QEMU fail to start with "phys-bits too low" @@ -154,9 +137,6 @@ def test_phybits_low_nonpse36(self): # now lets test some 64-bit CPU cases. def test_phybits_low_tcg_q35_70_amd(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - For q35 7.1 machines and above, there is a HT window that starts at 1024 GiB and ends at 1 TiB - 1. If the max GPA falls in this range, "above_4G" memory is adjusted to start at 1 TiB boundary for AMD cpus @@ -183,9 +163,6 @@ def test_phybits_low_tcg_q35_70_amd(self): def test_phybits_low_tcg_q35_71_amd(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - AMD_HT_START is defined to be at 1012 GiB. So for q35 machines version > 7.0 and AMD cpus, instead of 1024 GiB limit for 40 bit processor address space, it has to be 1012 GiB , that is 12 GiB @@ -206,9 +183,6 @@ def test_phybits_low_tcg_q35_71_amd(self): def test_phybits_ok_tcg_q35_70_amd(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - Same as q35-7.0 AMD case except that here we check that QEMU can successfully start when maxmem is < 988G. """ @@ -225,9 +199,6 @@ def test_phybits_ok_tcg_q35_70_amd(self): def test_phybits_ok_tcg_q35_71_amd(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - Same as q35-7.1 AMD case except that here we check that QEMU can successfully start when maxmem is < 976G. """ @@ -244,9 +215,6 @@ def test_phybits_ok_tcg_q35_71_amd(self): def test_phybits_ok_tcg_q35_71_intel(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - Same parameters as test_phybits_low_tcg_q35_71_amd() but use Intel cpu instead. QEMU should start fine in this case as "above_4G" memory starts at 4G. @@ -265,9 +233,6 @@ def test_phybits_ok_tcg_q35_71_intel(self): def test_phybits_low_tcg_q35_71_amd_41bits(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - AMD processor with 41 bits. Max cpu hw address = 2 TiB. By setting maxram above 1012 GiB - 32 GiB - 4 GiB = 976 GiB, we can force "above_4G" memory to start at 1 TiB for q35-7.1 machines @@ -292,9 +257,6 @@ def test_phybits_low_tcg_q35_71_amd_41bits(self): def test_phybits_ok_tcg_q35_71_amd_41bits(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - AMD processor with 41 bits. Max cpu hw address = 2 TiB. Same as above but by setting maxram between 976 GiB and 992 Gib, QEMU should start fine. @@ -313,9 +275,6 @@ def test_phybits_ok_tcg_q35_71_amd_41bits(self): def test_phybits_low_tcg_q35_intel_cxl(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - cxl memory window starts after memory device range. Here, we use 1 GiB of cxl window memory. 4G_mem end aligns at 4G. pci64_hole is 32 GiB and starts after the cxl memory window. @@ -336,9 +295,6 @@ def test_phybits_low_tcg_q35_intel_cxl(self): def test_phybits_ok_tcg_q35_intel_cxl(self): """ - :avocado: tags=machine:q35 - :avocado: tags=arch:x86_64 - Same as above but here we do not reserve any cxl memory window. Hence, with the exact same parameters as above, QEMU should start fine even with cxl enabled. @@ -353,3 +309,6 @@ def test_phybits_ok_tcg_q35_intel_cxl(self): time.sleep(self.DELAY_Q35_BOOT_SEQUENCE) self.vm.shutdown() self.assertNotRegex(self.vm.get_log(), r'phys-bits too low') + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/pc_cpu_hotplug_props.py b/tests/functional/test_pc_cpu_hotplug_props.py old mode 100644 new mode 100755 similarity index 90% rename from tests/avocado/pc_cpu_hotplug_props.py rename to tests/functional/test_pc_cpu_hotplug_props.py index 4bd3e02665..9d5a37cb17 --- a/tests/avocado/pc_cpu_hotplug_props.py +++ b/tests/functional/test_pc_cpu_hotplug_props.py @@ -1,3 +1,4 @@ +#!/usr/bin/env python3 # # Ensure CPU die-id can be omitted on -device # @@ -20,16 +21,16 @@ # License along with this library; if not, see . # -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest class OmittedCPUProps(QemuSystemTest): - """ - :avocado: tags=arch:x86_64 - :avocado: tags=cpu:qemu64 - """ + def test_no_die_id(self): self.vm.add_args('-nodefaults', '-S') self.vm.add_args('-smp', '1,sockets=2,cores=2,threads=2,maxcpus=8') self.vm.add_args('-device', 'qemu64-x86_64-cpu,socket-id=1,core-id=0,thread-id=0') self.vm.launch() self.assertEqual(len(self.vm.cmd('query-cpus-fast')), 2) + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/virtio_version.py b/tests/functional/test_virtio_version.py old mode 100644 new mode 100755 similarity index 98% rename from tests/avocado/virtio_version.py rename to tests/functional/test_virtio_version.py index afe5e828b5..eb23060564 --- a/tests/avocado/virtio_version.py +++ b/tests/functional/test_virtio_version.py @@ -1,3 +1,4 @@ +#!/usr/bin/env python3 """ Check compatibility of virtio device types """ @@ -12,7 +13,7 @@ import os from qemu.machine import QEMUMachine -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest # Virtio Device IDs: VIRTIO_NET = 1 @@ -60,8 +61,6 @@ class VirtioVersionCheck(QemuSystemTest): Check if virtio-version-specific device types result in the same device tree created by `disable-modern` and `disable-legacy`. - - :avocado: tags=arch:x86_64 """ # just in case there are failures, show larger diff: @@ -173,3 +172,6 @@ def test_modern_only_devs(self): self.check_modern_only('virtio-mouse-pci', VIRTIO_INPUT) self.check_modern_only('virtio-tablet-pci', VIRTIO_INPUT) self.check_modern_only('virtio-keyboard-pci', VIRTIO_INPUT) + +if __name__ == '__main__': + QemuSystemTest.main() From patchwork Tue Jul 16 11:26:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 13734473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CA366C3DA59 for ; Tue, 16 Jul 2024 11:30:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sTgKS-0000NE-7Q; Tue, 16 Jul 2024 07:26:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKQ-0000BE-Ft for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKN-0003eM-LQ for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721129199; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oWk6OiijvNFxzzsXUSJ3tpnyPsRwaP/81mITTt4wWbY=; b=DqxfA6VSvveUlCZdlsE0pEkBn0FSeee1+lFH+BAL9MP61KMN7rRiNk3Qspq0qeBZ1FC8TC jTyAHJn8A6rz6qotIsqkd50/O0mnmAm9iBTNEzDYIs1/WNUZx2PDi+71bUv9kf/vdAIkDh qC8FxnYOnQxikQ549011a8+s6LFikyQ= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-376--5NRluNtN7KM8eg3D6n7-Q-1; Tue, 16 Jul 2024 07:26:34 -0400 X-MC-Unique: -5NRluNtN7KM8eg3D6n7-Q-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 627211956080; Tue, 16 Jul 2024 11:26:33 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.192.154]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B9F5E1955E76; Tue, 16 Jul 2024 11:26:28 +0000 (UTC) From: Thomas Huth To: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Cc: Ani Sinha , Richard Henderson , Paolo Bonzini , "Daniel P . Berrange" , John Snow , =?utf-8?q?C=C3=A9dric_Le_Goater?= , Fabiano Rosas Subject: [PATCH 03/11] tests/functional: Convert avocado tests that just need a small adjustment Date: Tue, 16 Jul 2024 13:26:06 +0200 Message-ID: <20240716112614.1755692-4-thuth@redhat.com> In-Reply-To: <20240716112614.1755692-1-thuth@redhat.com> References: <20240716112614.1755692-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org These simple tests can be converted to stand-alone tests quite easily, e.g. by just setting the machine to 'none' now manually or by adding "-cpu" command line parameters, since we don't support the corresponding avocado tags in the new python test framework. Signed-off-by: Thomas Huth Reviewed-by: Daniel P. Berrangé --- .../test_info_usernet.py} | 11 ++- .../test_ppc_74xx.py} | 74 ++++++++----------- .../version.py => functional/test_version.py} | 13 ++-- 3 files changed, 47 insertions(+), 51 deletions(-) rename tests/{avocado/info_usernet.py => functional/test_info_usernet.py} (87%) mode change 100644 => 100755 rename tests/{avocado/ppc_74xx.py => functional/test_ppc_74xx.py} (74%) mode change 100644 => 100755 rename tests/{avocado/version.py => functional/test_version.py} (78%) mode change 100644 => 100755 diff --git a/tests/avocado/info_usernet.py b/tests/functional/test_info_usernet.py old mode 100644 new mode 100755 similarity index 87% rename from tests/avocado/info_usernet.py rename to tests/functional/test_info_usernet.py index e1aa7a6e0a..ded973253a --- a/tests/avocado/info_usernet.py +++ b/tests/functional/test_info_usernet.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Test for the hmp command "info usernet" # # Copyright (c) 2021 Red Hat, Inc. @@ -8,18 +10,16 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest from qemu.utils import get_info_usernet_hostfwd_port class InfoUsernet(QemuSystemTest): - """ - :avocado: tags=machine:none - """ def test_hostfwd(self): self.require_netdev('user') + self.machine = 'none' self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22') self.vm.launch() res = self.vm.cmd('human-monitor-command', @@ -31,3 +31,6 @@ def test_hostfwd(self): self.assertGreater(port, 0, ('Found a redirected port that is not greater than' ' zero')) + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/ppc_74xx.py b/tests/functional/test_ppc_74xx.py old mode 100644 new mode 100755 similarity index 74% rename from tests/avocado/ppc_74xx.py rename to tests/functional/test_ppc_74xx.py index f54757c243..5386016f26 --- a/tests/avocado/ppc_74xx.py +++ b/tests/functional/test_ppc_74xx.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Smoke tests for 74xx cpus (aka G4). # # Copyright (c) 2021, IBM Corp. @@ -5,132 +7,120 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern +from qemu_test import QemuSystemTest +from qemu_test import wait_for_console_pattern class ppc74xxCpu(QemuSystemTest): - """ - :avocado: tags=arch:ppc - :avocado: tags=accel:tcg - """ + timeout = 5 def test_ppc_7400(self): - """ - :avocado: tags=cpu:7400 - """ self.require_accelerator("tcg") + self.set_machine('g3beige') self.vm.set_console() + self.vm.add_args('-cpu', '7400') self.vm.launch() wait_for_console_pattern(self, '>> OpenBIOS') wait_for_console_pattern(self, '>> CPU type PowerPC,G4') def test_ppc_7410(self): - """ - :avocado: tags=cpu:7410 - """ self.require_accelerator("tcg") + self.set_machine('g3beige') self.vm.set_console() + self.vm.add_args('-cpu', '7410') self.vm.launch() wait_for_console_pattern(self, '>> OpenBIOS') wait_for_console_pattern(self, '>> CPU type PowerPC,74xx') def test_ppc_7441(self): - """ - :avocado: tags=cpu:7441 - """ self.require_accelerator("tcg") + self.set_machine('g3beige') self.vm.set_console() + self.vm.add_args('-cpu', '7441') self.vm.launch() wait_for_console_pattern(self, '>> OpenBIOS') wait_for_console_pattern(self, '>> CPU type PowerPC,G4') def test_ppc_7445(self): - """ - :avocado: tags=cpu:7445 - """ self.require_accelerator("tcg") + self.set_machine('g3beige') self.vm.set_console() + self.vm.add_args('-cpu', '7445') self.vm.launch() wait_for_console_pattern(self, '>> OpenBIOS') wait_for_console_pattern(self, '>> CPU type PowerPC,G4') def test_ppc_7447(self): - """ - :avocado: tags=cpu:7447 - """ self.require_accelerator("tcg") + self.set_machine('g3beige') self.vm.set_console() + self.vm.add_args('-cpu', '7447') self.vm.launch() wait_for_console_pattern(self, '>> OpenBIOS') wait_for_console_pattern(self, '>> CPU type PowerPC,G4') def test_ppc_7447a(self): - """ - :avocado: tags=cpu:7447a - """ self.require_accelerator("tcg") + self.set_machine('g3beige') self.vm.set_console() + self.vm.add_args('-cpu', '7447a') self.vm.launch() wait_for_console_pattern(self, '>> OpenBIOS') wait_for_console_pattern(self, '>> CPU type PowerPC,G4') def test_ppc_7448(self): - """ - :avocado: tags=cpu:7448 - """ self.require_accelerator("tcg") + self.set_machine('g3beige') self.vm.set_console() + self.vm.add_args('-cpu', '7448') self.vm.launch() wait_for_console_pattern(self, '>> OpenBIOS') wait_for_console_pattern(self, '>> CPU type PowerPC,MPC86xx') def test_ppc_7450(self): - """ - :avocado: tags=cpu:7450 - """ self.require_accelerator("tcg") + self.set_machine('g3beige') self.vm.set_console() + self.vm.add_args('-cpu', '7450') self.vm.launch() wait_for_console_pattern(self, '>> OpenBIOS') wait_for_console_pattern(self, '>> CPU type PowerPC,G4') def test_ppc_7451(self): - """ - :avocado: tags=cpu:7451 - """ self.require_accelerator("tcg") + self.set_machine('g3beige') self.vm.set_console() + self.vm.add_args('-cpu', '7451') self.vm.launch() wait_for_console_pattern(self, '>> OpenBIOS') wait_for_console_pattern(self, '>> CPU type PowerPC,G4') def test_ppc_7455(self): - """ - :avocado: tags=cpu:7455 - """ self.require_accelerator("tcg") + self.set_machine('g3beige') self.vm.set_console() + self.vm.add_args('-cpu', '7455') self.vm.launch() wait_for_console_pattern(self, '>> OpenBIOS') wait_for_console_pattern(self, '>> CPU type PowerPC,G4') def test_ppc_7457(self): - """ - :avocado: tags=cpu:7457 - """ self.require_accelerator("tcg") + self.set_machine('g3beige') self.vm.set_console() + self.vm.add_args('-cpu', '7457') self.vm.launch() wait_for_console_pattern(self, '>> OpenBIOS') wait_for_console_pattern(self, '>> CPU type PowerPC,G4') def test_ppc_7457a(self): - """ - :avocado: tags=cpu:7457a - """ self.require_accelerator("tcg") + self.set_machine('g3beige') self.vm.set_console() + self.vm.add_args('-cpu', '7457a') self.vm.launch() wait_for_console_pattern(self, '>> OpenBIOS') wait_for_console_pattern(self, '>> CPU type PowerPC,G4') + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/version.py b/tests/functional/test_version.py old mode 100644 new mode 100755 similarity index 78% rename from tests/avocado/version.py rename to tests/functional/test_version.py index c6139568a1..5e566d76b1 --- a/tests/avocado/version.py +++ b/tests/functional/test_version.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Version check example test # # Copyright (c) 2018 Red Hat, Inc. @@ -9,17 +11,18 @@ # later. See the COPYING file in the top-level directory. -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest class Version(QemuSystemTest): - """ - :avocado: tags=quick - :avocado: tags=machine:none - """ + def test_qmp_human_info_version(self): + self.machine = 'none' self.vm.add_args('-nodefaults') self.vm.launch() res = self.vm.cmd('human-monitor-command', command_line='info version') self.assertRegex(res, r'^(\d+\.\d+\.\d)') + +if __name__ == '__main__': + QemuSystemTest.main() From patchwork Tue Jul 16 11:26:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 13734462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8B236C3DA59 for ; Tue, 16 Jul 2024 11:27:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sTgKS-0000Pf-RR; Tue, 16 Jul 2024 07:26:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKR-0000Js-Fe for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKP-0003eb-Pl for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721129201; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CbeK7o2Avj9RP/nuuizYkz3rpCRVuPxR4fikDUeYbMw=; b=TPtuyzcDxmslnunUvtxPwUNLJHmljpJbXluohPTXgj8iqq7AHnZbsSZzjfZgG1kGE9V9oK OgDwnzMruZCYZye8b+hDTTESEtHge1Yq+mVBqW2oEeSngda1lP3DlSJLTGlE1+tHlFfY+c LEemwe5q6/tWbswkkHOG301EHTBagj0= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-66-kUWn3jTQNauKBvXuQNwphw-1; Tue, 16 Jul 2024 07:26:38 -0400 X-MC-Unique: kUWn3jTQNauKBvXuQNwphw-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B83871944A9B; Tue, 16 Jul 2024 11:26:36 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.192.154]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6E7101955F40; Tue, 16 Jul 2024 11:26:33 +0000 (UTC) From: Thomas Huth To: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Cc: Ani Sinha , Richard Henderson , Paolo Bonzini , "Daniel P . Berrange" , John Snow , =?utf-8?q?C=C3=A9dric_Le_Goater?= , Fabiano Rosas Subject: [PATCH 04/11] tests/functional: Add python-based tests to the meson build system Date: Tue, 16 Jul 2024 13:26:07 +0200 Message-ID: <20240716112614.1755692-5-thuth@redhat.com> In-Reply-To: <20240716112614.1755692-1-thuth@redhat.com> References: <20240716112614.1755692-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Integrate the new python-based test framework with the meson build system. Since these tests now require the pycotap module, make sure that it gets installed in the venv. The changes to the meson.build files are partly based on an earlier patch by Ani Sinha (but heavily modified by Thomas Huth e.g. to use pycotap for running the tests instead). Signed-off-by: Thomas Huth Reviewed-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé --- pythondeps.toml | 3 +- tests/Makefile.include | 18 ++++++++- tests/functional/meson.build | 75 ++++++++++++++++++++++++++++++++++++ tests/meson.build | 1 + 4 files changed, 95 insertions(+), 2 deletions(-) create mode 100644 tests/functional/meson.build diff --git a/pythondeps.toml b/pythondeps.toml index f6e590fdd8..c018b4d74a 100644 --- a/pythondeps.toml +++ b/pythondeps.toml @@ -26,9 +26,10 @@ meson = { accepted = ">=1.1.0", installed = "1.2.3", canary = "meson" } sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" } sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" } -[avocado] +[tests] # Note that qemu.git/python/ is always implicitly installed. # Prefer an LTS version when updating the accepted versions of # avocado-framework, for example right now the limit is 92.x. avocado-framework = { accepted = "(>=88.1, <93.0)", installed = "88.1", canary = "avocado" } pycdlib = { accepted = ">=1.11.0" } +pycotap = { accepted = ">=1.1.0" } diff --git a/tests/Makefile.include b/tests/Makefile.include index d39d5dd6a4..2bdf607977 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -9,6 +9,8 @@ check-help: @echo "Individual test suites:" @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target" @echo " $(MAKE) check-qtest Run qtest tests" + @echo " $(MAKE) check-functional Run python-based functional tests" + @echo " $(MAKE) check-functional-TARG Run functional tests for a given target" @echo " $(MAKE) check-unit Run qobject tests" @echo " $(MAKE) check-qapi-schema Run QAPI schema tests" @echo " $(MAKE) check-block Run block tests" @@ -111,7 +113,7 @@ quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \ $(TESTS_VENV_TOKEN): $(SRC_PATH)/pythondeps.toml $(call quiet-venv-pip,install -e "$(SRC_PATH)/python/") - $(MKVENV_ENSUREGROUP) $< avocado + $(MKVENV_ENSUREGROUP) $< tests $(call quiet-command, touch $@) $(TESTS_RESULTS_DIR): @@ -152,6 +154,20 @@ check-acceptance-deprecated-warning: check-acceptance: check-acceptance-deprecated-warning | check-avocado +# Make sure that pycotap is installed before running any functional tests: +ifneq ($(filter check-func%,$(MAKECMDGOALS))$(filter check,$(MAKECMDGOALS)),) +do-meson-check: check-venv +endif + +FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$(TARGETS))) +.PHONY: $(FUNCTIONAL_TARGETS) +$(FUNCTIONAL_TARGETS): + @make SPEED=thorough $(subst -functional,-func,$@) + +.PHONY: check-functional +check-functional: + @make SPEED=thorough check-func check-func-quick + # Consolidated targets .PHONY: check check-clean get-vm-images diff --git a/tests/functional/meson.build b/tests/functional/meson.build new file mode 100644 index 0000000000..11352b5bb5 --- /dev/null +++ b/tests/functional/meson.build @@ -0,0 +1,75 @@ +# QEMU functional tests: +# Tests that are put in the 'quick' category are run by default during +# 'make check'. Everything that should not be run during 'make check' +# (e.g. tests that fetch assets from the internet) should be put into +# the 'thorough' category instead. + +# Most tests run too slow with TCI enabled, so skip the functional tests there +if get_option('tcg_interpreter') + subdir_done() +endif + +# Timeouts for individual tests that can be slow e.g. with debugging enabled +test_timeouts = { + 'ppc_74xx' : 90, +} + +tests_generic = [ + 'empty_cpu_model', + 'info_usernet', + 'version', +] + +tests_ppc_quick = [ + 'ppc_74xx', +] + +tests_x86_64_quick = [ + 'cpu_queries', + 'mem_addr_space', + 'pc_cpu_hotplug_props', + 'virtio_version', +] + +foreach speed : ['quick', 'thorough'] + foreach dir : target_dirs + if not dir.endswith('-softmmu') + continue + endif + + target_base = dir.split('-')[0] + test_emulator = emulators['qemu-system-' + target_base] + + if speed == 'quick' + suites = ['func-quick', 'func-' + target_base] + target_tests = get_variable('tests_' + target_base + '_quick', []) + tests_generic + else + suites = ['func-' + speed, 'func-' + target_base + '-' + speed, speed] + target_tests = get_variable('tests_' + target_base + '_' + speed, []) + endif + + test_deps = roms + test_env = environment() + if have_tools + test_env.set('QEMU_TEST_QEMU_IMG', meson.global_build_root() / 'qemu-img') + test_deps += [qemu_img] + endif + test_env.set('QEMU_TEST_QEMU_BINARY', + meson.global_build_root() / 'qemu-system-' + target_base) + test_env.set('QEMU_BUILD_ROOT', meson.project_build_root()) + test_env.set('PYTHONPATH', meson.project_source_root() / 'python:' + + meson.current_source_dir()) + + foreach test : target_tests + test('func-@0@/@1@'.format(target_base, test), + python, + depends: [test_deps, test_emulator, emulator_modules], + env: test_env, + args: [meson.current_source_dir() / 'test_' + test + '.py'], + protocol: 'tap', + timeout: test_timeouts.get(test, 60), + priority: test_timeouts.get(test, 60), + suite: suites) + endforeach + endforeach +endforeach diff --git a/tests/meson.build b/tests/meson.build index acb6807094..3345ad2098 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -85,3 +85,4 @@ subdir('unit') subdir('qapi-schema') subdir('qtest') subdir('migration') +subdir('functional') From patchwork Tue Jul 16 11:26:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 13734472 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E9176C3DA49 for ; Tue, 16 Jul 2024 11:30:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sTgKW-0000eL-I0; Tue, 16 Jul 2024 07:26:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKV-0000Z7-6C for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKT-0003fB-OF for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721129205; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XHN7AYgHuHljyH4exkRaU7dMz5pCsBSrCydjHlqdXLk=; b=gSuD+2iN/cZXJcH2wgJCSaIFxP+WGI9d2dSUee4PDRbS0jVzlaGP/5v0M9T8hyDdVl2zbY wDwbDtL+OinA+JMZJrWyOFSnNcZUJgCgcEEYwj4Q3ic9hmB70ezN1iOSLVcztsbrJcVf28 qyHLcjqFCay7L9VhqEV4pGYZLKQxKlk= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-14--R4wowlLNDi4WVuRkOwxlw-1; Tue, 16 Jul 2024 07:26:41 -0400 X-MC-Unique: -R4wowlLNDi4WVuRkOwxlw-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A345E1944B25; Tue, 16 Jul 2024 11:26:40 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.192.154]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2A2B31955F40; Tue, 16 Jul 2024 11:26:36 +0000 (UTC) From: Thomas Huth To: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Cc: Ani Sinha , Richard Henderson , Paolo Bonzini , "Daniel P . Berrange" , John Snow , =?utf-8?q?C=C3=A9dric_Le_Goater?= , Fabiano Rosas Subject: [PATCH 05/11] tests/functional: Implement fetch_asset() method for downloading assets Date: Tue, 16 Jul 2024 13:26:08 +0200 Message-ID: <20240716112614.1755692-6-thuth@redhat.com> In-Reply-To: <20240716112614.1755692-1-thuth@redhat.com> References: <20240716112614.1755692-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In the new python test framework, we cannot use the fetch_asset() function from Avocado anymore, so we have to provide our own implementation now instead. Thus add such a function based on the urllib python module for this purpose. Signed-off-by: Thomas Huth --- tests/functional/qemu_test/__init__.py | 36 ++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py index e73705d40a..77c3b14d34 100644 --- a/tests/functional/qemu_test/__init__.py +++ b/tests/functional/qemu_test/__init__.py @@ -11,6 +11,8 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. +import hashlib +import urllib.request import logging import os import pycotap @@ -23,6 +25,7 @@ import unittest from pathlib import Path +from shutil import copyfileobj from qemu.machine import QEMUMachine from qemu.utils import (get_info_usernet_hostfwd_port, kvm_available, tcg_available) @@ -215,6 +218,39 @@ def setUp(self, bin_prefix): if not os.path.exists(self.workdir): os.makedirs(self.workdir) + def check_hash(self, file_name, expected_hash): + if not expected_hash: + return True + if len(expected_hash) == 40: + sum_prog = 'sha1sum' + elif len(expected_hash) == 64: + sum_prog = 'sha256sum' + elif len(expected_hash) == 128: + sum_prog = 'sha512sum' + else: + raise Exception("unknown hash type") + checksum = subprocess.check_output([sum_prog, file_name]).split()[0] + return expected_hash == checksum.decode("utf-8") + + def fetch_asset(self, url, asset_hash): + cache_dir = os.path.expanduser("~/.cache/qemu/download") + if not os.path.exists(cache_dir): + os.makedirs(cache_dir) + fname = os.path.join(cache_dir, + hashlib.sha256(url.encode("utf-8")).hexdigest()) + if os.path.exists(fname) and self.check_hash(fname, asset_hash): + return fname + self.log.info("Downloading %s to %s...", url, fname) + dl_fname = fname + ".download" + with urllib.request.urlopen(url) as src: + with open(dl_fname, "wb+") as dst: + copyfileobj(src, dst) + if not self.check_hash(dl_fname, asset_hash): + os.remove(dl_fname) + raise Exception("Hash of " + url + " does not match") + os.rename(dl_fname, fname) + return fname + def main(): tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError, test_output_log = pycotap.LogMode.LogToError) From patchwork Tue Jul 16 11:26:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 13734476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69FBBC3DA49 for ; Tue, 16 Jul 2024 11:30:32 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sTgLO-00021R-2T; Tue, 16 Jul 2024 07:27:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKa-0000tN-By for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:57 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKX-0003fk-KQ for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721129208; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dDxgDCCPrRxr8FRSrXz9EbtvhGE0KGOGi82LXJLMCZQ=; b=cPHeUQSfgTBTTab04bqJCVX8DDF3xtNyc3koilmXJgDAX9mBrtVJDcxs9hnqPLtG+eo/0S VCxSiBe7pZSAOtOgYK4eNJrN6PvugKgmoDpAmXFViQt+zWv9xd3uQmrpRSxGshp1h8qC+e W+8223nFB1Oj7CNoKkRmAKKW8UhCo6k= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-548-olWGSEYEM5GnpODS5kxVgw-1; Tue, 16 Jul 2024 07:26:45 -0400 X-MC-Unique: olWGSEYEM5GnpODS5kxVgw-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 605F41944D33; Tue, 16 Jul 2024 11:26:44 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.192.154]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id F00F81955F40; Tue, 16 Jul 2024 11:26:40 +0000 (UTC) From: Thomas Huth To: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Cc: Ani Sinha , Richard Henderson , Paolo Bonzini , "Daniel P . Berrange" , John Snow , =?utf-8?q?C=C3=A9dric_Le_Goater?= , Fabiano Rosas Subject: [PATCH 06/11] tests/functional: Convert some tests that download files via fetch_asset() Date: Tue, 16 Jul 2024 13:26:09 +0200 Message-ID: <20240716112614.1755692-7-thuth@redhat.com> In-Reply-To: <20240716112614.1755692-1-thuth@redhat.com> References: <20240716112614.1755692-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Now that we've got a working fetch_asset() function, we can convert some Avocado tests that use this function for downloading their required files. Signed-off-by: Thomas Huth Reviewed-by: Daniel P. Berrangé --- MAINTAINERS | 12 +++---- tests/functional/meson.build | 25 +++++++++++++++ .../test_arm_n8x0.py} | 25 +++++++-------- .../test_avr_mega2560.py} | 11 ++++--- .../test_loongarch64_virt.py} | 16 ++++++---- .../test_mips64el_loongson3v.py} | 26 +++++++-------- .../test_netdev_ethtool.py} | 32 ++++++------------- .../ppc_405.py => functional/test_ppc_405.py} | 19 ++++++----- 8 files changed, 89 insertions(+), 77 deletions(-) rename tests/{avocado/machine_arm_n8x0.py => functional/test_arm_n8x0.py} (71%) mode change 100644 => 100755 rename tests/{avocado/machine_avr6.py => functional/test_avr_mega2560.py} (90%) mode change 100644 => 100755 rename tests/{avocado/machine_loongarch.py => functional/test_loongarch64_virt.py} (89%) mode change 100644 => 100755 rename tests/{avocado/machine_mips_loongson3v.py => functional/test_mips64el_loongson3v.py} (55%) mode change 100644 => 100755 rename tests/{avocado/netdev-ethtool.py => functional/test_netdev_ethtool.py} (81%) mode change 100644 => 100755 rename tests/{avocado/ppc_405.py => functional/test_ppc_405.py} (73%) mode change 100644 => 100755 diff --git a/MAINTAINERS b/MAINTAINERS index 7d9811458c..533e22b548 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -221,7 +221,7 @@ S: Maintained F: docs/system/target-avr.rst F: gdb-xml/avr-cpu.xml F: target/avr/ -F: tests/avocado/machine_avr6.py +F: tests/functional/test_avr_mega2560.py CRIS TCG CPUs M: Edgar E. Iglesias @@ -264,7 +264,7 @@ M: Song Gao S: Maintained F: target/loongarch/ F: tests/tcg/loongarch64/ -F: tests/avocado/machine_loongarch.py +F: tests/functional/test_loongarch64_virt.py M68K TCG CPUs M: Laurent Vivier @@ -883,7 +883,7 @@ F: include/hw/display/blizzard.h F: include/hw/input/lm832x.h F: include/hw/input/tsc2xxx.h F: include/hw/misc/cbus.h -F: tests/avocado/machine_arm_n8x0.py +F: tests/functional/test_arm_n8x0.py F: docs/system/arm/nseries.rst Palm @@ -1392,7 +1392,7 @@ F: hw/mips/loongson3_bootp.h F: hw/mips/loongson3_virt.c F: include/hw/intc/loongson_ipi.h F: include/hw/intc/loongson_liointc.h -F: tests/avocado/machine_mips_loongson3v.py +F: tests/functional/test_mips64el_loongson3v.py Boston M: Paul Burton @@ -1418,7 +1418,7 @@ PowerPC Machines L: qemu-ppc@nongnu.org S: Orphan F: hw/ppc/ppc405* -F: tests/avocado/ppc_405.py +F: tests/functional/test_ppc_405.py Bamboo L: qemu-ppc@nongnu.org @@ -2483,7 +2483,7 @@ R: Sriram Yagnaraman S: Maintained F: docs/system/devices/igb.rst F: hw/net/igb* -F: tests/avocado/netdev-ethtool.py +F: tests/functional/test_netdev_ethtool.py F: tests/qtest/igb-test.c F: tests/qtest/libqos/igb.c diff --git a/tests/functional/meson.build b/tests/functional/meson.build index 11352b5bb5..947f9ac74f 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -11,6 +11,7 @@ endif # Timeouts for individual tests that can be slow e.g. with debugging enabled test_timeouts = { + 'netdev_ethtool' : 180, 'ppc_74xx' : 90, } @@ -20,10 +21,30 @@ tests_generic = [ 'version', ] +tests_arm_thorough = [ + 'arm_n8x0', +] + +tests_avr_thorough = [ + 'avr_mega2560', +] + +tests_loongarch64_thorough = [ + 'loongarch64_virt', +] + +tests_mips64el_thorough = [ + 'mips64el_loongson3v', +] + tests_ppc_quick = [ 'ppc_74xx', ] +tests_ppc_thorough = [ + 'ppc_405', +] + tests_x86_64_quick = [ 'cpu_queries', 'mem_addr_space', @@ -31,6 +52,10 @@ tests_x86_64_quick = [ 'virtio_version', ] +tests_x86_64_thorough = [ + 'netdev_ethtool', +] + foreach speed : ['quick', 'thorough'] foreach dir : target_dirs if not dir.endswith('-softmmu') diff --git a/tests/avocado/machine_arm_n8x0.py b/tests/functional/test_arm_n8x0.py old mode 100644 new mode 100755 similarity index 71% rename from tests/avocado/machine_arm_n8x0.py rename to tests/functional/test_arm_n8x0.py index 12e9a6803b..d451c80a73 --- a/tests/avocado/machine_arm_n8x0.py +++ b/tests/functional/test_arm_n8x0.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Functional test that boots a Linux kernel and checks the console # # Copyright (c) 2020 Red Hat, Inc. @@ -10,9 +12,9 @@ import os -from avocado import skipUnless -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern +from unittest import skipUnless +from qemu_test import QemuSystemTest +from qemu_test import wait_for_console_pattern class N8x0Machine(QemuSystemTest): """Boots the Linux kernel and checks that the console is operational""" @@ -32,18 +34,15 @@ def __do_test_n8x0(self): self.vm.launch() wait_for_console_pattern(self, 'TSC2005 driver initializing') - @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code') + @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code') def test_n800(self): - """ - :avocado: tags=arch:arm - :avocado: tags=machine:n800 - """ + self.set_machine('n800') self.__do_test_n8x0() - @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code') + @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code') def test_n810(self): - """ - :avocado: tags=arch:arm - :avocado: tags=machine:n810 - """ + self.set_machine('n810') self.__do_test_n8x0() + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/machine_avr6.py b/tests/functional/test_avr_mega2560.py old mode 100644 new mode 100755 similarity index 90% rename from tests/avocado/machine_avr6.py rename to tests/functional/test_avr_mega2560.py index 5485db79c6..b9b89367c6 --- a/tests/avocado/machine_avr6.py +++ b/tests/functional/test_avr_mega2560.py @@ -1,3 +1,4 @@ +#!/usr/bin/env python3 # # QEMU AVR integration tests # @@ -19,16 +20,12 @@ import time -from avocado_qemu import QemuSystemTest +from qemu_test import QemuSystemTest class AVR6Machine(QemuSystemTest): timeout = 5 def test_freertos(self): - """ - :avocado: tags=arch:avr - :avocado: tags=machine:arduino-mega-2560-v3 - """ """ https://github.com/seharris/qemu-avr-tests/raw/master/free-rtos/Demo/AVR_ATMega2560_GCC/demo.elf constantly prints out 'ABCDEFGHIJKLMNOPQRSTUVWXABCDEFGHIJKLMNOPQRSTUVWX' @@ -39,6 +36,7 @@ def test_freertos(self): rom_hash = '7eb521f511ca8f2622e0a3c5e8dd686efbb911d4' rom_path = self.fetch_asset(rom_url, asset_hash=rom_hash) + self.set_machine('arduino-mega-2560-v3') self.vm.add_args('-bios', rom_path) self.vm.add_args('-nographic') self.vm.launch() @@ -48,3 +46,6 @@ def test_freertos(self): self.assertIn('ABCDEFGHIJKLMNOPQRSTUVWXABCDEFGHIJKLMNOPQRSTUVWX', self.vm.get_log()) + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/machine_loongarch.py b/tests/functional/test_loongarch64_virt.py old mode 100644 new mode 100755 similarity index 89% rename from tests/avocado/machine_loongarch.py rename to tests/functional/test_loongarch64_virt.py index 8de308f2d6..3dc8bb3516 --- a/tests/avocado/machine_loongarch.py +++ b/tests/functional/test_loongarch64_virt.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # SPDX-License-Identifier: GPL-2.0-or-later # # LoongArch virt test. @@ -5,9 +7,9 @@ # Copyright (c) 2023 Loongson Technology Corporation Limited # -from avocado_qemu import QemuSystemTest -from avocado_qemu import exec_command_and_wait_for_pattern -from avocado_qemu import wait_for_console_pattern +from qemu_test import QemuSystemTest +from qemu_test import exec_command_and_wait_for_pattern +from qemu_test import wait_for_console_pattern class LoongArchMachine(QemuSystemTest): KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 ' @@ -21,10 +23,7 @@ def wait_for_console_pattern(self, success_message, vm=None): def test_loongarch64_devices(self): - """ - :avocado: tags=arch:loongarch64 - :avocado: tags=machine:virt - """ + self.set_machine('virt') kernel_url = ('https://github.com/yangxiaojuan-loongson/qemu-binary/' 'releases/download/2024-05-30/vmlinuz.efi') @@ -56,3 +55,6 @@ def test_loongarch64_devices(self): self.wait_for_console_pattern('Run /sbin/init as init process') exec_command_and_wait_for_pattern(self, 'cat /proc/cpuinfo', 'processor : 3') + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/machine_mips_loongson3v.py b/tests/functional/test_mips64el_loongson3v.py old mode 100644 new mode 100755 similarity index 55% rename from tests/avocado/machine_mips_loongson3v.py rename to tests/functional/test_mips64el_loongson3v.py index 5194cf18c9..f8d40b48a9 --- a/tests/avocado/machine_mips_loongson3v.py +++ b/tests/functional/test_mips64el_loongson3v.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Functional tests for the Generic Loongson-3 Platform. # # Copyright (c) 2021 Jiaxun Yang @@ -10,30 +12,26 @@ import os import time -from avocado import skipUnless -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern +from unittest import skipUnless +from qemu_test import QemuSystemTest +from qemu_test import wait_for_console_pattern class MipsLoongson3v(QemuSystemTest): timeout = 60 - @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code') + @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code') def test_pmon_serial_console(self): - """ - :avocado: tags=arch:mips64el - :avocado: tags=endian:little - :avocado: tags=machine:loongson3-virt - :avocado: tags=cpu:Loongson-3A1000 - :avocado: tags=device:liointc - :avocado: tags=device:goldfish_rtc - """ + self.set_machine('loongson3-virt') - pmon_hash = '7c8b45dd81ccfc55ff28f5aa267a41c3' + pmon_hash = 'fcdf6bb2cb7885a4a62f31fcb0d5e368bac7b6cea28f40c6dfa678af22fea20a' pmon_path = self.fetch_asset('https://github.com/loongson-community/pmon/' 'releases/download/20210112/pmon-3avirt.bin', - asset_hash=pmon_hash, algorithm='md5') + asset_hash=pmon_hash) self.vm.set_console() self.vm.add_args('-bios', pmon_path) self.vm.launch() wait_for_console_pattern(self, 'CPU GODSON3 BogoMIPS:') + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/netdev-ethtool.py b/tests/functional/test_netdev_ethtool.py old mode 100644 new mode 100755 similarity index 81% rename from tests/avocado/netdev-ethtool.py rename to tests/functional/test_netdev_ethtool.py index 5f33288f81..341c087e36 --- a/tests/avocado/netdev-ethtool.py +++ b/tests/functional/test_netdev_ethtool.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # ethtool tests for emulated network devices # # This test leverages ethtool's --test sequence to validate network @@ -5,15 +7,11 @@ # # SPDX-License-Identifier: GPL-2.0-or-late -from avocado import skip -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern +from unittest import skip +from qemu_test import QemuSystemTest +from qemu_test import wait_for_console_pattern class NetDevEthtool(QemuSystemTest): - """ - :avocado: tags=arch:x86_64 - :avocado: tags=machine:q35 - """ # Runs in about 17s under KVM, 19s under TCG, 25s under GCOV timeout = 45 @@ -25,11 +23,10 @@ def get_asset(self, name, sha1): 'kE4nCFLdQcoBF9t/download?' 'path=%2Fnetdev-ethtool&files=' ) url = base_url + name - # use explicit name rather than failing to neatly parse the - # URL into a unique one - return self.fetch_asset(name=name, locations=(url), asset_hash=sha1) + return self.fetch_asset(url, asset_hash=sha1) def common_test_code(self, netdev, extra_args=None): + self.set_machine('q35') # This custom kernel has drivers for all the supported network # devices we can emulate in QEMU @@ -68,15 +65,9 @@ def common_test_code(self, netdev, extra_args=None): self.vm.kill() def test_igb(self): - """ - :avocado: tags=device:igb - """ self.common_test_code("igb") def test_igb_nomsi(self): - """ - :avocado: tags=device:igb - """ self.common_test_code("igb", "pci=nomsi") # It seems the other popular cards we model in QEMU currently fail @@ -88,14 +79,11 @@ def test_igb_nomsi(self): @skip("Incomplete reg 0x00178 support") def test_e1000(self): - """ - :avocado: tags=device:e1000 - """ self.common_test_code("e1000") @skip("Incomplete reg 0x00178 support") def test_i82550(self): - """ - :avocado: tags=device:i82550 - """ self.common_test_code("i82550") + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/ppc_405.py b/tests/functional/test_ppc_405.py old mode 100644 new mode 100755 similarity index 73% rename from tests/avocado/ppc_405.py rename to tests/functional/test_ppc_405.py index 4e7e01aa76..f2368ada80 --- a/tests/avocado/ppc_405.py +++ b/tests/functional/test_ppc_405.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Test that the U-Boot firmware boots on ppc 405 machines and check the console # # Copyright (c) 2021 Red Hat, Inc. @@ -5,10 +7,9 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado.utils import archive -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern -from avocado_qemu import exec_command_and_wait_for_pattern +from qemu_test import QemuSystemTest +from qemu_test import wait_for_console_pattern +from qemu_test import exec_command_and_wait_for_pattern class Ppc405Machine(QemuSystemTest): @@ -26,11 +27,9 @@ def do_test_ppc405(self): exec_command_and_wait_for_pattern(self, 'reset', 'AMCC PowerPC 405EP') def test_ppc_ref405ep(self): - """ - :avocado: tags=arch:ppc - :avocado: tags=machine:ref405ep - :avocado: tags=cpu:405ep - :avocado: tags=accel:tcg - """ self.require_accelerator("tcg") + self.set_machine('ref405ep') self.do_test_ppc405() + +if __name__ == '__main__': + QemuSystemTest.main() From patchwork Tue Jul 16 11:26:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 13734464 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 762A6C3DA59 for ; Tue, 16 Jul 2024 11:28:29 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sTgL6-00018e-Hw; Tue, 16 Jul 2024 07:27:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKf-0000xC-RT for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:27:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKd-0003gJ-Vh for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721129214; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DSvA7AYriRHsYeaPvJp6kCACcsTi90Hf/v9y37k+a1Y=; b=HxkTt4sxD8I6mmrZuQPHuWApXNEzrU67P7QVqfMmBlBsqSjNdBbUkKjTdN2urG2pklZrrb maIFU2PTTZ+yrLzHh+RHMpts6BZ8uxNAeY6XQ/4SShUccH35vY7zZOof9snncKvtiWXQ2b ES9zhvSxjrPuiSyC4u/3DfQsgaB2SJ4= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-480-2wCU1QEUO0qjoRASY9iOhg-1; Tue, 16 Jul 2024 07:26:49 -0400 X-MC-Unique: 2wCU1QEUO0qjoRASY9iOhg-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 35EBE192DE01; Tue, 16 Jul 2024 11:26:48 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.192.154]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id AD0231955F40; Tue, 16 Jul 2024 11:26:44 +0000 (UTC) From: Thomas Huth To: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Cc: Ani Sinha , Richard Henderson , Paolo Bonzini , "Daniel P . Berrange" , John Snow , =?utf-8?q?C=C3=A9dric_Le_Goater?= , Fabiano Rosas Subject: [PATCH 07/11] tests/functional: Add a function for extracting files from an archive Date: Tue, 16 Jul 2024 13:26:10 +0200 Message-ID: <20240716112614.1755692-8-thuth@redhat.com> In-Reply-To: <20240716112614.1755692-1-thuth@redhat.com> References: <20240716112614.1755692-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Some Avocado-based tests use the "archive" module from avocado.utils to extract files from an archive. To be able to use these tests without Avocado, we have to provide our own function for extracting files. Fortunately, there is already the tarfile module that will provide us with this functionality, so let's just add a nice wrapper function around that. Signed-off-by: Thomas Huth Reviewed-by: Daniel P. Berrangé --- tests/functional/qemu_test/utils.py | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 tests/functional/qemu_test/utils.py diff --git a/tests/functional/qemu_test/utils.py b/tests/functional/qemu_test/utils.py new file mode 100644 index 0000000000..4eb5e5d5e5 --- /dev/null +++ b/tests/functional/qemu_test/utils.py @@ -0,0 +1,21 @@ +# Utilities for python-based QEMU tests +# +# Copyright 2024 Red Hat, Inc. +# +# Authors: +# Thomas Huth +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import tarfile + +def archive_extract(archive, dest_dir, member=None): + with tarfile.open(archive) as tf: + if hasattr(tarfile, 'data_filter'): + tf.extraction_filter = getattr(tarfile, 'data_filter', + (lambda member, path: member)) + if member: + tf.extract(member=member, path=dest_dir) + else: + tf.extractall(path=dest_dir) From patchwork Tue Jul 16 11:26:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 13734463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A450CC3DA49 for ; Tue, 16 Jul 2024 11:27:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sTgKu-00010L-RC; Tue, 16 Jul 2024 07:27:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKh-0000xV-93 for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKf-0003gS-2d for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:26:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721129216; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ut7/B3YSWJMlNJhfuXedKJXIltv1yhmAq/VM2uBUkSM=; b=D5z4xl10J4d73mqDzCsvoz/Q8Cfrh6Kod6qNZR919x2OEILlCBCfVte7iPn4VJnw3CJCc3 8bV09Ikujn1yWlM7t1HJSl+FasOn4UvPkTVaR+yy83mwSZo4YfNnOjsEDvpmlOgksZR5uz Mqijz+IO4uAhEAFJkXHWf+WxROd40eg= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-246-YT5X2P7ROOOf0rRAD1LDdA-1; Tue, 16 Jul 2024 07:26:53 -0400 X-MC-Unique: YT5X2P7ROOOf0rRAD1LDdA-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E997E1944AB2; Tue, 16 Jul 2024 11:26:51 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.192.154]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 836151955F40; Tue, 16 Jul 2024 11:26:48 +0000 (UTC) From: Thomas Huth To: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Cc: Ani Sinha , Richard Henderson , Paolo Bonzini , "Daniel P . Berrange" , John Snow , =?utf-8?q?C=C3=A9dric_Le_Goater?= , Fabiano Rosas Subject: [PATCH 08/11] tests/functional: Convert some avocado tests that needed avocado.utils.archive Date: Tue, 16 Jul 2024 13:26:11 +0200 Message-ID: <20240716112614.1755692-9-thuth@redhat.com> In-Reply-To: <20240716112614.1755692-1-thuth@redhat.com> References: <20240716112614.1755692-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Instead of using the "archive" module from avocado.utils, switch these tests to use the new wrapper function that is based on the "tarfile" module instead. Signed-off-by: Thomas Huth Reviewed-by: Daniel P. Berrangé --- MAINTAINERS | 6 ++--- tests/functional/meson.build | 6 +++++ .../test_arm_canona1100.py} | 21 +++++++++------- .../test_ppc_bamboo.py} | 23 ++++++++--------- .../test_sparc64_sun4u.py} | 25 +++++++++++-------- 5 files changed, 46 insertions(+), 35 deletions(-) rename tests/{avocado/machine_arm_canona1100.py => functional/test_arm_canona1100.py} (71%) mode change 100644 => 100755 rename tests/{avocado/ppc_bamboo.py => functional/test_ppc_bamboo.py} (75%) mode change 100644 => 100755 rename tests/{avocado/machine_sparc64_sun4u.py => functional/test_sparc64_sun4u.py} (60%) mode change 100644 => 100755 diff --git a/MAINTAINERS b/MAINTAINERS index 533e22b548..025227954c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -730,7 +730,7 @@ S: Odd Fixes F: include/hw/arm/digic.h F: hw/*/digic* F: include/hw/*/digic* -F: tests/avocado/machine_arm_canona1100.py +F: tests/functional/test_arm_canona1100.py F: docs/system/arm/digic.rst Goldfish RTC @@ -1425,7 +1425,7 @@ L: qemu-ppc@nongnu.org S: Orphan F: hw/ppc/ppc440_bamboo.c F: hw/pci-host/ppc4xx_pci.c -F: tests/avocado/ppc_bamboo.py +F: tests/functional/test_ppc_bamboo.py e500 L: qemu-ppc@nongnu.org @@ -1711,7 +1711,7 @@ F: include/hw/pci-host/sabre.h F: hw/pci-bridge/simba.c F: include/hw/pci-bridge/simba.h F: pc-bios/openbios-sparc64 -F: tests/avocado/machine_sparc64_sun4u.py +F: tests/functional/test_sparc64_sun4u.py Sun4v M: Artyom Tarasenko diff --git a/tests/functional/meson.build b/tests/functional/meson.build index 947f9ac74f..c8fc9f6c07 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -22,6 +22,7 @@ tests_generic = [ ] tests_arm_thorough = [ + 'arm_canona1100', 'arm_n8x0', ] @@ -43,6 +44,11 @@ tests_ppc_quick = [ tests_ppc_thorough = [ 'ppc_405', + 'ppc_bamboo', +] + +tests_sparc64_thorough = [ + 'sparc64_sun4u', ] tests_x86_64_quick = [ diff --git a/tests/avocado/machine_arm_canona1100.py b/tests/functional/test_arm_canona1100.py old mode 100644 new mode 100755 similarity index 71% rename from tests/avocado/machine_arm_canona1100.py rename to tests/functional/test_arm_canona1100.py index a42d8b0f2b..de6169f2af --- a/tests/avocado/machine_arm_canona1100.py +++ b/tests/functional/test_arm_canona1100.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Functional test that boots the canon-a1100 machine with firmware # # Copyright (c) 2020 Red Hat, Inc. @@ -8,9 +10,9 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern -from avocado.utils import archive +from qemu_test import QemuSystemTest +from qemu_test import wait_for_console_pattern +from qemu_test.utils import archive_extract class CanonA1100Machine(QemuSystemTest): """Boots the barebox firmware and checks that the console is operational""" @@ -18,18 +20,19 @@ class CanonA1100Machine(QemuSystemTest): timeout = 90 def test_arm_canona1100(self): - """ - :avocado: tags=arch:arm - :avocado: tags=machine:canon-a1100 - :avocado: tags=device:pflash_cfi02 - """ + self.set_machine('canon-a1100') + tar_url = ('https://qemu-advcal.gitlab.io' '/qac-best-of-multiarch/download/day18.tar.xz') tar_hash = '068b5fc4242b29381acee94713509f8a876e9db6' file_path = self.fetch_asset(tar_url, asset_hash=tar_hash) - archive.extract(file_path, self.workdir) + archive_extract(file_path, dest_dir=self.workdir, + member="day18/barebox.canon-a1100.bin") self.vm.set_console() self.vm.add_args('-bios', self.workdir + '/day18/barebox.canon-a1100.bin') self.vm.launch() wait_for_console_pattern(self, 'running /env/bin/init') + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/ppc_bamboo.py b/tests/functional/test_ppc_bamboo.py old mode 100644 new mode 100755 similarity index 75% rename from tests/avocado/ppc_bamboo.py rename to tests/functional/test_ppc_bamboo.py index a81be3d608..652b31e95a --- a/tests/avocado/ppc_bamboo.py +++ b/tests/functional/test_ppc_bamboo.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Test that Linux kernel boots on the ppc bamboo board and check the console # # Copyright (c) 2021 Red Hat @@ -5,30 +7,24 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. -from avocado.utils import archive -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern -from avocado_qemu import exec_command_and_wait_for_pattern +from qemu_test.utils import archive_extract +from qemu_test import QemuSystemTest +from qemu_test import wait_for_console_pattern +from qemu_test import exec_command_and_wait_for_pattern class BambooMachine(QemuSystemTest): timeout = 90 def test_ppc_bamboo(self): - """ - :avocado: tags=arch:ppc - :avocado: tags=machine:bamboo - :avocado: tags=cpu:440epb - :avocado: tags=device:rtl8139 - :avocado: tags=accel:tcg - """ + self.set_machine('bamboo') self.require_accelerator("tcg") self.require_netdev('user') tar_url = ('http://landley.net/aboriginal/downloads/binaries/' 'system-image-powerpc-440fp.tar.gz') tar_hash = '53e5f16414b195b82d2c70272f81c2eedb39bad9' file_path = self.fetch_asset(tar_url, asset_hash=tar_hash) - archive.extract(file_path, self.workdir) + archive_extract(file_path, self.workdir) self.vm.set_console() self.vm.add_args('-kernel', self.workdir + '/system-image-powerpc-440fp/linux', @@ -40,3 +36,6 @@ def test_ppc_bamboo(self): exec_command_and_wait_for_pattern(self, 'ping 10.0.2.2', '10.0.2.2 is alive!') exec_command_and_wait_for_pattern(self, 'halt', 'System Halted') + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/machine_sparc64_sun4u.py b/tests/functional/test_sparc64_sun4u.py old mode 100644 new mode 100755 similarity index 60% rename from tests/avocado/machine_sparc64_sun4u.py rename to tests/functional/test_sparc64_sun4u.py index d333c0ae91..cd89d4a1d6 --- a/tests/avocado/machine_sparc64_sun4u.py +++ b/tests/functional/test_sparc64_sun4u.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Functional test that boots a Linux kernel and checks the console # # Copyright (c) 2020 Red Hat, Inc. @@ -10,27 +12,28 @@ import os -from avocado_qemu import wait_for_console_pattern -from avocado.utils import archive -from boot_linux_console import LinuxKernelTest +from qemu_test import QemuSystemTest +from qemu_test import wait_for_console_pattern +from qemu_test.utils import archive_extract -class Sun4uMachine(LinuxKernelTest): +class Sun4uMachine(QemuSystemTest): """Boots the Linux kernel and checks that the console is operational""" timeout = 90 def test_sparc64_sun4u(self): - """ - :avocado: tags=arch:sparc64 - :avocado: tags=machine:sun4u - """ + self.set_machine('sun4u') tar_url = ('https://qemu-advcal.gitlab.io' '/qac-best-of-multiarch/download/day23.tar.xz') tar_hash = '142db83cd974ffadc4f75c8a5cad5bcc5722c240' file_path = self.fetch_asset(tar_url, asset_hash=tar_hash) - archive.extract(file_path, self.workdir) + kernel_name = 'day23/vmlinux' + archive_extract(file_path, self.workdir, kernel_name) self.vm.set_console() - self.vm.add_args('-kernel', self.workdir + '/day23/vmlinux', - '-append', self.KERNEL_COMMON_COMMAND_LINE) + self.vm.add_args('-kernel', os.path.join(self.workdir, kernel_name), + '-append', 'printk.time=0') self.vm.launch() wait_for_console_pattern(self, 'Starting logging: OK') + +if __name__ == '__main__': + QemuSystemTest.main() From patchwork Tue Jul 16 11:26:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 13734467 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 18643C3DA59 for ; Tue, 16 Jul 2024 11:28:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sTgLE-0001Gp-NW; Tue, 16 Jul 2024 07:27:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKp-00010J-53 for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:27:09 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKk-0003gz-EN for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:27:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721129221; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tWnfJwiTMYAfP+Hg+ct7OhcKr/q01e7XtGCaGzltuGY=; b=eT+MHNTWOyKQ+1r1XEDoovoK5knwSg2dAUtrN2sWNSc6pR50ajPdksIxYwDZ59brVSiakR gV84W0sHVwrYVrvD8WeGBTqSXWRg65CSVpn0AfgQBUWVbRvmBcTDaThK7Y/lfInTCb+eMG UZekUnSEHKBp3a9uFDdGoo25MOgc1n8= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-150-xGdQoJNzOGiOf-1pD98cng-1; Tue, 16 Jul 2024 07:26:56 -0400 X-MC-Unique: xGdQoJNzOGiOf-1pD98cng-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id AF8E01944B30; Tue, 16 Jul 2024 11:26:55 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.192.154]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 42AD41955F40; Tue, 16 Jul 2024 11:26:52 +0000 (UTC) From: Thomas Huth To: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Cc: Ani Sinha , Richard Henderson , Paolo Bonzini , "Daniel P . Berrange" , John Snow , =?utf-8?q?C=C3=A9dric_Le_Goater?= , Fabiano Rosas Subject: [PATCH 09/11] tests/functional: Set up logging Date: Tue, 16 Jul 2024 13:26:12 +0200 Message-ID: <20240716112614.1755692-10-thuth@redhat.com> In-Reply-To: <20240716112614.1755692-1-thuth@redhat.com> References: <20240716112614.1755692-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Create log files for each test separately, one file that contains the basic logging and one that contains the console output. Signed-off-by: Thomas Huth Reviewed-by: Daniel P. Berrangé --- tests/functional/qemu_test/__init__.py | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py index 77c3b14d34..cc49fd4c94 100644 --- a/tests/functional/qemu_test/__init__.py +++ b/tests/functional/qemu_test/__init__.py @@ -208,7 +208,7 @@ class QemuBaseTest(unittest.TestCase): BUILD_DIR = _build_dir() workdir = None - log = logging.getLogger('qemu-test') + log = None def setUp(self, bin_prefix): self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set') @@ -218,6 +218,18 @@ def setUp(self, bin_prefix): if not os.path.exists(self.workdir): os.makedirs(self.workdir) + self.log = logging.getLogger('qemu-test') + self.log.setLevel(logging.DEBUG) + self._log_fh = logging.FileHandler(os.path.join(self.workdir, + 'base.log'), mode='w') + self._log_fh.setLevel(logging.DEBUG) + fileFormatter = logging.Formatter('%(asctime)s - %(levelname)s: %(message)s') + self._log_fh.setFormatter(fileFormatter) + self.log.addHandler(self._log_fh) + + def tearDown(self): + self.log.removeHandler(self._log_fh) + def check_hash(self, file_name, expected_hash): if not expected_hash: return True @@ -269,6 +281,15 @@ def setUp(self): super().setUp('qemu-system-') + console_log = logging.getLogger('console') + console_log.setLevel(logging.DEBUG) + self._console_log_fh = logging.FileHandler(os.path.join(self.workdir, + 'console.log'), mode='w') + self._console_log_fh.setLevel(logging.DEBUG) + fileFormatter = logging.Formatter('%(asctime)s: %(message)s') + self._console_log_fh.setFormatter(fileFormatter) + console_log.addHandler(self._console_log_fh) + def set_machine(self, machinename): # TODO: We should use QMP to get the list of available machines if not self._machinehelp: @@ -359,4 +380,5 @@ def set_vm_arg(self, arg, value): def tearDown(self): for vm in self._vms.values(): vm.shutdown() + logging.getLogger('console').removeHandler(self._console_log_fh) super().tearDown() From patchwork Tue Jul 16 11:26:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 13734474 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 00F05C3DA61 for ; Tue, 16 Jul 2024 11:30:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sTgLO-00029i-V2; Tue, 16 Jul 2024 07:27:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKu-000113-Mx for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:27:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKp-0003hA-VV for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:27:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721129224; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cQC9YgpQ0Go3a4yLyP+3PWN3jDPG9o3cRP2hX1hIGP0=; b=gsr2fogGJKgx4UQ3imwhrl4/mm2In0pyj94kHViP0dJ1QE828uaqstNvrCaaIapt5eqR36 TwEIBl5noJoCWxR/43OiJ232CzxQqen94ZdYNDa12VKVvbAFdtrBMjXa5jf4jhG/Z0VkG4 jxXKGvPLuMDyD71Q7E79DB3WyLq/1H4= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-193-0XJ36HnjMGKvkaxXfL0Ybw-1; Tue, 16 Jul 2024 07:27:00 -0400 X-MC-Unique: 0XJ36HnjMGKvkaxXfL0Ybw-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7083B1944A8F; Tue, 16 Jul 2024 11:26:59 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.192.154]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0A7281955F40; Tue, 16 Jul 2024 11:26:55 +0000 (UTC) From: Thomas Huth To: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Cc: Ani Sinha , Richard Henderson , Paolo Bonzini , "Daniel P . Berrange" , John Snow , =?utf-8?q?C=C3=A9dric_Le_Goater?= , Fabiano Rosas Subject: [PATCH 10/11] tests/functional: Convert the s390x avocado tests into standalone tests Date: Tue, 16 Jul 2024 13:26:13 +0200 Message-ID: <20240716112614.1755692-11-thuth@redhat.com> In-Reply-To: <20240716112614.1755692-1-thuth@redhat.com> References: <20240716112614.1755692-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org These tests use archive.lzma_uncompress() from the Avocado utils, so provide a small helper function for this, based on the standard lzma module from Python instead. And while we're at it, replace the MD5 hashes in the topology test with proper SHA256 hashes, since MD5 should not be used anymore nowadays. Signed-off-by: Thomas Huth --- MAINTAINERS | 4 +- tests/functional/meson.build | 6 ++ tests/functional/qemu_test/utils.py | 7 ++ .../test_s390x_ccw_virtio.py} | 32 ++++----- .../test_s390x_topology.py} | 70 +++++++------------ 5 files changed, 52 insertions(+), 67 deletions(-) rename tests/{avocado/machine_s390_ccw_virtio.py => functional/test_s390x_ccw_virtio.py} (95%) mode change 100644 => 100755 rename tests/{avocado/s390_topology.py => functional/test_s390x_topology.py} (90%) mode change 100644 => 100755 diff --git a/MAINTAINERS b/MAINTAINERS index 025227954c..cbefb6fb81 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1739,7 +1739,7 @@ S: Supported F: hw/s390x/ F: include/hw/s390x/ F: configs/devices/s390x-softmmu/default.mak -F: tests/avocado/machine_s390_ccw_virtio.py +F: tests/functional/test_s390x_ccw_virtio.py T: git https://github.com/borntraeger/qemu.git s390-next L: qemu-s390x@nongnu.org @@ -1802,7 +1802,7 @@ F: hw/s390x/cpu-topology.c F: target/s390x/kvm/stsi-topology.c F: docs/devel/s390-cpu-topology.rst F: docs/system/s390x/cpu-topology.rst -F: tests/avocado/s390_topology.py +F: tests/functional/test_s390x_topology.py X86 Machines ------------ diff --git a/tests/functional/meson.build b/tests/functional/meson.build index c8fc9f6c07..f6de9af8a2 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -13,6 +13,7 @@ endif test_timeouts = { 'netdev_ethtool' : 180, 'ppc_74xx' : 90, + 's390x_ccw_virtio' : 180, } tests_generic = [ @@ -47,6 +48,11 @@ tests_ppc_thorough = [ 'ppc_bamboo', ] +tests_s390x_thorough = [ + 's390x_ccw_virtio', + 's390x_topology', +] + tests_sparc64_thorough = [ 'sparc64_sun4u', ] diff --git a/tests/functional/qemu_test/utils.py b/tests/functional/qemu_test/utils.py index 4eb5e5d5e5..8115d9d1da 100644 --- a/tests/functional/qemu_test/utils.py +++ b/tests/functional/qemu_test/utils.py @@ -8,6 +8,8 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. +import lzma +import shutil import tarfile def archive_extract(archive, dest_dir, member=None): @@ -19,3 +21,8 @@ def archive_extract(archive, dest_dir, member=None): tf.extract(member=member, path=dest_dir) else: tf.extractall(path=dest_dir) + +def lzma_uncompress(xz_path, output_path): + with lzma.open(xz_path, 'rb') as lzma_in: + with open(output_path, 'wb') as raw_out: + shutil.copyfileobj(lzma_in, raw_out) diff --git a/tests/avocado/machine_s390_ccw_virtio.py b/tests/functional/test_s390x_ccw_virtio.py old mode 100644 new mode 100755 similarity index 95% rename from tests/avocado/machine_s390_ccw_virtio.py rename to tests/functional/test_s390x_ccw_virtio.py index 26e938c9e9..b2cbaacd21 --- a/tests/avocado/machine_s390_ccw_virtio.py +++ b/tests/functional/test_s390x_ccw_virtio.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Functional test that boots an s390x Linux guest with ccw and PCI devices # attached and checks whether the devices are recognized by Linux # @@ -12,11 +14,10 @@ import os import tempfile -from avocado import skipUnless -from avocado_qemu import QemuSystemTest -from avocado_qemu import exec_command_and_wait_for_pattern -from avocado_qemu import wait_for_console_pattern -from avocado.utils import archive +from qemu_test import QemuSystemTest +from qemu_test import exec_command_and_wait_for_pattern +from qemu_test import wait_for_console_pattern +from qemu_test.utils import lzma_uncompress class S390CCWVirtioMachine(QemuSystemTest): KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 ' @@ -41,11 +42,7 @@ def clear_guest_dmesg(self): self.dmesg_clear_count += 1 def test_s390x_devices(self): - - """ - :avocado: tags=arch:s390x - :avocado: tags=machine:s390-ccw-virtio - """ + self.set_machine('s390-ccw-virtio') kernel_url = ('https://snapshot.debian.org/archive/debian/' '20201126T092837Z/dists/buster/main/installer-s390x/' @@ -160,15 +157,7 @@ def test_s390x_devices(self): def test_s390x_fedora(self): - - """ - :avocado: tags=arch:s390x - :avocado: tags=machine:s390-ccw-virtio - :avocado: tags=device:virtio-gpu - :avocado: tags=device:virtio-crypto - :avocado: tags=device:virtio-net - :avocado: tags=flaky - """ + self.set_machine('s390-ccw-virtio') kernel_url = ('https://archives.fedoraproject.org/pub/archive' '/fedora-secondary/releases/31/Server/s390x/os' @@ -182,7 +171,7 @@ def test_s390x_fedora(self): initrd_hash = '3de45d411df5624b8d8ef21cd0b44419ab59b12f' initrd_path_xz = self.fetch_asset(initrd_url, asset_hash=initrd_hash) initrd_path = os.path.join(self.workdir, 'initrd-raw.img') - archive.lzma_uncompress(initrd_path_xz, initrd_path) + lzma_uncompress(initrd_path_xz, initrd_path) self.vm.set_console() kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + ' audit=0 ' @@ -275,3 +264,6 @@ def test_s390x_fedora(self): exec_command_and_wait_for_pattern(self, 'while ! (dmesg -c | grep Start.virtcrypto_remove) ; do' ' sleep 1 ; done', 'Start virtcrypto_remove.') + +if __name__ == '__main__': + QemuSystemTest.main() diff --git a/tests/avocado/s390_topology.py b/tests/functional/test_s390x_topology.py old mode 100644 new mode 100755 similarity index 90% rename from tests/avocado/s390_topology.py rename to tests/functional/test_s390x_topology.py index 9154ac8776..39c5ce0b3d --- a/tests/avocado/s390_topology.py +++ b/tests/functional/test_s390x_topology.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python3 +# # Functional test that boots a Linux kernel and checks the console # # Copyright IBM Corp. 2023 @@ -9,16 +11,13 @@ # later. See the COPYING file in the top-level directory. import os -import shutil import time -from avocado_qemu import QemuSystemTest -from avocado_qemu import exec_command -from avocado_qemu import exec_command_and_wait_for_pattern -from avocado_qemu import interrupt_interactive_console_until_pattern -from avocado_qemu import wait_for_console_pattern -from avocado.utils import process -from avocado.utils import archive +from qemu_test import QemuSystemTest +from qemu_test import exec_command +from qemu_test import exec_command_and_wait_for_pattern +from qemu_test import wait_for_console_pattern +from qemu_test.utils import lzma_uncompress class S390CPUTopology(QemuSystemTest): @@ -81,18 +80,16 @@ def kernel_init(self): kernel_url = ('https://archives.fedoraproject.org/pub/archive' '/fedora-secondary/releases/35/Server/s390x/os' '/images/kernel.img') - kernel_hash = '0d1aaaf303f07cf0160c8c48e56fe638' - kernel_path = self.fetch_asset(kernel_url, algorithm='md5', - asset_hash=kernel_hash) + kernel_hash = '1f2dddfd11bb1393dd2eb2e784036fbf6fc11057a6d7d27f9eb12d3edc67ef73' + kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash) initrd_url = ('https://archives.fedoraproject.org/pub/archive' '/fedora-secondary/releases/35/Server/s390x/os' '/images/initrd.img') - initrd_hash = 'a122057d95725ac030e2ec51df46e172' - initrd_path_xz = self.fetch_asset(initrd_url, algorithm='md5', - asset_hash=initrd_hash) + initrd_hash = '1100145fbca00240c8c372ae4b89b48c99844bc189b3dfbc3f481dc60055ca46' + initrd_path_xz = self.fetch_asset(initrd_url, asset_hash=initrd_hash) initrd_path = os.path.join(self.workdir, 'initrd-raw.img') - archive.lzma_uncompress(initrd_path_xz, initrd_path) + lzma_uncompress(initrd_path_xz, initrd_path) self.vm.set_console() kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE @@ -115,10 +112,8 @@ def system_init(self): def test_single(self): """ This test checks the simplest topology with a single CPU. - - :avocado: tags=arch:s390x - :avocado: tags=machine:s390-ccw-virtio """ + self.set_machine('s390-ccw-virtio') self.kernel_init() self.vm.launch() self.wait_until_booted() @@ -127,10 +122,8 @@ def test_single(self): def test_default(self): """ This test checks the implicit topology. - - :avocado: tags=arch:s390x - :avocado: tags=machine:s390-ccw-virtio """ + self.set_machine('s390-ccw-virtio') self.kernel_init() self.vm.add_args('-smp', '13,drawers=2,books=2,sockets=3,cores=2,maxcpus=24') @@ -154,10 +147,8 @@ def test_move(self): """ This test checks the topology modification by moving a CPU to another socket: CPU 0 is moved from socket 0 to socket 2. - - :avocado: tags=arch:s390x - :avocado: tags=machine:s390-ccw-virtio """ + self.set_machine('s390-ccw-virtio') self.kernel_init() self.vm.add_args('-smp', '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24') @@ -174,10 +165,8 @@ def test_dash_device(self): """ This test verifies that a CPU defined with the '-device' command line option finds its right place inside the topology. - - :avocado: tags=arch:s390x - :avocado: tags=machine:s390-ccw-virtio """ + self.set_machine('s390-ccw-virtio') self.kernel_init() self.vm.add_args('-smp', '1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24') @@ -221,10 +210,8 @@ def test_polarization(self): """ This test verifies that QEMU modifies the entitlement change after several guest polarization change requests. - - :avocado: tags=arch:s390x - :avocado: tags=machine:s390-ccw-virtio """ + self.set_machine('s390-ccw-virtio') self.kernel_init() self.vm.launch() self.wait_until_booted() @@ -267,10 +254,8 @@ def test_entitlement(self): """ This test verifies that QEMU modifies the entitlement after a guest request and that the guest sees the change. - - :avocado: tags=arch:s390x - :avocado: tags=machine:s390-ccw-virtio """ + self.set_machine('s390-ccw-virtio') self.kernel_init() self.vm.launch() self.wait_until_booted() @@ -313,10 +298,8 @@ def test_dedicated(self): CPU is made dedicated. QEMU retains the entitlement value when horizontal polarization is in effect. For the guest, the field shows the effective value of the entitlement. - - :avocado: tags=arch:s390x - :avocado: tags=machine:s390-ccw-virtio """ + self.set_machine('s390-ccw-virtio') self.kernel_init() self.vm.launch() self.wait_until_booted() @@ -345,10 +328,8 @@ def test_socket_full(self): This test verifies that QEMU does not accept to overload a socket. The socket-id 0 on book-id 0 already contains CPUs 0 and 1 and can not accept any new CPU while socket-id 0 on book-id 1 is free. - - :avocado: tags=arch:s390x - :avocado: tags=machine:s390-ccw-virtio """ + self.set_machine('s390-ccw-virtio') self.kernel_init() self.vm.add_args('-smp', '3,drawers=2,books=2,sockets=3,cores=2,maxcpus=24') @@ -369,10 +350,8 @@ def test_dedicated_error(self): """ This test verifies that QEMU refuses to lower the entitlement of a dedicated CPU - - :avocado: tags=arch:s390x - :avocado: tags=machine:s390-ccw-virtio """ + self.set_machine('s390-ccw-virtio') self.kernel_init() self.vm.launch() self.wait_until_booted() @@ -417,10 +396,8 @@ def test_move_error(self): """ This test verifies that QEMU refuses to move a CPU to an nonexistent location - - :avocado: tags=arch:s390x - :avocado: tags=machine:s390-ccw-virtio """ + self.set_machine('s390-ccw-virtio') self.kernel_init() self.vm.launch() self.wait_until_booted() @@ -437,3 +414,6 @@ def test_move_error(self): self.assertEqual(res['error']['class'], 'GenericError') self.check_topology(0, 0, 0, 0, 'medium', False) + +if __name__ == '__main__': + QemuSystemTest.main() From patchwork Tue Jul 16 11:26:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 13734465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D4AFCC3DA59 for ; Tue, 16 Jul 2024 11:28:43 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sTgLW-0002hN-A9; Tue, 16 Jul 2024 07:27:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKw-00012S-2d for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:27:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sTgKr-0003hg-GK for qemu-devel@nongnu.org; Tue, 16 Jul 2024 07:27:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1721129228; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PkB61+6tDH/3jxHpJUmWndWzqnh9iRnjS743PT395Dw=; b=Nd725xUgaN6XFosY13dS9KqIXuZNgQJbecSNwy3dzAa4U2ckRM9P2c1FZAFFTA4AZfgwfZ eRX1UXmXsXO/dsGlPUQy5UbG9tHuPYcbCPKo5ZT8/+ItlHKr9SSAipzEhkfotjXybQwzXB 9sia0JYCeUPJWwgEPnMnadhYEQYbcFY= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-658-J3x5kLa-MWCMDHYAwQfZ0g-1; Tue, 16 Jul 2024 07:27:04 -0400 X-MC-Unique: J3x5kLa-MWCMDHYAwQfZ0g-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 303381944AB5; Tue, 16 Jul 2024 11:27:03 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.192.154]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BE0231955F40; Tue, 16 Jul 2024 11:26:59 +0000 (UTC) From: Thomas Huth To: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Cc: Ani Sinha , Richard Henderson , Paolo Bonzini , "Daniel P . Berrange" , John Snow , =?utf-8?q?C=C3=A9dric_Le_Goater?= , Fabiano Rosas Subject: [PATCH 11/11] gitlab-ci: Add "check-functional" to the build tests Date: Tue, 16 Jul 2024 13:26:14 +0200 Message-ID: <20240716112614.1755692-12-thuth@redhat.com> In-Reply-To: <20240716112614.1755692-1-thuth@redhat.com> References: <20240716112614.1755692-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Now that we converted many tests from the "check-avocado" test suite to the "check-functional" test suite, we should make sure that these also get tested in the CI. Signed-off-by: Thomas Huth Reviewed-by: Daniel P. Berrangé --- .gitlab-ci.d/buildtest-template.yml | 3 +- .gitlab-ci.d/buildtest.yml | 60 ++++++++++++++--------------- 2 files changed, 32 insertions(+), 31 deletions(-) diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml index 8f7ebfaed8..54cae74a8e 100644 --- a/.gitlab-ci.d/buildtest-template.yml +++ b/.gitlab-ci.d/buildtest-template.yml @@ -71,7 +71,7 @@ reports: junit: build/meson-logs/testlog.junit.xml -.avocado_test_job_template: +.functional_test_job_template: extends: .common_test_job_template cache: key: "${CI_JOB_NAME}-cache" @@ -98,6 +98,7 @@ du -chs ${CI_PROJECT_DIR}/avocado-cache ; fi - export AVOCADO_ALLOW_UNTRUSTED_CODE=1 + - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1 after_script: - cd build - du -chs ${CI_PROJECT_DIR}/avocado-cache diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index e3a0758bd9..8b1d61911c 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -22,14 +22,14 @@ check-system-alpine: IMAGE: alpine MAKE_CHECK_ARGS: check-unit check-qtest -avocado-system-alpine: - extends: .avocado_test_job_template +functional-system-alpine: + extends: .functional_test_job_template needs: - job: build-system-alpine artifacts: true variables: IMAGE: alpine - MAKE_CHECK_ARGS: check-avocado + MAKE_CHECK_ARGS: check-avocado check-functional AVOCADO_TAGS: arch:avr arch:loongarch64 arch:mips64 arch:mipsel build-system-ubuntu: @@ -53,14 +53,14 @@ check-system-ubuntu: IMAGE: ubuntu2204 MAKE_CHECK_ARGS: check -avocado-system-ubuntu: - extends: .avocado_test_job_template +functional-system-ubuntu: + extends: .functional_test_job_template needs: - job: build-system-ubuntu artifacts: true variables: IMAGE: ubuntu2204 - MAKE_CHECK_ARGS: check-avocado + MAKE_CHECK_ARGS: check-avocado check-functional AVOCADO_TAGS: arch:alpha arch:microblazeel arch:mips64el build-system-debian: @@ -85,14 +85,14 @@ check-system-debian: IMAGE: debian MAKE_CHECK_ARGS: check -avocado-system-debian: - extends: .avocado_test_job_template +functional-system-debian: + extends: .functional_test_job_template needs: - job: build-system-debian artifacts: true variables: IMAGE: debian - MAKE_CHECK_ARGS: check-avocado + MAKE_CHECK_ARGS: check-avocado check-functional AVOCADO_TAGS: arch:arm arch:i386 arch:riscv64 arch:sh4 arch:sparc arch:xtensa crash-test-debian: @@ -129,14 +129,14 @@ check-system-fedora: IMAGE: fedora MAKE_CHECK_ARGS: check -avocado-system-fedora: - extends: .avocado_test_job_template +functional-system-fedora: + extends: .functional_test_job_template needs: - job: build-system-fedora artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-avocado + MAKE_CHECK_ARGS: check-avocado check-functional AVOCADO_TAGS: arch:microblaze arch:mips arch:xtensa arch:m68k arch:riscv32 arch:ppc arch:sparc64 @@ -243,14 +243,14 @@ check-system-centos: IMAGE: centos9 MAKE_CHECK_ARGS: check -avocado-system-centos: - extends: .avocado_test_job_template +functional-system-centos: + extends: .functional_test_job_template needs: - job: build-system-centos artifacts: true variables: IMAGE: centos9 - MAKE_CHECK_ARGS: check-avocado + MAKE_CHECK_ARGS: check-avocado check-functional AVOCADO_TAGS: arch:ppc64 arch:or1k arch:s390x arch:x86_64 arch:rx arch:sh4 @@ -274,14 +274,14 @@ check-system-opensuse: IMAGE: opensuse-leap MAKE_CHECK_ARGS: check -avocado-system-opensuse: - extends: .avocado_test_job_template +functional-system-opensuse: + extends: .functional_test_job_template needs: - job: build-system-opensuse artifacts: true variables: IMAGE: opensuse-leap - MAKE_CHECK_ARGS: check-avocado + MAKE_CHECK_ARGS: check-avocado check-functional AVOCADO_TAGS: arch:s390x arch:x86_64 arch:aarch64 # @@ -302,15 +302,15 @@ build-system-flaky: ppc64-softmmu rx-softmmu s390x-softmmu sh4-softmmu x86_64-softmmu MAKE_CHECK_ARGS: check-build -avocado-system-flaky: - extends: .avocado_test_job_template +functional-system-flaky: + extends: .functional_test_job_template needs: - job: build-system-flaky artifacts: true allow_failure: true variables: IMAGE: debian - MAKE_CHECK_ARGS: check-avocado + MAKE_CHECK_ARGS: check-avocado check-functional QEMU_JOB_OPTIONAL: 1 QEMU_TEST_FLAKY_TESTS: 1 AVOCADO_TAGS: flaky @@ -484,14 +484,14 @@ check-cfi-aarch64: IMAGE: fedora MAKE_CHECK_ARGS: check -avocado-cfi-aarch64: - extends: .avocado_test_job_template +functional-cfi-aarch64: + extends: .functional_test_job_template needs: - job: build-cfi-aarch64 artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-avocado + MAKE_CHECK_ARGS: check-avocado check-functional build-cfi-ppc64-s390x: extends: @@ -522,14 +522,14 @@ check-cfi-ppc64-s390x: IMAGE: fedora MAKE_CHECK_ARGS: check -avocado-cfi-ppc64-s390x: - extends: .avocado_test_job_template +functional-cfi-ppc64-s390x: + extends: .functional_test_job_template needs: - job: build-cfi-ppc64-s390x artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-avocado + MAKE_CHECK_ARGS: check-avocado check-functional build-cfi-x86_64: extends: @@ -556,14 +556,14 @@ check-cfi-x86_64: IMAGE: fedora MAKE_CHECK_ARGS: check -avocado-cfi-x86_64: - extends: .avocado_test_job_template +functional-cfi-x86_64: + extends: .functional_test_job_template needs: - job: build-cfi-x86_64 artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-avocado + MAKE_CHECK_ARGS: check-avocado check-functional tsan-build: extends: .native_build_job_template