From patchwork Mon Apr 14 11:30:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050218 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B5EDBC3DA4A for ; Mon, 14 Apr 2025 11:31:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I1Y-0003Tg-JD; Mon, 14 Apr 2025 07:30:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1T-0003Sj-Fs for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1R-0003sn-FR for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630238; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v3SvVrRB8XRc9sJMLrlmZk6r9s+sgxFxLvTivA6M6PY=; b=Bv6rKkbo7eDmOKtv/dOghaeQ99pGb8jRnjmO2sA9iPgnE56Aux8v/iPC2I+5oK+g/OhJ5M nqE0hFoM5n6hGIxsl+xogRneHVDQ/kC+lyioyha/5y6tegALSzm9KoslhvBV+WsZt1IN/0 9uh57N0vWYB017SuX7bRdp3ADa8r2g0= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-615-vOmbQf5INKmBKjuu1igDrw-1; Mon, 14 Apr 2025 07:30:36 -0400 X-MC-Unique: vOmbQf5INKmBKjuu1igDrw-1 X-Mimecast-MFC-AGG-ID: vOmbQf5INKmBKjuu1igDrw_1744630236 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 07785195608D; Mon, 14 Apr 2025 11:30:36 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CA9941955BC1; Mon, 14 Apr 2025 11:30:34 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 01/15] gitlab-ci: Remove the avocado tests from the CI pipelines Date: Mon, 14 Apr 2025 13:30:15 +0200 Message-ID: <20250414113031.151105-2-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth We are going to move the remaining Avocado tests step by step into the functional test framework. Unfortunately, Avocado fails with an error if it cannot determine a test to run, so disable the tests here now to avoid failures in the Gitlab-CI during the next steps. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- .gitlab-ci.d/buildtest-template.yml | 11 ---------- .gitlab-ci.d/buildtest.yml | 33 +++++++++++------------------ 2 files changed, 12 insertions(+), 32 deletions(-) diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml index 39da7698b09..13fa4f4a4fc 100644 --- a/.gitlab-ci.d/buildtest-template.yml +++ b/.gitlab-ci.d/buildtest-template.yml @@ -95,7 +95,6 @@ cache: key: "${CI_JOB_NAME}-cache" paths: - - ${CI_PROJECT_DIR}/avocado-cache - ${CI_PROJECT_DIR}/functional-cache policy: pull-push artifacts: @@ -109,16 +108,6 @@ reports: junit: build/tests/results/latest/results.xml before_script: - - mkdir -p ~/.config/avocado - - echo "[datadir.paths]" > ~/.config/avocado/avocado.conf - - echo "cache_dirs = ['${CI_PROJECT_DIR}/avocado-cache']" - >> ~/.config/avocado/avocado.conf - - echo -e '[job.output.testlogs]\nstatuses = ["FAIL", "INTERRUPT"]' - >> ~/.config/avocado/avocado.conf - - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then - du -chs ${CI_PROJECT_DIR}/*-cache ; - fi - - export AVOCADO_ALLOW_UNTRUSTED_CODE=1 - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1 - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache after_script: diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index 00f4bfcd9f3..431bc07d8fb 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -29,8 +29,7 @@ functional-system-alpine: artifacts: true variables: IMAGE: alpine - MAKE_CHECK_ARGS: check-avocado check-functional - AVOCADO_TAGS: arch:avr arch:loongarch64 arch:mips64 arch:mipsel + MAKE_CHECK_ARGS: check-functional build-system-ubuntu: extends: @@ -60,8 +59,7 @@ functional-system-ubuntu: artifacts: true variables: IMAGE: ubuntu2204 - MAKE_CHECK_ARGS: check-avocado check-functional - AVOCADO_TAGS: arch:alpha arch:microblazeel arch:mips64el + MAKE_CHECK_ARGS: check-functional build-system-debian: extends: @@ -92,8 +90,7 @@ functional-system-debian: artifacts: true variables: IMAGE: debian - MAKE_CHECK_ARGS: check-avocado check-functional - AVOCADO_TAGS: arch:arm arch:i386 arch:riscv64 arch:sh4 arch:sparc arch:xtensa + MAKE_CHECK_ARGS: check-functional crash-test-debian: extends: .native_test_job_template @@ -155,9 +152,7 @@ functional-system-fedora: artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-avocado check-functional - AVOCADO_TAGS: arch:microblaze arch:mips arch:xtensa arch:m68k - arch:riscv32 arch:ppc arch:sparc64 + MAKE_CHECK_ARGS: check-functional crash-test-fedora: extends: .native_test_job_template @@ -278,9 +273,7 @@ functional-system-centos: artifacts: true variables: IMAGE: centos9 - MAKE_CHECK_ARGS: check-avocado check-functional - AVOCADO_TAGS: arch:ppc64 arch:or1k arch:s390x arch:x86_64 arch:rx - arch:sh4 + MAKE_CHECK_ARGS: check-functional build-system-opensuse: extends: @@ -309,8 +302,7 @@ functional-system-opensuse: artifacts: true variables: IMAGE: opensuse-leap - MAKE_CHECK_ARGS: check-avocado check-functional - AVOCADO_TAGS: arch:s390x arch:x86_64 arch:aarch64 + MAKE_CHECK_ARGS: check-functional # # Flaky tests. We don't run these by default and they are allow fail @@ -338,10 +330,9 @@ functional-system-flaky: allow_failure: true variables: IMAGE: debian - MAKE_CHECK_ARGS: check-avocado check-functional + MAKE_CHECK_ARGS: check-functional QEMU_JOB_OPTIONAL: 1 QEMU_TEST_FLAKY_TESTS: 1 - AVOCADO_TAGS: flaky # This jobs explicitly disable TCG (--disable-tcg), KVM is detected by # the configure script. The container doesn't contain Xen headers so @@ -482,8 +473,8 @@ clang-user: # Since slirp callbacks are used in QEMU Timers, we cannot use libslirp with # CFI builds, and thus have to disable it here. # -# Split in three sets of build/check/avocado to limit the execution time of each -# job +# Split in three sets of build/check/functional to limit the execution time +# of each job build-cfi-aarch64: extends: - .native_build_job_template @@ -520,7 +511,7 @@ functional-cfi-aarch64: artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-avocado check-functional + MAKE_CHECK_ARGS: check-functional build-cfi-ppc64-s390x: extends: @@ -558,7 +549,7 @@ functional-cfi-ppc64-s390x: artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-avocado check-functional + MAKE_CHECK_ARGS: check-functional build-cfi-x86_64: extends: @@ -592,7 +583,7 @@ functional-cfi-x86_64: artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-avocado check-functional + MAKE_CHECK_ARGS: check-functional tsan-build: extends: .native_build_job_template From patchwork Mon Apr 14 11:30:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050220 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B4000C369B5 for ; Mon, 14 Apr 2025 11:31:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I1b-0003Tr-II; Mon, 14 Apr 2025 07:30:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1W-0003T4-8C for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1U-0003tZ-Ah for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630243; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lAAL71y2YK6MR6mwduHfer8ymBodpbgASuKvdSN7bB8=; b=heiDCGN7xFqmw0TXXEbAEWrb+qMC9i/9c26SIzGPlluLAFpgT7eh9VzqZgYzNPcSE9PNuc BdM/l2oHryMHzQSFTli00z3PNE94ca9FQ9B0OLBmAVdCzfiy9nhM11B3FonqsY689XPEPQ FLq02WrR2VPK+ftL556MVlZVwxFDay4= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-674-rjCklVa2PGKZXwSJpfYwKA-1; Mon, 14 Apr 2025 07:30:40 -0400 X-MC-Unique: rjCklVa2PGKZXwSJpfYwKA-1 X-Mimecast-MFC-AGG-ID: rjCklVa2PGKZXwSJpfYwKA_1744630238 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 89E0B195608A; Mon, 14 Apr 2025 11:30:38 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6C3AC1955BC1; Mon, 14 Apr 2025 11:30:36 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 02/15] tests/functional: Move the check for the parameters from avocado to functional Date: Mon, 14 Apr 2025 13:30:16 +0200 Message-ID: <20250414113031.151105-3-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth test_x86_64_pc in tests/avocado/boot_linux_console.py only checks whether the kernel parameters have correctly been passed to the kernel in the guest by looking for them in the console output of the guest. Let's move that to the functional test framework now, but instead of doing it in a separate test, let's do it for all tuxrun tests instead, so it is done automatically for all targets that have a tuxrun test. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/avocado/boot_linux_console.py | 34 ------------------------ tests/functional/qemu_test/tuxruntest.py | 9 ++++--- 2 files changed, 5 insertions(+), 38 deletions(-) diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py index c15f39ae1f3..cbb1e2fb506 100644 --- a/tests/avocado/boot_linux_console.py +++ b/tests/avocado/boot_linux_console.py @@ -9,17 +9,9 @@ # later. See the COPYING file in the top-level directory. import os -import lzma -import gzip import shutil -from avocado import skip -from avocado import skipUnless -from avocado import skipUnless from avocado_qemu import QemuSystemTest -from avocado_qemu import exec_command -from avocado_qemu import exec_command_and_wait_for_pattern -from avocado_qemu import interrupt_interactive_console_until_pattern from avocado_qemu import wait_for_console_pattern from avocado.utils import process from avocado.utils import archive @@ -68,29 +60,3 @@ def extract_from_rpm(self, rpm, path): process.run("rpm2cpio %s | cpio -id %s" % (rpm, path), shell=True) os.chdir(cwd) return os.path.normpath(os.path.join(self.workdir, path)) - -class BootLinuxConsole(LinuxKernelTest): - """ - Boots a Linux kernel and checks that the console is operational and the - kernel command line is properly passed from QEMU to the kernel - """ - timeout = 90 - - def test_x86_64_pc(self): - """ - :avocado: tags=arch:x86_64 - :avocado: tags=machine:pc - """ - kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora' - '/linux/releases/29/Everything/x86_64/os/images/pxeboot' - '/vmlinuz') - kernel_hash = '23bebd2680757891cf7adedb033532163a792495' - kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash) - - self.vm.set_console() - kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0' - self.vm.add_args('-kernel', kernel_path, - '-append', kernel_command_line) - self.vm.launch() - console_pattern = 'Kernel command line: %s' % kernel_command_line - self.wait_for_console_pattern(console_pattern) diff --git a/tests/functional/qemu_test/tuxruntest.py b/tests/functional/qemu_test/tuxruntest.py index ad74156f9c5..c2bd5baaae9 100644 --- a/tests/functional/qemu_test/tuxruntest.py +++ b/tests/functional/qemu_test/tuxruntest.py @@ -77,12 +77,12 @@ def prepare_run(self, kernel, disk, drive, dtb=None, console_index=0): blockdev = "driver=raw,file.driver=file," \ + f"file.filename={disk},node-name=hd0" - kcmd_line = self.KERNEL_COMMON_COMMAND_LINE - kcmd_line += f" root=/dev/{self.root}" - kcmd_line += f" console={self.console}" + self.kcmd_line = self.KERNEL_COMMON_COMMAND_LINE + self.kcmd_line += f" root=/dev/{self.root}" + self.kcmd_line += f" console={self.console}" self.vm.add_args('-kernel', kernel, - '-append', kcmd_line, + '-append', self.kcmd_line, '-blockdev', blockdev) # Sometimes we need extra devices attached @@ -103,6 +103,7 @@ def run_tuxtest_tests(self, haltmsg): wait to exit cleanly. """ ps1='root@tuxtest:~#' + self.wait_for_console_pattern(self.kcmd_line) self.wait_for_console_pattern('tuxtest login:') exec_command_and_wait_for_pattern(self, 'root', ps1) exec_command_and_wait_for_pattern(self, 'cat /proc/interrupts', ps1) From patchwork Mon Apr 14 11:30:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050230 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E8F9CC369B2 for ; Mon, 14 Apr 2025 11:35:21 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I1z-0003YS-Ml; Mon, 14 Apr 2025 07:31:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1c-0003UV-Dx for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1Y-0003u5-Dp for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630247; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8DOSS9IVMvZ8b33XF5KfvbMXvheoSGL306thDeeG1kM=; b=WDxN3VjfNyKwr21vuzPbT0KW7dnY0DF/bmGfyx+nmRlXeWNjqkWz7GC3wtpdUsrjVeZISq 0zcuRreEoJdoKctNGeBJvZ5HL23H3UNSH6hTutfumS8vKMN8L3tVZEEeW2m/6BPH9L+Qgf VSQ86UsYIvhW5rqeRsLvuG8PPnMHm64= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-491-H1yWFk3QPgOZ5oQzJ406yg-1; Mon, 14 Apr 2025 07:30:41 -0400 X-MC-Unique: H1yWFk3QPgOZ5oQzJ406yg-1 X-Mimecast-MFC-AGG-ID: H1yWFk3QPgOZ5oQzJ406yg_1744630241 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 35F601955BC5; Mon, 14 Apr 2025 11:30:40 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id EFEB41955BC1; Mon, 14 Apr 2025 11:30:38 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 03/15] tests/functional: Convert reverse_debugging tests to the functional framework Date: Mon, 14 Apr 2025 13:30:17 +0200 Message-ID: <20250414113031.151105-4-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth These tests are using the gdb-related library functions from the Avocado framework which we don't have in the functional framework yet. So for the time being, keep those imports and skip the test if the Avocado framework is not installed on the host. Signed-off-by: Thomas Huth Reviewed-by: Daniel P. Berrangé --- MAINTAINERS | 2 +- tests/functional/meson.build | 4 + .../reverse_debugging.py | 114 +++--------------- .../functional/test_aarch64_reverse_debug.py | 38 ++++++ tests/functional/test_ppc64_reverse_debug.py | 41 +++++++ tests/functional/test_x86_64_reverse_debug.py | 36 ++++++ 6 files changed, 139 insertions(+), 96 deletions(-) rename tests/{avocado => functional}/reverse_debugging.py (66%) create mode 100755 tests/functional/test_aarch64_reverse_debug.py create mode 100755 tests/functional/test_ppc64_reverse_debug.py create mode 100755 tests/functional/test_x86_64_reverse_debug.py diff --git a/MAINTAINERS b/MAINTAINERS index d54b5578f88..af1d847f8ed 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3671,7 +3671,7 @@ F: docs/system/replay.rst F: stubs/replay.c F: tests/avocado/replay_kernel.py F: tests/avocado/replay_linux.py -F: tests/avocado/reverse_debugging.py +F: tests/functional/*reverse_debug*.py F: tests/functional/*replay*.py F: qapi/replay.json diff --git a/tests/functional/meson.build b/tests/functional/meson.build index 0f8be30fe27..7b0f4ab0b1d 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -13,6 +13,7 @@ endif test_timeouts = { 'aarch64_aspeed' : 600, 'aarch64_raspi4' : 480, + 'aarch64_reverse_debug' : 180, 'aarch64_rme_virt' : 1200, 'aarch64_rme_sbsaref' : 1200, 'aarch64_sbsaref_alpine' : 1200, @@ -78,6 +79,7 @@ tests_aarch64_system_thorough = [ 'aarch64_raspi3', 'aarch64_raspi4', 'aarch64_replay', + 'aarch64_reverse_debug', 'aarch64_rme_virt', 'aarch64_rme_sbsaref', 'aarch64_sbsaref', @@ -229,6 +231,7 @@ tests_ppc64_system_thorough = [ 'ppc64_powernv', 'ppc64_pseries', 'ppc64_replay', + 'ppc64_reverse_debug', 'ppc64_tuxrun', 'ppc64_mac99', ] @@ -311,6 +314,7 @@ tests_x86_64_system_thorough = [ 'x86_64_hotplug_cpu', 'x86_64_kvm_xen', 'x86_64_replay', + 'x86_64_reverse_debug', 'x86_64_tuxrun', ] diff --git a/tests/avocado/reverse_debugging.py b/tests/functional/reverse_debugging.py similarity index 66% rename from tests/avocado/reverse_debugging.py rename to tests/functional/reverse_debugging.py index f24287cd0a0..f9a1d395f1d 100644 --- a/tests/avocado/reverse_debugging.py +++ b/tests/functional/reverse_debugging.py @@ -1,5 +1,7 @@ # Reverse debugging test # +# SPDX-License-Identifier: GPL-2.0-or-later +# # Copyright (c) 2020 ISP RAS # # Author: @@ -10,14 +12,9 @@ import os import logging -from avocado import skipUnless -from avocado_qemu import BUILD_DIR -from avocado.utils import datadrainer -from avocado.utils import gdb -from avocado.utils import process -from avocado.utils.network.ports import find_free_port -from avocado.utils.path import find_command -from boot_linux_console import LinuxKernelTest +from qemu_test import LinuxKernelTest, get_qemu_img +from qemu_test.ports import Ports + class ReverseDebugging(LinuxKernelTest): """ @@ -36,8 +33,10 @@ class ReverseDebugging(LinuxKernelTest): endian_is_le = True def run_vm(self, record, shift, args, replay_path, image_path, port): + from avocado.utils import datadrainer + logger = logging.getLogger('replay') - vm = self.get_vm() + vm = self.get_vm(name='record' if record else 'replay') vm.set_console() if record: logger.info('recording the execution...') @@ -100,25 +99,25 @@ def vm_get_icount(vm): return vm.qmp('query-replay')['return']['icount'] def reverse_debugging(self, shift=7, args=None): + from avocado.utils import gdb + from avocado.utils import process + logger = logging.getLogger('replay') # create qcow2 for snapshots logger.info('creating qcow2 image for VM snapshots') image_path = os.path.join(self.workdir, 'disk.qcow2') - qemu_img = os.path.join(BUILD_DIR, 'qemu-img') - if not os.path.exists(qemu_img): - qemu_img = find_command('qemu-img', False) - if qemu_img is False: - self.cancel('Could not find "qemu-img", which is required to ' - 'create the temporary qcow2 image') + qemu_img = get_qemu_img(self) + if qemu_img is None: + self.skipTest('Could not find "qemu-img", which is required to ' + 'create the temporary qcow2 image') cmd = '%s create -f qcow2 %s 128M' % (qemu_img, image_path) process.run(cmd) replay_path = os.path.join(self.workdir, 'replay.bin') - port = find_free_port() # record the log - vm = self.run_vm(True, shift, args, replay_path, image_path, port) + vm = self.run_vm(True, shift, args, replay_path, image_path, -1) while self.vm_get_icount(vm) <= self.STEPS: pass last_icount = self.vm_get_icount(vm) @@ -127,7 +126,9 @@ def reverse_debugging(self, shift=7, args=None): logger.info("recorded log with %s+ steps" % last_icount) # replay and run debug commands - vm = self.run_vm(False, shift, args, replay_path, image_path, port) + with Ports() as ports: + port = ports.find_free_port() + vm = self.run_vm(False, shift, args, replay_path, image_path, port) logger.info('connecting to gdbstub') g = gdb.GDBRemote('127.0.0.1', port, False, False) g.connect() @@ -193,80 +194,3 @@ def reverse_debugging(self, shift=7, args=None): logger.info('exiting gdb and qemu') vm.shutdown() - -class ReverseDebugging_X86_64(ReverseDebugging): - """ - :avocado: tags=accel:tcg - """ - - REG_PC = 0x10 - REG_CS = 0x12 - def get_pc(self, g): - return self.get_reg_le(g, self.REG_PC) \ - + self.get_reg_le(g, self.REG_CS) * 0x10 - - # unidentified gitlab timeout problem - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab') - def test_x86_64_pc(self): - """ - :avocado: tags=arch:x86_64 - :avocado: tags=machine:pc - """ - # start with BIOS only - self.reverse_debugging() - -class ReverseDebugging_AArch64(ReverseDebugging): - """ - :avocado: tags=accel:tcg - """ - - REG_PC = 32 - - # unidentified gitlab timeout problem - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab') - def test_aarch64_virt(self): - """ - :avocado: tags=arch:aarch64 - :avocado: tags=machine:virt - :avocado: tags=cpu:cortex-a53 - """ - kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora' - '/linux/releases/29/Everything/aarch64/os/images/pxeboot' - '/vmlinuz') - kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493' - kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash) - - self.reverse_debugging( - args=('-kernel', kernel_path)) - -class ReverseDebugging_ppc64(ReverseDebugging): - """ - :avocado: tags=accel:tcg - """ - - REG_PC = 0x40 - - # unidentified gitlab timeout problem - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab') - def test_ppc64_pseries(self): - """ - :avocado: tags=arch:ppc64 - :avocado: tags=machine:pseries - :avocado: tags=flaky - """ - # SLOF branches back to its entry point, which causes this test - # to take the 'hit a breakpoint again' path. That's not a problem, - # just slightly different than the other machines. - self.endian_is_le = False - self.reverse_debugging() - - # See https://gitlab.com/qemu-project/qemu/-/issues/1992 - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab') - def test_ppc64_powernv(self): - """ - :avocado: tags=arch:ppc64 - :avocado: tags=machine:powernv - :avocado: tags=flaky - """ - self.endian_is_le = False - self.reverse_debugging() diff --git a/tests/functional/test_aarch64_reverse_debug.py b/tests/functional/test_aarch64_reverse_debug.py new file mode 100755 index 00000000000..58d45328350 --- /dev/null +++ b/tests/functional/test_aarch64_reverse_debug.py @@ -0,0 +1,38 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Reverse debugging test +# +# Copyright (c) 2020 ISP RAS +# +# Author: +# Pavel Dovgalyuk +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +from qemu_test import Asset, skipIfMissingImports, skipFlakyTest +from reverse_debugging import ReverseDebugging + + +@skipIfMissingImports('avocado.utils') +class ReverseDebugging_AArch64(ReverseDebugging): + + REG_PC = 32 + + KERNEL_ASSET = Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/' + 'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'), + '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7') + + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/2921") + def test_aarch64_virt(self): + self.set_machine('virt') + self.cpu = 'cortex-a53' + kernel_path = self.KERNEL_ASSET.fetch() + self.reverse_debugging(args=('-kernel', kernel_path)) + + +if __name__ == '__main__': + ReverseDebugging.main() diff --git a/tests/functional/test_ppc64_reverse_debug.py b/tests/functional/test_ppc64_reverse_debug.py new file mode 100755 index 00000000000..5931adef5a9 --- /dev/null +++ b/tests/functional/test_ppc64_reverse_debug.py @@ -0,0 +1,41 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Reverse debugging test +# +# Copyright (c) 2020 ISP RAS +# +# Author: +# Pavel Dovgalyuk +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +from qemu_test import skipIfMissingImports, skipFlakyTest +from reverse_debugging import ReverseDebugging + + +@skipIfMissingImports('avocado.utils') +class ReverseDebugging_ppc64(ReverseDebugging): + + REG_PC = 0x40 + + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992") + def test_ppc64_pseries(self): + self.set_machine('pseries') + # SLOF branches back to its entry point, which causes this test + # to take the 'hit a breakpoint again' path. That's not a problem, + # just slightly different than the other machines. + self.endian_is_le = False + self.reverse_debugging() + + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992") + def test_ppc64_powernv(self): + self.set_machine('powernv') + self.endian_is_le = False + self.reverse_debugging() + + +if __name__ == '__main__': + ReverseDebugging.main() diff --git a/tests/functional/test_x86_64_reverse_debug.py b/tests/functional/test_x86_64_reverse_debug.py new file mode 100755 index 00000000000..d713e91e144 --- /dev/null +++ b/tests/functional/test_x86_64_reverse_debug.py @@ -0,0 +1,36 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Reverse debugging test +# +# Copyright (c) 2020 ISP RAS +# +# Author: +# Pavel Dovgalyuk +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +from qemu_test import skipIfMissingImports, skipFlakyTest +from reverse_debugging import ReverseDebugging + + +@skipIfMissingImports('avocado.utils') +class ReverseDebugging_X86_64(ReverseDebugging): + + REG_PC = 0x10 + REG_CS = 0x12 + def get_pc(self, g): + return self.get_reg_le(g, self.REG_PC) \ + + self.get_reg_le(g, self.REG_CS) * 0x10 + + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/2922") + def test_x86_64_pc(self): + self.set_machine('pc') + # start with BIOS only + self.reverse_debugging() + + +if __name__ == '__main__': + ReverseDebugging.main() From patchwork Mon Apr 14 11:30:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050228 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E553DC369B5 for ; Mon, 14 Apr 2025 11:33:09 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I1z-0003YI-CK; Mon, 14 Apr 2025 07:31:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1a-0003UE-Ky for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1Y-0003u2-89 for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630247; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EFT2WxGEQfpsHO7alU6ylF9NtWckt/VI7w+fHgt2l7w=; b=SCqtfE4wXGXc8S3NfU5zswWUZhmwxlf2SwJBpdGo8Ob2K6TeaYUUpOprWw5v3s2GVoq8Xe CCmuGaQqUZuUaIanKixW5ZHgsOjNNwhS3M0p2JLQl1LdSStMwjAxuz4zCzBNJ/WCn0ch1G 9TwYm51836evkF5GdqwcdTlUsCETOls= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-221-gZSVycs5P7SJiiEix8Ef3g-1; Mon, 14 Apr 2025 07:30:44 -0400 X-MC-Unique: gZSVycs5P7SJiiEix8Ef3g-1 X-Mimecast-MFC-AGG-ID: gZSVycs5P7SJiiEix8Ef3g_1744630241 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A2E651956048; Mon, 14 Apr 2025 11:30:41 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7E0FE1956094; Mon, 14 Apr 2025 11:30:40 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 04/15] tests/functional: Convert the i386 replay avocado test Date: Mon, 14 Apr 2025 13:30:18 +0200 Message-ID: <20250414113031.151105-5-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth Since this was the last test in tests/avocado/replay_kernel.py, we can remove that Avocado file now. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -import os -import lzma -import shutil -import logging -import time -import subprocess - -from avocado import skip -from avocado import skipUnless -from avocado import skipUnless -from avocado_qemu import wait_for_console_pattern -from avocado.utils import archive -from avocado.utils import process -from boot_linux_console import LinuxKernelTest - -class ReplayKernelBase(LinuxKernelTest): - """ - Boots a Linux kernel in record mode and checks that the console - is operational and the kernel command line is properly passed - from QEMU to the kernel. - Then replays the same scenario and verifies, that QEMU correctly - terminates. - """ - - timeout = 180 - KERNEL_COMMON_COMMAND_LINE = 'printk.time=1 panic=-1 ' - - def run_vm(self, kernel_path, kernel_command_line, console_pattern, - record, shift, args, replay_path): - # icount requires TCG to be available - self.require_accelerator('tcg') - - logger = logging.getLogger('replay') - start_time = time.time() - vm = self.get_vm() - vm.set_console() - if record: - logger.info('recording the execution...') - mode = 'record' - else: - logger.info('replaying the execution...') - mode = 'replay' - vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' % - (shift, mode, replay_path), - '-kernel', kernel_path, - '-append', kernel_command_line, - '-net', 'none', - '-no-reboot') - if args: - vm.add_args(*args) - vm.launch() - self.wait_for_console_pattern(console_pattern, vm) - if record: - vm.shutdown() - logger.info('finished the recording with log size %s bytes' - % os.path.getsize(replay_path)) - self.run_replay_dump(replay_path) - logger.info('successfully tested replay-dump.py') - else: - vm.wait() - logger.info('successfully finished the replay') - elapsed = time.time() - start_time - logger.info('elapsed time %.2f sec' % elapsed) - return elapsed - - def run_replay_dump(self, replay_path): - try: - subprocess.check_call(["./scripts/replay-dump.py", - "-f", replay_path], - stdout=subprocess.DEVNULL) - except subprocess.CalledProcessError: - self.fail('replay-dump.py failed') - - def run_rr(self, kernel_path, kernel_command_line, console_pattern, - shift=7, args=None): - replay_path = os.path.join(self.workdir, 'replay.bin') - t1 = self.run_vm(kernel_path, kernel_command_line, console_pattern, - True, shift, args, replay_path) - t2 = self.run_vm(kernel_path, kernel_command_line, console_pattern, - False, shift, args, replay_path) - logger = logging.getLogger('replay') - logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1)) - -class ReplayKernelNormal(ReplayKernelBase): - - def test_i386_pc(self): - """ - :avocado: tags=arch:i386 - :avocado: tags=machine:pc - """ - kernel_url = ('https://storage.tuxboot.com/20230331/i386/bzImage') - kernel_hash = 'a3e5b32a354729e65910f5a1ffcda7c14a6c12a55e8213fb86e277f1b76ed956' - kernel_path = self.fetch_asset(kernel_url, - asset_hash=kernel_hash, - algorithm = "sha256") - - kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0' - console_pattern = 'VFS: Cannot open root device' - - self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5) diff --git a/tests/functional/meson.build b/tests/functional/meson.build index 7b0f4ab0b1d..4113b221de7 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -151,6 +151,7 @@ tests_i386_system_quick = [ ] tests_i386_system_thorough = [ + 'i386_replay', 'i386_tuxrun', ] diff --git a/tests/functional/test_i386_replay.py b/tests/functional/test_i386_replay.py new file mode 100755 index 00000000000..7c4c2602da9 --- /dev/null +++ b/tests/functional/test_i386_replay.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python3 +# +# Replay test that boots a Linux kernel on a i386 machine +# and checks the console +# +# SPDX-License-Identifier: GPL-2.0-or-later + +from qemu_test import Asset +from replay_kernel import ReplayKernelBase + + +class I386Replay(ReplayKernelBase): + + ASSET_KERNEL = Asset( + 'https://storage.tuxboot.com/20230331/i386/bzImage', + 'a3e5b32a354729e65910f5a1ffcda7c14a6c12a55e8213fb86e277f1b76ed956') + + def test_pc(self): + self.set_machine('pc') + kernel_url = () + kernel_path = self.ASSET_KERNEL.fetch() + kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0' + console_pattern = 'VFS: Cannot open root device' + self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5) + + +if __name__ == '__main__': + ReplayKernelBase.main() From patchwork Mon Apr 14 11:30:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 83C0AC369B8 for ; Mon, 14 Apr 2025 11:32:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I2D-0003jB-SZ; Mon, 14 Apr 2025 07:31:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1Z-0003Tq-QZ for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1Y-0003u0-0p for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630247; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S97ls/REyKax5kRch48GSdItlPTuqCpNJsorhHCsEfE=; b=ANFJSlMxPPiwITA97Hho0WMeMxFDbiWYYPmN3zZAU3lCW6wG+GZlFk53t6NhrRR3sDi2MZ cBZzJEnhtO6gkykfEr1lu+PmyOj3GIVUEkW7RjkI/VSehdYSO3PDZGcJin08Fp4KZrxZ5o YN1P8aBwhZoS0gtodnJrC84Mz2wiFFA= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-346-VSbJgsd5PA-xqEPuL5wzKQ-1; Mon, 14 Apr 2025 07:30:44 -0400 X-MC-Unique: VSbJgsd5PA-xqEPuL5wzKQ-1 X-Mimecast-MFC-AGG-ID: VSbJgsd5PA-xqEPuL5wzKQ_1744630243 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 30E901955DC5; Mon, 14 Apr 2025 11:30:43 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 12C121956094; Mon, 14 Apr 2025 11:30:41 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 05/15] tests/avocado: Remove the LinuxKernelTest class Date: Mon, 14 Apr 2025 13:30:19 +0200 Message-ID: <20250414113031.151105-6-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth All tests that used this class have been converted to the functional framework, so we can remove the boot_linux_console.py file now. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/avocado/boot_linux_console.py | 62 ----------------------------- 1 file changed, 62 deletions(-) delete mode 100644 tests/avocado/boot_linux_console.py diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py deleted file mode 100644 index cbb1e2fb506..00000000000 --- a/tests/avocado/boot_linux_console.py +++ /dev/null @@ -1,62 +0,0 @@ -# Functional test that boots a Linux kernel and checks the console -# -# Copyright (c) 2018 Red Hat, Inc. -# -# Author: -# Cleber Rosa -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -import os -import shutil - -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern -from avocado.utils import process -from avocado.utils import archive - -class LinuxKernelTest(QemuSystemTest): - KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 ' - - def wait_for_console_pattern(self, success_message, vm=None): - wait_for_console_pattern(self, success_message, - failure_message='Kernel panic - not syncing', - vm=vm) - - def extract_from_deb(self, deb, path): - """ - Extracts a file from a deb package into the test workdir - - :param deb: path to the deb archive - :param path: path within the deb archive of the file to be extracted - :returns: path of the extracted file - """ - cwd = os.getcwd() - os.chdir(self.workdir) - file_path = process.run("ar t %s" % deb).stdout_text.split()[2] - process.run("ar x %s %s" % (deb, file_path)) - archive.extract(file_path, self.workdir) - os.chdir(cwd) - # Return complete path to extracted file. Because callers to - # extract_from_deb() specify 'path' with a leading slash, it is - # necessary to use os.path.relpath() as otherwise os.path.join() - # interprets it as an absolute path and drops the self.workdir part. - return os.path.normpath(os.path.join(self.workdir, - os.path.relpath(path, '/'))) - - def extract_from_rpm(self, rpm, path): - """ - Extracts a file from an RPM package into the test workdir. - - :param rpm: path to the rpm archive - :param path: path within the rpm archive of the file to be extracted - needs to be a relative path (starting with './') because - cpio(1), which is used to extract the file, expects that. - :returns: path of the extracted file - """ - cwd = os.getcwd() - os.chdir(self.workdir) - process.run("rpm2cpio %s | cpio -id %s" % (rpm, path), shell=True) - os.chdir(cwd) - return os.path.normpath(os.path.join(self.workdir, path)) From patchwork Mon Apr 14 11:30:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7FAF6C369B5 for ; Mon, 14 Apr 2025 11:31:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I2C-0003gv-K6; Mon, 14 Apr 2025 07:31:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1c-0003UX-Ot for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1a-0003uQ-RT for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630250; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PwvGpx7Z/Q9SsEMSl5BbEBr4H1P4O/AUnFaAB5CS8Cs=; b=AHY2b2EoYasrpRT16QoblwROMAfNzwYDNS1PbyAmnkhb/WRaogHCIybraxI30IAbJYpcW0 FfKfS4q5b//sTIcStc3UQGv6ql7h+Dx65QsFTLxIXFe0M/zHKg4L5KcOTyJ/vzuOhtlD4V T6JKvQu9G6PJjM3/dFU+2llzn5kuq+M= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-392-AuWz9QKGP723yEDDbAp7tA-1; Mon, 14 Apr 2025 07:30:45 -0400 X-MC-Unique: AuWz9QKGP723yEDDbAp7tA-1 X-Mimecast-MFC-AGG-ID: AuWz9QKGP723yEDDbAp7tA_1744630244 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C3DEB1800259; Mon, 14 Apr 2025 11:30:44 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 95A6A1956094; Mon, 14 Apr 2025 11:30:43 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 06/15] tests/functional: Convert the 32-bit big endian Wheezy mips test Date: Mon, 14 Apr 2025 13:30:20 +0200 Message-ID: <20250414113031.151105-7-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth The test checks some entries in /proc and the output of some commands ... we put these checks into exportable functions now so that they can be reused more easily. Additionally the linux_ssh_mips_malta.py uses SSH to test the networking of the guest. Since we don't have a SSH module in the functional framework yet, let's use the check_http_download() function here instead. And while we're at it, also switch the NIC to e1000 now to get some more test coverage, since the "pcnet" device is already tested in the test test_mips_malta_cpio. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/avocado/linux_ssh_mips_malta.py | 8 -- tests/functional/meson.build | 2 +- tests/functional/test_mips_malta.py | 108 +++++++++++++++++++++++++- 3 files changed, 107 insertions(+), 11 deletions(-) diff --git a/tests/avocado/linux_ssh_mips_malta.py b/tests/avocado/linux_ssh_mips_malta.py index d9bb525ad9c..73d294ad5cf 100644 --- a/tests/avocado/linux_ssh_mips_malta.py +++ b/tests/avocado/linux_ssh_mips_malta.py @@ -172,14 +172,6 @@ def check_mips_malta(self, uname_m, endianess): # Wait for VM to shut down gracefully self.vm.wait() - def test_mips_malta32eb_kernel3_2_0(self): - """ - :avocado: tags=arch:mips - :avocado: tags=endian:big - :avocado: tags=device:pcnet32 - """ - self.check_mips_malta('mips', 'be') - def test_mips_malta32el_kernel3_2_0(self): """ :avocado: tags=arch:mipsel diff --git a/tests/functional/meson.build b/tests/functional/meson.build index 4113b221de7..4dedfc7b9f7 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -39,7 +39,7 @@ test_timeouts = { 'arm_tuxrun' : 240, 'arm_sx1' : 360, 'intel_iommu': 300, - 'mips_malta' : 120, + 'mips_malta' : 480, 'mipsel_replay' : 480, 'mips64el_replay' : 180, 'netdev_ethtool' : 180, diff --git a/tests/functional/test_mips_malta.py b/tests/functional/test_mips_malta.py index 9697c7d63f8..89b9556f30d 100755 --- a/tests/functional/test_mips_malta.py +++ b/tests/functional/test_mips_malta.py @@ -6,10 +6,93 @@ # # SPDX-License-Identifier: GPL-2.0-or-later -from qemu_test import LinuxKernelTest, Asset +import os + +from qemu_test import LinuxKernelTest, Asset, wait_for_console_pattern from qemu_test import exec_command_and_wait_for_pattern +def mips_run_common_commands(test, prompt='#'): + exec_command_and_wait_for_pattern(test, + 'uname -m', + 'mips') + exec_command_and_wait_for_pattern(test, + 'grep XT-PIC /proc/interrupts', + 'timer') + wait_for_console_pattern(test, prompt) + exec_command_and_wait_for_pattern(test, + 'grep XT-PIC /proc/interrupts', + 'serial') + wait_for_console_pattern(test, prompt) + exec_command_and_wait_for_pattern(test, + 'grep XT-PIC /proc/interrupts', + 'ata_piix') + wait_for_console_pattern(test, prompt) + exec_command_and_wait_for_pattern(test, + 'grep XT-PIC /proc/interrupts', + 'rtc') + wait_for_console_pattern(test, prompt) + exec_command_and_wait_for_pattern(test, + 'cat /proc/devices', + 'input') + wait_for_console_pattern(test, prompt) + exec_command_and_wait_for_pattern(test, + 'cat /proc/devices', + 'fb') + wait_for_console_pattern(test, prompt) + exec_command_and_wait_for_pattern(test, + 'cat /proc/ioports', + ' : serial') + wait_for_console_pattern(test, prompt) + exec_command_and_wait_for_pattern(test, + 'cat /proc/ioports', + ' : ata_piix') + wait_for_console_pattern(test, prompt) + +def mips_check_wheezy(test, kernel_path, image_path, kernel_command_line, + dl_file, hsum, nic='pcnet', cpuinfo='MIPS 24Kc'): + test.require_netdev('user') + test.require_device(nic) + test.set_machine('malta') + + port=8080 + test.vm.add_args('-kernel', kernel_path, + '-append', kernel_command_line, + '-drive', 'file=%s,snapshot=on' % image_path, + '-netdev', 'user,id=n1' + + ',tftp=' + os.path.basename(kernel_path) + + ',hostfwd=tcp:127.0.0.1:0-:%d' % port, + '-device', f'{nic},netdev=n1', + '-no-reboot') + test.vm.set_console() + test.vm.launch() + + wait_for_console_pattern(test, 'login: ', 'Oops') + exec_command_and_wait_for_pattern(test, 'root', 'Password:') + exec_command_and_wait_for_pattern(test, 'root', ':~# ') + mips_run_common_commands(test) + + exec_command_and_wait_for_pattern(test, 'cd /', '# ') + test.check_http_download(dl_file, hsum, port, + pythoncmd='python -m SimpleHTTPServer') + + exec_command_and_wait_for_pattern(test, 'cat /proc/cpuinfo', cpuinfo) + exec_command_and_wait_for_pattern(test, 'cat /proc/devices', 'usb') + exec_command_and_wait_for_pattern(test, 'cat /proc/ioports', + ' : piix4_smbus') + # lspci for the host bridge does not work on big endian targets: + # https://gitlab.com/qemu-project/qemu/-/issues/2826 + # exec_command_and_wait_for_pattern(test, 'lspci -d 11ab:4620', + # 'GT-64120') + exec_command_and_wait_for_pattern(test, + 'cat /sys/bus/i2c/devices/i2c-0/name', + 'SMBus PIIX4 adapter') + exec_command_and_wait_for_pattern(test, 'cat /proc/mtd', 'YAMON') + # Empty 'Board Config' (64KB) + exec_command_and_wait_for_pattern(test, 'md5sum /dev/mtd2ro', + '0dfbe8aa4c20b52e1b8bf3cb6cbdf193') + + class MaltaMachineConsole(LinuxKernelTest): ASSET_KERNEL_2_63_2 = Asset( @@ -70,7 +153,8 @@ def test_mips_malta_cpio(self): exec_command_and_wait_for_pattern(self, 'cat /proc/cpuinfo', 'BogoMIPS') exec_command_and_wait_for_pattern(self, 'uname -a', - 'Debian') + '4.5.0-2-4kc-malta #1 Debian') + mips_run_common_commands(self) exec_command_and_wait_for_pattern(self, 'ip link set eth0 up', 'eth0: link up') @@ -89,6 +173,26 @@ def test_mips_malta_cpio(self): # Wait for VM to shut down gracefully self.vm.wait() + ASSET_WHEEZY_KERNEL = Asset( + ('https://people.debian.org/~aurel32/qemu/mips/' + 'vmlinux-3.2.0-4-4kc-malta'), + '0377fcda31299213c10b8e5babe7260ef99188b3ae1aca6f56594abb71e7f67e') + + ASSET_WHEEZY_DISK = Asset( + ('https://people.debian.org/~aurel32/qemu/mips/' + 'debian_wheezy_mips_standard.qcow2'), + 'de03599285b8382ad309309a6c4869f6c6c42a5cfc983342bab9ec0dfa7849a2') + + def test_wheezy(self): + kernel_path = self.ASSET_WHEEZY_KERNEL.fetch() + image_path = self.ASSET_WHEEZY_DISK.fetch() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 root=/dev/sda1') + mips_check_wheezy(self, + kernel_path, image_path, kernel_command_line, nic='e1000', + dl_file='/boot/initrd.img-3.2.0-4-4kc-malta', + hsum='ff0c0369143d9bbb9a6e6bc79322a2be535619df639e84103237f406e87493dc') + if __name__ == '__main__': LinuxKernelTest.main() From patchwork Mon Apr 14 11:30:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6E67FC369B2 for ; Mon, 14 Apr 2025 11:36:29 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I2K-0003na-LU; Mon, 14 Apr 2025 07:31:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1k-0003Vk-B7 for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:31:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1i-0003wB-ID for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:31:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630257; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aF74lNmpX+5QNfKZxlD+JtFGLxYnqnODpVhAJIrKW/s=; b=GYEAkS7iSZu1yPXZDhimS8fCI/Q5SR1+D/IITPIQMsyiC+/g39xNwds/38rZTuBvmzynzh Joq2PyUj3mKpmTjsT94kTpZ+4x/GOT6mGGa6vOFBwmjzDfZ9dAt7lMMjn5t3HwP+QmtnPT gXigVU1QMiMJYdh9xZF+ekBNoEDxKd4= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-589-AWkx1PB3MQGka3RE5dujmw-1; Mon, 14 Apr 2025 07:30:49 -0400 X-MC-Unique: AWkx1PB3MQGka3RE5dujmw-1 X-Mimecast-MFC-AGG-ID: AWkx1PB3MQGka3RE5dujmw_1744630248 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 44A11195609F; Mon, 14 Apr 2025 11:30:46 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 26B541956094; Mon, 14 Apr 2025 11:30:44 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 07/15] tests/functional: Convert the 32-bit little endian Wheezy mips test Date: Mon, 14 Apr 2025 13:30:21 +0200 Message-ID: <20250414113031.151105-8-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth Reuse the test function from the big endian test to easily convert the 32-bit little endian Wheezy mips test. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/avocado/linux_ssh_mips_malta.py | 8 -------- tests/functional/meson.build | 1 + tests/functional/test_mipsel_malta.py | 22 ++++++++++++++++++++++ 3 files changed, 23 insertions(+), 8 deletions(-) diff --git a/tests/avocado/linux_ssh_mips_malta.py b/tests/avocado/linux_ssh_mips_malta.py index 73d294ad5cf..c1300aec961 100644 --- a/tests/avocado/linux_ssh_mips_malta.py +++ b/tests/avocado/linux_ssh_mips_malta.py @@ -172,14 +172,6 @@ def check_mips_malta(self, uname_m, endianess): # Wait for VM to shut down gracefully self.vm.wait() - def test_mips_malta32el_kernel3_2_0(self): - """ - :avocado: tags=arch:mipsel - :avocado: tags=endian:little - :avocado: tags=device:pcnet32 - """ - self.check_mips_malta('mips', 'le') - def test_mips_malta64eb_kernel3_2_0(self): """ :avocado: tags=arch:mips64 diff --git a/tests/functional/meson.build b/tests/functional/meson.build index 4dedfc7b9f7..cba0abf2920 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -40,6 +40,7 @@ test_timeouts = { 'arm_sx1' : 360, 'intel_iommu': 300, 'mips_malta' : 480, + 'mipsel_malta' : 420, 'mipsel_replay' : 480, 'mips64el_replay' : 180, 'netdev_ethtool' : 180, diff --git a/tests/functional/test_mipsel_malta.py b/tests/functional/test_mipsel_malta.py index fe9c3a172e6..9ee2884da8e 100755 --- a/tests/functional/test_mipsel_malta.py +++ b/tests/functional/test_mipsel_malta.py @@ -13,6 +13,8 @@ from qemu_test import interrupt_interactive_console_until_pattern from qemu_test import wait_for_console_pattern +from test_mips_malta import mips_check_wheezy + class MaltaMachineConsole(LinuxKernelTest): @@ -57,6 +59,26 @@ def test_mips_malta32el_nanomips_16k_up(self): def test_mips_malta32el_nanomips_64k_dbg(self): self.do_test_mips_malta32el_nanomips(self.ASSET_KERNEL_64K) + ASSET_WHEEZY_KERNEL = Asset( + ('https://people.debian.org/~aurel32/qemu/mipsel/' + 'vmlinux-3.2.0-4-4kc-malta'), + 'dc8a3648305b0201ca7a5cd135fe2890067a65d93c38728022bb0e656ad2bf9a') + + ASSET_WHEEZY_DISK = Asset( + ('https://people.debian.org/~aurel32/qemu/mipsel/' + 'debian_wheezy_mipsel_standard.qcow2'), + '454f09ae39f7e6461c84727b927100d2c7813841f2a0a5dce328114887ecf914') + + def test_wheezy(self): + kernel_path = self.ASSET_WHEEZY_KERNEL.fetch() + image_path = self.ASSET_WHEEZY_DISK.fetch() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 root=/dev/sda1') + mips_check_wheezy(self, + kernel_path, image_path, kernel_command_line, + dl_file='/boot/initrd.img-3.2.0-4-4kc-malta', + hsum='9fc9f250ed56a74e35e704ddfd5a1c5a5625adefc5c9da91f649288d3ca000f0') + class MaltaMachineYAMON(QemuSystemTest): From patchwork Mon Apr 14 11:30:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 22D9BC369B5 for ; Mon, 14 Apr 2025 11:32:06 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I24-0003Yn-Vq; Mon, 14 Apr 2025 07:31:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1h-0003VO-Op for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:31:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1f-0003v6-DA for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630253; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ks7zB8+UXPocpjnbLqRfBX7yiiMVfQgEztwKi8rXsU8=; b=TDuyhrPkjpbQTFaevTfhDdvm36V37xjxcn8Lq81fGZUsSLhNFwdupS/q1PDzp0KLkFz3ti 8i4x0fYxv5z+tP3slFH29Kx3cbW8vkyiMPsrD4t+MeN7xoKo3Vi2kTHAVWcqJP2RmJ4sUN K4OODB44ABY8K0wdHqj8WSq0en9r6v4= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-653-qV-Ic9xNODyXoebia3YAxg-1; Mon, 14 Apr 2025 07:30:48 -0400 X-MC-Unique: qV-Ic9xNODyXoebia3YAxg-1 X-Mimecast-MFC-AGG-ID: qV-Ic9xNODyXoebia3YAxg_1744630248 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C9A13180035E; Mon, 14 Apr 2025 11:30:47 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id AAA0B1956094; Mon, 14 Apr 2025 11:30:46 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 08/15] tests/functional: Convert the 64-bit little endian Wheezy mips test Date: Mon, 14 Apr 2025 13:30:22 +0200 Message-ID: <20250414113031.151105-9-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth Reuse the test function from the 32-bit big endian test to easily convert the 64-bit little endian Wheezy mips test. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/avocado/linux_ssh_mips_malta.py | 8 -------- tests/functional/meson.build | 1 + tests/functional/test_mips64el_malta.py | 22 ++++++++++++++++++++++ 3 files changed, 23 insertions(+), 8 deletions(-) diff --git a/tests/avocado/linux_ssh_mips_malta.py b/tests/avocado/linux_ssh_mips_malta.py index c1300aec961..2fa5cf9a6c1 100644 --- a/tests/avocado/linux_ssh_mips_malta.py +++ b/tests/avocado/linux_ssh_mips_malta.py @@ -179,11 +179,3 @@ def test_mips_malta64eb_kernel3_2_0(self): :avocado: tags=device:pcnet32 """ self.check_mips_malta('mips64', 'be') - - def test_mips_malta64el_kernel3_2_0(self): - """ - :avocado: tags=arch:mips64el - :avocado: tags=endian:little - :avocado: tags=device:pcnet32 - """ - self.check_mips_malta('mips64', 'le') diff --git a/tests/functional/meson.build b/tests/functional/meson.build index cba0abf2920..a4a317115b3 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -42,6 +42,7 @@ test_timeouts = { 'mips_malta' : 480, 'mipsel_malta' : 420, 'mipsel_replay' : 480, + 'mips64el_malta' : 420, 'mips64el_replay' : 180, 'netdev_ethtool' : 180, 'ppc_40p' : 240, diff --git a/tests/functional/test_mips64el_malta.py b/tests/functional/test_mips64el_malta.py index a8da15a26b3..dd37212f9df 100755 --- a/tests/functional/test_mips64el_malta.py +++ b/tests/functional/test_mips64el_malta.py @@ -16,6 +16,8 @@ from qemu_test import exec_command_and_wait_for_pattern from qemu_test import skipIfMissingImports, skipFlakyTest, skipUntrustedTest +from test_mips_malta import mips_check_wheezy + class MaltaMachineConsole(LinuxKernelTest): @@ -90,6 +92,26 @@ def test_mips64el_malta_5KEc_cpio(self): # Wait for VM to shut down gracefully self.vm.wait() + ASSET_WHEEZY_KERNEL = Asset( + ('https://people.debian.org/~aurel32/qemu/mipsel/' + 'vmlinux-3.2.0-4-5kc-malta'), + '5e8b725244c59745bb8b64f5d8f49f25fecfa549f3395fb6d19a3b9e5065b85b') + + ASSET_WHEEZY_DISK = Asset( + ('https://people.debian.org/~aurel32/qemu/mipsel/' + 'debian_wheezy_mipsel_standard.qcow2'), + '454f09ae39f7e6461c84727b927100d2c7813841f2a0a5dce328114887ecf914') + + def test_wheezy(self): + kernel_path = self.ASSET_WHEEZY_KERNEL.fetch() + image_path = self.ASSET_WHEEZY_DISK.fetch() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 root=/dev/sda1') + mips_check_wheezy(self, + kernel_path, image_path, kernel_command_line, cpuinfo='MIPS 20Kc', + dl_file='/boot/initrd.img-3.2.0-4-5kc-malta', + hsum='7579f8b56c1187c7c04d0dc3c0c56c7a6314c5ddd3a9bf8803ecc7cf8a3be9f8') + @skipIfMissingImports('numpy', 'cv2') class MaltaMachineFramebuffer(LinuxKernelTest): From patchwork Mon Apr 14 11:30:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0C47DC369B2 for ; Mon, 14 Apr 2025 11:34:09 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I1z-0003YU-SX; Mon, 14 Apr 2025 07:31:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1i-0003VP-3E for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:31:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1e-0003v3-LG for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630253; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eYlrb1JVcm8Wju4j1WOTQs1ZvsOuLGubR+3mj5+Icqk=; b=dhRS17xctsuTeKwjazrmFLy90LCkP69PCIxDA9FakEK0JtBvHv4aGo5qmnwx8puswd3V9G i7Vn2wWjKCKaicL3Qt483u1qRRTCOqOxD5evVXvoL+dKZLS14MXSXnwwrfBimHFdD1AKXZ HBuiBRyU0jAE4FZqM6re4h2t4TipS8E= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-62-0SMugsULNZWhz6aEH73izQ-1; Mon, 14 Apr 2025 07:30:51 -0400 X-MC-Unique: 0SMugsULNZWhz6aEH73izQ-1 X-Mimecast-MFC-AGG-ID: 0SMugsULNZWhz6aEH73izQ_1744630249 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 56AF31800263; Mon, 14 Apr 2025 11:30:49 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 3933F1956094; Mon, 14 Apr 2025 11:30:48 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 09/15] tests/functional: Convert the 64-bit big endian Wheezy mips test Date: Mon, 14 Apr 2025 13:30:23 +0200 Message-ID: <20250414113031.151105-10-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth Reuse the test function from the 32-bit big endian test to easily convert the 64-bit big endian Wheezy mips test. Since this was the last test in tests/avocado/linux_ssh_mips_malta.py, we can remove this avocado file now, too. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- MAINTAINERS | 1 - tests/avocado/linux_ssh_mips_malta.py | 181 -------------------------- tests/functional/meson.build | 2 + tests/functional/test_mips64_malta.py | 35 +++++ 4 files changed, 37 insertions(+), 182 deletions(-) delete mode 100644 tests/avocado/linux_ssh_mips_malta.py create mode 100755 tests/functional/test_mips64_malta.py diff --git a/MAINTAINERS b/MAINTAINERS index 42348df9d68..72f6b208f5a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1355,7 +1355,6 @@ F: hw/acpi/piix4.c F: hw/mips/malta.c F: hw/pci-host/gt64120.c F: include/hw/southbridge/piix.h -F: tests/avocado/linux_ssh_mips_malta.py F: tests/functional/test_mips*_malta.py F: tests/functional/test_mips*_tuxrun.py diff --git a/tests/avocado/linux_ssh_mips_malta.py b/tests/avocado/linux_ssh_mips_malta.py deleted file mode 100644 index 2fa5cf9a6c1..00000000000 --- a/tests/avocado/linux_ssh_mips_malta.py +++ /dev/null @@ -1,181 +0,0 @@ -# Functional test that boots a VM and run commands via a SSH session -# -# Copyright (c) Philippe Mathieu-Daudé -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -import os -import re -import base64 -import logging -import time - -from avocado import skipUnless -from avocado_qemu import LinuxSSHMixIn -from avocado_qemu import QemuSystemTest -from avocado_qemu import wait_for_console_pattern -from avocado.utils import process -from avocado.utils import archive -from avocado.utils import ssh - - -@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout') -@skipUnless(ssh.SSH_CLIENT_BINARY, 'No SSH client available') -class LinuxSSH(QemuSystemTest, LinuxSSHMixIn): - """ - :avocado: tags=accel:tcg - """ - - timeout = 150 # Not for 'configure --enable-debug --enable-debug-tcg' - - KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 ' - VM_IP = '127.0.0.1' - - BASE_URL = 'https://people.debian.org/~aurel32/qemu/' - IMAGE_INFO = { - 'be': {'base_url': 'mips', - 'image_name': 'debian_wheezy_mips_standard.qcow2', - 'image_hash': '8987a63270df67345b2135a6b7a4885a35e392d5', - 'kernel_hash': { - 32: '592e384a4edc16dade52a6cd5c785c637bcbc9ad', - 64: 'db6eea7de35d36c77d8c165b6bcb222e16eb91db'} - }, - 'le': {'base_url': 'mipsel', - 'image_name': 'debian_wheezy_mipsel_standard.qcow2', - 'image_hash': '7866764d9de3ef536ffca24c9fb9f04ffdb45802', - 'kernel_hash': { - 32: 'a66bea5a8adaa2cb3d36a1d4e0ccdb01be8f6c2a', - 64: '6a7f77245acf231415a0e8b725d91ed2f3487794'} - } - } - CPU_INFO = { - 32: {'cpu': 'MIPS 24Kc', 'kernel_release': '3.2.0-4-4kc-malta'}, - 64: {'cpu': 'MIPS 20Kc', 'kernel_release': '3.2.0-4-5kc-malta'} - } - - def get_url(self, endianess, path=''): - qkey = {'le': 'el', 'be': ''} - return '%s/mips%s/%s' % (self.BASE_URL, qkey[endianess], path) - - def get_image_info(self, endianess): - dinfo = self.IMAGE_INFO[endianess] - image_url = self.get_url(endianess, dinfo['image_name']) - image_hash = dinfo['image_hash'] - return (image_url, image_hash) - - def get_kernel_info(self, endianess, wordsize): - minfo = self.CPU_INFO[wordsize] - kernel_url = self.get_url(endianess, - 'vmlinux-%s' % minfo['kernel_release']) - kernel_hash = self.IMAGE_INFO[endianess]['kernel_hash'][wordsize] - return kernel_url, kernel_hash - - def ssh_disconnect_vm(self): - self.ssh_session.quit() - - def boot_debian_wheezy_image_and_ssh_login(self, endianess, kernel_path): - image_url, image_hash = self.get_image_info(endianess) - image_path = self.fetch_asset(image_url, asset_hash=image_hash) - - self.vm.set_console() - kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE - + 'console=ttyS0 root=/dev/sda1') - self.vm.add_args('-no-reboot', - '-kernel', kernel_path, - '-append', kernel_command_line, - '-drive', 'file=%s,snapshot=on' % image_path, - '-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22', - '-device', 'pcnet,netdev=vnet') - self.vm.launch() - - self.log.info('VM launched, waiting for sshd') - console_pattern = 'Starting OpenBSD Secure Shell server: sshd' - wait_for_console_pattern(self, console_pattern, 'Oops') - self.log.info('sshd ready') - - self.ssh_connect('root', 'root', False) - - def shutdown_via_ssh(self): - self.ssh_command('poweroff') - self.ssh_disconnect_vm() - wait_for_console_pattern(self, 'Power down', 'Oops') - - def run_common_commands(self, wordsize): - self.ssh_command_output_contains( - 'cat /proc/cpuinfo', - self.CPU_INFO[wordsize]['cpu']) - self.ssh_command_output_contains( - 'uname -m', - 'mips') - self.ssh_command_output_contains( - 'uname -r', - self.CPU_INFO[wordsize]['kernel_release']) - self.ssh_command_output_contains( - 'cat /proc/interrupts', - 'XT-PIC timer') - self.ssh_command_output_contains( - 'cat /proc/interrupts', - 'XT-PIC i8042') - self.ssh_command_output_contains( - 'cat /proc/interrupts', - 'XT-PIC serial') - self.ssh_command_output_contains( - 'cat /proc/interrupts', - 'XT-PIC ata_piix') - self.ssh_command_output_contains( - 'cat /proc/interrupts', - 'XT-PIC eth0') - self.ssh_command_output_contains( - 'cat /proc/devices', - 'input') - self.ssh_command_output_contains( - 'cat /proc/devices', - 'usb') - self.ssh_command_output_contains( - 'cat /proc/devices', - 'fb') - self.ssh_command_output_contains( - 'cat /proc/ioports', - ' : serial') - self.ssh_command_output_contains( - 'cat /proc/ioports', - ' : ata_piix') - self.ssh_command_output_contains( - 'cat /proc/ioports', - ' : piix4_smbus') - self.ssh_command_output_contains( - 'lspci -d 11ab:4620', - 'GT-64120') - self.ssh_command_output_contains( - 'cat /sys/bus/i2c/devices/i2c-0/name', - 'SMBus PIIX4 adapter') - self.ssh_command_output_contains( - 'cat /proc/mtd', - 'YAMON') - # Empty 'Board Config' (64KB) - self.ssh_command_output_contains( - 'md5sum /dev/mtd2ro', - '0dfbe8aa4c20b52e1b8bf3cb6cbdf193') - - def check_mips_malta(self, uname_m, endianess): - wordsize = 64 if '64' in uname_m else 32 - kernel_url, kernel_hash = self.get_kernel_info(endianess, wordsize) - kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash) - self.boot_debian_wheezy_image_and_ssh_login(endianess, kernel_path) - - stdout, _ = self.ssh_command('uname -a') - self.assertIn(True, [uname_m + " GNU/Linux" in line for line in stdout]) - - self.run_common_commands(wordsize) - self.shutdown_via_ssh() - # Wait for VM to shut down gracefully - self.vm.wait() - - def test_mips_malta64eb_kernel3_2_0(self): - """ - :avocado: tags=arch:mips64 - :avocado: tags=endian:big - :avocado: tags=device:pcnet32 - """ - self.check_mips_malta('mips64', 'be') diff --git a/tests/functional/meson.build b/tests/functional/meson.build index a4a317115b3..985ac5c27fe 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -42,6 +42,7 @@ test_timeouts = { 'mips_malta' : 480, 'mipsel_malta' : 420, 'mipsel_replay' : 480, + 'mips64_malta' : 240, 'mips64el_malta' : 420, 'mips64el_replay' : 180, 'netdev_ethtool' : 180, @@ -191,6 +192,7 @@ tests_mipsel_system_thorough = [ ] tests_mips64_system_thorough = [ + 'mips64_malta', 'mips64_tuxrun', ] diff --git a/tests/functional/test_mips64_malta.py b/tests/functional/test_mips64_malta.py new file mode 100755 index 00000000000..53c3e0c1221 --- /dev/null +++ b/tests/functional/test_mips64_malta.py @@ -0,0 +1,35 @@ +#!/usr/bin/env python3 +# +# Functional tests for the big-endian 64-bit MIPS Malta board +# +# SPDX-License-Identifier: GPL-2.0-or-later + +from qemu_test import LinuxKernelTest, Asset +from test_mips_malta import mips_check_wheezy + + +class MaltaMachineConsole(LinuxKernelTest): + + ASSET_WHEEZY_KERNEL = Asset( + ('https://people.debian.org/~aurel32/qemu/mips/' + 'vmlinux-3.2.0-4-5kc-malta'), + '3e4ec154db080b3f1839f04dde83120654a33e5e1716863de576c47cb94f68f6') + + ASSET_WHEEZY_DISK = Asset( + ('https://people.debian.org/~aurel32/qemu/mips/' + 'debian_wheezy_mips_standard.qcow2'), + 'de03599285b8382ad309309a6c4869f6c6c42a5cfc983342bab9ec0dfa7849a2') + + def test_wheezy(self): + kernel_path = self.ASSET_WHEEZY_KERNEL.fetch() + image_path = self.ASSET_WHEEZY_DISK.fetch() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 root=/dev/sda1') + mips_check_wheezy(self, + kernel_path, image_path, kernel_command_line, cpuinfo='MIPS 20Kc', + dl_file='/boot/initrd.img-3.2.0-4-5kc-malta', + hsum='d98b953bb4a41c0fc0fd8d19bbc691c08989ac52568c1d3054d92dfd890d3f06') + + +if __name__ == '__main__': + LinuxKernelTest.main() From patchwork Mon Apr 14 11:30:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1E310C369B2 for ; Mon, 14 Apr 2025 11:32:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I2F-0003m1-JY; Mon, 14 Apr 2025 07:31:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1j-0003VU-Fu for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:31:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1g-0003vH-F7 for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630254; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N+iAdaQ+NI5jREqyJw1BczLgIl/Xxx+7leiQHYSN6Rc=; b=COTc8lq6xwHFaAluqIfOcnrFd/KiprgCuIfo98zrUUzEWpb3JILV48m1Bjuel3NDmlRte+ gqzClgnd/LMZr6LHAS+yNKBYAlizzuZuP4JxQ8qkHuH2NstRJbuVmISoc10XLqHiubIdl8 xBRheM4L5NSCyLy25lZUz19DzaWWhLo= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-605-eFl2C2d9NXiAqvZ5fzVWZA-1; Mon, 14 Apr 2025 07:30:51 -0400 X-MC-Unique: eFl2C2d9NXiAqvZ5fzVWZA-1 X-Mimecast-MFC-AGG-ID: eFl2C2d9NXiAqvZ5fzVWZA_1744630251 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DC7F2180025B; Mon, 14 Apr 2025 11:30:50 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BD5C61956094; Mon, 14 Apr 2025 11:30:49 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 10/15] tests/avocado: Remove the boot_linux.py tests Date: Mon, 14 Apr 2025 13:30:24 +0200 Message-ID: <20250414113031.151105-11-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth These tests are based on the cloudinit functions from Avocado. The cloudinit is very, very slow compared to our other tests, so most of these Avocado tests have either been disabled by default with a decorator, or have been marked to only run with KVM. We won't include this sluggish cloudinit stuff in the functional framework, and we've already got plenty of other tests there that check pretty much the same things, so let's simply get rid of these old tests now. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/avocado/boot_linux.py | 132 ------------------------------------ 1 file changed, 132 deletions(-) delete mode 100644 tests/avocado/boot_linux.py diff --git a/tests/avocado/boot_linux.py b/tests/avocado/boot_linux.py deleted file mode 100644 index a029ef4ad1e..00000000000 --- a/tests/avocado/boot_linux.py +++ /dev/null @@ -1,132 +0,0 @@ -# Functional test that boots a complete Linux system via a cloud image -# -# Copyright (c) 2018-2020 Red Hat, Inc. -# -# Author: -# Cleber Rosa -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -import os - -from avocado_qemu.linuxtest import LinuxTest -from avocado_qemu import BUILD_DIR - -from avocado import skipUnless - - -class BootLinuxX8664(LinuxTest): - """ - :avocado: tags=arch:x86_64 - """ - timeout = 480 - - def test_pc_i440fx_tcg(self): - """ - :avocado: tags=machine:pc - :avocado: tags=accel:tcg - """ - self.require_accelerator("tcg") - self.vm.add_args("-accel", "tcg") - self.launch_and_wait(set_up_ssh_connection=False) - - def test_pc_i440fx_kvm(self): - """ - :avocado: tags=machine:pc - :avocado: tags=accel:kvm - """ - self.require_accelerator("kvm") - self.vm.add_args("-accel", "kvm") - self.launch_and_wait(set_up_ssh_connection=False) - - def test_pc_q35_tcg(self): - """ - :avocado: tags=machine:q35 - :avocado: tags=accel:tcg - """ - self.require_accelerator("tcg") - self.vm.add_args("-accel", "tcg") - self.launch_and_wait(set_up_ssh_connection=False) - - def test_pc_q35_kvm(self): - """ - :avocado: tags=machine:q35 - :avocado: tags=accel:kvm - """ - self.require_accelerator("kvm") - self.vm.add_args("-accel", "kvm") - self.launch_and_wait(set_up_ssh_connection=False) - - -# For Aarch64 we only boot KVM tests in CI as booting the current -# Fedora OS in TCG tests is very heavyweight. There are lighter weight -# distros which we use in the machine_aarch64_virt.py tests. -class BootLinuxAarch64(LinuxTest): - """ - :avocado: tags=arch:aarch64 - :avocado: tags=machine:virt - """ - timeout = 720 - - def test_virt_kvm(self): - """ - :avocado: tags=accel:kvm - :avocado: tags=cpu:host - """ - self.require_accelerator("kvm") - self.vm.add_args("-accel", "kvm") - self.vm.add_args("-machine", "virt,gic-version=host") - self.vm.add_args('-bios', - os.path.join(BUILD_DIR, 'pc-bios', - 'edk2-aarch64-code.fd')) - self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0') - self.vm.add_args('-object', 'rng-random,id=rng0,filename=/dev/urandom') - self.launch_and_wait(set_up_ssh_connection=False) - - -# See the tux_baseline.py tests for almost the same coverage in a lot -# less time. -class BootLinuxPPC64(LinuxTest): - """ - :avocado: tags=arch:ppc64 - """ - - timeout = 360 - - @skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited') - def test_pseries_tcg(self): - """ - :avocado: tags=machine:pseries - :avocado: tags=accel:tcg - """ - self.require_accelerator("tcg") - self.vm.add_args("-accel", "tcg") - self.launch_and_wait(set_up_ssh_connection=False) - - def test_pseries_kvm(self): - """ - :avocado: tags=machine:pseries - :avocado: tags=accel:kvm - """ - self.require_accelerator("kvm") - self.vm.add_args("-accel", "kvm") - self.vm.add_args("-machine", "cap-ccf-assist=off") - self.launch_and_wait(set_up_ssh_connection=False) - -class BootLinuxS390X(LinuxTest): - """ - :avocado: tags=arch:s390x - """ - - timeout = 240 - - @skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited') - def test_s390_ccw_virtio_tcg(self): - """ - :avocado: tags=machine:s390-ccw-virtio - :avocado: tags=accel:tcg - """ - self.require_accelerator("tcg") - self.vm.add_args("-accel", "tcg") - self.launch_and_wait(set_up_ssh_connection=False) From patchwork Mon Apr 14 11:30:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050224 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D8DDEC369B2 for ; Mon, 14 Apr 2025 11:32:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I27-0003ZZ-1z; Mon, 14 Apr 2025 07:31:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1j-0003Vd-Qj for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:31:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1h-0003vk-Q5 for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:30:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630257; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BhfTQ5TT4fi6f3OHTYsSSPVnpSGAUF6fIZjJer489Io=; b=JjY6hXxwXn+g47FKLA+/EEpB7KeYJVHFRSwaVGFR6j+tcK00f9Rc0/ufOUCYcSWxHiMbdK +236QrDrZHOPKc20DaN9UWHX+T5VsWHakmFeBr1iRkmA1seFI3Lt5VfDpgldXKoksgavdh Oliwj7/Qb+c/apKoXVKQ7Zm2Jf7gdxA= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-168-0-8MpzboO0-ork51C6-Oqw-1; Mon, 14 Apr 2025 07:30:53 -0400 X-MC-Unique: 0-8MpzboO0-ork51C6-Oqw-1 X-Mimecast-MFC-AGG-ID: 0-8MpzboO0-ork51C6-Oqw_1744630252 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 99B8919560B0; Mon, 14 Apr 2025 11:30:52 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7CC7D1956094; Mon, 14 Apr 2025 11:30:51 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 11/15] tests/functional: Use the tuxrun kernel for the x86 replay test Date: Mon, 14 Apr 2025 13:30:25 +0200 Message-ID: <20250414113031.151105-12-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from tests/avocado/replay_linux.py. Thus remove the x86 avocado replay_linux test now. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/avocado/replay_linux.py | 46 -------------------------- tests/functional/test_x86_64_replay.py | 43 ++++++++++++++++++------ 2 files changed, 33 insertions(+), 56 deletions(-) diff --git a/tests/avocado/replay_linux.py b/tests/avocado/replay_linux.py index 59169224353..6ba283d3bf6 100644 --- a/tests/avocado/replay_linux.py +++ b/tests/avocado/replay_linux.py @@ -118,52 +118,6 @@ def run_replay_dump(self, replay_path): except subprocess.CalledProcessError: self.fail('replay-dump.py failed') -@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout') -class ReplayLinuxX8664(ReplayLinux): - """ - :avocado: tags=arch:x86_64 - :avocado: tags=accel:tcg - """ - - chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0' - - def test_pc_i440fx(self): - """ - :avocado: tags=machine:pc - """ - self.run_rr(shift=1) - - def test_pc_q35(self): - """ - :avocado: tags=machine:q35 - """ - self.run_rr(shift=3) - -@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout') -class ReplayLinuxX8664Virtio(ReplayLinux): - """ - :avocado: tags=arch:x86_64 - :avocado: tags=virtio - :avocado: tags=accel:tcg - """ - - hdd = 'virtio-blk-pci' - cd = 'virtio-blk-pci' - bus = None - - chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0' - - def test_pc_i440fx(self): - """ - :avocado: tags=machine:pc - """ - self.run_rr(shift=1) - - def test_pc_q35(self): - """ - :avocado: tags=machine:q35 - """ - self.run_rr(shift=3) @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout') class ReplayLinuxAarch64(ReplayLinux): diff --git a/tests/functional/test_x86_64_replay.py b/tests/functional/test_x86_64_replay.py index 180f23a60c5..27287d452dc 100755 --- a/tests/functional/test_x86_64_replay.py +++ b/tests/functional/test_x86_64_replay.py @@ -5,30 +5,53 @@ # # SPDX-License-Identifier: GPL-2.0-or-later -from qemu_test import Asset, skipFlakyTest +from subprocess import check_call, DEVNULL + +from qemu_test import Asset, skipFlakyTest, get_qemu_img from replay_kernel import ReplayKernelBase class X86Replay(ReplayKernelBase): ASSET_KERNEL = Asset( - ('https://archives.fedoraproject.org/pub/archive/fedora/linux' - '/releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz'), - '8f237d84712b1b411baf3af2aeaaee10b9aae8e345ec265b87ab3a39639eb143') + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/bzImage', + 'f57bfc6553bcd6e0a54aab86095bf642b33b5571d14e3af1731b18c87ed5aef8') + + ASSET_ROOTFS = Asset( + 'https://storage.tuxboot.com/buildroot/20241119/x86_64/rootfs.ext4.zst', + '4b8b2a99117519c5290e1202cb36eb6c7aaba92b357b5160f5970cf5fb78a751') - def do_test_x86(self, machine): + def do_test_x86(self, machine, blkdevice, devroot): + self.require_netdev('user') self.set_machine(machine) + self.cpu="Nehalem" kernel_path = self.ASSET_KERNEL.fetch() - kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0' - console_pattern = 'VFS: Cannot open root device' - self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5) + + raw_disk = self.uncompress(self.ASSET_ROOTFS) + disk = self.scratch_file('scratch.qcow2') + qemu_img = get_qemu_img(self) + check_call([qemu_img, 'create', '-f', 'qcow2', '-b', raw_disk, + '-F', 'raw', disk], stdout=DEVNULL, stderr=DEVNULL) + + args = ('-drive', 'file=%s,snapshot=on,id=hd0,if=none' % disk, + '-drive', 'driver=blkreplay,id=hd0-rr,if=none,image=hd0', + '-device', '%s,drive=hd0-rr' % blkdevice, + '-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22', + '-device', 'virtio-net,netdev=vnet', + '-object', 'filter-replay,id=replay,netdev=vnet') + + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + f"console=ttyS0 root=/dev/{devroot}") + console_pattern = 'Welcome to TuxTest' + self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5, + args=args) @skipFlakyTest('https://gitlab.com/qemu-project/qemu/-/issues/2094') def test_pc(self): - self.do_test_x86('pc') + self.do_test_x86('pc', 'virtio-blk', 'vda') def test_q35(self): - self.do_test_x86('q35') + self.do_test_x86('q35', 'ide-hd', 'sda') if __name__ == '__main__': From patchwork Mon Apr 14 11:30:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050222 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 66415C369B5 for ; Mon, 14 Apr 2025 11:31:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I2F-0003m5-R5; Mon, 14 Apr 2025 07:31:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1l-0003Vr-M5 for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:31:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1j-0003wN-DT for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:31:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630258; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UhZFs1R3Ux+K0IjbRYck7691sJ3ftnwY5ZrXuGH66II=; b=fNt8XqxdpBbkUIVyLIeS++vLvRAlRPZzODKhMVKghDuEBvXGMg+v4dnlv2/qb/6rM8LXOB Q3J4oxlY/H8qJjo9MqWvz+Hgv5YKgXGOjw6eILyQQqMpxGhriGR+QoXh+7CS8nLTCe8c7g ENZGJJxpY33qjTwaJrlebCZqmZaN/f0= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-274-te3AeiVhPR6dlrBGV_47Cw-1; Mon, 14 Apr 2025 07:30:55 -0400 X-MC-Unique: te3AeiVhPR6dlrBGV_47Cw-1 X-Mimecast-MFC-AGG-ID: te3AeiVhPR6dlrBGV_47Cw_1744630254 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 51BB919560BC; Mon, 14 Apr 2025 11:30:54 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0D3501956094; Mon, 14 Apr 2025 11:30:52 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 12/15] tests/functional: Use the tuxrun kernel for the aarch64 replay test Date: Mon, 14 Apr 2025 13:30:26 +0200 Message-ID: <20250414113031.151105-13-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth This way we can do a full boot in record-replay mode and should get a similar test coverage compared to the old replay test from tests/avocado/replay_linux.py. Since the aarch64 test was the last avocado test in the tests/avocado/replay_linux.py file, we can remove this file now completely. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- MAINTAINERS | 1 - tests/avocado/replay_linux.py | 160 ------------------------ tests/functional/test_aarch64_replay.py | 37 ++++-- 3 files changed, 29 insertions(+), 169 deletions(-) delete mode 100644 tests/avocado/replay_linux.py diff --git a/MAINTAINERS b/MAINTAINERS index 72f6b208f5a..e29910cbd77 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3668,7 +3668,6 @@ F: include/system/replay.h F: docs/devel/replay.rst F: docs/system/replay.rst F: stubs/replay.c -F: tests/avocado/replay_linux.py F: tests/functional/*reverse_debug*.py F: tests/functional/*replay*.py F: qapi/replay.json diff --git a/tests/avocado/replay_linux.py b/tests/avocado/replay_linux.py deleted file mode 100644 index 6ba283d3bf6..00000000000 --- a/tests/avocado/replay_linux.py +++ /dev/null @@ -1,160 +0,0 @@ -# Record/replay test that boots a complete Linux system via a cloud image -# -# Copyright (c) 2020 ISP RAS -# -# Author: -# Pavel Dovgalyuk -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -import os -import logging -import time - -from avocado import skipUnless -from avocado_qemu import BUILD_DIR -from avocado.utils import cloudinit -from avocado.utils import network -from avocado.utils import vmimage -from avocado.utils import datadrainer -from avocado.utils.path import find_command -from avocado_qemu.linuxtest import LinuxTest - -class ReplayLinux(LinuxTest): - """ - Boots a Linux system, checking for a successful initialization - """ - - timeout = 1800 - chksum = None - hdd = 'ide-hd' - cd = 'ide-cd' - bus = 'ide' - - def setUp(self): - # LinuxTest does many replay-incompatible things, but includes - # useful methods. Do not setup LinuxTest here and just - # call some functions. - super(LinuxTest, self).setUp() - self._set_distro() - self.boot_path = self.download_boot() - self.phone_server = cloudinit.PhoneHomeServer(('0.0.0.0', 0), - self.name) - ssh_pubkey, self.ssh_key = self.set_up_existing_ssh_keys() - self.cloudinit_path = self.prepare_cloudinit(ssh_pubkey) - - def vm_add_disk(self, vm, path, id, device): - bus_string = '' - if self.bus: - bus_string = ',bus=%s.%d' % (self.bus, id,) - vm.add_args('-drive', 'file=%s,snapshot=on,id=disk%s,if=none' % (path, id)) - vm.add_args('-drive', - 'driver=blkreplay,id=disk%s-rr,if=none,image=disk%s' % (id, id)) - vm.add_args('-device', - '%s,drive=disk%s-rr%s' % (device, id, bus_string)) - - def vm_add_cdrom(self, vm, path, id, device): - vm.add_args('-drive', 'file=%s,id=disk%s,if=none,media=cdrom' % (path, id)) - - def launch_and_wait(self, record, args, shift): - self.require_netdev('user') - vm = self.get_vm() - vm.add_args('-smp', '1') - vm.add_args('-m', '1024') - vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22', - '-device', 'virtio-net,netdev=vnet') - vm.add_args('-object', 'filter-replay,id=replay,netdev=vnet') - if args: - vm.add_args(*args) - self.vm_add_disk(vm, self.boot_path, 0, self.hdd) - self.vm_add_cdrom(vm, self.cloudinit_path, 1, self.cd) - logger = logging.getLogger('replay') - if record: - logger.info('recording the execution...') - mode = 'record' - else: - logger.info('replaying the execution...') - mode = 'replay' - replay_path = os.path.join(self.workdir, 'replay.bin') - vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s' % - (shift, mode, replay_path)) - - start_time = time.time() - - vm.set_console() - vm.launch() - console_drainer = datadrainer.LineLogger(vm.console_socket.fileno(), - logger=self.log.getChild('console'), - stop_check=(lambda : not vm.is_running())) - console_drainer.start() - if record: - while not self.phone_server.instance_phoned_back: - self.phone_server.handle_request() - vm.shutdown() - logger.info('finished the recording with log size %s bytes' - % os.path.getsize(replay_path)) - self.run_replay_dump(replay_path) - logger.info('successfully tested replay-dump.py') - else: - vm.event_wait('SHUTDOWN', self.timeout) - vm.wait() - logger.info('successfully finished the replay') - elapsed = time.time() - start_time - logger.info('elapsed time %.2f sec' % elapsed) - return elapsed - - def run_rr(self, args=None, shift=7): - t1 = self.launch_and_wait(True, args, shift) - t2 = self.launch_and_wait(False, args, shift) - logger = logging.getLogger('replay') - logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1)) - - def run_replay_dump(self, replay_path): - try: - subprocess.check_call(["./scripts/replay-dump.py", - "-f", replay_path], - stdout=subprocess.DEVNULL) - except subprocess.CalledProcessError: - self.fail('replay-dump.py failed') - - -@skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout') -class ReplayLinuxAarch64(ReplayLinux): - """ - :avocado: tags=accel:tcg - :avocado: tags=arch:aarch64 - :avocado: tags=machine:virt - :avocado: tags=cpu:max - """ - - chksum = '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49' - - hdd = 'virtio-blk-device' - cd = 'virtio-blk-device' - bus = None - - def get_common_args(self): - return ('-bios', - os.path.join(BUILD_DIR, 'pc-bios', 'edk2-aarch64-code.fd'), - "-cpu", "max,lpa2=off", - '-device', 'virtio-rng-pci,rng=rng0', - '-object', 'rng-builtin,id=rng0') - - def test_virt_gicv2(self): - """ - :avocado: tags=machine:gic-version=2 - """ - - self.run_rr(shift=3, - args=(*self.get_common_args(), - "-machine", "virt,gic-version=2")) - - def test_virt_gicv3(self): - """ - :avocado: tags=machine:gic-version=3 - """ - - self.run_rr(shift=3, - args=(*self.get_common_args(), - "-machine", "virt,gic-version=3")) diff --git a/tests/functional/test_aarch64_replay.py b/tests/functional/test_aarch64_replay.py index 029fef3cbf8..8afe95668e2 100755 --- a/tests/functional/test_aarch64_replay.py +++ b/tests/functional/test_aarch64_replay.py @@ -5,27 +5,48 @@ # # SPDX-License-Identifier: GPL-2.0-or-later -from qemu_test import Asset, skipIfOperatingSystem +from subprocess import check_call, DEVNULL + +from qemu_test import Asset, skipIfOperatingSystem, get_qemu_img from replay_kernel import ReplayKernelBase class Aarch64Replay(ReplayKernelBase): ASSET_KERNEL = Asset( - ('https://archives.fedoraproject.org/pub/archive/fedora/linux/' - 'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'), - '7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7') + 'https://storage.tuxboot.com/buildroot/20241119/arm64/Image', + 'b74743c5e89e1cea0f73368d24ae0ae85c5204ff84be3b5e9610417417d2f235') + + ASSET_ROOTFS = Asset( + 'https://storage.tuxboot.com/buildroot/20241119/arm64/rootfs.ext4.zst', + 'a1acaaae2068df4648d04ff75f532aaa8c5edcd6b936122b6f0db4848a07b465') # Failing on Darwin: https://gitlab.com/qemu-project/qemu/-/issues/2907 @skipIfOperatingSystem('Darwin') def test_aarch64_virt(self): + self.require_netdev('user') self.set_machine('virt') - self.cpu = 'cortex-a53' + self.cpu = 'cortex-a57' kernel_path = self.ASSET_KERNEL.fetch() + + raw_disk = self.uncompress(self.ASSET_ROOTFS) + disk = self.scratch_file('scratch.qcow2') + qemu_img = get_qemu_img(self) + check_call([qemu_img, 'create', '-f', 'qcow2', '-b', raw_disk, + '-F', 'raw', disk], stdout=DEVNULL, stderr=DEVNULL) + + args = ('-drive', 'file=%s,snapshot=on,id=hd0,if=none' % disk, + '-drive', 'driver=blkreplay,id=hd0-rr,if=none,image=hd0', + '-device', 'virtio-blk-device,drive=hd0-rr', + '-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22', + '-device', 'virtio-net,netdev=vnet', + '-object', 'filter-replay,id=replay,netdev=vnet') + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + - 'console=ttyAMA0') - console_pattern = 'VFS: Cannot open root device' - self.run_rr(kernel_path, kernel_command_line, console_pattern) + 'console=ttyAMA0 root=/dev/vda') + console_pattern = 'Welcome to TuxTest' + self.run_rr(kernel_path, kernel_command_line, console_pattern, + args=args) if __name__ == '__main__': From patchwork Mon Apr 14 11:30:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0C9A9C369B2 for ; Mon, 14 Apr 2025 11:32:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I2F-0003lx-FD; Mon, 14 Apr 2025 07:31:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1o-0003WG-JD for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:31:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1l-0003wp-Ku for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:31:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630261; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qE5+1agrnfvvNBEzWyTUjZfk1vmSCnVio/NUUteHxOQ=; b=C1ST4CW9HcLrV+ej0E08hLmJhMeU6Hd2fLMYK7j1Z49pwKTfdtZsyUEJwhJH0MIHPMmrPr N6GhkVAhDVsrkn5t2KA9ZJRu2iAIZ+FmKWplD3rFQdkA9SVjOtFFOuvJQ24veEamojyQJS utiksIJ/+MXqLbtmvNj2PCBi46ZfvEY= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-220-echUdEWRO5Wrh7ykpYqYKw-1; Mon, 14 Apr 2025 07:30:56 -0400 X-MC-Unique: echUdEWRO5Wrh7ykpYqYKw-1 X-Mimecast-MFC-AGG-ID: echUdEWRO5Wrh7ykpYqYKw_1744630256 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EBD9F180025C; Mon, 14 Apr 2025 11:30:55 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 990151956094; Mon, 14 Apr 2025 11:30:54 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 13/15] tests/functional: Convert the SMMU test to the functional framework Date: Mon, 14 Apr 2025 13:30:27 +0200 Message-ID: <20250414113031.151105-14-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth This test was using cloudinit and a "dnf install" command in the guest to exercise the NIC with SMMU enabled. Since we don't have the cloudinit stuff in the functional framework and we should not rely on having access to external networks (once our ASSETs have been cached), we rather boot into the initrd first, manually mount the root disk and then use the check_http_download() function from the functional framework here instead for testing whether the network works as expected. Unfortunately, there seems to be a small race when using the files from Fedora 33: To enter the initrd shell, we have to send a "return" once. But it does not seem to work if we send it too early. Using a sleep(0.2) makes it work reliably for me, but to make it even more unlikely to trigger this situation, let's better limit the Fedora 33 tests to only run with KVM. Finally, while we're at it, we also add some lines for testing writes to the hard disk, as we already do it in the test_intel_iommu test. Reviewed-by: Eric Auger Tested-by: Eric Auger Signed-off-by: Thomas Huth --- MAINTAINERS | 2 +- tests/avocado/smmu.py | 139 ----------------- tests/functional/meson.build | 2 + tests/functional/test_aarch64_smmu.py | 205 ++++++++++++++++++++++++++ 4 files changed, 208 insertions(+), 140 deletions(-) delete mode 100644 tests/avocado/smmu.py create mode 100755 tests/functional/test_aarch64_smmu.py diff --git a/MAINTAINERS b/MAINTAINERS index e29910cbd77..07f77c048e2 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -211,7 +211,7 @@ L: qemu-arm@nongnu.org S: Maintained F: hw/arm/smmu* F: include/hw/arm/smmu* -F: tests/avocado/smmu.py +F: tests/functional/test_aarch64_smmu.py AVR TCG CPUs M: Michael Rolnik diff --git a/tests/avocado/smmu.py b/tests/avocado/smmu.py deleted file mode 100644 index 83fd79e922e..00000000000 --- a/tests/avocado/smmu.py +++ /dev/null @@ -1,139 +0,0 @@ -# SMMUv3 Functional tests -# -# Copyright (c) 2021 Red Hat, Inc. -# -# Author: -# Eric Auger -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. -import os - -from avocado import skipUnless -from avocado_qemu import BUILD_DIR -from avocado_qemu.linuxtest import LinuxTest - -@skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab') -class SMMU(LinuxTest): - """ - :avocado: tags=accel:kvm - :avocado: tags=cpu:host - :avocado: tags=arch:aarch64 - :avocado: tags=machine:virt - :avocado: tags=distro:fedora - :avocado: tags=smmu - :avocado: tags=flaky - """ - - IOMMU_ADDON = ',iommu_platform=on,disable-modern=off,disable-legacy=on' - kernel_path = None - initrd_path = None - kernel_params = None - - def set_up_boot(self): - path = self.download_boot() - self.vm.add_args('-device', 'virtio-blk-pci,bus=pcie.0,' + - 'drive=drv0,id=virtio-disk0,bootindex=1,' - 'werror=stop,rerror=stop' + self.IOMMU_ADDON) - self.vm.add_args('-drive', - 'file=%s,if=none,cache=writethrough,id=drv0' % path) - - def setUp(self): - super(SMMU, self).setUp(None, 'virtio-net-pci' + self.IOMMU_ADDON) - - def common_vm_setup(self, custom_kernel=False): - self.require_accelerator("kvm") - self.vm.add_args("-accel", "kvm") - self.vm.add_args("-cpu", "host") - self.vm.add_args("-machine", "iommu=smmuv3") - self.vm.add_args("-d", "guest_errors") - self.vm.add_args('-bios', os.path.join(BUILD_DIR, 'pc-bios', - 'edk2-aarch64-code.fd')) - self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0') - self.vm.add_args('-object', - 'rng-random,id=rng0,filename=/dev/urandom') - - if custom_kernel is False: - return - - kernel_url = self.distro.pxeboot_url + 'vmlinuz' - initrd_url = self.distro.pxeboot_url + 'initrd.img' - self.kernel_path = self.fetch_asset(kernel_url) - self.initrd_path = self.fetch_asset(initrd_url) - - def run_and_check(self): - if self.kernel_path: - self.vm.add_args('-kernel', self.kernel_path, - '-append', self.kernel_params, - '-initrd', self.initrd_path) - self.launch_and_wait() - self.ssh_command('cat /proc/cmdline') - self.ssh_command('dnf -y install numactl-devel') - - - # 5.3 kernel without RIL # - - def test_smmu_noril(self): - """ - :avocado: tags=smmu_noril - :avocado: tags=smmu_noril_tests - :avocado: tags=distro_version:31 - """ - self.common_vm_setup() - self.run_and_check() - - def test_smmu_noril_passthrough(self): - """ - :avocado: tags=smmu_noril_passthrough - :avocado: tags=smmu_noril_tests - :avocado: tags=distro_version:31 - """ - self.common_vm_setup(True) - self.kernel_params = (self.distro.default_kernel_params + - ' iommu.passthrough=on') - self.run_and_check() - - def test_smmu_noril_nostrict(self): - """ - :avocado: tags=smmu_noril_nostrict - :avocado: tags=smmu_noril_tests - :avocado: tags=distro_version:31 - """ - self.common_vm_setup(True) - self.kernel_params = (self.distro.default_kernel_params + - ' iommu.strict=0') - self.run_and_check() - - # 5.8 kernel featuring range invalidation - # >= v5.7 kernel - - def test_smmu_ril(self): - """ - :avocado: tags=smmu_ril - :avocado: tags=smmu_ril_tests - :avocado: tags=distro_version:33 - """ - self.common_vm_setup() - self.run_and_check() - - def test_smmu_ril_passthrough(self): - """ - :avocado: tags=smmu_ril_passthrough - :avocado: tags=smmu_ril_tests - :avocado: tags=distro_version:33 - """ - self.common_vm_setup(True) - self.kernel_params = (self.distro.default_kernel_params + - ' iommu.passthrough=on') - self.run_and_check() - - def test_smmu_ril_nostrict(self): - """ - :avocado: tags=smmu_ril_nostrict - :avocado: tags=smmu_ril_tests - :avocado: tags=distro_version:33 - """ - self.common_vm_setup(True) - self.kernel_params = (self.distro.default_kernel_params + - ' iommu.strict=0') - self.run_and_check() diff --git a/tests/functional/meson.build b/tests/functional/meson.build index 985ac5c27fe..b317ad42c5a 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -18,6 +18,7 @@ test_timeouts = { 'aarch64_rme_sbsaref' : 1200, 'aarch64_sbsaref_alpine' : 1200, 'aarch64_sbsaref_freebsd' : 720, + 'aarch64_smmu' : 720, 'aarch64_tuxrun' : 240, 'aarch64_virt' : 360, 'aarch64_virt_gpu' : 480, @@ -88,6 +89,7 @@ tests_aarch64_system_thorough = [ 'aarch64_sbsaref', 'aarch64_sbsaref_alpine', 'aarch64_sbsaref_freebsd', + 'aarch64_smmu', 'aarch64_tcg_plugins', 'aarch64_tuxrun', 'aarch64_virt', diff --git a/tests/functional/test_aarch64_smmu.py b/tests/functional/test_aarch64_smmu.py new file mode 100755 index 00000000000..c65d0f28178 --- /dev/null +++ b/tests/functional/test_aarch64_smmu.py @@ -0,0 +1,205 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: GPL-2.0-or-later +# +# SMMUv3 Functional tests +# +# Copyright (c) 2021 Red Hat, Inc. +# +# Author: +# Eric Auger +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import os +import time + +from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pattern +from qemu_test import BUILD_DIR +from qemu.utils import kvm_available + + +class SMMU(LinuxKernelTest): + + default_kernel_params = ('earlyprintk=pl011,0x9000000 no_timer_check ' + 'printk.time=1 rd_NO_PLYMOUTH net.ifnames=0 ' + 'console=ttyAMA0 rd.rescue') + IOMMU_ADDON = ',iommu_platform=on,disable-modern=off,disable-legacy=on' + kernel_path = None + initrd_path = None + kernel_params = None + + GUEST_PORT = 8080 + + def set_up_boot(self, path): + self.vm.add_args('-device', 'virtio-blk-pci,bus=pcie.0,' + + 'drive=drv0,id=virtio-disk0,bootindex=1,' + 'werror=stop,rerror=stop' + self.IOMMU_ADDON) + self.vm.add_args('-drive', + f'file={path},if=none,cache=writethrough,id=drv0,snapshot=on') + + self.vm.add_args('-netdev', + 'user,id=n1,hostfwd=tcp:127.0.0.1:0-:%d' % + self.GUEST_PORT) + self.vm.add_args('-device', 'virtio-net,netdev=n1' + self.IOMMU_ADDON) + + def common_vm_setup(self, kernel, initrd, disk): + self.require_accelerator("kvm") + self.require_netdev('user') + self.set_machine("virt") + self.vm.add_args('-m', '1G') + self.vm.add_args("-accel", "kvm") + self.vm.add_args("-cpu", "host") + self.vm.add_args("-machine", "iommu=smmuv3") + self.vm.add_args("-d", "guest_errors") + self.vm.add_args('-bios', os.path.join(BUILD_DIR, 'pc-bios', + 'edk2-aarch64-code.fd')) + self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0') + self.vm.add_args('-object', + 'rng-random,id=rng0,filename=/dev/urandom') + + self.kernel_path = kernel.fetch() + self.initrd_path = initrd.fetch() + self.set_up_boot(disk.fetch()) + + def run_and_check(self, filename, hashsum): + self.vm.add_args('-initrd', self.initrd_path) + self.vm.add_args('-append', self.kernel_params) + self.launch_kernel(self.kernel_path, initrd=self.initrd_path, + wait_for='attach it to a bug report.') + prompt = '# ' + # Fedora 33 requires 'return' to be pressed to enter the shell. + # There seems to be a small race between detecting the previous ':' + # and sending the newline, so we need to add a small delay here. + self.wait_for_console_pattern(':') + time.sleep(0.2) + exec_command_and_wait_for_pattern(self, '\n', prompt) + exec_command_and_wait_for_pattern(self, 'cat /proc/cmdline', + self.kernel_params) + + # Checking for SMMU enablement: + self.log.info("Checking whether SMMU has been enabled...") + exec_command_and_wait_for_pattern(self, 'dmesg | grep smmu', + 'arm-smmu-v3') + self.wait_for_console_pattern(prompt) + exec_command_and_wait_for_pattern(self, + 'find /sys/kernel/iommu_groups/ -type l', + 'devices/0000:00:') + self.wait_for_console_pattern(prompt) + + # Copy a file (checked later), umount afterwards to drop disk cache: + self.log.info("Checking hard disk...") + exec_command_and_wait_for_pattern(self, + "while ! (dmesg -c | grep vda:) ; do sleep 1 ; done", + "vda2") + exec_command_and_wait_for_pattern(self, 'mount /dev/vda2 /sysroot', + 'mounted filesystem') + exec_command_and_wait_for_pattern(self, 'cp /bin/vi /sysroot/root/vi', + prompt) + exec_command_and_wait_for_pattern(self, 'umount /sysroot', prompt) + # Switch from initrd to the cloud image filesystem: + exec_command_and_wait_for_pattern(self, 'mount /dev/vda2 /sysroot', + prompt) + exec_command_and_wait_for_pattern(self, + ('for d in dev proc sys run ; do ' + 'mount -o bind /$d /sysroot/$d ; done'), prompt) + exec_command_and_wait_for_pattern(self, 'chroot /sysroot', prompt) + # Check files on the hard disk: + exec_command_and_wait_for_pattern(self, + ('if diff -q /root/vi /usr/bin/vi ; then echo "file" "ok" ; ' + 'else echo "files differ"; fi'), 'file ok') + self.wait_for_console_pattern(prompt) + exec_command_and_wait_for_pattern(self, f'sha256sum {filename}', + hashsum) + + # Check virtio-net via HTTP: + exec_command_and_wait_for_pattern(self, 'dhclient eth0', prompt) + self.check_http_download(filename, hashsum, self.GUEST_PORT) + + + # 5.3 kernel without RIL # + + ASSET_KERNEL_F31 = Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/' + 'releases/31/Server/aarch64/os/images/pxeboot/vmlinuz'), + '3ae07fcafbfc8e4abeb693035a74fe10698faae15e9ccd48882a9167800c1527') + + ASSET_INITRD_F31 = Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/' + 'releases/31/Server/aarch64/os/images/pxeboot/initrd.img'), + '9f3146b28bc531c689f3c5f114cb74e4bd7bd548e0ba19fa77921d8bd256755a') + + ASSET_DISK_F31 = Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/releases' + '/31/Cloud/aarch64/images/Fedora-Cloud-Base-31-1.9.aarch64.qcow2'), + '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49') + + F31_FILENAME = '/boot/initramfs-5.3.7-301.fc31.aarch64.img' + F31_HSUM = '1a4beec6607d94df73d9dd1b4985c9c23dd0fdcf4e6ca1351d477f190df7bef9' + + def test_smmu_noril(self): + self.common_vm_setup(self.ASSET_KERNEL_F31, self.ASSET_INITRD_F31, + self.ASSET_DISK_F31) + self.kernel_params = self.default_kernel_params + self.run_and_check(self.F31_FILENAME, self.F31_HSUM) + + def test_smmu_noril_passthrough(self): + self.common_vm_setup(self.ASSET_KERNEL_F31, self.ASSET_INITRD_F31, + self.ASSET_DISK_F31) + self.kernel_params = (self.default_kernel_params + + ' iommu.passthrough=on') + self.run_and_check(self.F31_FILENAME, self.F31_HSUM) + + def test_smmu_noril_nostrict(self): + self.common_vm_setup(self.ASSET_KERNEL_F31, self.ASSET_INITRD_F31, + self.ASSET_DISK_F31) + self.kernel_params = (self.default_kernel_params + + ' iommu.strict=0') + self.run_and_check(self.F31_FILENAME, self.F31_HSUM) + + + # 5.8 kernel featuring range invalidation + # >= v5.7 kernel + + ASSET_KERNEL_F33 = Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/' + 'releases/33/Server/aarch64/os/images/pxeboot/vmlinuz'), + 'd8b1e6f7241f339d8e7609c456cf0461ffa4583ed07e0b55c7d1d8a0c154aa89') + + ASSET_INITRD_F33 = Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/' + 'releases/33/Server/aarch64/os/images/pxeboot/initrd.img'), + '92513f55295c2c16a777f7b6c35ccd70a438e9e1e40b6ba39e0e60900615b3df') + + ASSET_DISK_F33 = Asset( + ('https://archives.fedoraproject.org/pub/archive/fedora/linux/releases' + '/33/Cloud/aarch64/images/Fedora-Cloud-Base-33-1.2.aarch64.qcow2'), + 'e7f75cdfd523fe5ac2ca9eeece68edc1a81f386a17f969c1d1c7c87031008a6b') + + F33_FILENAME = '/boot/initramfs-5.8.15-301.fc33.aarch64.img' + F33_HSUM = '079cfad0caa82e84c8ca1fb0897a4999dd769f262216099f518619e807a550d9' + + def test_smmu_ril(self): + self.common_vm_setup(self.ASSET_KERNEL_F33, self.ASSET_INITRD_F33, + self.ASSET_DISK_F33) + self.kernel_params = self.default_kernel_params + self.run_and_check(self.F33_FILENAME, self.F33_HSUM) + + def test_smmu_ril_passthrough(self): + self.common_vm_setup(self.ASSET_KERNEL_F33, self.ASSET_INITRD_F33, + self.ASSET_DISK_F33) + self.kernel_params = (self.default_kernel_params + + ' iommu.passthrough=on') + self.run_and_check(self.F33_FILENAME, self.F33_HSUM) + + def test_smmu_ril_nostrict(self): + self.common_vm_setup(self.ASSET_KERNEL_F33, self.ASSET_INITRD_F33, + self.ASSET_DISK_F33) + self.kernel_params = (self.default_kernel_params + + ' iommu.strict=0') + self.run_and_check(self.F33_FILENAME, self.F33_HSUM) + + +if __name__ == '__main__': + LinuxKernelTest.main() From patchwork Mon Apr 14 11:30:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050232 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70FFFC369B2 for ; Mon, 14 Apr 2025 11:37:10 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I2A-0003Zg-8o; Mon, 14 Apr 2025 07:31:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1n-0003WC-I3 for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:31:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I1l-0003wl-MW for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:31:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630260; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ehssBvTBsCznQI3l0ZFsD4u7xuCreomGBJBuvFctdBo=; b=Y8YCethKJkAA5elaxK0PEYm5WMQQ9BdXr/welEzQtn8SAZrOXlv6KKhBI5FMTNUS6UOw2z 0T46qQn0AoaDfUZVajU06fMbWkR5wzj7XqlJxNfvOXLQ7E4wxcOkTWNpvJpoZGSlvRyMDM YaVQiBo2R2ZFw0Py97YoDWJOQYxBUjA= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-601-htv33V5LNquKHBFqzMnZCA-1; Mon, 14 Apr 2025 07:30:58 -0400 X-MC-Unique: htv33V5LNquKHBFqzMnZCA-1 X-Mimecast-MFC-AGG-ID: htv33V5LNquKHBFqzMnZCA_1744630257 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 78FE4180035C; Mon, 14 Apr 2025 11:30:57 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5C8B71956094; Mon, 14 Apr 2025 11:30:56 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 14/15] gitlab-ci: Update QEMU_JOB_AVOCADO and QEMU_CI_AVOCADO_TESTING Date: Mon, 14 Apr 2025 13:30:28 +0200 Message-ID: <20250414113031.151105-15-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth Since we don't run the Avocado jobs in the CI anymore, rename these variables to QEMU_JOB_FUNCTIONAL and QEMU_CI_FUNCTIONAL. Also, there was a mismatch between the documentation and the implementation of QEMU_CI_AVOCADO_TESTING: While the documentation said that you had to "Set this variable to have the tests using the Avocado framework run automatically", you indeed needed to set it to make the pipelines appear in your dashboard - but they were never run automatically in forks and had to be triggered manually. Let's improve this now: No need to hide these pipelines from the users by default anymore (the functional tests should be stable enough nowadays), and rather allow the users to run the pipelines auto- matically with this switch now instead, as was documented. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- docs/devel/testing/ci-jobs.rst.inc | 19 +++++++++---------- .gitlab-ci.d/base.yml | 8 ++------ .gitlab-ci.d/buildtest-template.yml | 2 +- 3 files changed, 12 insertions(+), 17 deletions(-) diff --git a/docs/devel/testing/ci-jobs.rst.inc b/docs/devel/testing/ci-jobs.rst.inc index 3756bbe3554..f1c541cc25b 100644 --- a/docs/devel/testing/ci-jobs.rst.inc +++ b/docs/devel/testing/ci-jobs.rst.inc @@ -126,10 +126,10 @@ QEMU_JOB_PUBLISH The job is for publishing content after a branch has been merged into the upstream default branch. -QEMU_JOB_AVOCADO -~~~~~~~~~~~~~~~~ +QEMU_JOB_FUNCTIONAL +~~~~~~~~~~~~~~~~~~~ -The job runs the Avocado integration test suite +The job runs the functional test suite Contributor controlled runtime variables ---------------------------------------- @@ -149,13 +149,12 @@ the jobs to be manually started from the UI Set this variable to 2 to create the pipelines and run all the jobs immediately, as was the historical behaviour -QEMU_CI_AVOCADO_TESTING -~~~~~~~~~~~~~~~~~~~~~~~ -By default, tests using the Avocado framework are not run automatically in -the pipelines (because multiple artifacts have to be downloaded, and if -these artifacts are not already cached, downloading them make the jobs -reach the timeout limit). Set this variable to have the tests using the -Avocado framework run automatically. +QEMU_CI_FUNCTIONAL +~~~~~~~~~~~~~~~~~~ +By default, tests using the functional framework are not run automatically +in the pipelines (because multiple artifacts have to be downloaded, which +might cause a lot of network traffic). Set this variable to have the tests +using the functional framework run automatically. Other misc variables -------------------- diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml index 25b88aaa06a..60a24a9d14d 100644 --- a/.gitlab-ci.d/base.yml +++ b/.gitlab-ci.d/base.yml @@ -69,10 +69,6 @@ variables: - if: '$QEMU_CI != "1" && $QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM' when: never - # Avocado jobs don't run in forks unless $QEMU_CI_AVOCADO_TESTING is set - - if: '$QEMU_JOB_AVOCADO && $QEMU_CI_AVOCADO_TESTING != "1" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM' - when: never - ############################################################# # Stage 2: fine tune execution of jobs in specific scenarios @@ -101,8 +97,8 @@ variables: when: manual allow_failure: true - # Avocado jobs can be manually start in forks if $QEMU_CI_AVOCADO_TESTING is unset - - if: '$QEMU_JOB_AVOCADO && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM' + # Functional jobs can be manually started in forks + - if: '$QEMU_JOB_FUNCTIONAL && $QEMU_CI_FUNCTIONAL != "1" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM' when: manual allow_failure: true diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml index 13fa4f4a4fc..d4f145fdb58 100644 --- a/.gitlab-ci.d/buildtest-template.yml +++ b/.gitlab-ci.d/buildtest-template.yml @@ -114,4 +114,4 @@ - cd build - du -chs ${CI_PROJECT_DIR}/*-cache variables: - QEMU_JOB_AVOCADO: 1 + QEMU_JOB_FUNCTIONAL: 1 From patchwork Mon Apr 14 11:30:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 14050233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 72CCBC369B5 for ; Mon, 14 Apr 2025 11:37:10 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u4I4f-0007Xr-EJ; Mon, 14 Apr 2025 07:34:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I3p-000749-Vt for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:33:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u4I3k-00047J-5c for qemu-devel@nongnu.org; Mon, 14 Apr 2025 07:33:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744630381; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UZ7eraPc4LRNcJ4W6UJXFHjHAAWowDgTwG9GAb4Ng8Q=; b=PWSRYLpVeeMxi3WKob3yNPJMWe3rCTvvCTebn4w4iErawlblXmEahiiYNgyzYfGAemwbID VuWkZlMFKtjggGlCBV+iiJW2PXnFPUf+RE12blhi15Rs2CY3KoawV6B+T92ktYdnsfsPNz 5u3+P8RfdblWw4FkJpxZkLNUuKE2Wic= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-608-ESEsLy9ROcO1sstMbZ3Bfg-1; Mon, 14 Apr 2025 07:31:01 -0400 X-MC-Unique: ESEsLy9ROcO1sstMbZ3Bfg-1 X-Mimecast-MFC-AGG-ID: ESEsLy9ROcO1sstMbZ3Bfg_1744630259 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4379E19560B3; Mon, 14 Apr 2025 11:30:59 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-219.str.redhat.com [10.33.192.219]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DFB421956094; Mon, 14 Apr 2025 11:30:57 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Subject: [PATCH v2 15/15] Remove the remainders of the Avocado tests Date: Mon, 14 Apr 2025 13:30:29 +0200 Message-ID: <20250414113031.151105-16-thuth@redhat.com> In-Reply-To: <20250414113031.151105-1-thuth@redhat.com> References: <20250414113031.151105-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.133.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, WEIRD_QUOTING=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth Now that all Avocado tests have been converted to or been replaced by other functional tests, we can delete the remainders of the Avocado tests from the QEMU source tree. Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- MAINTAINERS | 8 +- docs/about/build-platforms.rst | 10 +- docs/devel/build-system.rst | 11 +- docs/devel/codebase.rst | 5 - docs/devel/testing/avocado.rst | 581 ------------------------ docs/devel/testing/functional.rst | 3 - docs/devel/testing/index.rst | 1 - docs/devel/testing/main.rst | 15 - tests/avocado/README.rst | 10 - configure | 2 +- pythondeps.toml | 8 +- tests/Makefile.include | 60 +-- tests/avocado/avocado_qemu/__init__.py | 424 ----------------- tests/avocado/avocado_qemu/linuxtest.py | 253 ----------- 14 files changed, 16 insertions(+), 1375 deletions(-) delete mode 100644 docs/devel/testing/avocado.rst delete mode 100644 tests/avocado/README.rst delete mode 100644 tests/avocado/avocado_qemu/__init__.py delete mode 100644 tests/avocado/avocado_qemu/linuxtest.py diff --git a/MAINTAINERS b/MAINTAINERS index 07f77c048e2..5fd757c5ddb 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2073,7 +2073,7 @@ S: Supported F: hw/acpi/viot.c F: hw/acpi/viot.h -ACPI/AVOCADO/BIOSBITS +ACPI/FUNCTIONAL/BIOSBITS M: Ani Sinha M: Michael S. Tsirkin S: Supported @@ -4246,12 +4246,6 @@ R: Philippe Mathieu-Daudé S: Maintained F: tests/tcg/Makefile.target -Integration Testing with the Avocado framework -W: https://trello.com/b/6Qi1pxVn/avocado-qemu -R: Cleber Rosa -S: Odd Fixes -F: tests/avocado/ - GitLab custom runner (Works On Arm Sponsored) M: Alex Bennée M: Philippe Mathieu-Daudé diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst index 1552b1a7044..52521552c8a 100644 --- a/docs/about/build-platforms.rst +++ b/docs/about/build-platforms.rst @@ -123,11 +123,11 @@ Rust build dependencies to build QEMU. Optional build dependencies - Build components whose absence does not affect the ability to build - QEMU may not be available in distros, or may be too old for QEMU's - requirements. Many of these, such as the Avocado testing framework - or various linters, are written in Python and therefore can also - be installed using ``pip``. Cross compilers are another example + Build components whose absence does not affect the ability to build QEMU + may not be available in distros, or may be too old for our requirements. + Many of these, such as additional modules for the functional testing + framework or various linters, are written in Python and therefore can + also be installed using ``pip``. Cross compilers are another example of optional build-time dependency; in this case it is possible to download them from repositories such as EPEL, to use container-based cross compilation using ``docker`` or ``podman``, or to use pre-built diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst index a759982f45c..258cfad3fe7 100644 --- a/docs/devel/build-system.rst +++ b/docs/devel/build-system.rst @@ -134,7 +134,7 @@ in how the build process runs Python code. At this stage, ``configure`` also queries the chosen Python interpreter about QEMU's build dependencies. Note that the build process does *not* -look for ``meson``, ``sphinx-build`` or ``avocado`` binaries in the PATH; +look for ``meson`` or ``sphinx-build`` binaries in the PATH; likewise, there are no options such as ``--meson`` or ``--sphinx-build``. This avoids a potential mismatch, where Meson and Sphinx binaries on the PATH might operate in a different Python environment than the one chosen @@ -151,7 +151,7 @@ virtual environment with ``pip``, either from wheels in ``python/wheels`` or by downloading the package with PyPI. Downloading can be disabled with ``--disable-download``; and anyway, it only happens when a ``configure`` option (currently, only ``--enable-docs``) is explicitly enabled but -the dependencies are not present\ [#pip]_. +the dependencies are not present. .. [#distlib] The scripts are created based on the package's metadata, specifically the ``console_script`` entry points. This is the @@ -164,10 +164,6 @@ the dependencies are not present\ [#pip]_. because the Python Packaging Authority provides a package ``distlib.scripts`` to perform this task. -.. [#pip] ``pip`` might also be used when running ``make check-avocado`` - if downloading is enabled, to ensure that Avocado is - available. - The required versions of the packages are stored in a configuration file ``pythondeps.toml``. The format is custom to QEMU, but it is documented at the top of the file itself and it should be easy to understand. The @@ -497,8 +493,7 @@ number of dynamically created files listed later. ``pyvenv/bin``, and calling ``pip`` to install dependencies. ``tests/Makefile.include`` - Rules for external test harnesses. These include the TCG tests - and the Avocado-based integration tests. + Rules for external test harnesses like the TCG tests. ``tests/docker/Makefile.include`` Rules for Docker tests. Like ``tests/Makefile.include``, this file is diff --git a/docs/devel/codebase.rst b/docs/devel/codebase.rst index 1b09953197b..1da17f40494 100644 --- a/docs/devel/codebase.rst +++ b/docs/devel/codebase.rst @@ -175,11 +175,6 @@ yet, so sometimes the source code is all you have. * `tests `_: QEMU `test ` suite - - `avocado `_: - Functional tests booting full VM using `Avocado framework `. - Those tests will be transformed and moved into - `tests/functional `_ - in the future. - `data `_: Data for various tests. - `decode `_: diff --git a/docs/devel/testing/avocado.rst b/docs/devel/testing/avocado.rst deleted file mode 100644 index eda76fe2dbd..00000000000 --- a/docs/devel/testing/avocado.rst +++ /dev/null @@ -1,581 +0,0 @@ -.. _checkavocado-ref: - - -Integration testing with Avocado -================================ - -The ``tests/avocado`` directory hosts integration tests. They're usually -higher level tests, and may interact with external resources and with -various guest operating systems. - -These tests are written using the Avocado Testing Framework (which must be -installed separately) in conjunction with a the ``avocado_qemu.QemuSystemTest`` -class, implemented at ``tests/avocado/avocado_qemu``. - -Tests based on ``avocado_qemu.QemuSystemTest`` can easily: - - * Customize the command line arguments given to the convenience - ``self.vm`` attribute (a QEMUMachine instance) - - * Interact with the QEMU monitor, send QMP commands and check - their results - - * Interact with the guest OS, using the convenience console device - (which may be useful to assert the effectiveness and correctness of - command line arguments or QMP commands) - - * Interact with external data files that accompany the test itself - (see ``self.get_data()``) - - * Download (and cache) remote data files, such as firmware and kernel - images - - * Have access to a library of guest OS images (by means of the - ``avocado.utils.vmimage`` library) - - * Make use of various other test related utilities available at the - test class itself and at the utility library: - - - http://avocado-framework.readthedocs.io/en/latest/api/test/avocado.html#avocado.Test - - http://avocado-framework.readthedocs.io/en/latest/api/utils/avocado.utils.html - -Running tests -------------- - -You can run the avocado tests simply by executing: - -.. code:: - - make check-avocado - -This involves the automatic installation, from PyPI, of all the -necessary avocado-framework dependencies into the QEMU venv within the -build tree (at ``./pyvenv``). Test results are also saved within the -build tree (at ``tests/results``). - -Note: the build environment must be using a Python 3 stack, and have -the ``venv`` and ``pip`` packages installed. If necessary, make sure -``configure`` is called with ``--python=`` and that those modules are -available. On Debian and Ubuntu based systems, depending on the -specific version, they may be on packages named ``python3-venv`` and -``python3-pip``. - -It is also possible to run tests based on tags using the -``make check-avocado`` command and the ``AVOCADO_TAGS`` environment -variable: - -.. code:: - - make check-avocado AVOCADO_TAGS=quick - -Note that tags separated with commas have an AND behavior, while tags -separated by spaces have an OR behavior. For more information on Avocado -tags, see: - - https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html - -To run a single test file, a couple of them, or a test within a file -using the ``make check-avocado`` command, set the ``AVOCADO_TESTS`` -environment variable with the test files or test names. To run all -tests from a single file, use: - - .. code:: - - make check-avocado AVOCADO_TESTS=$FILEPATH - -The same is valid to run tests from multiple test files: - - .. code:: - - make check-avocado AVOCADO_TESTS='$FILEPATH1 $FILEPATH2' - -To run a single test within a file, use: - - .. code:: - - make check-avocado AVOCADO_TESTS=$FILEPATH:$TESTCLASS.$TESTNAME - -The same is valid to run single tests from multiple test files: - - .. code:: - - make check-avocado AVOCADO_TESTS='$FILEPATH1:$TESTCLASS1.$TESTNAME1 $FILEPATH2:$TESTCLASS2.$TESTNAME2' - -The scripts installed inside the virtual environment may be used -without an "activation". For instance, the Avocado test runner -may be invoked by running: - - .. code:: - - pyvenv/bin/avocado run $OPTION1 $OPTION2 tests/avocado/ - -Note that if ``make check-avocado`` was not executed before, it is -possible to create the Python virtual environment with the dependencies -needed running: - - .. code:: - - make check-venv - -It is also possible to run tests from a single file or a single test within -a test file. To run tests from a single file within the build tree, use: - - .. code:: - - pyvenv/bin/avocado run tests/avocado/$TESTFILE - -To run a single test within a test file, use: - - .. code:: - - pyvenv/bin/avocado run tests/avocado/$TESTFILE:$TESTCLASS.$TESTNAME - -Valid test names are visible in the output from any previous execution -of Avocado or ``make check-avocado``, and can also be queried using: - - .. code:: - - pyvenv/bin/avocado list tests/avocado - -Manual Installation -------------------- - -To manually install Avocado and its dependencies, run: - -.. code:: - - pip install --user avocado-framework - -Alternatively, follow the instructions on this link: - - https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/installing.html - -Overview --------- - -The ``tests/avocado/avocado_qemu`` directory provides the -``avocado_qemu`` Python module, containing the ``avocado_qemu.QemuSystemTest`` -class. Here's a simple usage example: - -.. code:: - - from avocado_qemu import QemuSystemTest - - - class Version(QemuSystemTest): - """ - :avocado: tags=quick - """ - def test_qmp_human_info_version(self): - self.vm.launch() - res = self.vm.cmd('human-monitor-command', - command_line='info version') - self.assertRegex(res, r'^(\d+\.\d+\.\d)') - -To execute your test, run: - -.. code:: - - avocado run version.py - -Tests may be classified according to a convention by using docstring -directives such as ``:avocado: tags=TAG1,TAG2``. To run all tests -in the current directory, tagged as "quick", run: - -.. code:: - - avocado run -t quick . - -The ``avocado_qemu.QemuSystemTest`` base test class -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The ``avocado_qemu.QemuSystemTest`` class has a number of characteristics -that are worth being mentioned right away. - -First of all, it attempts to give each test a ready to use QEMUMachine -instance, available at ``self.vm``. Because many tests will tweak the -QEMU command line, launching the QEMUMachine (by using ``self.vm.launch()``) -is left to the test writer. - -The base test class has also support for tests with more than one -QEMUMachine. The way to get machines is through the ``self.get_vm()`` -method which will return a QEMUMachine instance. The ``self.get_vm()`` -method accepts arguments that will be passed to the QEMUMachine creation -and also an optional ``name`` attribute so you can identify a specific -machine and get it more than once through the tests methods. A simple -and hypothetical example follows: - -.. code:: - - from avocado_qemu import QemuSystemTest - - - class MultipleMachines(QemuSystemTest): - def test_multiple_machines(self): - first_machine = self.get_vm() - second_machine = self.get_vm() - self.get_vm(name='third_machine').launch() - - first_machine.launch() - second_machine.launch() - - first_res = first_machine.cmd( - 'human-monitor-command', - command_line='info version') - - second_res = second_machine.cmd( - 'human-monitor-command', - command_line='info version') - - third_res = self.get_vm(name='third_machine').cmd( - 'human-monitor-command', - command_line='info version') - - self.assertEqual(first_res, second_res, third_res) - -At test "tear down", ``avocado_qemu.QemuSystemTest`` handles all the -QEMUMachines shutdown. - -The ``avocado_qemu.LinuxTest`` base test class -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The ``avocado_qemu.LinuxTest`` is further specialization of the -``avocado_qemu.QemuSystemTest`` class, so it contains all the characteristics -of the later plus some extra features. - -First of all, this base class is intended for tests that need to -interact with a fully booted and operational Linux guest. At this -time, it uses a Fedora 31 guest image. The most basic example looks -like this: - -.. code:: - - from avocado_qemu import LinuxTest - - - class SomeTest(LinuxTest): - - def test(self): - self.launch_and_wait() - self.ssh_command('some_command_to_be_run_in_the_guest') - -Please refer to tests that use ``avocado_qemu.LinuxTest`` under -``tests/avocado`` for more examples. - -QEMUMachine ------------ - -The QEMUMachine API is already widely used in the Python iotests, -device-crash-test and other Python scripts. It's a wrapper around the -execution of a QEMU binary, giving its users: - - * the ability to set command line arguments to be given to the QEMU - binary - - * a ready to use QMP connection and interface, which can be used to - send commands and inspect its results, as well as asynchronous - events - - * convenience methods to set commonly used command line arguments in - a more succinct and intuitive way - -QEMU binary selection -^^^^^^^^^^^^^^^^^^^^^ - -The QEMU binary used for the ``self.vm`` QEMUMachine instance will -primarily depend on the value of the ``qemu_bin`` parameter. If it's -not explicitly set, its default value will be the result of a dynamic -probe in the same source tree. A suitable binary will be one that -targets the architecture matching host machine. - -Based on this description, test writers will usually rely on one of -the following approaches: - -1) Set ``qemu_bin``, and use the given binary - -2) Do not set ``qemu_bin``, and use a QEMU binary named like - "qemu-system-${arch}", either in the current - working directory, or in the current source tree. - -The resulting ``qemu_bin`` value will be preserved in the -``avocado_qemu.QemuSystemTest`` as an attribute with the same name. - -Attribute reference -------------------- - -Test -^^^^ - -Besides the attributes and methods that are part of the base -``avocado.Test`` class, the following attributes are available on any -``avocado_qemu.QemuSystemTest`` instance. - -vm -"" - -A QEMUMachine instance, initially configured according to the given -``qemu_bin`` parameter. - -arch -"""" - -The architecture can be used on different levels of the stack, e.g. by -the framework or by the test itself. At the framework level, it will -currently influence the selection of a QEMU binary (when one is not -explicitly given). - -Tests are also free to use this attribute value, for their own needs. -A test may, for instance, use the same value when selecting the -architecture of a kernel or disk image to boot a VM with. - -The ``arch`` attribute will be set to the test parameter of the same -name. If one is not given explicitly, it will either be set to -``None``, or, if the test is tagged with one (and only one) -``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``. - -cpu -""" - -The cpu model that will be set to all QEMUMachine instances created -by the test. - -The ``cpu`` attribute will be set to the test parameter of the same -name. If one is not given explicitly, it will either be set to -``None ``, or, if the test is tagged with one (and only one) -``:avocado: tags=cpu:VALUE`` tag, it will be set to ``VALUE``. - -machine -""""""" - -The machine type that will be set to all QEMUMachine instances created -by the test. - -The ``machine`` attribute will be set to the test parameter of the same -name. If one is not given explicitly, it will either be set to -``None``, or, if the test is tagged with one (and only one) -``:avocado: tags=machine:VALUE`` tag, it will be set to ``VALUE``. - -qemu_bin -"""""""" - -The preserved value of the ``qemu_bin`` parameter or the result of the -dynamic probe for a QEMU binary in the current working directory or -source tree. - -LinuxTest -^^^^^^^^^ - -Besides the attributes present on the ``avocado_qemu.QemuSystemTest`` base -class, the ``avocado_qemu.LinuxTest`` adds the following attributes: - -distro -"""""" - -The name of the Linux distribution used as the guest image for the -test. The name should match the **Provider** column on the list -of images supported by the avocado.utils.vmimage library: - -https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images - -distro_version -"""""""""""""" - -The version of the Linux distribution as the guest image for the -test. The name should match the **Version** column on the list -of images supported by the avocado.utils.vmimage library: - -https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images - -distro_checksum -""""""""""""""" - -The sha256 hash of the guest image file used for the test. - -If this value is not set in the code or by a test parameter (with the -same name), no validation on the integrity of the image will be -performed. - -Parameter reference -------------------- - -To understand how Avocado parameters are accessed by tests, and how -they can be passed to tests, please refer to:: - - https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#accessing-test-parameters - -Parameter values can be easily seen in the log files, and will look -like the following: - -.. code:: - - PARAMS (key=qemu_bin, path=*, default=./qemu-system-x86_64) => './qemu-system-x86_64 - -Test -^^^^ - -arch -"""" - -The architecture that will influence the selection of a QEMU binary -(when one is not explicitly given). - -Tests are also free to use this parameter value, for their own needs. -A test may, for instance, use the same value when selecting the -architecture of a kernel or disk image to boot a VM with. - -This parameter has a direct relation with the ``arch`` attribute. If -not given, it will default to None. - -cpu -""" - -The cpu model that will be set to all QEMUMachine instances created -by the test. - -machine -""""""" - -The machine type that will be set to all QEMUMachine instances created -by the test. - -qemu_bin -"""""""" - -The exact QEMU binary to be used on QEMUMachine. - -LinuxTest -^^^^^^^^^ - -Besides the parameters present on the ``avocado_qemu.QemuSystemTest`` base -class, the ``avocado_qemu.LinuxTest`` adds the following parameters: - -distro -"""""" - -The name of the Linux distribution used as the guest image for the -test. The name should match the **Provider** column on the list -of images supported by the avocado.utils.vmimage library: - -https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images - -distro_version -"""""""""""""" - -The version of the Linux distribution as the guest image for the -test. The name should match the **Version** column on the list -of images supported by the avocado.utils.vmimage library: - -https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images - -distro_checksum -""""""""""""""" - -The sha256 hash of the guest image file used for the test. - -If this value is not set in the code or by this parameter no -validation on the integrity of the image will be performed. - -Skipping tests --------------- - -The Avocado framework provides Python decorators which allow for easily skip -tests running under certain conditions. For example, on the lack of a binary -on the test system or when the running environment is a CI system. For further -information about those decorators, please refer to:: - - https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#skipping-tests - -While the conditions for skipping tests are often specifics of each one, there -are recurring scenarios identified by the QEMU developers and the use of -environment variables became a kind of standard way to enable/disable tests. - -Here is a list of the most used variables: - -AVOCADO_ALLOW_LARGE_STORAGE -^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Tests which are going to fetch or produce assets considered *large* are not -going to run unless that ``AVOCADO_ALLOW_LARGE_STORAGE=1`` is exported on -the environment. - -The definition of *large* is a bit arbitrary here, but it usually means an -asset which occupies at least 1GB of size on disk when uncompressed. - -SPEED -^^^^^ -Tests which have a long runtime will not be run unless ``SPEED=slow`` is -exported on the environment. - -The definition of *long* is a bit arbitrary here, and it depends on the -usefulness of the test too. A unique test is worth spending more time on, -small variations on existing tests perhaps less so. As a rough guide, -a test or set of similar tests which take more than 100 seconds to -complete. - -AVOCADO_ALLOW_UNTRUSTED_CODE -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -There are tests which will boot a kernel image or firmware that can be -considered not safe to run on the developer's workstation, thus they are -skipped by default. The definition of *not safe* is also arbitrary but -usually it means a blob which either its source or build process aren't -public available. - -You should export ``AVOCADO_ALLOW_UNTRUSTED_CODE=1`` on the environment in -order to allow tests which make use of those kind of assets. - -AVOCADO_TIMEOUT_EXPECTED -^^^^^^^^^^^^^^^^^^^^^^^^ -The Avocado framework has a timeout mechanism which interrupts tests to avoid the -test suite of getting stuck. The timeout value can be set via test parameter or -property defined in the test class, for further details:: - - https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#setting-a-test-timeout - -Even though the timeout can be set by the test developer, there are some tests -that may not have a well-defined limit of time to finish under certain -conditions. For example, tests that take longer to execute when QEMU is -compiled with debug flags. Therefore, the ``AVOCADO_TIMEOUT_EXPECTED`` variable -has been used to determine whether those tests should run or not. - -QEMU_TEST_FLAKY_TESTS -^^^^^^^^^^^^^^^^^^^^^ -Some tests are not working reliably and thus are disabled by default. -This includes tests that don't run reliably on GitLab's CI which -usually expose real issues that are rarely seen on developer machines -due to the constraints of the CI environment. If you encounter a -similar situation then raise a bug and then mark the test as shown on -the code snippet below: - -.. code:: - - # See https://gitlab.com/qemu-project/qemu/-/issues/nnnn - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab') - def test(self): - do_something() - -You can also add ``:avocado: tags=flaky`` to the test meta-data so -only the flaky tests can be run as a group: - -.. code:: - - env QEMU_TEST_FLAKY_TESTS=1 ./pyvenv/bin/avocado \ - run tests/avocado -filter-by-tags=flaky - -Tests should not live in this state forever and should either be fixed -or eventually removed. - - -Uninstalling Avocado --------------------- - -If you've followed the manual installation instructions above, you can -easily uninstall Avocado. Start by listing the packages you have -installed:: - - pip list --user - -And remove any package you want with:: - - pip uninstall - -If you've used ``make check-avocado``, the Python virtual environment where -Avocado is installed will be cleaned up as part of ``make check-clean``. diff --git a/docs/devel/testing/functional.rst b/docs/devel/testing/functional.rst index 9bc973392aa..8030cb4299f 100644 --- a/docs/devel/testing/functional.rst +++ b/docs/devel/testing/functional.rst @@ -6,9 +6,6 @@ Functional testing with Python The ``tests/functional`` directory hosts functional tests written in Python. They are usually higher level tests, and may interact with external resources and with various guest operating systems. -The functional tests have initially evolved from the Avocado tests, so there -is a lot of similarity to those tests here (see :ref:`checkavocado-ref` for -details about the Avocado tests). The tests should be written in the style of the Python `unittest`_ framework, using stdio for the TAP protocol. The folder ``tests/functional/qemu_test`` diff --git a/docs/devel/testing/index.rst b/docs/devel/testing/index.rst index 1171f7db8f0..ccc2fc6cbc1 100644 --- a/docs/devel/testing/index.rst +++ b/docs/devel/testing/index.rst @@ -10,7 +10,6 @@ testing infrastructure. main qtest functional - avocado acpi-bits ci fuzzing diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst index 9869bcf0341..3c03d769539 100644 --- a/docs/devel/testing/main.rst +++ b/docs/devel/testing/main.rst @@ -894,21 +894,6 @@ Python. You can run the functional tests simply by executing: See :ref:`checkfunctional-ref` for more details. -Integration tests using the Avocado Framework ---------------------------------------------- - -The ``tests/avocado`` directory hosts integration tests. They're usually -higher level tests, and may interact with external resources and with -various guest operating systems. - -You can run the avocado tests simply by executing: - -.. code:: - - make check-avocado - -See :ref:`checkavocado-ref` for more details. - .. _checktcg-ref: Testing with "make check-tcg" diff --git a/tests/avocado/README.rst b/tests/avocado/README.rst deleted file mode 100644 index 94488371bbe..00000000000 --- a/tests/avocado/README.rst +++ /dev/null @@ -1,10 +0,0 @@ -============================================= -Integration tests using the Avocado Framework -============================================= - -This directory contains integration tests. They're usually higher -level, and may interact with external resources and with various -guest operating systems. - -For more information, please refer to ``docs/devel/testing.rst``, -section "Integration tests using the Avocado Framework". diff --git a/configure b/configure index 02f1dd2311f..000309cf610 100755 --- a/configure +++ b/configure @@ -1685,7 +1685,7 @@ LINKS="$LINKS pc-bios/optionrom/Makefile" LINKS="$LINKS pc-bios/s390-ccw/Makefile" LINKS="$LINKS pc-bios/vof/Makefile" LINKS="$LINKS .gdbinit scripts" # scripts needed by relative path in .gdbinit -LINKS="$LINKS tests/avocado tests/data" +LINKS="$LINKS tests/data" LINKS="$LINKS tests/qemu-iotests/check tests/qemu-iotests/Makefile" LINKS="$LINKS python" for f in $LINKS ; do diff --git a/pythondeps.toml b/pythondeps.toml index c03c9df81b5..7eaaa0fed10 100644 --- a/pythondeps.toml +++ b/pythondeps.toml @@ -27,9 +27,5 @@ pycotap = { accepted = ">=1.1.0", installed = "1.3.1" } sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" } sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" } -[avocado] -# Note that qemu.git/python/ is always implicitly installed. -# Prefer an LTS version when updating the accepted versions of -# avocado-framework, for example right now the limit is 92.x. -avocado-framework = { accepted = "(>=103.0, <104.0)", installed = "103.0", canary = "avocado" } -pycdlib = { accepted = ">=1.11.0" } +[testdeps] +qemu.qmp = { accepted = ">=0.0.3", installed = "0.0.3" } diff --git a/tests/Makefile.include b/tests/Makefile.include index 010369bd3ac..23fb722d426 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -18,7 +18,6 @@ ifneq ($(filter $(all-check-targets), check-softfloat),) @echo " $(MAKE) check-tcg Run TCG tests" @echo " $(MAKE) check-softfloat Run FPU emulation tests" endif - @echo " $(MAKE) check-avocado Run avocado (integration) tests for currently configured targets" @echo @echo " $(MAKE) check-report.junit.xml Generates an aggregated XML test report" @echo " $(MAKE) check-venv Creates a Python venv for tests" @@ -26,7 +25,6 @@ endif @echo @echo "The following are useful for CI builds" @echo " $(MAKE) check-build Build most test binaries" - @echo " $(MAKE) get-vm-images Downloads all images used by avocado tests, according to configured targets (~350 MB each, 1.5 GB max)" @echo @echo @echo "The variable SPEED can be set to control the gtester speed setting." @@ -86,26 +84,12 @@ distclean-tcg: $(DISTCLEAN_TCG_TARGET_RULES) # Python venv for running tests -.PHONY: check-venv check-avocado check-acceptance check-acceptance-deprecated-warning +.PHONY: check-venv # Build up our target list from the filtered list of ninja targets TARGETS=$(patsubst libqemu-%.a, %, $(filter libqemu-%.a, $(ninja-targets))) TESTS_VENV_TOKEN=$(BUILD_DIR)/pyvenv/tests.group -TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results -ifndef AVOCADO_TESTS - AVOCADO_TESTS=tests/avocado -endif -# Controls the output generated by Avocado when running tests. -# Any number of command separated loggers are accepted. For more -# information please refer to "avocado --help". -AVOCADO_SHOW?=app -ifndef AVOCADO_TAGS - AVOCADO_CMDLINE_TAGS=$(patsubst %-softmmu,-t arch:%, \ - $(filter %-softmmu,$(TARGETS))) -else - AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS)) -endif quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \ $(PYTHON) -m pip -q --disable-pip-version-check $1, \ @@ -113,47 +97,11 @@ quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \ $(TESTS_VENV_TOKEN): $(SRC_PATH)/pythondeps.toml $(call quiet-venv-pip,install -e "$(SRC_PATH)/python/") - $(MKVENV_ENSUREGROUP) $< avocado + $(MKVENV_ENSUREGROUP) $< testdeps $(call quiet-command, touch $@) -$(TESTS_RESULTS_DIR): - $(call quiet-command, mkdir -p $@, \ - MKDIR, $@) - check-venv: $(TESTS_VENV_TOKEN) -FEDORA_31_ARCHES_TARGETS=$(patsubst %-softmmu,%, $(filter %-softmmu,$(TARGETS))) -FEDORA_31_ARCHES_CANDIDATES=$(patsubst ppc64,ppc64le,$(FEDORA_31_ARCHES_TARGETS)) -FEDORA_31_ARCHES := x86_64 aarch64 ppc64le s390x -FEDORA_31_DOWNLOAD=$(filter $(FEDORA_31_ARCHES),$(FEDORA_31_ARCHES_CANDIDATES)) - -# download one specific Fedora 31 image -get-vm-image-fedora-31-%: check-venv - $(call quiet-command, \ - $(PYTHON) -m avocado vmimage get \ - --distro=fedora --distro-version=31 --arch=$*, \ - "AVOCADO", "Downloading avocado tests VM image for $*") - -# download all vm images, according to defined targets -get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD)) - -check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images - $(call quiet-command, \ - $(PYTHON) -m avocado \ - --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \ - $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \ - --filter-by-tags-include-empty-key) \ - $(AVOCADO_CMDLINE_TAGS) --max-parallel-tasks=1 \ - $(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \ - "AVOCADO", "tests/avocado") - -check-acceptance-deprecated-warning: - @echo - @echo "Note '$(MAKE) check-acceptance' is deprecated, use '$(MAKE) check-avocado' instead." - @echo - -check-acceptance: check-acceptance-deprecated-warning | check-avocado - FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$(TARGETS))) .PHONY: $(FUNCTIONAL_TARGETS) $(FUNCTIONAL_TARGETS): @@ -166,13 +114,13 @@ check-functional: # Consolidated targets -.PHONY: check check-clean get-vm-images +.PHONY: check check-clean check: check-build: run-ninja check-clean: - rm -rf $(TESTS_RESULTS_DIR) + rm -rf $(BUILD_DIR)/tests/functional clean: check-clean clean-tcg distclean: distclean-tcg diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py deleted file mode 100644 index 93c34602428..00000000000 --- a/tests/avocado/avocado_qemu/__init__.py +++ /dev/null @@ -1,424 +0,0 @@ -# Test class and utilities for functional tests -# -# Copyright (c) 2018 Red Hat, Inc. -# -# Author: -# Cleber Rosa -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -import logging -import os -import subprocess -import sys -import tempfile -import time -import uuid - -import avocado -from avocado.utils import ssh -from avocado.utils.path import find_command - -from qemu.machine import QEMUMachine -from qemu.utils import (get_info_usernet_hostfwd_port, kvm_available, - tcg_available) - - -#: The QEMU build root directory. It may also be the source directory -#: if building from the source dir, but it's safer to use BUILD_DIR for -#: that purpose. Be aware that if this code is moved outside of a source -#: and build tree, it will not be accurate. -BUILD_DIR = os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(__file__)))) - - -def has_cmd(name, args=None): - """ - This function is for use in a @avocado.skipUnless decorator, e.g.: - - @skipUnless(*has_cmd('sudo -n', ('sudo', '-n', 'true'))) - def test_something_that_needs_sudo(self): - ... - """ - - if args is None: - args = ('which', name) - - try: - _, stderr, exitcode = run_cmd(args) - except Exception as e: - exitcode = -1 - stderr = str(e) - - if exitcode != 0: - cmd_line = ' '.join(args) - err = f'{name} required, but "{cmd_line}" failed: {stderr.strip()}' - return (False, err) - else: - return (True, '') - -def has_cmds(*cmds): - """ - This function is for use in a @avocado.skipUnless decorator and - allows checking for the availability of multiple commands, e.g.: - - @skipUnless(*has_cmds(('cmd1', ('cmd1', '--some-parameter')), - 'cmd2', 'cmd3')) - def test_something_that_needs_cmd1_and_cmd2(self): - ... - """ - - for cmd in cmds: - if isinstance(cmd, str): - cmd = (cmd,) - - ok, errstr = has_cmd(*cmd) - if not ok: - return (False, errstr) - - return (True, '') - -def run_cmd(args): - subp = subprocess.Popen(args, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - universal_newlines=True) - stdout, stderr = subp.communicate() - ret = subp.returncode - - return (stdout, stderr, ret) - -def is_readable_executable_file(path): - return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK) - - -def pick_default_qemu_bin(bin_prefix='qemu-system-', arch=None): - """ - Picks the path of a QEMU binary, starting either in the current working - directory or in the source tree root directory. - - :param arch: the arch to use when looking for a QEMU binary (the target - will match the arch given). If None (the default), arch - will be the current host system arch (as given by - :func:`os.uname`). - :type arch: str - :returns: the path to the default QEMU binary or None if one could not - be found - :rtype: str or None - """ - if arch is None: - arch = os.uname()[4] - # qemu binary path does not match arch for powerpc, handle it - if 'ppc64le' in arch: - arch = 'ppc64' - qemu_bin_name = bin_prefix + arch - qemu_bin_paths = [ - os.path.join(".", qemu_bin_name), - os.path.join(BUILD_DIR, qemu_bin_name), - os.path.join(BUILD_DIR, "build", qemu_bin_name), - ] - for path in qemu_bin_paths: - if is_readable_executable_file(path): - return path - return None - - -def _console_interaction(test, success_message, failure_message, - send_string, keep_sending=False, vm=None): - assert not keep_sending or send_string - if vm is None: - vm = test.vm - console = vm.console_file - console_logger = logging.getLogger('console') - while True: - if send_string: - vm.console_socket.sendall(send_string.encode()) - if not keep_sending: - send_string = None # send only once - - # Only consume console output if waiting for something - if success_message is None and failure_message is None: - if send_string is None: - break - continue - - try: - msg = console.readline().decode().strip() - except UnicodeDecodeError: - msg = None - if not msg: - continue - console_logger.debug(msg) - if success_message is None or success_message in msg: - break - if failure_message and failure_message in msg: - console.close() - fail = 'Failure message found in console: "%s". Expected: "%s"' % \ - (failure_message, success_message) - test.fail(fail) - -def interrupt_interactive_console_until_pattern(test, success_message, - failure_message=None, - interrupt_string='\r'): - """ - Keep sending a string to interrupt a console prompt, while logging the - console output. Typical use case is to break a boot loader prompt, such: - - Press a key within 5 seconds to interrupt boot process. - 5 - 4 - 3 - 2 - 1 - Booting default image... - - :param test: an Avocado test containing a VM that will have its console - read and probed for a success or failure message - :type test: :class:`avocado_qemu.QemuSystemTest` - :param success_message: if this message appears, test succeeds - :param failure_message: if this message appears, test fails - :param interrupt_string: a string to send to the console before trying - to read a new line - """ - _console_interaction(test, success_message, failure_message, - interrupt_string, True) - -def wait_for_console_pattern(test, success_message, failure_message=None, - vm=None): - """ - Waits for messages to appear on the console, while logging the content - - :param test: an Avocado test containing a VM that will have its console - read and probed for a success or failure message - :type test: :class:`avocado_qemu.QemuSystemTest` - :param success_message: if this message appears, test succeeds - :param failure_message: if this message appears, test fails - """ - _console_interaction(test, success_message, failure_message, None, vm=vm) - -def exec_command(test, command): - """ - Send a command to a console (appending CRLF characters), while logging - the content. - - :param test: an Avocado test containing a VM. - :type test: :class:`avocado_qemu.QemuSystemTest` - :param command: the command to send - :type command: str - """ - _console_interaction(test, None, None, command + '\r') - -def exec_command_and_wait_for_pattern(test, command, - success_message, failure_message=None): - """ - Send a command to a console (appending CRLF characters), then wait - for success_message to appear on the console, while logging the. - content. Mark the test as failed if failure_message is found instead. - - :param test: an Avocado test containing a VM that will have its console - read and probed for a success or failure message - :type test: :class:`avocado_qemu.QemuSystemTest` - :param command: the command to send - :param success_message: if this message appears, test succeeds - :param failure_message: if this message appears, test fails - """ - _console_interaction(test, success_message, failure_message, command + '\r') - -class QemuBaseTest(avocado.Test): - - # default timeout for all tests, can be overridden - timeout = 120 - - def _get_unique_tag_val(self, tag_name): - """ - Gets a tag value, if unique for a key - """ - vals = self.tags.get(tag_name, []) - if len(vals) == 1: - return vals.pop() - return None - - def setUp(self, bin_prefix): - self.arch = self.params.get('arch', - default=self._get_unique_tag_val('arch')) - - self.cpu = self.params.get('cpu', - default=self._get_unique_tag_val('cpu')) - - default_qemu_bin = pick_default_qemu_bin(bin_prefix, arch=self.arch) - self.qemu_bin = self.params.get('qemu_bin', - default=default_qemu_bin) - if self.qemu_bin is None: - self.cancel("No QEMU binary defined or found in the build tree") - - def fetch_asset(self, name, - asset_hash, algorithm=None, - locations=None, expire=None, - find_only=False, cancel_on_missing=True): - return super().fetch_asset(name, - asset_hash=asset_hash, - algorithm=algorithm, - locations=locations, - expire=expire, - find_only=find_only, - cancel_on_missing=cancel_on_missing) - - -class QemuSystemTest(QemuBaseTest): - """Facilitates system emulation tests.""" - - def setUp(self): - self._vms = {} - - super().setUp('qemu-system-') - - accel_required = self._get_unique_tag_val('accel') - if accel_required: - self.require_accelerator(accel_required) - - self.machine = self.params.get('machine', - default=self._get_unique_tag_val('machine')) - - def require_accelerator(self, accelerator): - """ - Requires an accelerator to be available for the test to continue - - It takes into account the currently set qemu binary. - - If the check fails, the test is canceled. If the check itself - for the given accelerator is not available, the test is also - canceled. - - :param accelerator: name of the accelerator, such as "kvm" or "tcg" - :type accelerator: str - """ - checker = {'tcg': tcg_available, - 'kvm': kvm_available}.get(accelerator) - if checker is None: - self.cancel("Don't know how to check for the presence " - "of accelerator %s" % accelerator) - if not checker(qemu_bin=self.qemu_bin): - self.cancel("%s accelerator does not seem to be " - "available" % accelerator) - - def require_netdev(self, netdevname): - netdevhelp = run_cmd([self.qemu_bin, - '-M', 'none', '-netdev', 'help'])[0]; - if netdevhelp.find('\n' + netdevname + '\n') < 0: - self.cancel('no support for user networking') - - def _new_vm(self, name, *args): - self._sd = tempfile.TemporaryDirectory(prefix="qemu_") - vm = QEMUMachine(self.qemu_bin, base_temp_dir=self.workdir, - log_dir=self.logdir) - self.log.debug('QEMUMachine "%s" created', name) - self.log.debug('QEMUMachine "%s" temp_dir: %s', name, vm.temp_dir) - self.log.debug('QEMUMachine "%s" log_dir: %s', name, vm.log_dir) - if args: - vm.add_args(*args) - return vm - - def get_qemu_img(self): - self.log.debug('Looking for and selecting a qemu-img binary') - - # If qemu-img has been built, use it, otherwise the system wide one - # will be used. - qemu_img = os.path.join(BUILD_DIR, 'qemu-img') - if not os.path.exists(qemu_img): - qemu_img = find_command('qemu-img', False) - if qemu_img is False: - self.cancel('Could not find "qemu-img"') - - return qemu_img - - @property - def vm(self): - return self.get_vm(name='default') - - def get_vm(self, *args, name=None): - if not name: - name = str(uuid.uuid4()) - if self._vms.get(name) is None: - self._vms[name] = self._new_vm(name, *args) - if self.cpu is not None: - self._vms[name].add_args('-cpu', self.cpu) - if self.machine is not None: - self._vms[name].set_machine(self.machine) - return self._vms[name] - - def set_vm_arg(self, arg, value): - """ - Set an argument to list of extra arguments to be given to the QEMU - binary. If the argument already exists then its value is replaced. - - :param arg: the QEMU argument, such as "-cpu" in "-cpu host" - :type arg: str - :param value: the argument value, such as "host" in "-cpu host" - :type value: str - """ - if not arg or not value: - return - if arg not in self.vm.args: - self.vm.args.extend([arg, value]) - else: - idx = self.vm.args.index(arg) + 1 - if idx < len(self.vm.args): - self.vm.args[idx] = value - else: - self.vm.args.append(value) - - def tearDown(self): - for vm in self._vms.values(): - vm.shutdown() - self._sd = None - super().tearDown() - - -class LinuxSSHMixIn: - """Contains utility methods for interacting with a guest via SSH.""" - - def ssh_connect(self, username, credential, credential_is_key=True): - self.ssh_logger = logging.getLogger('ssh') - res = self.vm.cmd('human-monitor-command', - command_line='info usernet') - port = get_info_usernet_hostfwd_port(res) - self.assertIsNotNone(port) - self.assertGreater(port, 0) - self.log.debug('sshd listening on port: %d', port) - if credential_is_key: - self.ssh_session = ssh.Session('127.0.0.1', port=port, - user=username, key=credential) - else: - self.ssh_session = ssh.Session('127.0.0.1', port=port, - user=username, password=credential) - for i in range(10): - try: - self.ssh_session.connect() - return - except: - time.sleep(i) - self.fail('ssh connection timeout') - - def ssh_command(self, command): - self.ssh_logger.info(command) - result = self.ssh_session.cmd(command) - stdout_lines = [line.rstrip() for line - in result.stdout_text.splitlines()] - for line in stdout_lines: - self.ssh_logger.info(line) - stderr_lines = [line.rstrip() for line - in result.stderr_text.splitlines()] - for line in stderr_lines: - self.ssh_logger.warning(line) - - self.assertEqual(result.exit_status, 0, - f'Guest command failed: {command}') - return stdout_lines, stderr_lines - - def ssh_command_output_contains(self, cmd, exp): - stdout, _ = self.ssh_command(cmd) - for line in stdout: - if exp in line: - break - else: - self.fail('"%s" output does not contain "%s"' % (cmd, exp)) diff --git a/tests/avocado/avocado_qemu/linuxtest.py b/tests/avocado/avocado_qemu/linuxtest.py deleted file mode 100644 index 66fb9f15079..00000000000 --- a/tests/avocado/avocado_qemu/linuxtest.py +++ /dev/null @@ -1,253 +0,0 @@ -# Test class and utilities for functional Linux-based tests -# -# Copyright (c) 2018 Red Hat, Inc. -# -# Author: -# Cleber Rosa -# -# This work is licensed under the terms of the GNU GPL, version 2 or -# later. See the COPYING file in the top-level directory. - -import os -import shutil - -from avocado.utils import cloudinit, datadrainer, process, vmimage - -from avocado_qemu import LinuxSSHMixIn -from avocado_qemu import QemuSystemTest - -if os.path.islink(os.path.dirname(os.path.dirname(__file__))): - # The link to the avocado tests dir in the source code directory - lnk = os.path.dirname(os.path.dirname(__file__)) - #: The QEMU root source directory - SOURCE_DIR = os.path.dirname(os.path.dirname(os.readlink(lnk))) -else: - SOURCE_DIR = BUILD_DIR - -class LinuxDistro: - """Represents a Linux distribution - - Holds information of known distros. - """ - #: A collection of known distros and their respective image checksum - KNOWN_DISTROS = { - 'fedora': { - '31': { - 'x86_64': - {'checksum': ('e3c1b309d9203604922d6e255c2c5d09' - '8a309c2d46215d8fc026954f3c5c27a0'), - 'pxeboot_url': ('https://archives.fedoraproject.org/' - 'pub/archive/fedora/linux/releases/31/' - 'Everything/x86_64/os/images/pxeboot/'), - 'kernel_params': ('root=UUID=b1438b9b-2cab-4065-a99a-' - '08a96687f73c ro no_timer_check ' - 'net.ifnames=0 console=tty1 ' - 'console=ttyS0,115200n8'), - }, - 'aarch64': - {'checksum': ('1e18d9c0cf734940c4b5d5ec592facae' - 'd2af0ad0329383d5639c997fdf16fe49'), - 'pxeboot_url': 'https://archives.fedoraproject.org/' - 'pub/archive/fedora/linux/releases/31/' - 'Everything/aarch64/os/images/pxeboot/', - 'kernel_params': ('root=UUID=b6950a44-9f3c-4076-a9c2-' - '355e8475b0a7 ro earlyprintk=pl011,0x9000000' - ' ignore_loglevel no_timer_check' - ' printk.time=1 rd_NO_PLYMOUTH' - ' console=ttyAMA0'), - }, - 'ppc64': - {'checksum': ('7c3528b85a3df4b2306e892199a9e1e4' - '3f991c506f2cc390dc4efa2026ad2f58')}, - 's390x': - {'checksum': ('4caaab5a434fd4d1079149a072fdc789' - '1e354f834d355069ca982fdcaf5a122d')}, - }, - '32': { - 'aarch64': - {'checksum': ('b367755c664a2d7a26955bbfff985855' - 'adfa2ca15e908baf15b4b176d68d3967'), - 'pxeboot_url': ('http://dl.fedoraproject.org/pub/fedora/linux/' - 'releases/32/Server/aarch64/os/images/' - 'pxeboot/'), - 'kernel_params': ('root=UUID=3df75b65-be8d-4db4-8655-' - '14d95c0e90c5 ro no_timer_check net.ifnames=0' - ' console=tty1 console=ttyS0,115200n8'), - }, - }, - '33': { - 'aarch64': - {'checksum': ('e7f75cdfd523fe5ac2ca9eeece68edc1' - 'a81f386a17f969c1d1c7c87031008a6b'), - 'pxeboot_url': ('http://dl.fedoraproject.org/pub/fedora/linux/' - 'releases/33/Server/aarch64/os/images/' - 'pxeboot/'), - 'kernel_params': ('root=UUID=d20b3ffa-6397-4a63-a734-' - '1126a0208f8a ro no_timer_check net.ifnames=0' - ' console=tty1 console=ttyS0,115200n8' - ' console=tty0'), - }, - }, - } - } - - def __init__(self, name, version, arch): - self.name = name - self.version = version - self.arch = arch - try: - info = self.KNOWN_DISTROS.get(name).get(version).get(arch) - except AttributeError: - # Unknown distro - info = None - self._info = info or {} - - @property - def checksum(self): - """Gets the cloud-image file checksum""" - return self._info.get('checksum', None) - - @checksum.setter - def checksum(self, value): - self._info['checksum'] = value - - @property - def pxeboot_url(self): - """Gets the repository url where pxeboot files can be found""" - return self._info.get('pxeboot_url', None) - - @property - def default_kernel_params(self): - """Gets the default kernel parameters""" - return self._info.get('kernel_params', None) - - -class LinuxTest(LinuxSSHMixIn, QemuSystemTest): - """Facilitates having a cloud-image Linux based available. - - For tests that intend to interact with guests, this is a better choice - to start with than the more vanilla `QemuSystemTest` class. - """ - - distro = None - username = 'root' - password = 'password' - smp = '2' - memory = '1024' - - def _set_distro(self): - distro_name = self.params.get( - 'distro', - default=self._get_unique_tag_val('distro')) - if not distro_name: - distro_name = 'fedora' - - distro_version = self.params.get( - 'distro_version', - default=self._get_unique_tag_val('distro_version')) - if not distro_version: - distro_version = '31' - - self.distro = LinuxDistro(distro_name, distro_version, self.arch) - - # The distro checksum behaves differently than distro name and - # version. First, it does not respect a tag with the same - # name, given that it's not expected to be used for filtering - # (distro name versions are the natural choice). Second, the - # order of precedence is: parameter, attribute and then value - # from KNOWN_DISTROS. - distro_checksum = self.params.get('distro_checksum', - default=None) - if distro_checksum: - self.distro.checksum = distro_checksum - - def setUp(self, ssh_pubkey=None, network_device_type='virtio-net'): - super().setUp() - self.require_netdev('user') - self._set_distro() - self.vm.add_args('-smp', self.smp) - self.vm.add_args('-m', self.memory) - # The following network device allows for SSH connections - self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22', - '-device', '%s,netdev=vnet' % network_device_type) - self.set_up_boot() - if ssh_pubkey is None: - ssh_pubkey, self.ssh_key = self.set_up_existing_ssh_keys() - self.set_up_cloudinit(ssh_pubkey) - - def set_up_existing_ssh_keys(self): - ssh_public_key = os.path.join(SOURCE_DIR, 'tests', 'keys', 'id_rsa.pub') - source_private_key = os.path.join(SOURCE_DIR, 'tests', 'keys', 'id_rsa') - ssh_dir = os.path.join(self.workdir, '.ssh') - os.mkdir(ssh_dir, mode=0o700) - ssh_private_key = os.path.join(ssh_dir, - os.path.basename(source_private_key)) - shutil.copyfile(source_private_key, ssh_private_key) - os.chmod(ssh_private_key, 0o600) - return (ssh_public_key, ssh_private_key) - - def download_boot(self): - # Set the qemu-img binary. - # If none is available, the test will cancel. - vmimage.QEMU_IMG = super().get_qemu_img() - - self.log.info('Downloading/preparing boot image') - # Fedora 31 only provides ppc64le images - image_arch = self.arch - if self.distro.name == 'fedora': - if image_arch == 'ppc64': - image_arch = 'ppc64le' - - try: - boot = vmimage.get( - self.distro.name, arch=image_arch, version=self.distro.version, - checksum=self.distro.checksum, - algorithm='sha256', - cache_dir=self.cache_dirs[0], - snapshot_dir=self.workdir) - except: - self.cancel('Failed to download/prepare boot image') - return boot.path - - def prepare_cloudinit(self, ssh_pubkey=None): - self.log.info('Preparing cloudinit image') - try: - cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso') - pubkey_content = None - if ssh_pubkey: - with open(ssh_pubkey) as pubkey: - pubkey_content = pubkey.read() - cloudinit.iso(cloudinit_iso, self.name, - username=self.username, - password=self.password, - # QEMU's hard coded usermode router address - phone_home_host='10.0.2.2', - phone_home_port=self.phone_server.server_port, - authorized_key=pubkey_content) - except Exception: - self.cancel('Failed to prepare the cloudinit image') - return cloudinit_iso - - def set_up_boot(self): - path = self.download_boot() - self.vm.add_args('-drive', 'file=%s' % path) - - def set_up_cloudinit(self, ssh_pubkey=None): - self.phone_server = cloudinit.PhoneHomeServer(('0.0.0.0', 0), - self.name) - cloudinit_iso = self.prepare_cloudinit(ssh_pubkey) - self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso) - - def launch_and_wait(self, set_up_ssh_connection=True): - self.vm.set_console() - self.vm.launch() - console_drainer = datadrainer.LineLogger(self.vm.console_socket.fileno(), - logger=self.log.getChild('console')) - console_drainer.start() - self.log.info('VM launched, waiting for boot confirmation from guest') - while not self.phone_server.instance_phoned_back: - self.phone_server.handle_request() - - if set_up_ssh_connection: - self.log.info('Setting up the SSH connection') - self.ssh_connect(self.username, self.ssh_key)