From patchwork Thu Jan 17 18:56:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 10768823 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A4F1313A4 for ; Thu, 17 Jan 2019 19:06:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 94FE72BC2A for ; Thu, 17 Jan 2019 19:06:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8573F2BCA1; Thu, 17 Jan 2019 19:06:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 108952BC2A for ; Thu, 17 Jan 2019 19:06:56 +0000 (UTC) Received: from localhost ([127.0.0.1]:51754 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gkD0B-0005D0-B0 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 17 Jan 2019 14:06:55 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42890) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gkCqo-0005Jx-Ia for qemu-devel@nongnu.org; Thu, 17 Jan 2019 13:57:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gkCqk-00066K-S7 for qemu-devel@nongnu.org; Thu, 17 Jan 2019 13:57:12 -0500 Received: from mx1.redhat.com ([209.132.183.28]:52582) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gkCqh-0005RO-KF; Thu, 17 Jan 2019 13:57:08 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CFCE080F9F; Thu, 17 Jan 2019 18:56:35 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-121-148.rdu2.redhat.com [10.10.121.148]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8031B5D6AA; Thu, 17 Jan 2019 18:56:29 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org Date: Thu, 17 Jan 2019 13:56:10 -0500 Message-Id: <20190117185628.21862-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Thu, 17 Jan 2019 18:56:36 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 00/18] Acceptance Tests: target architecture support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Cleber Rosa , Fam Zheng , Eduardo Habkost , Stefan Markovic , =?utf-8?q?Alex_Benn=C3=A9e?= , Cornelia Huck , Wainer dos Santos Moschetta , Aleksandar Rikalo , qemu-s390x@nongnu.org, Aleksandar Markovic , Caio Carrara , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Aurelien Jarno Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP The current acceptance tests don't provide any type of architecture information that can be used to influence the selection of the QEMU binary used on them[1]. If one is running tests on a x86_64 host, the default QEMU binary will be "x86_64-softmmu/qemu-system-x86_64". Given the nature of QEMU, some tests will be architecture agnostic, while others will be architecture dependent. The "check-qtest" and "check-qtest-TARGET" make targets exemplify that pattern. For the acceptance tests, the same requirement exists. Tests should be allowed to influence the binary used, and when they don't, a default selection mechanism should kick in[2]. The proposed solution here requires only that an Avocado tag is set, such as: class My(Test): def test_nx_cpu_flag(self): """ :avocado: tags=arch:x86_64 """ test_code() The value of the "arch" key, in this case, "x86_64" will be used when selecting the QEMU binary to use in the test. At the same time, if "x86_64-softmmu" is not a built target, the test will be filtered out by "make check-acceptance"[3]. Besides the convention explained above, where the binary will be selected from the "arch" tag, it's also possible to set an "arch" *parameter* that will also influence the QEMU binary selection: $ avocado run -p arch=aarch64 works-on-many-arches.py Finally, it's also posible to set the "qemu_bin" parameter, which will define (instead of just influencing) the QEMU binary to be used: $ avocado run -p qemu_bin=qemu-bin-aarch64 test.py As examples for the idea proposed here, a number of "boot linux console" tests have been added, for a number of different target architectures. When the build environment includes them (as it has been added to Travis CI jobs) the architecture specific tests will be automatically executed. As mentioned previously, this patch series include ideas present in other patch series, and from different authors. I tried by best to include the information about authorship, but if I missed any, please accept my apologies and let me know. --- [1] - The "boot_linux_console.py" contains a "x86_64" test tag, but that is informational only, in the sense that it's not consumed by the test itself, or used by "make check-acceptance" to filter out tests. [2] - This patch series doesn't attempt to change the default selection mechanism. Possible changes in this area may include looking for any one built binary first, no matter the host architecture. [3] - On a previous proposed version, the test class would look at the "arch" parameter given, and would cancel the test if there wasn't a match. --- Git Info: - URI: https://github.com/clebergnu/qemu/tree/sent/target_arch - Remote: https://github.com/clebergnu/qemu - Branch: sent/target_arch Cleber Rosa (17): scripts/qemu.py: log QEMU launch command line Acceptance tests: show avocado test execution by default Acceptance tests: improve docstring on pick_default_qemu_bin() Acceptance tests: fix doc reference to avocado_qemu directory Acceptance tests: introduce arch parameter and attribute Acceptance tests: use "arch:" tag to filter target specific tests Acceptance tests: look for target architecture in test tags first Boot Linux Console Test: rename the x86_64 after the arch and machine Boot Linux Console Test: update the x86_64 kernel Boot Linux Console Test: refactor the console watcher into utility method scripts/qemu.py: support adding a console with the default serial device Boot Linux Console Test: add a test for mips64el + malta Boot Linux Console Test: add a test for ppc64 + pseries Boot Linux Console Test: add a test for aarch64 + virt Boot Linux Console Test: add a test for arm + virt Boot Linux Console Test: add a test for s390x + s390-ccw-virtio Boot Linux Console Test: add a test for alpha + clipper Philippe Mathieu-Daudé (1): Boot Linux Console Test: add a test for mips + malta .travis.yml | 4 +- docs/devel/testing.rst | 23 ++- scripts/qemu.py | 32 ++-- tests/Makefile.include | 5 +- tests/acceptance/avocado_qemu/__init__.py | 22 ++- tests/acceptance/boot_linux_console.py | 208 ++++++++++++++++++++-- tests/requirements.txt | 2 +- 7 files changed, 254 insertions(+), 42 deletions(-)