From patchwork Fri Jun 21 06:09:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 11008461 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A65F1395 for ; Fri, 21 Jun 2019 06:13:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 27D972003F for ; Fri, 21 Jun 2019 06:13:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1B6C5289BE; Fri, 21 Jun 2019 06:13:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BE83C289A3 for ; Fri, 21 Jun 2019 06:13:04 +0000 (UTC) Received: from localhost ([::1]:54966 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1heCnH-0008Lf-R0 for patchwork-qemu-devel@patchwork.kernel.org; Fri, 21 Jun 2019 02:13:03 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47109) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1heCku-0005C7-Ez for qemu-devel@nongnu.org; Fri, 21 Jun 2019 02:10:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1heCkZ-00048I-8Q for qemu-devel@nongnu.org; Fri, 21 Jun 2019 02:10:16 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55104) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1heCkY-0003G6-Uc for qemu-devel@nongnu.org; Fri, 21 Jun 2019 02:10:15 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BEE1DC05A1D8; Fri, 21 Jun 2019 06:09:36 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-204.rdu2.redhat.com [10.10.120.204]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9C2DB5D739; Fri, 21 Jun 2019 06:09:27 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org Date: Fri, 21 Jun 2019 02:09:23 -0400 Message-Id: <20190621060925.16214-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 21 Jun 2019 06:09:42 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 0/2] Acceptance tests: exclude "flaky" tests and introduce SPICE test X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Eduardo Habkost , Aleksandar Rikalo , =?utf-8?q?Alex_Benn=C3=A9e?= , Wainer dos Santos Moschetta , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Aurelien Jarno Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP These two seemingly unrelated topics have been added together in the same patch series because both deal with Avocado tags, and how their use is expand here to exclude tests. For tests which sometimes can fail or hang, this adds the "flaky" tag, and doesn't run them as part of "make check-acceptance". For the SPICE test, it depends on QEMU being configured with it, and if not, it will exclude it from the set of tests. Cleber Rosa (2): Acceptance tests: exclude "flaky" tests Acceptance tests: add SPICE protocol check .travis.yml | 5 ++- docs/devel/testing.rst | 17 ++++++++ tests/Makefile.include | 12 +++++- tests/acceptance/boot_linux_console.py | 2 + tests/acceptance/linux_ssh_mips_malta.py | 2 + tests/acceptance/spice.py | 54 ++++++++++++++++++++++++ tests/requirements.txt | 2 +- 7 files changed, 91 insertions(+), 3 deletions(-) create mode 100644 tests/acceptance/spice.py