From patchwork Wed Apr 19 10:13:00 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Marta Lofstedt X-Patchwork-Id: 9687239 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id EEFD160375 for ; Wed, 19 Apr 2017 10:08:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E0CD9283F4 for ; Wed, 19 Apr 2017 10:08:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D5D41283FB; Wed, 19 Apr 2017 10:08:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4E2EE283F4 for ; Wed, 19 Apr 2017 10:08:08 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7F1136E037; Wed, 19 Apr 2017 10:08:07 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by gabe.freedesktop.org (Postfix) with ESMTPS id 23FA86E037 for ; Wed, 19 Apr 2017 10:08:06 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 19 Apr 2017 03:08:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.37,220,1488873600"; d="scan'208"; a="1137589037" Received: from mlofsted-dev-hsw.fi.intel.com (HELO localhost.localdomain) ([10.237.72.167]) by fmsmga001.fm.intel.com with ESMTP; 19 Apr 2017 03:08:04 -0700 From: Marta Lofstedt To: intel-gfx@lists.freedesktop.org Date: Wed, 19 Apr 2017 13:13:00 +0300 Message-Id: <20170419101300.17914-1-marta.lofstedt@intel.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170405101153.18525-1-marta.lofstedt@intel.com> References: <20170405101153.18525-1-marta.lofstedt@intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH i-g-t v2] tests/meta_test: Add a meta test for sanity checks of CI systems X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP From: Marta Löfstedt The intention of this test is use it to test that the CI system that runs IGT is collecting the results correctly. For: VIZ-10281 v2: minor edits Signed-off-by: Marta Lofstedt Reviewed-by: Petri Latvala --- tests/Makefile.sources | 1 + tests/intel-ci/meta.testlist | 7 ++ tests/meta_test.c | 149 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 157 insertions(+) create mode 100644 tests/intel-ci/meta.testlist create mode 100644 tests/meta_test.c diff --git a/tests/Makefile.sources b/tests/Makefile.sources index 45c21a0c..7fa9b8f2 100644 --- a/tests/Makefile.sources +++ b/tests/Makefile.sources @@ -143,6 +143,7 @@ TESTS_progs_M = \ template \ vgem_basic \ vgem_slow \ + meta_test \ $(NULL) if HAVE_CHAMELIUM diff --git a/tests/intel-ci/meta.testlist b/tests/intel-ci/meta.testlist new file mode 100644 index 00000000..b3e29235 --- /dev/null +++ b/tests/intel-ci/meta.testlist @@ -0,0 +1,7 @@ +igt@meta_test@pass-result +igt@meta_test@fail-result +igt@meta_test@dmesg-pass +igt@meta_test@dmesg-warn +igt@meta_test@user-crash +igt@meta_test@piglit-timeout +igt@meta_test@generate-panic diff --git a/tests/meta_test.c b/tests/meta_test.c new file mode 100644 index 00000000..e09efba0 --- /dev/null +++ b/tests/meta_test.c @@ -0,0 +1,149 @@ +/* + * Copyright © 2017 Intel Corporation + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice (including the next + * paragraph) shall be included in all copies or substantial portions of the + * Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS + * IN THE SOFTWARE. + * + */ + +#include +#include +#include "igt.h" + +/* + * The purpose of this test is to test the CI system that we have + * for running the tests. The test should generate all possible + * exit states for igt tests. + * + * Possible exit-states of igt tests: + * 1. pass - subtest: pass-result + * 2. fail - subtest: fail-result + * 3. dmesg warn - subtest: dmesg-pass + * - subtest: dmesg-warn + * The purpose is to check that certain kernel log activity + * gets correctly reported in the test result, and that normal + * activity doesn't. + * 4. crash - subtest: user-crash + * 5. piglit timeout - subtest: piglit-timeout + * 6. incomplete - subtest: generate-panic + * NOTE: inorder for this to generate the incomplete state + * the kernel must be configured to reboot on panic. + * NOTE: if the tested CI system have features such as + * PSTORE and/or kexec/kdump enabled. This test could be + * used to make sure that the CI system stores the generated + * log/dumps as expected. + * 7. incomplete - where user hang is not caught by piglit timeout. + * This would be caught by a user-side softdog daemon, + * such as owatch by ezbench. However, I don't know + * how to trigger this state, so it will not be tested. + * 8. incomplete - system requires hard reboot : + * This state could be triggered by calling an evil kernel + * module that was developed hang the system. Such + * a module will not be developed for this purpose, + * so this "exit state" will not be tested. + * + * TODO: If this test was deployed on a CI system that + * was able to pick up testing again after reboot, + * such as ezbench, a post-analyze test should be added + * that collected and analyzed the result of the tests + * run before reboot. + */ + +__attribute__((format(printf, 1, 2))) +static void kmsg(const char *format, ...) +#define KERN_EMER "<0>" +#define KERN_ALERT "<1>" +#define KERN_CRIT "<2>" +#define KERN_ERR "<3>" +#define KERN_WARNING "<4>" +#define KERN_NOTICE "<5>" +#define KERN_INFO "<6>" +#define KERN_DEBUG "<7>" +{ + va_list ap; + FILE *file; + + file = fopen("/dev/kmsg", "w"); + if (file == NULL) + return; + + va_start(ap, format); + vfprintf(file, format, ap); + va_end(ap); + fclose(file); +} + +static void test_result(bool result) +{ + igt_assert_eq(result, true); +} + +static void test_dmesg(bool pass) +{ + if (pass) + kmsg(KERN_DEBUG "drm: IGT inserted string."); + else + kmsg(KERN_WARNING "drm: IGT inserted string."); +} + +static void test_user_crash(void) +{ + raise(SIGSEGV); +} + +static void test_piglit_timeout(void) +{ + sleep(605); +} + +static void test_panic(void) +{ + system("echo c > /proc/sysrq-trigger"); +} + +igt_main +{ + + igt_fixture { + igt_skip_on_f(!getenv("IGT_CI_META_TEST"), + "Only for meta-testing of CI systems"); + } + + igt_subtest("pass-result") + test_result(true); + + igt_subtest("fail-result") + test_result(false); + + igt_subtest("dmesg-pass") + test_dmesg(true); + + igt_subtest("dmesg-warn") + test_dmesg(false); + + igt_subtest("user-crash") + test_user_crash(); + + igt_subtest("piglit-timeout") + test_piglit_timeout(); + + igt_subtest("generate-panic") + test_panic(); +} +