From patchwork Fri Apr 8 15:07:27 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas Meneghel Rodrigues X-Patchwork-Id: 694801 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter1.kernel.org (8.14.4/8.14.3) with ESMTP id p38F7c0E002073 for ; Fri, 8 Apr 2011 15:07:38 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757489Ab1DHPHg (ORCPT ); Fri, 8 Apr 2011 11:07:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53798 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757204Ab1DHPHf (ORCPT ); Fri, 8 Apr 2011 11:07:35 -0400 Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id p38F7Ylp005614 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 8 Apr 2011 11:07:34 -0400 Received: from freedom.redhat.com (vpn-10-185.rdu.redhat.com [10.11.10.185]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id p38F7V9P004524; Fri, 8 Apr 2011 11:07:33 -0400 From: Lucas Meneghel Rodrigues To: autotest@test.kernel.org Cc: kvm@vger.kernel.org, Lucas Meneghel Rodrigues Subject: [PATCH 1/4] client job: Introduce a run_test_detail method Date: Fri, 8 Apr 2011 12:07:27 -0300 Message-Id: <1302275250-6215-2-git-send-email-lmr@redhat.com> In-Reply-To: <1302275250-6215-1-git-send-email-lmr@redhat.com> References: <1302275250-6215-1-git-send-email-lmr@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.6 (demeter1.kernel.org [140.211.167.41]); Fri, 08 Apr 2011 15:07:38 +0000 (UTC) For the KVM test, we have for a long time a system of 'test dependencies': If a dependency test, such as guest installation has failed, it is not possible to run subsequent tests for obvious reasons. There are minor failures though, that don't necessarily block other tests, such as some specific types of image problems found on a guest after the routine check. However, job.run_test() returns a boolean, making it not possible to fail a test, so we are not ignoring problems, and at the same time, execute dependent tests, because the failure was deemed to be non-fatal. Therefore, introduce and additional method, job.run_test_detail(), that returns the test status (see all possible statuses on client/common_lib/error.py), rather than a boolean. With this finer grained control, we can implement the kvm subtest dependency system in a better way. Strategy: refactor the run_test method into _run_test_base and creating the 2 implementations, run_test, and run_test_detail in terms of _run_test_base. Risk: High (touches a very very visible test writer API) Tests executed: run_test continues to work, run_test_detail provides the correct return codes and change does not break job_unittest. Signed-off-by: Lucas Meneghel Rodrigues --- client/bin/job.py | 54 +++++++++++++++++++++++++++++++++++++++++++++++++--- 1 files changed, 50 insertions(+), 4 deletions(-) diff --git a/client/bin/job.py b/client/bin/job.py index 97412c0..9effcdb 100644 --- a/client/bin/job.py +++ b/client/bin/job.py @@ -539,10 +539,9 @@ class base_client_job(base_job.base_job): raise error.UnhandledTestError(e) - @_run_test_complete_on_exit - def run_test(self, url, *args, **dargs): + def _run_test_base(self, url, *args, **dargs): """ - Summon a test object and run it. + Prepares arguments and run functions to run_test and run_test_detail. @param url A url that identifies the test to run. @param tag An optional keyword argument that will be added to the @@ -550,7 +549,11 @@ class base_client_job(base_job.base_job): @param subdir_tag An optional keyword argument that will be added to the subdir name. - @returns True if the test passes, False otherwise. + @returns: + subdir: Test subdirectory + testname: Test name + group_func: Actual test run function + timeout: Test timeout """ group, testname = self.pkgmgr.get_package_name(url, 'test') testname, subdir, tag = self._build_tagged_test_name(testname, dargs) @@ -573,6 +576,25 @@ class base_client_job(base_job.base_job): else: self.record('GOOD', subdir, testname, 'completed successfully') + return (subdir, testname, group_func, timeout) + + + @_run_test_complete_on_exit + def run_test(self, url, *args, **dargs): + """ + Summon a test object and run it. + + @param url A url that identifies the test to run. + @param tag An optional keyword argument that will be added to the + test and subdir name. + @param subdir_tag An optional keyword argument that will be added + to the subdir name. + + @returns True if the test passes, False otherwise. + """ + (subdir, testname, group_func, timeout) = self._run_test_base(url, + *args, + **dargs) try: self._rungroup(subdir, testname, group_func, timeout) return True @@ -585,6 +607,30 @@ class base_client_job(base_job.base_job): # UnhandledTestError that is caught above. + @_run_test_complete_on_exit + def run_test_detail(self, url, *args, **dargs): + """ + Summon a test object and run it, returning test status. + + @param url A url that identifies the test to run. + @param tag An optional keyword argument that will be added to the + test and subdir name. + @param subdir_tag An optional keyword argument that will be added + to the subdir name. + + @returns Test status + @see: client/common_lib/error.py, exit_status + """ + (subdir, testname, group_func, timeout) = self._run_test_base(url, + *args, + **dargs) + try: + self._rungroup(subdir, testname, group_func, timeout) + return 'GOOD' + except error.TestBaseException, detail: + return detail.exit_status + + def _rungroup(self, subdir, testname, function, timeout, *args, **dargs): """\ subdir: