From patchwork Mon Mar 23 13:36:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Norbert Manthey X-Patchwork-Id: 11452983 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8BE881392 for ; Mon, 23 Mar 2020 13:39:44 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 66F5720735 for ; Mon, 23 Mar 2020 13:39:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.de header.i=@amazon.de header.b="qpZfeq0n" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 66F5720735 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jGNHe-0000cb-Nv; Mon, 23 Mar 2020 13:38:26 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jGNHc-0000cD-Pa for xen-devel@lists.xenproject.org; Mon, 23 Mar 2020 13:38:24 +0000 X-Inumbo-ID: 91471062-6d0b-11ea-82d5-12813bfff9fa Received: from smtp-fw-33001.amazon.com (unknown [207.171.190.10]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 91471062-6d0b-11ea-82d5-12813bfff9fa; Mon, 23 Mar 2020 13:38:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.de; i=@amazon.de; q=dns/txt; s=amazon201209; t=1584970704; x=1616506704; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=z/DoFgB6ZCcaWgWbVny7toMvgGDC5SXUHgrw+cdeKno=; b=qpZfeq0nB/hYgLAD7UuBZAMVULcUnKlGgL/vhFqS70daTXgvafyggFZe DQcE+kJqwoQA+VkPf8UIvDliczwVVdEUCqeqVckOylkvwP4l549lUEfBq jNmsQwooJPX1QL68r/HN0muVVwd3pGAozzClUWjV0bLcCjtoo1OGI7jaZ Q=; IronPort-SDR: 41ky9xrDQqUFQg0UYXLck/YPsU/jruoM5vqOnjMMom1EQPdEy8f/g8AO1hd8rQ+srvwV6ysPjM aaksbrhqtzSQ== X-IronPort-AV: E=Sophos;i="5.72,296,1580774400"; d="scan'208";a="34295816" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1a-67b371d8.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP; 23 Mar 2020 13:38:22 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan3.iad.amazon.com [10.40.159.166]) by email-inbound-relay-1a-67b371d8.us-east-1.amazon.com (Postfix) with ESMTPS id 9574DA299B; Mon, 23 Mar 2020 13:38:21 +0000 (UTC) Received: from EX13D05EUC001.ant.amazon.com (10.43.164.118) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1236.3; Mon, 23 Mar 2020 13:38:21 +0000 Received: from EX13MTAUWC001.ant.amazon.com (10.43.162.135) by EX13D05EUC001.ant.amazon.com (10.43.164.118) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 23 Mar 2020 13:38:19 +0000 Received: from u6fc700a6f3c650.ant.amazon.com (10.95.138.2) by mail-relay.amazon.com (10.43.162.232) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Mon, 23 Mar 2020 13:38:16 +0000 From: Norbert Manthey To: Andrew Cooper , Date: Mon, 23 Mar 2020 14:36:58 +0100 Message-ID: <20200323133701.21728-2-nmanthey@amazon.de> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200323133701.21728-1-nmanthey@amazon.de> References: <20200323133701.21728-1-nmanthey@amazon.de> MIME-Version: 1.0 Precedence: Bulk Subject: [Xen-devel] [PATCH XTF benchmark v1 1/4] categories: add benchmark X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Pawel Wieczorkiewicz , Norbert Manthey Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" As XTF allows to write tests that interact with the hypervisor, we would like to use this capability to implement micro benchmarks, so that we can measure the performance impact of modifications to the hypervisor. This change introduces a category benchmark, which can be used as container for tests of this kind. Signed-off-by: Norbert Manthey --- build/common.mk | 2 +- xtf-runner | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/build/common.mk b/build/common.mk --- a/build/common.mk +++ b/build/common.mk @@ -1,4 +1,4 @@ -ALL_CATEGORIES := special functional xsa utility in-development +ALL_CATEGORIES := special functional xsa utility in-development benchmark ALL_ENVIRONMENTS := pv64 pv32pae hvm64 hvm32pae hvm32pse hvm32 diff --git a/xtf-runner b/xtf-runner --- a/xtf-runner +++ b/xtf-runner @@ -43,7 +43,7 @@ def exit_code(state): # All test categories default_categories = set(("functional", "xsa")) -non_default_categories = set(("special", "utility", "in-development")) +non_default_categories = set(("special", "utility", "in-development", "benchmark")) all_categories = default_categories | non_default_categories # All test environments