From patchwork Fri Apr 28 18:26:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mohit Gambhir X-Patchwork-Id: 9705247 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5595760225 for ; Fri, 28 Apr 2017 18:29:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4BA5628693 for ; Fri, 28 Apr 2017 18:29:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 404762869B; Fri, 28 Apr 2017 18:29:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6F63528693 for ; Fri, 28 Apr 2017 18:29:18 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1d4AbP-0002Le-TV; Fri, 28 Apr 2017 18:26:47 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1d4AbO-0002LN-2n for xen-devel@lists.xen.org; Fri, 28 Apr 2017 18:26:46 +0000 Received: from [85.158.137.68] by server-6.bemta-3.messagelabs.com id 88/0F-02189-56983095; Fri, 28 Apr 2017 18:26:45 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrLLMWRWlGSWpSXmKPExsXSO6nOVTepkzn S4O8VPYslHxezODB6HN39mymAMYo1My8pvyKBNePf1rusBQ9yKj6dvsrawNjv38XIxSEk0MEk cWfZaiYI5xujxPGlr6CcjYwSvxceY4ZwdjNKLDv9laWLkZODTUBPYtOaxewgtoiAmcTsCS9YQ WxmgTqJf983MoPYwgL2Er2f29lAbBYBVYmj3c1gNq+AjcSnb3fBbAkBOYlL276A1XMK2Eqcvz KFEcQWAqp523KJBaLGWKJvVh/LBEa+BYwMqxg1ilOLylKLdI0N9JKKMtMzSnITM3N0DQ2M9XJ Ti4sT01NzEpOK9ZLzczcxAoOlnoGBcQdj5wm/Q4ySHExKoryV7syRQnxJ+SmVGYnFGfFFpTmp xYcYZTg4lCR4N3QA5QSLUtNTK9Iyc4BhC5OW4OBREuG9A5LmLS5IzC3OTIdInWJUlBLnnQWSE ABJZJTmwbXBYuUSo6yUMC8jAwODEE9BalFuZgmq/CtGcQ5GJWHeXJApPJl5JXDTXwEtZgJazO LCALK4JBEhJdXAGPeLW0D46ma/XedOVGbNuVwW8aU+KMrjsLncm69XAs9oC96pDJefzHdai/W GsdQ3zfpPTAxdmwQ8Li9w8FQ+GzT5j5jyYu8rDcu2n3VYvjCvRKjqElst6y/r/yZXv74OWZF/ TvvZM/V7E/NPMGyTPrf/xE7u+H+vpdynGpzI1FLrZc31bZVrVGIpzkg01GIuKk4EAH1Li3SQA gAA X-Env-Sender: mohit.gambhir@oracle.com X-Msg-Ref: server-3.tower-31.messagelabs.com!1493404000!97945386!1 X-Originating-IP: [141.146.126.69] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTQxLjE0Ni4xMjYuNjkgPT4gMjc3MjE4\n X-StarScan-Received: X-StarScan-Version: 9.4.12; banners=-,-,- X-VirusChecked: Checked Received: (qmail 24993 invoked from network); 28 Apr 2017 18:26:42 -0000 Received: from aserp1040.oracle.com (HELO aserp1040.oracle.com) (141.146.126.69) by server-3.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 28 Apr 2017 18:26:42 -0000 Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v3SIQct6032467 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 28 Apr 2017 18:26:38 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.13.8/8.14.4) with ESMTP id v3SIQcWA008709 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 28 Apr 2017 18:26:38 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id v3SIQc3M008687; Fri, 28 Apr 2017 18:26:38 GMT Received: from dhcp-burlington7-2nd-B-west-10-152-54-159.usdhcp.oraclecorp.com.com (/10.152.54.159) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 28 Apr 2017 11:26:37 -0700 From: Mohit Gambhir To: andrew.cooper3@citrix.com, xen-devel@lists.xen.org Date: Fri, 28 Apr 2017 14:26:01 -0400 Message-Id: <20170428182601.5649-3-mohit.gambhir@oracle.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20170428182601.5649-1-mohit.gambhir@oracle.com> References: <20170428182601.5649-1-mohit.gambhir@oracle.com> X-Source-IP: aserv0021.oracle.com [141.146.126.233] Cc: boris.ostrovsky@oracle.com, mgambhir@outlook.com, Mohit Gambhir Subject: [Xen-devel] [PATCH v2 2/2] xtf/vpmu: MSR read/write tests for VPMU X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch tests VPMU functionality in the hypervisor on Intel machines. The tests write to all valid bits in the MSRs that get exposed to the guests when VPMU is enabled. The tests also write invalid values to the bits that should be masked and expect the wrmsr call to fault. The tests are currently unsupported for AMD machines and PV guests. Signed-off-by: Mohit Gambhir --- tests/vpmu/Makefile | 9 ++ tests/vpmu/main.c | 442 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 451 insertions(+) create mode 100644 tests/vpmu/Makefile create mode 100644 tests/vpmu/main.c diff --git a/tests/vpmu/Makefile b/tests/vpmu/Makefile new file mode 100644 index 0000000..57eb29d --- /dev/null +++ b/tests/vpmu/Makefile @@ -0,0 +1,9 @@ +include $(ROOT)/build/common.mk + +NAME := vpmu +CATEGORY := functional +TEST-ENVS := hvm64 hvm32 + +obj-perenv += main.o + +include $(ROOT)/build/gen.mk diff --git a/tests/vpmu/main.c b/tests/vpmu/main.c new file mode 100644 index 0000000..b7f9398 --- /dev/null +++ b/tests/vpmu/main.c @@ -0,0 +1,442 @@ +/** + * @file tests/vpmu/main.c + * @ref test-vpmu + * + * @page test-vpmu vpmu + * + * Test Virtual Performance Monitoring Unit implementation, which allows guests + * to program fixed and general purpose performance counters to measure hardware + * performance. + * + * The tests read and write MSRs that are exposed by VPMU for various versions + * of Architectural Performance Monitoring + * + * @see tests/vpmu/main.c + */ + +#include +#include +#include + +#define EVENT_UOPS_RETIRED 0x004101c2 +#define EVENT_UOPS_RETIRED_ANYTHREAD 0x006101c2 +#define FIXED_CTR_CTL_BITS 4 +#define FIXED_CTR_ENABLE 0x0000000A +#define FIXED_CTR_ENABLE_ANYTHREAD 0x0000000E +#define PERFEVENTSELx_ENABLE_ANYTHREAD (1ull << 21) +#define PERFEVENTSELx_ENABLE_PCB (1ull << 19) +#define DEBUGCTL_Freeze_LBR_ON_PMI (1ull << 11) +#define DEBUGCTL_Freeze_PerfMon_On_PMI (1ull << 12) +#define PERF_CAPABILITIES_Full_Width (1ull << 13) + +const char test_title[] = "Test vpmu"; + +static void test_valid_msr_write(uint32_t idx, uint64_t wval) +{ + uint64_t rval = 0; + + if( wrmsr_safe(idx, wval) ) + xtf_failure("Fail: wrmsr(0x%08x, 0x%016"PRIx64") got unexpected fault" + "\n", idx, wval); + + /* check to see if the values were written correctly */ + if ( rdmsr_safe(idx, &rval) ) + xtf_failure("Fail: rdmsr(0x%08x, 0x016%"PRIxPTR") got unexpected fault" + "\n", idx, (uintptr_t) &rval); + + if ( rval != wval ) + xtf_failure("Fail: rdmsr mismatch idx 0x%08x, wval 0x%016"PRIx64 + ", rval 0x%016"PRIx64"\n", idx, wval, rval); +} + +static void test_invalid_msr_write(uint32_t idx, uint64_t wval) +{ + /* wrmsr_safe must return false after faulting */ + if( !wrmsr_safe(idx, wval) ) + xtf_failure("Fail: wrmsr(0x%08x, 0x%016"PRIx64") did not fault.\n", + idx, wval); +} + +static void test_transparent_msr_write(uint32_t idx, uint64_t wval) +{ + uint64_t rval1 = 0, rval2 = 0; + + /* read current value */ + if ( rdmsr_safe(idx, &rval1) ) + xtf_failure("Fail: rdmsr(0x%08x, 0x016%"PRIxPTR") got unexpected fault" + "\n", idx, (uintptr_t)&rval1); + + /* wrmsr should not fault but should not write anything either */ + if ( wrmsr_safe(idx, wval) ) + xtf_failure("Fail: wrmsr(0x%08x, 0x%016"PRIx64") got unexpected fault" + "\n", idx, wval); + + /* read new value */ + if ( rdmsr_safe(idx, &rval2) ) + xtf_failure("Fail: rdmsr(0x%08x, 0x016%"PRIxPTR") got unexpected fault" + "\n", idx, (uintptr_t) &rval2); + + /* Check if the new value is the same as the one before wrmsr */ + if ( rval1 != rval2 ) + xtf_failure("Fail: rdmsr mismatch idx 0x%08x, wval 0x%016"PRIx64 + ", rval 0x%016"PRIx64"\n", idx, rval1, rval2); +} + +static void test_intel_pmu_ver1(uint8_t ng) +{ + /* + * Intel Sofwtare Development Manual Vol. 3B, + * Section 18.2.1 - Architectural Performance Monitoring Version 1 + * + * MSRs made available by the VPMU + * + * PMCx (start at address 0xc1) + * PERFEVENTSELx (start at address 0x186) + */ + + uint32_t idx; + uint64_t wval = 0; + unsigned i; + + printk("Testing version 1\n"); + + /* for all general purpose counters */ + for ( i = 0; i < ng; i++ ) + { + /* test writing to PMCx */ + idx = MSR_PMC(i); + + /* + * test we can write to all valid bits in the counters + * don't set bit 31 since that gets sign-extended + */ + wval = ((1ull << 31) - 1) ; + + test_valid_msr_write(idx, wval); + + /* set all valid bits in MSR_EVENTSELx */ + idx = MSR_PERFEVTSEL(i); + wval = ((1ull << 32) - 1) ^ (PERFEVENTSELx_ENABLE_ANYTHREAD | + PERFEVENTSELx_ENABLE_PCB); + + test_valid_msr_write(idx, wval); + + /* test writing an invalid value and assert that it faults */ + wval = ~((1ull << 32) - 1); + + test_invalid_msr_write(idx, wval); + } +} + +static void test_intel_pmu_ver2(uint8_t ng, uint8_t nf) +{ + /* + * Intel Sofwtare Development Manual Vol. 3B, + * Section 18.2.2 - Architectural Performance Monitoring Version 2 + * + * MSRs made available by the VPMU - + * + * FIXED_CTRx (start at address 0x309) + * FIXED_CTR_CTRL + * PERF_GLOBAL_CTRL + * PERF_GLOBAL_STATUS (read-only) + * PERF_GLOBAL_OVF_CTRL + * DEBUGCTL_Freeze_LBR_ON_PMI + * DEBUGCTL_Freeze_PerfMon_On_PMI + */ + + uint32_t idx; + uint64_t wval; + uint64_t rval; + unsigned i; + + printk("Testing version 2\n"); + + for ( i = 0; i < nf; i++ ) + { + /* test writing to FIXED_CTRx */ + idx = MSR_FIXED_CTR(i); + + wval = (1ull << 32) - 1; + + test_valid_msr_write(idx, wval); + + /* test invalid write to FIXED_CTRx */ + wval = ~wval; + + test_invalid_msr_write(idx, wval); + } + + /* test FIXED_CTR_CTRL */ + idx = MSR_FIXED_CTR_CTRL; + + /* enable all fixed counters */ + wval = 0; + + for ( i = 0; i < nf; i++ ) + wval |= (FIXED_CTR_ENABLE << (FIXED_CTR_CTL_BITS * i)); + + test_valid_msr_write(idx, wval); + + /* invert wval to test writing an invalid value */ + wval = ~wval; + + test_invalid_msr_write(idx, wval); + + /* test PERF_GLOBAL_CTRL */ + idx = MSR_PERF_GLOBAL_CTRL; + + wval = 0; + + /* set all fixed function counters enable bits */ + for ( i=0; i < nf; i ++ ) + wval |= ((uint64_t)1 << (32 + i)); + + /* set all general purpose counters enable bits*/ + for ( i = 0; i < ng; i++ ) + wval |= (1 << i); + + test_valid_msr_write(idx, wval); + + /* invert wval to test invalid write to MSR_PERF_GLOBAL_CTRL*/ + wval = ~wval; + + test_invalid_msr_write(idx, wval); + + /* test PERF_GLOBAL_OVF_CTRL */ + idx = MSR_PERF_GLOBAL_OVF_CTRL; + + /* set all valid bits in MSR_PERF_GLOBAL_OVF_CTRL */ + wval = 0xC000000000000000 | (((1ULL << nf) - 1) << 32) | ((1ULL << ng) - 1); + + /* + * Writing to MSR_PERF_GLOBAL_OVF_CTRL clears + * MSR_PERF_GLOBAL_STATUS but but always returns 0 when read so + * it is tested as a transparent write + */ + + test_transparent_msr_write(idx, wval); + + /* invert wval to test invalid write to MSR_PERF_GLOBAL_OVF_CTRL*/ + wval = ~wval; + + test_invalid_msr_write(idx, wval); + + /* test PERF_GLOBAL_STATUS (read-only) */ + idx = MSR_PERF_GLOBAL_STATUS; + + if ( rdmsr_safe(idx, &rval) ) + { + xtf_failure("Error: test_intel_pmu_ver2: " + "rdmsr_safe for MSR 0x%x resulted in a fault!\n", idx); + } + + /* try to write the PERF_GLOBAL_STATUS register and expect it to fail*/ + test_invalid_msr_write(idx, wval); + + /* test DEBUGCTL */ + idx = MSR_DEBUGCTL; + + /* Test DEBUGCTL facilities enabled by v2 */ + wval = DEBUGCTL_Freeze_LBR_ON_PMI | DEBUGCTL_Freeze_PerfMon_On_PMI; + + /* + * XENBUG: This test currently fails and needs to be fixed in Xen. Once + * fixed, call test_valid_msr_write(idx, wval) to fully test MSR write + */ + if( wrmsr_safe(idx, wval) ) + xtf_error("Fail: wrmsr(0x%08x, 0x%016"PRIx64") got unexpected fault" + "\n", idx, wval); +} + +static void test_intel_pmu_ver3(uint8_t ng, uint8_t nf) +{ + /* + * Intel Sofwtare Development Manual Vol. 3B + * Section 18.2.3 Architectural Performance Monitoring Version 3 + * + * MSRs made available by the VPMU + * + * PERFEVENTSELx.ANYTHREAD (Bit 21) + * FIXED_CTR_CTRL.ANYTHREADx (Bit 2, 6, 11) + * + * Version 3 introduces ANYTHREAD bit but VPMU does not support it to + * ensure that a VCPU isn't able to read values from physical resources that + * are not allocated to it. This test case validates that we are unable to + * write to .ANYTHREAD bit in PERFEVENTSELx and FIXED_CTR_CTRL + */ + + uint64_t wval; + uint32_t idx; + unsigned i; + + printk("Testing version 3\n"); + + /* test PERFEVENTSELx.ANYTHREAD is disabled */ + for ( i = 0; i < ng; i++ ) + { + idx = MSR_PERFEVTSEL(i); + + wval = EVENT_UOPS_RETIRED_ANYTHREAD; + + test_invalid_msr_write(idx, wval); + } + + /* test FIXED_CTR_CTL.ANYTHREAD is disabled */ + idx = MSR_FIXED_CTR_CTRL; + + wval = 0; + + for ( i = 0; i < nf; i++ ) + wval |= (FIXED_CTR_ENABLE_ANYTHREAD << (4 * i)) ; + + test_invalid_msr_write(idx, wval); +} + +static void test_full_width_cnts(uint8_t ng, uint8_t ngb, uint8_t pdcm) +{ + uint64_t caps, wval; + uint32_t idx; + unsigned i; + + if ( rdmsr_safe(MSR_PERF_CAPABILITIES, &caps) ) + { + if (pdcm) { + xtf_failure("Fail: Fault while reading MSR_PERF_CAPABILITIES\n"); + } + return; + } + else if ( !pdcm ) + { + /* + * XENBUG: MSR_PERF_CAPABILITIES should nto be readable without PDCM + * bit set in CPUID. + */ + xtf_error("Error: MSR_PERF_CAPABILITIES readable while PDCM is 0\n"); + } + + + if ( !(caps & PERF_CAPABILITIES_Full_Width) ) + { + printk("Info: Full width counters not supported\n"); + return; + } + + /* test writes to full width counters */ + for ( i = 0; i < ng; i++) + { + idx = MSR_A_PMC(i); + + wval = ((1ull << ngb) - 1) ; + + test_valid_msr_write(idx, wval); + + /* invert wval to test invalid write to MSR_PERF_GLOBAL_OVF_CTRL */ + wval = ~wval; + + test_invalid_msr_write(idx, wval); + } +} + +void test_main(void) +{ + /* Architectural Performance Monitoring Version */ + uint8_t ver; + /* Number of general purpose counter registers */ + uint8_t ngregs; + /* Number of fixed function counter registers */ + uint8_t nfregs; + /* Bit width of general-purpose, performance monitoring counter */ + uint8_t ngbits; + /* Performance and debug capabilties MSR*/ + uint8_t pdcm; + + /* cpuid variables */ + uint32_t leaf = 0x0a, subleaf = 0; + uint32_t eax, ebx, ecx, edx; + + if ( vendor_is_amd ) + return xtf_skip("Skip: VPMU testing for AMD currently unsupported\n"); + + if (max_leaf < leaf) + return xtf_skip("Skip: Unable to retrieve PMU information from cpuid\n"); + + /* + * Inter Software Development Manual Vol 2A + * Table 3-8 Information Returned by CPUID Instruction + */ + + cpuid_count(leaf, subleaf, &eax, &ebx, &ecx, &edx); + + /* Extract the version ID - EAX 7:0 */ + ver = (eax & 0xff); + + /* Extract number of general purpose counter regs - EAX 15:8 */ + ngregs = (eax >> 8) & 0xff; + + /* Extract number of fixed function counter regs - EDX 4:0 */ + nfregs = edx & 0x1f; + + /* Extract number of bits in general purpose counter registers bits */ + ngbits = (eax >> 16) & 0xff; + + /* Retrieve Perf & Debug Capabilties MSR availability*/ + leaf = 0x01; + + printk("Info: PMU Version %u, Genereal purpose counters %u, Fixed function " + "counters %u, Counter width %u\n", ver, ngregs, nfregs, ngbits); + + cpuid_count(leaf, subleaf, &eax, &ebx, &ecx, &edx); + + /* Extract Performance & Debug Capabilties MSR from ECX bit 15 */ + pdcm = (ecx >> 15) & 0x01; + + printk("Info: Perf & Debug Capability MSR %u\n", pdcm); + + switch (ver) + { + + default: + /* + * Version 4 facilities are not supported by Xen. + * VPMU emulates version 3. Fall through + */ + xtf_warning("Test doesn't support version %u\n", ver); + + case 3: + /* test version 3 facilities */ + test_intel_pmu_ver3( ngregs, nfregs); + + /* fall through */ + + case 2: + /* test version 2 facilities */ + test_intel_pmu_ver2(ngregs, nfregs); + + /* test version 1 facilities */ + test_intel_pmu_ver1(ngregs); + + /* test full width counters */ + test_full_width_cnts(ngregs, ngbits, pdcm); + + break; + + case 1: + return xtf_skip("Skip: VPMU version 1 unsupported in Xen\n"); + + case 0: + return xtf_skip("Skip: VPMU not enabled in Xen\n"); + } + + return xtf_success(NULL); +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */