From patchwork Sun Jul 9 08:09:04 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai Huang X-Patchwork-Id: 9831693 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 04AA260318 for ; Sun, 9 Jul 2017 08:12:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EDF7C237A5 for ; Sun, 9 Jul 2017 08:12:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E259027F82; Sun, 9 Jul 2017 08:12:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 24E0A237A5 for ; Sun, 9 Jul 2017 08:12:02 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dU7Ha-0004ai-KH; Sun, 09 Jul 2017 08:09:34 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dU7HY-0004aO-S8 for xen-devel@lists.xen.org; Sun, 09 Jul 2017 08:09:33 +0000 Received: from [85.158.139.211] by server-10.bemta-5.messagelabs.com id 42/73-01732-CB4E1695; Sun, 09 Jul 2017 08:09:32 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrPIsWRWlGSWpSXmKPExsXiVRvsrLv7SWK kwdlNAhZLPi5mcWD0OLr7N1MAYxRrZl5SfkUCa8abefuZC9qDK5ZsMWpgfOPQxcjFISQwkVFi +/tZTCAOi0AXq8TETx9YQRwJgWWsEjM7vgFlOIGcOIlNLz6yQdhVElOW32MEsYUElCW6vh1lh xi1iEni3p9TrCAJNgE1ia1L2lkgGmwlFpz7wgxiiwhIS1z7fBmsmVnAQ+Ljx6VgtrCAm0T/jb 1gy1gEVCUWrlkCFucViJeY9LOHFWKOvMSutotANgcHp0CCRPOFCIgb4iV+XfwJdhsn0Kov+xc wQ8RtJH4dW880gVF4ASPDKkaN4tSistQiXUNDvaSizPSMktzEzBxdQwNTvdzU4uLE9NScxKRi veT83E2MwABlAIIdjCvbnQ8xSnIwKYnyivUmRArxJeWnVGYkFmfEF5XmpBYfYpTh4FCS4BV+n BgpJFiUmp5akZaZA4wVmLQEB4+SCK/zdKA0b3FBYm5xZjpE6hSjJceG1eu/MHFMOrAdSL6a8P 8bkxBLXn5eqpQ479tHQA0CIA0ZpXlw42DxfIlRVkqYlxHoQCGegtSi3MwSVPlXjOIcjErCvDd ApvBk5pXAbX0FdBAT0EFsdQkgB5UkIqSkGhhTDTb45C150KQZtZ/Faz2f6LX3/7Ur1Vi5Wr6m vC6cMjVrCkvzqqDiqqvV9lHPdr+z3zO54peox3oWcaHWqXwT/39VfX3jt9j5RwImZxwPvPipK 1Uyc/q0aQuDtzMqN1Y7X/zwL2TJ+juTzCvuZHX//zQ53aj3kfgvrUSm/e3RBmfWLcow2nVDia U4I9FQi7moOBEANeFCbuICAAA= X-Env-Sender: kaih.linux@gmail.com X-Msg-Ref: server-11.tower-206.messagelabs.com!1499587769!82252860!1 X-Originating-IP: [74.125.83.67] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 9.4.25; banners=-,-,- X-VirusChecked: Checked Received: (qmail 18438 invoked from network); 9 Jul 2017 08:09:30 -0000 Received: from mail-pg0-f67.google.com (HELO mail-pg0-f67.google.com) (74.125.83.67) by server-11.tower-206.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 9 Jul 2017 08:09:30 -0000 Received: by mail-pg0-f67.google.com with SMTP id j186so9086618pge.1 for ; Sun, 09 Jul 2017 01:09:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=tGMAp30e7jF5JKUeXBvBTf1HINrz/1LilizokU82uTc=; b=RZbiPfHd2dkuUGHLusigH0ibBEWJFVFsmFvu01ApoU1Rr/a/xputTOOddWzqQWaD1H 241fqgPG8Yd8emhlwTIV7LKuU/baZ/6A3dhGr+eUs0I/YN5+yzZwnLmWr2BiniaW8ZSe 7ry1huOhT49Vj7DgPjjvgvGX3ZjGg8tPcyjCDaY/uRBoZjqBLkuaA7YkWsYoP2QhKSWU Gjr7JOlJ3QQPssIVOHMKrVZF9u8Nl7MdRURYYBBY3sLD2kby+AamlWHKN9IAu6CaNuUr gTR/DroeiI5m+nAF+IVqP3z+Ny9990F3H+K1Lhd7UuIfdIdvUHR9zBesM/g7OirQs6T9 m0UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=tGMAp30e7jF5JKUeXBvBTf1HINrz/1LilizokU82uTc=; b=kxAnLIDUA5QUVXQnTf+MRvkDqli5Ppz+fdh8s4ixVcpPB0o+gMh/lIL8qfm8jZJHWM A9bjKq34nW6tz2VEUGTpGtq2AqZAXlPNxWBQc69jOuqWsv74d0+DvD3l7rUezuRRbsb4 +BgSDrss4Jb+oupb3mb4pqrcrOrYc0UJSVLrx9I3hD16HVi3f4AJOLkFOu3vGIRtgfgk 7IggxDma71loSE1RzyQPRKAwMikA7f/1IQxoLfpWSMl3m+LdwBMgJe9qVvVHgrbGIKKN x/87uc4PBS9fY+JwcGJNHv7wF5L2UYU+nTU3L68x0U+XvhMnc70ChFVKGQB0dvM152/u J+2w== X-Gm-Message-State: AIVw110jgWe02AnSYnnbGCLFEIX2nNtqgv8YjQlQLrCIlbP81J8pLeAO PFNmuYjpTV720K+K X-Received: by 10.98.66.147 with SMTP id h19mr39907579pfd.178.1499587769080; Sun, 09 Jul 2017 01:09:29 -0700 (PDT) Received: from localhost.localdomain (118-92-234-57.dsl.dyn.ihug.co.nz. [118.92.234.57]) by smtp.gmail.com with ESMTPSA id d70sm22148172pga.49.2017.07.09.01.09.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 09 Jul 2017 01:09:28 -0700 (PDT) From: Kai Huang X-Google-Original-From: Kai Huang To: xen-devel@lists.xen.org Date: Sun, 9 Jul 2017 20:09:04 +1200 Message-Id: X-Mailer: git-send-email 2.11.0 In-Reply-To: <4b8baf9779038897e6ba2ed4ac0a3e9663db2756.1499586046.git.kai.huang@linux.intel.com> References: <4b8baf9779038897e6ba2ed4ac0a3e9663db2756.1499586046.git.kai.huang@linux.intel.com> In-Reply-To: References: Cc: andrew.cooper3@citrix.com, kevin.tian@intel.com, jbeulich@suse.com Subject: [Xen-devel] [PATCH 03/15] xen: x86: add early stage SGX feature detection X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch adds early stage SGX feature detection via SGX CPUID 0x12. Function detect_sgx is added to detect SGX info on each CPU (called from vmx_cpu_up). SDM says SGX info returned by CPUID is per-thread, and we cannot assume all threads will return the same SGX info, so we have to detect SGX for each CPU. For simplicity, currently SGX is only supported when all CPUs reports the same SGX info. SDM also says it's possible to have multiple EPC sections but this is only for multiple-socket server, which we don't support now (there are other things need to be done, ex, NUMA EPC, scheduling, etc, as well), so currently only one EPC is supported. Dedicated files sgx.c and sgx.h are added (under vmx directory as SGX is Intel specific) for bulk of above SGX detection code detection code, and for further SGX code as well. Signed-off-by: Kai Huang --- xen/arch/x86/hvm/vmx/Makefile | 1 + xen/arch/x86/hvm/vmx/sgx.c | 208 ++++++++++++++++++++++++++++++++++++++ xen/arch/x86/hvm/vmx/vmcs.c | 4 + xen/include/asm-x86/cpufeature.h | 1 + xen/include/asm-x86/hvm/vmx/sgx.h | 45 +++++++++ 5 files changed, 259 insertions(+) create mode 100644 xen/arch/x86/hvm/vmx/sgx.c create mode 100644 xen/include/asm-x86/hvm/vmx/sgx.h diff --git a/xen/arch/x86/hvm/vmx/Makefile b/xen/arch/x86/hvm/vmx/Makefile index 04a29ce59d..f6bcf0d143 100644 --- a/xen/arch/x86/hvm/vmx/Makefile +++ b/xen/arch/x86/hvm/vmx/Makefile @@ -4,3 +4,4 @@ obj-y += realmode.o obj-y += vmcs.o obj-y += vmx.o obj-y += vvmx.o +obj-y += sgx.o diff --git a/xen/arch/x86/hvm/vmx/sgx.c b/xen/arch/x86/hvm/vmx/sgx.c new file mode 100644 index 0000000000..6b41469371 --- /dev/null +++ b/xen/arch/x86/hvm/vmx/sgx.c @@ -0,0 +1,208 @@ +/* + * Intel Software Guard Extensions support + * + * Author: Kai Huang + */ + +#include +#include +#include +#include +#include + +static struct sgx_cpuinfo __read_mostly sgx_cpudata[NR_CPUS]; +static struct sgx_cpuinfo __read_mostly boot_sgx_cpudata; + +static bool_t sgx_enabled_in_bios(void) +{ + uint64_t val, sgx_enabled = IA32_FEATURE_CONTROL_SGX_ENABLE | + IA32_FEATURE_CONTROL_LOCK; + + rdmsrl(MSR_IA32_FEATURE_CONTROL, val); + + return (val & sgx_enabled) == sgx_enabled; +} + +static void __detect_sgx(int cpu) +{ + struct sgx_cpuinfo *sgxinfo = &sgx_cpudata[cpu]; + u32 eax, ebx, ecx, edx; + + memset(sgxinfo, 0, sizeof(*sgxinfo)); + + /* + * In reality if SGX is not enabled in BIOS, SGX CPUID should report + * invalid SGX info, but we do the check anyway to make sure. + */ + if ( !sgx_enabled_in_bios() ) + { + printk("CPU%d: SGX disabled in BIOS.\n", cpu); + goto not_supported; + } + + /* + * CPUID.0x12.0x0: + * + * EAX [0]: whether SGX1 is supported. + * [1]: whether SGX2 is supported. + * EBX [31:0]: miscselect + * ECX [31:0]: reserved + * EDX [7:0]: MaxEnclaveSize_Not64 + * [15:8]: MaxEnclaveSize_64 + */ + cpuid_count(SGX_CPUID, 0x0, &eax, &ebx, &ecx, &edx); + sgxinfo->cap = eax & (SGX_CAP_SGX1 | SGX_CAP_SGX2); + sgxinfo->miscselect = ebx; + sgxinfo->max_enclave_size32 = edx & 0xff; + sgxinfo->max_enclave_size64 = (edx & 0xff00) >> 8; + + if ( !(eax & SGX_CAP_SGX1) ) + { + /* We may reach here if BIOS doesn't enable SGX */ + printk("CPU%d: CPUID.0x12.0x0 reports not SGX support.\n", cpu); + goto not_supported; + } + + /* + * CPUID.0x12.0x1: + * + * EAX [31:0]: bitmask of 1-setting of SECS.ATTRIBUTES[31:0] + * EBX [31:0]: bitmask of 1-setting of SECS.ATTRIBUTES[63:32] + * ECX [31:0]: bitmask of 1-setting of SECS.ATTRIBUTES[95:64] + * EDX [31:0]: bitmask of 1-setting of SECS.ATTRIBUTES[127:96] + */ + cpuid_count(SGX_CPUID, 0x1, &eax, &ebx, &ecx, &edx); + sgxinfo->secs_attr_bitmask[0] = eax; + sgxinfo->secs_attr_bitmask[1] = ebx; + sgxinfo->secs_attr_bitmask[2] = ecx; + sgxinfo->secs_attr_bitmask[3] = edx; + + /* + * CPUID.0x12.0x2: + * + * EAX [3:0]: 0000: this sub-leaf is invalid + * 0001: this sub-leaf enumerates EPC resource + * [11:4]: reserved + * [31:12]: bits 31:12 of physical address of EPC base (when + * EAX[3:0] is 0001, which applies to following) + * EBX [19:0]: bits 51:32 of physical address of EPC base + * [31:20]: reserved + * ECX [3:0]: 0000: EDX:ECX are 0 + * 0001: this is EPC section. + * [11:4]: reserved + * [31:12]: bits 31:12 of EPC size + * EDX [19:0]: bits 51:32 of EPC size + * [31:20]: reserved + * + * TODO: So far assume there's only one EPC resource. + */ + cpuid_count(SGX_CPUID, 0x2, &eax, &ebx, &ecx, &edx); + if ( !(eax & 0x1) || !(ecx & 0x1) ) + { + /* We may reach here if BIOS doesn't enable SGX */ + printk("CPU%d: CPUID.0x12.0x2 reports invalid EPC resource.\n", cpu); + goto not_supported; + } + sgxinfo->epc_base = (((u64)(ebx & 0xfffff)) << 32) | (eax & 0xfffff000); + sgxinfo->epc_size = (((u64)(edx & 0xfffff)) << 32) | (ecx & 0xfffff000); + + return; + +not_supported: + memset(sgxinfo, 0, sizeof(*sgxinfo)); +} + +void detect_sgx(int cpu) +{ + /* Caller (vmx_cpu_up) has checked cpu_has_vmx_encls */ + if ( !cpu_has_sgx || boot_cpu_data.cpuid_level < SGX_CPUID ) + { + setup_clear_cpu_cap(X86_FEATURE_SGX); + return; + } + + __detect_sgx(cpu); +} + +static void __init disable_sgx(void) +{ + memset(&boot_sgx_cpudata, 0, sizeof (struct sgx_cpuinfo)); + /* + * X86_FEATURE_SGX is cleared in boot_cpu_data so that cpu_has_sgx + * can be used anywhere to check whether SGX is supported by Xen. + * + * FIXME: also adjust boot_cpu_data.cpuid_level ? + */ + setup_clear_cpu_cap(X86_FEATURE_SGX); +} + +static void __init print_sgx_cpuinfo(struct sgx_cpuinfo *sgxinfo) +{ + printk("SGX: \n" + "\tCAP: %s,%s\n" + "\tEPC: [0x%"PRIx64", 0x%"PRIx64")\n", + boot_sgx_cpudata.cap & SGX_CAP_SGX1 ? "SGX1" : "", + boot_sgx_cpudata.cap & SGX_CAP_SGX2 ? "SGX2" : "", + boot_sgx_cpudata.epc_base, + boot_sgx_cpudata.epc_base + boot_sgx_cpudata.epc_size); +} + +/* + * Check SGX CPUID info all for all CPUs, and only support SGX when all CPUs + * report the same SGX info. SDM (37.7.2 Intel SGX Resource Enumeration Leaves) + * says "software should not assume that if Intel SGX instructions are + * supported on one hardware thread, they are also supported elsewhere.". + * For simplicity, we only support SGX when all CPUs reports consistent SGX + * info. + * + * boot_sgx_cpudata is set to store the *common* SGX CPUID info. + */ +static bool_t __init check_sgx_consistency(void) +{ + int i; + + for_each_online_cpu ( i ) + { + struct sgx_cpuinfo *s = &sgx_cpudata[i]; + + if ( memcmp(&boot_sgx_cpudata, s, sizeof (*s)) ) + { + printk("SGX inconsistency between CPU 0 and CPU %d. " + "Disable SGX.\n", i); + memset(&boot_sgx_cpudata, 0, sizeof (*s)); + return false; + } + } + + return true; +} + +static int __init sgx_init(void) +{ + /* Assume CPU 0 is always online */ + boot_sgx_cpudata = sgx_cpudata[0]; + + if ( !(boot_sgx_cpudata.cap & SGX_CAP_SGX1) ) + goto not_supported; + + if ( !check_sgx_consistency() ) + goto not_supported; + + print_sgx_cpuinfo(&boot_sgx_cpudata); + + return 0; +not_supported: + disable_sgx(); + return -EINVAL; +} +__initcall(sgx_init); + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c index ae7e6f9321..518133bbfd 100644 --- a/xen/arch/x86/hvm/vmx/vmcs.c +++ b/xen/arch/x86/hvm/vmx/vmcs.c @@ -40,6 +40,7 @@ #include #include #include +#include static bool_t __read_mostly opt_vpid_enabled = 1; boolean_param("vpid", opt_vpid_enabled); @@ -696,6 +697,9 @@ int vmx_cpu_up(void) vmx_pi_per_cpu_init(cpu); + if ( cpu_has_vmx_encls ) + detect_sgx(cpu); + return 0; } diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h index 84cc51d2bd..9793f8c1c5 100644 --- a/xen/include/asm-x86/cpufeature.h +++ b/xen/include/asm-x86/cpufeature.h @@ -85,6 +85,7 @@ /* CPUID level 0x00000007:0.ebx */ #define cpu_has_fsgsbase boot_cpu_has(X86_FEATURE_FSGSBASE) +#define cpu_has_sgx boot_cpu_has(X86_FEATURE_SGX) #define cpu_has_bmi1 boot_cpu_has(X86_FEATURE_BMI1) #define cpu_has_hle boot_cpu_has(X86_FEATURE_HLE) #define cpu_has_avx2 boot_cpu_has(X86_FEATURE_AVX2) diff --git a/xen/include/asm-x86/hvm/vmx/sgx.h b/xen/include/asm-x86/hvm/vmx/sgx.h new file mode 100644 index 0000000000..5414d8237e --- /dev/null +++ b/xen/include/asm-x86/hvm/vmx/sgx.h @@ -0,0 +1,45 @@ +/* + * Intel Software Guard Extensions support + * + * Copyright (c) 2016, Intel Corporation. + * + * Author: Kai Huang + */ +#ifndef __ASM_X86_HVM_VMX_SGX_H__ +#define __ASM_X86_HVM_VMX_SGX_H__ + +#include +#include +#include +#include + +#define SGX_CPUID 0x12 + +/* + * SGX info reported by SGX CPUID. + * + * TODO: + * + * SDM (37.7.2 Intel SGX Resource Enumeration Leaves) actually says it's + * possible there are multiple EPC resources on the machine (CPUID.0x12, + * ECX starting with 0x2 enumerates available EPC resources until invalid + * EPC resource is returned). But this is only for multiple socket server, + * which we current don't support now (there are additional things need to + * be done as well). So far for simplicity we assume there is only one EPC. + */ +struct sgx_cpuinfo { +#define SGX_CAP_SGX1 (1UL << 0) +#define SGX_CAP_SGX2 (1UL << 1) + uint32_t cap; + uint32_t miscselect; + uint8_t max_enclave_size64; + uint8_t max_enclave_size32; + uint32_t secs_attr_bitmask[4]; + uint64_t epc_base; + uint64_t epc_size; +}; + +/* Detect SGX info for particular CPU via SGX CPUID */ +void detect_sgx(int cpu); + +#endif /* __ASM_X86_HVM_VMX_SGX_H__ */