From patchwork Fri Nov 6 01:16:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DEA2C4742C for ; Fri, 6 Nov 2020 01:06:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DF02D2078E for ; Fri, 6 Nov 2020 01:06:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732920AbgKFBGP (ORCPT ); Thu, 5 Nov 2020 20:06:15 -0500 Received: from mga18.intel.com ([134.134.136.126]:38177 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730895AbgKFBGP (ORCPT ); Thu, 5 Nov 2020 20:06:15 -0500 IronPort-SDR: JAJPrEjjqNDGIO+MnRXRmQFygsnShQDMhaL6YT++RSqt2/U/YN56vRPKmuO18shbFZmfDrqxMF LOk680jaWUQw== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264655" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264655" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:14 -0800 IronPort-SDR: Q4hvVxaRqSr/I/a5GXXIZA7j94gfYRLGTF9YOG3AjfbmpoA6FRvheyuvAzfvslRseOQtHGIxeo kTkDXvXjmXQQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874430" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:12 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com Subject: [PATCH v14 01/13] KVM: x86: Report XSS as an MSR to be saved if there are supported features Date: Fri, 6 Nov 2020 09:16:25 +0800 Message-Id: <20201106011637.14289-2-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Sean Christopherson Add MSR_IA32_XSS to the list of MSRs reported to userspace if supported_xss is non-zero, i.e. KVM supports at least one XSS based feature. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 397f599b20e5..fce18a2537ea 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1237,6 +1237,8 @@ static const u32 msrs_to_save_all[] = { MSR_ARCH_PERFMON_EVENTSEL0 + 12, MSR_ARCH_PERFMON_EVENTSEL0 + 13, MSR_ARCH_PERFMON_EVENTSEL0 + 14, MSR_ARCH_PERFMON_EVENTSEL0 + 15, MSR_ARCH_PERFMON_EVENTSEL0 + 16, MSR_ARCH_PERFMON_EVENTSEL0 + 17, + + MSR_IA32_XSS, }; static u32 msrs_to_save[ARRAY_SIZE(msrs_to_save_all)]; @@ -5728,6 +5730,10 @@ static void kvm_init_msr_list(void) min(INTEL_PMC_MAX_GENERIC, x86_pmu.num_counters_gp)) continue; break; + case MSR_IA32_XSS: + if (!supported_xss) + continue; + break; default: break; } From patchwork Fri Nov 6 01:16:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A36CFC55178 for ; Fri, 6 Nov 2020 01:07:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5F6A12075A for ; Fri, 6 Nov 2020 01:07:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733046AbgKFBGV (ORCPT ); Thu, 5 Nov 2020 20:06:21 -0500 Received: from mga18.intel.com ([134.134.136.126]:38177 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732838AbgKFBGR (ORCPT ); Thu, 5 Nov 2020 20:06:17 -0500 IronPort-SDR: A9Hpz6ytA5JxCwj9vYJCg/547eOuo/YVrCfaMalTeArw7Pz6uoCTq/5UsDKZfPHK+bg3ejnAE4 oW5AXUz/zbcg== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264660" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264660" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:16 -0800 IronPort-SDR: PB2ijGxdsWcgfgTUKQ+StnqpC9Cpk7tzCEkJHr2OJAmmnBzruJsYnuJ6g4crt64l2JPHQwU7fU hlwhUwVmzXsw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874441" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:14 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v14 02/13] KVM: x86: Refresh CPUID on writes to MSR_IA32_XSS Date: Fri, 6 Nov 2020 09:16:26 +0800 Message-Id: <20201106011637.14289-3-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Updated CPUID.0xD.0x1, which reports the current required storage size of all features enabled via XCR0 | XSS, when the guest's XSS is modified. Note, KVM does not yet support any XSS based features, i.e. supported_xss is guaranteed to be zero at this time. Co-developed-by: Zhang Yi Z Signed-off-by: Zhang Yi Z Signed-off-by: Yang Weijiang --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/cpuid.c | 21 ++++++++++++++++++--- arch/x86/kvm/x86.c | 7 +++++-- 3 files changed, 24 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index d44858b69353..1620a2cca781 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -611,6 +611,7 @@ struct kvm_vcpu_arch { u64 xcr0; u64 guest_supported_xcr0; + u64 guest_supported_xss; struct kvm_pio_request pio; void *pio_data; diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 06a278b3701d..2c737337f466 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -115,9 +115,24 @@ void kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu) best->ebx = xstate_required_size(vcpu->arch.xcr0, false); best = kvm_find_cpuid_entry(vcpu, 0xD, 1); - if (best && (cpuid_entry_has(best, X86_FEATURE_XSAVES) || - cpuid_entry_has(best, X86_FEATURE_XSAVEC))) - best->ebx = xstate_required_size(vcpu->arch.xcr0, true); + if (best) { + if (cpuid_entry_has(best, X86_FEATURE_XSAVES) || + cpuid_entry_has(best, X86_FEATURE_XSAVEC)) { + u64 xstate = vcpu->arch.xcr0 | vcpu->arch.ia32_xss; + + best->ebx = xstate_required_size(xstate, true); + } + + if (!cpuid_entry_has(best, X86_FEATURE_XSAVES)) { + best->ecx = 0; + best->edx = 0; + } + vcpu->arch.guest_supported_xss = + (((u64)best->edx << 32) | best->ecx) & supported_xss; + + } else { + vcpu->arch.guest_supported_xss = 0; + } best = kvm_find_cpuid_entry(vcpu, KVM_CPUID_FEATURES, 0); if (kvm_hlt_in_guest(vcpu->kvm) && best && diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index fce18a2537ea..0823bf1e2977 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3124,9 +3124,12 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) * IA32_XSS[bit 8]. Guests have to use RDMSR/WRMSR rather than * XSAVES/XRSTORS to save/restore PT MSRs. */ - if (data & ~supported_xss) + if (data & ~vcpu->arch.guest_supported_xss) return 1; - vcpu->arch.ia32_xss = data; + if (vcpu->arch.ia32_xss != data) { + vcpu->arch.ia32_xss = data; + kvm_update_cpuid_runtime(vcpu); + } break; case MSR_SMI_COUNT: if (!msr_info->host_initiated) From patchwork Fri Nov 6 01:16:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885639 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DC79C4742C for ; Fri, 6 Nov 2020 01:07:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4B44E2075A for ; Fri, 6 Nov 2020 01:07:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733005AbgKFBGU (ORCPT ); Thu, 5 Nov 2020 20:06:20 -0500 Received: from mga18.intel.com ([134.134.136.126]:38182 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732964AbgKFBGT (ORCPT ); Thu, 5 Nov 2020 20:06:19 -0500 IronPort-SDR: LaMx+uLJ2DeIp6rHtrmKfMFJBKsn714lzNkRjbO0u9vOs99vhvtg1wzGbIOwJokIZup78/dC/h HemPgee7wVzQ== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264665" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264665" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:18 -0800 IronPort-SDR: KSLUCBpBK+6hLApwjYes3WGPgU1Tx46WahCnnrzhVEfvEOOrpnFBVVAB9qRycq3F9K/8J/jJqG QIPM2qpxng7g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874453" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:16 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v14 03/13] KVM: x86: Add #CP support in guest exception dispatch Date: Fri, 6 Nov 2020 09:16:27 +0800 Message-Id: <20201106011637.14289-4-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add handling for Control Protection (#CP) exceptions, vector 21, used and introduced by Intel's Control-Flow Enforcement Technology (CET). relevant CET violation case. See Intel's SDM for details. Signed-off-by: Yang Weijiang --- arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/x86.c | 1 + arch/x86/kvm/x86.h | 2 +- 3 files changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 89e5f3d1bba8..9dc2c58d894b 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -31,6 +31,7 @@ #define MC_VECTOR 18 #define XM_VECTOR 19 #define VE_VECTOR 20 +#define CP_VECTOR 21 /* Select x86 specific features in */ #define __KVM_HAVE_PIT diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 0823bf1e2977..0433015ee443 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -433,6 +433,7 @@ static int exception_class(int vector) case NP_VECTOR: case SS_VECTOR: case GP_VECTOR: + case CP_VECTOR: return EXCPT_CONTRIBUTORY; default: break; diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 3900ab0c6004..74858c18978a 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -115,7 +115,7 @@ static inline bool x86_exception_has_error_code(unsigned int vector) { static u32 exception_has_error_code = BIT(DF_VECTOR) | BIT(TS_VECTOR) | BIT(NP_VECTOR) | BIT(SS_VECTOR) | BIT(GP_VECTOR) | - BIT(PF_VECTOR) | BIT(AC_VECTOR); + BIT(PF_VECTOR) | BIT(AC_VECTOR) | BIT(CP_VECTOR); return (1U << vector) & exception_has_error_code; } From patchwork Fri Nov 6 01:16:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885637 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37058C4742C for ; Fri, 6 Nov 2020 01:07:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ED83E20782 for ; Fri, 6 Nov 2020 01:07:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387433AbgKFBHS (ORCPT ); Thu, 5 Nov 2020 20:07:18 -0500 Received: from mga18.intel.com ([134.134.136.126]:38191 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733032AbgKFBGV (ORCPT ); Thu, 5 Nov 2020 20:06:21 -0500 IronPort-SDR: e9zGetl6oDXxIqs5wGq9YPifQlFF0AAB9XJ0KzFfJIwybosSLLH+2u/5tRGs+dEotZyTbq5FKO +Mx5+no2KGUg== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264673" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264673" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:20 -0800 IronPort-SDR: qU4cLHOb7IVhc+5YJ9o9ncykJS0Dh9apDZGJUaaezw4rbFwdsavCZHefIp5NIGw6U6B+RKS/B8 /aibKoRm8Zqw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874471" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:18 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v14 04/13] KVM: VMX: Introduce CET VMCS fields and flags Date: Fri, 6 Nov 2020 09:16:28 +0800 Message-Id: <20201106011637.14289-5-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org CET (Control-flow Enforcement Technology) is a CPU feature used to prevent Return/Jump-Oriented Programming (ROP/JOP) attacks. CET introduces a new exception type, Control Protection (#CP), and two sub-features to defend against ROP/JOP style control-flow subversion attacks: Shadow Stack (SHSTK): A shadow stack is a second stack used exclusively for control transfer operations. The shadow stack is separate from the data/normal stack and can be enabled individually in user and kernel mode. When shadow stacks are enabled, CALL pushes the return address on both the data and shadow stack. RET pops the return address from both stacks and compares them. If the return addresses from the two stacks do not match, the processor signals a #CP. Indirect Branch Tracking (IBT): IBT adds a new instrution, ENDBRANCH, that is used to mark valid target addresses of indirect branches (CALL, JMP, ENCLU[EEXIT], etc...). If an indirect branch is executed and the next instruction is _not_ an ENDBRANCH, the processor signals a #CP. Several new CET MSRs are defined to support CET: MSR_IA32_{U,S}_CET: Controls the CET settings for user mode and kernel mode respectively. MSR_IA32_PL{0,1,2,3}_SSP: Stores shadow stack pointers for CPL-0,1,2,3 protection respectively. MSR_IA32_INT_SSP_TAB: Stores base address of shadow stack pointer table. Two XSAVES state bits are introduced for CET: IA32_XSS:[bit 11]: Control saving/restoring user mode CET states IA32_XSS:[bit 12]: Control saving/restoring kernel mode CET states. Six VMCS fields are introduced for CET: {HOST,GUEST}_S_CET: Stores CET settings for kernel mode. {HOST,GUEST}_SSP: Stores shadow stack pointer of current task/thread. {HOST,GUEST}_INTR_SSP_TABLE: Stores base address of shadow stack pointer table. If VM_EXIT_LOAD_HOST_CET_STATE = 1, the host CET states are restored from the following VMCS fields at VM-Exit: HOST_S_CET HOST_SSP HOST_INTR_SSP_TABLE If VM_ENTRY_LOAD_GUEST_CET_STATE = 1, the guest CET states are loaded from the following VMCS fields at VM-Entry: GUEST_S_CET GUEST_SSP GUEST_INTR_SSP_TABLE Co-developed-by: Zhang Yi Z Signed-off-by: Zhang Yi Z Signed-off-by: Yang Weijiang --- arch/x86/include/asm/vmx.h | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index f8ba5289ecb0..2b1c55f32d5b 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -94,6 +94,7 @@ #define VM_EXIT_CLEAR_BNDCFGS 0x00800000 #define VM_EXIT_PT_CONCEAL_PIP 0x01000000 #define VM_EXIT_CLEAR_IA32_RTIT_CTL 0x02000000 +#define VM_EXIT_LOAD_CET_STATE 0x10000000 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR 0x00036dff @@ -107,6 +108,7 @@ #define VM_ENTRY_LOAD_BNDCFGS 0x00010000 #define VM_ENTRY_PT_CONCEAL_PIP 0x00020000 #define VM_ENTRY_LOAD_IA32_RTIT_CTL 0x00040000 +#define VM_ENTRY_LOAD_CET_STATE 0x00100000 #define VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR 0x000011ff @@ -328,6 +330,9 @@ enum vmcs_field { GUEST_PENDING_DBG_EXCEPTIONS = 0x00006822, GUEST_SYSENTER_ESP = 0x00006824, GUEST_SYSENTER_EIP = 0x00006826, + GUEST_S_CET = 0x00006828, + GUEST_SSP = 0x0000682a, + GUEST_INTR_SSP_TABLE = 0x0000682c, HOST_CR0 = 0x00006c00, HOST_CR3 = 0x00006c02, HOST_CR4 = 0x00006c04, @@ -340,6 +345,9 @@ enum vmcs_field { HOST_IA32_SYSENTER_EIP = 0x00006c12, HOST_RSP = 0x00006c14, HOST_RIP = 0x00006c16, + HOST_S_CET = 0x00006c18, + HOST_SSP = 0x00006c1a, + HOST_INTR_SSP_TABLE = 0x00006c1c }; /* From patchwork Fri Nov 6 01:16:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04D6AC5517A for ; Fri, 6 Nov 2020 01:07:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B45A520759 for ; Fri, 6 Nov 2020 01:07:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733107AbgKFBGY (ORCPT ); Thu, 5 Nov 2020 20:06:24 -0500 Received: from mga18.intel.com ([134.134.136.126]:38191 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730895AbgKFBGW (ORCPT ); Thu, 5 Nov 2020 20:06:22 -0500 IronPort-SDR: DVkEKgvm2WiAWYdcx8h0+yBS4aafl2S+MVGA7qoWoAYuCBQpYoe+wS/O5e9BA2Im+yOg2abPIs 0XUgF0KR5cIg== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264685" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264685" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:22 -0800 IronPort-SDR: JTtHd9+0mqb8zk8b9juZcRAEKQ/ZnNgsViHuSkKWhjfgdrST5m0d9/Qm7dTSr2jcfGiXnnWboJ CZxv8jTVLuCg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874488" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:20 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v14 05/13] KVM: x86: Add fault checks for CR4.CET Date: Fri, 6 Nov 2020 09:16:29 +0800 Message-Id: <20201106011637.14289-6-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add the fault checks for CR4.CET, which is the master control for all CET features (SHSTK and IBT). In addition to basic support checks, CET can be enabled if and only if CR0.WP==1, i.e. setting CR4.CET=1 faults if CR0.WP==0 and setting CR0.WP=0 fails if CR4.CET==1. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang Reported-by: kernel test robot --- arch/x86/kvm/x86.c | 6 ++++++ arch/x86/kvm/x86.h | 3 +++ 2 files changed, 9 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 0433015ee443..8c9d631d7842 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -847,6 +847,9 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) if (!(cr0 & X86_CR0_PG) && kvm_read_cr4_bits(vcpu, X86_CR4_PCIDE)) return 1; + if (!(cr0 & X86_CR0_WP) && kvm_read_cr4_bits(vcpu, X86_CR4_CET)) + return 1; + kvm_x86_ops.set_cr0(vcpu, cr0); if ((cr0 ^ old_cr0) & X86_CR0_PG) { @@ -1008,6 +1011,9 @@ int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) return 1; } + if ((cr4 & X86_CR4_CET) && !(kvm_read_cr0(vcpu) & X86_CR0_WP)) + return 1; + if (kvm_x86_ops.set_cr4(vcpu, cr4)) return 1; diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 74858c18978a..50386318a382 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -398,6 +398,9 @@ bool kvm_msr_allowed(struct kvm_vcpu *vcpu, u32 index, u32 type); __reserved_bits |= X86_CR4_UMIP; \ if (!__cpu_has(__c, X86_FEATURE_VMX)) \ __reserved_bits |= X86_CR4_VMXE; \ + if (!__cpu_has(__c, X86_FEATURE_SHSTK) && \ + !__cpu_has(__c, X86_FEATURE_IBT)) \ + __reserved_bits |= X86_CR4_CET; \ __reserved_bits; \ }) From patchwork Fri Nov 6 01:16:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1661C4742C for ; Fri, 6 Nov 2020 01:07:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A5BBC2075A for ; Fri, 6 Nov 2020 01:07:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733139AbgKFBG0 (ORCPT ); Thu, 5 Nov 2020 20:06:26 -0500 Received: from mga18.intel.com ([134.134.136.126]:38191 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733109AbgKFBGZ (ORCPT ); Thu, 5 Nov 2020 20:06:25 -0500 IronPort-SDR: L10PPhAzhW2FHb7UYIxNs/uAeEiqmMc+gXMJZNMz/oqvier7D18zzkE/HhLt67xKt1HVRuTNkN A4PkgQoGlXbA== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264689" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264689" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:24 -0800 IronPort-SDR: RZaPzcy5WY8uihHD3lffneYXEMAvzjFTpTnXGZTN9o2YCWm8VH2rPaBju5fWe/ScJyU/zvzdEP o6BLvmiAf1yA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874502" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:22 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v14 06/13] KVM: x86: Load guest fpu state when accessing MSRs managed by XSAVES Date: Fri, 6 Nov 2020 09:16:30 +0800 Message-Id: <20201106011637.14289-7-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Sean Christopherson A handful of CET MSRs are not context switched through "traditional" methods, e.g. VMCS or manual switching, but rather are passed through to the guest and are saved and restored by XSAVES/XRSTORS, i.e. in the guest's FPU state. Load the guest's FPU state if userspace is accessing MSRs whose values are managed by XSAVES so that the MSR helper, e.g. vmx_{get,set}_msr(), can simply do {RD,WR}MSR to access the guest's value. Because is also used for the KVM_GET_MSRS device ioctl(), explicitly check that @vcpu is non-null before attempting to load guest state. The CET MSRs cannot be retrieved via the device ioctl() without loading guest FPU state (which doesn't exist). Note that guest_cpuid_has() is not queried as host userspace is allowed to access MSRs that have not been exposed to the guest, e.g. it might do KVM_SET_MSRS prior to KVM_SET_CPUID2. Signed-off-by: Sean Christopherson Co-developed-by: Yang Weijiang Signed-off-by: Yang Weijiang --- arch/x86/kvm/x86.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 8c9d631d7842..751b62e871e5 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -109,6 +109,8 @@ static void enter_smm(struct kvm_vcpu *vcpu); static void __kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags); static void store_regs(struct kvm_vcpu *vcpu); static int sync_regs(struct kvm_vcpu *vcpu); +static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu); +static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu); struct kvm_x86_ops kvm_x86_ops __read_mostly; EXPORT_SYMBOL_GPL(kvm_x86_ops); @@ -3582,6 +3584,12 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) } EXPORT_SYMBOL_GPL(kvm_get_msr_common); +static bool is_xsaves_msr(u32 index) +{ + return index == MSR_IA32_U_CET || + (index >= MSR_IA32_PL0_SSP && index <= MSR_IA32_PL3_SSP); +} + /* * Read or write a bunch of msrs. All parameters are kernel addresses. * @@ -3592,11 +3600,20 @@ static int __msr_io(struct kvm_vcpu *vcpu, struct kvm_msrs *msrs, int (*do_msr)(struct kvm_vcpu *vcpu, unsigned index, u64 *data)) { + bool fpu_loaded = false; int i; - for (i = 0; i < msrs->nmsrs; ++i) + for (i = 0; i < msrs->nmsrs; ++i) { + if (vcpu && !fpu_loaded && supported_xss && + is_xsaves_msr(entries[i].index)) { + kvm_load_guest_fpu(vcpu); + fpu_loaded = true; + } if (do_msr(vcpu, entries[i].index, &entries[i].data)) break; + } + if (fpu_loaded) + kvm_put_guest_fpu(vcpu); return i; } From patchwork Fri Nov 6 01:16:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885617 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1389C55178 for ; Fri, 6 Nov 2020 01:06:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 844D020759 for ; Fri, 6 Nov 2020 01:06:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733163AbgKFBGa (ORCPT ); Thu, 5 Nov 2020 20:06:30 -0500 Received: from mga18.intel.com ([134.134.136.126]:38191 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733072AbgKFBG3 (ORCPT ); Thu, 5 Nov 2020 20:06:29 -0500 IronPort-SDR: ZicaO5qdlZqfiE1OM22QIo4f8RESe/4HYDqX2k5M+d16DZ+jJST+WxLAbG+tdRtckcHN5/btec urK1Cm4eZSZg== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264700" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264700" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:28 -0800 IronPort-SDR: 7ScEavtQZnmvzchfgxl69/h7dA0cgyuuULnzEcP+BwNFJTXgWpznkRXnBzzvHkxgyYo28hIUdn lKw337bec8WQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874508" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:24 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v14 07/13] KVM: VMX: Emulate reads and writes to CET MSRs Date: Fri, 6 Nov 2020 09:16:31 +0800 Message-Id: <20201106011637.14289-8-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add support for emulating read and write accesses to CET MSRs. CET MSRs are universally "special" as they are either context switched via dedicated VMCS fields or via XSAVES, i.e. no additional in-memory tracking is needed, but emulated reads/writes are more expensive. MSRs that are switched through XSAVES are especially annoying due to the possibility of the kernel's FPU being used in IRQ context. Disable IRQs and ensure the guest's FPU state is loaded when accessing such MSRs. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang --- arch/x86/kvm/vmx/vmx.c | 105 +++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.h | 5 ++ 2 files changed, 110 insertions(+) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index aef73dd3de4f..dd78d3a79e79 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1788,6 +1788,66 @@ static int vmx_get_msr_feature(struct kvm_msr_entry *msr) } } +static void vmx_get_xsave_msr(struct msr_data *msr_info) +{ + local_irq_disable(); + if (test_thread_flag(TIF_NEED_FPU_LOAD)) + switch_fpu_return(); + rdmsrl(msr_info->index, msr_info->data); + local_irq_enable(); +} + +static void vmx_set_xsave_msr(struct msr_data *msr_info) +{ + local_irq_disable(); + if (test_thread_flag(TIF_NEED_FPU_LOAD)) + switch_fpu_return(); + wrmsrl(msr_info->index, msr_info->data); + local_irq_enable(); +} + +static bool cet_is_ssp_msr_accessible(struct kvm_vcpu *vcpu, + struct msr_data *msr) +{ + u64 mask; + + if (!kvm_cet_supported()) + return false; + + if (msr->host_initiated) + return true; + + if (!guest_cpuid_has(vcpu, X86_FEATURE_SHSTK)) + return false; + + if (msr->index == MSR_IA32_INT_SSP_TAB) + return false; + + mask = (msr->index == MSR_IA32_PL3_SSP) ? XFEATURE_MASK_CET_USER : + XFEATURE_MASK_CET_KERNEL; + return !!(vcpu->arch.guest_supported_xss & mask); +} + +static bool cet_is_control_msr_accessible(struct kvm_vcpu *vcpu, + struct msr_data *msr) +{ + u64 mask; + + if (!kvm_cet_supported()) + return false; + + if (msr->host_initiated) + return true; + + if (!guest_cpuid_has(vcpu, X86_FEATURE_SHSTK) && + !guest_cpuid_has(vcpu, X86_FEATURE_IBT)) + return false; + + mask = (msr->index == MSR_IA32_U_CET) ? XFEATURE_MASK_CET_USER : + XFEATURE_MASK_CET_KERNEL; + return !!(vcpu->arch.guest_supported_xss & mask); +} + /* * Reads an msr value (of 'msr_index') into 'pdata'. * Returns 0 on success, non-0 otherwise. @@ -1920,6 +1980,26 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) else msr_info->data = vmx->pt_desc.guest.addr_a[index / 2]; break; + case MSR_IA32_S_CET: + if (!cet_is_control_msr_accessible(vcpu, msr_info)) + return 1; + msr_info->data = vmcs_readl(GUEST_S_CET); + break; + case MSR_IA32_U_CET: + if (!cet_is_control_msr_accessible(vcpu, msr_info)) + return 1; + vmx_get_xsave_msr(msr_info); + break; + case MSR_IA32_INT_SSP_TAB: + if (!cet_is_ssp_msr_accessible(vcpu, msr_info)) + return 1; + msr_info->data = vmcs_readl(GUEST_INTR_SSP_TABLE); + break; + case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + if (!cet_is_ssp_msr_accessible(vcpu, msr_info)) + return 1; + vmx_get_xsave_msr(msr_info); + break; case MSR_TSC_AUX: if (!msr_info->host_initiated && !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP)) @@ -2189,6 +2269,31 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) else vmx->pt_desc.guest.addr_a[index / 2] = data; break; + case MSR_IA32_S_CET: + case MSR_IA32_U_CET: + if (!cet_is_control_msr_accessible(vcpu, msr_info)) + return 1; + if (data & GENMASK(9, 6)) + return 1; + if (msr_index == MSR_IA32_S_CET) + vmcs_writel(GUEST_S_CET, data); + else + vmx_set_xsave_msr(msr_info); + break; + case MSR_IA32_INT_SSP_TAB: + if (!cet_is_control_msr_accessible(vcpu, msr_info)) + return 1; + if (is_noncanonical_address(data, vcpu)) + return 1; + vmcs_writel(GUEST_INTR_SSP_TABLE, data); + break; + case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + if (!cet_is_ssp_msr_accessible(vcpu, msr_info)) + return 1; + if ((data & GENMASK(2, 0)) || is_noncanonical_address(data, vcpu)) + return 1; + vmx_set_xsave_msr(msr_info); + break; case MSR_TSC_AUX: if (!msr_info->host_initiated && !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP)) diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 50386318a382..d05c3d11161e 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -286,6 +286,11 @@ static inline bool kvm_mpx_supported(void) == (XFEATURE_MASK_BNDREGS | XFEATURE_MASK_BNDCSR); } +static inline bool kvm_cet_supported(void) +{ + return supported_xss & XFEATURE_MASK_CET_USER; +} + extern unsigned int min_timer_period_us; extern bool enable_vmware_backdoor; From patchwork Fri Nov 6 01:16:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885619 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1D4FC388F7 for ; Fri, 6 Nov 2020 01:07:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 675B920759 for ; Fri, 6 Nov 2020 01:07:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730895AbgKFBHE (ORCPT ); Thu, 5 Nov 2020 20:07:04 -0500 Received: from mga18.intel.com ([134.134.136.126]:38191 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733170AbgKFBGb (ORCPT ); Thu, 5 Nov 2020 20:06:31 -0500 IronPort-SDR: oSQ9i817q+zr6wLTDS3yb5K8TPur5XPlcV4Ms2Zdz1oK4Luee9BhonEIxTvakfbCidg4uubSXb 5w0yYvHygX0w== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264703" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264703" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:30 -0800 IronPort-SDR: CZAhIzJEVwQhOlWEoWIlHUhCB8mv7hmduS/uifpTFDSqLzvjIwrI3OQXiYEuknpq/PFE9wxhLi oyGIqDE4qwDQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874514" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:28 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v14 08/13] KVM: VMX: Add a synthetic MSR to allow userspace VMM to access GUEST_SSP Date: Fri, 6 Nov 2020 09:16:32 +0800 Message-Id: <20201106011637.14289-9-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Introduce a host-only synthetic MSR, MSR_KVM_GUEST_SSP so that the VMM can read/write the guest's SSP, e.g. to migrate CET state. Use a synthetic MSR, e.g. as opposed to a VCPU_REG_, as GUEST_SSP is subject to the same consistency checks as the PL*_SSP MSRs, i.e. can share code. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang --- arch/x86/include/uapi/asm/kvm_para.h | 1 + arch/x86/kvm/vmx/vmx.c | 14 ++++++++++++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h index 812e9b4c1114..5203dc084125 100644 --- a/arch/x86/include/uapi/asm/kvm_para.h +++ b/arch/x86/include/uapi/asm/kvm_para.h @@ -53,6 +53,7 @@ #define MSR_KVM_POLL_CONTROL 0x4b564d05 #define MSR_KVM_ASYNC_PF_INT 0x4b564d06 #define MSR_KVM_ASYNC_PF_ACK 0x4b564d07 +#define MSR_KVM_GUEST_SSP 0x4b564d08 struct kvm_steal_time { __u64 steal; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index dd78d3a79e79..28ba8414a7a3 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1817,7 +1817,8 @@ static bool cet_is_ssp_msr_accessible(struct kvm_vcpu *vcpu, if (msr->host_initiated) return true; - if (!guest_cpuid_has(vcpu, X86_FEATURE_SHSTK)) + if (!guest_cpuid_has(vcpu, X86_FEATURE_SHSTK) || + msr->index == MSR_KVM_GUEST_SSP) return false; if (msr->index == MSR_IA32_INT_SSP_TAB) @@ -1995,6 +1996,11 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) return 1; msr_info->data = vmcs_readl(GUEST_INTR_SSP_TABLE); break; + case MSR_KVM_GUEST_SSP: + if (!cet_is_ssp_msr_accessible(vcpu, msr_info)) + return 1; + msr_info->data = vmcs_readl(GUEST_SSP); + break; case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: if (!cet_is_ssp_msr_accessible(vcpu, msr_info)) return 1; @@ -2287,12 +2293,16 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) return 1; vmcs_writel(GUEST_INTR_SSP_TABLE, data); break; + case MSR_KVM_GUEST_SSP: case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: if (!cet_is_ssp_msr_accessible(vcpu, msr_info)) return 1; if ((data & GENMASK(2, 0)) || is_noncanonical_address(data, vcpu)) return 1; - vmx_set_xsave_msr(msr_info); + if (msr_index == MSR_KVM_GUEST_SSP) + vmcs_writel(GUEST_SSP, data); + else + vmx_set_xsave_msr(msr_info); break; case MSR_TSC_AUX: if (!msr_info->host_initiated && From patchwork Fri Nov 6 01:16:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A4C5C388F7 for ; Fri, 6 Nov 2020 01:07:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E6C7C20759 for ; Fri, 6 Nov 2020 01:07:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733193AbgKFBGe (ORCPT ); Thu, 5 Nov 2020 20:06:34 -0500 Received: from mga18.intel.com ([134.134.136.126]:38191 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733138AbgKFBGd (ORCPT ); Thu, 5 Nov 2020 20:06:33 -0500 IronPort-SDR: DHhCoO8Oboy8EctR1TdeVd0njltL41IpMaXgRzxd33zhymWJvv/3vCtfsb9+tPFNAsq4PxPBru xBm5LXObslWg== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264709" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264709" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:32 -0800 IronPort-SDR: zNjkvGqVCsAUQu7ianTu/Nn/4Ia3SDi+35/hz0sKkbdmGsSFoqaHvO/K8PLEIM0jG9fw24TNzt 6AGpUEUP/nWw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874528" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:30 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v14 09/13] KVM: x86: Report CET MSRs as to-be-saved if CET is supported Date: Fri, 6 Nov 2020 09:16:33 +0800 Message-Id: <20201106011637.14289-10-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Report all CET MSRs, including the synthetic GUEST_SSP MSR, as to-be-saved, e.g. for migration, if CET is supported by KVM. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang --- arch/x86/kvm/x86.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 751b62e871e5..d573cadf5baf 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1248,6 +1248,8 @@ static const u32 msrs_to_save_all[] = { MSR_ARCH_PERFMON_EVENTSEL0 + 16, MSR_ARCH_PERFMON_EVENTSEL0 + 17, MSR_IA32_XSS, + MSR_IA32_U_CET, MSR_IA32_S_CET, MSR_IA32_INT_SSP_TAB, MSR_KVM_GUEST_SSP, + MSR_IA32_PL0_SSP, MSR_IA32_PL1_SSP, MSR_IA32_PL2_SSP, MSR_IA32_PL3_SSP, }; static u32 msrs_to_save[ARRAY_SIZE(msrs_to_save_all)]; @@ -5761,6 +5763,13 @@ static void kvm_init_msr_list(void) if (!supported_xss) continue; break; + case MSR_IA32_U_CET: + case MSR_IA32_S_CET: + case MSR_IA32_INT_SSP_TAB: + case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + if (!kvm_cet_supported()) + continue; + break; default: break; } From patchwork Fri Nov 6 01:16:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2FC7C4742C for ; Fri, 6 Nov 2020 01:06:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8898F2075A for ; Fri, 6 Nov 2020 01:06:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733208AbgKFBGg (ORCPT ); Thu, 5 Nov 2020 20:06:36 -0500 Received: from mga18.intel.com ([134.134.136.126]:38191 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733197AbgKFBGf (ORCPT ); Thu, 5 Nov 2020 20:06:35 -0500 IronPort-SDR: W3yLWLwXCozOTqUs9IKy9HPnK4OVwoetx85udnAPUHSzBAwJEkZeD1d5o0NYbAYTEm8wjgYWt1 ZlCvzSpa9YyA== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264711" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264711" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:34 -0800 IronPort-SDR: r0+OHauq2PfQBI4YHMUZkciIcs6e2RJZYKI3nywGVbbIb6aRqX0/u/B9ZMdSDv9jsz2C4GQyxY 1Kqfw6RQvANA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874556" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:33 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v14 10/13] KVM: x86: Enable CET virtualization for VMX and advertise CET to userspace Date: Fri, 6 Nov 2020 09:16:34 +0800 Message-Id: <20201106011637.14289-11-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Set the feature bits so that CET capabilities can be seen in guest via CPUID enumeration. Add CR4.CET bit support in order to allow guest set CET master control bit(CR4.CET). Disable KVM CET feature if unrestricted_guest is unsupported/disabled as KVM does not support emulating CET. Reset guest CET states in vmcs so as to avoid vmentry failure when guest toggles CR4.CET bit, e.g. during guest reboot. Don't expose CET feature if dependent CET bits are cleared in host XSS, or if XSAVES isn't supported. Updating the CET features in common x86 is a little ugly, but there is on clean solution without risking breakage of SVM if SVM hardware ever gains support for CET, e.g. moving everything to common x86 would prematurely expose CET on SVM. The alternative is to put all the logic in VMX, but that means rereading host_xss in VMX and duplicating the XSAVES check across VMX and SVM. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang Reported-by: kernel test robot --- arch/x86/include/asm/kvm_host.h | 3 +- arch/x86/kvm/cpuid.c | 5 +-- arch/x86/kvm/vmx/capabilities.h | 5 +++ arch/x86/kvm/vmx/vmx.c | 64 +++++++++++++++++++++++++++++++-- arch/x86/kvm/x86.c | 8 +++++ 5 files changed, 79 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 1620a2cca781..b3b5cb44e75b 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -100,7 +100,8 @@ | X86_CR4_PGE | X86_CR4_PCE | X86_CR4_OSFXSR | X86_CR4_PCIDE \ | X86_CR4_OSXSAVE | X86_CR4_SMEP | X86_CR4_FSGSBASE \ | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_VMXE \ - | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP)) + | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP \ + | X86_CR4_CET)) #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 2c737337f466..6a888ee1c7db 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -405,7 +405,8 @@ void kvm_set_cpu_caps(void) F(AVX512VBMI) | F(LA57) | F(PKU) | 0 /*OSPKE*/ | F(RDPID) | F(AVX512_VPOPCNTDQ) | F(UMIP) | F(AVX512_VBMI2) | F(GFNI) | F(VAES) | F(VPCLMULQDQ) | F(AVX512_VNNI) | F(AVX512_BITALG) | - F(CLDEMOTE) | F(MOVDIRI) | F(MOVDIR64B) | 0 /*WAITPKG*/ + F(CLDEMOTE) | F(MOVDIRI) | F(MOVDIR64B) | 0 /*WAITPKG*/ | + F(SHSTK) ); /* Set LA57 based on hardware capability. */ if (cpuid_ecx(7) & F(LA57)) @@ -422,7 +423,7 @@ void kvm_set_cpu_caps(void) F(AVX512_4VNNIW) | F(AVX512_4FMAPS) | F(SPEC_CTRL) | F(SPEC_CTRL_SSBD) | F(ARCH_CAPABILITIES) | F(INTEL_STIBP) | F(MD_CLEAR) | F(AVX512_VP2INTERSECT) | F(FSRM) | - F(SERIALIZE) | F(TSXLDTRK) + F(SERIALIZE) | F(TSXLDTRK) | F(IBT) ); /* TSC_ADJUST and ARCH_CAPABILITIES are emulated in software. */ diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h index 3a1861403d73..58cb57b08697 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -103,6 +103,11 @@ static inline bool cpu_has_load_perf_global_ctrl(void) (vmcs_config.vmexit_ctrl & VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL); } +static inline bool cpu_has_load_cet_ctrl(void) +{ + return (vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_CET_STATE) && + (vmcs_config.vmexit_ctrl & VM_EXIT_LOAD_CET_STATE); +} static inline bool cpu_has_vmx_mpx(void) { return (vmcs_config.vmexit_ctrl & VM_EXIT_CLEAR_BNDCFGS) && diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 28ba8414a7a3..c88a6e1721b1 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -2279,7 +2279,7 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) case MSR_IA32_U_CET: if (!cet_is_control_msr_accessible(vcpu, msr_info)) return 1; - if (data & GENMASK(9, 6)) + if ((data & GENMASK(9, 6)) || is_noncanonical_address(data, vcpu)) return 1; if (msr_index == MSR_IA32_S_CET) vmcs_writel(GUEST_S_CET, data); @@ -2594,7 +2594,8 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf, VM_EXIT_LOAD_IA32_EFER | VM_EXIT_CLEAR_BNDCFGS | VM_EXIT_PT_CONCEAL_PIP | - VM_EXIT_CLEAR_IA32_RTIT_CTL; + VM_EXIT_CLEAR_IA32_RTIT_CTL | + VM_EXIT_LOAD_CET_STATE; if (adjust_vmx_controls(min, opt, MSR_IA32_VMX_EXIT_CTLS, &_vmexit_control) < 0) return -EIO; @@ -2618,7 +2619,8 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf, VM_ENTRY_LOAD_IA32_EFER | VM_ENTRY_LOAD_BNDCFGS | VM_ENTRY_PT_CONCEAL_PIP | - VM_ENTRY_LOAD_IA32_RTIT_CTL; + VM_ENTRY_LOAD_IA32_RTIT_CTL | + VM_ENTRY_LOAD_CET_STATE; if (adjust_vmx_controls(min, opt, MSR_IA32_VMX_ENTRY_CTLS, &_vmentry_control) < 0) return -EIO; @@ -2646,6 +2648,15 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf, } } + /* + * The CET entry and exit controls need to be synchronized, e.g. to + * avoid loading guest state but not restoring host state. + */ + if (!(_vmentry_control & VM_ENTRY_LOAD_CET_STATE) || + !(_vmexit_control & VM_EXIT_LOAD_CET_STATE)) { + _vmentry_control &= ~VM_ENTRY_LOAD_CET_STATE; + _vmexit_control &= ~VM_EXIT_LOAD_CET_STATE; + } rdmsr(MSR_IA32_VMX_BASIC, vmx_msr_low, vmx_msr_high); @@ -3217,7 +3228,9 @@ int vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) * this bit, even if host CR4.MCE == 0. */ unsigned long hw_cr4; + unsigned long old_cr4; + old_cr4 = vmcs_readl(CR4_READ_SHADOW); hw_cr4 = (cr4_read_shadow() & X86_CR4_MCE) | (cr4 & ~X86_CR4_MCE); if (is_unrestricted_guest(vcpu)) hw_cr4 |= KVM_VM_CR4_ALWAYS_ON_UNRESTRICTED_GUEST; @@ -3281,6 +3294,13 @@ int vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) vmcs_writel(CR4_READ_SHADOW, cr4); vmcs_writel(GUEST_CR4, hw_cr4); + + if (((cr4 ^ old_cr4) & X86_CR4_CET) && kvm_cet_supported()) { + vmcs_writel(GUEST_SSP, 0); + vmcs_writel(GUEST_S_CET, 0); + vmcs_writel(GUEST_INTR_SSP_TABLE, 0); + } + return 0; } @@ -5961,6 +5981,12 @@ void dump_vmcs(void) pr_err("InterruptStatus = %04x\n", vmcs_read16(GUEST_INTR_STATUS)); + if (vmentry_ctl & VM_ENTRY_LOAD_CET_STATE) { + pr_err("S_CET = 0x%016lx\n", vmcs_readl(GUEST_S_CET)); + pr_err("SSP = 0x%016lx\n", vmcs_readl(GUEST_SSP)); + pr_err("SSP TABLE = 0x%016lx\n", + vmcs_readl(GUEST_INTR_SSP_TABLE)); + } pr_err("*** Host State ***\n"); pr_err("RIP = 0x%016lx RSP = 0x%016lx\n", vmcs_readl(HOST_RIP), vmcs_readl(HOST_RSP)); @@ -6035,6 +6061,12 @@ void dump_vmcs(void) if (secondary_exec_control & SECONDARY_EXEC_ENABLE_VPID) pr_err("Virtual processor ID = 0x%04x\n", vmcs_read16(VIRTUAL_PROCESSOR_ID)); + if (vmexit_ctl & VM_EXIT_LOAD_CET_STATE) { + pr_err("S_CET = 0x%016lx\n", vmcs_readl(HOST_S_CET)); + pr_err("SSP = 0x%016lx\n", vmcs_readl(HOST_SSP)); + pr_err("SSP TABLE = 0x%016lx\n", + vmcs_readl(HOST_INTR_SSP_TABLE)); + } } /* @@ -7409,6 +7441,15 @@ static __init void vmx_set_cpu_caps(void) if (cpu_has_vmx_waitpkg()) kvm_cpu_cap_check_and_set(X86_FEATURE_WAITPKG); + + if (!cpu_has_load_cet_ctrl() || !enable_unrestricted_guest) { + kvm_cpu_cap_clear(X86_FEATURE_SHSTK); + kvm_cpu_cap_clear(X86_FEATURE_IBT); + } else if (kvm_cpu_cap_has(X86_FEATURE_SHSTK) || + kvm_cpu_cap_has(X86_FEATURE_IBT)) { + supported_xss |= XFEATURE_MASK_CET_USER | + XFEATURE_MASK_CET_KERNEL; + } } static void vmx_request_immediate_exit(struct kvm_vcpu *vcpu) @@ -7837,6 +7878,8 @@ static __init int hardware_setup(void) unsigned long host_bndcfgs; struct desc_ptr dt; int r, i, ept_lpage_level; + u64 cet_msr; + bool accessible; store_idt(&dt); host_idt_base = dt.address; @@ -7850,6 +7893,21 @@ static __init int hardware_setup(void) if (boot_cpu_has(X86_FEATURE_NX)) kvm_enable_efer_bits(EFER_NX); + accessible = (supported_xss & XFEATURE_MASK_CET_KERNEL) && + (boot_cpu_has(X86_FEATURE_IBT) || + boot_cpu_has(X86_FEATURE_SHSTK)); + if (accessible) { + rdmsrl(MSR_IA32_S_CET, cet_msr); + WARN_ONCE(cet_msr, "KVM: CET S_CET in host will be lost!\n"); + } + + accessible = (supported_xss & XFEATURE_MASK_CET_KERNEL) && + boot_cpu_has(X86_FEATURE_SHSTK); + if (accessible) { + rdmsrl(MSR_IA32_PL0_SSP, cet_msr); + WARN_ONCE(cet_msr, "KVM: CET PL0_SSP in host will be lost!\n"); + } + if (boot_cpu_has(X86_FEATURE_MPX)) { rdmsrl(MSR_IA32_BNDCFGS, host_bndcfgs); WARN_ONCE(host_bndcfgs, "KVM: BNDCFGS in host will be lost"); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index d573cadf5baf..a500c4e260af 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10197,6 +10197,14 @@ int kvm_arch_hardware_setup(void *opaque) if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES)) supported_xss = 0; + else + supported_xss &= host_xss; + + /* Update CET features now that supported_xss is finalized. */ + if (!kvm_cet_supported()) { + kvm_cpu_cap_clear(X86_FEATURE_SHSTK); + kvm_cpu_cap_clear(X86_FEATURE_IBT); + } #define __kvm_cpu_cap_has(UNUSED_, f) kvm_cpu_cap_has(f) cr4_reserved_bits = __cr4_reserved_bits(__kvm_cpu_cap_has, UNUSED_); From patchwork Fri Nov 6 01:16:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68D99C55178 for ; Fri, 6 Nov 2020 01:06:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1E4FB2075A for ; Fri, 6 Nov 2020 01:06:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733221AbgKFBGi (ORCPT ); Thu, 5 Nov 2020 20:06:38 -0500 Received: from mga18.intel.com ([134.134.136.126]:38191 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733213AbgKFBGh (ORCPT ); Thu, 5 Nov 2020 20:06:37 -0500 IronPort-SDR: 55qWWIVJN+rGaPPppx9me/JevqZvgq85xLBYfRonKapqkushk1dF2mpQjo9JrcudCJlUveIUJK 849B6wpJFGwg== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264714" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264714" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:36 -0800 IronPort-SDR: oRowV/6fvjKU/L0uJHkiEOvegywWRZvq7nMK6TVFxShQWHXqRYWN//sCNlqAZswoYEaW6MCRAU YI0y64BIhrsA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874563" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:35 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v14 11/13] KVM: VMX: Pass through CET MSRs to the guest when supported Date: Fri, 6 Nov 2020 09:16:35 +0800 Message-Id: <20201106011637.14289-12-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Pass through all CET MSRs when the associated CET component (kernel vs. user) is enabled to improve guest performance. All CET MSRs are context switched, either via dedicated VMCS fields or XSAVES. Co-developed-by: Zhang Yi Z Signed-off-by: Zhang Yi Z Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang --- arch/x86/kvm/vmx/vmx.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index c88a6e1721b1..6ba2027a3d44 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7366,6 +7366,32 @@ static void update_intel_pt_cfg(struct kvm_vcpu *vcpu) vmx->pt_desc.ctl_bitmask &= ~(0xfULL << (32 + i * 4)); } +static bool is_cet_state_supported(struct kvm_vcpu *vcpu, u32 xss_state) +{ + return (vcpu->arch.guest_supported_xss & xss_state) && + (guest_cpuid_has(vcpu, X86_FEATURE_SHSTK) || + guest_cpuid_has(vcpu, X86_FEATURE_IBT)); +} + +static void vmx_update_intercept_for_cet_msr(struct kvm_vcpu *vcpu) +{ + bool incpt = !is_cet_state_supported(vcpu, XFEATURE_MASK_CET_USER); + + vmx_set_intercept_for_msr(vcpu, MSR_IA32_U_CET, MSR_TYPE_RW, incpt); + + incpt |= !guest_cpuid_has(vcpu, X86_FEATURE_SHSTK); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_PL3_SSP, MSR_TYPE_RW, incpt); + + incpt = !is_cet_state_supported(vcpu, XFEATURE_MASK_CET_KERNEL); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_S_CET, MSR_TYPE_RW, incpt); + + incpt |= !guest_cpuid_has(vcpu, X86_FEATURE_SHSTK); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_INT_SSP_TAB, MSR_TYPE_RW, incpt); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_PL0_SSP, MSR_TYPE_RW, incpt); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_PL1_SSP, MSR_TYPE_RW, incpt); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_PL2_SSP, MSR_TYPE_RW, incpt); +} + static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -7409,6 +7435,9 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) /* Refresh #PF interception to account for MAXPHYADDR changes. */ update_exception_bitmap(vcpu); + + if (kvm_cet_supported()) + vmx_update_intercept_for_cet_msr(vcpu); } static __init void vmx_set_cpu_caps(void) From patchwork Fri Nov 6 01:16:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6ED14C388F7 for ; Fri, 6 Nov 2020 01:06:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3185C2075A for ; Fri, 6 Nov 2020 01:06:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733271AbgKFBGm (ORCPT ); Thu, 5 Nov 2020 20:06:42 -0500 Received: from mga18.intel.com ([134.134.136.126]:38191 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733237AbgKFBGj (ORCPT ); Thu, 5 Nov 2020 20:06:39 -0500 IronPort-SDR: 6AkZnlBe7JYycstSJ/IKTDDfQvZxd+QaWQKAHRmmAIWf2mMrBf6hSAPmw5oq/KCs07fHdpy8jW XNPFBceJ43vA== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264716" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264716" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:38 -0800 IronPort-SDR: Cq9sbFqjcWPBe5rvAf6GTguXqh1M0OAtxTYrlkp6IlUlR61kH1ncpyMnJkfrjhQE/5HNWqAxqM ZdsqOB+goWRw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874567" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:36 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v14 12/13] KVM: nVMX: Add helper to check the vmcs01 MSR bitmap for MSR pass-through Date: Fri, 6 Nov 2020 09:16:36 +0800 Message-Id: <20201106011637.14289-13-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a helper to perform the check on the vmcs01/l01 MSR bitmap when disabling interception of an MSR for L2. This reduces the boilerplate for the existing cases, and will be used heavily in a future patch for CET MSRs. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang --- arch/x86/kvm/vmx/nested.c | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 89af692deb7e..8abc7bdd94f7 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -544,6 +544,17 @@ static void nested_vmx_disable_intercept_for_msr(unsigned long *msr_bitmap_l1, } } +static void nested_vmx_cond_disable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, + unsigned long *bitmap_12, + unsigned long *bitmap_02, + int type) +{ + if (msr_write_intercepted_l01(vcpu, msr)) + return; + + nested_vmx_disable_intercept_for_msr(bitmap_12, bitmap_02, msr, type); +} + static inline void enable_x2apic_msr_intercepts(unsigned long *msr_bitmap) { int msr; @@ -640,17 +651,13 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, * updated to reflect this when L1 (or its L2s) actually write to * the MSR. */ - if (!msr_write_intercepted_l01(vcpu, MSR_IA32_SPEC_CTRL)) - nested_vmx_disable_intercept_for_msr( - msr_bitmap_l1, msr_bitmap_l0, - MSR_IA32_SPEC_CTRL, - MSR_TYPE_R | MSR_TYPE_W); + nested_vmx_cond_disable_intercept_for_msr(vcpu, MSR_IA32_SPEC_CTRL, + msr_bitmap_l1, msr_bitmap_l0, + MSR_TYPE_R | MSR_TYPE_W); - if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PRED_CMD)) - nested_vmx_disable_intercept_for_msr( - msr_bitmap_l1, msr_bitmap_l0, - MSR_IA32_PRED_CMD, - MSR_TYPE_W); + nested_vmx_cond_disable_intercept_for_msr(vcpu, MSR_IA32_PRED_CMD, + msr_bitmap_l1, msr_bitmap_l0, + MSR_TYPE_W); kvm_vcpu_unmap(vcpu, &to_vmx(vcpu)->nested.msr_bitmap_map, false); From patchwork Fri Nov 6 01:16:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11885627 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA8A2C388F7 for ; Fri, 6 Nov 2020 01:06:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9DFB22075A for ; Fri, 6 Nov 2020 01:06:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733262AbgKFBGm (ORCPT ); Thu, 5 Nov 2020 20:06:42 -0500 Received: from mga18.intel.com ([134.134.136.126]:38191 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733219AbgKFBGl (ORCPT ); Thu, 5 Nov 2020 20:06:41 -0500 IronPort-SDR: 48iqFFLbCvi1liTU60IUZpDaFEFL0m506AwQgcXNKIgjy5Q7tAc1AEYBwi927mfGUP0pdXaQx3 PMhDL6pHIrIA== X-IronPort-AV: E=McAfee;i="6000,8403,9796"; a="157264721" X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="157264721" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2020 17:06:40 -0800 IronPort-SDR: W4hsUtTSzDLH7xlqOYx7acicCeYuCuZAwYVPPIepDZMs4KXRXdpPmqdeucjSTqDT4fu4Mq94E7 Uwv4pw3iGweg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,454,1596524400"; d="scan'208";a="471874572" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.156]) by orsmga004.jf.intel.com with ESMTP; 05 Nov 2020 17:06:38 -0800 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, sean.j.christopherson@intel.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v14 13/13] KVM: nVMX: Enable CET support for nested VMX Date: Fri, 6 Nov 2020 09:16:37 +0800 Message-Id: <20201106011637.14289-14-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20201106011637.14289-1-weijiang.yang@intel.com> References: <20201106011637.14289-1-weijiang.yang@intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add vmcs12 fields for all CET fields, pass-through CET MSRs to L2 when possible, and enumerate the VMCS controls and CR4 bit as supported. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang --- arch/x86/kvm/vmx/nested.c | 30 ++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/vmcs12.c | 6 ++++++ arch/x86/kvm/vmx/vmcs12.h | 14 +++++++++++++- arch/x86/kvm/vmx/vmx.c | 1 + 4 files changed, 48 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 8abc7bdd94f7..0fe18986c259 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -638,6 +638,29 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, nested_vmx_disable_intercept_for_msr(msr_bitmap_l1, msr_bitmap_l0, MSR_KERNEL_GS_BASE, MSR_TYPE_RW); + /* Pass CET MSRs to nested VM if L0 and L1 are set to pass-through. */ + nested_vmx_cond_disable_intercept_for_msr(vcpu, MSR_IA32_U_CET, + msr_bitmap_l1, msr_bitmap_l0, + MSR_TYPE_RW); + nested_vmx_cond_disable_intercept_for_msr(vcpu, MSR_IA32_PL3_SSP, + msr_bitmap_l1, msr_bitmap_l0, + MSR_TYPE_RW); + nested_vmx_cond_disable_intercept_for_msr(vcpu, MSR_IA32_S_CET, + msr_bitmap_l1, msr_bitmap_l0, + MSR_TYPE_RW); + nested_vmx_cond_disable_intercept_for_msr(vcpu, MSR_IA32_PL0_SSP, + msr_bitmap_l1, msr_bitmap_l0, + MSR_TYPE_RW); + nested_vmx_cond_disable_intercept_for_msr(vcpu, MSR_IA32_PL1_SSP, + msr_bitmap_l1, msr_bitmap_l0, + MSR_TYPE_RW); + nested_vmx_cond_disable_intercept_for_msr(vcpu, MSR_IA32_PL2_SSP, + msr_bitmap_l1, msr_bitmap_l0, + MSR_TYPE_RW); + nested_vmx_cond_disable_intercept_for_msr(vcpu, MSR_IA32_INT_SSP_TAB, + msr_bitmap_l1, msr_bitmap_l0, + MSR_TYPE_RW); + /* * Checking the L0->L1 bitmap is trying to verify two things: * @@ -6336,7 +6359,9 @@ void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps) VM_EXIT_HOST_ADDR_SPACE_SIZE | #endif VM_EXIT_LOAD_IA32_PAT | VM_EXIT_SAVE_IA32_PAT | - VM_EXIT_CLEAR_BNDCFGS | VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL; + VM_EXIT_CLEAR_BNDCFGS | VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL | + VM_EXIT_LOAD_CET_STATE; + msrs->exit_ctls_high |= VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR | VM_EXIT_LOAD_IA32_EFER | VM_EXIT_SAVE_IA32_EFER | @@ -6356,7 +6381,8 @@ void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps) VM_ENTRY_IA32E_MODE | #endif VM_ENTRY_LOAD_IA32_PAT | VM_ENTRY_LOAD_BNDCFGS | - VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL; + VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL | VM_ENTRY_LOAD_CET_STATE; + msrs->entry_ctls_high |= (VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR | VM_ENTRY_LOAD_IA32_EFER); diff --git a/arch/x86/kvm/vmx/vmcs12.c b/arch/x86/kvm/vmx/vmcs12.c index c8e51c004f78..8fd8e0beecef 100644 --- a/arch/x86/kvm/vmx/vmcs12.c +++ b/arch/x86/kvm/vmx/vmcs12.c @@ -137,6 +137,9 @@ const unsigned short vmcs_field_to_offset_table[] = { FIELD(GUEST_PENDING_DBG_EXCEPTIONS, guest_pending_dbg_exceptions), FIELD(GUEST_SYSENTER_ESP, guest_sysenter_esp), FIELD(GUEST_SYSENTER_EIP, guest_sysenter_eip), + FIELD(GUEST_S_CET, guest_s_cet), + FIELD(GUEST_SSP, guest_ssp), + FIELD(GUEST_INTR_SSP_TABLE, guest_ssp_tbl), FIELD(HOST_CR0, host_cr0), FIELD(HOST_CR3, host_cr3), FIELD(HOST_CR4, host_cr4), @@ -149,5 +152,8 @@ const unsigned short vmcs_field_to_offset_table[] = { FIELD(HOST_IA32_SYSENTER_EIP, host_ia32_sysenter_eip), FIELD(HOST_RSP, host_rsp), FIELD(HOST_RIP, host_rip), + FIELD(HOST_S_CET, host_s_cet), + FIELD(HOST_SSP, host_ssp), + FIELD(HOST_INTR_SSP_TABLE, host_ssp_tbl), }; const unsigned int nr_vmcs12_fields = ARRAY_SIZE(vmcs_field_to_offset_table); diff --git a/arch/x86/kvm/vmx/vmcs12.h b/arch/x86/kvm/vmx/vmcs12.h index 80232daf00ff..016896c9e701 100644 --- a/arch/x86/kvm/vmx/vmcs12.h +++ b/arch/x86/kvm/vmx/vmcs12.h @@ -115,7 +115,13 @@ struct __packed vmcs12 { natural_width host_ia32_sysenter_eip; natural_width host_rsp; natural_width host_rip; - natural_width paddingl[8]; /* room for future expansion */ + natural_width host_s_cet; + natural_width host_ssp; + natural_width host_ssp_tbl; + natural_width guest_s_cet; + natural_width guest_ssp; + natural_width guest_ssp_tbl; + natural_width paddingl[2]; /* room for future expansion */ u32 pin_based_vm_exec_control; u32 cpu_based_vm_exec_control; u32 exception_bitmap; @@ -295,6 +301,12 @@ static inline void vmx_check_vmcs12_offsets(void) CHECK_OFFSET(host_ia32_sysenter_eip, 656); CHECK_OFFSET(host_rsp, 664); CHECK_OFFSET(host_rip, 672); + CHECK_OFFSET(host_s_cet, 680); + CHECK_OFFSET(host_ssp, 688); + CHECK_OFFSET(host_ssp_tbl, 696); + CHECK_OFFSET(guest_s_cet, 704); + CHECK_OFFSET(guest_ssp, 712); + CHECK_OFFSET(guest_ssp_tbl, 720); CHECK_OFFSET(pin_based_vm_exec_control, 744); CHECK_OFFSET(cpu_based_vm_exec_control, 748); CHECK_OFFSET(exception_bitmap, 752); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 6ba2027a3d44..5a3e2ff4d7ad 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7276,6 +7276,7 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) cr4_fixed1_update(X86_CR4_PKE, ecx, feature_bit(PKU)); cr4_fixed1_update(X86_CR4_UMIP, ecx, feature_bit(UMIP)); cr4_fixed1_update(X86_CR4_LA57, ecx, feature_bit(LA57)); + cr4_fixed1_update(X86_CR4_CET, ecx, feature_bit(SHSTK)); #undef cr4_fixed1_update }