From patchwork Fri Mar 20 03:43:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11448361 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8A61C139A for ; Fri, 20 Mar 2020 03:40:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6AB4D2072D for ; Fri, 20 Mar 2020 03:40:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727069AbgCTDko (ORCPT ); Thu, 19 Mar 2020 23:40:44 -0400 Received: from mga01.intel.com ([192.55.52.88]:27908 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726614AbgCTDko (ORCPT ); Thu, 19 Mar 2020 23:40:44 -0400 IronPort-SDR: L/XhEyHv43c494j6BqfoIDsswtbbYtNHwmviatzda9Agl1fpFvu6OdAx/6W7LX4kcOlxzwCk/c N5Lw4caBayiQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2020 20:40:44 -0700 IronPort-SDR: YctU/yzzEcXJC+yGogNmsjubAa8wNJJ8mzA9mdvsWe1YSZvvrOHRO+tMCvwQNDgyG1lWAklWCb 2PaovIA9v0rQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,283,1580803200"; d="scan'208";a="263945576" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.128]) by orsmga002.jf.intel.com with ESMTP; 19 Mar 2020 20:40:41 -0700 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, sean.j.christopherson@intel.com, pbonzini@redhat.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v10 1/8] KVM: VMX: Introduce CET VMX fields and flags Date: Fri, 20 Mar 2020 11:43:34 +0800 Message-Id: <20200320034342.26610-2-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20200320034342.26610-1-weijiang.yang@intel.com> References: <20200320034342.26610-1-weijiang.yang@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org CET(Control-flow Enforcement Technology) is a CPU feature used to prevent Return/Jump-Oriented Programming(ROP/JOP) attacks. It provides the following sub-features to defend against ROP/JOP style control-flow subversion attacks: Shadow Stack (SHSTK): A second stack for program which is used exclusively for control transfer operations. Indirect Branch Tracking (IBT): Code branching protection to defend against jump/call oriented programming. Several new CET MSRs are defined in kernel to support CET: MSR_IA32_{U,S}_CET: Controls the CET settings for user mode and suervisor mode respectively. MSR_IA32_PL{0,1,2,3}_SSP: Stores shadow stack pointers for CPL-0,1,2,3 level respectively. MSR_IA32_INT_SSP_TAB: Stores base address of shadow stack pointer table. Two XSAVES state bits are introduced for CET: IA32_XSS:[bit 11]: For saving/restoring user mode CET states IA32_XSS:[bit 12]: For saving/restoring supervisor mode CET states. Six VMCS fields are introduced for CET: {HOST,GUEST}_S_CET: Stores CET settings for supervisor mode. {HOST,GUEST}_SSP: Stores shadow stack pointer for supervisor mode. {HOST,GUEST}_INTR_SSP_TABLE: Stores base address of shadow stack pointer table. If VM_EXIT_LOAD_HOST_CET_STATE = 1, the host's CET MSRs are restored from below VMCS fields at VM-Exit: HOST_S_CET HOST_SSP HOST_INTR_SSP_TABLE If VM_ENTRY_LOAD_GUEST_CET_STATE = 1, the guest's CET MSRs are loaded from below VMCS fields at VM-Entry: GUEST_S_CET GUEST_SSP GUEST_INTR_SSP_TABLE Co-developed-by: Zhang Yi Z Signed-off-by: Zhang Yi Z Signed-off-by: Yang Weijiang --- arch/x86/include/asm/vmx.h | 8 ++++++++ arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/x86.c | 4 ++++ arch/x86/kvm/x86.h | 2 +- 4 files changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index 5e090d1f03f8..e938bc6c37aa 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -94,6 +94,7 @@ #define VM_EXIT_CLEAR_BNDCFGS 0x00800000 #define VM_EXIT_PT_CONCEAL_PIP 0x01000000 #define VM_EXIT_CLEAR_IA32_RTIT_CTL 0x02000000 +#define VM_EXIT_LOAD_HOST_CET_STATE 0x10000000 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR 0x00036dff @@ -107,6 +108,7 @@ #define VM_ENTRY_LOAD_BNDCFGS 0x00010000 #define VM_ENTRY_PT_CONCEAL_PIP 0x00020000 #define VM_ENTRY_LOAD_IA32_RTIT_CTL 0x00040000 +#define VM_ENTRY_LOAD_GUEST_CET_STATE 0x00100000 #define VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR 0x000011ff @@ -328,6 +330,9 @@ enum vmcs_field { GUEST_PENDING_DBG_EXCEPTIONS = 0x00006822, GUEST_SYSENTER_ESP = 0x00006824, GUEST_SYSENTER_EIP = 0x00006826, + GUEST_S_CET = 0x00006828, + GUEST_SSP = 0x0000682a, + GUEST_INTR_SSP_TABLE = 0x0000682c, HOST_CR0 = 0x00006c00, HOST_CR3 = 0x00006c02, HOST_CR4 = 0x00006c04, @@ -340,6 +345,9 @@ enum vmcs_field { HOST_IA32_SYSENTER_EIP = 0x00006c12, HOST_RSP = 0x00006c14, HOST_RIP = 0x00006c16, + HOST_S_CET = 0x00006c18, + HOST_SSP = 0x00006c1a, + HOST_INTR_SSP_TABLE = 0x00006c1c }; /* diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 3f3f780c8c65..78e5c4266270 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -31,6 +31,7 @@ #define MC_VECTOR 18 #define XM_VECTOR 19 #define VE_VECTOR 20 +#define CP_VECTOR 21 /* Select x86 specific features in */ #define __KVM_HAVE_PIT diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 40c6768942ae..830afe5038d1 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -186,6 +186,9 @@ static struct kvm_shared_msrs __percpu *shared_msrs; | XFEATURE_MASK_BNDCSR | XFEATURE_MASK_AVX512 \ | XFEATURE_MASK_PKRU) +#define KVM_SUPPORTED_XSS (XFEATURE_MASK_CET_USER | \ + XFEATURE_MASK_CET_KERNEL) + u64 __read_mostly host_efer; EXPORT_SYMBOL_GPL(host_efer); @@ -402,6 +405,7 @@ static int exception_class(int vector) case NP_VECTOR: case SS_VECTOR: case GP_VECTOR: + case CP_VECTOR: return EXCPT_CONTRIBUTORY; default: break; diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index c1954e216b41..8f0baa6fa72f 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -115,7 +115,7 @@ static inline bool x86_exception_has_error_code(unsigned int vector) { static u32 exception_has_error_code = BIT(DF_VECTOR) | BIT(TS_VECTOR) | BIT(NP_VECTOR) | BIT(SS_VECTOR) | BIT(GP_VECTOR) | - BIT(PF_VECTOR) | BIT(AC_VECTOR); + BIT(PF_VECTOR) | BIT(AC_VECTOR) | BIT(CP_VECTOR); return (1U << vector) & exception_has_error_code; } From patchwork Fri Mar 20 03:43:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11448363 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F0ED213B1 for ; Fri, 20 Mar 2020 03:40:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D99392072C for ; Fri, 20 Mar 2020 03:40:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727192AbgCTDkr (ORCPT ); Thu, 19 Mar 2020 23:40:47 -0400 Received: from mga01.intel.com ([192.55.52.88]:27908 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727175AbgCTDkq (ORCPT ); Thu, 19 Mar 2020 23:40:46 -0400 IronPort-SDR: ZzOAajjLdf0ksnTm8e0FOVyfQyvEvmZKwY46lt8rSp9b+/F7yQCVzPOZnZ4784GHwWu42alwqm abBDaW//JhKA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2020 20:40:46 -0700 IronPort-SDR: cZZymfSosMDqlAQZiXXjOazLOfIDdq+RqSop6KhhUk9I0TVOc8RqCOYaNM099s2G41b9zMHrk3 UH19CRgJvyCw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,283,1580803200"; d="scan'208";a="263945582" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.128]) by orsmga002.jf.intel.com with ESMTP; 19 Mar 2020 20:40:44 -0700 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, sean.j.christopherson@intel.com, pbonzini@redhat.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v10 2/8] KVM: VMX: Set up guest CET MSRs per KVM and host configuration Date: Fri, 20 Mar 2020 11:43:35 +0800 Message-Id: <20200320034342.26610-3-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20200320034342.26610-1-weijiang.yang@intel.com> References: <20200320034342.26610-1-weijiang.yang@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org CET MSRs pass through guest directly to enhance performance. CET runtime control settings are stored in MSR_IA32_{U,S}_CET, Shadow Stack Pointer(SSP) are stored in MSR_IA32_PL{0,1,2,3}_SSP, SSP table base address is stored in MSR_IA32_INT_SSP_TAB, these MSRs are defined in kernel and re-used here. MSR_IA32_U_CET and MSR_IA32_PL3_SSP are used for user-mode protection, the MSR contents are switched between threads during scheduling, it makes sense to pass through them so that the guest kernel can use xsaves/xrstors to operate them efficiently. Other MSRs are used for non-user mode protection. See SDM for detailed info. The difference between CET VMCS fields and CET MSRs is that,the former are used during VMEnter/VMExit, whereas the latter are used for CET state storage between task/thread scheduling. Co-developed-by: Zhang Yi Z Signed-off-by: Zhang Yi Z Signed-off-by: Yang Weijiang --- arch/x86/kvm/vmx/vmx.c | 43 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 860e5f4a9f7b..61d2a4bf9eb6 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3033,6 +3033,19 @@ void vmx_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) vmcs_writel(GUEST_CR3, guest_cr3); } +static bool is_cet_mode_allowed(struct kvm_vcpu *vcpu, u32 mode_mask) +{ + return ((supported_xss & mode_mask) && + (guest_cpuid_has(vcpu, X86_FEATURE_SHSTK) || + guest_cpuid_has(vcpu, X86_FEATURE_IBT))); +} + +static bool is_cet_supported(struct kvm_vcpu *vcpu) +{ + return is_cet_mode_allowed(vcpu, XFEATURE_MASK_CET_USER | + XFEATURE_MASK_CET_KERNEL); +} + int vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -7064,6 +7077,35 @@ static void update_intel_pt_cfg(struct kvm_vcpu *vcpu) vmx->pt_desc.ctl_bitmask &= ~(0xfULL << (32 + i * 4)); } +static void vmx_update_intercept_for_cet_msr(struct kvm_vcpu *vcpu) +{ + struct vcpu_vmx *vmx = to_vmx(vcpu); + unsigned long *msr_bitmap = vmx->vmcs01.msr_bitmap; + bool flag; + + flag = !is_cet_mode_allowed(vcpu, XFEATURE_MASK_CET_USER); + /* + * U_CET is required for USER CET, and U_CET, PL3_SPP are bound as + * one component and controlled by IA32_XSS[bit 11]. + */ + vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_U_CET, MSR_TYPE_RW, flag); + vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_PL3_SSP, MSR_TYPE_RW, flag); + + flag = !is_cet_mode_allowed(vcpu, XFEATURE_MASK_CET_KERNEL); + /* + * S_CET is required for KERNEL CET, and PL0_SSP ... PL2_SSP are + * bound as one component and controlled by IA32_XSS[bit 12]. + */ + vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_S_CET, MSR_TYPE_RW, flag); + vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_PL0_SSP, MSR_TYPE_RW, flag); + vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_PL1_SSP, MSR_TYPE_RW, flag); + vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_PL2_SSP, MSR_TYPE_RW, flag); + + flag |= !guest_cpuid_has(vcpu, X86_FEATURE_SHSTK); + /* SSP_TAB is only available for KERNEL SHSTK.*/ + vmx_set_intercept_for_msr(msr_bitmap, MSR_IA32_INT_SSP_TAB, MSR_TYPE_RW, flag); +} + static void vmx_cpuid_update(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -7102,6 +7144,7 @@ static void vmx_cpuid_update(struct kvm_vcpu *vcpu) vmx_set_guest_msr(vmx, msr, enabled ? 0 : TSX_CTRL_RTM_DISABLE); } } + vmx_update_intercept_for_cet_msr(vcpu); } static __init void vmx_set_cpu_caps(void) From patchwork Fri Mar 20 03:43:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11448365 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6BE8792A for ; Fri, 20 Mar 2020 03:40:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 547342072C for ; Fri, 20 Mar 2020 03:40:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726738AbgCTDku (ORCPT ); Thu, 19 Mar 2020 23:40:50 -0400 Received: from mga01.intel.com ([192.55.52.88]:27908 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727227AbgCTDkt (ORCPT ); Thu, 19 Mar 2020 23:40:49 -0400 IronPort-SDR: 76wiKmcbvVgPFQqbR4IifNkQpv/JCdjgyz1s+ZBHCnLqPK2i6yF1Kmfh2Fvb0CoJ/gD0T8RwUS rBjtAC+nI9eQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2020 20:40:48 -0700 IronPort-SDR: 1KkkAk0vAJKRBQ/yio5nsHt6+uzEJmhw99Td8JNvFXxlIp2Jv6wswIEgiFEwXDJudekdQFSSj+ fjy+dDOfKcJQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,283,1580803200"; d="scan'208";a="263945589" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.128]) by orsmga002.jf.intel.com with ESMTP; 19 Mar 2020 20:40:46 -0700 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, sean.j.christopherson@intel.com, pbonzini@redhat.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v10 3/8] KVM: VMX: Load CET states on vmentry/vmexit Date: Fri, 20 Mar 2020 11:43:36 +0800 Message-Id: <20200320034342.26610-4-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20200320034342.26610-1-weijiang.yang@intel.com> References: <20200320034342.26610-1-weijiang.yang@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org "Load {guest,host} CET state" bit controls whether guest/host CET states will be loaded at VM entry/exit. There're mutual constrains between CR0.WP and CR4.CET, so need to check the dependent bit while changing the control registers. Note: 1)The processor does not allow CR4.CET to be set if CR0.WP = 0, similarly, it does not allow CR0.WP to be cleared while CR4.CET = 1. In either case, KVM would inject #GP to guest. 2)SHSTK and IBT features share one control MSR: MSR_IA32_{U,S}_CET, which means it's difficult to hide one feature from another in the case of SHSTK != IBT, after discussed in community, it's agreed to allow Guest control two features independently as it won't introduce security hole. Co-developed-by: Zhang Yi Z Signed-off-by: Zhang Yi Z Signed-off-by: Yang Weijiang Reported-by: kbuild test robot --- arch/x86/kvm/vmx/capabilities.h | 10 ++++++++++ arch/x86/kvm/vmx/vmx.c | 25 +++++++++++++++++++++++-- arch/x86/kvm/x86.c | 3 +++ 3 files changed, 36 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h index 8903475f751e..565340352260 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -107,6 +107,16 @@ static inline bool cpu_has_vmx_mpx(void) (vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_BNDCFGS); } +static inline bool cpu_has_cet_guest_load_ctrl(void) +{ + return (vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_GUEST_CET_STATE); +} + +static inline bool cpu_has_cet_host_load_ctrl(void) +{ + return (vmcs_config.vmexit_ctrl & VM_EXIT_LOAD_HOST_CET_STATE); +} + static inline bool cpu_has_vmx_tpr_shadow(void) { return vmcs_config.cpu_based_exec_ctrl & CPU_BASED_TPR_SHADOW; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 61d2a4bf9eb6..e7ac776c808f 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -44,6 +44,7 @@ #include #include #include +#include #include "capabilities.h" #include "cpuid.h" @@ -2456,7 +2457,8 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf, VM_EXIT_LOAD_IA32_EFER | VM_EXIT_CLEAR_BNDCFGS | VM_EXIT_PT_CONCEAL_PIP | - VM_EXIT_CLEAR_IA32_RTIT_CTL; + VM_EXIT_CLEAR_IA32_RTIT_CTL | + VM_EXIT_LOAD_HOST_CET_STATE; if (adjust_vmx_controls(min, opt, MSR_IA32_VMX_EXIT_CTLS, &_vmexit_control) < 0) return -EIO; @@ -2480,7 +2482,8 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf, VM_ENTRY_LOAD_IA32_EFER | VM_ENTRY_LOAD_BNDCFGS | VM_ENTRY_PT_CONCEAL_PIP | - VM_ENTRY_LOAD_IA32_RTIT_CTL; + VM_ENTRY_LOAD_IA32_RTIT_CTL | + VM_ENTRY_LOAD_GUEST_CET_STATE; if (adjust_vmx_controls(min, opt, MSR_IA32_VMX_ENTRY_CTLS, &_vmentry_control) < 0) return -EIO; @@ -3086,6 +3089,10 @@ int vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) return 1; } + if ((cr4 & X86_CR4_CET) && (!is_cet_supported(vcpu) || + !(kvm_read_cr0(vcpu) & X86_CR0_WP))) + return 1; + if (vmx->nested.vmxon && !nested_cr4_valid(vcpu, cr4)) return 1; @@ -3945,6 +3952,12 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx) if (cpu_has_load_ia32_efer()) vmcs_write64(HOST_IA32_EFER, host_efer); + + if (cpu_has_cet_host_load_ctrl()) { + vmcs_writel(HOST_S_CET, 0); + vmcs_writel(HOST_INTR_SSP_TABLE, 0); + vmcs_writel(HOST_SSP, 0); + } } void set_cr4_guest_host_mask(struct vcpu_vmx *vmx) @@ -6541,6 +6554,14 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu) vmx->loaded_vmcs->host_state.cr3 = cr3; } + if (cpu_has_cet_guest_load_ctrl() && is_cet_supported(vcpu)) + vmcs_set_bits(VM_ENTRY_CONTROLS, + VM_ENTRY_LOAD_GUEST_CET_STATE); + + if (cpu_has_cet_host_load_ctrl() && is_cet_supported(vcpu)) + vmcs_set_bits(VM_EXIT_CONTROLS, + VM_EXIT_LOAD_HOST_CET_STATE); + cr4 = cr4_read_shadow(); if (unlikely(cr4 != vmx->loaded_vmcs->host_state.cr4)) { vmcs_writel(HOST_CR4, cr4); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 830afe5038d1..90acdbbb8a5a 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -804,6 +804,9 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) if (!(cr0 & X86_CR0_PG) && kvm_read_cr4_bits(vcpu, X86_CR4_PCIDE)) return 1; + if (!(cr0 & X86_CR0_WP) && kvm_read_cr4_bits(vcpu, X86_CR4_CET)) + return 1; + kvm_x86_ops->set_cr0(vcpu, cr0); if ((cr0 ^ old_cr0) & X86_CR0_PG) { From patchwork Fri Mar 20 03:43:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11448375 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3BAF5139A for ; Fri, 20 Mar 2020 03:41:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 269232076C for ; Fri, 20 Mar 2020 03:41:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727301AbgCTDkz (ORCPT ); Thu, 19 Mar 2020 23:40:55 -0400 Received: from mga01.intel.com ([192.55.52.88]:27908 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727260AbgCTDkv (ORCPT ); Thu, 19 Mar 2020 23:40:51 -0400 IronPort-SDR: tJHduf5hFrMiOf0apiid2CyYJsY8nhsvCeSwf1kgN59Yl1L4KJgeBbvoy2zAjMEzkCKElwdplg DQbZqU2nhoHQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2020 20:40:50 -0700 IronPort-SDR: Y7bEvzzjGFuBXS4T7AazVxPvEeOlusBWvwbhI/OmD10zfDDNsPoeYKCx83AKtUndG0hYg9I9Qz ozlWA/n7VMRg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,283,1580803200"; d="scan'208";a="263945594" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.128]) by orsmga002.jf.intel.com with ESMTP; 19 Mar 2020 20:40:48 -0700 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, sean.j.christopherson@intel.com, pbonzini@redhat.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v10 4/8] KVM: X86: Refresh CPUID on guest XSS change Date: Fri, 20 Mar 2020 11:43:37 +0800 Message-Id: <20200320034342.26610-5-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20200320034342.26610-1-weijiang.yang@intel.com> References: <20200320034342.26610-1-weijiang.yang@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org CPUID(0xd, 1) reports the current required storage size of XCR0 | XSS, when guest updates the XSS, it's necessary to update the CPUID leaf, otherwise guest will fetch stale state, this result into some WARNs during guest running. Co-developed-by: Zhang Yi Z Signed-off-by: Zhang Yi Z Signed-off-by: Yang Weijiang --- arch/x86/kvm/cpuid.c | 18 +++++++++++++++--- arch/x86/kvm/x86.c | 7 ++++++- 2 files changed, 21 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 78d461be2102..71703d9277ee 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -95,9 +95,21 @@ int kvm_update_cpuid(struct kvm_vcpu *vcpu) } best = kvm_find_cpuid_entry(vcpu, 0xD, 1); - if (best && (cpuid_entry_has(best, X86_FEATURE_XSAVES) || - cpuid_entry_has(best, X86_FEATURE_XSAVEC))) - best->ebx = xstate_required_size(vcpu->arch.xcr0, true); + if (best) { + if (best->eax & (F(XSAVES) | F(XSAVEC))) { + u64 xstate = vcpu->arch.xcr0 | vcpu->arch.ia32_xss; + + best->ebx = xstate_required_size(xstate, true); + } + + if (best->eax & F(XSAVES)) { + supported_xss &= best->ecx | ((u64)best->edx << 32); + } else { + best->ecx = 0; + best->edx = 0; + supported_xss = 0; + } + } /* * The existing code assumes virtual address is 48-bit or 57-bit in the diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 90acdbbb8a5a..5be6fad6e08d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2838,7 +2838,10 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) */ if (data & ~supported_xss) return 1; - vcpu->arch.ia32_xss = data; + if (vcpu->arch.ia32_xss != data) { + vcpu->arch.ia32_xss = data; + kvm_update_cpuid(vcpu); + } break; case MSR_SMI_COUNT: if (!msr_info->host_initiated) @@ -9635,6 +9638,8 @@ int kvm_arch_hardware_setup(void) if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES)) supported_xss = 0; + else + supported_xss = host_xss & KVM_SUPPORTED_XSS; cr4_reserved_bits = kvm_host_cr4_reserved_bits(&boot_cpu_data); From patchwork Fri Mar 20 03:43:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11448377 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 112EC139A for ; Fri, 20 Mar 2020 03:41:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EF9F12076C for ; Fri, 20 Mar 2020 03:41:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727452AbgCTDlU (ORCPT ); Thu, 19 Mar 2020 23:41:20 -0400 Received: from mga01.intel.com ([192.55.52.88]:27908 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726789AbgCTDkx (ORCPT ); Thu, 19 Mar 2020 23:40:53 -0400 IronPort-SDR: pCT/WNIrKvK1W3k4aKIYTWpInTOuGwJettrcVbQ2f702esrNxGDDUq1N14mT5RUrQYKQOgYDJU WvTO0Hsmt8Fw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2020 20:40:53 -0700 IronPort-SDR: yyamiEG1I9WRd/Ovice+J1Y/dGo78B1yYGRXopKHoL+pOjaG1Dt9xPaLFNJztXN6u5ZZyk1NZq szbxRdSGetHg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,283,1580803200"; d="scan'208";a="263945602" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.128]) by orsmga002.jf.intel.com with ESMTP; 19 Mar 2020 20:40:51 -0700 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, sean.j.christopherson@intel.com, pbonzini@redhat.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v10 5/8] KVM: X86: Load guest fpu state when accessing MSRs managed by XSAVES Date: Fri, 20 Mar 2020 11:43:38 +0800 Message-Id: <20200320034342.26610-6-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20200320034342.26610-1-weijiang.yang@intel.com> References: <20200320034342.26610-1-weijiang.yang@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Sean Christopherson A handful of CET MSRs are not context switched through "traditional" methods, e.g. VMCS or manual switching, but rather are passed through to the guest and are saved and restored by XSAVES/XRSTORS, i.e. in the guest's FPU state. Load the guest's FPU state if userspace is accessing MSRs whose values are managed by XSAVES so that the MSR helper, e.g. vmx_{get,set}_msr(), can simply do {RD,WR}MSR to access the guest's value. Note that guest_cpuid_has() is not queried as host userspace is allowed to access MSRs that have not been exposed to the guest, e.g. it might do KVM_SET_MSRS prior to KVM_SET_CPUID2. Signed-off-by: Sean Christopherson Co-developed-by: Yang Weijiang Signed-off-by: Yang Weijiang --- arch/x86/kvm/x86.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 5be6fad6e08d..99e5b1df8555 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -109,6 +109,8 @@ static void enter_smm(struct kvm_vcpu *vcpu); static void __kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags); static void store_regs(struct kvm_vcpu *vcpu); static int sync_regs(struct kvm_vcpu *vcpu); +static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu); +static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu); struct kvm_x86_ops *kvm_x86_ops __read_mostly; EXPORT_SYMBOL_GPL(kvm_x86_ops); @@ -3249,6 +3251,12 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) } EXPORT_SYMBOL_GPL(kvm_get_msr_common); +static bool is_xsaves_msr(u32 index) +{ + return index == MSR_IA32_U_CET || + (index >= MSR_IA32_PL0_SSP && index <= MSR_IA32_PL3_SSP); +} + /* * Read or write a bunch of msrs. All parameters are kernel addresses. * @@ -3259,11 +3267,20 @@ static int __msr_io(struct kvm_vcpu *vcpu, struct kvm_msrs *msrs, int (*do_msr)(struct kvm_vcpu *vcpu, unsigned index, u64 *data)) { + bool fpu_loaded = false; int i; - for (i = 0; i < msrs->nmsrs; ++i) + for (i = 0; i < msrs->nmsrs; ++i) { + if (vcpu && !fpu_loaded && supported_xss && + is_xsaves_msr(entries[i].index)) { + kvm_load_guest_fpu(vcpu); + fpu_loaded = true; + } if (do_msr(vcpu, entries[i].index, &entries[i].data)) break; + } + if (fpu_loaded) + kvm_put_guest_fpu(vcpu); return i; } From patchwork Fri Mar 20 03:43:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11448371 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C76C8139A for ; Fri, 20 Mar 2020 03:41:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B18C12072C for ; Fri, 20 Mar 2020 03:41:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727260AbgCTDk6 (ORCPT ); Thu, 19 Mar 2020 23:40:58 -0400 Received: from mga01.intel.com ([192.55.52.88]:27908 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727297AbgCTDkz (ORCPT ); Thu, 19 Mar 2020 23:40:55 -0400 IronPort-SDR: asnKp8lZZhCTf9Yo8Z10t3fjNiJfbSyuOHvZ9BaOOFZFVwFOIIzixjzNAtvvbVGY+1I9/OUdRa MKYmDwKPXCbQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2020 20:40:55 -0700 IronPort-SDR: lWVD00z1ek6ShRPwbtlz0FlWrtVEnNPy2iA5RC+eyKDUqgKmse4ThHbOBG3TvVUDUXGBLT/ziW oZ5wv+dHZClg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,283,1580803200"; d="scan'208";a="263945605" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.128]) by orsmga002.jf.intel.com with ESMTP; 19 Mar 2020 20:40:53 -0700 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, sean.j.christopherson@intel.com, pbonzini@redhat.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v10 6/8] KVM: X86: Add userspace access interface for CET MSRs Date: Fri, 20 Mar 2020 11:43:39 +0800 Message-Id: <20200320034342.26610-7-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20200320034342.26610-1-weijiang.yang@intel.com> References: <20200320034342.26610-1-weijiang.yang@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org There're two different places storing Guest CET states, states managed with XSAVES/XRSTORS, as restored/saved in previous patch, can be read/write directly from/to the MSRs. For those stored in VMCS fields, they're access via vmcs_read/ vmcs_write. To correctly read/write the CET MSRs, it's necessary to check whether the kernel FPU context switch happened and reload guest FPU context if needed. Suggested-by: Sean Christopherson Signed-off-by: Yang Weijiang Reported-by: kbuild test robot --- arch/x86/kvm/vmx/vmx.c | 133 +++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 11 ++++ 2 files changed, 144 insertions(+) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index e7ac776c808f..2654bd099fe6 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1809,6 +1809,91 @@ static int vmx_get_msr_feature(struct kvm_msr_entry *msr) } } +static void vmx_get_xsave_msr(struct msr_data *msr_info) +{ + local_irq_disable(); + if (test_thread_flag(TIF_NEED_FPU_LOAD)) + switch_fpu_return(); + rdmsrl(msr_info->index, msr_info->data); + local_irq_enable(); +} + +static void vmx_set_xsave_msr(struct msr_data *msr_info) +{ + local_irq_disable(); + if (test_thread_flag(TIF_NEED_FPU_LOAD)) + switch_fpu_return(); + wrmsrl(msr_info->index, msr_info->data); + local_irq_enable(); +} + +#define CET_MSR_RSVD_BITS_1 GENMASK(1, 0) +#define CET_MSR_RSVD_BITS_2 GENMASK(9, 6) + +static bool cet_check_msr_write(struct kvm_vcpu *vcpu, + struct msr_data *msr, + u64 mask) +{ + u64 data = msr->data; + u32 high_word = data >> 32; + + if (data & mask) + return false; + + if (!is_64_bit_mode(vcpu) && high_word) + return false; + + return true; +} + +static bool cet_check_ssp_msr_access(struct kvm_vcpu *vcpu, + struct msr_data *msr) +{ + u32 index = msr->index; + + if (!boot_cpu_has(X86_FEATURE_SHSTK)) + return false; + + if (!msr->host_initiated && + !guest_cpuid_has(vcpu, X86_FEATURE_SHSTK)) + return false; + + if (index == MSR_IA32_INT_SSP_TAB) + return true; + + if (index == MSR_IA32_PL3_SSP) { + if (!(supported_xss & XFEATURE_MASK_CET_USER)) + return false; + } else if (!(supported_xss & XFEATURE_MASK_CET_KERNEL)) { + return false; + } + + return true; +} + +static bool cet_check_ctl_msr_access(struct kvm_vcpu *vcpu, + struct msr_data *msr) +{ + u32 index = msr->index; + + if (!boot_cpu_has(X86_FEATURE_SHSTK) && + !boot_cpu_has(X86_FEATURE_IBT)) + return false; + + if (!msr->host_initiated && + !guest_cpuid_has(vcpu, X86_FEATURE_SHSTK) && + !guest_cpuid_has(vcpu, X86_FEATURE_IBT)) + return false; + + if (index == MSR_IA32_U_CET) { + if (!(supported_xss & XFEATURE_MASK_CET_USER)) + return false; + } else if (!(supported_xss & XFEATURE_MASK_CET_KERNEL)) { + return false; + } + + return true; +} /* * Reads an msr value (of 'msr_index') into 'pdata'. * Returns 0 on success, non-0 otherwise. @@ -1941,6 +2026,26 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) else msr_info->data = vmx->pt_desc.guest.addr_a[index / 2]; break; + case MSR_IA32_S_CET: + if (!cet_check_ctl_msr_access(vcpu, msr_info)) + return 1; + msr_info->data = vmcs_readl(GUEST_S_CET); + break; + case MSR_IA32_INT_SSP_TAB: + if (!cet_check_ssp_msr_access(vcpu, msr_info)) + return 1; + msr_info->data = vmcs_readl(GUEST_INTR_SSP_TABLE); + break; + case MSR_IA32_U_CET: + if (!cet_check_ctl_msr_access(vcpu, msr_info)) + return 1; + vmx_get_xsave_msr(msr_info); + break; + case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + if (!cet_check_ssp_msr_access(vcpu, msr_info)) + return 1; + vmx_get_xsave_msr(msr_info); + break; case MSR_TSC_AUX: if (!msr_info->host_initiated && !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP)) @@ -2197,6 +2302,34 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) else vmx->pt_desc.guest.addr_a[index / 2] = data; break; + case MSR_IA32_S_CET: + if (!cet_check_ctl_msr_access(vcpu, msr_info)) + return 1; + if (!cet_check_msr_write(vcpu, msr_info, CET_MSR_RSVD_BITS_2)) + return 1; + vmcs_writel(GUEST_S_CET, data); + break; + case MSR_IA32_INT_SSP_TAB: + if (!cet_check_ctl_msr_access(vcpu, msr_info)) + return 1; + if (!is_64_bit_mode(vcpu)) + return 1; + vmcs_writel(GUEST_INTR_SSP_TABLE, data); + break; + case MSR_IA32_U_CET: + if (!cet_check_ctl_msr_access(vcpu, msr_info)) + return 1; + if (!cet_check_msr_write(vcpu, msr_info, CET_MSR_RSVD_BITS_2)) + return 1; + vmx_set_xsave_msr(msr_info); + break; + case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + if (!cet_check_ssp_msr_access(vcpu, msr_info)) + return 1; + if (!cet_check_msr_write(vcpu, msr_info, CET_MSR_RSVD_BITS_1)) + return 1; + vmx_set_xsave_msr(msr_info); + break; case MSR_TSC_AUX: if (!msr_info->host_initiated && !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP)) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 99e5b1df8555..3d5049faac58 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1229,6 +1229,10 @@ static const u32 msrs_to_save_all[] = { MSR_ARCH_PERFMON_EVENTSEL0 + 12, MSR_ARCH_PERFMON_EVENTSEL0 + 13, MSR_ARCH_PERFMON_EVENTSEL0 + 14, MSR_ARCH_PERFMON_EVENTSEL0 + 15, MSR_ARCH_PERFMON_EVENTSEL0 + 16, MSR_ARCH_PERFMON_EVENTSEL0 + 17, + + MSR_IA32_XSS, MSR_IA32_U_CET, MSR_IA32_S_CET, + MSR_IA32_PL0_SSP, MSR_IA32_PL1_SSP, MSR_IA32_PL2_SSP, + MSR_IA32_PL3_SSP, MSR_IA32_INT_SSP_TAB, }; static u32 msrs_to_save[ARRAY_SIZE(msrs_to_save_all)]; @@ -1504,6 +1508,13 @@ static int __kvm_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data, * invokes 64-bit SYSENTER. */ data = get_canonical(data, vcpu_virt_addr_bits(vcpu)); + break; + case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + case MSR_IA32_U_CET: + case MSR_IA32_S_CET: + case MSR_IA32_INT_SSP_TAB: + if (is_noncanonical_address(data, vcpu)) + return 1; } msr.data = data; From patchwork Fri Mar 20 03:43:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11448373 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8DBD592A for ; Fri, 20 Mar 2020 03:41:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6FE582076C for ; Fri, 20 Mar 2020 03:41:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727391AbgCTDlM (ORCPT ); Thu, 19 Mar 2020 23:41:12 -0400 Received: from mga01.intel.com ([192.55.52.88]:27908 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726842AbgCTDk6 (ORCPT ); Thu, 19 Mar 2020 23:40:58 -0400 IronPort-SDR: C8fOYa9/pSrTcR9FxR6dQw7+eqGsrCh+oo3Wct3yKLpBL+22sSdOh5a3iUEbj1ihY/3FQezBT4 ulwjQwk3AyhQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2020 20:40:57 -0700 IronPort-SDR: W5ZstaUtz9maRJDfTqM6VJdpg6BetG4lmAq+nuI1GkdcH+I+YVx08Xm0bxJkb599L2/8GxUelu r/QYBLjoHNgQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,283,1580803200"; d="scan'208";a="263945619" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.128]) by orsmga002.jf.intel.com with ESMTP; 19 Mar 2020 20:40:55 -0700 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, sean.j.christopherson@intel.com, pbonzini@redhat.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v10 7/8] KVM: VMX: Enable CET support for nested VM Date: Fri, 20 Mar 2020 11:43:40 +0800 Message-Id: <20200320034342.26610-8-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20200320034342.26610-1-weijiang.yang@intel.com> References: <20200320034342.26610-1-weijiang.yang@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org CET MSRs are passed to guests for performance consideration. Configure the MSRs to match L0/L1 settings so that nested VM is able to run with CET. Signed-off-by: Yang Weijiang --- arch/x86/kvm/vmx/nested.c | 41 +++++++++++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/vmcs12.c | 6 ++++++ arch/x86/kvm/vmx/vmcs12.h | 14 ++++++++++++- arch/x86/kvm/vmx/vmx.c | 1 + 4 files changed, 59 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index e47eb7c0fbae..a71ef33de55f 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -627,6 +627,41 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, nested_vmx_disable_intercept_for_msr(msr_bitmap_l1, msr_bitmap_l0, MSR_KERNEL_GS_BASE, MSR_TYPE_RW); + /* Pass CET MSRs to nested VM if L0 and L1 are set to pass-through. */ + if (!msr_write_intercepted_l01(vcpu, MSR_IA32_U_CET)) + nested_vmx_disable_intercept_for_msr( + msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_U_CET, MSR_TYPE_RW); + + if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL3_SSP)) + nested_vmx_disable_intercept_for_msr( + msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_PL3_SSP, MSR_TYPE_RW); + + if (!msr_write_intercepted_l01(vcpu, MSR_IA32_S_CET)) + nested_vmx_disable_intercept_for_msr( + msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_S_CET, MSR_TYPE_RW); + + if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL0_SSP)) + nested_vmx_disable_intercept_for_msr( + msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_PL0_SSP, MSR_TYPE_RW); + + if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL1_SSP)) + nested_vmx_disable_intercept_for_msr( + msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_PL1_SSP, MSR_TYPE_RW); + + if (!msr_write_intercepted_l01(vcpu, MSR_IA32_PL2_SSP)) + nested_vmx_disable_intercept_for_msr( + msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_PL2_SSP, MSR_TYPE_RW); + + if (!msr_write_intercepted_l01(vcpu, MSR_IA32_INT_SSP_TAB)) + nested_vmx_disable_intercept_for_msr( + msr_bitmap_l1, msr_bitmap_l0, + MSR_IA32_INT_SSP_TAB, MSR_TYPE_RW); /* * Checking the L0->L1 bitmap is trying to verify two things: * @@ -6040,7 +6075,8 @@ void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps) msrs->exit_ctls_high |= VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR | VM_EXIT_LOAD_IA32_EFER | VM_EXIT_SAVE_IA32_EFER | - VM_EXIT_SAVE_VMX_PREEMPTION_TIMER | VM_EXIT_ACK_INTR_ON_EXIT; + VM_EXIT_SAVE_VMX_PREEMPTION_TIMER | VM_EXIT_ACK_INTR_ON_EXIT | + VM_EXIT_LOAD_HOST_CET_STATE; /* We support free control of debug control saving. */ msrs->exit_ctls_low &= ~VM_EXIT_SAVE_DEBUG_CONTROLS; @@ -6057,7 +6093,8 @@ void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps) #endif VM_ENTRY_LOAD_IA32_PAT; msrs->entry_ctls_high |= - (VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR | VM_ENTRY_LOAD_IA32_EFER); + (VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR | VM_ENTRY_LOAD_IA32_EFER | + VM_ENTRY_LOAD_GUEST_CET_STATE); /* We support free control of debug control loading. */ msrs->entry_ctls_low &= ~VM_ENTRY_LOAD_DEBUG_CONTROLS; diff --git a/arch/x86/kvm/vmx/vmcs12.c b/arch/x86/kvm/vmx/vmcs12.c index 53dfb401316d..82b82bebeee0 100644 --- a/arch/x86/kvm/vmx/vmcs12.c +++ b/arch/x86/kvm/vmx/vmcs12.c @@ -141,6 +141,9 @@ const unsigned short vmcs_field_to_offset_table[] = { FIELD(GUEST_PENDING_DBG_EXCEPTIONS, guest_pending_dbg_exceptions), FIELD(GUEST_SYSENTER_ESP, guest_sysenter_esp), FIELD(GUEST_SYSENTER_EIP, guest_sysenter_eip), + FIELD(GUEST_S_CET, guest_s_cet), + FIELD(GUEST_SSP, guest_ssp), + FIELD(GUEST_INTR_SSP_TABLE, guest_ssp_tbl), FIELD(HOST_CR0, host_cr0), FIELD(HOST_CR3, host_cr3), FIELD(HOST_CR4, host_cr4), @@ -153,5 +156,8 @@ const unsigned short vmcs_field_to_offset_table[] = { FIELD(HOST_IA32_SYSENTER_EIP, host_ia32_sysenter_eip), FIELD(HOST_RSP, host_rsp), FIELD(HOST_RIP, host_rip), + FIELD(HOST_S_CET, host_s_cet), + FIELD(HOST_SSP, host_ssp), + FIELD(HOST_INTR_SSP_TABLE, host_ssp_tbl), }; const unsigned int nr_vmcs12_fields = ARRAY_SIZE(vmcs_field_to_offset_table); diff --git a/arch/x86/kvm/vmx/vmcs12.h b/arch/x86/kvm/vmx/vmcs12.h index d0c6df373f67..62b7be68f05c 100644 --- a/arch/x86/kvm/vmx/vmcs12.h +++ b/arch/x86/kvm/vmx/vmcs12.h @@ -118,7 +118,13 @@ struct __packed vmcs12 { natural_width host_ia32_sysenter_eip; natural_width host_rsp; natural_width host_rip; - natural_width paddingl[8]; /* room for future expansion */ + natural_width host_s_cet; + natural_width host_ssp; + natural_width host_ssp_tbl; + natural_width guest_s_cet; + natural_width guest_ssp; + natural_width guest_ssp_tbl; + natural_width paddingl[2]; /* room for future expansion */ u32 pin_based_vm_exec_control; u32 cpu_based_vm_exec_control; u32 exception_bitmap; @@ -301,6 +307,12 @@ static inline void vmx_check_vmcs12_offsets(void) CHECK_OFFSET(host_ia32_sysenter_eip, 656); CHECK_OFFSET(host_rsp, 664); CHECK_OFFSET(host_rip, 672); + CHECK_OFFSET(host_s_cet, 680); + CHECK_OFFSET(host_ssp, 688); + CHECK_OFFSET(host_ssp_tbl, 696); + CHECK_OFFSET(guest_s_cet, 704); + CHECK_OFFSET(guest_ssp, 712); + CHECK_OFFSET(guest_ssp_tbl, 720); CHECK_OFFSET(pin_based_vm_exec_control, 744); CHECK_OFFSET(cpu_based_vm_exec_control, 748); CHECK_OFFSET(exception_bitmap, 752); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 2654bd099fe6..1de03da2bda6 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7141,6 +7141,7 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) cr4_fixed1_update(X86_CR4_PKE, ecx, feature_bit(PKU)); cr4_fixed1_update(X86_CR4_UMIP, ecx, feature_bit(UMIP)); cr4_fixed1_update(X86_CR4_LA57, ecx, feature_bit(LA57)); + cr4_fixed1_update(X86_CR4_CET, ecx, feature_bit(SHSTK)); #undef cr4_fixed1_update } From patchwork Fri Mar 20 03:43:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 11448369 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0BF3C139A for ; Fri, 20 Mar 2020 03:41:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EC5C82076C for ; Fri, 20 Mar 2020 03:41:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727380AbgCTDlA (ORCPT ); Thu, 19 Mar 2020 23:41:00 -0400 Received: from mga01.intel.com ([192.55.52.88]:27908 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727120AbgCTDlA (ORCPT ); Thu, 19 Mar 2020 23:41:00 -0400 IronPort-SDR: WO4nwzeyq+4PvF7Rza97GKV/TbRalAAGy3xeSb7CZRGTOhD2Ce14y52345UGJkOlfz7oGuwalw IaMvB8F9HPHQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2020 20:40:59 -0700 IronPort-SDR: A0suKYNLrI9mEJUhorY+5BRwYs46z+GHZJ1e3nAGsh46rBqP+zVHMtT8TPwQ0FPtKc8sDadN04 ZyWh8m4VyBww== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,283,1580803200"; d="scan'208";a="263945626" Received: from local-michael-cet-test.sh.intel.com ([10.239.159.128]) by orsmga002.jf.intel.com with ESMTP; 19 Mar 2020 20:40:57 -0700 From: Yang Weijiang To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, sean.j.christopherson@intel.com, pbonzini@redhat.com, jmattson@google.com Cc: yu.c.zhang@linux.intel.com, Yang Weijiang Subject: [PATCH v10 8/8] KVM: X86: Set CET feature bits for CPUID enumeration Date: Fri, 20 Mar 2020 11:43:41 +0800 Message-Id: <20200320034342.26610-9-weijiang.yang@intel.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20200320034342.26610-1-weijiang.yang@intel.com> References: <20200320034342.26610-1-weijiang.yang@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Set the feature bits so that CET capabilities can be see in guest via CPUID enumeration. Add CR4.CET bit support in order to allow guest set CET master control bit. Signed-off-by: Yang Weijiang Reported-by: kbuild test robot --- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/cpuid.c | 4 ++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 24c90ea5ddbd..3b6dba7e610e 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -95,7 +95,8 @@ | X86_CR4_PGE | X86_CR4_PCE | X86_CR4_OSFXSR | X86_CR4_PCIDE \ | X86_CR4_OSXSAVE | X86_CR4_SMEP | X86_CR4_FSGSBASE \ | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_VMXE \ - | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP)) + | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP \ + | X86_CR4_CET)) #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 71703d9277ee..9e19775c4105 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -363,6 +363,10 @@ void kvm_set_cpu_caps(void) kvm_cpu_cap_set(X86_FEATURE_INTEL_STIBP); if (boot_cpu_has(X86_FEATURE_AMD_SSBD)) kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL_SSBD); + if (boot_cpu_has(X86_FEATURE_IBT)) + kvm_cpu_cap_set(X86_FEATURE_IBT); + if (boot_cpu_has(X86_FEATURE_SHSTK)) + kvm_cpu_cap_set(X86_FEATURE_SHSTK); kvm_cpu_cap_mask(CPUID_7_1_EAX, F(AVX512_BF16)