| Message ID | 20230914063325.85503-23-weijiang.yang@intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Enable CET Virtualization | expand |
On Thu, 2023-09-14 at 02:33 -0400, Yang Weijiang wrote: > Save constant values to HOST_{S_CET,SSP,INTR_SSP_TABLE} field explicitly. > Kernel IBT is supported and the setting in MSR_IA32_S_CET is static after > post-boot(The exception is BIOS call case but vCPU thread never across it) > and KVM doesn't need to refresh HOST_S_CET field before every VM-Enter/ > VM-Exit sequence. > > Host supervisor shadow stack is not enabled now and SSP is not accessible > to kernel mode, thus it's safe to set host IA32_INT_SSP_TAB/SSP VMCS field > to 0s. When shadow stack is enabled for CPL3, SSP is reloaded from PL3_SSP > before it exits to userspace. Check SDM Vol 2A/B Chapter 3/4 for SYSCALL/ > SYSRET/SYSENTER SYSEXIT/RDSSP/CALL etc. > > Prevent KVM module loading if host supervisor shadow stack SHSTK_EN is set > in MSR_IA32_S_CET as KVM cannot co-exit with it correctly. > > Suggested-by: Sean Christopherson <seanjc@google.com> > Suggested-by: Chao Gao <chao.gao@intel.com> > Signed-off-by: Yang Weijiang <weijiang.yang@intel.com> > --- > arch/x86/kvm/vmx/capabilities.h | 4 ++++ > arch/x86/kvm/vmx/vmx.c | 15 +++++++++++++++ > arch/x86/kvm/x86.c | 14 ++++++++++++++ > arch/x86/kvm/x86.h | 1 + > 4 files changed, 34 insertions(+) > > diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h > index 41a4533f9989..ee8938818c8a 100644 > --- a/arch/x86/kvm/vmx/capabilities.h > +++ b/arch/x86/kvm/vmx/capabilities.h > @@ -106,6 +106,10 @@ static inline bool cpu_has_load_perf_global_ctrl(void) > return vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL; > } > > +static inline bool cpu_has_load_cet_ctrl(void) > +{ > + return (vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_CET_STATE); > +} > static inline bool cpu_has_vmx_mpx(void) > { > return vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_BNDCFGS; > diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c > index 30373258573d..9ccc2c552f55 100644 > --- a/arch/x86/kvm/vmx/vmx.c > +++ b/arch/x86/kvm/vmx/vmx.c > @@ -4375,6 +4375,21 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx) > > if (cpu_has_load_ia32_efer()) > vmcs_write64(HOST_IA32_EFER, host_efer); > + > + /* > + * Supervisor shadow stack is not enabled on host side, i.e., > + * host IA32_S_CET.SHSTK_EN bit is guaranteed to 0 now, per SDM > + * description(RDSSP instruction), SSP is not readable in CPL0, > + * so resetting the two registers to 0s at VM-Exit does no harm > + * to kernel execution. When execution flow exits to userspace, > + * SSP is reloaded from IA32_PL3_SSP. Check SDM Vol.2A/B Chapter > + * 3 and 4 for details. > + */ > + if (cpu_has_load_cet_ctrl()) { > + vmcs_writel(HOST_S_CET, host_s_cet); > + vmcs_writel(HOST_SSP, 0); > + vmcs_writel(HOST_INTR_SSP_TABLE, 0); > + } > } > > void set_cr4_guest_host_mask(struct vcpu_vmx *vmx) > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index c85ee42ab4f1..231d4a7b6f3d 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -114,6 +114,8 @@ static u64 __read_mostly efer_reserved_bits = ~((u64)EFER_SCE); > #endif > > static u64 __read_mostly cr4_reserved_bits = CR4_RESERVED_BITS; > +u64 __read_mostly host_s_cet; > +EXPORT_SYMBOL_GPL(host_s_cet); > > #define KVM_EXIT_HYPERCALL_VALID_MASK (1 << KVM_HC_MAP_GPA_RANGE) > > @@ -9618,6 +9620,18 @@ static int __kvm_x86_vendor_init(struct kvm_x86_init_ops *ops) > return -EIO; > } > > + if (boot_cpu_has(X86_FEATURE_SHSTK)) { > + rdmsrl(MSR_IA32_S_CET, host_s_cet); > + /* > + * Linux doesn't yet support supervisor shadow stacks (SSS), so > + * KVM doesn't save/restore the associated MSRs, i.e. KVM may > + * clobber the host values. Yell and refuse to load if SSS is > + * unexpectedly enabled, e.g. to avoid crashing the host. > + */ > + if (WARN_ON_ONCE(host_s_cet & CET_SHSTK_EN)) > + return -EIO; This is a good idea. > + } > + > x86_emulator_cache = kvm_alloc_emulator_cache(); > if (!x86_emulator_cache) { > pr_err("failed to allocate cache for x86 emulator\n"); > diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h > index 9a8e3a84eaf4..0d5f673338dd 100644 > --- a/arch/x86/kvm/x86.h > +++ b/arch/x86/kvm/x86.h > @@ -324,6 +324,7 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu); > extern u64 host_xcr0; > extern u64 host_xss; > extern u64 host_arch_capabilities; > +extern u64 host_s_cet; > > extern struct kvm_caps kvm_caps; > Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com> Best regards, Maxim Levitsky
diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h index 41a4533f9989..ee8938818c8a 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -106,6 +106,10 @@ static inline bool cpu_has_load_perf_global_ctrl(void) return vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL; } +static inline bool cpu_has_load_cet_ctrl(void) +{ + return (vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_CET_STATE); +} static inline bool cpu_has_vmx_mpx(void) { return vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_BNDCFGS; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 30373258573d..9ccc2c552f55 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -4375,6 +4375,21 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx) if (cpu_has_load_ia32_efer()) vmcs_write64(HOST_IA32_EFER, host_efer); + + /* + * Supervisor shadow stack is not enabled on host side, i.e., + * host IA32_S_CET.SHSTK_EN bit is guaranteed to 0 now, per SDM + * description(RDSSP instruction), SSP is not readable in CPL0, + * so resetting the two registers to 0s at VM-Exit does no harm + * to kernel execution. When execution flow exits to userspace, + * SSP is reloaded from IA32_PL3_SSP. Check SDM Vol.2A/B Chapter + * 3 and 4 for details. + */ + if (cpu_has_load_cet_ctrl()) { + vmcs_writel(HOST_S_CET, host_s_cet); + vmcs_writel(HOST_SSP, 0); + vmcs_writel(HOST_INTR_SSP_TABLE, 0); + } } void set_cr4_guest_host_mask(struct vcpu_vmx *vmx) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c85ee42ab4f1..231d4a7b6f3d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -114,6 +114,8 @@ static u64 __read_mostly efer_reserved_bits = ~((u64)EFER_SCE); #endif static u64 __read_mostly cr4_reserved_bits = CR4_RESERVED_BITS; +u64 __read_mostly host_s_cet; +EXPORT_SYMBOL_GPL(host_s_cet); #define KVM_EXIT_HYPERCALL_VALID_MASK (1 << KVM_HC_MAP_GPA_RANGE) @@ -9618,6 +9620,18 @@ static int __kvm_x86_vendor_init(struct kvm_x86_init_ops *ops) return -EIO; } + if (boot_cpu_has(X86_FEATURE_SHSTK)) { + rdmsrl(MSR_IA32_S_CET, host_s_cet); + /* + * Linux doesn't yet support supervisor shadow stacks (SSS), so + * KVM doesn't save/restore the associated MSRs, i.e. KVM may + * clobber the host values. Yell and refuse to load if SSS is + * unexpectedly enabled, e.g. to avoid crashing the host. + */ + if (WARN_ON_ONCE(host_s_cet & CET_SHSTK_EN)) + return -EIO; + } + x86_emulator_cache = kvm_alloc_emulator_cache(); if (!x86_emulator_cache) { pr_err("failed to allocate cache for x86 emulator\n"); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 9a8e3a84eaf4..0d5f673338dd 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -324,6 +324,7 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu); extern u64 host_xcr0; extern u64 host_xss; extern u64 host_arch_capabilities; +extern u64 host_s_cet; extern struct kvm_caps kvm_caps;
Save constant values to HOST_{S_CET,SSP,INTR_SSP_TABLE} field explicitly. Kernel IBT is supported and the setting in MSR_IA32_S_CET is static after post-boot(The exception is BIOS call case but vCPU thread never across it) and KVM doesn't need to refresh HOST_S_CET field before every VM-Enter/ VM-Exit sequence. Host supervisor shadow stack is not enabled now and SSP is not accessible to kernel mode, thus it's safe to set host IA32_INT_SSP_TAB/SSP VMCS field to 0s. When shadow stack is enabled for CPL3, SSP is reloaded from PL3_SSP before it exits to userspace. Check SDM Vol 2A/B Chapter 3/4 for SYSCALL/ SYSRET/SYSENTER SYSEXIT/RDSSP/CALL etc. Prevent KVM module loading if host supervisor shadow stack SHSTK_EN is set in MSR_IA32_S_CET as KVM cannot co-exit with it correctly. Suggested-by: Sean Christopherson <seanjc@google.com> Suggested-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Yang Weijiang <weijiang.yang@intel.com> --- arch/x86/kvm/vmx/capabilities.h | 4 ++++ arch/x86/kvm/vmx/vmx.c | 15 +++++++++++++++ arch/x86/kvm/x86.c | 14 ++++++++++++++ arch/x86/kvm/x86.h | 1 + 4 files changed, 34 insertions(+)