[v11,73/81] KVM: introspection: restore the state of MSR interception on unhook

Message ID	20201207204622.15258-74-alazar@bitdefender.com (mailing list archive)
State	New, archived
Headers	show Return-Path: <kvm-owner@kernel.org> From: =?utf-8?q?Adalbert_Laz=C4=83r?= <alazar@bitdefender.com> To: kvm@vger.kernel.org Cc: virtualization@lists.linux-foundation.org, Paolo Bonzini <pbonzini@redhat.com>, =?utf-8?b?TmljdciZb3IgQ8OuyJt1?= <nicu.citu@icloud.com>, =?utf-8?q?Adalbert?= =?utf-8?q?_Laz=C4=83r?= <alazar@bitdefender.com> Subject: [PATCH v11 73/81] KVM: introspection: restore the state of MSR interception on unhook Date: Mon, 7 Dec 2020 22:46:14 +0200 Message-Id: <20201207204622.15258-74-alazar@bitdefender.com> In-Reply-To: <20201207204622.15258-1-alazar@bitdefender.com> References: <20201207204622.15258-1-alazar@bitdefender.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk
Series	VM introspection \| expand [v11,00/81] VM introspection [v11,01/81] KVM: UAPI: add error codes used by the VM introspection code [v11,02/81] KVM: add kvm_vcpu_kick_and_wait() [v11,03/81] KVM: add kvm_get_max_gfn() [v11,04/81] KVM: doc: fix the hypercalls numbering [v11,05/81] KVM: x86: add kvm_arch_vcpu_get_regs() and kvm_arch_vcpu_get_sregs() [v11,06/81] KVM: x86: add kvm_arch_vcpu_set_regs() [v11,07/81] KVM: x86: avoid injecting #PF when emulate the VMCALL instruction [v11,08/81] KVM: x86: add kvm_x86_ops.bp_intercepted() [v11,09/81] KVM: x86: add kvm_x86_ops.control_cr3_intercept() [v11,10/81] KVM: x86: add kvm_x86_ops.cr3_write_intercepted() [v11,11/81] KVM: x86: add kvm_x86_ops.desc_ctrl_supported() [v11,12/81] KVM: svm: add support for descriptor-table VM-exits [v11,13/81] KVM: x86: add kvm_x86_ops.control_desc_intercept() [v11,14/81] KVM: x86: add kvm_x86_ops.desc_intercepted() [v11,15/81] KVM: x86: add kvm_x86_ops.msr_write_intercepted() [v11,16/81] KVM: x86: svm: use the vmx convention to control the MSR interception [v11,17/81] KVM: x86: add kvm_x86_ops.control_msr_intercept() [v11,18/81] KVM: x86: vmx: use a symbolic constant when checking the exit qualifications [v11,19/81] KVM: x86: save the error code during EPT/NPF exits handling [v11,20/81] KVM: x86: add kvm_x86_ops.fault_gla() [v11,21/81] KVM: x86: add kvm_x86_ops.control_singlestep() [v11,22/81] KVM: x86: export kvm_arch_vcpu_set_guest_debug() [v11,23/81] KVM: x86: extend kvm_mmu_gva_to_gpa_system() with the 'access' parameter [v11,24/81] KVM: x86: export kvm_inject_pending_exception() [v11,25/81] KVM: x86: export kvm_vcpu_ioctl_x86_get_xsave() [v11,26/81] KVM: x86: export kvm_vcpu_ioctl_x86_set_xsave() [v11,27/81] KVM: x86: page track: provide all callbacks with the guest virtual address [v11,28/81] KVM: x86: page track: add track_create_slot() callback [v11,29/81] KVM: x86: page_track: add support for preread, prewrite and preexec [v11,30/81] KVM: x86: wire in the preread/prewrite/preexec page trackers [v11,31/81] KVM: x86: disable gpa_available optimization for fetch and page-walk SPT violations [v11,32/81] KVM: introduce VM introspection [v11,33/81] KVM: introspection: add hook/unhook ioctls [v11,34/81] KVM: introspection: add permission access ioctls [v11,35/81] KVM: introspection: add the read/dispatch message function [v11,36/81] KVM: introspection: add KVMI_GET_VERSION [v11,37/81] KVM: introspection: add KVMI_VM_CHECK_COMMAND and KVMI_VM_CHECK_EVENT [v11,38/81] KVM: introspection: add KVMI_VM_GET_INFO [v11,39/81] KVM: introspection: add KVM_INTROSPECTION_PREUNHOOK [v11,40/81] KVM: introspection: add KVMI_VM_EVENT_UNHOOK [v11,41/81] KVM: introspection: add KVMI_VM_CONTROL_EVENTS [v11,42/81] KVM: introspection: add KVMI_VM_READ_PHYSICAL/KVMI_VM_WRITE_PHYSICAL [v11,43/81] KVM: introspection: add vCPU related data [v11,44/81] KVM: introspection: add a jobs list to every introspected vCPU [v11,45/81] KVM: introspection: handle vCPU introspection requests [v11,46/81] KVM: introspection: handle vCPU commands [v11,47/81] KVM: introspection: add KVMI_VCPU_GET_INFO [v11,48/81] KVM: introspection: add KVMI_VM_PAUSE_VCPU [v11,49/81] KVM: introspection: add support for vCPU events [v11,50/81] KVM: introspection: add KVMI_VCPU_EVENT_PAUSE [v11,51/81] KVM: introspection: add the crash action handling on the event reply [v11,52/81] KVM: introspection: add KVMI_VCPU_CONTROL_EVENTS [v11,53/81] KVM: introspection: add KVMI_VCPU_GET_REGISTERS [v11,54/81] KVM: introspection: add KVMI_VCPU_SET_REGISTERS [v11,55/81] KVM: introspection: add KVMI_VCPU_GET_CPUID [v11,56/81] KVM: introspection: add KVMI_VCPU_EVENT_HYPERCALL [v11,57/81] KVM: introspection: add KVMI_VCPU_EVENT_BREAKPOINT [v11,58/81] KVM: introspection: add cleanup support for vCPUs [v11,59/81] KVM: introspection: restore the state of #BP interception on unhook [v11,60/81] KVM: introspection: add KVMI_VM_CONTROL_CLEANUP [v11,61/81] KVM: introspection: add KVMI_VCPU_CONTROL_CR and KVMI_VCPU_EVENT_CR [v11,62/81] KVM: introspection: restore the state of CR3 interception on unhook [v11,63/81] KVM: introspection: add KVMI_VCPU_INJECT_EXCEPTION + KVMI_VCPU_EVENT_TRAP [v11,64/81] KVM: introspection: add KVMI_VM_GET_MAX_GFN [v11,65/81] KVM: introspection: add KVMI_VCPU_EVENT_XSETBV [v11,66/81] KVM: introspection: add KVMI_VCPU_GET_XCR [v11,67/81] KVM: introspection: add KVMI_VCPU_GET_XSAVE [v11,68/81] KVM: introspection: add KVMI_VCPU_SET_XSAVE [v11,69/81] KVM: introspection: add KVMI_VCPU_GET_MTRR_TYPE [v11,70/81] KVM: introspection: add KVMI_VCPU_EVENT_DESCRIPTOR [v11,71/81] KVM: introspection: restore the state of descriptor-table register interception on unho… [v11,72/81] KVM: introspection: add KVMI_VCPU_CONTROL_MSR and KVMI_VCPU_EVENT_MSR [v11,73/81] KVM: introspection: restore the state of MSR interception on unhook [v11,74/81] KVM: introspection: add KVMI_VM_SET_PAGE_ACCESS [v11,75/81] KVM: introspection: add KVMI_VCPU_EVENT_PF [v11,76/81] KVM: introspection: extend KVMI_GET_VERSION with struct kvmi_features [v11,77/81] KVM: introspection: add KVMI_VCPU_CONTROL_SINGLESTEP [v11,78/81] KVM: introspection: add KVMI_VCPU_EVENT_SINGLESTEP [v11,79/81] KVM: introspection: add KVMI_VCPU_TRANSLATE_GVA [v11,80/81] KVM: introspection: emulate a guest page table walk on SPT violations due to A/D bit up… [v11,81/81] KVM: x86: call the page tracking code on emulation failure

diff --git a/arch/x86/include/asm/kvmi_host.h b/arch/x86/include/asm/kvmi_host.h index 5a4fc5b80907..8822f0310156 100644 --- a/arch/x86/include/asm/kvmi_host.h +++ b/arch/x86/include/asm/kvmi_host.h @@ -26,6 +26,12 @@ struct kvmi_interception { DECLARE_BITMAP(low, KVMI_NUM_MSR); DECLARE_BITMAP(high, KVMI_NUM_MSR); } kvmi_mask; + struct { + DECLARE_BITMAP(low, KVMI_NUM_MSR); + DECLARE_BITMAP(high, KVMI_NUM_MSR); + } kvm_mask; + bool (*monitor_fct)(struct kvm_vcpu *vcpu, u32 msr, + bool enable); } msrw; }; @@ -61,6 +67,8 @@ void kvmi_xsetbv_event(struct kvm_vcpu *vcpu, u8 xcr, bool kvmi_monitor_desc_intercept(struct kvm_vcpu *vcpu, bool enable); bool kvmi_descriptor_event(struct kvm_vcpu *vcpu, u8 descriptor, bool write); bool kvmi_msr_event(struct kvm_vcpu *vcpu, struct msr_data *msr); +bool kvmi_monitor_msrw_intercept(struct kvm_vcpu *vcpu, u32 msr, bool enable); +bool kvmi_msrw_intercept_originator(struct kvm_vcpu *vcpu); #else /* CONFIG_KVM_INTROSPECTION */ @@ -82,6 +90,10 @@ static inline bool kvmi_descriptor_event(struct kvm_vcpu *vcpu, u8 descriptor, bool write) { return true; } static inline bool kvmi_msr_event(struct kvm_vcpu *vcpu, struct msr_data *msr) { return true; } +static inline bool kvmi_monitor_msrw_intercept(struct kvm_vcpu *vcpu, u32 msr, + bool enable) { return false; } +static inline bool kvmi_msrw_intercept_originator(struct kvm_vcpu *vcpu) + { return false; } #endif /* CONFIG_KVM_INTROSPECTION */ diff --git a/arch/x86/kvm/kvmi.c b/arch/x86/kvm/kvmi.c index ce29e01ba7a6..e325dad88dbb 100644 --- a/arch/x86/kvm/kvmi.c +++ b/arch/x86/kvm/kvmi.c @@ -345,22 +345,25 @@ static void kvmi_arch_disable_desc_intercept(struct kvm_vcpu *vcpu) vcpu->arch.kvmi->descriptor.kvm_intercepted = false; } -static unsigned long *msr_mask(struct kvm_vcpu *vcpu, unsigned int *msr) +static unsigned long *msr_mask(struct kvm_vcpu *vcpu, unsigned int *msr, + bool kvmi) { switch (*msr) { case 0 ... 0x1fff: - return vcpu->arch.kvmi->msrw.kvmi_mask.low; + return kvmi ? vcpu->arch.kvmi->msrw.kvmi_mask.low : + vcpu->arch.kvmi->msrw.kvm_mask.low; case 0xc0000000 ... 0xc0001fff: *msr &= 0x1fff; - return vcpu->arch.kvmi->msrw.kvmi_mask.high; + return kvmi ? vcpu->arch.kvmi->msrw.kvmi_mask.high : + vcpu->arch.kvmi->msrw.kvm_mask.high; } return NULL; } -static bool test_msr_mask(struct kvm_vcpu *vcpu, unsigned int msr) +static bool test_msr_mask(struct kvm_vcpu *vcpu, unsigned int msr, bool kvmi) { - unsigned long *mask = msr_mask(vcpu, &msr); + unsigned long *mask = msr_mask(vcpu, &msr, kvmi); if (!mask) return false; @@ -368,9 +371,27 @@ static bool test_msr_mask(struct kvm_vcpu *vcpu, unsigned int msr) return !!test_bit(msr, mask); } -static bool msr_control(struct kvm_vcpu *vcpu, unsigned int msr, bool enable) +/* + * Returns true if one side (kvm or kvmi) tries to disable the MSR write + * interception while the other side is still tracking it. + */ +bool kvmi_monitor_msrw_intercept(struct kvm_vcpu *vcpu, u32 msr, bool enable) { - unsigned long *mask = msr_mask(vcpu, &msr); + struct kvmi_interception *arch_vcpui; + + if (!vcpu) + return false; + + arch_vcpui = READ_ONCE(vcpu->arch.kvmi); + + return (arch_vcpui && arch_vcpui->msrw.monitor_fct(vcpu, msr, enable)); +} +EXPORT_SYMBOL(kvmi_monitor_msrw_intercept); + +static bool msr_control(struct kvm_vcpu *vcpu, unsigned int msr, bool enable, + bool kvmi) +{ + unsigned long *mask = msr_mask(vcpu, &msr, kvmi); if (!mask) return false; @@ -383,6 +404,63 @@ static bool msr_control(struct kvm_vcpu *vcpu, unsigned int msr, bool enable) return true; } +static bool msr_intercepted_by_kvmi(struct kvm_vcpu *vcpu, u32 msr) +{ + return test_msr_mask(vcpu, msr, true); +} + +static bool msr_intercepted_by_kvm(struct kvm_vcpu *vcpu, u32 msr) +{ + return test_msr_mask(vcpu, msr, false); +} + +static void record_msr_intercept_status_for_kvmi(struct kvm_vcpu *vcpu, u32 msr, + bool enable) +{ + msr_control(vcpu, msr, enable, true); +} + +static void record_msr_intercept_status_for_kvm(struct kvm_vcpu *vcpu, u32 msr, + bool enable) +{ + msr_control(vcpu, msr, enable, false); +} + +static bool monitor_msrw_fct_kvmi(struct kvm_vcpu *vcpu, u32 msr, bool enable) +{ + bool ret = false; + + if (enable) { + if (kvm_x86_ops.msr_write_intercepted(vcpu, msr)) + record_msr_intercept_status_for_kvm(vcpu, msr, true); + } else { + if (unlikely(!msr_intercepted_by_kvmi(vcpu, msr))) + ret = true; + + if (msr_intercepted_by_kvm(vcpu, msr)) + ret = true; + } + + record_msr_intercept_status_for_kvmi(vcpu, msr, enable); + + return ret; +} + +static bool monitor_msrw_fct_kvm(struct kvm_vcpu *vcpu, u32 msr, bool enable) +{ + bool ret = false; + + if (!(msr_intercepted_by_kvmi(vcpu, msr))) + return false; + + if (!enable) + ret = true; + + record_msr_intercept_status_for_kvm(vcpu, msr, enable); + + return ret; +} + static unsigned int msr_mask_to_base(struct kvm_vcpu *vcpu, unsigned long *mask) { if (mask == vcpu->arch.kvmi->msrw.kvmi_mask.high) @@ -393,12 +471,13 @@ static unsigned int msr_mask_to_base(struct kvm_vcpu *vcpu, unsigned long *mask) void kvmi_control_msrw_intercept(struct kvm_vcpu *vcpu, u32 msr, bool enable) { + vcpu->arch.kvmi->msrw.monitor_fct = monitor_msrw_fct_kvmi; kvm_x86_ops.control_msr_intercept(vcpu, msr, MSR_TYPE_W, enable); - msr_control(vcpu, msr, enable); + vcpu->arch.kvmi->msrw.monitor_fct = monitor_msrw_fct_kvm; } -static void kvmi_arch_disable_msr_intercept(struct kvm_vcpu *vcpu, - unsigned long *mask) +static void kvmi_arch_disable_msrw_intercept(struct kvm_vcpu *vcpu, + unsigned long *mask) { unsigned int msr_base = msr_mask_to_base(vcpu, mask); int offset = -1; @@ -409,8 +488,7 @@ static void kvmi_arch_disable_msr_intercept(struct kvm_vcpu *vcpu, if (offset >= KVMI_NUM_MSR) break; - kvm_x86_ops.control_msr_intercept(vcpu, msr_base + offset, - MSR_TYPE_W, false); + kvmi_control_msrw_intercept(vcpu, msr_base + offset, false); } bitmap_zero(mask, KVMI_NUM_MSR); @@ -462,8 +540,8 @@ static void kvmi_arch_restore_interception(struct kvm_vcpu *vcpu) kvmi_arch_disable_bp_intercept(vcpu); kvmi_arch_disable_cr3w_intercept(vcpu); kvmi_arch_disable_desc_intercept(vcpu); - kvmi_arch_disable_msr_intercept(vcpu, arch_vcpui->msrw.kvmi_mask.low); - kvmi_arch_disable_msr_intercept(vcpu, arch_vcpui->msrw.kvmi_mask.high); + kvmi_arch_disable_msrw_intercept(vcpu, arch_vcpui->msrw.kvmi_mask.low); + kvmi_arch_disable_msrw_intercept(vcpu, arch_vcpui->msrw.kvmi_mask.high); } bool kvmi_arch_clean_up_interception(struct kvm_vcpu *vcpu) @@ -490,12 +568,14 @@ bool kvmi_arch_vcpu_alloc_interception(struct kvm_vcpu *vcpu) arch_vcpui->breakpoint.monitor_fct = monitor_bp_fct_kvm; arch_vcpui->cr3w.monitor_fct = monitor_cr3w_fct_kvm; arch_vcpui->descriptor.monitor_fct = monitor_desc_fct_kvm; + arch_vcpui->msrw.monitor_fct = monitor_msrw_fct_kvm; /* * paired with: * - kvmi_monitor_bp_intercept() * - kvmi_monitor_cr3w_intercept() * - kvmi_monitor_desc_intercept() + * - kvmi_monitor_msrw_intercept() */ smp_wmb(); WRITE_ONCE(vcpu->arch.kvmi, arch_vcpui); @@ -777,6 +857,20 @@ bool kvmi_descriptor_event(struct kvm_vcpu *vcpu, u8 descriptor, bool write) } EXPORT_SYMBOL(kvmi_descriptor_event); +bool kvmi_msrw_intercept_originator(struct kvm_vcpu *vcpu) +{ + struct kvmi_interception *arch_vcpui; + + if (!vcpu) + return false; + + arch_vcpui = READ_ONCE(vcpu->arch.kvmi); + + return (arch_vcpui && + arch_vcpui->msrw.monitor_fct == monitor_msrw_fct_kvmi); +} +EXPORT_SYMBOL(kvmi_msrw_intercept_originator); + static bool __kvmi_msr_event(struct kvm_vcpu *vcpu, struct msr_data *msr) { struct msr_data old_msr = { @@ -787,7 +881,7 @@ static bool __kvmi_msr_event(struct kvm_vcpu *vcpu, struct msr_data *msr) u32 action; bool ret; - if (!test_msr_mask(vcpu, msr->index)) + if (!test_msr_mask(vcpu, msr->index, true)) return true; if (kvm_x86_ops.get_msr(vcpu, &old_msr)) return true; diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 834e4b6c4112..08189e814f59 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -641,6 +641,16 @@ static void set_msr_interception_bitmap(struct kvm_vcpu *vcpu, u32 *msrpm, unsigned long tmp; u32 offset; +#ifdef CONFIG_KVM_INTROSPECTION + if ((type & MSR_TYPE_W) && + kvmi_monitor_msrw_intercept(vcpu, msr, !value)) + type &= ~MSR_TYPE_W; + + /* + * Avoid the below warning for kvmi intercepted msrs. + */ + if (!kvmi_msrw_intercept_originator(vcpu)) +#endif /* CONFIG_KVM_INTROSPECTION */ /* * If this warning triggers extend the direct_access_msrs list at the * beginning of the file diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 245332ce91a5..01d18c9243bc 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3755,6 +3755,12 @@ static __always_inline void vmx_disable_intercept_for_msr(struct kvm_vcpu *vcpu, if (!cpu_has_vmx_msr_bitmap()) return; +#ifdef CONFIG_KVM_INTROSPECTION + if ((type & MSR_TYPE_W) && + kvmi_monitor_msrw_intercept(vcpu, msr, false)) + type &= ~MSR_TYPE_W; +#endif /* CONFIG_KVM_INTROSPECTION */ + if (static_branch_unlikely(&enable_evmcs)) evmcs_touch_msr_bitmap(); @@ -3801,6 +3807,11 @@ static __always_inline void vmx_enable_intercept_for_msr(struct kvm_vcpu *vcpu, if (!cpu_has_vmx_msr_bitmap()) return; +#ifdef CONFIG_KVM_INTROSPECTION + if (type & MSR_TYPE_W) + kvmi_monitor_msrw_intercept(vcpu, msr, true); +#endif /* CONFIG_KVM_INTROSPECTION */ + if (static_branch_unlikely(&enable_evmcs)) evmcs_touch_msr_bitmap();

[v11,73/81] KVM: introspection: restore the state of MSR interception on unhook

Commit Message

Patch