[v2,17/18] KVM: nVMX: add option to perform early consistency checks via H/W

KVM defers many VMX consistency checks to the CPU, ostensibly for
performance reasons[1], including checks that result in VMFail (as
opposed to VMExit).  This behavior may be undesirable for some users
since this means KVM detects certain classes of VMFail only after it
has processed guest state, e.g. emulated MSR load-on-entry.  Because
there is a strict ordering between checks that cause VMFail and those
that cause VMExit, i.e. all VMFail checks are performed before any
checks that cause VMExit, we can detect all VMFail conditions via a
dry run of sorts.

After preparing vmcs02 with all state needed to pass the VMFail
consistency checks, optionally do a "test" VMEnter with an invalid
GUEST_RIP.  If the VMEnter results in a VMExit (due to bad guest
state), then we can safely say that the nested VMEnter should not
VMFail, i.e. any VMFail encountered in nested_vmx_vmexit() must
be due to an L0 bug.

Unfortunately, since the "passing" case causes a VMExit, KVM must
be extra diligent to ensure that host state is restored, e.g. DR7
and RFLAGS are reset on VMExit.  Failure to restore RFLAGS.IF is
particularly fatal.

And of course the extra VMEnter and VMExit impacts performance.
The raw overhead of the early consistency checks is ~6% on modern
hardware (though this could easily vary based on configuration),
while the added latency observed from the L1 VMM is ~10%.  The
early consistency checks do not occur in a vacuum, e.g. spending
more time in L0 can lead to more interrupts being serviced while
emulating VMEnter, thereby increasing the latency observed by L1.

Add a module param, early_consistency_checks, to provide control
over whether or not VMX performs the early consistency checks.
In addition to standard on/off behavior, the param accepts a value
of -1, which is essentialy an "auto" setting whereby KVM does
the early checks only when it thinks it's running on bare metal.
When running nested, doing early checks is of dubious value since
the resulting behavior is heavily dependent on L0.  In the future,
the "auto" setting could also be used to default to skipping the
early hardware checks for certain configurations/platforms if KVM
reaches a state where it has 100% coverage of VMFail conditions.

[1] To my knowledge no one has implemented and tested full software
    emulation of the VMFail consistency checks.  Until that happens,
    one can only speculate about the actual performance overhead of
    doing all VMFail consistency checks in software.  Obviously any
    code is slower than no code, but in the grand scheme of nested
    virtualization it's entirely possible the overhead is negligible.

Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
---
 arch/x86/kvm/vmx.c | 151 +++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 146 insertions(+), 5 deletions(-)

Message ID	20180828160459.14093-18-sean.j.christopherson@intel.com (mailing list archive)
State	New, archived
Headers	show Return-Path: <kvm-owner@kernel.org> From: Sean Christopherson <sean.j.christopherson@intel.com> To: Paolo Bonzini <pbonzini@redhat.com>, =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= <rkrcmar@redhat.com> Cc: kvm@vger.kernel.org, Jim Mattson <jmattson@google.com>, Sean Christopherson <sean.j.christopherson@intel.com> Subject: [PATCH v2 17/18] KVM: nVMX: add option to perform early consistency checks via H/W Date: Tue, 28 Aug 2018 09:04:58 -0700 Message-Id: <20180828160459.14093-18-sean.j.christopherson@intel.com> In-Reply-To: <20180828160459.14093-1-sean.j.christopherson@intel.com> References: <20180828160459.14093-1-sean.j.christopherson@intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk
Series	KVM: nVMX: add option to perform early consistency checks via H/W \| expand [v2,00/18] KVM: nVMX: add option to perform early consistency checks via H/W [v2,01/18] KVM: nVMX: move host EFER consistency checks to VMFail path [v2,02/18] KVM: nVMX: move vmcs12 EPTP consistency check to check_vmentry_prereqs() [v2,03/18] KVM: nVMX: use vm_exit_controls_init() to write exit controls for vmcs02 [v2,04/18] KVM: nVMX: reset cache/shadows on nested consistency check VMExit [v2,05/18] KVM: vmx: do not unconditionally clear EFER switching [v2,06/18] KVM: nVMX: try to set EFER bits correctly when init'ing entry controls [v2,07/18] KVM: nVMX: rename enter_vmx_non_root_mode to nested_vmx_enter_non_root_mode [v2,08/18] KVM: nVMX: move check_vmentry_postreqs() call to nested_vmx_enter_non_root_mode() [v2,09/18] KVM: nVMX: assimilate nested_vmx_entry_failure() into nested_vmx_enter_non_root_mode() [v2,10/18] KVM: nVMX: split pieces of prepare_vmcs02() to prepare_vmcs02_early() [v2,11/18] KVM: nVMX: do early preparation of vmcs02 before check_vmentry_postreqs() [v2,12/18] KVM: vVMX: rename label for post-enter_guest_mode consistency check [v2,13/18] KVM: nVMX: do not skip VMEnter instruction that succeeds [v2,14/18] KVM: nVMX: do not call nested_vmx_succeed() for consistency check VMExit [v2,15/18] KVM: nVMX: call kvm_skip_emulated_instruction in nested_vmx_{fail,succeed} [v2,16/18] KVM: vmx: write HOST_IA32_EFER in vmx_set_constant_host_state() [v2,17/18] KVM: nVMX: add option to perform early consistency checks via H/W [v2,18/18] KVM: nVMX: WARN if nested run hits VMFail with early consistency checks enabled

[v2,17/18] KVM: nVMX: add option to perform early consistency checks via H/W

Commit Message

Comments

Patch