[v2,00/14] KVM: x86: Remove emulation_result enums
mbox series

Message ID 20190827214040.18710-1-sean.j.christopherson@intel.com
Headers show
Series
  • KVM: x86: Remove emulation_result enums
Related show

Message

Sean Christopherson Aug. 27, 2019, 9:40 p.m. UTC
Rework the emulator and its users to handle failure scenarios entirely
within the emulator.

{x86,kvm}_emulate_instruction() currently returns a tri-state value to
indicate success/continue, userspace exit needed, and failure.  The
intent of returning EMULATE_FAIL is to let the caller handle failure in
a manner that is appropriate for the current context.  In practice,
the emulator has ended up with a mixture of failure handling, i.e.
whether or not the emulator takes action on failure is dependent on the
specific flavor of emulation.

The mixed handling has proven to be rather fragile, e.g. many flows
incorrectly assume their specific flavor of emulation cannot fail or
that the emulator sets state to report the failure back to userspace.

Move everything inside the emulator, piece by piece, so that the
emulation routines can return '0' for exit to userspace and '1' for
resume the guest, just like every other VM-Exit handler.

Patch 13/14 is a tangentially related bug fix that conflicts heavily with
this series, so I tacked it on here.

Patch 14/14 documents the emulation types.  I added it as a separate
patch at the very end so that the comments could reference the final
state of the code base, e.g. incorporate the rule change for using
EMULTYPE_SKIP that is introduced in patch 13/14.

v1:
  - https://patchwork.kernel.org/cover/11110331/

v2:
  - Collect reviews. [Vitaly and Liran]
  - Squash VMware emultype changes into a single patch. [Liran]
  - Add comments in VMX/SVM for VMware #GP handling. [Vitaly]
  - Tack on the EPT misconfig bug fix.
  - Add a patch to comment/document the emultypes. [Liran]

Sean Christopherson (14):
  KVM: x86: Relocate MMIO exit stats counting
  KVM: x86: Clean up handle_emulation_failure()
  KVM: x86: Refactor kvm_vcpu_do_singlestep() to remove out param
  KVM: x86: Don't attempt VMWare emulation on #GP with non-zero error
    code
  KVM: x86: Move #GP injection for VMware into x86_emulate_instruction()
  KVM: x86: Add explicit flag for forced emulation on #UD
  KVM: x86: Move #UD injection for failed emulation into emulation code
  KVM: x86: Exit to userspace on emulation skip failure
  KVM: x86: Handle emulation failure directly in kvm_task_switch()
  KVM: x86: Move triple fault request into RM int injection
  KVM: VMX: Remove EMULATE_FAIL handling in handle_invalid_guest_state()
  KVM: x86: Remove emulation_result enums, EMULATE_{DONE,FAIL,USER_EXIT}
  KVM: VMX: Handle single-step #DB for EMULTYPE_SKIP on EPT misconfig
  KVM: x86: Add comments to document various emulation types

 arch/x86/include/asm/kvm_host.h |  40 +++++++--
 arch/x86/kvm/mmu.c              |  16 +---
 arch/x86/kvm/svm.c              |  62 ++++++--------
 arch/x86/kvm/vmx/vmx.c          | 147 +++++++++++++-------------------
 arch/x86/kvm/x86.c              | 133 ++++++++++++++++-------------
 arch/x86/kvm/x86.h              |   2 +-
 6 files changed, 195 insertions(+), 205 deletions(-)

Comments

Paolo Bonzini Sept. 17, 2019, 3:14 p.m. UTC | #1
On 27/08/19 23:40, Sean Christopherson wrote:
> Rework the emulator and its users to handle failure scenarios entirely
> within the emulator.
> 
> {x86,kvm}_emulate_instruction() currently returns a tri-state value to
> indicate success/continue, userspace exit needed, and failure.  The
> intent of returning EMULATE_FAIL is to let the caller handle failure in
> a manner that is appropriate for the current context.  In practice,
> the emulator has ended up with a mixture of failure handling, i.e.
> whether or not the emulator takes action on failure is dependent on the
> specific flavor of emulation.
> 
> The mixed handling has proven to be rather fragile, e.g. many flows
> incorrectly assume their specific flavor of emulation cannot fail or
> that the emulator sets state to report the failure back to userspace.
> 
> Move everything inside the emulator, piece by piece, so that the
> emulation routines can return '0' for exit to userspace and '1' for
> resume the guest, just like every other VM-Exit handler.
> 
> Patch 13/14 is a tangentially related bug fix that conflicts heavily with
> this series, so I tacked it on here.
> 
> Patch 14/14 documents the emulation types.  I added it as a separate
> patch at the very end so that the comments could reference the final
> state of the code base, e.g. incorporate the rule change for using
> EMULTYPE_SKIP that is introduced in patch 13/14.
> 
> v1:
>   - https://patchwork.kernel.org/cover/11110331/
> 
> v2:
>   - Collect reviews. [Vitaly and Liran]
>   - Squash VMware emultype changes into a single patch. [Liran]
>   - Add comments in VMX/SVM for VMware #GP handling. [Vitaly]
>   - Tack on the EPT misconfig bug fix.
>   - Add a patch to comment/document the emultypes. [Liran]
> 
> Sean Christopherson (14):
>   KVM: x86: Relocate MMIO exit stats counting
>   KVM: x86: Clean up handle_emulation_failure()
>   KVM: x86: Refactor kvm_vcpu_do_singlestep() to remove out param
>   KVM: x86: Don't attempt VMWare emulation on #GP with non-zero error
>     code
>   KVM: x86: Move #GP injection for VMware into x86_emulate_instruction()
>   KVM: x86: Add explicit flag for forced emulation on #UD
>   KVM: x86: Move #UD injection for failed emulation into emulation code
>   KVM: x86: Exit to userspace on emulation skip failure
>   KVM: x86: Handle emulation failure directly in kvm_task_switch()
>   KVM: x86: Move triple fault request into RM int injection
>   KVM: VMX: Remove EMULATE_FAIL handling in handle_invalid_guest_state()
>   KVM: x86: Remove emulation_result enums, EMULATE_{DONE,FAIL,USER_EXIT}
>   KVM: VMX: Handle single-step #DB for EMULTYPE_SKIP on EPT misconfig
>   KVM: x86: Add comments to document various emulation types
> 
>  arch/x86/include/asm/kvm_host.h |  40 +++++++--
>  arch/x86/kvm/mmu.c              |  16 +---
>  arch/x86/kvm/svm.c              |  62 ++++++--------
>  arch/x86/kvm/vmx/vmx.c          | 147 +++++++++++++-------------------
>  arch/x86/kvm/x86.c              | 133 ++++++++++++++++-------------
>  arch/x86/kvm/x86.h              |   2 +-
>  6 files changed, 195 insertions(+), 205 deletions(-)
> 

Queued, thanks (a couple conflicts had to be sorted out, but nothing
requiring a respin).

Paolo
Alexander Graf Oct. 25, 2019, 11 a.m. UTC | #2
On 17.09.19 17:14, Paolo Bonzini wrote:
> On 27/08/19 23:40, Sean Christopherson wrote:
>> Rework the emulator and its users to handle failure scenarios entirely
>> within the emulator.
>>
>> {x86,kvm}_emulate_instruction() currently returns a tri-state value to
>> indicate success/continue, userspace exit needed, and failure.  The
>> intent of returning EMULATE_FAIL is to let the caller handle failure in
>> a manner that is appropriate for the current context.  In practice,
>> the emulator has ended up with a mixture of failure handling, i.e.
>> whether or not the emulator takes action on failure is dependent on the
>> specific flavor of emulation.
>>
>> The mixed handling has proven to be rather fragile, e.g. many flows
>> incorrectly assume their specific flavor of emulation cannot fail or
>> that the emulator sets state to report the failure back to userspace.
>>
>> Move everything inside the emulator, piece by piece, so that the
>> emulation routines can return '0' for exit to userspace and '1' for
>> resume the guest, just like every other VM-Exit handler.
>>
>> Patch 13/14 is a tangentially related bug fix that conflicts heavily with
>> this series, so I tacked it on here.
>>
>> Patch 14/14 documents the emulation types.  I added it as a separate
>> patch at the very end so that the comments could reference the final
>> state of the code base, e.g. incorporate the rule change for using
>> EMULTYPE_SKIP that is introduced in patch 13/14.
>>
>> v1:
>>    - https://patchwork.kernel.org/cover/11110331/
>>
>> v2:
>>    - Collect reviews. [Vitaly and Liran]
>>    - Squash VMware emultype changes into a single patch. [Liran]
>>    - Add comments in VMX/SVM for VMware #GP handling. [Vitaly]
>>    - Tack on the EPT misconfig bug fix.
>>    - Add a patch to comment/document the emultypes. [Liran]
>>
>> Sean Christopherson (14):
>>    KVM: x86: Relocate MMIO exit stats counting
>>    KVM: x86: Clean up handle_emulation_failure()
>>    KVM: x86: Refactor kvm_vcpu_do_singlestep() to remove out param
>>    KVM: x86: Don't attempt VMWare emulation on #GP with non-zero error
>>      code
>>    KVM: x86: Move #GP injection for VMware into x86_emulate_instruction()
>>    KVM: x86: Add explicit flag for forced emulation on #UD
>>    KVM: x86: Move #UD injection for failed emulation into emulation code
>>    KVM: x86: Exit to userspace on emulation skip failure
>>    KVM: x86: Handle emulation failure directly in kvm_task_switch()
>>    KVM: x86: Move triple fault request into RM int injection
>>    KVM: VMX: Remove EMULATE_FAIL handling in handle_invalid_guest_state()
>>    KVM: x86: Remove emulation_result enums, EMULATE_{DONE,FAIL,USER_EXIT}
>>    KVM: VMX: Handle single-step #DB for EMULTYPE_SKIP on EPT misconfig
>>    KVM: x86: Add comments to document various emulation types
>>
>>   arch/x86/include/asm/kvm_host.h |  40 +++++++--
>>   arch/x86/kvm/mmu.c              |  16 +---
>>   arch/x86/kvm/svm.c              |  62 ++++++--------
>>   arch/x86/kvm/vmx/vmx.c          | 147 +++++++++++++-------------------
>>   arch/x86/kvm/x86.c              | 133 ++++++++++++++++-------------
>>   arch/x86/kvm/x86.h              |   2 +-
>>   6 files changed, 195 insertions(+), 205 deletions(-)
>>
> 
> Queued, thanks (a couple conflicts had to be sorted out, but nothing
> requiring a respin).

Ugh, I just stumbled over this commit. Is this really the right 
direction to move towards?

I appreciate the move to reduce the emulator logic from the many-fold 
enum into a simple binary "worked" or "needs a user space exit". But are 
"0" and "1" really the right names for that? I find the readability of 
the current intercept handlers bad enough, trickling that into even more 
code sounds like a situation that will decrease readability even more.

Why can't we just use names throughout? Something like

enum kvm_return {
     KVM_RET_USER_EXIT = 0,
     KVM_RET_GUEST = 1,
};

and then consistently use them as return values? That way anyone who has 
not worked on kvm before can still make sense of the code.


Alex



Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
Sean Christopherson Nov. 6, 2019, 12:58 a.m. UTC | #3
On Fri, Oct 25, 2019 at 01:00:03PM +0200, Alexander Graf wrote:
> On 17.09.19 17:14, Paolo Bonzini wrote:
> >On 27/08/19 23:40, Sean Christopherson wrote:
> >>Rework the emulator and its users to handle failure scenarios entirely
> >>within the emulator.
> >>
> >>{x86,kvm}_emulate_instruction() currently returns a tri-state value to
> >>indicate success/continue, userspace exit needed, and failure.  The
> >>intent of returning EMULATE_FAIL is to let the caller handle failure in
> >>a manner that is appropriate for the current context.  In practice,
> >>the emulator has ended up with a mixture of failure handling, i.e.
> >>whether or not the emulator takes action on failure is dependent on the
> >>specific flavor of emulation.
> >>
> >>The mixed handling has proven to be rather fragile, e.g. many flows
> >>incorrectly assume their specific flavor of emulation cannot fail or
> >>that the emulator sets state to report the failure back to userspace.
> >>
> >>Move everything inside the emulator, piece by piece, so that the
> >>emulation routines can return '0' for exit to userspace and '1' for
> >>resume the guest, just like every other VM-Exit handler.
> >>
> >>Patch 13/14 is a tangentially related bug fix that conflicts heavily with
> >>this series, so I tacked it on here.
> >>
> >>Patch 14/14 documents the emulation types.  I added it as a separate
> >>patch at the very end so that the comments could reference the final
> >>state of the code base, e.g. incorporate the rule change for using
> >>EMULTYPE_SKIP that is introduced in patch 13/14.
> >>
> >>v1:
> >>   - https://patchwork.kernel.org/cover/11110331/
> >>
> >>v2:
> >>   - Collect reviews. [Vitaly and Liran]
> >>   - Squash VMware emultype changes into a single patch. [Liran]
> >>   - Add comments in VMX/SVM for VMware #GP handling. [Vitaly]
> >>   - Tack on the EPT misconfig bug fix.
> >>   - Add a patch to comment/document the emultypes. [Liran]
> >>
> >>Sean Christopherson (14):
> >>   KVM: x86: Relocate MMIO exit stats counting
> >>   KVM: x86: Clean up handle_emulation_failure()
> >>   KVM: x86: Refactor kvm_vcpu_do_singlestep() to remove out param
> >>   KVM: x86: Don't attempt VMWare emulation on #GP with non-zero error
> >>     code
> >>   KVM: x86: Move #GP injection for VMware into x86_emulate_instruction()
> >>   KVM: x86: Add explicit flag for forced emulation on #UD
> >>   KVM: x86: Move #UD injection for failed emulation into emulation code
> >>   KVM: x86: Exit to userspace on emulation skip failure
> >>   KVM: x86: Handle emulation failure directly in kvm_task_switch()
> >>   KVM: x86: Move triple fault request into RM int injection
> >>   KVM: VMX: Remove EMULATE_FAIL handling in handle_invalid_guest_state()
> >>   KVM: x86: Remove emulation_result enums, EMULATE_{DONE,FAIL,USER_EXIT}
> >>   KVM: VMX: Handle single-step #DB for EMULTYPE_SKIP on EPT misconfig
> >>   KVM: x86: Add comments to document various emulation types
> >>
> >>  arch/x86/include/asm/kvm_host.h |  40 +++++++--
> >>  arch/x86/kvm/mmu.c              |  16 +---
> >>  arch/x86/kvm/svm.c              |  62 ++++++--------
> >>  arch/x86/kvm/vmx/vmx.c          | 147 +++++++++++++-------------------
> >>  arch/x86/kvm/x86.c              | 133 ++++++++++++++++-------------
> >>  arch/x86/kvm/x86.h              |   2 +-
> >>  6 files changed, 195 insertions(+), 205 deletions(-)
> >>
> >
> >Queued, thanks (a couple conflicts had to be sorted out, but nothing
> >requiring a respin).
> 
> Ugh, I just stumbled over this commit. Is this really the right direction to
> move towards?

As you basically surmised below, removing the enum was just a side effect
of cleaning up the emulation error handling, it wasn't really a goal in
and of itself.

> I appreciate the move to reduce the emulator logic from the many-fold enum
> into a simple binary "worked" or "needs a user space exit". But are "0" and
> "1" really the right names for that? I find the readability of the current
> intercept handlers bad enough, trickling that into even more code sounds
> like a situation that will decrease readability even more.
> 
> Why can't we just use names throughout? Something like
> 
> enum kvm_return {
>     KVM_RET_USER_EXIT = 0,
>     KVM_RET_GUEST = 1,
> };
> 
> and then consistently use them as return values? That way anyone who has not
> worked on kvm before can still make sense of the code.

Hmm, I think it'd make more sense to use #define instead of enum to
hopefully make it clear that they aren't the *only* values that can be
returned.  That'd also prevent anyone from changing the return types from
'int' to 'enum kvm_return', which IMO would hurt readability overall.

And maybe KVM_EXIT_TO_USERSPACE and KVM_RETURN_TO_GUEST?
Paolo Bonzini Nov. 6, 2019, 12:17 p.m. UTC | #4
On 06/11/19 01:58, Sean Christopherson wrote:
>> enum kvm_return {
>>     KVM_RET_USER_EXIT = 0,
>>     KVM_RET_GUEST = 1,
>> };
>>
>> and then consistently use them as return values? That way anyone who has not
>> worked on kvm before can still make sense of the code.
> Hmm, I think it'd make more sense to use #define instead of enum to
> hopefully make it clear that they aren't the *only* values that can be
> returned.  That'd also prevent anyone from changing the return types from
> 'int' to 'enum kvm_return', which IMO would hurt readability overall.
> 
> And maybe KVM_EXIT_TO_USERSPACE and KVM_RETURN_TO_GUEST?

That would be quite some work.  Right now there is some consistency
between all of:

- x86_emulate_instruction and its callers

- vcpu->arch.complete_userspace_io

- vcpu_enter_guest/vcpu_block

- kvm_x86_ops->handle_exit

so it would be very easy to end up with a half-int-half-enum state that
is more confusing than before...

I'm more worried about cases where we have functions returning either 0
or -errno, but 0 lets you enter the guest.  I'm not sure if the only one
is kvm_mmu_reload or there are others.

Paolo
Sean Christopherson Nov. 6, 2019, 10:27 p.m. UTC | #5
On Wed, Nov 06, 2019 at 01:17:40PM +0100, Paolo Bonzini wrote:
> On 06/11/19 01:58, Sean Christopherson wrote:
> >> enum kvm_return {
> >>     KVM_RET_USER_EXIT = 0,
> >>     KVM_RET_GUEST = 1,
> >> };
> >>
> >> and then consistently use them as return values? That way anyone who has not
> >> worked on kvm before can still make sense of the code.
> > Hmm, I think it'd make more sense to use #define instead of enum to
> > hopefully make it clear that they aren't the *only* values that can be
> > returned.  That'd also prevent anyone from changing the return types from
> > 'int' to 'enum kvm_return', which IMO would hurt readability overall.
> > 
> > And maybe KVM_EXIT_TO_USERSPACE and KVM_RETURN_TO_GUEST?
> 
> That would be quite some work.  Right now there is some consistency
> between all of:
> 
> - x86_emulate_instruction and its callers
> 
> - vcpu->arch.complete_userspace_io
> 
> - vcpu_enter_guest/vcpu_block
> 
> - kvm_x86_ops->handle_exit
> 
> so it would be very easy to end up with a half-int-half-enum state that
> is more confusing than before...
 
Ya, my thought was to update the obvious cases, essentially the ones you
listed above, to use the define.  So almost intentionally end up in a
half-n-half state, at least in the short term, the thought being that the
extra annotation would do more harm than good.  But there's really no way
to determine whether or not it would actually be a net positive without
writing the code...

> I'm more worried about cases where we have functions returning either 0
> or -errno, but 0 lets you enter the guest.  I'm not sure if the only one
> is kvm_mmu_reload or there are others.

There are lots of those, e.g. basically all of the helpers for nested
consistency checks.