diff mbox series

[1/4] KVM: x86: Handle TIF_NEED_FPU_LOAD in kvm_{load,put}_guest_fpu()

Message ID 20200117062628.6233-2-sean.j.christopherson@intel.com (mailing list archive)
State New, archived
Headers show
Series KVM: x86: TIF_NEED_FPU_LOAD bug fixes | expand

Commit Message

Sean Christopherson Jan. 17, 2020, 6:26 a.m. UTC 
Handle TIF_NEED_FPU_LOAD similar to how fpu__copy() handles the flag
when duplicating FPU state to a new task struct.  TIF_NEED_FPU_LOAD can
be set any time control is transferred out of KVM, be it voluntarily,
e.g. if I/O is triggered during a KVM call to get_user_pages, or
involuntarily, e.g. if softirq runs after an IRQ occurs.  Therefore,
KVM must account for TIF_NEED_FPU_LOAD whenever it is (potentially)
accessing CPU FPU state.

Fixes: 5f409e20b7945 ("x86/fpu: Defer FPU state load until return to userspace")
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
---
 arch/x86/kvm/x86.c | 27 ++++++++++++++++++++++++---
 1 file changed, 24 insertions(+), 3 deletions(-)

Comments

Dave Hansen Jan. 17, 2020, 6:31 p.m. UTC | #1 
On 1/16/20 10:26 PM, Sean Christopherson wrote:
> Handle TIF_NEED_FPU_LOAD similar to how fpu__copy() handles the flag
> when duplicating FPU state to a new task struct.  TIF_NEED_FPU_LOAD can
> be set any time control is transferred out of KVM, be it voluntarily,
> e.g. if I/O is triggered during a KVM call to get_user_pages, or
> involuntarily, e.g. if softirq runs after an IRQ occurs.  Therefore,
> KVM must account for TIF_NEED_FPU_LOAD whenever it is (potentially)
> accessing CPU FPU state.
> 
> Fixes: 5f409e20b7945 ("x86/fpu: Defer FPU state load until return to userspace")
> Cc: stable@vger.kernel.org
> Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
> ---
>  arch/x86/kvm/x86.c | 27 ++++++++++++++++++++++++---
>  1 file changed, 24 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index cf917139de6b..0c7211491f98 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -8476,8 +8476,20 @@ static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu)
>  {
>  	fpregs_lock();
>  
> -	copy_fpregs_to_fpstate(vcpu->arch.user_fpu);
> -	/* PKRU is separately restored in kvm_x86_ops->run.  */
> +	/*
> +	 * If userspace's FPU state is not resident in the CPU registers, just
> +	 * memcpy() from current, else save CPU state directly to user_fpu.
> +	 */
> +	if (test_thread_flag(TIF_NEED_FPU_LOAD))
> +		memcpy(&vcpu->arch.user_fpu->state, &current->thread.fpu.state,
> +		       fpu_kernel_xstate_size);
> +	else
> +		copy_fpregs_to_fpstate(vcpu->arch.user_fpu);
> +
> +	/*
> +	 * Load guest's FPU state to the CPU registers.  PKRU is separately
> +	 * loaded in kvm_x86_ops->run.
> +	 */
>  	__copy_kernel_to_fpregs(&vcpu->arch.guest_fpu->state,
>  				~XFEATURE_MASK_PKRU);

Nit: it took me a minute to realize that there is both:

	vcpu->arch.user_fpu
and
	vcpu->arch.guest_fpu

It might help readability to have local variables for those, or at least
a comment to help differentiate the two.


> @@ -8492,7 +8504,16 @@ static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu)
>  {
>  	fpregs_lock();
>  
> -	copy_fpregs_to_fpstate(vcpu->arch.guest_fpu);
> +	/*
> +	 * If guest's FPU state is not resident in the CPU registers, just
> +	 * memcpy() from current, else save CPU state directly to guest_fpu.
> +	 */
> +	if (test_thread_flag(TIF_NEED_FPU_LOAD))
> +		memcpy(&vcpu->arch.guest_fpu->state, &current->thread.fpu.state,
> +		       fpu_kernel_xstate_size);
> +	else
> +		copy_fpregs_to_fpstate(vcpu->arch.guest_fpu);
> +
>  	copy_kernel_to_fpregs(&vcpu->arch.user_fpu->state);
>  
>  	fpregs_mark_activate();

This also makes me wonder if we want to have copy_fpregs_to_fpstate()
check for TIF_NEED_FPU_LOAD and complain if it's set.
Sean Christopherson Jan. 17, 2020, 6:43 p.m. UTC | #2 
On Fri, Jan 17, 2020 at 10:31:53AM -0800, Dave Hansen wrote:
> On 1/16/20 10:26 PM, Sean Christopherson wrote:
> > Handle TIF_NEED_FPU_LOAD similar to how fpu__copy() handles the flag
> > when duplicating FPU state to a new task struct.  TIF_NEED_FPU_LOAD can
> > be set any time control is transferred out of KVM, be it voluntarily,
> > e.g. if I/O is triggered during a KVM call to get_user_pages, or
> > involuntarily, e.g. if softirq runs after an IRQ occurs.  Therefore,
> > KVM must account for TIF_NEED_FPU_LOAD whenever it is (potentially)
> > accessing CPU FPU state.
> > 
> > Fixes: 5f409e20b7945 ("x86/fpu: Defer FPU state load until return to userspace")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
> > ---
> >  arch/x86/kvm/x86.c | 27 ++++++++++++++++++++++++---
> >  1 file changed, 24 insertions(+), 3 deletions(-)
> > 
> > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> > index cf917139de6b..0c7211491f98 100644
> > --- a/arch/x86/kvm/x86.c
> > +++ b/arch/x86/kvm/x86.c
> > @@ -8476,8 +8476,20 @@ static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu)
> >  {
> >  	fpregs_lock();
> >  
> > -	copy_fpregs_to_fpstate(vcpu->arch.user_fpu);
> > -	/* PKRU is separately restored in kvm_x86_ops->run.  */
> > +	/*
> > +	 * If userspace's FPU state is not resident in the CPU registers, just
> > +	 * memcpy() from current, else save CPU state directly to user_fpu.
> > +	 */
> > +	if (test_thread_flag(TIF_NEED_FPU_LOAD))
> > +		memcpy(&vcpu->arch.user_fpu->state, &current->thread.fpu.state,
> > +		       fpu_kernel_xstate_size);
> > +	else
> > +		copy_fpregs_to_fpstate(vcpu->arch.user_fpu);
> > +
> > +	/*
> > +	 * Load guest's FPU state to the CPU registers.  PKRU is separately
> > +	 * loaded in kvm_x86_ops->run.
> > +	 */
> >  	__copy_kernel_to_fpregs(&vcpu->arch.guest_fpu->state,
> >  				~XFEATURE_MASK_PKRU);
> 
> Nit: it took me a minute to realize that there is both:
> 
> 	vcpu->arch.user_fpu
> and
> 	vcpu->arch.guest_fpu
> 
> It might help readability to have local variables for those, or at least
> a comment to help differentiate the two.

Or even better, add a helper to wrap the logic instead of copy+paste, e.g.:

static void kvm_save_current_fpu(struct fpu *fpu)
{
	if (test_thread_flag(TIF_NEED_FPU_LOAD))
		memcpy(&fpu->state, &current->thread.fpu.state,
		       fpu_kernel_xstate_size);
	else
		copy_fpregs_to_fpstate(fpu);
}

> 
> 
> > @@ -8492,7 +8504,16 @@ static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu)
> >  {
> >  	fpregs_lock();
> >  
> > -	copy_fpregs_to_fpstate(vcpu->arch.guest_fpu);
> > +	/*
> > +	 * If guest's FPU state is not resident in the CPU registers, just
> > +	 * memcpy() from current, else save CPU state directly to guest_fpu.
> > +	 */
> > +	if (test_thread_flag(TIF_NEED_FPU_LOAD))
> > +		memcpy(&vcpu->arch.guest_fpu->state, &current->thread.fpu.state,
> > +		       fpu_kernel_xstate_size);
> > +	else
> > +		copy_fpregs_to_fpstate(vcpu->arch.guest_fpu);
> > +
> >  	copy_kernel_to_fpregs(&vcpu->arch.user_fpu->state);
> >  
> >  	fpregs_mark_activate();
> 
> This also makes me wonder if we want to have copy_fpregs_to_fpstate()
> check for TIF_NEED_FPU_LOAD and complain if it's set.
diff mbox series

Patch

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index cf917139de6b..0c7211491f98 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8476,8 +8476,20 @@  static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu)
 {
 	fpregs_lock();
 
-	copy_fpregs_to_fpstate(vcpu->arch.user_fpu);
-	/* PKRU is separately restored in kvm_x86_ops->run.  */
+	/*
+	 * If userspace's FPU state is not resident in the CPU registers, just
+	 * memcpy() from current, else save CPU state directly to user_fpu.
+	 */
+	if (test_thread_flag(TIF_NEED_FPU_LOAD))
+		memcpy(&vcpu->arch.user_fpu->state, &current->thread.fpu.state,
+		       fpu_kernel_xstate_size);
+	else
+		copy_fpregs_to_fpstate(vcpu->arch.user_fpu);
+
+	/*
+	 * Load guest's FPU state to the CPU registers.  PKRU is separately
+	 * loaded in kvm_x86_ops->run.
+	 */
 	__copy_kernel_to_fpregs(&vcpu->arch.guest_fpu->state,
 				~XFEATURE_MASK_PKRU);
 
@@ -8492,7 +8504,16 @@  static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu)
 {
 	fpregs_lock();
 
-	copy_fpregs_to_fpstate(vcpu->arch.guest_fpu);
+	/*
+	 * If guest's FPU state is not resident in the CPU registers, just
+	 * memcpy() from current, else save CPU state directly to guest_fpu.
+	 */
+	if (test_thread_flag(TIF_NEED_FPU_LOAD))
+		memcpy(&vcpu->arch.guest_fpu->state, &current->thread.fpu.state,
+		       fpu_kernel_xstate_size);
+	else
+		copy_fpregs_to_fpstate(vcpu->arch.guest_fpu);
+
 	copy_kernel_to_fpregs(&vcpu->arch.user_fpu->state);
 
 	fpregs_mark_activate();