diff mbox series

[6.1.y] KVM: x86: fire timer when it is migrated and expired, and in oneshot mode

Message ID 20240820053229.2858-1-david.hunter.linux@gmail.com (mailing list archive)
State New, archived
Headers show
Series [6.1.y] KVM: x86: fire timer when it is migrated and expired, and in oneshot mode | expand

Commit Message

David Hunter Aug. 20, 2024, 5:32 a.m. UTC
From: Li RongQing <lirongqing@baidu.com>

[ Upstream Commit 8e6ed96cdd5001c55fccc80a17f651741c1ca7d2]

when the vCPU was migrated, if its timer is expired, KVM _should_ fire
the timer ASAP, zeroing the deadline here will cause the timer to
immediately fire on the destination

Cc: Sean Christopherson <seanjc@google.com>
Cc: Peter Shier <pshier@google.com>
Cc: Jim Mattson <jmattson@google.com>
Cc: Wanpeng Li <wanpengli@tencent.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Li RongQing <lirongqing@baidu.com>
Link: https://lore.kernel.org/r/20230106040625.8404-1-lirongqing@baidu.com
Signed-off-by: Sean Christopherson <seanjc@google.com>

(cherry picked from commit 8e6ed96cdd5001c55fccc80a17f651741c1ca7d2)
The code was able to compile without errors or warnings. 
Signed-off-by: David Hunter <david.hunter.linux@gmail.com>
---
 arch/x86/kvm/lapic.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

Comments

Li RongQing Aug. 20, 2024, 6:18 a.m. UTC | #1
> 
> From: Li RongQing <lirongqing@baidu.com>
> 
> [ Upstream Commit 8e6ed96cdd5001c55fccc80a17f651741c1ca7d2]
> 
> when the vCPU was migrated, if its timer is expired, KVM _should_ fire the
> timer ASAP, zeroing the deadline here will cause the timer to immediately fire
> on the destination
> 

This patch increased the reproduce ratio of lapic timer interrupt losing, which has been fixed by the following patch;
so I think patch should not merge it into 6.1


commit 9cfec6d097c607e36199cf0cfbb8cf5acbd8e9b2
Author: Haitao Shan <hshan@google.com>
Date:   Tue Sep 12 16:55:45 2023 -0700

    KVM: x86: Fix lapic timer interrupt lost after loading a snapshot.

    When running android emulator (which is based on QEMU 2.12) on
    certain Intel hosts with kernel version 6.3-rc1 or above, guest
    will freeze after loading a snapshot. This is almost 100%
    reproducible. By default, the android emulator will use snapshot
    to speed up the next launching of the same android guest. So
    this breaks the android emulator badly.

    I tested QEMU 8.0.4 from Debian 12 with an Ubuntu 22.04 guest by
    running command "loadvm" after "savevm". The same issue is
    observed. At the same time, none of our AMD platforms is impacted.
    More experiments show that loading the KVM module with
    "enable_apicv=false" can workaround it.

    The issue started to show up after commit 8e6ed96cdd50 ("KVM: x86:
    fire timer when it is migrated and expired, and in oneshot mode").
    However, as is pointed out by Sean Christopherson, it is introduced
    by commit 967235d32032 ("KVM: vmx: clear pending interrupts on
    KVM_SET_LAPIC"). commit 8e6ed96cdd50 ("KVM: x86: fire timer when
    it is migrated and expired, and in oneshot mode") just makes it
    easier to hit the issue.

    Having both commits, the oneshot lapic timer gets fired immediately
    inside the KVM_SET_LAPIC call when loading the snapshot. On Intel
    platforms with APIC virtualization and posted interrupt processing,
    this eventually leads to setting the corresponding PIR bit. However,
    the whole PIR bits get cleared later in the same KVM_SET_LAPIC call
    by apicv_post_state_restore. This leads to timer interrupt lost.

    The fix is to move vmx_apicv_post_state_restore to the beginning of
    the KVM_SET_LAPIC call and rename to vmx_apicv_pre_state_restore.
    What vmx_apicv_post_state_restore does is actually clearing any
    former apicv state and this behavior is more suitable to carry out
    in the beginning.

    Fixes: 967235d32032 ("KVM: vmx: clear pending interrupts on KVM_SET_LAPIC")
    Cc: stable@vger.kernel.org
    Suggested-by: Sean Christopherson <seanjc@google.com>
    Signed-off-by: Haitao Shan <hshan@google.com>
    Link: https://lore.kernel.org/r/20230913000215.478387-1-hshan@google.com
    Signed-off-by: Sean Christopherson <seanjc@google.com>



> Cc: Sean Christopherson <seanjc@google.com>
> Cc: Peter Shier <pshier@google.com>
> Cc: Jim Mattson <jmattson@google.com>
> Cc: Wanpeng Li <wanpengli@tencent.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Li RongQing <lirongqing@baidu.com>
> Link:
> https://lore.kernel.org/r/20230106040625.8404-1-lirongqing@baidu.com
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> 
> (cherry picked from commit 8e6ed96cdd5001c55fccc80a17f651741c1ca7d2)
> The code was able to compile without errors or warnings.
> Signed-off-by: David Hunter <david.hunter.linux@gmail.com>
> ---
>  arch/x86/kvm/lapic.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index
> c90fef0258c5..3cd590ace95a 100644
> --- a/arch/x86/kvm/lapic.c
> +++ b/arch/x86/kvm/lapic.c
> @@ -1843,8 +1843,12 @@ static bool set_target_expiration(struct kvm_lapic
> *apic, u32 count_reg)
>  		if (unlikely(count_reg != APIC_TMICT)) {
>  			deadline = tmict_to_ns(apic,
>  				     kvm_lapic_get_reg(apic, count_reg));
> -			if (unlikely(deadline <= 0))
> -				deadline = apic->lapic_timer.period;
> +			if (unlikely(deadline <= 0)) {
> +				if (apic_lvtt_period(apic))
> +					deadline = apic->lapic_timer.period;
> +				else
> +					deadline = 0;
> +			}
>  			else if (unlikely(deadline > apic->lapic_timer.period)) {
>  				pr_info_ratelimited(
>  				    "kvm: vcpu %i: requested lapic timer restore with "
> --
> 2.43.0
Sean Christopherson Aug. 20, 2024, 2:07 p.m. UTC | #2
On Tue, Aug 20, 2024, Li,Rongqing wrote:
> > 
> > From: Li RongQing <lirongqing@baidu.com>
> > 
> > [ Upstream Commit 8e6ed96cdd5001c55fccc80a17f651741c1ca7d2]
> > 
> > when the vCPU was migrated, if its timer is expired, KVM _should_ fire the
> > timer ASAP, zeroing the deadline here will cause the timer to immediately fire
> > on the destination
> > 
> 
> This patch increased the reproduce ratio of lapic timer interrupt losing,

Yep, this caused a painful amount of fallout in our environment.

> which has been fixed by the following patch; so I think patch should not
> merge it into 6.1

David, can you prep a small series with both this patch and the fix below?

Thanks!

> commit 9cfec6d097c607e36199cf0cfbb8cf5acbd8e9b2
> Author: Haitao Shan <hshan@google.com>
> Date:   Tue Sep 12 16:55:45 2023 -0700
diff mbox series

Patch

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index c90fef0258c5..3cd590ace95a 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -1843,8 +1843,12 @@  static bool set_target_expiration(struct kvm_lapic *apic, u32 count_reg)
 		if (unlikely(count_reg != APIC_TMICT)) {
 			deadline = tmict_to_ns(apic,
 				     kvm_lapic_get_reg(apic, count_reg));
-			if (unlikely(deadline <= 0))
-				deadline = apic->lapic_timer.period;
+			if (unlikely(deadline <= 0)) {
+				if (apic_lvtt_period(apic))
+					deadline = apic->lapic_timer.period;
+				else
+					deadline = 0;
+			}
 			else if (unlikely(deadline > apic->lapic_timer.period)) {
 				pr_info_ratelimited(
 				    "kvm: vcpu %i: requested lapic timer restore with "