| Message ID | 1498219190-6457-1-git-send-email-wanpeng.li@hotmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 23/06/2017 13:59, Wanpeng Li wrote: > From: Wanpeng Li <wanpeng.li@hotmail.com> > > Linux as a guest will prevent to program the next event to the clock event > device when the event is in the past. However, it is not guaranteed by all > the guests, the guest like kvm-unit-tests/apic.flat will write current tsc > value visible in guest to MSR_IA32_TSCDEADLINE, lapic timer which is emulated > by vmx preemption timer will program the absolute target tsc value to vmcs > preemption timer field w/ delta == 0, then plays a vmentry and an upcoming > vmx preemption timer fire vmexit dance, the lapic timer injection is delayed > for this duration. Actually the lapic timer which is emulated by hrtimer can > handle this correctly. > > This patch fixes it by firing the lapic timer and injecting a timer interrupt > immediately during the next vmentry if guest programs an expired timer to > the emulated timer device. This saves ~300 cycles on the tsc_deadline_timer > test of apic.flat. If it doesn't matter for actual guests, I'd rather not complicate the logic. Paolo
2017-06-28 0:10 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>: > > > On 23/06/2017 13:59, Wanpeng Li wrote: >> From: Wanpeng Li <wanpeng.li@hotmail.com> >> >> Linux as a guest will prevent to program the next event to the clock event >> device when the event is in the past. However, it is not guaranteed by all >> the guests, the guest like kvm-unit-tests/apic.flat will write current tsc >> value visible in guest to MSR_IA32_TSCDEADLINE, lapic timer which is emulated >> by vmx preemption timer will program the absolute target tsc value to vmcs >> preemption timer field w/ delta == 0, then plays a vmentry and an upcoming >> vmx preemption timer fire vmexit dance, the lapic timer injection is delayed >> for this duration. Actually the lapic timer which is emulated by hrtimer can >> handle this correctly. >> >> This patch fixes it by firing the lapic timer and injecting a timer interrupt >> immediately during the next vmentry if guest programs an expired timer to >> the emulated timer device. This saves ~300 cycles on the tsc_deadline_timer >> test of apic.flat. > > If it doesn't matter for actual guests, I'd rather not complicate the logic. Sorry, maybe my description is not that accurate. It can also benefit if the tsc deadline timer is programmed really close to the deadline. That's why we introduce commit 9175d2e97b ("KVM: vmx: fix underflow in TSC deadline calculation"). In addition, I capture there is frequently programmed close to the deadline in vmx_set_hv_timer(delta_tsc == 0): qemu-system-x86-2352 [003] .... 457.709435: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [004] .... 457.941090: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [005] .... 458.381340: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [005] .... 458.381565: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [005] .... 458.382079: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [005] .... 458.382277: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [005] .... 458.382576: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [005] .... 458.383065: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [005] .... 458.565506: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [005] .... 458.565718: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [005] .... 458.566199: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [005] .... 458.566431: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [005] .... 458.566680: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [005] .... 458.567154: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [006] .... 458.989358: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [006] .... 458.989554: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [006] .... 458.990075: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [003] .... 459.397456: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [003] .... 459.397618: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [003] .... 459.397877: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [003] .... 459.398091: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [003] .... 459.398307: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [003] .... 459.398672: vmx_set_hv_timer: expired timer qemu-system-x86-2352 [003] .... 459.502826: vmx_set_hv_timer: expired timer Regards, Wanpeng Li
2017-06-28 6:09 GMT+08:00 Wanpeng Li <kernellwp@gmail.com>: > 2017-06-28 0:10 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>: >> >> >> On 23/06/2017 13:59, Wanpeng Li wrote: >>> From: Wanpeng Li <wanpeng.li@hotmail.com> >>> >>> Linux as a guest will prevent to program the next event to the clock event >>> device when the event is in the past. However, it is not guaranteed by all >>> the guests, the guest like kvm-unit-tests/apic.flat will write current tsc >>> value visible in guest to MSR_IA32_TSCDEADLINE, lapic timer which is emulated >>> by vmx preemption timer will program the absolute target tsc value to vmcs >>> preemption timer field w/ delta == 0, then plays a vmentry and an upcoming >>> vmx preemption timer fire vmexit dance, the lapic timer injection is delayed >>> for this duration. Actually the lapic timer which is emulated by hrtimer can >>> handle this correctly. >>> >>> This patch fixes it by firing the lapic timer and injecting a timer interrupt >>> immediately during the next vmentry if guest programs an expired timer to >>> the emulated timer device. This saves ~300 cycles on the tsc_deadline_timer >>> test of apic.flat. >> >> If it doesn't matter for actual guests, I'd rather not complicate the logic. > > Sorry, maybe my description is not that accurate. It can also benefit > if the tsc deadline timer is programmed really close to the deadline. > That's why we introduce commit 9175d2e97b ("KVM: vmx: fix underflow in > TSC deadline calculation"). In addition, I capture there is frequently > programmed close to the deadline in vmx_set_hv_timer(delta_tsc == 0): I test a linux guest. > > qemu-system-x86-2352 [003] .... 457.709435: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [004] .... 457.941090: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [005] .... 458.381340: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [005] .... 458.381565: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [005] .... 458.382079: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [005] .... 458.382277: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [005] .... 458.382576: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [005] .... 458.383065: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [005] .... 458.565506: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [005] .... 458.565718: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [005] .... 458.566199: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [005] .... 458.566431: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [005] .... 458.566680: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [005] .... 458.567154: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [006] .... 458.989358: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [006] .... 458.989554: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [006] .... 458.990075: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [003] .... 459.397456: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [003] .... 459.397618: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [003] .... 459.397877: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [003] .... 459.398091: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [003] .... 459.398307: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [003] .... 459.398672: vmx_set_hv_timer: expired timer > qemu-system-x86-2352 [003] .... 459.502826: vmx_set_hv_timer: expired timer > > Regards, > Wanpeng Li
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index d24c874..08cf73d 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -1504,12 +1504,17 @@ static void cancel_hv_timer(struct kvm_lapic *apic) static bool start_hv_timer(struct kvm_lapic *apic) { u64 tscdeadline = apic->lapic_timer.tscdeadline; + int ret = 0; if ((atomic_read(&apic->lapic_timer.pending) && !apic_lvtt_period(apic)) || - kvm_x86_ops->set_hv_timer(apic->vcpu, tscdeadline)) { + (ret = kvm_x86_ops->set_hv_timer(apic->vcpu, tscdeadline))) { if (apic->lapic_timer.hv_timer_in_use) cancel_hv_timer(apic); + if (ret == 1) { + apic_timer_expired(apic); + return true; + } } else { apic->lapic_timer.hv_timer_in_use = true; hrtimer_cancel(&apic->lapic_timer.timer); diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 2e906cf..2008e9b 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -11149,6 +11149,9 @@ static int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc) u64 guest_tscl = kvm_read_l1_tsc(vcpu, tscl); u64 delta_tsc = max(guest_deadline_tsc, guest_tscl) - guest_tscl; + if (delta_tsc == 0) + return 1; + /* Convert to host delta tsc if tsc scaling is enabled */ if (vcpu->arch.tsc_scaling_ratio != kvm_default_tsc_scaling_ratio && u64_shl_div_u64(delta_tsc,