| Message ID | 20170321192239.GD25540@potion (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Tue, Mar 21, 2017 at 08:22:39PM +0100, Radim Krčmář wrote: > 2017-03-21 10:29-0700, Nadav Amit: > > > > > On Mar 21, 2017, at 9:58 AM, Radim Krčmář <rkrcmar@redhat.com> wrote: > > > > > In '-smp 2', the writing VCPU always does 10000 wakeups by writing into > > > monitored memory, but the mwaiting VCPU can be also woken up by host > > > interrupts, which might add a few exits depending on timing. > > > > > > I didn't spend much time in making the PASS/FAIL mean much, or ensuring > > > that we only get 10000 wakeups ... it is nothing to be worried about. > > > > > > Hint 240 behaves as nop even on my system, so I still don't find > > > anything insane on that machine (if OS X is exluded) ... And I get the exact same results on the MacBookAir4,2 (which exhibits no freezing or extreme sluggishness when running OS X 10.7 smp with Michael's KVM MWAIT-in-L1 patch)... > > > > From my days in Intel (10 years ago), I can say that MWAIT wakes for many > > microarchitecural events beside interrupts. > > > > Out of curiosity, aren’t you worried that on OS X the wbinvd causes an exit > > after the monitor and before the mwait? > > VM entry clears the monitoring, so it should behave just like an MWAIT > without MONITOR, which is NOP according to the spec. It does so on > modern hardware, but it definitely is a good thing to try ... > (I am worried about disabling MWAIT exits by default and it's a no-go > until we understand why OS X doesn't work.) > > Gabriel, how does testing with this change behave on the old machine? > > Thanks. > > ---8<--- > This should be the same as "wbinvd", because "wbinvd" does nothing > without non-coherent vfio. > Simply replacing "vmcall" with "wbinvd" is an option if the "vmcall" > version works as expected. > --- > diff --git a/x86/mwait.c b/x86/mwait.c > index 20f4dcbff8ae..19f988b94541 100644 > --- a/x86/mwait.c > +++ b/x86/mwait.c > @@ -54,6 +54,7 @@ int main(int argc, char **argv) > > while ((smp ? *page : resumes) < TARGET_RESUMES) { > asm volatile("monitor" :: "a" (page), "c" (0), "d" (0)); > + asm volatile("vmcall" :: "a"(-1)); > asm volatile("mwait" :: "a" (eax), "c" (ecx)); > resumes++; > } Sure thing, here's the MacPro1,1 results: [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 0' timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 0 enabling apic PASS: resumed from mwait 10000 times SUMMARY: 1 tests real 0m1.709s user 0m0.547s sys 0m0.243s [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 1' timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 1 enabling apic PASS: resumed from mwait 10000 times SUMMARY: 1 tests real 0m0.752s user 0m0.545s sys 0m0.218s [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 0' -smp 2 timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 0 -smp 2 enabling apic enabling apic FAIL: resumed from mwait 10004 times SUMMARY: 1 tests, 1 unexpected failures real 0m0.753s user 0m0.554s sys 0m0.227s [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 1' -smp 2 timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 1 -smp 2 enabling apic enabling apic PASS: resumed from mwait 10000 times SUMMARY: 1 tests real 0m0.755s user 0m0.562s sys 0m0.221s For comparison, the resuls including 'vmcall' on the MacBookAir4,2 (interesting, the results for the last test, "-append '240 1 1' -smp 2", are different): [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 0' timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 0 enabling apic PASS: resumed from mwait 10000 times SUMMARY: 1 tests real 0m0.622s user 0m0.501s sys 0m0.130s [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 1' timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 1 enabling apic PASS: resumed from mwait 10000 times SUMMARY: 1 tests real 0m0.624s user 0m0.504s sys 0m0.127s [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 0' -smp 2 timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 0 -smp 2 enabling apic enabling apic FAIL: resumed from mwait 10023 times SUMMARY: 1 tests, 1 unexpected failures real 0m0.623s user 0m0.544s sys 0m0.110s [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 1' -smp 2 timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 1 -smp 2 enabling apic enabling apic FAIL: resumed from mwait 10006 times SUMMARY: 1 tests, 1 unexpected failures real 0m0.618s user 0m0.527s sys 0m0.121s HTH, --Gabriel
> On Mar 21, 2017, at 3:51 PM, Gabriel Somlo <gsomlo@gmail.com> wrote: > > And I get the exact same results on the MacBookAir4,2 (which exhibits > no freezing or extreme sluggishness when running OS X 10.7 smp with > Michael's KVM MWAIT-in-L1 patch)... Sorry for my confusion. I didn’t read the entire thread and thought that the problem is spurious wake-ups. Since that is not the case, I would just suggest two things that you can freely ignore: 1. According to the SDM, when an interrupt is delivered, the interrupt is only delivered on the following instruction, so you may consider skipping the MWAIT first. 2. Perhaps the CPU changes for some reason GUEST_ACTIVITY_STATE (which is not according to the SDM). That is it. No more BS from me. Nadav
On Tue, Mar 21, 2017 at 05:02:25PM -0700, Nadav Amit wrote: > > > On Mar 21, 2017, at 3:51 PM, Gabriel Somlo <gsomlo@gmail.com> wrote: > > > > And I get the exact same results on the MacBookAir4,2 (which exhibits > > no freezing or extreme sluggishness when running OS X 10.7 smp with > > Michael's KVM MWAIT-in-L1 patch)... > > Sorry for my confusion. I didn’t read the entire thread and thought that > the problem is spurious wake-ups. > > Since that is not the case, I would just suggest two things that you can > freely ignore: > > 1. According to the SDM, when an interrupt is delivered, the interrupt > is only delivered on the following instruction, so you may consider > skipping the MWAIT first. > > 2. Perhaps the CPU changes for some reason GUEST_ACTIVITY_STATE (which > is not according to the SDM). > > That is it. No more BS from me. > > Nadav Intersting. I found this errata: A REP STOS/MOVS to a MONITOR/MWAIT Address Range May Prevent Triggering of the Monitoring Hardware Could the macbook CPU be affected?
On Wed, Mar 22, 2017 at 03:35:18PM +0200, Michael S. Tsirkin wrote: > On Tue, Mar 21, 2017 at 05:02:25PM -0700, Nadav Amit wrote: > > > > > On Mar 21, 2017, at 3:51 PM, Gabriel Somlo <gsomlo@gmail.com> wrote: > > > > > > And I get the exact same results on the MacBookAir4,2 (which exhibits > > > no freezing or extreme sluggishness when running OS X 10.7 smp with > > > Michael's KVM MWAIT-in-L1 patch)... > > > > Sorry for my confusion. I didn’t read the entire thread and thought that > > the problem is spurious wake-ups. > > > > Since that is not the case, I would just suggest two things that you can > > freely ignore: > > > > 1. According to the SDM, when an interrupt is delivered, the interrupt > > is only delivered on the following instruction, so you may consider > > skipping the MWAIT first. > > > > 2. Perhaps the CPU changes for some reason GUEST_ACTIVITY_STATE (which > > is not according to the SDM). > > > > That is it. No more BS from me. > > > > Nadav > > Intersting. I found this errata: > A REP STOS/MOVS to a MONITOR/MWAIT Address Range May Prevent Triggering of > the Monitoring Hardware Any way to tell if they mean that for L0, or L>=1, or all of them? > Could the macbook CPU be affected? I ran a grep on the log file I collected when disassembling AppleIntelCPUPowerManagement.kext (where the MWAIT-based idle thread lives) a few days ago, and didn't find any "rep stos" or "rep movs" instances.
On Wed, Mar 22, 2017 at 10:10:05AM -0400, Gabriel L. Somlo wrote: > On Wed, Mar 22, 2017 at 03:35:18PM +0200, Michael S. Tsirkin wrote: > > On Tue, Mar 21, 2017 at 05:02:25PM -0700, Nadav Amit wrote: > > > > > > > On Mar 21, 2017, at 3:51 PM, Gabriel Somlo <gsomlo@gmail.com> wrote: > > > > > > > > And I get the exact same results on the MacBookAir4,2 (which exhibits > > > > no freezing or extreme sluggishness when running OS X 10.7 smp with > > > > Michael's KVM MWAIT-in-L1 patch)... > > > > > > Sorry for my confusion. I didn’t read the entire thread and thought that > > > the problem is spurious wake-ups. > > > > > > Since that is not the case, I would just suggest two things that you can > > > freely ignore: > > > > > > 1. According to the SDM, when an interrupt is delivered, the interrupt > > > is only delivered on the following instruction, so you may consider > > > skipping the MWAIT first. > > > > > > 2. Perhaps the CPU changes for some reason GUEST_ACTIVITY_STATE (which > > > is not according to the SDM). > > > > > > That is it. No more BS from me. > > > > > > Nadav > > > > Intersting. I found this errata: > > A REP STOS/MOVS to a MONITOR/MWAIT Address Range May Prevent Triggering of > > the Monitoring Hardware > > Any way to tell if they mean that for L0, or L>=1, or all of them? > > > Could the macbook CPU be affected? > > I ran a grep on the log file I collected when disassembling > AppleIntelCPUPowerManagement.kext (where the MWAIT-based idle > thread lives) a few days ago, and didn't find any "rep stos" or > "rep movs" instances. > Right but that would be on the waking side, not the one that does mwait.
diff --git a/x86/mwait.c b/x86/mwait.c index 20f4dcbff8ae..19f988b94541 100644 --- a/x86/mwait.c +++ b/x86/mwait.c @@ -54,6 +54,7 @@ int main(int argc, char **argv) while ((smp ? *page : resumes) < TARGET_RESUMES) { asm volatile("monitor" :: "a" (page), "c" (0), "d" (0)); + asm volatile("vmcall" :: "a"(-1)); asm volatile("mwait" :: "a" (eax), "c" (ecx)); resumes++; }