| Message ID | 1239616545-25199-6-git-send-email-gleb@redhat.com (mailing list archive) |
|---|---|
| State | Accepted |
| Headers | show |
Gleb Natapov wrote: > Start to use interrupt/exception queues like VMX does. This also fix the > bug that if exit was caused by a guest internal exception access to IDT > the exception was not reinjected. This patch broke KVM for me: after it is applied (to the tip of avi's git tree), linux inside KVM (version 84 from Debian) stops booting, moaning about lost interrupts from ide. The KVM is executed inside qemu-system-x86_64, version 0.10.2.
On Tue, Apr 14, 2009 at 02:14:04PM +0000, Dmitry Eremin-Solenikov wrote: > Gleb Natapov wrote: > > > Start to use interrupt/exception queues like VMX does. This also fix the > > bug that if exit was caused by a guest internal exception access to IDT > > the exception was not reinjected. > > This patch broke KVM for me: after it is applied (to the tip of avi's git tree), > linux inside KVM (version 84 from Debian) stops booting, moaning about lost > interrupts from ide. The KVM is executed inside qemu-system-x86_64, > version 0.10.2. > Please apply next patch in the series too. This one will not work without it. But better yet can you please test entire series. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2009/4/14 Gleb Natapov <gleb@redhat.com>: > On Tue, Apr 14, 2009 at 02:14:04PM +0000, Dmitry Eremin-Solenikov wrote: >> Gleb Natapov wrote: >> >> > Start to use interrupt/exception queues like VMX does. This also fix the >> > bug that if exit was caused by a guest internal exception access to IDT >> > the exception was not reinjected. >> >> This patch broke KVM for me: after it is applied (to the tip of avi's git tree), >> linux inside KVM (version 84 from Debian) stops booting, moaning about lost >> interrupts from ide. The KVM is executed inside qemu-system-x86_64, >> version 0.10.2. >> > Please apply next patch in the series too. This one will not work > without it. But better yet can you please test entire series. After applying the next patch (or the whole serie), I get the following messages during initramfs drivers probe: Clocksource tsc unstable (delta...) no cont in shutdown! floppy0: FDC access conflict! Then kernel boot stalls. I'll try gdbing into kernel but this may require lots of efforts. I don't quite understand how do these two patches influence FDC emulation, but they do. Tell me if you need any additional info.
On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: > 2009/4/14 Gleb Natapov <gleb@redhat.com>: > > On Tue, Apr 14, 2009 at 02:14:04PM +0000, Dmitry Eremin-Solenikov wrote: > >> Gleb Natapov wrote: > >> > >> > Start to use interrupt/exception queues like VMX does. This also fix the > >> > bug that if exit was caused by a guest internal exception access to IDT > >> > the exception was not reinjected. > >> > >> This patch broke KVM for me: after it is applied (to the tip of avi's git tree), > >> linux inside KVM (version 84 from Debian) stops booting, moaning about lost > >> interrupts from ide. The KVM is executed inside qemu-system-x86_64, > >> version 0.10.2. > >> > > Please apply next patch in the series too. This one will not work > > without it. But better yet can you please test entire series. > > After applying the next patch (or the whole serie), I get the following messages > during initramfs drivers probe: > > Clocksource tsc unstable (delta...) > no cont in shutdown! > floppy0: FDC access conflict! > > Then kernel boot stalls. I'll try gdbing into kernel but this may > require lots of efforts. > I don't quite understand how do these two patches influence FDC emulation, but > they do. Tell me if you need any additional info. > What guest is this? What kernel? Does the whole series works? -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Apr 14, 2009 at 05:55:36PM +0300, Gleb Natapov wrote: > > After applying the next patch (or the whole serie), I get the following messages > > during initramfs drivers probe: > > > > Clocksource tsc unstable (delta...) > > no cont in shutdown! > > floppy0: FDC access conflict! > > > > Then kernel boot stalls. I'll try gdbing into kernel but this may > > require lots of efforts. > > I don't quite understand how do these two patches influence FDC emulation, but > > they do. Tell me if you need any additional info. > > > What guest is this? What kernel? Does the whole series works? > And while you are at it can you try to run with -no-kvm-irqchip? -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Gleb Natapov wrote: > On Tue, Apr 14, 2009 at 02:14:04PM +0000, Dmitry Eremin-Solenikov wrote: > >> Gleb Natapov wrote: >> >> >>> Start to use interrupt/exception queues like VMX does. This also fix the >>> bug that if exit was caused by a guest internal exception access to IDT >>> the exception was not reinjected. >>> >> This patch broke KVM for me: after it is applied (to the tip of avi's git tree), >> linux inside KVM (version 84 from Debian) stops booting, moaning about lost >> interrupts from ide. The KVM is executed inside qemu-system-x86_64, >> version 0.10.2. >> >> > Please apply next patch in the series too. This one will not work > without it. But better yet can you please test entire series. > > Er, I'd much rather have a bisectable series.
On Tue, Apr 14, 2009 at 07:10:40PM +0300, Avi Kivity wrote: > Gleb Natapov wrote: >> On Tue, Apr 14, 2009 at 02:14:04PM +0000, Dmitry Eremin-Solenikov wrote: >> >>> Gleb Natapov wrote: >>> >>> >>>> Start to use interrupt/exception queues like VMX does. This also fix the >>>> bug that if exit was caused by a guest internal exception access to IDT >>>> the exception was not reinjected. >>>> >>> This patch broke KVM for me: after it is applied (to the tip of avi's git tree), >>> linux inside KVM (version 84 from Debian) stops booting, moaning about lost >>> interrupts from ide. The KVM is executed inside qemu-system-x86_64, >>> version 0.10.2. >>> >>> >> Please apply next patch in the series too. This one will not work >> without it. But better yet can you please test entire series. >> >> > > Er, I'd much rather have a bisectable series. > The next patch (use eveninj to inject interrupts) is really small. We can coalesce those two. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2009/4/14 Gleb Natapov <gleb@redhat.com>: > On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: >> 2009/4/14 Gleb Natapov <gleb@redhat.com>: >> > On Tue, Apr 14, 2009 at 02:14:04PM +0000, Dmitry Eremin-Solenikov wrote: >> >> Gleb Natapov wrote: >> >> >> >> > Start to use interrupt/exception queues like VMX does. This also fix the >> >> > bug that if exit was caused by a guest internal exception access to IDT >> >> > the exception was not reinjected. >> >> >> >> This patch broke KVM for me: after it is applied (to the tip of avi's git tree), >> >> linux inside KVM (version 84 from Debian) stops booting, moaning about lost >> >> interrupts from ide. The KVM is executed inside qemu-system-x86_64, >> >> version 0.10.2. >> >> >> > Please apply next patch in the series too. This one will not work >> > without it. But better yet can you please test entire series. >> >> After applying the next patch (or the whole serie), I get the following messages >> during initramfs drivers probe: >> >> Clocksource tsc unstable (delta...) >> no cont in shutdown! >> floppy0: FDC access conflict! >> >> Then kernel boot stalls. I'll try gdbing into kernel but this may >> require lots of efforts. >> I don't quite understand how do these two patches influence FDC emulation, but >> they do. Tell me if you need any additional info. >> > What guest is this? What kernel? Does the whole series works? Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide config or bzImage + initrd). The whole serie doesn't work too (that's why I started bisecting). And BTW, I got the same results with -no-kvm-irqchip
On Tue, Apr 14, 2009 at 11:29:49PM +0400, Dmitry Eremin-Solenikov wrote: > 2009/4/14 Gleb Natapov <gleb@redhat.com>: > > On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: > >> 2009/4/14 Gleb Natapov <gleb@redhat.com>: > >> > On Tue, Apr 14, 2009 at 02:14:04PM +0000, Dmitry Eremin-Solenikov wrote: > >> >> Gleb Natapov wrote: > >> >> > >> >> > Start to use interrupt/exception queues like VMX does. This also fix the > >> >> > bug that if exit was caused by a guest internal exception access to IDT > >> >> > the exception was not reinjected. > >> >> > >> >> This patch broke KVM for me: after it is applied (to the tip of avi's git tree), > >> >> linux inside KVM (version 84 from Debian) stops booting, moaning about lost > >> >> interrupts from ide. The KVM is executed inside qemu-system-x86_64, > >> >> version 0.10.2. > >> >> > >> > Please apply next patch in the series too. This one will not work > >> > without it. But better yet can you please test entire series. > >> > >> After applying the next patch (or the whole serie), I get the following messages > >> during initramfs drivers probe: > >> > >> Clocksource tsc unstable (delta...) > >> no cont in shutdown! > >> floppy0: FDC access conflict! > >> > >> Then kernel boot stalls. I'll try gdbing into kernel but this may > >> require lots of efforts. > >> I don't quite understand how do these two patches influence FDC emulation, but > >> they do. Tell me if you need any additional info. > >> > > What guest is this? What kernel? Does the whole series works? > > Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide > config or bzImage + initrd). Yes please provide. Debian lenny (x86_64) is my default guest :) And I just booted it fine on AMD barcelona CPU. What is you host cpu? "cat /proc/cpuinfo" I just noticed that my kernel is different. Will install 2.6.26 and retest, but provide me yours anyway. > The whole serie doesn't work too (that's why I started bisecting). > > And BTW, I got the same results with -no-kvm-irqchip > -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Apr 14, 2009 at 10:41:03PM +0300, Gleb Natapov wrote: > > Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide > > config or bzImage + initrd). > Yes please provide. Debian lenny (x86_64) is my default guest :) And I > just booted it fine on AMD barcelona CPU. What is you host cpu? > "cat /proc/cpuinfo" > > I just noticed that my kernel is different. Will install 2.6.26 and retest, > but provide me yours anyway. 2.6.26-2-amd64 works for me too. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2009/4/14 Gleb Natapov <gleb@redhat.com>: > On Tue, Apr 14, 2009 at 11:29:49PM +0400, Dmitry Eremin-Solenikov wrote: >> 2009/4/14 Gleb Natapov <gleb@redhat.com>: >> > On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: >> >> 2009/4/14 Gleb Natapov <gleb@redhat.com>: >> >> > On Tue, Apr 14, 2009 at 02:14:04PM +0000, Dmitry Eremin-Solenikov wrote: >> >> >> Gleb Natapov wrote: >> >> >> >> >> >> > Start to use interrupt/exception queues like VMX does. This also fix the >> >> >> > bug that if exit was caused by a guest internal exception access to IDT >> >> >> > the exception was not reinjected. >> >> >> >> >> >> This patch broke KVM for me: after it is applied (to the tip of avi's git tree), >> >> >> linux inside KVM (version 84 from Debian) stops booting, moaning about lost >> >> >> interrupts from ide. The KVM is executed inside qemu-system-x86_64, >> >> >> version 0.10.2. >> >> >> >> >> > Please apply next patch in the series too. This one will not work >> >> > without it. But better yet can you please test entire series. >> >> >> >> After applying the next patch (or the whole serie), I get the following messages >> >> during initramfs drivers probe: >> >> >> >> Clocksource tsc unstable (delta...) >> >> no cont in shutdown! >> >> floppy0: FDC access conflict! >> >> >> >> Then kernel boot stalls. I'll try gdbing into kernel but this may >> >> require lots of efforts. >> >> I don't quite understand how do these two patches influence FDC emulation, but >> >> they do. Tell me if you need any additional info. >> >> >> > What guest is this? What kernel? Does the whole series works? >> >> Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide >> config or bzImage + initrd). > Yes please provide. Debian lenny (x86_64) is my default guest :) And I > just booted it fine on AMD barcelona CPU. Â What is you host cpu? > "cat /proc/cpuinfo" qemu-x86_64 version 0.10.2 running on i386 Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. qemu-64:~# cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 6 model : 2 model name : QEMU Virtual CPU version 0.10.2 stepping : 3 cpu MHz : 1828.754 cache size : 512 KB fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm 3dnowext 3dnow up pni svm bogomips : 3700.32 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: > > I just noticed that my kernel is different. Will install 2.6.26 and retest, > but provide me yours anyway. > >> The whole serie doesn't work too (that's why I started bisecting). >> >> And BTW, I got the same results with -no-kvm-irqchip >> > > -- > Â Â Â Â Â Â Â Â Â Â Â Â Gleb. >
On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: > 2009/4/14 Gleb Natapov <gleb@redhat.com>: > > On Tue, Apr 14, 2009 at 11:29:49PM +0400, Dmitry Eremin-Solenikov wrote: > >> 2009/4/14 Gleb Natapov <gleb@redhat.com>: > >> > On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: > >> >> 2009/4/14 Gleb Natapov <gleb@redhat.com>: > >> >> > On Tue, Apr 14, 2009 at 02:14:04PM +0000, Dmitry Eremin-Solenikov wrote: > >> >> >> Gleb Natapov wrote: > >> >> >> > >> >> >> > Start to use interrupt/exception queues like VMX does. This also fix the > >> >> >> > bug that if exit was caused by a guest internal exception access to IDT > >> >> >> > the exception was not reinjected. > >> >> >> > >> >> >> This patch broke KVM for me: after it is applied (to the tip of avi's git tree), > >> >> >> linux inside KVM (version 84 from Debian) stops booting, moaning about lost > >> >> >> interrupts from ide. The KVM is executed inside qemu-system-x86_64, > >> >> >> version 0.10.2. > >> >> >> > >> >> > Please apply next patch in the series too. This one will not work > >> >> > without it. But better yet can you please test entire series. > >> >> > >> >> After applying the next patch (or the whole serie), I get the following messages > >> >> during initramfs drivers probe: > >> >> > >> >> Clocksource tsc unstable (delta...) > >> >> no cont in shutdown! > >> >> floppy0: FDC access conflict! > >> >> > >> >> Then kernel boot stalls. I'll try gdbing into kernel but this may > >> >> require lots of efforts. > >> >> I don't quite understand how do these two patches influence FDC emulation, but > >> >> they do. Tell me if you need any additional info. > >> >> > >> > What guest is this? What kernel? Does the whole series works? > >> > >> Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide > >> config or bzImage + initrd). > > Yes please provide. Debian lenny (x86_64) is my default guest :) And I > > just booted it fine on AMD barcelona CPU. Â What is you host cpu? > > "cat /proc/cpuinfo" > > qemu-x86_64 version 0.10.2 running on i386 > Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. > qemu-64:~# cat /proc/cpuinfo > processor : 0 > vendor_id : AuthenticAMD > cpu family : 6 > model : 2 > model name : QEMU Virtual CPU version 0.10.2 > stepping : 3 > cpu MHz : 1828.754 > cache size : 512 KB > fpu : yes > fpu_exception : yes > cpuid level : 2 > wp : yes > flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca > cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm 3dnowext 3dnow > up pni svm > bogomips : 3700.32 > TLB size : 1024 4K pages > clflush size : 64 > cache_alignment : 64 > address sizes : 40 bits physical, 48 bits virtual > power management: > > I need _host_ cpu info. Do the same on the host please. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: > qemu-x86_64 version 0.10.2 running on i386 > Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. Do you mean boot 'guest' kernel with noapic? The guest is what runs inside qemu. So you are able to boot guest with 'noapic'? What is the command line you are using. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Gleb Natapov wrote: > On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: >> 2009/4/14 Gleb Natapov <gleb@redhat.com>: >>> On Tue, Apr 14, 2009 at 11:29:49PM +0400, Dmitry Eremin-Solenikov wrote: >>>> 2009/4/14 Gleb Natapov <gleb@redhat.com>: >>>>> On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: >>>>>> 2009/4/14 Gleb Natapov <gleb@redhat.com>: >>>>>>> On Tue, Apr 14, 2009 at 02:14:04PM +0000, Dmitry Eremin-Solenikov wrote: >>>>>>>> Gleb Natapov wrote: >>>>>>>> >>>>>>>>> Start to use interrupt/exception queues like VMX does. This also fix the >>>>>>>>> bug that if exit was caused by a guest internal exception access to IDT >>>>>>>>> the exception was not reinjected. >>>>>>>> This patch broke KVM for me: after it is applied (to the tip of avi's git tree), >>>>>>>> linux inside KVM (version 84 from Debian) stops booting, moaning about lost >>>>>>>> interrupts from ide. The KVM is executed inside qemu-system-x86_64, >>>>>>>> version 0.10.2. >>>>>>>> >>>>>>> Please apply next patch in the series too. This one will not work >>>>>>> without it. But better yet can you please test entire series. >>>>>> After applying the next patch (or the whole serie), I get the following messages >>>>>> during initramfs drivers probe: >>>>>> >>>>>> Clocksource tsc unstable (delta...) >>>>>> no cont in shutdown! >>>>>> floppy0: FDC access conflict! >>>>>> >>>>>> Then kernel boot stalls. I'll try gdbing into kernel but this may >>>>>> require lots of efforts. >>>>>> I don't quite understand how do these two patches influence FDC emulation, but >>>>>> they do. Tell me if you need any additional info. >>>>>> >>>>> What guest is this? What kernel? Does the whole series works? >>>> Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide >>>> config or bzImage + initrd). >>> Yes please provide. Debian lenny (x86_64) is my default guest :) And I >>> just booted it fine on AMD barcelona CPU. What is you host cpu? >>> "cat /proc/cpuinfo" >> qemu-x86_64 version 0.10.2 running on i386 >> Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. >> qemu-64:~# cat /proc/cpuinfo >> processor : 0 >> vendor_id : AuthenticAMD >> cpu family : 6 >> model : 2 >> model name : QEMU Virtual CPU version 0.10.2 >> stepping : 3 >> cpu MHz : 1828.754 >> cache size : 512 KB >> fpu : yes >> fpu_exception : yes >> cpuid level : 2 >> wp : yes >> flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca >> cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm 3dnowext 3dnow >> up pni svm >> bogomips : 3700.32 >> TLB size : 1024 4K pages >> clflush size : 64 >> cache_alignment : 64 >> address sizes : 40 bits physical, 48 bits virtual >> power management: >> >> > I need _host_ cpu info. Do the same on the host please. That _is_ his host - qemu in emulation mode (ie. nested virtualization). Maybe there is an issue with qemu's emulation of svm or, rather, with the apic emulation. The fact that he has to boot the first-level guest with noapic is fairly suspicious. Dmitry, what is your first level-guest distro/kernel, also Lenny? And what is the top-level qemu command line? Let's focus on this first, leaving KVM and this patch series aside for a while. Jan
On Wed, Apr 15, 2009 at 12:22:34PM +0200, Jan Kiszka wrote: > Gleb Natapov wrote: > > On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: > >> 2009/4/14 Gleb Natapov <gleb@redhat.com>: > >>> On Tue, Apr 14, 2009 at 11:29:49PM +0400, Dmitry Eremin-Solenikov wrote: > >>>> 2009/4/14 Gleb Natapov <gleb@redhat.com>: > >>>>> On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: > >>>>>> 2009/4/14 Gleb Natapov <gleb@redhat.com>: > >>>>>>> On Tue, Apr 14, 2009 at 02:14:04PM +0000, Dmitry Eremin-Solenikov wrote: > >>>>>>>> Gleb Natapov wrote: > >>>>>>>> > >>>>>>>>> Start to use interrupt/exception queues like VMX does. This also fix the > >>>>>>>>> bug that if exit was caused by a guest internal exception access to IDT > >>>>>>>>> the exception was not reinjected. > >>>>>>>> This patch broke KVM for me: after it is applied (to the tip of avi's git tree), > >>>>>>>> linux inside KVM (version 84 from Debian) stops booting, moaning about lost > >>>>>>>> interrupts from ide. The KVM is executed inside qemu-system-x86_64, > >>>>>>>> version 0.10.2. > >>>>>>>> > >>>>>>> Please apply next patch in the series too. This one will not work > >>>>>>> without it. But better yet can you please test entire series. > >>>>>> After applying the next patch (or the whole serie), I get the following messages > >>>>>> during initramfs drivers probe: > >>>>>> > >>>>>> Clocksource tsc unstable (delta...) > >>>>>> no cont in shutdown! > >>>>>> floppy0: FDC access conflict! > >>>>>> > >>>>>> Then kernel boot stalls. I'll try gdbing into kernel but this may > >>>>>> require lots of efforts. > >>>>>> I don't quite understand how do these two patches influence FDC emulation, but > >>>>>> they do. Tell me if you need any additional info. > >>>>>> > >>>>> What guest is this? What kernel? Does the whole series works? > >>>> Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide > >>>> config or bzImage + initrd). > >>> Yes please provide. Debian lenny (x86_64) is my default guest :) And I > >>> just booted it fine on AMD barcelona CPU. What is you host cpu? > >>> "cat /proc/cpuinfo" > >> qemu-x86_64 version 0.10.2 running on i386 > >> Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. > >> qemu-64:~# cat /proc/cpuinfo > >> processor : 0 > >> vendor_id : AuthenticAMD > >> cpu family : 6 > >> model : 2 > >> model name : QEMU Virtual CPU version 0.10.2 > >> stepping : 3 > >> cpu MHz : 1828.754 > >> cache size : 512 KB > >> fpu : yes > >> fpu_exception : yes > >> cpuid level : 2 > >> wp : yes > >> flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca > >> cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm 3dnowext 3dnow > >> up pni svm > >> bogomips : 3700.32 > >> TLB size : 1024 4K pages > >> clflush size : 64 > >> cache_alignment : 64 > >> address sizes : 40 bits physical, 48 bits virtual > >> power management: > >> > >> > > I need _host_ cpu info. Do the same on the host please. > > That _is_ his host - qemu in emulation mode (ie. nested virtualization). Ah, now I noticed svm in cpu flags. Does qemu support svm in TCG? > Maybe there is an issue with qemu's emulation of svm or, rather, with > the apic emulation. The fact that he has to boot the first-level guest > with noapic is fairly suspicious. > > Dmitry, what is your first level-guest distro/kernel, also Lenny? And > what is the top-level qemu command line? Let's focus on this first, > leaving KVM and this patch series aside for a while. > If KVM runs inside a guest that is definitely a good idea :) -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Gleb Natapov wrote: > On Wed, Apr 15, 2009 at 12:22:34PM +0200, Jan Kiszka wrote: >> Gleb Natapov wrote: >>> On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: >>>> 2009/4/14 Gleb Natapov <gleb@redhat.com>: >>>>> On Tue, Apr 14, 2009 at 11:29:49PM +0400, Dmitry Eremin-Solenikov wrote: >>>>>> 2009/4/14 Gleb Natapov <gleb@redhat.com>: >>>>>>> On Tue, Apr 14, 2009 at 06:32:29PM +0400, Dmitry Eremin-Solenikov wrote: >>>>>>>> 2009/4/14 Gleb Natapov <gleb@redhat.com>: >>>>>>>>> On Tue, Apr 14, 2009 at 02:14:04PM +0000, Dmitry Eremin-Solenikov wrote: >>>>>>>>>> Gleb Natapov wrote: >>>>>>>>>> >>>>>>>>>>> Start to use interrupt/exception queues like VMX does. This also fix the >>>>>>>>>>> bug that if exit was caused by a guest internal exception access to IDT >>>>>>>>>>> the exception was not reinjected. >>>>>>>>>> This patch broke KVM for me: after it is applied (to the tip of avi's git tree), >>>>>>>>>> linux inside KVM (version 84 from Debian) stops booting, moaning about lost >>>>>>>>>> interrupts from ide. The KVM is executed inside qemu-system-x86_64, >>>>>>>>>> version 0.10.2. >>>>>>>>>> >>>>>>>>> Please apply next patch in the series too. This one will not work >>>>>>>>> without it. But better yet can you please test entire series. >>>>>>>> After applying the next patch (or the whole serie), I get the following messages >>>>>>>> during initramfs drivers probe: >>>>>>>> >>>>>>>> Clocksource tsc unstable (delta...) >>>>>>>> no cont in shutdown! >>>>>>>> floppy0: FDC access conflict! >>>>>>>> >>>>>>>> Then kernel boot stalls. I'll try gdbing into kernel but this may >>>>>>>> require lots of efforts. >>>>>>>> I don't quite understand how do these two patches influence FDC emulation, but >>>>>>>> they do. Tell me if you need any additional info. >>>>>>>> >>>>>>> What guest is this? What kernel? Does the whole series works? >>>>>> Guest: Debian lenny. Linux 2.6.26, Debian version (I can provide >>>>>> config or bzImage + initrd). >>>>> Yes please provide. Debian lenny (x86_64) is my default guest :) And I >>>>> just booted it fine on AMD barcelona CPU. What is you host cpu? >>>>> "cat /proc/cpuinfo" >>>> qemu-x86_64 version 0.10.2 running on i386 >>>> Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. >>>> qemu-64:~# cat /proc/cpuinfo >>>> processor : 0 >>>> vendor_id : AuthenticAMD >>>> cpu family : 6 >>>> model : 2 >>>> model name : QEMU Virtual CPU version 0.10.2 >>>> stepping : 3 >>>> cpu MHz : 1828.754 >>>> cache size : 512 KB >>>> fpu : yes >>>> fpu_exception : yes >>>> cpuid level : 2 >>>> wp : yes >>>> flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca >>>> cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm 3dnowext 3dnow >>>> up pni svm >>>> bogomips : 3700.32 >>>> TLB size : 1024 4K pages >>>> clflush size : 64 >>>> cache_alignment : 64 >>>> address sizes : 40 bits physical, 48 bits virtual >>>> power management: >>>> >>>> >>> I need _host_ cpu info. Do the same on the host please. >> That _is_ his host - qemu in emulation mode (ie. nested virtualization). > Ah, now I noticed svm in cpu flags. Does qemu support svm in TCG? Yes, and KVM seems to have been fine without the patch. But that may not exclude remaining bugs in QEMU (as first-level hypervisor here). On the other hand, it wouldn't be the first time QEMU, with its extreme delays, triggers some nasty race in its guest... > >> Maybe there is an issue with qemu's emulation of svm or, rather, with >> the apic emulation. The fact that he has to boot the first-level guest >> with noapic is fairly suspicious. >> >> Dmitry, what is your first level-guest distro/kernel, also Lenny? And >> what is the top-level qemu command line? Let's focus on this first, >> leaving KVM and this patch series aside for a while. >> > If KVM runs inside a guest that is definitely a good idea :) > > -- > Gleb. Jan
On Wed, Apr 15, 2009 at 12:51:00PM +0200, Jan Kiszka wrote: > >>> I need _host_ cpu info. Do the same on the host please. > >> That _is_ his host - qemu in emulation mode (ie. nested virtualization). > > Ah, now I noticed svm in cpu flags. Does qemu support svm in TCG? > > Yes, and KVM seems to have been fine without the patch. But that may not > exclude remaining bugs in QEMU (as first-level hypervisor here). > > On the other hand, it wouldn't be the first time QEMU, with its extreme > delays, triggers some nasty race in its guest... > It doesn't look like race to me. The failure is 100% reproducible. I'll try to reproduce locally and see what is going on. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Gleb Natapov wrote: > On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: >> qemu-x86_64 version 0.10.2 running on i386 >> Due to problems with qemu-x86_64 I have to boot the 'host' kernel with 'noapic'. > Do you mean boot 'guest' kernel with noapic? The guest is what runs > inside qemu. So you are able to boot guest with 'noapic'? > > What is the command line you are using. Well, since this caused lot's of questions, here is my setup: Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), qemu-system-x86_64 version 0.10.2 KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c from Avi's tree (KVM: x86 emulator: fix call near emulation) + these patches. I have to boot the kernels (both this kernel and 2.6.26 from debian) with noapic to w/around APIC problems (I dunno if it's qemu or bochsbios problem). system inside qemu: 64-bit debian lenny KVM userspace: debian 84+dfsg-2 inside kvm I run 32-bit debian lenny with plain debian 2.6.26 kernel.
Dmitry Eremin-Solenikov wrote: > Gleb Natapov wrote: >> On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: >>> qemu-x86_64 version 0.10.2 running on i386 >>> Due to problems with qemu-x86_64 I have to boot the 'host' kernel >>> with 'noapic'. >> Do you mean boot 'guest' kernel with noapic? The guest is what runs >> inside qemu. So you are able to boot guest with 'noapic'? >> >> What is the command line you are using. > > Well, since this caused lot's of questions, here is my setup: > > Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), > qemu-system-x86_64 version 0.10.2 > > KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c > from Avi's tree (KVM: x86 emulator: fix call near emulation) + these > patches. I have to boot the kernels (both this kernel and 2.6.26 from > debian) with noapic to w/around APIC problems (I dunno if it's qemu or > bochsbios problem). And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, specify explicitly with -bios and/or -L)? Then this would be a QEMU upstream bug. Jan
Jan Kiszka пишет: > Dmitry Eremin-Solenikov wrote: >> Gleb Natapov wrote: >>> On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: >>>> qemu-x86_64 version 0.10.2 running on i386 >>>> Due to problems with qemu-x86_64 I have to boot the 'host' kernel >>>> with 'noapic'. >>> Do you mean boot 'guest' kernel with noapic? The guest is what runs >>> inside qemu. So you are able to boot guest with 'noapic'? >>> >>> What is the command line you are using. >> Well, since this caused lot's of questions, here is my setup: >> >> Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), >> qemu-system-x86_64 version 0.10.2 >> >> KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c >> from Avi's tree (KVM: x86 emulator: fix call near emulation) + these >> patches. I have to boot the kernels (both this kernel and 2.6.26 from >> debian) with noapic to w/around APIC problems (I dunno if it's qemu or >> bochsbios problem). > > And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, > specify explicitly with -bios and/or -L)? Then this would be a QEMU > upstream bug. Indeed, there seem to be problems with upstream qemu bios. I was using the image from the debian's bochsbios package. I asked qemu to use the bios from 0.10.2 release and got slightly different messages. Attached the kernel log
Dmitry Eremin-Solenikov пишет: > Jan Kiszka пишет: >> Dmitry Eremin-Solenikov wrote: >>> Gleb Natapov wrote: >>>> On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov >>>> wrote: >>>>> qemu-x86_64 version 0.10.2 running on i386 >>>>> Due to problems with qemu-x86_64 I have to boot the 'host' kernel >>>>> with 'noapic'. >>>> Do you mean boot 'guest' kernel with noapic? The guest is what runs >>>> inside qemu. So you are able to boot guest with 'noapic'? >>>> >>>> What is the command line you are using. >>> Well, since this caused lot's of questions, here is my setup: >>> >>> Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), >>> qemu-system-x86_64 version 0.10.2 >>> >>> KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c >>> from Avi's tree (KVM: x86 emulator: fix call near emulation) + these >>> patches. I have to boot the kernels (both this kernel and 2.6.26 from >>> debian) with noapic to w/around APIC problems (I dunno if it's qemu or >>> bochsbios problem). >> >> And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, >> specify explicitly with -bios and/or -L)? Then this would be a QEMU >> upstream bug. > > Indeed, there seem to be problems with upstream qemu bios. I was using > the image from the debian's bochsbios package. I asked qemu to use the > bios from 0.10.2 release and got slightly different messages. Attached > the kernel log > Moreover, using bios from 0.10.2 I can't boot linux even with noapic: ACPI: PM-Timer IO Port: 0xb008 ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) ACPI: Skipping IOAPIC probe due to 'noapic' option. Using ACPI for processor (LAPIC) configuration information ACPI: HPET id: 0x8086a201 base: 0xfed00000 Intel MultiProcessor Specification v1.4 MPTABLE: OEM ID: QEMUCPU MPTABLE: Product ID: 0.1 MPTABLE: APIC at: 0xFEE00000 I/O APIC #1 Version 17 at 0xFEC00000. Processors: 1 SMP: Allowing 1 CPUs, 0 hotplug CPUs Allocating PCI resources starting at 20000000 (gap: 10000000:effc0000) NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:1 nr_node_ids:1 PERCPU: Embedded 25 pages at ffff880001033000, static data 70880 bytes Built 1 zonelists in Zone order, mobility grouping on. Total pages: 62771 Kernel command line: root=/dev/sda1 ro console=ttyS0 noapic Initializing CPU#0 NR_IRQS:512 PID hash table entries: 1024 (order: 10, 8192 bytes) Fast TSC calibration using PIT Detected 1828.371 MHz processor. Console: colour VGA+ 80x25 console [ttyS0] enabled Dentry cache hash table entries: 32768 (order: 6, 262144 bytes) Inode-cache hash table entries: 16384 (order: 5, 131072 bytes) Checking aperture... No AGP bridge found Memory: 249848k/262080k available (4048k kernel code, 388k absent, 11528k reserved, 1626k data, 436k init) SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 Calibrating delay loop (skipped), value calculated using timer frequency.. 3656.74 BogoMIPS (lpj=7313484) Mount-cache hash table entries: 256 CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) CPU: L2 Cache: 512K (64 bytes/line) SMP alternatives: switching to UP code Freeing SMP alternatives: 29k freed ACPI: Core revision 20081204 ACPI: setting ELCR to 0200 (from 0a00) Setting APIC routing to flat CPU0: AMD QEMU Virtual CPU version 0.10.2 stepping 03 And after that qemu stalls.
On Wed, Apr 15, 2009 at 03:53:40PM +0400, Dmitry Eremin-Solenikov wrote: > Jan Kiszka пишет: >> Dmitry Eremin-Solenikov wrote: >>> Gleb Natapov wrote: >>>> On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: >>>>> qemu-x86_64 version 0.10.2 running on i386 >>>>> Due to problems with qemu-x86_64 I have to boot the 'host' kernel >>>>> with 'noapic'. >>>> Do you mean boot 'guest' kernel with noapic? The guest is what runs >>>> inside qemu. So you are able to boot guest with 'noapic'? >>>> >>>> What is the command line you are using. >>> Well, since this caused lot's of questions, here is my setup: >>> >>> Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), >>> qemu-system-x86_64 version 0.10.2 >>> >>> KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c >>> from Avi's tree (KVM: x86 emulator: fix call near emulation) + these >>> patches. I have to boot the kernels (both this kernel and 2.6.26 from >>> debian) with noapic to w/around APIC problems (I dunno if it's qemu or >>> bochsbios problem). >> >> And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, >> specify explicitly with -bios and/or -L)? Then this would be a QEMU >> upstream bug. > > Indeed, there seem to be problems with upstream qemu bios. I was using > the image from the debian's bochsbios package. I asked qemu to use the > bios from 0.10.2 release and got slightly different messages. Attached > the kernel log > Now it seems to be a problem with KVM bios. KVM will not work with upstream bochs or qemu bios only with its own version. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Gleb Natapov пишет: > On Wed, Apr 15, 2009 at 03:53:40PM +0400, Dmitry Eremin-Solenikov wrote: >> Jan Kiszka пишет: >>> Dmitry Eremin-Solenikov wrote: >>>> Gleb Natapov wrote: >>>>> On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote: >>>>>> qemu-x86_64 version 0.10.2 running on i386 >>>>>> Due to problems with qemu-x86_64 I have to boot the 'host' kernel >>>>>> with 'noapic'. >>>>> Do you mean boot 'guest' kernel with noapic? The guest is what runs >>>>> inside qemu. So you are able to boot guest with 'noapic'? >>>>> >>>>> What is the command line you are using. >>>> Well, since this caused lot's of questions, here is my setup: >>>> >>>> Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), >>>> qemu-system-x86_64 version 0.10.2 >>>> >>>> KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c >>>> from Avi's tree (KVM: x86 emulator: fix call near emulation) + these >>>> patches. I have to boot the kernels (both this kernel and 2.6.26 from >>>> debian) with noapic to w/around APIC problems (I dunno if it's qemu or >>>> bochsbios problem). >>> And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, >>> specify explicitly with -bios and/or -L)? Then this would be a QEMU >>> upstream bug. >> Indeed, there seem to be problems with upstream qemu bios. I was using >> the image from the debian's bochsbios package. I asked qemu to use the >> bios from 0.10.2 release and got slightly different messages. Attached >> the kernel log >> > Now it seems to be a problem with KVM bios. KVM will not work with > upstream bochs or qemu bios only with its own version. I was talking about qemu-system_x86-64, not about KVM.
Dmitry Eremin-Solenikov wrote: > Jan Kiszka пишет: >> Dmitry Eremin-Solenikov wrote: >>> Gleb Natapov wrote: >>>> On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov >>>> wrote: >>>>> qemu-x86_64 version 0.10.2 running on i386 >>>>> Due to problems with qemu-x86_64 I have to boot the 'host' kernel >>>>> with 'noapic'. >>>> Do you mean boot 'guest' kernel with noapic? The guest is what runs >>>> inside qemu. So you are able to boot guest with 'noapic'? >>>> >>>> What is the command line you are using. >>> Well, since this caused lot's of questions, here is my setup: >>> >>> Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), >>> qemu-system-x86_64 version 0.10.2 >>> >>> KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c >>> from Avi's tree (KVM: x86 emulator: fix call near emulation) + these >>> patches. I have to boot the kernels (both this kernel and 2.6.26 from >>> debian) with noapic to w/around APIC problems (I dunno if it's qemu or >>> bochsbios problem). >> >> And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, >> specify explicitly with -bios and/or -L)? Then this would be a QEMU >> upstream bug. > > Indeed, there seem to be problems with upstream qemu bios. I was using > the image from the debian's bochsbios package. Bochsbios is typically lacking some patches qemu needs, therefore that bios patch queue in qemu. > I asked qemu to use the > bios from 0.10.2 release and got slightly different messages. Attached > the kernel log > ... > init IO_APIC IRQs > 1-0 (apicid-pin) not connected > IOAPIC[0]: Set routing entry (1-1 -> 0x31 -> IRQ 1 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-2 -> 0x30 -> IRQ 0 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-3 -> 0x33 -> IRQ 3 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-4 -> 0x34 -> IRQ 4 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-5 -> 0x35 -> IRQ 5 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-6 -> 0x36 -> IRQ 6 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-7 -> 0x37 -> IRQ 7 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-8 -> 0x38 -> IRQ 8 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-9 -> 0x39 -> IRQ 9 Mode:1 Active:1) > IOAPIC[0]: Set routing entry (1-10 -> 0x3a -> IRQ 10 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-11 -> 0x3b -> IRQ 11 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-12 -> 0x3c -> IRQ 12 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-13 -> 0x3d -> IRQ 13 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-14 -> 0x3e -> IRQ 14 Mode:0 Active:0) > IOAPIC[0]: Set routing entry (1-15 -> 0x3f -> IRQ 15 Mode:0 Active:0) > 1-16 1-17 1-18 1-19 1-20 1-21 1-22 1-23 (apicid-pin) not connected > ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 > ..MP-BIOS bug: 8254 timer not connected to IO-APIC > ...trying to set up timer (IRQ0) through the 8259A ... > ..... (found apic 0 pin 2) ... > ....... failed. > ...trying to set up timer as Virtual Wire IRQ... > ..... failed. > ...trying to set up timer as ExtINT IRQ... > ..... failed :( . > Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option. This looks a bit like [1, 2] on first glance... Jan [1] http://permalink.gmane.org/gmane.comp.emulators.qemu/41300 [2] http://permalink.gmane.org/gmane.comp.emulators.qemu/41433
Jan Kiszka пишет: > Dmitry Eremin-Solenikov wrote: >> Jan Kiszka пишет: >>> Dmitry Eremin-Solenikov wrote: >>>> Gleb Natapov wrote: >>>>> On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov >>>>> wrote: >>>>>> qemu-x86_64 version 0.10.2 running on i386 >>>>>> Due to problems with qemu-x86_64 I have to boot the 'host' kernel >>>>>> with 'noapic'. >>>>> Do you mean boot 'guest' kernel with noapic? The guest is what runs >>>>> inside qemu. So you are able to boot guest with 'noapic'? >>>>> >>>>> What is the command line you are using. >>>> Well, since this caused lot's of questions, here is my setup: >>>> >>>> Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), >>>> qemu-system-x86_64 version 0.10.2 >>>> >>>> KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c >>>> from Avi's tree (KVM: x86 emulator: fix call near emulation) + these >>>> patches. I have to boot the kernels (both this kernel and 2.6.26 from >>>> debian) with noapic to w/around APIC problems (I dunno if it's qemu or >>>> bochsbios problem). >>> And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, >>> specify explicitly with -bios and/or -L)? Then this would be a QEMU >>> upstream bug. >> Indeed, there seem to be problems with upstream qemu bios. I was using >> the image from the debian's bochsbios package. > > Bochsbios is typically lacking some patches qemu needs, therefore that > bios patch queue in qemu. Debian's bochsbios provides two bios versions: one for bochs and one patched with qemu (maybe not the latest patches though) >> I asked qemu to use the >> bios from 0.10.2 release and got slightly different messages. Attached >> the kernel log >> > > ... > >> init IO_APIC IRQs >> 1-0 (apicid-pin) not connected >> IOAPIC[0]: Set routing entry (1-1 -> 0x31 -> IRQ 1 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-2 -> 0x30 -> IRQ 0 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-3 -> 0x33 -> IRQ 3 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-4 -> 0x34 -> IRQ 4 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-5 -> 0x35 -> IRQ 5 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-6 -> 0x36 -> IRQ 6 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-7 -> 0x37 -> IRQ 7 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-8 -> 0x38 -> IRQ 8 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-9 -> 0x39 -> IRQ 9 Mode:1 Active:1) >> IOAPIC[0]: Set routing entry (1-10 -> 0x3a -> IRQ 10 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-11 -> 0x3b -> IRQ 11 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-12 -> 0x3c -> IRQ 12 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-13 -> 0x3d -> IRQ 13 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-14 -> 0x3e -> IRQ 14 Mode:0 Active:0) >> IOAPIC[0]: Set routing entry (1-15 -> 0x3f -> IRQ 15 Mode:0 Active:0) >> 1-16 1-17 1-18 1-19 1-20 1-21 1-22 1-23 (apicid-pin) not connected >> ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 >> ..MP-BIOS bug: 8254 timer not connected to IO-APIC >> ...trying to set up timer (IRQ0) through the 8259A ... >> ..... (found apic 0 pin 2) ... >> ....... failed. >> ...trying to set up timer as Virtual Wire IRQ... >> ..... failed. >> ...trying to set up timer as ExtINT IRQ... >> ..... failed :( . >> Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option. > > This looks a bit like [1, 2] on first glance... > > Jan > > [1] http://permalink.gmane.org/gmane.comp.emulators.qemu/41300 > [2] http://permalink.gmane.org/gmane.comp.emulators.qemu/41433 Looks like a part of this changes. However I don't quite understand: these patches should address non-ACPI OS, but linux is surely and ACPI os!
Dmitry Eremin-Solenikov wrote: > Jan Kiszka пишет: >> Dmitry Eremin-Solenikov wrote: >>> Jan Kiszka пишет: >>>> Dmitry Eremin-Solenikov wrote: >>>>> Gleb Natapov wrote: >>>>>> On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov >>>>>> wrote: >>>>>>> qemu-x86_64 version 0.10.2 running on i386 >>>>>>> Due to problems with qemu-x86_64 I have to boot the 'host' kernel >>>>>>> with 'noapic'. >>>>>> Do you mean boot 'guest' kernel with noapic? The guest is what runs >>>>>> inside qemu. So you are able to boot guest with 'noapic'? >>>>>> >>>>>> What is the command line you are using. >>>>> Well, since this caused lot's of questions, here is my setup: >>>>> >>>>> Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter), >>>>> qemu-system-x86_64 version 0.10.2 >>>>> >>>>> KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c >>>>> from Avi's tree (KVM: x86 emulator: fix call near emulation) + these >>>>> patches. I have to boot the kernels (both this kernel and 2.6.26 from >>>>> debian) with noapic to w/around APIC problems (I dunno if it's qemu or >>>>> bochsbios problem). >>>> And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt, >>>> specify explicitly with -bios and/or -L)? Then this would be a QEMU >>>> upstream bug. >>> Indeed, there seem to be problems with upstream qemu bios. I was using >>> the image from the debian's bochsbios package. >> >> Bochsbios is typically lacking some patches qemu needs, therefore that >> bios patch queue in qemu. > > Debian's bochsbios provides two bios versions: one for bochs and one > patched with qemu (maybe not the latest patches though) > >>> I asked qemu to use the >>> bios from 0.10.2 release and got slightly different messages. Attached >>> the kernel log >>> >> >> ... >> >>> init IO_APIC IRQs >>> 1-0 (apicid-pin) not connected >>> IOAPIC[0]: Set routing entry (1-1 -> 0x31 -> IRQ 1 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-2 -> 0x30 -> IRQ 0 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-3 -> 0x33 -> IRQ 3 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-4 -> 0x34 -> IRQ 4 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-5 -> 0x35 -> IRQ 5 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-6 -> 0x36 -> IRQ 6 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-7 -> 0x37 -> IRQ 7 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-8 -> 0x38 -> IRQ 8 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-9 -> 0x39 -> IRQ 9 Mode:1 Active:1) >>> IOAPIC[0]: Set routing entry (1-10 -> 0x3a -> IRQ 10 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-11 -> 0x3b -> IRQ 11 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-12 -> 0x3c -> IRQ 12 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-13 -> 0x3d -> IRQ 13 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-14 -> 0x3e -> IRQ 14 Mode:0 Active:0) >>> IOAPIC[0]: Set routing entry (1-15 -> 0x3f -> IRQ 15 Mode:0 Active:0) >>> 1-16 1-17 1-18 1-19 1-20 1-21 1-22 1-23 (apicid-pin) not connected >>> ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 >>> ..MP-BIOS bug: 8254 timer not connected to IO-APIC >>> ...trying to set up timer (IRQ0) through the 8259A ... >>> ..... (found apic 0 pin 2) ... >>> ....... failed. >>> ...trying to set up timer as Virtual Wire IRQ... >>> ..... failed. >>> ...trying to set up timer as ExtINT IRQ... >>> ..... failed :( . >>> Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with >>> apic=debug and send a report. Then try booting with the 'noapic' >>> option. >> >> This looks a bit like [1, 2] on first glance... >> >> Jan >> >> [1] http://permalink.gmane.org/gmane.comp.emulators.qemu/41300 >> [2] http://permalink.gmane.org/gmane.comp.emulators.qemu/41433 > > Looks like a part of this changes. You mean the kernel boots for you now? > However I don't quite understand: > these patches should address non-ACPI OS, but linux is surely and ACPI os! That's what I also do not understand ATM. I've once seen the above error as well, but with a !CONFIG_ACPI kernel. However, I'd suggest to move the issue (if it still exists) to qemu-devel until we reach KVM specifics again. Jan
Gleb Natapov wrote: > Start to use interrupt/exception queues like VMX does. > This also fix the bug that if exit was caused by a guest > internal exception access to IDT the exception was not > reinjected. > > Signed-off-by: Gleb Natapov <gleb@redhat.com> > --- > arch/x86/kvm/svm.c | 176 ++++++++++++++++++++++------------------------------ > 1 files changed, 75 insertions(+), 101 deletions(-) > > diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c > index 52c41aa..053370d 100644 > --- a/arch/x86/kvm/svm.c > +++ b/arch/x86/kvm/svm.c > @@ -70,7 +70,6 @@ module_param(npt, int, S_IRUGO); > static int nested = 0; > module_param(nested, int, S_IRUGO); > > -static void kvm_reput_irq(struct vcpu_svm *svm); > static void svm_flush_tlb(struct kvm_vcpu *vcpu); > > static int nested_svm_exit_handled(struct vcpu_svm *svm, bool kvm_override); > @@ -199,9 +198,7 @@ static void svm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr, > > static bool svm_exception_injected(struct kvm_vcpu *vcpu) > { > - struct vcpu_svm *svm = to_svm(vcpu); > - > - return !(svm->vmcb->control.exit_int_info & SVM_EXITINTINFO_VALID); > + return false; > } > > static int is_external_interrupt(u32 info) > @@ -976,12 +973,9 @@ static int svm_guest_debug(struct kvm_vcpu *vcpu, struct kvm_guest_debug *dbg) > > static int svm_get_irq(struct kvm_vcpu *vcpu) > { > - struct vcpu_svm *svm = to_svm(vcpu); > - u32 exit_int_info = svm->vmcb->control.exit_int_info; > - > - if (is_external_interrupt(exit_int_info)) > - return exit_int_info & SVM_EVTINJ_VEC_MASK; > - return -1; > + if (!vcpu->arch.interrupt.pending) > + return -1; > + return vcpu->arch.interrupt.nr; > } > > static void load_host_msrs(struct kvm_vcpu *vcpu) > @@ -1088,17 +1082,8 @@ static void svm_set_dr(struct kvm_vcpu *vcpu, int dr, unsigned long value, > > static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) > { > - u32 exit_int_info = svm->vmcb->control.exit_int_info; > - struct kvm *kvm = svm->vcpu.kvm; > u64 fault_address; > u32 error_code; > - bool event_injection = false; > - > - if (!irqchip_in_kernel(kvm) && > - is_external_interrupt(exit_int_info)) { > - event_injection = true; > - kvm_push_irq(&svm->vcpu, exit_int_info & SVM_EVTINJ_VEC_MASK); > - } > > fault_address = svm->vmcb->control.exit_info_2; > error_code = svm->vmcb->control.exit_info_1; > @@ -1118,9 +1103,11 @@ static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) > */ > if (npt_enabled) > svm_flush_tlb(&svm->vcpu); > - > - if (!npt_enabled && event_injection) > - kvm_mmu_unprotect_page_virt(&svm->vcpu, fault_address); > + else { > + if (svm->vcpu.arch.interrupt.pending || > + svm->vcpu.arch.exception.pending) > + kvm_mmu_unprotect_page_virt(&svm->vcpu, fault_address); > + } > return kvm_mmu_page_fault(&svm->vcpu, fault_address, error_code); > } > > @@ -2187,7 +2174,6 @@ static int handle_exit(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu) > } > } > > - kvm_reput_irq(svm); > > if (svm->vmcb->control.exit_code == SVM_EXIT_ERR) { > kvm_run->exit_reason = KVM_EXIT_FAIL_ENTRY; > @@ -2289,98 +2275,47 @@ static int svm_interrupt_allowed(struct kvm_vcpu *vcpu) > (svm->vcpu.arch.hflags & HF_GIF_MASK); > } > > -static void svm_intr_assist(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) > +static void enable_irq_window(struct kvm_vcpu *vcpu) > { > - struct vcpu_svm *svm = to_svm(vcpu); > - struct vmcb *vmcb = svm->vmcb; > - int intr_vector = -1; > - > - if ((vmcb->control.exit_int_info & SVM_EVTINJ_VALID) && > - ((vmcb->control.exit_int_info & SVM_EVTINJ_TYPE_MASK) == 0)) { > - intr_vector = vmcb->control.exit_int_info & > - SVM_EVTINJ_VEC_MASK; > - vmcb->control.exit_int_info = 0; > - svm_inject_irq(svm, intr_vector); > - goto out; > - } > - > - if (vmcb->control.int_ctl & V_IRQ_MASK) > - goto out; > - > - if (!kvm_cpu_has_interrupt(vcpu)) > - goto out; > - > - if (nested_svm_intr(svm)) > - goto out; > - > - if (!(svm->vcpu.arch.hflags & HF_GIF_MASK)) > - goto out; > - > - if (!(vmcb->save.rflags & X86_EFLAGS_IF) || > - (vmcb->control.int_state & SVM_INTERRUPT_SHADOW_MASK) || > - (vmcb->control.event_inj & SVM_EVTINJ_VALID)) { > - /* unable to deliver irq, set pending irq */ > - svm_set_vintr(svm); > - svm_inject_irq(svm, 0x0); > - goto out; > - } > - /* Okay, we can deliver the interrupt: grab it and update PIC state. */ > - intr_vector = kvm_cpu_get_interrupt(vcpu); > - svm_inject_irq(svm, intr_vector); > -out: > - update_cr8_intercept(vcpu); > + svm_set_vintr(to_svm(vcpu)); > + svm_inject_irq(to_svm(vcpu), 0x0); > } > > -static void kvm_reput_irq(struct vcpu_svm *svm) > +static void svm_intr_inject(struct kvm_vcpu *vcpu) > { > - struct vmcb_control_area *control = &svm->vmcb->control; > - > - if ((control->int_ctl & V_IRQ_MASK) > - && !irqchip_in_kernel(svm->vcpu.kvm)) { > - control->int_ctl &= ~V_IRQ_MASK; > - kvm_push_irq(&svm->vcpu, control->int_vector); > + /* try to reinject previous events if any */ > + if (vcpu->arch.interrupt.pending) { > + svm_inject_irq(to_svm(vcpu), vcpu->arch.interrupt.nr); > + return; > } > > - svm->vcpu.arch.interrupt_window_open = > - !(control->int_state & SVM_INTERRUPT_SHADOW_MASK) && > - (svm->vcpu.arch.hflags & HF_GIF_MASK); > -} > - > -static void svm_do_inject_vector(struct vcpu_svm *svm) > -{ > - svm_inject_irq(svm, kvm_pop_irq(&svm->vcpu)); > + /* try to inject new event if pending */ > + if (kvm_cpu_has_interrupt(vcpu)) { > + if (vcpu->arch.interrupt_window_open) { > + kvm_queue_interrupt(vcpu, kvm_cpu_get_interrupt(vcpu)); > + svm_inject_irq(to_svm(vcpu), vcpu->arch.interrupt.nr); > + } > + } > } > > -static void do_interrupt_requests(struct kvm_vcpu *vcpu, > - struct kvm_run *kvm_run) > +static void svm_intr_assist(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) > { > struct vcpu_svm *svm = to_svm(vcpu); > - struct vmcb_control_area *control = &svm->vmcb->control; > + bool req_int_win = !irqchip_in_kernel(vcpu->kvm) && > + kvm_run->request_interrupt_window; > > if (nested_svm_intr(svm)) > - return; > + goto out; > > - svm->vcpu.arch.interrupt_window_open = > - (!(control->int_state & SVM_INTERRUPT_SHADOW_MASK) && > - (svm->vmcb->save.rflags & X86_EFLAGS_IF) && > - (svm->vcpu.arch.hflags & HF_GIF_MASK)); > + svm->vcpu.arch.interrupt_window_open = svm_interrupt_allowed(vcpu); > > - if (svm->vcpu.arch.interrupt_window_open && > - kvm_cpu_has_interrupt(&svm->vcpu)) > - /* > - * If interrupts enabled, and not blocked by sti or mov ss. Good. > - */ > - svm_do_inject_vector(svm); > + svm_intr_inject(vcpu); > > - /* > - * Interrupts blocked. Wait for unblock. > - */ > - if (!svm->vcpu.arch.interrupt_window_open && > - (kvm_cpu_has_interrupt(&svm->vcpu) || > - kvm_run->request_interrupt_window)) > - svm_set_vintr(svm); > - else > - svm_clear_vintr(svm); > + if (kvm_cpu_has_interrupt(vcpu) || req_int_win) > + enable_irq_window(vcpu); > + > +out: > + update_cr8_intercept(vcpu); > } > > static int svm_set_tss_addr(struct kvm *kvm, unsigned int addr) > @@ -2420,6 +2355,43 @@ static inline void sync_lapic_to_cr8(struct kvm_vcpu *vcpu) > svm->vmcb->control.int_ctl |= cr8 & V_TPR_MASK; > } > > +static void svm_complete_interrupts(struct vcpu_svm *svm) > +{ > + u8 vector; > + int type; > + u32 exitintinfo = svm->vmcb->control.exit_int_info; > + > + svm->vcpu.arch.nmi_injected = false; > + kvm_clear_exception_queue(&svm->vcpu); > + kvm_clear_interrupt_queue(&svm->vcpu); > + > + if (!(exitintinfo & SVM_EXITINTINFO_VALID)) > + return; > + > + vector = exitintinfo & SVM_EXITINTINFO_VEC_MASK; > + type = exitintinfo & SVM_EXITINTINFO_TYPE_MASK; > + > + switch (type) { > + case SVM_EXITINTINFO_TYPE_NMI: > + svm->vcpu.arch.nmi_injected = true; > + break; > + case SVM_EXITINTINFO_TYPE_EXEPT: > + if (exitintinfo & SVM_EXITINTINFO_VALID_ERR) { > + u32 err = svm->vmcb->control.exit_int_info_err; > + kvm_queue_exception_e(&svm->vcpu, vector, err); > + stgit remarked "trailing whitespace". checkpatch.pl would probably do so, too. :) And there is another one in patch 14. > + } else > + kvm_queue_exception(&svm->vcpu, vector); > + break; > + case SVM_EXITINTINFO_TYPE_SOFT: > + case SVM_EXITINTINFO_TYPE_INTR: > + kvm_queue_interrupt(&svm->vcpu, vector); > + break; > + default: > + break; > + } > +} > + > #ifdef CONFIG_X86_64 > #define R "r" > #else > @@ -2548,6 +2520,8 @@ static void svm_vcpu_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) > sync_cr8_to_lapic(vcpu); > > svm->next_rip = 0; > + > + svm_complete_interrupts(svm); > } > > #undef R > @@ -2669,7 +2643,7 @@ static struct kvm_x86_ops svm_x86_ops = { > .queue_exception = svm_queue_exception, > .exception_injected = svm_exception_injected, > .inject_pending_irq = svm_intr_assist, > - .inject_pending_vectors = do_interrupt_requests, > + .inject_pending_vectors = svm_intr_assist, > .interrupt_allowed = svm_interrupt_allowed, > > .set_tss_addr = svm_set_tss_addr, Jan
Jan Kiszka wrote: > Gleb Natapov wrote: >> + switch (type) { >> + case SVM_EXITINTINFO_TYPE_NMI: >> + svm->vcpu.arch.nmi_injected = true; >> + break; >> + case SVM_EXITINTINFO_TYPE_EXEPT: >> + if (exitintinfo & SVM_EXITINTINFO_VALID_ERR) { >> + u32 err = svm->vmcb->control.exit_int_info_err; >> + kvm_queue_exception_e(&svm->vcpu, vector, err); >> + > > stgit remarked "trailing whitespace". checkpatch.pl would probably do > so, too. :) > > And there is another one in patch 14. Sorry, the latter one was due to /me using the old version. Jan
Gleb Natapov wrote: > Start to use interrupt/exception queues like VMX does. This also fix the > bug that if exit was caused by a guest internal exception access to IDT > the exception was not reinjected. > OK. On real SVM HW this seems to work. However now i'm stumbled upon another problem wrt. NMI. See another mail.
Gleb Natapov wrote: > Start to use interrupt/exception queues like VMX does. > This also fix the bug that if exit was caused by a guest > internal exception access to IDT the exception was not > reinjected. > > Signed-off-by: Gleb Natapov <gleb@redhat.com> > --- > arch/x86/kvm/svm.c | 176 ++++++++++++++++++++++------------------------------ > 1 files changed, 75 insertions(+), 101 deletions(-) > > diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c > index 52c41aa..053370d 100644 > --- a/arch/x86/kvm/svm.c > +++ b/arch/x86/kvm/svm.c > @@ -70,7 +70,6 @@ module_param(npt, int, S_IRUGO); > static int nested = 0; > module_param(nested, int, S_IRUGO); > > -static void kvm_reput_irq(struct vcpu_svm *svm); > static void svm_flush_tlb(struct kvm_vcpu *vcpu); > > static int nested_svm_exit_handled(struct vcpu_svm *svm, bool kvm_override); > @@ -199,9 +198,7 @@ static void svm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr, > > static bool svm_exception_injected(struct kvm_vcpu *vcpu) > { > - struct vcpu_svm *svm = to_svm(vcpu); > - > - return !(svm->vmcb->control.exit_int_info & SVM_EXITINTINFO_VALID); > + return false; > } > > static int is_external_interrupt(u32 info) > @@ -976,12 +973,9 @@ static int svm_guest_debug(struct kvm_vcpu *vcpu, struct kvm_guest_debug *dbg) > > static int svm_get_irq(struct kvm_vcpu *vcpu) > { > - struct vcpu_svm *svm = to_svm(vcpu); > - u32 exit_int_info = svm->vmcb->control.exit_int_info; > - > - if (is_external_interrupt(exit_int_info)) > - return exit_int_info & SVM_EVTINJ_VEC_MASK; > - return -1; > + if (!vcpu->arch.interrupt.pending) > + return -1; > + return vcpu->arch.interrupt.nr; > } > > static void load_host_msrs(struct kvm_vcpu *vcpu) > @@ -1088,17 +1082,8 @@ static void svm_set_dr(struct kvm_vcpu *vcpu, int dr, unsigned long value, > > static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) > { > - u32 exit_int_info = svm->vmcb->control.exit_int_info; > - struct kvm *kvm = svm->vcpu.kvm; > u64 fault_address; > u32 error_code; > - bool event_injection = false; > - > - if (!irqchip_in_kernel(kvm) && > - is_external_interrupt(exit_int_info)) { > - event_injection = true; > - kvm_push_irq(&svm->vcpu, exit_int_info & SVM_EVTINJ_VEC_MASK); > - } > > fault_address = svm->vmcb->control.exit_info_2; > error_code = svm->vmcb->control.exit_info_1; > @@ -1118,9 +1103,11 @@ static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) > */ > if (npt_enabled) > svm_flush_tlb(&svm->vcpu); > - > - if (!npt_enabled && event_injection) > - kvm_mmu_unprotect_page_virt(&svm->vcpu, fault_address); > + else { > + if (svm->vcpu.arch.interrupt.pending || > + svm->vcpu.arch.exception.pending) > + kvm_mmu_unprotect_page_virt(&svm->vcpu, fault_address); > + } Without understanding yet why kvm_mmu_unprotect_page_virt is required here, this looks like it is lacking '|| svm->vcpu.arch.nmi_injected'. Interrupts and exceptions are re-queued on fault-during-injection, therefore they are now pending again, right? Jan
Dmitry Eremin-Solenikov wrote: > Gleb Natapov wrote: > >> Start to use interrupt/exception queues like VMX does. This also fix the >> bug that if exit was caused by a guest internal exception access to IDT >> the exception was not reinjected. >> > > > OK. On real SVM HW this seems to work. However now i'm stumbled upon another > problem wrt. NMI. See another mail. > I can confirm that this series (probably this patch) breaks the following scenario: qemu-system-x86_64 (qemu trunk, emulation mode) linux (kvm.git) with kvm-amd + kvm-userspace linux (the same kvm.git) The second-level guest hangs in the mid of its boot, maybe no longer receiving some or any interrupt. So this patch may either expose a bug in the svm emulation of qemu or comes with a subtle regression that only triggers due to qemu's timing. This needs to be understood. Gleb, any progress on reproducing it on your side? Jan
On Sat, Apr 18, 2009 at 11:05:10AM +0200, Jan Kiszka wrote: > Gleb Natapov wrote: > > Start to use interrupt/exception queues like VMX does. > > This also fix the bug that if exit was caused by a guest > > internal exception access to IDT the exception was not > > reinjected. > > > > Signed-off-by: Gleb Natapov <gleb@redhat.com> > > --- > > arch/x86/kvm/svm.c | 176 ++++++++++++++++++++++------------------------------ > > 1 files changed, 75 insertions(+), 101 deletions(-) > > > > diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c > > index 52c41aa..053370d 100644 > > --- a/arch/x86/kvm/svm.c > > +++ b/arch/x86/kvm/svm.c > > @@ -70,7 +70,6 @@ module_param(npt, int, S_IRUGO); > > static int nested = 0; > > module_param(nested, int, S_IRUGO); > > > > -static void kvm_reput_irq(struct vcpu_svm *svm); > > static void svm_flush_tlb(struct kvm_vcpu *vcpu); > > > > static int nested_svm_exit_handled(struct vcpu_svm *svm, bool kvm_override); > > @@ -199,9 +198,7 @@ static void svm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr, > > > > static bool svm_exception_injected(struct kvm_vcpu *vcpu) > > { > > - struct vcpu_svm *svm = to_svm(vcpu); > > - > > - return !(svm->vmcb->control.exit_int_info & SVM_EXITINTINFO_VALID); > > + return false; > > } > > > > static int is_external_interrupt(u32 info) > > @@ -976,12 +973,9 @@ static int svm_guest_debug(struct kvm_vcpu *vcpu, struct kvm_guest_debug *dbg) > > > > static int svm_get_irq(struct kvm_vcpu *vcpu) > > { > > - struct vcpu_svm *svm = to_svm(vcpu); > > - u32 exit_int_info = svm->vmcb->control.exit_int_info; > > - > > - if (is_external_interrupt(exit_int_info)) > > - return exit_int_info & SVM_EVTINJ_VEC_MASK; > > - return -1; > > + if (!vcpu->arch.interrupt.pending) > > + return -1; > > + return vcpu->arch.interrupt.nr; > > } > > > > static void load_host_msrs(struct kvm_vcpu *vcpu) > > @@ -1088,17 +1082,8 @@ static void svm_set_dr(struct kvm_vcpu *vcpu, int dr, unsigned long value, > > > > static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) > > { > > - u32 exit_int_info = svm->vmcb->control.exit_int_info; > > - struct kvm *kvm = svm->vcpu.kvm; > > u64 fault_address; > > u32 error_code; > > - bool event_injection = false; > > - > > - if (!irqchip_in_kernel(kvm) && > > - is_external_interrupt(exit_int_info)) { > > - event_injection = true; > > - kvm_push_irq(&svm->vcpu, exit_int_info & SVM_EVTINJ_VEC_MASK); > > - } > > > > fault_address = svm->vmcb->control.exit_info_2; > > error_code = svm->vmcb->control.exit_info_1; > > @@ -1118,9 +1103,11 @@ static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) > > */ > > if (npt_enabled) > > svm_flush_tlb(&svm->vcpu); > > - > > - if (!npt_enabled && event_injection) > > - kvm_mmu_unprotect_page_virt(&svm->vcpu, fault_address); > > + else { > > + if (svm->vcpu.arch.interrupt.pending || > > + svm->vcpu.arch.exception.pending) > > + kvm_mmu_unprotect_page_virt(&svm->vcpu, fault_address); > > + } > > Without understanding yet why kvm_mmu_unprotect_page_virt is required > here, this looks like it is lacking '|| svm->vcpu.arch.nmi_injected'. > Interrupts and exceptions are re-queued on fault-during-injection, > therefore they are now pending again, right? > Yes right, and we discussed this with Avi already. I'll send another patch after this series will be applied (exactly the same issue exists for VMX btw). -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, Apr 18, 2009 at 11:16:47AM +0200, Jan Kiszka wrote: > Dmitry Eremin-Solenikov wrote: > > Gleb Natapov wrote: > > > >> Start to use interrupt/exception queues like VMX does. This also fix the > >> bug that if exit was caused by a guest internal exception access to IDT > >> the exception was not reinjected. > >> > > > > > > OK. On real SVM HW this seems to work. However now i'm stumbled upon another > > problem wrt. NMI. See another mail. > > > > I can confirm that this series (probably this patch) breaks the > following scenario: > > qemu-system-x86_64 (qemu trunk, emulation mode) > linux (kvm.git) with kvm-amd + kvm-userspace > linux (the same kvm.git) > > The second-level guest hangs in the mid of its boot, maybe no longer > receiving some or any interrupt. > > So this patch may either expose a bug in the svm emulation of qemu or > comes with a subtle regression that only triggers due to qemu's timing. > This needs to be understood. Gleb, any progress on reproducing it on > your side? > I reproduced it and I am debugging it. In my case the boot hangs on sti;hlt sequence. Instrumentation thus far shows that at this point interrupts no longer injected because ppr value is too big. Need to see why, but tpr handling is not complete in qemu svm. May be this is the reason. Will know more tomorrow. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Jan Kiszka wrote: >> - >> - if (!npt_enabled && event_injection) >> - kvm_mmu_unprotect_page_virt(&svm->vcpu, fault_address); >> + else { >> + if (svm->vcpu.arch.interrupt.pending || >> + svm->vcpu.arch.exception.pending) >> + kvm_mmu_unprotect_page_virt(&svm->vcpu, fault_address); >> + } >> > > Without understanding yet why kvm_mmu_unprotect_page_virt is required > here, That is needed in case interrupt injection failed because the process of injecting the interrupt (usually pushing data on the stack) writes to a write-protected page table. I guess we need nmi_pending here as well.
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 52c41aa..053370d 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -70,7 +70,6 @@ module_param(npt, int, S_IRUGO); static int nested = 0; module_param(nested, int, S_IRUGO); -static void kvm_reput_irq(struct vcpu_svm *svm); static void svm_flush_tlb(struct kvm_vcpu *vcpu); static int nested_svm_exit_handled(struct vcpu_svm *svm, bool kvm_override); @@ -199,9 +198,7 @@ static void svm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr, static bool svm_exception_injected(struct kvm_vcpu *vcpu) { - struct vcpu_svm *svm = to_svm(vcpu); - - return !(svm->vmcb->control.exit_int_info & SVM_EXITINTINFO_VALID); + return false; } static int is_external_interrupt(u32 info) @@ -976,12 +973,9 @@ static int svm_guest_debug(struct kvm_vcpu *vcpu, struct kvm_guest_debug *dbg) static int svm_get_irq(struct kvm_vcpu *vcpu) { - struct vcpu_svm *svm = to_svm(vcpu); - u32 exit_int_info = svm->vmcb->control.exit_int_info; - - if (is_external_interrupt(exit_int_info)) - return exit_int_info & SVM_EVTINJ_VEC_MASK; - return -1; + if (!vcpu->arch.interrupt.pending) + return -1; + return vcpu->arch.interrupt.nr; } static void load_host_msrs(struct kvm_vcpu *vcpu) @@ -1088,17 +1082,8 @@ static void svm_set_dr(struct kvm_vcpu *vcpu, int dr, unsigned long value, static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) { - u32 exit_int_info = svm->vmcb->control.exit_int_info; - struct kvm *kvm = svm->vcpu.kvm; u64 fault_address; u32 error_code; - bool event_injection = false; - - if (!irqchip_in_kernel(kvm) && - is_external_interrupt(exit_int_info)) { - event_injection = true; - kvm_push_irq(&svm->vcpu, exit_int_info & SVM_EVTINJ_VEC_MASK); - } fault_address = svm->vmcb->control.exit_info_2; error_code = svm->vmcb->control.exit_info_1; @@ -1118,9 +1103,11 @@ static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) */ if (npt_enabled) svm_flush_tlb(&svm->vcpu); - - if (!npt_enabled && event_injection) - kvm_mmu_unprotect_page_virt(&svm->vcpu, fault_address); + else { + if (svm->vcpu.arch.interrupt.pending || + svm->vcpu.arch.exception.pending) + kvm_mmu_unprotect_page_virt(&svm->vcpu, fault_address); + } return kvm_mmu_page_fault(&svm->vcpu, fault_address, error_code); } @@ -2187,7 +2174,6 @@ static int handle_exit(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu) } } - kvm_reput_irq(svm); if (svm->vmcb->control.exit_code == SVM_EXIT_ERR) { kvm_run->exit_reason = KVM_EXIT_FAIL_ENTRY; @@ -2289,98 +2275,47 @@ static int svm_interrupt_allowed(struct kvm_vcpu *vcpu) (svm->vcpu.arch.hflags & HF_GIF_MASK); } -static void svm_intr_assist(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) +static void enable_irq_window(struct kvm_vcpu *vcpu) { - struct vcpu_svm *svm = to_svm(vcpu); - struct vmcb *vmcb = svm->vmcb; - int intr_vector = -1; - - if ((vmcb->control.exit_int_info & SVM_EVTINJ_VALID) && - ((vmcb->control.exit_int_info & SVM_EVTINJ_TYPE_MASK) == 0)) { - intr_vector = vmcb->control.exit_int_info & - SVM_EVTINJ_VEC_MASK; - vmcb->control.exit_int_info = 0; - svm_inject_irq(svm, intr_vector); - goto out; - } - - if (vmcb->control.int_ctl & V_IRQ_MASK) - goto out; - - if (!kvm_cpu_has_interrupt(vcpu)) - goto out; - - if (nested_svm_intr(svm)) - goto out; - - if (!(svm->vcpu.arch.hflags & HF_GIF_MASK)) - goto out; - - if (!(vmcb->save.rflags & X86_EFLAGS_IF) || - (vmcb->control.int_state & SVM_INTERRUPT_SHADOW_MASK) || - (vmcb->control.event_inj & SVM_EVTINJ_VALID)) { - /* unable to deliver irq, set pending irq */ - svm_set_vintr(svm); - svm_inject_irq(svm, 0x0); - goto out; - } - /* Okay, we can deliver the interrupt: grab it and update PIC state. */ - intr_vector = kvm_cpu_get_interrupt(vcpu); - svm_inject_irq(svm, intr_vector); -out: - update_cr8_intercept(vcpu); + svm_set_vintr(to_svm(vcpu)); + svm_inject_irq(to_svm(vcpu), 0x0); } -static void kvm_reput_irq(struct vcpu_svm *svm) +static void svm_intr_inject(struct kvm_vcpu *vcpu) { - struct vmcb_control_area *control = &svm->vmcb->control; - - if ((control->int_ctl & V_IRQ_MASK) - && !irqchip_in_kernel(svm->vcpu.kvm)) { - control->int_ctl &= ~V_IRQ_MASK; - kvm_push_irq(&svm->vcpu, control->int_vector); + /* try to reinject previous events if any */ + if (vcpu->arch.interrupt.pending) { + svm_inject_irq(to_svm(vcpu), vcpu->arch.interrupt.nr); + return; } - svm->vcpu.arch.interrupt_window_open = - !(control->int_state & SVM_INTERRUPT_SHADOW_MASK) && - (svm->vcpu.arch.hflags & HF_GIF_MASK); -} - -static void svm_do_inject_vector(struct vcpu_svm *svm) -{ - svm_inject_irq(svm, kvm_pop_irq(&svm->vcpu)); + /* try to inject new event if pending */ + if (kvm_cpu_has_interrupt(vcpu)) { + if (vcpu->arch.interrupt_window_open) { + kvm_queue_interrupt(vcpu, kvm_cpu_get_interrupt(vcpu)); + svm_inject_irq(to_svm(vcpu), vcpu->arch.interrupt.nr); + } + } } -static void do_interrupt_requests(struct kvm_vcpu *vcpu, - struct kvm_run *kvm_run) +static void svm_intr_assist(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) { struct vcpu_svm *svm = to_svm(vcpu); - struct vmcb_control_area *control = &svm->vmcb->control; + bool req_int_win = !irqchip_in_kernel(vcpu->kvm) && + kvm_run->request_interrupt_window; if (nested_svm_intr(svm)) - return; + goto out; - svm->vcpu.arch.interrupt_window_open = - (!(control->int_state & SVM_INTERRUPT_SHADOW_MASK) && - (svm->vmcb->save.rflags & X86_EFLAGS_IF) && - (svm->vcpu.arch.hflags & HF_GIF_MASK)); + svm->vcpu.arch.interrupt_window_open = svm_interrupt_allowed(vcpu); - if (svm->vcpu.arch.interrupt_window_open && - kvm_cpu_has_interrupt(&svm->vcpu)) - /* - * If interrupts enabled, and not blocked by sti or mov ss. Good. - */ - svm_do_inject_vector(svm); + svm_intr_inject(vcpu); - /* - * Interrupts blocked. Wait for unblock. - */ - if (!svm->vcpu.arch.interrupt_window_open && - (kvm_cpu_has_interrupt(&svm->vcpu) || - kvm_run->request_interrupt_window)) - svm_set_vintr(svm); - else - svm_clear_vintr(svm); + if (kvm_cpu_has_interrupt(vcpu) || req_int_win) + enable_irq_window(vcpu); + +out: + update_cr8_intercept(vcpu); } static int svm_set_tss_addr(struct kvm *kvm, unsigned int addr) @@ -2420,6 +2355,43 @@ static inline void sync_lapic_to_cr8(struct kvm_vcpu *vcpu) svm->vmcb->control.int_ctl |= cr8 & V_TPR_MASK; } +static void svm_complete_interrupts(struct vcpu_svm *svm) +{ + u8 vector; + int type; + u32 exitintinfo = svm->vmcb->control.exit_int_info; + + svm->vcpu.arch.nmi_injected = false; + kvm_clear_exception_queue(&svm->vcpu); + kvm_clear_interrupt_queue(&svm->vcpu); + + if (!(exitintinfo & SVM_EXITINTINFO_VALID)) + return; + + vector = exitintinfo & SVM_EXITINTINFO_VEC_MASK; + type = exitintinfo & SVM_EXITINTINFO_TYPE_MASK; + + switch (type) { + case SVM_EXITINTINFO_TYPE_NMI: + svm->vcpu.arch.nmi_injected = true; + break; + case SVM_EXITINTINFO_TYPE_EXEPT: + if (exitintinfo & SVM_EXITINTINFO_VALID_ERR) { + u32 err = svm->vmcb->control.exit_int_info_err; + kvm_queue_exception_e(&svm->vcpu, vector, err); + + } else + kvm_queue_exception(&svm->vcpu, vector); + break; + case SVM_EXITINTINFO_TYPE_SOFT: + case SVM_EXITINTINFO_TYPE_INTR: + kvm_queue_interrupt(&svm->vcpu, vector); + break; + default: + break; + } +} + #ifdef CONFIG_X86_64 #define R "r" #else @@ -2548,6 +2520,8 @@ static void svm_vcpu_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) sync_cr8_to_lapic(vcpu); svm->next_rip = 0; + + svm_complete_interrupts(svm); } #undef R @@ -2669,7 +2643,7 @@ static struct kvm_x86_ops svm_x86_ops = { .queue_exception = svm_queue_exception, .exception_injected = svm_exception_injected, .inject_pending_irq = svm_intr_assist, - .inject_pending_vectors = do_interrupt_requests, + .inject_pending_vectors = svm_intr_assist, .interrupt_allowed = svm_interrupt_allowed, .set_tss_addr = svm_set_tss_addr,
Start to use interrupt/exception queues like VMX does. This also fix the bug that if exit was caused by a guest internal exception access to IDT the exception was not reinjected. Signed-off-by: Gleb Natapov <gleb@redhat.com> --- arch/x86/kvm/svm.c | 176 ++++++++++++++++++++++------------------------------ 1 files changed, 75 insertions(+), 101 deletions(-)