Message ID | 200904081026.02816.sheng@linux.intel.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Hi Sheng, On Wed, 2009-04-08 at 10:26 +0800, Sheng Yang wrote: > On Wednesday 08 April 2009 00:38:10 Alex Williamson wrote: > > On Tue, 2009-04-07 at 14:09 +0800, Sheng Yang wrote: > > > Could you enable DEVICE_ASSSIGNMENT_DEBUG=1 in > > > qemu/hw/device-assignment.c and post the output? > > > > Yup, see below. The error comes after I 'ifdown eth0; ifup eth0' in the > > guest. Note bnx2 appears to only turn on MSIX for SMP systems. Thanks, > > > > Alex > > Seems your "ifdown/ifup" script reload the module? No, the bnx2 module isn't unloaded on ifdown. > Oh god, I found one bug > after checked the spec: > > System software reads this field to determine the MSI-X Table Size *N*, which > is encoded as *N-1*. For example, a returned value of “00000000011†indicates > a table size of 4. > > But it seems still can't explain the problem...(OK, it may affect the guest in > a unknown way as well...) I would post a fix for it soon. [snip] > > The writing to MMIO have been intercepted, but code fail to count it? > Strange... > > Could you try this debug? I added the debug printfs, plus the MSI-X table size patch, and printed the value of msg_ctrl as we loop through. Output below. This is what made me think the MSI-X state isn't getting cleared when the driver closes the interface. Let me know what you think. Thanks, Alex init_assigned_device: Registering real physical device 03:00.0 (bus=3 dev=0 func=0) get_real_device: region 0 size 33554432 start 0xf4000000 type 512 resource_fd 19 assigned_dev_pci_read_config: (4.0): address=0000 val=0x000014e4 len=2 assigned_dev_pci_read_config: (4.0): address=0002 val=0x00001639 len=2 assigned_dev_pci_read_config: (4.0): address=0000 val=0x000014e4 len=2 assigned_dev_pci_read_config: (4.0): address=0002 val=0x00001639 len=2 assigned_dev_pci_read_config: (4.0): address=0000 val=0x000014e4 len=2 assigned_dev_pci_read_config: (4.0): address=0002 val=0x00001639 len=2 assigned_dev_pci_read_config: (4.0): address=000a val=0x00000200 len=2 assigned_dev_pci_read_config: (4.0): address=0000 val=0x000014e4 len=2 assigned_dev_pci_read_config: (4.0): address=0002 val=0x00001639 len=2 assigned_dev_pci_write_config: (4.0): address=0010 val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=0010 val=0xfe000000 len=4 assigned_dev_pci_read_config: (4.0): address=0010 val=0xfe000000 len=4 assigned_dev_pci_write_config: (4.0): address=0010 val=0xf4000000 len=4 assigned_dev_iomem_map: e_phys=f4000000 r_virt=0x7f4fd9bfc000 type=0 len=02000000 region_num=0 assigned_dev_iomem_map: munmap done, virt_base 0x0x7f4fd9c08000 assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000442 len=2 assigned_dev_pci_write_config: (4.0): address=0004 val=0x00000442 len=2 assigned_dev_pci_write_config: NON BAR (4.0): address=0004 val=0x00000442 len=2 assigned_dev_pci_write_config: (4.0): address=0014 val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=0014 val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=0018 val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=0018 val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=001c val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=001c val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=0020 val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=0020 val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=0024 val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=0024 val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=0030 val=0xffffffff len=4 assigned_dev_pci_write_config: NON BAR (4.0): address=0030 val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=0030 val=0xffff0001 len=4 assigned_dev_pci_read_config: (4.0): address=0030 val=0xffff0001 len=4 assigned_dev_pci_write_config: (4.0): address=0030 val=0x00010000 len=4 assigned_dev_pci_write_config: NON BAR (4.0): address=0030 val=0x00010000 len=4 assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000442 len=2 assigned_dev_pci_write_config: (4.0): address=0004 val=0x00000442 len=2 assigned_dev_pci_write_config: NON BAR (4.0): address=0004 val=0x00000442 len=2 assigned_dev_pci_read_config: (4.0): address=003d val=0x00000001 len=1 assigned_dev_pci_write_config: (4.0): address=003c val=0x0000000b len=1 assigned_dev_pci_read_config: (4.0): address=000a val=0x00000200 len=2 assigned_dev_pci_read_config: (4.0): address=0000 val=0x000014e4 len=2 assigned_dev_pci_read_config: (4.0): address=0002 val=0x00001639 len=2 assigned_dev_pci_read_config: (4.0): address=000e val=0x00000080 len=1 assigned_dev_pci_read_config: (4.0): address=0000 val=0x163914e4 len=4 assigned_dev_pci_read_config: (4.0): address=000e val=0x00000080 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_read_config: (4.0): address=0008 val=0x02000020 len=4 assigned_dev_pci_read_config: (4.0): address=003d val=0x00000001 len=1 assigned_dev_pci_read_config: (4.0): address=003c val=0x0000000b len=1 assigned_dev_pci_read_config: (4.0): address=0010 val=0xf4000000 len=4 assigned_dev_pci_write_config: (4.0): address=0010 val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=0010 val=0xfe000000 len=4 assigned_dev_pci_write_config: (4.0): address=0010 val=0xf4000000 len=4 assigned_dev_iomem_map: e_phys=f4000000 r_virt=0x7f4fd9bfc000 type=0 len=02000000 region_num=0 BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1) assigned_dev_iomem_map: munmap done, virt_base 0x0x7f4fd9c08000 assigned_dev_pci_read_config: (4.0): address=0014 val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=0014 val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=0014 val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=0014 val=0x00000000 len=4 assigned_dev_pci_read_config: (4.0): address=0018 val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=0018 val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=0018 val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=0018 val=0x00000000 len=4 assigned_dev_pci_read_config: (4.0): address=001c val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=001c val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=001c val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=001c val=0x00000000 len=4 assigned_dev_pci_read_config: (4.0): address=0020 val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=0020 val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=0020 val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=0020 val=0x00000000 len=4 assigned_dev_pci_read_config: (4.0): address=0024 val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=0024 val=0xffffffff len=4 assigned_dev_pci_read_config: (4.0): address=0024 val=0x00000000 len=4 assigned_dev_pci_write_config: (4.0): address=0024 val=0x00000000 len=4 assigned_dev_pci_read_config: (4.0): address=0030 val=0x00010000 len=4 assigned_dev_pci_write_config: (4.0): address=0030 val=0xfffffffe len=4 assigned_dev_pci_write_config: NON BAR (4.0): address=0030 val=0xfffffffe len=4 assigned_dev_pci_read_config: (4.0): address=0030 val=0xffff0000 len=4 assigned_dev_pci_write_config: (4.0): address=0030 val=0x00010000 len=4 assigned_dev_pci_write_config: NON BAR (4.0): address=0030 val=0x00010000 len=4 assigned_dev_pci_read_config: (4.0): address=002c val=0x0000103c len=2 assigned_dev_pci_read_config: (4.0): address=002e val=0x00007055 len=2 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000442 len=2 assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000442 len=2 assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000442 len=2 assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000442 len=2 assigned_dev_pci_write_config: (4.0): address=0004 val=0x00000446 len=2 assigned_dev_pci_write_config: NON BAR (4.0): address=0004 val=0x00000446 len=2 assigned_dev_pci_read_config: (4.0): address=000d val=0x00000000 len=1 assigned_dev_pci_write_config: (4.0): address=000d val=0x00000040 len=1 assigned_dev_pci_write_config: NON BAR (4.0): address=000d val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0000 val=0x163914e4 len=4 assigned_dev_pci_read_config: (4.0): address=0004 val=0x00100446 len=4 assigned_dev_pci_read_config: (4.0): address=0008 val=0x02000020 len=4 assigned_dev_pci_read_config: (4.0): address=000c val=0x00800010 len=4 assigned_dev_pci_read_config: (4.0): address=0010 val=0xf4000000 len=4 assigned_dev_pci_read_config: (4.0): address=0014 val=0x00000000 len=4 assigned_dev_pci_read_config: (4.0): address=0018 val=0x00000000 len=4 assigned_dev_pci_read_config: (4.0): address=001c val=0x00000000 len=4 assigned_dev_pci_read_config: (4.0): address=0020 val=0x00000000 len=4 assigned_dev_pci_read_config: (4.0): address=0024 val=0x00000000 len=4 assigned_dev_pci_read_config: (4.0): address=0028 val=0x00000000 len=4 assigned_dev_pci_read_config: (4.0): address=002c val=0x7055103c len=4 assigned_dev_pci_read_config: (4.0): address=0030 val=0x00010000 len=4 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=4 assigned_dev_pci_read_config: (4.0): address=0038 val=0x00000000 len=4 assigned_dev_pci_read_config: (4.0): address=003c val=0x0000010b len=4 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_write_config: (4.0): address=0068 val=0x00000088 len=4 assigned_dev_pci_write_config: NON BAR (4.0): address=0068 val=0x00000088 len=4 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2 assigned_dev_pci_write_config: (4.0): address=0052 val=0x00000008 len=2 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2 assigned_dev_pci_read_config: (4.0): address=0054 val=0x0000c000 len=4 msix_mmio_writel: write to MSI-X entry table mmio offset 0x0, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x4, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x8, val 0x4149 msix_mmio_writel: write to MSI-X entry table mmio offset 0x10, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x14, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x18, val 0x4151 msix_mmio_writel: write to MSI-X entry table mmio offset 0x20, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x24, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x28, val 0x4159 msix_mmio_writel: write to MSI-X entry table mmio offset 0x30, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x34, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x38, val 0x4161 msix_mmio_writel: write to MSI-X entry table mmio offset 0x40, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x44, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x48, val 0x4169 msix_mmio_writel: write to MSI-X entry table mmio offset 0x50, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x54, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x58, val 0x4171 msix_mmio_writel: write to MSI-X entry table mmio offset 0x60, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x64, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x68, val 0x4179 msix_mmio_writel: write to MSI-X entry table mmio offset 0x70, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x74, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x78, val 0x4181 msix_mmio_writel: write to MSI-X entry table mmio offset 0x80, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x84, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x88, val 0x4189 assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000046 len=2 assigned_dev_pci_write_config: (4.0): address=0004 val=0x00000446 len=2 assigned_dev_pci_write_config: NON BAR (4.0): address=0004 val=0x00000446 len=2 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2 assigned_dev_pci_write_config: (4.0): address=0052 val=0x00008008 len=2 the MSIX capabilty position is 0x50 the MSIX entries_max_nr is 0x9 0: msg_ctrl: 0000 1: msg_ctrl: 0000 2: msg_ctrl: 0000 3: msg_ctrl: 0000 4: msg_ctrl: 0000 5: msg_ctrl: 0000 6: msg_ctrl: 0000 7: msg_ctrl: 0000 8: msg_ctrl: 0000 assigned_dev_update_msix_mmio: MSI-X data 0x4149, MSI-X addr_lo 0xfeeff00c !assigned_dev_update_msix_mmio: MSI-X entry gsi 0x18, entry 0 !assigned_dev_update_msix_mmio: MSI-X data 0x4151, MSI-X addr_lo 0xfeeff00c !assigned_dev_update_msix_mmio: MSI-X entry gsi 0x19, entry 1 !assigned_dev_update_msix_mmio: MSI-X data 0x4159, MSI-X addr_lo 0xfeeff00c !assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1a, entry 2 !assigned_dev_update_msix_mmio: MSI-X data 0x4161, MSI-X addr_lo 0xfeeff00c !assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1b, entry 3 !assigned_dev_update_msix_mmio: MSI-X data 0x4169, MSI-X addr_lo 0xfeeff00c !assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1c, entry 4 !assigned_dev_update_msix_mmio: MSI-X data 0x4171, MSI-X addr_lo 0xfeeff00c !assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1d, entry 5 !assigned_dev_update_msix_mmio: MSI-X data 0x4179, MSI-X addr_lo 0xfeeff00c !assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1e, entry 6 !assigned_dev_update_msix_mmio: MSI-X data 0x4181, MSI-X addr_lo 0xfeeff00c !assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1f, entry 7 !assigned_dev_update_msix_mmio: MSI-X data 0x4189, MSI-X addr_lo 0xfeeff00c !assigned_dev_update_msix_mmio: MSI-X entry gsi 0x20, entry 8 !msix_mmio_writel: write to MSI-X entry table mmio offset 0xc, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x1c, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x2c, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x3c, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x4c, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x5c, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x6c, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x7c, val 0x0 assigned_dev_pci_write_config: (4.0): address=0068 val=0x00000088 len=4 assigned_dev_pci_write_config: NON BAR (4.0): address=0068 val=0x00000088 len=4 assigned_dev_pci_write_config: (4.0): address=0068 val=0x00000088 len=4 assigned_dev_pci_write_config: NON BAR (4.0): address=0068 val=0x00000088 len=4 msix_mmio_writel: write to MSI-X entry table mmio offset 0xc, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x1c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x2c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x3c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x4c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x5c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x6c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x7c, val 0x1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0052 val=0x00008008 len=2 assigned_dev_pci_write_config: (4.0): address=0052 val=0x00000008 len=2 assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000046 len=2 msix_mmio_writel: write to MSI-X entry table mmio offset 0xc, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x1c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x2c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x3c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x4c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x5c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x6c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x7c, val 0x1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x8c, val 0x1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2 assigned_dev_pci_write_config: (4.0): address=0052 val=0x00000008 len=2 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2 assigned_dev_pci_read_config: (4.0): address=0054 val=0x0000c000 len=4 msix_mmio_writel: write to MSI-X entry table mmio offset 0x0, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x4, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x8, val 0x4191 msix_mmio_writel: write to MSI-X entry table mmio offset 0x10, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x14, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x18, val 0x4199 msix_mmio_writel: write to MSI-X entry table mmio offset 0x20, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x24, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x28, val 0x41a1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x30, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x34, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x38, val 0x41a9 msix_mmio_writel: write to MSI-X entry table mmio offset 0x40, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x44, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x48, val 0x41b1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x50, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x54, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x58, val 0x41b9 msix_mmio_writel: write to MSI-X entry table mmio offset 0x60, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x64, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x68, val 0x41c1 msix_mmio_writel: write to MSI-X entry table mmio offset 0x70, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x74, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x78, val 0x41c9 msix_mmio_writel: write to MSI-X entry table mmio offset 0x80, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table mmio offset 0x84, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x88, val 0x41d1 assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000046 len=2 assigned_dev_pci_write_config: (4.0): address=0004 val=0x00000446 len=2 assigned_dev_pci_write_config: NON BAR (4.0): address=0004 val=0x00000446 len=2 assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2 assigned_dev_pci_write_config: (4.0): address=0052 val=0x00008008 len=2 the MSIX capabilty position is 0x50 the MSIX entries_max_nr is 0x9 0: msg_ctrl: 0001 1: msg_ctrl: 0001 2: msg_ctrl: 0001 3: msg_ctrl: 0001 4: msg_ctrl: 0001 5: msg_ctrl: 0001 6: msg_ctrl: 0001 7: msg_ctrl: 0001 8: msg_ctrl: 0001 MSI-X entry number is zero! assigned_dev_update_msix_mmio: No such device or address msix_mmio_writel: write to MSI-X entry table mmio offset 0xc, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x1c, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x2c, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x3c, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x4c, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x5c, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x6c, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x7c, val 0x0 assigned_dev_pci_write_config: (4.0): address=0068 val=0x00000088 len=4 assigned_dev_pci_write_config: NON BAR (4.0): address=0068 val=0x00000088 len=4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thursday 09 April 2009 00:13:56 Alex Williamson wrote: > Hi Sheng, > > On Wed, 2009-04-08 at 10:26 +0800, Sheng Yang wrote: > > On Wednesday 08 April 2009 00:38:10 Alex Williamson wrote: > > > On Tue, 2009-04-07 at 14:09 +0800, Sheng Yang wrote: > > > > Could you enable DEVICE_ASSSIGNMENT_DEBUG=1 in > > > > qemu/hw/device-assignment.c and post the output? > > > > > > Yup, see below. The error comes after I 'ifdown eth0; ifup eth0' in > > > the guest. Note bnx2 appears to only turn on MSIX for SMP systems. > > > Thanks, > > > > > > Alex > > > > Seems your "ifdown/ifup" script reload the module? > > No, the bnx2 module isn't unloaded on ifdown. > > > Oh god, I found one bug > > after checked the spec: > > > > System software reads this field to determine the MSI-X Table Size *N*, > > which is encoded as *N-1*. For example, a returned value of “00000000011†> > indicates a table size of 4. > > > > But it seems still can't explain the problem...(OK, it may affect the > > guest in a unknown way as well...) I would post a fix for it soon. > > [snip] > > > The writing to MMIO have been intercepted, but code fail to count it? > > Strange... > > > > Could you try this debug? > > I added the debug printfs, plus the MSI-X table size patch, and printed > the value of msg_ctrl as we loop through. Output below. This is what > made me think the MSI-X state isn't getting cleared when the driver > closes the interface. Let me know what you think. Thanks, > Thanks Alex, now I know where the problem is. Part of functional haven't been implemented... > address=0052 val=0x00008008 len=2 the MSIX capabilty position is 0x50 > the MSIX entries_max_nr is 0x9 > 0: msg_ctrl: 0001 > 1: msg_ctrl: 0001 > 2: msg_ctrl: 0001 > 3: msg_ctrl: 0001 > 4: msg_ctrl: 0001 > 5: msg_ctrl: 0001 > 6: msg_ctrl: 0001 > 7: msg_ctrl: 0001 > 8: msg_ctrl: 0001 > MSI-X entry number is zero! > assigned_dev_update_msix_mmio: No such device or address Driver write to the vectors at first, then enable MSI-X, > msix_mmio_writel: write to MSI-X entry table mmio offset 0xc, val 0x0 > msix_mmio_writel: write to MSI-X entry table mmio offset 0x1c, val 0x0 > msix_mmio_writel: write to MSI-X entry table mmio offset 0x2c, val 0x0 > msix_mmio_writel: write to MSI-X entry table mmio offset 0x3c, val 0x0 > msix_mmio_writel: write to MSI-X entry table mmio offset 0x4c, val 0x0 > msix_mmio_writel: write to MSI-X entry table mmio offset 0x5c, val 0x0 > msix_mmio_writel: write to MSI-X entry table mmio offset 0x6c, val 0x0 > msix_mmio_writel: write to MSI-X entry table mmio offset 0x7c, val 0x0 And finally clear the mask bit. For current we didn't implement mask capability in MSI-X vectors, so it won't work... OK. I'd like to remove the check of mask bit and only ignored unused vector when msg data is zero now(hope it won't cause more problems). And we would add support for per-vector mask later. Thanks for help to debug!
diff --git a/qemu/hw/device-assignment.c b/qemu/hw/device-assignment.c index 09e54ae..ba31bed 100644 --- a/qemu/hw/device-assignment.c +++ b/qemu/hw/device-assignment.c @@ -45,7 +45,7 @@ #define IORESOURCE_DMA 0x00000800 #define IORESOURCE_PREFETCH 0x00001000 /* No side effects */ -/* #define DEVICE_ASSIGNMENT_DEBUG 1 */ +#define DEVICE_ASSIGNMENT_DEBUG 1 #ifdef DEVICE_ASSIGNMENT_DEBUG #define DEBUG(fmt, ...) \ @@ -816,9 +816,13 @@ static int assigned_dev_update_msix_mmio(PCIDevice *pci_dev) else pos = pci_dev->cap.start; + fprintf(stderr, "the MSIX capabilty position is 0x%lx\n", pos); + entries_max_nr = pci_dev->config[pos + 2]; entries_max_nr &= PCI_MSIX_TABSIZE; + fprintf(stderr, "the MSIX entries_max_nr is 0x%lx\n", entries_max_nr); + /* Get the usable entry number for allocating */ for (i = 0; i < entries_max_nr; i++) {