PCI passthrough of USB controllers on Xen 4.8.1, Linux 4.9.29, stubdomain

Commit Message

Marek Marczykowski-Górecki June 6, 2017, 1:14 p.m. UTC

On Tue, Jun 06, 2017 at 04:55:20AM -0600, Jan Beulich wrote:
> >>> On 06.06.17 at 12:41, <[e-mail]> wrote:
> > [root@dom0 ~]# lspci -s 00:14.0 -vvv
> > 00:14.0 USB controller: Intel Corporation Wildcat Point-LP USB xHCI 
> > Controller (rev 03) (prog-if 30 [XHCI])
> > 	Subsystem: Intel Corporation Device 7270
> > 	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
> > Stepping- SERR- FastB2B- DisINTx+
> > 	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> > 	Latency: 0
> > 	Interrupt: pin A routed to IRQ 170
> > 	Region 0: Memory at b2200000 (64-bit, non-prefetchable) [size=64K]
> > 	Capabilities: [70] Power Management version 2
> > 		Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
> > 		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
> > 	Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
> > 		Address: 00000000fee00338  Data: 0000
> 
> So as I did expect the field accessed is the MSI capability structure.
> Such accesses shouldn't reach pciback, but instead be taken care
> of by qemu. I can't really comment on the qemu side though, but
> this at least makes me think the underlying cause of the problems
> you see with the two controllers is the same.
> 
> Is this a regression of some sort, i.e. did the same setup work in
> earlier Xen versions?

This was working with pure PV, but this is probably irrelevant here.

I can't test if switching only Xen version changes anything (that would
require a lot of recompiling), but very similar setup with mini-os based
stubdom on Xen 4.6.5 and Xen 4.8.1 also doesn't work. Even with a qemu
patch disabling MSI reporting (attached). In that case, I don't get any
message from qemu, but very similar messages from xhci driver.

BTW Other patches:
https://github.com/QubesOS/qubes-vmm-xen/tree/xen-4.8/patches.misc

I've also checked Xen 4.8.1 with qemu-upstream in dom0 (instead of
Linux-based stubdom) and it also doesn't work, but with slightly
different messages:

qemu:
    [00:05.0] Write-back to unknown field 0xd8 (partially) inhibited (0x00000000)
    [00:05.0] If the device doesn't work, try enabling permissive mode
    [00:05.0] (unsafe) and if it helps report the problem to xen-devel
    [00:06.0] Write-back to unknown field 0x6c (partially) inhibited (0x00000000)
    [00:06.0] If the device doesn't work, try enabling permissive mode
    [00:06.0] (unsafe) and if it helps report the problem to xen-devel

Linux in domU:

    [   51.679188] xhci_hcd 0000:00:05.0: Error while assigning device slot ID
    [   51.679264] xhci_hcd 0000:00:05.0: Max number of devices this xHCI host supports is 32.
    [   51.679359] usb usb1-port2: couldn't allocate usb_device

(and nothing about ehci, no devices connected to it is visible)

Patch

--- a/tools/qemu-xen-traditional/hw/pass-through.c	2016-10-31 12:48:42.924026468 -0400
+++ b/tools/qemu-xen-traditional/hw/pass-through.c	2016-10-31 12:50:52.953026468 -0400
@@ -874,6 +874,10 @@ 
         .grp_size   = 0x04,
         .size_init  = pt_reg_grp_size_init,
     },
+#ifndef CONFIG_STUBDOM
+    /* At present stubdom doesn't support MSI for passthrough, so let's not
+     * expose MSI capability to stubdom HVM guest for now.
+     */
 #ifndef __ia64__
     /* At present IA64 Xen doesn't support MSI for passthrough, so let's not
      * expose MSI capability to IA64 HVM guest for now.
@@ -886,7 +890,8 @@ 
         .size_init  = pt_msi_size_init,
         .emu_reg_tbl= pt_emu_reg_msi_tbl,
     },
-#endif
+#endif /* __ia64__ */
+#endif /* !CONFIG_STUBDOM */
     /* PCI-X Capabilities List Item reg group */
     {
         .grp_id     = PCI_CAP_ID_PCIX,
@@ -931,6 +936,10 @@ 
         .size_init  = pt_pcie_size_init,
         .emu_reg_tbl= pt_emu_reg_pcie_tbl,
     },
+#ifndef CONFIG_STUBDOM
+    /* At present stubdom doesn't support MSI for passthrough, so let's not
+     * expose MSI-X capability to stubdom HVM guest for now.
+     */
 #ifndef __ia64__
     /* At present IA64 Xen doesn't support MSI for passthrough, so let's not
      * expose MSI-X capability to IA64 HVM guest for now.
@@ -943,7 +952,8 @@ 
         .size_init  = pt_msix_size_init,
         .emu_reg_tbl= pt_emu_reg_msix_tbl,
     },
-#endif
+#endif /* __ia64__ */
+#endif /* !CONFIG_STUBDOM */
     {
         .grp_size = 0,
     },