[BUG] pci-passthrough generates "xen:events: Failed to obtain physical IRQ" for some devices

Commit Message

Konrad Rzeszutek Wilk Feb. 3, 2016, 3:26 p.m. UTC

On Wed, Feb 03, 2016 at 03:22:30PM +0100, Marek Marczykowski-Górecki wrote:
> On Mon, Feb 01, 2016 at 09:50:53AM -0500, Konrad Rzeszutek Wilk wrote:
> > > The second bullet looks at first pretty interesting from this PoV,
> > > see http://xenbits.xen.org/xsa/advisory-157.html for info on the XSA and
> > > the various patches. Konrad is on the CC already so hopefully he has some
> > > ideas.
> > 
> > Thanks. I will try to reproduce this with the upstream kernel first as
> > those patches are there.
> 
> According to one Qubes OS user report[1], the bug was introduced between
> version, which differs only by XSA-155 patches (including one for
> pciback), especially not XSA-157. 
> Maybe on some code path, some value is not copied back to pdev->sh_info->op?

I found two bugs (attached the draft not-compiled patches). Upstream
wise I seem to be tripping over another issue.

There is also some more work required in there to fix the MSI-x enable op.
From 34c754a209747826336f6e1f0477a55d214fcc28 Mon Sep 17 00:00:00 2001
From: Konrad Rzeszutek Wilk <[e-mail]>
Date: Wed, 3 Feb 2016 10:18:18 -0500
Subject: [PATCH 1/2] xen/pciback: Check PF instead of VF for
 PCI_COMMAND_MEMORY

c/s 408fb0e5aa7fda0059db282ff58c3b2a4278baa0
"xen/pciback: Don't allow MSI-X ops if PCI_COMMAND_MEMORY is not set."
would check the device for PCI_COMMAND_MEMORY which is great.
Except that VF devices are unique - for example they have no
legacy interrupts, and also any writes to PCI_COMMAND_MEMORY
are silently ignored (by the hardware).

CC: [e-mail]
Signed-off-by: Konrad Rzeszutek Wilk <[e-mail]>
---
 drivers/xen/xen-pciback/pciback_ops.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

Comments

Marek Marczykowski-Górecki Feb. 8, 2016, 5:39 p.m. UTC | #1

On Wed, Feb 03, 2016 at 10:26:58AM -0500, Konrad Rzeszutek Wilk wrote:
> On Wed, Feb 03, 2016 at 03:22:30PM +0100, Marek Marczykowski-Górecki wrote:
> > On Mon, Feb 01, 2016 at 09:50:53AM -0500, Konrad Rzeszutek Wilk wrote:
> > > > The second bullet looks at first pretty interesting from this PoV,
> > > > see http://xenbits.xen.org/xsa/advisory-157.html for info on the XSA and
> > > > the various patches. Konrad is on the CC already so hopefully he has some
> > > > ideas.
> > > 
> > > Thanks. I will try to reproduce this with the upstream kernel first as
> > > those patches are there.
> > 
> > According to one Qubes OS user report[1], the bug was introduced between
> > version, which differs only by XSA-155 patches (including one for
> > pciback), especially not XSA-157. 
> > Maybe on some code path, some value is not copied back to pdev->sh_info->op?
> 
> I found two bugs (attached the draft not-compiled patches). Upstream
> wise I seem to be tripping over another issue.
> 
> There is also some more work required in there to fix the MSI-x enable op.

What exactly do you have in mind here? That four patches in your next
email? Or something not yet fixed?

Patch

diff --git a/drivers/xen/xen-pciback/pciback_ops.c b/drivers/xen/xen-pciback/pciback_ops.c
index 73dafdc..8c86a53 100644
--- a/drivers/xen/xen-pciback/pciback_ops.c
+++ b/drivers/xen/xen-pciback/pciback_ops.c
@@ -213,6 +213,7 @@  int xen_pcibk_enable_msix(struct xen_pcibk_device *pdev,
 	int i, result;
 	struct msix_entry *entries;
 	u16 cmd;
+	struct pci_dev *phys_dev;
 
 	if (unlikely(verbose_request))
 		printk(KERN_DEBUG DRV_NAME ": %s: enable MSI-X\n",
@@ -227,8 +228,10 @@  int xen_pcibk_enable_msix(struct xen_pcibk_device *pdev,
 	/*
 	 * PCI_COMMAND_MEMORY must be enabled, otherwise we may not be able
 	 * to access the BARs where the MSI-X entries reside.
+	 * But VF devices are unique in which the PF needs to be checked.
 	 */
-	pci_read_config_word(dev, PCI_COMMAND, &cmd);
+	phys_dev = pci_physfn(dev);
+	pci_read_config_word(phys_dev, PCI_COMMAND, &cmd);
 	if (dev->msi_enabled || !(cmd & PCI_COMMAND_MEMORY))
 		return -ENXIO;