| Message ID | alpine.DEB.2.10.1701121338410.14454@sstabellini-ThinkPad-X260 (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 01/12/2017 04:39 PM, Stefano Stabellini wrote: > The following commit: > > commit 72a9b186292d98494f222226cfd24a1621796209 > Author: KarimAllah Ahmed <karahmed@amazon.de> > Date: Fri Aug 26 23:55:36 2016 +0200 > > xen: Remove event channel notification through Xen PCI platform device > > broke Linux when booting as Dom0 on Xen in a nested Xen environment (Xen > installed inside a Xen VM). In this scenario, Linux is a PV guest, but > at the same time it uses the platform-pci driver to receive > notifications from L0 Xen. vector callbacks are not available because L1 > Xen doesn't allow them. > > Partially revert the offending commit, by restoring IRQ based > notifications for PV guests only. I restored only the code which is > strictly needed and replaced the xen_have_vector_callback checks within > it with xen_pv_domain() checks. > > Signed-off-by: Stefano Stabellini <sstabellini@kernel.org> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
On Fri, 13 Jan 2017, Boris Ostrovsky wrote: > On 01/12/2017 04:39 PM, Stefano Stabellini wrote: > > The following commit: > > > > commit 72a9b186292d98494f222226cfd24a1621796209 > > Author: KarimAllah Ahmed <karahmed@amazon.de> > > Date: Fri Aug 26 23:55:36 2016 +0200 > > > > xen: Remove event channel notification through Xen PCI platform device > > > > broke Linux when booting as Dom0 on Xen in a nested Xen environment (Xen > > installed inside a Xen VM). In this scenario, Linux is a PV guest, but > > at the same time it uses the platform-pci driver to receive > > notifications from L0 Xen. vector callbacks are not available because L1 > > Xen doesn't allow them. > > > > Partially revert the offending commit, by restoring IRQ based > > notifications for PV guests only. I restored only the code which is > > strictly needed and replaced the xen_have_vector_callback checks within > > it with xen_pv_domain() checks. > > > > Signed-off-by: Stefano Stabellini <sstabellini@kernel.org> > > Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> Applied to xentip/for-linus-4.10, improving the commit message as suggested before. Do you plan on sending out another pull request for 4.10?
On 01/13/2017 01:26 PM, Stefano Stabellini wrote: > On Fri, 13 Jan 2017, Boris Ostrovsky wrote: >> On 01/12/2017 04:39 PM, Stefano Stabellini wrote: >>> The following commit: >>> >>> commit 72a9b186292d98494f222226cfd24a1621796209 >>> Author: KarimAllah Ahmed <karahmed@amazon.de> >>> Date: Fri Aug 26 23:55:36 2016 +0200 >>> >>> xen: Remove event channel notification through Xen PCI platform device >>> >>> broke Linux when booting as Dom0 on Xen in a nested Xen environment (Xen >>> installed inside a Xen VM). In this scenario, Linux is a PV guest, but >>> at the same time it uses the platform-pci driver to receive >>> notifications from L0 Xen. vector callbacks are not available because L1 >>> Xen doesn't allow them. >>> >>> Partially revert the offending commit, by restoring IRQ based >>> notifications for PV guests only. I restored only the code which is >>> strictly needed and replaced the xen_have_vector_callback checks within >>> it with xen_pv_domain() checks. >>> >>> Signed-off-by: Stefano Stabellini <sstabellini@kernel.org> >> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> > Applied to xentip/for-linus-4.10, improving the commit message as > suggested before. > > Do you plan on sending out another pull request for 4.10? Juergen? BTW, this probably wants to go to 4.9-stable too? -boris
On 13/01/17 19:44, Boris Ostrovsky wrote: > On 01/13/2017 01:26 PM, Stefano Stabellini wrote: >> On Fri, 13 Jan 2017, Boris Ostrovsky wrote: >>> On 01/12/2017 04:39 PM, Stefano Stabellini wrote: >>>> The following commit: >>>> >>>> commit 72a9b186292d98494f222226cfd24a1621796209 >>>> Author: KarimAllah Ahmed <karahmed@amazon.de> >>>> Date: Fri Aug 26 23:55:36 2016 +0200 >>>> >>>> xen: Remove event channel notification through Xen PCI platform device >>>> >>>> broke Linux when booting as Dom0 on Xen in a nested Xen environment (Xen >>>> installed inside a Xen VM). In this scenario, Linux is a PV guest, but >>>> at the same time it uses the platform-pci driver to receive >>>> notifications from L0 Xen. vector callbacks are not available because L1 >>>> Xen doesn't allow them. >>>> >>>> Partially revert the offending commit, by restoring IRQ based >>>> notifications for PV guests only. I restored only the code which is >>>> strictly needed and replaced the xen_have_vector_callback checks within >>>> it with xen_pv_domain() checks. >>>> >>>> Signed-off-by: Stefano Stabellini <sstabellini@kernel.org> >>> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> >> Applied to xentip/for-linus-4.10, improving the commit message as >> suggested before. >> >> Do you plan on sending out another pull request for 4.10? > > Juergen? Originally I didn't plan to, but I'm not opposed to sending out one. Juergen
On Mon, 16 Jan 2017, Juergen Gross wrote: > On 13/01/17 19:44, Boris Ostrovsky wrote: > > On 01/13/2017 01:26 PM, Stefano Stabellini wrote: > >> On Fri, 13 Jan 2017, Boris Ostrovsky wrote: > >>> On 01/12/2017 04:39 PM, Stefano Stabellini wrote: > >>>> The following commit: > >>>> > >>>> commit 72a9b186292d98494f222226cfd24a1621796209 > >>>> Author: KarimAllah Ahmed <karahmed@amazon.de> > >>>> Date: Fri Aug 26 23:55:36 2016 +0200 > >>>> > >>>> xen: Remove event channel notification through Xen PCI platform device > >>>> > >>>> broke Linux when booting as Dom0 on Xen in a nested Xen environment (Xen > >>>> installed inside a Xen VM). In this scenario, Linux is a PV guest, but > >>>> at the same time it uses the platform-pci driver to receive > >>>> notifications from L0 Xen. vector callbacks are not available because L1 > >>>> Xen doesn't allow them. > >>>> > >>>> Partially revert the offending commit, by restoring IRQ based > >>>> notifications for PV guests only. I restored only the code which is > >>>> strictly needed and replaced the xen_have_vector_callback checks within > >>>> it with xen_pv_domain() checks. > >>>> > >>>> Signed-off-by: Stefano Stabellini <sstabellini@kernel.org> > >>> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> > >> Applied to xentip/for-linus-4.10, improving the commit message as > >> suggested before. > >> > >> Do you plan on sending out another pull request for 4.10? > > > > Juergen? > > Originally I didn't plan to, but I'm not opposed to sending out one. I think that would be best. Thanks, Stefano
diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c index b59c9455..549c618 100644 --- a/drivers/xen/platform-pci.c +++ b/drivers/xen/platform-pci.c @@ -42,6 +42,7 @@ static unsigned long platform_mmio; static unsigned long platform_mmio_alloc; static unsigned long platform_mmiolen; +static uint64_t callback_via; static unsigned long alloc_xen_mmio(unsigned long len) { @@ -54,6 +55,51 @@ static unsigned long alloc_xen_mmio(unsigned long len) return addr; } +static uint64_t get_callback_via(struct pci_dev *pdev) +{ + u8 pin; + int irq; + + irq = pdev->irq; + if (irq < 16) + return irq; /* ISA IRQ */ + + pin = pdev->pin; + + /* We don't know the GSI. Specify the PCI INTx line instead. */ + return ((uint64_t)0x01 << HVM_CALLBACK_VIA_TYPE_SHIFT) | /* PCI INTx identifier */ + ((uint64_t)pci_domain_nr(pdev->bus) << 32) | + ((uint64_t)pdev->bus->number << 16) | + ((uint64_t)(pdev->devfn & 0xff) << 8) | + ((uint64_t)(pin - 1) & 3); +} + +static irqreturn_t do_hvm_evtchn_intr(int irq, void *dev_id) +{ + xen_hvm_evtchn_do_upcall(); + return IRQ_HANDLED; +} + +static int xen_allocate_irq(struct pci_dev *pdev) +{ + return request_irq(pdev->irq, do_hvm_evtchn_intr, + IRQF_NOBALANCING | IRQF_TRIGGER_RISING, + "xen-platform-pci", pdev); +} + +static int platform_pci_resume(struct pci_dev *pdev) +{ + int err; + if (!xen_pv_domain()) + return 0; + err = xen_set_callback_via(callback_via); + if (err) { + dev_err(&pdev->dev, "platform_pci_resume failure!\n"); + return err; + } + return 0; +} + static int platform_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) { @@ -92,6 +138,28 @@ static int platform_pci_probe(struct pci_dev *pdev, platform_mmio = mmio_addr; platform_mmiolen = mmio_len; + /* + * Xen HVM guests always use the vector callback mechanism. + * L1 Dom0 in a nested Xen environment is a PV guest inside in an + * HVM environment. It needs the platform-pci driver to get + * notifications from L0 Xen, but it cannot use the vector callback + * as it is not exported by L1 Xen. + */ + if (xen_pv_domain()) { + ret = xen_allocate_irq(pdev); + if (ret) { + dev_warn(&pdev->dev, "request_irq failed err=%d\n", ret); + goto out; + } + callback_via = get_callback_via(pdev); + ret = xen_set_callback_via(callback_via); + if (ret) { + dev_warn(&pdev->dev, "Unable to set the evtchn callback " + "err=%d\n", ret); + goto out; + } + } + max_nr_gframes = gnttab_max_grant_frames(); grant_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes); ret = gnttab_setup_auto_xlat_frames(grant_frames); @@ -123,6 +191,9 @@ static int platform_pci_probe(struct pci_dev *pdev, .name = DRV_NAME, .probe = platform_pci_probe, .id_table = platform_pci_tbl, +#ifdef CONFIG_PM + .resume_early = platform_pci_resume, +#endif }; static int __init platform_pci_init(void)
The following commit: commit 72a9b186292d98494f222226cfd24a1621796209 Author: KarimAllah Ahmed <karahmed@amazon.de> Date: Fri Aug 26 23:55:36 2016 +0200 xen: Remove event channel notification through Xen PCI platform device broke Linux when booting as Dom0 on Xen in a nested Xen environment (Xen installed inside a Xen VM). In this scenario, Linux is a PV guest, but at the same time it uses the platform-pci driver to receive notifications from L0 Xen. vector callbacks are not available because L1 Xen doesn't allow them. Partially revert the offending commit, by restoring IRQ based notifications for PV guests only. I restored only the code which is strictly needed and replaced the xen_have_vector_callback checks within it with xen_pv_domain() checks. Signed-off-by: Stefano Stabellini <sstabellini@kernel.org> --- Changes in v2: - in code comment - use HVM_CALLBACK_VIA_TYPE_SHIFT ---