From patchwork Thu Jan 12 21:39:52 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Stabellini X-Patchwork-Id: 9514251 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id DD06660476 for ; Thu, 12 Jan 2017 21:42:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DF2C928703 for ; Thu, 12 Jan 2017 21:42:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D11BF28702; Thu, 12 Jan 2017 21:42:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6382E28702 for ; Thu, 12 Jan 2017 21:42:34 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cRn6F-0003yz-SB; Thu, 12 Jan 2017 21:39:59 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cRn6E-0003yh-8w for xen-devel@lists.xenproject.org; Thu, 12 Jan 2017 21:39:58 +0000 Received: from [85.158.139.211] by server-11.bemta-5.messagelabs.com id 70/D3-14064-DA7F7785; Thu, 12 Jan 2017 21:39:57 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrIIsWRWlGSWpSXmKPExsVybKJsh+6a7+U RBseWmll83zKZyYHR4/CHKywBjFGsmXlJ+RUJrBkzr01hL3gmW7F+4wbmBsaHEl2MXBxCAlMZ JR7eeM4I4cxmkni+cDV7FyMnB4uAtsSGO3+ZQGw2AUOJv082sXUxcnBIANlLPnOAhEUE5CSa3 r5jAellFpjPKPF35QoWkISwQJ7Et/P3wXp5BbwlNm2dBhYXFdCVOPTvDxtEXFDi5MwnYHFmAS 2J5dO3gdkSAhkS83rmsELYXhKLblyCstUkrp7bxDyBkX8WkvZZSNoXMDKtYtQoTi0qSy3SNTL USyrKTM8oyU3MzNE1NDDVy00tLk5MT81JTCrWS87P3cQIDLh6BgbGHYx3J/sdYpTkYFIS5ZXZ Uh4hxJeUn1KZkVicEV9UmpNafIhRhoNDSYI34xtQTrAoNT21Ii0zBxj6MGkJDh4lEd57IGne4 oLE3OLMdIjUKUZjjlM3Tr9k4uh7duUlkxBLXn5eqpQ4ryNIqQBIaUZpHtwgWExeYpSVEuZlZG BgEOIpSC3KzSxBlX/FKM7BqCTMuxxkCk9mXgncvldApzABnXLRBuyUkkSElFQDo63WnFxOFQ+ xWwkv9jw+ei/VWbj1w8nugIq8p3Vx64tb39qpmKzknXp1xdqZxeGFpx7kZh5TCyyIvjcxqYRp yk+rX/ut400Mom0Uk71nLd7juuvV05bLKTe2zkr5UDnP3P6S8/p51TpHJ7dt/c0l8P1FHk8IH 5P91HkTX2yvbLw4KySC580qdSWW4oxEQy3mouJEAArylcTEAgAA X-Env-Sender: sstabellini@kernel.org X-Msg-Ref: server-15.tower-206.messagelabs.com!1484257195!65672516!1 X-Originating-IP: [198.145.29.136] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 9.1.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 24308 invoked from network); 12 Jan 2017 21:39:56 -0000 Received: from mail.kernel.org (HELO mail.kernel.org) (198.145.29.136) by server-15.tower-206.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 12 Jan 2017 21:39:56 -0000 Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 179842020F; Thu, 12 Jan 2017 21:39:54 +0000 (UTC) Received: from [10.1.10.56] (96-82-76-110-static.hfc.comcastbusiness.net [96.82.76.110]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AADD920340; Thu, 12 Jan 2017 21:39:52 +0000 (UTC) Date: Thu, 12 Jan 2017 13:39:52 -0800 (PST) From: Stefano Stabellini X-X-Sender: sstabellini@sstabellini-ThinkPad-X260 To: boris.ostrovsky@oracle.com Message-ID: User-Agent: Alpine 2.10 (DEB 1266 2009-07-14) MIME-Version: 1.0 X-Virus-Scanned: ClamAV using ClamSMTP Cc: jgross@suse.com, sstabellini@kernel.org, karahmed@amazon.de, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Subject: [Xen-devel] [PATCH v2] partially revert "xen: Remove event channel notification through Xen PCI platform device" X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP The following commit: commit 72a9b186292d98494f222226cfd24a1621796209 Author: KarimAllah Ahmed Date: Fri Aug 26 23:55:36 2016 +0200 xen: Remove event channel notification through Xen PCI platform device broke Linux when booting as Dom0 on Xen in a nested Xen environment (Xen installed inside a Xen VM). In this scenario, Linux is a PV guest, but at the same time it uses the platform-pci driver to receive notifications from L0 Xen. vector callbacks are not available because L1 Xen doesn't allow them. Partially revert the offending commit, by restoring IRQ based notifications for PV guests only. I restored only the code which is strictly needed and replaced the xen_have_vector_callback checks within it with xen_pv_domain() checks. Signed-off-by: Stefano Stabellini Reviewed-by: Boris Ostrovsky --- Changes in v2: - in code comment - use HVM_CALLBACK_VIA_TYPE_SHIFT --- diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c index b59c9455..549c618 100644 --- a/drivers/xen/platform-pci.c +++ b/drivers/xen/platform-pci.c @@ -42,6 +42,7 @@ static unsigned long platform_mmio; static unsigned long platform_mmio_alloc; static unsigned long platform_mmiolen; +static uint64_t callback_via; static unsigned long alloc_xen_mmio(unsigned long len) { @@ -54,6 +55,51 @@ static unsigned long alloc_xen_mmio(unsigned long len) return addr; } +static uint64_t get_callback_via(struct pci_dev *pdev) +{ + u8 pin; + int irq; + + irq = pdev->irq; + if (irq < 16) + return irq; /* ISA IRQ */ + + pin = pdev->pin; + + /* We don't know the GSI. Specify the PCI INTx line instead. */ + return ((uint64_t)0x01 << HVM_CALLBACK_VIA_TYPE_SHIFT) | /* PCI INTx identifier */ + ((uint64_t)pci_domain_nr(pdev->bus) << 32) | + ((uint64_t)pdev->bus->number << 16) | + ((uint64_t)(pdev->devfn & 0xff) << 8) | + ((uint64_t)(pin - 1) & 3); +} + +static irqreturn_t do_hvm_evtchn_intr(int irq, void *dev_id) +{ + xen_hvm_evtchn_do_upcall(); + return IRQ_HANDLED; +} + +static int xen_allocate_irq(struct pci_dev *pdev) +{ + return request_irq(pdev->irq, do_hvm_evtchn_intr, + IRQF_NOBALANCING | IRQF_TRIGGER_RISING, + "xen-platform-pci", pdev); +} + +static int platform_pci_resume(struct pci_dev *pdev) +{ + int err; + if (!xen_pv_domain()) + return 0; + err = xen_set_callback_via(callback_via); + if (err) { + dev_err(&pdev->dev, "platform_pci_resume failure!\n"); + return err; + } + return 0; +} + static int platform_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) { @@ -92,6 +138,28 @@ static int platform_pci_probe(struct pci_dev *pdev, platform_mmio = mmio_addr; platform_mmiolen = mmio_len; + /* + * Xen HVM guests always use the vector callback mechanism. + * L1 Dom0 in a nested Xen environment is a PV guest inside in an + * HVM environment. It needs the platform-pci driver to get + * notifications from L0 Xen, but it cannot use the vector callback + * as it is not exported by L1 Xen. + */ + if (xen_pv_domain()) { + ret = xen_allocate_irq(pdev); + if (ret) { + dev_warn(&pdev->dev, "request_irq failed err=%d\n", ret); + goto out; + } + callback_via = get_callback_via(pdev); + ret = xen_set_callback_via(callback_via); + if (ret) { + dev_warn(&pdev->dev, "Unable to set the evtchn callback " + "err=%d\n", ret); + goto out; + } + } + max_nr_gframes = gnttab_max_grant_frames(); grant_frames = alloc_xen_mmio(PAGE_SIZE * max_nr_gframes); ret = gnttab_setup_auto_xlat_frames(grant_frames); @@ -123,6 +191,9 @@ static int platform_pci_probe(struct pci_dev *pdev, .name = DRV_NAME, .probe = platform_pci_probe, .id_table = platform_pci_tbl, +#ifdef CONFIG_PM + .resume_early = platform_pci_resume, +#endif }; static int __init platform_pci_init(void)