From patchwork Sun Sep 6 15:32:35 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 7132031 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 1707BBEEC1 for ; Sun, 6 Sep 2015 15:32:44 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 1E9D320715 for ; Sun, 6 Sep 2015 15:32:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D0CB420685 for ; Sun, 6 Sep 2015 15:32:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752438AbbIFPck (ORCPT ); Sun, 6 Sep 2015 11:32:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:34270 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750888AbbIFPck (ORCPT ); Sun, 6 Sep 2015 11:32:40 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id BF3F58CF55; Sun, 6 Sep 2015 15:32:39 +0000 (UTC) Received: from redhat.com (ovpn-116-28.ams2.redhat.com [10.36.116.28]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with SMTP id t86FWaEl007045; Sun, 6 Sep 2015 11:32:36 -0400 Date: Sun, 6 Sep 2015 18:32:35 +0300 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Fam Zheng , Bjorn Helgaas , Yinghai Lu , Ulrich Obergfell , Rusty Russell , "Eric W. Biederman" , linux-pci@vger.kernel.org, virtualization@lists.linux-foundation.org Subject: [PATCH v7] pci: quirk to skip msi disable on shutdown Message-ID: <1441553385-3810-1-git-send-email-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline X-Mutt-Fcc: =sent X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-7.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On some hypervisors, virtio devices tend to generate spurious interrupts when switching between MSI and non-MSI mode. Normally, either MSI or non-MSI is used and all is well, but during shutdown, linux disables MSI which then causes an "irq %d: nobody cared" message, with irq being subsequently disabled. Since bus mastering is already disabled at this point, disabling MSI isn't actually useful for spec compliant devices: MSI interrupts are in-bus memory writes so disabling Bus Master (which is already done) disables these as well: after some research, it appears to be there for the benefit of devices that ignore the bus master bit. As it's not clear how common this kind of bug is, this patch simply adds a quirk, to be set by devices that wish to skip disabling msi on shutdown, relying on bus master instead. We set this quirk in virtio core. Reported-by: Fam Zheng Cc: Bjorn Helgaas Cc: Yinghai Lu Cc: Ulrich Obergfell Cc: Rusty Russell Cc: "Eric W. Biederman" Signed-off-by: Michael S. Tsirkin --- changes from v6: limit changes to virtio only changes from v5: rebased on top of pci/msi fixed commit log, including comments by Bjorn and adding explanation to address comments/questions by Eric dropped stable Cc, this patch does not seem to qualify for stable changes from v4: Yijing Wang noted that early fixups rely on pci_msi_off. Split out the functionality and move off the required part to run early during pci_device_setup. Changes from v3: fix a copy-and-paste error in pci: drop some duplicate code other patches are unchanged drop Cc stable for now Changes from v2: move code from probe to device enumeration add patches to unexport pci_msi_off include/linux/pci.h | 2 ++ drivers/pci/pci-driver.c | 6 ++++-- drivers/virtio/virtio_pci_common.c | 4 ++++ 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/include/linux/pci.h b/include/linux/pci.h index 860c751..80f3494 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -180,6 +180,8 @@ enum pci_dev_flags { PCI_DEV_FLAGS_NO_BUS_RESET = (__force pci_dev_flags_t) (1 << 6), /* Do not use PM reset even if device advertises NoSoftRst- */ PCI_DEV_FLAGS_NO_PM_RESET = (__force pci_dev_flags_t) (1 << 7), + /* Do not disable MSI on shutdown, disable bus master instead */ + PCI_DEV_FLAGS_NO_MSI_SHUTDOWN = (__force pci_dev_flags_t) (1 << 8), }; enum pci_irq_reroute_variant { diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index 3cb2210..59d9e40 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -450,8 +450,10 @@ static void pci_device_shutdown(struct device *dev) if (drv && drv->shutdown) drv->shutdown(pci_dev); - pci_msi_shutdown(pci_dev); - pci_msix_shutdown(pci_dev); + if (!(pci_dev->dev_flags & PCI_DEV_FLAGS_NO_MSI_SHUTDOWN)) { + pci_msi_shutdown(pci_dev); + pci_msix_shutdown(pci_dev); + } #ifdef CONFIG_KEXEC /* diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c index 78f804a..26f46c3 100644 --- a/drivers/virtio/virtio_pci_common.c +++ b/drivers/virtio/virtio_pci_common.c @@ -528,6 +528,8 @@ static int virtio_pci_probe(struct pci_dev *pci_dev, if (rc) goto err_register; + pci_dev->dev_flags |= PCI_DEV_FLAGS_NO_MSI_SHUTDOWN; + return 0; err_register: @@ -546,6 +548,8 @@ static void virtio_pci_remove(struct pci_dev *pci_dev) { struct virtio_pci_device *vp_dev = pci_get_drvdata(pci_dev); + pci_dev->dev_flags &= ~PCI_DEV_FLAGS_NO_MSI_SHUTDOWN; + unregister_virtio_device(&vp_dev->vdev); if (vp_dev->ioaddr)