From patchwork Fri Jul 19 19:14:17 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yinghai Lu X-Patchwork-Id: 2830671 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id AF4E2C0319 for ; Fri, 19 Jul 2013 19:14:40 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id A618220254 for ; Fri, 19 Jul 2013 19:14:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A1B7920265 for ; Fri, 19 Jul 2013 19:14:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752181Ab3GSTOe (ORCPT ); Fri, 19 Jul 2013 15:14:34 -0400 Received: from userp1040.oracle.com ([156.151.31.81]:26775 "EHLO userp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751926Ab3GSTOd (ORCPT ); Fri, 19 Jul 2013 15:14:33 -0400 Received: from ucsinet22.oracle.com (ucsinet22.oracle.com [156.151.31.94]) by userp1040.oracle.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.1) with ESMTP id r6JJEMbD006321 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 19 Jul 2013 19:14:23 GMT Received: from aserz7022.oracle.com (aserz7022.oracle.com [141.146.126.231]) by ucsinet22.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id r6JJELmt000185 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 19 Jul 2013 19:14:22 GMT Received: from abhmt119.oracle.com (abhmt119.oracle.com [141.146.116.71]) by aserz7022.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id r6JJELT1026678; Fri, 19 Jul 2013 19:14:21 GMT Received: from linux-siqj.site (/10.159.181.233) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 19 Jul 2013 12:14:21 -0700 From: Yinghai Lu To: Bjorn Helgaas Cc: linux-pci@vger.kernel.org, Yinghai Lu , Yijing Wang , Jiang Liu Subject: [PATCH] PCI: Separate stop and remove devices in pciehp Date: Fri, 19 Jul 2013 12:14:17 -0700 Message-Id: <1374261258-23036-2-git-send-email-yinghai@kernel.org> X-Mailer: git-send-email 1.8.1.4 In-Reply-To: <1374261258-23036-1-git-send-email-yinghai@kernel.org> References: <1374261258-23036-1-git-send-email-yinghai@kernel.org> X-Source-IP: ucsinet22.oracle.com [156.151.31.94] Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-7.3 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP After commit dc087f2f6a2925e81831f3016b9cbb6e470e7423 (PCI: Simplify IOV implementation and fix reference count races) VF need to be removed via virtfn_remove to make sure ref to PF is put back. So we can not call stop_and_remove for VF before PF, that will make VF get removed directly before PF's driver try to use virtfn_remove to do it. Solution is separating stop and remove in two iterations. In first iteration, we stop VF driver at first with iterating devices reversely, and later during stop PF driver, disable_sriov will use virtfn_remove to remove VFs. Also some driver (like mlx4_core) need VF's driver get stopped before PF. To make it simple, separate VGA checking out and do that at first, if there is VGA in the chain, do even try to stop or remove any device under that bus. Need this one for v3.11. Signed-off-by: Yinghai Lu Cc: Yijing Wang Cc: Jiang Liu --- drivers/pci/hotplug/pciehp_pci.c | 46 ++++++++++++++++++++++++++------------- drivers/pci/remove.c | 6 +++-- include/linux/pci.h | 2 + 3 files changed, 37 insertions(+), 17 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Index: linux-2.6/drivers/pci/hotplug/pciehp_pci.c =================================================================== --- linux-2.6.orig/drivers/pci/hotplug/pciehp_pci.c +++ linux-2.6/drivers/pci/hotplug/pciehp_pci.c @@ -92,26 +92,37 @@ int pciehp_unconfigure_device(struct slo if (ret) presence = 0; + /* check if VGA is around */ + if (presence) { + list_for_each_entry(dev, &parent->devices, bus_list) { + if (dev->hdr_type == PCI_HEADER_TYPE_BRIDGE) { + pci_read_config_byte(dev, PCI_BRIDGE_CONTROL, + &bctl); + if (bctl & PCI_BRIDGE_CTL_VGA) { + ctrl_err(ctrl, + "Cannot remove display device %s\n", + pci_name(dev)); + return -EINVAL; + } + } + } + } + /* - * Need to iterate device reversely, as during + * Now VF need to be removed via virtfn_remove to make + * sure ref to PF is put back. Some driver (mlx4_core) need + * VF's driver get stopped before PF. + * So we need to stop VF driver at first, that means + * loop reversely, and later during stop PF driver, + * disable_sriov will use virtfn_remove to remove VFs. + * Also we can not loop without reverse, as during * stop PF driver, VF will be removed, the list_for_each * could point to removed VF with temp. */ list_for_each_entry_safe_reverse(dev, temp, &parent->devices, - bus_list) { - pci_dev_get(dev); - if (dev->hdr_type == PCI_HEADER_TYPE_BRIDGE && presence) { - pci_read_config_byte(dev, PCI_BRIDGE_CONTROL, &bctl); - if (bctl & PCI_BRIDGE_CTL_VGA) { - ctrl_err(ctrl, - "Cannot remove display device %s\n", - pci_name(dev)); - pci_dev_put(dev); - rc = -EINVAL; - break; - } - } - pci_stop_and_remove_bus_device(dev); + bus_list) { + pci_stop_bus_device(dev); + /* * Ensure that no new Requests will be generated from * the device. @@ -122,6 +133,11 @@ int pciehp_unconfigure_device(struct slo command |= PCI_COMMAND_INTX_DISABLE; pci_write_config_word(dev, PCI_COMMAND, command); } + } + + list_for_each_entry_safe(dev, temp, &parent->devices, bus_list) { + pci_dev_get(dev); + pci_remove_bus_device(dev); pci_dev_put(dev); } Index: linux-2.6/drivers/pci/remove.c =================================================================== --- linux-2.6.orig/drivers/pci/remove.c +++ linux-2.6/drivers/pci/remove.c @@ -56,7 +56,7 @@ void pci_remove_bus(struct pci_bus *bus) } EXPORT_SYMBOL(pci_remove_bus); -static void pci_stop_bus_device(struct pci_dev *dev) +void pci_stop_bus_device(struct pci_dev *dev) { struct pci_bus *bus = dev->subordinate; struct pci_dev *child, *tmp; @@ -75,8 +75,9 @@ static void pci_stop_bus_device(struct p pci_stop_dev(dev); } +EXPORT_SYMBOL(pci_stop_bus_device); -static void pci_remove_bus_device(struct pci_dev *dev) +void pci_remove_bus_device(struct pci_dev *dev) { struct pci_bus *bus = dev->subordinate; struct pci_dev *child, *tmp; @@ -92,6 +93,7 @@ static void pci_remove_bus_device(struct pci_destroy_dev(dev); } +EXPORT_SYMBOL(pci_remove_bus_device); /** * pci_stop_and_remove_bus_device - remove a PCI device and any children Index: linux-2.6/include/linux/pci.h =================================================================== --- linux-2.6.orig/include/linux/pci.h +++ linux-2.6/include/linux/pci.h @@ -754,6 +754,8 @@ u8 pci_common_swizzle(struct pci_dev *de struct pci_dev *pci_dev_get(struct pci_dev *dev); void pci_dev_put(struct pci_dev *dev); void pci_remove_bus(struct pci_bus *b); +void pci_stop_bus_device(struct pci_dev *dev); +void pci_remove_bus_device(struct pci_dev *dev); void pci_stop_and_remove_bus_device(struct pci_dev *dev); void pci_stop_root_bus(struct pci_bus *bus); void pci_remove_root_bus(struct pci_bus *bus);