From patchwork Sun Jan 13 23:27:37 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 1970851 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id 83E3B3FCDE for ; Sun, 13 Jan 2013 23:33:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754978Ab3AMXdh (ORCPT ); Sun, 13 Jan 2013 18:33:37 -0500 Received: from mail-wg0-f74.google.com ([74.125.82.74]:55089 "EHLO mail-wg0-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751663Ab3AMXdg (ORCPT ); Sun, 13 Jan 2013 18:33:36 -0500 Received: by mail-wg0-f74.google.com with SMTP id dt14so213970wgb.5 for ; Sun, 13 Jan 2013 15:33:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:subject:to:from:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-type :content-transfer-encoding; bh=CPW0R/tC4bxTKSdreeujShlikiIPcL7Evv81j5z5zdI=; b=Me/GRcdrHr4jJd5Ngm13aowMQpqTh517sPkAm98OrxXZBgMm8QaRngp7Usk1sUtCJJ iWmboHtymPGCFdfe7VhzGP3H5IGFgwTcbopV+tbund8QcpODRVBOdxi7Ho/GNzDagc9X Bca+I7ex5nVfstDD6mX7E+pzlZ4TxMgq6rH7b2m6hF5rlbYtRs0QGEgZfyXzFm/Sux67 qeBUu7wVIlaIF0vWpByRcycpJQvXnyu6Ra9PoQAqwo1lnh1zbWTkChpNLEph6TZXs6U5 2Pypmpl0ZmBiFi6qFCtnVSxsalJVX/Y3RW5QRScNDah2acFdfZkVbjVUkr0HUfsXEdwO e4WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:subject:to:from:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-type :content-transfer-encoding:x-gm-message-state; bh=CPW0R/tC4bxTKSdreeujShlikiIPcL7Evv81j5z5zdI=; b=FX/fKk/hQYruKicpQU7ASQ5BQfLHKSFd2WjATKMQUwIuzxWYHHaY56maFb3cLlS4J8 QeBwRqCuGGbZJSe3m50JscU32b80Cu/3QouaRuD8GyM1i1reRAj4Jyo0LWfNlgXPXgoX cUyr4LykmsqcooP/cAAU280x91IY/xaKjrtEzVvYuHqiIkgVsW97eN53vXokuFXmHwmS yZ4HzoVO/QpQSNbZUGZwTfoexRTftZATNcpo6/ytWpYWnVl3y9Gdc7CH4/VVEOdnGLwU 4XDM+QObUauUhdCyVxQERlkTDNYWclmFiC4rhNFUz5X625rKIe7tH7ox2rx+ugMLm0YJ GywA== X-Received: by 10.14.180.2 with SMTP id i2mr19613880eem.1.1358119659341; Sun, 13 Jan 2013 15:27:39 -0800 (PST) Received: from hpza10.eem.corp.google.com ([74.125.121.33]) by gmr-mx.google.com with ESMTPS id z44si4441785een.0.2013.01.13.15.27.39 (version=TLSv1 cipher=AES128-SHA bits=128/128); Sun, 13 Jan 2013 15:27:39 -0800 (PST) Received: from bhelgaas.mtv.corp.google.com (bhelgaas.mtv.corp.google.com [172.17.131.112]) by hpza10.eem.corp.google.com (Postfix) with ESMTP id 92799200057; Sun, 13 Jan 2013 15:27:38 -0800 (PST) Received: from bhelgaas.mtv.corp.google.com (unknown [IPv6:::1]) by bhelgaas.mtv.corp.google.com (Postfix) with ESMTP id EAD3618024D; Sun, 13 Jan 2013 15:27:37 -0800 (PST) Subject: [PATCH v7 1/4] PCI: pciehp: Use per-slot workqueues to avoid deadlock To: Yijing Wang , linux-pci@vger.kernel.org From: Bjorn Helgaas Cc: linux-kernel@vger.kernel.org, stable@vger.kernel.org, Daniel J Blueman , Kenji Kaneshige , Tejun Heo , Yinghai Lu , Jiang Liu Date: Sun, 13 Jan 2013 16:27:37 -0700 Message-ID: <20130113232737.7225.97136.stgit@bhelgaas.mtv.corp.google.com> In-Reply-To: <20130113232606.7225.94802.stgit@bhelgaas.mtv.corp.google.com> References: <20130113232606.7225.94802.stgit@bhelgaas.mtv.corp.google.com> User-Agent: StGit/0.15 MIME-Version: 1.0 X-Gm-Message-State: ALoCoQnojlF4HS4bDAgHAtoW0lFcyey1FOwzsFS/mm8EoH3TYUSL8QwgcsSdIprppsA2Q3DRCLyVstVuyD4ypA2KvE5ueb9a1HJfa2WwIrEa52blilwhAJBl8+nobNxvWpizxAjJBl1OdJ3vkk7ZEQdASQfIGGj+TTl2qLx6JoijBgkT+xeJz5kUowFavoYM7SzUxoQ8rbkIIlblupoA8q+PKTK03VSjow== Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Yijing Wang When we have a hotplug-capable PCIe port with a second hotplug-capable PCIe port below it, removing the device below the upstream port causes a deadlock. The deadlock happens because we use the pciehp_wq workqueue to run pciehp_power_thread(), which uses pciehp_disable_slot() to remove devices below the upstream port. When we remove the downstream PCIe port, we call pciehp_remove(), the pciehp driver's .remove() method. That calls flush_workqueue(pciehp_wq), which deadlocks because the pciehp_power_thread() work item is still running. This patch avoids the deadlock by creating a workqueue for every PCIe port and removing the single shared workqueue. Here's the call path that leads to the deadlock: pciehp_queue_pushbutton_work queue_work(pciehp_wq) # queue pciehp_power_thread ... pciehp_power_thread pciehp_disable_slot remove_board pciehp_unconfigure_device pci_stop_and_remove_bus_device ... pciehp_remove # pciehp driver .remove method pciehp_release_ctrl pcie_cleanup_slot flush_workqueue(pciehp_wq) This is fairly urgent because it can be caused by simply unplugging a Thunderbolt adapter, as reported by Daniel below. [bhelgaas: changelog] Reference: http://lkml.kernel.org/r/CAMVG2ssiRgcTD1bej2tkUUfsWmpL5eNtPcNif9va2-Gzb2u8nQ@mail.gmail.com Reported-and-tested-by: Daniel J Blueman Reviewed-by: Kenji Kaneshige Signed-off-by: Yijing Wang Signed-off-by: Bjorn Helgaas CC: stable@vger.kernel.org --- drivers/pci/hotplug/pciehp.h | 2 +- drivers/pci/hotplug/pciehp_core.c | 11 ++--------- drivers/pci/hotplug/pciehp_ctrl.c | 8 ++++---- drivers/pci/hotplug/pciehp_hpc.c | 11 ++++++++++- 4 files changed, 17 insertions(+), 15 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/pci/hotplug/pciehp.h b/drivers/pci/hotplug/pciehp.h index 26ffd3e..2c113de 100644 --- a/drivers/pci/hotplug/pciehp.h +++ b/drivers/pci/hotplug/pciehp.h @@ -44,7 +44,6 @@ extern bool pciehp_poll_mode; extern int pciehp_poll_time; extern bool pciehp_debug; extern bool pciehp_force; -extern struct workqueue_struct *pciehp_wq; #define dbg(format, arg...) \ do { \ @@ -78,6 +77,7 @@ struct slot { struct hotplug_slot *hotplug_slot; struct delayed_work work; /* work for button event */ struct mutex lock; + struct workqueue_struct *wq; }; struct event_info { diff --git a/drivers/pci/hotplug/pciehp_core.c b/drivers/pci/hotplug/pciehp_core.c index 916bf4f..939bd1d 100644 --- a/drivers/pci/hotplug/pciehp_core.c +++ b/drivers/pci/hotplug/pciehp_core.c @@ -42,7 +42,6 @@ bool pciehp_debug; bool pciehp_poll_mode; int pciehp_poll_time; bool pciehp_force; -struct workqueue_struct *pciehp_wq; #define DRIVER_VERSION "0.4" #define DRIVER_AUTHOR "Dan Zink , Greg Kroah-Hartman , Dely Sy " @@ -340,18 +339,13 @@ static int __init pcied_init(void) { int retval = 0; - pciehp_wq = alloc_workqueue("pciehp", 0, 0); - if (!pciehp_wq) - return -ENOMEM; - pciehp_firmware_init(); retval = pcie_port_service_register(&hpdriver_portdrv); dbg("pcie_port_service_register = %d\n", retval); info(DRIVER_DESC " version: " DRIVER_VERSION "\n"); - if (retval) { - destroy_workqueue(pciehp_wq); + if (retval) dbg("Failure to register service\n"); - } + return retval; } @@ -359,7 +353,6 @@ static void __exit pcied_cleanup(void) { dbg("unload_pciehpd()\n"); pcie_port_service_unregister(&hpdriver_portdrv); - destroy_workqueue(pciehp_wq); info(DRIVER_DESC " version: " DRIVER_VERSION " unloaded\n"); } diff --git a/drivers/pci/hotplug/pciehp_ctrl.c b/drivers/pci/hotplug/pciehp_ctrl.c index 27f4429..38f0186 100644 --- a/drivers/pci/hotplug/pciehp_ctrl.c +++ b/drivers/pci/hotplug/pciehp_ctrl.c @@ -49,7 +49,7 @@ static int queue_interrupt_event(struct slot *p_slot, u32 event_type) info->p_slot = p_slot; INIT_WORK(&info->work, interrupt_event_handler); - queue_work(pciehp_wq, &info->work); + queue_work(p_slot->wq, &info->work); return 0; } @@ -344,7 +344,7 @@ void pciehp_queue_pushbutton_work(struct work_struct *work) kfree(info); goto out; } - queue_work(pciehp_wq, &info->work); + queue_work(p_slot->wq, &info->work); out: mutex_unlock(&p_slot->lock); } @@ -377,7 +377,7 @@ static void handle_button_press_event(struct slot *p_slot) if (ATTN_LED(ctrl)) pciehp_set_attention_status(p_slot, 0); - queue_delayed_work(pciehp_wq, &p_slot->work, 5*HZ); + queue_delayed_work(p_slot->wq, &p_slot->work, 5*HZ); break; case BLINKINGOFF_STATE: case BLINKINGON_STATE: @@ -439,7 +439,7 @@ static void handle_surprise_event(struct slot *p_slot) else p_slot->state = POWERON_STATE; - queue_work(pciehp_wq, &info->work); + queue_work(p_slot->wq, &info->work); } static void interrupt_event_handler(struct work_struct *work) diff --git a/drivers/pci/hotplug/pciehp_hpc.c b/drivers/pci/hotplug/pciehp_hpc.c index 13b2eaf..5127f3f 100644 --- a/drivers/pci/hotplug/pciehp_hpc.c +++ b/drivers/pci/hotplug/pciehp_hpc.c @@ -773,23 +773,32 @@ static void pcie_shutdown_notification(struct controller *ctrl) static int pcie_init_slot(struct controller *ctrl) { struct slot *slot; + char name[32]; slot = kzalloc(sizeof(*slot), GFP_KERNEL); if (!slot) return -ENOMEM; + snprintf(name, sizeof(name), "pciehp-%u", PSN(ctrl)); + slot->wq = alloc_workqueue(name, 0, 0); + if (!slot->wq) + goto abort; + slot->ctrl = ctrl; mutex_init(&slot->lock); INIT_DELAYED_WORK(&slot->work, pciehp_queue_pushbutton_work); ctrl->slot = slot; return 0; +abort: + kfree(slot); + return -ENOMEM; } static void pcie_cleanup_slot(struct controller *ctrl) { struct slot *slot = ctrl->slot; cancel_delayed_work(&slot->work); - flush_workqueue(pciehp_wq); + destroy_workqueue(slot->wq); kfree(slot); }