From patchwork Tue Dec 12 08:29:44 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiongfeng Wang X-Patchwork-Id: 10106495 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D690F602B3 for ; Tue, 12 Dec 2017 08:38:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C3D6D298DB for ; Tue, 12 Dec 2017 08:38:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B89ED29B1C; Tue, 12 Dec 2017 08:38:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3BD1F298DB for ; Tue, 12 Dec 2017 08:38:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:To :From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=lR7OrlgAxO6ZsBGHGYM4cHjKMKlEm/N//BXmtO5mHYs=; b=PXfnhDonm6FOOc RyOrLasLxGFBxVEc4Ruaf5BH/ionRqf3+rKmuLhUI1tkHGgR59DARsbybLsfwbvZKH0H7LSmTHp7t 5ecntiFg9dCgfJkXGi6VX7I328N/4s5qg0kuaRC36h/vUig5F5mPZey/YAZ7g11E2bWzlIhXcdN/q tm0KdDluMV/kt5h5+DyPrlyfOD2/Ct796JFwhpG5f4U8uZJJXnUvp1df6rJvr5Elwcj0S7Ie6qe7A nfQqQ5fiNMszy6LzvjyDqOH/1JMgJCGs02AHmcu8JkT622YjkQmIkVfcgKWiU6hjFYnANsUfMs5h4 8g8hVkIzqqe+bx0dMs3g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1eOg5U-0005xL-Ix; Tue, 12 Dec 2017 08:38:52 +0000 Received: from szxga05-in.huawei.com ([45.249.212.191]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1eOg5P-0005nd-Ux for linux-arm-kernel@lists.infradead.org; Tue, 12 Dec 2017 08:38:50 +0000 Received: from 172.30.72.59 (EHLO DGGEMS414-HUB.china.huawei.com) ([172.30.72.59]) by dggrg05-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id DMB75539; Tue, 12 Dec 2017 16:38:14 +0800 (CST) Received: from linux-ibm.site (10.175.102.37) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.361.1; Tue, 12 Dec 2017 16:38:06 +0800 From: Xiongfeng Wang To: , Subject: [RFC PATCH][resend] pciehp: fix a race between pciehp and removing operations by sysfs Date: Tue, 12 Dec 2017 16:29:44 +0800 Message-ID: <1513067384-10914-1-git-send-email-wangxiongfeng2@huawei.com> X-Mailer: git-send-email 1.7.12.4 MIME-Version: 1.0 X-Originating-IP: [10.175.102.37] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020202.5A2F9576.00AF, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: defc94be521d4adc054f05ddbbc365f5 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20171212_003848_434837_6EC443EC X-CRM114-Status: GOOD ( 10.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-pci@vger.kernel.org, liudongdong3@huawei.com, linux-arm-kernel@lists.infradead.org, arnd@arndb.de, linux-kernel@vger.kernel.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP When the Attention button on a PCIE slot is pressed, 5 seconds later, pciehp_power_thread() will be scheduled on slot->wq. This function will get a global mutex lock 'pci_rescan_remove_lock' in pciehp_unconfigure_device(). At the same time, we remove the pcie port by sysfs, which results in pci_stop_and_remove_bus_device_locked() called. This function will get the global mutex lock 'pci_rescan_remove_lock', and then release the struct 'ctrl', which will wait until the work_struct on slot->wq is finished. If pci_stop_and_remove_bus_device_locked() got the mutex lock, and before it drains workqueue slot->wq, pciehp_power_thread() is scheduled on slot->wq and tries to get the mutex lock. Then pci_stop_and_remove_bus_device_locked() tries to drain workqueue slot->wq and wait until work struct 'pciehp_power_thread()' is finished. Then a hung_task happens. This patch solve this problem by schedule 'pciehp_power_thread()' on a system workqueue instead of slot->wq. The Call Trace we got is as following. INFO: task kworker/0:2:4413 blocked for more than 120 seconds. Tainted: P W O 4.12.0-rc1 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kworker/0:2 D 0 4413 2 0x00000000 Workqueue: pciehp-0 pciehp_power_thread Call trace: [] __switch_to+0x94/0xa8 [] __schedule+0x1b0/0x708 [] schedule+0x40/0xa4 [] schedule_preempt_disabled+0x28/0x40 [] __mutex_lock.isra.8+0x148/0x50c [] __mutex_lock_slowpath+0x24/0x30 [] mutex_lock+0x48/0x54 [] pci_lock_rescan_remove+0x20/0x28 [] pciehp_unconfigure_device+0x54/0x1cc [] pciehp_disable_slot+0x4c/0xbc [] pciehp_power_thread+0xa0/0xb8 [] process_one_work+0x13c/0x3f8 [] worker_thread+0x60/0x3e4 [] kthread+0x10c/0x138 [] ret_from_fork+0x10/0x50 INFO: task bash:31732 blocked for more than 120 seconds. Tainted: P W O 4.12.0-rc1 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. bash D 0 31732 1 0x00000009 Call trace: [] __switch_to+0x94/0xa8 [] __schedule+0x1b0/0x708 [] schedule+0x40/0xa4 [] schedule_timeout+0x1a0/0x340 [] wait_for_common+0x108/0x1bc [] wait_for_completion+0x28/0x34 [] flush_workqueue+0x130/0x488 [] drain_workqueue+0xc4/0x164 [] destroy_workqueue+0x28/0x1f4 [] pciehp_release_ctrl+0x34/0xe0 [] pciehp_remove+0x30/0x3c [] pcie_port_remove_service+0x3c/0x54 [] device_release_driver_internal+0x150/0x1d0 [] device_release_driver+0x28/0x34 [] bus_remove_device+0xe0/0x11c [] device_del+0x200/0x304 [] device_unregister+0x20/0x38 [] remove_iter+0x44/0x54 [] device_for_each_child+0x4c/0x90 [] pcie_port_device_remove+0x2c/0x48 [] pcie_portdrv_remove+0x60/0x6c [] pci_device_remove+0x48/0x110 [] device_release_driver_internal+0x150/0x1d0 [] device_release_driver+0x28/0x34 [] pci_stop_bus_device+0x9c/0xac [] pci_stop_and_remove_bus_device_locked+0x24/0x3c [] remove_store+0x74/0x80 [] dev_attr_store+0x44/0x5c [] sysfs_kf_write+0x5c/0x74 [] kernfs_fop_write+0xcc/0x1dc [] __vfs_write+0x48/0x13c [] vfs_write+0xa8/0x198 [] SyS_write+0x54/0xb0 [] el0_svc_naked+0x24/0x28 Signed-off-by: Xiongfeng Wang --- drivers/pci/hotplug/pciehp_ctrl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/pci/hotplug/pciehp_ctrl.c b/drivers/pci/hotplug/pciehp_ctrl.c index 83f3d4a..9d39d85 100644 --- a/drivers/pci/hotplug/pciehp_ctrl.c +++ b/drivers/pci/hotplug/pciehp_ctrl.c @@ -221,7 +221,7 @@ static void pciehp_queue_power_work(struct slot *p_slot, int req) info->p_slot = p_slot; INIT_WORK(&info->work, pciehp_power_thread); info->req = req; - queue_work(p_slot->wq, &info->work); + schedule_work(&info->work); } void pciehp_queue_pushbutton_work(struct work_struct *work)