From patchwork Fri Aug 11 01:52:51 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shawn Lin X-Patchwork-Id: 9894811 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id DE98A602DA for ; Fri, 11 Aug 2017 01:53:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CFA7828BD7 for ; Fri, 11 Aug 2017 01:53:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C396328BF2; Fri, 11 Aug 2017 01:53:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1BC9628BD7 for ; Fri, 11 Aug 2017 01:53:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751908AbdHKBxr (ORCPT ); Thu, 10 Aug 2017 21:53:47 -0400 Received: from lucky1.263xmail.com ([211.157.147.133]:47467 "EHLO lucky1.263xmail.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751487AbdHKBxq (ORCPT ); Thu, 10 Aug 2017 21:53:46 -0400 Received: from shawn.lin?rock-chips.com (unknown [192.168.167.139]) by lucky1.263xmail.com (Postfix) with ESMTP id 2F3498F499; Fri, 11 Aug 2017 09:53:42 +0800 (CST) X-263anti-spam: KSV:0; X-MAIL-GRAY: 1 X-MAIL-DELIVERY: 0 X-KSVirus-check: 0 X-ABS-CHECKED: 4 Received: from localhost.localdomain (localhost [127.0.0.1]) by smtp.263.net (Postfix) with ESMTPA id 9C403819; Fri, 11 Aug 2017 09:53:35 +0800 (CST) X-RL-SENDER: shawn.lin@rock-chips.com X-FST-TO: bhelgaas@google.com X-SENDER-IP: 58.22.7.114 X-LOGIN-NAME: shawn.lin@rock-chips.com X-UNIQUE-TAG: <7a967ebfa3873b3ccecab8cd0c482aa7> X-ATTACHMENT-NUM: 0 X-SENDER: lintao@rock-chips.com X-DNS-TYPE: 0 Received: from localhost.localdomain (unknown [58.22.7.114]) by smtp.263.net (Postfix) whith ESMTP id 17411P9RL2Z; Fri, 11 Aug 2017 09:53:39 +0800 (CST) From: Shawn Lin To: Bjorn Helgaas Cc: Heiko Stuebner , linux-pci@vger.kernel.org, linux-rockchip@lists.infradead.org, Douglas Anderson , Brian Norris , Jeffy Chen , Shawn Lin Subject: [PATCH v3] PCI: rockchip: fix system hang up if activating CONFIG_DEBUG_SHIRQ Date: Fri, 11 Aug 2017 09:52:51 +0800 Message-Id: <1502416371-185632-1-git-send-email-shawn.lin@rock-chips.com> X-Mailer: git-send-email 1.9.1 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With CONFIG_DEBUG_SHIRQ enabled, the irq tear down routine would still access the irq handler registed as a shard irq. Per the comment within the function of __free_irq, it says "It's a shared IRQ -- the driver ought to be prepared for an IRQ event to happen even now it's being freed". However when failing to probe the driver, it may disable the clock for accessing the register and the following check for shared irq state would call the irq handler which accesses the register w/o the clk enabled. That will hang the system forever. With adding some dump_stack we could see how that happened. calling rockchip_pcie_driver_init+0x0/0x28 @ 1 rockchip-pcie f8000000.pcie: no vpcie3v3 regulator found rockchip-pcie f8000000.pcie: no vpcie1v8 regulator found rockchip-pcie f8000000.pcie: no vpcie0v9 regulator found rockchip-pcie f8000000.pcie: PCIe link training gen1 timeout! CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.13.0-rc3-next-20170807-ARCH+ #189 Hardware name: Firefly-RK3399 Board (DT) Call trace: [] dump_backtrace+0x0/0x250 [] show_stack+0x20/0x28 [] dump_stack+0x90/0xb0 [] rockchip_pcie_read.isra.11+0x54/0x58 [] rockchip_pcie_client_irq_handler+0x30/0x1a0 [] __free_irq+0x1c8/0x2dc [] free_irq+0x44/0x74 [] devm_irq_release+0x24/0x2c [] release_nodes+0x1d8/0x30c [] devres_release_all+0x3c/0x5c [] driver_probe_device+0x244/0x494 [] __driver_attach+0x120/0x124 [] bus_for_each_dev+0x6c/0xac [] driver_attach+0x2c/0x34 [] bus_add_driver+0x244/0x2b0 [] driver_register+0x70/0x110 [] platform_driver_register+0x60/0x6c [] rockchip_pcie_driver_init+0x20/0x28 [] do_one_initcall+0xc8/0x130 [] kernel_init_freeable+0x1a0/0x238 [] kernel_init+0x18/0x108 [] ret_from_fork+0x10/0x50 In order to fix this, we remove all the clock-disabling from the error handle path and driver's remove function. And replying on the devm_add_action_or_reset to fire the clock-disabling at the appropriate time. Also split out rockchip_pcie_setup_irq and move requesting irq after enabling clks to avoid this kind of issues. Signed-off-by: Shawn Lin --- Changes in v3: - check the return value of devm_add_action_or_reset and spilt out rockchip_pcie_setup_irq in order to move requesting irq after enabling clks. Changes in v2: - use devm_add_action_or_reset to fix this ordering suggested by Heiko and Jeffy. Thanks! drivers/pci/host/pcie-rockchip.c | 136 +++++++++++++++++++++++---------------- 1 file changed, 79 insertions(+), 57 deletions(-) diff --git a/drivers/pci/host/pcie-rockchip.c b/drivers/pci/host/pcie-rockchip.c index 39aafe2..c12c5bc 100644 --- a/drivers/pci/host/pcie-rockchip.c +++ b/drivers/pci/host/pcie-rockchip.c @@ -939,6 +939,51 @@ static int rockchip_pcie_get_phys(struct rockchip_pcie *rockchip) return 0; } +static int rockchip_pcie_setup_irq(struct rockchip_pcie *rockchip) +{ + int irq, err; + struct device *dev = rockchip->dev; + struct platform_device *pdev = to_platform_device(dev); + + irq = platform_get_irq_byname(pdev, "sys"); + if (irq < 0) { + dev_err(dev, "missing sys IRQ resource\n"); + return -EINVAL; + } + + err = devm_request_irq(dev, irq, rockchip_pcie_subsys_irq_handler, + IRQF_SHARED, "pcie-sys", rockchip); + if (err) { + dev_err(dev, "failed to request PCIe subsystem IRQ\n"); + return err; + } + + irq = platform_get_irq_byname(pdev, "legacy"); + if (irq < 0) { + dev_err(dev, "missing legacy IRQ resource\n"); + return -EINVAL; + } + + irq_set_chained_handler_and_data(irq, + rockchip_pcie_legacy_int_handler, + rockchip); + + irq = platform_get_irq_byname(pdev, "client"); + if (irq < 0) { + dev_err(dev, "missing client IRQ resource\n"); + return -EINVAL; + } + + err = devm_request_irq(dev, irq, rockchip_pcie_client_irq_handler, + IRQF_SHARED, "pcie-client", rockchip); + if (err) { + dev_err(dev, "failed to request PCIe client IRQ\n"); + return err; + } + + return 0; +} + /** * rockchip_pcie_parse_dt - Parse Device Tree * @rockchip: PCIe port information @@ -951,7 +996,6 @@ static int rockchip_pcie_parse_dt(struct rockchip_pcie *rockchip) struct platform_device *pdev = to_platform_device(dev); struct device_node *node = dev->of_node; struct resource *regs; - int irq; int err; regs = platform_get_resource_byname(pdev, @@ -1065,42 +1109,6 @@ static int rockchip_pcie_parse_dt(struct rockchip_pcie *rockchip) return PTR_ERR(rockchip->clk_pcie_pm); } - irq = platform_get_irq_byname(pdev, "sys"); - if (irq < 0) { - dev_err(dev, "missing sys IRQ resource\n"); - return -EINVAL; - } - - err = devm_request_irq(dev, irq, rockchip_pcie_subsys_irq_handler, - IRQF_SHARED, "pcie-sys", rockchip); - if (err) { - dev_err(dev, "failed to request PCIe subsystem IRQ\n"); - return err; - } - - irq = platform_get_irq_byname(pdev, "legacy"); - if (irq < 0) { - dev_err(dev, "missing legacy IRQ resource\n"); - return -EINVAL; - } - - irq_set_chained_handler_and_data(irq, - rockchip_pcie_legacy_int_handler, - rockchip); - - irq = platform_get_irq_byname(pdev, "client"); - if (irq < 0) { - dev_err(dev, "missing client IRQ resource\n"); - return -EINVAL; - } - - err = devm_request_irq(dev, irq, rockchip_pcie_client_irq_handler, - IRQF_SHARED, "pcie-client", rockchip); - if (err) { - dev_err(dev, "failed to request PCIe client IRQ\n"); - return err; - } - rockchip->vpcie12v = devm_regulator_get_optional(dev, "vpcie12v"); if (IS_ERR(rockchip->vpcie12v)) { if (PTR_ERR(rockchip->vpcie12v) == -EPROBE_DEFER) @@ -1460,6 +1468,16 @@ static int __maybe_unused rockchip_pcie_resume_noirq(struct device *dev) return err; } +static void rockchip_pcie_disable_clocks(void *data) +{ + struct rockchip_pcie *rockchip = data; + + clk_disable_unprepare(rockchip->clk_pcie_pm); + clk_disable_unprepare(rockchip->hclk_pcie); + clk_disable_unprepare(rockchip->aclk_perf_pcie); + clk_disable_unprepare(rockchip->aclk_pcie); +} + static int rockchip_pcie_probe(struct platform_device *pdev) { struct rockchip_pcie *rockchip; @@ -1493,31 +1511,49 @@ static int rockchip_pcie_probe(struct platform_device *pdev) err = clk_prepare_enable(rockchip->aclk_pcie); if (err) { dev_err(dev, "unable to enable aclk_pcie clock\n"); - goto err_aclk_pcie; + return err; } err = clk_prepare_enable(rockchip->aclk_perf_pcie); if (err) { dev_err(dev, "unable to enable aclk_perf_pcie clock\n"); - goto err_aclk_perf_pcie; + clk_disable_unprepare(rockchip->aclk_pcie); + return err; } err = clk_prepare_enable(rockchip->hclk_pcie); if (err) { dev_err(dev, "unable to enable hclk_pcie clock\n"); - goto err_hclk_pcie; + clk_disable_unprepare(rockchip->aclk_pcie); + clk_disable_unprepare(rockchip->aclk_perf_pcie); + return err; } err = clk_prepare_enable(rockchip->clk_pcie_pm); if (err) { - dev_err(dev, "unable to enable hclk_pcie clock\n"); - goto err_pcie_pm; + dev_err(dev, "unable to enable clk_pcie_pm clock\n"); + clk_disable_unprepare(rockchip->aclk_pcie); + clk_disable_unprepare(rockchip->aclk_perf_pcie); + clk_disable_unprepare(rockchip->hclk_pcie); + return err; } + err = devm_add_action_or_reset(dev, + rockchip_pcie_disable_clocks, + rockchip); + if (err) { + dev_err(dev, "unable to add action or reset\n"); + return err; + } + + err = rockchip_pcie_setup_irq(rockchip); + if (err) + return err; + err = rockchip_pcie_set_vpcie(rockchip); if (err) { dev_err(dev, "failed to set vpcie regulator\n"); - goto err_set_vpcie; + return err; } err = rockchip_pcie_init_port(rockchip); @@ -1614,15 +1650,6 @@ static int rockchip_pcie_probe(struct platform_device *pdev) regulator_disable(rockchip->vpcie1v8); if (!IS_ERR(rockchip->vpcie0v9)) regulator_disable(rockchip->vpcie0v9); -err_set_vpcie: - clk_disable_unprepare(rockchip->clk_pcie_pm); -err_pcie_pm: - clk_disable_unprepare(rockchip->hclk_pcie); -err_hclk_pcie: - clk_disable_unprepare(rockchip->aclk_perf_pcie); -err_aclk_perf_pcie: - clk_disable_unprepare(rockchip->aclk_pcie); -err_aclk_pcie: return err; } @@ -1644,11 +1671,6 @@ static int rockchip_pcie_remove(struct platform_device *pdev) phy_exit(rockchip->phys[i]); } - clk_disable_unprepare(rockchip->clk_pcie_pm); - clk_disable_unprepare(rockchip->hclk_pcie); - clk_disable_unprepare(rockchip->aclk_perf_pcie); - clk_disable_unprepare(rockchip->aclk_pcie); - if (!IS_ERR(rockchip->vpcie12v)) regulator_disable(rockchip->vpcie12v); if (!IS_ERR(rockchip->vpcie3v3))