From patchwork Thu May 4 11:35:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AngeloGioacchino Del Regno X-Patchwork-Id: 13231061 X-Patchwork-Delegate: kw@linux.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A5A4C77B7C for ; Thu, 4 May 2023 11:35:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230017AbjEDLfV (ORCPT ); Thu, 4 May 2023 07:35:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51756 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229872AbjEDLfU (ORCPT ); Thu, 4 May 2023 07:35:20 -0400 Received: from madras.collabora.co.uk (madras.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e5ab]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF15F44B5; Thu, 4 May 2023 04:35:18 -0700 (PDT) Received: from IcarusMOD.eternityproject.eu (unknown [IPv6:2001:b07:2ed:14ed:c5f8:7372:f042:90a2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kholk11) by madras.collabora.co.uk (Postfix) with ESMTPSA id 981BD66032D1; Thu, 4 May 2023 12:35:16 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1683200117; bh=oYDe/VXfJGa+IBEXNZZL8Wjnx24xldbrD++GRNp7wf8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SSY9a5QIjnOdxPA/nZt/sT2nUHh81bauNVLFJmH9cSX+HGZD4TACaot8H0bR8D23L ySp40BxxMq+oXusx1z8QEq56Gcy6IwgsSYbl8bEpH/6dpRebt9lznmGBTTtk5NwX7t Q97j8u24JQfFH/K1/zSpXU+s4RP/+JZxhw8kwvCbM7UbgSaF4yGncb6JdbFCGJeyT4 8e5GVRzCvv0KNWKK1hyjwS5jZjNxCCYUbkYdBMXSbjTqoFjyrBF0JuwESQRoU6weIn 8ioIlSAUata3gYwRmFZLJMd1FKYksDMex1zYtBGsgfRQruvoPRAy3A1jJ15lIPheG4 qcjCPyNSKplQQ== From: AngeloGioacchino Del Regno To: ryder.lee@mediatek.com Cc: jianjun.wang@mediatek.com, lpieralisi@kernel.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, p.zabel@pengutronix.de, matthias.bgg@gmail.com, angelogioacchino.delregno@collabora.com, linux-pci@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 1/2] PCI: mediatek-gen3: Stop acquiring spinlocks in {suspend,resume}_noirq Date: Thu, 4 May 2023 13:35:08 +0200 Message-Id: <20230504113509.184633-2-angelogioacchino.delregno@collabora.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230504113509.184633-1-angelogioacchino.delregno@collabora.com> References: <20230504113509.184633-1-angelogioacchino.delregno@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org In mtk_pcie_suspend_noirq() and mtk_pcie_resume_noirq() we are, respectively, disabling and enabling generation of interrupts and then saving and restoring the enabled interrupts register: since we're using noirq PM callbacks, that can be safely done without holding any spin lock. That was noticed because of, and solves, the following issue: <4>[ 74.185982] ======================================================== <4>[ 74.192629] WARNING: possible irq lock inversion dependency detected <4>[ 74.199276] 6.3.0-next-20230428+ #51 Tainted: G W <4>[ 74.205664] -------------------------------------------------------- <4>[ 74.212309] systemd-sleep/809 just changed the state of lock: <4>[ 74.218347] ffff65a5c34c65a0 (&pcie->irq_lock){+...}-{2:2}, at: mtk_pcie_resume+0x50/0xa8 <4>[ 74.226870] but this lock was taken by another, HARDIRQ-safe lock in the past: <4>[ 74.234389] (&irq_desc_lock_class){-.-.}-{2:2} <4>[ 74.234409] <4>[ 74.234409] <4>[ 74.234409] and interrupts could create inverse lock ordering between them. <4>[ 74.234409] <4>[ 74.251704] <4>[ 74.251704] other info that might help us debug this: <4>[ 74.258785] Possible interrupt unsafe locking scenario: <4>[ 74.258785] <4>[ 74.266126] CPU0 CPU1 <4>[ 74.270942] ---- ---- <4>[ 74.275758] lock(&pcie->irq_lock); <4>[ 74.279627] local_irq_disable(); <4>[ 74.285836] lock(&irq_desc_lock_class); <4>[ 74.292667] lock(&pcie->irq_lock); <4>[ 74.299061] <4>[ 74.301960] lock(&irq_desc_lock_class); <4>[ 74.306438] <4>[ 74.306438] *** DEADLOCK *** Fixes: d537dc125f07 ("PCI: mediatek-gen3: Add system PM support") Signed-off-by: AngeloGioacchino Del Regno --- drivers/pci/controller/pcie-mediatek-gen3.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/drivers/pci/controller/pcie-mediatek-gen3.c b/drivers/pci/controller/pcie-mediatek-gen3.c index b8612ce5f4d0..52f52ca5db71 100644 --- a/drivers/pci/controller/pcie-mediatek-gen3.c +++ b/drivers/pci/controller/pcie-mediatek-gen3.c @@ -963,8 +963,6 @@ static void mtk_pcie_irq_save(struct mtk_gen3_pcie *pcie) { int i; - raw_spin_lock(&pcie->irq_lock); - pcie->saved_irq_state = readl_relaxed(pcie->base + PCIE_INT_ENABLE_REG); for (i = 0; i < PCIE_MSI_SET_NUM; i++) { @@ -973,16 +971,12 @@ static void mtk_pcie_irq_save(struct mtk_gen3_pcie *pcie) msi_set->saved_irq_state = readl_relaxed(msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); } - - raw_spin_unlock(&pcie->irq_lock); } static void mtk_pcie_irq_restore(struct mtk_gen3_pcie *pcie) { int i; - raw_spin_lock(&pcie->irq_lock); - writel_relaxed(pcie->saved_irq_state, pcie->base + PCIE_INT_ENABLE_REG); for (i = 0; i < PCIE_MSI_SET_NUM; i++) { @@ -991,8 +985,6 @@ static void mtk_pcie_irq_restore(struct mtk_gen3_pcie *pcie) writel_relaxed(msi_set->saved_irq_state, msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); } - - raw_spin_unlock(&pcie->irq_lock); } static int mtk_pcie_turn_off_link(struct mtk_gen3_pcie *pcie) From patchwork Thu May 4 11:35:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AngeloGioacchino Del Regno X-Patchwork-Id: 13231062 X-Patchwork-Delegate: kw@linux.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F03CCC7EE23 for ; Thu, 4 May 2023 11:35:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229482AbjEDLfV (ORCPT ); Thu, 4 May 2023 07:35:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229993AbjEDLfV (ORCPT ); Thu, 4 May 2023 07:35:21 -0400 Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 714FA448F; Thu, 4 May 2023 04:35:19 -0700 (PDT) Received: from IcarusMOD.eternityproject.eu (unknown [IPv6:2001:b07:2ed:14ed:c5f8:7372:f042:90a2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kholk11) by madras.collabora.co.uk (Postfix) with ESMTPSA id 814F966056D2; Thu, 4 May 2023 12:35:17 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1683200118; bh=lNz84nkQPkKr7TAz+egLElMi8V+budLVrygOS7c2k94=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Sq9Gg07A2+ggbFLIA2eh0yDroW+O7rNKaLQTrhpuhVgPfdsgC2GGb5nlNTUFZrf4Y RW5Bsn91z2/l7+bd6kxQRU/3YjMczFDwim5t4RKVaJIB6s7TZkJg82b/fIDMR8vWms iPL2epMOl1nVgJ8dX4MA+SgbENgspRuUvZvWsVqhTGVaSlYAw2D+aa+x3BmNf6gqTw fNDEsNYix78N1x2Ns8guR6k+3cB4jZ+8ZvvRU+gOzGfB32NrrrqFEhnzrvX9IGamj+ Icoi2cuhLYeprWoCyLi759E/eYB28LWHmSQCZ9zzRwxkQVWWJJn0qa/rf6gto0aDmA P7Uxg+/ISHT2A== From: AngeloGioacchino Del Regno To: ryder.lee@mediatek.com Cc: jianjun.wang@mediatek.com, lpieralisi@kernel.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, p.zabel@pengutronix.de, matthias.bgg@gmail.com, angelogioacchino.delregno@collabora.com, linux-pci@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 2/2] PCI: mediatek-gen3: Assert MAC reset only if PHY reset also present Date: Thu, 4 May 2023 13:35:09 +0200 Message-Id: <20230504113509.184633-3-angelogioacchino.delregno@collabora.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230504113509.184633-1-angelogioacchino.delregno@collabora.com> References: <20230504113509.184633-1-angelogioacchino.delregno@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Some SoCs have two PCI-Express controllers: in the case of MT8195, one of them is using a dedicated PHY, but the other uses a combo PHY that is shared with USB and in that case the PHY cannot be reset from the PCIe driver, or USB functionality will be unable to resume. Resetting the PCIe MAC without also resetting the PHY will result in a full system lockup at PCIe resume time and the only option to resume operation is to hard reboot the system (with a PMIC cut-off). To resolve this issue, check if we've got both a PHY and a MAC reset and, if not, never assert resets at PM suspend time: in that case, the link is still getting powered down as both the clocks and the power domains will go down anyway. Fixes: d537dc125f07 ("PCI: mediatek-gen3: Add system PM support") Signed-off-by: AngeloGioacchino Del Regno --- drivers/pci/controller/pcie-mediatek-gen3.c | 25 ++++++++++++++------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/drivers/pci/controller/pcie-mediatek-gen3.c b/drivers/pci/controller/pcie-mediatek-gen3.c index 52f52ca5db71..480621ca1450 100644 --- a/drivers/pci/controller/pcie-mediatek-gen3.c +++ b/drivers/pci/controller/pcie-mediatek-gen3.c @@ -859,17 +859,26 @@ static int mtk_pcie_power_up(struct mtk_gen3_pcie *pcie) return err; } -static void mtk_pcie_power_down(struct mtk_gen3_pcie *pcie) +static void mtk_pcie_power_down(struct mtk_gen3_pcie *pcie, bool is_suspend) { + bool suspend_reset_supported = pcie->mac_reset && pcie->phy_reset; + clk_bulk_disable_unprepare(pcie->num_clks, pcie->clks); pm_runtime_put_sync(pcie->dev); pm_runtime_disable(pcie->dev); - reset_control_assert(pcie->mac_reset); + + /* + * Assert MAC reset only if we also got a PHY reset, otherwise + * the system will lockup at PM resume time. + */ + if (is_suspend && suspend_reset_supported) + reset_control_assert(pcie->mac_reset); phy_power_off(pcie->phy); phy_exit(pcie->phy); - reset_control_assert(pcie->phy_reset); + if (is_suspend && suspend_reset_supported) + reset_control_assert(pcie->phy_reset); } static int mtk_pcie_setup(struct mtk_gen3_pcie *pcie) @@ -905,7 +914,7 @@ static int mtk_pcie_setup(struct mtk_gen3_pcie *pcie) return 0; err_setup: - mtk_pcie_power_down(pcie); + mtk_pcie_power_down(pcie, false); return err; } @@ -936,7 +945,7 @@ static int mtk_pcie_probe(struct platform_device *pdev) err = pci_host_probe(host); if (err) { mtk_pcie_irq_teardown(pcie); - mtk_pcie_power_down(pcie); + mtk_pcie_power_down(pcie, false); return err; } @@ -954,7 +963,7 @@ static int mtk_pcie_remove(struct platform_device *pdev) pci_unlock_rescan_remove(); mtk_pcie_irq_teardown(pcie); - mtk_pcie_power_down(pcie); + mtk_pcie_power_down(pcie, false); return 0; } @@ -1023,7 +1032,7 @@ static int mtk_pcie_suspend_noirq(struct device *dev) dev_dbg(pcie->dev, "entered L2 states successfully"); mtk_pcie_irq_save(pcie); - mtk_pcie_power_down(pcie); + mtk_pcie_power_down(pcie, true); return 0; } @@ -1039,7 +1048,7 @@ static int mtk_pcie_resume_noirq(struct device *dev) err = mtk_pcie_startup_port(pcie); if (err) { - mtk_pcie_power_down(pcie); + mtk_pcie_power_down(pcie, false); return err; }