From patchwork Thu May 4 11:35:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AngeloGioacchino Del Regno X-Patchwork-Id: 13231067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A68FBC7EE21 for ; Thu, 4 May 2023 11:36:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=XanPCdf/QzluUYaXVnffgGHNjdwZdtz7w222UbKmTtw=; b=ILA8dBq3RHVm5T OOuvwp74onrpWhR2YsTH1JFOFVV8BVSC3ytBKe2MaCu+X5MDuvrM0uOgXWCzhEaRwbczAdnFiRv+L TxnQHRyBraiRrKEWlVsOExbS4xHDhHaBw2fKCrPbVa6NbxQ4I8qJiiDUW0KIDAsx6ueuCxANKi2hO 6HiPrLmwfa6w+HvuGiIfDVs4QHtqTAlrf6uQ3LGi/BzZjtNg2MVgQy3sRhFmkGJWLwe+D4NNf7g67 n+OfP4k0AkUrcz9HVxcrdoq4vlAlGIKqRhLbiY50LZCOhOhCCpOwR1UKz2A69uBfZ3l6CKPJxIsME TpGYb54G7JMpcJF7kFPQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1puXF7-007ff6-2A; Thu, 04 May 2023 11:35:25 +0000 Received: from madras.collabora.co.uk ([46.235.227.172]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1puXF4-007fcz-1O; Thu, 04 May 2023 11:35:23 +0000 Received: from IcarusMOD.eternityproject.eu (unknown [IPv6:2001:b07:2ed:14ed:c5f8:7372:f042:90a2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kholk11) by madras.collabora.co.uk (Postfix) with ESMTPSA id 981BD66032D1; Thu, 4 May 2023 12:35:16 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1683200117; bh=oYDe/VXfJGa+IBEXNZZL8Wjnx24xldbrD++GRNp7wf8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SSY9a5QIjnOdxPA/nZt/sT2nUHh81bauNVLFJmH9cSX+HGZD4TACaot8H0bR8D23L ySp40BxxMq+oXusx1z8QEq56Gcy6IwgsSYbl8bEpH/6dpRebt9lznmGBTTtk5NwX7t Q97j8u24JQfFH/K1/zSpXU+s4RP/+JZxhw8kwvCbM7UbgSaF4yGncb6JdbFCGJeyT4 8e5GVRzCvv0KNWKK1hyjwS5jZjNxCCYUbkYdBMXSbjTqoFjyrBF0JuwESQRoU6weIn 8ioIlSAUata3gYwRmFZLJMd1FKYksDMex1zYtBGsgfRQruvoPRAy3A1jJ15lIPheG4 qcjCPyNSKplQQ== From: AngeloGioacchino Del Regno To: ryder.lee@mediatek.com Cc: jianjun.wang@mediatek.com, lpieralisi@kernel.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, p.zabel@pengutronix.de, matthias.bgg@gmail.com, angelogioacchino.delregno@collabora.com, linux-pci@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 1/2] PCI: mediatek-gen3: Stop acquiring spinlocks in {suspend,resume}_noirq Date: Thu, 4 May 2023 13:35:08 +0200 Message-Id: <20230504113509.184633-2-angelogioacchino.delregno@collabora.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230504113509.184633-1-angelogioacchino.delregno@collabora.com> References: <20230504113509.184633-1-angelogioacchino.delregno@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230504_043522_603303_5E583190 X-CRM114-Status: UNSURE ( 9.77 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In mtk_pcie_suspend_noirq() and mtk_pcie_resume_noirq() we are, respectively, disabling and enabling generation of interrupts and then saving and restoring the enabled interrupts register: since we're using noirq PM callbacks, that can be safely done without holding any spin lock. That was noticed because of, and solves, the following issue: <4>[ 74.185982] ======================================================== <4>[ 74.192629] WARNING: possible irq lock inversion dependency detected <4>[ 74.199276] 6.3.0-next-20230428+ #51 Tainted: G W <4>[ 74.205664] -------------------------------------------------------- <4>[ 74.212309] systemd-sleep/809 just changed the state of lock: <4>[ 74.218347] ffff65a5c34c65a0 (&pcie->irq_lock){+...}-{2:2}, at: mtk_pcie_resume+0x50/0xa8 <4>[ 74.226870] but this lock was taken by another, HARDIRQ-safe lock in the past: <4>[ 74.234389] (&irq_desc_lock_class){-.-.}-{2:2} <4>[ 74.234409] <4>[ 74.234409] <4>[ 74.234409] and interrupts could create inverse lock ordering between them. <4>[ 74.234409] <4>[ 74.251704] <4>[ 74.251704] other info that might help us debug this: <4>[ 74.258785] Possible interrupt unsafe locking scenario: <4>[ 74.258785] <4>[ 74.266126] CPU0 CPU1 <4>[ 74.270942] ---- ---- <4>[ 74.275758] lock(&pcie->irq_lock); <4>[ 74.279627] local_irq_disable(); <4>[ 74.285836] lock(&irq_desc_lock_class); <4>[ 74.292667] lock(&pcie->irq_lock); <4>[ 74.299061] <4>[ 74.301960] lock(&irq_desc_lock_class); <4>[ 74.306438] <4>[ 74.306438] *** DEADLOCK *** Fixes: d537dc125f07 ("PCI: mediatek-gen3: Add system PM support") Signed-off-by: AngeloGioacchino Del Regno --- drivers/pci/controller/pcie-mediatek-gen3.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/drivers/pci/controller/pcie-mediatek-gen3.c b/drivers/pci/controller/pcie-mediatek-gen3.c index b8612ce5f4d0..52f52ca5db71 100644 --- a/drivers/pci/controller/pcie-mediatek-gen3.c +++ b/drivers/pci/controller/pcie-mediatek-gen3.c @@ -963,8 +963,6 @@ static void mtk_pcie_irq_save(struct mtk_gen3_pcie *pcie) { int i; - raw_spin_lock(&pcie->irq_lock); - pcie->saved_irq_state = readl_relaxed(pcie->base + PCIE_INT_ENABLE_REG); for (i = 0; i < PCIE_MSI_SET_NUM; i++) { @@ -973,16 +971,12 @@ static void mtk_pcie_irq_save(struct mtk_gen3_pcie *pcie) msi_set->saved_irq_state = readl_relaxed(msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); } - - raw_spin_unlock(&pcie->irq_lock); } static void mtk_pcie_irq_restore(struct mtk_gen3_pcie *pcie) { int i; - raw_spin_lock(&pcie->irq_lock); - writel_relaxed(pcie->saved_irq_state, pcie->base + PCIE_INT_ENABLE_REG); for (i = 0; i < PCIE_MSI_SET_NUM; i++) { @@ -991,8 +985,6 @@ static void mtk_pcie_irq_restore(struct mtk_gen3_pcie *pcie) writel_relaxed(msi_set->saved_irq_state, msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); } - - raw_spin_unlock(&pcie->irq_lock); } static int mtk_pcie_turn_off_link(struct mtk_gen3_pcie *pcie) From patchwork Thu May 4 11:35:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AngeloGioacchino Del Regno X-Patchwork-Id: 13231068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8449DC77B7C for ; Thu, 4 May 2023 11:36:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2hhOE0CoShUf0ZDyfZp7RkQED0cGR6+ZWkmC3+aN1MM=; b=nusvyLMRsqZEko OZS8apa4oHD6BDk18Q8eh/yUjdjUf1MqXQW8OO0FMA0wWlcPGm+yWYroBaXRsPGB3i4rS9fppyE3e 7lq2czl3LlJN+nG54OKnDAK+vaGT+rUdDlgNRJq21PQ/wZ7ADUcFvofU3dT2fAA7Kuo121JUNHHZq A0IT7H+BSq+Wjfhvdo2dOOxw5z3QGWPUBdo0ZSZCKkOJbsJjKLIxG1s/IPEDpYYLeknt15nOkknib Jkl8C1KfU5alFMlEMMUTJsT2IhuChpCPjlhpa9ZRnOBIe7jhoSFUS/aSyvXC7XRHMQn8LzkYvBYv1 mE6yQoAZRdZuVvS/z0iQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1puXF8-007ffU-28; Thu, 04 May 2023 11:35:26 +0000 Received: from madras.collabora.co.uk ([46.235.227.172]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1puXF4-007fd0-1V; Thu, 04 May 2023 11:35:23 +0000 Received: from IcarusMOD.eternityproject.eu (unknown [IPv6:2001:b07:2ed:14ed:c5f8:7372:f042:90a2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kholk11) by madras.collabora.co.uk (Postfix) with ESMTPSA id 814F966056D2; Thu, 4 May 2023 12:35:17 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1683200118; bh=lNz84nkQPkKr7TAz+egLElMi8V+budLVrygOS7c2k94=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Sq9Gg07A2+ggbFLIA2eh0yDroW+O7rNKaLQTrhpuhVgPfdsgC2GGb5nlNTUFZrf4Y RW5Bsn91z2/l7+bd6kxQRU/3YjMczFDwim5t4RKVaJIB6s7TZkJg82b/fIDMR8vWms iPL2epMOl1nVgJ8dX4MA+SgbENgspRuUvZvWsVqhTGVaSlYAw2D+aa+x3BmNf6gqTw fNDEsNYix78N1x2Ns8guR6k+3cB4jZ+8ZvvRU+gOzGfB32NrrrqFEhnzrvX9IGamj+ Icoi2cuhLYeprWoCyLi759E/eYB28LWHmSQCZ9zzRwxkQVWWJJn0qa/rf6gto0aDmA P7Uxg+/ISHT2A== From: AngeloGioacchino Del Regno To: ryder.lee@mediatek.com Cc: jianjun.wang@mediatek.com, lpieralisi@kernel.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, p.zabel@pengutronix.de, matthias.bgg@gmail.com, angelogioacchino.delregno@collabora.com, linux-pci@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 2/2] PCI: mediatek-gen3: Assert MAC reset only if PHY reset also present Date: Thu, 4 May 2023 13:35:09 +0200 Message-Id: <20230504113509.184633-3-angelogioacchino.delregno@collabora.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230504113509.184633-1-angelogioacchino.delregno@collabora.com> References: <20230504113509.184633-1-angelogioacchino.delregno@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230504_043522_632047_90A375E1 X-CRM114-Status: GOOD ( 14.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Some SoCs have two PCI-Express controllers: in the case of MT8195, one of them is using a dedicated PHY, but the other uses a combo PHY that is shared with USB and in that case the PHY cannot be reset from the PCIe driver, or USB functionality will be unable to resume. Resetting the PCIe MAC without also resetting the PHY will result in a full system lockup at PCIe resume time and the only option to resume operation is to hard reboot the system (with a PMIC cut-off). To resolve this issue, check if we've got both a PHY and a MAC reset and, if not, never assert resets at PM suspend time: in that case, the link is still getting powered down as both the clocks and the power domains will go down anyway. Fixes: d537dc125f07 ("PCI: mediatek-gen3: Add system PM support") Signed-off-by: AngeloGioacchino Del Regno --- drivers/pci/controller/pcie-mediatek-gen3.c | 25 ++++++++++++++------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/drivers/pci/controller/pcie-mediatek-gen3.c b/drivers/pci/controller/pcie-mediatek-gen3.c index 52f52ca5db71..480621ca1450 100644 --- a/drivers/pci/controller/pcie-mediatek-gen3.c +++ b/drivers/pci/controller/pcie-mediatek-gen3.c @@ -859,17 +859,26 @@ static int mtk_pcie_power_up(struct mtk_gen3_pcie *pcie) return err; } -static void mtk_pcie_power_down(struct mtk_gen3_pcie *pcie) +static void mtk_pcie_power_down(struct mtk_gen3_pcie *pcie, bool is_suspend) { + bool suspend_reset_supported = pcie->mac_reset && pcie->phy_reset; + clk_bulk_disable_unprepare(pcie->num_clks, pcie->clks); pm_runtime_put_sync(pcie->dev); pm_runtime_disable(pcie->dev); - reset_control_assert(pcie->mac_reset); + + /* + * Assert MAC reset only if we also got a PHY reset, otherwise + * the system will lockup at PM resume time. + */ + if (is_suspend && suspend_reset_supported) + reset_control_assert(pcie->mac_reset); phy_power_off(pcie->phy); phy_exit(pcie->phy); - reset_control_assert(pcie->phy_reset); + if (is_suspend && suspend_reset_supported) + reset_control_assert(pcie->phy_reset); } static int mtk_pcie_setup(struct mtk_gen3_pcie *pcie) @@ -905,7 +914,7 @@ static int mtk_pcie_setup(struct mtk_gen3_pcie *pcie) return 0; err_setup: - mtk_pcie_power_down(pcie); + mtk_pcie_power_down(pcie, false); return err; } @@ -936,7 +945,7 @@ static int mtk_pcie_probe(struct platform_device *pdev) err = pci_host_probe(host); if (err) { mtk_pcie_irq_teardown(pcie); - mtk_pcie_power_down(pcie); + mtk_pcie_power_down(pcie, false); return err; } @@ -954,7 +963,7 @@ static int mtk_pcie_remove(struct platform_device *pdev) pci_unlock_rescan_remove(); mtk_pcie_irq_teardown(pcie); - mtk_pcie_power_down(pcie); + mtk_pcie_power_down(pcie, false); return 0; } @@ -1023,7 +1032,7 @@ static int mtk_pcie_suspend_noirq(struct device *dev) dev_dbg(pcie->dev, "entered L2 states successfully"); mtk_pcie_irq_save(pcie); - mtk_pcie_power_down(pcie); + mtk_pcie_power_down(pcie, true); return 0; } @@ -1039,7 +1048,7 @@ static int mtk_pcie_resume_noirq(struct device *dev) err = mtk_pcie_startup_port(pcie); if (err) { - mtk_pcie_power_down(pcie); + mtk_pcie_power_down(pcie, false); return err; }