From patchwork Wed Jul 27 01:32:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 12929944 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88387C00144 for ; Wed, 27 Jul 2022 01:33:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231334AbiG0Bdv (ORCPT ); Tue, 26 Jul 2022 21:33:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230245AbiG0Bdv (ORCPT ); Tue, 26 Jul 2022 21:33:51 -0400 Received: from smtp-relay-canonical-0.canonical.com (smtp-relay-canonical-0.canonical.com [185.125.188.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE5743AE74; Tue, 26 Jul 2022 18:33:49 -0700 (PDT) Received: from localhost.localdomain (unknown [10.101.196.174]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id D66123F0E1; Wed, 27 Jul 2022 01:33:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1658885628; bh=zAZzDf5aGoTBpAzJoH3PK3AquzA6pEp7Sdhc0ZGtT3Q=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=n08e71jppZrm/5ZH++ydry++70qnxTirBD69Wny/y3E2uHFaUroEpOAOPrqmkHq5T n6n9q4J2SWNsX0KlGsJbgyd3hdlrKL2rA/Yz/TwzHHnLKLbHKhGKHSELVPEPwgwcjc CEe1bX5+HtqkFQTK+yqmqaHiF3qWkscGAHSx5wHYcYfxqikHApXBNv1lziGhPFZOXH sMJM+bAIO+K4iNWktotjLWijfNVK2lcyKJMJmWbwNEFtRyVJaBkXGtIAdy3O7BhD3Q UzJZ6KruMm6y2tqzXO5ee2koXLAmTp1O3+Yb0VvFUkNMsJ8DlLNeTSHQmKzWr9029d qLBqNrLMcMXFw== From: Kai-Heng Feng To: bhelgaas@google.com Cc: mika.westerberg@linux.intel.com, koba.ko@canonical.com, sathyanarayanan.kuppuswamy@linux.intel.com, Kai-Heng Feng , Lukas Wunner , Jan Kiszka , Stuart Hayes , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/3] PCI/portdrv: Flag services when IRQ is shared with PME Date: Wed, 27 Jul 2022 09:32:50 +0800 Message-Id: <20220727013255.269815-1-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org After commit cb1f65c1e142 ("PM: s2idle: ACPI: Fix wakeup interrupts handling"), there's a system that always gets woken up by spurious PME event when one of the root port is put to D3cold. '/sys/power/pm_wakeup_irq' shows 122, which is an IRQ shared between PME, AER and DPC: pcieport 0000:00:01.0: PME: Signaling with IRQ 122 pcieport 0000:00:01.0: AER: enabled with IRQ 122 pcieport 0000:00:01.0: DPC: enabled with IRQ 122 Disabling services one by one and the issue goes away when PCIE_PORT_SERVICE_AER is not enabled. Following the lead, more info can be found on resume when pci_aer_clear_status() is removed from pci_restore_state() to print out what happened: pcieport 0000:00:01.0: AER: Corrected error received: 0000:00:01.0 pcieport 0000:00:01.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID) pcieport 0000:00:01.0: device [8086:4c01] error status/mask=00000001/00002000 pcieport 0000:00:01.0: [ 0] RxErr Since the corrected AER error happens at physical layer when the root port is transitioning to D3cold, making system be able to suspend is more important than reporting issues like this. So introduce a new flag to indicate when IRQ is shared with PME, therefore AER and DPC can be suspended to prevent any spurious wakeup. HP already has its own suspend routine so it doesn't need to use this flag. Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=216295 Signed-off-by: Kai-Heng Feng --- drivers/pci/pcie/portdrv.h | 11 ++++++----- drivers/pci/pcie/portdrv_core.c | 21 +++++++++++++++------ 2 files changed, 21 insertions(+), 11 deletions(-) diff --git a/drivers/pci/pcie/portdrv.h b/drivers/pci/pcie/portdrv.h index 0ef4bf5f811d9..97a9d15638cc8 100644 --- a/drivers/pci/pcie/portdrv.h +++ b/drivers/pci/pcie/portdrv.h @@ -57,11 +57,12 @@ static inline int pcie_dpc_init(void) { return 0; } #define PCIE_ANY_PORT (~0) struct pcie_device { - int irq; /* Service IRQ/MSI/MSI-X Vector */ - struct pci_dev *port; /* Root/Upstream/Downstream Port */ - u32 service; /* Port service this device represents */ - void *priv_data; /* Service Private Data */ - struct device device; /* Generic Device Interface */ + int irq; /* Service IRQ/MSI/MSI-X Vector */ + struct pci_dev *port; /* Root/Upstream/Downstream Port */ + u32 service; /* Port service this device represents */ + bool shared_pme_irq; /* Service IRQ shared with PME */ + void *priv_data; /* Service Private Data */ + struct device device; /* Generic Device Interface */ }; #define to_pcie_device(d) container_of(d, struct pcie_device, device) diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c index 604feeb84ee40..ddc6854cdde2d 100644 --- a/drivers/pci/pcie/portdrv_core.c +++ b/drivers/pci/pcie/portdrv_core.c @@ -274,7 +274,7 @@ static int get_port_device_capability(struct pci_dev *dev) * @service: Type of service to associate with the service device * @irq: Interrupt vector to associate with the service device */ -static int pcie_device_init(struct pci_dev *pdev, int service, int irq) +static struct pcie_device *pcie_device_init(struct pci_dev *pdev, int service, int irq) { int retval; struct pcie_device *pcie; @@ -282,7 +282,7 @@ static int pcie_device_init(struct pci_dev *pdev, int service, int irq) pcie = kzalloc(sizeof(*pcie), GFP_KERNEL); if (!pcie) - return -ENOMEM; + return ERR_PTR(-ENOMEM); pcie->port = pdev; pcie->irq = irq; pcie->service = service; @@ -300,12 +300,12 @@ static int pcie_device_init(struct pci_dev *pdev, int service, int irq) retval = device_register(device); if (retval) { put_device(device); - return retval; + return ERR_PTR(retval); } pm_runtime_no_callbacks(device); - return 0; + return pcie; } /** @@ -350,10 +350,19 @@ int pcie_port_device_register(struct pci_dev *dev) nr_service = 0; for (i = 0; i < PCIE_PORT_DEVICE_MAXSERVICES; i++) { int service = 1 << i; + struct pcie_device *pcie; if (!(capabilities & service)) continue; - if (!pcie_device_init(dev, service, irqs[i])) - nr_service++; + + pcie = pcie_device_init(dev, service, irqs[i]); + if (IS_ERR(pcie)) + continue; + + nr_service++; + + if (i != PCIE_PORT_SERVICE_PME_SHIFT && + irqs[i] == irqs[PCIE_PORT_SERVICE_PME_SHIFT]) + pcie->shared_pme_irq = true; } if (!nr_service) goto error_cleanup_irqs; From patchwork Wed Jul 27 01:32:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 12929945 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAE55C19F21 for ; Wed, 27 Jul 2022 01:34:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240096AbiG0Bd7 (ORCPT ); Tue, 26 Jul 2022 21:33:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240145AbiG0Bd6 (ORCPT ); Tue, 26 Jul 2022 21:33:58 -0400 Received: from smtp-relay-canonical-0.canonical.com (smtp-relay-canonical-0.canonical.com [185.125.188.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 363153B944; Tue, 26 Jul 2022 18:33:55 -0700 (PDT) Received: from localhost.localdomain (unknown [10.101.196.174]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id 1508E3F395; Wed, 27 Jul 2022 01:33:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1658885634; bh=J7Mo7gRC9aUnXUlERL8fcZBWoJb2JAks8HFFUOO/0qw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=acyN2D9yFbUZ/Di08hQd138lFj++ds1vlYZ+Yl49z3HG+jdfU29MdKH+i3syPYvvW jMeejKm069QBpOOIQ88GvibA3gP6GHjBSbeugvUV/j2Y8a8luyT9yyff0Rn0zoRIrj k/P1Lf5C8n17MNcgSuBZB6LdaY/w/VkDLMOfM+7r35laPtNsvYRghH8L+CtTZBiMpK 3a1iWObZEdTXFhfrKpzFhEPgGEe6bBPREQ5qp+pro2mnXsJwB/bx62SUGi3kbvfk0x C0kwAiwb2i3KXtB/ON81AQhdcguRpPJWR5GVC3/Yod9zWEfMTiSsMITi+TN4n0LLPM KXrElqQEzq+SQ== From: Kai-Heng Feng To: bhelgaas@google.com Cc: mika.westerberg@linux.intel.com, koba.ko@canonical.com, sathyanarayanan.kuppuswamy@linux.intel.com, Kai-Heng Feng , Russell Currey , "Oliver O'Halloran" , linuxppc-dev@lists.ozlabs.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/3] PCI/AER: Disable AER service on suspend when IRQ is shared with PME Date: Wed, 27 Jul 2022 09:32:51 +0800 Message-Id: <20220727013255.269815-2-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220727013255.269815-1-kai.heng.feng@canonical.com> References: <20220727013255.269815-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org PCIe service that shares IRQ with PME may cause spurious wakeup on system suspend. PCIe Base Spec 5.0, section 5.2 "Link State Power Management" states that TLP and DLLP transmission is disabled for a Link in L2/L3 Ready (D3hot), L2 (D3cold with aux power) and L3 (D3cold), so we don't lose much here to disable AER during system suspend. This is very similar to previous attempts to suspend AER and DPC [1], but with a different reason. [1] https://lore.kernel.org/linux-pci/20220408153159.106741-1-kai.heng.feng@canonical.com/ Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=216295 Signed-off-by: Kai-Heng Feng --- drivers/pci/pcie/aer.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 7952e5efd6cf3..60cc373754af2 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -1372,6 +1372,26 @@ static int aer_probe(struct pcie_device *dev) return 0; } +static int aer_suspend(struct pcie_device *dev) +{ + struct aer_rpc *rpc = get_service_data(dev); + + if (dev->shared_pme_irq) + aer_disable_rootport(rpc); + + return 0; +} + +static int aer_resume(struct pcie_device *dev) +{ + struct aer_rpc *rpc = get_service_data(dev); + + if (dev->shared_pme_irq) + aer_enable_rootport(rpc); + + return 0; +} + /** * aer_root_reset - reset Root Port hierarchy, RCEC, or RCiEP * @dev: pointer to Root Port, RCEC, or RCiEP @@ -1441,8 +1461,9 @@ static struct pcie_port_service_driver aerdriver = { .name = "aer", .port_type = PCIE_ANY_PORT, .service = PCIE_PORT_SERVICE_AER, - .probe = aer_probe, + .suspend = aer_suspend, + .resume = aer_resume, .remove = aer_remove, }; From patchwork Wed Jul 27 01:32:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 12929946 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D586C00144 for ; Wed, 27 Jul 2022 01:34:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240153AbiG0BeL (ORCPT ); Tue, 26 Jul 2022 21:34:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46316 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240173AbiG0BeI (ORCPT ); Tue, 26 Jul 2022 21:34:08 -0400 Received: from smtp-relay-canonical-0.canonical.com (smtp-relay-canonical-0.canonical.com [185.125.188.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E52163B954; Tue, 26 Jul 2022 18:34:02 -0700 (PDT) Received: from localhost.localdomain (unknown [10.101.196.174]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id E199F3F39B; Wed, 27 Jul 2022 01:33:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1658885640; bh=DsZRX8j/q4Cpjzn1W11I3hAQXEWJrBN86NgoVVaLV3A=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=TFT3im8fedDV8eqhGPtwSn5vtGFvE76IEoZRXyiGAyzytfZ8PqyKDLyMGQMWenpoz EdSIjBu6PSxYV7wq2N2wFN1sjp2XAE+SK1jV7fNMAAz8baJle8ANICSqr+hTaycaEh FuYrwAQ2Di/3kKJWgn6RN/72amg9Xwr4/uATsenmxJ0BgIRSXeod0tz+cHOwRMQnu1 /R68+i0BdBJgtsIjpvbw9QwfsdOwIwgSUDlTyRXpn9DA6Sh2gOy/CNheuZLoPyvSq5 a4cN9IRMFVgkW+Ij64VEIj9aOxWqBDJLEO93w7suSgZPd0+IMpRG85rwJZ24vvL62M 1rBCFxj6z5rGg== From: Kai-Heng Feng To: bhelgaas@google.com Cc: mika.westerberg@linux.intel.com, koba.ko@canonical.com, sathyanarayanan.kuppuswamy@linux.intel.com, Kai-Heng Feng , Russell Currey , "Oliver O'Halloran" , linuxppc-dev@lists.ozlabs.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] PCI/DPC: Disable DPC service on suspend when IRQ is shared with PME Date: Wed, 27 Jul 2022 09:32:52 +0800 Message-Id: <20220727013255.269815-3-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220727013255.269815-1-kai.heng.feng@canonical.com> References: <20220727013255.269815-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org PCIe service that shares IRQ with PME may cause spurious wakeup on system suspend. Since AER is conditionally disabled in previous patch, also apply the same condition to disable DPC which depends on AER to work. PCIe Base Spec 5.0, section 5.2 "Link State Power Management" states that TLP and DLLP transmission is disabled for a Link in L2/L3 Ready (D3hot), L2 (D3cold with aux power) and L3 (D3cold), so we don't lose much here to disable DPC during system suspend. This is very similar to previous attempts to suspend AER and DPC [1], but with a different reason. [1] https://lore.kernel.org/linux-pci/20220408153159.106741-1-kai.heng.feng@canonical.com/ Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=216295 Signed-off-by: Kai-Heng Feng --- drivers/pci/pcie/dpc.c | 52 +++++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 11 deletions(-) diff --git a/drivers/pci/pcie/dpc.c b/drivers/pci/pcie/dpc.c index 3e9afee02e8d1..542f282c43f75 100644 --- a/drivers/pci/pcie/dpc.c +++ b/drivers/pci/pcie/dpc.c @@ -343,13 +343,33 @@ void pci_dpc_init(struct pci_dev *pdev) } } +static void dpc_enable(struct pcie_device *dev) +{ + struct pci_dev *pdev = dev->port; + u16 ctl; + + pci_read_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, &ctl); + ctl = (ctl & 0xfff4) | PCI_EXP_DPC_CTL_EN_FATAL | PCI_EXP_DPC_CTL_INT_EN; + pci_write_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, ctl); +} + +static void dpc_disable(struct pcie_device *dev) +{ + struct pci_dev *pdev = dev->port; + u16 ctl; + + pci_read_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, &ctl); + ctl &= ~(PCI_EXP_DPC_CTL_EN_FATAL | PCI_EXP_DPC_CTL_INT_EN); + pci_write_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, ctl); +} + #define FLAG(x, y) (((x) & (y)) ? '+' : '-') static int dpc_probe(struct pcie_device *dev) { struct pci_dev *pdev = dev->port; struct device *device = &dev->device; int status; - u16 ctl, cap; + u16 cap; if (!pcie_aer_is_native(pdev) && !pcie_ports_dpc_native) return -ENOTSUPP; @@ -364,10 +384,7 @@ static int dpc_probe(struct pcie_device *dev) } pci_read_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CAP, &cap); - pci_read_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, &ctl); - - ctl = (ctl & 0xfff4) | PCI_EXP_DPC_CTL_EN_FATAL | PCI_EXP_DPC_CTL_INT_EN; - pci_write_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, ctl); + dpc_enable(dev); pci_info(pdev, "enabled with IRQ %d\n", dev->irq); pci_info(pdev, "error containment capabilities: Int Msg #%d, RPExt%c PoisonedTLP%c SwTrigger%c RP PIO Log %d, DL_ActiveErr%c\n", @@ -380,14 +397,25 @@ static int dpc_probe(struct pcie_device *dev) return status; } -static void dpc_remove(struct pcie_device *dev) +static int dpc_suspend(struct pcie_device *dev) { - struct pci_dev *pdev = dev->port; - u16 ctl; + if (dev->shared_pme_irq) + dpc_disable(dev); - pci_read_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, &ctl); - ctl &= ~(PCI_EXP_DPC_CTL_EN_FATAL | PCI_EXP_DPC_CTL_INT_EN); - pci_write_config_word(pdev, pdev->dpc_cap + PCI_EXP_DPC_CTL, ctl); + return 0; +} + +static int dpc_resume(struct pcie_device *dev) +{ + if (dev->shared_pme_irq) + dpc_enable(dev); + + return 0; +} + +static void dpc_remove(struct pcie_device *dev) +{ + dpc_disable(dev); } static struct pcie_port_service_driver dpcdriver = { @@ -395,6 +423,8 @@ static struct pcie_port_service_driver dpcdriver = { .port_type = PCIE_ANY_PORT, .service = PCIE_PORT_SERVICE_DPC, .probe = dpc_probe, + .suspend = dpc_suspend, + .resume = dpc_resume, .remove = dpc_remove, };