From patchwork Mon Feb 11 22:55:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 10806927 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3639D1575 for ; Mon, 11 Feb 2019 22:55:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 25C3C2B426 for ; Mon, 11 Feb 2019 22:55:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 19F5C2B42D; Mon, 11 Feb 2019 22:55:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AC4D82B426 for ; Mon, 11 Feb 2019 22:55:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727284AbfBKWzm (ORCPT ); Mon, 11 Feb 2019 17:55:42 -0500 Received: from mail.kernel.org ([198.145.29.99]:38416 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727830AbfBKWzl (ORCPT ); Mon, 11 Feb 2019 17:55:41 -0500 Received: from localhost (unknown [64.22.249.253]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 3F5D92083B; Mon, 11 Feb 2019 22:55:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1549925740; bh=ZhRxEp9WNhE3yzMFnOonscjkSmwRIyF643uiLMom3TM=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=STKH6PceXjVHGZCZAMKWz+EEBmN9g+E9X/B4s/k3O6GpQ0Ll893eu7LMWnGI+0ocC e0yYakF2AwdwOJJkbBtHi7ihs+Vk7HkWmy2CiCESmiZJt/R0WnhS2Zo64Z3G/OSJBH CNobCPbGqVT5796YJPqcz+a0ZDZyC+DpwoGAOTC0= Subject: [PATCH v1 1/2] PCI/ASPM: Use LTR if already enabled by platform From: Bjorn Helgaas To: linux-pci@vger.kernel.org Cc: RussianNeuroMancer , David Ward , Frederick Lawler , Patrick Talbert , Lukas Wunner , Srinath Mannam , "Rafael J. Wysocki" , Vidya Sagar , Emmanuel Grumbach , linux-kernel@vger.kernel.org Date: Mon, 11 Feb 2019 16:55:39 -0600 Message-ID: <154992573941.7864.17514554950814966890.stgit@bhelgaas-glaptop.roam.corp.google.com> In-Reply-To: <154992561743.7864.14353823538864422094.stgit@bhelgaas-glaptop.roam.corp.google.com> References: <154992561743.7864.14353823538864422094.stgit@bhelgaas-glaptop.roam.corp.google.com> User-Agent: StGit/0.18 MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Bjorn Helgaas RussianNeuroMancer reported that the Intel 7265 wifi on a Dell Venue 11 Pro 7140 table stopped working after wakeup from suspend and bisected the problem to 9ab105deb60f ("PCI/ASPM: Disable ASPM L1.2 Substate if we don't have LTR"). David Ward reported the same problem on a Dell Latitude 7350. After af8bb9f89838 ("PCI/ACPI: Request LTR control from platform before using it"), we don't enable LTR unless the platform has granted LTR control to us. In addition, we don't notice if the platform had already enabled LTR itself. After 9ab105deb60f ("PCI/ASPM: Disable ASPM L1.2 Substate if we don't have LTR"), we avoid using LTR if we don't think the path to the device has LTR enabled. The combination means that if the platform itself enables LTR but declines to give the OS control over LTR, we unnecessarily avoided using ASPM L1.2. Link: https://bugzilla.kernel.org/show_bug.cgi?id=201469 Fixes: 9ab105deb60f ("PCI/ASPM: Disable ASPM L1.2 Substate if we don't have LTR") Fixes: af8bb9f89838 ("PCI/ACPI: Request LTR control from platform before using it") Reported-by: RussianNeuroMancer Reported-by: David Ward CC: stable@vger.kernel.org # v4.18+ --- drivers/pci/probe.c | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 257b9f6f2ebb..c46a3fcb341e 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2071,11 +2071,8 @@ static void pci_configure_ltr(struct pci_dev *dev) { #ifdef CONFIG_PCIEASPM struct pci_host_bridge *host = pci_find_host_bridge(dev->bus); - u32 cap; struct pci_dev *bridge; - - if (!host->native_ltr) - return; + u32 cap, ctl; if (!pci_is_pcie(dev)) return; @@ -2084,22 +2081,35 @@ static void pci_configure_ltr(struct pci_dev *dev) if (!(cap & PCI_EXP_DEVCAP2_LTR)) return; - /* - * Software must not enable LTR in an Endpoint unless the Root - * Complex and all intermediate Switches indicate support for LTR. - * PCIe r3.1, sec 6.18. - */ - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) - dev->ltr_path = 1; - else { + pcie_capability_read_dword(dev, PCI_EXP_DEVCTL2, &ctl); + if (ctl & PCI_EXP_DEVCTL2_LTR_EN) { + if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) { + dev->ltr_path = 1; + return; + } + bridge = pci_upstream_bridge(dev); if (bridge && bridge->ltr_path) dev->ltr_path = 1; + + return; } - if (dev->ltr_path) + if (!host->native_ltr) + return; + + /* + * Software must not enable LTR in an Endpoint unless the Root + * Complex and all intermediate Switches indicate support for LTR. + * PCIe r4.0, sec 6.18. + */ + if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT || + ((bridge = pci_upstream_bridge(dev)) && + bridge->ltr_path)) { pcie_capability_set_word(dev, PCI_EXP_DEVCTL2, PCI_EXP_DEVCTL2_LTR_EN); + dev->ltr_path = 1; + } #endif } From patchwork Mon Feb 11 22:55:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 10806929 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7653813B4 for ; Mon, 11 Feb 2019 22:55:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 627802B426 for ; Mon, 11 Feb 2019 22:55:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 53D1C2B42D; Mon, 11 Feb 2019 22:55:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DC32E2B426 for ; Mon, 11 Feb 2019 22:55:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727874AbfBKWzs (ORCPT ); Mon, 11 Feb 2019 17:55:48 -0500 Received: from mail.kernel.org ([198.145.29.99]:38522 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726866AbfBKWzr (ORCPT ); Mon, 11 Feb 2019 17:55:47 -0500 Received: from localhost (unknown [64.22.249.253]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6FE242083B; Mon, 11 Feb 2019 22:55:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1549925746; bh=Y0njGnl61OeTB9QnGN215mmGplKAnxdWOuj/aYkR+m0=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=BUsmEuIyAlU5JO3MQ5UI3OQD5ji6l79XEyDo70noPBY90ZYTzroJ265H02jciNj5k qGsKYZMzaCXDY0Yjnzc3/qxsWU7HqMCNrRNbiYQ8QPFXRnDbxSKkTAu2O72EJYx2rZ L0kVZISlX4i5pxpuQ3kGeEP0ICRT9EbZUnh2yEAo= Subject: [PATCH v1 2/2] PCI/ASPM: Save LTR Capability for suspend/resume From: Bjorn Helgaas To: linux-pci@vger.kernel.org Cc: RussianNeuroMancer , David Ward , Frederick Lawler , Patrick Talbert , Lukas Wunner , Srinath Mannam , "Rafael J. Wysocki" , Vidya Sagar , Emmanuel Grumbach , linux-kernel@vger.kernel.org Date: Mon, 11 Feb 2019 16:55:45 -0600 Message-ID: <154992574559.7864.8459750458248568194.stgit@bhelgaas-glaptop.roam.corp.google.com> In-Reply-To: <154992561743.7864.14353823538864422094.stgit@bhelgaas-glaptop.roam.corp.google.com> References: <154992561743.7864.14353823538864422094.stgit@bhelgaas-glaptop.roam.corp.google.com> User-Agent: StGit/0.18 MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Bjorn Helgaas Latency Tolerance Reporting (LTR) allows Endpoints and Switch Upstream Ports to report their latency requirements to upstream components. If ASPM L1 PM substates are enabled, the LTR information helps determine when a Link enters L1.2 [1]. Software must set the maximum latency values in the LTR Capability based on characteristics of the platform, then set LTR Mechanism Enable in the Device Control 2 register in the PCIe Capability. The device can then use LTR to report its latency tolerance. If the device reports a maximum latency value of zero, that means the device requires the highest possible performance and the ASPM L1.2 substate is effectively disabled. We put devices in D3 for suspend, and we assume their internal state is lost. On resume, previously we did not restore the LTR Capability, but we did restore the LTR Mechanism Enable bit, so devices would request the highest possible performance and ASPM L1.2 wouldn't be used. [1] PCIe r4.0, sec 5.5.1 Link: https://bugzilla.kernel.org/show_bug.cgi?id=201469 Signed-off-by: Bjorn Helgaas --- drivers/pci/pci.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 51 insertions(+), 2 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index c9d8e3c837de..13d65991c77b 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1233,7 +1233,6 @@ static void pci_restore_pcie_state(struct pci_dev *dev) pcie_capability_write_word(dev, PCI_EXP_SLTCTL2, cap[i++]); } - static int pci_save_pcix_state(struct pci_dev *dev) { int pos; @@ -1270,6 +1269,45 @@ static void pci_restore_pcix_state(struct pci_dev *dev) pci_write_config_word(dev, pos + PCI_X_CMD, cap[i++]); } +static void pci_save_ltr_state(struct pci_dev *dev) +{ + int ltr; + struct pci_cap_saved_state *save_state; + u16 *cap; + + if (!pci_is_pcie(dev)) + return; + + ltr = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_LTR); + if (!ltr) + return; + + save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR); + if (!save_state) { + pci_err(dev, "no suspend buffer for LTR; ASPM issues possible after resume\n"); + return; + } + + cap = (u16 *)&save_state->cap.data[0]; + pci_read_config_word(dev, ltr + PCI_LTR_MAX_SNOOP_LAT, cap++); + pci_read_config_word(dev, ltr + PCI_LTR_MAX_NOSNOOP_LAT, cap++); +} + +static void pci_restore_ltr_state(struct pci_dev *dev) +{ + struct pci_cap_saved_state *save_state; + int ltr; + u16 *cap; + + save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR); + ltr = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_LTR); + if (!save_state || !ltr) + return; + + cap = (u16 *)&save_state->cap.data[0]; + pci_write_config_word(dev, ltr + PCI_LTR_MAX_SNOOP_LAT, *cap++); + pci_write_config_word(dev, ltr + PCI_LTR_MAX_NOSNOOP_LAT, *cap++); +} /** * pci_save_state - save the PCI configuration space of a device before suspending @@ -1291,6 +1329,7 @@ int pci_save_state(struct pci_dev *dev) if (i != 0) return i; + pci_save_ltr_state(dev); pci_save_dpc_state(dev); return pci_save_vc_state(dev); } @@ -1390,7 +1429,12 @@ void pci_restore_state(struct pci_dev *dev) if (!dev->state_saved) return; - /* PCI Express register must be restored first */ + /* + * Restore max latencies (in the LTR capability) before enabling + * LTR itself (in the PCIe capability). + */ + pci_restore_ltr_state(dev); + pci_restore_pcie_state(dev); pci_restore_pasid_state(dev); pci_restore_pri_state(dev); @@ -2998,6 +3042,11 @@ void pci_allocate_cap_save_buffers(struct pci_dev *dev) if (error) pci_err(dev, "unable to preallocate PCI-X save buffer\n"); + error = pci_add_ext_cap_save_buffer(dev, PCI_EXT_CAP_ID_LTR, + 2 * sizeof(u16)); + if (error) + pci_err(dev, "unable to allocate suspend buffer for LTR\n"); + pci_allocate_vc_save_buffers(dev); }