From patchwork Tue Jan 12 04:02:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victor Ding X-Patchwork-Id: 12012321 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79586C433E9 for ; Tue, 12 Jan 2021 04:03:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4D4DB22A83 for ; Tue, 12 Jan 2021 04:03:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726564AbhALEDd (ORCPT ); Mon, 11 Jan 2021 23:03:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43458 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728386AbhALEDc (ORCPT ); Mon, 11 Jan 2021 23:03:32 -0500 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4ADC9C0617A2 for ; Mon, 11 Jan 2021 20:02:16 -0800 (PST) Received: by mail-pj1-x104a.google.com with SMTP id c1so655635pjo.6 for ; Mon, 11 Jan 2021 20:02:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=ak768gEI6XCH+aPGM5U6EpkG7B0YhsCWJJw1G4ZyE7I=; b=JG+Sg8xWVV6o2H7jEpk7k4T+BYMj0g9b6yD/Ot9mOHxD3cId9PvCBaP793TPOW6UDE NemddI4QWvvzitpZfeFW+h/oZk7T6Jnj+EEIilx9OvKxUllf+a15pi4wBluJwif15orG lxt0Iwhipfxrk6ZW89+/Fvs9cjOHvOjWU7z5Ydl+jyjAJMe1yjselzRW72578Bx2Fyqc d3fSSxw4Snb1Eucb/fUPnADWmIUQDqlRl6E6WwlguCH12WUI4466mfh2+DLUIKH0GubQ Z+ijve1yEtXLZp0i1ZdOM1OSbmUX0fmUjCnlyPY9JvwvvDhbZnup8zgujXBoFuyz5geq 1Dyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ak768gEI6XCH+aPGM5U6EpkG7B0YhsCWJJw1G4ZyE7I=; b=YoOJSWLWvPwfMhF3i4xUcDytPblOcvWJvJ48hispZzv08eCXuxwb1S/eLfPEmwoww8 mgv/2ItHxwGNoPwDYy2E+5Zooc/FoKGGeqsk5nVAR6TtRLMeG6vCUwg6AKG6CwkfSyyE NbSPtpP/XgHt12xMR/r4pfZgGvKp17ychUojng4J5GRcG91xHeFLnPrrQG1CNaDYdnv9 ktWEipjh9jv98h/B0a8qgd3QuGohJWmsL1hveEKPB5k2yb8vGVmpQ4wR2WVybMRKVFsu zsBbtXy5da2iAf4Ve/FkBrPQKvcqjPsg5IT6fTs3twX/pxnnBRCrzuCipcimPkXafTAY i1RA== X-Gm-Message-State: AOAM530+hPpUeghlKRhMptrwx/LtJ5wmBRuSua8IWa3X/PhHl+9Vj3St pet0iwNx5FNhYCiQfOyzqMx8Zk9xqsCskav8 X-Google-Smtp-Source: ABdhPJyeGpHSwrKQpN/leiMW8w46NVzHLITPTcbviWLsyHqYnfuKcC5BbfnSxl+8REx+Rif+d0Rdxchqd99GuCMG Sender: "victording via sendgmr" X-Received: from victording.c.googlers.com ([fda3:e722:ac3:10:24:72f4:c0a8:65c7]) (user=victording job=sendgmr) by 2002:a17:902:c144:b029:dc:292e:a8a1 with SMTP id 4-20020a170902c144b02900dc292ea8a1mr3103608plj.13.1610424135755; Mon, 11 Jan 2021 20:02:15 -0800 (PST) Date: Tue, 12 Jan 2021 04:02:04 +0000 In-Reply-To: <20210112040205.4117303-1-victording@google.com> Message-Id: <20210112040146.1.I9aa2b9dd39a6ac9235ec55a8c56020538aa212fb@changeid> Mime-Version: 1.0 References: <20210112040205.4117303-1-victording@google.com> X-Mailer: git-send-email 2.30.0.284.gd98b1dd5eaa7-goog Subject: [PATCH 1/2] PCI/ASPM: Disable ASPM until its LTR and L1ss state is restored From: Victor Ding To: Ulf Hansson , Adrian Hunter Cc: Ben Chuang , Bjorn Helgaas , linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-mmc@vger.kernel.org, Victor Ding , Alex Levin , Bjorn Helgaas , Kai-Heng Feng , "Saheed O. Bolarinwa" , Sean Paul , Sukumar Ghorai , Yicong Yang Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Right after powering up, the device may have ASPM enabled; however, its LTR and/or L1ss controls may not be in the desired states; hence, the device may enter L1.2 undesirably and cause resume performance penalty. This is especially problematic if ASPM related control registers are modified before a suspension. Therefore, ASPM should disabled until its LTR and L1ss states are fully restored. Signed-off-by: Victor Ding --- drivers/pci/pci.c | 11 +++++++++++ drivers/pci/pci.h | 2 ++ drivers/pci/pcie/aspm.c | 2 +- 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index eb323af34f1e..428de433f2e6 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1660,6 +1660,17 @@ void pci_restore_state(struct pci_dev *dev) if (!dev->state_saved) return; + /* + * Right after powering up, the device may have ASPM enabled; + * however, its LTR and/or L1ss controls may not be in the desired + * states; as a result, the device may enter L1.2 undesirably and + * cause resume performance penalty. + * Therefore, ASPM is disabled until its LTR and L1ss states are + * fully restored. + * (enabling ASPM is part of pci_restore_pcie_state) + */ + pcie_config_aspm_dev(dev, 0); + /* * Restore max latencies (in the LTR capability) before enabling * LTR itself (in the PCIe capability). diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index e9ea5dfaa3e0..f774bd6d2555 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -564,6 +564,7 @@ void pcie_aspm_init_link_state(struct pci_dev *pdev); void pcie_aspm_exit_link_state(struct pci_dev *pdev); void pcie_aspm_pm_state_change(struct pci_dev *pdev); void pcie_aspm_powersave_config_link(struct pci_dev *pdev); +void pcie_config_aspm_dev(struct pci_dev *pdev, u32 val); void pci_save_aspm_l1ss_state(struct pci_dev *dev); void pci_restore_aspm_l1ss_state(struct pci_dev *dev); #else @@ -571,6 +572,7 @@ static inline void pcie_aspm_init_link_state(struct pci_dev *pdev) { } static inline void pcie_aspm_exit_link_state(struct pci_dev *pdev) { } static inline void pcie_aspm_pm_state_change(struct pci_dev *pdev) { } static inline void pcie_aspm_powersave_config_link(struct pci_dev *pdev) { } +static inline void pcie_config_aspm_dev(struct pci_dev *pdev, u32 val) { } static inline void pci_save_aspm_l1ss_state(struct pci_dev *dev) { } static inline void pci_restore_aspm_l1ss_state(struct pci_dev *dev) { } #endif diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index a08e7d6dc248..45535b4e1595 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -778,7 +778,7 @@ void pci_restore_aspm_l1ss_state(struct pci_dev *dev) pci_write_config_dword(dev, aspm_l1ss + PCI_L1SS_CTL2, *cap++); } -static void pcie_config_aspm_dev(struct pci_dev *pdev, u32 val) +void pcie_config_aspm_dev(struct pci_dev *pdev, u32 val) { pcie_capability_clear_and_set_word(pdev, PCI_EXP_LNKCTL, PCI_EXP_LNKCTL_ASPMC, val); From patchwork Tue Jan 12 04:02:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victor Ding X-Patchwork-Id: 12012323 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62A88C433E6 for ; Tue, 12 Jan 2021 04:03:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 30C2C22C7B for ; Tue, 12 Jan 2021 04:03:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729061AbhALEDd (ORCPT ); Mon, 11 Jan 2021 23:03:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728688AbhALEDc (ORCPT ); Mon, 11 Jan 2021 23:03:32 -0500 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0F00C0617A4 for ; Mon, 11 Jan 2021 20:02:20 -0800 (PST) Received: by mail-pj1-x1049.google.com with SMTP id s14so660023pjk.4 for ; Mon, 11 Jan 2021 20:02:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=X7CTBa0CXHOMqqTGqsqUqty/+Ht2Bje2c2Eq+Eb4V0I=; b=dosHyd1xU5Jxa86wRgk5aAlB3weKCxWCFviYJFQ5ts0KGDbckGx5opGQ9PLbHJ5ifJ tIx+WKUaCrT1bO4ZigiP/f8LvgOpoR0509E71Qmx3WsN9pNKinGTFVzU/eSeXcR3MEFo pyH2d+ZDMVfsWs16oKFRAVN0K68ioK92ODhM45kYJpAEummKurbw+PnoHkcS84hgBtjy O/p8Zb1VEpMHkikB2uWbC20guJy5HMkUPcimdCcqIR/FuGrxq2UzRqLO8uT3KP3Sfl3Q EfaL6y3GAFHULAWVUHDTfc1mTnOOyYvy4gw+IbLuPPN+xfWnajuLPR+y3vR22v8DHHY8 MsAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=X7CTBa0CXHOMqqTGqsqUqty/+Ht2Bje2c2Eq+Eb4V0I=; b=SBPUaeLdoXeyy+pj//8ePZRjifo71SMcrZ+wNok89yUCZc+V4nA1vXG/+fm8qaZ/zM +ZUiAiTKVC8WdnT/CS23yPwVb5WfPsRqX8IvPM5A6rIEd6OEX/Wp4Ey82OuhGANb1cOH BuACqzzPCKh8Zp7G8//RUUH+gva3d9Htp/UrjiAWNuQfLAcOJWpXlchilBqkXkE6ck2j cacNCIJa7Rt6S5FExQEw8jJrdSNTrnti4c7h6eHhNw23Hr4U25wf/MDCdrG5bcRuslun Eii6vR9w6t7OLGorhIlPziPCM+BwXfZvNXxvx48QUEQHqNMIGGlSE/pVMepqRetrOWcQ dBNw== X-Gm-Message-State: AOAM531fWtulp+gkVTaUuNabxK9sW96s8m0wAKmG+FifSZi4VCWaRp04 nOLo9PyjmfPGRp0W7Ta8sstF+OYcg62INOoT X-Google-Smtp-Source: ABdhPJyXGja+0Cr6qcY6OW3IG/D9g+yhPgqXQIJ7KYT6DxWIWIIz08eY6z+Wu8RStj7uYyMxpSEhnjEkGAwCOm5s Sender: "victording via sendgmr" X-Received: from victording.c.googlers.com ([fda3:e722:ac3:10:24:72f4:c0a8:65c7]) (user=victording job=sendgmr) by 2002:a17:902:fe07:b029:dc:43e4:fcbf with SMTP id g7-20020a170902fe07b02900dc43e4fcbfmr2668307plj.63.1610424140274; Mon, 11 Jan 2021 20:02:20 -0800 (PST) Date: Tue, 12 Jan 2021 04:02:05 +0000 In-Reply-To: <20210112040205.4117303-1-victording@google.com> Message-Id: <20210112040146.2.Ic902bbd9f04e2d82ac578411e7fafc77b6c750e2@changeid> Mime-Version: 1.0 References: <20210112040205.4117303-1-victording@google.com> X-Mailer: git-send-email 2.30.0.284.gd98b1dd5eaa7-goog Subject: [PATCH 2/2] mmc: sdhci-pci-gli: Disable ASPM during a suspension From: Victor Ding To: Ulf Hansson , Adrian Hunter Cc: Ben Chuang , Bjorn Helgaas , linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-mmc@vger.kernel.org, Victor Ding Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org GL9750 has a 3100us PortTPowerOnTime; however, it enters L1.2 after only ~4us inactivity per PCIe trace. During a suspend/resume process, PCI access operations are frequently longer than 4us apart. Therefore, the device frequently enters and leaves L1.2 during this process, causing longer than desirable suspend/resume time. The total time cost due to this L1.2 exit latency could add up to ~200ms. Considering that PCI access operations are fairly close to each other (though sometimes > 4us), the actual time the device could stay in L1.2 is negligible. Therefore, the little power-saving benefit from ASPM during suspend/resume does not overweight the performance degradation caused by long L1.2 exit latency. Therefore, this patch proposes to disable ASPM during a suspend/resume process. Signed-off-by: Victor Ding --- drivers/mmc/host/sdhci-pci-core.c | 2 +- drivers/mmc/host/sdhci-pci-gli.c | 46 +++++++++++++++++++++++++++++-- drivers/mmc/host/sdhci-pci.h | 1 + 3 files changed, 46 insertions(+), 3 deletions(-) diff --git a/drivers/mmc/host/sdhci-pci-core.c b/drivers/mmc/host/sdhci-pci-core.c index 9552708846ca..fd7544a498c0 100644 --- a/drivers/mmc/host/sdhci-pci-core.c +++ b/drivers/mmc/host/sdhci-pci-core.c @@ -67,7 +67,7 @@ static int sdhci_pci_init_wakeup(struct sdhci_pci_chip *chip) return 0; } -static int sdhci_pci_suspend_host(struct sdhci_pci_chip *chip) +int sdhci_pci_suspend_host(struct sdhci_pci_chip *chip) { int i, ret; diff --git a/drivers/mmc/host/sdhci-pci-gli.c b/drivers/mmc/host/sdhci-pci-gli.c index 9887485a4134..c7b788b0e22e 100644 --- a/drivers/mmc/host/sdhci-pci-gli.c +++ b/drivers/mmc/host/sdhci-pci-gli.c @@ -109,6 +109,12 @@ #define GLI_MAX_TUNING_LOOP 40 +#ifdef CONFIG_PM_SLEEP +struct gli_host { + u16 linkctl_saved; +}; +#endif + /* Genesys Logic chipset */ static inline void gl9750_wt_on(struct sdhci_host *host) { @@ -577,14 +583,48 @@ static u32 sdhci_gl9750_readl(struct sdhci_host *host, int reg) } #ifdef CONFIG_PM_SLEEP +static int sdhci_pci_gli_suspend(struct sdhci_pci_chip *chip) +{ + int ret; + struct sdhci_pci_slot *slot = chip->slots[0]; + struct pci_dev *pdev = slot->chip->pdev; + struct gli_host *gli_host = sdhci_pci_priv(slot); + + ret = pcie_capability_read_word(pdev, PCI_EXP_LNKCTL, + &gli_host->linkctl_saved); + if (ret) + goto exit; + + ret = pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, + gli_host->linkctl_saved & ~PCI_EXP_LNKCTL_ASPMC); + if (ret) + goto exit; + + ret = sdhci_pci_suspend_host(chip); + +exit: + return ret; +} + static int sdhci_pci_gli_resume(struct sdhci_pci_chip *chip) { + int ret; struct sdhci_pci_slot *slot = chip->slots[0]; + struct pci_dev *pdev = slot->chip->pdev; + struct gli_host *gli_host = sdhci_pci_priv(slot); - pci_free_irq_vectors(slot->chip->pdev); + pci_free_irq_vectors(pdev); gli_pcie_enable_msi(slot); - return sdhci_pci_resume_host(chip); + ret = sdhci_pci_resume_host(chip); + if (ret) + goto exit; + + ret = pcie_capability_clear_and_set_word(pdev, PCI_EXP_LNKCTL, + PCI_EXP_LNKCTL_ASPMC, gli_host->linkctl_saved); + +exit: + return ret; } static int sdhci_cqhci_gli_resume(struct sdhci_pci_chip *chip) @@ -834,7 +874,9 @@ const struct sdhci_pci_fixes sdhci_gl9750 = { .probe_slot = gli_probe_slot_gl9750, .ops = &sdhci_gl9750_ops, #ifdef CONFIG_PM_SLEEP + .suspend = sdhci_pci_gli_suspend, .resume = sdhci_pci_gli_resume, + .priv_size = sizeof(struct gli_host), #endif }; diff --git a/drivers/mmc/host/sdhci-pci.h b/drivers/mmc/host/sdhci-pci.h index d0ed232af0eb..16187a265e63 100644 --- a/drivers/mmc/host/sdhci-pci.h +++ b/drivers/mmc/host/sdhci-pci.h @@ -187,6 +187,7 @@ static inline void *sdhci_pci_priv(struct sdhci_pci_slot *slot) } #ifdef CONFIG_PM_SLEEP +int sdhci_pci_suspend_host(struct sdhci_pci_chip *chip); int sdhci_pci_resume_host(struct sdhci_pci_chip *chip); #endif int sdhci_pci_enable_dma(struct sdhci_host *host);