From patchwork Tue Mar 1 04:19:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "David E. Box" X-Patchwork-Id: 12764045 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2D92C4332F for ; Tue, 1 Mar 2022 04:19:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232259AbiCAEUa (ORCPT ); Mon, 28 Feb 2022 23:20:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232261AbiCAEU1 (ORCPT ); Mon, 28 Feb 2022 23:20:27 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5E9F2C111; Mon, 28 Feb 2022 20:19:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1646108387; x=1677644387; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AIyzSJ/ML6P0dJudfUlrNMtxGvJXJ1cbc80FGTGSMus=; b=LNOtWDx1OyUdGhDkoye7ee4CkcxOs4Al6IAS5dHwk1qCOQebSp4pAwL2 4SxxiHQIrLxXQktBENt6xdFpe/un8YUrt3uMhQ7RxATvpcSmr0ksqKq6r 0P4IbRZZffGXuDnmcV9Ny24fYiFJMaUbDnpb1bxl0qDKdrSkvxleFwqdU 7NQ/kNGCthNFGRFroP7LDtD4QLNcnKAuy8aa956Ub0greSy8Q+FLyIfPI Qz/7NGNArpB4xfqGf+qIQGqXGUO62pRRoyJNa+MPWkTekhHfbtGOTnZVj LfH/i41yrbvabhY+8v79ZHxiBcFPNLCcozbyIt4ZaQ9XgERMn4kG85Z2e A==; X-IronPort-AV: E=McAfee;i="6200,9189,10272"; a="240463932" X-IronPort-AV: E=Sophos;i="5.90,144,1643702400"; d="scan'208";a="240463932" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Feb 2022 20:19:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,144,1643702400"; d="scan'208";a="641123501" Received: from linux.intel.com ([10.54.29.200]) by orsmga004.jf.intel.com with ESMTP; 28 Feb 2022 20:19:44 -0800 Received: from debox1-desk4.lan (unknown [10.251.9.202]) by linux.intel.com (Postfix) with ESMTP id C8F3C580C20; Mon, 28 Feb 2022 20:19:43 -0800 (PST) From: "David E. Box" To: nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev, lorenzo.pieralisi@arm.com, hch@infradead.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, david.e.box@linux.intel.com, michael.a.bottini@linux.intel.com, rafael@kernel.org, me@adhityamohan.in Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V6 1/3] PCI/ASPM: Add pci_enable_default_link_state() Date: Mon, 28 Feb 2022 20:19:41 -0800 Message-Id: <20220301041943.2935892-2-david.e.box@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220301041943.2935892-1-david.e.box@linux.intel.com> References: <20220301041943.2935892-1-david.e.box@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Michael Bottini Add pci_enable_default_link_state() to allow devices to change the default BIOS configured states. Clears the BIOS default settings then sets the new states and reconfigures the link under the semaphore. Also add PCIE_LINK_STATE_ALL macro for convenience for callers that want to enable all link states. Signed-off-by: Michael Bottini Signed-off-by: David E. Box Acked-by: Bjorn Helgaas --- V6 - No change V5 - Rename to pci_enable_default_link_state and model after pci_disable_link_state() as suggested by Bjorn. - Add helper PCIE_LINK_STATE_ALL which sets bits for all links states and clock pm. - Clarify commit language to indicate the function changes the default link states (not ASPM policy). V4 - Refactor vmd_enable_apsm() to exit early, making the lines shorter and more readable. Suggested by Christoph. V3 - No changes V2 - Use return status to print pci_info message if ASPM cannot be enabled. - Add missing static declaration, caught by lkp@intel.com drivers/pci/pcie/aspm.c | 54 +++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 7 ++++++ 2 files changed, 61 insertions(+) diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index a96b7424c9bc..b2752851b1ba 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -1132,6 +1132,60 @@ int pci_disable_link_state(struct pci_dev *pdev, int state) } EXPORT_SYMBOL(pci_disable_link_state); +/** + * pci_enable_default_link_state - Clears and sets the default device link state + * so that the link may be allowed to enter the specified states. Note that + * if the BIOS didn't grant ASPM control to the OS, this does nothing because + * we can't touch the LNKCTL register. Also note that this does not enable + * states disabled by pci_disable_link_state(). Returns 0 or a negative errno. + * + * @pdev: PCI device + * @state: Mask of ASPM link states to enable + */ +int pci_enable_default_link_state(struct pci_dev *pdev, int state) +{ + struct pcie_link_state *link = pcie_aspm_get_link(pdev); + + if (!link) + return -EINVAL; + /* + * A driver requested that ASPM be enabled on this device, but + * if we don't have permission to manage ASPM (e.g., on ACPI + * systems we have to observe the FADT ACPI_FADT_NO_ASPM bit and + * the _OSC method), we can't honor that request. + */ + if (aspm_disabled) { + pci_warn(pdev, "can't override BIOS ASPM; OS doesn't have ASPM control\n"); + return -EPERM; + } + + down_read(&pci_bus_sem); + mutex_lock(&aspm_lock); + link->aspm_default = 0; + if (state & PCIE_LINK_STATE_L0S) + link->aspm_default |= ASPM_STATE_L0S; + if (state & PCIE_LINK_STATE_L1) + /* L1 PM substates require L1 */ + link->aspm_default |= ASPM_STATE_L1 | ASPM_STATE_L1SS; + if (state & PCIE_LINK_STATE_L1_1) + link->aspm_default |= ASPM_STATE_L1_1; + if (state & PCIE_LINK_STATE_L1_2) + link->aspm_default |= ASPM_STATE_L1_2; + if (state & PCIE_LINK_STATE_L1_1_PCIPM) + link->aspm_default |= ASPM_STATE_L1_1_PCIPM; + if (state & PCIE_LINK_STATE_L1_2_PCIPM) + link->aspm_default |= ASPM_STATE_L1_2_PCIPM; + pcie_config_aspm_link(link, policy_to_aspm_state(link)); + + link->clkpm_default = (state & PCIE_LINK_STATE_CLKPM) ? 1 : 0; + pcie_set_clkpm(link, policy_to_clkpm_state(link)); + mutex_unlock(&aspm_lock); + up_read(&pci_bus_sem); + + return 0; +} +EXPORT_SYMBOL(pci_enable_default_link_state); + static int pcie_aspm_set_policy(const char *val, const struct kernel_param *kp) { diff --git a/include/linux/pci.h b/include/linux/pci.h index 8253a5413d7c..fd710afe0209 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1641,10 +1641,15 @@ extern bool pcie_ports_native; #define PCIE_LINK_STATE_L1_2 BIT(4) #define PCIE_LINK_STATE_L1_1_PCIPM BIT(5) #define PCIE_LINK_STATE_L1_2_PCIPM BIT(6) +#define PCIE_LINK_STATE_ALL (PCIE_LINK_STATE_L0S | PCIE_LINK_STATE_L1 |\ + PCIE_LINK_STATE_CLKPM | PCIE_LINK_STATE_L1_1 |\ + PCIE_LINK_STATE_L1_2 | PCIE_LINK_STATE_L1_1_PCIPM |\ + PCIE_LINK_STATE_L1_2_PCIPM) #ifdef CONFIG_PCIEASPM int pci_disable_link_state(struct pci_dev *pdev, int state); int pci_disable_link_state_locked(struct pci_dev *pdev, int state); +int pci_enable_default_link_state(struct pci_dev *pdev, int state); void pcie_no_aspm(void); bool pcie_aspm_support_enabled(void); bool pcie_aspm_enabled(struct pci_dev *pdev); @@ -1653,6 +1658,8 @@ static inline int pci_disable_link_state(struct pci_dev *pdev, int state) { return 0; } static inline int pci_disable_link_state_locked(struct pci_dev *pdev, int state) { return 0; } +static inline int pci_enable_default_link_state(struct pci_dev *pdev, int state) +{ return 0; } static inline void pcie_no_aspm(void) { } static inline bool pcie_aspm_support_enabled(void) { return false; } static inline bool pcie_aspm_enabled(struct pci_dev *pdev) { return false; } From patchwork Tue Mar 1 04:19:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "David E. Box" X-Patchwork-Id: 12764046 X-Patchwork-Delegate: lorenzo.pieralisi@arm.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F4079C433EF for ; Tue, 1 Mar 2022 04:20:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232229AbiCAEVR (ORCPT ); Mon, 28 Feb 2022 23:21:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232331AbiCAEVH (ORCPT ); Mon, 28 Feb 2022 23:21:07 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0EE272F00E; Mon, 28 Feb 2022 20:19:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1646108394; x=1677644394; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jVLzZIkE06jbqrtxiCO2PmKZxUVeTSb1DUOuP2uR+ZY=; b=QA0RyfD37RTtapH5eQUCZutZvoYPNiuaQbclisSUZqvVI6X3RNfXq5wi eAA2K7NCXDH2Y4u6Q1ZchOO+rJqj6c/HYqP2wnBxEF2qVH6r6VMTKM/iw PFdG2LxpPaZ/jTReMTZO8SMHOYxD5567vbLl5K8QDI2IgSDWVslrtg5So 1JZo40sZlZ17KLUwlAiJySVrlNZ2/XeAW7un3XxxvvHHfW5OGdc0CIEQz 4+nWxTB8+xKGyVNyeGPU3XjkzyIidrulyLJeodkeOD6b1SsHvATRDHMDq IUdRzHBubYkcRUUGH4cx8WnlfoJQckKFx5l69ylaeZGdOyoxF1XYJAYnz Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10272"; a="253243225" X-IronPort-AV: E=Sophos;i="5.90,144,1643702400"; d="scan'208";a="253243225" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Feb 2022 20:19:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,144,1643702400"; d="scan'208";a="550546074" Received: from linux.intel.com ([10.54.29.200]) by orsmga008.jf.intel.com with ESMTP; 28 Feb 2022 20:19:44 -0800 Received: from debox1-desk4.lan (unknown [10.251.9.202]) by linux.intel.com (Postfix) with ESMTP id 1965B5808DB; Mon, 28 Feb 2022 20:19:44 -0800 (PST) From: "David E. Box" To: nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev, lorenzo.pieralisi@arm.com, hch@infradead.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, david.e.box@linux.intel.com, michael.a.bottini@linux.intel.com, rafael@kernel.org, me@adhityamohan.in Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V6 2/3] PCI: vmd: Add vmd_device_data Date: Mon, 28 Feb 2022 20:19:42 -0800 Message-Id: <20220301041943.2935892-3-david.e.box@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220301041943.2935892-1-david.e.box@linux.intel.com> References: <20220301041943.2935892-1-david.e.box@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Add vmd_device_data to allow adding additional info for driver data. Also refactor the PCI ID list to use PCI_VDEVICE. Signed-off-by: David E. Box --- V6 - Inline the declarations for driver data in the vmd_ids list. Suggested by Jonathan V5 - New patch drivers/pci/controller/vmd.c | 76 ++++++++++++++++++++++++------------ 1 file changed, 50 insertions(+), 26 deletions(-) diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index cc166c683638..cde6e2cba210 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -69,6 +69,10 @@ enum vmd_features { VMD_FEAT_CAN_BYPASS_MSI_REMAP = (1 << 4), }; +struct vmd_device_data { + enum vmd_features features; +}; + static DEFINE_IDA(vmd_instance_ida); /* @@ -710,11 +714,12 @@ static void vmd_copy_host_bridge_flags(struct pci_host_bridge *root_bridge, vmd_bridge->native_dpc = root_bridge->native_dpc; } -static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features) +static int vmd_enable_domain(struct vmd_dev *vmd, struct vmd_device_data *info) { struct pci_sysdata *sd = &vmd->sysdata; struct resource *res; u32 upper_bits; + unsigned long features = info->features; unsigned long flags; LIST_HEAD(resources); resource_size_t offset[2] = {0}; @@ -881,7 +886,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features) static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id) { - unsigned long features = (unsigned long) id->driver_data; + struct vmd_device_data *info = (struct vmd_device_data *)id->driver_data; + unsigned long features = info->features; struct vmd_dev *vmd; int err; @@ -925,7 +931,7 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id) spin_lock_init(&vmd->cfg_lock); pci_set_drvdata(dev, vmd); - err = vmd_enable_domain(vmd, features); + err = vmd_enable_domain(vmd, info); if (err) goto out_release_instance; @@ -994,29 +1000,47 @@ static int vmd_resume(struct device *dev) static SIMPLE_DEV_PM_OPS(vmd_dev_pm_ops, vmd_suspend, vmd_resume); static const struct pci_device_id vmd_ids[] = { - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_201D), - .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP,}, - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_28C0), - .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW | - VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_CAN_BYPASS_MSI_REMAP,}, - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x467f), - .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | - VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR,}, - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x4c3d), - .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | - VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR,}, - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xa77f), - .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | - VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR,}, - {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B), - .driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | - VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR,}, - {0,} + { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_201D), + (kernel_ulong_t)&(struct vmd_device_data) { + .features = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP, + }, + }, + { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_28C0), + (kernel_ulong_t)&(struct vmd_device_data) { + .features = VMD_FEAT_HAS_MEMBAR_SHADOW | + VMD_FEAT_HAS_BUS_RESTRICTIONS | + VMD_FEAT_CAN_BYPASS_MSI_REMAP, + }, + }, + { PCI_VDEVICE(INTEL, 0x467f), + (kernel_ulong_t)&(struct vmd_device_data) { + .features = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | + VMD_FEAT_HAS_BUS_RESTRICTIONS | + VMD_FEAT_OFFSET_FIRST_VECTOR, + }, + }, + { PCI_VDEVICE(INTEL, 0x4c3d), + (kernel_ulong_t)&(struct vmd_device_data) { + .features = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | + VMD_FEAT_HAS_BUS_RESTRICTIONS | + VMD_FEAT_OFFSET_FIRST_VECTOR, + }, + }, + { PCI_VDEVICE(INTEL, 0xa77f), + (kernel_ulong_t)&(struct vmd_device_data) { + .features = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | + VMD_FEAT_HAS_BUS_RESTRICTIONS | + VMD_FEAT_OFFSET_FIRST_VECTOR, + }, + }, + { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B), + (kernel_ulong_t)&(struct vmd_device_data) { + .features = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | + VMD_FEAT_HAS_BUS_RESTRICTIONS | + VMD_FEAT_OFFSET_FIRST_VECTOR, + }, + }, + { } }; MODULE_DEVICE_TABLE(pci, vmd_ids); From patchwork Tue Mar 1 04:19:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "David E. Box" X-Patchwork-Id: 12764043 X-Patchwork-Delegate: lorenzo.pieralisi@arm.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE8E4C433F5 for ; Tue, 1 Mar 2022 04:19:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232200AbiCAEU2 (ORCPT ); Mon, 28 Feb 2022 23:20:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47364 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232251AbiCAEU1 (ORCPT ); Mon, 28 Feb 2022 23:20:27 -0500 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 728232B24F; Mon, 28 Feb 2022 20:19:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1646108385; x=1677644385; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4t+PvdGlz2cLuHS96n7ALdgqcJnX7IA3lO1HM5B6eiM=; b=Nj2NKC3EaLpQD3OCUtg+B0vv69naBb4uZxFd8F8R4rfxluMY+mdsGLwh TFwM8Mz7N/gSyIOL7dz8FjZufpGgDn14zcdgriu7vOLZAUf/LVH2L6RgC HMIaegnQa5bINWWW9u6l2BcLhYah9hYRJeQO3eTh7u3Q2CCvyyBZ5X2jN EAzaAVw/lKj2Mu0CkUwRVHvE4u3XYtwl9F+/7+avTbhG/cmQixC76wnu4 YtCjjzEPhQiexBfi/9B4PFbqAIiQ2aO4ESdJ7sTG+SwJuBoAIktshOHtM 5Hg4neycUdb/KbzfC3if/Ms1knehXTQXVFCa8Hggx9DdhvWY6ssNfwEjs Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10272"; a="250634178" X-IronPort-AV: E=Sophos;i="5.90,144,1643702400"; d="scan'208";a="250634178" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Feb 2022 20:19:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,144,1643702400"; d="scan'208";a="507667403" Received: from linux.intel.com ([10.54.29.200]) by orsmga002.jf.intel.com with ESMTP; 28 Feb 2022 20:19:44 -0800 Received: from debox1-desk4.lan (unknown [10.251.9.202]) by linux.intel.com (Postfix) with ESMTP id 73195580C20; Mon, 28 Feb 2022 20:19:44 -0800 (PST) From: "David E. Box" To: nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev, lorenzo.pieralisi@arm.com, hch@infradead.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, david.e.box@linux.intel.com, michael.a.bottini@linux.intel.com, rafael@kernel.org, me@adhityamohan.in Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V6 3/3] PCI: vmd: Configure PCIe ASPM and LTR Date: Mon, 28 Feb 2022 20:19:43 -0800 Message-Id: <20220301041943.2935892-4-david.e.box@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220301041943.2935892-1-david.e.box@linux.intel.com> References: <20220301041943.2935892-1-david.e.box@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org PCIe ports reserved for VMD use are not visible to BIOS and therefore not configured to enable PCIE ASPM. Additionally, PCIE LTR values may be left unset since BIOS will set a default maximum LTR value on endpoints to ensure that they don't block SoC power management. Lack of this programming results in high power consumption on laptops as reported in bugzilla [1]. For currently affected products, use pci_enable_default_link_state to set the allowed link states for devices on the root ports. Also set the LTR value to the maximum value needed for the SoC. Per the VMD hardware team future products using VMD will enable BIOS configuration of these capabilities. This solution is a workaround for current products that mainly targets laptops. Support is not provided if a switch used nor for hotplug. [1] https://bugzilla.kernel.org/show_bug.cgi?id=213717 Signed-off-by: Michael Bottini Signed-off-by: David E. Box --- V6 - Set ASPM first before setting LTR. This is needed because some devices may only have LTR set by BIOS and not ASPM - Skip setting the LTR if the current LTR in non-zero. V5 - Provide the LTR value as driver data. - Use DWORD for the config space write to avoid PCI WORD access bug. - Set ASPM links firsts, enabling all link states, before setting a default LTR if the capability is present - Add kernel message that VMD is setting the device LTR. V4 - Refactor vmd_enable_apsm() to exit early, making the lines shorter and more readable. Suggested by Christoph. V3 - No changes V2 - Use return status to print pci_info message if ASPM cannot be enabled. - Add missing static declaration, caught by lkp@intel.com drivers/pci/controller/vmd.c | 66 +++++++++++++++++++++++++++++++++--- 1 file changed, 62 insertions(+), 4 deletions(-) diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index cde6e2cba210..8525bb8312f2 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -67,10 +67,19 @@ enum vmd_features { * interrupt handling. */ VMD_FEAT_CAN_BYPASS_MSI_REMAP = (1 << 4), + + /* + * Enable ASPM on the PCIE root ports and set the default LTR of the + * storage devices on platforms where these values are not configured by + * BIOS. This is needed for laptops, which require these settings for + * proper power management of the SoC. + */ + VMD_FEAT_BIOS_PM_QUIRK = (1 << 5), }; struct vmd_device_data { enum vmd_features features; + u16 ltr; }; static DEFINE_IDA(vmd_instance_ida); @@ -714,6 +723,45 @@ static void vmd_copy_host_bridge_flags(struct pci_host_bridge *root_bridge, vmd_bridge->native_dpc = root_bridge->native_dpc; } +/* + * Enable ASPM and LTR settings on devices that aren't configured by BIOS. + */ +static int vmd_pm_enable_quirk(struct pci_dev *pdev, void *userdata) +{ + struct vmd_device_data *info = userdata; + u32 ltr_reg; + int pos; + + if (!(info->features & VMD_FEAT_BIOS_PM_QUIRK)) + return 0; + + pci_enable_default_link_state(pdev, PCIE_LINK_STATE_ALL); + + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_LTR); + if (!pos) + return 0; + + /* + * Skip if the max snoop LTR is non-zero, indicating BIOS has set it + * so the LTR quirk is not needed. + */ + pci_read_config_dword(pdev, pos + PCI_LTR_MAX_SNOOP_LAT, <r_reg); + if (!!(ltr_reg & (PCI_LTR_VALUE_MASK | PCI_LTR_SCALE_MASK))) + return 0; + + /* + * Set the default values to the maximum required by the platform to + * allow the deepest power management savings. Write as a DWORD where + * the lower word is the max snoop latency and the upper word is the + * max non-snoop latency. + */ + ltr_reg = (info->ltr << 16) | info->ltr; + pci_write_config_dword(pdev, pos + PCI_LTR_MAX_SNOOP_LAT, ltr_reg); + pci_info(pdev, "VMD: Default LTR set\n"); + + return 0; +} + static int vmd_enable_domain(struct vmd_dev *vmd, struct vmd_device_data *info) { struct pci_sysdata *sd = &vmd->sysdata; @@ -867,6 +915,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd, struct vmd_device_data *info) pci_reset_bus(child->self); pci_assign_unassigned_bus_resources(vmd->bus); + pci_walk_bus(vmd->bus, vmd_pm_enable_quirk, info); + /* * VMD root buses are virtual and don't return true on pci_is_pcie() * and will fail pcie_bus_configure_settings() early. It can instead be @@ -1016,28 +1066,36 @@ static const struct pci_device_id vmd_ids[] = { (kernel_ulong_t)&(struct vmd_device_data) { .features = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR, + VMD_FEAT_OFFSET_FIRST_VECTOR | + VMD_FEAT_BIOS_PM_QUIRK, + .ltr = 0x1003, /* 3145728 ns */ }, }, { PCI_VDEVICE(INTEL, 0x4c3d), (kernel_ulong_t)&(struct vmd_device_data) { .features = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR, + VMD_FEAT_OFFSET_FIRST_VECTOR | + VMD_FEAT_BIOS_PM_QUIRK, + .ltr = 0x1003, /* 3145728 ns */ }, }, { PCI_VDEVICE(INTEL, 0xa77f), (kernel_ulong_t)&(struct vmd_device_data) { .features = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR, + VMD_FEAT_OFFSET_FIRST_VECTOR | + VMD_FEAT_BIOS_PM_QUIRK, + .ltr = 0x1003, /* 3145728 ns */ }, }, { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B), (kernel_ulong_t)&(struct vmd_device_data) { .features = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | VMD_FEAT_HAS_BUS_RESTRICTIONS | - VMD_FEAT_OFFSET_FIRST_VECTOR, + VMD_FEAT_OFFSET_FIRST_VECTOR | + VMD_FEAT_BIOS_PM_QUIRK, + .ltr = 0x1003, /* 3145728 ns */ }, }, { }