From patchwork Tue Feb 4 05:37:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13958661 Received: from out199-2.us.a.mail.aliyun.com (out199-2.us.a.mail.aliyun.com [47.90.199.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7AAD25A626; Tue, 4 Feb 2025 06:09:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=47.90.199.2 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738649353; cv=none; b=NYuchFqgvBXnjanIiYLJe6N6gpJclmnJiu6iyWr1eFw4cCSqoh84yAqqXBZ7HMU05RIEkEGsnGdl7lQGQ4PZAwq7Z4g2KwPhW1aWIOmX/edP8aNngTqdi+ajo3XtctBtNCRTDOltnQrVEmReCAQ3dM/BaMWXDrTR8MNxVoAORxM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738649353; c=relaxed/simple; bh=V7odFs4tcKIWLBm/ALEmvYBCpIMoMGf+iuY56dgvY2E=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=nvlke+U8PlrJIELuX8HOKZjCw2vtJ5jEnWRcwJ23F7VI9/f1XYCySVyO5fPRcxuFx0OXRNmO57H7hX6dHL4PPQJC/hupYBB7LplREaEnoWBYSio054NmXRFjT41PLeBrmjvtL7sZKOc0SQBO8/nzB58CAgK8SgNxwJIreniLH+E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=KcjTDZB1; arc=none smtp.client-ip=47.90.199.2 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="KcjTDZB1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1738649337; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=bSrmfWIDiCB2JPQM4jHmdym0boCxfT1KwmWYr/hPKro=; b=KcjTDZB1zFv39Qa/WRGqMPaF72mgaFN5GcElWQD9IDfq9NZuXROlHjqFj7h49DgicLCTX2Ez2ivuXXLUgOrtqywTfbIOaOhGK9labSpfxzvDndOtTlXioHdo7q/OKE/lU/pN2PB/rLvfBLVqd+t/bVGJHdNrz8giFcFi0kiHtdE= Received: from localhost(mailfrom:feng.tang@linux.alibaba.com fp:SMTPD_---0WOjUGtb_1738647478 cluster:ay36) by smtp.aliyun-inc.com; Tue, 04 Feb 2025 13:37:59 +0800 From: Feng Tang To: Bjorn Helgaas Cc: Jonathan Cameron , ilpo.jarvinen@linux.intel.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH 1/2] PCI/portdrv: Add necessary delay for disabling hotplug events Date: Tue, 4 Feb 2025 13:37:57 +0800 Message-Id: <20250204053758.6025-1-feng.tang@linux.alibaba.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 According to PCIe 6.1 spec, section 6.7.3.2, software need to wait at least 1 second for the command-complete event, before resending the cmd or sending a new cmd. Currently get_port_device_capability() sends slot control cmd to disable PCIe hotplug interrupts without waiting for its completion and there was real problem reported for the lack of waiting. Add the necessary wait to comply with PCIe spec. The waiting logic refers existing pcie_poll_cmd(). Signed-off-by: Feng Tang --- drivers/pci/pci.h | 2 ++ drivers/pci/pcie/portdrv.c | 33 +++++++++++++++++++++++++++++++-- 2 files changed, 33 insertions(+), 2 deletions(-) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 01e51db8d285..c1e234d1b81d 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -759,12 +759,14 @@ static inline void pcie_ecrc_get_policy(char *str) { } #ifdef CONFIG_PCIEPORTBUS void pcie_reset_lbms_count(struct pci_dev *port); int pcie_lbms_count(struct pci_dev *port, unsigned long *val); +void pcie_disable_hp_interrupts_early(struct pci_dev *dev); #else static inline void pcie_reset_lbms_count(struct pci_dev *port) {} static inline int pcie_lbms_count(struct pci_dev *port, unsigned long *val) { return -EOPNOTSUPP; } +static inline void pcie_disable_hp_interrupts_early(struct pci_dev *dev) {} #endif struct pci_dev_reset_methods { diff --git a/drivers/pci/pcie/portdrv.c b/drivers/pci/pcie/portdrv.c index 02e73099bad0..16010973bfe2 100644 --- a/drivers/pci/pcie/portdrv.c +++ b/drivers/pci/pcie/portdrv.c @@ -18,6 +18,7 @@ #include #include #include +#include #include "../pci.h" #include "portdrv.h" @@ -205,6 +206,35 @@ static int pcie_init_service_irqs(struct pci_dev *dev, int *irqs, int mask) return 0; } +static int pcie_wait_sltctl_cmd_raw(struct pci_dev *pdev) +{ + u16 slot_status; + /* 1000 ms, according toPCIe spec 6.1, section 6.7.3.2 */ + int timeout = 1000; + + do { + pcie_capability_read_word(pdev, PCI_EXP_SLTSTA, &slot_status); + if (slot_status & PCI_EXP_SLTSTA_CC) { + pcie_capability_write_word(pdev, PCI_EXP_SLTSTA, + PCI_EXP_SLTSTA_CC); + return 0; + } + msleep(10); + timeout -= 10; + } while (timeout); + + /* Timeout */ + return -1; +} + +void pcie_disable_hp_interrupts_early(struct pci_dev *dev) +{ + pcie_capability_clear_word(dev, PCI_EXP_SLTCTL, + PCI_EXP_SLTCTL_CCIE | PCI_EXP_SLTCTL_HPIE); + if (pcie_wait_sltctl_cmd_raw(dev)) + pci_info(dev, "Timeout on disabling hot-plug interrupts\n"); +} + /** * get_port_device_capability - discover capabilities of a PCI Express port * @dev: PCI Express port to examine @@ -230,8 +260,7 @@ static int get_port_device_capability(struct pci_dev *dev) * Disable hot-plug interrupts in case they have been enabled * by the BIOS and the hot-plug service driver is not loaded. */ - pcie_capability_clear_word(dev, PCI_EXP_SLTCTL, - PCI_EXP_SLTCTL_CCIE | PCI_EXP_SLTCTL_HPIE); + pcie_disable_hp_interrupts_early(dev); } #ifdef CONFIG_PCIEAER From patchwork Tue Feb 4 05:37:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13958620 Received: from out199-10.us.a.mail.aliyun.com (out199-10.us.a.mail.aliyun.com [47.90.199.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B3AF139579; Tue, 4 Feb 2025 05:53:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=47.90.199.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738648435; cv=none; b=uVhhkq+LxaJPDR1jpUm/3pMVDHRUQRGajiHMHmWY7sOxABUnzKoAQrRjXNAyLGxWMxPBInA1HCvVMUfp2LbKunpvuGRejGk1h9RtWXpuXGxQMFVTEMK+az76VVzQvFR1Wv+Ch2O47i7pGNdwmng/+a73lwd3vk1xHeXpKBQ1pF8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738648435; c=relaxed/simple; bh=iCct0DBbP2W1vpi6THSTibm+AfRcjlGe0lKnkdNC8X8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QzR3odS+cKJsts4/CtHPJt7VojZ1ZfPQoR5Cuv9TPDb1O+1XBsWrmocW9mveG+nbm9cjnohX7nCwiqWmpTkVGcgWsy+a8Q3HH0ULAZ2nKB6LNetZ+KNTNxS4WIhoLQUMigYmGy2QIm2cexe3/44DeBmV03fQ0MI73/xwP7biTgg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=H7GPRfKI; arc=none smtp.client-ip=47.90.199.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="H7GPRfKI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1738648418; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=74IslISy4mrgKNiJWv1O1Q22rD/RFRZQPMLY7Tnw3a4=; b=H7GPRfKIQmgQGgLVsOEZSYmGv5JXh768BThMiv93ezFWbezU77TALKx7r+B26TL5WibStCBGnj8Z6ktMlh18VDkhq7mXYbHF4o0MzCqAaSjSgDXhhtyfFJ3Sy0nZV5DXUFDhSW12+otpKn9aA42dSG5TW0AIfKqaCbuh3kyPpzY= Received: from localhost(mailfrom:feng.tang@linux.alibaba.com fp:SMTPD_---0WOjXThs_1738647479 cluster:ay36) by smtp.aliyun-inc.com; Tue, 04 Feb 2025 13:37:59 +0800 From: Feng Tang To: Bjorn Helgaas Cc: Jonathan Cameron , ilpo.jarvinen@linux.intel.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH 2/2] PCI: Disable PCIE hotplug interrupts early when msi is disabled Date: Tue, 4 Feb 2025 13:37:58 +0800 Message-Id: <20250204053758.6025-2-feng.tang@linux.alibaba.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250204053758.6025-1-feng.tang@linux.alibaba.com> References: <20250204053758.6025-1-feng.tang@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 There was a irq storm bug when testing "pci=nomsi" case, and the root cause is: 'nomsi' will disable MSI and let devices and root ports use legacy INTX inerrupt, and likely make several devices/ports share one interrupt. In the failure case, BIOS doesn't disable the PCIE hotplug interrupts, and actually asserts the command-complete interrupt. As MSI is disabled, ACPI initialization code will not enumerate root port's PCIE hotplug capability, and pciehp service driver wont' be enabled for the root port to handle that interrupt, later on when it is shared and enabled by other device driver like NVME or NIC, the "nobody care irq storm" happens. So disable the pcie hotplug CCIE/HPIE interrupt in early boot phase when MSI is not enbaled. Signed-off-by: Feng Tang --- drivers/pci/probe.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index b6536ed599c3..10d72156da9a 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1664,6 +1664,15 @@ void set_pcie_hotplug_bridge(struct pci_dev *pdev) pcie_capability_read_dword(pdev, PCI_EXP_SLTCAP, ®32); if (reg32 & PCI_EXP_SLTCAP_HPC) pdev->is_hotplug_bridge = 1; + + /* + * When MSI is disabled, root port will use legacy INTX, and likely + * share INTX interrupt line with other devices like NIC/NVME. There + * was real world issue that the CCIE IRQ is asserted afer boot, but + * will not be handled well and cause IRQ storm. So disable it early. + */ + if (!pci_msi_enabled()) + pcie_disable_hp_interrupts_early(pdev); } static void set_pcie_thunderbolt(struct pci_dev *dev)