From patchwork Thu Mar 27 01:47:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2071.outbound.protection.outlook.com [40.107.244.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A663B169AE6; Thu, 27 Mar 2025 01:47:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.71 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040062; cv=fail; b=PjzOFU5tuGl0RBGG6XPpEp9Dp8W1uh/WhElav0Twt/esoNkGMhdP1vp0Qt8xB576ANBjx2VsO94HlDTX/Y4tc9/TA5gkVvEO0ODS7Gr8Ko4yperPvJwZCYC4XapKpvE20xruB4hpXcpVhyY5lybhNI6FNn6H/T1YdhuBlMR2FQI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040062; c=relaxed/simple; bh=N7QditvkKZ0vgfJsq02Vy0Vo4tQLeoqVN2iJGdt+sjY=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=t33GdfUABvPDBHtqi2N9fm7y8BNWNAwnV6KzcyEvXmY+W0mTPzoOVzjBFXrYJCOgPHqn6HlJTKQ+VV6zqonpAM83HhjiFa8hIzwW22A4XSLQTDTyQHby6VPis3dPfijDr3aBqz8wVF42OSZqiXu+MwsqdDXo40zDKmh0TjNEfyY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Dh1FS3Gz; arc=fail smtp.client-ip=40.107.244.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Dh1FS3Gz" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=sIjsPTLHAgq4itbBtXGJkPAZmFJdCMRjifUp7pp6hmxoQwIsBLKrH1eBRUh5EWBbN4METVJtuUMbqEItNPgakKGVwwIzHW752a4E4puuxfrDwpvAwPBaJQW3BLCYOhAiItGNybWnwFvNw1oyLbX2RrX5vusHUjICFz3NZuwuyDdgif7GvoqsOQdUNk8qGxit4bIhBHrsnzAGpHBS+bV8yKagoDRtft8s7CjF+Jo1fUiTzkhr6JWV8Ai/RfeAkiFR3FpurrzleDBXU19XgEK+ExJXdxqSX1/66EhTeiU4DoSVLjzUTDF5DaPKU9A5FwECkUPO+vSeFM9CVMj4GpRL5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=88PTImvXJu0Nh+jE/jX2UbJN5DaUdt3mwrlHRQcMWSQ=; b=MtNmWXFUtCsDi5YYktSZqxwViTONRAGY6UtdniTCn4g4mHdrsyBIcJAOjcqO4w/PLupCm2vDWEScrbilRv3PEEs2bsGcGqv4RUu/LWV4QjeNPqMU4t/xFAON5Th7bFR8pF1GyN9l2E7wkump9/ILCBRZrvFNXiDJOztCK4XL+9YGhkGJkpvIsU1GWJNS8lkwH8tYZ/rqaLPZyV2KKuCKLbDjYu9Tj3ff7jYXnA/vbY0gPh7qh0DSztWVadMoQnkJJ/5Knqz5hc5EZnqBosvnRSkdmKBbBIfcBFkngRz5qAdfBGDqAVNxRtVSfW0jWexnNKiyrMFsO3gfmuT9zkvPfQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=88PTImvXJu0Nh+jE/jX2UbJN5DaUdt3mwrlHRQcMWSQ=; b=Dh1FS3GzBqgRtjftc7NNcNw7akgtNd3JrwHrc8xyGcY5d1xBIjHGoiO4YlCLo3PYv0BVBdEuQYuLqB+6VZOza3IJm9/2tw47axNiJ6WBraK6arvQSbwmpnjVRK/zCwRdOGUACgbKmQmjEKXgL0x59aCEqbkdrjOKodAtaCETzVw= Received: from MN2PR05CA0009.namprd05.prod.outlook.com (2603:10b6:208:c0::22) by MW4PR12MB7264.namprd12.prod.outlook.com (2603:10b6:303:22e::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Thu, 27 Mar 2025 01:47:36 +0000 Received: from BL6PEPF00022571.namprd02.prod.outlook.com (2603:10b6:208:c0:cafe::93) by MN2PR05CA0009.outlook.office365.com (2603:10b6:208:c0::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8583.27 via Frontend Transport; Thu, 27 Mar 2025 01:47:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022571.mail.protection.outlook.com (10.167.249.39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:47:35 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:47:34 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 01/16] PCI/CXL: Introduce PCIe helper function pcie_is_cxl() Date: Wed, 26 Mar 2025 20:47:02 -0500 Message-ID: <20250327014717.2988633-2-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022571:EE_|MW4PR12MB7264:EE_ X-MS-Office365-Filtering-Correlation-Id: 5370a4dc-19f1-49b5-385a-08dd6cd15941 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|36860700013|82310400026|921020; X-Microsoft-Antispam-Message-Info: ujPsz5zOIgeADq8J+wr/PtOoh+5WnFqzGi+VwLbFPbj6wMg4twG1bxfmog76e0zFvXCjXJYh9N6HXG5fdu9n3evPswk2fTE55bLEImfO7G9fz1wUuDQIWsnA5o/bEkD51+G1+UFEZb0fnELrJCrmHrurar811vSEbkz4vQYVXbA2mH8L8nBX+eJF3Gjd83C5p6yrSFl3cZWCs7xYZ0nGpdmxd0obl50qVSTitqnas/mnZ8dpPG60PNRnqvw1nCbeUvCuzOg4wR5rG5TnQvYE7SUKI7ciyE+YetaLD3d+dlhtAucO8ebJB62ksG/v56nNQqcoSKFQcCCUDxsnxvk1qUYKI9LzTkQjzezSuHCDTCEt24F+aWXv95lRawnH2WYIHcw/bF8QN9AhwMraw0WJ87mY6hQLWgNqSSJxVOzH2cWy806mJLbQt4SaOr5nPKzon425YykqTkMbCWap9umE1GGuUv3Mp1N2OSc0sdJLp54ielYlX5Qtbs1R/IGB79pK+uGXrVBko7dTT7QcY7Q4IguDln6cmZ9U8t5K8tQee2brybMKxSjDgkC40wxcK8G2PECJRXf/rQeOscSCyxtPWn6XN1f7CCq7lN8WYVXl69b6l59dZOWJKDS7P+eflGZRTOz2m5kLb2qejlSnK6bFTlfkcQMQ3ROlNvAhVpQ5JO85Mulnmij7/6KuW7joAzasmDw8dgvzhLgyafeybsTFzs1dHh4+wdX0/Kod7kSQDg8tHDzl2ek+V5mt7FRfg1YBLHvbz/RioFJN/W3b74+3r17kv3QFSc9LzAsNR00/9xX/VJ/4hRW9jsqEdKr6JOZvnM40u1ANpY+CtlGu8CK/aM1nHYKbN/KPCPrqudVJYZTrrNOaJY73raw7KRjUVuP8odAc3pDu4g5rycfm8hxTu516++ZBWmzTPaY4bVCzPZpNbqTksDXwtAhaB1X8K/2e/Is7j9P1FdtBlmwcumk9PXyeBIpvHdBMoGch86k0F5evRQqh2nBbz35pnQgX3lu2y4p7G0G+t8TtonZTvC0yiCmlpv/GZQQIpEORQMrK/EXFRL8UjHZLOOQ1kRD3O5JhYrGCDnxmT6X50Mq7AotAkXesnLjDAEbkwKs4rd8blyPBKoBKSpFHn4ul7x4HAPcH4t8rcG+frNcE+gGK4sgutnpzjOHeTVLoNOeEhl0j/ugGK2qHiyK/OvMJ8ePyYymmWvg8rE8tb2VIQbJKJfDfITneTlbrZjRfYb05QabRQGM0Cwv1UZNIa3Ql0MKjOIAHCu7wiYQJsw4ctSFLHtuEwOtP8plET9pQ2BfMNGxGvxdEmUcV/+MEMm9+csjxrCFoS4o4cWRGD9BILfLkZfbcwKtQo0f3ZLlyhAWwqKZxh776yC8H60qAnFqhpknNcHzR8vmd+rbLdkaHJ0Yo+npnIdYXyp5dZ8+k9v2VVODPsNzzScrCuwtuXR9qxzbNAk9tVDPf7R335ohdv9yUYYGqry9DXxooewJNtynP60hTedK74CiMluK3omHleimWTe+2 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(36860700013)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:47:35.7577 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5370a4dc-19f1-49b5-385a-08dd6cd15941 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022571.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7264 CXL and AER drivers need the ability to identify CXL devices. Add set_pcie_cxl() with logic checking for CXL Flexbus DVSEC presence. The CXL Flexbus DVSEC presence is used because it is required for all the CXL PCIe devices.[1] Add boolean 'struct pci_dev::is_cxl' with the purpose to cache the CXL Flexbus presence. Add function pcie_is_cxl() to return 'struct pci_dev::is_cxl'. [1] CXL 3.1 Spec, 8.1.1 PCIe Designated Vendor-Specific Extended Capability (DVSEC) ID Assignment, Table 8-2 Signed-off-by: Terry Bowman Reviewed-by: Ira Weiny --- drivers/pci/pci.c | 5 +++++ drivers/pci/probe.c | 10 ++++++++++ include/linux/pci.h | 3 +++ include/uapi/linux/pci_regs.h | 8 +++++++- 4 files changed, 25 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 869d204a70a3..a1d75f40017e 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -5032,6 +5032,11 @@ static u16 cxl_port_dvsec(struct pci_dev *dev) PCI_DVSEC_CXL_PORT); } +inline bool pcie_is_cxl(struct pci_dev *pci_dev) +{ + return pci_dev->is_cxl; +} + static bool cxl_sbr_masked(struct pci_dev *dev) { u16 dvsec, reg; diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index b6536ed599c3..7737b9ce7a83 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1676,6 +1676,14 @@ static void set_pcie_thunderbolt(struct pci_dev *dev) dev->is_thunderbolt = 1; } +static void set_pcie_cxl(struct pci_dev *dev) +{ + u16 dvsec = pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL, + PCI_DVSEC_CXL_FLEXBUS); + if (dvsec) + dev->is_cxl = 1; +} + static void set_pcie_untrusted(struct pci_dev *dev) { struct pci_dev *parent = pci_upstream_bridge(dev); @@ -2006,6 +2014,8 @@ int pci_setup_device(struct pci_dev *dev) /* Need to have dev->cfg_size ready */ set_pcie_thunderbolt(dev); + set_pcie_cxl(dev); + set_pcie_untrusted(dev); if (pci_is_pcie(dev)) diff --git a/include/linux/pci.h b/include/linux/pci.h index 47b31ad724fa..af83230bef1a 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -452,6 +452,7 @@ struct pci_dev { unsigned int is_hotplug_bridge:1; unsigned int shpc_managed:1; /* SHPC owned by shpchp */ unsigned int is_thunderbolt:1; /* Thunderbolt controller */ + unsigned int is_cxl:1; /* Compute Express Link (CXL) */ /* * Devices marked being untrusted are the ones that can potentially * execute DMA attacks and similar. They are typically connected @@ -741,6 +742,8 @@ static inline bool pci_is_vga(struct pci_dev *pdev) return false; } +bool pcie_is_cxl(struct pci_dev *pci_dev); + #define for_each_pci_bridge(dev, bus) \ list_for_each_entry(dev, &bus->devices, bus_list) \ if (!pci_is_bridge(dev)) {} else diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h index 3445c4970e4d..7ccb3b2fcc38 100644 --- a/include/uapi/linux/pci_regs.h +++ b/include/uapi/linux/pci_regs.h @@ -1208,9 +1208,15 @@ #define PCI_DOE_DATA_OBJECT_DISC_RSP_3_PROTOCOL 0x00ff0000 #define PCI_DOE_DATA_OBJECT_DISC_RSP_3_NEXT_INDEX 0xff000000 -/* Compute Express Link (CXL r3.1, sec 8.1.5) */ +/* Compute Express Link (CXL r3.1, sec 8.1) + * + * Note that CXL DVSEC id 3 and 7 to be ignored when the CXL link state + * is "disconnected" (CXL r3.1, sec 9.12.3). Re-enumerate these + * registers on downstream link-up events. + */ #define PCI_DVSEC_CXL_PORT 3 #define PCI_DVSEC_CXL_PORT_CTL 0x0c #define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001 +#define PCI_DVSEC_CXL_FLEXBUS 7 #endif /* LINUX_PCI_REGS_H */ From patchwork Thu Mar 27 01:47:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030701 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2065.outbound.protection.outlook.com [40.107.244.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2AEE9190664; Thu, 27 Mar 2025 01:47:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.65 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040072; cv=fail; b=LLq9rNQ2di5eYgHFOR4oubnlsE+ClIW8VC+ekGmkW7y7dzHJTPfkL4g2Nthoesl5kH7IjQFE1Tw5y8Jl7FvSWSl3Vfi2eXmxnei5XjUF8owatMX5FUsxksmn+hV8EshFPukI3kwL8iDI/nBlozyZTUpwwKyL5BqlFcC1ibsxaTE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040072; c=relaxed/simple; bh=XwvEiS8cDx/EuhtklG73Cp/2T4HpXtjIkooQdFNfbUc=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hKmawSfqN6xmawi7UBltcyvuhuJYjG1+4PaqxdQaZGd2lYmv4YbMJLVx0cBczN6eWmYDH6E4NMCIH++H70XN/z8sm55W9hrBqfGToOqN2VBw0XuWGB31wVbz1wqFW31Cca+u3LBoBof7KSnFsZpvusR1hbTGERxilCRmeDBeg+I= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=3ngZ6wam; arc=fail smtp.client-ip=40.107.244.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="3ngZ6wam" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=yDtuKJXBkDqJFXRJF0YKDYLAk16EHwCrCIu/3/L9VBJBwVGh10bbRe+cKL7G23OHGNVsPEQz5YtLgrH2EsdcvaKWq6MS2awkEayjgCWZluwiOmUpf6lA+JWTwAecFZo/izTraR4WBp2uYbX6Iyb+ZPLZt+gBc6CT6UF4zhMXF8FlrMF29BtKsqcMdo8+3IRFhEqzISNVV+XM3lwY7C0sniLTojInRPv6H5kLhQnSnjH0Kql81ho23Ofr2ISN9mZLsuvOgFtOTlZZ5Tmi+pcXmnLGIYmqDsOIyzZ917evkQK8Pg0CMVq19YRlL+EQ/l7u0UmGOtzlXlNVi4t76kpNVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jH2P37Wz2Xs60Q9HyBzu4urL0zeiyKS7pjZmXbu5+RM=; b=eGHt/+ZPbHpenX5HBQBzKn+DhbnkLE5GiPrhGmRxtc2TQnzlik/+4S5JRsCmwytTIOCXnAuJGdCfe1tBlbeBKYdOAjzfGHIbAPIM+YB6morFa5QGQtp22rPBhV0FXq24GvjO7tY9Qsy5OS7/NzI1Yk/oZa2cqnZqmFjTen9Bg9VkucaozdRrk7sHx6+vuXWCTI5nTM+w7xGbxXPfeYcTLo6heq/cgPjkYD1N3DjGARmzwYvPj5rvsYJc3eKDfOwyrHBwMaPo3kzAivZidx/6jCk+gpYkWlrANpaQ01no+srUYTC9KtllPWX8ucG6IwVli3hqZy8jZiKnx263VWEbcQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jH2P37Wz2Xs60Q9HyBzu4urL0zeiyKS7pjZmXbu5+RM=; b=3ngZ6wam4Q7Bvi9gPg2v5u+iwWWxlrnppNs9iLZ/y54oPex1vfNTzh9Y8Nx26DZrnAgegeFzoYP0MKdypjwW6AINwEVwKNHhk3rzlJKBHd21EqRzAkpfjWQ8ZQ71VG7Th013rPmombSAkpw7Il/aLfgTg7yLH7QHsY8cvBA0LC8= Received: from BL0PR1501CA0021.namprd15.prod.outlook.com (2603:10b6:207:17::34) by IA0PPF170E97DF1.namprd12.prod.outlook.com (2603:10b6:20f:fc04::bc9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Thu, 27 Mar 2025 01:47:47 +0000 Received: from BL6PEPF00022575.namprd02.prod.outlook.com (2603:10b6:207:17:cafe::24) by BL0PR1501CA0021.outlook.office365.com (2603:10b6:207:17::34) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.33 via Frontend Transport; Thu, 27 Mar 2025 01:47:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022575.mail.protection.outlook.com (10.167.249.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:47:47 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:47:46 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 02/16] PCI/AER: Modify AER driver logging to report CXL or PCIe bus error type Date: Wed, 26 Mar 2025 20:47:03 -0500 Message-ID: <20250327014717.2988633-3-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022575:EE_|IA0PPF170E97DF1:EE_ X-MS-Office365-Filtering-Correlation-Id: 36f7651a-e431-49d7-eb40-08dd6cd16018 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|1800799024|376014|7416014|921020; X-Microsoft-Antispam-Message-Info: z1OTxdnmsx/krG7+9WE/vRF26CEdPtw+HsJX7Fs8rJMnvEAq4WWxNuGq0kaKOUSbnv2f0BDEa+INSgVSxI1eEnm+N00KRPCCeFr7B0cPfpuZwdXY8pChmXE//AAjdt6q8Nl0eDvtyUt7uR6zIDe37xcT5cuWs/yFVcH3fDgLYBOCCctWqNT/k09pkAD7GteBMPl7kIbvYVqdanb/jGyRzy7yRFZ+7QqU54y1KkZtnJys1oqOSQ3pnJBr2TOTXNM8OMxAAzGZl9FCYWC9mpfMf/e3SezTNIVC5fhTsBnjBYAIy+GwhoTwUBjo/cH8ee6wq3S/A7iKxmQIe2zKoY4A2o06IZUgh1zWq1pheLU0Rje1LnBDeC8C65Wm3QebLAUWVKkP1wXDGNiRJkHeX2i+pxSOk++XyleA1Cy6GZUpjHu7PxcyikJjjqLk88BvUmK0Tqb8PEy6UZoeu55W9ixgQvQH3nQWJ5ghVItRQfo+8iAcqtqduUh/gI5+nc0LfXmntolA9Zn0WDvDpmqGlDsw1AZS3MfHgq4w8EMT3kcyp8dUbJsrokkJgsWJ9fnzoDIapAbt+rPdtXEwOszninANyW3kaR8F+VZioyDdobp/UaV8UUZJ3MS/Y9jz4NOkwOpN3LcnfTbycZ8KGx0LDGPWhPE/fZmrlSAo8twNuEcC7hnbJg8tAu6oMzdNG3p4qKzM2waG6gVPsIzF2be/jtG7Tmt4wfd9zOB2eqwpOadtEOD1lLcNxqOpcYoCtd0I8aVsZbRG/WKNOtXiUiNhIZbZytA3A7lecME3i1vfeFxKHIHEP5FhBR879YkQg+C1idG8Z4X/4lNzjJQVNB4j3SThIhYvJkXcMnoK0obh7bQVnVfkZqSVvRgEPprI1Bbf10tfW385howGAluOySxrQ1HWuTBoMMtBFpej34afywcGvzyzi7mld5GC+Yl8SFfD7G4QfLHv/do3pfRH68Un5V1TnskDEBBKE1xissOvoLBdodzL5OzWcL36wxSxEkxohjqbsGPvRhvxrRNrOgNsZ2P7ZRTwHwC0NQDDVoyPFaJMp7H/cAKBA1sVK9C0ABKbEseyYNIgL6mp2rr3IWHBvucoO//V/ZhrkgrGcA/mDiPu+aFCvFAPwqT7K/gShUAzasnn+12Zjh3EWGLpl0dUlVD1lRVFlY9H18ZfUN0Ui1euyr8VXjNe9aZ+BDTAL258IYx6/rAsucg0ng/ZbNKlMuumeCZ6Ohsf8OR1Cwx18Rnd96cIMpa3t8dQVlnJJE5okdxHBC4AsDBFvNqLt13lspJy0+LWGXR3oBH5l3BzuYqqQjd5d/liu2nVIOZoRSkSK4pvRDefs+UvbX46Vsq8Jfw8UuudpJd9paQJCw2Y5dKMb2mNxjbMUCccQRvZczQ8SM8NjJS0ev8AKYKT1j37c6fHbc8Zft0QQJ5M1iAIvN4qXxrwlfCK7SPrx+7K7oSanC6QMj0hyZykpp1qg7wzPKaaVsD+/amFcbw2hDpf4lwu5oR2Ly0UZcTPS3NFrSxqpXZX X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(1800799024)(376014)(7416014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:47:47.2376 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 36f7651a-e431-49d7-eb40-08dd6cd16018 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022575.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PPF170E97DF1 The AER service driver and aer_event tracing currently log 'PCIe Bus Type' for all errors. Update the driver and aer_event tracing to log 'CXL Bus Type' for CXL device errors. This requires the AER can identify and distinguish between PCIe errors and CXL errors. Introduce boolean 'is_cxl' to 'struct aer_err_info'. Add assignment in aer_get_device_error_info() and pci_print_aer(). Update the aer_event trace routine to accept a bus type string parameter. Signed-off-by: Terry Bowman Reviewed-by: Ira Weiny --- drivers/pci/pci.h | 6 ++++++ drivers/pci/pcie/aer.c | 18 ++++++++++++------ include/ras/ras_event.h | 9 ++++++--- 3 files changed, 24 insertions(+), 9 deletions(-) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 01e51db8d285..eed098c134a6 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -533,6 +533,7 @@ static inline bool pci_dev_test_and_set_removed(struct pci_dev *dev) struct aer_err_info { struct pci_dev *dev[AER_MAX_MULTI_ERR_DEVICES]; int error_dev_num; + bool is_cxl; unsigned int id:16; @@ -549,6 +550,11 @@ struct aer_err_info { struct pcie_tlp_log tlp; /* TLP Header */ }; +static inline const char *aer_err_bus(struct aer_err_info *info) +{ + return info->is_cxl ? "CXL" : "PCIe"; +} + int aer_get_device_error_info(struct pci_dev *dev, struct aer_err_info *info); void aer_print_error(struct pci_dev *dev, struct aer_err_info *info); diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 508474e17183..83f2069f111e 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -694,13 +694,14 @@ static void __aer_print_error(struct pci_dev *dev, void aer_print_error(struct pci_dev *dev, struct aer_err_info *info) { + const char *bus_type = aer_err_bus(info); int layer, agent; int id = pci_dev_id(dev); const char *level; if (!info->status) { - pci_err(dev, "PCIe Bus Error: severity=%s, type=Inaccessible, (Unregistered Agent ID)\n", - aer_error_severity_string[info->severity]); + pci_err(dev, "%s Bus Error: severity=%s, type=Inaccessible, (Unregistered Agent ID)\n", + bus_type, aer_error_severity_string[info->severity]); goto out; } @@ -709,8 +710,8 @@ void aer_print_error(struct pci_dev *dev, struct aer_err_info *info) level = (info->severity == AER_CORRECTABLE) ? KERN_WARNING : KERN_ERR; - pci_printk(level, dev, "PCIe Bus Error: severity=%s, type=%s, (%s)\n", - aer_error_severity_string[info->severity], + pci_printk(level, dev, "%s Bus Error: severity=%s, type=%s, (%s)\n", + bus_type, aer_error_severity_string[info->severity], aer_error_layer[layer], aer_agent_string[agent]); pci_printk(level, dev, " device [%04x:%04x] error status/mask=%08x/%08x\n", @@ -725,7 +726,7 @@ void aer_print_error(struct pci_dev *dev, struct aer_err_info *info) if (info->id && info->error_dev_num > 1 && info->id == id) pci_err(dev, " Error of this Agent is reported first\n"); - trace_aer_event(dev_name(&dev->dev), (info->status & ~info->mask), + trace_aer_event(dev_name(&dev->dev), bus_type, (info->status & ~info->mask), info->severity, info->tlp_header_valid, &info->tlp); } @@ -759,6 +760,7 @@ EXPORT_SYMBOL_GPL(cper_severity_to_aer); void pci_print_aer(struct pci_dev *dev, int aer_severity, struct aer_capability_regs *aer) { + const char *bus_type; int layer, agent, tlp_header_valid = 0; u32 status, mask; struct aer_err_info info; @@ -780,6 +782,9 @@ void pci_print_aer(struct pci_dev *dev, int aer_severity, info.status = status; info.mask = mask; info.first_error = PCI_ERR_CAP_FEP(aer->cap_control); + info.is_cxl = pcie_is_cxl(dev); + + bus_type = aer_err_bus(&info); pci_err(dev, "aer_status: 0x%08x, aer_mask: 0x%08x\n", status, mask); __aer_print_error(dev, &info); @@ -793,7 +798,7 @@ void pci_print_aer(struct pci_dev *dev, int aer_severity, if (tlp_header_valid) pcie_print_tlp_log(dev, &aer->header_log, dev_fmt(" ")); - trace_aer_event(dev_name(&dev->dev), (status & ~mask), + trace_aer_event(dev_name(&dev->dev), bus_type, (status & ~mask), aer_severity, tlp_header_valid, &aer->header_log); } EXPORT_SYMBOL_NS_GPL(pci_print_aer, "CXL"); @@ -1211,6 +1216,7 @@ int aer_get_device_error_info(struct pci_dev *dev, struct aer_err_info *info) /* Must reset in this function */ info->status = 0; info->tlp_header_valid = 0; + info->is_cxl = pcie_is_cxl(dev); /* The device might not support AER */ if (!aer) diff --git a/include/ras/ras_event.h b/include/ras/ras_event.h index e5f7ee0864e7..1bf8e7050ba8 100644 --- a/include/ras/ras_event.h +++ b/include/ras/ras_event.h @@ -297,15 +297,17 @@ TRACE_EVENT(non_standard_event, TRACE_EVENT(aer_event, TP_PROTO(const char *dev_name, + const char *bus_type, const u32 status, const u8 severity, const u8 tlp_header_valid, struct pcie_tlp_log *tlp), - TP_ARGS(dev_name, status, severity, tlp_header_valid, tlp), + TP_ARGS(dev_name, bus_type, status, severity, tlp_header_valid, tlp), TP_STRUCT__entry( __string( dev_name, dev_name ) + __string( bus_type, bus_type ) __field( u32, status ) __field( u8, severity ) __field( u8, tlp_header_valid) @@ -314,6 +316,7 @@ TRACE_EVENT(aer_event, TP_fast_assign( __assign_str(dev_name); + __assign_str(bus_type); __entry->status = status; __entry->severity = severity; __entry->tlp_header_valid = tlp_header_valid; @@ -325,8 +328,8 @@ TRACE_EVENT(aer_event, } ), - TP_printk("%s PCIe Bus Error: severity=%s, %s, TLP Header=%s\n", - __get_str(dev_name), + TP_printk("%s %s Bus Error: severity=%s, %s, TLP Header=%s\n", + __get_str(dev_name), __get_str(bus_type), __entry->severity == AER_CORRECTABLE ? "Corrected" : __entry->severity == AER_FATAL ? "Fatal" : "Uncorrected, non-fatal", From patchwork Thu Mar 27 01:47:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030702 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2057.outbound.protection.outlook.com [40.107.93.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BE601A239A; Thu, 27 Mar 2025 01:48:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.57 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040085; cv=fail; b=QXVXebo0/cmP4fFJgJsi+1cNJKpZwY6v9S7QFyB8ETskR4P/x3ZARsmUYNGIPSjgSrpWPPjhJRJjUy7pZ26bz0nPnEbUlabNVCRe5VzqUnwJAsEfC2vb662GsuYOWqARc+4lb7kDAlOfmQOT5/fhRp+tVXfdc+Oms4AsPAlChdA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040085; c=relaxed/simple; bh=SBj5d5I1sClvT1svuTrCfmF8BhSgKqWo15vw+t7qA48=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=glm5LxkWENjIr9L8+9/1CFyvXmauCGx2jt0yCqugDog+76yIlmRt4FTZBxOJ9XAnZHBaym5qiTERWpjJ7sjtCnCH197w1ZjIs0fSsMXwTegd1EoQ3mxkZuYc7RNlbi4tZ2b6MPCi25q52nF4hyvwNUVY0bOgmOpAaqbhhCKMAbc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=3FSET8Mm; arc=fail smtp.client-ip=40.107.93.57 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="3FSET8Mm" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=xwJxYC7s7ey5ZJoyG9U6gCPtdS9dsSyvl86sBjJiVrK7GOhUk85BBS+EfHIQiuiFD5rxGlcFIvS3Y5ZJ0R31VEsVIv2HfmukSAQPdpYoYAIJhKoxRrd+R0ssN0eCznUX4F4IXjAlR5mQqEveg+wDoF+Mj4XZKnsTPFZ/B7RlCgrfAb1Vs70LPLbM1uCzv6ZtqNGP+rmhaneV/G6jLUpZTwdbhbrNv8RCwg2nd5aBj91JwA0be7q9mKptv9grP6uu5hS+nHdGmtIlABdOSJSgwK3uyq3WgyTTENUJ8VwGtnD6uW9rQDWodm/Me09rdLC2CTc4mZh6iCCGJJfG4IQzGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sxWG9JxvSxjfV1iPOfvMwj7sVWNlHSEELFOyA9lJN1k=; b=mesrUhWTtLcbql3yV7vrj9H4m+RFhRms2cJsp2F6N+IpBfdBsZubDv7N/fXbVDgc1EARjT8BE//sJpE+EIL+4yjGf5Zb6G5+ySiF6onOU+/gFzkfb0vIm8xQjxYN8ux117TXIDzRgGz6+OANKKo5akKkOhXNOEg09BaeQw3bXNq9UUsTkwdHoMpnEU8ew1Ks1dZ1nMgtu11RxfI0oqkPJVBK2/o36Bq02SfWlGw9m2Spkflppc8UP5Z2/BgqIE+IRvurvJhos/y3HTdlmv61OcNo+aN5abAlqGiTBrpt5M6dZ9e7Ch+/6SddKNfq9mQFu5tAxXe7kM3ANMp5rK/M1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sxWG9JxvSxjfV1iPOfvMwj7sVWNlHSEELFOyA9lJN1k=; b=3FSET8MmbZXug9hevybw88JHG3Knnx39FVDiM78m5azpzBMFHEDCIeKPKt9jSAvRVPwX4VwGVHJsjPfRV1x/GfCQVSkjJ2BKM+JWqxHNGnMrB7eUciCr8BP2YmVqbObU8zA7AtLcC4DxfXGsdB06h68Qe5oy5yFOU+eTeaAepmE= Received: from BL0PR1501CA0011.namprd15.prod.outlook.com (2603:10b6:207:17::24) by PH0PR12MB7077.namprd12.prod.outlook.com (2603:10b6:510:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Thu, 27 Mar 2025 01:47:59 +0000 Received: from BL6PEPF00022575.namprd02.prod.outlook.com (2603:10b6:207:17:cafe::d) by BL0PR1501CA0011.outlook.office365.com (2603:10b6:207:17::24) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.34 via Frontend Transport; Thu, 27 Mar 2025 01:47:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022575.mail.protection.outlook.com (10.167.249.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:47:58 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:47:57 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 03/16] CXL/AER: Introduce Kfifo for forwarding CXL errors Date: Wed, 26 Mar 2025 20:47:04 -0500 Message-ID: <20250327014717.2988633-4-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022575:EE_|PH0PR12MB7077:EE_ X-MS-Office365-Filtering-Correlation-Id: 8327ed30-da9d-4b67-892a-08dd6cd166ec X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700013|7416014|376014|921020; X-Microsoft-Antispam-Message-Info: 5talz5bcjj2xJl+ZD14fLTb25F72ciZKHaFBNyDzEduXzwiwLeqDDUyPsGQTeiJwPFALrKDIP9POien6nmSq8AESVQgEf6f0012kHqsy+mn0hy7PrHbuUIy0mpGsI8Nbku6JA/X76mko1P3qBcUX8MwJzYh6tDw0T2BboKRckuFwgkulkCHyfbAVXaBjx8X8f+vDUoyDBp8L4we4NH+eSEICz86xk2FBz/fSYYvovialDyLGGFbwMskzlqEaAmmKqr0NEQQT55kcNHXnwp7nu6IkldTgpGsf9oUI+PkPqniG+6OdB948LYTCIXWCcIGkVkeb5XXv9uLURSWn71hzhzXBU7GiUGQrBGvcLZYRNc8fJDKTo3o/nk661Bu9tAr+Nd5axCrhHumcTTLRiknFSHfg2I91fVgAWrUpXWXPGaQye1reye4pr+wg+hlM1BS9h8CjvjTqQae17j2hDPL3kjAh5waZfZrza5hyG9MUC+PQE+QTADIWFTzYYYGZEr5HkOC2Kqzv+Wfi3a5hUzcAr+cqbbrkePyktGDbOq/LR/gqWSnqNbSvmHMq/xrcvFK2SRl63MAN+RT8242maKCXzR20VWqoYxLpldtg+NrKOd5pBvGkym43DRGiueo/cIKU2svM9n6JLx1Q9YMLimykTLe1c325mfJAlmVOWEJmsuGUnhHkU1qLZ3WuxCGqcRPBo7+iQFUJQD0ky8nkp/LqZVyNwiMVFSte63Ko/3ZbWyY8Qllw7US0x39X9Jmu4riaKiInrQELwBjMvXkcfKiLtlZkE2lX2/ne1W29FvwXlEqueZTwefkGgodOW5nKlLdIy5gjdpux5X3rl8o5Uhd+l2/pr+sHJnvvgM0psTecm9LzfDx/w7Rv/lbKW8C3li7yJGbYH5cvZLvIpTJrxzperotnLu64kljKBZLSuGChcnitLg1CyuP3FI60HgZViKp0MBfDcjxSDLBoVhwvGUhch1bdBATroH4zmoiB3iX5KCVCZOgBgvC/skz9xrTCpR6nPcOy3ib3lDcNxqi+jLhCWVC5RISu0UqAPCyOa/u1/Ccq2JOX1vvCeG8gCtOzhPezn2cmhJyRx3EuP7FGjisygre9vt2416Dwenw5sGMoc94WBb562VQ3+D7lJjU/ygjg7dZnSnEdJeExnN4fnRhi36OCmWjJj20ZPlyO/vixcrAHDUJ7+osrIzegc6Q6FJrf1y1hIVHQxZwFp2Cyp88G2+0djRpjezKX6LHrbkQzUdaAJnFsSh7QHXm0S3kL1XlDhBAVo/lPRdSomu1kz5SVkAR+VDmqCtK2nPxleP7zczch4vpYFh612cRus/Z+tMJiM3saOuEUdIMg9wIyphpIK/9Pw5eKy05paxRafThrvkDwm9utWVmMRq/fFgQsvnDyzXAhEa0kn1xVwUZB0Gw2K9qNaY8P8HEQL11xOdCXHu4JbSp+iFs831yYC3IHZpkGIP+orUCFKd+KMJE8npufWaPsYKO+29zPdWSVIejK/ZSDP4SsRuWHjlypgJwKkgP3 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700013)(7416014)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:47:58.6752 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8327ed30-da9d-4b67-892a-08dd6cd166ec X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022575.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7077 CXL error handling will soon be moved from the AER driver into the CXL driver. This requires a notification mechanism for the AER driver to share the AER interrupt details with CXL driver. The notification is required for the CXL drivers to then handle CXL RAS errors. Add a kfifo work queue to be used by the AER driver and CXL driver. The AER driver will be the sole kfifo producer adding work. The cxl_core will be the sole kfifo consumer removing work. Add the boilerplate kfifo support. Add CXL work queue handler registration functions in the AER driver. Export the functions allowing CXL driver to access. Implement the registration functions for the CXL driver to assign or clear the work handler function. Create a work queue handler function, cxl_prot_err_work_fn(), as a stub for now. The CXL specific handling will be added in future patch. Introduce 'struct cxl_prot_err_info'. This structure caches CXL error details used in completing error handling. This avoid duplicating some function calls and allows the error to be treated generically when possible. Signed-off-by: Terry Bowman --- drivers/cxl/core/ras.c | 54 +++++++++++++++++++++++++++++++++++++++++- drivers/cxl/cxlpci.h | 3 +++ drivers/pci/pcie/aer.c | 39 ++++++++++++++++++++++++++++++ include/linux/aer.h | 37 +++++++++++++++++++++++++++++ 4 files changed, 132 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c index 485a831695c7..ecb60a5962de 100644 --- a/drivers/cxl/core/ras.c +++ b/drivers/cxl/core/ras.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "trace.h" static void cxl_cper_trace_corr_port_prot_err(struct pci_dev *pdev, @@ -107,13 +108,64 @@ static void cxl_cper_prot_err_work_fn(struct work_struct *work) } static DECLARE_WORK(cxl_cper_prot_err_work, cxl_cper_prot_err_work_fn); +int cxl_create_prot_err_info(struct pci_dev *_pdev, int severity, + struct cxl_prot_error_info *err_info) +{ + struct pci_dev *pdev __free(pci_dev_put) = pci_dev_get(_pdev); + struct cxl_dev_state *cxlds; + + if (!pdev || !err_info) { + pr_warn_once("Error: parameter is NULL"); + return -ENODEV; + } + + if ((pci_pcie_type(pdev) != PCI_EXP_TYPE_ENDPOINT) && + (pci_pcie_type(pdev) != PCI_EXP_TYPE_RC_END)) { + pci_warn_once(pdev, "Error: Unsupported device type (%X)", pci_pcie_type(pdev)); + return -ENODEV; + } + + cxlds = pci_get_drvdata(pdev); + struct device *dev __free(put_device) = get_device(&cxlds->cxlmd->dev); + + if (!dev) + return -ENODEV; + + *err_info = (struct cxl_prot_error_info){ 0 }; + err_info->ras_base = cxlds->regs.ras; + err_info->severity = severity; + err_info->pdev = pdev; + err_info->dev = dev; + + return 0; +} + +struct work_struct cxl_prot_err_work; + int cxl_ras_init(void) { - return cxl_cper_register_prot_err_work(&cxl_cper_prot_err_work); + int rc; + + rc = cxl_cper_register_prot_err_work(&cxl_cper_prot_err_work); + if (rc) { + pr_err("Failed to register CPER kfifo with AER driver"); + return rc; + } + + rc = cxl_register_prot_err_work(&cxl_prot_err_work, cxl_create_prot_err_info); + if (rc) { + pr_err("Failed to register kfifo with AER driver"); + return rc; + } + + return rc; } void cxl_ras_exit(void) { cxl_cper_unregister_prot_err_work(&cxl_cper_prot_err_work); cancel_work_sync(&cxl_cper_prot_err_work); + + cxl_unregister_prot_err_work(); + cancel_work_sync(&cxl_prot_err_work); } diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h index 54e219b0049e..92d72c0423ab 100644 --- a/drivers/cxl/cxlpci.h +++ b/drivers/cxl/cxlpci.h @@ -4,6 +4,7 @@ #define __CXL_PCI_H__ #include #include "cxl.h" +#include "linux/aer.h" #define CXL_MEMORY_PROGIF 0x10 @@ -135,4 +136,6 @@ void read_cdat_data(struct cxl_port *port); void cxl_cor_error_detected(struct pci_dev *pdev); pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, pci_channel_state_t state); +int cxl_create_prot_err_info(struct pci_dev *_pdev, int severity, + struct cxl_prot_error_info *err_info); #endif /* __CXL_PCI_H__ */ diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 83f2069f111e..46123b70f496 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -110,6 +110,16 @@ struct aer_stats { static int pcie_aer_disable; static pci_ers_result_t aer_root_reset(struct pci_dev *dev); +#if defined(CONFIG_PCIEAER_CXL) +#define CXL_ERROR_SOURCES_MAX 128 +static DEFINE_KFIFO(cxl_prot_err_fifo, struct cxl_prot_err_work_data, + CXL_ERROR_SOURCES_MAX); +static DEFINE_SPINLOCK(cxl_prot_err_fifo_lock); +struct work_struct *cxl_prot_err_work; +static int (*cxl_create_prot_err_info)(struct pci_dev*, int severity, + struct cxl_prot_error_info*); +#endif + void pci_no_aer(void) { pcie_aer_disable = 1; @@ -1577,6 +1587,35 @@ static pci_ers_result_t aer_root_reset(struct pci_dev *dev) return rc ? PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_RECOVERED; } + +#if defined(CONFIG_PCIEAER_CXL) +int cxl_register_prot_err_work(struct work_struct *work, + int (*_cxl_create_prot_err_info)(struct pci_dev*, int, + struct cxl_prot_error_info*)) +{ + guard(spinlock)(&cxl_prot_err_fifo_lock); + cxl_prot_err_work = work; + cxl_create_prot_err_info = _cxl_create_prot_err_info; + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_register_prot_err_work, "CXL"); + +int cxl_unregister_prot_err_work(void) +{ + guard(spinlock)(&cxl_prot_err_fifo_lock); + cxl_prot_err_work = NULL; + cxl_create_prot_err_info = NULL; + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_unregister_prot_err_work, "CXL"); + +int cxl_prot_err_kfifo_get(struct cxl_prot_err_work_data *wd) +{ + return kfifo_get(&cxl_prot_err_fifo, wd); +} +EXPORT_SYMBOL_NS_GPL(cxl_prot_err_kfifo_get, "CXL"); +#endif + static struct pcie_port_service_driver aerdriver = { .name = "aer", .port_type = PCIE_ANY_PORT, diff --git a/include/linux/aer.h b/include/linux/aer.h index 947b63091902..761d6f5cd792 100644 --- a/include/linux/aer.h +++ b/include/linux/aer.h @@ -10,6 +10,7 @@ #include #include +#include #define AER_NONFATAL 0 #define AER_FATAL 1 @@ -45,6 +46,24 @@ struct aer_capability_regs { u16 uncor_err_source; }; +/** + * struct cxl_prot_err_info - Error information used in CXL error handling + * @pdev: PCI device with CXL error + * @dev: CXL device with error. From CXL topology using ACPI/platform discovery + * @ras_base: Mapped address of CXL RAS registers + * @severity: CXL AER/RAS severity: AER_CORRECTABLE, AER_FATAL, AER_NONFATAL + */ +struct cxl_prot_error_info { + struct pci_dev *pdev; + struct device *dev; + void __iomem *ras_base; + int severity; +}; + +struct cxl_prot_err_work_data { + struct cxl_prot_error_info err_info; +}; + #if defined(CONFIG_PCIEAER) int pci_aer_clear_nonfatal_status(struct pci_dev *dev); int pcie_aer_is_native(struct pci_dev *dev); @@ -56,6 +75,24 @@ static inline int pci_aer_clear_nonfatal_status(struct pci_dev *dev) static inline int pcie_aer_is_native(struct pci_dev *dev) { return 0; } #endif +#if defined(CONFIG_PCIEAER_CXL) +int cxl_register_prot_err_work(struct work_struct *work, + int (*_cxl_create_proto_err_info)(struct pci_dev*, int, + struct cxl_prot_error_info*)); +int cxl_unregister_prot_err_work(void); +int cxl_prot_err_kfifo_get(struct cxl_prot_err_work_data *wd); +#else +static inline int +cxl_register_prot_err_work(struct work_struct *work, + int (*_cxl_create_proto_err_info)(struct pci_dev*, int, + struct cxl_prot_error_info*)) +{ + return 0; +} +static inline int cxl_unregister_prot_err_work(void) { return 0; } +static inline int cxl_prot_err_kfifo_get(struct cxl_prot_err_work_data *wd) { return 0; } +#endif + void pci_print_aer(struct pci_dev *dev, int aer_severity, struct aer_capability_regs *aer); int cper_severity_to_aer(int cper_severity); From patchwork Thu Mar 27 01:47:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030703 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2065.outbound.protection.outlook.com [40.107.92.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B81B71B3929; Thu, 27 Mar 2025 01:48:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.65 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040095; cv=fail; b=oE/LHcSFcZmGdAmjCsN1FCSJOlCIEoN+af5ljTd0H5HHpIm1Yh/jSb5vGPKYrOzZXek78XghpqWaDj83h4S2UgX1XF0kF4Hsbfxlrou2ysjCigPOVyVYDXF70I9OG1+0MYXVvaG6WJETAyjvWmCZENREiYFuoIt2XsSn4PSb2S0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040095; c=relaxed/simple; bh=AFdaFQHrS8tvNKjfSBTOp0dFvzggiERR0t4JLPXh/eE=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NIxRwJc27VOky2P/fYXG7N7GQdALrSnKqd54ZoQebV1Bp7nlZ1AxmlHaYLWIAmq2HquprFW3P0h/zpBFB6L0nOuDlKSy5Hxz+Q1/Jt+XW8JWMqLmbou7rr9/QBD6okmdB7PbjrEyjHYzV+iPToRFWD/FAjEAlFGw1zTEuZIhaIQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=mpYMhkc7; arc=fail smtp.client-ip=40.107.92.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="mpYMhkc7" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pLe/jYrztaxFmziE43ktNQExQn+2ayTWxJjYRfaObp0EEWKH7GC/8Re7U9NBbZZrwTaPfqk/iy04xw/X6h+blTgyBZIhKBYusiD+yFS0+2rKdtzNzJN7MC93jzTSA6lZ/yCH9psR+OiF7qsTLFA50/Hs2TWc+VWYkon5y0cgszUQEqeNxsOjLFTESU+ZCcNNlScjxcMaEJEHDoYeDwY7bOzC3Kd+Lx9GD8/4JdNk7lE6bY28u5z1mfr04I8hB+AgjwRYSSQD+O0jytDZPx9xjalHkpo4WSzSaGOXBNbYqdZOxLxzLABZPJObdg9+tkJIr2E/Wp+VtSADUdn+zuS+hA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+FBXYXLUNXYNpWZxDXAdJ8CY3Gp66SNMtfpYpKTCbaA=; b=QdabNlXqI/lkBfPYRZJKcB72e7SEZIyTvej97+CNPtNO9zOe43p8iiT9kxjLpzuliROD3QttmMXrvsVc0lJUX4sdklQdtERKUpwHwnk/JveH29Aja95skboitDisvDYQmDCstJ3EA2O1YE32mSemC+SgAhSNYUCwLgHOhE5/Yw93DO11l2OmjoQiIF2GBkVkkLY6PF0fflT6dsLQ38YKxCaGaDXVbk/2zmeIRntlcTXTs/Yb9sd/DDEtGwvFOiYvH9rbr99jROjqDhIIvW13SUqcabM5gItx8+zo0KeegGyi263NopYDrnXrCxzIMfceZ6H8YgFuWWDLOWfFCRuTew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+FBXYXLUNXYNpWZxDXAdJ8CY3Gp66SNMtfpYpKTCbaA=; b=mpYMhkc7Fg2A4dr1/R6P8KZgKQf4J6afalfD8+f1QW/wQgE6PTP4X/EzXg+IscmkQsr1eE4FMKZP/E01vOVoWK5+nrR5cQmQ1a9WKg6e4X8YV/MveCrzbRqJaB6ExQdNvFvfqgrS7R9DhR8u3z2JkI/UPsjnQ88K6zmDXA2gj6g= Received: from BL0PR1501CA0013.namprd15.prod.outlook.com (2603:10b6:207:17::26) by IA1PR12MB8517.namprd12.prod.outlook.com (2603:10b6:208:449::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.42; Thu, 27 Mar 2025 01:48:10 +0000 Received: from BL6PEPF00022575.namprd02.prod.outlook.com (2603:10b6:207:17:cafe::3c) by BL0PR1501CA0013.outlook.office365.com (2603:10b6:207:17::26) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.31 via Frontend Transport; Thu, 27 Mar 2025 01:48:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022575.mail.protection.outlook.com (10.167.249.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:48:09 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:48:08 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 04/16] cxl/aer: AER service driver forwards CXL error to CXL driver Date: Wed, 26 Mar 2025 20:47:05 -0500 Message-ID: <20250327014717.2988633-5-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022575:EE_|IA1PR12MB8517:EE_ X-MS-Office365-Filtering-Correlation-Id: e2a66f8f-e99f-4ed6-b1f0-08dd6cd16daa X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|82310400026|7416014|376014|921020; X-Microsoft-Antispam-Message-Info: qk05w4JlyR6wB4EoTaSxgbzruQb8dDhNAt/ajTYcrN80ghenDcgnoo8twWNMwRUUqV8vnVIyYCVUE+DeVkRLI0DuaAB7CXk+Ol7uAKew/3WgPPioavVdgEymYqEL79UAquWFWffCKnUbnNFz1QnRDrBDxHNcR8ZJaFUpMvBmfJeWPTAIHg3kur/BKopcCXsVc5CHrjWc9cShRESRieb8j0rXurUwzy6FagSOWD6NjgqKLkf3LS/cwZ4YqI6d8ANnn+xVjChN8cuZdI4gVa585qkKGmt1JC3V81y0jognK8AeSav1nawZR8dS60VWip61JlGC8E5aUu+yENtytDqNGaWERbcpL69iasK/oO/00w/lDjkkug8IfjFQzxAN0iiuL22WyEYRTIgFpnPmEet4d85gMlr97NdZRgur1dB2D2hKbMsPa5gQJRIlYE+LAiCZK4l3vrCqjTieD5gpy0tL9NA7nFfGva+Eg/cgfhtvdv9eqwhipjtVNzX9wzvL3Jv24Ecz9k3mzVPaolOs6BLJMNmZVpo/6He+o3RS00628Mgd0iU7C1Cg5CrseEsjEYFrNhbm3yShv6LZqM8EM+pLICJTGpFSf+J+AxR96Q/s6yJK/oOCC2JwKuqsATiwPDrl54yfuKuXGO9s9M08tp6BXLOfjutx8mToNCjd/l8Fp8L3Xviksiz7BAJmG5DUCMHZRrqySw9WohyQSG6/l+JEXF2Da3VSW1kJ9uceXgqHWQgdNPmCqIZjV2GaWv0OW4kxh4ncLY3dnFtLG/pVtBJRVSYRD8WpXwhfSlx4DScz9EZun/qhg22qzKyzXQuI5+kpMRc/65sxBDLaG+dGYRJH9pPeOn08wnBEgGOpy2bH/o//3PcbeJuPnpbY+m/gSyeQrdrYOjXz0H6ZAOkJOI3KJJ6aNffMeTOX84j3p66mdvUS1HzjQuxVQZQwXwOrQKG45tETn8HtDTJgNQA7vTryRXUCPQZFL1zpKlGA9RXa9PnS2vwG4LXzAsJ93mX+0NNA8vbE6WWl7spFChTd24KU8p2/8Qv4ANzxp0tEUfKQttKNvHmNpLlcGEe3IhrJAub9iK6CAhH9ToEA18a+ouoszeZMZa00/VN3oKiTQHN7mT36Q++B4nkn4Q4i0LaXikndzWt6tJlAOHUjR4vpkozMtn+GJnrhuzeUGem5i20BY+B36WcCkwd2uUTKhSzTCHFAEZTY/A0z5QDwudLNC4gBhCjUBaC29RRNVOU+QAMVl0zlHi4NIsb46WCgpyNgta6ElMTjWxBu1hBz12rGbcyYa+ji1/Kau7Qe2ZiyY8ejETml8TgdQSAgfcalebcCqS2wrds0HHMGqw5ZB+xDH46KdtKvSjGcLsi9u4LaxPgnn/6V8LbPgNvI+XUntHYCDKX/rJ8i3myNEWH8XhEDv0Bp0VZacyAnNJRCRabk0NcPuMLdkbGrLP5T/J252hbQKHtEAHO+VoOZk2LmiqQs4k8fc/8SvPLlCpYkZ3J3ZB1OIhCl+EjdVYiGrYgPxPuUMLAR X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(82310400026)(7416014)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:48:09.9877 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e2a66f8f-e99f-4ed6-b1f0-08dd6cd16daa X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022575.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8517 The AER service driver includes a CXL-specific kfifo, intended to forward CXL errors to the CXL driver. However, the forwarding functionality is currently unimplemented. Update the AER driver to enable error forwarding to the CXL driver. Modify the AER service driver's handle_error_source(), which is called from process_aer_err_devices(), to distinguish between PCIe and CXL errors. Rename and update is_internal_error() to is_cxl_error(). Ensuring it checks both the 'struct aer_info::is_cxl' flag and the AER internal error masks. If the error is a standard PCIe error then continue calling pcie_aer_handle_error() as done in the current AER driver. If the error is a CXL-related error then forward it to the CXL driver for handling using the kfifo mechanism. Introduce a new function forward_cxl_error(), which constructs a CXL protocol error context using cxl_create_prot_err_info(). This context is then passed to the CXL driver via kfifo using a 'struct work_struct'. Signed-off-by: Terry Bowman --- drivers/pci/pcie/aer.c | 61 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 55 insertions(+), 6 deletions(-) diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 46123b70f496..d1df751cfe4b 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -1010,6 +1010,14 @@ static bool is_internal_error(struct aer_err_info *info) return info->status & PCI_ERR_UNC_INTN; } +static bool is_cxl_error(struct aer_err_info *info) +{ + if (!info || !info->is_cxl) + return false; + + return is_internal_error(info); +} + static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data) { struct aer_err_info *info = (struct aer_err_info *)data; @@ -1062,13 +1070,17 @@ static int handles_cxl_error_iter(struct pci_dev *dev, void *data) return *handles_cxl; } -static bool handles_cxl_errors(struct pci_dev *rcec) +static bool handles_cxl_errors(struct pci_dev *dev) { bool handles_cxl = false; - if (pci_pcie_type(rcec) == PCI_EXP_TYPE_RC_EC && - pcie_aer_is_native(rcec)) - pcie_walk_rcec(rcec, handles_cxl_error_iter, &handles_cxl); + if (!pcie_aer_is_native(dev)) + return false; + + if (pci_pcie_type(dev) == PCI_EXP_TYPE_RC_EC) + pcie_walk_rcec(dev, handles_cxl_error_iter, &handles_cxl); + else + handles_cxl = pcie_is_cxl(dev); return handles_cxl; } @@ -1082,10 +1094,44 @@ static void cxl_rch_enable_rcec(struct pci_dev *rcec) pci_info(rcec, "CXL: Internal errors unmasked"); } +static void forward_cxl_error(struct pci_dev *_pdev, struct aer_err_info *info) +{ + int severity = info->severity; + struct cxl_prot_err_work_data wd; + struct cxl_prot_error_info *err_info = &wd.err_info; + struct pci_dev *pdev __free(pci_dev_put) = pci_dev_get(_pdev); + + if (!cxl_create_prot_err_info) { + pci_err(pdev, "Failed. CXL-AER interface not initialized."); + return; + } + + if (cxl_create_prot_err_info(pdev, severity, err_info)) { + pci_err(pdev, "Failed to create CXL protocol error information"); + return; + } + + struct device *cxl_dev __free(put_device) = get_device(err_info->dev); + + if (!kfifo_put(&cxl_prot_err_fifo, wd)) { + pr_err_ratelimited("CXL kfifo overflow\n"); + return; + } + + schedule_work(cxl_prot_err_work); +} + #else static inline void cxl_rch_enable_rcec(struct pci_dev *dev) { } static inline void cxl_rch_handle_error(struct pci_dev *dev, struct aer_err_info *info) { } +static inline void forward_cxl_error(struct pci_dev *dev, + struct aer_err_info *info) { } +static inline bool handles_cxl_errors(struct pci_dev *dev) +{ + return false; +} +static bool is_cxl_error(struct aer_err_info *info) { return 0; }; #endif /** @@ -1123,8 +1169,11 @@ static void pci_aer_handle_error(struct pci_dev *dev, struct aer_err_info *info) static void handle_error_source(struct pci_dev *dev, struct aer_err_info *info) { - cxl_rch_handle_error(dev, info); - pci_aer_handle_error(dev, info); + if (is_cxl_error(info)) + forward_cxl_error(dev, info); + else + pci_aer_handle_error(dev, info); + pci_dev_put(dev); } From patchwork Thu Mar 27 01:47:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030704 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2040.outbound.protection.outlook.com [40.107.92.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 486F44594A; Thu, 27 Mar 2025 01:48:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.40 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040107; cv=fail; b=FYYdWt0X24ZcRWBtzqYe/h9foAi98hnzxYC0K1D9S/838CIttXmpTVLt09NmE4346BQ9NMxaMA9LFWUMyFh0mv/0ijTl8/yPbATkQ+HgzjgHDEx2zOwKbLvZcxtJWu5dIVpLh/ODMOEp/4k8CwvLnD/jAcfBNt5RpganjzEN6hQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040107; c=relaxed/simple; bh=5kVqgDril1L2zZeKlWa0RV6Cmn6NLmquwG+4f+GDOw8=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=VwNXrpCov4LP9joSTC/WhaKZuxVIWq5ZydZofWJJjsWgTDOFTSRg+jT2O453WSWcwIl8HGeHRdArfP8xx0y9ZATK+pNCGGhCUjPAlWuQcDZreXW+LEZQ9A87Y+LozFRWiO3U1zb07lsvwFyaOg0F+gOgjshYs8sioMZBWtN7TwY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Q1OSP4Kv; arc=fail smtp.client-ip=40.107.92.40 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Q1OSP4Kv" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=EHXzhHAQ0zydJinv1jvmQ8KQ7+FkmabBbD8QmO0taEXHNMFE/NyJfk1MpHgJMy/mz09gbNcZ2w6AwRr4Ex3A9sJDmkIr8rjLSt+W6LzwgFGZDPRFSllD/wbeD6FRw+Bs7R4zD3X0+kKTLfRNN2dndurO1o/px6yBWOQgRfh6V2GSUVgyZuQrpDTfePH8edmmq5E6WrQX9fjNbPw+N0P0eohZR0snYHpp4BdXs5I87qmI7zopOee02wrgmidKENGfZle8sFDndXkkHHkTsRiozMjq2Ye8yG+lXeTm75ZCW8Afk1O4TIqwfnKCQAAPhzB2NKdm2y+mN22P9dvW+OcKXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0xVjEnmkSKzQ57PPSgGQvl7Y89B1rCHgdZHe/jkIgps=; b=gxX0dD9Cx2aCgRDaKzZDiQuFlBGwBuy4UukBzsSPhiXt+doOD2u6t5Su3xyyFbNmGGvau9RCm9zCTmlQLBoCE6s4pCtgbnLo0ykxn+F30hYN4RRu3LUCF57x9ln+GIHQ6sSFFjAiEgrsyBlYFl0a38vfqtEcR22kne4oLUWPRYyYExF4W8Vc4SAXA/ODjrM0RZvNlP3jqzXgFRCjyFdTbPpnV6URfLeFbY35Kze0cOAQ2NsrwJkvKXOc4342/1aDCZAtsaFYhfW4kOVb8AQuII0K6UQoK5E8ewH2Fes3SPOt/JHmX0hENQ/v5piYaX0Z2HtucvIspMh7FbwcU7128w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0xVjEnmkSKzQ57PPSgGQvl7Y89B1rCHgdZHe/jkIgps=; b=Q1OSP4KvKrEMYjTUEMtAnzIRksvmsjDTYM8PxL0XWlzCBrxVSlw/c4yas+dTMnwZOOsdByC6+VukrDQVFkGbNI9fV1rmP+aqeGqM6GfRv+/QlgwWLkMOlLAUmPIVQC7wXEgPS5rQ9NhLhDy8KN+tt7r1n7BDW1MxtaALlx8pv+4= Received: from BL0PR1501CA0004.namprd15.prod.outlook.com (2603:10b6:207:17::17) by BN7PPF02710D35B.namprd12.prod.outlook.com (2603:10b6:40f:fc02::6c4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8511.31; Thu, 27 Mar 2025 01:48:21 +0000 Received: from BL6PEPF00022575.namprd02.prod.outlook.com (2603:10b6:207:17:cafe::dd) by BL0PR1501CA0004.outlook.office365.com (2603:10b6:207:17::17) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.33 via Frontend Transport; Thu, 27 Mar 2025 01:48:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022575.mail.protection.outlook.com (10.167.249.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:48:21 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:48:20 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 05/16] PCI/AER: CXL driver dequeues CXL error forwarded from AER service driver Date: Wed, 26 Mar 2025 20:47:06 -0500 Message-ID: <20250327014717.2988633-6-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022575:EE_|BN7PPF02710D35B:EE_ X-MS-Office365-Filtering-Correlation-Id: ad31afa6-fae4-44b8-b23d-08dd6cd17480 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|36860700013|1800799024|7416014|921020; X-Microsoft-Antispam-Message-Info: bKlEbB12KfKGNfZ0h8GOqJlliYEWGn+OwgB/lX26ykQL3RxonoJ/dlasVjPiCQX8pnkQ/tL24s75rQP6oon3VwCiPcDvC0GmW1zB2qPMu2v0BJo3vtHBRv6RIrhKBQz0fumg5G7gEBGKd51cSoN0VlkPC37xnctODCWmgK06PHZstTBBUOknyg4AlsviSmKijPO5AVEX79loig4Ms15RefqZKWIm20DsX7b6mRP++XcY7VguvLhkUC2fAQH+te1QuHq8o6psnopK0YeNDZ7K7qI/G7I2FayhvF7/prMrgqJ3mvVlHjwR9PYxuB3lv7uoGfhBZmtMh1l4jzWkGM6Cem5gZ9dYRahAyv6cwOx9OwY2B+1eJ+eZvJLr4fdAMX9YGd+2tO/IMICFCx0knAchPcygva/jlYbvdCvefYuWxMV5Ta2mhDmdNmWkVc4Z5um6CS+06DLZLXWJucskq9Eo7QedYutmbjMy2QotAr+W/lSvFStB9Mw5N4S0qEfOdPXxu0px0roxRW542CDR5c8TEOSwlVqcyx1KkaOPxtLVpdp3JXurGR/5mt8EkBq0J28feNgztL8MlwvPOm0RsL1N6smn0IDoq1jic2PSEHZXhjLTANVQSMV5kB9fF/wj+E44v6goMwTgHFYuoncsAwqUWxxwqefDdmogIpWmO2Q8SblCzxdfitmU0wRt/1JZ+FxNnDLrUyoofPdRn1nkOOFzc5aw9ZosCrzCAIXytddGqu1ZF36M5K5I5d3OIBhnYV51azVa8IbYGt5bFJquDiEm7wXNtTHFpjRHOnzFyHn16jn9eY7s9hrO4lBiOJuNs8Cxdx2quJImvAlM2dJSLBm9uXdtviz4mNLmVKXyReQXGKQQLFifAr14FltQDPUI3GNyfeOO5Elmkx7GGPOrR2yatWtdVcpc0Rj5IBeZB72MUQRJHEaw1hv1BAY1rtzpejyX0Kzh2V+EaWfw+RdyugSpLUebVpjGQHXUzmyjSBj4ReC5TXGnkLWySFGGqooUvg1mu0op523qA0q6uYsCk+PJQb2s0k2kGYv/Jy9Uf86eFKLgfUBg7NUHN27DIk4eXaJIckFXoHcVm0ZWjJCvkmxHuNt6tI2PukFX7XPKcUEh3Gbm6885M/kY0iYm7JnCjSFzkLuindPLlNM/LFPyZaelpTwajukKUQn1eE5KkDrPmsx/857GJHxTuM/pZmSOwuceunTiBFt/loJXPwIh2mW9QFHqZ5Q66LQQajbH7pHKj+bZwraw1WKMSP8CM1IM9WMH5xP85jz0qsTeiiwnOoAzigLjCVwGwqYgX9OkgUUupSwkXvw5VyEbIDOVZq5R2puMVA5cuvJc2UhpBsI1hkbilavM0bQcN00C2VIRgH0fCU8DOa+/AMAR1o1W4z/pELFo8ZbqisbgcadermLq97l8E3g72/X9Ldhzh8CQU7IUdP6cZ2+77qIfH7Ns9HG/6BBQ/IBca7u4dKL173EpSNH08XFmbzdgVmpindaQ7Vfp+wV7pFpjL1hkF9JWb7n1od7a X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024)(7416014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:48:21.4565 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ad31afa6-fae4-44b8-b23d-08dd6cd17480 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022575.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PPF02710D35B The AER driver is now designed to forward CXL protocol errors to the CXL driver. Update the CXL driver with functionality to dequeue the forwarded CXL error from the kfifo. Also, update the CXL driver to process the CXL protocol errors using CXL protocol error handlers. First, move cxl_rch_handle_error_iter() from aer.c to cxl/core/ras.c. Remove and drop the cxl_rch_handle_error() in aer.c as it is not needed. Introduce function cxl_prot_err_work_fn() to dequeue work forwarded by the AER service driver. This will begin the CXL protocol error processing with the call to cxl_handle_prot_error(). Introduce cxl_handle_prot_error() to differntiate between Restricted CXL Host (RCH) protocol errors and CXL virtual host (VH) protocol errors. RCH errors will be processed with a call to walk the associated Root Complex Event Collector's (RCEC) secondary bus looking for the Root Complex Integrated Endpoint (RCiEP) to handle the RCH error. Export pcie_walk_rcec() so the CXL driver can walk the RCEC's downstream bus, searching for the RCiEP. VH correctable error (CE) processing will call the CXL CE handler if present. VH uncorrectable errors (UCE) will call cxl_do_recovery(), implemented as a stub for now and to be updated in future patch. Export pci_aer_clean_fatal_status() and pci_clean_device_status() used to clean up AER status after handling. Create cxl_driver::error_handler structure similar to pci_driver::error_handlers. Add handlers for CE and UCE CXL.io errors. Add 'struct cxl_prot_error_info' as a parameter to the CXL CE and UCE error handlers. Signed-off-by: Terry Bowman --- drivers/cxl/core/ras.c | 102 +++++++++++++++++++++++++++++++++++++++- drivers/cxl/cxl.h | 17 +++++++ drivers/pci/pci.c | 1 + drivers/pci/pci.h | 6 --- drivers/pci/pcie/aer.c | 42 +---------------- drivers/pci/pcie/rcec.c | 1 + include/linux/aer.h | 2 + include/linux/pci.h | 10 ++++ 8 files changed, 133 insertions(+), 48 deletions(-) diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c index ecb60a5962de..eca8f11a05d9 100644 --- a/drivers/cxl/core/ras.c +++ b/drivers/cxl/core/ras.c @@ -139,8 +139,108 @@ int cxl_create_prot_err_info(struct pci_dev *_pdev, int severity, return 0; } +EXPORT_SYMBOL_NS_GPL(cxl_create_prot_err_info, "CXL"); -struct work_struct cxl_prot_err_work; +static void cxl_do_recovery(struct pci_dev *pdev) { } + +static int cxl_rch_handle_error_iter(struct pci_dev *pdev, void *data) +{ + struct cxl_prot_error_info *err_info = data; + const struct cxl_error_handlers *err_handler; + struct device *dev = err_info->dev; + struct cxl_driver *pdrv; + + /* + * The capability, status, and control fields in Device 0, + * Function 0 DVSEC control the CXL functionality of the + * entire device (CXL 3.0, 8.1.3). + */ + if (pdev->devfn != PCI_DEVFN(0, 0)) + return 0; + + /* + * CXL Memory Devices must have the 502h class code set (CXL + * 3.0, 8.1.12.1). + */ + if ((pdev->class >> 8) != PCI_CLASS_MEMORY_CXL) + return 0; + + if (!is_cxl_memdev(dev) || !dev->driver) + return 0; + + pdrv = to_cxl_drv(dev->driver); + if (!pdrv || !pdrv->err_handler) + return 0; + + err_handler = pdrv->err_handler; + if (err_info->severity == AER_CORRECTABLE) { + if (err_handler->cor_error_detected) + err_handler->cor_error_detected(dev, err_info); + } else if (err_handler->error_detected) { + cxl_do_recovery(pdev); + } + + return 0; +} + +static void cxl_handle_prot_error(struct pci_dev *pdev, struct cxl_prot_error_info *err_info) +{ + if (!pdev || !err_info) + return; + + /* + * Internal errors of an RCEC indicate an AER error in an + * RCH's downstream port. Check and handle them in the CXL.mem + * device driver. + */ + if (pci_pcie_type(pdev) == PCI_EXP_TYPE_RC_EC) + return pcie_walk_rcec(pdev, cxl_rch_handle_error_iter, err_info); + + if (err_info->severity == AER_CORRECTABLE) { + struct device *dev __free(put_device) = get_device(err_info->dev); + struct cxl_driver *pdrv; + int aer = pdev->aer_cap; + + if (!dev || !dev->driver) + return; + + if (aer) { + int ras_status; + + pci_read_config_dword(pdev, aer + PCI_ERR_COR_STATUS, &ras_status); + pci_write_config_dword(pdev, aer + PCI_ERR_COR_STATUS, + ras_status); + } + + pdrv = to_cxl_drv(dev->driver); + if (!pdrv || !pdrv->err_handler || + !pdrv->err_handler->cor_error_detected) + return; + + pdrv->err_handler->cor_error_detected(dev, err_info); + pcie_clear_device_status(pdev); + } else { + cxl_do_recovery(pdev); + } +} + +static void cxl_prot_err_work_fn(struct work_struct *work) +{ + struct cxl_prot_err_work_data wd; + + while (cxl_prot_err_kfifo_get(&wd)) { + struct cxl_prot_error_info *err_info = &wd.err_info; + struct device *dev __free(put_device) = get_device(err_info->dev); + struct pci_dev *pdev __free(pci_dev_put) = pci_dev_get(err_info->pdev); + + if (!dev || !pdev) + continue; + + cxl_handle_prot_error(pdev, err_info); + } +} + +static DECLARE_WORK(cxl_prot_err_work, cxl_prot_err_work_fn); int cxl_ras_init(void) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index be8a7dc77719..73cddd2c921e 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -11,6 +11,8 @@ #include #include #include +#include +#include extern const struct nvdimm_security_ops *cxl_security_ops; @@ -786,6 +788,20 @@ static inline int cxl_root_decoder_autoremove(struct device *host, } int cxl_endpoint_autoremove(struct cxl_memdev *cxlmd, struct cxl_port *endpoint); +int cxl_create_prot_err_info(struct pci_dev *pdev, int severity, + struct cxl_prot_error_info *err_info); + +/* CXL bus error event callbacks */ +struct cxl_error_handlers { + /* CXL bus error detected on this device */ + pci_ers_result_t (*error_detected)(struct device *dev, + struct cxl_prot_error_info *err_info); + + /* Allow device driver to record more details of a correctable error */ + void (*cor_error_detected)(struct device *dev, + struct cxl_prot_error_info *err_info); +}; + /** * struct cxl_endpoint_dvsec_info - Cached DVSEC info * @mem_enabled: cached value of mem_enabled in the DVSEC at init time @@ -820,6 +836,7 @@ struct cxl_driver { void (*remove)(struct device *dev); struct device_driver drv; int id; + const struct cxl_error_handlers *err_handler; }; #define to_cxl_drv(__drv) container_of_const(__drv, struct cxl_driver, drv) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index a1d75f40017e..d80c705d683c 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -2320,6 +2320,7 @@ void pcie_clear_device_status(struct pci_dev *dev) pcie_capability_read_word(dev, PCI_EXP_DEVSTA, &sta); pcie_capability_write_word(dev, PCI_EXP_DEVSTA, sta); } +EXPORT_SYMBOL_NS_GPL(pcie_clear_device_status, "CXL"); #endif /** diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index eed098c134a6..c32eab22c0b2 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -593,16 +593,10 @@ static inline bool pci_dpc_recovered(struct pci_dev *pdev) { return false; } void pci_rcec_init(struct pci_dev *dev); void pci_rcec_exit(struct pci_dev *dev); void pcie_link_rcec(struct pci_dev *rcec); -void pcie_walk_rcec(struct pci_dev *rcec, - int (*cb)(struct pci_dev *, void *), - void *userdata); #else static inline void pci_rcec_init(struct pci_dev *dev) { } static inline void pci_rcec_exit(struct pci_dev *dev) { } static inline void pcie_link_rcec(struct pci_dev *rcec) { } -static inline void pcie_walk_rcec(struct pci_dev *rcec, - int (*cb)(struct pci_dev *, void *), - void *userdata) { } #endif #ifdef CONFIG_PCI_ATS diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index d1df751cfe4b..763ec6aa1a9a 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -288,6 +288,7 @@ void pci_aer_clear_fatal_status(struct pci_dev *dev) if (status) pci_write_config_dword(dev, aer + PCI_ERR_UNCOR_STATUS, status); } +EXPORT_SYMBOL_GPL(pci_aer_clear_fatal_status); /** * pci_aer_raw_clear_status - Clear AER error registers. @@ -1018,47 +1019,6 @@ static bool is_cxl_error(struct aer_err_info *info) return is_internal_error(info); } -static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data) -{ - struct aer_err_info *info = (struct aer_err_info *)data; - const struct pci_error_handlers *err_handler; - - if (!is_cxl_mem_dev(dev) || !cxl_error_is_native(dev)) - return 0; - - /* protect dev->driver */ - device_lock(&dev->dev); - - err_handler = dev->driver ? dev->driver->err_handler : NULL; - if (!err_handler) - goto out; - - if (info->severity == AER_CORRECTABLE) { - if (err_handler->cor_error_detected) - err_handler->cor_error_detected(dev); - } else if (err_handler->error_detected) { - if (info->severity == AER_NONFATAL) - err_handler->error_detected(dev, pci_channel_io_normal); - else if (info->severity == AER_FATAL) - err_handler->error_detected(dev, pci_channel_io_frozen); - } -out: - device_unlock(&dev->dev); - return 0; -} - -static void cxl_rch_handle_error(struct pci_dev *dev, struct aer_err_info *info) -{ - /* - * Internal errors of an RCEC indicate an AER error in an - * RCH's downstream port. Check and handle them in the CXL.mem - * device driver. - */ - if (pci_pcie_type(dev) == PCI_EXP_TYPE_RC_EC && - is_internal_error(info)) - pcie_walk_rcec(dev, cxl_rch_handle_error_iter, info); -} - static int handles_cxl_error_iter(struct pci_dev *dev, void *data) { bool *handles_cxl = data; diff --git a/drivers/pci/pcie/rcec.c b/drivers/pci/pcie/rcec.c index d0bcd141ac9c..fb6cf6449a1d 100644 --- a/drivers/pci/pcie/rcec.c +++ b/drivers/pci/pcie/rcec.c @@ -145,6 +145,7 @@ void pcie_walk_rcec(struct pci_dev *rcec, int (*cb)(struct pci_dev *, void *), walk_rcec(walk_rcec_helper, &rcec_data); } +EXPORT_SYMBOL_NS_GPL(pcie_walk_rcec, "CXL"); void pci_rcec_init(struct pci_dev *dev) { diff --git a/include/linux/aer.h b/include/linux/aer.h index 761d6f5cd792..8f815f34d447 100644 --- a/include/linux/aer.h +++ b/include/linux/aer.h @@ -66,12 +66,14 @@ struct cxl_prot_err_work_data { #if defined(CONFIG_PCIEAER) int pci_aer_clear_nonfatal_status(struct pci_dev *dev); +void pci_aer_clear_fatal_status(struct pci_dev *dev); int pcie_aer_is_native(struct pci_dev *dev); #else static inline int pci_aer_clear_nonfatal_status(struct pci_dev *dev) { return -EINVAL; } +static inline void pci_aer_clear_fatal_status(struct pci_dev *dev) { } static inline int pcie_aer_is_native(struct pci_dev *dev) { return 0; } #endif diff --git a/include/linux/pci.h b/include/linux/pci.h index af83230bef1a..56015721be22 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1802,6 +1802,9 @@ extern bool pcie_ports_native; int pcie_set_target_speed(struct pci_dev *port, enum pci_bus_speed speed_req, bool use_lt); +void pcie_walk_rcec(struct pci_dev *rcec, + int (*cb)(struct pci_dev *, void *), + void *userdata); #else #define pcie_ports_disabled true #define pcie_ports_native false @@ -1812,8 +1815,15 @@ static inline int pcie_set_target_speed(struct pci_dev *port, { return -EOPNOTSUPP; } + +static inline void pcie_walk_rcec(struct pci_dev *rcec, + int (*cb)(struct pci_dev *, void *), + void *userdata) { } + #endif +void pcie_clear_device_status(struct pci_dev *dev); + #define PCIE_LINK_STATE_L0S (BIT(0) | BIT(1)) /* Upstr/dwnstr L0s */ #define PCIE_LINK_STATE_L1 BIT(2) /* L1 state */ #define PCIE_LINK_STATE_L1_1 BIT(3) /* ASPM L1.1 state */ From patchwork Thu Mar 27 01:47:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030705 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2076.outbound.protection.outlook.com [40.107.94.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0116B13B5A0; Thu, 27 Mar 2025 01:48:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.76 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040118; cv=fail; b=r0l8lB2/z/Ko8zQIE8z4JUDGOO0hMRbkLMzqOIPb7nR1S0o1EXvYVpdTxS3s2/xsC5ctg/8T+cc5iRrCKbnxjBg1s9J/Wjk33ThMpAe/40eaXr9ywK1Au2D7DowEpUS0NNgwU5/Ub7y0agYOYy0xJ+/W/y1XHUsEkmdQ+NnAvsM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040118; c=relaxed/simple; bh=GRBceh4bE/kleCkOXroWMqYw4oaon/gtJ2UBz6UBSDs=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MprM+GGmZudsJUUIlljNUxkFrw+dgXXef7yd5/rVPf9KF4X1AzRIu6/9duzaYr2X/0OVlD3+TQKBvwwbEFKtBfiE1So8Fq/fOzmAP4ECfpy5RbekG8yej9Au7HWKfwxftPKJRGkQJz0Owz2/2XAmbcZE95iVRDscctFfXFvd7gk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=SxeCwkBu; arc=fail smtp.client-ip=40.107.94.76 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="SxeCwkBu" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ENBgQ9Kms9dXXabt/ER5hYL54vFs0LZuI2resRZvagsPbLdOFuN5vK7swk5/xIKFO3ZtSaVSWrcr7m45EfVY5IJX1U9HE+69Zd/+gVqBnhErU10FXBLr5BHgvja6+SmFV07T0YA/25hp5U3Plqxo8EQ3KBGRWK44AGii1EQChhCw6tIQYUxm+5bq5GEF0GHYBca6GiUhITacnrDLYf8wyWXXGGOKigNtn2EoSF+YNZMRb/oI4zRe89JDvzJ4ujTyEPIEbh1T685Fy3LSfLiHm8TXFiam3Y6QZontnGzI1daF/ywS12fIUqeFa4YReNs4sYnmqCvEGh1CV9NFzbnJGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JaiQ++Kacp3McQZM6yx8PgYKOkS5kKCYWXJoCM0iPOA=; b=lQMauw8u/dMIQhsJzY/dLlwBMtN003sBHtEvn+q35248Jo3tXyLFidgP5LQbVYCBxlsX61Lx7ydZEsdhA1qgiaLBypflaVCr2+ksMeh/k8Qoc62NTWpaynzgfY1SL83IHj3jFMu1IGgQRDIktPOgSQdVe4gH6NJ/e4D9j76VOyGGiX8wyhE1lHsxhij4mA31lqdUoaU7ECpJmBmOUkGkR/HEeyeTyu1Xo5zGhqbboYX+BuU8Jm10Ty/tL1mMkW7+sn7sMo3B5aBiVaj6Q7DNu1PQbp/PYCXOgHvAWbEbu3AIrGkt1U3vSbU+7e0IPpgtXDi/g4CaC75sZqMJeGWr5g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JaiQ++Kacp3McQZM6yx8PgYKOkS5kKCYWXJoCM0iPOA=; b=SxeCwkBud/FP6jAaVUbqS0b1LHrR+81zokwTHWDv58oIMgq4L0IGJkJbun6lzA3jfgwbom6zyxM29B+K6n3jdsRranDyjUQvo1SYksHXVMn1bNny4m/1Kavr0+i5XeaXgfABat6o2MPw3bXSyXKfPsptvqCgxtr1yKnVoiOAmig= Received: from MN2PR05CA0012.namprd05.prod.outlook.com (2603:10b6:208:c0::25) by PH7PR12MB5998.namprd12.prod.outlook.com (2603:10b6:510:1da::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.43; Thu, 27 Mar 2025 01:48:33 +0000 Received: from BL6PEPF00022571.namprd02.prod.outlook.com (2603:10b6:208:c0:cafe::ff) by MN2PR05CA0012.outlook.office365.com (2603:10b6:208:c0::25) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8583.26 via Frontend Transport; Thu, 27 Mar 2025 01:48:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022571.mail.protection.outlook.com (10.167.249.39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:48:32 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:48:31 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 06/16] CXL/PCI: Introduce CXL uncorrectable protocol error 'recovery' Date: Wed, 26 Mar 2025 20:47:07 -0500 Message-ID: <20250327014717.2988633-7-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022571:EE_|PH7PR12MB5998:EE_ X-MS-Office365-Filtering-Correlation-Id: f71523a5-d232-41ce-1694-08dd6cd17b3d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|82310400026|1800799024|36860700013|921020; X-Microsoft-Antispam-Message-Info: f6g7OYP4PG3e/HbYygf7xl/iNOklMWZZWjEAZI8JD8fdiAee2l2CG31OuIq9GGhhqkeeYFGYgB8oMk19W6H6qxxmsYM4n8X7U8O/HywkysI48uadN5aSDlADhCPY64w/UwgitH9OBsRVjAPqzI2yOEPqWLlbOa/RG6IDkAFUOTCPHLj2VlKK36k7SFPdGi71MqCHOgiujA0Lr4EG3oQzfuWZbVtdmJdVAFDWarAOzOc8Foo0wggJzH9y8t494nNyEyXgEI/mG+uF0EFShW3K4zQHr2v6R762FxIiARHVSp3PE6S/RnC88zF8TVFwF+YP7cIu8gYEDngmK0KF5YUmSVbNsawgoGKHNTQwcVgtOFxz4s1Iyv5nqP33EofVHLmtsRbK6KqbKcKNL7uTQtRtbM1mtfp4giybiwJ3szwogNXOQXjl8faPtQKE5LW+QgH8k1SMsPeFycM1tmTmSZKlj0HSoGZKykfdY8WzrSyrJXxqoz5Et2kauu7GrlqWb9caOWrf0Ahrs9g+/R9dsIwMzcOMzbHU9ZyHXWLQCAbynj51u76P+JpQUX5D+j2NDF7aTkeo29ruhHZVWnhOOySAeDxVhV1f9l/CmnFzScWgZWjb4RZvEpZ6hHR+MSWfW3Oxd1rMayMJMspm/mF/2Wd/1IwTQa5QctvxCOcF2IGHtO9Z/fVWw4vR8cMBeAYZMNYdOEijz6VUKS6QHG8hFZgFGYugPm5gWZe8/oke61eaYCeXs+9HvY3hHgaXKUI20ItzlCeLpgiYMpCl1Oubo0u1v9N5Q0CqeKz0ODNgYcK326YG3GJcJzb4gyYIcqXS4gu8HzKFlDE7p4eoDcywDgHKRtu39FKlCKPdxqBVLpkAtRYO1s6B03wtfSgdMiSifPS/jHomOcMnOWm7ie05qiFPvDC2NfITrg45tOzUW88MCNFKD9jF5rlJeNQpf4DGQP3lhyt2DKQ7PN298euiLnts09/9jhVzId1+8h3PnheBA0htRSCqmStiJRgyaWpeuwGLWYESGgGhTSAAjtmUrGQTTa7WG8A+00OjlJ7PTcZE0Iwcvg/WNuLs+zwiJ7ez3iSMMhBTtZ0jjAWkYXaQ9YawNHEI+4nmudGE4nppy3VipUrvCJGoi3Xu/1QUkCOFWVzkmm9vzq0NQ4S4OoIKVUaLrO773SG/slzhRq8sN0Zo3GHtGkLrT1m5bZHyNzDShUexcERBJ8jiyCQoLY6a6N6dkdhAnkac4BSJZjNoRN8ShtRMmq1ScyryRfHlq7cQlXEHAjgN2iM59Ci4zMyxF/ynZ+XUcNdVX4qmeAizGHuCL3WMSEqwknDlhmmT7kTuDx0PaLpX5scIDaBVY9UWiGClH7B1Rwrfri4RALE4vKWDT73maGOZ2AVjNaGBCZ//SmoAgE1mJM8XrzPIVxjCF4oC5s0yRzSz/qNZaBmt/K5YwDFQU/TzML8F9AYHGs1VOclDYT7lDPTeO3MGgeDu/ey0TYNU8BEli+Pbzr67t4ez6QddqmrHC1XlzwoWA7yYe0+Z X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(7416014)(82310400026)(1800799024)(36860700013)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:48:32.7731 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f71523a5-d232-41ce-1694-08dd6cd17b3d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022571.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5998 Create cxl_do_recovery() to provide uncorrectable protocol error (UCE) handling. Follow similar design as found in PCIe error driver, pcie_do_recovery(). One difference is that cxl_do_recovery() will treat all UCEs as fatal with a kernel panic. This is to prevent corruption on CXL memory. Copy the PCIe error handlers merge_result(). Introduce PCI_ERS_RESULT_PANIC and add support in the merge_result() routine. Copy pci_walk_bridge() to cxl_walk_bridge(). Make a change to walk the first device in all cases. Copy report_error_detected() to cxl_report_error_detected(). Update this function to populate the CXL error information structure, 'struct cxl_prot_error_info', before calling the device error handler. Call panic() to halt the system in the case of uncorrectable errors (UCE) in cxl_do_recovery(). Export pci_aer_clear_fatal_status() for CXL to use if a UCE is not found. In this case the AER status must be cleared and uses pci_aer_clear_fatal_status(). Signed-off-by: Terry Bowman --- drivers/cxl/core/ras.c | 92 +++++++++++++++++++++++++++++++++++++++++- drivers/pci/pci.h | 2 - include/linux/pci.h | 5 +++ 3 files changed, 96 insertions(+), 3 deletions(-) diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c index eca8f11a05d9..1f94fc08e72b 100644 --- a/drivers/cxl/core/ras.c +++ b/drivers/cxl/core/ras.c @@ -141,7 +141,97 @@ int cxl_create_prot_err_info(struct pci_dev *_pdev, int severity, } EXPORT_SYMBOL_NS_GPL(cxl_create_prot_err_info, "CXL"); -static void cxl_do_recovery(struct pci_dev *pdev) { } + +static pci_ers_result_t merge_result(enum pci_ers_result orig, + enum pci_ers_result new) +{ + if (new == PCI_ERS_RESULT_PANIC) + return PCI_ERS_RESULT_PANIC; + + if (new == PCI_ERS_RESULT_NO_AER_DRIVER) + return PCI_ERS_RESULT_NO_AER_DRIVER; + + if (new == PCI_ERS_RESULT_NONE) + return orig; + + switch (orig) { + case PCI_ERS_RESULT_CAN_RECOVER: + case PCI_ERS_RESULT_RECOVERED: + orig = new; + break; + case PCI_ERS_RESULT_DISCONNECT: + if (new == PCI_ERS_RESULT_NEED_RESET) + orig = PCI_ERS_RESULT_NEED_RESET; + break; + default: + break; + } + + return orig; +} + +static void cxl_walk_bridge(struct pci_dev *bridge, + int (*cb)(struct pci_dev *, void *), + void *userdata) +{ + if (cb(bridge, userdata)) + return; + + if (bridge->subordinate) + pci_walk_bus(bridge->subordinate, cb, userdata); +} + + +static int cxl_report_error_detected(struct pci_dev *pdev, void *data) +{ + struct cxl_driver *pdrv; + pci_ers_result_t vote, *result = data; + struct cxl_prot_error_info err_info = { 0 }; + const struct cxl_error_handlers *cxl_err_handler; + + if (cxl_create_prot_err_info(pdev, AER_FATAL, &err_info)) + return 0; + + struct device *dev __free(put_device) = get_device(err_info.dev); + if (!dev) + return 0; + + pdrv = to_cxl_drv(dev->driver); + if (!pdrv || !pdrv->err_handler || + !pdrv->err_handler->error_detected) + return 0; + + cxl_err_handler = pdrv->err_handler; + vote = cxl_err_handler->error_detected(dev, &err_info); + + *result = merge_result(*result, vote); + + return 0; +} + +static void cxl_do_recovery(struct pci_dev *pdev) +{ + struct pci_host_bridge *host = pci_find_host_bridge(pdev->bus); + pci_ers_result_t status = PCI_ERS_RESULT_CAN_RECOVER; + + cxl_walk_bridge(pdev, cxl_report_error_detected, &status); + if (status == PCI_ERS_RESULT_PANIC) + panic("CXL cachemem error."); + + /* + * If we have native control of AER, clear error status in the device + * that detected the error. If the platform retained control of AER, + * it is responsible for clearing this status. In that case, the + * signaling device may not even be visible to the OS. + */ + if (host->native_aer) { + pcie_clear_device_status(pdev); + pci_aer_clear_nonfatal_status(pdev); + pci_aer_clear_fatal_status(pdev); + } + + pci_info(pdev, "CXL uncorrectable error.\n"); +} static int cxl_rch_handle_error_iter(struct pci_dev *pdev, void *data) { diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index c32eab22c0b2..1354c7cfedeb 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -886,7 +886,6 @@ void pci_no_aer(void); void pci_aer_init(struct pci_dev *dev); void pci_aer_exit(struct pci_dev *dev); extern const struct attribute_group aer_stats_attr_group; -void pci_aer_clear_fatal_status(struct pci_dev *dev); int pci_aer_clear_status(struct pci_dev *dev); int pci_aer_raw_clear_status(struct pci_dev *dev); void pci_save_aer_state(struct pci_dev *dev); @@ -895,7 +894,6 @@ void pci_restore_aer_state(struct pci_dev *dev); static inline void pci_no_aer(void) { } static inline void pci_aer_init(struct pci_dev *d) { } static inline void pci_aer_exit(struct pci_dev *d) { } -static inline void pci_aer_clear_fatal_status(struct pci_dev *dev) { } static inline int pci_aer_clear_status(struct pci_dev *dev) { return -EINVAL; } static inline int pci_aer_raw_clear_status(struct pci_dev *dev) { return -EINVAL; } static inline void pci_save_aer_state(struct pci_dev *dev) { } diff --git a/include/linux/pci.h b/include/linux/pci.h index 56015721be22..0aee5846b95c 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -862,6 +862,9 @@ enum pci_ers_result { /* No AER capabilities registered for the driver */ PCI_ERS_RESULT_NO_AER_DRIVER = (__force pci_ers_result_t) 6, + + /* System is unstable, panic */ + PCI_ERS_RESULT_PANIC = (__force pci_ers_result_t) 7, }; /* PCI bus error event callbacks */ @@ -1864,8 +1867,10 @@ static inline bool pcie_aspm_enabled(struct pci_dev *pdev) { return false; } #ifdef CONFIG_PCIEAER bool pci_aer_available(void); +void pci_aer_clear_fatal_status(struct pci_dev *dev); #else static inline bool pci_aer_available(void) { return false; } +void pci_aer_clear_fatal_status(struct pci_dev *dev) { }; #endif bool pci_ats_disabled(void); From patchwork Thu Mar 27 01:47:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030706 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2058.outbound.protection.outlook.com [40.107.237.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 504C613B5A0; Thu, 27 Mar 2025 01:48:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.58 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040135; cv=fail; b=j/BxHXhTzfvJ4ZvukgivtYcJ6l+YGDd/AWUgS5UnECjaplMR+iLCu/c3PthGwGtu4/yEsqSbOWM+oTFscHODesHQuNJgRJlBXYF9r23SSxiVYzXhT8H0Ya7yDYivAhf3wShGFMayuhZEpppJCXjn0I94CfkmoShDHOSKncvHbkM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040135; c=relaxed/simple; bh=sqR5yahNyqmNJfiop7IPYm9UTRaFaYODMSuHzFECLaU=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=j7E9wvcTHRAHiOdFW9Vx2fFfjHEdxW7x51utLxhv7s/SM7SLJVW9VvQMdDz2k1pw9ttaFESbyNUzFQyH1WAPYWsFq50eG593Q2jbAatCt7ypab8T1ib7OuOgwzl8O9E3dSRLh6UFJ6dkQH2/QD5PD3gbaCZ8awniQkoqfIkhRgw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=hU+TxITW; arc=fail smtp.client-ip=40.107.237.58 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="hU+TxITW" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HDouoolWRyWBTPorL+BZb0noaY/tpi/n/efZNtUa/P8nOQCv+rIePw21cThh9gz2THIc4X3WrHcQefiS2Gco8XfRvtTc0c5w76Km+kwTTzv9w5lZwjIkSxzETNFV8FxopxvvHQlmtrrD+0v5HkOFA7Lvg0CPF4zTyiVRqxLPQxgMAlLrcyqU7Zu+cLgAp77ArsLGFZKPJ7fVcBKbu9d4790/AIzpxdtWMgpm8EdiOtvjM+wPCJCfIpwJoMi03Y4KR8IUYTAOIGi421xA/6II28ftswd73VQJWOJ0BmoeMpVnhMb0CIlE/2wMq0r14dn/6TrDacIBbDNI8zXmof2lGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cxZRgW4p4bQDvnFeIGlVFuNyT5lewFZIawA3zxvobVI=; b=D/en420PimeYiiVOASbfieCag9R0oK0jZgidps2D8qohFqJO31xPa1nG7b3RtJU3FTBLfyda8DfzLYTF0eVm9q7Ns0iI3HisUpUx3YmeD1FPTC/CyoNJ8/FhImiwJ4d0DoQjcjZsYYW9wkMghwSDbU+omUD9EMYbRtAGaRhjpNJ9X2vb3DghOLOxuHYll+NBwM6Rdd5XptdTrk0l+kyBM66KY6GRMlH9y8ITlaGW7pxom1dubinJ/5jjwfgwmL2V1auyWRLGwMNtWoz3/9oHNtYqXvZ3DrxeRWDfnhYvmEgOAbpVSnNtzheoBcVU3hvJXffcH9mMwJm/oSAnedcbbw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cxZRgW4p4bQDvnFeIGlVFuNyT5lewFZIawA3zxvobVI=; b=hU+TxITWPuihz4CTZqDdQZZIr6qFDQGEGCeazcBz5wjVQS1lSL26xKBffH/SPeH9RJuTorlrk5o85ZirhPCUCUGnfYjhWHO5ByX/SJ2m2mYP4eX7wNnQvrA6rNj+8h1zmgrZ3rhA2hs9yD5fn2zukhikqk1BOKZyN+GNevE9gWs= Received: from BL1PR13CA0145.namprd13.prod.outlook.com (2603:10b6:208:2bb::30) by SJ5PPF0C60B25BF.namprd12.prod.outlook.com (2603:10b6:a0f:fc02::98a) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Thu, 27 Mar 2025 01:48:45 +0000 Received: from BL6PEPF00022575.namprd02.prod.outlook.com (2603:10b6:208:2bb:cafe::4c) by BL1PR13CA0145.outlook.office365.com (2603:10b6:208:2bb::30) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.42 via Frontend Transport; Thu, 27 Mar 2025 01:48:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022575.mail.protection.outlook.com (10.167.249.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:48:44 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:48:43 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 07/16] cxl/pci: Move existing CXL RAS initialization to CXL's cxl_port driver Date: Wed, 26 Mar 2025 20:47:08 -0500 Message-ID: <20250327014717.2988633-8-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022575:EE_|SJ5PPF0C60B25BF:EE_ X-MS-Office365-Filtering-Correlation-Id: b35d1f79-6987-4f9f-edd8-08dd6cd18227 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|1800799024|82310400026|376014|36860700013|921020; X-Microsoft-Antispam-Message-Info: ULo/SNJsIEJFNefoweXVXx3TJPDeX+rOD4ErMvDULO78Rtz04wDzGlxqCfZa0tnVzuinwIJqUgL/v7LsrM5zRQRkD9b+XG5OBo9ntHqF97q/viNQdQw/ieG1BGF0dP1uT+xn8YtxpPuUYovsJ66lFbgYy3/U2XAurSL2R0/4wnc6ydHLyR4WIL9QF2sdSojp6ysO7/uDZx5lGdf4EZvqqApl+iYcbOF7w18XJJSKSOI4ZziVF/sU9lHCrtH9QIeK3z/fjHvKtMAr24f+pPJ7FlzA0rwNHGapQWotT+Ib7ZITbRTPQ0bQtiXvhuTZZTrioBeWy6STLursmdSPCBl4yt5qzdw30qmHSM4QJhFquXFFcr3wPw7WIgsJEAgGMT2jcHATSNCNFSV6ChOWy49d+c3rzOX9FaXHiilLUPCBccURPUF7MmeuWa/1ebQQJwQCLpXTEmFin5QiiwdBqhqVS01IwEnKvWJlUznuZE6hx4Ym9psSVNfV+2K7XObnqOhF+Yqswxm3KssReQHMVvoRkLK/ME5WPYbLl5QYRAHEV8+xeOJCk8yeB+dCWx1tBjikwojOtXBia4rnNlT+5jvFrbi+TDBGSJeegsz12rfgd6g81uKwK9ot5YoxL/5VMz99QikprYRMv/jyunERxXj9RuvHRe4l10zH1ltGvpKQXMLhgydgTxLrc9SS3g3iZoU63fgE9CaCNgpVQ5C4P0M3/fGQ2dtFGfUo0pBsfW1sl4WdPY/ZnilfgVswwFHcFcjESZqbyglZwz3LzmtVm+p6LUb9meGj8g6mH+XyI/c9lK6dTNeXViZTdDen9UvRWT+Q+9JtlB6jZjD++Nt3cAUGF/D431UvTiwX5DaYPvfUwlqRi9Js3xi3dDHn/P9nblTan/iAey1BAwwBykALZ2rPXK9xKhUNgcHk59Y1fs4lGfSWddB4xSqfw/iDU2tDNnW8JK+ISP9IcUi0qmf788eRYnGmT6oBS827wmPfA7ckiOSJcjhRni4uE3kgNA56ecU15EcRD2ykc4Hs4JBLwkrWgfvrtID5BJVLpYrFbgoSz4Gryj/WqxGKzFnU0v2CrMCcJMLOoQ9kss1j6AHxNogy4easjKBNmHNuJdg48XTJ3Lt9c1Nrlmk5GwfqkAHneoQ3BDNYkqxwjouHGpgWOTETZ6g6xssZUo4NcNkR0QDFsjcKmo0V47A36SoG6dvhFsWPtGhqm/UTaQPh52649gn5vpW7XFiO6RBzoQmafHH1Ml14jQBsX4lcWmYeoZD3ZZ29B/o+BVcuI8Wn33cEA2qFROokTwZzEmMDndUxE5U6UuDWTI3i6LRkLT5gD3yAa+O7+OhV1gOnVAI2gRxApaz+3qFmBCEnyoAXP+nn8/foUO4xSBCrPx6fLHCMOVr3huMQ+mk/FT1JpARPhPaEQ8GVXJP0iBh34O7CMBfFqWAJS/4/zJCAfo2gapzkfvzyRNinSO181c903s6+a7OLDrVGw3KveS1zWhZCIf2NtgbDXBmh2X+hfxRzXf3yQrEj+SEB X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(7416014)(1800799024)(82310400026)(376014)(36860700013)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:48:44.3629 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b35d1f79-6987-4f9f-edd8-08dd6cd18227 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022575.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF0C60B25BF Restricted CXL Host (RCH) Downstream Port RAS initialization currently resides in cxl/core/pci.c. The PCI source file is not otherwise associated with CXL port management. Additional CXL Port RAS initialization will be added in future patches to support CXL Port devices' CXL errors. Move existing RAS initialization to the cxl_port driver. The cxl_port driver is intended to manage CXL Endpoint and CXL Switch Ports. Signed-off-by: Terry Bowman --- drivers/cxl/core/pci.c | 73 -------------------------------------- drivers/cxl/core/regs.c | 2 ++ drivers/cxl/cxl.h | 6 ++++ drivers/cxl/port.c | 78 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 86 insertions(+), 73 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 96fecb799cbc..27ef3d55a6f1 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -734,79 +734,6 @@ static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) #ifdef CONFIG_PCIEAER_CXL -static void cxl_dport_map_rch_aer(struct cxl_dport *dport) -{ - resource_size_t aer_phys; - struct device *host; - u16 aer_cap; - - aer_cap = cxl_rcrb_to_aer(dport->dport_dev, dport->rcrb.base); - if (aer_cap) { - host = dport->reg_map.host; - aer_phys = aer_cap + dport->rcrb.base; - dport->regs.dport_aer = devm_cxl_iomap_block(host, aer_phys, - sizeof(struct aer_capability_regs)); - } -} - -static void cxl_dport_map_ras(struct cxl_dport *dport) -{ - struct cxl_register_map *map = &dport->reg_map; - struct device *dev = dport->dport_dev; - - if (!map->component_map.ras.valid) - dev_dbg(dev, "RAS registers not found\n"); - else if (cxl_map_component_regs(map, &dport->regs.component, - BIT(CXL_CM_CAP_CAP_ID_RAS))) - dev_dbg(dev, "Failed to map RAS capability.\n"); -} - -static void cxl_disable_rch_root_ints(struct cxl_dport *dport) -{ - void __iomem *aer_base = dport->regs.dport_aer; - u32 aer_cmd_mask, aer_cmd; - - if (!aer_base) - return; - - /* - * Disable RCH root port command interrupts. - * CXL 3.0 12.2.1.1 - RCH Downstream Port-detected Errors - * - * This sequence may not be necessary. CXL spec states disabling - * the root cmd register's interrupts is required. But, PCI spec - * shows these are disabled by default on reset. - */ - aer_cmd_mask = (PCI_ERR_ROOT_CMD_COR_EN | - PCI_ERR_ROOT_CMD_NONFATAL_EN | - PCI_ERR_ROOT_CMD_FATAL_EN); - aer_cmd = readl(aer_base + PCI_ERR_ROOT_COMMAND); - aer_cmd &= ~aer_cmd_mask; - writel(aer_cmd, aer_base + PCI_ERR_ROOT_COMMAND); -} - -/** - * cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport - * @dport: the cxl_dport that needs to be initialized - * @host: host device for devm operations - */ -void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) -{ - dport->reg_map.host = host; - cxl_dport_map_ras(dport); - - if (dport->rch) { - struct pci_host_bridge *host_bridge = to_pci_host_bridge(dport->dport_dev); - - if (!host_bridge->native_aer) - return; - - cxl_dport_map_rch_aer(dport); - cxl_disable_rch_root_ints(dport); - } -} -EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL"); - static void cxl_handle_rdport_cor_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index 117c2e94c761..f3f85a753460 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -199,6 +199,7 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr, return ret_val; } +EXPORT_SYMBOL_NS_GPL(devm_cxl_iomap_block, "CXL"); int cxl_map_component_regs(const struct cxl_register_map *map, struct cxl_component_regs *regs, @@ -517,6 +518,7 @@ u16 cxl_rcrb_to_aer(struct device *dev, resource_size_t rcrb) return offset; } +EXPORT_SYMBOL_NS_GPL(cxl_rcrb_to_aer, "CXL"); static resource_size_t cxl_rcrb_to_linkcap(struct device *dev, struct cxl_dport *dport) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 73cddd2c921e..b2b55083886a 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -315,6 +315,12 @@ int cxl_setup_regs(struct cxl_register_map *map); struct cxl_dport; resource_size_t cxl_rcd_component_reg_phys(struct device *dev, struct cxl_dport *dport); + +u16 cxl_rcrb_to_aer(struct device *dev, resource_size_t rcrb); + +void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr, + resource_size_t length); + int cxl_dport_map_rcd_linkcap(struct pci_dev *pdev, struct cxl_dport *dport); #define CXL_RESOURCE_NONE ((resource_size_t) -1) diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index d2bfd1ff5492..d5ea8b04400b 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -6,6 +6,7 @@ #include "cxlmem.h" #include "cxlpci.h" +#include "cxl.h" /** * DOC: cxl port @@ -57,6 +58,83 @@ static int discover_region(struct device *dev, void *root) return 0; } +#ifdef CONFIG_PCIEAER_CXL + +static void cxl_dport_map_rch_aer(struct cxl_dport *dport) +{ + resource_size_t aer_phys; + struct device *host; + u16 aer_cap; + + aer_cap = cxl_rcrb_to_aer(dport->dport_dev, dport->rcrb.base); + if (aer_cap) { + host = dport->reg_map.host; + aer_phys = aer_cap + dport->rcrb.base; + dport->regs.dport_aer = devm_cxl_iomap_block(host, aer_phys, + sizeof(struct aer_capability_regs)); + } +} + +static void cxl_dport_map_ras(struct cxl_dport *dport) +{ + struct cxl_register_map *map = &dport->reg_map; + struct device *dev = dport->dport_dev; + + if (!map->component_map.ras.valid) + dev_dbg(dev, "RAS registers not found\n"); + else if (cxl_map_component_regs(map, &dport->regs.component, + BIT(CXL_CM_CAP_CAP_ID_RAS))) + dev_dbg(dev, "Failed to map RAS capability.\n"); +} + +static void cxl_disable_rch_root_ints(struct cxl_dport *dport) +{ + void __iomem *aer_base = dport->regs.dport_aer; + u32 aer_cmd_mask, aer_cmd; + + if (!aer_base) + return; + + /* + * Disable RCH root port command interrupts. + * CXL 3.0 12.2.1.1 - RCH Downstream Port-detected Errors + * + * This sequence may not be necessary. CXL spec states disabling + * the root cmd register's interrupts is required. But, PCI spec + * shows these are disabled by default on reset. + */ + aer_cmd_mask = (PCI_ERR_ROOT_CMD_COR_EN | + PCI_ERR_ROOT_CMD_NONFATAL_EN | + PCI_ERR_ROOT_CMD_FATAL_EN); + aer_cmd = readl(aer_base + PCI_ERR_ROOT_COMMAND); + aer_cmd &= ~aer_cmd_mask; + writel(aer_cmd, aer_base + PCI_ERR_ROOT_COMMAND); +} + +/** + * cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport + * @dport: the cxl_dport that needs to be initialized + * @host: host device for devm operations + */ +void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) +{ + dport->reg_map.host = host; + cxl_dport_map_ras(dport); + + if (dport->rch) { + struct pci_host_bridge *host_bridge = to_pci_host_bridge(dport->dport_dev); + + if (!host_bridge->native_aer) + return; + + cxl_dport_map_rch_aer(dport); + cxl_disable_rch_root_ints(dport); + } +} +EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL"); + +#endif /* CONFIG_PCIEAER_CXL */ + static int cxl_switch_port_probe(struct cxl_port *port) { struct cxl_hdm *cxlhdm; From patchwork Thu Mar 27 01:47:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030707 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2081.outbound.protection.outlook.com [40.107.236.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D98E145FE8; Thu, 27 Mar 2025 01:49:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.81 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040144; cv=fail; b=j7626C824xtI4oEPOXU6HC8MZ8CZZHJdcVxxXujFymzYea3JcHA09o3xEgDRTClzE46GVWh4NALRDCVCeb69OI7J9/E2JreR2GCjSWohuOSunUpyoc/3fgxs3VzXDEcqR9efKEBDCle/8nFMw/gDIKn8iBAVx+XHVeaTYL8GZ6s= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040144; c=relaxed/simple; bh=bSEUIdqsyB747ePPRvkFgaQ7VOiO4shF5t5E6i8IKLY=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=gV+hhkCt1zAPpFSjL0o81Wjf2lQDQZJr7XI578l6zF1ciesXynuJEszbnyhbWDGG65xLOs2y4dJwWnlRVbyBxCv4sdageSFZm55WWJxDdJ0Y00PzByha7mDdv7ej9lsTPJVsA3KSPwM29XWD+BAAxOd4NJj2NkrkXBbH7dFAsIQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=ppLN3OI4; arc=fail smtp.client-ip=40.107.236.81 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="ppLN3OI4" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kXmUq1bZJc3sS5sTzkCFjlrbn0Al/o/obNxzFA0LN17oPlTvR6nvQjRjHPGccuoidLpLDn57KB15XY8lhWdkwrmMKOA4u2vB6JUiJkqqeE6sXZUoW1ozc6PVujgZceDOC0BCsz9efFa0TZCapvigUgDnQ0KtaB+79IhElnfj5dncIcb4GuMA0Outykh41hfvAtJy+OrCY18/t2GDhhggbocsNuiqGRWYSJRuRKIIhSN9qugCuGyXvwonk9+U+B5ZevR5Ngne0gsT4WWxfx1Y2soVoWUQvzI7mqJkfCDZYhTkwrj/YHrldwiyl1Mq011Ba4mrNN6Ypz9u6IyYrTowQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DCdaIMK0zvBHMQ/KWNARZqDvqzGFnL4ri8TgCcrn5rY=; b=tgBu+aAVwYer+20rO9J4H1M2d4mKpks9OO2HvBZBtuoUmpUtQXNBIUMVt6SZip0XdcdIcsOoWeWwqy3NclN15ac/S+jw2klGmYBD4QlAiHZumGQ7Gxv9OjMWHa+hjuiVn3Zp3KLx2EKZFLJ6O33b2JXlTkwWq5xh6SAt68ey/U7o9q0zN01GgozSpvvPFMPsHyoRZriKr7e1rqsy6htH33wPgpJQZRduGbHoe4BA6V51cSiWNg5qX688dIcLQiZnVrGbk+2VdoghwLbk5OfmLBi/QlPi9Ey90LF3849V2JUu8pxokAtEP4nI0iO6C6Vw7xZ0i2+K05maeDQvY/PAOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DCdaIMK0zvBHMQ/KWNARZqDvqzGFnL4ri8TgCcrn5rY=; b=ppLN3OI4TWSPysjsf4qfTxPDQ2gsxyLvT0num1Fk6CFxRSeCg0fBG9N4soZvK0XBhHGFPZJ1nFZvI9UdspPdrFiNXUfUrGIa3PVn+74+Moe2Z9S+KnUplWdD0yM/wR4uItENPvnABzFFp8y8/YEwkSngDSxPx8grKPuiFRsPjAQ= Received: from BN9PR03CA0269.namprd03.prod.outlook.com (2603:10b6:408:ff::34) by SJ2PR12MB9162.namprd12.prod.outlook.com (2603:10b6:a03:555::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.42; Thu, 27 Mar 2025 01:48:56 +0000 Received: from BL6PEPF00022570.namprd02.prod.outlook.com (2603:10b6:408:ff:cafe::c0) by BN9PR03CA0269.outlook.office365.com (2603:10b6:408:ff::34) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.43 via Frontend Transport; Thu, 27 Mar 2025 01:48:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022570.mail.protection.outlook.com (10.167.249.38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:48:55 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:48:54 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 08/16] cxl/pci: Map CXL Endpoint Port and CXL Switch Port RAS registers Date: Wed, 26 Mar 2025 20:47:09 -0500 Message-ID: <20250327014717.2988633-9-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022570:EE_|SJ2PR12MB9162:EE_ X-MS-Office365-Filtering-Correlation-Id: f79d6f01-2975-4e7a-d089-08dd6cd188e6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|1800799024|376014|7416014|921020; X-Microsoft-Antispam-Message-Info: aIsaTqafNg42mjgK+TCZWsSRN8JrxaySzW+26ZKay0Idsl5LjFEgwG9pQ70KTjZ2p5ZZcjswnHVyR+ZQqHqvbatREgrhy5FOmtuBst/Lh+DDpuLj9rcQxPXDNM6fRukhi3TmbJaunucEx96vq35m6SkqufDESo+OVt19biw86ZSGa9IpnUUmdsF7s7NhmI0QzDJcQIJ2U7Jwp6Um2Z7uJNaF87A1+nnYeYNvoTS8/SAvzST3/5s2Jar8hJNApnXcr2nuOB6Pj5a1x96vuS4Hmjp7wikQtkCyn5+UJ2VhpVMD2P6eBBzl2G8F3ZeNqPd2nVBoJXGaPEJZ7ZnguChP3uLM/zAqlc0vwV0ZbNxLXhR/2gBkDAhgMJ3a5EwcHv0Po32A5Q3Nl7IuTCp9dg96PFY8c5VnOtIgW9ZWn8VBuCF9bemte4e3U8KMNCINvffn07qzUMDxCRm92mK9dy10GUvibsCwIVMyJsttsT6ScGLWY0vYfoEZU97RR1jb1P/I3CcGptGONVrNZSi1owuTuj6aqoDhWzaz5lTtbRrcbUPIabhDvK4crG3jKOjqSIp8w0TkRHiH2jnLrOcBzLLL7bC2piyV/BFDojK12m3dj6Z+qjCh3EW0a9BiV1GP2ayxZsgrBbt+VK9QMoAxwtPM3DhjcOHukgnuhQgNbIe0V+y9M1DJh2R7IwdcsZhlRD+wF/oVtjBodYNai70PRFBUD8CLdOWoKCISjrRnNDmeBsuyc4K6B7kCX2uQN20Lj9w3Q3BsuSe4fAkUUnG3GiwEinFJ6t4tSvbIzwJBgIj51LEFQ58ALJANQWgcQ2BKM6DqU+8HD82BK/zbyl/x1f0XavM5PN2De5PxK2KMK+boxtMadJGHipsFMTh+hAwnKj08CODKSSBq/BQemmGb/rC/TfWrmANI0lUuqG+CO7s4b1Fe2S0Avp8gnwSPjxkaD/OkOeEO8UlEurgromyRUFHLP1tdcNP8ad7eH10kEtFxg5TRkXi32UUgAl+susciSz5jYNTT/xvT+B/WT60s8UtRijOv+iQGmXXfMRYJcgmu4BLzDIiVLnaPAtK1KubF5hYyKT6zAxtKlj5VR5iCmM9mO/O6C8HminyTGm2V7lMMk7/4WAB4q7Fb8JXUuH48XJCCBTj5azYaog6g80b44xO/4EsCVrDf7hISXvAIugJ7TCB5pz7qlQf4P6IsdfVZb76ffF42SKsSb7WUSm6LdYbUEzaM1vGYmzrf3y+AHHHBPqkBNwUvcMoRbTB1eTQl6anHkhCe2j32YOHXyoB1FVcDL7X8XbK3T1Bs66c+nHJlLzpH/WqFTLnXrTcDzex50DIhFw4oav9thEpsaNBJz38yG/VfIgeA/7fHNJNQ2//Hx3SvXItWhwsNS+UrS7ckblq6Qu98LgIAWbSOIaY0JOriCYEwTadty24zR49Wpq7Sp1rU8SrX+59Umrv2BiuvrdnhQHnxCTsKwK+oeVdYsJOt6nUKRIkixjEEydTCKNEZ/rsKQ4jSgfTYGvm3Blz9k6Hh X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(1800799024)(376014)(7416014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:48:55.6802 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f79d6f01-2975-4e7a-d089-08dd6cd188e6 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022570.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB9162 CXL Endpoint (EP) Ports may include Root Ports (RP) or Downstream Switch Ports (DSP). CXL RPs and DSPs contain RAS registers that require memory mapping to enable RAS logging. This initialization is currently missing and must be added for CXL RPs and DSPs. Update cxl_dport_init_ras_reporting() to support RP and DSP RAS mapping. Add alongside the existing Restricted CXL Host Downstream Port RAS mapping. Update cxl_endpoint_port_probe() to invoke cxl_dport_init_ras_reporting(). This will initiate the RAS mapping for CXL RPs and DSPs when each CXL EP is created and added to the EP port. Signed-off-by: Terry Bowman --- drivers/cxl/cxl.h | 2 ++ drivers/cxl/mem.c | 3 ++- drivers/cxl/port.c | 56 +++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 59 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index b2b55083886a..0d05d5449f97 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -592,6 +592,7 @@ struct cxl_dax_region { * @parent_dport: dport that points to this port in the parent * @decoder_ida: allocator for decoder ids * @reg_map: component and ras register mapping parameters + * @uport_regs: mapped component registers * @nr_dports: number of entries in @dports * @hdm_end: track last allocated HDM decoder instance for allocation ordering * @commit_end: cursor to track highest committed decoder for commit ordering @@ -613,6 +614,7 @@ struct cxl_port { struct cxl_dport *parent_dport; struct ida decoder_ida; struct cxl_register_map reg_map; + struct cxl_component_regs uport_regs; int nr_dports; int hdm_end; int commit_end; diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 9675243bd05b..29dc4a624b15 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -166,7 +166,8 @@ static int cxl_mem_probe(struct device *dev) else endpoint_parent = &parent_port->dev; - cxl_dport_init_ras_reporting(dport, dev); + if (dport->rch) + cxl_dport_init_ras_reporting(dport, dev); scoped_guard(device, endpoint_parent) { if (!endpoint_parent->driver) { diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index d5ea8b04400b..1b8dc161428f 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -111,6 +111,17 @@ static void cxl_disable_rch_root_ints(struct cxl_dport *dport) writel(aer_cmd, aer_base + PCI_ERR_ROOT_COMMAND); } +static void cxl_uport_init_ras_reporting(struct cxl_port *port, + struct device *host) +{ + struct cxl_register_map *map = &port->reg_map; + + map->host = host; + if (cxl_map_component_regs(map, &port->uport_regs, + BIT(CXL_CM_CAP_CAP_ID_RAS))) + dev_dbg(&port->dev, "Failed to map RAS capability\n"); +} + /** * cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport * @dport: the cxl_dport that needs to be initialized @@ -119,7 +130,6 @@ static void cxl_disable_rch_root_ints(struct cxl_dport *dport) void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) { dport->reg_map.host = host; - cxl_dport_map_ras(dport); if (dport->rch) { struct pci_host_bridge *host_bridge = to_pci_host_bridge(dport->dport_dev); @@ -127,12 +137,52 @@ void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) if (!host_bridge->native_aer) return; + cxl_dport_map_ras(dport); cxl_dport_map_rch_aer(dport); cxl_disable_rch_root_ints(dport); + return; } + + if (cxl_map_component_regs(&dport->reg_map, &dport->regs.component, + BIT(CXL_CM_CAP_CAP_ID_RAS))) + dev_dbg(dport->dport_dev, "Failed to map RAS capability\n"); + } EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL"); +static void cxl_switch_port_init_ras(struct cxl_port *port) +{ + struct device *dev __free(put_device) = get_device(&port->dev); + + if (is_cxl_root(to_cxl_port(port->dev.parent))) + return; + + /* Check for parent DSP */ + if (port->parent_dport) + cxl_dport_init_ras_reporting(port->parent_dport, dev); + + cxl_uport_init_ras_reporting(port, dev); +} + +static void cxl_endpoint_port_init_ras(struct cxl_port *port) +{ + struct cxl_dport *dport; + struct cxl_memdev *cxlmd = to_cxl_memdev(port->uport_dev); + struct cxl_port *parent_port __free(put_cxl_port) = + cxl_mem_find_port(cxlmd, &dport); + struct device *cxlmd_dev __free(put_device) = &cxlmd->dev; + + if (!dport || !dev_is_pci(dport->dport_dev)) { + dev_err(&port->dev, "CXL port topology not found\n"); + return; + } + + cxl_dport_init_ras_reporting(dport, cxlmd_dev); +} + +#else +static void cxl_endpoint_port_init_ras(struct cxl_port *port) { } +static void cxl_switch_port_init_ras(struct cxl_port *port) { } #endif /* CONFIG_PCIEAER_CXL */ static int cxl_switch_port_probe(struct cxl_port *port) @@ -149,6 +199,8 @@ static int cxl_switch_port_probe(struct cxl_port *port) cxl_switch_parse_cdat(port); + cxl_switch_port_init_ras(port); + cxlhdm = devm_cxl_setup_hdm(port, NULL); if (!IS_ERR(cxlhdm)) return devm_cxl_enumerate_decoders(cxlhdm, NULL); @@ -204,6 +256,8 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) if (rc) return rc; + cxl_endpoint_port_init_ras(port); + /* * This can't fail in practice as CXL root exit unregisters all * descendant ports and that in turn synchronizes with cxl_port_probe() From patchwork Thu Mar 27 01:47:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030708 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2054.outbound.protection.outlook.com [40.107.92.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DFF3D1474B8; Thu, 27 Mar 2025 01:49:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.54 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040155; cv=fail; b=UPBflMuPLZjdVfeLqp/etncpV6xhZuPdShoSGAVQ2lpwTlBJedwvGRB3ZHrIQofWGCa8xiehM6+Jt5YsoT/K4NxM1+naTJYP3j//UzBBGjElbJ5zJu4VgyiTqFIbTclo5bUKCKXlxUzmZVtGNkTRCiZVe7HuwPbtk0LEVbJuw6c= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040155; c=relaxed/simple; bh=EOao7UuycJIjU4sQciwSEqKuY4hiujkyspG182Raixs=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=bdCPlIYHc3tov/rF6CNyM0vx5WvNYsKVyOpZQxapFSyH18MIyquBL4nmGynt+4sWngfcsrDfBTrZ6Q0XK8ZEu7xa66PdXf9K7SyonR85OEqILeN0A5Vr+LLT3sNFA19g8e5lAaJ5q0s7E68TqDgbxAtG4oWORDUX7quWOnnuSD8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=RfKjA0gz; arc=fail smtp.client-ip=40.107.92.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="RfKjA0gz" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KMB+ZRZjxy3ziDrCWCDodMT+gILW9QgSWGl01MNDbfyfbS8eeY0pebwxu+gmmTKOWxJaJanJ/NbG6/xmUVGUWW87OLT0tFobeA04pp2/eEd8OZwcpP9VTd5GXFxuXsTbFFDYIH3PXHcKfzZTk4Pz88H36oaSYCWKcjwQDn0TN3mD1wwLJVs8htImWMXZf5ZWLxMg8PEVxEjDsQN9z/aPlpjVZ2SpOyvkKW2Zt/t+bDNiP5sY3i0F8uc4qLduM3tc3WJscC/1yw0cLvVU1GKtRDW7CFpFiIc4TRLI4KDWvo99q9Tp0XzJbAAjER5C8HcgKSHkEyqLljhs9qNbJNhkcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lr9RGJcGudcjfMDMdf5UoKSOS7NvOBm3XEo9KYK6trs=; b=Z3CcVzg22biiOAiI7w8X8FKNunbe3cR43sely7hhffpuMnTqJU66ZYXwvAH3Hx40JwNtFIlczTkKLZIxs8xthRIWh7oW3fAIus1l6HG0JYW7OiLOHaC1K1el2y31PPwUCZaWUpi27tO/whtEfHxdfTwrEbBBljK1yk/vSzSsbDBLcGgiLWNenI9PcI024Dw5unff2i9Dk/79C5hTs6cbg7ZuwFd4ECv5FFvwwwFRTb7EomskeQjQ2CstC8a7thUvEJx5tR6C2yDgOU2Z650+V/pmDdopHEc5lu9qRZBUoVY1JQ5ClmVzbM1HE7D+/ZbkYsFn+SJw73M+9S90Ihkr+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lr9RGJcGudcjfMDMdf5UoKSOS7NvOBm3XEo9KYK6trs=; b=RfKjA0gzKVSm4fsQQpeYZlmAz0ZL28ecAlGJmMdINfbllvZIGWDkHIiKJj4EbZyvE3qnplLghTRAgYL8cRzyatm/57ukJCf4sfrtG3r7hSB4Zm+Nc569djlqPWgJ3LzaDTHMiUGq2t06Z5nAZ5L1nqWcGZKFYeBeLp/IE7hXgFY= Received: from BN0PR04CA0004.namprd04.prod.outlook.com (2603:10b6:408:ee::9) by CY8PR12MB7611.namprd12.prod.outlook.com (2603:10b6:930:9b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Thu, 27 Mar 2025 01:49:09 +0000 Received: from BL6PEPF00022574.namprd02.prod.outlook.com (2603:10b6:408:ee:cafe::cb) by BN0PR04CA0004.outlook.office365.com (2603:10b6:408:ee::9) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.43 via Frontend Transport; Thu, 27 Mar 2025 01:49:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022574.mail.protection.outlook.com (10.167.249.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:49:08 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:49:06 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 09/16] cxl/pci: Update RAS handler interfaces to also support CXL PCIe Ports Date: Wed, 26 Mar 2025 20:47:10 -0500 Message-ID: <20250327014717.2988633-10-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022574:EE_|CY8PR12MB7611:EE_ X-MS-Office365-Filtering-Correlation-Id: 340b9a15-94ad-4714-a16e-08dd6cd190ac X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700013|7416014|376014|921020; X-Microsoft-Antispam-Message-Info: op2wFLx+ydZlhL1kRHOPekKkwTC80UP1bIxKcB9V7bIiCs81NCzfzkDnx7BWa60bO8QDR3y340Sm9O0a3sdQxwB+mH14r7Fe6inzSC2rN6LQfFZVBFcNVRz7YKtK3JGwoPnJdYX8k34Og0iLTsG6iXE3aztG6Db63yCYfxvA75v5U1v8TC3oyxjIvUfbx3a51uBr13S8B7tUq/YyP/6aRISIFya+pX7F9PLHzrbbP6vKahkVJkZaGevfQ5fiQ+ajeKQB8d9F7QH7L3wi/XcwZ1uZtfWRSNIjdZ4wTXWDF5Toh8FJ1UgnfqM3lU/zGLO92r3ByWtd+edWvWg5shapLvU/VwuMD6aZxPg7jpkz+clyUbuaU0/NTvEZUfrV1A37AAUov5jaibHlokZxJ39mZ4DjRPaFwnVaI/l8y2JOaDrmY5jV6VuENVDW5RKDXN8Yz69RTn1f9CQbFHoxp1kolCfTXNNdW/FBl40Kd2lmrDBb1rB2Nii9n+2Z1p/ReTdrZSbXiIJOkPbdt3XTK4pvcsiu9WICjIEnBVjBrZAIE4jcLmdnMkmdnmLwkBtshwFUY0hvxUubzda9oPFO0X7cjhcdxrtv1Q+QN/5zi1nb49GQMJvu84Ce6ktlRNFYR0ne6sgOasXSHvzkGhWe4KqaMHnvzU2C4zc4BY6PwwDyFD3bt/XFKluytZ8KAlaciyLF1hWd1+AWmYq2/9MjS6Ei6z5QI2AOlVQcpq7Ifc5ChcSbfFdlSSPgH33FVgjpkj9EL9WxLbmV3oB0j24qIGpy5fMaUjoGCmVRrsenpo+9oDXVL11f6Jf2LAoJ/ITz/6f6wtjXsrxsUGuprdC+lzIxPYh4nRqG6Dbp0Cm30GwXAYyksQHvcm6NXXKL563p5GK/ZtwphxVQfJrQb98z6gbW3heDRvnG9OUemdO4P4qKP6tc/PqALQ+QGcRWhpA7T7/B40NLucjdUOPfWmuR2e12fNLkQoE5lvBV0alhAe3jIYKlG2EVSHiGn/73mawybgN0vwUzZj0w2OxbqKSpidhlkRJTPwryxAxMWgI8rHrEEAi4IVmHN0hoEIW7dCknHPLHbdPYPlKzqrL/s9GlN87GjCInPtLye4BEqLSgNqcguJeuG4hF4Gn2ZzOJwSRDcT3zY7GMek+05vU4nAwYEnL63kGLpCCmLKoXQ/ZpGh6JceH05POTI4dnB7TLiTLuDDv6/TflhGnle+3cNoVLa3jgwsnF7N22wb+pKPrv2SzSuC/qObq5KSfKrxQMq02oSNh+yfNwzZJushL7KH41RsCkfx7m9EiJ/qdrajtpV0ZH7OBImQu3sSaIbCTGtGnK3lcczuq41GhA/GTGKTp1Ly3PkxrzZ8mRPwe8Eh5CKn7gGwrZfBWyWVcq5qI/LngFoOOgWVQDxRcSKMcxplCKfUoUKtBMIJKHh0kF37GYAYbaYpmStdvzQKB4NWuwce7feoBsZCJhH/KYRaDmzE+eK93eTJAXkuvr5VS48vxdgg+804oGR/Mvg86ZvR1pftff4EKE X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700013)(7416014)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:49:08.7207 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 340b9a15-94ad-4714-a16e-08dd6cd190ac X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022574.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7611 CXL PCIe Port Protocol Error handling support will be added to the CXL drivers in the future. In preparation, rename the existing interfaces to support handling all CXL PCIe Port Protocol Errors. The driver's RAS support functions currently rely on a 'struct cxl_dev_state' type parameter, which is not available for CXL Port devices. However, since the same CXL RAS capability structure is needed across most CXL components and devices, a common handling approach should be adopted. To accommodate this, update the __cxl_handle_cor_ras() and __cxl_handle_ras() functions to use a `struct device` instead of `struct cxl_dev_state`. No functional changes are introduced. [1] CXL 3.1 Spec, 8.2.4 CXL.cache and CXL.mem Registers Signed-off-by: Terry Bowman Reviewed-by: Alejandro Lucero Reviewed-by: Ira Weiny Reviewed-by: Gregory Price Reviewed-by: Dave Jiang Reviewed-by: Jonathan Cameron --- drivers/cxl/core/pci.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 27ef3d55a6f1..1cf1ab4d9160 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -650,7 +650,7 @@ void read_cdat_data(struct cxl_port *port) } EXPORT_SYMBOL_NS_GPL(read_cdat_data, "CXL"); -static void __cxl_handle_cor_ras(struct cxl_dev_state *cxlds, +static void __cxl_handle_cor_ras(struct device *dev, void __iomem *ras_base) { void __iomem *addr; @@ -663,13 +663,13 @@ static void __cxl_handle_cor_ras(struct cxl_dev_state *cxlds, status = readl(addr); if (status & CXL_RAS_CORRECTABLE_STATUS_MASK) { writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); - trace_cxl_aer_correctable_error(cxlds->cxlmd, status); + trace_cxl_aer_correctable_error(to_cxl_memdev(dev), status); } } static void cxl_handle_endpoint_cor_ras(struct cxl_dev_state *cxlds) { - return __cxl_handle_cor_ras(cxlds, cxlds->regs.ras); + return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, cxlds->regs.ras); } /* CXL spec rev3.0 8.2.4.16.1 */ @@ -693,8 +693,7 @@ static void header_log_copy(void __iomem *ras_base, u32 *log) * Log the state of the RAS status registers and prepare them to log the * next error status. Return 1 if reset needed. */ -static bool __cxl_handle_ras(struct cxl_dev_state *cxlds, - void __iomem *ras_base) +static bool __cxl_handle_ras(struct device *dev, void __iomem *ras_base) { u32 hl[CXL_HEADERLOG_SIZE_U32]; void __iomem *addr; @@ -721,7 +720,7 @@ static bool __cxl_handle_ras(struct cxl_dev_state *cxlds, } header_log_copy(ras_base, hl); - trace_cxl_aer_uncorrectable_error(cxlds->cxlmd, status, fe, hl); + trace_cxl_aer_uncorrectable_error(to_cxl_memdev(dev), status, fe, hl); writel(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK, addr); return true; @@ -729,7 +728,7 @@ static bool __cxl_handle_ras(struct cxl_dev_state *cxlds, static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) { - return __cxl_handle_ras(cxlds, cxlds->regs.ras); + return __cxl_handle_ras(&cxlds->cxlmd->dev, cxlds->regs.ras); } #ifdef CONFIG_PCIEAER_CXL @@ -737,13 +736,13 @@ static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) static void cxl_handle_rdport_cor_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { - return __cxl_handle_cor_ras(cxlds, dport->regs.ras); + return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, dport->regs.ras); } static bool cxl_handle_rdport_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { - return __cxl_handle_ras(cxlds, dport->regs.ras); + return __cxl_handle_ras(&cxlds->cxlmd->dev, dport->regs.ras); } /* From patchwork Thu Mar 27 01:47:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030709 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2041.outbound.protection.outlook.com [40.107.244.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4AB481474B8; Thu, 27 Mar 2025 01:49:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.41 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040164; cv=fail; b=MlXxFLtXpixI4mMaFbiiBNZd2FuVHnoAXbjG9BJsVZJ0whjpfqa2Ln44r1byVcnP/Y3HqxPaXZ21/iJUnv/fJhadKHXGjQ0OJVTnLmlG1q+o70fbgRMOQ1jRMN1mTBO3JT9OEmgGhSEjYe7gaSShLHS1eegruehGiIBGYUi6ihQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040164; c=relaxed/simple; bh=MYmOxnTC4B0GUlauHCM7cnbpWb7bdEr4UbE9Ni7l3xM=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WthyJQaD2V5/FZT28ym5ahXzFI6rr3X4dplc4wvae1MXtF7JNg2p1PZGh8vh6eDKq5jDZrO/6ROLuA5VCvtfRyva2npNcAY16O01UStQ8rzQ2NUyLnbrSnvb8ziNEcUBbbJz8MbmKzQX+aaVUXvVa5ieXrkBIDnKunndwsen29s= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=AfbdFrJA; arc=fail smtp.client-ip=40.107.244.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="AfbdFrJA" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qEmrXEqpwMJ3xRnMrUL4n99E0RMGGjKBEsEFFf6Lhv99S9QaikZ7s6es9QCtD1RzhmqtFU6sH/JH9ikmKG467BoImB7yPx6d8/L0sYgAdojZ+zpsArDtA9A4Gags7BlOEp2EwmUtV6aTXmxnPlqLbkJC4IAutgR3ojDWLUriuF9QO9oW5277h8m2HqNPcKHc58Gj7cluE02Zsydq1ShZbWe4UuPyiGvql12edK1k/pZn+umaYBabFxQD8sPYu3zhM64CT7EVeibPXCqqbjvIeODf2T+KuEwHoQBk4ne+/LnuXOL7ApMG6b/MUTSFb5N03CJ47RFt9NGgngJP+f9cDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YlrbCBKi48nAXR62tJrL6X5Ze0TjJ+j8s+o4y9ZOSes=; b=dW9SF01Tc3XD5tX8sIlvSjENyNccFtjT2EaF0S0iqWfDpkhQW7p5rNnwnA+YgXigNkzAy3XQNJbC/45rQL140KXonzprbSCSxAawn8myR/5IxUAgIqYPa32GL8IWie4LCx/yAG1DTmuxH+6w6VlZw2LQO8OXJWtphuhmaCYJjO6cARacLqApgL9pAKMhCM5seA6M8uct678aO8/X3Q/V7NWmM4fq9spFnPOD2e4WKxpBJKVKlnaWG1zzLmfMrdafaUKUDy8x8SKT0uRDXsDRotypCGgkEwaZOftZUsPdYzpifpJ6crxfQFf4ifwZ6YmMtkj/b9lDeZdv/xQz1WH4QQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YlrbCBKi48nAXR62tJrL6X5Ze0TjJ+j8s+o4y9ZOSes=; b=AfbdFrJAnHSzSD/Q7YGjikrdJC4mX3MOSjQ+DH6sfuFWQjk5xw2PaDbbrNGa3pv+ml2yYku5hv2hrDxXQoTuOoBzXIsqMNWh34wQs7YhkESoiUqk8o4pIOdScNhabuXjArt8tm+Dqcfa70e3tKp2YRA2CpMNob+bAkPX+ervUqc= Received: from MN0P222CA0008.NAMP222.PROD.OUTLOOK.COM (2603:10b6:208:531::12) by SN7PR12MB7785.namprd12.prod.outlook.com (2603:10b6:806:346::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.42; Thu, 27 Mar 2025 01:49:19 +0000 Received: from BL6PEPF0002256F.namprd02.prod.outlook.com (2603:10b6:208:531:cafe::ac) by MN0P222CA0008.outlook.office365.com (2603:10b6:208:531::12) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.43 via Frontend Transport; Thu, 27 Mar 2025 01:49:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0002256F.mail.protection.outlook.com (10.167.249.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:49:18 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:49:17 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 10/16] cxl/pci: Add log message if RAS registers are not mapped Date: Wed, 26 Mar 2025 20:47:11 -0500 Message-ID: <20250327014717.2988633-11-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0002256F:EE_|SN7PR12MB7785:EE_ X-MS-Office365-Filtering-Correlation-Id: 0cd39bec-1d9c-4007-b172-08dd6cd1967b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|376014|1800799024|7416014|921020; X-Microsoft-Antispam-Message-Info: Ga/i5Z+1F8La8C+KqoP2opQ+TqgI47avlaUH8QEl8wZSUFiaCjSSbm5fz4eqgF63Eh1SxrEMD249dy/GTMld6oRcNmH7ec3ukdlEpzIh+eQj4V8LaepeG9uCr2dmTNIqvZQUVMqS6ezlrmeDG2nOt1L6dCOQ7FKFcsR27JAMWCZ8vQ5DCB92bUt5FuCiaRD6igNxnB4iJzaJADMsg3agCpmU6KjBvAVU2iMWKkI6mccttuNqTESVkPwtbYnbJy3xs6ouvNp2JsY76GvSuGsGy2Zuw/EoH8mQH6FcP0+FQAUn/uHX7KGjoZ4YbeEWQcmnW4OMIdIPYW/ASMdQZca1RbVeSV23qgVJgE3H9+OjJAEcU4+YzeA8LpNvr/Z/bDhkr6U6tM0bxw/A2TE/LJCOjOfEVZaQCYzlu4XluuPfYKtLWI5j+hbNKieSE802yWA4PC9MM1N1IKNg+u/DJNONQvW2FFO+PKwKFUtSDa9nQyantnHhitu2itdNReLaaSVelqZMTU09yxGbeibUaG/3RwK/F+n0/ihGi88MMKtUU3eJVN3pc/2etnggj5jwXV9TWIbQohkKlQfO0UESz/FDqtQbdkEyZJSkooi1FYhp0EtInu5CdGBtkHCkIDTZUHkLxWh/nKGX1Ba5bn72Fs13K3I4/u4TdL+6Li9WZt2qZ+ym9D44gS/Tw1KvtQ4uCEZjg5KOYjFT2UiZGv9Ynrw6StFFhd3TsprpOKY/urf3o7bQY9nUuZ1qET9ALd/9vfr1QuXHj9ehymG9FUmaYZGHlJvkdK8t1YKua3XSs/yOEr42VH0Ug7iMu1PvhorQNoC3mQcrtX/IoeaGRny0GWk/IKh7tA5sVPcSNG12/sdWI0GpHN2r2VZgsqGItLBmkVLBPrMvYUAeZNjBWv4qAKJ8SnQJV7NoluN+M/OW9uLETmthVuCYIhdUkSCFae8e3QNbyjUj07OtW2Bxy5leOGfA7cuqa6CNJk/AD2C4e6sswMHpOoVKjTg3Duu68qFGRqBqz8X+YlQNJbMxJoevuOSkLusxoBBKcSFwFEVCDcIEGmWXtTRQG8xTzKNDptcdyenPeGeKJCi1ICces3VY2ETBcIIaU7gk/Yjn24qBmCJoTPo7XH4IAk719JEMy5kvrSdf7EvSfDA9wMbzmwd5FThgBCGKHq5cfoXvMoNgXrME/CTcimTZ1m3dc8iZMAl0LjkpmPoVjdsEaBcQdqsA/xzH+MHY0CsbdapY+MQBPA+6EVXZHrIXH6p4gI2h1L00m1+Q06CjWRVOMYvKlwqYVZfhMi6m7WqUlWp9lAxmlJ3cYxZY11qwLwOFEWEasZwrnpwM7O45AMsrRpMlm+iMnkKanA9mAWuG+qsoeKYwHzEvXRSU/n7o3fNzvIsmyQ5zGZ8ZaUsYB5xXQ8CFM/4a/tIoqxKOxDIF1ybgULG76a2tujEMa6qWoy4RuJmH+8j6VxBAbsaLDudXs2EyT5ECBV5JMFh4uIpw/WBMC5FPc4Me+PkRIiwssH1/iR7kNW6Jf2Pr X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(376014)(1800799024)(7416014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:49:18.4819 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0cd39bec-1d9c-4007-b172-08dd6cd1967b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0002256F.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7785 The CXL RAS handlers do not currently log if the RAS registers are unmapped. This is needed in order to help debug CXL error handling. Update the CXL driver to log a warning message if the RAS register block is unmapped during RAS error handling. Also, refactor the __cxl_handle_cor_ras() functions check for status. Change it to be consistent with the same status check in __cxl_handle_cor_ras(). Signed-off-by: Terry Bowman --- drivers/cxl/core/pci.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 1cf1ab4d9160..4770810b2138 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -656,15 +656,18 @@ static void __cxl_handle_cor_ras(struct device *dev, void __iomem *addr; u32 status; - if (!ras_base) + if (!ras_base) { + dev_warn_once(dev, "CXL RAS register block is not mapped"); return; + } addr = ras_base + CXL_RAS_CORRECTABLE_STATUS_OFFSET; status = readl(addr); - if (status & CXL_RAS_CORRECTABLE_STATUS_MASK) { - writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); - trace_cxl_aer_correctable_error(to_cxl_memdev(dev), status); - } + if (!(status & CXL_RAS_CORRECTABLE_STATUS_MASK)) + return; + writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); + + trace_cxl_aer_correctable_error(to_cxl_memdev(dev), status); } static void cxl_handle_endpoint_cor_ras(struct cxl_dev_state *cxlds) @@ -700,8 +703,10 @@ static bool __cxl_handle_ras(struct device *dev, void __iomem *ras_base) u32 status; u32 fe; - if (!ras_base) + if (!ras_base) { + dev_warn_once(dev, "CXL RAS register block is not mapped"); return false; + } addr = ras_base + CXL_RAS_UNCORRECTABLE_STATUS_OFFSET; status = readl(addr); From patchwork Thu Mar 27 01:47:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030710 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2086.outbound.protection.outlook.com [40.107.212.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31A3B2E3392; Thu, 27 Mar 2025 01:49:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.212.86 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040181; cv=fail; b=l07Qm3u6ljCRIO41qwOJXofGDhOppz4erpRUq+NMpudzg3RC7Z/9oK/TwAbrgIbyXWt57Ej/6xx+rZaABoVBgo5HpAA3Cd2WMSvmiN1Zouxti2O/SspNFweDkFG2F0tLzrWT9mhEJ3S4gGrsf+YV5UHXtcTDKW3A8flLxYejvV0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040181; c=relaxed/simple; bh=RtjahhEuyI2PEtZ/8x2v3UGI3uNUDt27Q3nWlHboTyU=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=a7O2SIEqxoX0riulwJVVP17h8J2j0d6SO4ZCbH7RIGRUZeI5FSvvgwVW8Kb4Pb25DKQ9QicZP+da9kF9RkZvMl0BQFgCfKDgft8OJ6PusxGTDHaT//vrp5F4BxyKnFbEprHFiqEHQar2VH/2rbgiqyf+cVvTVbxeJGtQBSWGRCw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=see/Vo3j; arc=fail smtp.client-ip=40.107.212.86 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="see/Vo3j" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XTVQC8LQ4GWcXXnIsU7zl2xrPrIHDQyfZJ+/XM8NEfEahRcyabPZe9+/No+KUpKxlF7aElGy4HHLZgA20mEHy60TyFm+FCYSrQpwRwe8MjCBwox6oEUtjLdVGiruwAQmNEsD20XOGSrQY81i73pynTrxJvQeOBEnVzQErt9zj0j7zSfIn7JcN6SyzR6ioQslW1L3f1Vq0glFLXQ3CX8pklhR42Db95mcsMuzvWJAXIWWSD4/DTsOQmdCRgJd8XcNJkHdZa1uAc+h7D7SiOm9M8XXbzzNL1d6HneHG2TvpVT1oPXOCDuXbwjmcLUNp8S9sTx+JCdlfgNGG7isYA4bhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FBN3Uvr6gqdDkhua96mIzBLs8tNfyPNw8AmSDhaN8bw=; b=xV5wEgA9MATFRm708jAAjDSFkgHiSW9IJoZmrRgsaCRp2vfFCfLXRNbziGjPTlK/dzYS/K8wIDPDdcA/ChsiWuoJk55S4Yof0z4yNsNNOmzMIYC6/q07/Kx+fUN4F9Prm6UxWnEt7JYBE6rfpcGJveboEucrhFGAUQrFEJFF987Pu1wZqfO7zfYXfnrMejJI6+DAs1W5VHZvnkMswFn5hJZeYH1LqOVpDqjp+0nbufy6EiLzC3fxkTfShOtEr4XC2psXj7nU0X29d0UezMH2WMAW0pfrUN8sZjkAbkT3xpTJMyBNABwL88dBC8O3jUu/vNnnT/ndMyk4FNpPn4ELQg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FBN3Uvr6gqdDkhua96mIzBLs8tNfyPNw8AmSDhaN8bw=; b=see/Vo3jTGXcpMLp6ig0dy93ZkAVzLPoyIYT5J5beiUc88w8AMZnCJHSEbU0mMwQ6FukZgh+AxpS8KSAMZ8rth+iUHf4eyfbqX2q2Rt1Z3mLTkKF2FX5mLg0VHXkuRC7r6VWkQHFEBdMwuKnF6ro7pdKWEEAD0gw8bm8ZWrk1MQ= Received: from BN9PR03CA0261.namprd03.prod.outlook.com (2603:10b6:408:ff::26) by MW4PR12MB5626.namprd12.prod.outlook.com (2603:10b6:303:169::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Thu, 27 Mar 2025 01:49:31 +0000 Received: from BL6PEPF00022570.namprd02.prod.outlook.com (2603:10b6:408:ff:cafe::d6) by BN9PR03CA0261.outlook.office365.com (2603:10b6:408:ff::26) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.43 via Frontend Transport; Thu, 27 Mar 2025 01:49:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022570.mail.protection.outlook.com (10.167.249.38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:49:30 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:49:28 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 11/16] cxl/pci: Unifi CXL trace logging for CXL Endpoints and CXL Ports Date: Wed, 26 Mar 2025 20:47:12 -0500 Message-ID: <20250327014717.2988633-12-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022570:EE_|MW4PR12MB5626:EE_ X-MS-Office365-Filtering-Correlation-Id: 6c8451d0-bf38-47f8-1495-08dd6cd19dd4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|376014|7416014|82310400026|1800799024|921020; X-Microsoft-Antispam-Message-Info: nKH+8CQQTXpl7Yr+fwNpwGeBIYUpR7dQs4Qo+Qg9OQ9JjhGxQfEaLNYJX3WMGt66DX3cLnUIRd/w7+qzjTJhQ4xeMXDqdS3lkCiVbra3uHDp6VHeWIm+Dh9y/kaTQnLhv1K0J0ev0n7LYP/Z03vjZg/iLsYpRNmrxtsKGqh/ja17TR64QkzsCbT/LUjaaLUO4EHJbAHFOSgjKv9kdvCKeWijUFuZTghWl6B2nBkTNB4nzDDKmCYLSDarGZyGfo2yq1Mfh7Hdt2KSubs3m+Eb5B085nmmgk3WdJvmFrC8z/TdOL1re445DFisDLFxHYKfyEeC1GzBaSfpOqokQapbBw2cUncS4d24jMR/CrGmBAWG9a59Z7FPlp68r+B9UFkx1ip1XWJvvpkVrGe24yQGdsONfw6cXXbxMPVPP/Wxr8jKjpG63j70QtGrrtnCJ6Cas2dvtrYpx112Y4m/6ZiMhWiOa6mX5J9lLOIt9LvFA0LLyom9CikkEZfKT+S2Ao6ybsKkiL5Fjz7GW5I9X8i4Sb6Yr4B8BkZGSmwBYX4sBwnMBMlRTrQxxmun1skRaZ+UuifqnJuztboYPqqoc+bAaaYJvSxGlnkeEvvX0CykiV95rCzmb6UjF+RV3f1av7NzrH1KJzguayJ3F9SzTyHiL/yGJJfKEtYcMztFdhY+I0AQmMqdLc6L+n1/MYdA2Z8ISqn5/au2JHrpnbwgnmOETtzc5GOY2FKrT5rra3cTUauo28PvtbpbpfAdfX1V1kS8Sib3UefVttnQhat8n+g4lCz3WP5ZLW2nsqJnK3Gds6lrpWDSfeA9JzDEDThfYCwaykQIBXctqkjoY/0uCy00XVb5l+mS+hj+WMylyf3f6598MlaZGsIxzpR6pcyOaNIrhyWMCliFnRorwivxg6QSzL6KD8nvzzeNI8KeNIg/nmF1PhFpntxqltIin0yohp1AMD4B3G6u09TKqI8sqVnuRrBdBCwklYy0prdqNfGcFBUJriUrvi3HQOcf6yvTpkIMv8RgnMbDa0o95dUg8Gx2t/Grv1nNnojLkvrQRvByBtj2jZVUQgzMtx6i+xoB5a1h6VmfuzPY5md+s9Evk5fCvqirgsPWS7Mz2RocEe5dJjMCImQDggF2GtPjv+st85V+tA4HEAzmfipDCVPFzB2VqlI7zsqzprtJCCUiaGrcfcyk2faaBUopfewM2wKx6DjA6qTKFL9G5m0SVMuZpezZj82fgpNupQh7/Jq1/UHjh5l/cGe4s7c9r/vOwQ67FFH+e+m3J8HKEnYbGrn8Adb8Z1rZbGdtskzUV0tIFs26olu1rFfony9nqZeZeDVozpfNKN2R5iPzh2VR8e+lwXoooduA1iS6r1XkUorgyibW4gIuc/dmjsKlvjVnmJAUPuxMpFuuGkGdIJ+elGG2P9d9zuHfxZ5TqWdsB3kIORAk0CrfX7AszgUge+xVhZfOlPc71hwSsODoCTC+5CfX85s1LCACLGpf19n7WOizQ14VoiakXIz4hcbJPRoD4fnKvXFX X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(376014)(7416014)(82310400026)(1800799024)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:49:30.7896 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6c8451d0-bf38-47f8-1495-08dd6cd19dd4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022570.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB5626 CXL currently has separate trace routines for CXL Port errors and CXL Endpoint errors. This is inconvnenient for the user because they must enable 2 sets of trace routines. Make updates to the trace logging such that a single trace routine logs both CXL Endpoint and CXL Port protocol errors. Also, CXL RAS errors are currently logged using the associated CXL port's name returned from devname(). They are typically named with 'port1', 'port2', etc. to indicate the hierarchial location in the CXL topology. But, this doesn't clearly indicate the CXL card or slot reporting the error. Update the logging to also log the corresponding PCIe devname. This will give a PCIe SBDF or ACPI object name (in case of CXL HB). This will provide details helping users understand which physical slot and card has the error. Below is example output after making these changes. Correctable error example output: cxl_port_aer_correctable_error: device=port1 (0000:0c:00.0) parent=root0 (pci0000:0c) status='Received Error From Physical Layer' Uncorrectable error example output: cxl_port_aer_uncorrectable_error: device=port1 (0000:0c:00.0) parent=root0 (pci0000:0c) status: 'Memory Byte Enable Parity Error' first_error: 'Memory Byte Enable Parity Error' Signed-off-by: Terry Bowman --- drivers/cxl/core/pci.c | 29 ++++++------ drivers/cxl/core/ras.c | 14 +++--- drivers/cxl/core/trace.h | 100 +++++++++++++-------------------------- 3 files changed, 55 insertions(+), 88 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 4770810b2138..10b2abfb0e64 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -650,14 +650,14 @@ void read_cdat_data(struct cxl_port *port) } EXPORT_SYMBOL_NS_GPL(read_cdat_data, "CXL"); -static void __cxl_handle_cor_ras(struct device *dev, - void __iomem *ras_base) +static void __cxl_handle_cor_ras(struct device *cxl_dev, struct device *pcie_dev, + u64 serial, void __iomem *ras_base) { void __iomem *addr; u32 status; if (!ras_base) { - dev_warn_once(dev, "CXL RAS register block is not mapped"); + dev_warn_once(cxl_dev, "CXL RAS register block is not mapped"); return; } @@ -667,12 +667,12 @@ static void __cxl_handle_cor_ras(struct device *dev, return; writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); - trace_cxl_aer_correctable_error(to_cxl_memdev(dev), status); + trace_cxl_aer_correctable_error(cxl_dev, pcie_dev, serial, status); } static void cxl_handle_endpoint_cor_ras(struct cxl_dev_state *cxlds) { - return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, cxlds->regs.ras); + return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, NULL, cxlds->serial, cxlds->regs.ras); } /* CXL spec rev3.0 8.2.4.16.1 */ @@ -696,7 +696,8 @@ static void header_log_copy(void __iomem *ras_base, u32 *log) * Log the state of the RAS status registers and prepare them to log the * next error status. Return 1 if reset needed. */ -static bool __cxl_handle_ras(struct device *dev, void __iomem *ras_base) +static pci_ers_result_t __cxl_handle_ras(struct device *cxl_dev, struct device *pcie_dev, + u64 serial, void __iomem *ras_base) { u32 hl[CXL_HEADERLOG_SIZE_U32]; void __iomem *addr; @@ -704,14 +705,14 @@ static bool __cxl_handle_ras(struct device *dev, void __iomem *ras_base) u32 fe; if (!ras_base) { - dev_warn_once(dev, "CXL RAS register block is not mapped"); - return false; + dev_warn_once(cxl_dev, "CXL RAS register block is not mapped"); + return PCI_ERS_RESULT_NONE; } addr = ras_base + CXL_RAS_UNCORRECTABLE_STATUS_OFFSET; status = readl(addr); if (!(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK)) - return false; + return PCI_ERS_RESULT_NONE; /* If multiple errors, log header points to first error from ctrl reg */ if (hweight32(status) > 1) { @@ -725,15 +726,15 @@ static bool __cxl_handle_ras(struct device *dev, void __iomem *ras_base) } header_log_copy(ras_base, hl); - trace_cxl_aer_uncorrectable_error(to_cxl_memdev(dev), status, fe, hl); + trace_cxl_aer_uncorrectable_error(cxl_dev, pcie_dev, serial, status, fe, hl); writel(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK, addr); - return true; + return PCI_ERS_RESULT_PANIC; } static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) { - return __cxl_handle_ras(&cxlds->cxlmd->dev, cxlds->regs.ras); + return __cxl_handle_ras(&cxlds->cxlmd->dev, NULL, cxlds->serial, cxlds->regs.ras); } #ifdef CONFIG_PCIEAER_CXL @@ -741,13 +742,13 @@ static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) static void cxl_handle_rdport_cor_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { - return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, dport->regs.ras); + return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, NULL, cxlds->serial, dport->regs.ras); } static bool cxl_handle_rdport_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { - return __cxl_handle_ras(&cxlds->cxlmd->dev, dport->regs.ras); + return __cxl_handle_ras(&cxlds->cxlmd->dev, NULL, cxlds->serial, dport->regs.ras); } /* diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c index 1f94fc08e72b..f18cb568eabd 100644 --- a/drivers/cxl/core/ras.c +++ b/drivers/cxl/core/ras.c @@ -13,7 +13,7 @@ static void cxl_cper_trace_corr_port_prot_err(struct pci_dev *pdev, { u32 status = ras_cap.cor_status & ~ras_cap.cor_mask; - trace_cxl_port_aer_correctable_error(&pdev->dev, status); + trace_cxl_aer_correctable_error(&pdev->dev, &pdev->dev, 0, status); } static void cxl_cper_trace_uncorr_port_prot_err(struct pci_dev *pdev, @@ -28,8 +28,8 @@ static void cxl_cper_trace_uncorr_port_prot_err(struct pci_dev *pdev, else fe = status; - trace_cxl_port_aer_uncorrectable_error(&pdev->dev, status, fe, - ras_cap.header_log); + trace_cxl_aer_uncorrectable_error(&pdev->dev, &pdev->dev, 0, + status, fe, ras_cap.header_log); } static void cxl_cper_trace_corr_prot_err(struct pci_dev *pdev, @@ -42,7 +42,8 @@ static void cxl_cper_trace_corr_prot_err(struct pci_dev *pdev, if (!cxlds) return; - trace_cxl_aer_correctable_error(cxlds->cxlmd, status); + trace_cxl_aer_correctable_error(&cxlds->cxlmd->dev, &pdev->dev, + cxlds->serial, status); } static void cxl_cper_trace_uncorr_prot_err(struct pci_dev *pdev, @@ -62,8 +63,9 @@ static void cxl_cper_trace_uncorr_prot_err(struct pci_dev *pdev, else fe = status; - trace_cxl_aer_uncorrectable_error(cxlds->cxlmd, status, fe, - ras_cap.header_log); + trace_cxl_aer_uncorrectable_error(&cxlds->cxlmd->dev, &pdev->dev, + cxlds->serial, status, + fe, ras_cap.header_log); } static void cxl_cper_handle_prot_err(struct cxl_cper_prot_err_work_data *data) diff --git a/drivers/cxl/core/trace.h b/drivers/cxl/core/trace.h index 25ebfbc1616c..399e0b8bf0f2 100644 --- a/drivers/cxl/core/trace.h +++ b/drivers/cxl/core/trace.h @@ -48,49 +48,26 @@ { CXL_RAS_UC_IDE_RX_ERR, "IDE Rx Error" } \ ) -TRACE_EVENT(cxl_port_aer_uncorrectable_error, - TP_PROTO(struct device *dev, u32 status, u32 fe, u32 *hl), - TP_ARGS(dev, status, fe, hl), - TP_STRUCT__entry( - __string(device, dev_name(dev)) - __string(host, dev_name(dev->parent)) - __field(u32, status) - __field(u32, first_error) - __array(u32, header_log, CXL_HEADERLOG_SIZE_U32) - ), - TP_fast_assign( - __assign_str(device); - __assign_str(host); - __entry->status = status; - __entry->first_error = fe; - /* - * Embed the 512B headerlog data for user app retrieval and - * parsing, but no need to print this in the trace buffer. - */ - memcpy(__entry->header_log, hl, CXL_HEADERLOG_SIZE); - ), - TP_printk("device=%s host=%s status: '%s' first_error: '%s'", - __get_str(device), __get_str(host), - show_uc_errs(__entry->status), - show_uc_errs(__entry->first_error) - ) -); - TRACE_EVENT(cxl_aer_uncorrectable_error, - TP_PROTO(const struct cxl_memdev *cxlmd, u32 status, u32 fe, u32 *hl), - TP_ARGS(cxlmd, status, fe, hl), + TP_PROTO(struct device *cxl_dev, struct device *pcie_dev, u64 serial, + u32 status, u32 fe, u32 *hl), + TP_ARGS(cxl_dev, pcie_dev, serial, status, fe, hl), TP_STRUCT__entry( - __string(memdev, dev_name(&cxlmd->dev)) - __string(host, dev_name(cxlmd->dev.parent)) + __string(cxl_name, dev_name(cxl_dev)) + __string(cxl_parent_name, dev_name(cxl_dev->parent)) + __string(pcie_name, dev_name(pcie_dev)) + __string(pcie_parent_name, dev_name(pcie_dev->parent)) __field(u64, serial) __field(u32, status) __field(u32, first_error) __array(u32, header_log, CXL_HEADERLOG_SIZE_U32) ), TP_fast_assign( - __assign_str(memdev); - __assign_str(host); - __entry->serial = cxlmd->cxlds->serial; + __assign_str(cxl_name); + __assign_str(cxl_parent_name); + __assign_str(pcie_name); + __assign_str(pcie_parent_name); + __entry->serial = serial; __entry->status = status; __entry->first_error = fe; /* @@ -99,10 +76,11 @@ TRACE_EVENT(cxl_aer_uncorrectable_error, */ memcpy(__entry->header_log, hl, CXL_HEADERLOG_SIZE); ), - TP_printk("memdev=%s host=%s serial=%lld: status: '%s' first_error: '%s'", - __get_str(memdev), __get_str(host), __entry->serial, - show_uc_errs(__entry->status), - show_uc_errs(__entry->first_error) + TP_printk("device=%s (%s) parent=%s (%s) serial: %lld status: '%s' first_error: '%s'", + __get_str(cxl_name), __get_str(pcie_name), + __get_str(cxl_parent_name), __get_str(pcie_parent_name), + __entry->serial, show_uc_errs(__entry->status), + show_uc_errs(__entry->first_error) ) ); @@ -124,43 +102,29 @@ TRACE_EVENT(cxl_aer_uncorrectable_error, { CXL_RAS_CE_PHYS_LAYER_ERR, "Received Error From Physical Layer" } \ ) -TRACE_EVENT(cxl_port_aer_correctable_error, - TP_PROTO(struct device *dev, u32 status), - TP_ARGS(dev, status), - TP_STRUCT__entry( - __string(device, dev_name(dev)) - __string(host, dev_name(dev->parent)) - __field(u32, status) - ), - TP_fast_assign( - __assign_str(device); - __assign_str(host); - __entry->status = status; - ), - TP_printk("device=%s host=%s status='%s'", - __get_str(device), __get_str(host), - show_ce_errs(__entry->status) - ) -); - TRACE_EVENT(cxl_aer_correctable_error, - TP_PROTO(const struct cxl_memdev *cxlmd, u32 status), - TP_ARGS(cxlmd, status), + TP_PROTO(struct device *cxl_dev, struct device *pcie_dev, u64 serial, u32 status), + TP_ARGS(cxl_dev, pcie_dev, serial, status), TP_STRUCT__entry( - __string(memdev, dev_name(&cxlmd->dev)) - __string(host, dev_name(cxlmd->dev.parent)) + __string(cxl_name, dev_name(cxl_dev)) + __string(cxl_parent_name, dev_name(cxl_dev->parent)) + __string(pcie_name, dev_name(pcie_dev)) + __string(pcie_parent_name, dev_name(pcie_dev->parent)) __field(u64, serial) __field(u32, status) ), TP_fast_assign( - __assign_str(memdev); - __assign_str(host); - __entry->serial = cxlmd->cxlds->serial; + __assign_str(cxl_name); + __assign_str(cxl_parent_name); + __assign_str(pcie_name); + __assign_str(pcie_parent_name); + __entry->serial = serial; __entry->status = status; ), - TP_printk("memdev=%s host=%s serial=%lld: status: '%s'", - __get_str(memdev), __get_str(host), __entry->serial, - show_ce_errs(__entry->status) + TP_printk("device=%s (%s) parent=%s (%s) serieal=%lld status='%s'", + __get_str(cxl_name), __get_str(pcie_name), + __get_str(cxl_parent_name), __get_str(pcie_parent_name), + __entry->serial, show_ce_errs(__entry->status) ) ); From patchwork Thu Mar 27 01:47:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030711 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2087.outbound.protection.outlook.com [40.107.244.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C2951991C9; Thu, 27 Mar 2025 01:49:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.87 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040187; cv=fail; b=XlzN/wlVHb4LUAJEqxIFx+zFh0vMjPFXQCd5oWqksmFV0Is6zdgpcN6wv7Ja8Rizyg5cTRofDN0XHwZse2on8Xqry82Y5JoZ2U4WmwZAg/VNjQHOg+EXeH5IdrYkHpFzT08XdAH+ePoBqqSlQOP7cHBLz2xRt+LI4+Fzuu0Knj4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040187; c=relaxed/simple; bh=WepXOcLM1uEs9pESDv87Ym/p6BYt8twNuiNZ9FdmfRo=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=pnn9lIlm0T0cHy+fJc1t8F79GkwEx6Yk4wf8Px9KLuZuB8G8yVrjle0byT0yhhLah8NGRpHg2CUMsHy7+Bi3tZpVnRfinoXJx0nTSfYGIlzg618YOabu3KsiEgbPvP/RvsR2uDr1yX70zP3cP8hz+fY/n8HYY/czfEghyiAJREw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=SNsGsSUi; arc=fail smtp.client-ip=40.107.244.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="SNsGsSUi" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gzJob80QicYLlGbvTJrwwyNAV12GTpxPkBo9bvF3Y2P+836yViaghVjE6sAPazjtU8NZ5u4TCg1LusvB/TUsCTND8WjOVHbMywXOuIU7VZxOArzeuws0hcr2YjzGpzrzQ4SovAxyvO7VUdmF3THagRpjUhg0tIixbATQIWqmpj/jZToFGTgYJjj3P+4gtY6DlI9I3wwjyW4JBLndSw/yKfeUbvk2i6BXz8X6jL3FSHpv7z+OsYEslc8MapFPeJjR5UCQYQB9BUpxkqPX9P5Fy7ZvnO4yYHvsDSsH8JmScgI3YWG7QrOANwSsteKzR9E19k3Sw1E4NoLWjvYYcYlt/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=r1dA8A3QiVtXTMHvQGMJBvGycEyDTnNvWurcj9uH3qg=; b=CsjUMLA727tbcX+Jekvy25J0YCic8oGIycmwXEApuAD87HCUIxlW5tolW5aneDLR08Q3YG0OpkE3tJCY4DZxtDBAYAXtVw2lqzGaFdMO2fMrM7NJPJDBLuWZBoGgvBTsYGEPEoHumvTH+kvqp6PqJQrryWRYWVoKhHitvWaXWtKseMpA0We15mmN1wn1qzNdN7mam1RQy1Z1Lqm1/78wjX/cDcMESUBS9/PZpAuIfOQWkZIjEeNuFbcW1iPhyZtCls5eHbD/DtiX8q24H59S3QgTvF3Z+pPeBRq+vuul4IssxY0ceVrtLXn0hecbhjcLmwwhEsFgENb8Mm5NDuX+zg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=r1dA8A3QiVtXTMHvQGMJBvGycEyDTnNvWurcj9uH3qg=; b=SNsGsSUibrBBIengxi7D7kOPKSjHCxJ6rDAzOdB+v8Rs7eIw/z6fJWfNB14nmTbubfldfclvBelzgBzy/otG40+tEqVtnfYdENAm+dzeSohbvhVwPtjpiRJEvAzI1RcEtHuP/xefqvXVIbY39jbfvyZ5fa1S/Nrzj4to7K9sInY= Received: from BLAPR03CA0068.namprd03.prod.outlook.com (2603:10b6:208:329::13) by LV3PR12MB9329.namprd12.prod.outlook.com (2603:10b6:408:21c::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Thu, 27 Mar 2025 01:49:41 +0000 Received: from BL6PEPF00022572.namprd02.prod.outlook.com (2603:10b6:208:329:cafe::97) by BLAPR03CA0068.outlook.office365.com (2603:10b6:208:329::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.43 via Frontend Transport; Thu, 27 Mar 2025 01:49:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022572.mail.protection.outlook.com (10.167.249.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:49:41 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:49:40 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 12/16] cxl/pci: Assign CXL Port protocol error handlers Date: Wed, 26 Mar 2025 20:47:13 -0500 Message-ID: <20250327014717.2988633-13-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022572:EE_|LV3PR12MB9329:EE_ X-MS-Office365-Filtering-Correlation-Id: ab9c80a4-2070-4d5d-6f3f-08dd6cd1a40b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|7416014|376014|1800799024|921020; X-Microsoft-Antispam-Message-Info: 2vwxi2BKYzuWbmvL7W5d6uoWnqz9GDXU0z97BTd5XKZ1sZpP0YjI0M84hZp5GGsgSpp602mAvcu0TpRZ3kFADzFxtTiU/IRNxmx3b4Us/ymh2+zxAS2zqrt6jE0xYUvosFNccHcviGafXK7raQDtyEFWYm2ag77hObRWm8yKI2zpTaEF5biLKuXe0Ek3wnd31uFUbp9yBwI6xa6MNJhXS0qpj9bT+VvRVUNEUKtpOP5YZQP1Arsa+1rdYSNhZerOIG/m6sJt8JuHS0CmX3XdiTiv4iIs445EboIZIv6GO0vl0CMi2TspdPkWbO+SudZgWD2n3KVzKep5DBsFyWy9PAZJDlA3IzpNI5bL1hGPdJbSpJkGrIiemnGMZb5KwpeXzvMfDz7/fz6fhMsU4xJhr2zzPNmZarUycnNMXGwh2lrMcowvfeWELEAs/BOLXk72BJrGvB9fG7JHKvFz0y3gQR2/zpLbvT/k+JV+arC0Rf4UoatfaEF0tvWedf7T2yj4qSsK5X39eiBXhsmj9usHnlH29+q4Pgaipyg1AX5Zwbv6lkGBMhl/Zb7tVnJBQ1ODN/1nXzAxdBBcdPhwIrm3kJlY8PcCf6avtVZ4r4j5TLRzBu89U6I5kn1Keeu7Q6P3DDX7KPuEUnQojC0mXEbirTAolpsGPCdgHsX3SDjjRGe3V5BMT/MTY0V0B4QGO5MLC0GfRsEoN9dwohYg3GxTTzL3H4qX7gS54LCUwlBzvx68fZDcfybOFzYMV+rdGW6htX+m4R5aM7tB5NLGnbP1NJxX771iXmz2MxUG2LD0kbNMzQzCI74KlUIwNQKtKnvz3WinQsP4YbW8HNsQ4xBVEY1OVX0NIRzUqVuXNGJxFi/Phw6SYqxTo+iwMKxlcJ3x47yKROVj7EppTBC5wHQV0mQ/m3Kd3k67P+SxKDW2bOQzpgQy/ENAk+JiN04R7IZAmcwakgHyX1A246f+WFr5008sEXWcRGLC12l2qBmrMYPVp5zLDYe1tXZVuqr6iOQRBpWC9TnfMCt4pGrrfRGcEl+0IpRdcCGGZZcawCphti3+qdwRvrUMys4qQnOaK/4mr7DJOZQA0v66DtO6MulbQUp8TKdOXxI7PXQngYzK8v1q57RcqupuUgacx9racPhZInMC6ZWT5CK5YM3q9nKiN0IxaaBUfnkT9s1CApsWVMRMxiyeTRwrhOXQ9p87+cq+X67yerhurpUCjXKGPHR1f5mKsrrZaxHnmweGLhQicfDs6av4IUenT1fEEIu2tJX06M8Pz91VedgMnhbTTXRw/EVrMNn5qi7y3FxfKRuQhjWZRq4yIMEXn+D5RkuVQ0YXZB/GFHbWm3pXiqjX4hFZjyTHpvwCoSMqzD6mpSsCMoUaXOoAZVSmW8ZpQRBEVK2ROlFtrXootwuedP2M8ZSXPMFLdYQh7I7bEWQ22Z24iWaZ6PLnkGUXfmLALMmIpgDookBsj33pFLd3URs6lPQYisuCOSzlqQiXJHd9tbXDulw9ewya3TXV8ZnDHbRexD89uFLATj0rS9GbjDiRWIXpmw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(7416014)(376014)(1800799024)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:49:41.2343 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ab9c80a4-2070-4d5d-6f3f-08dd6cd1a40b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022572.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9329 Introduce CXL error handlers for CXL Port devices. These are needed to handle and log CXL protocol errors. Update cxl_create_prot_err_info() with support for CXL Root Ports (RP), CXL Upstream Switch Ports (USP) and CXL Downstreasm Switch ports (DSP). Add functions cxl_port_error_detected() and cxl_port_cor_error_detected(). Add cxl_assign_error_handlers() and use to assign the CXL Port error handlers for CXL RP, CXL USP, and CXL DSP. Make the assignments in cxl_uport_init_ras() and cxl_dport_init_ras() after mapping RAS registers. Signed-off-by: Terry Bowman --- drivers/cxl/core/core.h | 2 ++ drivers/cxl/core/pci.c | 23 +++++++++++++ drivers/cxl/core/port.c | 4 +-- drivers/cxl/core/ras.c | 76 +++++++++++++++++++++++++++++++++-------- drivers/cxl/cxl.h | 5 +++ drivers/cxl/port.c | 29 ++++++++++++++-- 6 files changed, 120 insertions(+), 19 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 15699299dc11..5ce7269e5f13 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -122,6 +122,8 @@ void cxl_ras_exit(void); int cxl_gpf_port_setup(struct device *dport_dev, struct cxl_port *port); int cxl_acpi_get_extended_linear_cache_size(struct resource *backing_res, int nid, resource_size_t *size); +struct cxl_port *find_cxl_port(struct device *dport_dev, + struct cxl_dport **dport); #ifdef CONFIG_CXL_FEATURES size_t cxl_get_feature(struct cxl_mailbox *cxl_mbox, const uuid_t *feat_uuid, diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 10b2abfb0e64..9ed6f700e132 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -739,6 +739,29 @@ static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) #ifdef CONFIG_PCIEAER_CXL + +void cxl_port_cor_error_detected(struct device *cxl_dev, + struct cxl_prot_error_info *err_info) +{ + void __iomem *ras_base = err_info->ras_base; + struct device *pci_dev = &err_info->pdev->dev; + u64 serial = 0; + + __cxl_handle_cor_ras(cxl_dev, pci_dev, serial, ras_base); +} +EXPORT_SYMBOL_NS_GPL(cxl_port_cor_error_detected, "CXL"); + +pci_ers_result_t cxl_port_error_detected(struct device *cxl_dev, + struct cxl_prot_error_info *err_info) +{ + void __iomem *ras_base = err_info->ras_base; + struct device *pci_dev = &err_info->pdev->dev; + u64 serial = 0; + + return __cxl_handle_ras(cxl_dev, pci_dev, serial, ras_base); +} +EXPORT_SYMBOL_NS_GPL(cxl_port_error_detected, "CXL"); + static void cxl_handle_rdport_cor_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 0fd6646c1a2e..83d331c82d91 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1348,8 +1348,8 @@ static struct cxl_port *__find_cxl_port(struct cxl_find_port_ctx *ctx) return NULL; } -static struct cxl_port *find_cxl_port(struct device *dport_dev, - struct cxl_dport **dport) +struct cxl_port *find_cxl_port(struct device *dport_dev, + struct cxl_dport **dport) { struct cxl_find_port_ctx ctx = { .dport_dev = dport_dev, diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c index f18cb568eabd..fe38e76f2d1a 100644 --- a/drivers/cxl/core/ras.c +++ b/drivers/cxl/core/ras.c @@ -110,34 +110,80 @@ static void cxl_cper_prot_err_work_fn(struct work_struct *work) } static DECLARE_WORK(cxl_cper_prot_err_work, cxl_cper_prot_err_work_fn); +static int match_uport(struct device *dev, const void *data) +{ + const struct device *uport_dev = data; + struct cxl_port *port; + + if (!is_cxl_port(dev)) + return 0; + + port = to_cxl_port(dev); + + return port->uport_dev == uport_dev; +} + int cxl_create_prot_err_info(struct pci_dev *_pdev, int severity, struct cxl_prot_error_info *err_info) { struct pci_dev *pdev __free(pci_dev_put) = pci_dev_get(_pdev); - struct cxl_dev_state *cxlds; if (!pdev || !err_info) { pr_warn_once("Error: parameter is NULL"); return -ENODEV; } - if ((pci_pcie_type(pdev) != PCI_EXP_TYPE_ENDPOINT) && - (pci_pcie_type(pdev) != PCI_EXP_TYPE_RC_END)) { + *err_info = (struct cxl_prot_error_info){ 0 }; + err_info->severity = severity; + err_info->pdev = pdev; + + switch (pci_pcie_type(pdev)) { + case PCI_EXP_TYPE_ROOT_PORT: + case PCI_EXP_TYPE_DOWNSTREAM: + { + struct cxl_dport *dport = NULL; + struct cxl_port *port __free(put_cxl_port) = + find_cxl_port(&pdev->dev, &dport); + + if (!port || !is_cxl_port(&port->dev)) + return -ENODEV; + + err_info->ras_base = dport ? dport->regs.ras : NULL; + err_info->dev = &port->dev; + break; + } + case PCI_EXP_TYPE_UPSTREAM: + { + struct cxl_port *port; + struct device *port_dev __free(put_device) = + bus_find_device(&cxl_bus_type, NULL, &pdev->dev, + match_uport); + + if (!port_dev || !is_cxl_port(port_dev)) + return -ENODEV; + + port = to_cxl_port(port_dev); + err_info->ras_base = port ? port->uport_regs.ras : NULL; + err_info->dev = port_dev; + break; + } + case PCI_EXP_TYPE_ENDPOINT: + case PCI_EXP_TYPE_RC_END: + { + struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); + struct cxl_memdev *cxlmd = cxlds->cxlmd; + struct device *dev __free(put_device) = get_device(&cxlmd->dev); + + err_info->ras_base = cxlds->regs.ras; + err_info->dev = &cxlds->cxlmd->dev; + break; + } + default: + { pci_warn_once(pdev, "Error: Unsupported device type (%X)", pci_pcie_type(pdev)); return -ENODEV; } - - cxlds = pci_get_drvdata(pdev); - struct device *dev __free(put_device) = get_device(&cxlds->cxlmd->dev); - - if (!dev) - return -ENODEV; - - *err_info = (struct cxl_prot_error_info){ 0 }; - err_info->ras_base = cxlds->regs.ras; - err_info->severity = severity; - err_info->pdev = pdev; - err_info->dev = dev; + } return 0; } diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 0d05d5449f97..512cc38892ed 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -810,6 +810,11 @@ struct cxl_error_handlers { struct cxl_prot_error_info *err_info); }; +void cxl_port_cor_error_detected(struct device *dev, + struct cxl_prot_error_info *err_info); +pci_ers_result_t cxl_port_error_detected(struct device *dev, + struct cxl_prot_error_info *err_info); + /** * struct cxl_endpoint_dvsec_info - Cached DVSEC info * @mem_enabled: cached value of mem_enabled in the DVSEC at init time diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 1b8dc161428f..30a4bdb88c31 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -60,6 +60,24 @@ static int discover_region(struct device *dev, void *root) #ifdef CONFIG_PCIEAER_CXL +static const struct cxl_error_handlers cxl_port_error_handlers = { + .error_detected = cxl_port_error_detected, + .cor_error_detected = cxl_port_cor_error_detected, +}; + +static void cxl_assign_error_handlers(struct device *_dev, + const struct cxl_error_handlers *handlers) +{ + struct device *dev __free(put_device) = get_device(_dev); + struct cxl_driver *pdrv; + + if (!dev) + return; + + pdrv = to_cxl_drv(dev->driver); + pdrv->err_handler = handlers; +} + static void cxl_dport_map_rch_aer(struct cxl_dport *dport) { resource_size_t aer_phys; @@ -118,8 +136,12 @@ static void cxl_uport_init_ras_reporting(struct cxl_port *port, map->host = host; if (cxl_map_component_regs(map, &port->uport_regs, - BIT(CXL_CM_CAP_CAP_ID_RAS))) + BIT(CXL_CM_CAP_CAP_ID_RAS))) { dev_dbg(&port->dev, "Failed to map RAS capability\n"); + return; + } + + cxl_assign_error_handlers(&port->dev, &cxl_port_error_handlers); } /** @@ -144,9 +166,12 @@ void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) } if (cxl_map_component_regs(&dport->reg_map, &dport->regs.component, - BIT(CXL_CM_CAP_CAP_ID_RAS))) + BIT(CXL_CM_CAP_CAP_ID_RAS))) { dev_dbg(dport->dport_dev, "Failed to map RAS capability\n"); + return; + } + cxl_assign_error_handlers(dport->dport_dev, &cxl_port_error_handlers); } EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL"); From patchwork Thu Mar 27 01:47:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030712 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2071.outbound.protection.outlook.com [40.107.236.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5AEA61991C9; Thu, 27 Mar 2025 01:49:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.71 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040199; cv=fail; b=pUHhYfoD027zl+TkATawP3E17rWqIRr7cd79mwNac7BLHg61K3Kr5E83PgrJqfqLW5wKgrW/ZFzbypWxrXL+uZzfnPPnc9cTk1luhVJkNOMgBNNZYRjd0TplDsxGM5Ut6XZRo8tvNRPk+NlfIfryQOH3BCXJ6QqsJbQmxq6zfLk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040199; c=relaxed/simple; bh=LvB1BJjsJpVmt2ROl0EBiceE273rigb7xnvc1pIntUI=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=qPGkq4MRRooXPE6rH7SS7/ofILXg3fdAGTliQMqbGG+rZ3+k/KNGFAceIoUWSv1uHEPxjUT+v5SOWRtmPQ1AxT9Oq9dkBlLpewhFdxhlu6XPC6LafY6F7gxr+Da0bdMbqpivDWM3fXPbtmuAIt2FSSz3pkzgy/eLLqIVvcSocz0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=13ntWkvp; arc=fail smtp.client-ip=40.107.236.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="13ntWkvp" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=b66O36h5LgQG+kbPn8D/8op9anb+6BR1Yh8d0wSFDK8v9ac6AIA3Ahn3pPgvozFrRibKe9pYgTTMjdEjpe9H+hmyWye8N/iElsZJAGXTowZl91OTq8xAoMykfRngIwsQtH01ws1QRTRixz/athdnB0Iyt/Vb5SQ9vKioKM40sCV8AVHY97F4PnaHlrvgGC/cpRBnbcD6YjqbuwXblNb19D1+YjumjBAvqkIzo1PLEn6zVEF/n/6EBYgeHZM7gU7VAx+OVZiTvGKSSwLR5vPGHvb1Em0UsyqUoFUEE4Pxe7ka0X38eOU177kywa5IOTejrm+45bp8rsmjISl4eoYtrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6+Zx4TJRlY7mSb2pr2TmqDd0lZL4IF2OJ78fwP+tYvo=; b=KliEwqyj01LOr3PTdJKT/7i918xwhaFHAP1ZOiTGE8aPN0IZVEsSA53l0YtEZxilUapdPijpINkeP3ZywKL9rSE1Ma93Z5SQ4rgFpCl2MHaIh8o1QOLf1dIEi1u36ZL4HvXflWjAwELzDH8p9se4etMBc2ttE5ynbGo8L9LKzgQTGP1RYfDHcN5CKevXNxYln9kpJLbVSdg9YlcHiMuXBYGPa4n4GhdRw/knKH3pmamd0NgUQEwQWfqFV3h6X6Mj5VLbZCuDXACqc+Z6/xflDTd0qcxmEixJpjSJPxz3ZZYYCfy5dMBy/VrljsUx+cf83LQD0mcDOndBvoIKj0TMOw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6+Zx4TJRlY7mSb2pr2TmqDd0lZL4IF2OJ78fwP+tYvo=; b=13ntWkvpgnkwEsmeLsRRpJujMesVsivLkw6AIT5YIy8RaoAMam18iQZU/BPKkj7W9oPCgE7M4j5xyKwoKA1LKGqGdwoOomL+yQ1XzaG/mQjQJzSpl6OxslNVC/etmJ9ZkWSRgdB7zV1/XYA5o6Bl49urCNynQ77LkKSPqVjFaOc= Received: from MN0P222CA0001.NAMP222.PROD.OUTLOOK.COM (2603:10b6:208:531::6) by SJ2PR12MB8926.namprd12.prod.outlook.com (2603:10b6:a03:53b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.42; Thu, 27 Mar 2025 01:49:53 +0000 Received: from BL6PEPF0002256F.namprd02.prod.outlook.com (2603:10b6:208:531:cafe::2e) by MN0P222CA0001.outlook.office365.com (2603:10b6:208:531::6) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.43 via Frontend Transport; Thu, 27 Mar 2025 01:49:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0002256F.mail.protection.outlook.com (10.167.249.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:49:52 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:49:51 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 13/16] cxl/pci: Assign CXL Endpoint protocol error handlers Date: Wed, 26 Mar 2025 20:47:14 -0500 Message-ID: <20250327014717.2988633-14-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0002256F:EE_|SJ2PR12MB8926:EE_ X-MS-Office365-Filtering-Correlation-Id: 66b080a7-5421-40d5-c304-08dd6cd1aad2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|82310400026|376014|36860700013|1800799024|921020; X-Microsoft-Antispam-Message-Info: VbHedtpVJZJ5hRh8Q5KheyMs5IcuiwxL+cwux4Ca6r13KrME4gz+qJ/3FwxD1DqzQxtrbLEPgTjw+DLgAc1f0afr9mCVouj2JuKz4LbfbaY+Kb3ex422zWkWF9uEqV3hRI3LNTcFcIAnPB9DZ2cTTN/0bEed5pPNZYRkKRSD3vVGhi8Z5U7YsD9zDn8kXxIg4Cslgpoo9aThfkqEJgoMuk0kmOF1KXwC2ohuSpjwUWTD1yFb7lfYoNSJk1JAtFDTYz8CM6uSeUm2eeOgKS/yrEfDsawJl5N2NVQ5LSHjovuh3AmZkkg185mdRudRPzFDtlrAJJl2/JmpF6+dGJ3iko/SFgUOchoftRk8WpsryezkU1XvwR9KULyDCyHSvcgdI2rlpfSWcyEWI8HNifnVb0NdK7uzC3dJ0Fug19qDjj/ZXcC3Y5k6W3RXmpAKwplmKObo0zT+4C02VDlcnc/DdelElVk/A6XA9IgsXTth/RFFgv/37tqdVHxCArq7Dmt5H5auw6h9fnhqebBBcy6JaJqpwixsc/PRO/7CT9i4HKobCZ35DjmquaqRcToHY7FCw4Enb75bSnb9G6xxlYWU6VAqGH45v2rFkR7zpqCi4vd8j7xCAk2Rz1MhvgZ3T0IZ68gPVIWwglVQow99ApWtuSeWP9ZZxIuWAGl/5bkaH4bNvZFsHOg/nNBJR3nkKrJEWO3KD8IbHs285HiQtl6gibiL92Hw7k1X6TsYO3RxL0j2nMk9A5K54h8fMoKMxbddrIyAkMnR+2V547L6U5vRJRszwgcMTJy7ULLTbqJyK8dVsAMXCNnzW6pJqu7c48Zu6Q0LOOePRFT6SmSeXTsMgsKmIPd5wwxIrMZi3i5+9MVSGC2CTMS5Kd8VdQNyCeuTlZw1+QfcyxAaUf+mB/eTpsNzyrX4pOO27GDebDT1IrnQFGyaQOSBZk1dWkrI/1N9Rq2mHBKFBbUNI6T/xh/icn/Gb/yqfVMb6ZVaLtdFcjbUgKMAQ2W2Di0epvu8kIpwP0IPIA4iebSKQ7QFWLo0Yqf3GLLpRehpIpVPwFZkxsJWaC93Ud0zCDMb9cMGGe2H0MJAwuL3Yc304UJciu7S+bKph6/WCNVvzfnY3BRoY3BaXaZCzbRnf54bCiva9nJ6IwCX83nko2vidmP8fTXtknNGC6hH/9oHzQQhiEfYpLW6rRWCqTU8FgVJWa+hg8TB+ENnQpsZhD4b1j98RYIClskAdfzw8u0DLKIbmbn0qCh93JqmUIyjXg0qMkTxHocCH9YMPX3xs4toLw4ML/TTDTZ/qcHO2yqsPrJnHSEs3hcfDKwKlVYyk8IBu0q2+LwoayvuSq+xDjFv2TCDZG6/iUDKPiK6ZjQqT4bYQcSD1s8SNwj4g6HEUEQEVS1dLtlrFJi1SnQ4vBAT6Eqh5MbmqvNqXdr2wh1KngeFpVHl6Fi7kny2M+GjWCN0dDtC0HYzaL9fnfcUHaa+mkqbf6VaTCPKyMULKXbRloEQ4admRwbWj8cQ/fPx0fX/KPOIHXm8 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(7416014)(82310400026)(376014)(36860700013)(1800799024)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:49:52.5913 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 66b080a7-5421-40d5-c304-08dd6cd1aad2 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0002256F.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8926 CXL Endpoint protocol errors are currently handled using PCI error handlers. The CXL Endpoint requires CXL specific handling in the case of uncorrectable error handling not provided by the PCI handlers. Add CXL specific handlers for CXL Endpoints. Assign the CXL handlers during Endpoint Port initialization. Keep the PCI Endpoint handlers. PCI handlers can be called if the CXL device is not trained for alternate protocol (CXL). Update the CXL Endpoint PCI handlers to call the CXL handler. If the CXL uncorrectable handler returns PCI_ERS_RESULT_PANIC then the PCI handler invokes panic(). Signed-off-by: Terry Bowman --- drivers/cxl/core/pci.c | 65 ++++++++++++++++++++++++------------------ drivers/cxl/cxl.h | 5 ++++ drivers/cxl/cxlpci.h | 4 +-- drivers/cxl/pci.c | 8 +++--- drivers/cxl/port.c | 7 +++++ 5 files changed, 56 insertions(+), 33 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 9ed6f700e132..f2139b382839 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -852,10 +852,10 @@ static void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds) static void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds) { } #endif -void cxl_cor_error_detected(struct pci_dev *pdev) +void cxl_cor_error_detected(struct device *dev, struct cxl_prot_error_info *err_info) { + struct pci_dev *pdev = err_info->pdev; struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); - struct device *dev = &cxlds->cxlmd->dev; scoped_guard(device, dev) { if (!dev->driver) { @@ -873,20 +873,30 @@ void cxl_cor_error_detected(struct pci_dev *pdev) } EXPORT_SYMBOL_NS_GPL(cxl_cor_error_detected, "CXL"); -pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, - pci_channel_state_t state) +void pci_cor_error_detected(struct pci_dev *pdev) +{ + struct cxl_prot_error_info err_info; + + if (cxl_create_prot_err_info(pdev, AER_CORRECTABLE, &err_info)) + return; + + cxl_cor_error_detected(err_info.dev, &err_info); +} +EXPORT_SYMBOL_NS_GPL(pci_cor_error_detected, "CXL"); + +pci_ers_result_t cxl_error_detected(struct device *dev, + struct cxl_prot_error_info *err_info) { - struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); - struct cxl_memdev *cxlmd = cxlds->cxlmd; - struct device *dev = &cxlmd->dev; bool ue; + struct pci_dev *pdev = err_info->pdev; + struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); scoped_guard(device, dev) { if (!dev->driver) { dev_warn(&pdev->dev, "%s: memdev disabled, abort error handling\n", dev_name(dev)); - return PCI_ERS_RESULT_DISCONNECT; + return PCI_ERS_RESULT_PANIC; } if (cxlds->rcd) @@ -900,29 +910,30 @@ pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, ue = cxl_handle_endpoint_ras(cxlds); } + if (ue) + return PCI_ERS_RESULT_PANIC; - switch (state) { - case pci_channel_io_normal: - if (ue) { - device_release_driver(dev); - return PCI_ERS_RESULT_NEED_RESET; - } - return PCI_ERS_RESULT_CAN_RECOVER; - case pci_channel_io_frozen: - dev_warn(&pdev->dev, - "%s: frozen state error detected, disable CXL.mem\n", - dev_name(dev)); - device_release_driver(dev); - return PCI_ERS_RESULT_NEED_RESET; - case pci_channel_io_perm_failure: - dev_warn(&pdev->dev, - "failure state error detected, request disconnect\n"); - return PCI_ERS_RESULT_DISCONNECT; - } - return PCI_ERS_RESULT_NEED_RESET; + return PCI_ERS_RESULT_CAN_RECOVER; } EXPORT_SYMBOL_NS_GPL(cxl_error_detected, "CXL"); +pci_ers_result_t pci_error_detected(struct pci_dev *pdev, + pci_channel_state_t error) +{ + struct cxl_prot_error_info err_info; + pci_ers_result_t rc; + + if (cxl_create_prot_err_info(pdev, AER_FATAL, &err_info)) + return PCI_ERS_RESULT_DISCONNECT; + + rc = cxl_error_detected(err_info.dev, &err_info); + if (rc == PCI_ERS_RESULT_PANIC) + panic("CXL cachemem error."); + + return rc; +} +EXPORT_SYMBOL_NS_GPL(pci_error_detected, "CXL"); + static int cxl_flit_size(struct pci_dev *pdev) { if (cxl_pci_flit_256(pdev)) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 512cc38892ed..c1adf8a3cb9e 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -815,6 +815,11 @@ void cxl_port_cor_error_detected(struct device *dev, pci_ers_result_t cxl_port_error_detected(struct device *dev, struct cxl_prot_error_info *err_info); +void cxl_cor_error_detected(struct device *dev, + struct cxl_prot_error_info *err_info); +pci_ers_result_t cxl_error_detected(struct device *dev, + struct cxl_prot_error_info *err_info); + /** * struct cxl_endpoint_dvsec_info - Cached DVSEC info * @mem_enabled: cached value of mem_enabled in the DVSEC at init time diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h index 92d72c0423ab..d277cf048eba 100644 --- a/drivers/cxl/cxlpci.h +++ b/drivers/cxl/cxlpci.h @@ -133,8 +133,8 @@ struct cxl_dev_state; int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, struct cxl_endpoint_dvsec_info *info); void read_cdat_data(struct cxl_port *port); -void cxl_cor_error_detected(struct pci_dev *pdev); -pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, +void pci_cor_error_detected(struct pci_dev *pdev); +pci_ers_result_t pci_error_detected(struct pci_dev *pdev, pci_channel_state_t state); int cxl_create_prot_err_info(struct pci_dev *_pdev, int severity, struct cxl_prot_error_info *err_info); diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 4288f4814cc5..c5be4422748e 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -1108,11 +1108,11 @@ static void cxl_reset_done(struct pci_dev *pdev) } } -static const struct pci_error_handlers cxl_error_handlers = { - .error_detected = cxl_error_detected, +static const struct pci_error_handlers pci_error_handlers = { + .error_detected = pci_error_detected, .slot_reset = cxl_slot_reset, .resume = cxl_error_resume, - .cor_error_detected = cxl_cor_error_detected, + .cor_error_detected = pci_cor_error_detected, .reset_done = cxl_reset_done, }; @@ -1120,7 +1120,7 @@ static struct pci_driver cxl_pci_driver = { .name = KBUILD_MODNAME, .id_table = cxl_mem_pci_tbl, .probe = cxl_pci_probe, - .err_handler = &cxl_error_handlers, + .err_handler = &pci_error_handlers, .dev_groups = cxl_rcd_groups, .driver = { .probe_type = PROBE_PREFER_ASYNCHRONOUS, diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 30a4bdb88c31..8e2b70e73582 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -65,6 +65,11 @@ static const struct cxl_error_handlers cxl_port_error_handlers = { .cor_error_detected = cxl_port_cor_error_detected, }; +const struct cxl_error_handlers cxl_ep_error_handlers = { + .error_detected = cxl_error_detected, + .cor_error_detected = cxl_cor_error_detected, +}; + static void cxl_assign_error_handlers(struct device *_dev, const struct cxl_error_handlers *handlers) { @@ -203,6 +208,8 @@ static void cxl_endpoint_port_init_ras(struct cxl_port *port) } cxl_dport_init_ras_reporting(dport, cxlmd_dev); + + cxl_assign_error_handlers(cxlmd_dev, &cxl_ep_error_handlers); } #else From patchwork Thu Mar 27 01:47:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030713 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2064.outbound.protection.outlook.com [40.107.243.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08CE61531E1; Thu, 27 Mar 2025 01:50:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.64 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040212; cv=fail; b=mKfbSKADrp0Zxv5tipY/zvV7lZGci3us+XT3vzXg7QAy9SohjMd3+DQUFdxQBWrvg4FrNulrQJcpsQy/Tz2J7YoQNfzI+eqt94CuI1LvdcaVv7656sip8f6mAIa8K0grMwS6qlTha8xjdzUqts9aghpx+3Fma6xLGnC4nLPC/tE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040212; c=relaxed/simple; bh=1/a8P2ADIFvqFQqoMTdw3h7cZBiNNdm1LMCQzEKRY+w=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MnproHJgymEkjpYs2Id+9+BS3P9PityiaYdu12j59d1x8QGWtP1Y9+ljghOLaWfTakYyOVe2v7WRyQIdth7YkmeVJLMwLFzcugZcbGvtzipth4i4/9NhA8GAIhS6echyL5t+P18tc88Km8S6GJXiCE++tqEacfod4B/sFwpgkMo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Oglm2pG7; arc=fail smtp.client-ip=40.107.243.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Oglm2pG7" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OG7uBnaG0JEPm318sBqQWhL8dFVl+2uwmYej9taLwjZBYowfRPPShblA3zVnifs+JVScO/hPoZU0xqaqsba3y5lrBEqEVbxmmQ8RVrEgEs7rWydNdo24szl4mfcxUeRIeT5zjzURzSMLwxklU8/kJuTNwu4giPtYDhT1yOKf/n1livoRayzISy8GmXVcpInAVFMBsG2DLCT942AywxVPLJOPNbpZKbf1LyKnanihvqpL0FjMT5WP0BQYhKRr621LITWlw6LnrLdniuJU2O3WeklaW7gSK0Vhe0gEQZtLdU946QJ/vkrBGABc7rOyaUGOTQdLPjl35K2QIzmu6P3nrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KccQmjsMUYgdF0Fd3PhmMVSrOYuiGPTwG6CX+zqOyNQ=; b=vK+dUC1EmjNzL/2HbCXQbqiY6ZbdVvxeasBrNIDWz8mcppfbV6NG4t/9Xo+Whz6zvfYOB99C3vWYcwZ5VVqHFqtzF4TdIWSxQ55ZtRJEztAsixPUAjRIqmlEAmp+f4n/qVE7Rop8Dlk7HjYak0oeEOLSFs8Ov4twVU+OGeBGf3BT4J3p+5aEyLVDds9KGASDVhAL/JPrOW39f4BjmlR4Xl34nJROP8McQPwfeU+Lj2nObYseS7OHTwcemRpKue/uekxZmFF4l2yH8Seb3DZP7X6BqkKGsJS2qwKUfGabd5QYuhYGwbfkDVGQqFCzZHWG9ThVEB+v0qR4/jQa7sgw0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KccQmjsMUYgdF0Fd3PhmMVSrOYuiGPTwG6CX+zqOyNQ=; b=Oglm2pG7MKJdsgZRiWjojghIpUHG+PuddoeMIHlqxwqqOKVF2S0RmLUzFT50bMXp+HD+0u6YJc6xm30ZEYU67vbNqwg0pnvQ8x8uwRu4jrwPyfrzIaVwLISEZSzYwzZa7x4+9g/z6BwAenoA2qyZd1bgzBn4++aB9wA7WIufD5s= Received: from BL1PR13CA0305.namprd13.prod.outlook.com (2603:10b6:208:2c1::10) by IA0PR12MB8695.namprd12.prod.outlook.com (2603:10b6:208:485::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Thu, 27 Mar 2025 01:50:06 +0000 Received: from BL6PEPF00022573.namprd02.prod.outlook.com (2603:10b6:208:2c1:cafe::cf) by BL1PR13CA0305.outlook.office365.com (2603:10b6:208:2c1::10) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8583.26 via Frontend Transport; Thu, 27 Mar 2025 01:50:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022573.mail.protection.outlook.com (10.167.249.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:50:06 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:50:02 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 14/16] cxl/pci: Remove unnecessary CXL Endpoint handling helper functions Date: Wed, 26 Mar 2025 20:47:15 -0500 Message-ID: <20250327014717.2988633-15-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022573:EE_|IA0PR12MB8695:EE_ X-MS-Office365-Filtering-Correlation-Id: d30beb38-5b21-401b-9e2a-08dd6cd1b327 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|30052699003|1800799024|36860700013|82310400026|7416014|376014|921020; X-Microsoft-Antispam-Message-Info: C8wprnFZHgUuocyMLQamk/Qy90IAWSRUotiltvtN/bkF1A7liZXhRxni7vrvmDTTBklZIp/xUJrOSFOnUh9CyJLArLkht9OarqVbfhmLk6aDMQdiN1KnMfudC/lvvnaYqee9bMDNj9TLMLxucWqrAcxlLBO/3GJUhKTkUO7lsjtwg508zDV6LAE4Z0kT+bjGDmPcGMzalH4R8eKrcd40VRpiofBh/j5uGCvPDOf1Ce4FFBhqg9bIdQiUUE5dZl9UqCd4eWRwpJhcSjPF9WCPZI4+r5aiV1xZ/VvAjBIEN2Jp0PmImtMRe3sb9MU61zwNWk41bQoF4nhhTEnWQkYH1AhsskbcGYVcqdGq+84yuB6b5Bu/WQYrLIvWXYh39bxW0Av1oc1CdWHlcBF3v5D9d8B/wn4AEnuLzhdzWW6bjrGliJ4sq5SpO1zwDpLaytDk0ynzbahJx3LSJXUwPMIb2oH0xYvvqgmFXVdb7WBPN7wtfy88s/qK4K4xZhrPMe0Pq5FMCLA/tyMqfAYX21rkchbNoVwvPbMiTqNyc3gxTab/xpqRLfUfAYWF9jLPp3EPq5510AfWvdGjiCBY98jruHbPPLZh7nFt2eRnyAh/W390q6oS921ypjQNNIKEL/dzBRotpGHgymU8nu6l6W7btWPXi2uwpyTLK/6dQ0uV2fDpHbIE/YJQlespe2UE0UmNL0YhspfFQ0nYOVkqNxadUvUtfQRVw3Bu4MPp8gUWrum3FzANjN7nCwR9mJN8s7epk+08BKkoVQqBeyWumdqujCa/cm+JjzPI8Gt8rOyEUQEygj/ntR4Lo2bu3LrQe/i+EmM/Uxln1k5k+YCH1JOWylxUZ1bjRKVLYBgp4Nmjivki7j54HKEPKiAPxPvsNwDEzkedJ5VF7MgI6DQY6usvlYWLDN33QCtOAlawFV0XNds5GCvGXTzPquRODrVfTQ1ICtttxAuBv09W0E/mZVw8H3NU9gTKXN0SxWsvwy3I9wnyGRxc00KI5xOwwgTkRvyrJQFBdcpcQsSe0i5AhMbSb4ORu18d+9cF/pcy1X6QIKKUpJX8/re3bqvXIX9EtyP+DLWsxGioJRFIaAUxZYOTcGSeIIgtxvG26Kc504U0iYZGPDoqa3MvjLUJMyJ7wHaj6sqO8tiIoTguYujep9Ipw3d46Xq0ZFG0ibSFWFv8WGoWmM7B1SUn0RRYIvdcyHERw6hd9r5r+vYRfbAK9gnZAahzclp0IUKB8r7zVEzl/rWsPW2MWy+5WaKgby+rfjQ12h5Tc9YIR3Z6dQNZgzbYYEa8HGA74HlOJJXPjjA8DMNzbTFg1AesJbPNs61qXwAi+wa6gmvEKzpPpEvB7keLRim0VlKys4wzrqtjtubjFl8WoDwHBDCimEUIWJnBSsuBZ5zPmB1PbVPbjBUUpjVFPZfsopVPkUGSesXAzVIof0b9he+13bRIeWo46RZh+oDGrGhT268V6dNU2C7pP2XcDCvJV2Ex+d7sctJz9jthxbeI4ZpCkG4xUGD/SiG+lHd/ X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(30052699003)(1800799024)(36860700013)(82310400026)(7416014)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:50:06.5849 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d30beb38-5b21-401b-9e2a-08dd6cd1b327 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022573.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8695 The cxl_handle_endpoint_cor_ras()/cxl_handle_endpoint_ras() functions are unnecessary helper function and only used for Endpoints. Remove these functions because they are not necessary and do not align with a common handling API for all CXL devices' errors. Signed-off-by: Terry Bowman --- drivers/cxl/core/pci.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index f2139b382839..a67925dfdbe1 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -670,11 +670,6 @@ static void __cxl_handle_cor_ras(struct device *cxl_dev, struct device *pcie_dev trace_cxl_aer_correctable_error(cxl_dev, pcie_dev, serial, status); } -static void cxl_handle_endpoint_cor_ras(struct cxl_dev_state *cxlds) -{ - return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, NULL, cxlds->serial, cxlds->regs.ras); -} - /* CXL spec rev3.0 8.2.4.16.1 */ static void header_log_copy(void __iomem *ras_base, u32 *log) { @@ -732,14 +727,8 @@ static pci_ers_result_t __cxl_handle_ras(struct device *cxl_dev, struct device * return PCI_ERS_RESULT_PANIC; } -static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) -{ - return __cxl_handle_ras(&cxlds->cxlmd->dev, NULL, cxlds->serial, cxlds->regs.ras); -} - #ifdef CONFIG_PCIEAER_CXL - void cxl_port_cor_error_detected(struct device *cxl_dev, struct cxl_prot_error_info *err_info) { @@ -868,7 +857,8 @@ void cxl_cor_error_detected(struct device *dev, struct cxl_prot_error_info *err_ if (cxlds->rcd) cxl_handle_rdport_errors(cxlds); - cxl_handle_endpoint_cor_ras(cxlds); + __cxl_handle_cor_ras(&cxlds->cxlmd->dev, &pdev->dev, + cxlds->serial, cxlds->regs.ras); } } EXPORT_SYMBOL_NS_GPL(cxl_cor_error_detected, "CXL"); @@ -907,7 +897,8 @@ pci_ers_result_t cxl_error_detected(struct device *dev, * chance the situation is recoverable dump the status of the RAS * capability registers and bounce the active state of the memdev. */ - ue = cxl_handle_endpoint_ras(cxlds); + ue = __cxl_handle_ras(&cxlds->cxlmd->dev, &pdev->dev, + cxlds->serial, cxlds->regs.ras); } if (ue) From patchwork Thu Mar 27 01:47:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030714 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2082.outbound.protection.outlook.com [40.107.243.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCCF11A08DB; Thu, 27 Mar 2025 01:50:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.82 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040220; cv=fail; b=QViXWUN6RFoNLvPhw3Si+J21VVGmDZJOIai4vSnZ7oRTwhlhbVxXehQEF8DQCxDJChmho00HQaX0gPyXBEJ509kQrIFoNZYyPpO1NDB+ClECTK0/PMyUupWdFAmr7S9uh8yR9jZ3Rlyb0v/stdI7MjWM1/S8aAaJitTWPg9kMdk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040220; c=relaxed/simple; bh=3k0sA1XrRBCTtjz7qcYJXdvlLeroL0F5Y4nw09WrZC4=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hEEk0fNp1PqswiacJgjc4tfnljXenbHeno6jWtXCnZKCFbkMn+oIxDO9VWJzG9LL/9B8296fk9RRkmymH9TvCKLS6LFvEFw4SPdo1TuRdWkZDo3wFipGmbwTzO3eXa3nsBEbI+7eRsxPIec8nems4fTB5VrZqFCmTUgQBSXSG7I= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=aiUxsdx0; arc=fail smtp.client-ip=40.107.243.82 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="aiUxsdx0" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=S/HlF6fRz2c284LJ3qy+nsRSmk1ptCc/w14ng6vHJ7Cv87/9i0K/KTv/6Q4Yf0gERKFTwwYGqfXU/GYJgtXYApZqgwHNLrGUnITgQA/uYiafH5w+Qi15X1ZBzrJs3G83f2XiETgLQrxN1TZcFwIeskPjgroz5ZDKD8sBgVE2gPPzj42mDLbGnS8N1dQBzJp/kiWLI54rIuQqmCcu/dG6KEx5kYXPkMsc4AbUHSV2JQhMQSAF8MrjJRqTkCT384L9NcXoEn2Z1fKs+JxvJk7oGUR7Vi432r74s8vCuflpRptmNhvxL0IGkA0ImXYD2E9m1ZjHrG88JL7btzAnwkZogQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=C1UhMaoTbL81h8kvBAE8ojSWBsM18f0yfeJPfXdSKRc=; b=wcNdEov0Fb8mwnmvXi80mM4c8ZMZC19zPemggyt6UCRFmlNNojDuGyR1VNyT7cg8oSuayo3cotDipHS8RY3eSXhM7PNseS5dvv8GXE1yDVKmpCCpBZ0vk73ntpvzWNTF3yOWs9W/x00F0hY8MJ6axJCCK0VhWp1GjKjjCcI+7wQdAXEp8iWsUUELBt+OyqQct+4bPEQ6aZrxeVNg3N+p/y7dd0BdEf2LOiFH25gE5oh6u6Vri9gpmoVHIDG5G9SeIlBvgL1SW9rGN19CfWGM4h8/p/o77aerbTFZKHUpb1ahlxivZpYonXmalzni1QA+Lej2H4z5kSvpgSZL1+AB9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C1UhMaoTbL81h8kvBAE8ojSWBsM18f0yfeJPfXdSKRc=; b=aiUxsdx0BDm9q7ocsyGGUE4tyC4hoTo25MrdHDc9fiKNqAJltYO7y0RF/wZTUzZwkUBh49SwB3EpC7NKxa1hy65WlPUZXhVXI/ufV3fkZUEu/MHxJb72K5ZBf/j13I7a2pGjwhsqZ1UE2DTLeQ5wsxA35zYjlXUoch9jbTr8a38= Received: from BN9PR03CA0250.namprd03.prod.outlook.com (2603:10b6:408:ff::15) by CY1PR12MB9673.namprd12.prod.outlook.com (2603:10b6:930:104::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Thu, 27 Mar 2025 01:50:16 +0000 Received: from BL6PEPF00022570.namprd02.prod.outlook.com (2603:10b6:408:ff:cafe::3) by BN9PR03CA0250.outlook.office365.com (2603:10b6:408:ff::15) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.43 via Frontend Transport; Thu, 27 Mar 2025 01:50:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022570.mail.protection.outlook.com (10.167.249.38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:50:15 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:50:14 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 15/16] CXL/PCI: Enable CXL protocol errors during CXL Port probe Date: Wed, 26 Mar 2025 20:47:16 -0500 Message-ID: <20250327014717.2988633-16-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022570:EE_|CY1PR12MB9673:EE_ X-MS-Office365-Filtering-Correlation-Id: ce55ee71-bd25-41e9-da3c-08dd6cd1b876 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|36860700013|376014|7416014|921020; X-Microsoft-Antispam-Message-Info: k/f3Bem4HoKKmE6i2k3LiE31lTsXfCfbUNMvRbh3vGfpHLh1f6jiLkaGv3Wf7dIbPlpiMNAR/hU+B3iVMPf9nsfK2Q51txn7japSbzFDVBCOn+CWDXaC8mpQ6FAm8qQXb9evgm3EdTq4Dr/2sWZp4ad2Fdmtes4b9E/S+EG8Nqzt3twYnW1LX/Sl2xgDwf5v6Anrz5LUsShrrAR5RbJnPCR0sjMGsBTYt/2XE8kbvhikZTY5vpiCqT63gWACKvOqnepZ/1dVvmEMRVIx2rjKksbrapN+UhSSkgcQEm4cdlxWfrJVma09K8luwDkIN5CPDJSKojI0WfN4ybLOhSq3f2JICXOP51YXj1d3T4GACLjQBp4Np6j3k+spiTQlfo/7hKYiRGCg7OEpXxttU5dIDReg10tbHBS5xACHyQ4fdsKiam5C6/UnXrcgbrK94G/OjAE77Q0JbAPy1kFofeSUQmicKd/PsN5LRPlvTXgUEM7iImmfF02S4KljfYrQgsfgBHWu4GJK+02V5/kWHau+hs+KQea5dBIrWbFD2u0nCsP+oti1of7HvMz4BLUjzOusAXbd4E/wKLlZMsjYqVpaJpqShfE/vdTngqt+4sh5WLCkxsD6rpdwU4Fdj20Ghhx4dlpTq1yUh+oChBPzW9B9A5Wf13LPlCG+tHPa0O4dn2ZwNYjErZAYIg+Ftxa0hmwfUtSM+ZoMbgnBDPslaU1YIh9eTld+rEVY7UhIYFKy2no+ELPXoJvd3/awP8vjazOaDIulon0EyOmzGnTMXGD75qVW4h40qBI8BJFcgpA5DtolF2wxfHw05gpwYoLAoUE9tyjy2niXfmyum8lGDOVikqz3IXPnaZKQZ6TblLlZE7sPHhVvCyolXl+o217XC9FaGS3zHTvpZFOEC9MWDg//4dcW6Y8k8EkLdIo3nGrY9UiPiLCgqtw5eApwxJEr4w8VNlK7pSfgpTr3dXVLic2+h/hQ/WzglasXEigIqGu5JuypSOcp0JYxthB6BmtBjxyx8fen0QBI3BDvfN3571/YOjUnPqrRksFte1hDQqAy8jlcNEFqaAji+3lQ91ghD1PeLLNhAYl+AlGY42c9ZdegokpIpBZzw91prlXM80SVuFb44lQa9l9ULBYMkJjM4MMbx8EnoPyPMarCxjGdoedYc6BnT3N/jAu1/mSobyzlYv596nh3EoGLTX2pfo3e6a/0ee0NB+1mH3j1P7y625q1vTRnz1nI1iS5qXe+9p2jza5kS4Yer2mhfdDgvfIkMYsa97YVFXb9Sd3aWXTxNbBqByYu3Yfg4j/l3xoJhF/wObJRd7qfpiNZnHZnIP3sZZZx7hUvIfeNtdG6TG7lkjLX7fWA4m4rroGsT+UBDCdbl+oShFGVbfx3ShSdVBTSkplpP9lYcLgWstcJjwtQ9uQo6aXDCJ/n4xDQ/6IvSnkM2FNUHe8yco4FdKQVH5bH8X+1iy9PoLrOtOIojBi2+QtrA2fZKsUJJI4CeCEztg73NAnSgC2J9oeBuH941xyh9jU9 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(36860700013)(376014)(7416014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:50:15.4771 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ce55ee71-bd25-41e9-da3c-08dd6cd1b876 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022570.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR12MB9673 CXL protocol errors are not enabled for all CXL devices at boot. These must be enabled inorder to process CXL protocol errors. Export the AER service driver's pci_aer_unmask_internal_errors(). Introduce cxl_enable_port_errors() to call pci_aer_unmask_internal_errors(). pci_aer_unmask_internal_errors() expects the pdev->aer_cap is initialized. But, dev->aer_cap is not initialized for CXL Upstream Switch Ports and CXL Downstream Switch Ports. Initialize the dev->aer_cap if necessary. Enable AER correctable internal errors and uncorrectable internal errors for all CXL devices. Signed-off-by: Terry Bowman --- drivers/cxl/cxl.h | 2 ++ drivers/cxl/port.c | 22 ++++++++++++++++++++++ drivers/pci/pcie/aer.c | 3 ++- include/linux/aer.h | 1 + 4 files changed, 27 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index c1adf8a3cb9e..473267c19cd0 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -769,9 +769,11 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port, #ifdef CONFIG_PCIEAER_CXL void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport); void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host); +void cxl_enable_prot_errors(struct device *dev); #else static inline void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) { } +static inline void cxl_enable_prot_errors(struct device *dev) { } #endif struct cxl_decoder *to_cxl_decoder(struct device *dev); diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 8e2b70e73582..bb7a0526e609 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -83,6 +83,24 @@ static void cxl_assign_error_handlers(struct device *_dev, pdrv->err_handler = handlers; } +void cxl_enable_prot_errors(struct device *dev) +{ + struct pci_dev *pdev = to_pci_dev(dev); + struct device *pci_dev __free(put_device) = get_device(&pdev->dev); + + if (!pci_dev) + return; + + if (!pdev->aer_cap) { + pdev->aer_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_ERR); + if (!pdev->aer_cap) + return; + } + + pci_aer_unmask_internal_errors(pdev); +} +EXPORT_SYMBOL_NS_GPL(cxl_enable_prot_errors, "CXL"); + static void cxl_dport_map_rch_aer(struct cxl_dport *dport) { resource_size_t aer_phys; @@ -147,6 +165,7 @@ static void cxl_uport_init_ras_reporting(struct cxl_port *port, } cxl_assign_error_handlers(&port->dev, &cxl_port_error_handlers); + cxl_enable_prot_errors(port->uport_dev); } /** @@ -177,6 +196,7 @@ void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) } cxl_assign_error_handlers(dport->dport_dev, &cxl_port_error_handlers); + cxl_enable_prot_errors(dport->dport_dev); } EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL"); @@ -201,6 +221,7 @@ static void cxl_endpoint_port_init_ras(struct cxl_port *port) struct cxl_port *parent_port __free(put_cxl_port) = cxl_mem_find_port(cxlmd, &dport); struct device *cxlmd_dev __free(put_device) = &cxlmd->dev; + struct cxl_dev_state *cxlds = cxlmd->cxlds; if (!dport || !dev_is_pci(dport->dport_dev)) { dev_err(&port->dev, "CXL port topology not found\n"); @@ -210,6 +231,7 @@ static void cxl_endpoint_port_init_ras(struct cxl_port *port) cxl_dport_init_ras_reporting(dport, cxlmd_dev); cxl_assign_error_handlers(cxlmd_dev, &cxl_ep_error_handlers); + cxl_enable_prot_errors(cxlds->dev); } #else diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 763ec6aa1a9a..d3068f5cc767 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -962,7 +962,7 @@ static bool find_source_device(struct pci_dev *parent, * Note: AER must be enabled and supported by the device which must be * checked in advance, e.g. with pcie_aer_is_native(). */ -static void pci_aer_unmask_internal_errors(struct pci_dev *dev) +void pci_aer_unmask_internal_errors(struct pci_dev *dev) { int aer = dev->aer_cap; u32 mask; @@ -975,6 +975,7 @@ static void pci_aer_unmask_internal_errors(struct pci_dev *dev) mask &= ~PCI_ERR_COR_INTERNAL; pci_write_config_dword(dev, aer + PCI_ERR_COR_MASK, mask); } +EXPORT_SYMBOL_NS_GPL(pci_aer_unmask_internal_errors, "CXL"); static bool is_cxl_mem_dev(struct pci_dev *dev) { diff --git a/include/linux/aer.h b/include/linux/aer.h index 8f815f34d447..a65fe324fad2 100644 --- a/include/linux/aer.h +++ b/include/linux/aer.h @@ -100,5 +100,6 @@ void pci_print_aer(struct pci_dev *dev, int aer_severity, int cper_severity_to_aer(int cper_severity); void aer_recover_queue(int domain, unsigned int bus, unsigned int devfn, int severity, struct aer_capability_regs *aer_regs); +void pci_aer_unmask_internal_errors(struct pci_dev *dev); #endif //_AER_H_ From patchwork Thu Mar 27 01:47:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 14030715 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2043.outbound.protection.outlook.com [40.107.223.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C700155312; Thu, 27 Mar 2025 01:50:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.43 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040238; cv=fail; b=SbDtFgxSHkJuKP3+bXJPJX0egmPw4mCue97qgCV8ICSaf2gZZ585wLKAyOewwxm7iH10AdOxXXI8VYMZ7rlVEaA49rJafAxZcXapx0m2F+xOyRSIM+891Hk1qZWqrul1Td+tzfxwUqngzxDRi2u8hYjEvE7qj/Z3nsOQjkU7MCE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743040238; c=relaxed/simple; bh=yHBaSlWEfdy1u7SWeDPDDA2TIV+AeYPW1bBMdITrQ1o=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=KpRqDzUkQBMq5ULTf7quuHGdOqYeTzoy3GB5cieKHd89zDXididgcywEDYB1jzpvgmbe7PjIGnWpUnXFARGv+QVuer2FqoabaKK98ODLfICgriypqW1fHm83iaKe6xnOC9vPV76//QM77sXCn2pQQq6uBNipr4P4x1PDh1ygVbc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=umfOEVja; arc=fail smtp.client-ip=40.107.223.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="umfOEVja" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HxoGiidTS1Z0pwimIscpKnRm/sCo/Q9MTSDiMCm+qFVZ6e2/taeFnPnSXq+TsyWMReY1BXztomiNIHZNETxqGWNo087KPAe4/Oc1JlLxxNNCqdImo4Nv8/3v5HaqNUY03FLWJRWW8bniTHuisQG1pPzXvzZCBaFDEgaylEy0/54y+o/OvZk9XAZ7wnUKuRiKR/x0AzYCMYAVjgTKYcwMj74iD4ScVRp8mB2mgCIpyv/kewKTbbbfSzctZQ9JCHTIw7GBJQVI8TivPc7Dx9+hjux2aS7WYMOvBPRq+TrfxT3fi1C05sP8udIXby+2CtP6LOUTtmb+apEhupLE5pj8hw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jJuKtnYZNx7iAixvtqlSRDhMR7Fzck0vwEHY/3dH6BE=; b=VxqW/HqRhKhCu3sOYIsPVhQEEVJ0TARceOYUgTxMceYqGJz5hlc8vVcNjZOV6DK/a7Vdea0iBUNYeGDZ6+zyH2jHlvknDNhesWtzTSErrYkW8YRLcxrdMMP/lx1at340D9pKgqSpZuQ7EZAoJPWQmKJHLuCH03xfGArqPPUfMlQtdLFSaCi+w88GyYkRi+CTRo49YbL7083WxdDaKI5uDlXoTdm5Milz3aBHGBI+C+3bKWAp+y6yiX1Ntp3KEmIe5yWrukqc+wSxYr1dZ9aRElqbbpfSMQe5NC2GjmD3HaUj9JhN+C7Ey2vg2+t5n2NuHuaLVeddh3vPdwCX21Qanw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jJuKtnYZNx7iAixvtqlSRDhMR7Fzck0vwEHY/3dH6BE=; b=umfOEVjaaUYSNAbDQxAdoOMDNYMHzAfqIjcYECj/i4wIhSlqB3LpXDH3+AKH41ySqCzU5SbkgV8t2p+xCHvkUXa6Zjh3z4UBkYC2nnv6n16TrdsxKlbiU2thKr6K5nj5SGtVoOHLvABWIhbb5mQopSifLsijQsqbIYynIyXJyIk= Received: from BL1PR13CA0124.namprd13.prod.outlook.com (2603:10b6:208:2bb::9) by PH8PR12MB7278.namprd12.prod.outlook.com (2603:10b6:510:222::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Thu, 27 Mar 2025 01:50:27 +0000 Received: from BL6PEPF00022575.namprd02.prod.outlook.com (2603:10b6:208:2bb:cafe::4c) by BL1PR13CA0124.outlook.office365.com (2603:10b6:208:2bb::9) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8583.26 via Frontend Transport; Thu, 27 Mar 2025 01:50:26 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF00022575.mail.protection.outlook.com (10.167.249.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Thu, 27 Mar 2025 01:50:26 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Mar 2025 20:50:25 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v8 16/16] CXL/PCI: Disable CXL protocol errors during CXL Port cleanup Date: Wed, 26 Mar 2025 20:47:17 -0500 Message-ID: <20250327014717.2988633-17-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250327014717.2988633-1-terry.bowman@amd.com> References: <20250327014717.2988633-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00022575:EE_|PH8PR12MB7278:EE_ X-MS-Office365-Filtering-Correlation-Id: 513bf1be-d400-4448-5859-08dd6cd1bf34 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|82310400026|7416014|376014|921020; X-Microsoft-Antispam-Message-Info: ECT1wnSENI/KRc9Mn9jBNOSxt/yTMjrkPiBOS0sa5TW0dwChXTFnBO+Pzx9Fk80olVAfXthQyhnPNy9ITNItkQFuca053fcdbQuM8Ea/XBX1c9hw50bNxssG1lZXNsho+ejxm28NcfaRgEjc9veSTMt48MT65G2bK0JHwJxLsheuPD5i8AzKfrkz9arfeuQHX0+35tscpam+oJib5jjRPOPyxGj0jZu7NlEoskoVZd03UoGpi/Dy+xU6Wg6IT9xyPe/fHMzLtxIo7TJ4Q3JmV0ZqR5BVM+17sMucYa1yDQmt1XvyoQ56bJpF6OAZ8V8c8LSQGvkd7eSugX4sYtpUlm3m+q76he7xMsdGZC/uzVajj0p/FDZzi7rUWyRaHZQFyVJqiRiveLH5zEso31HJONhWr3z3iLQnPazanOl65UWQ2iza8ibxoTqqEI7Mumjdrwyt/SOaXUC68uhncY9tm3lw7DnEj29WIn8AUge3SUEpknaqvPpZVT9ZIJxaGhsXNGiCe/z0ETLBT/p7JW/jmGoIPv7R/t0LVRthanWilKPtgeMBT4l/uaIl5t1dzgJlJvYDv287nrTGwc9M6aJ/Nwy/efYCNTJ1lxoW+AlsWKgMqJwRWXb4fkMrXf8rWVkmKD6s417++X4oh8wQBUFrodpMvP9a22VO1vNUinyB+eYgzXJSUOCS5au2gPNBoI5nuivTr9Q4gZapcC54DvRsQeutT440dUOcsDesMdY7bLrejb8JeJkiv0v7DhEZCqhS7MWbF65x9umBae6lt+fWPhyLUcMKIb8zXaZOrCOf8ivrM3bCIJC5/Rgslx9Q05mXAQ+6goq7GX2HBA1ounCcRrIgypbuE9L6BE270XrR1u6YRdSAPSkQ04lzimd5bXcpM9XUlS9VHpS31bn5okSa2hs2bePfwlSIHCUOckKR45dZre+ZDj3gGad2ahdS98ZA51D+GROTxpIHaLUmiKQpOSvxWFdvL0h1HcQGBD5u28apUTNFL/K7bs3Ufmhq7OGSMtNu82eSwD4p+cr+Y6kT9IIFgSLfLRXG3pz/o6+bHgprLhmBZ9tgmEZJfAp/rtSSmTPgd50GBfnanuWf+PH7gSIRJtxh9W8ZnnAG8mVVlrNBi5SoHBZgNhJDHk4pJlTPVVSIaj2cEnwkp98P3BSgsjQmQy7RgAlWEXNg2frTtVny9m3q/MB1mpzk/elmjUoEdogaIJOmwad6i+Fw4Oq7CdrGXt6zZ3QfG9QmV7Ls3hTblu2b18Ad84IaX9uG13zfMm3GOvuJdwRG21Z51WiV1u6xJyJzp/gWTqsdQVzkNAb9XuthzfhkpW2tMqTkaSBXPXSFuowjyKaAfnUNdba4SX32UCaiauq8dW54syxcEy4Q/h/s4bQJiIoNW609oes65DvKEndTOyZF3QRr9X1NrAEbt4daHc+dP/CboyaL8eMjTxiMzFBYcZMX99AUpQlj46wLI22/6T/q56CTwBN/mK5RkwxPJi4rNWqcIIEItJ5Ux1wNdafT8ALgbK7JUHdH X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(82310400026)(7416014)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 01:50:26.7853 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 513bf1be-d400-4448-5859-08dd6cd1bf34 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00022575.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7278 During CXL device cleanup the CXL PCIe Port device interrupts may remain enabled. This can potentialy allow unnecessary interrupt processing on behalf of the CXL errors while the device is destroyed. Disable CXL protocol errors by setting the CXL devices' AER mask register. Introduce pci_aer_mask_internal_errors() similar to pci_aer_unmask_internal_errors(). Next, introduce cxl_disable_prot_errors() to call pci_aer_mask_internal_errors(). Register cxl_disable_prot_errors() to run at CXL device cleanup. Register for CXL Root Ports, CXL Downstream Ports, CXL Upstream Ports, and CXL Endpoints. Signed-off-by: Terry Bowman --- drivers/cxl/port.c | 18 +++++++++++++++++- drivers/pci/pcie/aer.c | 25 +++++++++++++++++++++++++ include/linux/aer.h | 1 + 3 files changed, 43 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index bb7a0526e609..7e3efd8be8eb 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -101,6 +101,19 @@ void cxl_enable_prot_errors(struct device *dev) } EXPORT_SYMBOL_NS_GPL(cxl_enable_prot_errors, "CXL"); +void cxl_disable_prot_errors(void *_dev) +{ + struct device *dev = _dev; + struct pci_dev *pdev = to_pci_dev(dev); + struct device *pci_dev __free(put_device) = get_device(&pdev->dev); + + if (!pci_dev || !pdev->aer_cap) + return; + + pci_aer_mask_internal_errors(pdev); +} +EXPORT_SYMBOL_NS_GPL(cxl_disable_prot_errors, "CXL"); + static void cxl_dport_map_rch_aer(struct cxl_dport *dport) { resource_size_t aer_phys; @@ -166,6 +179,7 @@ static void cxl_uport_init_ras_reporting(struct cxl_port *port, cxl_assign_error_handlers(&port->dev, &cxl_port_error_handlers); cxl_enable_prot_errors(port->uport_dev); + devm_add_action_or_reset(host, cxl_disable_prot_errors, port->uport_dev); } /** @@ -197,6 +211,7 @@ void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) cxl_assign_error_handlers(dport->dport_dev, &cxl_port_error_handlers); cxl_enable_prot_errors(dport->dport_dev); + devm_add_action_or_reset(host, cxl_disable_prot_errors, dport->dport_dev); } EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL"); @@ -223,7 +238,7 @@ static void cxl_endpoint_port_init_ras(struct cxl_port *port) struct device *cxlmd_dev __free(put_device) = &cxlmd->dev; struct cxl_dev_state *cxlds = cxlmd->cxlds; - if (!dport || !dev_is_pci(dport->dport_dev)) { + if (!dport || !dev_is_pci(dport->dport_dev) || !dev_is_pci(cxlds->dev)) { dev_err(&port->dev, "CXL port topology not found\n"); return; } @@ -232,6 +247,7 @@ static void cxl_endpoint_port_init_ras(struct cxl_port *port) cxl_assign_error_handlers(cxlmd_dev, &cxl_ep_error_handlers); cxl_enable_prot_errors(cxlds->dev); + devm_add_action_or_reset(cxlds->dev, cxl_disable_prot_errors, cxlds->dev); } #else diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index d3068f5cc767..d1ef0c676ff8 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -977,6 +977,31 @@ void pci_aer_unmask_internal_errors(struct pci_dev *dev) } EXPORT_SYMBOL_NS_GPL(pci_aer_unmask_internal_errors, "CXL"); +/** + * pci_aer_mask_internal_errors - mask internal errors + * @dev: pointer to the pcie_dev data structure + * + * Masks internal errors in the Uncorrectable and Correctable Error + * Mask registers. + * + * Note: AER must be enabled and supported by the device which must be + * checked in advance, e.g. with pcie_aer_is_native(). + */ +void pci_aer_mask_internal_errors(struct pci_dev *dev) +{ + int aer = dev->aer_cap; + u32 mask; + + pci_read_config_dword(dev, aer + PCI_ERR_UNCOR_MASK, &mask); + mask |= PCI_ERR_UNC_INTN; + pci_write_config_dword(dev, aer + PCI_ERR_UNCOR_MASK, mask); + + pci_read_config_dword(dev, aer + PCI_ERR_COR_MASK, &mask); + mask |= PCI_ERR_COR_INTERNAL; + pci_write_config_dword(dev, aer + PCI_ERR_COR_MASK, mask); +} +EXPORT_SYMBOL_NS_GPL(pci_aer_mask_internal_errors, "CXL"); + static bool is_cxl_mem_dev(struct pci_dev *dev) { /* diff --git a/include/linux/aer.h b/include/linux/aer.h index a65fe324fad2..f0c84db466e5 100644 --- a/include/linux/aer.h +++ b/include/linux/aer.h @@ -101,5 +101,6 @@ int cper_severity_to_aer(int cper_severity); void aer_recover_queue(int domain, unsigned int bus, unsigned int devfn, int severity, struct aer_capability_regs *aer_regs); void pci_aer_unmask_internal_errors(struct pci_dev *dev); +void pci_aer_mask_internal_errors(struct pci_dev *dev); #endif //_AER_H_