From patchwork Mon Jun 17 20:04:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13701345 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2077.outbound.protection.outlook.com [40.107.92.77]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9DE1EE542; Mon, 17 Jun 2024 20:04:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.77 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654681; cv=fail; b=FExAF3cMRij72Z+/CoQ2NgcQz/cQ8AzDmRqyg5rqbqp+FxHqz5CnJgLMwI1jOM1UlYHmVgrXqssAB/irdPyiHl/1Z/j+t9PbB4KssTAiGEKoCdK4YHxehhKZ/2Nn40/w8y8ySsD2qJt6SukGlo0DVYshvo+rwJbFvJ9va5142MM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654681; c=relaxed/simple; bh=iIiWo71BMYN6t41ECHNmO95zl1J+plKfADdgKrkqbuM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=gpGCtPDOZW9iP/dym/ROaF9go8Pix+nB/BMqFBPUCFQ4b00cpmcxxa5PKUD7Qraho0qI2PeSY/EsqvClscSclNM/LdzBbgk1sc/F2QxmmKHbplfVwJ76on15pbuwFcLK9LGMfPJ0suhBpZt5KX3oh9nA7pMZjx1QnzwyOpJRHnw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=d+4hLdFp; arc=fail smtp.client-ip=40.107.92.77 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="d+4hLdFp" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nOhHJqwMb96627X8NvtgHGqBETDBZXsBbO6wOvCDaIiER5kzdDPmTs0DF6fmf6/h5G8HQJ4IY1agFbhzyvLPX7WOgXU1ngNPLB7chlv3Fht1R+Sq4B6eTpSFvVgTgGEvPsu4aTCsvdXnxWj36DBjs63C+x+boAKmX/xvGN3xquFX9adh3f6loQdHgZBn9mKRZ/UIs9+en9lvsPh0Nt9nP6epXE7ngsLj4RYh92BrV4570lO4ZduObObU2+CcFjE8NXlYWyXwQCoC+QVQaTs19rTYq1LtLg6O0OHIL6ozTe49TkhM3b5br1SE/5EactitH0JEVu5YOLymAxw0r3wj7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=l7tuKWOBSNUKoTf6tuLn8VCStoGfSAplcLpQhO77+xE=; b=A6CAPtmCz3/WgNYe/fPU0G4UQabrYa9rgnJNrA9AKiMRt0fYP7fR4MWLxprvsebby/c16GgZx7kaTA3DljsXuy+0zeMO/s+01/cEh+UsgDo1eZ0oH6qjotWF0X2bQwAp7ATdl/h1Y1wG015sbPNHRDBch7TC6NjtjQ/m5UuMmzZmgR3KBp3Hyvbg2PI3GdGLTCDRfoWpqRe33A/rErPGY4XKSpO3DSigOYQ2gVpdday3gOVesO20wT0N676wXgvayyV4iVXnp4l7jj5QUm9aKTvBXxVzlIByN4sEyQAXMfi8YpTJg+AwK3Hb9TI8oNlqUYXKQJYAfRrhAviJpSzY5Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=l7tuKWOBSNUKoTf6tuLn8VCStoGfSAplcLpQhO77+xE=; b=d+4hLdFpZpRa3vEUPtNqjrmMhWOFS7ImTJffyE/zcLWnDIB9AlkLra4Lod/EWboTvraJ5Ox2iTfbYwjFiwo5rEyoF1KghpkDZsNQ/rYOLIQiZ+Z2GAFtAtxIez7acwL2634/bCl/uoNwZKtpq8IdMCr/057lzRZ99LHSvqMfUxM= Received: from SJ0PR03CA0001.namprd03.prod.outlook.com (2603:10b6:a03:33a::6) by DS0PR12MB7771.namprd12.prod.outlook.com (2603:10b6:8:138::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30; Mon, 17 Jun 2024 20:04:36 +0000 Received: from CO1PEPF000044F3.namprd05.prod.outlook.com (2603:10b6:a03:33a:cafe::aa) by SJ0PR03CA0001.outlook.office365.com (2603:10b6:a03:33a::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30 via Frontend Transport; Mon, 17 Jun 2024 20:04:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000044F3.mail.protection.outlook.com (10.167.241.73) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7677.15 via Frontend Transport; Mon, 17 Jun 2024 20:04:35 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 17 Jun 2024 15:04:34 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , CC: Bjorn Helgaas , Subject: [RFC PATCH 1/9] PCI/AER: Update AER driver to call root port and downstream port UCE handlers Date: Mon, 17 Jun 2024 15:04:03 -0500 Message-ID: <20240617200411.1426554-2-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617200411.1426554-1-terry.bowman@amd.com> References: <20240617200411.1426554-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044F3:EE_|DS0PR12MB7771:EE_ X-MS-Office365-Filtering-Correlation-Id: e58444e7-03bc-45b9-5c50-08dc8f08b63a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|376011|7416011|36860700010|1800799021|82310400023|921017; X-Microsoft-Antispam-Message-Info: iD7a1SFXbbi2gOuYIOFa8M/2RtMT0ZA2kuz2Qo5SnypwJZ8/SeCtucrZU02otwHs057ziJLqLqevlYrCdW40mlvCFaxOEBTSAk7ScW8IPM7f66dtvPsHkhdEnQ6/YFXV7tXgLwbr/Z7S1nOF582w3o/PEGY20GsBaHjQUrVmd8H94cyaclbLgbdSMa3aP2lBMqkTx9vkKmRV5hpf/bmbLii+dCSZiD+q68pa+KWbNU8UHTgDbPy0NucOSGLl8zygA/Ao1FVSmT7a37NpHudwXF7NeFDd5GYNXfVWnAm/PDSCGynsEf0jCwGRAHpGUOzAqYzjkP/dCShK2ygxdKksHOri2QpWJEkdo3w8yJIZRj5KEOD2dDzKpyUtcPBuKrZNWSMgV3dJKnyuzMpMsppYi8rjL7fD3comybDzecC3Idp9jvL6E9VtmI0KMxoA9vpOMCCIYz9Zbb0J2KVfpJnJSJ5crmTkV5x9nMReVh4l8OZn0izMwHqUyJ6AYfMkdvLiaalld9/p47SUidWgXhWsMlFYzKCM0uGNJ67PYxbiR2GFDg0U751OSGerRSJXLcQ9qUzlrKZvgirk0JhJ9nsxuQe5cXhP595JCPHRcfD32D4ecFp24Xs4QRQvaUZ3wqX3AM03nrBNDQ7VgfLpcLjeAnArWkx/SSSzbzuEf6FmcPmWWOo+NdGEH5R0mnYq7k02JMYyZqch32WH7oJr380qgO4wQcA593PUWfywyZ783uSaD0CL4BR6oRl3CCAft/ZVaIu5Kl8cuKen9HuGLIjvGG6P9lJVBA0EevJj7v0eUcLQIvzVB5DT4wt5om399jTyV7upkud70E9bztB5kQ5SoO0WRY697pNwd+Z+wAdbbuVN4iZucpO/FCsetG8/YfF6HOxx/QIJpUQuohjguxvRMXvRDgvYgAziGoGuMPzK1xD5NF5lcLh0cVktYRwMEFjgQ1mrzSPKg5ErSamesw1pkfNJ9ghWsAE2/5TFXLqyynqLPam+//dL12oOaDSa04x1V8YMM90FHH+sUps3th5z+9tweTocmfNUk3Oay+ZUFUTEBoSLsj5AYZmAAlQTU56w57w/1nDkz3z7GmxZ1NSkwHOCgf7o5bQU0Tn1AW9PDzz6DKZpP2FlHJbQwEI0qi93uMELyEJlPZYHHgms0lpwLIVpRrhWZQb+XpDWVzPSdwbZI4bU+iKz032F7pW6xA9BSbz4lDXMWu9M10N5N/t6/BJpMmBDzF9Ffk8ru1XxyiYlzSmIssmOCV593QjudLASMhwcpImZ7EPodmuaCqrSZ2CPS3q52sAL4yy0MaUJjCVIHR+EE4XEc5S2IeMRp4y/h+E/rF3yXfpszv2mXLEuJ3OYX6CTwrAmka+wOZOhPiKRyimH1hKlg5mOE2w0un0rtPWoZy+6dXFrHF8kCfU1uKvpVNIzfUhF2TixJoRnKno= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230037)(376011)(7416011)(36860700010)(1800799021)(82310400023)(921017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2024 20:04:35.7993 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e58444e7-03bc-45b9-5c50-08dc8f08b63a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044F3.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7771 The AER service driver does not currently call a handler for AER uncorrectable errors (UCE) detected in root ports or downstream ports. This is not needed in most cases because common PCIe port functionality is handled by portdrv service drivers. CXL root ports include CXL specific RAS registers that need logging before starting do_recovery() in the UCE case. Update the AER service driver to call the UCE handler for root ports and downstream ports. These PCIe port devices are bound to the portdrv driver that includes a CE and UCE handler to be called. Signed-off-by: Terry Bowman Cc: Bjorn Helgaas Cc: linux-pci@vger.kernel.org --- drivers/pci/pcie/err.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/drivers/pci/pcie/err.c b/drivers/pci/pcie/err.c index 705893b5f7b0..a4db474b2be5 100644 --- a/drivers/pci/pcie/err.c +++ b/drivers/pci/pcie/err.c @@ -203,6 +203,26 @@ pci_ers_result_t pcie_do_recovery(struct pci_dev *dev, pci_ers_result_t status = PCI_ERS_RESULT_CAN_RECOVER; struct pci_host_bridge *host = pci_find_host_bridge(dev->bus); + /* + * PCIe ports may include functionality beyond the standard + * extended port capabilities. This may present a need to log and + * handle errors not addressed in this driver. Examples are CXL + * root ports and CXL downstream switch ports using AER UIE to + * indicate CXL UCE RAS protocol errors. + */ + if (type == PCI_EXP_TYPE_ROOT_PORT || + type == PCI_EXP_TYPE_DOWNSTREAM) { + struct pci_driver *pdrv = dev->driver; + + if (pdrv && pdrv->err_handler && + pdrv->err_handler->error_detected) { + const struct pci_error_handlers *err_handler; + + err_handler = pdrv->err_handler; + status = err_handler->error_detected(dev, state); + } + } + /* * If the error was detected by a Root Port, Downstream Port, RCEC, * or RCiEP, recovery runs on the device itself. For Ports, that From patchwork Mon Jun 17 20:04:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13701346 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2042.outbound.protection.outlook.com [40.107.236.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F9BE18F2DD; Mon, 17 Jun 2024 20:04:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.42 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654698; cv=fail; b=IgM5TctwvPex0KmFJXp61y2vgW26AWeiFjIfpu4zbozGpEFieGzFjcWZgCLDICh2ATrmFPtG1lSovfdTM46ycU0eyWkC06I75eqmP6Tna03XryXej9VIny2kPqH/QhVBluVKoGJM9rasyUxZ1VAzzLRuuX41Ti4AxxOsebcbptE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654698; c=relaxed/simple; bh=I8YbldKXiWn3t7cWyoVjtC8qAgtObcRa6GNqAbqkigU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FGH+mqHFd5W/T26BrRUoRixbqzFA1Hhs/crZSNPl+P7dQ8htPinaM8hHgCy26flmkwyPtPJiLWdNL2VaAB3kOxopt19cvMfBGgLWlWWVk6rCp1xzAHtiJ7wP7Coi/H7m3KbQhatn1uGwKOpOTO4SOKAzrm8/pOWescS1S/et3wo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=0E4gTj28; arc=fail smtp.client-ip=40.107.236.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="0E4gTj28" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kdHzhDwX9JjzvRCfPPwKASHLHHnJaJll5jUVYTy0186W65yhAMlUdkmZwLNPnUm6nsn4f+EaCj85zRbyHtKX1acmCS9MBRmxb8S08Mgxry337sUmeTtMrNFY/qnDwfMgTgRj0B/wSsIowbE+v6956FcnWOtxGDRsuqfKB35bTyS6ccyyrHFGcQ9GKwPCRf6OSPltpF7tqHUfaTYklVkga4/7Vm/ZmKp/hAU8XQOSZwIjE2M0UUghhQ1jpiQHgp1bFfX+wiGxmiQeQgzZ/LwHsNKLu2PVMuD4r1U9qzq4W7XkwSCYoAWxFgmYHjjXea2NLdIcSzvdFfeNz83FgY4f/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fMRJJqWCuuwDTeuS9t112XMcqjKsFD0n0c+Z5H69HRk=; b=oBVH8SfGvbFnWc0Z18tV/VLK/fkJerzeWyM6iwT6YR3dLUxtcqTz+wD+Qz88R6kZH6SMMubLQ354J5Dmtj8vpQ6OItWNXV24s3WSP500TXVrB1ucfOb/9VXhO55N+xTYfiOd4vxpeeBYOYjIe092EodEZVdNq81Xng/tVli/fFLG/weSoTfCJARi2jH8MhcONygF81zLbFWMWGKNzkO4WZJN7faEN90yMJWqDIn9BBfPr+OunbRlcGnrQ4n1mBLfhNPR0oX0ehtMH32VCpbBsqRIkixAB/sh4k2+TZT7isOqefG8y+41PDOTM4p24L9SHTCviLyCnEJzlHLnOZT/QA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fMRJJqWCuuwDTeuS9t112XMcqjKsFD0n0c+Z5H69HRk=; b=0E4gTj28KMUTwGXAov1VZvfzuqSiakzVjWK17za3/R4GJud46CA9E5deQO2UXzCaWFWB4djwlhI1EaWc+1dVnu3V5/cqLBtHKK0Tv9/bRGgv3SluG/gjxuygLT3+Dk8gunBWZ2/kC/ZnybBoZw0kx3gDn4jYwtdMBQgwXXK73RI= Received: from BYAPR04CA0029.namprd04.prod.outlook.com (2603:10b6:a03:40::42) by MW4PR12MB6949.namprd12.prod.outlook.com (2603:10b6:303:208::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.31; Mon, 17 Jun 2024 20:04:50 +0000 Received: from CO1PEPF000066E6.namprd05.prod.outlook.com (2603:10b6:a03:40:cafe::85) by BYAPR04CA0029.outlook.office365.com (2603:10b6:a03:40::42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.31 via Frontend Transport; Mon, 17 Jun 2024 20:04:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000066E6.mail.protection.outlook.com (10.167.249.4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7677.15 via Frontend Transport; Mon, 17 Jun 2024 20:04:50 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 17 Jun 2024 15:04:45 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , CC: Bjorn Helgaas , Subject: [RFC PATCH 2/9] PCI/AER: Call AER CE handler before clearing AER CE status register Date: Mon, 17 Jun 2024 15:04:04 -0500 Message-ID: <20240617200411.1426554-3-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617200411.1426554-1-terry.bowman@amd.com> References: <20240617200411.1426554-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000066E6:EE_|MW4PR12MB6949:EE_ X-MS-Office365-Filtering-Correlation-Id: eefab12b-fdee-46f4-7899-08dc8f08beb4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|36860700010|376011|7416011|1800799021|82310400023|921017; X-Microsoft-Antispam-Message-Info: rLfLrvpUkQ1LPlFMrydcq6bAEdhRwrm8FZVITc63Uo3eqzQcHDnU++AHtMDYbKMz5lqfuimZQThzc/cUU+hQtVYDPH5d8P7S70aGadhOEmkLA2miDkCFx3DCRc8BP5/dD2xKTJJ0e1GfERDh/nFtW4AJc4F6M3GkeKKfOZM6yiincQ9rylVAUgt47WvNYxhevJkS9C3i3qSPMHg/GZlMR/gqGy8EtRcHVZlTGrtxXbZLHJb9bWE2qvIykM2NGmgmX/whFY1pW2mVZsetC0TmSpwnLT59K+v3ROr+bTEgOJoPNcVR150f5ijcLFTi6AEZX6UCJ9bQB+8S5x7l+QXJegfedpUBm4yiJMymz5umI4LlBhUZy+2Td7PSDlLkzPudazBYKvoegofTg0yJI1+z0BOxGyoqXtt/RJVjxysV6XGss5TG87cmCvphA6Z0nkC6n3/bQQ8OSkegsWpe4ZAEnJpWeuMSoTJLpZWiPaVggY2172uMyl7RbK3qVd5Gqb/Bn0s8mrGfXGrnVlPp8OKIXzknVpu7u8qlbK8+wDAgMM1MBzUNt83tbC5j5EfSdjvTb43rh944BCJgQh7+Os4buVzbrQ8n2J1jChPfJiCJwGtz88E5i5VhOCpz9wiQmWAOPeYKNmhwyF6xeAfuNNuLD0yBtxhmAi8wJdYTseCWFf5GoF7odCKUdKj7EEzrWlX3Kcc9Zg1erF5Schtcf5V3kHvENzVo+vB6k249cf99TzhO07y4maywvz9a1HrAXcmlTKhY3iJnVN6Mfld8UA4UrQeVoZAl2iJWqmK6B6LxA9EpexKCPm3vHXkgIt8VQ89qlUiCxgEw3le0VxmUJuQXexRsngi/SU7kMD4oLe7nGy69myQloG9WcjYZ/qFFAz2rnLX7oksRyx2TEBM+zabN0AMBKLmCqIn2LZeZnEe3AVw8hOZIK3OUEginKVDV0u8ANcQdWbqH4m3F0ENv1nMeLCvm/gQvQgzqfFbcvMWxnHBHIuITIISDrl5IMn8Q0unMiq1/Sn3wY5lzIEG13CNm0yM3mEq7vYq/Ce8lsDYChUCdi0fDxCzar4fwwpURbOhdxM/v4PO7h4h7EYxSU6bZk20BrO3yzdNNFxR8nbDKFGcBD2cpIJJejmGpchVl2e2ebJayPWOChLLJbpBShFpeti8bvARW0Uyco9n7J9LJ+Qrds/7GnKraQDQfYirw7vOPZKGSSqGGSYLt3S6HuOYZQ8hCtsKtHp2LkHQtkq+NIQHVPbHkusrjo/zafCZMDtE6aM6x7m+6FnchiiVFrARaehS4I5+FsRmH4yHANnv7ipt89xj7+OUtKEE6OxLS6qnKLyJaSugzGhat9gzJCJyGMhBUq5OiQ2feq35htJwXmxRHYst4Kq+zhn69I5g0VywfGdbsxm424Vp07c9k6qw3rusUWJRmvEm8EZH/H14S2sg= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230037)(36860700010)(376011)(7416011)(1800799021)(82310400023)(921017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2024 20:04:50.0057 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: eefab12b-fdee-46f4-7899-08dc8f08beb4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000066E6.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6949 The AER service driver clears the AER correctable error (CE) status before calling the correctable error handler. This results in the error's status not correctly reflected if read from the CE handler. The AER CE status is needed by the portdrv's CE handler. The portdrv's CE handler is intended to only call the registered notifier callbacks if the CE error status has correctable internal error (CIE) set. This is not a problem for AER uncorrrectbale errors (UCE). The UCE status is still present in the AER capability and available for reading, if needed, when the UCE handler is called. Change the order of clearing the CE status and calling the CE handler. Make it to call the CE handler first and then clear the CE status after returning. Signed-off-by: Terry Bowman Cc: Bjorn Helgaas Cc: linux-pci@vger.kernel.org Reviewed-by: Jonathan Cameron Acked-by: Dan Williams --- drivers/pci/pcie/aer.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index ac6293c24976..4dc03cb9aff0 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -1094,9 +1094,6 @@ static void pci_aer_handle_error(struct pci_dev *dev, struct aer_err_info *info) * Correctable error does not need software intervention. * No need to go through error recovery process. */ - if (aer) - pci_write_config_dword(dev, aer + PCI_ERR_COR_STATUS, - info->status); if (pcie_aer_is_native(dev)) { struct pci_driver *pdrv = dev->driver; @@ -1105,6 +1102,10 @@ static void pci_aer_handle_error(struct pci_dev *dev, struct aer_err_info *info) pdrv->err_handler->cor_error_detected(dev); pcie_clear_device_status(dev); } + if (aer) + pci_write_config_dword(dev, aer + PCI_ERR_COR_STATUS, + info->status); + } else if (info->severity == AER_NONFATAL) pcie_do_recovery(dev, pci_channel_io_normal, aer_root_reset); else if (info->severity == AER_FATAL) From patchwork Mon Jun 17 20:04:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13701347 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2041.outbound.protection.outlook.com [40.107.212.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD76718F2DD; Mon, 17 Jun 2024 20:05:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.212.41 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654710; cv=fail; b=KQEZqOVMEEa8jCbi4kMIws1rwUEqEgrMHL365VFkFEYRF1MBgwGKMJ4grLGmVbf2GUVYymC0sKQJvKDRpf+AJMsbPbejO5pRefcqd1wC8TQOdmaz5W6ibw/NuIFTvDtyMP953/Hi6xGb9zRXr0GrSuW6mwGHFVR7MBT3hY7bm20= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654710; c=relaxed/simple; bh=bGWHOWnYdZzpBnr1Ho0/KSj4cSY2AxW0AZjSa8HiLzI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=OQxfSbPOdqV5LzbFnMjcCtFPvb4AZ9J06qjZuACUK1CZahjHu0Xtf0+asl+YJzapmEAcw9k9F3uy4CQ4sgqU+kgcYT//Cg2MTELA+PC+AgmW8Xtm1s8ItcT9u/4ywQIWUlT9uMUmi8f1PY55A6dARHuJuo1rdsxsvzL+817YYt4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=4wFUsUeP; arc=fail smtp.client-ip=40.107.212.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="4wFUsUeP" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CYi0YiiHkv3zCdAWXahIjWdsBCbm1K5b3U446CuY5OmwIPl3/IUeazr16/ZSkcR3P947Hfv8GlPXJLulBFM6DYRnmLtVB2yVrCs3Q4aGOvrk2lJG8meHiM/BwOWrBbY7an5aNjnY/SvR7NzKKV+lTcG3DBYKBNMWYTbAqPD1v467WUIonkjTvCuswxxMirlAx+m7fY6sN63BwIVUa/BJmIZx+BTvzQNIwQPtEoHQsgz+DKapeCOl/EsPqTDQjZpzlQp1SAWCj4kxszIQJ4tGDUXYrcL6vFfG8+Z4R+xn35MX5qg33Ob/YRSvmDiFVdaiy/q/LHWs66zr/rov9cQlTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Xh49tAef+SQQ7oq6oMN+78JJGbyZNM8nnh0QiqIgiMs=; b=UN4IrlvJ8VNtu0379SJacnmhJzQBXiV1Rq1CHiSS3ee86KprkNSveE+H6nsawsTlec7BoNvinkBpNg9DuVJhqlEix566BtTNJCbVvxfivwFVv0KnRDv7mcUuoKvpUED0CQYtcS8IdvXSIQb5m9qbIGO4TaUP4hn2iz0rXb32FL6PrG4GwiXZGPaWsZ4QwgZiecKu3F7UTkmNrURrB9f38mjnMtlvs75+0oCDfV51RnltXVEt+f894hiQPF5eQaq1nrbv1LCj5P2YRqamBpk6m2V/gNvUYZ/DnX4Z+gIU9TsGldaE/e/RPQ2xOMILQOvuI2J7KiT3dcP4J1OUDZTR+Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Xh49tAef+SQQ7oq6oMN+78JJGbyZNM8nnh0QiqIgiMs=; b=4wFUsUePTWr6D8TCIrtXC3tkx66R/ZrfOWff52SLloAkXl4d3PdgS2JmvnPVx/T2c9p8q2IOGYOjbEL3O9sKvN6pIiw0HpEWd6wVH6RXu8B293nJNtv3gs158ZREFin3BawWCy36enKNJZ/tIasMsLPNMXohGZmjS73JYji8y/E= Received: from MW4PR03CA0104.namprd03.prod.outlook.com (2603:10b6:303:b7::19) by DS0PR12MB8294.namprd12.prod.outlook.com (2603:10b6:8:f4::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30; Mon, 17 Jun 2024 20:04:59 +0000 Received: from CO1PEPF000066E8.namprd05.prod.outlook.com (2603:10b6:303:b7:cafe::23) by MW4PR03CA0104.outlook.office365.com (2603:10b6:303:b7::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30 via Frontend Transport; Mon, 17 Jun 2024 20:04:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000066E8.mail.protection.outlook.com (10.167.249.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7677.15 via Frontend Transport; Mon, 17 Jun 2024 20:04:58 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 17 Jun 2024 15:04:56 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , CC: Bjorn Helgaas , Subject: [RFC PATCH 3/9] PCI/portdrv: Update portdrv with an atomic notifier for reporting AER internal errors Date: Mon, 17 Jun 2024 15:04:05 -0500 Message-ID: <20240617200411.1426554-4-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617200411.1426554-1-terry.bowman@amd.com> References: <20240617200411.1426554-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000066E8:EE_|DS0PR12MB8294:EE_ X-MS-Office365-Filtering-Correlation-Id: a44170a4-087f-413c-cbfa-08dc8f08c380 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|376011|7416011|82310400023|36860700010|1800799021|921017; X-Microsoft-Antispam-Message-Info: oYjPdXbLzUKiFcskX3EkWEhXU+QEBOBXmy3M3SINpgwxQOC3nclBJUtJQuBEj5TVpRQPXc32SSlDPGjJLQmgtJWOZVQmOwEgBaohg1NAYMwA9JJJUCBlIxGKrbWFB8B1ZoL0x/JTQQQ/SWb/7Q2cpWEzu1QiXMO4Q4Dl9L02vQ5bQRw/kwycLtwh3W6vgrnanKE5r9rReIzRBnkf5eD3pzRoiCTkXJ3sQPknU4S2zYSnJjLb8iQY0k6IeCGD6dxDl3a7znicXrRcUiEfLmodK5d11e27fQSnKYhiSoUijVU74c2xfJ8s7zyh5y44LTV0GenTv5TPDBTIsjS/pHKqJM0nYJwavXEZOWJS88xA0SzBW6qDBVdtfzcmOJbpi6tdT702hG2Nvs0aGlzk6iUHSvB3glkBUpvvOvM+ZbvRTjy4oL1Upn+g3RHvdAzvAkBdmYATA2sXVdBb+6/v0Uh8uTQMAf4nXroMUtMkmy35Ma1NSqAtDmA79ksedHZyhz69MWGeF4EZupgUANNpuJgylAPnaprseVu8g70W5OrZxaBPorbKiinrfXfySHC7B/LJ/n7R+N2K2nkb2nU+IqjRmmRicoiqN7y78ZEFXTXeLqFgyZlxeHAMyddzfwH54OkIsnT4wuHxxlDNpbeMJSmGPh6Imua29EXqdc58klOKBb2DrjM7ae/1w3M9hWV2flqIDJu5+1yoU0cS/+3E+1rgKWZi7dFeTSR0c2hpC2uY4A+JbvDW+WqCZqSmtNiHivwjvASFv/PKDiO6pYH0uX5qIRB34Wrqry6j38QS53jU0+yEQrW2BJBiR5RSMVZY9bknMnsgQF6w9pC06rsUG1PbuWGNWswHIvIrxt1pH013Trnfkgz+4Xxr32JO/bBP3S3d9hnJkSwUcTLatGpXySEF6I9dLNLmDT6WFNUV1dVx92OF3Hq+4d+mSfv1Cwgkq37rR3UEtfkWbShwHDW8nMvNi3LAZXFMRXwvfTvNcQFLA/Kvt6nHdDyBCVpm6xa+ovrGr8Jyjh5VRI6leE8QWq0NvipBHshIGdwj/4nuKwVfUq1MZ5MqGi2JX1CcUoDtB3D2J5kfg+VacMtDwJ3NjKk817pbkWFc0KqYO3ex+9K802kSiNB+xvi7U089hK1gxNSvDcTzXPSse9bQn5fUdYnD2OHuFAen7xMUkHD4MaLVjDc5H4V8G0y6VxMp7l2ybfFBRbkWIn72zzT+qX6w7hR2cNnFJWQFUg7tYFHipvTcX+jQoKRsv9/qdT7a5xZ1Jp4mAdkRCA439DweaUa55HFvtPYEVg0I9PRaRRfBoEWEAboHHagvpMAvH3SodyfiLJVhewNnRSt+7hCjFO3uG3rSHePxG+nqlnLdzNEaqSPolxqWUgtM/7Y3RFEs3k7mnZdO8zx4ziYtM9N9YSmcE7PeCHORbY8jhPdOIU9Ex0diF/o= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230037)(376011)(7416011)(82310400023)(36860700010)(1800799021)(921017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2024 20:04:58.0713 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a44170a4-087f-413c-cbfa-08dc8f08c380 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000066E8.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8294 PCIe port devices are bound to portdrv, the PCIe port bus driver. portdrv does not implement an AER correctable handler (CE) but does implement the AER uncorrectable error (UCE). The UCE handler is fairly straightforward in that it only checks for frozen error state and returns the next step for recovery accordingly. As a result, port devices relying on AER correctable internal errors (CIE) and AER uncorrectable internal errors (UIE) will not be handled. Note, the PCIe spec indicates AER CIE/UIE can be used to report implementation specific errors.[1] CXL root ports, CXL downstream switch ports, and CXL upstream switch ports are examples of devices using the AER CIE/UIE for implementation specific purposes. These CXL ports use the AER interrupt and AER CIE/UIE status to report CXL RAS errors.[2] Add an atomic notifier to portdrv's CE/UCE handlers. Use the atomic notifier to report CIE/UIE errors to the registered functions. This will require adding a CE handler and updating the existing UCE handler. For the UCE handler, the CXL spec states UIE errors should return need reset: "The only method of recovering from an Uncorrectable Internal Error is reset or hardware replacement."[1] [1] PCI6.0 - 6.2.10 Internal Errors [2] CXL3.1 - 12.2.2 CXL Root Ports, Downstream Switch Ports, and Upstream Switch Ports Signed-off-by: Terry Bowman Cc: Bjorn Helgaas Cc: linux-pci@vger.kernel.org --- drivers/pci/pcie/portdrv.c | 32 ++++++++++++++++++++++++++++++++ drivers/pci/pcie/portdrv.h | 2 ++ 2 files changed, 34 insertions(+) diff --git a/drivers/pci/pcie/portdrv.c b/drivers/pci/pcie/portdrv.c index 14a4b89a3b83..86d80e0e9606 100644 --- a/drivers/pci/pcie/portdrv.c +++ b/drivers/pci/pcie/portdrv.c @@ -37,6 +37,9 @@ struct portdrv_service_data { u32 service; }; +ATOMIC_NOTIFIER_HEAD(portdrv_aer_internal_err_chain); +EXPORT_SYMBOL_GPL(portdrv_aer_internal_err_chain); + /** * release_pcie_device - free PCI Express port service device structure * @dev: Port service device to release @@ -745,11 +748,39 @@ static void pcie_portdrv_shutdown(struct pci_dev *dev) static pci_ers_result_t pcie_portdrv_error_detected(struct pci_dev *dev, pci_channel_state_t error) { + if (dev->aer_cap) { + u32 status; + + pci_read_config_dword(dev, dev->aer_cap + PCI_ERR_UNCOR_STATUS, + &status); + + if (status & PCI_ERR_UNC_INTN) { + atomic_notifier_call_chain(&portdrv_aer_internal_err_chain, + AER_FATAL, (void *)dev); + return PCI_ERS_RESULT_NEED_RESET; + } + } + if (error == pci_channel_io_frozen) return PCI_ERS_RESULT_NEED_RESET; return PCI_ERS_RESULT_CAN_RECOVER; } +static void pcie_portdrv_cor_error_detected(struct pci_dev *dev) +{ + u32 status; + + if (!dev->aer_cap) + return; + + pci_read_config_dword(dev, dev->aer_cap + PCI_ERR_COR_STATUS, + &status); + + if (status & PCI_ERR_COR_INTERNAL) + atomic_notifier_call_chain(&portdrv_aer_internal_err_chain, + AER_CORRECTABLE, (void *)dev); +} + static pci_ers_result_t pcie_portdrv_slot_reset(struct pci_dev *dev) { size_t off = offsetof(struct pcie_port_service_driver, slot_reset); @@ -780,6 +811,7 @@ static const struct pci_device_id port_pci_ids[] = { static const struct pci_error_handlers pcie_portdrv_err_handler = { .error_detected = pcie_portdrv_error_detected, + .cor_error_detected = pcie_portdrv_cor_error_detected, .slot_reset = pcie_portdrv_slot_reset, .mmio_enabled = pcie_portdrv_mmio_enabled, }; diff --git a/drivers/pci/pcie/portdrv.h b/drivers/pci/pcie/portdrv.h index 12c89ea0313b..8a39197f0203 100644 --- a/drivers/pci/pcie/portdrv.h +++ b/drivers/pci/pcie/portdrv.h @@ -121,4 +121,6 @@ static inline void pcie_pme_interrupt_enable(struct pci_dev *dev, bool en) {} #endif /* !CONFIG_PCIE_PME */ struct device *pcie_port_find_device(struct pci_dev *dev, u32 service); + +extern struct atomic_notifier_head portdrv_aer_internal_err_chain; #endif /* _PORTDRV_H_ */ From patchwork Mon Jun 17 20:04:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13701348 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2072.outbound.protection.outlook.com [40.107.93.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 694251990A1; Mon, 17 Jun 2024 20:05:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.72 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654717; cv=fail; b=Qes0/PvdyyoS6KMqOISLLZrCJ1qrdnTx/Jz3id5kO1bNBd7jWQvZF6hwpg7PDu87mjWEwZ9zj5IuZM0O1ypDRY4pCFymAwRkmPDK5YumMtbAdevHSs2/967NrGMIOD27X5e0uWvUhNXyuMfKQEst+dzAK9IBTUCNIZcRbN81oeA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654717; c=relaxed/simple; bh=vpAjLN0Q2W03GmpM1WSixDUGVhbjHYnHoYotzOBU4bs=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=plwpYNuAtS19IvDdyitknAzXLiMFkomsF7wbsTfWtsTgY3Sql9hAS+W2kVpgR6oAtUiQz3HpqC6nUp/q0oU0si9Mc5wRqJh5g+pUjqU4Ay3/jbIjqhisCb1ydxl1f8gFS8jHYji6CxiXvwFdYFU0Aq7TQR2qA1psViP/sYPkeuE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=uFac7V+N; arc=fail smtp.client-ip=40.107.93.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="uFac7V+N" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RUVV+4MhU/Tk13iyQQb7LSk6eOyGMmonSswYE03x8cn/17DESgKvoALVIiiQrTcYvmLMgouXp7BXzOJuYi+d6NmMjfh17BTmNcqvmnTZ9p40Zd2BgK8vOKxgUtBt9LaRKB450jYmC9OapXWsG2kGOr6zIE4pvIY1z+l+jLHqz4jH2GZmvzxqPaHWA4XmUKZxGr45XcjtSCuzTfAqBVgZT0KD0C3RgeQJRzGbP8JXdt8Gsjl5STcvtyN/2m8x1L9zIvSfZjXLq7Iv4WTJin3qLsnJnbzM31fkcTwsr9ggAomW+sfhDZWcvv4plwmj+I6F0HpMgEy0dapLT9uZbGmovw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pVtoOEcp5KRdOrLEHoxVb2q71bT6xZkcV470l7qosps=; b=U0i7pM1S482V1ebEt/EFlSm556dusv3bYxE20MdzUF7D4qMbLd3rXyqZfTLItJ/IEX2HTHZ9TJvHI8CLcU4iM/9E7TgoYpqN0loZOfPlyMPmkI11UjDoODNhynAKGFhl2cU4nbN+YkbEmx0jCeI/j+dSYGkMSAFBWIKH8i108tr+jgxsq2tIsisMuvjCI0ksRzJ0qFc6zq+eQnO0HMiXTw9UIrO1L8jLBGwihIQtwQrS4lGFKmQhzygLrdgqV8dloMIBlROQ8Mrfzue04GocgeAJzeWqPmAypafVYMAiREFZ7QKBqPk35ZFkce0E3bgR113GsVyO+TUftSy3guf32w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pVtoOEcp5KRdOrLEHoxVb2q71bT6xZkcV470l7qosps=; b=uFac7V+N+x+WLQyWjhrvMHhwTMhu5xrx6eRLj8L49fGWkQYOg7X7BddpzHvh2DPqqTb4J3zoVlYUwLm545GDUaVkbge6x6Ygy/5zFfWDzpOFWlRII93rebg0Ga3rFkP+1aHzSSZE8i7zViJFIPQcCKimn+Fc9TTnZ6OINkz07A4= Received: from BYAPR04CA0019.namprd04.prod.outlook.com (2603:10b6:a03:40::32) by PH7PR12MB7796.namprd12.prod.outlook.com (2603:10b6:510:275::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30; Mon, 17 Jun 2024 20:05:11 +0000 Received: from CO1PEPF000066E6.namprd05.prod.outlook.com (2603:10b6:a03:40:cafe::55) by BYAPR04CA0019.outlook.office365.com (2603:10b6:a03:40::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30 via Frontend Transport; Mon, 17 Jun 2024 20:05:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000066E6.mail.protection.outlook.com (10.167.249.4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7677.15 via Frontend Transport; Mon, 17 Jun 2024 20:05:10 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 17 Jun 2024 15:05:07 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , Subject: [RFC PATCH 4/9] cxl/pci: Map CXL PCIe ports' RAS registers Date: Mon, 17 Jun 2024 15:04:06 -0500 Message-ID: <20240617200411.1426554-5-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617200411.1426554-1-terry.bowman@amd.com> References: <20240617200411.1426554-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000066E6:EE_|PH7PR12MB7796:EE_ X-MS-Office365-Filtering-Correlation-Id: 59206555-cac9-4393-9a4c-08dc8f08caff X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|36860700010|376011|7416011|82310400023|1800799021|921017; X-Microsoft-Antispam-Message-Info: zHyIAmcRJdst9+yZQ9ZEZGxHcWtbE7ZCHvWQqkvmExFvitvJ6YmpKXRVUfJ9tGryRUU9zVIdDyuIIk1XzlPC+k/rCHhTnEhhAijQZkBebqifXbm1EY1kfHqOGyr5ww1wz2kEw+Zb4Xac3fd8VwlN9vv4iVAEXnUmvy3eEBHelRAfhzrNxMZ9KTjzwEhO0pDdZ+To18vR55P9SNbY5adUCbvPPwWjAsVgdWMCTXcxZM5zxrHLmD3LuXtpRK/1V7Ntw5LDPEA8z96KsPGLszAcpAXx+PnNgtQpDb+dD0bzYcVHSUwXx+IyONbQeEWX/2nAIpRjfbatVFTeF3zBSmWeGtFpHoJMGDa4YW6aS7Vi+1d3vwIufHyPYJwYuHjVDTxMuKLJesnnqr57E9mLVT3sxEWKr0EMaCXCs6e19HUzao5UdnKYycFgHZEiP8Ghgzx4jJTTN1dkgUhp87HAH1IqgMbSzqJfTFlrwGNnG3a1KFoW4Hq6me/e1wcMnU0J5lb6rTqzdiwugsYtcT3vyS4Tzlbb4v+MHEhyVSWYYQTCOAvm6pIxopxcHfmKVTCqvLez7MYJKA0dI1CPO/2J64uIZllcAYxZInVcKJf47dmqJW/dt5zQ7AYkzPFyZkBUGftHiIFZLEGL1ZCVUkfQkgouN8otPPrplsvV2Yomw/6ayNon+PYhed/jXT/0lkoFZ3T5HufNnB4C1SFs+nQjDAj0nvfNsPKzkUjrkaAbfD+gsCYFzm1lz3DlGRrzJcqnKIl8dV9F0rr5V4sPRKhQ4EsEJuzkgMpNvN1OdiQfaI3poPhwCq5DYKKunCQrvgZzC2ROFaR4+spBrQzANwF7Ux2fMuVvL7Rt0jUg7gbMTIBOoclWAKt0fIvdUFvDKKOzphEXJ/y+jva/rnnWt2kxK29oF2KfqI3bkRBX1WlSA7OllRjqF5rQITSHl5ozxz7rlcb7Z3BXkViN/ydllFIxdC2HjXlYhXyt5xSKezc7iQ/SQOLDHdNKywaXIy1efdTTwBFTxvLb1LAlAJOGRv3k2tkrFpa9Ms04wdhHnro73u72xd+7aFIniPabcmEYQecc1NUPiOUgvWFHFkMFoT9Fwf4EjTE5rnbzyKFTe6IY/CyMyPVYLt6VQBzr747A6rIQdgZfMfiZDPT6gGFGMZUR+LEatiN1TnNsN+nRbn/oc2clHsnqo4bdgHpMVE8di7/uJ+9NJ34QulZs0zdGXXY6op7canm1qcCGdIWbSQyAZb43vw3tCJXnpkykv3lTRDRkteAUWkVgDycH6LgustRZ8rJp/QvowDDm2BvEU7zOUGGPZOY6uPJ/HQC3oPEwVEICq0i0HEYFeGQplWweIOsQd+Tm9dtPXuEXiIseileAo7e0JZiB8NxjW4TuaNKK+VeIkxRDzt43c48gCiVkCs0+ekX7VA95wwz1Cuh9i+9G1YbvRfg= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230037)(36860700010)(376011)(7416011)(82310400023)(1800799021)(921017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2024 20:05:10.6620 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 59206555-cac9-4393-9a4c-08dc8f08caff X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000066E6.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7796 RAS registers are not currently mapped for CXL root ports, CXL downstream switch ports, and CXL upstream switch ports. Update the driver to map the ports' RAS registers in preparation for RAS logging and handling to be added in the future. Add a 'struct cxl_regs' variable to 'struct cxl_port'. This will be used to store a pointer to the upstream port's mapped RAS registers. Invoke the RAS mapping logic from the CXL memory device probe routine after the endpoint is added. This ensures the ports have completed training and the RAS registers are present in CXL.cachemem. Refactor the cxl_dport_map_regs() function to support mapping the CXL PCIe ports. Also, check for previously mapped registers in the topology including CXL switch. Endpoints under a CXL switch share a CXL root port and will be iterated for each endpoint. Only map once. Signed-off-by: Terry Bowman --- drivers/cxl/core/pci.c | 30 +++++++++++++++++++++++++----- drivers/cxl/cxl.h | 5 +++++ drivers/cxl/mem.c | 32 ++++++++++++++++++++++++++++++-- 3 files changed, 60 insertions(+), 7 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 0df09bd79408..e6c91b3dfccf 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -787,16 +787,26 @@ static void cxl_dport_map_rch_aer(struct cxl_dport *dport) dport->regs.dport_aer = dport_aer; } -static void cxl_dport_map_regs(struct cxl_dport *dport) +static void cxl_port_map_regs(struct device *dev, + struct cxl_register_map *map, + struct cxl_regs *regs) { - struct cxl_register_map *map = &dport->reg_map; - struct device *dev = dport->dport_dev; - if (!map->component_map.ras.valid) dev_dbg(dev, "RAS registers not found\n"); - else if (cxl_map_component_regs(map, &dport->regs.component, + else if (regs->ras) + dev_dbg(dev, "RAS registers already initialized\n"); + else if (cxl_map_component_regs(map, ®s->component, BIT(CXL_CM_CAP_CAP_ID_RAS))) dev_dbg(dev, "Failed to map RAS capability.\n"); +} + +static void cxl_dport_map_regs(struct cxl_dport *dport) +{ + struct cxl_register_map *map = &dport->reg_map; + struct cxl_regs *regs = &dport->regs; + struct device *dev = dport->dport_dev; + + cxl_port_map_regs(dev, map, regs); if (dport->rch) cxl_dport_map_rch_aer(dport); @@ -831,6 +841,16 @@ static void cxl_disable_rch_root_ints(struct cxl_dport *dport) } } +void cxl_setup_parent_uport(struct device *host, struct cxl_port *port) +{ + struct cxl_register_map *map = &port->reg_map; + struct cxl_regs *regs = &port->regs; + struct device *uport_dev = port->uport_dev; + + cxl_port_map_regs(uport_dev, map, regs); +} +EXPORT_SYMBOL_NS_GPL(cxl_setup_parent_uport, CXL); + void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport) { struct device *dport_dev = dport->dport_dev; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 036d17db68e0..7cee678fdb75 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -587,6 +587,7 @@ struct cxl_dax_region { * @parent_dport: dport that points to this port in the parent * @decoder_ida: allocator for decoder ids * @reg_map: component and ras register mapping parameters + * @regs: mapped component registers * @nr_dports: number of entries in @dports * @hdm_end: track last allocated HDM decoder instance for allocation ordering * @commit_end: cursor to track highest committed decoder for commit ordering @@ -607,6 +608,7 @@ struct cxl_port { struct cxl_dport *parent_dport; struct ida decoder_ida; struct cxl_register_map reg_map; + struct cxl_regs regs; int nr_dports; int hdm_end; int commit_end; @@ -757,9 +759,12 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port, #ifdef CONFIG_PCIEAER_CXL void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport); +void cxl_setup_parent_uport(struct device *host, struct cxl_port *port); #else static inline void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport) { } +static inline void cxl_setup_parent_uport(struct device *host, + struct cxl_port *port) { } #endif struct cxl_decoder *to_cxl_decoder(struct device *dev); diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 0c79d9ce877c..51a4641fc9a6 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -45,10 +45,39 @@ static int cxl_mem_dpa_show(struct seq_file *file, void *data) return 0; } +static bool cxl_dev_is_pci_type(struct device *dev, u32 pcie_type) +{ + struct pci_dev *pdev; + + if (!dev_is_pci(dev)) + return false; + + pdev = to_pci_dev(dev); + if (pci_pcie_type(pdev) != pcie_type) + return false; + + return pci_find_dvsec_capability(pdev, PCI_DVSEC_VENDOR_ID_CXL, + CXL_DVSEC_REG_LOCATOR); +} + +static void cxl_setup_ep_parent_ports(struct cxl_ep *ep, struct device *host) +{ + struct cxl_dport *dport = ep->dport; + + if (cxl_dev_is_pci_type(dport->dport_dev, PCI_EXP_TYPE_DOWNSTREAM) || + cxl_dev_is_pci_type(dport->dport_dev, PCI_EXP_TYPE_ROOT_PORT)) + cxl_setup_parent_dport(host, ep->dport); + + if (cxl_dev_is_pci_type(dport->port->uport_dev, PCI_EXP_TYPE_UPSTREAM)) + cxl_setup_parent_uport(host, ep->dport->port); +} + static int devm_cxl_add_endpoint(struct device *host, struct cxl_memdev *cxlmd, struct cxl_dport *parent_dport) { struct cxl_port *parent_port = parent_dport->port; + struct cxl_dev_state *cxlds = cxlmd->cxlds; + struct pci_dev *pdev = to_pci_dev(cxlds->dev); struct cxl_port *endpoint, *iter, *down; int rc; @@ -62,6 +91,7 @@ static int devm_cxl_add_endpoint(struct device *host, struct cxl_memdev *cxlmd, ep = cxl_ep_load(iter, cxlmd); ep->next = down; + cxl_setup_ep_parent_ports(ep, &pdev->dev); } /* Note: endpoint port component registers are derived from @cxlds */ @@ -157,8 +187,6 @@ static int cxl_mem_probe(struct device *dev) else endpoint_parent = &parent_port->dev; - cxl_setup_parent_dport(dev, dport); - device_lock(endpoint_parent); if (!endpoint_parent->driver) { dev_err(dev, "CXL port topology %s not enabled\n", From patchwork Mon Jun 17 20:04:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13701349 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2076.outbound.protection.outlook.com [40.107.237.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6FC01E542; Mon, 17 Jun 2024 20:05:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.76 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654726; cv=fail; b=V7fvOXUiWxNDLJQB01bWGFWDbAMNOqFwK74a2mM0UJ1IFo2Vgha8nV9k1NVfUQKeLIUY3wLQyN1LcGmV2kYz3tCQwmNrCICzyNkwrvIJ0oan6QndqhLlaspQhz+kfaNVeqgvRbBuMjIwhJTNn7sDc6JdgGAjqews4sfKpaVWhbM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654726; c=relaxed/simple; bh=CyIysawuifkYpDi6xy6GdfbMN7O++SqYG6o5mN44n6o=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=uEa6MlUaznO44ZNLjIWPPwCCjJ/YUwgPn9d2YMM8WHmlhPOTsSi/Hit1MP6fzah0OzhV5xKZbYpEH3gmpAGcAF3UPgPFUjvhRaeqqivf4UqyItmsfBzAt5LyHPyvEChOv5vOFvw6N/44Agw6OS7SHmCokelWG15jnRjMs2OFxO0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=klCr5T+Z; arc=fail smtp.client-ip=40.107.237.76 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="klCr5T+Z" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WqcM6spnx2UOvuKzfvPFeRGrJgxD0imo3ktsyd9FYTd1kQYw7Qz+48hvWKm1S5PCkCSVnMOGi6lsujPb/Wm4t3GcycoYQ2WEN/VTg1MMmDfC1asGuPfkPxl/EoEYD0Fbor5PBQ7PHHqD8veZHNA7rx7m915vt5EnG7JH14SjuuAVHgrZcfh27EWon9lyExFS6JLuV+lvXk75SqyJlE99s1SSJKhhoa2fRRGzfuEEmUDC5jJb5tpQLWeRLhhM2+/1KSI5HbsMqETVXDYJONPIu4QzHwypxzfE0YVAiPREl4l2z45htaxwmlxt/680yVJ1/Qqojnj/CCxyndvztv5u+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xBXM8oohfM7lGqcdJlam74a7SPVH+VPxP7kMXhf228I=; b=Sl+pYCwOr+o+Qb6PQVb0Ymxl0RB7OBavzorOZ+VJXCMtK601Evwh43HnuD+KVhQ4U/aAWGMrpl7fIUdS3jm5dw0znltCJWmhCqAOSiTlfbsEX+KhzUyP1VNuVbtTfe9/QZoKy2U+4LiBzC7IArGsihtyAtP/LmVEMxxpV5AsxZQ4/QVHe33HKj+BR8B5j9jSobgSMh4roaEzAp1uzFA/tBPT4lJQbDjJvh98J59OLHb6VVv/4mDvoVgFbF+X/hNjAR+a/5QGXqM0aWmWl+/t82VBf3Q63NdJg9PVBITcaXY7Y/AG8CBh5EOydPRZyiiU55X0e3S4KbVkM0Ubq9+SWQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xBXM8oohfM7lGqcdJlam74a7SPVH+VPxP7kMXhf228I=; b=klCr5T+ZniXYRXsTCmzKTIjPQSE/beB5FyK6smzCMO4qwqMXVwbni+eXuZpQdnQictszg8h8gRe5oaf1dy+AS/vDXSElCXZgqY8fPi33xJuodBT/XKoftBliV/S51afQAFX5sP9jDsWSGZdencD+gAjJ3g87OUsd+Ug0XNyHiFY= Received: from BYAPR04CA0030.namprd04.prod.outlook.com (2603:10b6:a03:40::43) by MN0PR12MB5787.namprd12.prod.outlook.com (2603:10b6:208:376::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30; Mon, 17 Jun 2024 20:05:21 +0000 Received: from CO1PEPF000066E6.namprd05.prod.outlook.com (2603:10b6:a03:40:cafe::50) by BYAPR04CA0030.outlook.office365.com (2603:10b6:a03:40::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.31 via Frontend Transport; Mon, 17 Jun 2024 20:05:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000066E6.mail.protection.outlook.com (10.167.249.4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7677.15 via Frontend Transport; Mon, 17 Jun 2024 20:05:21 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 17 Jun 2024 15:05:18 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , Subject: [RFC PATCH 5/9] cxl/pci: Update RAS handler interfaces to support CXL PCIe ports Date: Mon, 17 Jun 2024 15:04:07 -0500 Message-ID: <20240617200411.1426554-6-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617200411.1426554-1-terry.bowman@amd.com> References: <20240617200411.1426554-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000066E6:EE_|MN0PR12MB5787:EE_ X-MS-Office365-Filtering-Correlation-Id: 01df0e58-23b7-416e-29ce-08dc8f08d144 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|36860700010|7416011|376011|82310400023|1800799021|921017; X-Microsoft-Antispam-Message-Info: IbKvEtC0pHNxShxFgcp3D//9P+WLxElhL2kqLrbWw45q83PlbOl0cVir48O2YDppasMV1+eNBIv0pQedjL8uCYpZHfcEF491vRMLp3jYWr5c8HgEdJcvipTqBVz+JYVQGV/j1gY6pjuVXKjVUXJzb6xSmkH0SWh10OU3gyWPkmfVKISisZywcY7z48Ttqm8pOD6azLK0WPHen4mlPURV0Ku+wBZQndROAVOma2rLaDAoBtGklq0masF6Z/Md6KR0GMEtA5aW6c7EVkuq5zMiDkuAt81jbihmoWeOz9NIgs7Y9X4VB5cr5o2/KQ65SpRIrkVAgASnYs8OILslo/BHiHLv1efo+OK0FCSApE+3fzoMVrUaS7hX1AvQFb5jxXiGI+/LsZeJPwoFbq55VcTQTYToxzEnBMsbpBT0swU/SsMR45QuU2F5xW0a5oIdRy+u6YfPE+7EkacozAKbzg5QZFJks4eRlEzzDO9Q4UVUFvta3JWcfzoIPsHHi5woQhnORFXOfWAt4UEHOUun/cI+K/lrqMDUsiriVozfMuC6Nnh4HkGBZ5l8xDdEyjBC+AYyYrZcTjRuYx1RaDlVpiaFWyjWo12T1hMDk0S3x3iOdo9454XxWkWjR8DvH6akhwyzgRFmC9KDdMUnPUqMIMKAH+Ox3Dv32RRz282mZJv9AGb3eBhfyWBbyb6YU5l5beyBiMi5XxEU9L56y8Y2qEysY5GEdR4ealSp4L+DskAkw9XxrMwQm7Ky92d7K43Qisydo2HNcFCK0Bq1XacGupbajm/on16JZ0RyWShVuFBjIc1eyAfNIn31/AWjmEq7jNhf9hYHQQPfcqsDl4NhlWV2DFRE46BRq0egAoQaym9I15G3TUV6dHp4kZ6gwfma1iYVB6BBs/DYjBSw7bZD0QD4lYUYILPX910XOut7+qR/r8Z1o3m0hxF1iAeb0QHXHyNwduxuPqAIZ8rUc5+xgjADXHG6x9LN/fFAS7mFwOJFHL97qf3REknl6vZOhuMfN8O66m34gchF8njP6RjGlTwqkjB57Ex6tFzHMblpC3FRsD9mo5qXRPMPjGgxe2AXgG6DqAPXkSvTMJH84zIMCEX9Sjchpb4Ecgc8hXItK43tGaAlmxpOTk8OcsVl/XV43Xo23yYjQBkwM1canS4leKo32Ap3H4IHumqap/M5xYT/dc/c+d0+qsd4+IZr9tSniAxmrWMDLhRWLKQ2KlPZdZrcjoDwDyf/tTsemb4X/WmSwy0OOUynMg2JwHA4B5nZcFMT4NB0dPG95z+5roNemdMwGdONMMwkp+N1GJszFGPAHSce5SSv0tVldIqTpO5fKIoQQ/n6ZxeLkrUmh/pKlbfVdXv6rkkOiPNjsm/xfAP4HAYCeW30imYjP0zT8zFO2D0hv9BJn1WCXxy9ffqWhnnjUkm8U0cyUXVJD8kYvPkMJlQ= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230037)(36860700010)(7416011)(376011)(82310400023)(1800799021)(921017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2024 20:05:21.1464 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 01df0e58-23b7-416e-29ce-08dc8f08d144 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000066E6.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB5787 CXL RAS error handling includes support for endpoints and RCH downstream ports. The same support is missing for CXL root ports, CXL downstream switch ports, and CXL upstream switch ports. This patch is in preparation for adding CXL ports' RAS handling. The cxl_pci driver's RAS support functions use the 'struct cxl_dev_state' type parameter that is not available in CXL port devices. The same CXL RAS capability structure is required for most CXL components/devices and should have common handling where possible.[1] Update __cxl_handle_cor_ras() and __cxl_handle_ras() to use 'struct device' instead of 'struct cxl_dev_state'. Add function call to translate device to CXL device state where needed. [1] CXL3.1 - 8.2.4 CXL.cache and CXL.mem Registers Signed-off-by: Terry Bowman Reviewed-by: Jonathan Cameron --- drivers/cxl/core/pci.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index e6c91b3dfccf..59a317ab84bb 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -686,9 +686,10 @@ void read_cdat_data(struct cxl_port *port) } EXPORT_SYMBOL_NS_GPL(read_cdat_data, CXL); -static void __cxl_handle_cor_ras(struct cxl_dev_state *cxlds, +static void __cxl_handle_cor_ras(struct device *dev, void __iomem *ras_base) { + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); void __iomem *addr; u32 status; @@ -699,13 +700,13 @@ static void __cxl_handle_cor_ras(struct cxl_dev_state *cxlds, status = readl(addr); if (status & CXL_RAS_CORRECTABLE_STATUS_MASK) { writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); - trace_cxl_aer_correctable_error(cxlds->cxlmd, status); + trace_cxl_aer_correctable_error(cxlmd, status); } } static void cxl_handle_endpoint_cor_ras(struct cxl_dev_state *cxlds) { - return __cxl_handle_cor_ras(cxlds, cxlds->regs.ras); + return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, cxlds->regs.ras); } /* CXL spec rev3.0 8.2.4.16.1 */ @@ -729,9 +730,10 @@ static void header_log_copy(void __iomem *ras_base, u32 *log) * Log the state of the RAS status registers and prepare them to log the * next error status. Return 1 if reset needed. */ -static bool __cxl_handle_ras(struct cxl_dev_state *cxlds, - void __iomem *ras_base) +static bool __cxl_handle_ras(struct device *dev, + void __iomem *ras_base) { + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); u32 hl[CXL_HEADERLOG_SIZE_U32]; void __iomem *addr; u32 status; @@ -757,7 +759,7 @@ static bool __cxl_handle_ras(struct cxl_dev_state *cxlds, } header_log_copy(ras_base, hl); - trace_cxl_aer_uncorrectable_error(cxlds->cxlmd, status, fe, hl); + trace_cxl_aer_uncorrectable_error(cxlmd, status, fe, hl); writel(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK, addr); return true; @@ -765,7 +767,7 @@ static bool __cxl_handle_ras(struct cxl_dev_state *cxlds, static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) { - return __cxl_handle_ras(cxlds, cxlds->regs.ras); + return __cxl_handle_ras(&cxlds->cxlmd->dev, cxlds->regs.ras); } #ifdef CONFIG_PCIEAER_CXL @@ -871,13 +873,13 @@ EXPORT_SYMBOL_NS_GPL(cxl_setup_parent_dport, CXL); static void cxl_handle_rdport_cor_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { - return __cxl_handle_cor_ras(cxlds, dport->regs.ras); + return __cxl_handle_cor_ras(&cxlds->cxlmd->dev, dport->regs.ras); } static bool cxl_handle_rdport_ras(struct cxl_dev_state *cxlds, struct cxl_dport *dport) { - return __cxl_handle_ras(cxlds, dport->regs.ras); + return __cxl_handle_ras(&cxlds->cxlmd->dev, dport->regs.ras); } /* From patchwork Mon Jun 17 20:04:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13701350 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2072.outbound.protection.outlook.com [40.107.212.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F5C51991B4; Mon, 17 Jun 2024 20:05:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.212.72 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654741; cv=fail; b=R/XFJW9J42gb8g4x2KQNEllG6X+vttRePEkRxp3RrwLniO98DRcIiAqyXgwhJEQhYR+m9QNLPvdVVhBiqHL6NNOxAN+Tt82b8N7ZIVJ6lZB5g8ibiIHWbwPX+mDVyeXiRMWaRoBy6JtnVd9vEvrMZ+bRZPtZMX+hYtETjD9BXH0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654741; c=relaxed/simple; bh=YhwI4ehx/hIFI3OEBGt82LbwNZCVpIwpbCBc8WMcFuI=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Kd7rqaNFLFg+aymEEi3tqHUQYhPTElH7fvg+ZvhEoPsJQgqzWO3fzClN8WLTEMBWwHc4m7LcW30TGTru/saQQ5/KHnCl5AqsOuAMiStUCzK/dmASgFnttJM5myPOeMRB3960VpVRgLoJXyTnmdqb4VGd1iHGFdYQqy6hsLbOeSU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=G5qqNolS; arc=fail smtp.client-ip=40.107.212.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="G5qqNolS" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mwnrNTqig2ZQm8yO9YF/cGMevVKe+tzzireNMB72CsmnQvCKcZ92sl1yKmHcZWvleDEUspsOGY5PaNuwqojNvyasupazosMVE4pcc30bveF8gJ4+2ZrAssqaQW0DK4y4G7ju7A2sGH6S/eSqcweq/76cN1T5slU73Eq93lLRrof9HmlkDhgaETB+/f428RFIyuFS9Qbl+c06zy+0It6j/cowxJlBXhzDW1sKZcMW9lCBsgp0pBJ5UsArd2CxegAUPRpNl8KzMXg2PbvwjPTVvPMW++0O3zySnsn/j6DN4P1j2+833+0SWmptNYEqMiCKoyMfR3LLmaldXG/odqwmBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=shfodHw6jfZM88Zzznv+XtjFFw271NPr29HmNje4DcM=; b=VW59GajNtlW4H0pB7agVXxcqzVq6EBFXpE5fXxC22Q0N16/UbuFQzhm569HUgKhbdDOiJWSYRqlCz5xXzhDG+4rbesurfe/MxSrhSYW4O3jdUgzW+/YfCwQBR5gC/IEKcG9UUvtdFzbgb0gQ9dFBK7FsWu2vWZfGl2iwNoPVWV39m552qm+YYoC9l86y6DK6RFYomMK/EM35Jimzit2bPL33X5SDaVsdn2WHLZrO/Gu3K7mjvORpzsiI9ZJrgHURyZkmiLwQ/eEmYgBIZPuyAIi3olcpbY08Xqz5mn1SgyORfcmc81mYaqVkloxckHSydq7IA94R4gX7XmpSQYD/4g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=shfodHw6jfZM88Zzznv+XtjFFw271NPr29HmNje4DcM=; b=G5qqNolSeJm7qIjeDlOd1dwO7ca9i8yY9pbDFcy067n0f4LleBqmoUKN1rpxyTuHEbHMdoT14/fG2rFuEsICZ465o4n8Qx+btdSBsfy6ZaYqj9KS9ryzT1QtTckiga6SSuYwYk4UOQhgtqt2OnCXaG3E0w012ZhHm5YL3fmg/fE= Received: from BYAPR03CA0015.namprd03.prod.outlook.com (2603:10b6:a02:a8::28) by CH3PR12MB9316.namprd12.prod.outlook.com (2603:10b6:610:1ce::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30; Mon, 17 Jun 2024 20:05:32 +0000 Received: from CO1PEPF000066EC.namprd05.prod.outlook.com (2603:10b6:a02:a8:cafe::59) by BYAPR03CA0015.outlook.office365.com (2603:10b6:a02:a8::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.31 via Frontend Transport; Mon, 17 Jun 2024 20:05:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000066EC.mail.protection.outlook.com (10.167.249.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7677.15 via Frontend Transport; Mon, 17 Jun 2024 20:05:31 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 17 Jun 2024 15:05:29 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , Subject: [RFC PATCH 6/9] cxl/pci: Add trace logging for CXL PCIe port RAS errors Date: Mon, 17 Jun 2024 15:04:08 -0500 Message-ID: <20240617200411.1426554-7-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617200411.1426554-1-terry.bowman@amd.com> References: <20240617200411.1426554-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000066EC:EE_|CH3PR12MB9316:EE_ X-MS-Office365-Filtering-Correlation-Id: dffeb003-480b-4610-1455-08dc8f08d750 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|36860700010|376011|7416011|82310400023|1800799021|921017; X-Microsoft-Antispam-Message-Info: r2UYczTk3XNFDdBNxSyg55ALPKe+LISxe74koy9CCUcX7KudDPQ4I+zYeGtixRL+wBMiJ8ms0VutyBd7VzeW2yRKIsMV8kUIc1NptN+Z9iCB0ccFjjFOrM44DjS48wRbBeI4ESKbVPoZ6n+KTzlwZbbot+6tA3Y9r2/SVmkVagJvdvqNpbmakp8dedEnxAwLycspppMQdVWEjswitfDXwVK/G4BXMEZ7RzwsoriZ+n4ulC/4p9lOr2swDqHDJmob8e//Lm2xZ5i2/oV/U9HaR7/GUps2yfyt+Yqgl2E3XsGaq07XJCsZb5X2o2PB5jTsZKXlc3WAoA6auA7Nvn0l8asSEiXWcbprppuEuGKR6JQrPa06mXrRTMl09kstVsKXCEubXR7qp2zBbDDYpXBN716r0CWExUqaeyorwOYU6GYVBUUR4jQ6T7jHTC7hnXAJJipej06v+T7OH8432P2lpB+EM7fc2IpUH5ue3DBgMRBg0DL0aHV1c/jAZe3MDMJicTJvzxrV0GyH8G2BHoSYRDIeiSENvUwSW+BpzxopjRS9S+XLZkYKibZrO3wg0O6syQfDX68bfsKarGHDP7tc84YxQgb7RHxF7zs9C7HkqSniLoMODdTvNUp2Qx+YH6USgmnIX2us4V6qLbBmfV/9uqFaXCFn3+fnRXX8IBofNjVzcgKMyPVL1W/xNi+MwiBzuMGavtA/5iml01jc5SLdwFXafJbAjueEHTUDd6hBnvNB7S6BsASUoEffhLa/n99lajTSzDSslX483OboGWiQ+6dxnMyb6JG+4C0McS8wLBH7xFOg2myrY2HrAk85KrRDlqjbagKuQFJsACTNq42GRAprVw8bm/4VfrRTLqaITahI1blevlcmCGPSEwc4Ih1Cpeoc1mVA5wuc1m3isEpY0dhMZUyOyDB8BWoKjvKfvpH3rqlRTxqgFz2XXddDsBsU/MsGTCdyY+k2s2jTVPHezySlO6r3oWWf2ft9Z8zsUps3m/wDps+ttL8U4emgWvetfINic59cyKluIDQ33kUEllC6ESlv4YbI69JL6fcp5QLj+gzZbP0SgcobDrCx5aCcJcuNl2h98gHbqfZdJHvxZUI7xL3PgQPR1tvdabqLXwj67hZ0ONfaBUYyeuAuHO9zKwx24sjn2JpE54/blD9CwKDFDIFSRlJIaSkg0UebiINwhfkYadit+aoKAEFlXKjQ1FUAzhu5VtrNZzXdQexeMKnfHYbz9lIgtpuDZVfjxUJPcVV29DfmpKcP/2cnZ5CIiOyxKlIHrSVb9p2tFDTxtLz5toV60/kGWe44q7VZDh6hQmwkmJrsSmJ7Mf0ECW8+YZPN2xlUH34oLU3RBoy+rftvXbgJTnwCBrre3SRCKur16xanKwU0WCjrPIQH0Z2f9sTZx7Zbvg2pRBmvDp/lhbqcm0S3r0QpGb5kmQa0wz4= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230037)(36860700010)(376011)(7416011)(82310400023)(1800799021)(921017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2024 20:05:31.3065 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dffeb003-480b-4610-1455-08dc8f08d750 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000066EC.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9316 The cxl_pci driver uses kernel trace functions to log RAS errors for endpoints and RCH downstream ports. The same is needed for CXL root ports, CXL downstream switch ports, and CXL upstream switch ports. Add RAS correctable and RAS uncorrectable trace logging functions for CXL PCIE ports. Signed-off-by: Terry Bowman --- drivers/cxl/core/trace.h | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/drivers/cxl/core/trace.h b/drivers/cxl/core/trace.h index e5f13260fc52..5cfd9952d88a 100644 --- a/drivers/cxl/core/trace.h +++ b/drivers/cxl/core/trace.h @@ -48,6 +48,23 @@ { CXL_RAS_UC_IDE_RX_ERR, "IDE Rx Error" } \ ) +TRACE_EVENT(cxl_port_aer_uncorrectable_error, + TP_PROTO(struct device *dev, u32 status), + TP_ARGS(dev, status), + TP_STRUCT__entry( + __string(devname, dev_name(dev)) + __field(u32, status) + ), + TP_fast_assign( + __assign_str(devname, dev_name(dev)); + __entry->status = status; + ), + TP_printk("device=%s status='%s'", + __get_str(devname), + show_uc_errs(__entry->status) + ) +); + TRACE_EVENT(cxl_aer_uncorrectable_error, TP_PROTO(const struct cxl_memdev *cxlmd, u32 status, u32 fe, u32 *hl), TP_ARGS(cxlmd, status, fe, hl), @@ -96,6 +113,23 @@ TRACE_EVENT(cxl_aer_uncorrectable_error, { CXL_RAS_CE_PHYS_LAYER_ERR, "Received Error From Physical Layer" } \ ) +TRACE_EVENT(cxl_port_aer_correctable_error, + TP_PROTO(struct device *dev, u32 status), + TP_ARGS(dev, status), + TP_STRUCT__entry( + __string(devname, dev_name(dev)) + __field(u32, status) + ), + TP_fast_assign( + __assign_str(devname, dev_name(dev)); + __entry->status = status; + ), + TP_printk("device=%s status='%s'", + __get_str(devname), + show_ce_errs(__entry->status) + ) +); + TRACE_EVENT(cxl_aer_correctable_error, TP_PROTO(const struct cxl_memdev *cxlmd, u32 status), TP_ARGS(cxlmd, status), From patchwork Mon Jun 17 20:04:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13701351 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2056.outbound.protection.outlook.com [40.107.243.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4A3C1990AB; Mon, 17 Jun 2024 20:05:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.56 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654748; cv=fail; b=aKBFkEbUN0sQ0WdUzu1zxVZt2+CF7VsNfGn9NqzuMkKnbPRk+0oVos79eLAxMKuzCornxKrGPB0CMvxmHlHbMXnFA4N83+gI9naLUE08Oc29Wnss6ImcyCGSBnPStgdXdQI9pDJV55Yl/WjGjNx2lubEqXPmml60DyAaEKzGTME= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654748; c=relaxed/simple; bh=AQznya0ccgoWLuT4eXFlv4qzKbvV87kJv4Achwny0Vg=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=EfB6o4sI/nXzoFlMfSHkSEazkmIFVVdJM4NEMAPlNDtZj2nyY0bOux/gCOmEWqr9QX3z3rqYC5wn3kvJOkCcdFDG1ZkkAcGaQdkN5+bETsYTsBgBY5sph0rNR4TkzIbESs1wffc6nVnNZbU4syueHyfPQouBT77n7oQN14k7GDg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=iu7CfVu5; arc=fail smtp.client-ip=40.107.243.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="iu7CfVu5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fQ0aeVh9KMCmJuInjI+7ctRakgIAtAY9rVx0LlcTIle63fCjm1VvJ8XSj+rVakGfcc46NdMDc5ZCsofCfC644zbBDxCqWznlS6IcTLT+mcx1hRB7Ph1ls1pGBt9b5O2rNLY5k+LzQZDlJHU/mxHG5kXOG5X7HEo/A1jt76isw4UbYwjqv+SDaQEXTkqf7JlNGGW/s5OFr6Yx6p0de+EXTpTOObM4+7Dk+VbuTgqP1qcFQ07ni2C+2J0FR+HMyTONcSKl9/zF0+mSBLSJu6OpLMiAJAe7HW9UKGKbEAEf9IC9G0MtMmIzFJRQbUi5xQiBkBlKMsHM16SjJjQXuLf5+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8HvJ/jRE1dq/SEfTEIhVL/EFl9NGIKorrAXaRQWcz0I=; b=Jb4NmccgKQZ6DstWxkeWF1dGLFqBycM1iTiZH1papRifDegVVZaDPkB77WxNM/65MxEwq033h+j3h6S6pYauDpE/uAnnYqAl/mIW7nZS9v4AN5e5u0Qcs8sNFKIhGjzr95TSpS9Y6sbttfcuzSnvz128ON55ZTl3Z4+lUyKGWIc6HEwrF9nurAF1J5OEmg/jn0SaFb7YZEJ/EQUAoAVal3VbLngozKFaR5ohb4JhpCFsNzKDkc1RArWnWRhxDCuRmIwBTz/ZKJ/rwmGCrm3COz6JKkR70yyWtP3Y3Of4O8o0jkaW7Yq5T5EDv1aFjCNbDXrbPf9LicGhgjcxpTfimw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8HvJ/jRE1dq/SEfTEIhVL/EFl9NGIKorrAXaRQWcz0I=; b=iu7CfVu5ag8wCGeeNmEB0ble4T3mnuHTGzv7IeW/4To2Am6iZLRaycqjr0zauSYOQsMQ0IXGBT5vJUUl1PWguuYSZCZZYsnw3RhSPxpzP0FDqnTiuTdxHXoAv3O4KCFAIvilPeS2t5pUltqyECBuoEvM0vz6kDlJUgMoocV0YWo= Received: from SJ0PR05CA0068.namprd05.prod.outlook.com (2603:10b6:a03:332::13) by IA1PR12MB6234.namprd12.prod.outlook.com (2603:10b6:208:3e6::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30; Mon, 17 Jun 2024 20:05:43 +0000 Received: from CO1PEPF000066ED.namprd05.prod.outlook.com (2603:10b6:a03:332:cafe::2f) by SJ0PR05CA0068.outlook.office365.com (2603:10b6:a03:332::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.26 via Frontend Transport; Mon, 17 Jun 2024 20:05:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000066ED.mail.protection.outlook.com (10.167.249.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7677.15 via Frontend Transport; Mon, 17 Jun 2024 20:05:41 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 17 Jun 2024 15:05:40 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , Subject: [RFC PATCH 7/9] cxl/pci: Add atomic notifier callback for CXL PCIe port AER internal errors Date: Mon, 17 Jun 2024 15:04:09 -0500 Message-ID: <20240617200411.1426554-8-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617200411.1426554-1-terry.bowman@amd.com> References: <20240617200411.1426554-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000066ED:EE_|IA1PR12MB6234:EE_ X-MS-Office365-Filtering-Correlation-Id: 4e47d74f-2619-452a-e377-08dc8f08dd9b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|36860700010|7416011|376011|82310400023|1800799021|921017; X-Microsoft-Antispam-Message-Info: 0SfnFi0MNvlF6WXy6A4u+nGNnCQsKzo7LSRINx/eD8geWVycy1MTsaaoJyyBkCnAx41xFipe5fYlY9fzKVzNOXkigL62chO5ctkqZt4WQKal2+D9NHKeGviCqxnDx99U+1Tl/saRFaEhP3CBEtUd587tnQcl463zC7+V3+isKxnyrd2ZmqcXnbtRRmjw/p69ABtWpDaG+BTpDXXlbDbAorCVNHJMbL/lfgNxxvlCpMVAiyGivseNKjTHz6Ijsfv0/wCP0W9x6MUPI8+iAlVSXzNXEQJ16nlyQ0Pberv9ZzE0ZvMl+f+7If3JihexWZ79i8zK94AXKofVFRsWEP+E/G2ch/qh8QAOlLh1mGrsYPbOopmt2Kkwq1R+JURu/bfkuyq9WhkWV2ugrBgW6kZHB2MfjtQG8aBtcOTFvpElUOQh9yEnwbQIo7pM0NkwMRo8IYJYa1s9jXZjjH+IBgBD3yy4Ks/i81SebmlWswl8jOyRbJ8hhrSdrNARI6aJipIpydImZb4uzKMaHIGhE6kytBZTDDkonUrigUo6cug7NSvun/8//t2odCELVOmhaAhhTNaWlg/aadjePfK9MQ0Tg4e+BfYbzVRWiNeR+ZEJxVq2zhK+dyw1IX9atcxNv6E6V9OH3QfxjxaBY2eNTiPExbjTVcLhjhW7TmOtoJUxjXM7LMfjJ+JMz+t0jysvjaC7I0NXpMxTwjLuD6a1BbfEpwU65YNeabwqbPDgTrlZxs/Vf713D4mcdFAifOKHoZ1cisD0Ef5g5d9MAgY59T6Q2k8FTJ4r8wLrr/tSXH8/5tvmn5GjCXpFMXH8pesdZzQ+ft0Zgw0Um5QOCr3n1uXAUyw6qlrsr1ILLF9GpIEA2jUJUa2ZTTh3bFRCLTvO5KMXXP5lXXI4QtWMeyLG586iwZGIgs2rPQjLiygyxkN9/FjxNDGETVIvmqMhnv2l1T5inh3tKGayXrn5jOaxwnlNcHSA0VbGZQMZhTB778al3BlaOIHlKmxdCqYqZItIdQiJtFrdHnrzGKVrCLr0Kq22n4XoTlbqwiEbZ20kEIDaPgBeuHUeficyYd3zoEICoi7HqkGFMR3yGjpOS8esPCeLrCZjC6KiV7me689oRdJO3J2sHmhTF1aHCDGQ8a1zG74NpAXlDuCDgyKZbkG1riei4S5Sg4ofvW7+/NDJqE/wk6VeO9UJmLRIl0veAM8jnqpY9UqW16zA8FUm8xAHSu/BwKialeR3MzQ34AQkvfFwJL0ep+PHbVyG1Y43rL2vH/6p2TGKGEGHowusIvVO0VZs1aHtnDbU1uVXHsFoxj94ba7N8rmq3Yca5ZdF7JCJiTXQvuOnAFFO1IdR2hSfqz87GrcSv3XBCMnlvZQ19uB7WgXwFmh5A3R6+JlSFFj++EGrD74iGmrwdzZSVPtR4npVJyefob5WFYpwZq39SskO9EI= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230037)(36860700010)(7416011)(376011)(82310400023)(1800799021)(921017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2024 20:05:41.8229 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4e47d74f-2619-452a-e377-08dc8f08dd9b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000066ED.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6234 CXL root ports, CXL downstream switch ports, and CXL upstream switch ports are bound to the PCIe port bus driver, portdrv. portdrv provides an atomic notifier chain for reporting PCIe port device AER correctable internal errors (CIE) and AER uncorrectable internal errors (UIE). CXL PCIe port devices use AER CIE/UIE to report CXL RAS.[1] Add a cxl_pci atomic notification callback for handling the portdrv's AER UIE/CIE notifications. Register the atomic notification callback in the cxl_pci module's load. Unregister the callback in the cxl_pci driver's unload. Implement the callback to check if the device parameter is a valid CXL PCIe port. If it is valid then make the notification callback call __cxl_handle_cor_ras() or __cxl_handle_ras() depending on the AER type. [1] CXL3.1 - 12.2.2 CXL Root Ports, Downstream Switch Ports, and Upstream Switch Ports Signed-off-by: Terry Bowman --- drivers/cxl/core/core.h | 4 ++ drivers/cxl/core/pci.c | 97 ++++++++++++++++++++++++++++++++++++++--- drivers/cxl/core/port.c | 6 +-- drivers/cxl/cxl.h | 5 +++ drivers/cxl/cxlpci.h | 2 + drivers/cxl/pci.c | 19 +++++++- 6 files changed, 123 insertions(+), 10 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index bc5a95665aa0..69bef1db6ee0 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -94,4 +94,8 @@ int cxl_update_hmat_access_coordinates(int nid, struct cxl_region *cxlr, enum access_coordinate_class access); bool cxl_need_node_perf_attrs_update(int nid); +struct cxl_dport *find_dport(struct cxl_port *port, int id); +struct cxl_port *find_cxl_port(struct device *dport_dev, + struct cxl_dport **dport); + #endif /* __CXL_CORE_H__ */ diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 59a317ab84bb..e630eccb733d 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -689,7 +689,6 @@ EXPORT_SYMBOL_NS_GPL(read_cdat_data, CXL); static void __cxl_handle_cor_ras(struct device *dev, void __iomem *ras_base) { - struct cxl_memdev *cxlmd = to_cxl_memdev(dev); void __iomem *addr; u32 status; @@ -698,10 +697,17 @@ static void __cxl_handle_cor_ras(struct device *dev, addr = ras_base + CXL_RAS_CORRECTABLE_STATUS_OFFSET; status = readl(addr); - if (status & CXL_RAS_CORRECTABLE_STATUS_MASK) { - writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); + + if (!(status & CXL_RAS_CORRECTABLE_STATUS_MASK)) + return; + + writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); + if (is_cxl_memdev(dev)) { + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + trace_cxl_aer_correctable_error(cxlmd, status); - } + } else if (dev_is_pci(dev)) + trace_cxl_port_aer_correctable_error(dev, status); } static void cxl_handle_endpoint_cor_ras(struct cxl_dev_state *cxlds) @@ -733,7 +739,6 @@ static void header_log_copy(void __iomem *ras_base, u32 *log) static bool __cxl_handle_ras(struct device *dev, void __iomem *ras_base) { - struct cxl_memdev *cxlmd = to_cxl_memdev(dev); u32 hl[CXL_HEADERLOG_SIZE_U32]; void __iomem *addr; u32 status; @@ -759,7 +764,13 @@ static bool __cxl_handle_ras(struct device *dev, } header_log_copy(ras_base, hl); - trace_cxl_aer_uncorrectable_error(cxlmd, status, fe, hl); + if (is_cxl_memdev(dev)) { + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + + trace_cxl_aer_uncorrectable_error(cxlmd, status, fe, hl); + } else if (dev_is_pci(dev)) + trace_cxl_port_aer_uncorrectable_error(dev, status); + writel(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK, addr); return true; @@ -882,6 +893,80 @@ static bool cxl_handle_rdport_ras(struct cxl_dev_state *cxlds, return __cxl_handle_ras(&cxlds->cxlmd->dev, dport->regs.ras); } +static int match_uport(struct device *dev, void *data) +{ + struct device *uport_dev = (struct device *)data; + struct cxl_port *port; + + if (!is_cxl_port(dev)) + return 0; + + port = to_cxl_port(dev); + + return (port->uport_dev == uport_dev); +} + +static struct cxl_port *pci_to_cxl_uport(struct pci_dev *pdev) +{ + struct cxl_dport *dport; + struct device *port_dev; + struct cxl_port *port; + + port = find_cxl_port(pdev->dev.parent, &dport); + if (!port) + return NULL; + put_device(&port->dev); + + port_dev = device_find_child(&port->dev, &pdev->dev, match_uport); + if (!port_dev) + return NULL; + put_device(port_dev); + + port = to_cxl_port(port_dev); + + return port; +} + +static void __iomem *cxl_pci_port_ras(struct pci_dev *pdev) +{ + void __iomem *ras_base = NULL; + + if ((pci_pcie_type(pdev) == PCI_EXP_TYPE_ROOT_PORT) || + (pci_pcie_type(pdev) == PCI_EXP_TYPE_DOWNSTREAM)) { + struct cxl_dport *dport; + + find_cxl_port(&pdev->dev, &dport); + ras_base = dport ? dport->regs.ras : NULL; + } else if (pci_pcie_type(pdev) == PCI_EXP_TYPE_UPSTREAM) { + struct cxl_port *port = pci_to_cxl_uport(pdev); + + ras_base = port ? port->regs.ras : NULL; + } + + return ras_base; +} + +int port_internal_err_cb(struct notifier_block *unused, + unsigned long event, void *ptr) +{ + struct pci_dev *pdev = (struct pci_dev *)ptr; + void __iomem *ras_base; + + if (!pdev) + return 0; + + if (event == AER_CORRECTABLE) { + ras_base = cxl_pci_port_ras(pdev); + __cxl_handle_cor_ras(&pdev->dev, ras_base); + } else if ((event == AER_FATAL) || (event == AER_NONFATAL)) { + ras_base = cxl_pci_port_ras(pdev); + __cxl_handle_ras(&pdev->dev, ras_base); + } + + return 0; +} +EXPORT_SYMBOL_NS_GPL(port_internal_err_cb, CXL); + /* * Copy the AER capability registers using 32 bit read accesses. * This is necessary because RCRB AER capability is MMIO mapped. Clear the diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 887ed6e358fb..d0f95c965ab4 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1027,7 +1027,7 @@ void put_cxl_root(struct cxl_root *cxl_root) } EXPORT_SYMBOL_NS_GPL(put_cxl_root, CXL); -static struct cxl_dport *find_dport(struct cxl_port *port, int id) +struct cxl_dport *find_dport(struct cxl_port *port, int id) { struct cxl_dport *dport; unsigned long index; @@ -1336,8 +1336,8 @@ static struct cxl_port *__find_cxl_port(struct cxl_find_port_ctx *ctx) return NULL; } -static struct cxl_port *find_cxl_port(struct device *dport_dev, - struct cxl_dport **dport) +struct cxl_port *find_cxl_port(struct device *dport_dev, + struct cxl_dport **dport) { struct cxl_find_port_ctx ctx = { .dport_dev = dport_dev, diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 7cee678fdb75..04725344393b 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -11,6 +11,7 @@ #include #include #include +#include "../pci/pcie/portdrv.h" /** * DOC: cxl objects @@ -760,11 +761,15 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port, #ifdef CONFIG_PCIEAER_CXL void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport); void cxl_setup_parent_uport(struct device *host, struct cxl_port *port); +int port_internal_err_cb(struct notifier_block *unused, + unsigned long event, void *ptr); #else static inline void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport) { } static inline void cxl_setup_parent_uport(struct device *host, struct cxl_port *port) { } +static inline int port_internal_err_cb(struct notifier_block *unused, + unsigned long event, void *ptr) { return 0; } #endif struct cxl_decoder *to_cxl_decoder(struct device *dev); diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h index 93992a1c8eec..6044955e1c48 100644 --- a/drivers/cxl/cxlpci.h +++ b/drivers/cxl/cxlpci.h @@ -130,4 +130,6 @@ void read_cdat_data(struct cxl_port *port); void cxl_cor_error_detected(struct pci_dev *pdev); pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, pci_channel_state_t state); +int port_err_nb_cb(struct notifier_block *unused, + unsigned long event, void *ptr); #endif /* __CXL_PCI_H__ */ diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 2ff361e756d6..f4183c5aea38 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -926,6 +926,10 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) return rc; } +struct notifier_block port_internal_err_nb = { + .notifier_call = port_internal_err_cb, +}; + static const struct pci_device_id cxl_mem_pci_tbl[] = { /* PCI class code for CXL.mem Type-3 Devices */ { PCI_DEVICE_CLASS((PCI_CLASS_MEMORY_CXL << 8 | CXL_MEMORY_PROGIF), ~0)}, @@ -974,6 +978,19 @@ static struct pci_driver cxl_pci_driver = { }, }; -module_pci_driver(cxl_pci_driver); +static int __init cxl_pci_init(void) +{ + atomic_notifier_chain_register(&portdrv_aer_internal_err_chain, &port_internal_err_nb); + return pci_register_driver(&cxl_pci_driver); +} +module_init(cxl_pci_init); + +static void __exit cxl_pci_exit(void) +{ + atomic_notifier_chain_unregister(&portdrv_aer_internal_err_chain, &port_internal_err_nb); + pci_unregister_driver(&cxl_pci_driver); +} +module_exit(cxl_pci_exit); + MODULE_LICENSE("GPL v2"); MODULE_IMPORT_NS(CXL); From patchwork Mon Jun 17 20:04:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13701352 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2045.outbound.protection.outlook.com [40.107.94.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E21518FDCD; Mon, 17 Jun 2024 20:05:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.45 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654758; cv=fail; b=Tu9LkcP+JnGhKVavJsk8chBOTN5XLhpvyXrlqcMYT69A+SCkhWRmDBthNGVQaZWwcxZPXMkqa5icXZAqfrBEbxPcovexoI89tnpd7J4JZzJga6Nm+Z9qqG7A0oycwqK/980depeMpjVy+SRFhYcEYJF51UmWWfGQsTNlT5cujYo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654758; c=relaxed/simple; bh=m2+M7Uq08F7HibZgF5W8wxOp5E1Evv8hjSsaJ7FLKlk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=STsU/R52ky1s8Lp8xAo0TMLGzvp2Q66o1GMLLdmYnvYg6hHJ2KI1XIzt7b5pSYURacx2hQFUmDmOk8JScB18a9PEpZ5H1klVTSG7myHGop2cyDaokFAyhLoqx1QAxalH5iSigzabRDwKccobGPE0Bna+E3X2b+rpUFQYSI6/RnU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=osNUnLv2; arc=fail smtp.client-ip=40.107.94.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="osNUnLv2" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KBS7bAxeYHjWN+6W9HeY41r16L/oX2cYqmmYNqJraTtMlwm/glKGTfP0wHEsuaAMEowHmuVRhNxtMikg9ZyvoDG1yEPnIzOy7RYIyXEBHU5gopXBNehgNyBdrJnFhcrmX3VGew6KFaSFg+5wF8hrWruQdaa8dicmWni62zS/RO/C/hcbCcAcqnMxaAtXA83xNdetaCMPigOg8E0yN3XCrDYuBSQZPdqhq7TOt7xBVJ9V6wlBKy2DeGVHyt+rJhTKn2eBf8pCdKiDyEOo+VcQeMLtAh+mv2bYC/SgoZEBFsE+foH4t9TEEUSED828IkD1/2BD5WR9l9hw3n94sm4bng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RRQup9KGzBjZ10NoITfMp0a+Us+E7VpZ8nL1w13Kflw=; b=AAR1X5W2bjapKEEmjFO10EtBOl09e0ZvSILR55f5naxMivPtKHeJwaK3hK848OLBez6qQM/bNwv3bdFBG4kk0IdhpYwl4Cy10xX3+wF6DgZHr2HoA0GzUa+r4Y8gIrviVXRTxWbl5hVXh4w8s4fjuS2eOA+JstEE7I8JlUVt8G6jLHT2VmTvgBmZKZPwCbFCOI91o/R4fb5WnaZYpz3ZvjgzR3dBbz1o2SaFMmSxzx4hrpOsYBm+yk3Vg+OjvgtfSXtYtwQ3GJVVbouusn94zjtaAuB2osQFtp9e/jcd/BHFBlzua1wHJfPI3wNIdrS73M8O90hQqoRZKemtQWRjZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RRQup9KGzBjZ10NoITfMp0a+Us+E7VpZ8nL1w13Kflw=; b=osNUnLv2kNhLDx9eAlAJxaJ+sIIct5mfJqzjd7JBXWzOF+rHDlm3J3i7DlJu6JBARzvVz98X9G9/8u7nx9oNGaBlsbmyiUInQXr/DxNx5HaT8ryppPuyYZqdEPBkU+cFy0BKaVoGkzhFLKU8CjJf+gFu/X4bjPWio34LMDsgZAs= Received: from SJ0PR05CA0074.namprd05.prod.outlook.com (2603:10b6:a03:332::19) by SA3PR12MB7878.namprd12.prod.outlook.com (2603:10b6:806:31e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30; Mon, 17 Jun 2024 20:05:53 +0000 Received: from CO1PEPF000066ED.namprd05.prod.outlook.com (2603:10b6:a03:332:cafe::a8) by SJ0PR05CA0074.outlook.office365.com (2603:10b6:a03:332::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30 via Frontend Transport; Mon, 17 Jun 2024 20:05:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000066ED.mail.protection.outlook.com (10.167.249.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7677.15 via Frontend Transport; Mon, 17 Jun 2024 20:05:53 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 17 Jun 2024 15:05:52 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , CC: Bjorn Helgaas , Subject: [RFC PATCH 8/9] PCI/AER: Export pci_aer_unmask_internal_errors() Date: Mon, 17 Jun 2024 15:04:10 -0500 Message-ID: <20240617200411.1426554-9-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617200411.1426554-1-terry.bowman@amd.com> References: <20240617200411.1426554-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000066ED:EE_|SA3PR12MB7878:EE_ X-MS-Office365-Filtering-Correlation-Id: 3fb1fedb-f156-4aca-6fc1-08dc8f08e47b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|36860700010|376011|7416011|1800799021|82310400023|921017; X-Microsoft-Antispam-Message-Info: OOaGlpVEo2NB2sHV/PUp8BhSUYlYHFI3i9RvhG/mipjYHrpQUeCpZttkoxYbttZSPIIk9eTouGxMcPuE1BaANfSmwzlEfCcL2GDMM3AGeLIinLFsaedm0LMYaCVx2kZLTFfwRG396RM3xvGxSgVRn2V2owgheuf2Kxw7kSbeC4oPLOCTju/I5GZK5IBC4RBp4hV9yXRzZT8SE3KCy9c6UW7t72D2xa92Ii9oxPMDDWjKbdq7hDjyLxIpMaPYiKfB0kRv/sLiMASPoYWHzz2Ww9qoREocaAE5b9HSp39pd8tnon6ERpY5PpqH7pYbbi8VqiugQ7HEHpvVjsjjgu0Hvjg+QAq0lZhYDz7FL+/Lbi4WdKyCHO807vDUuBxIYYBQfCZMvmL+WRKB2ilVVpNsIo/wSrFHz5v/5/TBecDPU9uGJl2pr6mLCjRsRBghbKy2kASWb/9+plI+rGwWkI6yGL6julQQbkI7e9MTAAs8eYetX0c9cc64KCo5YA9RH3alJyXBNFAD3qn8zzncA5MDDuT0Ptd9U/ecMiV4jEf5Nq6erYG66v5549hj69zvHhAlPH5dNwD2acnx2i9sBig/jD+sHI1TDBkk8GNIKC5Mv6hYJS8d27l4P+UmiUFio/fDAwxHwxGRFdT5jBQFA6yR2uSn0Dn+CNPUSix6R2RGnh4Q14vthiBTUi+zvbw5Pd0/Rs3eU3KMkg7WVVfdI6mhw6T5H0yMLy8xlYW4qoc65Wcvzk76M1FwvcsF1HhzNpv3moBfvjhXEI2nsQlwOOp0QpfLgGd+mI+P8GpS2/+omCY1gQYTDI/sF7J4W1BMH99KHHr7Xi0JKB5m4pnxziJCuKjcdp75alfKwRXvnFQXwDd0JQH3dxtJucdn4bBwxIoRoZVXv8w0oXN8G4MlwoxIMFihiIKwHM3rFqZEqenOXu8O4uZry3wTQXjTEwdOE0yIr+aLYLhIw+zb8odxGL7hqZmtuvHVh/cML2wTDEl1HOpxbpI7J/9oQyzafz0cFQk22cSSzl62JFk9gGTGo6CeWk9P7AUMF8IGRfJT5V3ZQodo/XG+9E5+rEC1rQXOh07TLnQ3KuHvHFJVscguoKeNPSHXel3uukYkPWjEkO8qv6P7fSO6w4BUZzOz6EivzBLAOwsDm2vAbWW4H6/DVjvTeyer/9mK0r46wqh1DPjiWT2rEEu5aNHTknoJJVIYYol0Li7oUrSKLOBFDw7axlHLO8nZXbKIB7GOkEInctuAm08zLbC+0jeEYBVS8ZojoyhpbW+1HshmMZZgX4ExzaeGypP5GDV7mFr/drG2KLtGn1th1IetOGzuzwUjHAhE7lj1vA7T/4YUdE7atE1BAQeXvTjjIWCKGhiPx5HGkSUBwKD/kjKFbm+G/GRuZCwrv30tU/2NmZ05JFI1PM0xMAhauEMjADe06AHM63/vm732PtU= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230037)(36860700010)(376011)(7416011)(1800799021)(82310400023)(921017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2024 20:05:53.3854 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3fb1fedb-f156-4aca-6fc1-08dc8f08e47b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000066ED.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB7878 AER correctable internal errors (CIE) and AER uncorrectable internal errors (UIE) are disabled through the AER mask register by default.[1] CXL PCIe ports use the CIE/UIE to report RAS errors and as a result need CIE/UIE enabled.[2] Change pci_aer_unmask_internal_errors() function to be exported for the CXL driver and other drivers to use. [1] PCI6.0 - 7.8.4.3 Uncorrectable [2] CXL3.1 - 12.2.2 CXL Root Ports, Downstream Switch Ports, and Upstream Switch Ports Signed-off-by: Terry Bowman Cc: Bjorn Helgaas Cc: linux-pci@vger.kernel.org --- drivers/pci/pcie/aer.c | 3 ++- include/linux/aer.h | 6 ++++++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 4dc03cb9aff0..d7a1982f0c50 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -951,7 +951,7 @@ static bool find_source_device(struct pci_dev *parent, * Note: AER must be enabled and supported by the device which must be * checked in advance, e.g. with pcie_aer_is_native(). */ -static void pci_aer_unmask_internal_errors(struct pci_dev *dev) +void pci_aer_unmask_internal_errors(struct pci_dev *dev) { int aer = dev->aer_cap; u32 mask; @@ -964,6 +964,7 @@ static void pci_aer_unmask_internal_errors(struct pci_dev *dev) mask &= ~PCI_ERR_COR_INTERNAL; pci_write_config_dword(dev, aer + PCI_ERR_COR_MASK, mask); } +EXPORT_SYMBOL_GPL(pci_aer_unmask_internal_errors); static bool is_cxl_mem_dev(struct pci_dev *dev) { diff --git a/include/linux/aer.h b/include/linux/aer.h index 4b97f38f3fcf..a4fd25ea0280 100644 --- a/include/linux/aer.h +++ b/include/linux/aer.h @@ -50,6 +50,12 @@ static inline int pci_aer_clear_nonfatal_status(struct pci_dev *dev) static inline int pcie_aer_is_native(struct pci_dev *dev) { return 0; } #endif +#ifdef CONFIG_PCIEAER_CXL +void pci_aer_unmask_internal_errors(struct pci_dev *dev); +#else +static inline void pci_aer_unmask_internal_errors(struct pci_dev *dev) { } +#endif + void pci_print_aer(struct pci_dev *dev, int aer_severity, struct aer_capability_regs *aer); int cper_severity_to_aer(int cper_severity); From patchwork Mon Jun 17 20:04:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13701353 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2068.outbound.protection.outlook.com [40.107.243.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7216018FDCD; Mon, 17 Jun 2024 20:06:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.68 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654768; cv=fail; b=CmT6V+Eww3U6mVxR9jOhvYtd1kfamd7HeupXPOoBxUII1PwSur1bXe+NK7YPLKENM8RR8ePOfU1KzJHishslQ0ZOoK0WGtQ6/QfXLvX60qVVUh9QcuncBLDJIURwm3FTlH+2phtbjJBDZ8RvX0PqZcjquov3NRNlnxdjbQ7wy7w= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718654768; c=relaxed/simple; bh=6FdwMlb9PTpqhBN7ZRCboi+YKCb8NjmX/vTY2UROd78=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=I6lVR6rdVSn+j2S9LWMpOTIqhOlF/PppbE9zaz5QDOjJm0/DA5m7BZ4sMnMStema2hngx3dLXhLtlwZ92wp5CZJCCyTd3xHqnibMa6cAJzf4XJBSvp9Yk1HNd6COJoPH2J0yYOIgrcfihD/G4czPHRqdxcR1+s3IbKdf6vqnZVE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=MmUlPfVX; arc=fail smtp.client-ip=40.107.243.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="MmUlPfVX" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Rh/TLs5wBTBPqKVjhnE6fjJOr5iCu8I+O/7CIarXvtJTH+9prQqdj2HR7o9bUNNaqd1gGcGUQbenBlQiNKcm9tiXRP/8O92IdrtL8aDzK2cDfzvNxXMRZzJUKA3NBwiBEXzkV3R7vuNLB0uKMVQ3m17ZlYGf0qB8wbcwt3eUbqkJgEyFuBr/OcbhxzTj7MgtLae9DYjdLEdlJU7MK/5nw5mjjwpaBpMo0FMAoZEf0no22N39d6aHauEMHQnBGpO1eqJJ/usYWiRfPHhn8Cv2zbdehEfocwAlut9LtTClhMHRGVU3CJ4rv9qhVoiTy5hDUBshFJTqhbmCsdD8zraKlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tfjI9rw9dxmXO1fk15wQcsHTJ74Ow2Vwn+HbohIAklY=; b=EC5hGO2YhpCb3mJPvvoqcXgiiBkZzwNchIf/QosP9tRvXWtTbrbHgPfubjrPoGuQKbGpvm/ZKzvlioC2u6E/PDGHyYjp81XvSZ/q1ENXic4PgRsTdSX/lNSzDUS9Tv+5jGhYT56LtCnKRWbALKVDZgxYFB7Oq4Im3zayR3MIioL7F70HmR3PiIulahMc0wATb55BUpfTP6YUg3q2YtBPK2e2LOcROurmNhgoSjldswpdYM/hRYedZCANbd6154uf7memi3OuEdjxmt4sTSXqbYA/HcuFb+/3Mf8LfZWu+TLg//C5Mm/osHU5vfLOIod27OVwZAnSaNPd6IIqz8dxRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tfjI9rw9dxmXO1fk15wQcsHTJ74Ow2Vwn+HbohIAklY=; b=MmUlPfVXINzfmp6SBmii9RZLgJoxMtsVzR5VIX6fJWJKu9nGGw+8YfqGOgCW0rZXFeSuSX65GzTBZNMq+EOGW1dxSQeF/DLrAqc1I6DmUaSncjdB2IjBkfmrzB96Ag/I0sMaOMNrvpCFgdxcDP7umx9tHGsX+7cbAiH1EeZ4LVE= Received: from BYAPR05CA0039.namprd05.prod.outlook.com (2603:10b6:a03:74::16) by DS0PR12MB8453.namprd12.prod.outlook.com (2603:10b6:8:157::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.31; Mon, 17 Jun 2024 20:06:04 +0000 Received: from CO1PEPF000066ED.namprd05.prod.outlook.com (2603:10b6:a03:74:cafe::8d) by BYAPR05CA0039.outlook.office365.com (2603:10b6:a03:74::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.31 via Frontend Transport; Mon, 17 Jun 2024 20:06:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000066ED.mail.protection.outlook.com (10.167.249.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7677.15 via Frontend Transport; Mon, 17 Jun 2024 20:06:04 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 17 Jun 2024 15:06:03 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , Subject: [RFC PATCH 9/9] cxl/pci: Enable interrupts for CXL PCIe ports' AER internal errors Date: Mon, 17 Jun 2024 15:04:11 -0500 Message-ID: <20240617200411.1426554-10-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240617200411.1426554-1-terry.bowman@amd.com> References: <20240617200411.1426554-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000066ED:EE_|DS0PR12MB8453:EE_ X-MS-Office365-Filtering-Correlation-Id: bf1ae36e-3097-488e-9f0f-08dc8f08eadc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|7416011|376011|36860700010|1800799021|82310400023|921017; X-Microsoft-Antispam-Message-Info: U8iCPKh2nNwoqVLMzpirLYCl+Q+TeRDT5HURUMePdoG3wj4hIWWfVQyJTZEbz6Ae/uRzj/V6ItMv9TfCJY1ZaW/wHHrrXsllpcQ19bJWYTAJQ/qx62xrPshosA0iMXvQ49nzb49iW/kOga6XRT+fYSW/E9S/NZtsTxBjI+id9pEyHBKhYRXom5NN/ZlHTrGN9lmmvKZPysTReImFH+tjzPAmhRmUOTlzFmbW2gtsESXxZhiVr3pA5tBQWCeVfW+1BjajgNASIiV+7dG4RSPxz8fgAe8TlqiK+TfSM7RaULUUSnuuU5u5yPrc3QAB0LngeeX7W9SIWvfoJzbpQJqkZUSJXW8AuyD89Q6Zla+3aDptiLo5DsyG0iPdLO+RYHheCNUWaOo1Y9Dl0hFVQYnJJ0RAzqMxOzAol+jxdSJP8Tu7H79qTIJcXanjD4ov1MO1uuYfhBuy/a22yeVLP12mtey5gPhH9oOMsNZJxjEZ2Ly8uqKjkEA+rQInX1eaxi5umlsibf1MvNVFQBUMB0Ga/jNIs7mOkgGHVBdLlGZKrMbFMn6RGQDfDT6OW5/SCdLgi/vymOTgvW6r+xR3BXBx5iZzTzEoL0ed0Q67fpaWezk/aV2cTwdQELRBVioD6DGzJ0HVOgU4mOjAzJRnFrtq7qlezsKfCH6BLdbK7PrZ5vm61unCKArcOp8MDaxiFwBVhAwHR2cp1giz0fuZdmMqy9PInrKLfLWRfaflLyKG6dc65EiENmAG8JzpoqRQzHrz4L8tz1OXKmupkSZCzNQas4QBswv61QllHL5KSJ85/+jNjwTG2IakU7nt+RGk65sQSEro1C1tDzaBapaIdds3G92UiOlSVjB4dGCaykwP7phP3gUH6IlbManN4ij+WcnwYMlZZFdDI2oANX0Wycc7Na28tTGZBDDUTJkqjhq1nOFcoiKBuKhqyyr8LsXt8Ec5oKrpUjecMPjqO1ayfj60fFZ4zFewsyS0OtDtAlTxI7vzSLWPtMWSDGfUjhy36KKVZ7pdlqRp33GNn/5JAHFzLHIfA3CA3yp4pkWzC6Up87VJ9tpqyOJVdchxp6cJ8UPbE189pKspO35kHZlaQzI3Swdqa6mMBqoocI1ObDaXLFLiqDjj8oU0X+hiVtXlEd2WfDLfXHDKl0GIfCv9O09GqPoweLwKifDEOVPHxsCQn/qn2zCC7gCMy9yDruKjbyPcjvPvB0dxk+LdReCN1ZIXmZNffXTO7AgwyijUvjL2KTyzZUGsoik/TU6LBF/F0qRbkhyEfdW2rDhRjfhVAR06CwkSe7UeBy0pNQQJEH+bhSKt07DI7POYJLmaqlkEvhC0Vx3AMnU9Sb3+X9k/69ReDarWA3s0j6x8pIes0dqTnutvqgLtfdhO4+/vDgUQVyIVPWJM6sb3JyYlmYk4S7EwwINXvJ6elrB54qdh23dJjus= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230037)(7416011)(376011)(36860700010)(1800799021)(82310400023)(921017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2024 20:06:04.0886 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bf1ae36e-3097-488e-9f0f-08dc8f08eadc X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000066ED.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8453 CXL RAS errors are reported through AER interrupts using the AER status: correctbale internal errors (CIE) and AER uncorrectable internal errors (UIE).[1] But, the AER CIE/UIE are disabled by default preventing notification of CXL RAS errors.[2] Enable CXL PCIe port RAS notification by unmasking the ports' AER CIE and UIE errors. [1] CXL3.1 - 12.2.2 CXL Root Ports, Downstream Switch Ports, and Upstream Switch Ports [2] PCI6.0 - 7.8.4.3 Uncorrectable Error Mask Register (Offset 08h), 7.8.4.6 Correctable Error Mask Register (Offset 14h) Signed-off-by: Terry Bowman --- drivers/cxl/core/pci.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index e630eccb733d..73637d39df0a 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -861,6 +861,12 @@ void cxl_setup_parent_uport(struct device *host, struct cxl_port *port) struct device *uport_dev = port->uport_dev; cxl_port_map_regs(uport_dev, map, regs); + + if (dev_is_pci(uport_dev)) { + struct pci_dev *pdev = to_pci_dev(uport_dev); + + pci_aer_unmask_internal_errors(pdev); + } } EXPORT_SYMBOL_NS_GPL(cxl_setup_parent_uport, CXL); @@ -878,6 +884,12 @@ void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport) if (dport->rch) cxl_disable_rch_root_ints(dport); + + if (dev_is_pci(dport_dev)) { + struct pci_dev *pdev = to_pci_dev(dport_dev); + + pci_aer_unmask_internal_errors(pdev); + } } EXPORT_SYMBOL_NS_GPL(cxl_setup_parent_dport, CXL);