From patchwork Fri Oct 21 18:56:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Bowman X-Patchwork-Id: 13015257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E656C433FE for ; Fri, 21 Oct 2022 18:56:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229489AbiJUS4i (ORCPT ); Fri, 21 Oct 2022 14:56:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229941AbiJUS4g (ORCPT ); Fri, 21 Oct 2022 14:56:36 -0400 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1anam02on2084.outbound.protection.outlook.com [40.107.96.84]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 950DB2764C1; Fri, 21 Oct 2022 11:56:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LxEkFB55DnzPo1BTK3SOT3X48EpnNeRgJuPq11fVO8JkLeVTzh2NCjBe/iGsdHPhYQxIsXrkklCuTfatLv+aufq7zIcLpl44d3FEGrNeOGveddtJWfzEFyEZOnvOHOBUzblFMRmUdJertkjMrB4SjpGBjx9SKel4FNflIeVdH87oXiiKLJCvkKVfRyENJ+pK1C4htw4Shv8eZNA8nMal5Fu2QrQyWCbeW5g6AIo605He2OjMLoF6jOMnoyemGFT6u6VjFdyVTZWgNqYsM3yu6PvfsYdTKCdBw67TULm+HaUxKgnGTr5hxji1MRr54fCff+//z4CKG0/IWan8uOqWPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NAtCwXB+tdj0y5nj+SlhChXbhHBEB+p+yA8uRqWcyKM=; b=cBKoZuO0dqQ9jwZ1bsq0UzJxk9ZafJdHhjE4tUP5FM39LMPg4owPFlMLjQKdE6aEV5TXnHz/t1OLZEbESYLIcTwXrt3h6Q9iI9KbyQUH1xfkzqVbDG5P63sJORNGw8km3YRidhjYEKqybhxOAvUkmZ6o5RA2WVcV9chtQnjAmrLlycQindMj01YfaPF5dD3OslV8AQqkrTperq1wXiUE7XSz0LY3930SaZisWo+FxxJauR4+vkaZofpOlxMAAiBr/FNfxKoJSht/0aRsRNguVpi7OIVfqbbYoWhumRYiND+DA2qyO1bRTopXttoXi9AllcgBIsxMdALdPR+E3zM4tg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NAtCwXB+tdj0y5nj+SlhChXbhHBEB+p+yA8uRqWcyKM=; b=zKDptJLcBqsQHKnf7Pvq6faX7bo1gcbN/F8/mGMiMHZzEabfRRD0A7xQV7jCEA31en28LjOiRtivqNiBG+RDB1saPXTKm2HDgVWNl8Xc85GXkS0vFxXjORUgq4mbgoTGq0WlQa7DZLfH0yolYrzGEChmrxJxYuLsGf4ZtY86c0M= Received: from BN9PR03CA0088.namprd03.prod.outlook.com (2603:10b6:408:fc::33) by CY5PR12MB6154.namprd12.prod.outlook.com (2603:10b6:930:26::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.34; Fri, 21 Oct 2022 18:56:33 +0000 Received: from BN8NAM11FT053.eop-nam11.prod.protection.outlook.com (2603:10b6:408:fc:cafe::8) by BN9PR03CA0088.outlook.office365.com (2603:10b6:408:fc::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.37 via Frontend Transport; Fri, 21 Oct 2022 18:56:33 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT053.mail.protection.outlook.com (10.13.177.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5746.16 via Frontend Transport; Fri, 21 Oct 2022 18:56:33 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 21 Oct 2022 13:56:31 -0500 From: Terry Bowman To: , , , , , CC: , , , , , , , , Subject: [PATCH 1/5] cxl/acpi: Set ACPI's CXL _OSC to indicate CXL1.1 support Date: Fri, 21 Oct 2022 13:56:11 -0500 Message-ID: <20221021185615.605233-2-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021185615.605233-1-terry.bowman@amd.com> References: <20221021185615.605233-1-terry.bowman@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT053:EE_|CY5PR12MB6154:EE_ X-MS-Office365-Filtering-Correlation-Id: a0832e09-4206-4961-6717-08dab395f8e9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: bwxvs1hFAGvKoEYyw4m23rVmbz24JgC72ZBXz/PYePqH7daEFCqvfwrSyns8y9zNSJjMAvKQJHNBcmEQgLiABvpWH8QermyUhRtNgYrOOouaTEg/AfImmWJwg1q4dEcqN0Rw7P7KcBRse1RMqSJ/w0QQYZhPSjweMAzt7+3SCWTRD6QcB3DINGKBBONYpeK4zj3WKS262Eca6rRFCEO5G27NM+AL1/uB8wYhrdiWegM//zZAMEgWqAFyXqLDh7TR+HGfQIZ/2GWMG9RjqQtVX5IKwGrHwZgC8AlSWKShzae5TeCog9ZornTx+E3IvppYU7IQOuTk7z0Sj6gkpri6ApAL3hkrGYKN/DTcd3breDcZGbTNaidO3zCosMRqovA/ztkYjw/Vs3UYbJby/J0lZnPL17ck97SezzAUfN2ciCF0CwX4UAaUSgQI7zOIArkDZ6tKEN/jqRvNgHkT/A9pvA/7m0/q8CIijIYWY9xjNJ1yRFfNoSFTOpdB28fSj3BdMygPLbfmKD2kV6B+ATxFqqrND8WH1NY4qDwz4MohK2qUH0MmUuDV0VFc6za+LPBsq/3gVHogpLoXCFadQpMTcjLQpVwgIcv5zl0/g1FoFlAQA2WPyvfnd28fAL+Awn+5EjIEKnDI30cIq8hyAmCSTHgdYlJWQOmKI5b04O8ZcD8q/HtxTAUwgW6FPgqTsPyOD34TiDpiU0brtTMkqJzOnywfOinpZS5dbnnIVXFpWdibHl7cCp79Las2Qb6F/mbK4962e+eetHn/T25rM7iKfQrMLNu/eJ7atD1Ejuza+Cuv9lyHMc6YpmWRzHEHvTsX X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(396003)(346002)(39860400002)(376002)(136003)(451199015)(46966006)(36840700001)(40470700004)(36756003)(8676002)(54906003)(36860700001)(110136005)(316002)(40480700001)(8936002)(7416002)(2906002)(2616005)(26005)(44832011)(426003)(47076005)(5660300002)(186003)(40460700003)(336012)(6666004)(70206006)(16526019)(70586007)(1076003)(4326008)(41300700001)(83380400001)(86362001)(7696005)(356005)(81166007)(82310400005)(82740400003)(478600001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2022 18:56:33.3578 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a0832e09-4206-4961-6717-08dab395f8e9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT053.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6154 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org ACPI includes a CXL _OSC support method to communicate the available CXL support to FW. The CXL support _OSC includes a field to indicate CXL1.1 RCH RCD support. The OS sets this bit to 1 if it supports access to RCD and RCH Port registers.[1] FW can potentially change it's operation depending on the _OSC support setting reported by the OS. The ACPI driver does not currently set the ACPI _OSC support to indicate CXL1.1 RCD RCH support. Change the capability reported to include CXL1.1. [1] CXL3.0 Table 9-26 'Interpretation of CXL _OSC Support Field' Signed-off-by: Terry Bowman --- drivers/acpi/pci_root.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c index c8385ef54c37..094a59b216ae 100644 --- a/drivers/acpi/pci_root.c +++ b/drivers/acpi/pci_root.c @@ -492,6 +492,7 @@ static u32 calculate_cxl_support(void) u32 support; support = OSC_CXL_2_0_PORT_DEV_REG_ACCESS_SUPPORT; + support |= OSC_CXL_1_1_PORT_REG_ACCESS_SUPPORT; if (pci_aer_available()) support |= OSC_CXL_PROTOCOL_ERR_REPORTING_SUPPORT; if (IS_ENABLED(CONFIG_HOTPLUG_PCI_PCIE)) From patchwork Fri Oct 21 18:56:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Bowman X-Patchwork-Id: 13015258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3533BC433FE for ; Fri, 21 Oct 2022 18:57:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229915AbiJUS46 (ORCPT ); Fri, 21 Oct 2022 14:56:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230018AbiJUS4v (ORCPT ); Fri, 21 Oct 2022 14:56:51 -0400 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1anam02on2042.outbound.protection.outlook.com [40.107.96.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 76DF628E06D; Fri, 21 Oct 2022 11:56:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JUFvp9ImtWMjH64DprJdPt+JwojGOr1uMiDek6bPgDfFAtdzDKNDlbAx6K6rDGAJ+VCeavTi9wU9dSSURr1zRBlnKIQjk8YzNeVsITCFwCZOzxyBy8ligmIBcothrbMrepz1+OlbwxEt1CV9auM1rHuHuMMSzi9Otvh+r92ALLpGOSZaTRseRKI7fWjC28OHcm2ZPJ9U8cntMwWON0AcO52NsK0hKtz0yAoEiViVoou5vQTOap6xEeeI2yRkxoDa4BualHfFIhWUhajkQ9Av6F/WAJzSLFlJQtyhNkrUy9H4KnskcRsaWOkHY7qWUSV7pqrqg+vHIqkx8mjbup5t+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=q+8ygDnWYxrVFXWLHgnB8BUyt5hlla6dTF1pKqeRd8g=; b=P65/D/ues+RtXgyywMTAy2LwY53aZLt0xy7xqbrUTeOx7NZeHYunypujIGFzL/64zZi9XBXYdTAeuLm7o3WndUuF7nVIhnz6SBDegXN5ZWA1HtXNnmniwyis7RtBmXAGMRwfFA2MoAJceXVI8oOWqY4La254+06l/RjEWtQBI0LNF7fGrCFqF2PEfGEM1VLytF9rFM5UCB4j4YB+ucgloECudg9hrn05CEktrwGnj4mfrltTZ/DOAQUlNfUpPztNoNWvDNBr6hjwcR2CjU/j+q0opW5ttrk7pT4iQZx3Lpg5Q2UMqIGv4NmcX8vmZMFQ5fNOhdcZxnVd0T8kMl8F/Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q+8ygDnWYxrVFXWLHgnB8BUyt5hlla6dTF1pKqeRd8g=; b=0DiHuCeE3kUfEzj4RFhrBXdwvDOUVjb41HySleLPSk6zuUp/azMJEhabBdgwUzETMDvSC0a4KPZzrO2Ds+P1HaYHohgVBZDwrUckhiJFVb0KZ6/RufJwIoG8ta6LEYjCIcz6AZnteO0HgA64ACyTv6Az1W5ZJe6n5iGBEkBmSEc= Received: from BN9PR03CA0964.namprd03.prod.outlook.com (2603:10b6:408:109::9) by CH3PR12MB7596.namprd12.prod.outlook.com (2603:10b6:610:14b::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.34; Fri, 21 Oct 2022 18:56:44 +0000 Received: from BN8NAM11FT105.eop-nam11.prod.protection.outlook.com (2603:10b6:408:109:cafe::6) by BN9PR03CA0964.outlook.office365.com (2603:10b6:408:109::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.21 via Frontend Transport; Fri, 21 Oct 2022 18:56:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT105.mail.protection.outlook.com (10.13.176.183) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5746.16 via Frontend Transport; Fri, 21 Oct 2022 18:56:43 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 21 Oct 2022 13:56:42 -0500 From: Terry Bowman To: , , , , , CC: , , , , , , , , Subject: [PATCH 2/5] cxl/pci: Discover and cache pointer to RCD dport's PCIe AER capability Date: Fri, 21 Oct 2022 13:56:12 -0500 Message-ID: <20221021185615.605233-3-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021185615.605233-1-terry.bowman@amd.com> References: <20221021185615.605233-1-terry.bowman@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT105:EE_|CH3PR12MB7596:EE_ X-MS-Office365-Filtering-Correlation-Id: d33bc205-64cd-4261-3b61-08dab395ff12 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +bNO+/Xx7t5gCcrmwTwkQFIIOdUdvjud+fTIvPSFt956x0cXtmnTexzqzcoNoJDuwpf+mryl6s4At7MTxFOCXYKgddZk9mFJVUbvuaTJWUjtklC05Hvb7Oh1tZSbnRP4OBr5FjkUhMrQKh6RsAmNHm16Jk4se/fSMI1zh/8ioo9nCo1lNU6BiyjFqgE94eHa/GWXK4rr91IlqEvaE9tGvapnNMbPIAVA2HVV8zstmt2hG+0TWG7SnOFQSZsFdV87OLamct+G8UxvJS3GrMEZUMhrD+5lYMULZwr9gNvPStvyrwtwvf0lgThoIPuyNTWu35B/W5mZu7wDuZenN1lMGqygWUti6HKG+1ZHawcJdcA8lTRpYha65w9UjxbbtIU5t3owsFoXEVCi7kQtHTxB/aO9P5GmgLVOxI1NaIC89W9F/NTlU4sI6x23oIfPWI9EQwaHHkB1KNyshv1uulRRQ3uk1+pfpmbcfcMsFyDgEZtRinm5a6OYKLV6Yl0ZPVYHxqYfMuF7GraT0Q9nzS5hjC0/fr3vBKQ8NDyuJK4berp+HQgzaa6FAdF9a2kr/OJWdrbFIaNrbJSjY5tuHNLubhfUR5jqhSgr+86Ud2ZnlBMsKTmQPoswCBbj9ofo2SZ4maHYt0jVMkm/GZm2s0TgFFo/WdeI3s0t/hf+XA868fk3VAyoVJ9MIUUofhl6CRA0UGPbZJ6m/oP8V3oI3qXk/Hph505V3xyXBYxoHLkhSMKhly5HWtyRg4IlPwXmOxxembiKu8tqnUc0rLPyaBEC+tpc+WX7rv+670De7bF3juoHLSu4mA+VU50wtUPkV3g3 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(136003)(376002)(346002)(39860400002)(396003)(451199015)(40470700004)(46966006)(36840700001)(41300700001)(83380400001)(44832011)(2906002)(40460700003)(316002)(1076003)(110136005)(16526019)(186003)(2616005)(54906003)(36860700001)(8936002)(36756003)(5660300002)(40480700001)(26005)(336012)(6666004)(70586007)(7416002)(7696005)(426003)(8676002)(70206006)(86362001)(47076005)(82310400005)(4326008)(82740400003)(478600001)(81166007)(356005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2022 18:56:43.6761 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d33bc205-64cd-4261-3b61-08dab395ff12 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT105.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7596 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org CXL downport PCIe AER information needs to be logged during error handling. The RCD downport/upport does not have a BDF and is not PCI enumerable. As a result the CXL PCIe driver is not aware of the AER in 'PCI Express' capability located in the RCRB downport/upport. Logic must be introduced to use the downport/upport AER information. Update the CXL driver to find the downport's PCIe AER capability and cache a pointer for using later. First, find the RCRB to provide the downport/upport memory region. The downport/upport are mapped as MMIO not PCI config space. Use readl/writel/readq/writeq as required by the CXL spec to find and operate on the AER registers.[1] Also, add function to detect if the device is a CXL1.1 RCD device. [1] CXL3.0, 8.2 'Memory Mapped Registers' Signed-off-by: Terry Bowman --- drivers/cxl/acpi.c | 56 ++++++++++++++ drivers/cxl/core/regs.c | 1 + drivers/cxl/cxl.h | 9 +++ drivers/cxl/cxlmem.h | 2 + drivers/cxl/mem.c | 2 + drivers/cxl/pci.c | 158 ++++++++++++++++++++++++++++++++++++++++ 6 files changed, 228 insertions(+) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index bf251a27e460..5d543c789e8d 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -232,6 +232,7 @@ struct cxl_chbs_context { struct device *dev; unsigned long long uid; struct acpi_cedt_chbs chbs; + resource_size_t chbcr; }; static int cxl_get_chbs(union acpi_subtable_headers *header, void *arg, @@ -417,6 +418,61 @@ static void remove_cxl_resources(void *data) } } +static const struct acpi_device_id cxl_host_ids[] = { + { "ACPI0016", 0 }, + { "PNP0A08", 0 }, + { }, +}; + +static int __cxl_get_rcrb(union acpi_subtable_headers *header, void *arg, + const unsigned long end) +{ + struct cxl_chbs_context *ctx = arg; + struct acpi_cedt_chbs *chbs; + + if (ctx->chbcr) + return 0; + + chbs = (struct acpi_cedt_chbs *)header; + + if (ctx->uid != chbs->uid) + return 0; + + if (chbs->cxl_version != ACPI_CEDT_CHBS_VERSION_CXL11) + return 0; + + if (chbs->length != SZ_8K) + return 0; + + ctx->chbcr = chbs->base; + + return 0; +} + +resource_size_t cxl_get_rcrb(struct cxl_memdev *cxlmd) +{ + struct pci_host_bridge *host = NULL; + struct cxl_chbs_context ctx = {0}; + struct cxl_dport *dport; + struct cxl_port *port; + + port = cxl_mem_find_port(cxlmd, NULL); + if (!port) + return 0; + + dport = port->parent_dport; + ctx.uid = dport ? dport->port_id : 0; + if (!dport) + return 0; + + acpi_table_parse_cedt(ACPI_CEDT_TYPE_CHBS, __cxl_get_rcrb, &ctx); + + dev_dbg(&host->dev, "RCRB found: 0x%08llx\n", (u64)ctx.chbcr); + + return ctx.chbcr; +} +EXPORT_SYMBOL_NS_GPL(cxl_get_rcrb, CXL); + /** * add_cxl_resources() - reflect CXL fixed memory windows in iomem_resource * @cxl_res: A standalone resource tree where each CXL window is a sibling diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index ec178e69b18f..0d4f633e5c01 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -184,6 +184,7 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr, return ret_val; } +EXPORT_SYMBOL_NS_GPL(devm_cxl_iomap_block, CXL); int cxl_map_component_regs(struct pci_dev *pdev, struct cxl_component_regs *regs, diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index ac8998b627b5..7d507ab80a78 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -204,6 +204,14 @@ struct cxl_register_map { }; }; +struct cxl_memdev; +int cxl_pci_probe_dport(struct cxl_memdev *cxlmd); + +void cxl_pci_aer_init(struct cxl_memdev *cxlmd); + +void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr, + resource_size_t length); + void cxl_probe_component_regs(struct device *dev, void __iomem *base, struct cxl_component_reg_map *map); void cxl_probe_device_regs(struct device *dev, void __iomem *base, @@ -549,6 +557,7 @@ static inline bool is_cxl_root(struct cxl_port *port) return port->uport == port->dev.parent; } +resource_size_t cxl_get_rcrb(struct cxl_memdev *cxlmd); bool is_cxl_port(struct device *dev); struct cxl_port *to_cxl_port(struct device *dev); struct pci_bus; diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 88e3a8e54b6a..079db2e15acc 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -242,6 +242,8 @@ struct cxl_dev_state { u64 next_volatile_bytes; u64 next_persistent_bytes; + struct cxl_register_map aer_map; + resource_size_t component_reg_phys; u64 serial; diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 64ccf053d32c..d1e663be43c2 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -74,6 +74,8 @@ static int cxl_mem_probe(struct device *dev) if (rc) return rc; + cxl_pci_aer_init(cxlmd); + parent_port = cxl_mem_find_port(cxlmd, &dport); if (!parent_port) { dev_err(dev, "CXL port topology not found\n"); diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index faeb5d9d7a7a..2287b5225862 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -35,6 +35,15 @@ (readl((cxlds)->regs.mbox + CXLDEV_MBOX_CTRL_OFFSET) & \ CXLDEV_MBOX_CTRL_DOORBELL) +/* PCI 5.0 - 7.8.4 Advanced Error Reporting Extended Capability */ +#define PCI_AER_CAP_SIZE 0x48 + +/* CXL 3.0 - 8.2.1.3.3, Offset to DVSEC Port Status */ +#define CXL_DVSEC_PORT_STATUS_OFF 0xE + +/* CXL 3.0 - 8.2.1.3.3 */ +#define CXL_DVSEC_VH_SUPPORT 0x20 + /* CXL 2.0 - 8.2.8.4 */ #define CXL_MAILBOX_TIMEOUT_MS (2 * HZ) @@ -428,6 +437,155 @@ static void devm_cxl_pci_create_doe(struct cxl_dev_state *cxlds) } } +static resource_size_t cxl_get_dport_ext_cap(struct cxl_memdev *cxlmd, u32 cap_id) +{ + resource_size_t rcrb, offset; + void *rcrb_mapped; + u32 cap_hdr; + + rcrb = cxl_get_rcrb(cxlmd); + if (!rcrb) + return 0; + + rcrb_mapped = ioremap(rcrb, SZ_4K); + if (!rcrb_mapped) + return 0; + + offset = PCI_CFG_SPACE_SIZE; + cap_hdr = readl(rcrb_mapped + offset); + + while (PCI_EXT_CAP_ID(cap_hdr)) { + if (PCI_EXT_CAP_ID(cap_hdr) == cap_id) + break; + + offset = PCI_EXT_CAP_NEXT(cap_hdr); + if (offset == 0) + break; + + cap_hdr = readl(rcrb_mapped + offset); + } + iounmap((void *)rcrb_mapped); + + if (PCI_EXT_CAP_ID(cap_hdr) != cap_id) + return 0; + + pr_debug("Found capability %X @ %llX (%X)\n", + cap_id, rcrb + offset, cap_hdr); + + return rcrb + offset; +} + +bool is_rcd(struct cxl_memdev *cxlmd) +{ + struct pci_dev *pdev; + resource_size_t dvsec; + void *dvsec_mapped; + u32 dvsec_data; + + if (!dev_is_pci(cxlmd->cxlds->dev)) + return false; + + pdev = to_pci_dev(cxlmd->cxlds->dev); + + /* + * 'CXL devices operating in this mode always set the Device/Port + * Type field in the PCI Express Capabilities register to RCiEP.' + * - CXL3.0 9.11.1 'RCD Mode' + */ + if (pci_pcie_type(pdev) != PCI_EXP_TYPE_RC_END) + return false; + + /* + * Check if VH is enabled + * - CXL3.0 8.2.1.3.1 'DVSEC Flex Bus Port Capability' + */ + dvsec = cxl_get_dport_ext_cap(cxlmd, PCI_EXT_CAP_ID_DVSEC); + if (!dvsec) + return false; + + dvsec_mapped = ioremap(dvsec, SZ_4K); + dvsec_data = readl(dvsec_mapped + CXL_DVSEC_PORT_STATUS_OFF); + iounmap(dvsec_mapped); + if (dvsec_data & CXL_DVSEC_VH_SUPPORT) + return false; + + return true; +} + +#define PCI_CAP_ID(header) (header & 0x000000ff) +#define PCI_CAP_NEXT(header) ((header >> 8) & 0xff) + +static resource_size_t cxl_get_dport_cap(struct cxl_memdev *cxlmd, int cap_id) +{ + resource_size_t offset, rcrb; + void *rcrb_mapped; + u32 cap_hdr; + + rcrb = cxl_get_rcrb(cxlmd); + if (!rcrb) + return 0; + + rcrb_mapped = ioremap(rcrb, SZ_4K); + if (!rcrb_mapped) + return 0; + + offset = readl(rcrb_mapped + PCI_CAPABILITY_LIST); + cap_hdr = readl(rcrb_mapped + offset); + + while (PCI_CAP_ID(cap_hdr)) { + if (PCI_CAP_ID(cap_hdr) == cap_id) + break; + + offset = PCI_CAP_NEXT(cap_hdr); + if (offset == 0) + break; + + cap_hdr = readl(rcrb_mapped + offset); + } + iounmap((void *)rcrb_mapped); + + if (PCI_CAP_ID(cap_hdr) != cap_id) + return 0; + + pr_debug("Found capability %X @ %llX (%X)\n", + cap_id, rcrb + offset, cap_hdr); + + return rcrb + offset; +} + +static int cxl_setup_dport_aer(struct cxl_memdev *cxlmd, resource_size_t cap_base) +{ + struct cxl_register_map *map = &cxlmd->cxlds->aer_map; + struct pci_dev *pdev = to_pci_dev(&cxlmd->dev); + + if (!cap_base) + return -ENODEV; + + map->base = devm_cxl_iomap_block(&pdev->dev, cap_base, + PCI_CAP_EXP_RC_ENDPOINT_SIZEOF_V1); + if (!map->base) + return -ENOMEM; + + return 0; +} + +void cxl_pci_aer_init(struct cxl_memdev *cxlmd) +{ + resource_size_t cap_base; + + /* CXL2.0 is enumerable and will use AER attached to `struct pci_dev` */ + if (!is_rcd(cxlmd)) + return; + + /* + * Read base address of the PCI express cap. Cache the cap's + * PCI_EXP_DEVCTL and PCI_EXP_DEVSTA for AER control and status. + */ + cap_base = cxl_get_dport_cap(cxlmd, PCI_CAP_ID_EXP); + cxl_setup_dport_aer(cxlmd, cap_base); +} +EXPORT_SYMBOL_NS_GPL(cxl_pci_aer_init, CXL); + static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { struct cxl_register_map map; From patchwork Fri Oct 21 18:56:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Bowman X-Patchwork-Id: 13015259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78BF1C38A2D for ; Fri, 21 Oct 2022 18:57:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230049AbiJUS5E (ORCPT ); Fri, 21 Oct 2022 14:57:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58446 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229982AbiJUS5A (ORCPT ); Fri, 21 Oct 2022 14:57:00 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2059.outbound.protection.outlook.com [40.107.93.59]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1715428E048; Fri, 21 Oct 2022 11:56:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=azUKMxPAQM9PdPdQ8ni0A72x61EfHzdWFAy5Kucaksi6R3AhVNIBIKEqNBUjRPb2lSTbWrjaF2xRRhZnBwzlu4nSIEAJyyF4uthF/1jXoD2xQ1oiYOUg0YR4Xcs8wBXUS1cV6w9bSnQB75myd2g9lW59mfh0vPwyTQVkm2sIs6uW/JjEWMPyGZ5TqVbrzfCJho2oLO6K6Ohx3ARlZNBO97bOVEDPrzjCUFc7Wkpeida2HUMuue0AHg/rpOZyDPxvURCicHDKzs4d3ytWNpo9PnTRSIgEqhZ1DN6snNaRoEaU2gUFOZF5E9TmKRJFcbqqv2jO3uc6mxghb23vnB5E0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tPLCm3ji3vI5vthM1SDorRmzlUNnDd2UN00R25Iao/k=; b=LLTwCKNu/mL9+IgY2dHB8T+hdKkJ9ohL2fWzvhmElnSSgXRwKQU5PBwXhE8PchE/bCI87FiMctyRmDcV3+BjxCjwoKjCkA/xtvc7C3QSgOQRYOMS4A9RhQhLnXsGPJ79o12joiyW/nGX6bMmngTStx4K8y7u1gL32leGutXXDxeoib2Aih/IHbx1CYrIPBpo4x3fuWBOvQ7E3DRTzZLOfc+H5V2eOCB21BIRZS4jFZU8Cuw+FYMBfkifKQU1Na9zQeinHwL0zuCktItPpzueR50VgmhaW2O7Um2+aEwrA7EP1bhU35nH0rSsTOwP9CKqyd/FQxgohywCmHz1pRMtGA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tPLCm3ji3vI5vthM1SDorRmzlUNnDd2UN00R25Iao/k=; b=2gKVKULONLOlsWCFztcv5Zl2EiCLX72Xy0A9pu/oiEF15dj+HUPAWner9fjvkRma683K3XdIT05wuo/UfStpcDrM8LmSY5paPknX1mewfIPgIAtbAEXQiTeooqEAKfSZLdEe/BOHfB+RH7ioS/Bx1Xy01Flu+O479xHrAAGvoU0= Received: from BN1PR10CA0026.namprd10.prod.outlook.com (2603:10b6:408:e0::31) by SA1PR12MB7197.namprd12.prod.outlook.com (2603:10b6:806:2bd::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.26; Fri, 21 Oct 2022 18:56:55 +0000 Received: from BN8NAM11FT065.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e0:cafe::ba) by BN1PR10CA0026.outlook.office365.com (2603:10b6:408:e0::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.21 via Frontend Transport; Fri, 21 Oct 2022 18:56:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT065.mail.protection.outlook.com (10.13.177.63) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5746.16 via Frontend Transport; Fri, 21 Oct 2022 18:56:54 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 21 Oct 2022 13:56:53 -0500 From: Terry Bowman To: , , , , , CC: , , , , , , , , Subject: [PATCH 3/5] cxl/pci: Discover and cache pointer to RCD dport's CXL RAS registers Date: Fri, 21 Oct 2022 13:56:13 -0500 Message-ID: <20221021185615.605233-4-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021185615.605233-1-terry.bowman@amd.com> References: <20221021185615.605233-1-terry.bowman@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT065:EE_|SA1PR12MB7197:EE_ X-MS-Office365-Filtering-Correlation-Id: b3d9c79e-f6db-4098-a0c0-08dab396059c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: UdLJBe3pl5wsW75ZgQ8ql0rTfwW/8f+UYH+IzY1qmyfjZ6oaBA3AKtJdlY/hVYORExGRkvGRROhcNUZlscV7g5FLCa2iDtm/0ACAjt59KGrVJUOb5G4c3i6cxOIzVUhg0woCqvA4Nta+7ZPeOav5HVtszu+Fu5W1SQ+Xf1oQPZl9l8Mmw/iscctHMFsbSbFfY9s2SmPIMl/opBEQYIpi2RYCw/jnGm0ZZl25peQuWVwE+CKbQZa8SE1d1On7aOAt7742UzXTtGq4N6z7AD85XjUm72UpGH4Ux4L8FuYjbSiF1QuRxfzHf55X2yzGKpOkpu06FjzS8TtuThsqnJeV9kljb/fFy/uiPdHYL/hf3ZTsCdESgkSh7yXxPvPWeUTtdU0U0U/TmfbWMkKhjLnuL/SloT4eTbidN771GYfSsphp5bPvlvkcTsodgnMfM0p6tJpR7I7KBJP27ISoHZlV6FeupyQDKJIU1gpTwql+rJ+ideqDLRmbbbj2FCdJWZGUYHZkn9LtAzi21cxdTsdyIlulI6h6kRncby33jtJ9ibSjAi2rx/uYQWxkmuiN6SKifL+caWU3xnndHrsbZdmnHC9Jnebr/srM3Ti/OOnaOk5ZEkkru1f+J/EJbgQtS245FB6cpjZuFNp3rLcE4lZ/UFsQqK1VkdyCe6DKxlnLHp9J4Bk4264yFUQeh5U4d8aCPWexrd1tKm9b1a7oZGqZEKRiFt8LyJN7Y0iMCy8DB/DNhZOEj6+LI9LjaSGoWw6zDjXc6ZeYYcmwJ+EcHfVx0XXTO2gU5GDw39Yadvh+ElJAJcUxeQYTjrTEVirdW4KC X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(396003)(346002)(376002)(39860400002)(136003)(451199015)(40470700004)(46966006)(36840700001)(8936002)(81166007)(110136005)(6666004)(54906003)(41300700001)(8676002)(70586007)(478600001)(36860700001)(4326008)(36756003)(1076003)(7696005)(2616005)(40480700001)(336012)(16526019)(186003)(82740400003)(40460700003)(356005)(86362001)(82310400005)(70206006)(26005)(47076005)(2906002)(316002)(83380400001)(426003)(7416002)(44832011)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2022 18:56:54.6494 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b3d9c79e-f6db-4098-a0c0-08dab396059c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT065.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7197 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org CXL RAS information resides in a RAS capability structure located in CXL.cache and CXL.mem registers.[1] The RAS capability provides CXL specific error information that can be helpful in debugging. This information is not currently logged but needs to be logged during PCIe AER error handling. Update the CXL driver to find and cache a pointer to the CXL RAS capability. The RAS registers resides in the downport's component register block. Note:RAS registers are not in the upport. The component registers can be found by first using the RCRB to goto the downport. Next, the downport's 64-bit BAR[0] will point to the component register block. [1] CXL3.0 Spec, '8.2.5 CXL.cache and CXL.mem Registers' Signed-off-by: Terry Bowman --- drivers/cxl/cxl.h | 4 +++ drivers/cxl/cxlmem.h | 1 + drivers/cxl/pci.c | 72 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 77 insertions(+) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 7d507ab80a78..69b50131ad86 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -36,6 +36,10 @@ #define CXL_CM_CAP_CAP_ID_HDM 0x5 #define CXL_CM_CAP_CAP_HDM_VERSION 1 +/* CXL 3.0 8.2.4.2 CXL RAS Capability Header */ +#define CXL_CM_CAP_ID_RAS 0x2 +#define CXL_CM_CAP_SIZE_RAS 0x5C + /* HDM decoders CXL 2.0 8.2.5.12 CXL HDM Decoder Capability Structure */ #define CXL_HDM_DECODER_CAP_OFFSET 0x0 #define CXL_HDM_DECODER_COUNT_MASK GENMASK(3, 0) diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 079db2e15acc..515273e224ea 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -243,6 +243,7 @@ struct cxl_dev_state { u64 next_persistent_bytes; struct cxl_register_map aer_map; + struct cxl_register_map ras_map; resource_size_t component_reg_phys; u64 serial; diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 2287b5225862..7f717fb47a36 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -586,6 +586,78 @@ void cxl_pci_aer_init(struct cxl_memdev *cxlmd) } EXPORT_SYMBOL_NS_GPL(cxl_pci_aer_init, CXL); +static resource_size_t cxl_get_dport_ras_base(struct cxl_memdev *cxlmd) +{ + resource_size_t component_reg_phys, offset = 0; + struct cxl_dev_state *cxlds = cxlmd->cxlds; + void *cap_hdr_addr, *comp_reg_mapped; + u32 cap_hdr, ras_cap_hdr; + int cap_ndx; + + comp_reg_mapped = ioremap(cxlds->component_reg_phys + + CXL_CM_OFFSET, CXL_COMPONENT_REG_BLOCK_SIZE); + if (!comp_reg_mapped) + return 0; + + cap_hdr_addr = comp_reg_mapped; + cap_hdr = readl(cap_hdr_addr); + for (cap_ndx = 0; + cap_ndx < FIELD_GET(CXL_CM_CAP_HDR_ARRAY_SIZE_MASK, cap_hdr); + cap_ndx++) { + ras_cap_hdr = readl(cap_hdr_addr + cap_ndx*sizeof(u32)); + + if (FIELD_GET(CXL_CM_CAP_HDR_ID_MASK, ras_cap_hdr) == CXL_CM_CAP_ID_RAS) { + pr_debug("RAS cap header = %X @ %pa, cap_ndx = %d\n", + ras_cap_hdr, cap_hdr_addr, cap_ndx); + break; + } + } + + offset = CXL_CM_OFFSET + PCI_EXT_CAP_NEXT(ras_cap_hdr); + + iounmap(comp_reg_mapped); + + if (FIELD_GET(CXL_CM_CAP_HDR_ID_MASK, ras_cap_hdr) != CXL_CM_CAP_ID_RAS) + return 0; + + pr_debug("Found RAS capability @ %llX (%X)\n", + component_reg_phys + offset, *((u32 *)(comp_reg_mapped + offset))); + + return component_reg_phys + offset; +} + +static int cxl_setup_dport_ras(struct cxl_memdev *cxlmd, resource_size_t resource) +{ + struct cxl_register_map *map = &cxlmd->cxlds->ras_map; + struct pci_dev *pdev = to_pci_dev(&cxlmd->dev); + + if (!resource) { + pr_err("%s():%d: RAS resource ptr is NULL\n", __func__, __LINE__); + return -EINVAL; + } + + map->base = devm_cxl_iomap_block(&pdev->dev, resource, CXL_CM_CAP_SIZE_RAS); + if (!map->base) + return -ENOMEM; + + return 0; +} + +void cxl_pci_ras_init(struct cxl_memdev *cxlmd) +{ + resource_size_t cap; + + /* + * TODO - CXL2.0 will need change to support PCI config space. + */ + if (!is_rcd(cxlmd)) + return; + + cap = cxl_get_dport_ras_base(cxlmd); + cxl_setup_dport_ras(cxlmd, cap); +} +EXPORT_SYMBOL_NS_GPL(cxl_pci_ras_init, CXL); + static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { struct cxl_register_map map; From patchwork Fri Oct 21 18:56:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Bowman X-Patchwork-Id: 13015260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D098C433FE for ; Fri, 21 Oct 2022 18:57:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230005AbiJUS5W (ORCPT ); Fri, 21 Oct 2022 14:57:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230126AbiJUS5N (ORCPT ); Fri, 21 Oct 2022 14:57:13 -0400 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2083.outbound.protection.outlook.com [40.107.95.83]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5CFAC28F278; Fri, 21 Oct 2022 11:57:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jnCi6dsnpvHeugPfwT2JqybkHgM/4itAn+vtKTgQBS+nrEIuyMHM5ivtCb5WbeaFh6OYcyqAzt7ZWQ0chh3+loVSWNrN198H+RIMp6HgX6gq9D2zsSAmvVxoDTww8qX4rS+nEyum4NKKfWkB+Q4cRoU67GYwfNq+rM/UxtFJAqQkdcCJ9Hdm4Ue9xqUnvUtk/9HG6yqYB2UXRXyNeR+s0pwSxXZ5O7ZOQ8y2RhgSYtgBS4eBo3d3T2zni7dp1fBKLAoQ2gdDb+n372PDuU0XB2lvh2VXtosmrBzz92juf6vJtOyRsPI7Nm2sdwk+x+G5OpDGd6atZoZG7t5K75F1Gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PEUepIvqVCto5d9DOAhi8VyT1xoFQF7b2RHIibve5pU=; b=Qb36Y9np3bMzLVvtWEiDtWiX2g6YdWam+4pCrOoBTeRvY95DKCtP3QqUjA9kXWWw4ghQ2K4m1qNrkqt2wLX83524IdCwZ/6Qfv4/j187TetUiYKLEVPLqPpZGHq0ZCXVN92IgaA8Jo3FFG0jlQFr+8M/XXKszpO2DUIfXhsLielgPvn2kvugCrKv4657/okdtT3KHu4W9xL+0WErmzP01sIHoULX7faNIstmrTy61M4f7U1B3KZmA9+/hMmuiU90/3kNztOVQ96mfuJT/OowX/7P0HX/xq6O4g2qKt49hHM0wfCmzAFi0BzqmRW4X9myT3gKtIxwUyA1kD594zCL2g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PEUepIvqVCto5d9DOAhi8VyT1xoFQF7b2RHIibve5pU=; b=shCPxl6RQx2IAsRHZDj/SAf4em2eTKoSPnmUX6i0BaUZyBue9EC2T407lUL0CP1FKrJJICHAPMSgpKDAyUgujTaebUzBTYUhReH1Q0y5YIdZpQgEcWzulv/HOiTGZVQvnLWgPpfQL87OHTP76In9ASSLHlLo4MekAEHnUAKNk5c= Received: from BN0PR04CA0095.namprd04.prod.outlook.com (2603:10b6:408:ec::10) by DS0PR12MB6416.namprd12.prod.outlook.com (2603:10b6:8:cb::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.21; Fri, 21 Oct 2022 18:57:06 +0000 Received: from BN8NAM11FT096.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ec:cafe::60) by BN0PR04CA0095.outlook.office365.com (2603:10b6:408:ec::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.21 via Frontend Transport; Fri, 21 Oct 2022 18:57:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT096.mail.protection.outlook.com (10.13.177.195) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5746.16 via Frontend Transport; Fri, 21 Oct 2022 18:57:05 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 21 Oct 2022 13:57:04 -0500 From: Terry Bowman To: , , , , , CC: , , , , , , , , Subject: [PATCH 4/5] cxl/pci: Enable RCD dport AER reporting Date: Fri, 21 Oct 2022 13:56:14 -0500 Message-ID: <20221021185615.605233-5-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021185615.605233-1-terry.bowman@amd.com> References: <20221021185615.605233-1-terry.bowman@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT096:EE_|DS0PR12MB6416:EE_ X-MS-Office365-Filtering-Correlation-Id: 51e11ef2-2359-45fa-6a88-08dab3960c28 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VE58TeLIba04z28DIZzytxZ8Xk/eJDXXsXNQOX2kXzpiKOu4vInWtLZsSWDLCCMHDrOMg7mSL4y86i4LByS7E5SeNRwO6O73Fy8Xc1GDioLYVtH5r41dJC678HeOqhLBW5Pj7GiiJWb/3bsOlf35Qh1/uUBYf4nKgsJBdYw0o+Z4sG+w35w3jngX5gRBZEGTKhmjRJMuEqrOb3SHVQmuMfMtzfCuCgz1oWTAZcTY+O6p8HfSwdz2PnHdILj/zvlghQqA/vdZnfuY4rEmsgCISvA3jZbuZlyLW5wUIpCuvgkcv9AROEECJJBu/gQwgflK5oeOAmJ4wchQwWku6uRE5fjpc4OHJD/h2gvl7kPX/CTwkLcATdN/mJzOwrf3vYXNIXvG45S3duqcosafG0fXIdwOhRCH1so7FgfqL8ZD0ykr0Z9clOntdPmnWFD51RCtlClsoqQPdStZF8qo+EeH0Z7ilarcfEfyrrQ1rQDEdRJWOC2kxxnJkdJ210kjnOdFD7n2q5EU+alioDPCjrqEFNP2vf/7BUifS2pxZZ7jK3siif9RyMt68MSl7qd/b1o2gnFIJI+h2DpK8dBFJZT0dQ5dNJv54Hluiv1SOxzYAkHkSW5iIkPGM6BuWdYeZysedTRvjASCcKhMnhIBhKlMWXVfFVFs+rsgWQOMKHv0FZQ6/EutocC+doYEhMewQCyA3JHR4XIeiSTUaThDGLbbaVimwEDPHSG0imwLFKZGyuqgWC5u0LGWNBYEtrTxQ6SnQjYH3N5UclVJ/t/5AOYGiCy5Lz0vma5kKtWNlJDmvCyRw5FYwsFTL02P7THIBrx5 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(396003)(346002)(39860400002)(136003)(376002)(451199015)(36840700001)(46966006)(40470700004)(36860700001)(40460700003)(82740400003)(316002)(81166007)(356005)(54906003)(4326008)(8676002)(70206006)(36756003)(70586007)(110136005)(82310400005)(2906002)(44832011)(41300700001)(8936002)(7416002)(5660300002)(426003)(47076005)(40480700001)(6666004)(7696005)(336012)(186003)(1076003)(16526019)(2616005)(26005)(478600001)(86362001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2022 18:57:05.6319 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 51e11ef2-2359-45fa-6a88-08dab3960c28 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT096.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB6416 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org The RCD downport/upport include 'PCI express' capability with AER registers. The PCI subsystem is not aware of RCD downport/upport AER because the downport/upport are not enumerable devices. Since the downport/upport are not enumerable the existing PCIe AER logic to enable AER reporting does not apply. Add logic to the CXL driver to enable AER reporting in the RCRB 'PCI express' capability. These must be set for correctly reporting the PCIe AER errors to the RCEC or root port. Signed-off-by: Terry Bowman --- drivers/cxl/pci.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 7f717fb47a36..80a01b304efe 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -553,6 +553,17 @@ static resource_size_t cxl_get_dport_cap(struct cxl_memdev *cxlmd, int cap_id) return rcrb + offset; } +static void cxl_enable_dport_aer(struct cxl_memdev *cxlmd) +{ + struct cxl_register_map *map = &cxlmd->cxlds->aer_map; + u32 devctl_cap; + + devctl_cap = readl(map->base + PCI_EXP_DEVCTL); + devctl_cap |= (PCI_EXP_DEVCTL_CERE | PCI_EXP_DEVCTL_NFERE | + PCI_EXP_DEVCTL_FERE | PCI_EXP_DEVCTL_URRE); + writel(devctl_cap, map->base + PCI_EXP_DEVCTL); +} + static int cxl_setup_dport_aer(struct cxl_memdev *cxlmd, resource_size_t cap_base) { struct cxl_register_map *map = &cxlmd->cxlds->aer_map; @@ -566,6 +577,8 @@ static int cxl_setup_dport_aer(struct cxl_memdev *cxlmd, resource_size_t cap_bas if (!map->base) return -ENOMEM; + cxl_enable_dport_aer(cxlmd); + return 0; } From patchwork Fri Oct 21 18:56:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Bowman X-Patchwork-Id: 13015261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22203C433FE for ; Fri, 21 Oct 2022 18:57:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229690AbiJUS5f (ORCPT ); Fri, 21 Oct 2022 14:57:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230086AbiJUS5d (ORCPT ); Fri, 21 Oct 2022 14:57:33 -0400 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2058.outbound.protection.outlook.com [40.107.237.58]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F77528F26F; Fri, 21 Oct 2022 11:57:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gyMpALwWrjRBo6Is60IBR5tQH+m54EfR20B2DbfIff/6gtSSluOkvNjQ32rCl3ql18xsnJkLLSScQzEzFvfqt32Qk5tDdZvozZiQVquHc+x3iP9pxVvkVk3WjIU8GmfBAcstUqhE1vjGxb95ANZke9nddX+JZ1UflTh0iibdO2b7LZdCW7qfKgJtI2uUUb46/gF8Tx68HvmBldztY/AdRMdwFVS7eUrVTr2gkskdDTq8ZZp6oIIgmXr6UxN37hyhNwa37dRvNcGyU9/aYZPeK/A/RZTP56z6hBfF25Vr6kIzzB8raOpZx3Lwa1LzBYFvdmPZuyvfWOziUmZunUZo2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=g6Dn92GPbW7gIcGIZtAondhwYlN+GFR0oASVtDZ8Nko=; b=PCDQIo/hFo9CjDZ8KOrxUuGkxyBi2eQszPeoOUo27rZgma33zWFRKapJp8bGQfBeNA4DIjRaEUHuxq64fqVplsxB+zyFfPcH04R3ElaknG0kuWVS67eaEk5pKZBYQAzEBUHYe41jTdQfAgdR20r6YbDvAED8kZ8im8zh5DiPSNc5NilEaJZEOYY+56CYMdDjP54WZwNkH54nqGr2WgpSp4GCHeMPbWf79KdRow91096624rg7bebKqDudMQO7QMcgDmOSd0A/Xfk/1+fuLQJafROxnmzdCl0DVfdpOdk/KYUFTjsLvAuVgbO2HgU/M1nKuFaNEJIBdjUOHMIEnVWuQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g6Dn92GPbW7gIcGIZtAondhwYlN+GFR0oASVtDZ8Nko=; b=u9wh0YDGD+sZNP4MOnniP3cYVxQM/ixG7OjRMWiR4HSUH33wMcP9gehhphts7UgEZmgaRejiMQ+jlOK9dLJbPUZ2bn08tRh9mO7XqmMF0TD8dZbrgRldRX85o+mTTXkxLj1U1ZXBKcmPLWUx2e1oJnjJIOsUn1eCp5dmUCiXq6M= Received: from BN9PR03CA0287.namprd03.prod.outlook.com (2603:10b6:408:f5::22) by BY5PR12MB4855.namprd12.prod.outlook.com (2603:10b6:a03:1dd::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.33; Fri, 21 Oct 2022 18:57:17 +0000 Received: from BN8NAM11FT031.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f5:cafe::60) by BN9PR03CA0287.outlook.office365.com (2603:10b6:408:f5::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.21 via Frontend Transport; Fri, 21 Oct 2022 18:57:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT031.mail.protection.outlook.com (10.13.177.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5746.16 via Frontend Transport; Fri, 21 Oct 2022 18:57:16 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 21 Oct 2022 13:57:15 -0500 From: Terry Bowman To: , , , , , CC: , , , , , , , , Subject: [PATCH 5/5] cxl/pci: Log CXL device's PCIe AER and CXL RAS error information Date: Fri, 21 Oct 2022 13:56:15 -0500 Message-ID: <20221021185615.605233-6-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021185615.605233-1-terry.bowman@amd.com> References: <20221021185615.605233-1-terry.bowman@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT031:EE_|BY5PR12MB4855:EE_ X-MS-Office365-Filtering-Correlation-Id: 473852d0-1a92-4430-d190-08dab39612be X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: d0oEYI4VX2aDCBazeUSDuJlT8ou582+5/qkH7QUGYE6T9sDkKZ2cIgaHVwBFeUCy7f/GwiZcvyelo1kAsj5wb19Xut0yvnfHq7Kup/STz/hHvWUrQH/p1FjuouIDQgzt1geIa/VKcR+HIjmMFZXH+etqLDuO72u3pr6YPIGVBT6Oe+7kKLI74shMCKX+vWrXvJ1VN/8TBqJ+js/PNj8ZOQ5dHEW8GQOOVIbt1CJQPc8TOkmao2Tj7llEbKfAX8yKRDOM63XezDecbZTkDriedZ0iBqtTFKLoefvc13KU226LHAs9aJLazSU1R7AKXtnz2tYKsH0YKAYP0Wav4EOjClfIfc2/cS+KdN3HcNrT1xkByPdP4YksHS8j9pClHMoyL2fQNUnKHml9QIR0ES6yK1nfru4DHYwIcHv4BJA8csfh8C1Ikr7+WtPMr2xvn9sdA4ljEOeuuY124r876duMuBQnxBJnistXrmigC4yv/cKumxe+clm38fxBQnQM6S+8MaCgHMhJZALe1gqnEbyC3bOZzpgIEla5b0DgyCjIiBLjQ2Ut2oVXWgFfW+PI+eISG3bIAZTNltJmHyJx9oD6TgDFTVcepLA48xxLhbnz7bEsh20Go5SUPBqNmhZ3GRbaK/2xOnYg6583NxcV+UiIzw9jzI3L+bpi3a58vn8Tk4pyrueRtyreOJu26nCz2C42TGxgE5K0hcllFlC3q9SK8DenNWkYGRq6/BA6u8Vq7u+QVZ0lZDKtNzSKrc7+2YeDYkGaZuSZ+cbga76N7Jb77cEjF3YCxg6yPZSONPum4LrDvRD25V2wwC1/cokOawo+ X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(136003)(39860400002)(346002)(376002)(396003)(451199015)(40470700004)(36840700001)(46966006)(2906002)(2616005)(40460700003)(186003)(36860700001)(16526019)(47076005)(7696005)(426003)(44832011)(40480700001)(26005)(5660300002)(1076003)(7416002)(41300700001)(110136005)(8676002)(8936002)(316002)(4326008)(478600001)(82310400005)(54906003)(336012)(70206006)(70586007)(86362001)(36756003)(356005)(81166007)(82740400003)(83380400001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2022 18:57:16.6839 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 473852d0-1a92-4430-d190-08dab39612be X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT031.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4855 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org The CXL downport PCIe AER and CXL RAS capability information needs to be logged during PCIe AER error handling. The existing PCIe AER error handler logs native AER errors but does not log upport/downport AER capability residing in the RCRB. The CXL1.1 RCRB does not have a BDF and is not enunmerable. The existing error handler logic does not display CXL RAS details either. Add a CXL error handler to the existing PCI error handlers. Add a call to the CXL error handler within the PCIe AER error handler. Implement the driver's CXL callback to log downport PCIe AER and CXL RAS capability information. Signed-off-by: Terry Bowman --- drivers/cxl/pci.c | 76 ++++++++++++++++++++++++++++++++++++++++++ drivers/pci/pcie/aer.c | 45 ++++++++++++++++++++++++- include/linux/pci.h | 4 +++ 3 files changed, 124 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 80a01b304efe..dceda9f9fc60 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include #include @@ -14,6 +15,9 @@ #include "cxlpci.h" #include "cxl.h" +extern void cxl_print_aer(struct pci_dev *dev, int aer_severity, + struct aer_capability_regs *aer); + /** * DOC: cxl pci * @@ -744,9 +748,80 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (resource_size(&cxlds->pmem_res) && IS_ENABLED(CONFIG_CXL_PMEM)) rc = devm_cxl_add_nvdimm(&pdev->dev, cxlmd); + dev_set_drvdata(&pdev->dev, cxlmd); + return rc; } +struct ras_cap { + u32 uc_error_status; + u32 uc_error_mask; + u32 uc_error_severity; + u32 c_error_status; + u32 c_error_mask; + u32 ctrl; + u32 log[]; +}; + +/* + * Log the state of the CXL downport AER and RAS status registers. + */ +static void cxl_error_report(struct cxl_memdev *cxlmd) +{ + struct pci_dev *pdev = to_pci_dev(cxlmd->cxlds->dev); + struct aer_capability_regs *aer_cap; + struct ras_cap *ras_cap; + + aer_cap = (struct aer_capability_regs *)cxlmd->cxlds->aer_map.base; + ras_cap = (struct ras_cap *)cxlmd->cxlds->ras_map.base; + + pci_err(pdev, "CXL Error Report\n"); + pci_err(pdev, "AER Errors:\n"); + if (aer_cap) { + cxl_print_aer(pdev, AER_CORRECTABLE, aer_cap); + cxl_print_aer(pdev, AER_NONFATAL, aer_cap); + cxl_print_aer(pdev, AER_FATAL, aer_cap); + } + + pci_err(pdev, "RAS Errors:\n"); + if (ras_cap) { + pci_err(pdev, "RAS: uc_error_status = %X\n", readl(&ras_cap->uc_error_status)); + pci_err(pdev, "RAS: uc_error_mask = %X\n", readl(&ras_cap->uc_error_mask)); + pci_err(pdev, "RAS: uc_error_severity = %X\n", readl(&ras_cap->uc_error_severity)); + pci_err(pdev, "RAS: c_error_status = %X\n", readl(&ras_cap->c_error_status)); + pci_err(pdev, "RAS: c_error_mask = %X\n", readl(&ras_cap->c_error_mask)); + pci_err(pdev, "RAS: ras_caps->ctrl = %X\n", readl(&ras_cap->ctrl)); + pci_err(pdev, "RAS: log = %X\n", readl(&ras_cap->log)); + } +} + +static void cxl_error_detected(struct pci_dev *pdev) +{ + struct cxl_memdev *cxlmd; + + if (!is_cxl_memdev(&pdev->dev)) { + pci_err(pdev, "CXL memory device is invalid\n"); + return; + } + + cxlmd = dev_get_drvdata(&pdev->dev); + if (!cxlmd) { + pci_err(pdev, "CXL memory device is NULL\n"); + return; + } + + if (!cxlmd->cxlds) { + pci_err(pdev, "CXL device state object is NULL\n"); + return; + } + + cxl_error_report(cxlmd); +} + +static struct pci_error_handlers cxl_error_handlers = { + .cxl_error_detected = cxl_error_detected, +}; + static const struct pci_device_id cxl_mem_pci_tbl[] = { /* PCI class code for CXL.mem Type-3 Devices */ { PCI_DEVICE_CLASS((PCI_CLASS_MEMORY_CXL << 8 | CXL_MEMORY_PROGIF), ~0)}, @@ -761,6 +836,7 @@ static struct pci_driver cxl_pci_driver = { .driver = { .probe_type = PROBE_PREFER_ASYNCHRONOUS, }, + .err_handler = &cxl_error_handlers, }; MODULE_LICENSE("GPL v2"); diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index e2d8a74f83c3..dea04d412406 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -811,6 +811,13 @@ void cper_print_aer(struct pci_dev *dev, int aer_severity, } #endif +void cxl_print_aer(struct pci_dev *dev, int aer_severity, + struct aer_capability_regs *aer) +{ + cper_print_aer(dev, aer_severity, aer); +} +EXPORT_SYMBOL_GPL(cxl_print_aer); + /** * add_error_device - list device to be handled * @e_info: pointer to error info @@ -1169,6 +1176,40 @@ static void aer_isr_one_error(struct aer_rpc *rpc, } } +static int report_cxl_errors_iter(struct pci_dev *pdev, void *data) +{ + struct pci_driver *pdrv = pdev->driver; + + if (pdrv && + pdrv->err_handler && + pdrv->err_handler->cxl_error_detected) + pdrv->err_handler->cxl_error_detected(pdev); + + return 0; +} + +static void report_cxl_errors(struct aer_rpc *rpc, + struct aer_err_source *e_src) +{ + struct pci_dev *pdev = rpc->rpd; + struct aer_err_info e_info; + u32 uncor_status, cor_status; + + pci_read_config_dword(pdev, pdev->aer_cap + PCI_ERR_UNCOR_STATUS, &uncor_status); + pci_read_config_dword(pdev, pdev->aer_cap + PCI_ERR_COR_STATUS, &cor_status); + + if (!uncor_status && !cor_status) + return; + + if (pci_pcie_type(pdev) == PCI_EXP_TYPE_RC_EC) + pcie_walk_rcec(pdev, report_cxl_errors_iter, &e_info); + else + pci_walk_bus(pdev->subordinate, report_cxl_errors_iter, &e_info); + + pci_write_config_dword(pdev, pdev->aer_cap + PCI_ERR_UNCOR_STATUS, uncor_status); + pci_write_config_dword(pdev, pdev->aer_cap + PCI_ERR_COR_STATUS, cor_status); +} + /** * aer_isr - consume errors detected by root port * @irq: IRQ assigned to Root Port @@ -1185,8 +1226,10 @@ static irqreturn_t aer_isr(int irq, void *context) if (kfifo_is_empty(&rpc->aer_fifo)) return IRQ_NONE; - while (kfifo_get(&rpc->aer_fifo, &e_src)) + while (kfifo_get(&rpc->aer_fifo, &e_src)) { + report_cxl_errors(rpc, &e_src); aer_isr_one_error(rpc, &e_src); + } return IRQ_HANDLED; } diff --git a/include/linux/pci.h b/include/linux/pci.h index 2bda4a4e47e8..4f4b3a8f5454 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -827,6 +827,10 @@ enum pci_ers_result { /* PCI bus error event callbacks */ struct pci_error_handlers { + + /* CXL error detected on this device */ + void (*cxl_error_detected)(struct pci_dev *dev); + /* PCI bus error detected on this device */ pci_ers_result_t (*error_detected)(struct pci_dev *dev, pci_channel_state_t error);