From patchwork Mon Jun 17 16:56:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Petr Machata X-Patchwork-Id: 13701082 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2071.outbound.protection.outlook.com [40.107.93.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 57163166306 for ; Mon, 17 Jun 2024 16:57:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.71 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718643426; cv=fail; b=SwL3fj0faFkXIW8YN01qyCfW3dsWqklDCqTM/zCOcrMFodh4TFeIZaTvMXATDPHmIX7157rhYAM4ug4Dpvwv1rMB9bVUIHHN5DRaxs29ay3btQie9dNqSI25O7X5fhRiTJnOe07XBHlobPriIvEdmxYofaZ0fDUHhGGfy2LCGP0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718643426; c=relaxed/simple; bh=dpLorgXv/Jt+f5TxRJTc6ErxmTh3ShUwp4q8m9NE8YM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=bBWYPd9kXRSePi7HTG9PWX+1+/iCRjNgTzk4Dg16lI8Vkq+9oJnFbHr67k/E6mPgHBTqILjyuRadbju2yA6EGkxSfsaMOVTFARciv8tUSwZahBiWEIyRKi+PcBaoHUMrGtMLL0aGxBqg6T0t5ZDn99r96OIWJhAxAKpF5lN2XwY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=gPS43wrU; arc=fail smtp.client-ip=40.107.93.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="gPS43wrU" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lH4h1ihbTRcmPobbGoIhdLWfS0pPTw7RmmI4djenwmVJncPOybUX4VOQlk2uVr79VvJLyd4C3LVU5VobnI1+5lhv+LTPhH4QyKej4kV9gkndTJCMp2YEosOf2F4Wjz4HY47yWJfWEQ9U9HNMKgyDKz5qoqk3LCmXFawenUtKwyr/6FrZQ0AYa2A1hu7+/SYByWwPVqEbTN3Pw+hrDIhhKX8wUCjwDOkfqu5CE3XEcbDtfaXuZ9bH79KVpWwLFm0joUcHaCk7PJWt3v2dDzqnVZEepXuHf/w9uwgbiudXCY9Qle9uqVGP0ld/wtlnC+lOQUxO6sGmuKL32DgNfviSyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=g3dexDeR1R3FAjSxQqJ3Ng8qPTEuD0xqsbpkuzPqxOk=; b=NQ+INKSbluWarVaRMC5R+QgcRY8MYGkcjI/qYdcYRbNm0UVz1AG3OYqeKucF1o3gIwNDmudGv2iuTgiNkNCiUdWWhp0mU8PaMUxm6Ww+BNuSwcf+rAY9wZFrU20O+0SMxG0GRZOdpFonH0drC3evlfZi8zW1NWzF1ONdf/Av8q3x2puX76QwrBmzfxLhKHp5Hmg5ZaXIWg6kvEu0F3XdnxVDuJ1Kmv/OinRVuGBLRTnoqKnloe+yh7HWRJeL5+eIB1Ac7ENUHRX1uVq7OKerBPEt47EOxqM2MKn9E/T57g7fi0uz6fPCSKYBUm9bf/tqqbZAEU5ZKVyA2TqYj1DaJQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g3dexDeR1R3FAjSxQqJ3Ng8qPTEuD0xqsbpkuzPqxOk=; b=gPS43wrURYPo0ItXlwoWGES+LY7mdK+HC8TzGCaCCuNqsFRZzEA4kOmpzSqGJFq5VXoPVgiVfjLkQTET6HExhQPMjDPf3wkXl7wanmTFFiMJrH7GKiR+O6BnY20/Cwu+6l/4ERbFRK2S4UpJoEmn8Jg2NUTodtU5fcZS+c1xbeQio571urMQYRkYI2d0OfuBXGIQhohWEKTn8tNMprB0jxdV/qeAvocFCRgqD8ebveFGVqJ0JTl8/P3YP58ikXFp3PmqaXgUyDwq0BHCqfCrOf7QeddrFbb00WnS+yTj4p3N1JswdJ3LjXTFaKA6kjc4/hPDmY8d3YHV0EjB+L/DPQ== Received: from BY5PR16CA0022.namprd16.prod.outlook.com (2603:10b6:a03:1a0::35) by CH3PR12MB8188.namprd12.prod.outlook.com (2603:10b6:610:120::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.31; Mon, 17 Jun 2024 16:57:00 +0000 Received: from SJ5PEPF000001D6.namprd05.prod.outlook.com (2603:10b6:a03:1a0:cafe::8) by BY5PR16CA0022.outlook.office365.com (2603:10b6:a03:1a0::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.31 via Frontend Transport; Mon, 17 Jun 2024 16:56:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SJ5PEPF000001D6.mail.protection.outlook.com (10.167.242.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.15 via Frontend Transport; Mon, 17 Jun 2024 16:56:58 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 17 Jun 2024 09:56:42 -0700 Received: from yaviefel.mtl.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 17 Jun 2024 09:56:38 -0700 From: Petr Machata To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , CC: Ido Schimmel , Petr Machata , , Simon Horman , Maksym Yaremchuk Subject: [PATCH net 1/3] mlxsw: pci: Fix driver initialization with Spectrum-4 Date: Mon, 17 Jun 2024 18:56:00 +0200 Message-ID: <782b8dfa5a4b6adb0ef9b56303037fb0cda19226.1718641468.git.petrm@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001D6:EE_|CH3PR12MB8188:EE_ X-MS-Office365-Filtering-Correlation-Id: c808120b-5d43-4730-2734-08dc8eee8087 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|36860700010|376011|1800799021|82310400023; X-Microsoft-Antispam-Message-Info: X6IkjN0JzNsSZ5WVsb8PR/XZDL2Yiq90573zHxK8+t0Ay6+fP0JxEVdLOJl1k9Yf7lqdxaKD4/AArN87FVJtca69ZVC5CXV96zuW44+CzWGBRGLgIQLq7c1BbkgX5aSMHkQhbY3cHWPX5XzTjshwQQU6AZYhxnWibjbczjIW2+HQ9MbyGNlZUMKIAPPA5uOXMZL0ncvz8yoloQbvwAEQyhB4fvRJfi8HnVaIyUT9ys1wVnuI3t0lC2uu7rW6MtSZX+TE+7PzxoIN3/4jeFDjEaba+m7f6YKf25oY0+jx+HPZ7nD3OZi181RHC4wS0K7GWgAr4wRO/kmP1h+R7BtYST8A/8suqjnsYce67r91kFgOMP/FEWZHNQZDd5yyj7CoFWpdFktbXMJK6HyRut6ENe2MhDbxiao6Crod+pACJRPNSQxRt9XitubrvNi46M+5RejT0FbglQ/VCqcwC9L46dkGj7amWar26qVb2F2l577OyY/rcyLPuHYX5uQmyyTSF3ae/pT9Hqx+ORO0jqgeVbnsRjteeUZ61Sr4jcOEgnv8qAbkRWlbdJxnqQdoPYL0NXUsRJk4v6oBpA0n/exjuuMjrD5GhcRyQfMRvdL5qW4TkJ0b591DU1tw8gLcBt6cZaGdO+yBAeDnQPFtjmZ06r8Kb467KxuHnfME4fid4dl7bUrE+8zEPP85uCFD3JUbSiU24ptVC14AzPBiRxXdFJo0nV/EpadZUsjwngOG5iYnnkvVFYxOycSfSXdz4a7MNpCk5eTWRLhVE8erF3Byg6Atb6GbcC0lHhDEYVwraRWZclw/f9Riucn8aKlL5fKUuw/kwqPRKkat0FI/QyPar0KV2D01VIWC7GxDZ0LPQYsJUMcOxCwaU8rz9moogeVcWvUBIdVypxfsqZEH+cXp6AYJRFlfgJTW5B3tx384siE7WquWqL5gCLJXDCjB8BOy6mknzUNCMoQTz5yg/Zlv7I2t325Tg0HQ8KI7tK+8JWRDCAG57TuR+ItpUQ/Ak5rKDCUpG8UsIyR+nVnG1uKNCejJj2gTEWQDKXAPyriJDInjyywSTIrUcFzGkEEaUr8T9bZTz0eHL3AUFH+rIP2egX2qryJJKpojQYgvYCrzPPXcUeJ8D0FSX9IVFx2i1sjapRx2YDpLguvrHVPaQolAJj06YiJcLKezRvtAaE4p1FUGiOVNYvduUmHDrcYmK50P1MXUFG6FPS7bNSemw6oRDggrB/R7KP4ef9aE3hpYwO8y90WemmdiR3+MT89mp79ejZe79lQQ8OM+YWoeqE0SVWDSv8WJyfVVwZqGTSiseuBhs3Awl3P0BeANA0lkCgC580RsBu5inSpuY4Ffg7BYjhv0YzBA4wJR/xg/QJpzBSOzhb//bkHEocvrX+H3P3zNbu6wCsdHQYIz7fNw5qfc+Q== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230037)(36860700010)(376011)(1800799021)(82310400023);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2024 16:56:58.9313 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c808120b-5d43-4730-2734-08dc8eee8087 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001D6.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8188 X-Patchwork-Delegate: kuba@kernel.org From: Ido Schimmel Cited commit added support for a new reset flow ("all reset") which is deeper than the existing reset flow ("software reset") and allows the device's PCI firmware to be upgraded. In the new flow the driver first tells the firmware that "all reset" is required by issuing a new reset command (i.e., MRSR.command=6) and then triggers the reset by having the PCI core issue a secondary bus reset (SBR). However, due to a race condition in the device's firmware the device is not always able to recover from this reset, resulting in initialization failures [1]. New firmware versions include a fix for the bug and advertise it using a new capability bit in the Management Capabilities Mask (MCAM) register. Avoid initialization failures by reading the new capability bit and triggering the new reset flow only if the bit is set. If the bit is not set, trigger a normal PCI hot reset by skipping the call to the Management Reset and Shutdown Register (MRSR). Normal PCI hot reset is weaker than "all reset", but it results in a fully operational driver and allows users to flash a new firmware, if they want to. [1] mlxsw_spectrum4 0000:01:00.0: not ready 1023ms after bus reset; waiting mlxsw_spectrum4 0000:01:00.0: not ready 2047ms after bus reset; waiting mlxsw_spectrum4 0000:01:00.0: not ready 4095ms after bus reset; waiting mlxsw_spectrum4 0000:01:00.0: not ready 8191ms after bus reset; waiting mlxsw_spectrum4 0000:01:00.0: not ready 16383ms after bus reset; waiting mlxsw_spectrum4 0000:01:00.0: not ready 32767ms after bus reset; waiting mlxsw_spectrum4 0000:01:00.0: not ready 65535ms after bus reset; giving up mlxsw_spectrum4 0000:01:00.0: PCI function reset failed with -25 mlxsw_spectrum4 0000:01:00.0: cannot register bus device mlxsw_spectrum4: probe of 0000:01:00.0 failed with error -25 Fixes: f257c73e5356 ("mlxsw: pci: Add support for new reset flow") Cc: Simon Horman Reported-by: Maksym Yaremchuk Signed-off-by: Ido Schimmel Tested-by: Maksym Yaremchuk Signed-off-by: Petr Machata Reviewed-by: Simon Horman --- drivers/net/ethernet/mellanox/mlxsw/pci.c | 18 +++++++++++++++--- drivers/net/ethernet/mellanox/mlxsw/reg.h | 2 ++ 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlxsw/pci.c b/drivers/net/ethernet/mellanox/mlxsw/pci.c index bf66d996e32e..c0ced4d315f3 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/pci.c +++ b/drivers/net/ethernet/mellanox/mlxsw/pci.c @@ -1594,18 +1594,25 @@ static int mlxsw_pci_sys_ready_wait(struct mlxsw_pci *mlxsw_pci, return -EBUSY; } -static int mlxsw_pci_reset_at_pci_disable(struct mlxsw_pci *mlxsw_pci) +static int mlxsw_pci_reset_at_pci_disable(struct mlxsw_pci *mlxsw_pci, + bool pci_reset_sbr_supported) { struct pci_dev *pdev = mlxsw_pci->pdev; char mrsr_pl[MLXSW_REG_MRSR_LEN]; int err; + if (!pci_reset_sbr_supported) { + pci_dbg(pdev, "Performing PCI hot reset instead of \"all reset\"\n"); + goto sbr; + } + mlxsw_reg_mrsr_pack(mrsr_pl, MLXSW_REG_MRSR_COMMAND_RESET_AT_PCI_DISABLE); err = mlxsw_reg_write(mlxsw_pci->core, MLXSW_REG(mrsr), mrsr_pl); if (err) return err; +sbr: device_lock_assert(&pdev->dev); pci_cfg_access_lock(pdev); @@ -1633,6 +1640,7 @@ static int mlxsw_pci_reset(struct mlxsw_pci *mlxsw_pci, const struct pci_device_id *id) { struct pci_dev *pdev = mlxsw_pci->pdev; + bool pci_reset_sbr_supported = false; char mcam_pl[MLXSW_REG_MCAM_LEN]; bool pci_reset_supported = false; u32 sys_status; @@ -1652,13 +1660,17 @@ mlxsw_pci_reset(struct mlxsw_pci *mlxsw_pci, const struct pci_device_id *id) mlxsw_reg_mcam_pack(mcam_pl, MLXSW_REG_MCAM_FEATURE_GROUP_ENHANCED_FEATURES); err = mlxsw_reg_query(mlxsw_pci->core, MLXSW_REG(mcam), mcam_pl); - if (!err) + if (!err) { mlxsw_reg_mcam_unpack(mcam_pl, MLXSW_REG_MCAM_PCI_RESET, &pci_reset_supported); + mlxsw_reg_mcam_unpack(mcam_pl, MLXSW_REG_MCAM_PCI_RESET_SBR, + &pci_reset_sbr_supported); + } if (pci_reset_supported) { pci_dbg(pdev, "Starting PCI reset flow\n"); - err = mlxsw_pci_reset_at_pci_disable(mlxsw_pci); + err = mlxsw_pci_reset_at_pci_disable(mlxsw_pci, + pci_reset_sbr_supported); } else { pci_dbg(pdev, "Starting software reset flow\n"); err = mlxsw_pci_reset_sw(mlxsw_pci); diff --git a/drivers/net/ethernet/mellanox/mlxsw/reg.h b/drivers/net/ethernet/mellanox/mlxsw/reg.h index 8adf86a6f5cc..3bb89045eaf5 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/reg.h +++ b/drivers/net/ethernet/mellanox/mlxsw/reg.h @@ -10671,6 +10671,8 @@ enum mlxsw_reg_mcam_mng_feature_cap_mask_bits { MLXSW_REG_MCAM_MCIA_128B = 34, /* If set, MRSR.command=6 is supported. */ MLXSW_REG_MCAM_PCI_RESET = 48, + /* If set, MRSR.command=6 is supported with Secondary Bus Reset. */ + MLXSW_REG_MCAM_PCI_RESET_SBR = 67, }; #define MLXSW_REG_BYTES_PER_DWORD 0x4 From patchwork Mon Jun 17 16:56:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Petr Machata X-Patchwork-Id: 13701083 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2063.outbound.protection.outlook.com [40.107.100.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78B26166317 for ; Mon, 17 Jun 2024 16:57:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.63 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718643428; cv=fail; b=dssRZJbKXkmvqRfbbRzvo6J7rj82vVG5d3Kij/CqNUBWOupN3ITsYiShJFebLVrj6Vs7aDJpF5V1ZrGrc4feDYxU16pqzjRLbxs91+1Ii+jdjU9t3kJDSdJT1sakOFnCqwiSml59ETdNrubGg6KeZM6zXgYxGiVQj2vxi+NKIEc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718643428; c=relaxed/simple; bh=KE2uzoEXHNSOvwsI/71hk1GIC4UvhSUBEKV+olAriq8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ObJOYvy0MkWDuCCku69j9uA4TwwYJZYsFtzziU8EalmWV0Vjm5z41pliHPWnsEyehTkmDCxpYB7WBRX7+UnpsxC9puBh6RCmkQx0uMkk0EHcICVgzV8HSyINcw+wSiczaPt9q5fgY6nbJUnVoaI3oYVUlgS/FWhl3MMq4wnXvF8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=CgqPw4U7; arc=fail smtp.client-ip=40.107.100.63 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="CgqPw4U7" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TLpeh/2N32ERAKb4lUZV1pvFBkA7vWfnZEVyB2QyucMarmQDA2qFIWfZsWTudXczrMsXn7wUL3FhbV1bE72QDMukkV3qyJXKpZeZn6RqNd46+XuRkJOSH13Q2dgS+Fh2+FlFadsNBk72Ud84XCv5QT4l5E73z0ZJL5scFoYs83PxyeWdBrSmBz6WjVMdQ3L/hOc8gNqKhhk6l/i0AhCIxGC6eg12zVG3X1Nj0aoZUXs1oxCgn68XLXCiq/rSN5+dLujqCJlGOnYrpcd3bX9dVYfO89UuQsPJRY8RTprWT9BXJPBKutnA91HUAMB60QFJjnNMBx2SCUwbvwhULR2f/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/IQmdjyyEz+O+y9khsISw09DC1ZRHIO5lc37l+N9xh4=; b=AWk51uBi4yk591XZO4fAfbwS4oZcS4mABURHY8TAJBZ5DKEMQ2xtqcw3Sblu8DDYCf/7ryRmZBPntrQpz5NRDW4HURJqoJ6UrfwpMAGF2qrSA/gmOeXAPoyl+x8OoL55JSqH9pRGjLUHV7iDMWIs1QNWyHZbTd5hYtIbuExVAsltkUTIF3Mt94E6VkYP/StJu7GmpOFCuhRICOdKQLdR5tJV/L5jeZ4hi6J0eNoTRIjFf4cnrgK6dOjbE0P8UuEl8Y83b7wIPpf9e/Hisi2S4UosSrCn3hNiSorz8CAPaDBTWcqWz2/8RXqB5J+FscXVPhNSKm0J20N9niTKyiUuQg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/IQmdjyyEz+O+y9khsISw09DC1ZRHIO5lc37l+N9xh4=; b=CgqPw4U7f31KBZY3V5+PLpsmoS+MAobXOaOeAK2yjrM/LmtNNfb8ytV38JPKxcRCpDU9nNWFk0E+FYFpiR4Wt2/6d9gvxQgGIRPUPKjf8+QzYexq7vgNQX78762XaLqyjrliQ5fxK2bBLJJ4l7NPctXdUtaoOsWADz9EH4hIE6JTNr/DLIZFX/Xk6WWU8tMlvPqS8d87VwEulgvY04WAJRU8vcD+whMlCvcdXBpFWQ2qz78Hpto/tcCfvuSeWOeiUj7vLTQxYKjWLObf890LOFCp1ROzCG9bTKw4oPSZIzKJUjKqazBR9LFdaEdmnxK4Gr7yHEdOaE/ifTzh9zzy9w== Received: from SJ0PR03CA0076.namprd03.prod.outlook.com (2603:10b6:a03:331::21) by SN7PR12MB7105.namprd12.prod.outlook.com (2603:10b6:806:2a0::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Mon, 17 Jun 2024 16:57:02 +0000 Received: from SJ5PEPF000001D2.namprd05.prod.outlook.com (2603:10b6:a03:331:cafe::4) by SJ0PR03CA0076.outlook.office365.com (2603:10b6:a03:331::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30 via Frontend Transport; Mon, 17 Jun 2024 16:57:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SJ5PEPF000001D2.mail.protection.outlook.com (10.167.242.54) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.15 via Frontend Transport; Mon, 17 Jun 2024 16:57:02 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 17 Jun 2024 09:56:48 -0700 Received: from yaviefel.mtl.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 17 Jun 2024 09:56:43 -0700 From: Petr Machata To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , CC: Ido Schimmel , Petr Machata , , "Rafael J. Wysocki" , "Lukasz Luba" , Daniel Lezcano , "Vadim Pasternak" Subject: [PATCH net 2/3] mlxsw: core_thermal: Fix driver initialization failure Date: Mon, 17 Jun 2024 18:56:01 +0200 Message-ID: X-Mailer: git-send-email 2.44.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001D2:EE_|SN7PR12MB7105:EE_ X-MS-Office365-Filtering-Correlation-Id: 1892c522-275d-4022-fe61-08dc8eee82b6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|1800799021|82310400023|376011|36860700010; X-Microsoft-Antispam-Message-Info: iFJUCiL5Ak163/plwHVWC4KVuFDkuCewqHQYLpkEvbX0DDpXaMnxdHd+y6ap6OkxFlCtcd/8NpZxgaABfXKrhxWY7cF+WISdQANd0pISL8vSA436cBqWpRA554V4OTipXR081KWrurm3O1I/OHw2JrMv1+W2TsErOwAuLhG39OcOUptObvlpmnSeLnvPF247oG6uLOVjNn/ozaH5K42mmNB4gwbSDSrP7x4uWvLMiiUzXFQ0wGHYZ5f2KAYnBmIURjcltatYCcQePNRQGFKpoy1PJpy7gmq3gvJhPvPnZ5IW59iUU7fmI+wNskb+wqxHP9TgXgwGi1AVmGLoLwBh2fRQWed82qKSEzdmCQCcJ2MHZNVrxYWWMQxUA4njV3iYJeqMfvBgHQXaYI2rWtVavLeNMr+kbQot62TlVKGbV2kOtTxYI1BA6dxEsP//hYuPM1nDmEZYzhxzGfGMAYW6Uy8kcp9lA2kK/Bpp1y6zgRwKqeIik1yQDzkhTjj2CLJn7ig1o5Z8D8qpcHEmo+rHRncFTj6qUI8wNmGtKnTrlKJfC+u0YzeSSVJmx/ryUKyOcEOcFjeXgZgmnvAEeyQBN46Be3734rj17fwAiINCdrvlewC2h6VeOxPufJ83YT1spCvwwDJWe/kf35gh6CBJFp5Pq2BARvdTMA+3NT3absuwlcis5sPNlh1MCzr43yGynED0TE2AXU5qQj+fciiEZPXB9QNW+/jn67oQgTK8I79dpfdl6PXY4iiVQnG2ITYPxCMUodePB7fWJ//vmIINAY/ohKG/VJG4Y4FcU3pASUJjAS2K0srzmSn8NibYeJiZ9w3PR6Tr2Ix/QgQ1iROagbS0fJ6tSiCjT1ghbYt0kCNSb4FyxZugruvPl+5I/He5w+2X9bsDnk0tk8zI88fJw8JSbwoaamMYpI3oMcwsEJJTY9LoWYST7vhusHSrx8BGOcqjCuWMP3OjGhL6q2APOk9fIsjlwxvff+K0uWke8AEmBIYuQyQwFnMFNq4oaVbJqjMTMJAM1cq/39CkfEw9SjNfFO5VNB/tFqb/aKmrcUoZkUV63HQq7iGvLh+OZnZvI/dhYbootA4MlB9J2/NgqEYdOajmQue7Hfvp33RhCZjMrc02TO2d4M+tZ+kBy5/jcA0pxqqjrz0iNAg9Hlqe4it0RH3Av4O0TwcXZBdzXiB2NCmo9EaXm6RQMwC/oeOCz92gBxcZRBtvZQeOJ+IWZKNTGunUyC4jWbhG4ml5AoYuIFKAXlSCdO6bUJeqPihbt+QFEs+EN6qWlGUzdbe20cnE8F+vN4ivSNgYo1Bi7vUVtAbNdk0A0UmbE0VyyOYnCiFWd+WvDq42YYUdrD/l8v+IJT12l4pb4/CSwbtQqLZP5dGVBfW2WAO7gNMOOT0VsP2tetMZotVQWdZQdj502Q== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230037)(1800799021)(82310400023)(376011)(36860700010);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2024 16:57:02.6002 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1892c522-275d-4022-fe61-08dc8eee82b6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001D2.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7105 X-Patchwork-Delegate: kuba@kernel.org From: Ido Schimmel Commit 31a0fa0019b0 ("thermal/debugfs: Pass cooling device state to thermal_debug_cdev_add()") changed the thermal core to read the current state of the cooling device as part of the cooling device's registration. This is incompatible with the current implementation of the cooling device operations in mlxsw, leading to initialization failure with errors such as: mlxsw_spectrum 0000:01:00.0: Failed to register cooling device mlxsw_spectrum 0000:01:00.0: cannot register bus device The reason for the failure is that when the get current state operation is invoked the driver tries to derive the index of the cooling device by walking a per thermal zone array and looking for the matching cooling device pointer. However, the pointer is returned from the registration function and therefore only set in the array after the registration. Fix by passing to the registration function a per cooling device private data that already has the cooling device index populated. Decided to fix the issue in the driver since as far as I can tell other drivers do not suffer from this problem. Fixes: 31a0fa0019b0 ("thermal/debugfs: Pass cooling device state to thermal_debug_cdev_add()") Fixes: 755113d76786 ("thermal/debugfs: Add thermal cooling device debugfs information") Cc: "Rafael J. Wysocki" Cc: Lukasz Luba Cc: Daniel Lezcano Signed-off-by: Ido Schimmel Reviewed-by: Vadim Pasternak Signed-off-by: Petr Machata --- .../ethernet/mellanox/mlxsw/core_thermal.c | 50 ++++++++++--------- 1 file changed, 26 insertions(+), 24 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_thermal.c b/drivers/net/ethernet/mellanox/mlxsw/core_thermal.c index 5c511e1a8efa..eee3e37983ca 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core_thermal.c +++ b/drivers/net/ethernet/mellanox/mlxsw/core_thermal.c @@ -100,6 +100,12 @@ static const struct mlxsw_cooling_states default_cooling_states[] = { struct mlxsw_thermal; +struct mlxsw_thermal_cooling_device { + struct mlxsw_thermal *thermal; + struct thermal_cooling_device *cdev; + unsigned int idx; +}; + struct mlxsw_thermal_module { struct mlxsw_thermal *parent; struct thermal_zone_device *tzdev; @@ -123,7 +129,7 @@ struct mlxsw_thermal { const struct mlxsw_bus_info *bus_info; struct thermal_zone_device *tzdev; int polling_delay; - struct thermal_cooling_device *cdevs[MLXSW_MFCR_PWMS_MAX]; + struct mlxsw_thermal_cooling_device cdevs[MLXSW_MFCR_PWMS_MAX]; struct thermal_trip trips[MLXSW_THERMAL_NUM_TRIPS]; struct mlxsw_cooling_states cooling_states[MLXSW_THERMAL_NUM_TRIPS]; struct mlxsw_thermal_area line_cards[]; @@ -147,7 +153,7 @@ static int mlxsw_get_cooling_device_idx(struct mlxsw_thermal *thermal, int i; for (i = 0; i < MLXSW_MFCR_PWMS_MAX; i++) - if (thermal->cdevs[i] == cdev) + if (thermal->cdevs[i].cdev == cdev) return i; /* Allow mlxsw thermal zone binding to an external cooling device */ @@ -352,17 +358,14 @@ static int mlxsw_thermal_get_cur_state(struct thermal_cooling_device *cdev, unsigned long *p_state) { - struct mlxsw_thermal *thermal = cdev->devdata; + struct mlxsw_thermal_cooling_device *mlxsw_cdev = cdev->devdata; + struct mlxsw_thermal *thermal = mlxsw_cdev->thermal; struct device *dev = thermal->bus_info->dev; char mfsc_pl[MLXSW_REG_MFSC_LEN]; - int err, idx; u8 duty; + int err; - idx = mlxsw_get_cooling_device_idx(thermal, cdev); - if (idx < 0) - return idx; - - mlxsw_reg_mfsc_pack(mfsc_pl, idx, 0); + mlxsw_reg_mfsc_pack(mfsc_pl, mlxsw_cdev->idx, 0); err = mlxsw_reg_query(thermal->core, MLXSW_REG(mfsc), mfsc_pl); if (err) { dev_err(dev, "Failed to query PWM duty\n"); @@ -378,22 +381,19 @@ static int mlxsw_thermal_set_cur_state(struct thermal_cooling_device *cdev, unsigned long state) { - struct mlxsw_thermal *thermal = cdev->devdata; + struct mlxsw_thermal_cooling_device *mlxsw_cdev = cdev->devdata; + struct mlxsw_thermal *thermal = mlxsw_cdev->thermal; struct device *dev = thermal->bus_info->dev; char mfsc_pl[MLXSW_REG_MFSC_LEN]; - int idx; int err; if (state > MLXSW_THERMAL_MAX_STATE) return -EINVAL; - idx = mlxsw_get_cooling_device_idx(thermal, cdev); - if (idx < 0) - return idx; - /* Normalize the state to the valid speed range. */ state = max_t(unsigned long, MLXSW_THERMAL_MIN_STATE, state); - mlxsw_reg_mfsc_pack(mfsc_pl, idx, mlxsw_state_to_duty(state)); + mlxsw_reg_mfsc_pack(mfsc_pl, mlxsw_cdev->idx, + mlxsw_state_to_duty(state)); err = mlxsw_reg_write(thermal->core, MLXSW_REG(mfsc), mfsc_pl); if (err) { dev_err(dev, "Failed to write PWM duty\n"); @@ -753,17 +753,21 @@ int mlxsw_thermal_init(struct mlxsw_core *core, } for (i = 0; i < MLXSW_MFCR_PWMS_MAX; i++) { if (pwm_active & BIT(i)) { + struct mlxsw_thermal_cooling_device *mlxsw_cdev; struct thermal_cooling_device *cdev; + mlxsw_cdev = &thermal->cdevs[i]; + mlxsw_cdev->thermal = thermal; + mlxsw_cdev->idx = i; cdev = thermal_cooling_device_register("mlxsw_fan", - thermal, + mlxsw_cdev, &mlxsw_cooling_ops); if (IS_ERR(cdev)) { err = PTR_ERR(cdev); dev_err(dev, "Failed to register cooling device\n"); goto err_thermal_cooling_device_register; } - thermal->cdevs[i] = cdev; + mlxsw_cdev->cdev = cdev; } } @@ -824,8 +828,8 @@ int mlxsw_thermal_init(struct mlxsw_core *core, err_thermal_zone_device_register: err_thermal_cooling_device_register: for (i = 0; i < MLXSW_MFCR_PWMS_MAX; i++) - if (thermal->cdevs[i]) - thermal_cooling_device_unregister(thermal->cdevs[i]); + if (thermal->cdevs[i].cdev) + thermal_cooling_device_unregister(thermal->cdevs[i].cdev); err_reg_write: err_reg_query: kfree(thermal); @@ -848,10 +852,8 @@ void mlxsw_thermal_fini(struct mlxsw_thermal *thermal) } for (i = 0; i < MLXSW_MFCR_PWMS_MAX; i++) { - if (thermal->cdevs[i]) { - thermal_cooling_device_unregister(thermal->cdevs[i]); - thermal->cdevs[i] = NULL; - } + if (thermal->cdevs[i].cdev) + thermal_cooling_device_unregister(thermal->cdevs[i].cdev); } kfree(thermal); From patchwork Mon Jun 17 16:56:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Petr Machata X-Patchwork-Id: 13701084 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2059.outbound.protection.outlook.com [40.107.223.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7AC816849D for ; Mon, 17 Jun 2024 16:57:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.59 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718643438; cv=fail; b=d2BhQwDPd4rrCarP8RED2ARpLiTBbu6mDVsZca+WkQrCbROrvRmJ3GPZjXboZifIyGA4Q3EFMYZ9qC4iwuNK74yTxVjwwxal6cS082j4IEn/U/Wmc2veXXLXgdPWz1Dnhtce+TSwE+uwYv9p5AImDw9erHwkvQTULu7oyU2TkdY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718643438; c=relaxed/simple; bh=gnP5De53z5NbYJp3+0NQ0s2+4ccCBgs/brSelbGks2I=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QgVTFIpl3r6Jcrc6XJyDEMM7UuLzDanF17Dm59zxDl8HxZG3BmisySlXiSz0dqkOOjlED99UjYXYys1nAQNXtp5zWTy79vYOg+qGP+3iKEZmbO5HCZL1qEajB2WJUDR9ml/sVlyWlfwPKtxLyjRWkmeWQTiwH+Iz8iycXlKm4bE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=d1GNOxxv; arc=fail smtp.client-ip=40.107.223.59 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="d1GNOxxv" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WVfq5J9KF6vlmd9j5dlNcK4YFeLot9gEp2cIfzLOLcXIbHBYozzDaqs3ifDHoBgYgidduGExqGbSQuDx+IKoHKqLoSDm62YcZ44zgng23qgAyRCZ4buxSwVsC1zrSbZQ19T+v2O31ehN7o8SdN6zW3I3AWB6IvzXvyn2VSbtr7qLWq5URMY8T+rzOormcCzn/NDi9iQ+fdqwRXU84Fwq52kvOk6ApdcUMI3tZw58Dqwpq4GuL8lUdV2J2Ph2fnfi8I+tiPwb7Dq/ApFwpSmJ+5rcco+TWgSnZhWJoPIbJpLkxV5obXplebtoPvv3uNVywG5qKlaunRgn3fQgHpD9qw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VOqlK/ILJV5Vig/F/104Gdns3eKOE6tH+3Y1V78+afs=; b=kOJi3WsFeKOAwmPDDmPJwgA1vVxHKr849InrNqNJnpvVa+gcY2QSV173cs5mHJyUdXWW6VDeOOiVpFzZoHoC6x63UU6QW9b0yoS6lOdDbG6orXCylbR8NlMNrmGt5Nasndyurn+lXUYbKNPHLAT4ic/jz5z9/w3Vi2dOxZ5UBdzun7YOWJREmgQd04vWMY0u1gFFOTp2A84h6d9pkNMcfK3oob/QBEWke/sc6JFAV7jobi4HJDm8At4StXJG5ERgp6Z84pVuT2oyCFjPjjy/tSLrveTr0U9Fs0AIj0PnqMr/S/A7V5Dc2pgTOij9F85c1AJrMMnb+e34QODsZvDx6g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VOqlK/ILJV5Vig/F/104Gdns3eKOE6tH+3Y1V78+afs=; b=d1GNOxxvMDbecO9Rr119LBrTYwt9QdN+E0VYAs3mmQdikq3bnqXtBs/NVRRm2tkOoOGV1ADdLP4KlY7JcX/vcVSUVrDyOwkJ87BjZ78wrWiNWdNmLoY2Q1x3gbwfmOwU582D7Md4EZlgCMDGM+VbeimNlqvPZpHSl1tYTDKf3tuWrBODmCEoAlJ9nEtCL0INGk+O6o7T3j6FmzByUo+3MFEe95bYYjfKUh8ovqizOtv1nnf46k7NJbDdCd3TviNGpl3Nqrrw9vXXF9p75pW8ktlv1UiaFYd1aNpPiGFL6O2ig0ljscuhk1SexfP2tl3aJngpx+Bmin2vDs4kA6TZEg== Received: from CH0PR03CA0262.namprd03.prod.outlook.com (2603:10b6:610:e5::27) by MW4PR12MB7143.namprd12.prod.outlook.com (2603:10b6:303:222::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.30; Mon, 17 Jun 2024 16:57:10 +0000 Received: from DS2PEPF00003442.namprd04.prod.outlook.com (2603:10b6:610:e5:cafe::f8) by CH0PR03CA0262.outlook.office365.com (2603:10b6:610:e5::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.31 via Frontend Transport; Mon, 17 Jun 2024 16:57:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003442.mail.protection.outlook.com (10.167.17.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.15 via Frontend Transport; Mon, 17 Jun 2024 16:57:08 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 17 Jun 2024 09:56:52 -0700 Received: from yaviefel.mtl.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 17 Jun 2024 09:56:48 -0700 From: Petr Machata To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , CC: Ido Schimmel , Petr Machata , , Amit Cohen Subject: [PATCH net 3/3] mlxsw: spectrum_buffers: Fix memory corruptions on Spectrum-4 systems Date: Mon, 17 Jun 2024 18:56:02 +0200 Message-ID: X-Mailer: git-send-email 2.44.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003442:EE_|MW4PR12MB7143:EE_ X-MS-Office365-Filtering-Correlation-Id: d74f74e8-6a7b-48d9-3bdf-08dc8eee8699 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|36860700010|376011|82310400023|1800799021; X-Microsoft-Antispam-Message-Info: 9e2nEHFBxIFo1w+mtVfJohzWIaeh0XaOwH6n38rTGoOw8hcl7f7TYkvVP3WmDaB41r45ptUfeYf76+9NQz+r24KOeJdtapIrK4YxmX+kBigyjTXu8n5+PHbsrBz9367kzaL+2JBzWVsc66SeDbo2XeMLVDWQLSDBVtZrWQiHUANCKIRulRxSRYrJjPYxHgBsoCSsaJECewcmTs9Tm9nWxd4WeT0VzLQPBWhc/MTFJyxgpjIszXRKtniqbfTXPG3rGlFPtdJVbGpVKlBpd1VT2gNubvpi/Ognse4XJk+L5cqmIXMWz7SJow98E1KvS1xLmqGzC5j96/9NHMywHDYeVrn2YEWB3SNF6uyNBmFyQvJ5f5lc1y6hBoXFNZGUV6/LzcVk1uka8Q/x8lG7s1MA2eeNMzFaSa19c8YnX78D41qTvKygcDbAy5451psqioy/JIZ2l5Ji2d/ddD0Pyf+25Ka4fH+GDq2ePtlT+LBew28SpQgC+gHutY0ENRX0G3d6Rm/D+F2gA1XLccxPtVa/CtzhQp90V6x+GRPeCnxWCRYq8wr+NVlPccLT4kCm3jFSatEMFLpmUCKwzPSQ0INpsiBgHsluaf+CpxBIm80zjIy3uJSwMwS+OjCEXO/SySV3/i/dj+2SkBnTtiIp8UbrJIPZulFOCI3INfdAxKNGmLGLvjiP/aVE84VfWLKSmonkzschrZKfgvKGuxMpZMcSSKPuToqXMCv4BHA7T0Xe5bo7JM85iGa+MFVLaXduWxIrwpyAJ+kTUfJCuLeB4xLf/2bRsd+xvW8gFumVm9s3CP8X6AORCiiJW4V6HN2tYuqzRNH/L6sbWTvWeegpsiDB1KdkBOmt/piZdV5RMF4AcvdVBO1dRu3tzSlyTnNUEumcnfKS3r65mpEDQ9/ylZl+aY7p5kjn9TLexGuwdv1/ht6qkS7HiYWFgWJHMzXTij6hHisuHGtn8hPJzBXqFR72KdBTnRDxXTZ+EKqScUiK0x/86QrIGZILt8lCsbKkjuyk9aTa1n54fLus57HvU6SvVh/ZkF7glkxPmoh8miS2ms0y1emHS1s9e2JtJuxn+siQ3eyj5MjWr1I9X9Yj7GOe3Yea8hIfL7OcxnKwkizZL7aBfvw8ptkG5lYHPfDHxupOAefTn3XodmThOYbs+wy3V0YlsPe4v5s1jDrTU9ThXf00WwGwfQ7QOxunPq71R73oCjtuQC0SukxV2Lfmd2iNMBXzVMcLoyfo3YjZ7mDj9/7ikt0GDjixFZ7LubruFlTyehVyGmAhBGOLME+E7xfPyxAnJ2BfCDXOBu6wDnIzAZnrcLh9BQCPJkydqcz+g5steud2yBwTE55PoQgieu8wXn3WbK49zOuGPhI3Q/lUzIcJkQ1XXDQ6CwQMjTQcgayuJA1wXal3SBK81680YONHkw== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230037)(36860700010)(376011)(82310400023)(1800799021);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2024 16:57:08.9755 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d74f74e8-6a7b-48d9-3bdf-08dc8eee8699 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003442.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7143 X-Patchwork-Delegate: kuba@kernel.org From: Ido Schimmel The following two shared buffer operations make use of the Shared Buffer Status Register (SBSR): # devlink sb occupancy snapshot pci/0000:01:00.0 # devlink sb occupancy clearmax pci/0000:01:00.0 The register has two masks of 256 bits to denote on which ingress / egress ports the register should operate on. Spectrum-4 has more than 256 ports, so the register was extended by cited commit with a new 'port_page' field. However, when filling the register's payload, the driver specifies the ports as absolute numbers and not relative to the first port of the port page, resulting in memory corruptions [1]. Fix by specifying the ports relative to the first port of the port page. [1] BUG: KASAN: slab-use-after-free in mlxsw_sp_sb_occ_snapshot+0xb6d/0xbc0 Read of size 1 at addr ffff8881068cb00f by task devlink/1566 [...] Call Trace: dump_stack_lvl+0xc6/0x120 print_report+0xce/0x670 kasan_report+0xd7/0x110 mlxsw_sp_sb_occ_snapshot+0xb6d/0xbc0 mlxsw_devlink_sb_occ_snapshot+0x75/0xb0 devlink_nl_sb_occ_snapshot_doit+0x1f9/0x2a0 genl_family_rcv_msg_doit+0x20c/0x300 genl_rcv_msg+0x567/0x800 netlink_rcv_skb+0x170/0x450 genl_rcv+0x2d/0x40 netlink_unicast+0x547/0x830 netlink_sendmsg+0x8d4/0xdb0 __sys_sendto+0x49b/0x510 __x64_sys_sendto+0xe5/0x1c0 do_syscall_64+0xc1/0x1d0 entry_SYSCALL_64_after_hwframe+0x77/0x7f [...] Allocated by task 1: kasan_save_stack+0x33/0x60 kasan_save_track+0x14/0x30 __kasan_kmalloc+0x8f/0xa0 copy_verifier_state+0xbc2/0xfb0 do_check_common+0x2c51/0xc7e0 bpf_check+0x5107/0x9960 bpf_prog_load+0xf0e/0x2690 __sys_bpf+0x1a61/0x49d0 __x64_sys_bpf+0x7d/0xc0 do_syscall_64+0xc1/0x1d0 entry_SYSCALL_64_after_hwframe+0x77/0x7f Freed by task 1: kasan_save_stack+0x33/0x60 kasan_save_track+0x14/0x30 kasan_save_free_info+0x3b/0x60 poison_slab_object+0x109/0x170 __kasan_slab_free+0x14/0x30 kfree+0xca/0x2b0 free_verifier_state+0xce/0x270 do_check_common+0x4828/0xc7e0 bpf_check+0x5107/0x9960 bpf_prog_load+0xf0e/0x2690 __sys_bpf+0x1a61/0x49d0 __x64_sys_bpf+0x7d/0xc0 do_syscall_64+0xc1/0x1d0 entry_SYSCALL_64_after_hwframe+0x77/0x7f Fixes: f8538aec88b4 ("mlxsw: Add support for more than 256 ports in SBSR register") Cc: Amit Cohen Signed-off-by: Ido Schimmel Reviewed-by: Petr Machata Signed-off-by: Petr Machata Reviewed-by: Simon Horman --- .../mellanox/mlxsw/spectrum_buffers.c | 20 +++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_buffers.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_buffers.c index 1b9ed393fbd4..2c0cfa79d138 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_buffers.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_buffers.c @@ -1611,8 +1611,8 @@ static void mlxsw_sp_sb_sr_occ_query_cb(struct mlxsw_core *mlxsw_core, int mlxsw_sp_sb_occ_snapshot(struct mlxsw_core *mlxsw_core, unsigned int sb_index) { + u16 local_port, local_port_1, first_local_port, last_local_port; struct mlxsw_sp *mlxsw_sp = mlxsw_core_driver_priv(mlxsw_core); - u16 local_port, local_port_1, last_local_port; struct mlxsw_sp_sb_sr_occ_query_cb_ctx cb_ctx; u8 masked_count, current_page = 0; unsigned long cb_priv = 0; @@ -1632,6 +1632,7 @@ int mlxsw_sp_sb_occ_snapshot(struct mlxsw_core *mlxsw_core, masked_count = 0; mlxsw_reg_sbsr_pack(sbsr_pl, false); mlxsw_reg_sbsr_port_page_set(sbsr_pl, current_page); + first_local_port = current_page * MLXSW_REG_SBSR_NUM_PORTS_IN_PAGE; last_local_port = current_page * MLXSW_REG_SBSR_NUM_PORTS_IN_PAGE + MLXSW_REG_SBSR_NUM_PORTS_IN_PAGE - 1; @@ -1649,9 +1650,12 @@ int mlxsw_sp_sb_occ_snapshot(struct mlxsw_core *mlxsw_core, if (local_port != MLXSW_PORT_CPU_PORT) { /* Ingress quotas are not supported for the CPU port */ mlxsw_reg_sbsr_ingress_port_mask_set(sbsr_pl, - local_port, 1); + local_port - first_local_port, + 1); } - mlxsw_reg_sbsr_egress_port_mask_set(sbsr_pl, local_port, 1); + mlxsw_reg_sbsr_egress_port_mask_set(sbsr_pl, + local_port - first_local_port, + 1); for (i = 0; i < mlxsw_sp->sb_vals->pool_count; i++) { err = mlxsw_sp_sb_pm_occ_query(mlxsw_sp, local_port, i, &bulk_list); @@ -1688,7 +1692,7 @@ int mlxsw_sp_sb_occ_max_clear(struct mlxsw_core *mlxsw_core, unsigned int sb_index) { struct mlxsw_sp *mlxsw_sp = mlxsw_core_driver_priv(mlxsw_core); - u16 local_port, last_local_port; + u16 local_port, first_local_port, last_local_port; LIST_HEAD(bulk_list); unsigned int masked_count; u8 current_page = 0; @@ -1706,6 +1710,7 @@ int mlxsw_sp_sb_occ_max_clear(struct mlxsw_core *mlxsw_core, masked_count = 0; mlxsw_reg_sbsr_pack(sbsr_pl, true); mlxsw_reg_sbsr_port_page_set(sbsr_pl, current_page); + first_local_port = current_page * MLXSW_REG_SBSR_NUM_PORTS_IN_PAGE; last_local_port = current_page * MLXSW_REG_SBSR_NUM_PORTS_IN_PAGE + MLXSW_REG_SBSR_NUM_PORTS_IN_PAGE - 1; @@ -1723,9 +1728,12 @@ int mlxsw_sp_sb_occ_max_clear(struct mlxsw_core *mlxsw_core, if (local_port != MLXSW_PORT_CPU_PORT) { /* Ingress quotas are not supported for the CPU port */ mlxsw_reg_sbsr_ingress_port_mask_set(sbsr_pl, - local_port, 1); + local_port - first_local_port, + 1); } - mlxsw_reg_sbsr_egress_port_mask_set(sbsr_pl, local_port, 1); + mlxsw_reg_sbsr_egress_port_mask_set(sbsr_pl, + local_port - first_local_port, + 1); for (i = 0; i < mlxsw_sp->sb_vals->pool_count; i++) { err = mlxsw_sp_sb_pm_occ_clear(mlxsw_sp, local_port, i, &bulk_list);