From patchwork Fri Aug 30 07:31:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13784442 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2057.outbound.protection.outlook.com [40.107.243.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EE31166F28 for ; Fri, 30 Aug 2024 07:31:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.57 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003114; cv=fail; b=sNDQKfA52zpV9T+PRgpxlx0Af4qQ4mT9QEAWk/OJkJrcV2CYiCZSgkurQfinDS37CiAQxBjNj5Cg6YTwEd62vvvT8O4f0GYEQexOzy9cLAz8d/8+wNCIdRAp1EwsLj7+mGQPO0aEX63qXK8lNVC5LhPMX6JL5BepU8ngcqNbLK4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003114; c=relaxed/simple; bh=EQFkdF98ImVIpiz+D5z6Qiggi1JO6HFXBAoFITwzt6c=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=JfpN2btoG3jS6bqCGupRefyeok8lPbDWC9OSKSkwhzVbtPFgdnnVUHXUFxfHPDCZZSMt3GFZ1PltZSLONcYFOwdmTg9+1ekwhRyRgK0ZjbrCayA/Jd1OZi5K6+jOpu2PeCU7vTSyRLwdpueMYoCbGYect6Z9IGFzyaOmfK2R3Kw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Il1AK3tq; arc=fail smtp.client-ip=40.107.243.57 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Il1AK3tq" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=EGdcJgQKEF53RgaYUyYc50Z7eURilnm8cWQef94CeGkJ3KRKPY/ro1ToEtsRJtPFj2QZTTq/5yqJM0BjgykGhIOo/JeFEsb5k/PML19Ua+8kQCBrc7ucWAl48X6etZsPMOuFI+KkDmgziF2zhIqKR01FVM/3u43QbfvH8l5DGGc1fvpVTahBzOFFh/rALIy3fBChn84VIAlMJvNjcg9sHSogMi0ZI7WhI/i+c106YqfV/HwIoHjDwy1mblWCU5thf+cuuEhBkbYzdlATm20nLdaRXqNDdYczg0qBuQ3ZXOS2m3vKqfXWqsPkuCVk34hxFyB6EhJqArqBNOtzmr4qSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cY+B4Nsuo7WtmAU1ohBNCsMm+kGOeHPENTxEA13abtY=; b=ZW1iLOuiPn4mVhwKsny1WkMdmGbAK0i+QePZ8153G9HeeHmisJXpeZmr+gPPQ2hbY/EAGfLPWa1+h2zUWOj1jGSSEkbazZtKZiWANKN0ueZrGN/jR0jamIGj6tref41/DeNBWm2Parocwi/y6TWBp+SaolRjAedIJjgfcfPWH96ljY4SYc3teR8yckbbeQry1jgU1b2yw6PWpiJ0pQvZGFd0EVL31KbNQr8RlS624HQhXsdRXX6k2NriEnb1wXOAuGAndf4PXNz5TgwKE4aMYA7a66mZlnIGcfpRqsqFTwU+GHX3ImCBapv1byKUep/ExFxTyDV//cFlruNN9Q92qg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cY+B4Nsuo7WtmAU1ohBNCsMm+kGOeHPENTxEA13abtY=; b=Il1AK3tql6AtJrvvbqN8Bd4xbcuhq8fspvT3yLPHIIgQPEWZuDlIPCPx9YOg2PkVRCKSWFd6c18garNwxnNM6749fstYW4L7Fq1AAMeQ9f9KMoVHQ4yC0EEIyRIB8ijrQW/XXq3cU8ulQp2VekVPSk8MMOGX/cwm1XK2jvucN5FBlhgm3QxdaYz9+1b0yISB1vLuRf3v5rVx3zGTjREOMosiToYkRmDuC8NXi/wv6VjFkYK3GRJyi04cjMr8+On/hreRMEr94/X37yq62J3yUGt+RvjLIVmI3WNQ75IcsfRbZHmbATciui10ijYNlpSfxMfVMx9hRRIB22yGUD3Tfw== Received: from BYAPR03CA0033.namprd03.prod.outlook.com (2603:10b6:a02:a8::46) by BL1PR12MB5873.namprd12.prod.outlook.com (2603:10b6:208:395::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.20; Fri, 30 Aug 2024 07:31:49 +0000 Received: from SJ5PEPF000001EF.namprd05.prod.outlook.com (2603:10b6:a02:a8:cafe::a1) by BYAPR03CA0033.outlook.office365.com (2603:10b6:a02:a8::46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.28 via Frontend Transport; Fri, 30 Aug 2024 07:31:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ5PEPF000001EF.mail.protection.outlook.com (10.167.242.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.13 via Frontend Transport; Fri, 30 Aug 2024 07:31:49 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:37 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:37 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 30 Aug 2024 00:31:35 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next v2 1/7] RDMA/mlx5: Check RoCE LAG status before getting netdev Date: Fri, 30 Aug 2024 10:31:24 +0300 Message-ID: <20240830073130.29982-2-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240830073130.29982-1-michaelgur@nvidia.com> References: <20240830073130.29982-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001EF:EE_|BL1PR12MB5873:EE_ X-MS-Office365-Filtering-Correlation-Id: 0f9ca0db-f13a-4dad-57ad-08dcc8c5cf90 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: 96FLl789/VRO6gAxU+GqTYgW/nIIgXYic4Knrg1u8oAOrw8eDCkAdlRcx1ZgAbbwri2S0HIQAwnJoUsD2JFxr6cbSpsHo6I7tdYhELLQRjhPOaoFaaocozzOi25VJn83EtlwIcdAin+Rbo+dICFLavB3yx1gXAEMZJf9avdD6bQCi8S0krgi3wBkHn+owmgZ6/YmseGdsqzTVmCqHGzdp96BX5536OXNzBOwdUZWFe3jpL2JI0waOqXeOHEWXA0rYSwY/FE0fUd0bVQi4Nx5OCHA/zGDuZ1y7hYODWEBl4UCtCIo4tk6Mjob3QN7iqLmQGfDPU/nT8kESqmKkEaSKSIh0gswPt5izqf9wCta56X8QORIte03339Dvace08m5E2ZQEuMV+WaTRQe340ZTmyGkmNkK5vSDhkHUNzws+6vtuKjw5QZrbWhT7sIX9wJu6mnXwKXYzQ3gTe3TXS3NlTSpBRGop4ZpqIRg22azUfPHUqIwxOY87KHkM+zjSQVy4eRzD8k2x1fURuYvKGe2I+KfanjbADTK3x/4iUqfVUfWRajT0acZHfbIxGw7ggPwMgUFgbvSHxu5EhXMN/AWbi6MQkzFZq0Uax8QzLz3Sw+qNWvLrKMgIOxAeA08xjXt8SaE/OJKPZMCDXSq/yB2VoFZ28XxYWfCbnPcTk58CCm0gwtwWl8dQLCNh/+4WI7Y3e2F9D7HZHx3FGbAD9XXFrZ1fVT0fqqIC5c/OlBC7Xjiw1gOidluBG5ET4w7BZSXYlbC69f6RXiL6tKlCTCyoPy6uF4CzuITcl3CJqnC2FOUuR8WVSnVAmPEBgVVXkdjlJbmkO7yOiiUZsEd3fRfZzMvQBOQQBtEv0B9AgIjJDLUimvqRT8xdY/vA7XbS+i9bZkACpL5dkUcFpZu2uLvDRQP70Vi6JhrWCTno9mJp7tkLfxNgbNk0/2Ya924pQD9RvYawsSzo8SKg60f3g3S+YKdlGrOlECZCbcbJ0XEAW3atxtZ9/x884XZXcl7xFO9tyKMoR9J3hLp0yZtRIkt1/bOLKF2LDY8lIPgMAeCCaHrAea3HjfxGOiib7Snubr+RPwnxsMqrD1L0LGVpUpAz1kN7HE8TrTqRwyf/8tKWGwMBr2HpI5vPB+QmcYv20BJ7C/UV2cutEDmGtaHrLy/zTyZML6XRwzN6Mw8yDP5FrNK063eo6ZZCMI7/j/tL4gGxVUDKWsn4YXIfxIVlJcEFwOBNbK+TyGmi1Q+CyrfnCQdbRalJF0W0MGgJKLLS+WWFKLmmv4H0vrFU63szXDZ38fqOLLpYpywbFPcHR4k5iKLZznV1NH0+Bsn4LBzKBoBkT6AOhQdYt5A8Pcr7xiJSiVqYBuuPYPLL7VDyT0l12UeNyDGlOhLZXpe3GReZW/SXTb2orzZHScj4uS/aYN1DFfiY1odAOqZB/J74AFm630wH0DWot6ukNGOQTX1O/O+ X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2024 07:31:49.5911 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0f9ca0db-f13a-4dad-57ad-08dcc8c5cf90 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001EF.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5873 From: Mark Bloch Check if RoCE LAG is active before calling the LAG layer for netdev. This clarifies if LAG is active. No behavior changes with this patch. Signed-off-by: Mark Bloch Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/main.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index b85ad3c0bfa1..cdf1ce0f6b34 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -198,12 +198,18 @@ static int mlx5_netdev_event(struct notifier_block *this, case NETDEV_CHANGE: case NETDEV_UP: case NETDEV_DOWN: { - struct net_device *lag_ndev = mlx5_lag_get_roce_netdev(mdev); struct net_device *upper = NULL; - if (lag_ndev) { - upper = netdev_master_upper_dev_get(lag_ndev); - dev_put(lag_ndev); + if (mlx5_lag_is_roce(mdev)) { + struct net_device *lag_ndev; + + lag_ndev = mlx5_lag_get_roce_netdev(mdev); + if (lag_ndev) { + upper = netdev_master_upper_dev_get(lag_ndev); + dev_put(lag_ndev); + } else { + goto done; + } } if (ibdev->is_rep) @@ -257,9 +263,10 @@ static struct net_device *mlx5_ib_get_netdev(struct ib_device *device, if (!mdev) return NULL; - ndev = mlx5_lag_get_roce_netdev(mdev); - if (ndev) + if (mlx5_lag_is_roce(mdev)) { + ndev = mlx5_lag_get_roce_netdev(mdev); goto out; + } /* Ensure ndev does not disappear before we invoke dev_hold() */ From patchwork Fri Aug 30 07:31:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13784443 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2050.outbound.protection.outlook.com [40.107.93.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 509C116B38E for ; Fri, 30 Aug 2024 07:31:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.50 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003116; cv=fail; b=D7gk7Hm/DQ0pVgpcm/Fu5z3t4X8O+hMi89uhZh3AA8Q+/zH/QJQON/0zbmc6696H8pt++XzKOvDuwuM/ZKQNJbNQIjXzOkLluT3YO5q96mUyE/31jWYa6/9dRAr6eEx9ikJPQr5aIOF/IeErW45iwx4ODaU0NMwnweTQNDeQ9Xo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003116; c=relaxed/simple; bh=lLEMnxsxyqW9JKAvem3D0RAljr/RlvLLG90QJIOYFXs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=g63N3d2QBzaN5GEN4bJVq8wNtibGWkugBGPcNm5Q97Vro5Pn844ZmGF+Ha1TNhwv/1BU52Mt4UnkPDzu5dPejBNib8uWVft3ygJmGpQ4dCavdlAmAQbx/FMCRsYLmWcPsJmiCemOrCN/y1sN9wRjfUXexSBcd52LwFxTu99XsSo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=KcyUprGu; arc=fail smtp.client-ip=40.107.93.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="KcyUprGu" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gGPHjXdKCXo5rMI5EZmAYBZK6SBM5X/hM6qvMNRSughH0wzqIctTZoIPcQx7TOQEsJYDUAKMv/5YDpBpjX8Sk3dEpJeEBbj4wtWcgzxAgi3omQ77T7SQ6AC1EmIiM0NvoP1E+7XfJtb6nue4I/uaW7/h1zWIx6wQ5xAJT1dYNBSIDI1ZNe+fKCMBk5bzlkW4Fy75ohiTa55IYLqoGErxbTatjQZ5XhCgCOEcamvs8wxNrqR2HagvxMerq5PKt0ALlk2u8Ccc1oVcLdA3BgXZqFJEtaU1O8yfHmr/d6NIzRqL3ZtYDHxZ4erOulmD37KbHNCHHYQaxiThv2eafkMY9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j2OqhVHk8El4CAjcA45dc2j34faTtRc0H2tjll4NbU0=; b=p27kUGByLy9E19MpbyGgqnklDr3+QKy2bID7V+tzkpFgFzqyuKJx5lP3jwv79SECARUidggDS6y1ld6IUAycASFeRCgcJy5RIVxTyden+/4zX94/Nz6eW8ultPMlA6nWiTm6vHNirNo/B/INE6CqYWfcok3eIY/fLeSPbsp5CTs7X+pzHbOJtvHGHDH+szgqwcYNQnpWqZtqqqdHV6mBE8+Q2+71bw5erd2WwA3ojnUDaoq39K2Yc2XbWVNNWcmzJvQPpKyF5ZxGfPAiuun9T6sjN/B7w3dIzANbSbc5k/klGScomdGSnAy+cSJw9FUBHszH8a1E86yAh4l1GfF0JA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j2OqhVHk8El4CAjcA45dc2j34faTtRc0H2tjll4NbU0=; b=KcyUprGul2+xfcUB8dEdJx67LtpcDkckaig0T0WYnPH+rekgJvAcZ9nYENfp8c+uIm7UuMaAuEtlaC1oYl39dqYF8NTOJpJi11SuH4kGppqktQgr8Ya0FIv15CztOqPpvdrn60ynS49ha1CEBYXIZc3g3hALyZVLZ5dVuCkRMhFSQDf0uKkP++3nGM+n+CRwIKTvAqQ2JDzk1tsC9X0BUmj8MEYxT7kBafz2lvjG9tzr/NaHoMOOWThSfyLJPbZ1ieaR9axd+ZMXWFcpMjXzxOfEGGNlrT98qpb3y7eAr/QrGfxK0UpQW5TQWLHEJmbrJJqCjJkzbspaEHawDtOW3g== Received: from BYAPR03CA0002.namprd03.prod.outlook.com (2603:10b6:a02:a8::15) by MW4PR12MB5603.namprd12.prod.outlook.com (2603:10b6:303:16a::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.26; Fri, 30 Aug 2024 07:31:52 +0000 Received: from SJ5PEPF000001EF.namprd05.prod.outlook.com (2603:10b6:a02:a8:cafe::7f) by BYAPR03CA0002.outlook.office365.com (2603:10b6:a02:a8::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.20 via Frontend Transport; Fri, 30 Aug 2024 07:31:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ5PEPF000001EF.mail.protection.outlook.com (10.167.242.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.13 via Frontend Transport; Fri, 30 Aug 2024 07:31:52 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:40 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:39 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 30 Aug 2024 00:31:37 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next v2 2/7] RDMA/mlx5: Obtain upper net device only when needed Date: Fri, 30 Aug 2024 10:31:25 +0300 Message-ID: <20240830073130.29982-3-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240830073130.29982-1-michaelgur@nvidia.com> References: <20240830073130.29982-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001EF:EE_|MW4PR12MB5603:EE_ X-MS-Office365-Filtering-Correlation-Id: 563eeb72-27dc-416c-931b-08dcc8c5d112 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700013|376014; X-Microsoft-Antispam-Message-Info: r9LWwGS/EbL+fN99Uum/fwpIIWMYg7ZM0AsyrPzS/f32Ym07tSUYL68YeOgh4D+Ix9SkPpLWtEaL7Bkp8Uwu1jTMCJG8Z2uijn+GzVFhJhWfoH3r3goj3Mi7mqHc4c7T4shHN2RZ8YI5Cmgf5CBarclcN7VwzB2Atfa8iiELkBvC0cjoqhxu2w8l6iZxE8CTFoId2CH47fv7lFFb9xsM8e+JSgTQU+bBNNYNmo0swRxbNTYPTLd6XJFwf/afCDf3J2GT3HyX+zCjDuLG10CJHqiB30bXIVGMaSz0ka7HPzLWjd/QzbRdzAx6XgAEhgHlQz5jx3tnLI1iobQkb0yLF1HLojlL1GPr9H3fD/zJIO6ED1GAA0JQzM7gc57b9vS2WNgabPaH9st8mCA7znmxXFsewIY17kzuM26wZBWEZwH/KYe51d9ev5+rRJF/+iRrziSctNu3MzPQ7ObwRkC6d38/mEDqRJDHtOJdrFzR01V/hrkq6BiVsSWfjVh/K06Sn3B/S/Sg7l+/eCzDxtCvMK7sMbpQ+IHJwA4sw10wfVNNEFzXPLGdP+ARo3CMcnIJY/dJ+mV3Cp9Ce2wOGNqb8Lb7mVw0ijqzDqblRXxdYMkk+R/OuniRbR27GS4GXWaO8BM711hi7WZISTxK2MqULZ05Mq4UKA0oO/OzImX9PkJVMz8LJ66ZUv0lqCwDutmEnNUt1+LxSRstM43/X1oYcoccpkA9KnNIBEcdoxM/itkTqoR7bkP8V+cKJ1ewAPSXQXguLrbj+KXBJa9iFF4nxvGjsPdRbgyYTbQ6uQjA95k/DRpmLdhuq3WsDyPW84Cv+5Pk0yXH/DIL+qzsI3b8mRMB1Y2KvNs5XRauMLc2J/kYaADjQmEs5qbEWN3P2gDMhmnCQQ1cn6mVQC10gCvgxRQDTdNq5G9zOx3+fEszQ5auRxqjLBA+ZWHy7XurFX6KGzo8f8QKsQZbBcnaAvvz0HfNokmNJVWTtD7dMw94chCnT5soKxDIzoLJWI4H+pYw+72tHAt+iPlGv1rdWOZPyxs9wVyP1HQdg2OZ4ss1Acb1WOlw924wppT0HVo5Jp6u+3EXhmDprb001biuEa/kmYkVKumjRdHf58Uh+uLBWv/7Dx+Sl3IMy6BwX95bjnlowqhH0Q6w2nV3LPemSfsXY+RgTXOD33xRBB2Lr3uADQYDpqdWGZ+PldVcorvFx6dt3XP6TJ82OKBHuQDfm2b6OyFTBoozDTNE303wzv4/ELMdMh7R2u4W2pzkie+GabIN+p8l5g0BESfAJKrATCvTNtwRdcQYPeos93Cx0YXybVwwopWbYLDQNF7sUKZZo8q4SJgHLxeWq/rbAJBB3CSJPbjZCJ8oZG3A097Lz2YhlP3FRylPylmpnqp0HnUHe92HcjCCE2sTNQlzVTNnfuDFoNoHlHkZoKXJ+VAFh0OE3RKemw+MIV0Il63k8ICihxRH X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700013)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2024 07:31:52.1067 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 563eeb72-27dc-416c-931b-08dcc8c5d112 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001EF.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB5603 From: Mark Bloch Report the upper device's state as the RDMA port state only in RoCE LAG or switchdev LAG. Fixes: 27f9e0ccb6da ("net/mlx5: Lag, Add single RDMA device in multiport mode") Signed-off-by: Mark Bloch Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index cdf1ce0f6b34..c75cc3d14e74 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -558,7 +558,7 @@ static int mlx5_query_port_roce(struct ib_device *device, u32 port_num, if (!ndev) goto out; - if (dev->lag_active) { + if (mlx5_lag_is_roce(mdev) || mlx5_lag_is_sriov(mdev)) { rcu_read_lock(); upper = netdev_master_upper_dev_get_rcu(ndev); if (upper) { From patchwork Fri Aug 30 07:31:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13784444 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2046.outbound.protection.outlook.com [40.107.92.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E4666166F28 for ; Fri, 30 Aug 2024 07:31:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.46 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003120; cv=fail; b=j1ZTLqNqiCzd+lvq16P+WgrFIR2NiSHKTvl7Iv/5FN+aK2mmpKmxvWvYpFczeXAd4+dqMtkmT2LxfwxpMc2x7btEfnMY4pk/oP1NXRDT5a0UaltYvVgDeFGVoE81OgBqHJuii3GsjSYjO3pD1cBZD+isIalHubRLbAT7q4gyopM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003120; c=relaxed/simple; bh=Je5C3f+CcZSAmB0AMT9meb69fQe7s/01w9SvWGMooE8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=g0BEReRgtm0Q4++4tyhSw2Q9sDg2nqp9Z6Nm1JQL06V8PlPYqEt1VMsap5ZkYYT2BOUHRiO8XnxsljZN2AdgX+G/eVp97AiPPO/fmsSeFGq7RXUWvnJLuk1pAwGrfQmmDbOkuuGAv7uDyajex7Bwl9gOWiB/96MpY/eSzOQ6jgU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Tcg9JYRF; arc=fail smtp.client-ip=40.107.92.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Tcg9JYRF" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dMhWo0L/L9sg+/jcrU2wZYLY6RQ0J3d1aPlukW4Oue7J38v8edTdqQBCzkZOiYDQ1m1ukABzjfKgb81K105qYfbmBNc3aMKqyN2HlfZvhoRe4CLrDBnGW/CUU+zCjMT5ZCAnFm5Y9WkveZGU5Ls9gmUEc04Z+NhTziW0QxQulZuhkKMzaOHc0ZV1oPB8MFUO3/FF6sgCIYeY4iMf51fChHYh38JLVZqswX/YVPsCOA15g5tmzXXNdldoOUd281fpgdXQuPdNPZaW1GNRmYLWOM8WXejb9YMoaJFMzL5JRTEXLIsoVjs7/Tp1CQBGzwt1a13QkqN0WjLOpP42Ru9jRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j3X8e2q5H7HzbLMNGEcnFtC+Vm0ucJAlldStCRceoaU=; b=ABKCDUAGk+W0DFpv40fB5m056nIkQq/o2tzDvGOwPLbArvsH4eCR7HqEIdjIJwSoz/urrdLoyvMDDZKIB8wFeq1N52PqaB0cSUJAoUL7NAvefhoTcUED1QqA2Gy3osnzjm1PMSznJYwvgbRvxQ60ozVd+fZHZMbxXu/cpVjO5fMfHNcokGfTmDtjZAKwMKCmKzKBEEAraJBHzFreleHrqWJJrrAntb0OXs4rfE0u78MH8Nj4mKGZjFar390T2ZPMged0igHzBo5+OTzb716izkmDjEXdyGfMkOF0noTA8OuAgw+hg4boS+/1tG4IEbRn8CyQWma6/T7P+9dXCwunLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j3X8e2q5H7HzbLMNGEcnFtC+Vm0ucJAlldStCRceoaU=; b=Tcg9JYRFrNniOhpYUtDe6RI8i5TaPQjjAgka5sbRkK/OLf4NMzr8ij5o/KQMCRss1prG1O5j2FI9OQ9bE7w2gFGoWngOCL4dVeCsANVWHB9G9sTJIj2af6bb+CFcJiB9Ty94GziV0fDlIf4O3cPKa9+NmiZMoV4K+r3WaShfQl1GphhccyY6rEg3J8PipmtBTdzRJRqNdG4oNXPigmytzuvnUq/jdnJJU0jP294gFXwmDPazCCaBFBUGtuGRg/XzTf6lXJ+nV06ro9UbFFsmMfyWoWwcom02tFq44cejTF3AnP6G/1G1umYTOsBUA+G1AaXiUOfGR5NjZUOOjB06BA== Received: from BYAPR08CA0066.namprd08.prod.outlook.com (2603:10b6:a03:117::43) by IA0PR12MB8976.namprd12.prod.outlook.com (2603:10b6:208:485::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.26; Fri, 30 Aug 2024 07:31:54 +0000 Received: from SJ5PEPF000001E8.namprd05.prod.outlook.com (2603:10b6:a03:117:cafe::95) by BYAPR08CA0066.outlook.office365.com (2603:10b6:a03:117::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.28 via Frontend Transport; Fri, 30 Aug 2024 07:31:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ5PEPF000001E8.mail.protection.outlook.com (10.167.242.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.13 via Frontend Transport; Fri, 30 Aug 2024 07:31:54 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:42 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:42 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 30 Aug 2024 00:31:40 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next v2 3/7] RDMA/mlx5: Initialize phys_port_cnt earlier in RDMA device creation Date: Fri, 30 Aug 2024 10:31:26 +0300 Message-ID: <20240830073130.29982-4-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240830073130.29982-1-michaelgur@nvidia.com> References: <20240830073130.29982-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001E8:EE_|IA0PR12MB8976:EE_ X-MS-Office365-Filtering-Correlation-Id: 4618a753-cd05-4400-c1c3-08dcc8c5d255 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|376014|1800799024; X-Microsoft-Antispam-Message-Info: HTST9H0dlQZNijcWCy09rZabPDkJmwHrFhZKPPF/WQjOcNoEeoqmjZTg6lMckA//suHHO0sLM8InYa9o650wJtB/G+mGIsdBHxrE/MOGMIj2MzKjhKNkISkFUm5+jNXA0ZadRe28hfq0EqtBJ1qVvhhOQEBx/bFcn+XppYtxO2t8RqWlf2RftyFcEHhNKzKQExhsz/IO/epk65ndual2+pB6bBERNb/0RN7s1aayRo61iPPIyKfv/DzcU3WucwYVRAdaCd7uy4nU82BS3wf6TXZc17Tmlg+vyBHZV2PgguyHCgjPSF85gRsXCdvAN4fTH21GiEm4yqiLZt/7e06qP4nknU4T0zgE4AI5wXajzK4D8PyTRGZN+Iuam4S0aCMysNYWkk4/PQN0rGGwOXYjKV+Hd/JDukC+pNp30OTv5vPNtJqApeRdEIwV/W9HnzXkYc+LGMtd8AOKO3r2z+dKf1IqUhKSJLdeRcyQZYYM/uD4XdZ+VmNky7QUkTw7ORP9OHdeufxMT+upWkKXBW6Wi/TPbbuMlW5koqqQPAr6qxkQA+Aa19Zg/3iKGl6zyqhQIB0+Zec/MFaX/w7JmXS3Nr8pxLqTEt5pmua9Xwuaq4np4enDhHHd1Pcc4AVFftnuTLsIjCh5FiSG3rCuEu1y+VJOxxw5eiLMAM/GnErCHaQKkaa8WTQ6gXC4LJO/weVTYujW1iG1XbHml0hl8Zk/ZgEgfHtKBbG8fjaqmCHxWfuNvPSnzabEGHKCk7z3gN7kYc4M/7+kuaGGdnaAPopMTMQzuQVbr+ZYuc/gMRY3bIfGKol4aFcQA/ptLyuNTD/E3j5Mj7VjHUx8AqoERbKsGamTBdClQda4ZuMdI4QjQSvBqJkyOtOXk9pVvzj23PTvlwtgN2fCei5DgL3dN4kLCY4wWmoSaP44Rl3BqEyNxXMRGzi2HNC/U22kE90TQQ9pimReP4wOc3/eOwvGFUEUd08IAGuV0sM/BR9JpP+8a145OMBWxSI1Yv8ouZTdQIrGXiZ0iKYzSXUpU9Yobt8HDP8tFZ6yED0N3ohl3HnxSAbwqCa5fKTLtEy4encG9OtUZFjYAkBdg0ZlY1QPulG5XLgwt428jmq/0rRLNOybOBjftvDjN/sCliQP/MRHLLfNhSIBXsG6X9we3ViAi8rB+e41TCJwRqvSSZxIrMlUldf+TN9ypyb9nbVgh66bJyrn6xYCywUjFfsR0ctBB3GSN9n9D0K5n5nyB1I6KMOS7JTl71QWUqHNZ8ugxCoRlG6+1vizqYKKXKlcPjEgdgaLLs/R5j4nLyH4exQvRgki7czSH3dwOjuGeeDrFcJ7E0d7JYz04H5jl7UR3zDRKlxn/NjOhn5xPUBWkztF/Kw0QO23LCCSYo2LK7f6Vt5YFsmjDXJ1OqwyhzotMy3Lv3wzqeKABMhssfjxJYBDfTZXkOFrWbrKobAlJnYw6CrunErc X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(376014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2024 07:31:54.2403 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4618a753-cd05-4400-c1c3-08dcc8c5d255 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001E8.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8976 From: Chiara Meiohas phys_port_cnt of the IB device must be initialized before calling ib_device_set_netdev(). Previously, phys_port_cnt was initialized in the mlx5_ib init function. Remove this initialization to allow setting it separately, providing the flexibility to call ib_device_set_netdev before registering the IB device. Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/ib_rep.c | 1 + drivers/infiniband/hw/mlx5/main.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c index c7a4ee896121..1ad934685d80 100644 --- a/drivers/infiniband/hw/mlx5/ib_rep.c +++ b/drivers/infiniband/hw/mlx5/ib_rep.c @@ -104,6 +104,7 @@ mlx5_ib_vport_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep) ibdev->is_rep = true; vport_index = rep->vport_index; ibdev->port[vport_index].rep = rep; + ibdev->ib_dev.phys_port_cnt = num_ports; ibdev->port[vport_index].roce.netdev = mlx5_ib_get_rep_netdev(lag_master->priv.eswitch, rep->vport); ibdev->mdev = lag_master; diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index c75cc3d14e74..1046c92212c7 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -3932,7 +3932,6 @@ static int mlx5_ib_stage_init_init(struct mlx5_ib_dev *dev) dev->ib_dev.node_type = RDMA_NODE_IB_CA; dev->ib_dev.local_dma_lkey = 0 /* not supported for now */; - dev->ib_dev.phys_port_cnt = dev->num_ports; dev->ib_dev.dev.parent = mdev->device; dev->ib_dev.lag_flags = RDMA_LAG_FLAGS_HASH_ALL_SLAVES; @@ -4647,6 +4646,7 @@ static struct ib_device *mlx5_ib_add_sub_dev(struct ib_device *parent, mplane->mdev = mparent->mdev; mplane->num_ports = mparent->num_plane; mplane->sub_dev_name = name; + mplane->ib_dev.phys_port_cnt = mplane->num_ports; ret = __mlx5_ib_add(mplane, &plane_profile); if (ret) @@ -4763,6 +4763,7 @@ static int mlx5r_probe(struct auxiliary_device *adev, dev->mdev = mdev; dev->num_ports = num_ports; + dev->ib_dev.phys_port_cnt = num_ports; if (ll == IB_LINK_LAYER_ETHERNET && !mlx5_get_roce_state(mdev)) profile = &raw_eth_profile; From patchwork Fri Aug 30 07:31:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13784445 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2076.outbound.protection.outlook.com [40.107.94.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8238168487 for ; Fri, 30 Aug 2024 07:32:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.76 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003125; cv=fail; b=thVp40d2UpIZix+mA1+MeeLKelV4cBlY7lKNJMDapgBFbYz0dJ88nQFhNiJizqg9Jl6/qqs37hkG6tTYkHMa58r9aMiTT7t62u5zgpySt7pbdtiwUUzsijK10WEFssqvY0mUr3LFelyqN7oMGao3ZuWxf0f7EYW7OlVjfGIOrC8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003125; c=relaxed/simple; bh=CnbQObHDd2dIFDF2Hs4buSYxYvltOmZzBzyMSFSd6YA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QJ4ecUyuU8e9c4G6ttaYR9A0nzc5HR9BFbLuH6AjskXAiq0UjsI2tUhnXgBfw803INsfZ/yBSfEKCDgecRrC4wwYTalQf8zWsfA50XD78kjGoJUs+LdhoqMESLseer7WorbWRvhysEqZxXwZ57WXpVSgYg+adjuNxHe3YY7/ftk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=gp3H4cmu; arc=fail smtp.client-ip=40.107.94.76 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="gp3H4cmu" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Uvohk/EJCVrZlWg3AOxbJRoqN3MB87xUKaCD1bbT03s5tfBudOzEW8pqkoIVjTb6h9b/dLpDR7k3nCK5j1f1FIwiD7P1tFjGWF4BCMOtcvQfJuvA19pmCOEbJL/FXjT+80QsWVvJtsZSMJvAv9zeIRXn3vHT9sb03eIUIfZN9tFwhTm7AABHqdzNIX6g1sJT/qbX8MQ7XvtjhLNr07oSpZWFdRoerhhMXfyDcCyXZGwpZZ3QMOjgH5oTvDruIKOWL9lUo1qoybiNxs7EhPAau8IHlZhwgketVCHtIttBwICxjz4noUmQDXzcwIX/OuH91Yl+lEWPEtqwqVozl9n65w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yBEcTykqEySuYHFc8FP+1MW7lCTwuoKKyg0UEtAYMgI=; b=mDL9V9mE8DTEvCdmP3s5WaY914GQJiJ11MHuA7zYCO1mhTAapBvzDCzzV2hc4Lz/d7BYCUQf3bMkcOPfNBmiacgwk9z3J6OMGcxysiDMJBnAVUsSMepd/cY+mQIg4U04YT/xygYaVl65N/AkoY+01dJ0l0wY93GIIU8mo9IHhBfu/sXGTADEQxVf3Li5RmrLRN/6JiGgFPVLvfbOhgnVuS1hb14vSnPmNjbM7t11ddtqtNTOrRTG7PsWjlTcHrJ1AhPUu8FqxFFaW+h0g4CVGZka2GxkFrCh9o13HJNaQKW4NjpIcMCyL5oeuOpt5o6MycfRA/VzspOZIiuZj0yvrg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yBEcTykqEySuYHFc8FP+1MW7lCTwuoKKyg0UEtAYMgI=; b=gp3H4cmu3Z64q6xswbDhxG44MW08B2el/nAgGVfzSdENdJi4VdBDd59Svo3LuBh8qWf1E1kyHeZ+iOV/m7Bej3C5GG6Ia6d3QxDTXfBV0frDPowrGWBjvwsT6Pyg7q9BGdsFABkei5piwY3MvGyJewSns+riLBhxz+lttwjMybcp34ZYlv6CkbqHMlu/FSx3tI7eBiPmDyYtBeH8u+0ManhMWOZVqe1N0XD0ucmXqRtEPCGkAPH5H3A4hzWIDTh8NAGESfBvrQS+xYkVBPR4vBcW6/b/FrCxUPFDt/70joU+ZDYP8RMdaoG9KyGrSFxZO2ZY4mG7kcSbNNbmxWe4Hw== Received: from MW4PR03CA0196.namprd03.prod.outlook.com (2603:10b6:303:b8::21) by PH0PR12MB7863.namprd12.prod.outlook.com (2603:10b6:510:28b::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.20; Fri, 30 Aug 2024 07:31:57 +0000 Received: from CO1PEPF000044FD.namprd21.prod.outlook.com (2603:10b6:303:b8:cafe::34) by MW4PR03CA0196.outlook.office365.com (2603:10b6:303:b8::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.28 via Frontend Transport; Fri, 30 Aug 2024 07:31:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF000044FD.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.2 via Frontend Transport; Fri, 30 Aug 2024 07:31:56 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:44 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:44 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 30 Aug 2024 00:31:42 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next v2 4/7] RDMA/device: Clear netdev mapping in ib_device_get_netdev on unregistration Date: Fri, 30 Aug 2024 10:31:27 +0300 Message-ID: <20240830073130.29982-5-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240830073130.29982-1-michaelgur@nvidia.com> References: <20240830073130.29982-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FD:EE_|PH0PR12MB7863:EE_ X-MS-Office365-Filtering-Correlation-Id: 031aa2fa-38d1-47c4-b620-08dcc8c5d3dc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|376014|1800799024; X-Microsoft-Antispam-Message-Info: hT0Hd8wFUttIsjnDOn5epxcDy5Olo1sXNGouoGrPWcVIMPO+HVjQjKoo/+j67jpUTiV23bcn2VKA2wBzou9BjZrlADmcJq43D0l6M2Hrb9cxTTU3JPAPE0EJftZbh+I+Ilg5uq+r8AKFT5lUtAT8hegCsO4fYv8NuTbvGTU+4KodCf8xmk/Ma/IWijKXtX+0duO7WBceFAwQBtfeT0iqz7IvbgRSAis7t9+zNH5X44x+48F++o7wR1hHKPge3Za1y64lmDLdBct1EwyHQkZFYPp/WzT88hZfZ0ET6329lvgpugnC8I2f90ymlz4cd54CVXCFiTzRgXCLYOZ5b/Byi6yA6lqtrNPs3tj2yhxDmwn1uVm23Dams5V7N2Lx2yW6k1Yc6dpBqhGU/kHEWx5vy4Hij1zRNYDchGLpZqWDYO+dEjmUJXWVUKtWTYMwsbJh39U7qvOgmZS4R0wOCDIEEgnEiDsAQMQ5hi3bjxRwYEK2oAPVy+O313hngyD1/s55+MG0yMAtmNXAC/jn34vskz3/ht5AkQ4v9JQSBpq07MMylqBfwZl24r9A2CKzMwFzkAGCABDusu3tkgkWnbfiEB/IKNjMqp/Olgynn/wYpG+53eHcqg9yp0LKUBU9H4uGTtHegQa5kejWRpeObQfzCJGqywUuefsX2f6rT56Ra8nvberQDOPsIP0RXtr3yRcD++SzAPjHO5hu/uNUSGIpAe45wZRHTX91AW5YpoMuKPH4WNoU3jLmXFVL4YyBT9ruNCg0RhPkr/ZIZ19dRx0BsHBdIGMlPKLpYWhvtz5Lw/uFpTYdJFXie1iexNPlsOvOb3ugJ73b0ol6X63QSVGKGq0Rrg3ab89A2Qt298NDUYNEofUnQMiJqxPgh5gFk44NDA7joC1H/4MJf3pHcIDfR+Wk67KjU3UQUvhT2n1ED4URvgdIUeQw7CMWqie518W+Gx+XhoNPYpDettQysA/FQyyKubOKMaGR8rm5euBYGQsCAiVNuQqw+KguArxqJJsiL4TUFCE5KVYHme4bJc9a9VRrcbD0K8vKiKn/ZgNJV6O825a6Aliju1Y1UBYxRqkUU86Zn2W2lfuNs3AraAfc1o8yp+TFt93JEagDtZs//vCewXkPUsQ9/VU20e/DXanxIn1EPTzdKdZ7JEQ5mZFSIWwHANKMjRebeIQqyrCIXfu/Yps9iMpTGT3P8kwBj76qLNLv8+paq3//yaEVeC35nUS18KHoXxqNFD8CvWVpvwLI1+JUK4bTBTEFscZnGUQ8F1/XaPo6J9wE6idJM8i8KW4riQS+hxCaiz2+4wPwxwwnAEAiMi7CeFmIHs3+IMRqkoS49QBMnNxFWOu29TOZbF/BBwgTGgz9skUfQNvM4Ew4KnDjIU/sSuepG08CxxXG22bOjMQeEHAW2XmrXOZP22xzYPVv9NYU3tyUdk4WLyWZ8Cw55Lpq+WvzLnXCa3HR X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(376014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2024 07:31:56.7399 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 031aa2fa-38d1-47c4-b620-08dcc8c5d3dc X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FD.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7863 From: Maher Sanalla The caller of ib_device_get_netdev() relies on its result to accurately match a given netdev with the ib device associated netdev. ib_device_get_netdev returns NULL when the IB device associated netdev is unregistering, preventing the caller of matching netdevs properly. Thus, revise ib_device_get_netdev to assign NULL to netdev when the netdev is undergoing unregistration, allowing matching by the caller. This change ensures proper netdev matching and reference count handling by the caller of ib_device_get_netdev/ib_device_set_netdev API. Fixes: c2261dd76b54 ("RDMA/device: Add ib_device_set_netdev() as an alternative to get_netdev" Signed-off-by: Maher Sanalla Signed-off-by: Michael Guralnik Reviewed-by: Mark Bloch --- drivers/infiniband/core/device.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 0290aca18d26..583047457de2 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -2257,6 +2257,7 @@ struct net_device *ib_device_get_netdev(struct ib_device *ib_dev, * propagation of an unregistering netdev. */ if (res && res->reg_state != NETREG_REGISTERED) { + ib_device_set_netdev(ib_dev, NULL, port); dev_put(res); return NULL; } From patchwork Fri Aug 30 07:31:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13784447 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2051.outbound.protection.outlook.com [40.107.243.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D69E16B38B for ; Fri, 30 Aug 2024 07:32:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003126; cv=fail; b=gsjWnuSY2nK5lkQjjU4zX1lWI92C/ibazlKc6aQ5H+FH+cfnTqzKPA1vD3kpGuh9lUasxxE0HF0c4O1yoq3htUGuM/HkRQhUn0RfweYQEPL81/iwlO2fMaITwQNoDnBnJkH5LWtVPdF+jY4er+VluDbeVyg0W9kPy7cWfU7T6UQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003126; c=relaxed/simple; bh=QiX9r8v8Am7Cw8S5F6697UK1l+U4FuTjeBwd3sQRyzM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=VBfSgaebvpC4fqwVh7S+wEQUlIqInwMPmrhViD20TtSFurJqvmZ1l5zhP0MHyUmweFKGF0UhOZEeZ456ta+ir0n0y77l/x3maX9j0teBPipSp0kLf6So3V6MH+RqhfoF8xnBOL/Vhptdv9149Me8mz2S2OPXahxn/H8DZg2ndkM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=D2mWfaiV; arc=fail smtp.client-ip=40.107.243.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="D2mWfaiV" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kI5XmcBc8v1e3opoWh1KrzamM6I+AmASyKdX4LHjmyjtA282hMxwXSTRwCfv5ZiKlz+kCdSArCkZsfGiMbnxT2ro+JNoKIXhVIYOgVFEW2Tw0RVGWOQ7s43ImiLcbsDtWlZxLskpFqDYjDpVqlJVaFaPs1MOarjU1vpbD7YXxfaIlO9L1uwu713ZOARArGNvt+0yCflHVoFkSV9TQ93pUuGlpZmFUsryzNv60IK42kcYtVZfMie6XSamjBFqIVgG+uBRRzWfbHxUVwr0UMMrTM/eXoLoz7TkwBd/QJjBUdAWsiQCV+xZ6IEGC3YIMMXBa1z8jR2oxseZWzccF+p8iA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WM98OpYaBWrGP/uyMHl9VBSg7oci8BT4A/3KzuA1+ag=; b=OFNUsLhG/f/4VDv36OS+v75hAsCF7EnLdzRw8RrWyTqwVkJeyiz1jEcBeTP/ueTF/eRZXbbbnzjjnC04Sjwcgmsdyz2T7VGnu1CEUgUmKMThjBxiwdzOC+odf77zatokpgN59Ux7Ml2oWm1E5atMsZIacwEZ6UhWAlbTJlB3atEMpTSdqJA8EMBb2GBcxbKkH7RT5PBv0HcLeOA72fR5VaJ4HF8deWkl406D7CJ3s9+dl/Y2+t4inkMjzBozahpABZtZ39CDhTjBo6jRNln3hZgdHCO7sUzPP9jRdJDRsk5lV7A1VMJowNxlVyLth17bTfvR4yy/M/rlIQoAOX4MLA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WM98OpYaBWrGP/uyMHl9VBSg7oci8BT4A/3KzuA1+ag=; b=D2mWfaiVoXiKhQPI3ZunY0Og4KoHL0oUu+FxOz3nhk3pxj8gSAOVc4iFyOEG8E7U8Wti5Ce/IZyztM24nR6IC4Blz6pwUwIHv8QKCKTMZ/yWUXwdNXQR/9QR8yP3hoSbICtCPhabnPyIab57rkfgFDA1o5QkLup5Nq0MUF0O5cF8KSHTxpwnmwTthjDbUX1bq/mZ+VrCgGqQmkc8YSZDp0LaLBZx2Ai/GG4e1I+uWGcx820qYbW4r4/xozVEzs2w5LAw1T8OwcBmWs3GLAIXMSkBUema/0SP69Rpf3qIsbH1rvxrc8F86ZXMsr0qGME+Sf9ksPbC1Gwt0STxfaYVhw== Received: from MW4PR03CA0182.namprd03.prod.outlook.com (2603:10b6:303:b8::7) by SA1PR12MB8917.namprd12.prod.outlook.com (2603:10b6:806:386::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.20; Fri, 30 Aug 2024 07:31:58 +0000 Received: from CO1PEPF000044FD.namprd21.prod.outlook.com (2603:10b6:303:b8:cafe::95) by MW4PR03CA0182.outlook.office365.com (2603:10b6:303:b8::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.28 via Frontend Transport; Fri, 30 Aug 2024 07:31:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF000044FD.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.2 via Frontend Transport; Fri, 30 Aug 2024 07:31:58 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:47 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:46 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 30 Aug 2024 00:31:44 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next v2 5/7] RDMA/mlx5: Use IB set_netdev and get_netdev functions Date: Fri, 30 Aug 2024 10:31:28 +0300 Message-ID: <20240830073130.29982-6-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240830073130.29982-1-michaelgur@nvidia.com> References: <20240830073130.29982-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FD:EE_|SA1PR12MB8917:EE_ X-MS-Office365-Filtering-Correlation-Id: 061eb3fc-b2f9-42af-e74b-08dcc8c5d4b0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: whMaKVH/qh2C9PpD4lCV6gyQM0+6+vjR+wOs15EH46qnVwdq+SZvsU9TY04iIfG8b3wLMTe7QTApoHgDMQUcNeyqAeBo5LQB2Fv4ahbcLEd3N5QlmRZ2RkEwviDBGPv+TyzsmzJgQLohFQ5laf5cVldqKaeQHOzYWPc9MYpxYGPEKaQyp4dwoidLum5ej6jY7rQs3J8TT0ku9xsaS/4Jjkplp5oNqy+5vSnfxa/pp/1UzSuLo6J+Co9YfJPNQ4ofRPtg1bCw3t+n0egGzXkd5zcg/yfBe98DXtazG0cgH/e9XCnnFC56z/A5GYkUTO1Q8+TZiO4vpq0M5P68Wc85W7RhJyESihweKFVfDCadjLO1SGVm/zK6TNgddZcolynuMTdmrijAr6TvjHEOZLcDoJKdZQkgl+7A+ylS48tN7ZBtvjiJgj7w/xGoG/zOlZSQ7I7Kmsy1/1w2jl+G+tzI/5mV2ToQDpLxfG0xw4lxf/8a4dUoKPOFxrVW5duYEAvFzIFsgF8ZQWNox4AjZN7sR9I87uL19zuKoX7lBh/I+z5E+eRKxoyh2BLCjv9q6ImTrZdbU0pBrZgP8YjXTVp4zQPBeZ/nMvU5AeNegMNCO9DcdaXnU1T20Yh9MK4lVzHwKVEaHASDA04cifGZiYQYbqCWQ2+Yq7gP6/84xEHDkpgNqFyMWCeOXLHdZBl1znUujdiTlt0LB/z9RCS2dHIFYyfVwxQ57Z5pJojD+Jh53RegZL2GmN6QZTWfdmNecHT4t9ik+YofsCgI1wWv86iaMggGW4NziYU8LgIM1+l9AznfyRWQ4L/g9zGd/pX5w3aZTAXi7n7jD2LwSUS72IhGdoypDMkbrJkz0gukz3s5h1gl6hi8Qdk5WRH7MOoviviHfLt8KWd4W57lVwR4etSKQynjDEIAim+MSlosSCHNky2wfdZiyoo7KriujMy26vPgHfSf7zETS3qcVqlSZT/QLSipus2mJyQGfjo9+F1SQHTpCe8retFCV7gszWxWbRlr6PoqdDhd/2vul0q+3mIvFnPvX4RygoFAx9msr+EZccHlBk7NTIsm3g+9KSytja8e/Su0rSxnh1yQPCbXJ+RhmXgANk8iHgdVSievXk+hVzcAByqV77DdyAwLfNsXohC2QuYU3oTr05aKFm82uhaRsOC4h7OlnG2pmNEtsHtuElRt+tCw3772g686rh6HxrFGCpW3+7PNghpwnOXzKCPgECUzyCOvZHuRjj4+CK2Ok9PWqPJ5/oyX193RbNKobrJ9NAGN34ubNITbPlUaL0y49as8PB2eeK4oYFR23yVxlcSYbgfuVZKWncYA7rt0Dahzk6F4lBXsdOUFV7ti1uYA8Sys3UxivuOFQBaEHysY7XD36AW09nNmrn4YrVIW4eLihOlVTuMs+6UiDvc3vVbISCYJ36KrMwYDCjoU27hCnTLGFXeSEnFALvSGkSwm3+wG X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2024 07:31:58.1149 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 061eb3fc-b2f9-42af-e74b-08dcc8c5d4b0 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FD.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8917 From: Chiara Meiohas The IB layer provides a common interface to store and get net devices associated to an IB device port (ib_device_set_netdev() and ib_device_get_netdev()). Previously, mlx5_ib stored and managed the associated net devices internally. Replace internal net device management in mlx5_ib with ib_device_set_netdev() when attaching/detaching a net device and ib_device_get_netdev() when retrieving the net device. Export ib_device_get_netdev(). For mlx5 representors/PFs/VFs and lag creation we replace the netdev assignments with the IB set/get netdev functions. In active-backup mode lag the active slave net device is stored in the lag itself. To assure the net device stored in a lag bond IB device is the active slave we implement the following: - mlx5_core: when modifying the slave of a bond we send the internal driver event MLX5_DRIVER_EVENT_ACTIVE_BACKUP_LAG_CHANGE_LOWERSTATE. - mlx5_ib: when catching the event call ib_device_set_netdev() This patch also ensures the correct IB events are sent in switchdev lag. While at it, when in multiport eswitch mode, only a single IB device is created for all ports. The said IB device will receive all netdev events of its VFs once loaded, thus to avoid overwriting the mapping of PF IB device to PF netdev, ignore NETDEV_REGISTER events if the ib device has already been mapped to a netdev. Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/core/device.c | 4 + drivers/infiniband/hw/mlx5/ib_rep.c | 23 +-- drivers/infiniband/hw/mlx5/main.c | 183 ++++++++++++------ drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 +- .../net/ethernet/mellanox/mlx5/core/lag/lag.c | 76 ++++---- include/linux/mlx5/device.h | 1 + include/linux/mlx5/driver.h | 2 +- include/rdma/ib_verbs.h | 2 + 8 files changed, 191 insertions(+), 103 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 583047457de2..b2fc5a13577c 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -2236,6 +2236,9 @@ struct net_device *ib_device_get_netdev(struct ib_device *ib_dev, if (!rdma_is_port_valid(ib_dev, port)) return NULL; + if (!ib_dev->port_data) + return NULL; + pdata = &ib_dev->port_data[port]; /* @@ -2264,6 +2267,7 @@ struct net_device *ib_device_get_netdev(struct ib_device *ib_dev, return res; } +EXPORT_SYMBOL(ib_device_get_netdev); /** * ib_device_get_by_netdev - Find an IB device associated with a netdev diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c index 1ad934685d80..49af1cfbe6d1 100644 --- a/drivers/infiniband/hw/mlx5/ib_rep.c +++ b/drivers/infiniband/hw/mlx5/ib_rep.c @@ -13,6 +13,7 @@ mlx5_ib_set_vport_rep(struct mlx5_core_dev *dev, int vport_index) { struct mlx5_ib_dev *ibdev; + struct net_device *ndev; ibdev = mlx5_eswitch_uplink_get_proto_dev(dev->priv.eswitch, REP_IB); if (!ibdev) @@ -20,12 +21,9 @@ mlx5_ib_set_vport_rep(struct mlx5_core_dev *dev, ibdev->port[vport_index].rep = rep; rep->rep_data[REP_IB].priv = ibdev; - write_lock(&ibdev->port[vport_index].roce.netdev_lock); - ibdev->port[vport_index].roce.netdev = - mlx5_ib_get_rep_netdev(rep->esw, rep->vport); - write_unlock(&ibdev->port[vport_index].roce.netdev_lock); + ndev = mlx5_ib_get_rep_netdev(rep->esw, rep->vport); - return 0; + return ib_device_set_netdev(&ibdev->ib_dev, ndev, vport_index + 1); } static void mlx5_ib_register_peer_vport_reps(struct mlx5_core_dev *mdev); @@ -104,11 +102,15 @@ mlx5_ib_vport_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep) ibdev->is_rep = true; vport_index = rep->vport_index; ibdev->port[vport_index].rep = rep; - ibdev->ib_dev.phys_port_cnt = num_ports; - ibdev->port[vport_index].roce.netdev = - mlx5_ib_get_rep_netdev(lag_master->priv.eswitch, rep->vport); ibdev->mdev = lag_master; ibdev->num_ports = num_ports; + ibdev->ib_dev.phys_port_cnt = num_ports; + ret = ib_device_set_netdev(&ibdev->ib_dev, + mlx5_ib_get_rep_netdev(lag_master->priv.eswitch, + rep->vport), + vport_index + 1); + if (ret) + goto fail_add; ret = __mlx5_ib_add(ibdev, profile); if (ret) @@ -161,9 +163,8 @@ mlx5_ib_vport_rep_unload(struct mlx5_eswitch_rep *rep) } port = &dev->port[vport_index]; - write_lock(&port->roce.netdev_lock); - port->roce.netdev = NULL; - write_unlock(&port->roce.netdev_lock); + + ib_device_set_netdev(&dev->ib_dev, NULL, vport_index + 1); rep->rep_data[REP_IB].priv = NULL; port->rep = NULL; diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index 1046c92212c7..a750f61513d4 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -147,16 +147,52 @@ static struct mlx5_roce *mlx5_get_rep_roce(struct mlx5_ib_dev *dev, if (upper && port->rep->vport == MLX5_VPORT_UPLINK) continue; - - read_lock(&port->roce.netdev_lock); - rep_ndev = mlx5_ib_get_rep_netdev(port->rep->esw, - port->rep->vport); - if (rep_ndev == ndev) { - read_unlock(&port->roce.netdev_lock); + rep_ndev = ib_device_get_netdev(&dev->ib_dev, i + 1); + if (rep_ndev && rep_ndev == ndev) { + dev_put(rep_ndev); *port_num = i + 1; return &port->roce; } - read_unlock(&port->roce.netdev_lock); + + dev_put(rep_ndev); + } + + return NULL; +} + +static bool mlx5_netdev_send_event(struct mlx5_ib_dev *dev, + struct net_device *ndev, + struct net_device *upper, + struct net_device *ib_ndev) +{ + if (!dev->ib_active) + return false; + + /* Event is about our upper device */ + if (upper == ndev) + return true; + + /* RDMA device is not in lag and not in switchdev */ + if (!dev->is_rep && !upper && ndev == ib_ndev) + return true; + + /* RDMA devie is in switchdev */ + if (dev->is_rep && ndev == ib_ndev) + return true; + + return false; +} + +static struct net_device *mlx5_ib_get_rep_uplink_netdev(struct mlx5_ib_dev *ibdev) +{ + struct mlx5_ib_port *port; + int i; + + for (i = 0; i < ibdev->num_ports; i++) { + port = &ibdev->port[i]; + if (port->rep && port->rep->vport == MLX5_VPORT_UPLINK) { + return ib_device_get_netdev(&ibdev->ib_dev, i + 1); + } } return NULL; @@ -168,6 +204,7 @@ static int mlx5_netdev_event(struct notifier_block *this, struct mlx5_roce *roce = container_of(this, struct mlx5_roce, nb); struct net_device *ndev = netdev_notifier_info_to_dev(ptr); u32 port_num = roce->native_port_num; + struct net_device *ib_ndev = NULL; struct mlx5_core_dev *mdev; struct mlx5_ib_dev *ibdev; @@ -181,29 +218,38 @@ static int mlx5_netdev_event(struct notifier_block *this, /* Should already be registered during the load */ if (ibdev->is_rep) break; - write_lock(&roce->netdev_lock); + + ib_ndev = ib_device_get_netdev(&ibdev->ib_dev, port_num); + /* Exit if already registered */ + if (ib_ndev) + goto put_ndev; + if (ndev->dev.parent == mdev->device) - roce->netdev = ndev; - write_unlock(&roce->netdev_lock); + ib_device_set_netdev(&ibdev->ib_dev, ndev, port_num); break; case NETDEV_UNREGISTER: /* In case of reps, ib device goes away before the netdevs */ - write_lock(&roce->netdev_lock); - if (roce->netdev == ndev) - roce->netdev = NULL; - write_unlock(&roce->netdev_lock); - break; + if (ibdev->is_rep) + break; + ib_ndev = ib_device_get_netdev(&ibdev->ib_dev, port_num); + if (ib_ndev == ndev) + ib_device_set_netdev(&ibdev->ib_dev, NULL, port_num); + goto put_ndev; case NETDEV_CHANGE: case NETDEV_UP: case NETDEV_DOWN: { struct net_device *upper = NULL; - if (mlx5_lag_is_roce(mdev)) { + if (mlx5_lag_is_roce(mdev) || mlx5_lag_is_sriov(mdev)) { struct net_device *lag_ndev; - lag_ndev = mlx5_lag_get_roce_netdev(mdev); + if(mlx5_lag_is_roce(mdev)) + lag_ndev = ib_device_get_netdev(&ibdev->ib_dev, 1); + else /* sriov lag */ + lag_ndev = mlx5_ib_get_rep_uplink_netdev(ibdev); + if (lag_ndev) { upper = netdev_master_upper_dev_get(lag_ndev); dev_put(lag_ndev); @@ -216,18 +262,19 @@ static int mlx5_netdev_event(struct notifier_block *this, roce = mlx5_get_rep_roce(ibdev, ndev, upper, &port_num); if (!roce) return NOTIFY_DONE; - if ((upper == ndev || - ((!upper || ibdev->is_rep) && ndev == roce->netdev)) && - ibdev->ib_active) { + + ib_ndev = ib_device_get_netdev(&ibdev->ib_dev, port_num); + + if (mlx5_netdev_send_event(ibdev, ndev, upper, ib_ndev)) { struct ib_event ibev = { }; enum ib_port_state port_state; if (get_port_state(&ibdev->ib_dev, port_num, &port_state)) - goto done; + goto put_ndev; if (roce->last_port_state == port_state) - goto done; + goto put_ndev; roce->last_port_state = port_state; ibev.device = &ibdev->ib_dev; @@ -236,7 +283,7 @@ static int mlx5_netdev_event(struct notifier_block *this, else if (port_state == IB_PORT_ACTIVE) ibev.event = IB_EVENT_PORT_ACTIVE; else - goto done; + goto put_ndev; ibev.element.port_num = port_num; ib_dispatch_event(&ibev); @@ -247,39 +294,13 @@ static int mlx5_netdev_event(struct notifier_block *this, default: break; } +put_ndev: + dev_put(ib_ndev); done: mlx5_ib_put_native_port_mdev(ibdev, port_num); return NOTIFY_DONE; } -static struct net_device *mlx5_ib_get_netdev(struct ib_device *device, - u32 port_num) -{ - struct mlx5_ib_dev *ibdev = to_mdev(device); - struct net_device *ndev; - struct mlx5_core_dev *mdev; - - mdev = mlx5_ib_get_native_port_mdev(ibdev, port_num, NULL); - if (!mdev) - return NULL; - - if (mlx5_lag_is_roce(mdev)) { - ndev = mlx5_lag_get_roce_netdev(mdev); - goto out; - } - - /* Ensure ndev does not disappear before we invoke dev_hold() - */ - read_lock(&ibdev->port[port_num - 1].roce.netdev_lock); - ndev = ibdev->port[port_num - 1].roce.netdev; - dev_hold(ndev); - read_unlock(&ibdev->port[port_num - 1].roce.netdev_lock); - -out: - mlx5_ib_put_native_port_mdev(ibdev, port_num); - return ndev; -} - struct mlx5_core_dev *mlx5_ib_get_native_port_mdev(struct mlx5_ib_dev *ibdev, u32 ib_port_num, u32 *native_port_num) @@ -554,7 +575,7 @@ static int mlx5_query_port_roce(struct ib_device *device, u32 port_num, if (!put_mdev) goto out; - ndev = mlx5_ib_get_netdev(device, port_num); + ndev = ib_device_get_netdev(device, port_num); if (!ndev) goto out; @@ -3185,6 +3206,60 @@ static void get_dev_fw_str(struct ib_device *ibdev, char *str) fw_rev_sub(dev->mdev)); } +static int lag_event(struct notifier_block *nb, unsigned long event, void *data) +{ + struct mlx5_ib_dev *dev = container_of(nb, struct mlx5_ib_dev, + lag_events); + struct mlx5_core_dev *mdev = dev->mdev; + struct mlx5_ib_port *port; + struct net_device *ndev; + int i, err; + int portnum; + + portnum = 0; + switch (event) { + case MLX5_DRIVER_EVENT_ACTIVE_BACKUP_LAG_CHANGE_LOWERSTATE: + ndev = data; + if (ndev) { + if (!mlx5_lag_is_roce(mdev)) { + // sriov lag + for (i = 0; i < dev->num_ports; i++) { + port = &dev->port[i]; + if (port->rep && port->rep->vport == + MLX5_VPORT_UPLINK) { + portnum = i; + break; + } + } + } + err = ib_device_set_netdev(&dev->ib_dev, ndev, + portnum + 1); + dev_put(ndev); + if (err) + return err; + /* Rescan gids after new netdev assignment */ + rdma_roce_rescan_device(&dev->ib_dev); + } + break; + default: + return NOTIFY_DONE; + } + return NOTIFY_OK; +} + +static void mlx5e_lag_event_register(struct mlx5_ib_dev *dev) +{ + dev->lag_events.notifier_call = lag_event; + blocking_notifier_chain_register(&dev->mdev->priv.lag_nh, + &dev->lag_events); +} + +static void mlx5e_lag_event_unregister(struct mlx5_ib_dev *dev) +{ + blocking_notifier_chain_unregister(&dev->mdev->priv.lag_nh, + &dev->lag_events); +} + static int mlx5_eth_lag_init(struct mlx5_ib_dev *dev) { struct mlx5_core_dev *mdev = dev->mdev; @@ -3206,6 +3281,7 @@ static int mlx5_eth_lag_init(struct mlx5_ib_dev *dev) goto err_destroy_vport_lag; } + mlx5e_lag_event_register(dev); dev->flow_db->lag_demux_ft = ft; dev->lag_ports = mlx5_lag_get_num_ports(mdev); dev->lag_active = true; @@ -3223,6 +3299,7 @@ static void mlx5_eth_lag_cleanup(struct mlx5_ib_dev *dev) if (dev->lag_active) { dev->lag_active = false; + mlx5e_lag_event_unregister(dev); mlx5_destroy_flow_table(dev->flow_db->lag_demux_ft); dev->flow_db->lag_demux_ft = NULL; @@ -3937,7 +4014,6 @@ static int mlx5_ib_stage_init_init(struct mlx5_ib_dev *dev) for (i = 0; i < dev->num_ports; i++) { spin_lock_init(&dev->port[i].mp.mpi_lock); - rwlock_init(&dev->port[i].roce.netdev_lock); dev->port[i].roce.dev = dev; dev->port[i].roce.native_port_num = i + 1; dev->port[i].roce.last_port_state = IB_PORT_DOWN; @@ -4202,7 +4278,6 @@ static const struct ib_device_ops mlx5_ib_dev_common_roce_ops = { .create_wq = mlx5_ib_create_wq, .destroy_rwq_ind_table = mlx5_ib_destroy_rwq_ind_table, .destroy_wq = mlx5_ib_destroy_wq, - .get_netdev = mlx5_ib_get_netdev, .modify_wq = mlx5_ib_modify_wq, INIT_RDMA_OBJ_SIZE(ib_rwq_ind_table, mlx5_ib_rwq_ind_table, diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index 47df41800ac5..262ade50be57 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -901,8 +901,6 @@ struct mlx5_roce { /* Protect mlx5_ib_get_netdev from invoking dev_hold() with a NULL * netdev pointer */ - rwlock_t netdev_lock; - struct net_device *netdev; struct notifier_block nb; struct netdev_net_notifier nn; struct notifier_block mdev_nb; @@ -1151,6 +1149,7 @@ struct mlx5_ib_dev { /* protect accessing data_direct_dev */ struct mutex data_direct_lock; struct notifier_block mdev_events; + struct notifier_block lag_events; int num_ports; /* serialize update of capability mask */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c index cf8045b92689..8577db3308cc 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c @@ -445,6 +445,34 @@ static int _mlx5_modify_lag(struct mlx5_lag *ldev, u8 *ports) return mlx5_cmd_modify_lag(dev0, ldev->ports, ports); } +static struct net_device *mlx5_lag_active_backup_get_netdev(struct mlx5_core_dev *dev) +{ + struct net_device *ndev = NULL; + struct mlx5_lag *ldev; + unsigned long flags; + int i; + + spin_lock_irqsave(&lag_lock, flags); + ldev = mlx5_lag_dev(dev); + + if (!ldev) + goto unlock; + + for (i = 0; i < ldev->ports; i++) + if (ldev->tracker.netdev_state[i].tx_enabled) + ndev = ldev->pf[i].netdev; + if (!ndev) + ndev = ldev->pf[ldev->ports - 1].netdev; + + if (ndev) + dev_hold(ndev); + +unlock: + spin_unlock_irqrestore(&lag_lock, flags); + + return ndev; +} + void mlx5_modify_lag(struct mlx5_lag *ldev, struct lag_tracker *tracker) { @@ -477,9 +505,18 @@ void mlx5_modify_lag(struct mlx5_lag *ldev, } } - if (tracker->tx_type == NETDEV_LAG_TX_TYPE_ACTIVEBACKUP && - !(ldev->mode == MLX5_LAG_MODE_ROCE)) - mlx5_lag_drop_rule_setup(ldev, tracker); + if (tracker->tx_type == NETDEV_LAG_TX_TYPE_ACTIVEBACKUP) { + struct net_device *ndev = mlx5_lag_active_backup_get_netdev(dev0); + + if(!(ldev->mode == MLX5_LAG_MODE_ROCE)) + mlx5_lag_drop_rule_setup(ldev, tracker); + /** Only sriov and roce lag should have tracker->tx_type set so + * no need to check the mode + */ + blocking_notifier_call_chain(&dev0->priv.lag_nh, + MLX5_DRIVER_EVENT_ACTIVE_BACKUP_LAG_CHANGE_LOWERSTATE, + ndev); + } } static int mlx5_lag_set_port_sel_mode_roce(struct mlx5_lag *ldev, @@ -613,6 +650,7 @@ static int mlx5_create_lag(struct mlx5_lag *ldev, mlx5_core_err(dev0, "Failed to deactivate RoCE LAG; driver restart required\n"); } + BLOCKING_INIT_NOTIFIER_HEAD(&dev0->priv.lag_nh); return err; } @@ -1492,38 +1530,6 @@ void mlx5_lag_enable_change(struct mlx5_core_dev *dev) mlx5_queue_bond_work(ldev, 0); } -struct net_device *mlx5_lag_get_roce_netdev(struct mlx5_core_dev *dev) -{ - struct net_device *ndev = NULL; - struct mlx5_lag *ldev; - unsigned long flags; - int i; - - spin_lock_irqsave(&lag_lock, flags); - ldev = mlx5_lag_dev(dev); - - if (!(ldev && __mlx5_lag_is_roce(ldev))) - goto unlock; - - if (ldev->tracker.tx_type == NETDEV_LAG_TX_TYPE_ACTIVEBACKUP) { - for (i = 0; i < ldev->ports; i++) - if (ldev->tracker.netdev_state[i].tx_enabled) - ndev = ldev->pf[i].netdev; - if (!ndev) - ndev = ldev->pf[ldev->ports - 1].netdev; - } else { - ndev = ldev->pf[MLX5_LAG_P1].netdev; - } - if (ndev) - dev_hold(ndev); - -unlock: - spin_unlock_irqrestore(&lag_lock, flags); - - return ndev; -} -EXPORT_SYMBOL(mlx5_lag_get_roce_netdev); - u8 mlx5_lag_get_slave_port(struct mlx5_core_dev *dev, struct net_device *slave) { diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h index 57c9b18c3adb..dac33cfe9c0c 100644 --- a/include/linux/mlx5/device.h +++ b/include/linux/mlx5/device.h @@ -371,6 +371,7 @@ enum mlx5_driver_event { MLX5_DRIVER_EVENT_SF_PEER_DEVLINK, MLX5_DRIVER_EVENT_AFFILIATION_DONE, MLX5_DRIVER_EVENT_AFFILIATION_REMOVED, + MLX5_DRIVER_EVENT_ACTIVE_BACKUP_LAG_CHANGE_LOWERSTATE, }; enum { diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index a96438ded15f..46a7a3d11048 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -643,6 +643,7 @@ struct mlx5_priv { struct mlx5_sf_hw_table *sf_hw_table; struct mlx5_sf_table *sf_table; #endif + struct blocking_notifier_head lag_nh; }; enum mlx5_device_state { @@ -1181,7 +1182,6 @@ bool mlx5_lag_mode_is_hash(struct mlx5_core_dev *dev); bool mlx5_lag_is_master(struct mlx5_core_dev *dev); bool mlx5_lag_is_shared_fdb(struct mlx5_core_dev *dev); bool mlx5_lag_is_mpesw(struct mlx5_core_dev *dev); -struct net_device *mlx5_lag_get_roce_netdev(struct mlx5_core_dev *dev); u8 mlx5_lag_get_slave_port(struct mlx5_core_dev *dev, struct net_device *slave); int mlx5_lag_query_cong_counters(struct mlx5_core_dev *dev, diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index a1dcf812d787..aa8ede439905 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -4453,6 +4453,8 @@ struct net_device *ib_get_net_dev_by_params(struct ib_device *dev, u32 port, const struct sockaddr *addr); int ib_device_set_netdev(struct ib_device *ib_dev, struct net_device *ndev, unsigned int port); +struct net_device *ib_device_get_netdev(struct ib_device *ib_dev, + u32 port); struct ib_wq *ib_create_wq(struct ib_pd *pd, struct ib_wq_init_attr *init_attr); int ib_destroy_wq_user(struct ib_wq *wq, struct ib_udata *udata); From patchwork Fri Aug 30 07:31:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13784446 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2082.outbound.protection.outlook.com [40.107.236.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 92FF716B39E for ; Fri, 30 Aug 2024 07:32:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.82 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003125; cv=fail; b=Ecf4OWDYmd/KP7eIkpErMIBymEV8JKFqg7AGqno+XSrC2vU77JItlwn8NetrIe1x2J3XjBtKbCNwkMqDKysDoWxcjJ4JbDpc1/odEQzQwuPYJGbrFlTniIo15R65nSe+rhjtQ22okm33nhM3I8u7CWrA4tpgPXfddNqdpvBjiqA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003125; c=relaxed/simple; bh=tI5Vqg6WttKNUXXE6sGs/gz6VzalD7hxB9dKPvGHL2c=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=oI9jfc6Fhu3PMVBCtFQLEiN5fi+/+FEwLw6OzcvPEDZ5h0lKhax49RmERPMhR6zwKONV3RXkBAUiGKOPkd+fmRnPChREyXh7lN5cp7+uxtR6dMeopDr1XkTNR2Q0k/HlrHWy5fIKK0DgYPLUXNxBUAQ8VyffwW9gKNvySprt/7c= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=J2Zeyz2F; arc=fail smtp.client-ip=40.107.236.82 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="J2Zeyz2F" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Ucg9v0Wk2qC7A81wN5QuLJS3T88I8gHIAN76pgWp46ALxDNqGXTwlsZJqeTloXZEqRv2LxvBOZqUx9XQsOw+OZl5yUk7iuUGPexQj+6d40VH/JbTA2zwDBg6M4F8aXZInNkkBMtxQZ3VirsFatbd7askuqctAAUdruiIo6G+6lFRixIpa0+YX3gV24Hz1d01ZezQxM0Kw6S2aYUoGjT8uI5h6WkACkqY+eG/j4CU8c4J465tDn1UWyArgpmwHIulKXG3iVlsxBQNXPRoe4hINRwFjr13sXPGL6AT82j9V6h6EEB4AUn3K6VxDa5KIqJ2s750nkt1LWuyagyREigfNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/lQKnX3gSF6vCXhZ0ooU9bw5mXb/7dnBK4LPNMxpKMY=; b=y1Su30ZufTTIOQJ1uL7/Fwdo21yqX9MN9HLOZYn+j24sUCTpa5ognB+L/sPEAeQ7CxTla6e/YBXFVW/8TfDl0/4CEg/WaRGLNMGAp4NFquUHG72Zk2SOK5gZVzPUfaSF0deh5n3dKrkfJ+/nb7ahk6SG3Rlgumso0F4yi/0kzuntqL7xJ7HI/rIa4/i6mfrt2mvj8EX7GdPxQUteQUwRMAjAka4swnFFLqmCh8c6SbLQTBqpZfiGLwqy/nGCwM5IMdwZTpBXBMo//8UXiTXKVqgYSb0a9m9jZ5LFuvego1HRKW2oL1A20lnWMTD3O5mt0Ii06OA5ch4t87WLRWBufA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/lQKnX3gSF6vCXhZ0ooU9bw5mXb/7dnBK4LPNMxpKMY=; b=J2Zeyz2FOswZJpDMbOHt64LHhbKQmB4tvk8YfsCE4M4wzSJ35+pdXtKlsuD51dwiNEHMfKfhN5lNz+WoWm2oIPYDLP2Ouc5cPVmJ3JyHUGOJeVp3/2N8X5hasK+04cRiSkma8Y1F8tmyfdbh2ZTUv9YfJJeZpAWOfCZSQOzxhJzEkhIM35m+bzbslIzpQafHw7KTItVKx3zfsMU+KAsakIdBrKd9bTsWjP2OMQ4MRGU5gEMxGqu3LUX5z3b8SnMWhUmQOKSfCJQNs1euKwuvoEYXSX9WOsw60AjSUqE07Pspiai0WEn+PNqDcqrAVU5QQiKzIW3LiVOOBsoaszl25Q== Received: from MW4PR04CA0078.namprd04.prod.outlook.com (2603:10b6:303:6b::23) by PH7PR12MB9065.namprd12.prod.outlook.com (2603:10b6:510:1f7::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.19; Fri, 30 Aug 2024 07:31:59 +0000 Received: from CO1PEPF000044FA.namprd21.prod.outlook.com (2603:10b6:303:6b:cafe::2d) by MW4PR04CA0078.outlook.office365.com (2603:10b6:303:6b::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.20 via Frontend Transport; Fri, 30 Aug 2024 07:31:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF000044FA.mail.protection.outlook.com (10.167.241.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.2 via Frontend Transport; Fri, 30 Aug 2024 07:31:58 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:49 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:49 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 30 Aug 2024 00:31:47 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next v2 6/7] RDMA/nldev: Add support for RDMA monitoring Date: Fri, 30 Aug 2024 10:31:29 +0300 Message-ID: <20240830073130.29982-7-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240830073130.29982-1-michaelgur@nvidia.com> References: <20240830073130.29982-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FA:EE_|PH7PR12MB9065:EE_ X-MS-Office365-Filtering-Correlation-Id: e84a3576-36f4-4926-8079-08dcc8c5d51e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|82310400026|376014; X-Microsoft-Antispam-Message-Info: 40AdfjrWNk6pL+iA2DgyPsFiwZkOk2zAOkTALMA52uxy3inWp5/uapPKXtD6ylMIFzmLKgnY3JDefAetGX6JUaRsQEjFeRbeX+qHiKX1I1xKST92bpFSQF5PVvTHqA4xHp3TPJFhVGSNGEbFXLZZulcE1tfoaj8y4ljAfWXFFID2AnBPTFjhh0pV8pVcGbXjSTrY7ex+NbfxFb368FC14IPpsqEhjItrdCeKDx7/uSdIUWTdwveVa+ia7zU+7zBJDcEygYRWEm5IYv4d/m1VtD1GnFTR3xq8QXEcpO47g3T/EaZPIxDYjqOkjv0Um5LV6Szyc5XQnk7jbxTJG7VAyYnDbU6IkjDDF4qPz48cOcmNda8vXZ9r7X9p8XbiaTTOY2wZNmP4Y5PCykMY5TiEoYUplsjZy3iodwcIKADzqm14nj5NshKgtCLS0XbNCzunhoFZcuqWxNSWAVwnqpAz7hlnZmibYNdFLTMlYU/+VI+7PoqaEDmuz7sZ4cudJAEWoxW8VZQfND8t/MECldNcHbmSayNiW3BuD/BuZqnFaZG9x111zsbd7xWDQ0SolCyXynXUY04pQyf+wIT5uxWfvXQFTMUK7K5Q48O4aoBQjMteWMDk0y8S0E1552mksmQqd6iR601jTRjlwLKhAN5G6CIWzdk5E1OQeQmpVo2L7nnfuDCMZsRxaCHhW1nSnJrbydGHFs+hJ0pK5w2vy/dSv41jhIciopzovUgZUQywSg1PC1UQbjcmmYjDOc7srSLG7HlOnIGxJdjXYC2ih7dfqzpwwozXn0+csaRGOK1wbaj/NCR4V+MkiF9WY3uTpldk9ZKDS1tzzWA4+klkxmu/7/Y5oIAOgnLNTVG3T7VTIM1Nj3GYiUxKA577gnx5YZpGz8L6IJT6MQY1Af49JxkOi29qYdHajZxt5Grkag+K+8Ih8K0tJLOYSW6x8Ka1EcL5DBSD3RfUG/TLZ3W5TRU2x7ZlwxDxziI0OZ/jChj5ikjee8yLMnmBC3YsbydFfDg4+QYAHVpzBMqIja8VwZpN4Lf9h7Ye9xGJlVYQ8lJpPDIqkm/l7ZkKo5iE70DU0dyFfh5FTxq77es+wyG2dJtCsPr+IlHeRyS6LS9sovgRBSNPG5YHqOX53+iRbHf/pRWVRBuY59VuL611yCdYnWKwN32/7AdvF2mCP4myZH+TYZIjBrGnEhmb50u4NY4OD+EGafOFNx5+rpSWLCBSqQdrnEDlwoBPWUgL21yXWtfYxmcZ+QrGMqyTLXXzy0QPONT2zDQQ9oOjqLWaSHTwTE3kO3umfYVdz6Wjm5nkR3JZalU0mrLwbTZ8QqvsHox/wnLh2/mMsE1jVqavQCT4RdBB4l7lvZTLKrHfuVWWODCMOl1hyQwQsPz1M+PgADdSZkSjhKsCyr1iGHY3duVG1KxtGbQUk81ZOwpvsRudrhW/9xidEMz7zLJkzJM2ScDmRw/F X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(82310400026)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2024 07:31:58.8488 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e84a3576-36f4-4926-8079-08dcc8c5d51e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FA.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB9065 From: Chiara Meiohas Introduce a new netlink command to allow rdma event monitoring. The rdma events supported now are IB device registration/unregistration and net device attachment/detachment. Example output of rdma monitor and the commands which trigger the events: $ rdma monitor $ rmmod mlx5_ib [UNREGISTER] dev 3 [UNREGISTER] dev 0 $modprobe mlx5_ib [REGISTER] dev 4 [NETDEV_ATTACH] dev 4 port 1 netdev 4 [REGISTER] dev 5 [NETDEV_ATTACH] dev 5 port 1 netdev 5 $ devlink dev eswitch set pci/0000:08:00.0 mode switchdev [UNREGISTER] dev 4 [REGISTER] dev 6 [NETDEV_ATTACH] dev 6 port 6 netdev 4 $ echo 4 > /sys/class/net/eth2/device/sriov_numvfs [NETDEV_ATTACH] dev 6 port 2 netdev 7 [NETDEV_ATTACH] dev 6 port 3 netdev 8 [NETDEV_ATTACH] dev 6 port 4 netdev 9 [NETDEV_ATTACH] dev 6 port 5 netdev 10 [REGISTER] dev 7 [NETDEV_ATTACH] dev 7 port 1 netdev 11 [REGISTER] dev 8 [NETDEV_ATTACH] dev 8 port 1 netdev 12 [REGISTER] dev 9 [NETDEV_ATTACH] dev 9 port 1 netdev 13 [REGISTER] dev 10 [NETDEV_ATTACH] dev 10 port 1 netdev 14 $ echo 0 > /sys/class/net/eth2/device/sriov_numvfs [UNREGISTER] dev 7 [UNREGISTER] dev 8 [UNREGISTER] dev 9 [UNREGISTER] dev 10 [NETDEV_DETACH] dev 6 port 2 [NETDEV_DETACH] dev 6 port 3 [NETDEV_DETACH] dev 6 port 4 [NETDEV_DETACH] dev 6 port 5 Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/core/device.c | 38 ++++++++++ drivers/infiniband/core/netlink.c | 1 + drivers/infiniband/core/nldev.c | 118 ++++++++++++++++++++++++++++++ include/rdma/rdma_netlink.h | 12 +++ include/uapi/rdma/rdma_netlink.h | 15 ++++ 5 files changed, 184 insertions(+) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index b2fc5a13577c..2113eb7c7573 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -1351,6 +1351,30 @@ static void prevent_dealloc_device(struct ib_device *ib_dev) { } +static void ib_device_notify_register(struct ib_device *device) +{ + struct net_device *netdev; + u32 port; + int ret; + + ret = rdma_nl_notify_event(device, 0, RDMA_REGISTER_EVENT); + if (ret) + return; + + rdma_for_each_port(device, port) { + netdev = ib_device_get_netdev(device, port); + if (!netdev) + continue; + + ret = rdma_nl_notify_event(device, port, + RDMA_NETDEV_ATTACH_EVENT); + dev_put(netdev); + if (ret) + return; + } + return; +} + /** * ib_register_device - Register an IB device with IB core * @device: Device to register @@ -1449,6 +1473,8 @@ int ib_register_device(struct ib_device *device, const char *name, dev_set_uevent_suppress(&device->dev, false); /* Mark for userspace that device is ready */ kobject_uevent(&device->dev.kobj, KOBJ_ADD); + + ib_device_notify_register(device); ib_device_put(device); return 0; @@ -1491,6 +1517,7 @@ static void __ib_unregister_device(struct ib_device *ib_dev) goto out; disable_device(ib_dev); + rdma_nl_notify_event(ib_dev, 0, RDMA_UNREGISTER_EVENT); /* Expedite removing unregistered pointers from the hash table */ free_netdevs(ib_dev); @@ -2159,6 +2186,7 @@ static void add_ndev_hash(struct ib_port_data *pdata) int ib_device_set_netdev(struct ib_device *ib_dev, struct net_device *ndev, u32 port) { + enum rdma_nl_notify_event_type etype; struct net_device *old_ndev; struct ib_port_data *pdata; unsigned long flags; @@ -2190,6 +2218,16 @@ int ib_device_set_netdev(struct ib_device *ib_dev, struct net_device *ndev, spin_unlock_irqrestore(&pdata->netdev_lock, flags); add_ndev_hash(pdata); + + down_read(&devices_rwsem); + if (xa_get_mark(&devices, ib_dev->index, DEVICE_REGISTERED) && + xa_load(&devices, ib_dev->index) == ib_dev) { + etype = ndev ? + RDMA_NETDEV_ATTACH_EVENT : RDMA_NETDEV_DETACH_EVENT; + rdma_nl_notify_event(ib_dev, port, etype); + } + up_read(&devices_rwsem); + return 0; } EXPORT_SYMBOL(ib_device_set_netdev); diff --git a/drivers/infiniband/core/netlink.c b/drivers/infiniband/core/netlink.c index ae2db0c70788..def14c54b648 100644 --- a/drivers/infiniband/core/netlink.c +++ b/drivers/infiniband/core/netlink.c @@ -311,6 +311,7 @@ int rdma_nl_net_init(struct rdma_dev_net *rnet) struct net *net = read_pnet(&rnet->net); struct netlink_kernel_cfg cfg = { .input = rdma_nl_rcv, + .flags = NL_CFG_F_NONROOT_RECV, }; struct sock *nls; diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index 4d4a1f90e484..b0354bb8ba0d 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -170,6 +170,7 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = { [RDMA_NLDEV_ATTR_DEV_TYPE] = { .type = NLA_U8 }, [RDMA_NLDEV_ATTR_PARENT_NAME] = { .type = NLA_NUL_STRING }, [RDMA_NLDEV_ATTR_NAME_ASSIGN_TYPE] = { .type = NLA_U8 }, + [RDMA_NLDEV_ATTR_EVENT_TYPE] = { .type = NLA_U8 }, }; static int put_driver_name_print_type(struct sk_buff *msg, const char *name, @@ -2722,6 +2723,123 @@ static const struct rdma_nl_cbs nldev_cb_table[RDMA_NLDEV_NUM_OPS] = { }, }; +static int fill_mon_netdev_association(struct sk_buff *msg, + struct ib_device *device, u32 port, + const struct net *net) +{ + struct net_device *netdev = ib_device_get_netdev(device, port); + int ret = 0; + + if (netdev && !net_eq(dev_net(netdev), net)) + goto out; + + ret = nla_put_u32(msg, RDMA_NLDEV_ATTR_DEV_INDEX, device->index); + if (ret) + goto out; + ret = nla_put_u32(msg, RDMA_NLDEV_ATTR_PORT_INDEX, port); + if (ret) + goto out; + if (netdev) + ret = nla_put_u32(msg, RDMA_NLDEV_ATTR_NDEV_INDEX, + netdev->ifindex); + +out: + dev_put(netdev); + return ret; +} + +static int fill_mon_register(struct sk_buff *msg, struct ib_device *device, + const struct net *net) +{ + return nla_put_u32(msg, RDMA_NLDEV_ATTR_DEV_INDEX, device->index); +} + +static void rdma_nl_notify_err_msg(struct ib_device *device, u32 port_num, + enum rdma_nl_notify_event_type type) +{ + struct net_device *netdev; + + switch (type) { + case RDMA_REGISTER_EVENT: + dev_warn_ratelimited(&device->dev, + "Failed to send RDMA monitor register device event\n"); + break; + case RDMA_UNREGISTER_EVENT: + dev_warn_ratelimited(&device->dev, + "Failed to send RDMA monitor unregister device event\n"); + break; + case RDMA_NETDEV_ATTACH_EVENT: + netdev = ib_device_get_netdev(device, port_num); + dev_warn_ratelimited(&device->dev, + "Failed to send RDMA monitor netdev attach event: port %d netdev %d\n", + port_num, netdev->ifindex); + dev_put(netdev); + break; + case RDMA_NETDEV_DETACH_EVENT: + dev_warn_ratelimited(&device->dev, + "Failed to send RDMA monitor netdev detach event: port %d\n", + port_num); + default: + break; + }; +} + +int rdma_nl_notify_event(struct ib_device *device, u32 port_num, + enum rdma_nl_notify_event_type type) +{ + struct sk_buff *skb; + struct net *net; + int ret = 0; + void *nlh; + + net = read_pnet(&device->coredev.rdma_net); + if (!net) + return -EINVAL; + + skb = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL); + if (!skb) + return -ENOMEM; + nlh = nlmsg_put(skb, 0, 0, + RDMA_NL_GET_TYPE(RDMA_NL_NLDEV, RDMA_NLDEV_CMD_MONITOR), + 0, 0); + + switch (type) { + case RDMA_REGISTER_EVENT: + case RDMA_UNREGISTER_EVENT: + ret = fill_mon_register(skb, device, net); + if (ret) + goto err_free; + break; + case RDMA_NETDEV_ATTACH_EVENT: + case RDMA_NETDEV_DETACH_EVENT: + ret = fill_mon_netdev_association(skb, device, + port_num, net); + if (ret) + goto err_free; + break; + default: + ret = -EINVAL; + goto err_free; + } + + ret = nla_put_u8(skb, RDMA_NLDEV_ATTR_EVENT_TYPE, type); + if (ret) + goto err_free; + + nlmsg_end(skb, nlh); + ret = rdma_nl_multicast(net, skb, RDMA_NL_GROUP_NOTIFY, GFP_KERNEL); + if (ret && ret != -ESRCH) { + skb = NULL; /* skb is freed in the netlink send-op handling */ + goto err_free; + } + return 0; + +err_free: + rdma_nl_notify_err_msg(device, port_num, type); + nlmsg_free(skb); + return ret; +} + void __init nldev_init(void) { rdma_nl_register(RDMA_NL_NLDEV, nldev_cb_table); diff --git a/include/rdma/rdma_netlink.h b/include/rdma/rdma_netlink.h index c2a79aeee113..326deaf56d5d 100644 --- a/include/rdma/rdma_netlink.h +++ b/include/rdma/rdma_netlink.h @@ -6,6 +6,8 @@ #include #include +struct ib_device; + enum { RDMA_NLDEV_ATTR_EMPTY_STRING = 1, RDMA_NLDEV_ATTR_ENTRY_STRLEN = 16, @@ -110,6 +112,16 @@ int rdma_nl_multicast(struct net *net, struct sk_buff *skb, */ bool rdma_nl_chk_listeners(unsigned int group); +/** + * Prepare and send an event message + * @ib: the IB device which triggered the event + * @port_num: the port number which triggered the event - 0 if unused + * @type: the event type + * Returns 0 on success or a negative error code + */ +int rdma_nl_notify_event(struct ib_device *ib, u32 port_num, + enum rdma_nl_notify_event_type type); + struct rdma_link_ops { struct list_head list; const char *type; diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h index 2f37568f5556..5f9636d26050 100644 --- a/include/uapi/rdma/rdma_netlink.h +++ b/include/uapi/rdma/rdma_netlink.h @@ -15,6 +15,7 @@ enum { enum { RDMA_NL_GROUP_IWPM = 2, RDMA_NL_GROUP_LS, + RDMA_NL_GROUP_NOTIFY, RDMA_NL_NUM_GROUPS }; @@ -305,6 +306,8 @@ enum rdma_nldev_command { RDMA_NLDEV_CMD_DELDEV, + RDMA_NLDEV_CMD_MONITOR, + RDMA_NLDEV_NUM_OPS }; @@ -574,6 +577,8 @@ enum rdma_nldev_attr { RDMA_NLDEV_ATTR_NAME_ASSIGN_TYPE, /* u8 */ + RDMA_NLDEV_ATTR_EVENT_TYPE, /* u8 */ + /* * Always the end */ @@ -624,4 +629,14 @@ enum rdma_nl_name_assign_type { RDMA_NAME_ASSIGN_TYPE_USER = 1, /* Provided by user-space */ }; +/* + * Supported rdma monitoring event types. + */ +enum rdma_nl_notify_event_type { + RDMA_REGISTER_EVENT, + RDMA_UNREGISTER_EVENT, + RDMA_NETDEV_ATTACH_EVENT, + RDMA_NETDEV_DETACH_EVENT, +}; + #endif /* _UAPI_RDMA_NETLINK_H */ From patchwork Fri Aug 30 07:31:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13784448 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2083.outbound.protection.outlook.com [40.107.95.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15E1C16B38E for ; Fri, 30 Aug 2024 07:32:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.95.83 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003126; cv=fail; b=sqluxtdMhUT9Y3Isvw4sr0qjQ7lteSXQY/8tTL6ICFimoKrFG/YqAf2WjZdK1bBkeXTBj9e9+mvQWUJXjr85sp3RUaVaq50IGOux5T007Dt1U1TlDEaNtpfZSzwTv71m+3atCaRoSmBiaHXLNxvY/Nt5TQrS2kYDBQdxcXHj1q0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725003126; c=relaxed/simple; bh=CZUQl0p/Ew155LkgJ0AXKQn2F7dZlBk4kaLbl1ajHXk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=cynP/dPgQ5WljfiqaG+tSpfzSUxYz7wLrvtrMen2hGyTR4jdyWexek9uceV+y/x7PS3Q9zrObCEfboKqMAFpBGalQoMTk9t/zcfCsCLhzA4Bzh/3x8Zoa8vgp9w46dDHtwbXJGDglfsB6W9WwoZwUpf63fP0wAVhJR1JMoyj4vQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=N4lRlvZa; arc=fail smtp.client-ip=40.107.95.83 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="N4lRlvZa" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aeEHhRVf97/1xoOeJcM1ywfaLb68mDo4HWRRBpZnoVfh6hF3BtONHe6Igjsbtce1JdBJpnz5ys8MoTUWzpxXfe67CAI+kEl8ubJvTz2ZnaBZtEQarkQFMP2TJpuMCYxDM09cx+9u0svbR7ZbWe3LTF39o4JlpJZzZT0qK1AP0TroZK7bgx8qJGGTmg86yDS2JZCjOywMwQjUlXP1CJpESQC+ZfML3YN0uXyKYtQn+JBKkT3wPCuCRyH9MOx92sLsiELurvc263U5YqMWoGKe7aYU6pgE5oLwApxSvgXh/FR47z+csrnJ6KuYOyY2t8599UmlIrZg8dejUV9eHGC4Zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4hTEamW2IgIuljHfKT1nTk28fOwxJW5ItzWsU+FaB+Y=; b=JivvYl8WzbOZZqDhOfOsaxPplk4KICyv+Da/5YSE77WuZ3lmTVfBdnf6a/+9Zv6DGfc79447ARvaO/MIk9k2Qs3zUlJAyEDE6KRiecdX06igfdNcATK5xFUOShfHIEXMat/JDzicnXToq2U7ScOVic2wILMQLPpCOO2S+17NlQ8Dr+LYP4MtNkHFrwT8Ar02EzUq0pw1kS/cIKPXlFuS+WudF1ZtmQDAOMqfgaIRKDr16EM3T48ULlsFSN0DpGvbwe9WaSv+OO/zahlCtYfT/aYF0NrGeUw0CoH2Ns56clvgKdq/aWOes9ABQCsDqP4y7J9T03NoJCPip9sfbSbdLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4hTEamW2IgIuljHfKT1nTk28fOwxJW5ItzWsU+FaB+Y=; b=N4lRlvZaDF+NoBRlZ+SSANgX/bP1rMV+VQnaxLk4EbO4KxucKeit3zHvQnsai6TynFHOpyI6UDyzQMM7pqiHN7z/cH5Raq7BVrG1j7q0iVq/peq5r2+Wd5Hg2cgOdayo9JOtm2S+47EpVY0ftj7ViSyVrQoGsz8Y/EXKyscgqHJ+2A3x5CoAkmweJMhP1ZOo8Dz7OPV+UGpkI5frGRmA7lh/DG8wS/wRcJ+FDe1gjeuWV8rLeM340faZ974ZXO14aPckkVKiN1K72LL92ugAvzoo4QpituVgHT0hUZ9+MpYk5m0RRdF5xl24RnQlXqolrBXRGVLDhdpwGb+jD9L61Q== Received: from MW4PR03CA0209.namprd03.prod.outlook.com (2603:10b6:303:b8::34) by DM4PR12MB7504.namprd12.prod.outlook.com (2603:10b6:8:110::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.20; Fri, 30 Aug 2024 07:32:01 +0000 Received: from CO1PEPF000044FD.namprd21.prod.outlook.com (2603:10b6:303:b8:cafe::8) by MW4PR03CA0209.outlook.office365.com (2603:10b6:303:b8::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.28 via Frontend Transport; Fri, 30 Aug 2024 07:32:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF000044FD.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.2 via Frontend Transport; Fri, 30 Aug 2024 07:32:01 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:51 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 30 Aug 2024 00:31:51 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 30 Aug 2024 00:31:49 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next v2 7/7] RDMA/nldev: Expose whether RDMA monitoring is supported Date: Fri, 30 Aug 2024 10:31:30 +0300 Message-ID: <20240830073130.29982-8-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240830073130.29982-1-michaelgur@nvidia.com> References: <20240830073130.29982-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FD:EE_|DM4PR12MB7504:EE_ X-MS-Office365-Filtering-Correlation-Id: 8bb76bb0-37a9-4de9-821b-08dcc8c5d692 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|376014|82310400026|1800799024; X-Microsoft-Antispam-Message-Info: 5ftHe8cexXUkQbU0Ff7uc6qceflpiZLTJKSPOElXLFx2wppY2xpLA5VcUXWMj58PcpCLwcy24b5JOLTJowAVSJ69rTTJpDLWRRxU2F5QR6BdRaadSRKeZM0ErDxP9bC2mlHdRMPGCgLQCPBIFBmiPdO7gz2Gu7mmx7we9c3GSArnTTi9eAAZAvN+YMjJNdYI5xDP3U+cH8eMHaLpNxxnb1ARle8O+Jc0VfqnuG+zzuXKv3F+4vmD+0t1H6s1rm+Ib8BPSyUkeX5SGsynIMw7qQujT75WhDpdU7TTTWf8B0AVFgCx4Yt/PlBVo0OVPGO4uvwErPKrSdHb0pBTQYkR7oJkeGZ0pW5NC5hfrCLhDj3i/tXIw/D+0kLPI0HfDVyrSAw/xvJ0190nyex2hacCM9CZb+hJZlGlHuqmwnYY4nwm8lvHI4DU6LbHlwpjT8FVLCv34TZlPMQd2yP1K9ax5m84f5Oc3kEGMkrpQHX8PplM5lN3N+Zyq65/K9TLwnmg/LT6FmF8RY3cWF3pfB2VkJOtN0b2X43F0JWwI4iNQfIiGwrIOgTrQcmjmYU/ToVQNesXmBUx1jMQifPpurtxb2s29yJjho+n5fczSCj907+weJ7CpTePg5UP9IAPohJRkfP8xjXvRNXGySUYmGM6Q7vYh/dZy+/b5io2pe57whuvuV30BwFR21n3VcAUVznqh/2QgtHxrUQWudxlgfBZdlKpbOYj6IOukIuAi9elbuYz2lF+5Sk063oHNta3Z2B+skJMFzs/TXpchW5kbMMuY6uL5gTcz87620HZDmPUU7aB5Cw2CZuTYI/6vChxlalWcOE64LSD86B9jL3TzZwxWml5g4LvgIDuKvNELoYv2Fax0w50wELJoXmGZBv1VywHiUSgRzLOtvzXdteeJETtU6EmCtu26oOADmLH1QQQjca4Bjr9F/O8Ktl6vgr2U6t+Jvf6OmWlgCGwx5ccj1+xlRx40C60x7IsWlmYYkMes6yH2JtS0lWSRcUGGnTZHiyepVhouYHplBLJxjz4iYZyw6PY5BJIHUoW7xYyzBxSoS/c72rvx1IaWBKHPrYpUPaxv1UKKid5ZJxF7hiEZbLdYOw7mtpaZdVED8LolD9QMKuZ8xGBUGoRdfUCGNH/EhEUZWvMQdVWHootbQmmb0CQ209wKS9RM5S+e+OIKk5Gr9b9rX0E9PhXgZhl5u+beeqyXSPfY0ZtaIn0FwajhzQsyQy9n6STRA1qCR/yqoYmgiCkRkknvq+QLbNp2jPI5/iWbKzvSZ9b5mbnnRZGFM5jAZjDtp0DZeVGlax+p5ML5cWpprBOFwW8FOypLJGYtj291dFjW+oeWNCYTS1TRFAsRvwN+gzoCoX0O37HwBPSm/4PC06AqwyBG/MK9kqjlTlJe/zKd383W8bqFBc6sB+qUsJPCqEhQI6w+UBQ6t3MoDlzpczKndtje84K9lhLCSrH X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(376014)(82310400026)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2024 07:32:01.2868 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8bb76bb0-37a9-4de9-821b-08dcc8c5d692 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FD.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7504 From: Chiara Meiohas Extend the "rdma sys" command to display whether RDMA monitoring is supported. RDMA monitoring is not supported in mlx4 because it does not use the ib_device_set_netdev() API, which sends the RDMA events. Example output for kernel where monitoring is supported: $ rdma sys show netns shared privileged-qkey off monitor on copy-on-fork on Example output for kernel where monitoring is not supported: $ rdma sys show netns shared privileged-qkey off monitor off copy-on-fork on Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/core/nldev.c | 6 ++++++ include/uapi/rdma/rdma_netlink.h | 1 + 2 files changed, 7 insertions(+) diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index b0354bb8ba0d..ee19bcd1cb1e 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -1952,6 +1952,12 @@ static int nldev_sys_get_doit(struct sk_buff *skb, struct nlmsghdr *nlh, nlmsg_free(msg); return err; } + + err = nla_put_u8(msg, RDMA_NLDEV_SYS_ATTR_MONITOR_MODE, 1); + if (err) { + nlmsg_free(msg); + return err; + } /* * Copy-on-fork is supported. * See commits: diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h index 5f9636d26050..39be09c0ffbb 100644 --- a/include/uapi/rdma/rdma_netlink.h +++ b/include/uapi/rdma/rdma_netlink.h @@ -579,6 +579,7 @@ enum rdma_nldev_attr { RDMA_NLDEV_ATTR_EVENT_TYPE, /* u8 */ + RDMA_NLDEV_SYS_ATTR_MONITOR_MODE, /* u8 */ /* * Always the end */