From patchwork Mon Sep 9 17:30:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13797376 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2066.outbound.protection.outlook.com [40.107.93.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 453E617C205 for ; Mon, 9 Sep 2024 17:31:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.66 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903068; cv=fail; b=hnRCm9HMUjA6+fi5D9fugZ2DyRJC1rL1l+ksTe10WCSBDH5oxiVoj1phClCu3C6MkKqHUy1aiVxLE/HZUN3oHD5HsruQgj89fUR49CtQoA291vuHHE3LZje14VMe2CGWiD6ku239tXe1v7c3p5oyM1XtKzUyweTHyuvQvvd09GI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903068; c=relaxed/simple; bh=EQFkdF98ImVIpiz+D5z6Qiggi1JO6HFXBAoFITwzt6c=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=qLTnvsnSeTQ7Q+9uM/JjCIwzJJ7WBJEOLuQRhR1kxIUjsgzrpdu4pi2tQBNpjR53fFJAUNpDuX1zEPi1nbEq6viBSSEvPzfQhBfppafA9w9dsYITzkHCB0o//XcqZbjturVJEXd0GjvEotYvzhqJ3Pxyt9NsSjr69cmVsFSOxe0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=h07oTF4k; arc=fail smtp.client-ip=40.107.93.66 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="h07oTF4k" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=D+LNBJ6E1+fL5nLx7rIFSUcyn/tyZExFx6ndGXPgWIkyk3Q3WrS/LkLHx99zMHXojjCPIdlUh4jq2zB7q4BwgFg1fEjhFN5d0a9L/rkQou4SUD4sHy1mdrkqMwFpTECh2HZTqFsNXskMl+hG2x4GkB2ssmE3RxTdkREv4f28v1hFXA0gAdSpn6EUadbOA+9kk4QYqsAN+vc817bOP8/b0BlhaiRRa2Wu5rlN2tMpDS7FnhRcYg1/UKDkH8tiT+n9KVhIgImtQ+7Bw4+FuwBKYxiZWGdOUS4i0GqqC4Y3lz6sUpx8RUNVNMkmljVvq/L/J8K1J1r3O6v0fsVg6wp2oQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cY+B4Nsuo7WtmAU1ohBNCsMm+kGOeHPENTxEA13abtY=; b=B4Uu0+TOmmueQuD9+90XLMLRNOTCU2GxINm9hB7aDBoq8AuXAgd7gAfN5zS6KOQJ7MuNEsO6Mn2xbu8PZMcfRjWglzJtXcvsoiWZGifCzza/ZjBk4qr6cAqzGTWzsEAkRIPRyXCRKef5KEa0/kc2Buv0/f/Kz46vl8Ur5G+kbHhz9bjYDPpFSM6mtR5/C13Sc7L8zTdhdeUf0xuTItEI1YabRCeWwONHlSRyxOkWjeYCRKvjIBHMwke7Z6PNDNjDpo17F18SIDbgy2v0jsaBtEZUy1KTcTJvws4uYvphJ7zPwMTWNEtwMxjbggOye0zW8CaE+PezzdOT88qMTkIy1w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cY+B4Nsuo7WtmAU1ohBNCsMm+kGOeHPENTxEA13abtY=; b=h07oTF4ksvgJSRhw9vQJsCUcT25sq7XEpqTGdvII1Fif1Z+r86IX0YVHcmTPSU7gCq8VXeRECNyQn3UfERfuh8YMCjreS6JIvONTHgAOiLIHTn2vsHL4vjXhYLHE1ASXfL94AkwPjKun4DJIBWSHDdRmVdOuOUKxfTdbVJGhxEwCC9boweT4y2U8Cf/3U/FL2g58XD754a7dBNfZ5ep98XfqxfxReB23gJ54QFJAIg6F0yMaTYGeY99xKWzC17JH9iXRTJsNfZ4D/TEHLi1iG7HT01M8Taz6B+jC6X1zG1p6PXzq69iX7sBMI4OrXD9u2LVjWE8YeHvJWJjafeDgxQ== Received: from MN2PR17CA0027.namprd17.prod.outlook.com (2603:10b6:208:15e::40) by SN7PR12MB7252.namprd12.prod.outlook.com (2603:10b6:806:2ac::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.27; Mon, 9 Sep 2024 17:31:04 +0000 Received: from BL02EPF0001A0FD.namprd03.prod.outlook.com (2603:10b6:208:15e:cafe::46) by MN2PR17CA0027.outlook.office365.com (2603:10b6:208:15e::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.24 via Frontend Transport; Mon, 9 Sep 2024 17:31:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BL02EPF0001A0FD.mail.protection.outlook.com (10.167.242.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.13 via Frontend Transport; Mon, 9 Sep 2024 17:31:03 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:35 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:35 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Mon, 9 Sep 2024 10:30:33 -0700 From: Michael Guralnik To: CC: , , , , , , "Michael Guralnik" Subject: [PATCH v3 rdma-next 1/7] RDMA/mlx5: Check RoCE LAG status before getting netdev Date: Mon, 9 Sep 2024 20:30:19 +0300 Message-ID: <20240909173025.30422-2-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240909173025.30422-1-michaelgur@nvidia.com> References: <20240909173025.30422-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FD:EE_|SN7PR12MB7252:EE_ X-MS-Office365-Filtering-Correlation-Id: 10fe414a-f125-4a89-1bc3-08dcd0f52dfd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: KG9vffdqRsyUS4D8J6HTcUlKx4Nb7WhLKbJK3pN+zVahPmwwHOMHuP8+LNsj+pxZGcTIFzrjI+ZPXaPypnxGaq0RffnhfKQJpVy6IiCvRZx/MRmuVuyBKnAj75BbYhdzQ5gTlh5TPV6kLimjrfgACW/Lwfid3D2+MpE+HSybvzcIZBbZG+SJTLDkzEYPFk1zMjpzSAtUs3ekwPLDBRTojipy4v2N1wJnF2UdCNX9H3pi8aIB7QZyITZZbBDl6L7FzA2AueTOTylzq+92Ji4MhbV3bsSZ52YpUtJtvXKM9Roij9rPSBrYTyEHo+HZOHpHyZiesuNKVjox7ZqAYFWeXMX3Pgq0ffe+TMLYWoBpUpeUJ95LD0nkaS/YH4SJb0rFiwzs6pdcKPrTPf5DKvpCJt07yWErlJPrDReZ1odJKoyAWOQflQNNnUNBXfFcBKiFxtCGA5H5aHemhX+6ZhhWi3y0gZHjIZAMU5bnv1p1WF5//iUOg5Rm1s0xQGUfNrkQ/f9XiYXVaJUxEbY76s/pNNnKvigsxtAURCskyjO4dxJlOf49sJGA/FBFkj9jLiylmJbsMp+GxuMxErvmn84cHuSFITkRpU++WTsMpNgChfa3dwdacFlaejOGcWD1u96hnVDcgf+TXmei2l5CbnX4qhUTb9nF+ersHchWfPBfZollAaIGsYNJVLiewVTDFOoBTXFu6ihWLU8NHUTD2q+7DOnAGLyptERfTaABPmHIiLO8YIrlB+q9x9ykkQwZ7x2TXKZTrlRKVzlclf3FsKYzMSuasI9QKHFN2ECjPRCDF/auAHm2Zo5stVrgONOOA8MKi2gyORtOtazCfsFBzqKz/emTo+gtpBvbXX2xjAQpWMHl1JvqL5eLoqI3859O68HxgSfMzehGRxrsGQmKO/JBI8ijSrWTakSRuDWGvtpVmZrhEpwQI/ZGScbg5oqIP2VQs191yavnOY0oiJIt++V44qb9I4atnkorP987vh5DnH5h/fwWwR5p3qB0UkzLhodI//SfyMmzSF4rzjIxF7APhFfPyx4vbhyLShHLSNHoUeqNXBPteqMKusucpVsFS/dbWLQb9Ej3h+di2rmuo11S6vrqxCrJhfR6sz1OgpUuFKgcS+kyf9Cc5DLCRsjKuNfsGBHNmSqA9K+hUWhxMtmXVROhfBmCe7V1okTYkyeEVONH/2RnDIovZaX+Owzkl47jUGsXsGQjEI73g4qZnkRuuP6cJotkcWfeJ/CfV6XOfytxsnoOFFkIe8k5Al/flAi9rapj3yZ/YQLAo7RQNr3rdk6wtfwXIaoKT2H59Xl4stePeWW/dNWKDe+4SXjPZl3sGSMj7uwSFYUqNsUVuZhaJSJjBBjlGJUFvKBHcHvF5GLeu1IhJd5ukJN4YjbN7v38SJEqGLLJsakEkxNTYRcU7SpAj2hdgdwGYZ5fmYUSyjXkYal4t87HQkdsifyD2cne X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(1800799024)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2024 17:31:03.4707 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 10fe414a-f125-4a89-1bc3-08dcd0f52dfd X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A0FD.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7252 From: Mark Bloch Check if RoCE LAG is active before calling the LAG layer for netdev. This clarifies if LAG is active. No behavior changes with this patch. Signed-off-by: Mark Bloch Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky Reviewed-by: Kalesh AP --- drivers/infiniband/hw/mlx5/main.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index b85ad3c0bfa1..cdf1ce0f6b34 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -198,12 +198,18 @@ static int mlx5_netdev_event(struct notifier_block *this, case NETDEV_CHANGE: case NETDEV_UP: case NETDEV_DOWN: { - struct net_device *lag_ndev = mlx5_lag_get_roce_netdev(mdev); struct net_device *upper = NULL; - if (lag_ndev) { - upper = netdev_master_upper_dev_get(lag_ndev); - dev_put(lag_ndev); + if (mlx5_lag_is_roce(mdev)) { + struct net_device *lag_ndev; + + lag_ndev = mlx5_lag_get_roce_netdev(mdev); + if (lag_ndev) { + upper = netdev_master_upper_dev_get(lag_ndev); + dev_put(lag_ndev); + } else { + goto done; + } } if (ibdev->is_rep) @@ -257,9 +263,10 @@ static struct net_device *mlx5_ib_get_netdev(struct ib_device *device, if (!mdev) return NULL; - ndev = mlx5_lag_get_roce_netdev(mdev); - if (ndev) + if (mlx5_lag_is_roce(mdev)) { + ndev = mlx5_lag_get_roce_netdev(mdev); goto out; + } /* Ensure ndev does not disappear before we invoke dev_hold() */ From patchwork Mon Sep 9 17:30:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13797378 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2069.outbound.protection.outlook.com [40.107.223.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E8EE188CD6 for ; Mon, 9 Sep 2024 17:31:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.69 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903071; cv=fail; b=upd+rIIIKHES1Q4gZ9qbar2xrW83SgbKn+71oJU9vZfwUlwk11OtzzAgGBMudd+oYgMC508dFnwMUt5Ta4rB4cjfLWQNyNUIRu+aOGEEYsllIIgijj1I8I/i4BQfZW9otQVNNPsQKf9iRouf8DtNoAL9gNSbWGoUyPgJfv9ANn8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903071; c=relaxed/simple; bh=lLEMnxsxyqW9JKAvem3D0RAljr/RlvLLG90QJIOYFXs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=IAnafF/HI0Z3qRyUe0d+p+wSbPo2FERo6Z6ZcjZm7vpR3B6OXCVqB0M9JpjvAriO0aopvon8aMzIigCjCMadXS8u0DP3Vn/7IECD51AlS3bNdTd2D2mSqtbMW9QTLijzgGEipJ3Nm8kQs0HSjFtgOawwSbJ7ToFcpDBc8jX5xUw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=SEXw96Qv; arc=fail smtp.client-ip=40.107.223.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="SEXw96Qv" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Pd95QgkC6ZVb2O2MseGkiyq0ZsMSYRE1dNUH5L20/CvxXMT0ccq/MKrNhaH0EWz+7shcLez52KnyqprrP3+wz5onPR9lNwJtFULiYm3z/h2AHCO1VuEh1mBJUV+eostz4omNwTA6enXx5q9oSkCeMxbksk2xuAJrbC8VH58C8SCKkTLiy17t7QeH7T3shJ7Z6AWqIKfWFbBHLa3mHONZfjUDyu6lU5ZDC0pyUsp+ZfohtZJ+xAwkt/RabncD4XwL4+Mtn8h5Ep7ARh/qg8JOXtO85yoXcLluTFlRO/JSgBVqrk7q8s/Eos98wUgM+WMv0p8wVNiQWMOHd2legn8kRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j2OqhVHk8El4CAjcA45dc2j34faTtRc0H2tjll4NbU0=; b=RMU4wxTDrMTgMgSct7Q0vsJxjhEO0wj5ZXD8A3ZJZ+3CN9Kp58fFOtR1+W2T5RewX8A2HDW6l8UU3mwHyhY7tyvaso1XUCyN4noPcruR1k4gM9A7rRzluDNd4QsoUeqaWeqzVlLVyjQWlHgo3RLxv75s/OHWJBsS6Bb9MzSZOXtnPt5Cno7YPxgfSYneGSvQ/ur3mch4kqWxqoMWml1nBWB85nhr9BgOiQf/bzipYWk4pVXBrlwbdL1XfKgWAYfL2hTS1HsruJKwZY7gbAaRkyFONBy2/7rjNc3R4rkiZOtnmtuwtiNC/iiBWVKnKwrpI42PVngdldCSwx+yazhZLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j2OqhVHk8El4CAjcA45dc2j34faTtRc0H2tjll4NbU0=; b=SEXw96QvvlUiXE1By+el1RI3Vusb7aYwFXcjMpHdSpHeIC+JoxXyIuTSjEEgIbNOjVZ1ZpIGJ1FoN/3dPIB6ZpW8AY7FmwkFuKg6fFJPuTTtHxtyKyqRZZCTgDR8+MZNVojutEcx/vd3VH06oAz157FGa3yAuS04MI6e+CIHOa9/FiTvIs0QrTqtBzvROYQtprlTy/GvFz8m1KqgOLXCwX3CbsXZIOs+gT49keXvG+9DWMvXCwj9sv5CBsb5ZaE34COD6mnxbfxE6FD2+wRcpivmSUXNp12UfpMQ+A0lqSi9S39EC80fNlZD39gVWxe56rsTGLuvMNQ/b/IBoL5d2A== Received: from MN2PR17CA0014.namprd17.prod.outlook.com (2603:10b6:208:15e::27) by SA1PR12MB7152.namprd12.prod.outlook.com (2603:10b6:806:2b2::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.25; Mon, 9 Sep 2024 17:31:05 +0000 Received: from BL02EPF0001A0FD.namprd03.prod.outlook.com (2603:10b6:208:15e:cafe::6e) by MN2PR17CA0014.outlook.office365.com (2603:10b6:208:15e::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.24 via Frontend Transport; Mon, 9 Sep 2024 17:31:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BL02EPF0001A0FD.mail.protection.outlook.com (10.167.242.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.13 via Frontend Transport; Mon, 9 Sep 2024 17:31:04 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:38 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:38 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Mon, 9 Sep 2024 10:30:36 -0700 From: Michael Guralnik To: CC: , , , , , , "Michael Guralnik" Subject: [PATCH v3 rdma-next 2/7] RDMA/mlx5: Obtain upper net device only when needed Date: Mon, 9 Sep 2024 20:30:20 +0300 Message-ID: <20240909173025.30422-3-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240909173025.30422-1-michaelgur@nvidia.com> References: <20240909173025.30422-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FD:EE_|SA1PR12MB7152:EE_ X-MS-Office365-Filtering-Correlation-Id: cbba309d-f556-44b0-c575-08dcd0f52ea8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|376014|82310400026; X-Microsoft-Antispam-Message-Info: qF7fa00f1uM3Iei/iKNfOZiONh7bX+FZhfmLCJHtK9PLT+f7E6BJn477ktXyCtFnLhvjDeb8fvWn3PLmzGC+JD5C1F48GnVwHXsZTSkr8VBizHWovC1czwOFF42TbUdNbGqYRcvJjzdrySsUnEZAEC+cAFe9rMs7nLg1GYdLz0j8U9J5mCrTorji0SauRsSN7zjR3lBlxT6ALybHNDmpbukkSJRxwJeYrXK28ALjbbWPQ2z+iH2FC0Joo1e6zyXoBw+5PrnY6B3GaI7RGKsxKUPKBDc4eliC75DLCD0VRM2ubPsaODEcHa3IjgXULomw/rz+kbAmGKmmppDqnbsmcG4iqNkvLkArl2CM3jY3NtUY3uS3rw0FYvfjAQZcqlS7VXwXB5d2GMcDgpnjoXL6vnuh/DdooSgQZegpMwsM6x0pV/eq3l0lAReab2gzEsmTh6u3wmlLJpmgzdmdFAFclrXQS6JYUJTzZbGlmjmaofOW0mCikWyUhBe7B8agL5OVSX9D95MMdcohmRq0HsPqgyeMVazFxLfBEbEhTuca4xdvVNfnAhKlhrHdrSoFGJ7zfrrN2CVfG3Ou2FFroBxU//nG3vOCeFSum8OvS0EAjQPTI33k0A1QeNzylxqN3RENE0YwdJ/9UB4uVAMa/3V70sphUH+k6y53JecyVDOyAy1e5cCGvTkyhXFCFbyKHaxNTPMBNKr2mocprrjUC/0dgWsS737yFV4sYtkyBFbKsH3yrGyo7PEYCURQeTD4emr9OW0B+Q6w2YUZDAEYfEU6PbC19t/LFjgwThYjgNnueFFwJkjhdLS/u99QYgk2zP2fs2AjKaT1z6GV1xUE+91h8lJ4dwPj6MhHUhgH5Ig2uCsfp5My6jPSGjdAUfhY6PT6ZtOqx5YRdcwpVP3IT48znp9TR2yoZRJOkx4bfmRO2lkTNNZbBvTdhIgHK61zR+/+rC2hpIrJU9Wg+/6EFWmYpymVLguTTF3qNx/IEVa60bzt/t1LCPklonLDGfI9wy5+M+2jNvGOIxOVYkz8jgQQ36UcEPIvZ9WrmokKlF2CjYiJi+90QzYq+/HQtjSZ0k84QwZv4kavFP+/97rHfsupbKt3H6GLSCEk0jAj3zb9J5XXdFsYfmr4ptKAQ7qRKp4fA07jBPmkXL3wtOJ+9mVo6bgf41bzyjaxUuWoXZtzQtSy3iwZgyeidqBfqKZM9DXG9NNZdF0FKCqnYVkUaFa2iix46dqA16wC6Fv5QqLnFpeAJkfGXBhUm0YJEIC5p3dIoB6J1KpEIcfgLqmhE9wNhb97Vl89JifkJxVv1sScjYVUQxEkSyJfb9t16qsoLCncTNVZrq5TWNK2sbkuze98gh1ye+C5Igd4qzjoHHq8LaqsujR0MbgWafykTzOMBfNFz69YQxgYI+YEgy3Nc4F45bHJZFnGMpXErLrESyGg2YRiULDPUdD02mryHM/W7y9O X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2024 17:31:04.6114 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cbba309d-f556-44b0-c575-08dcd0f52ea8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A0FD.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7152 From: Mark Bloch Report the upper device's state as the RDMA port state only in RoCE LAG or switchdev LAG. Fixes: 27f9e0ccb6da ("net/mlx5: Lag, Add single RDMA device in multiport mode") Signed-off-by: Mark Bloch Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky Reviewed-by: Kalesh AP --- drivers/infiniband/hw/mlx5/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index cdf1ce0f6b34..c75cc3d14e74 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -558,7 +558,7 @@ static int mlx5_query_port_roce(struct ib_device *device, u32 port_num, if (!ndev) goto out; - if (dev->lag_active) { + if (mlx5_lag_is_roce(mdev) || mlx5_lag_is_sriov(mdev)) { rcu_read_lock(); upper = netdev_master_upper_dev_get_rcu(ndev); if (upper) { From patchwork Mon Sep 9 17:30:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13797377 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2083.outbound.protection.outlook.com [40.107.236.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B1B3F17ADF6 for ; Mon, 9 Sep 2024 17:31:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.83 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903069; cv=fail; b=UfvrHJ9JVWeFWusIMhQSBXVyD76Wkd3LrcJz2LvevuQY72+oMzOwzsKj4T4JrtkbVsk//HPKU4GvNtBft09ZbV+13xmGqMQVJWP2MaE/rw+8V8FSMcEaQCv2D1N7dHpAhX0ec2mby/OQE3s+ePPDX9vW5gtalWax8hz3y+TZm3M= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903069; c=relaxed/simple; bh=Je5C3f+CcZSAmB0AMT9meb69fQe7s/01w9SvWGMooE8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=K5mOaXad5xF9BsaCCQEqNsv5URdHl86rJHQhsEr+XOA/BNouLqwVSiPUvaQnYPmnuMNiQmU9ZG1nlzsuRYz8dwATerHQtZNUyuNiRwmj1MSL1Ln/HqxFAYxr2SudyXxstc2bnBlId1wbgUEqR6K+MoukYpzsdQlMvDoNUgqHeoQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ZDWF8KcG; arc=fail smtp.client-ip=40.107.236.83 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ZDWF8KcG" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=juiU0owayxJtHYUAgCekgfFWL9DL8zXroKex6Pt9AgHTryMAJ1t2sKXSGffETeZ9JEqYFb/JSowuEaEoux2cAW5WpazCNmZkiVgjdt1SpWYRxOJQ8sQNMt9pF4soAIN/oV9kJyi1nsV9S9ia/lOrfpL0L+4jWoHho/XA8i67xsY07RvhtAKyp56Mg3Akq0krA58HMUVWpouIW+qQj+8Kn1mdPcDm2p5dxJd+txTmnNeFlgPRtppf5OMC3fzNiLkuqccZgOarXNKwlv3MhT1Eyh5el7a6sAWPvRfQcEF52nJEcD1w/WKv3/MOHJ0c6aY3pniS5A4mh9HEEZ6WVFmAnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j3X8e2q5H7HzbLMNGEcnFtC+Vm0ucJAlldStCRceoaU=; b=OrNLzjJBDDNDs5OJ9Dgj195S8Qd9VOY7Dd4n8dxDOdjpwsJ3Eni53ga9CRDoi2kj/cxkhPSSoSzSutAuh6++U3C1pIiGMEmXEAMx7TpORCnOsW4SL4O8wNI6imLPcqcH8VwP/+XHqzbtLXYAeDGUNcc3yt4XF1Us3Idvq/1BDtp+1oAD/ZL7obruhjCJCKLRyIeyVgvJmnsG8dE3B9T1t6bhaqdBiXDlV70cr+BMRO9J/9sZOsCsfyuuMbnSORgglZKsde52IK6dCjMm4D1/IfHhhda9UZuMHlJOFcU1B7oi/27PwP300fdYXET7XTYHTu023XZOJPqTuC7LKsXZZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j3X8e2q5H7HzbLMNGEcnFtC+Vm0ucJAlldStCRceoaU=; b=ZDWF8KcGeygP9FewDeD0jSmpiDLsE3yiQVryBFmRhwaXm+c551oXySOsUYqhbUpZM8DVfRry6g4LjlQvrBdr5aZzjY5b7BuhBUwYzUNkhKFFGGEv/ZRzuU5tBIRifalROXKPp38ye2nZnWE+u2gA60ZFBNPvs0BhuD5dFT0E1Aw3FzTZuaqv3mtOKDgTWl+JssFhTpEqqEXkNZFSFslJLexL+QTSyKqc2JhWabAKDuQggB9q+/1lvTD26t3hcXkDiXUzBIOegWj0EfkEeseRJdBCE8r+E9QlkGL9wiV/BO07tnkMea4tg7SSDJzuc2f+nSLxfVnTnjHI6hgfpfwL4Q== Received: from MN2PR05CA0043.namprd05.prod.outlook.com (2603:10b6:208:236::12) by PH7PR12MB7234.namprd12.prod.outlook.com (2603:10b6:510:205::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.27; Mon, 9 Sep 2024 17:30:59 +0000 Received: from BL02EPF0002992A.namprd02.prod.outlook.com (2603:10b6:208:236:cafe::9e) by MN2PR05CA0043.outlook.office365.com (2603:10b6:208:236::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.23 via Frontend Transport; Mon, 9 Sep 2024 17:30:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF0002992A.mail.protection.outlook.com (10.167.249.55) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.13 via Frontend Transport; Mon, 9 Sep 2024 17:30:59 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:41 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:40 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Mon, 9 Sep 2024 10:30:38 -0700 From: Michael Guralnik To: CC: , , , , , , "Michael Guralnik" Subject: [PATCH v3 rdma-next 3/7] RDMA/mlx5: Initialize phys_port_cnt earlier in RDMA device creation Date: Mon, 9 Sep 2024 20:30:21 +0300 Message-ID: <20240909173025.30422-4-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240909173025.30422-1-michaelgur@nvidia.com> References: <20240909173025.30422-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0002992A:EE_|PH7PR12MB7234:EE_ X-MS-Office365-Filtering-Correlation-Id: f2f8785a-9cad-4cb6-1330-08dcd0f52b8f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: dDRHD0IE56WymZsv8kf44eI9pjxY8Ip/ipH4R4RDmghwBW6RoG6JTKNpZ4U1fm/MUp73SgDaO3qCXY9vHZ3NeidHf2jF/4iQnuJJSK4IzkKW0+f5Fch6TflwI961Ify2Ba7JHkrHTNNsUH7gkU2l2eobUti8UY8JJ9OB+Kth3IflHOePRmUeypgOuokW3tAGthWXINPiNffELGRBQTsc4OmzPw4KOJy3QTdsuD0JGxp00unQsutns2IHpiFRimoPwKlqqAuZVjTa4kDNNhhf+rjGlU5gLqE5zM9bL2nFn0reM5//3h+nGEbxrAbMyX4BsMyU1wVrOvU7UqlSZxPiWzWZ7BnHcXJXUDLRFCIFlH9zHc45+XGcH+6+8s05yfdpNib8mkWdVNX8knpst8Fo9G1nkD5htiP0ntnfWLyi6pZVyIXylhnsFaNIozUfXKMZ13ikmpPAfyVV1ARCfb5j9OnX5EVZORG6wDILfOW9ltt0yRyPqY8UufU4m6QU2h1h9kqPUHYsRtxOfyJSKAmAMfF42G399nTjQZrBxYtPz7vU76LwXWe5Ayh5KN25I8+98q6GyzrUqcSvVy8s+hG6vIQl3SQscrvKmXp6equNk9cgXeieTkBD5IfFrE7H0AHYsPtAN0BLP2jQ33E93TdO7ZwsZfi3Hp0Nrl3iMI3zOpVudg1xF/QzDpKAg5liWrgZ2wfglmMV5vNXeQFbkv1K/OT6FD/DQNA092IPF6/XXbpAeOMm1grsdqz89db1ZGwILgjr54yHuykQH6a0AEWIVTKrpTg/NVZ0EwX5g+hENmpbwiO/M5+S5bZJb6vipEssd7ZfPSxH3y0L63AmI4nz9XAnPpeHD17ltwfzOeyJSKyDyx7Pj/uFtr8S/U4cPw1DVSKiPnNpQvdCCmxNp1vjIPzkfV6UM0WVPO/TAMdUwxX6kbzhcs6mDAw8+jz2MlYb1e53RXEd6cO/FmZA8dh21xdH1mwmpcmxJ7vGt9hlXMQ4ARWpsBwJWx0eLDYRpahncn/1WKceBvXMm0Nqg/e7U8J17r+az4paE3PZkQ6j/lll0Bz6sTVNUHkyD7hb5VZbwOXomgP6y6YWxFxJgUffv+Q8vJpPgJ2ECq5sKEadQt9PxPA6B5HkYWaB6vrOGsd/nvhnxs/eFRMYaPJ2ytX8QNZJ8frVakJbF/YejDz9iFpl1AtfG0V3L4PpYLo/8RxshN61NwbTK3wAOoTqDSMeN3NzWdgSWBMfuKDDxHB53ABHHTTg0junixSGZOTRMHyLn8rR9AHPSxdq7ZNuxF1rBVFdoEns+zuHLofwfPoEKOE+FEnC8HEWoT8x9h8AqsBGa5CW5KZlvdkQIQo5oKQt+G6Xx+EMolXJVzmX8xiLGauLQ3AsUll5kyx/Dv1XC9Cdy7m4hNd9rP18ip92M0m0vh09FZVZaItXvxNLW+Xq51rWWxffPDrXkF8G4lpkaDLI X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(82310400026)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2024 17:30:59.4098 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f2f8785a-9cad-4cb6-1330-08dcd0f52b8f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0002992A.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7234 From: Chiara Meiohas phys_port_cnt of the IB device must be initialized before calling ib_device_set_netdev(). Previously, phys_port_cnt was initialized in the mlx5_ib init function. Remove this initialization to allow setting it separately, providing the flexibility to call ib_device_set_netdev before registering the IB device. Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/ib_rep.c | 1 + drivers/infiniband/hw/mlx5/main.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c index c7a4ee896121..1ad934685d80 100644 --- a/drivers/infiniband/hw/mlx5/ib_rep.c +++ b/drivers/infiniband/hw/mlx5/ib_rep.c @@ -104,6 +104,7 @@ mlx5_ib_vport_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep) ibdev->is_rep = true; vport_index = rep->vport_index; ibdev->port[vport_index].rep = rep; + ibdev->ib_dev.phys_port_cnt = num_ports; ibdev->port[vport_index].roce.netdev = mlx5_ib_get_rep_netdev(lag_master->priv.eswitch, rep->vport); ibdev->mdev = lag_master; diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index c75cc3d14e74..1046c92212c7 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -3932,7 +3932,6 @@ static int mlx5_ib_stage_init_init(struct mlx5_ib_dev *dev) dev->ib_dev.node_type = RDMA_NODE_IB_CA; dev->ib_dev.local_dma_lkey = 0 /* not supported for now */; - dev->ib_dev.phys_port_cnt = dev->num_ports; dev->ib_dev.dev.parent = mdev->device; dev->ib_dev.lag_flags = RDMA_LAG_FLAGS_HASH_ALL_SLAVES; @@ -4647,6 +4646,7 @@ static struct ib_device *mlx5_ib_add_sub_dev(struct ib_device *parent, mplane->mdev = mparent->mdev; mplane->num_ports = mparent->num_plane; mplane->sub_dev_name = name; + mplane->ib_dev.phys_port_cnt = mplane->num_ports; ret = __mlx5_ib_add(mplane, &plane_profile); if (ret) @@ -4763,6 +4763,7 @@ static int mlx5r_probe(struct auxiliary_device *adev, dev->mdev = mdev; dev->num_ports = num_ports; + dev->ib_dev.phys_port_cnt = num_ports; if (ll == IB_LINK_LAYER_ETHERNET && !mlx5_get_roce_state(mdev)) profile = &raw_eth_profile; From patchwork Mon Sep 9 17:30:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13797379 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2089.outbound.protection.outlook.com [40.107.93.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F7B417E010 for ; Mon, 9 Sep 2024 17:31:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.89 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903076; cv=fail; b=ZkUnXCHzuEg0y7PUzt5LaZRqBmdnpufuy0alQWNR5sphYYMENXmm2skIduiX4TfBj/LP0NwAh4wGdiAlXSTCgmYdWokuUkAHUe8KNQ6rA3gN56AKoNNj6DD8N11povH5il+dy0Tp5YA6/c9cYBzVAic3kSmwNxn+ij3vtPsxHrI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903076; c=relaxed/simple; bh=E/UZ+SjKKOxBRCS0EvOUdXE6BA0cZu3zPMkBLOagI9U=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GJYvOYtNPOqHhY16JTeCJqLIli/v7hM/vrTSklyrVrPWM9plod2kjqoXF6eeIV/VE/sJTD4sBlKWAlx/8nv5nEq61h7dgWiwmpqvk9/xbgK4Iqp64zl2y160XLTogBu6p34FNzEHPSJBtWYNziN5CLpaS11mlUXBbtdwBPEEywc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=dnxxaOhK; arc=fail smtp.client-ip=40.107.93.89 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="dnxxaOhK" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XnzwzskwUxSEjJTX5OQZ14OGLPLr3RWE0f1Ry9HCvR+jbKNumIpW+hNQc0bPDyuU61ZNOcJYkG/Sfd2yu9Lbvdrh2vOCCAUIbVu0TrNbuIyfX5CnP+cRcne4hZ+WP0G3bxrXuYZ2b5DDmtsQ87qhgLMNMNtItMAJfySMfpoWAMuQozZKZwzVhDIfhzpQNSWrNT+flp5mwJn2vnVI6jRehIU701BEMjTdXNE0jynUbMbInaOGvi9Uxcs77viN9SKyWiY9630pwT5psDD88PQqOLkntzeb5oYCB8qhdFbeY/VBukZGJ5gVvMr/cWmovRFVsrTG5d6TBJfdf0ty379JIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3L6o0Xb8vdDzQOVm+4hNgsraing69YLMZpPhm5ff2qg=; b=MJkMyp+9947LXMCkqTzS1xwe8Zyb/h8a3x5YaP/EZHD6dRrOtEWu2od0BR4v5kBJxhjDiZ/KZqZiMiv5GJHVXNIxtjuV+m9tQZX8jSVxwm4i0GMiUIJmys1TtAlJ+0/5iTyTO+DoGXEShIXlSdemMtFpNBzvRidKjXY0ITWa36+FDuIzImJ5ij3SO/iQn5ROhQxIp7doU5vwOJbKxxiJPBH33Nh2Cso6DENEXCjDhMVMLxUwXayaStjRaQWfEVvxdofk0ZtGmm6o4n3D1dHbxgU/iBeqb2vnkUdHjespXsBoc4bylP/X0LAWbqz0GNeOfJgqMrXgG99K+8TSEXrzRQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3L6o0Xb8vdDzQOVm+4hNgsraing69YLMZpPhm5ff2qg=; b=dnxxaOhKafB59vPNUuR6qvz43N8KjOWh+0aOhCW5CwDpyVTe/Q8n+01jYZ6q5b/WJkBdRMGHFeTL/1tKmKmavANhw743LmdAgEUl4hf41aftWUKjCxfp22TMwBshqPN/5rutKLwrRwDQnet6Ydv1+PFnMjXOshIp1r4hc6bPfxrrV3r232SkF93cWEisy1k9sDnmEjXkcQ9Piu0+vESriaxb+nwfMbOeKvhdGXtiK/Sl2ZowdmG9DBpaFErxMtsL6xNXvtgSw9/8aLKsfwoGe0iY6RyzjRrHVGMUF1u6V/rtfnAi0Ktsa7qqhQlykKB2uLC2s0SD1LHHEOBneQxTQg== Received: from BL1PR13CA0309.namprd13.prod.outlook.com (2603:10b6:208:2c1::14) by BY5PR12MB4051.namprd12.prod.outlook.com (2603:10b6:a03:20c::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.24; Mon, 9 Sep 2024 17:31:08 +0000 Received: from BL02EPF0001A0FE.namprd03.prod.outlook.com (2603:10b6:208:2c1:cafe::f0) by BL1PR13CA0309.outlook.office365.com (2603:10b6:208:2c1::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.23 via Frontend Transport; Mon, 9 Sep 2024 17:31:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BL02EPF0001A0FE.mail.protection.outlook.com (10.167.242.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.13 via Frontend Transport; Mon, 9 Sep 2024 17:31:07 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:44 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:43 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Mon, 9 Sep 2024 10:30:41 -0700 From: Michael Guralnik To: CC: , , , , , , "Michael Guralnik" Subject: [PATCH v3 rdma-next 4/7] RDMA/device: Remove optimization in ib_device_get_netdev() Date: Mon, 9 Sep 2024 20:30:22 +0300 Message-ID: <20240909173025.30422-5-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240909173025.30422-1-michaelgur@nvidia.com> References: <20240909173025.30422-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FE:EE_|BY5PR12MB4051:EE_ X-MS-Office365-Filtering-Correlation-Id: 0ae55b81-c7f6-4ecd-af17-08dcd0f53051 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: EhR1Q5AzaPLkRUMtEQ4lQlpSpN497KhOleHTV/1+5s06TBSSUQKkBeGqTuS01nm/PBTdhVCYK69AiFgFYTwjXKkQ1c7fEBoWJfSWH/BXesxw5COz0HjGaHDUZlUOt3gPrVx5dCy/BB3gOhYivQSp3uVo7vFzivUomPTAczVPKSkaANmyVYzFze4n73rb2NFlwxOdj0bB34XUVdgSZQPYXxkYSsodSD6UkWuC6G0q5bk7EDCaJwBZuxjWb8K5Emg6rulNp1IgAwe4lA7VpMBC1o7I2IzZF4j8CvhQMk4rpD5Edc2kuKPUN29X4663URHjJDUFI8BwFdDq+HEhRaOviBCABxu2dbj6U1WQZBfi6nRb7DTPXfMLvNPmWQH/9iLWrEaEshB/jWmf6DivfBzCZKXwF79arfaJzFbxOxvIOgvW6TeE2zzX6ABrimmbYhxIRG1NkGcMwVKZhzUb08dq+8cf9/qfi05n5rfUujgOFXeSdS9MHjIPZvAOly6LiySiBi1exXkt1aP6XgbXHwz7rkvSegNqk+ZEU5/XgEK28aMTlUUsx3vk7LbP2hPpXKTSlJeKOhI95yElJCxc1s2ZokzLLp3k9ibnIB8RdSIK3BMtXImiMZMorRI/9QBjHnakQuCzYiQh2SMu1n0gV88qTWK0y5BPNzrDdo8VorYSzQBt2cNq0/Z27MUHXutvQCsZa1LSwBAmNa+QYdqGljYfZz072cZutu7PjqUEf6sj1o2SvpI5n02ba6o+TSuMa5JPjtm+FU8kD0s5pHGC7kPX+rGnQocDRNbrnSp7egTampYeYOx83PGO96S4IgykVAU4BFUVBgwgGF5qpgQt1QlDuqqtnSyv8rUrGnliF8WIRdW0nbHYyoPDS8a1EIMP1ZyOpmrSfG9aI3f6GacZQdj/F0ptgv30iIt7KU04eF95klvNEYrhv8RvXBKNgwcMs+Liuhs2iTkg2xVjjz+56pZveb0nykSCYAVkg9Bt+aYSy1499+M6ZKoLpKio4VTdooVZzNrcsXQzrkwb1dltIHmktOGzCPLubA+ZAN8n75wXGuUo3eOVBFIMWe/widcebugqCIj/5krcAGqbPcxGUomBIuA6FQLfGqWIbp6qrq+r6n9Hra7vYtREysEFjO7AIFyUSyEhli/Aw9ZZa4px0Bs8+TrSNaiyJew07XD3LKXG7gHl6yqaGgzs1k4xBiqFVOW30iNSfoBClCNxQzqJRo9O8eiKXUi1Qky3RtutnDg9Fr6Fr36ZPQlN0EetZHZdNCi6kZHo52zWGKkz2BS2repAoqZlguxZChgCB7Hk4IrDjfrcyxf0dCOMSZCSCtH5JRoiZuS5Yr2YD4wbEioah74z6U25YLJO6sQf5FLj89ZYODApH5LZu4k6CAhLJxNILtK158Ltr+sVYpXGQIQ76mt45sfbJ6c7M30Z9PsW6BFTNUOXpoHr9covIQ0gsDZib8lJ X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(1800799024)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2024 17:31:07.3804 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0ae55b81-c7f6-4ecd-af17-08dcd0f53051 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A0FE.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4051 From: Chiara Meiohas The caller of ib_device_get_netdev() relies on its result to accurately match a given netdev with the ib device associated netdev. ib_device_get_netdev returns NULL when the IB device associated netdev is unregistering, preventing the caller of matching netdevs properly. Thus, remove this optimization and return the netdev even if it is undergoing unregistration, allowing matching by the caller. This change ensures proper netdev matching and reference count handling by the caller of ib_device_get_netdev/ib_device_set_netdev API. Signed-off-by: Maher Sanalla Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky Reviewed-by: Kalesh AP --- drivers/infiniband/core/device.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 0290aca18d26..b1377503cb9d 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -2252,15 +2252,6 @@ struct net_device *ib_device_get_netdev(struct ib_device *ib_dev, spin_unlock(&pdata->netdev_lock); } - /* - * If we are starting to unregister expedite things by preventing - * propagation of an unregistering netdev. - */ - if (res && res->reg_state != NETREG_REGISTERED) { - dev_put(res); - return NULL; - } - return res; } From patchwork Mon Sep 9 17:30:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13797382 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2052.outbound.protection.outlook.com [40.107.94.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE0C2189511 for ; Mon, 9 Sep 2024 17:31:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.52 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903081; cv=fail; b=mxC5tEAAfJUIIXow9Tvp0rDnxrWkYW0w1ZASNuZwcYYHZGNjEs363HXgBQHShtsSOV6SpjsfsM8snqKOLDErPDWMWLa7yjmHLOyoI1ulZYMxppB+SRXu5+TZ2F1dl6NKQHRk6xOQRNvC+BClrTmVDa6/bzN0FBPqbegqbIpxdXY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903081; c=relaxed/simple; bh=VZiaL4Y3d+zpT/ihBYP1y8Ayp1drZCVgLlz99LGH0Gw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=bmKlpk1ku+s6D33mn2t+QeH8ASfi71WKXWDpgZ5mVOXzmOiUR5us4icAFmYZhzNbK46TgrDWVWyBvdPpWJLi6/jmG9uZ3tpG0yBdpjsa80uZeYCQQNxiZeUlychfvp5+NJRtQQMFk8leqrGuCK74IIorPuaW8IAtnpnrqNJqlvk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=gEmLamTf; arc=fail smtp.client-ip=40.107.94.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="gEmLamTf" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=DcXP7a+V3ishk+2Fznff54VMyfyjxRInu8MFHjLSSelJYZ9AWwShe2LQyzbhLal+fdLRyI0hXdhAibkX0T++7xvAWk45lqsaY0A6QHxnNDEiJEwcIaN5ncAQu2pl7iBFxwDBDbihgW8+8B0KMQup8lXJXQP/bFM9tre+XxtTyVoxJPbFLE9kvVkwYdkoltEoCQILvIkidi9JN6D0IsEjmu5A2dKsNV/poRnsJL+B14OfDHrcVTf5f3dO9W4FBLcU2LtbK4YGUy78KnsMb4cHMEeJsQBtty4fLiV1nceQcBU6OF6J3bHzrVUpL6va4gIWzLU2kNSH0vUIUUEoFamO4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=djegOwDop8e4Hm22kUFVHPVy+CRz2dJPTXW11WNhrqg=; b=Aa9lziG2JRFvP6gdWTpkv2wl1EINghMiJoKrtejN+jc59bEInupqwIE3cOytQnYGYa7jBbHBBIFth1XBIXu8hH80H894gwBZl2sWUc8AAhzQAGGbNYeQL/+iVzss4R5HCl1KDBzFIVsUV+LkjYxCOSKElWJblHhkyHoY2IbQPFW4SKtgmDmKeZ1SEscqJsI+gqourEySjROK5iuHXW8yA4ENG4AavBnfgsSkwNYunvMds6rOvPykT/VIMs5XlOtX0TojDUA9u4EqJ3qB+EXepq7IiDMPk5RFTEHqYDrt8rB9S758O/yaRz2WBwIso5zGXqAprtKDtdetakyXTueRFw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=djegOwDop8e4Hm22kUFVHPVy+CRz2dJPTXW11WNhrqg=; b=gEmLamTfSlQ7i9xNfdDFhZ1nqX/WIx7JNFFsA1cwJ/ZxyTzMRxyvwFyCRwmm2PcFTnsnc6Yaq2yjjm8XyWzDuEBN9hUXs8AN8sTqYWlpcIF5//WEb8RI4WDSNg/7/Nckfau7Fo9hW3t0ZJL6a9mpwsa1P11ki9ZYaimYvLZaCOHBm9pbCQGrN2xBSS4nK6I6T0NXQuuuf/tZMmSM07UM6bBrHQF/pn/EXN6CEE+n6a4eo7sQ/2S5+9Pj/+Ko3USdYSDSrQsVGn63pqRJhME88oWYSD/01nfHVs7QAZo0+eCeY0P5bZNanhc1ndXn2l2Uz8mCGyLvVkwVgZbvoNYaFQ== Received: from MN2PR17CA0036.namprd17.prod.outlook.com (2603:10b6:208:15e::49) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.24; Mon, 9 Sep 2024 17:31:10 +0000 Received: from BL02EPF0001A0FD.namprd03.prod.outlook.com (2603:10b6:208:15e:cafe::1a) by MN2PR17CA0036.outlook.office365.com (2603:10b6:208:15e::49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.27 via Frontend Transport; Mon, 9 Sep 2024 17:31:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BL02EPF0001A0FD.mail.protection.outlook.com (10.167.242.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.13 via Frontend Transport; Mon, 9 Sep 2024 17:31:09 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:47 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:46 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Mon, 9 Sep 2024 10:30:44 -0700 From: Michael Guralnik To: CC: , , , , , , "Michael Guralnik" Subject: [PATCH v3 rdma-next 5/7] RDMA/mlx5: Use IB set_netdev and get_netdev functions Date: Mon, 9 Sep 2024 20:30:23 +0300 Message-ID: <20240909173025.30422-6-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240909173025.30422-1-michaelgur@nvidia.com> References: <20240909173025.30422-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FD:EE_|MN2PR12MB4125:EE_ X-MS-Office365-Filtering-Correlation-Id: 69c6af07-10aa-46fb-7abc-08dcd0f531c7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: lUcd5isnnEH3jny8DIpFfENPNUlT/hfy71+73z5OWLioIavNXvk+yxvZvyOTGj0sb4XyWgyliJxb9nL1RByOxM1XdljA6kQaF/B1E4wRfFlQ1AL2N1Ww46pQ9maUcOgjJURH2H3kSWb8fLKo650YUaJh5+/+i750U9Ms7ZMkeXVSgzWdFNCdKkYiSn349xtbV/Msg11Q8nwsMCgHM3Sj0y3JnUETjkZHx0Jb6A8jOAjExvfWeiGow3RRWsJFPw5OBNK77oHFSPRLLzQ6NMLDXqeKR+RnhY8BuH90unA/2+w5omkzAQA9Hx+jczkpVQFiX+CYekszJgvV0frRKDZ4nDayosDWF4lbDo3ni352z8WGUwT6zqh7BVYKTBstEPMzDk+PSS7x/rLIzbVEQL5Rj7tlTO17SQigoc3hSqFQEfqlxGUmn/2RIHudei8zoMPg257LH40P2YuOD5sBLLtZrJvxEcLGu4h1SMD0bq1eQxFSKYmZdq9ZhCxAgWYAGJXAYKltADiXBirwj6dAxD6I/BOkQ1W8qKo9sqzvpN9uo7RUdv4su5Fy0mHgZI7UeI35eOSKWgpW5IKo8oR9jQR+u/6FySBWsWS6ZI49KVcaBOU6eXEdZk5UvX2TgFCJkktJDVCdxB9L7dAU+yzg0d0IfkEqviu57ppUxhMilx9gESnFNJTD1fq21VZSFjHcA94VCFLzufWrxKkyV0dSZUYZlSzmj2dcVee2smxGPLvoj+gAXgfoSfbarlw0bikvEVDL6M5euStYJ8lJ9KlGHttn7K8I0DV/m+W+uAWtTJgM3b9duBR0Hewqm6S59yBCVhaou5q18K5pOGDLESsdUGCDlssKnTBOKNSgyPzckm6VGzPt7n6bfvUCLW3VWGiS1gHLpH4A7KdeaFg6KXip5lfzpsp55c4e06+XUbKodt/XzWtMAmthvCDNjo66g9GmB6dviat5DcaC/T43ZJvmC31jJVHeia0ciTm5Bt0yhAFYITKDn+21tCQAg5RGqKhXSmJasvmzMQpMfYFpZErSZlEPXhtQBc/y3daO6ZovAZrMd6mv1qWJK6elXfNaeZdssghX59N5EmaiRXmoSnRT5zrUyE+QewJ3Cl4CZIKCY9ZOGzAYQ3ItuthY3iZEa7DYAQFy3+0C+HSY8wZUQzF7fW5iJPqsgWnbcOJGa2s7/XUtmKVnMj/IYdE7cAPrAW6RFpZrmFLVYS8w3L3RjPI2IXYw6FaKmOBuC8LclfRsNgC6WMJhzGcluSzBOOFdF9qsIDrIRqI+LMQEW1ORsxU7vm4ryQ7GFuLkmokianvdpq1EuBeR10r2kPz+2SFc98ZRD6S6FWbhzrenyB+nbMcoynpT4oENO9IQ2mu2XqfBqp5aGEGGGX+qOcXKOY0sEW1mKmGYoWTLdN86VrcLO/sDPuUr/NVsO2HrpPChahNjzWjBc+TyCdh4o9xPMxXSr2awqn1e X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(1800799024)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2024 17:31:09.8458 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 69c6af07-10aa-46fb-7abc-08dcd0f531c7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A0FD.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 From: Chiara Meiohas The IB layer provides a common interface to store and get net devices associated to an IB device port (ib_device_set_netdev() and ib_device_get_netdev()). Previously, mlx5_ib stored and managed the associated net devices internally. Replace internal net device management in mlx5_ib with ib_device_set_netdev() when attaching/detaching a net device and ib_device_get_netdev() when retrieving the net device. Export ib_device_get_netdev(). For mlx5 representors/PFs/VFs and lag creation we replace the netdev assignments with the IB set/get netdev functions. In active-backup mode lag the active slave net device is stored in the lag itself. To assure the net device stored in a lag bond IB device is the active slave we implement the following: - mlx5_core: when modifying the slave of a bond we send the internal driver event MLX5_DRIVER_EVENT_ACTIVE_BACKUP_LAG_CHANGE_LOWERSTATE. - mlx5_ib: when catching the event call ib_device_set_netdev() This patch also ensures the correct IB events are sent in switchdev lag. While at it, when in multiport eswitch mode, only a single IB device is created for all ports. The said IB device will receive all netdev events of its VFs once loaded, thus to avoid overwriting the mapping of PF IB device to PF netdev, ignore NETDEV_REGISTER events if the ib device has already been mapped to a netdev. Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/core/device.c | 4 + drivers/infiniband/hw/mlx5/ib_rep.c | 23 +-- drivers/infiniband/hw/mlx5/main.c | 183 ++++++++++++------ drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 +- .../net/ethernet/mellanox/mlx5/core/lag/lag.c | 76 ++++---- include/linux/mlx5/device.h | 1 + include/linux/mlx5/driver.h | 2 +- include/rdma/ib_verbs.h | 2 + 8 files changed, 191 insertions(+), 103 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index b1377503cb9d..9e765c79a892 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -2236,6 +2236,9 @@ struct net_device *ib_device_get_netdev(struct ib_device *ib_dev, if (!rdma_is_port_valid(ib_dev, port)) return NULL; + if (!ib_dev->port_data) + return NULL; + pdata = &ib_dev->port_data[port]; /* @@ -2254,6 +2257,7 @@ struct net_device *ib_device_get_netdev(struct ib_device *ib_dev, return res; } +EXPORT_SYMBOL(ib_device_get_netdev); /** * ib_device_get_by_netdev - Find an IB device associated with a netdev diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c index 1ad934685d80..49af1cfbe6d1 100644 --- a/drivers/infiniband/hw/mlx5/ib_rep.c +++ b/drivers/infiniband/hw/mlx5/ib_rep.c @@ -13,6 +13,7 @@ mlx5_ib_set_vport_rep(struct mlx5_core_dev *dev, int vport_index) { struct mlx5_ib_dev *ibdev; + struct net_device *ndev; ibdev = mlx5_eswitch_uplink_get_proto_dev(dev->priv.eswitch, REP_IB); if (!ibdev) @@ -20,12 +21,9 @@ mlx5_ib_set_vport_rep(struct mlx5_core_dev *dev, ibdev->port[vport_index].rep = rep; rep->rep_data[REP_IB].priv = ibdev; - write_lock(&ibdev->port[vport_index].roce.netdev_lock); - ibdev->port[vport_index].roce.netdev = - mlx5_ib_get_rep_netdev(rep->esw, rep->vport); - write_unlock(&ibdev->port[vport_index].roce.netdev_lock); + ndev = mlx5_ib_get_rep_netdev(rep->esw, rep->vport); - return 0; + return ib_device_set_netdev(&ibdev->ib_dev, ndev, vport_index + 1); } static void mlx5_ib_register_peer_vport_reps(struct mlx5_core_dev *mdev); @@ -104,11 +102,15 @@ mlx5_ib_vport_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep) ibdev->is_rep = true; vport_index = rep->vport_index; ibdev->port[vport_index].rep = rep; - ibdev->ib_dev.phys_port_cnt = num_ports; - ibdev->port[vport_index].roce.netdev = - mlx5_ib_get_rep_netdev(lag_master->priv.eswitch, rep->vport); ibdev->mdev = lag_master; ibdev->num_ports = num_ports; + ibdev->ib_dev.phys_port_cnt = num_ports; + ret = ib_device_set_netdev(&ibdev->ib_dev, + mlx5_ib_get_rep_netdev(lag_master->priv.eswitch, + rep->vport), + vport_index + 1); + if (ret) + goto fail_add; ret = __mlx5_ib_add(ibdev, profile); if (ret) @@ -161,9 +163,8 @@ mlx5_ib_vport_rep_unload(struct mlx5_eswitch_rep *rep) } port = &dev->port[vport_index]; - write_lock(&port->roce.netdev_lock); - port->roce.netdev = NULL; - write_unlock(&port->roce.netdev_lock); + + ib_device_set_netdev(&dev->ib_dev, NULL, vport_index + 1); rep->rep_data[REP_IB].priv = NULL; port->rep = NULL; diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index 1046c92212c7..a750f61513d4 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -147,16 +147,52 @@ static struct mlx5_roce *mlx5_get_rep_roce(struct mlx5_ib_dev *dev, if (upper && port->rep->vport == MLX5_VPORT_UPLINK) continue; - - read_lock(&port->roce.netdev_lock); - rep_ndev = mlx5_ib_get_rep_netdev(port->rep->esw, - port->rep->vport); - if (rep_ndev == ndev) { - read_unlock(&port->roce.netdev_lock); + rep_ndev = ib_device_get_netdev(&dev->ib_dev, i + 1); + if (rep_ndev && rep_ndev == ndev) { + dev_put(rep_ndev); *port_num = i + 1; return &port->roce; } - read_unlock(&port->roce.netdev_lock); + + dev_put(rep_ndev); + } + + return NULL; +} + +static bool mlx5_netdev_send_event(struct mlx5_ib_dev *dev, + struct net_device *ndev, + struct net_device *upper, + struct net_device *ib_ndev) +{ + if (!dev->ib_active) + return false; + + /* Event is about our upper device */ + if (upper == ndev) + return true; + + /* RDMA device is not in lag and not in switchdev */ + if (!dev->is_rep && !upper && ndev == ib_ndev) + return true; + + /* RDMA devie is in switchdev */ + if (dev->is_rep && ndev == ib_ndev) + return true; + + return false; +} + +static struct net_device *mlx5_ib_get_rep_uplink_netdev(struct mlx5_ib_dev *ibdev) +{ + struct mlx5_ib_port *port; + int i; + + for (i = 0; i < ibdev->num_ports; i++) { + port = &ibdev->port[i]; + if (port->rep && port->rep->vport == MLX5_VPORT_UPLINK) { + return ib_device_get_netdev(&ibdev->ib_dev, i + 1); + } } return NULL; @@ -168,6 +204,7 @@ static int mlx5_netdev_event(struct notifier_block *this, struct mlx5_roce *roce = container_of(this, struct mlx5_roce, nb); struct net_device *ndev = netdev_notifier_info_to_dev(ptr); u32 port_num = roce->native_port_num; + struct net_device *ib_ndev = NULL; struct mlx5_core_dev *mdev; struct mlx5_ib_dev *ibdev; @@ -181,29 +218,38 @@ static int mlx5_netdev_event(struct notifier_block *this, /* Should already be registered during the load */ if (ibdev->is_rep) break; - write_lock(&roce->netdev_lock); + + ib_ndev = ib_device_get_netdev(&ibdev->ib_dev, port_num); + /* Exit if already registered */ + if (ib_ndev) + goto put_ndev; + if (ndev->dev.parent == mdev->device) - roce->netdev = ndev; - write_unlock(&roce->netdev_lock); + ib_device_set_netdev(&ibdev->ib_dev, ndev, port_num); break; case NETDEV_UNREGISTER: /* In case of reps, ib device goes away before the netdevs */ - write_lock(&roce->netdev_lock); - if (roce->netdev == ndev) - roce->netdev = NULL; - write_unlock(&roce->netdev_lock); - break; + if (ibdev->is_rep) + break; + ib_ndev = ib_device_get_netdev(&ibdev->ib_dev, port_num); + if (ib_ndev == ndev) + ib_device_set_netdev(&ibdev->ib_dev, NULL, port_num); + goto put_ndev; case NETDEV_CHANGE: case NETDEV_UP: case NETDEV_DOWN: { struct net_device *upper = NULL; - if (mlx5_lag_is_roce(mdev)) { + if (mlx5_lag_is_roce(mdev) || mlx5_lag_is_sriov(mdev)) { struct net_device *lag_ndev; - lag_ndev = mlx5_lag_get_roce_netdev(mdev); + if(mlx5_lag_is_roce(mdev)) + lag_ndev = ib_device_get_netdev(&ibdev->ib_dev, 1); + else /* sriov lag */ + lag_ndev = mlx5_ib_get_rep_uplink_netdev(ibdev); + if (lag_ndev) { upper = netdev_master_upper_dev_get(lag_ndev); dev_put(lag_ndev); @@ -216,18 +262,19 @@ static int mlx5_netdev_event(struct notifier_block *this, roce = mlx5_get_rep_roce(ibdev, ndev, upper, &port_num); if (!roce) return NOTIFY_DONE; - if ((upper == ndev || - ((!upper || ibdev->is_rep) && ndev == roce->netdev)) && - ibdev->ib_active) { + + ib_ndev = ib_device_get_netdev(&ibdev->ib_dev, port_num); + + if (mlx5_netdev_send_event(ibdev, ndev, upper, ib_ndev)) { struct ib_event ibev = { }; enum ib_port_state port_state; if (get_port_state(&ibdev->ib_dev, port_num, &port_state)) - goto done; + goto put_ndev; if (roce->last_port_state == port_state) - goto done; + goto put_ndev; roce->last_port_state = port_state; ibev.device = &ibdev->ib_dev; @@ -236,7 +283,7 @@ static int mlx5_netdev_event(struct notifier_block *this, else if (port_state == IB_PORT_ACTIVE) ibev.event = IB_EVENT_PORT_ACTIVE; else - goto done; + goto put_ndev; ibev.element.port_num = port_num; ib_dispatch_event(&ibev); @@ -247,39 +294,13 @@ static int mlx5_netdev_event(struct notifier_block *this, default: break; } +put_ndev: + dev_put(ib_ndev); done: mlx5_ib_put_native_port_mdev(ibdev, port_num); return NOTIFY_DONE; } -static struct net_device *mlx5_ib_get_netdev(struct ib_device *device, - u32 port_num) -{ - struct mlx5_ib_dev *ibdev = to_mdev(device); - struct net_device *ndev; - struct mlx5_core_dev *mdev; - - mdev = mlx5_ib_get_native_port_mdev(ibdev, port_num, NULL); - if (!mdev) - return NULL; - - if (mlx5_lag_is_roce(mdev)) { - ndev = mlx5_lag_get_roce_netdev(mdev); - goto out; - } - - /* Ensure ndev does not disappear before we invoke dev_hold() - */ - read_lock(&ibdev->port[port_num - 1].roce.netdev_lock); - ndev = ibdev->port[port_num - 1].roce.netdev; - dev_hold(ndev); - read_unlock(&ibdev->port[port_num - 1].roce.netdev_lock); - -out: - mlx5_ib_put_native_port_mdev(ibdev, port_num); - return ndev; -} - struct mlx5_core_dev *mlx5_ib_get_native_port_mdev(struct mlx5_ib_dev *ibdev, u32 ib_port_num, u32 *native_port_num) @@ -554,7 +575,7 @@ static int mlx5_query_port_roce(struct ib_device *device, u32 port_num, if (!put_mdev) goto out; - ndev = mlx5_ib_get_netdev(device, port_num); + ndev = ib_device_get_netdev(device, port_num); if (!ndev) goto out; @@ -3185,6 +3206,60 @@ static void get_dev_fw_str(struct ib_device *ibdev, char *str) fw_rev_sub(dev->mdev)); } +static int lag_event(struct notifier_block *nb, unsigned long event, void *data) +{ + struct mlx5_ib_dev *dev = container_of(nb, struct mlx5_ib_dev, + lag_events); + struct mlx5_core_dev *mdev = dev->mdev; + struct mlx5_ib_port *port; + struct net_device *ndev; + int i, err; + int portnum; + + portnum = 0; + switch (event) { + case MLX5_DRIVER_EVENT_ACTIVE_BACKUP_LAG_CHANGE_LOWERSTATE: + ndev = data; + if (ndev) { + if (!mlx5_lag_is_roce(mdev)) { + // sriov lag + for (i = 0; i < dev->num_ports; i++) { + port = &dev->port[i]; + if (port->rep && port->rep->vport == + MLX5_VPORT_UPLINK) { + portnum = i; + break; + } + } + } + err = ib_device_set_netdev(&dev->ib_dev, ndev, + portnum + 1); + dev_put(ndev); + if (err) + return err; + /* Rescan gids after new netdev assignment */ + rdma_roce_rescan_device(&dev->ib_dev); + } + break; + default: + return NOTIFY_DONE; + } + return NOTIFY_OK; +} + +static void mlx5e_lag_event_register(struct mlx5_ib_dev *dev) +{ + dev->lag_events.notifier_call = lag_event; + blocking_notifier_chain_register(&dev->mdev->priv.lag_nh, + &dev->lag_events); +} + +static void mlx5e_lag_event_unregister(struct mlx5_ib_dev *dev) +{ + blocking_notifier_chain_unregister(&dev->mdev->priv.lag_nh, + &dev->lag_events); +} + static int mlx5_eth_lag_init(struct mlx5_ib_dev *dev) { struct mlx5_core_dev *mdev = dev->mdev; @@ -3206,6 +3281,7 @@ static int mlx5_eth_lag_init(struct mlx5_ib_dev *dev) goto err_destroy_vport_lag; } + mlx5e_lag_event_register(dev); dev->flow_db->lag_demux_ft = ft; dev->lag_ports = mlx5_lag_get_num_ports(mdev); dev->lag_active = true; @@ -3223,6 +3299,7 @@ static void mlx5_eth_lag_cleanup(struct mlx5_ib_dev *dev) if (dev->lag_active) { dev->lag_active = false; + mlx5e_lag_event_unregister(dev); mlx5_destroy_flow_table(dev->flow_db->lag_demux_ft); dev->flow_db->lag_demux_ft = NULL; @@ -3937,7 +4014,6 @@ static int mlx5_ib_stage_init_init(struct mlx5_ib_dev *dev) for (i = 0; i < dev->num_ports; i++) { spin_lock_init(&dev->port[i].mp.mpi_lock); - rwlock_init(&dev->port[i].roce.netdev_lock); dev->port[i].roce.dev = dev; dev->port[i].roce.native_port_num = i + 1; dev->port[i].roce.last_port_state = IB_PORT_DOWN; @@ -4202,7 +4278,6 @@ static const struct ib_device_ops mlx5_ib_dev_common_roce_ops = { .create_wq = mlx5_ib_create_wq, .destroy_rwq_ind_table = mlx5_ib_destroy_rwq_ind_table, .destroy_wq = mlx5_ib_destroy_wq, - .get_netdev = mlx5_ib_get_netdev, .modify_wq = mlx5_ib_modify_wq, INIT_RDMA_OBJ_SIZE(ib_rwq_ind_table, mlx5_ib_rwq_ind_table, diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index c0b1a9cd752b..6af8b4df9a5c 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -896,8 +896,6 @@ struct mlx5_roce { /* Protect mlx5_ib_get_netdev from invoking dev_hold() with a NULL * netdev pointer */ - rwlock_t netdev_lock; - struct net_device *netdev; struct notifier_block nb; struct netdev_net_notifier nn; struct notifier_block mdev_nb; @@ -1146,6 +1144,7 @@ struct mlx5_ib_dev { /* protect accessing data_direct_dev */ struct mutex data_direct_lock; struct notifier_block mdev_events; + struct notifier_block lag_events; int num_ports; /* serialize update of capability mask */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c index cf8045b92689..8577db3308cc 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c @@ -445,6 +445,34 @@ static int _mlx5_modify_lag(struct mlx5_lag *ldev, u8 *ports) return mlx5_cmd_modify_lag(dev0, ldev->ports, ports); } +static struct net_device *mlx5_lag_active_backup_get_netdev(struct mlx5_core_dev *dev) +{ + struct net_device *ndev = NULL; + struct mlx5_lag *ldev; + unsigned long flags; + int i; + + spin_lock_irqsave(&lag_lock, flags); + ldev = mlx5_lag_dev(dev); + + if (!ldev) + goto unlock; + + for (i = 0; i < ldev->ports; i++) + if (ldev->tracker.netdev_state[i].tx_enabled) + ndev = ldev->pf[i].netdev; + if (!ndev) + ndev = ldev->pf[ldev->ports - 1].netdev; + + if (ndev) + dev_hold(ndev); + +unlock: + spin_unlock_irqrestore(&lag_lock, flags); + + return ndev; +} + void mlx5_modify_lag(struct mlx5_lag *ldev, struct lag_tracker *tracker) { @@ -477,9 +505,18 @@ void mlx5_modify_lag(struct mlx5_lag *ldev, } } - if (tracker->tx_type == NETDEV_LAG_TX_TYPE_ACTIVEBACKUP && - !(ldev->mode == MLX5_LAG_MODE_ROCE)) - mlx5_lag_drop_rule_setup(ldev, tracker); + if (tracker->tx_type == NETDEV_LAG_TX_TYPE_ACTIVEBACKUP) { + struct net_device *ndev = mlx5_lag_active_backup_get_netdev(dev0); + + if(!(ldev->mode == MLX5_LAG_MODE_ROCE)) + mlx5_lag_drop_rule_setup(ldev, tracker); + /** Only sriov and roce lag should have tracker->tx_type set so + * no need to check the mode + */ + blocking_notifier_call_chain(&dev0->priv.lag_nh, + MLX5_DRIVER_EVENT_ACTIVE_BACKUP_LAG_CHANGE_LOWERSTATE, + ndev); + } } static int mlx5_lag_set_port_sel_mode_roce(struct mlx5_lag *ldev, @@ -613,6 +650,7 @@ static int mlx5_create_lag(struct mlx5_lag *ldev, mlx5_core_err(dev0, "Failed to deactivate RoCE LAG; driver restart required\n"); } + BLOCKING_INIT_NOTIFIER_HEAD(&dev0->priv.lag_nh); return err; } @@ -1492,38 +1530,6 @@ void mlx5_lag_enable_change(struct mlx5_core_dev *dev) mlx5_queue_bond_work(ldev, 0); } -struct net_device *mlx5_lag_get_roce_netdev(struct mlx5_core_dev *dev) -{ - struct net_device *ndev = NULL; - struct mlx5_lag *ldev; - unsigned long flags; - int i; - - spin_lock_irqsave(&lag_lock, flags); - ldev = mlx5_lag_dev(dev); - - if (!(ldev && __mlx5_lag_is_roce(ldev))) - goto unlock; - - if (ldev->tracker.tx_type == NETDEV_LAG_TX_TYPE_ACTIVEBACKUP) { - for (i = 0; i < ldev->ports; i++) - if (ldev->tracker.netdev_state[i].tx_enabled) - ndev = ldev->pf[i].netdev; - if (!ndev) - ndev = ldev->pf[ldev->ports - 1].netdev; - } else { - ndev = ldev->pf[MLX5_LAG_P1].netdev; - } - if (ndev) - dev_hold(ndev); - -unlock: - spin_unlock_irqrestore(&lag_lock, flags); - - return ndev; -} -EXPORT_SYMBOL(mlx5_lag_get_roce_netdev); - u8 mlx5_lag_get_slave_port(struct mlx5_core_dev *dev, struct net_device *slave) { diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h index ba875a619b97..a38a17f8c12c 100644 --- a/include/linux/mlx5/device.h +++ b/include/linux/mlx5/device.h @@ -370,6 +370,7 @@ enum mlx5_driver_event { MLX5_DRIVER_EVENT_SF_PEER_DEVLINK, MLX5_DRIVER_EVENT_AFFILIATION_DONE, MLX5_DRIVER_EVENT_AFFILIATION_REMOVED, + MLX5_DRIVER_EVENT_ACTIVE_BACKUP_LAG_CHANGE_LOWERSTATE, }; enum { diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index a96438ded15f..46a7a3d11048 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -643,6 +643,7 @@ struct mlx5_priv { struct mlx5_sf_hw_table *sf_hw_table; struct mlx5_sf_table *sf_table; #endif + struct blocking_notifier_head lag_nh; }; enum mlx5_device_state { @@ -1181,7 +1182,6 @@ bool mlx5_lag_mode_is_hash(struct mlx5_core_dev *dev); bool mlx5_lag_is_master(struct mlx5_core_dev *dev); bool mlx5_lag_is_shared_fdb(struct mlx5_core_dev *dev); bool mlx5_lag_is_mpesw(struct mlx5_core_dev *dev); -struct net_device *mlx5_lag_get_roce_netdev(struct mlx5_core_dev *dev); u8 mlx5_lag_get_slave_port(struct mlx5_core_dev *dev, struct net_device *slave); int mlx5_lag_query_cong_counters(struct mlx5_core_dev *dev, diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index a1dcf812d787..aa8ede439905 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -4453,6 +4453,8 @@ struct net_device *ib_get_net_dev_by_params(struct ib_device *dev, u32 port, const struct sockaddr *addr); int ib_device_set_netdev(struct ib_device *ib_dev, struct net_device *ndev, unsigned int port); +struct net_device *ib_device_get_netdev(struct ib_device *ib_dev, + u32 port); struct ib_wq *ib_create_wq(struct ib_pd *pd, struct ib_wq_init_attr *init_attr); int ib_destroy_wq_user(struct ib_wq *wq, struct ib_udata *udata); From patchwork Mon Sep 9 17:30:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13797381 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2048.outbound.protection.outlook.com [40.107.237.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2143717E010 for ; Mon, 9 Sep 2024 17:31:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.48 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903080; cv=fail; b=uFrRmsSt+/4PT22H92+c148et2LbEMEanW/1JoPVQm7aR4NZPYjSf0/QDl5Vb9xiJQ3IN1UCcnezkMsRReaHOGdap4AKjQoGBFYdqnyOyFwdsQhXf8Apgg6C6pN3VUp2n4Zf5S0PhQh3svzYcWknSK9Q8N0VsODyFKP1KkNGW6g= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903080; c=relaxed/simple; bh=HASvvHgjHHyklVdaWbDBea5XwO1dQEe5vXS++kQZABM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ORueApJoY3Gd8zGEqk5/Etmi3pD6EtILphnnlIA21oYtMnSaAsHO4a76ssHtci1MUkE6Lu2V6BibEDGVt6jm1eXdjFQE/7p63hD/8eenWHrvVF/144wUsPTqYb2CxOMDgsEz0gts4cQrA2o6ji/y4BTUtOOGFlWjWgUR5qC+Q1c= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=U85ybWto; arc=fail smtp.client-ip=40.107.237.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="U85ybWto" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=yBFhaWU77vrsJ585ZX2ipF9CLjtW1TW7s6nJgVxyuToM/wvI+EE3iyqcv13rgItBudXGIzwrI3TratjywhVNNzQURzSVB8XcEEI1XNFVzOvbWMHylE1kLGztSH3HLmG7CW1LEitkC9omXxvAG+KgazOTgBsPAOuiIkrK7AhBS3+DjRFQdhOvE10YSO3Pv6DXKc7Q/JDjNyM8Fhp+UhF7pXrGHy5wdlmjztdnT24psChFv5Vdg8GYeT7KIQIhwxTwP4SHdrRnx7GiETSXMsnhDpYA+znAw3Zqo1lNdd4rxQYNyI4oZY0euvZVVVvYq5mjQ4wez2vLTvYV6lPiIoFkkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=K475JZZ4hVhTQpymAscLug6pbEw0e9+LmpBWRjbRQnw=; b=B98VJRugTXdmmsfvVHBZ2/dvq/QtXLLujKYf+hbvqUgPfp3D2mGMog8PE4BKYiW/eSgYIdntdJZi11g9qYOTgflgqNQghPD9mO72Sx7JdDLJCqXIb+WHJlt4X9qCGj7l47N9OBVuOaF6FHPZNlCGVWz3+RNvHpggv9CXL45xRhIYBGrVC4ZOM+CUkGyhRYBkhv7U0zjt4tsPUnXlPl89l81t1ZRO78HJybhCuwJnH8XLV4pEUC05wkp2pcZ5C6Nra0kQ1CWLFUCIrNPNLQEbv1sNh6cbD0QtRDwt4XuK/ENsA1n+G1fO/sEStCeqL1x5FIdYdiDZDHNJ2LSplWNySw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K475JZZ4hVhTQpymAscLug6pbEw0e9+LmpBWRjbRQnw=; b=U85ybWto858Iam5QtbC2C68DlYIMHFNUl/CHu/7SnMq1qdnmzO2FmbIyHw+Odm3rbH/XmKxMop10qI40gotFsxbleWBumW9ws1a4h08opPZiPqK1r2PkSYGzJBtvSEuQXEUKl6pGV+Rb1HlKIphvKcl5qgqa6uQM8z8sHIGoPPw8NW7fJnc2/ljOtrW2OV1wGwID0PIYnGJQTgv0EkQBa9db02gH1PjDt1VMxzICw4V7VL0m8K4kUHsS5B90aII73bcYudUPOB/9RoIRKKhmdmfmQmO1vESINdWFxL3DnpVhAcaYr0E52kSZqVeEDkohUXvwY9WB72OFT+dHlrP2KA== Received: from MN2PR17CA0019.namprd17.prod.outlook.com (2603:10b6:208:15e::32) by IA1PR12MB7759.namprd12.prod.outlook.com (2603:10b6:208:420::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.27; Mon, 9 Sep 2024 17:31:12 +0000 Received: from BL02EPF0001A0FD.namprd03.prod.outlook.com (2603:10b6:208:15e:cafe::47) by MN2PR17CA0019.outlook.office365.com (2603:10b6:208:15e::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.27 via Frontend Transport; Mon, 9 Sep 2024 17:31:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BL02EPF0001A0FD.mail.protection.outlook.com (10.167.242.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.13 via Frontend Transport; Mon, 9 Sep 2024 17:31:11 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:49 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:49 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Mon, 9 Sep 2024 10:30:47 -0700 From: Michael Guralnik To: CC: , , , , , , "Michael Guralnik" Subject: [PATCH v3 rdma-next 6/7] RDMA/nldev: Add support for RDMA monitoring Date: Mon, 9 Sep 2024 20:30:24 +0300 Message-ID: <20240909173025.30422-7-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240909173025.30422-1-michaelgur@nvidia.com> References: <20240909173025.30422-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FD:EE_|IA1PR12MB7759:EE_ X-MS-Office365-Filtering-Correlation-Id: 34066451-016f-495a-6bbd-08dcd0f5331c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: Sznf8tncEQFQKP8mvASjmuqqj/vyFW3Eu1GlKUsZhyeLaQ8YJIJLLOSqkrGUgiihNBKU9vCJuYnnZtQs5UukdeCXKNCt3xE+naJMPL0wGrlIS3fPuDd67zlnalGUgGasPmVdbdwlEfcCycta8Rr7cHK0Ua4c4LJ9v5S7pAw2ooFieif4oHQbNND93heuJWo1aUX+loAtctQr58yVNwxG14zHcRwqkXuxkaGuycW/aYIGwG+3noP+1rHDz9kDvTrK3E71QPiP7vx5Z4ESAmFET5KqRG3Gk8MdK+DjevOd5O82aI07dDFgvbXfkmbi5d0VWWUYC9sxOc6gu25Lq+gJh56ihT4H7pfl62vun8D5BnAkHsfZV+CtnPURC5077T3jWpwedl7krYbprqAmdviA6T+99vwFmqPtGAfjY/ak+sGNJmf5LU6k0g952BMOAZyJCg9MDxYTmBiMwaHaPlgwZxpwFwrt9knLleyTNp/QtYiCJ7+QERduTsKZ489iZ3wiy7FxAup054mQNcj2lja2Dh+tsoHlpMgAez0EBgP4gUHbDj/WRh6by8qLMNFt2R6BpjKYYxTQsgNIqLc6XDXAlIMgGZIA++5IhX2AIxUDGRTJOwH9N9kfcUAjPG953AvlhacWbVINzlJ7pyYu51VA9CewTaQKi4jeV7KmGEJzS0sXwaly8+5vKM7UqkGmc9VS0X+DFL01MSjz2fGyr/FdhrSs6oTZj0Rwd+JMPE13R/1/rSS4SrcNj0UnmFik6HuKdkFX2f0/J4L/MUUJcOouwsdW+zkKuwyjs+wIbEVQ/VkJcmWG3fSwgFp4RAKAxDgLDptwrBZ6heIO5uchc09a5NFCL5T/rGBOjb7ZZHXWgnvn14nnxrv5fKHeuuKtH4Y6TqujBwYEjcJpEKa4/M9K7womAETW7FRkzLZvmWKQ/Px18jr3etjYSm/xU9GHj1ZivwXDeCiU+BLUTTrlxOHJNaAX3ZUgw6huT2/vaz9cFzznKzfxEjANSJDnmyNTCGUPm+fYz+/mMGQF3IXNkZkzs17eFfSl3+WYDr0fHPFuvrgftNPB0gLtrlpvwWfodB8pBmO0JRMlQoEtItceqTkyd5XfAu0bb2IBxabb5iJn6OqkbRceOvXTIKp8PCVrrTazcqjgNAOqWArs/C/lqz4O04Y/JuN2q4SO1fa5F2SJj93PBeamjvlir6hZc3WcSWtYy13hdve6CZn6NQV77xv7GqvwpJISghZOxD3GE17CZezkV2wRgKuyWvaVkO5UUfvKtHPlqUtobWncgJdla5tC4mM/PO04rxginH0BLQBUuXeuycgq/MzspKLsf8iwREIKaxleMbSP2ckCaBEBLgfQUznut6grwauH992Z0q94vP3MVUZLcr/Aqn5vmOZy5QEoFVhsQrXdNfwl2jP45+HigxGrxRIlKujFEKpeJOoProIO+7d1QJnh+ZPvRkKA6LN2 X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2024 17:31:11.9864 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 34066451-016f-495a-6bbd-08dcd0f5331c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A0FD.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7759 From: Chiara Meiohas Introduce a new netlink command to allow rdma event monitoring. The rdma events supported now are IB device registration/unregistration and net device attachment/detachment. Example output of rdma monitor and the commands which trigger the events: $ rdma monitor $ rmmod mlx5_ib [UNREGISTER] ibdev_idx 1 ibdev rocep8s0f1 [UNREGISTER] ibdev_idx 0 ibdev rocep8s0f0 $ modprobe mlx5_ib [REGISTER] ibdev_idx 2 ibdev mlx5_0 [NETDEV_ATTACH] ibdev_idx 2 ibdev mlx5_0 port 1 netdev_idx 4 netdev eth2 [REGISTER] ibdev_idx 3 ibdev mlx5_1 [NETDEV_ATTACH] ibdev_idx 3 ibdev mlx5_1 port 1 netdev_idx 5 netdev eth3 $ devlink dev eswitch set pci/0000:08:00.0 mode switchdev [UNREGISTER] ibdev_idx 2 ibdev rocep8s0f0 [REGISTER] ibdev_idx 4 ibdev mlx5_0 [NETDEV_ATTACH] ibdev_idx 4 ibdev mlx5_0 port 30 netdev_idx 4 netdev eth2 $ echo 4 > /sys/class/net/eth2/device/sriov_numvfs [NETDEV_ATTACH] ibdev_idx 4 ibdev rdmap8s0f0 port 2 netdev_idx 7 netdev eth4 [NETDEV_ATTACH] ibdev_idx 4 ibdev rdmap8s0f0 port 3 netdev_idx 8 netdev eth5 [NETDEV_ATTACH] ibdev_idx 4 ibdev rdmap8s0f0 port 4 netdev_idx 9 netdev eth6 [NETDEV_ATTACH] ibdev_idx 4 ibdev rdmap8s0f0 port 5 netdev_idx 10 netdev eth7 [REGISTER] ibdev_idx 5 ibdev mlx5_0 [NETDEV_ATTACH] ibdev_idx 5 ibdev mlx5_0 port 1 netdev_idx 11 netdev eth8 [REGISTER] ibdev_idx 6 ibdev mlx5_0 [NETDEV_ATTACH] ibdev_idx 6 ibdev mlx5_0 port 1 netdev_idx 12 netdev eth9 [REGISTER] ibdev_idx 7 ibdev mlx5_0 [NETDEV_ATTACH] ibdev_idx 7 ibdev mlx5_0 port 1 netdev_idx 13 netdev eth10 [REGISTER] ibdev_idx 8 ibdev mlx5_0 [NETDEV_ATTACH] ibdev_idx 8 ibdev mlx5_0 port 1 netdev_idx 14 netdev eth11 $ echo 0 > /sys/class/net/eth2/device/sriov_numvfs [UNREGISTER] ibdev_idx 5 ibdev rocep8s0f0v0 [UNREGISTER] ibdev_idx 6 ibdev rocep8s0f0v1 [UNREGISTER] ibdev_idx 7 ibdev rocep8s0f0v2 [UNREGISTER] ibdev_idx 8 ibdev rocep8s0f0v3 [NETDEV_DETACH] ibdev_idx 4 ibdev rdmap8s0f0 port 2 [NETDEV_DETACH] ibdev_idx 4 ibdev rdmap8s0f0 port 3 [NETDEV_DETACH] ibdev_idx 4 ibdev rdmap8s0f0 port 4 [NETDEV_DETACH] ibdev_idx 4 ibdev rdmap8s0f0 port 5 Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/core/device.c | 38 +++++++++ drivers/infiniband/core/netlink.c | 1 + drivers/infiniband/core/nldev.c | 124 ++++++++++++++++++++++++++++++ include/rdma/rdma_netlink.h | 12 +++ include/uapi/rdma/rdma_netlink.h | 15 ++++ 5 files changed, 190 insertions(+) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 9e765c79a892..d571b78d1bcc 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -1351,6 +1351,30 @@ static void prevent_dealloc_device(struct ib_device *ib_dev) { } +static void ib_device_notify_register(struct ib_device *device) +{ + struct net_device *netdev; + u32 port; + int ret; + + ret = rdma_nl_notify_event(device, 0, RDMA_REGISTER_EVENT); + if (ret) + return; + + rdma_for_each_port(device, port) { + netdev = ib_device_get_netdev(device, port); + if (!netdev) + continue; + + ret = rdma_nl_notify_event(device, port, + RDMA_NETDEV_ATTACH_EVENT); + dev_put(netdev); + if (ret) + return; + } + return; +} + /** * ib_register_device - Register an IB device with IB core * @device: Device to register @@ -1449,6 +1473,8 @@ int ib_register_device(struct ib_device *device, const char *name, dev_set_uevent_suppress(&device->dev, false); /* Mark for userspace that device is ready */ kobject_uevent(&device->dev.kobj, KOBJ_ADD); + + ib_device_notify_register(device); ib_device_put(device); return 0; @@ -1491,6 +1517,7 @@ static void __ib_unregister_device(struct ib_device *ib_dev) goto out; disable_device(ib_dev); + rdma_nl_notify_event(ib_dev, 0, RDMA_UNREGISTER_EVENT); /* Expedite removing unregistered pointers from the hash table */ free_netdevs(ib_dev); @@ -2159,6 +2186,7 @@ static void add_ndev_hash(struct ib_port_data *pdata) int ib_device_set_netdev(struct ib_device *ib_dev, struct net_device *ndev, u32 port) { + enum rdma_nl_notify_event_type etype; struct net_device *old_ndev; struct ib_port_data *pdata; unsigned long flags; @@ -2190,6 +2218,16 @@ int ib_device_set_netdev(struct ib_device *ib_dev, struct net_device *ndev, spin_unlock_irqrestore(&pdata->netdev_lock, flags); add_ndev_hash(pdata); + + down_read(&devices_rwsem); + if (xa_get_mark(&devices, ib_dev->index, DEVICE_REGISTERED) && + xa_load(&devices, ib_dev->index) == ib_dev) { + etype = ndev ? + RDMA_NETDEV_ATTACH_EVENT : RDMA_NETDEV_DETACH_EVENT; + rdma_nl_notify_event(ib_dev, port, etype); + } + up_read(&devices_rwsem); + return 0; } EXPORT_SYMBOL(ib_device_set_netdev); diff --git a/drivers/infiniband/core/netlink.c b/drivers/infiniband/core/netlink.c index ae2db0c70788..def14c54b648 100644 --- a/drivers/infiniband/core/netlink.c +++ b/drivers/infiniband/core/netlink.c @@ -311,6 +311,7 @@ int rdma_nl_net_init(struct rdma_dev_net *rnet) struct net *net = read_pnet(&rnet->net); struct netlink_kernel_cfg cfg = { .input = rdma_nl_rcv, + .flags = NL_CFG_F_NONROOT_RECV, }; struct sock *nls; diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index 4d4a1f90e484..30b0fd54a7d3 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -170,6 +170,7 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = { [RDMA_NLDEV_ATTR_DEV_TYPE] = { .type = NLA_U8 }, [RDMA_NLDEV_ATTR_PARENT_NAME] = { .type = NLA_NUL_STRING }, [RDMA_NLDEV_ATTR_NAME_ASSIGN_TYPE] = { .type = NLA_U8 }, + [RDMA_NLDEV_ATTR_EVENT_TYPE] = { .type = NLA_U8 }, }; static int put_driver_name_print_type(struct sk_buff *msg, const char *name, @@ -2722,6 +2723,129 @@ static const struct rdma_nl_cbs nldev_cb_table[RDMA_NLDEV_NUM_OPS] = { }, }; +static int fill_mon_netdev_association(struct sk_buff *msg, + struct ib_device *device, u32 port, + const struct net *net) +{ + struct net_device *netdev = ib_device_get_netdev(device, port); + int ret = 0; + + if (netdev && !net_eq(dev_net(netdev), net)) + goto out; + + ret = nla_put_u32(msg, RDMA_NLDEV_ATTR_DEV_INDEX, device->index); + if (ret) + goto out; + + ret = nla_put_string(msg, RDMA_NLDEV_ATTR_DEV_NAME, + dev_name(&device->dev)); + if (ret) + goto out; + + ret = nla_put_u32(msg, RDMA_NLDEV_ATTR_PORT_INDEX, port); + if (ret) + goto out; + + if (netdev) { + ret = nla_put_u32(msg, + RDMA_NLDEV_ATTR_NDEV_INDEX, netdev->ifindex); + if (ret) + goto out; + + ret = nla_put_string(msg, + RDMA_NLDEV_ATTR_NDEV_NAME, netdev->name); + } + +out: + dev_put(netdev); + return ret; +} + +static void rdma_nl_notify_err_msg(struct ib_device *device, u32 port_num, + enum rdma_nl_notify_event_type type) +{ + struct net_device *netdev; + + switch (type) { + case RDMA_REGISTER_EVENT: + dev_warn_ratelimited(&device->dev, + "Failed to send RDMA monitor register device event\n"); + break; + case RDMA_UNREGISTER_EVENT: + dev_warn_ratelimited(&device->dev, + "Failed to send RDMA monitor unregister device event\n"); + break; + case RDMA_NETDEV_ATTACH_EVENT: + netdev = ib_device_get_netdev(device, port_num); + dev_warn_ratelimited(&device->dev, + "Failed to send RDMA monitor netdev attach event: port %d netdev %d\n", + port_num, netdev->ifindex); + dev_put(netdev); + break; + case RDMA_NETDEV_DETACH_EVENT: + dev_warn_ratelimited(&device->dev, + "Failed to send RDMA monitor netdev detach event: port %d\n", + port_num); + default: + break; + }; +} + +int rdma_nl_notify_event(struct ib_device *device, u32 port_num, + enum rdma_nl_notify_event_type type) +{ + struct sk_buff *skb; + struct net *net; + int ret = 0; + void *nlh; + + net = read_pnet(&device->coredev.rdma_net); + if (!net) + return -EINVAL; + + skb = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL); + if (!skb) + return -ENOMEM; + nlh = nlmsg_put(skb, 0, 0, + RDMA_NL_GET_TYPE(RDMA_NL_NLDEV, RDMA_NLDEV_CMD_MONITOR), + 0, 0); + + switch (type) { + case RDMA_REGISTER_EVENT: + case RDMA_UNREGISTER_EVENT: + ret = fill_nldev_handle(skb, device); + if (ret) + goto err_free; + break; + case RDMA_NETDEV_ATTACH_EVENT: + case RDMA_NETDEV_DETACH_EVENT: + ret = fill_mon_netdev_association(skb, device, + port_num, net); + if (ret) + goto err_free; + break; + default: + break; + } + + ret = nla_put_u8(skb, RDMA_NLDEV_ATTR_EVENT_TYPE, type); + if (ret) + goto err_free; + + nlmsg_end(skb, nlh); + ret = rdma_nl_multicast(net, skb, RDMA_NL_GROUP_NOTIFY, GFP_KERNEL); + if (ret && ret != -ESRCH) { + skb = NULL; /* skb is freed in the netlink send-op handling */ + goto err_free; + } + return 0; + +err_free: + rdma_nl_notify_err_msg(device, port_num, type); + nlmsg_free(skb); + return ret; +} + void __init nldev_init(void) { rdma_nl_register(RDMA_NL_NLDEV, nldev_cb_table); diff --git a/include/rdma/rdma_netlink.h b/include/rdma/rdma_netlink.h index c2a79aeee113..326deaf56d5d 100644 --- a/include/rdma/rdma_netlink.h +++ b/include/rdma/rdma_netlink.h @@ -6,6 +6,8 @@ #include #include +struct ib_device; + enum { RDMA_NLDEV_ATTR_EMPTY_STRING = 1, RDMA_NLDEV_ATTR_ENTRY_STRLEN = 16, @@ -110,6 +112,16 @@ int rdma_nl_multicast(struct net *net, struct sk_buff *skb, */ bool rdma_nl_chk_listeners(unsigned int group); +/** + * Prepare and send an event message + * @ib: the IB device which triggered the event + * @port_num: the port number which triggered the event - 0 if unused + * @type: the event type + * Returns 0 on success or a negative error code + */ +int rdma_nl_notify_event(struct ib_device *ib, u32 port_num, + enum rdma_nl_notify_event_type type); + struct rdma_link_ops { struct list_head list; const char *type; diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h index 2f37568f5556..5f9636d26050 100644 --- a/include/uapi/rdma/rdma_netlink.h +++ b/include/uapi/rdma/rdma_netlink.h @@ -15,6 +15,7 @@ enum { enum { RDMA_NL_GROUP_IWPM = 2, RDMA_NL_GROUP_LS, + RDMA_NL_GROUP_NOTIFY, RDMA_NL_NUM_GROUPS }; @@ -305,6 +306,8 @@ enum rdma_nldev_command { RDMA_NLDEV_CMD_DELDEV, + RDMA_NLDEV_CMD_MONITOR, + RDMA_NLDEV_NUM_OPS }; @@ -574,6 +577,8 @@ enum rdma_nldev_attr { RDMA_NLDEV_ATTR_NAME_ASSIGN_TYPE, /* u8 */ + RDMA_NLDEV_ATTR_EVENT_TYPE, /* u8 */ + /* * Always the end */ @@ -624,4 +629,14 @@ enum rdma_nl_name_assign_type { RDMA_NAME_ASSIGN_TYPE_USER = 1, /* Provided by user-space */ }; +/* + * Supported rdma monitoring event types. + */ +enum rdma_nl_notify_event_type { + RDMA_REGISTER_EVENT, + RDMA_UNREGISTER_EVENT, + RDMA_NETDEV_ATTACH_EVENT, + RDMA_NETDEV_DETACH_EVENT, +}; + #endif /* _UAPI_RDMA_NETLINK_H */ From patchwork Mon Sep 9 17:30:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13797380 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2048.outbound.protection.outlook.com [40.107.244.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2BA518950E for ; Mon, 9 Sep 2024 17:31:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.48 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903079; cv=fail; b=s3Y/E/cMFcxlHqgpmPmbAQLAJCXpdi5ms5qeT8pMcOemPwwmco0+lDlFCcZXewPWGCBAFzxoKNY8cyPI9Pyi/qNo9VMRB40i/pk4W9dHmfUF1VvrfVZD1NvZkm5HD2cT1ZnhiM2vxrW+UWU/uOZdTfYwYbHDknlBmd7FPQCgRnQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725903079; c=relaxed/simple; bh=xWP2jeU5dPrV6UR509bMAe5e4I5o2qzEfiOGmI56xZU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UsEBaqII3U9ayl19puIKSYWn7Ri9FROxNECkJcsQwzTVfH9JaMPqo+KV2OOL/rlNxB5ToetTzqdshvXiwuvBao1kfLGBLifZS9v+aw9WK7UlUcJDqhh4puYWosOXPt+5ZLKrmVubqe8LXLgcB4xNpgP+HY9AuZucz9Jrq15ORAc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=NkGj/bfZ; arc=fail smtp.client-ip=40.107.244.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="NkGj/bfZ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=h0awmGJIci+m4BGnImOo2Yea7RqIHHIEOk23P6wb0+Znt+8ZspnQbrALEDYwz6TUomtk05jgcefQa7Xs2ssepyXZZhYi0JC0NYieu9xjw0e3ILrkSZrGoyQbszeDfRWs/13vPiHTSR59G/LPZOhahhq0Q5TmhsTK7gr3QlJc0nw9RX9dAsHmA/USWZ0uaZy+oLfa8iwAEQMYkEtB4f58CNlTNHBR2qVjDg0dO5JDWB50QWMeI5NGRzc3lH/nx/ahQFDePgmdK8CfN+b6nDi8xTRMrD1d5rQBkf19L9v0qlwl4N/tJ0BMa6Pbsr17Ez/ihGQzWQQ5qGf+vLuWo6uerQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GmzKhBQfEEZFhc4PuqNqyR/ZS5xARz64NwjVDxh3VtY=; b=S2w1PygkNkq7Ib8aOXW4xD86LfNgmc/gFH1rD9XCThDRqk2owA9Xm8vysBX/ZP6ky8VBZsPk+8Z1FLWxxaUwuvraJhKsPul0XzduS4scLE0LSs8K+bXOvAHS99HWm/99Ln1iOSpcmYej+ATTNZfrRLWwyxCCSCG4tBF+yQZMjXhax7bhtgjxK7KU2RuT24ueAwIfEXkHfJ2Jgo40Djy6uoxQ0g1PdIM6GW0JFEQcJzxJKI5hBpf0Nrv7BCUkOmrnIZ9T4lZWZUlF56QzHVO5sS7HjjyeHobdEIwq/05QZVUCM9fCLFcNzAgO/Wma0r8Q3MoHHcd6SLM2AuNgwEnIDQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GmzKhBQfEEZFhc4PuqNqyR/ZS5xARz64NwjVDxh3VtY=; b=NkGj/bfZ7Yv5lpR5LyHwcDH/cGixXnMPmFy91gYcW7wjNLX60FqDfwM6YMsdcgCKLGrqtPrHW8NTcWjq2j4ceIFzvPfLpraNsTYpkKOAooAzlF37aqqzBPklfUFKb7hfHyHd5zqKDOIvdJYiYFawC3NLbQ3J7pKmv3PUsuIrByUdq5+HzRhTmG2C9xkdKxacF6crat0e5XfJ57fqhkZ/T0Ada07fczY9xVp97XzIftKsew6pSJGIylfpZWJCulEjlica8Wo7nfepcjXgE3DYd1mu7rUMYdBvS4MatGMDEZMmannSlmGD4Pjxs0MeKOlgPWtoZL+MfE0mKc+ojpCL4Q== Received: from MN2PR06CA0007.namprd06.prod.outlook.com (2603:10b6:208:23d::12) by PH7PR12MB7869.namprd12.prod.outlook.com (2603:10b6:510:27e::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.24; Mon, 9 Sep 2024 17:31:13 +0000 Received: from BL02EPF00029929.namprd02.prod.outlook.com (2603:10b6:208:23d:cafe::c2) by MN2PR06CA0007.outlook.office365.com (2603:10b6:208:23d::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.24 via Frontend Transport; Mon, 9 Sep 2024 17:31:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF00029929.mail.protection.outlook.com (10.167.249.54) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.13 via Frontend Transport; Mon, 9 Sep 2024 17:31:13 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:52 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 9 Sep 2024 10:30:52 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Mon, 9 Sep 2024 10:30:49 -0700 From: Michael Guralnik To: CC: , , , , , , "Michael Guralnik" Subject: [PATCH v3 rdma-next 7/7] RDMA/nldev: Expose whether RDMA monitoring is supported Date: Mon, 9 Sep 2024 20:30:25 +0300 Message-ID: <20240909173025.30422-8-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240909173025.30422-1-michaelgur@nvidia.com> References: <20240909173025.30422-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF00029929:EE_|PH7PR12MB7869:EE_ X-MS-Office365-Filtering-Correlation-Id: c32bfc3d-8621-457b-3f8b-08dcd0f533b3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: fArufpEAlp4dvICosNJIcQT9dGHQGqmiXZ6/A975I38j2tLmMGQWmMaD6i4847RMVR60wPx3iwvCfKrTZqAAXPIv7oxzvjHf0VXrwynChsPEJ3x4vkFz5nFv+PDWAX9TjthPKRfNlGS86k+/KYAHz72oVdXYDsRIDScChIuv43OVuOqkIGCRN9tR01VNFDJIIrTAI0ah9rdO84OkL+fy8pNwHnSG6bN0JVUxgM3EV3Olb4T0BA6j2tAXf4abMXzZdNrQJwAciurFXgE32mxIGjknsGVttNNhzB/Reah6V6AEjzBX6KsiAUmZFNDluk2gRGUYmCQpBzS/Ag/BdG73Zm3mxiNax4mOchyeIuYgeNEQizodVVO01SuuxtnkcrlMqTYYa21/f2xLfjhzphDnrHojLfZrsQNIiuAoNX2Was/StiJ99yTqELeS3UkaTRpzJDBC6/VDd5ZEvj6InDqyi2mBIGAk+oYRU2+wOvc1BiHO6CqrLQ01ghL0ZOx0sXRJmQbUWAOyQ4pVZ5VT0pfjBCgsUeHp2E9TBgeL+GtUF2kkH37LnQbX5dc/dqpJACrGFCx9MGoxo5hlxvHJOSgOGhvUfq5gFfWfV2VKylq7HeEJdeCcqU0uKVt/NaiC6Ke5QykWzJytEW7D+nEAZwrcMmQK/XwG6AaMBnKoKwbITCnb3qTWtM6RV6nioQTpTiz/mNDkSL7cGi566pB8U3aJOgzsYJysbLBZkLhYgoicir94wO5MTVWvU55jf9rBVExZyjySC+wo0HoNncckzVvVkwaUsdv4oyXbjcdvFQ5ORlsRGgnAoYvfzsAZQ5f+0K/qZ8968C0csPHSPVxRUQ61ZVyGEaBTcj4+eHoycUy1qbhpQtPamvPSmFvDspIQAxgEfir4m5psR7Hw2i89LRdsJ8DautnnQIvlX7udVYePv021JYHl/LsuVUxkqxpqYNuRoNLHg6bE9/no0eXUcLIEQQiOQqcAY9pi/CHWRonptRV0FOUxndiunr6yQsmw4798i85I/rreawx+MYm1sxALjTIsg8u/fkUheIz5yBnMZjrUw7FssNbGPdv2sN9+OXTYEha/Mxmd5WZlrEQosB2EawbQB103hcZ3RbEDfQu0W3veH6+KOUXRd6HMMqPHwrZ0OltjDpD3BVjEAoIfGZKJn1ClwveFClIctzOVD0x18cP0KEHkUFu29hVFTfFDPOH3ButoyL4Cwp4igQU8h9oOW9jjtWo+OjRzJQ8xh4kUG6RjS4g3E9NHiqayJn8zBC7C2GtPuablO5jK38vFxcvijy+8LZiR9NAlUTsLgSzYq8Oiu4cpdhcuU44tjgs8XnB5uDaZiuo96MMl2dAsmKCwJx5Ifs0C3SNe3sKhe4T+HR/F4CKPTSNiV0lKX2tk3bS/n9RigVeMNWmbgLZTntyvcuMSZYDMO3axbT4vMPZwt0Xdh7kWEsCJVjXulat8rpKD X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(1800799024)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2024 17:31:13.0843 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c32bfc3d-8621-457b-3f8b-08dcd0f533b3 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00029929.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7869 From: Chiara Meiohas Extend the "rdma sys" command to display whether RDMA monitoring is supported. RDMA monitoring is not supported in mlx4 because it does not use the ib_device_set_netdev() API, which sends the RDMA events. Example output for kernel where monitoring is supported: $ rdma sys show netns shared privileged-qkey off monitor on copy-on-fork on Example output for kernel where monitoring is not supported: $ rdma sys show netns shared privileged-qkey off monitor off copy-on-fork on Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/core/nldev.c | 6 ++++++ include/uapi/rdma/rdma_netlink.h | 1 + 2 files changed, 7 insertions(+) diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index 30b0fd54a7d3..b2dca6aa531d 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -1952,6 +1952,12 @@ static int nldev_sys_get_doit(struct sk_buff *skb, struct nlmsghdr *nlh, nlmsg_free(msg); return err; } + + err = nla_put_u8(msg, RDMA_NLDEV_SYS_ATTR_MONITOR_MODE, 1); + if (err) { + nlmsg_free(msg); + return err; + } /* * Copy-on-fork is supported. * See commits: diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h index 5f9636d26050..39be09c0ffbb 100644 --- a/include/uapi/rdma/rdma_netlink.h +++ b/include/uapi/rdma/rdma_netlink.h @@ -579,6 +579,7 @@ enum rdma_nldev_attr { RDMA_NLDEV_ATTR_EVENT_TYPE, /* u8 */ + RDMA_NLDEV_SYS_ATTR_MONITOR_MODE, /* u8 */ /* * Always the end */