From patchwork Wed Aug 21 05:10:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13770784 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2064.outbound.protection.outlook.com [40.107.244.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5EE6A2599 for ; Wed, 21 Aug 2024 05:10:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.64 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217043; cv=fail; b=DQk+7YuNfpOTaUUJaQrXmTmQxXsMC8FLZRfcCftqACh8Z9QiGVgBWME5wXru5vj/5a7GgUYKc1K0L8N74npQdzsCrCwfTpfoKoFZPoRIKbQbAcl8Kp2NZnFFicwdUzZcFvmDm3S5oOrkF8QB8gBvrB9IBxuWBXTYWww2FkjxjkA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217043; c=relaxed/simple; bh=EQFkdF98ImVIpiz+D5z6Qiggi1JO6HFXBAoFITwzt6c=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=q7uQ428Wxbn2NEhXi6EomHhHSUVzrW3D0/QPYymqFzZDUvAyuU5HLVNAZGOS5LNElNGW8Rgz9OQVhh0fI1iXndzsG/fCqiAo0mCi1e8NyRFVofaepQ9QZXr8y2jC15tEHyJQOulb2G0gUdwfhDBn6KuFdfJPZhMga4eBQjUggUA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ols2QTFV; arc=fail smtp.client-ip=40.107.244.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ols2QTFV" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=xwZcdF8cYsgRvHIUigMsjqxNeT+hMQuv8pwP8ooE5eYF6XuWsEeUzqVjVJfXu8KfOyQldZ+yVW9+xgG/1D1OpaoffWoujw1wtMnl58trdzZrVnZdO7c3opi1jV13GkwFlINq3Aw/w3dk1If+p02kY6PHiLzGH1vmVEhisJzho1QnkqOwvDzIwMqDG5t3KoiKjIwhAEnfvTVCduyP53fFdZSTML5INFAu0ELwkzDepsiJmietZs20d0BnJ8dRhx7vB3VM+xw83cPnVR8dwofsX48e/Y3+5Pdmwopg1QTxtFxk9uTdDNSo18+Ln3nqIHhMKhCbX2V1sGsB4rjxwZON3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cY+B4Nsuo7WtmAU1ohBNCsMm+kGOeHPENTxEA13abtY=; b=n94LeZEx3VgRKTEN4IrQMEGBy8i+ljlJeXnNqrO9rIa3XKgZ1NPCIhoOX+edEI6pC7kfBNYAANze3jnhK+lJ7R2ajYt6cKdAukuJ/fKI7x0U2UWSv9LPW+1OD3it+A1lW+jT5IcSPXg0wOgJXVYV5JZ9eUjSReK7NN5yZNnAP/Ydm/mWE5rETXVV+7WnBtjTApzh+FAXCxoEbNEFjgGc4nwYJ+VzZQtt7Ty70hXQaX2lVLupv2kOphqxqjTj7UEtw0mwc8ZuBeOnzXmOgEBlc6UsH7gnWaeWxVjyjnuGYBJA2aNaWWWPleJSYVlfux8X6zP+Ci+J/wBK2o/ehQHtvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cY+B4Nsuo7WtmAU1ohBNCsMm+kGOeHPENTxEA13abtY=; b=ols2QTFV3VuU9fFaIQEDIv18B29WSprXNekth1pr1JkOc64lifa81GrCg9f2/c2JqfH0lNcDQEgXxHqxqSbKirTo2i0N10RW6kv0OviUZpzkXF37CeVxgcgCku2RpKazJajlhD3o19k+xFJR6LjeWfPTW5/qpzZvNXi5ZqnqlcdoxHsGRFaf5+jTNd38R3XJjTbbNghC5Q3Tg77NF519AzjilBt1YRQEpdmYGhHSjqUJkoilODIRrTZBwjjAgqBUzNg9lvtrLgRNa0s8g7z9BfkA9iqsdSmAaM9Y5MeFBoDqb7j62M5Lm2EAsf5qUHP6joIPNrVd/7Z4OVun5tVg/w== Received: from BYAPR07CA0062.namprd07.prod.outlook.com (2603:10b6:a03:60::39) by CH2PR12MB4295.namprd12.prod.outlook.com (2603:10b6:610:a7::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.16; Wed, 21 Aug 2024 05:10:37 +0000 Received: from SJ5PEPF000001EE.namprd05.prod.outlook.com (2603:10b6:a03:60:cafe::f1) by BYAPR07CA0062.outlook.office365.com (2603:10b6:a03:60::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.21 via Frontend Transport; Wed, 21 Aug 2024 05:10:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ5PEPF000001EE.mail.protection.outlook.com (10.167.242.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.11 via Frontend Transport; Wed, 21 Aug 2024 05:10:37 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:26 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:26 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 20 Aug 2024 22:10:24 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next 1/7] RDMA/mlx5: Check RoCE LAG status before getting netdev Date: Wed, 21 Aug 2024 08:10:11 +0300 Message-ID: <20240821051017.7730-2-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240821051017.7730-1-michaelgur@nvidia.com> References: <20240821051017.7730-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001EE:EE_|CH2PR12MB4295:EE_ X-MS-Office365-Filtering-Correlation-Id: 6ff5cd66-6823-4826-6386-08dcc19f97e1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|36860700013|376014; X-Microsoft-Antispam-Message-Info: YMl7Bk3/0T8i/bjHfxe2iHLl6kVu+P2JzYz7nQswvP+hG6Hn6VHzPcok0xnUDhkP8Zk7kqzrPSSR3SGauJFDh7HoKkyonuiyFgavtcbX/5zZTGnu5igw4Xu72MVYWKdBd9s7Upv7qy/gn8QQRrS+Vs+AgCnIeqbQjor7xB5UnnZw/eOUSM9GuOM3XSUpIyB0ikPQKzD/3NVz0wC0Sf6y0xv0DsK5hBwJrNe6tdmcCkMqrlJBANNpTFU+I+lCwhgH8Muz7IHFUIETathTz7vYf6P0wFl268PkI33QJMluajeE+wVKeI/5dLzBwPDqsA60GatBwo99VqNjP4SzJMTr7QSpuYIwY2S2JhoWNOOt0Y3xbvODN0NPsA03Q0F6o6zj8zyGREmxbIo3CoEu+0w9d7uoQUjjkpP6oC5IZrNh63dkKNNdjcsx+Tb+aGFZ9F0BZ6SEHsPdzmqc7f7+do+eaCTRpEIrCif4UZWepmzuunGJ7i2I+UE3qrmu881C5AowJD9TB0Be4eHkUU/AOeXRDk1ywcF2DFmouUyviR4qNoGnSvw+mNYNxn/xJo8qu12bZv1EEiprV/cmnQEwUGc5FvZxxVhZJjs0U+zUepSo9DljInxA8VSNxXHdKvJE++Gf+bC20TrpL1AfUXveZp5k/M1IV/VxkD9+a76WZjVg/Id9N0C+N2IwPjm1QP3hTvh9wQZxrtRblyqeHrHnbIrqIA+yj4WwccE0QJcZtsc5idkO2Lg2CDfLzi2RvglCPojKFegtCFZnHtr106SBM2IREtGwT9MBUccXe5pyNz3HVs9SE/NNCiS+/L82YePQpsmpTQBukfV/DoCogK3lCDSKUCvGjSVpv/Dd6wTDTWZ+lKVk0R1Pbpz41MnqJQaKVe4InGHbLML0UBCUqHQTMeH36EDLMfcW9wcfA9hrqVKeAhb2ox/AuGRe90Z+yRCS8tHlolh1uNffgmQ3W5lEuVnBc9rlAaHPdy6syKqCBCFXc1jPVvA7RNbl9JRe/WgSu4E7FLYJVyy/RyXycMiVh07XP6XxZAR3Zu9lnHfGntXPU+t/ZioL4GKLUPnOmFmah/CpEf0ly3O3+PCosor0BsCCjEwTQGih1cQtZ3wEDg/ILFFDZVy0GlLv9k7I5TsWEMA7/Q/ElcP8aM3ThWP6VL0kkgmEIodxlDj0Er20dcPkiyI/hKONQgcogL1lWuT6Yy3gv887obKFzMI+KBXABWgS5eXqkuR/Difg+TZqnJAFPLoIOueOhdbMwUHT//ouA2qg4vowIX9XPTeAiitGFwQ1n+PK2/DjBUyUHeVCBdfyPsyfpoL62A4BadsCg9x9wft4wXUcwxv6JpMLkOBMEUBIavivXbFt18/Gcq5EmdIFMAAL2yVf1rESUv8KFioqfHahQyTNsFWyIEG5UZMzDRMgpMz0LZxykN1rGdeKMKcnLg1AbRr9IuMotEED10KE7JBL X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(36860700013)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 05:10:37.1703 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6ff5cd66-6823-4826-6386-08dcc19f97e1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001EE.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4295 From: Mark Bloch Check if RoCE LAG is active before calling the LAG layer for netdev. This clarifies if LAG is active. No behavior changes with this patch. Signed-off-by: Mark Bloch Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/main.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index b85ad3c0bfa1..cdf1ce0f6b34 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -198,12 +198,18 @@ static int mlx5_netdev_event(struct notifier_block *this, case NETDEV_CHANGE: case NETDEV_UP: case NETDEV_DOWN: { - struct net_device *lag_ndev = mlx5_lag_get_roce_netdev(mdev); struct net_device *upper = NULL; - if (lag_ndev) { - upper = netdev_master_upper_dev_get(lag_ndev); - dev_put(lag_ndev); + if (mlx5_lag_is_roce(mdev)) { + struct net_device *lag_ndev; + + lag_ndev = mlx5_lag_get_roce_netdev(mdev); + if (lag_ndev) { + upper = netdev_master_upper_dev_get(lag_ndev); + dev_put(lag_ndev); + } else { + goto done; + } } if (ibdev->is_rep) @@ -257,9 +263,10 @@ static struct net_device *mlx5_ib_get_netdev(struct ib_device *device, if (!mdev) return NULL; - ndev = mlx5_lag_get_roce_netdev(mdev); - if (ndev) + if (mlx5_lag_is_roce(mdev)) { + ndev = mlx5_lag_get_roce_netdev(mdev); goto out; + } /* Ensure ndev does not disappear before we invoke dev_hold() */ From patchwork Wed Aug 21 05:10:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13770786 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2044.outbound.protection.outlook.com [40.107.244.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6DF5915747C for ; Wed, 21 Aug 2024 05:10:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.44 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217046; cv=fail; b=ALcLdu4nS51CAOLsx23fwxnHsBjbqw0UhwqLlpVKyYmZvdvNL/orcScC69S+zx1VQnGS8s6S7/0x3Yr6y4M9vAtj4pV9L0RlAj2jg6w6jiQa8csuJYy8CBMHU5/27SgmzFVhfx/XRB4z9zcspQyWvXrc4mqVhM0zoK49Vy5yP8Q= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217046; c=relaxed/simple; bh=lLEMnxsxyqW9JKAvem3D0RAljr/RlvLLG90QJIOYFXs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=mnY6PQMRx/pCSTI9AzOtc0lG67YfR28NvCU8f32Jv6wOHfqPDyvIbbKOjf8Tme8NSL2ZLo2sw8wN6l8VWF0KcD/RllX4OpLt5chFfHfsGBim55JQWq4qYFsOtktW8MVUsEOb43IMwDOvmskoCKi2rZzkk1liq10aHVbr3Qa3X+Q= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=YcHfZLMY; arc=fail smtp.client-ip=40.107.244.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="YcHfZLMY" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aa84oWmE0prvwfwqi5egm++1LlAB52WXhuokAtX9PnX+1Vta86mLdHQEaxanuFv+Ne/YeeQaDrHuCw85h+GCx8uSfQqgmzLu9aiFaKxwcPSQOt/pgX2jLjHFrNGpOMKC2MfA4WoPPDd8Cx1wahNU1Uxa/RAyNG7WQ+FZy2xnGC+3sQ+iP4oYc9cPclIIq1xKkrv9OFodEspcibRMcqTGE8knidqB9Hv9Ddga63OOnZctbKHzRz3LAhgGOGN4pfMghMPZnWefATqVBS/olsIoYpv2ozIKH1wn2lCRrYBUdX3aTFqcIEKxtOYQtmC+tcFgELQSbBaaWp9T72tsRepNOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j2OqhVHk8El4CAjcA45dc2j34faTtRc0H2tjll4NbU0=; b=N839+XdMXA0Wp27HZcBPA6xdzqVrgBrSkboCu59J9LqpbkkshfNShjntGxvqzDfPrJ0ieCgdX+HglvjvwU/ZzafPhbXRdSjFGs+AT36cpcjdfETHcUHZcUYgBJaJEovZQ1dWxfKOwexyt/ZwKCk6qRGbjFYegrbNcX3rE1aJhYzluBtFUmyvlRm+j7r6Gk2eRjERsCfqTxKsxuHJq3uaclXZYSbEkr04RknDjhrZMEErOG+Q3x3Ych6yohLcCEvVQ1vZYyrc78Sbg8WEJlLtTIWXwkXArCvtW/WloHusF792OkKZN8AmspUMxs5tcX4qXgMzP//q6bSJOdceTRvGsA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j2OqhVHk8El4CAjcA45dc2j34faTtRc0H2tjll4NbU0=; b=YcHfZLMYYNtdd+41wp1Bv5YcN1A54kkxBh544PTCINhYrm6H1do14SVVmS7XtJB2xi/0swdbv/VRIXPlY8yPar2lypwhvjUvhbCmEYI/txRDw5dwQ7T4DpIR8IYHbi2nr7o7prQ5OH8ZvWsTm5HJEMIr2OburuR1BYiq7+e1EWZRZy03qHzHLloiK2YX1nqc8PN4Zme0PmzU2VyNuf6NqafCO3xiZiV239pFGpFY3xdMecWLcrP2Eg0kLbtTm+6wS8ttgtcTzZt5Ho2P5wPMzMfu7Qu4csZDSdpK6OIYmluH/fcCSffSeeyUyd2hlTEMOhOw7iT8CAYiSGtgrA7Ktg== Received: from BYAPR01CA0063.prod.exchangelabs.com (2603:10b6:a03:94::40) by DM6PR12MB4073.namprd12.prod.outlook.com (2603:10b6:5:217::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.18; Wed, 21 Aug 2024 05:10:39 +0000 Received: from SJ5PEPF000001E9.namprd05.prod.outlook.com (2603:10b6:a03:94:cafe::5d) by BYAPR01CA0063.outlook.office365.com (2603:10b6:a03:94::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.21 via Frontend Transport; Wed, 21 Aug 2024 05:10:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ5PEPF000001E9.mail.protection.outlook.com (10.167.242.197) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.11 via Frontend Transport; Wed, 21 Aug 2024 05:10:39 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:29 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:29 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 20 Aug 2024 22:10:26 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next 2/7] RDMA/mlx5: Obtain upper net device only when needed Date: Wed, 21 Aug 2024 08:10:12 +0300 Message-ID: <20240821051017.7730-3-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240821051017.7730-1-michaelgur@nvidia.com> References: <20240821051017.7730-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001E9:EE_|DM6PR12MB4073:EE_ X-MS-Office365-Filtering-Correlation-Id: 00802951-2f4e-4eae-36f6-08dcc19f9919 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|376014|1800799024; X-Microsoft-Antispam-Message-Info: y1xV2ue2AdUXW+pR3QOjI431SsxK3JnHndtwsW6gcYUwpdCTiEw90fU1WO6RbsLo6Zt9Z8TKYWY9sWZRgwRo66z2hjnOlLkVoDDplBDj5IFJYqnXwTc+9PAeatKTgrdKvfFjFe+oAF5RzPu0LzoQA3LNTAa9TL98UGMwc0wsHCf5vzrlvnKplZS20SKD78BsrtSI2K4DRuiy4hUPharL6uo+S3recnAfp7NafePKVGq4Ymkv53X1ke0V29VJg6KdfBjyg5UmaGpqyqnUDXNVk+LvN0Wtg7/F7Q+AxkiMjnRiek3nOABb3KWRJZutlJJXvbBeaAnvbeF1rB9Hd1PzAbU49BWk0mQXdLWyLfIwUjxkNvf0tM/5b46U3hRSr8AVucs7uPlucB4LvyRKp9oz+Pn//O0bYJgwxKQT7SgPNomSQrV4SYC7MWjFhXy0SNvmy0x39IYW5tuJvHidnGCpMDIIlheLIBXPQFYtP9eznXH10Tn56phKSrlm65lADNHnBWyyoR4pGPPDWJ37VWsTDSmZEoXIe/SNxtfCv5FUGig76TgiFAT7cEGl8idnrXB7KNPCN7+qDUHRVfzoT3RwomP4+1EMbtOHsx26SenSW2ZbRjPh7LXqrhnvH9P5QwMLjOm4TVLWpEALcdvTgf/iRVxrDSSwfeDQ2+bs2o4g6QMzLIKOvfIfc3KoxnZ6UXZPlLp1chBOuU2h628R/jMbx5nk9c2X/eeIMaBXdtI67S7PdbCxCNx3L/ubPIJo0DtAfRAt285WoLoOU/bLUEstP/Wr2eWrtdshcgrjD/seizTcNYXJHf1HpV7RFCF/mB9suSVUFZLTPN68H7hortvcYpjh3oMKXiYWYhGf4254bRuxGyRYz3S0lQ7zQvQf3ivALhAcxMzQtk7J0Ed4O0oT932lB31deyEQZxrD9mV4ddcU1tnSqRCiK9qsCtTjYI5A3BEst4RO5hi8J1mJtUx0mHUqsuzJzvFaqAT13PqOWyqhnvi7ArZmm8aKbJ4Sd0E+C5duQ+1fjEYrC3oFsYh9AnCdEVRATihVD0giGxJMTS7dcdgzN3XEv3GNalz5S7VOCK4K5DU94blKZcMZxpJSrwJBRxMgh6HtY76dIDxCgngXsnlGMMGZG7cHVfsuDbzCZnjEOlr0UA0PsG/TOoskFI2fDGM+KDNgE0N3y+B4JMiL034hfzogPjLJuz25qadVINaNwBzvT/QqGpc010FoLGCIgpQ/WKa2xc2MW85KtPk3z2JSL07VB3VfzCtsBwkAyckTqAYNI+cJcICjjK+WDtgj+gr/qYPWnkIyrSHABhC7JKqJIjMc5PrxogvB6o6VKXvIC2tVLPXyMCAtgayYstAJZPze2epV9uEhdgOtghyWNRA8rBh+D49phAzp/SzI4oLT5MNXUVTl2vNywi1NGeCwljfm3hLWxsUJ8Sp7UYuSOmRj6KJ9u8dWRMrYOqHb X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(376014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 05:10:39.1982 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 00802951-2f4e-4eae-36f6-08dcc19f9919 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001E9.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4073 From: Mark Bloch Report the upper device's state as the RDMA port state only in RoCE LAG or switchdev LAG. Fixes: 27f9e0ccb6da ("net/mlx5: Lag, Add single RDMA device in multiport mode") Signed-off-by: Mark Bloch Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index cdf1ce0f6b34..c75cc3d14e74 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -558,7 +558,7 @@ static int mlx5_query_port_roce(struct ib_device *device, u32 port_num, if (!ndev) goto out; - if (dev->lag_active) { + if (mlx5_lag_is_roce(mdev) || mlx5_lag_is_sriov(mdev)) { rcu_read_lock(); upper = netdev_master_upper_dev_get_rcu(ndev); if (upper) { From patchwork Wed Aug 21 05:10:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13770785 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2080.outbound.protection.outlook.com [40.107.223.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA6162599 for ; Wed, 21 Aug 2024 05:10:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.80 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217046; cv=fail; b=QR10rdvDCla3jqJwYXik9893KEO3YmTJUcCajDvDSV2kLl06DyMsArkDtSrH6N7QcTwp8TMoVGEYNNG/3/SmVbMegUEFQruZM+Q+pal0YOi3uBdIX8CPQTTVS2zh8uSPaOF8kU5SJ3yfg0SHG/tA6Oa5tN+NJhfIzC+QzUZedx4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217046; c=relaxed/simple; bh=Je5C3f+CcZSAmB0AMT9meb69fQe7s/01w9SvWGMooE8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=A8QaCqOk5NVpVsiVTAKh56IIEIFYVVZwWrgKLaHtoiZvmP4WX+3KdKFEQqXwzm1gPSIKqSZFdC3GHklPUZdpLNAhn0+tlbHiW4TpF68sZJ6XjkSeoYlRfI9UWrAR9cjTq0qr5Oyrc1ZgY1pfCSkBVWLrg5eMdLpacv+7+EGDFPY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=IDe1dEFx; arc=fail smtp.client-ip=40.107.223.80 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="IDe1dEFx" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NzIboq57UxbcoOWPRR/tSbLj9iMjuC1dSJAIRtHnnIBQIa7S7g4M/IKFTKudfKt836Oehh9o5Q0eyz4/WGvpNUljrDfsxUAn1nZZYmIGS/fbUrwDN+TnRRwKe5wyfTl7toB7ixCbTGQSGyjSvIrLM6iyDsPstBKtfLfMMpew8/h80GL5hOSyTF8tviV4cr5kUTUB/QHILH8cealLrNw+OXG6D0mSannBk63tqYhY3UuRXhHmarKb7A06ZpNrK5KnNZcfmpyecSyJ46bOq4CxfAHCqGWRmW5xUGvU9m2bc+tYiFuhMiUX63qcYki8SSEKUiQyyC+dfU/y7+Jbj/Q7pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j3X8e2q5H7HzbLMNGEcnFtC+Vm0ucJAlldStCRceoaU=; b=Cq8LRQNmEUhifd2H4fNwsg3AE4+lKFYKdlm1fz1cnxeFUTOOivzlI3fZQAh7VePCHe6tz5O9zgzWKA1nHJ5mVgLd9+nHLpMP+t2mNapq1Bgn9/1xYWJu08n8aETjpIUVzgSbxECYhgeZttSwEzp2xn6Zj/JdJk6ijeM8EI/datC2WiIzFeq/fMK+z8E68BzkWRujMdx372t0Bi9cmlkXT4Op+paHcfhx6fP2x29S9oVWLSV4rx7OXsT2jz/Y7SLP45sk+tR5Sjhlq26IMY4BirN5Oyr8QQj853G/xuBC3muvZ27W0v05hSMlzoiRmRwlWax21NqiuSrH7/Jkj8w3Sg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j3X8e2q5H7HzbLMNGEcnFtC+Vm0ucJAlldStCRceoaU=; b=IDe1dEFxYw7qL1VooI/vYCLGT9lT+UWntQV4uubPtS3uo73zn8I7QNC38MyrDlo4iIS9k3wyRZDenOVGvZfgicQ4JUvxGrC5MzcSQp4Sh6raMsSRv3mrbr1eIlEunwxpdRt5Bgeh/NbS9iTXgOuq3xa1QHQkC2KKxx510nZx478sROa3mpxAgdq7ZjYVz4vPsaDYDI1tnnO4uLN8UUnA3aceOjCMYWFhP0SDpy8LsWXu2dgPAHX10zHHNI2Fk0f0nJQ+L+bLDJ5Us8dWM5d8LSx3ItU7yzuQ9WHv8rf1bGN4OGqMzfNfssU/t0uyXqZ4R2BdLnwyNyuQZ5EcJs0aZg== Received: from SJ0PR03CA0210.namprd03.prod.outlook.com (2603:10b6:a03:2ef::35) by DS0PR12MB8501.namprd12.prod.outlook.com (2603:10b6:8:15d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.20; Wed, 21 Aug 2024 05:10:41 +0000 Received: from SJ5PEPF000001ED.namprd05.prod.outlook.com (2603:10b6:a03:2ef:cafe::1c) by SJ0PR03CA0210.outlook.office365.com (2603:10b6:a03:2ef::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.21 via Frontend Transport; Wed, 21 Aug 2024 05:10:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ5PEPF000001ED.mail.protection.outlook.com (10.167.242.201) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.11 via Frontend Transport; Wed, 21 Aug 2024 05:10:41 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:32 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:31 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 20 Aug 2024 22:10:29 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next 3/7] RDMA/mlx5: Initialize phys_port_cnt earlier in RDMA device creation Date: Wed, 21 Aug 2024 08:10:13 +0300 Message-ID: <20240821051017.7730-4-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240821051017.7730-1-michaelgur@nvidia.com> References: <20240821051017.7730-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001ED:EE_|DS0PR12MB8501:EE_ X-MS-Office365-Filtering-Correlation-Id: 44767838-1a7e-4e05-dab5-08dcc19f9a62 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: iGCRKz3CxeCGptqCMgv2S+cAh8CSroocf+5F0EkuBb97eU7bgOFImrK4JmLUnbXHFRSU6nIe2XjnzkWNENE+7MxahdsMVVXOwLdi0HPV1cgWhcUdkD3aagS7//5P3iJ/lJqB667wnHaz+RZMN1ZYRTs0mT7CXPDSiW4LfdPnzmraAcrk5UKF+U/ViHRtpNCI3oUujXw5O50ZohvMw0oeG0qYjZ2i1I+42pnXMu+g7avMBoB0evs2V1D2ZUvFQx6q6RFVqPGrhCFc/30poaxNd5IArFIbENJ8/pP/q+aQHAWE1WGWZRykjlWyhBDx6ClEgZMrAlhvNLapdv1XJEVUe7y3QWlqx3+iqdhVHH8yC3AKox5rGD/rExn8aPf76WZZQYMpb+9dFF7sMLpSyW2CDJOzxmZOmBYUFtKm5X0cr+Fcnotr9dCIGz+ycwTVwXDNFV2AJLD1QtZE/X77gnggcRmmVrFF2eEyeunelh6gIgBWhXKKiKPLDuVtC3Il2Tqz40e26ltnXq9Z4dQkjwZj3Anr8Xfae8DRCc2mWKvAY67LK5unAu0qbLT6Za/3XvTREYEDrHdrSw5gc4D5QKb5ECSRiLmp77imrXg0a6kU/bMknvx520YgDfQCxKF36FP8iFxYc8TspHbFG3vBqHSoYNZsV1fpUuoWTgn1/ROlQXD5GTwBe5vZlv81huVMKnd04+/+xze+XsqDG9ucZn57Ghbd8ahDX16SaPxX/xVJxEjc89U2Lny0rCA/KWubidi+AThuw9vyB/kiL6XGa3cBzSxpgfVOw4hbRp2paETMaVJ4gzJMxqpRw7g/4i2kq9VhZRqTz7Q+VgaiZ09HAcM38rcEkDoDAm7LD+CiqHCDctyKdENhsxUC3uM9+gzvGG7uGAqScu5TzbUw3dfxEUuCN4wFCFZnThAptLQt5O0N8iMf4O3P75Nv7vtjehjPDX8ZRbC4EW0367OXa0aZQNgELPVCog1gr4PPpvD7BJNbyo/SlbC4UwN4vf89cX6z0G8WPZANnTT/wBbrqVNDZgtyXwFbAcA3nSrIUjrR3EJTL7VL/Vbpf8UrQL8s1mZ0Eib3Y8WnJ3acbOS9uYVUdurKb8DWilvujKm5n0n0jlroaYctcgCUO23lvFFrv5rxcMxn439rDpIYFMz3hjhwoOpgqHQvTgyMK8l2ikZ17GSinGgAPYZ/DT1fl0necf8OZUhr+uPI4vuQawXoKJyiSDQmAZkIvio6XEHRb6tbqSMpplzBmaz60VDZfDK4AJ8Pm8e+AHHyZU/VniZT4qMAlYay/rWgQzKmEgCZF6GqnWNBvv4dqWHv/t+0NeTfjOgcw5eoRw1RPt4GjMEz0IMbRcPJmP0EPXiI5D7a9qtlrHS0RqC00TVtgyGE+vy17uM+jT01U/EcuDdTDGBGrXyXpovHEVITPlp0FpUkj7F0hN6EIYq22+zvAZXNhJHSNv2wDWlC X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(1800799024)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 05:10:41.3585 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 44767838-1a7e-4e05-dab5-08dcc19f9a62 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001ED.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8501 From: Chiara Meiohas phys_port_cnt of the IB device must be initialized before calling ib_device_set_netdev(). Previously, phys_port_cnt was initialized in the mlx5_ib init function. Remove this initialization to allow setting it separately, providing the flexibility to call ib_device_set_netdev before registering the IB device. Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/ib_rep.c | 1 + drivers/infiniband/hw/mlx5/main.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c index c7a4ee896121..1ad934685d80 100644 --- a/drivers/infiniband/hw/mlx5/ib_rep.c +++ b/drivers/infiniband/hw/mlx5/ib_rep.c @@ -104,6 +104,7 @@ mlx5_ib_vport_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep) ibdev->is_rep = true; vport_index = rep->vport_index; ibdev->port[vport_index].rep = rep; + ibdev->ib_dev.phys_port_cnt = num_ports; ibdev->port[vport_index].roce.netdev = mlx5_ib_get_rep_netdev(lag_master->priv.eswitch, rep->vport); ibdev->mdev = lag_master; diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index c75cc3d14e74..1046c92212c7 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -3932,7 +3932,6 @@ static int mlx5_ib_stage_init_init(struct mlx5_ib_dev *dev) dev->ib_dev.node_type = RDMA_NODE_IB_CA; dev->ib_dev.local_dma_lkey = 0 /* not supported for now */; - dev->ib_dev.phys_port_cnt = dev->num_ports; dev->ib_dev.dev.parent = mdev->device; dev->ib_dev.lag_flags = RDMA_LAG_FLAGS_HASH_ALL_SLAVES; @@ -4647,6 +4646,7 @@ static struct ib_device *mlx5_ib_add_sub_dev(struct ib_device *parent, mplane->mdev = mparent->mdev; mplane->num_ports = mparent->num_plane; mplane->sub_dev_name = name; + mplane->ib_dev.phys_port_cnt = mplane->num_ports; ret = __mlx5_ib_add(mplane, &plane_profile); if (ret) @@ -4763,6 +4763,7 @@ static int mlx5r_probe(struct auxiliary_device *adev, dev->mdev = mdev; dev->num_ports = num_ports; + dev->ib_dev.phys_port_cnt = num_ports; if (ll == IB_LINK_LAYER_ETHERNET && !mlx5_get_roce_state(mdev)) profile = &raw_eth_profile; From patchwork Wed Aug 21 05:10:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13770788 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2067.outbound.protection.outlook.com [40.107.237.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F6C414B978 for ; Wed, 21 Aug 2024 05:10:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.67 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217056; cv=fail; b=cw/6g+wxhLefl8590Q5ZjeMpJEEddtrEKsI/lARrHAUEaiNNEizws1JNhRy6+0h5M5mV2Eh4qyffKXqz1PcbweP1rGR8zfYjVsEdbdALx/k0zifruMEGcQTq2/ofZ2oE3YeTk8St3B23ShBoXry/Vb7FK1QDoKe6bdS3HhfJRzk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217056; c=relaxed/simple; bh=3EonbINpHw2XGDVE+TI5/ERA3RnSH3/FbcxSU3hiL2Q=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=XLzMiIfO1SPrzJpSciqsA3BgrDJeEYCykXkXZmH7X58Zl/nN968YTFpW01vCFMEF8aj4DSdAivoqnpf4WSjq9NmHIqSf6SpauukuhkYVQTAWpa4f0ajnWfpsa90A/iuEqLTgWKN+rT7VSbGMeWxZn2YbyjK1w/inn9OqytOzy1E= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=jGYTX5S1; arc=fail smtp.client-ip=40.107.237.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="jGYTX5S1" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NqizZJSHCp4fQQSY28R7nt77WA5cYowCTxK9ecMAsF2GPFBJx+lwzaVnmpMd4CjGOgrllsm6/Qll1ulSIBp/bakx6rXrjAxgkHWI0vbtnKrBv6KHRhdsZj8pn7tNegKrjAA7oAq3kKnfQSqpZHOtJeb/0I9MpmJ31erhYAqNn4ivZ+HP2FLO+Ruhgj27xmStf5aLJsCzmLXFAZC/fDDjZo8Prn2U5bWu3kfBfxQUQfk+z7b5C5IW3nVVdy3SDRKZdwmKtfx1TmxrhYSK8TtyFOBrBS4uubLFXJNLMxwcJpQjDd+cmIKw0qxz1xkESqvJrUiMTJzgHnDYiGZibWd9uw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mxxWRhW6Lp/65L8b8OkeuZm1fxOpeKuu2U7x2wYvwuo=; b=v3Y8JhxkC5r2NWUeim25VerrsJM4SnxasIwssWFcvsBIBcwDE+4yBsAUdBFi3i979dADyeQ88Lne1RvAj6D2qxOKVsseU1od0u1Jh874mzVDVmKSqkXziCcF2gokicGc/oizGBX6BxNT33rMFSELjdztWZ2Cp2AAlzEtR7diGnR8TznuqjMIlQ7Qa77sYrnv/3lDH4SM4VjxnNlK5yLNuxV/A4uG0ahbsPeYvd0qtD2ZJdqvkyQZEB2VqSoOVfurH5/o0Tr/koVCHw2S29/l5CY3ZYhL/E5eji191+ftwTISGQJWkGWKxAFYe5XCSDVkuPbzJUR0PPFE1TKh+BmMQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mxxWRhW6Lp/65L8b8OkeuZm1fxOpeKuu2U7x2wYvwuo=; b=jGYTX5S1aC+0fQfdV3utTphJYE+0rIeprnIz05fqujzRVuPCXkdugBUrViN9dJC7oasEvsu2WDkRl4dyKIy/F+eh3nWRd7vog/eHZciLb7VE5JYk+1Rgf4t2WGEhfnOYVlBajS0L77JV+nsLwARImr7cowNmGkTj5OZfzf3QFb/mjhMAITrRLdU8YzQVhmtMj2XGwVgQ1Isy2xlTo2Z142+HcWjue1LpJwETUNFOxG7mKMc7wP6QH+fuk+vBc+xF7Fz9zam41RiVn1basU3pp6ywpJDFMjl/4/k4bV6KejdZ8MsnO0oJGIinfhHIvyps7FukqGK7tlzWXNFXk2uFdg== Received: from PH7PR10CA0016.namprd10.prod.outlook.com (2603:10b6:510:23d::8) by LV3PR12MB9118.namprd12.prod.outlook.com (2603:10b6:408:1a1::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.16; Wed, 21 Aug 2024 05:10:49 +0000 Received: from CY4PEPF0000FCBE.namprd03.prod.outlook.com (2603:10b6:510:23d:cafe::64) by PH7PR10CA0016.outlook.office365.com (2603:10b6:510:23d::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.18 via Frontend Transport; Wed, 21 Aug 2024 05:10:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CY4PEPF0000FCBE.mail.protection.outlook.com (10.167.242.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.11 via Frontend Transport; Wed, 21 Aug 2024 05:10:48 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:34 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:34 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 20 Aug 2024 22:10:32 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next 4/7] RDMA/device: Clear netdev mapping in ib_device_get_netdev on unregistration Date: Wed, 21 Aug 2024 08:10:14 +0300 Message-ID: <20240821051017.7730-5-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240821051017.7730-1-michaelgur@nvidia.com> References: <20240821051017.7730-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCBE:EE_|LV3PR12MB9118:EE_ X-MS-Office365-Filtering-Correlation-Id: 6510d2c7-320e-4d2f-9c4a-08dcc19f9ee5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|82310400026|376014; X-Microsoft-Antispam-Message-Info: iSVUUpkPg7jm4o1riUX8vX28qCMvjwDhdvo99QDc+xXTJEeGYMp3muANKSiQ8ZndwmiERDc+dsHr/GMijv4GX0wVyMXk34ySDLVLct6EP9UFbBtDDo9mBtbuYASW48AbBPMPgtzilYoT6OTJ39kn4mxea8f5i7rEOtbqYDBBX97kkctBDn/4EUSWvNivw+AGcQ1i8H+U5w8lWxKOeQNbapSh3rJt+RJf0wFnMqojwBqmEPYSo57mWpw+9+YcQRQwujOE7f8QZFUisggu9zai7RzWTrtMaEQC6Wei+4T7oojb0nJHxEbrRSKIcf1fQSihjNrTbILNRLrLXaK1CiskX30sFAh0mp8l53qZ0i5h29LNGVyJeY6Qdp3nyuwzSchI4MRivp2kpyxdCyEeJV+rfTDIlrSHC+Ufc/avFn4tDauJ6mI2w4ILy3xTZSZp2zoqDY5botpg63hCM3a9RAzKzO3tBISTSzE73HQg6J9p2BXNF02qwyj7V+pFWMBysQLEHhe8BTXULdhqXXW6F5Y0kN4U3P9pIp/wtw3l+rnGo32iyFODZn5yLJcHUgbiitr7/5ip+3qvrKZTubilbo14nOJzQFP6/XYcEv0/V7eTPUoYsw2mgw0tBrMknW6EJoCh5MxWIIXm4q4W3sGOlaL5B2UeL9Zhg/z1XUMHi/IKL7tnujOiFe2JMEaQwGSuQ9suE+iEVYAMN9gl0hFeNlKxBIs9x87ckU7jvpgV+IwcVO8rCwK+W/cqt8yFh3gqquDuznaRwLB0QKSxnmOKVGgV5Gx4xX3DDM/vOX2fPHIx2zvcUa0r/cBYBrYP9F1woc+dBmixqYPwmaAMRDb5R/AfNgO/2UfZq5d4pyGp2URBX0No/GL5qUrGHL1PlBEBo8icdOai+myuCXJbxwpRGgMKpmpvBhaV2QM9ygFKoAgn3OwXgGlHndHfAcP4tAAywaOuV+0NBGqnB0vDTYDE3kjA2mYFScZsF7hn8yT9x1ykkW99A1pUOt4kV1xnTFPf/SOyhW5evm2rG+a9Rt2rG8ljdPlkW2RFZWAxh6JqtN0QrSrJXhIFY+BexPfPCUFskNUkO6l+PS4ejJiDgxnCZpSLmVcUuPul6FhxckQ3x+d1/3k7I5JrjHb7SgZcrQk8hdS2tdSa4t0kjruULMoKVHSp3yaHKTXHaX9kkXNLC5t4RmUqlQcQWoxa2LiiGNJ7nySML7VdsvuQAFprZA7/KoXf3rOBeJfI0xF3vsu8MdPy/uTNNMx0J3uAszbPyXM2uKaRDHeHMPMtkYSyXY5mjwKlYo1NJC7i0itkUGdWOUaTLxkq5/nPTW/Q9YWDNiVP7bcqbQM4UTDV4KB1P7gNpiJE8PN0zJf71+z8fM+u9obA+DzmsTxk7ZrTL4Un9PJ4ctP3fLvcz3upJyIxT8DZWZNBuTrjXcvxBe6h//LIV8mQVzznSw+60bQxwFX4pA7IFa2m X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(82310400026)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 05:10:48.8482 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6510d2c7-320e-4d2f-9c4a-08dcc19f9ee5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000FCBE.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9118 From: Maher Sanalla The caller of ib_device_get_netdev() relies on its result to accurately match a given netdev with the ib device associated netdev. ib_device_get_netdev returns NULL when the IB device associated netdev is unregistering, preventing the caller of matching netdevs properly. Thus, revise ib_device_get_netdev to assign NULL to netdev when the netdev is undergoing unregistration, allowing matching by the caller. This change ensures proper netdev matching and reference count handling by the caller of ib_device_get_netdev/ib_device_set_netdev API. Fixes: c2261dd76b54 ("RDMA/device: Add ib_device_set_netdev() as an alternative to get_netdev" Signed-off-by: Maher Sanalla Reviewed-by: Mark Bloch Signed-off-by: Michael Guralnik --- drivers/infiniband/core/device.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 0290aca18d26..583047457de2 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -2257,6 +2257,7 @@ struct net_device *ib_device_get_netdev(struct ib_device *ib_dev, * propagation of an unregistering netdev. */ if (res && res->reg_state != NETREG_REGISTERED) { + ib_device_set_netdev(ib_dev, NULL, port); dev_put(res); return NULL; } From patchwork Wed Aug 21 05:10:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13770787 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2053.outbound.protection.outlook.com [40.107.243.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9BB54D8D1 for ; Wed, 21 Aug 2024 05:10:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.53 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217054; cv=fail; b=b75Uz1ROzFf2CrLi/9ohl2XavIL3LYjo2xi6I2PpRC71xR4zGqxm2zIZUmVLa2PQO4Ps+DYaBFcjkZpg92tv1g3oBMDPf0V7/C7CV7mJ/utNRp4Ym95zEeyFjVgj22DsglhC6t/YnBynidIyBuXHaRfg0i4xRUE4d9yzmOS+1eo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217054; c=relaxed/simple; bh=sJvEjVWr96soQmqJB+pINcTQmezBlixxFYQGq4CPeBg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MMv0TjZUWutk+wImpZTee88nKdu2x4l1AFfC3Jx1HR7cR+khI8L2FC2Poz35Ey/02cjvGJkx0CT8mOK7duTu+Bkla5DB5vHSwZFGZb9BFsx2oGKtoDs/JkgtbIFnxpqHRMz7O6LL+kTYmf9OYPWctJWEqkc8M/JJETIqRXDDUsw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=NDjg6RXL; arc=fail smtp.client-ip=40.107.243.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="NDjg6RXL" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=d25PAv2VWz4uD8p5+IhQEbpcUENo/os+XaH9FiNeW+WubnVowfqFBzSoHxnP0CLLvr9BAd//vPE5980oTX+3wCXI4dvdHb7X9p9pI7lyJDEjCrQoHr0eU14gp4MDFNRdkgF92123I2ytF2ry3tnea0SeXirwHofdtlBrXdsXgSe9k37FZ8jGV+l9/8/AbNc2PN6OWOUEZL5ixgazFMG65ol5oCOkb5otU2Is70ZTL4Xvnc3UXZoIq0Bozy02kQ+VGuVP23zdfKqkjLZXCG/JN2WPKm3Prx05UivASDCM1jIOuQ8a1TQRlXWFGmcHDw0S2RF47BrkbqnC2HfO5YFBsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9kxupg9sG3YStzvtgXxg9YfLaC+Gqe5PLx1kPAhSnxo=; b=wWT4yy7IyCXxARuwZJTJFGHYRqlpe8gqawc9nhHvt1uIpX8sPQwcmKuYlyEjA+4csxAzv9y8B9rsnuBSTDkbXAK6CqgXkWHRkUlp9KGUrwUHiL3x15b8Ip5RHprmMm3BIa0z7BHpEW5rvahH/BKDtfjZwmloA5Jw0HLGQLe5bcJIr7cjfhQw4iuKnMMRLw7eVC7FlIKMbUjqlRixpXOHFnwG2uMEF0+X/biff5PA+pGT7EUlquppgrY8vOXfh6347MiSURAZq0siOSxftKpwkDSVzXZ5UhjJBy+hNTHiXwh97GGHAK+RwEDwotYIuuoa2+Z/E/6xRwpSLnS09OySfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9kxupg9sG3YStzvtgXxg9YfLaC+Gqe5PLx1kPAhSnxo=; b=NDjg6RXLGUFjLTUQKB3kXgXC/L/Ec7p0JFzcdHD/82okNLFNd1LKwcFV7S9+9ME+AN6ZbPg/S+YInhpydde5W/D7671T7HoSTbPJPH5jk8XeVeP+/opvOFN0QZg3KK2LltmAkSuuhY532VMeLIUUtKHnBEsGNm6UIEeIrXsmHwTD1+oJYINoSGLO5q9JVWYV4ivCPFJFPFoOVwll3Sc91fpNcvkHt5oi4c01LoCt6C2SB4EuHXIIthix14YqEP18X4WB/XBUBRUbm7yKNTyAXalXLy8JSgLU8RwldzZy239y2co4YqXjsZS9xojhKgICXE1rsupZVMifuitboRpujw== Received: from SJ0PR03CA0209.namprd03.prod.outlook.com (2603:10b6:a03:2ef::34) by MW5PR12MB5682.namprd12.prod.outlook.com (2603:10b6:303:19f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.18; Wed, 21 Aug 2024 05:10:46 +0000 Received: from SJ5PEPF000001ED.namprd05.prod.outlook.com (2603:10b6:a03:2ef:cafe::e0) by SJ0PR03CA0209.outlook.office365.com (2603:10b6:a03:2ef::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.21 via Frontend Transport; Wed, 21 Aug 2024 05:10:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ5PEPF000001ED.mail.protection.outlook.com (10.167.242.201) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.11 via Frontend Transport; Wed, 21 Aug 2024 05:10:46 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:37 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:37 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 20 Aug 2024 22:10:35 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next 5/7] RDMA/mlx5: Use IB set_netdev and get_netdev functions Date: Wed, 21 Aug 2024 08:10:15 +0300 Message-ID: <20240821051017.7730-6-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240821051017.7730-1-michaelgur@nvidia.com> References: <20240821051017.7730-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001ED:EE_|MW5PR12MB5682:EE_ X-MS-Office365-Filtering-Correlation-Id: 326c5a9e-62d0-4f70-64e6-08dcc19f9d99 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: HNhDxcoIhOq1PLKLarnUgHZsT3dRbG0AkTLpJKupd2mnaHyVK3V59M0LR3VQF5nD1sNLfWXM02H0LlRrj96faz+pxocqYJRw2crtY1JAAwl7JPeIug5AbRWgIMPhbaxsyooWmT1VjCMe6EYOCQDoOii1pqR+SYetLiVQctAtID1Zwbe4tFKo8qc1XHYfyYvqKISR6XDWVD1UhAl8mEkVHSnmgVROZ3CriQV9cHhh3ZME1kOMZZSL0lEj+hJ1GXBh/NAouXYH//09XrT8/T98gJcopcR3BIgJSN8VH7wy2PIT104L1zrCZJPMHJGczJvIZUESdE1UgRp/eSeOyoItQ1liJClod1NgO0o2F55vlOpWvrgk31DdzLdDUbvOTOIXvm7+V7sfeTsdY+/A0zYasloCJk3kIvtqtxmpAXX/yzv8jeHIzMoonQo79KXRbcUvBiOcbNLTMuw6ih/1fEQHurCguiF8cvYSei4x5cZfuqHU1Uj9qoMaU8JQOlCoOW0t83dOA+9ZGx4mqi0j46NLKmpLnoPBp2QEdng4d04+1lXd1FGjp1w6xbUuPf5TZ7gR82+btHoN4fDWvGRNRYzmu078caFuU8kt0wamOI8NM3hTSOL3TPE1dXuADRLYSTwXaJmxuN2a1PvEEpClB7LtYRYdjRiVXd7BtLAv8Pj8L123CG2Ez0eUjKflOoipTVLJYFKVPIztCXTn4dqDOd6JXr37KEW3vUV2M+5mmBtUt33znoMtoHD4i0nrLFeKKr4ELrneBt5LqzDx1BRUYeJxHBUBYgsyNdvWY7oByQeIjAEOkcw7wiTNX6Qk4QWPEz69TipP2WgjKzdPNuSobG8VIsyQxcrAGxRx/tEuEA/kz+gBHMu4mKwpef34t8Kyrx5ZE1hqjmpmRatxBvl+PeUTr9MuGnLjs8mMcaSic/q2HCr+QWUOJM1CQyMfSP49JCv4xSrsE50bLnzY7I2d1mbT4wHxAkNjy5hlTvaL3VMcEoRFe/01BHnn5NXhLSDNHQFk/dfhfIHeURzPyuZpvpu4dHVd4YKVyIRXzc0YcSVI7L3lQzsGqfCmFPCITMvlkoUJ7g3hggIhvEA6G2QOMjpR8HwnnonVcyhUolevrwlvLZUkN1a0IOIMbhQsE8DLYbq4aYMd2RlCUzVtcvyVW2Uux9WsAL+Bh11JDPzQeQuvLMwNKRFoTnBrWUeae67jcbgumGok9bwl09QyKML+3qLsjgZIsTUndO6q8bJO8ynS5t0HfJfm718uRK0SCmexzo1PDSXQuOujxLqg3FCbBnCydr53DWVuFVOQ9HW5jQxO5kpd8kPBCdHgpbehEwFg9dIiu2k7QnJ5uw3MaxOjKv2yW8QH0LPle7x+qi0sFFOLmkVqzhTP4p4GSA1iZGiVB5JKsglacRWzNpeh9/ZuT7PexTgTRXXVK8DuInRGoRHThsAdlXCt3krp776BpM4nnnAH X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 05:10:46.7491 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 326c5a9e-62d0-4f70-64e6-08dcc19f9d99 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001ED.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW5PR12MB5682 From: Chiara Meiohas The IB layer provides a common interface to store and get net devices associated to an IB device port (ib_device_set_netdev() and ib_device_get_netdev()). Previously, mlx5_ib stored and managed the associated net devices internally. Replace internal net device management in mlx5_ib with ib_device_set_netdev() when attaching/detaching a net device and ib_device_get_netdev() when retrieving the net device. Export ib_device_get_netdev(). For mlx5 representors/PFs/VFs and lag creation we replace the netdev assignments with the IB set/get netdev functions. In active-backup mode lag the active slave net device is stored in the lag itself. To assure the net device stored in a lag bond IB device is the active slave we implement the following: - mlx5_core: when modifying the slave of a bond we send the internal driver event MLX5_DRIVER_EVENT_ACTIVE_BACKUP_LAG_CHANGE_LOWERSTATE. - mlx5_ib: when catching the event call ib_device_set_netdev() This patch also ensures the correct IB events are sent in switchdev lag. While at it, when in multiport eswitch mode, only a single IB device is created for all ports. The said IB device will receive all netdev events of its VFs once loaded, thus to avoid overwriting the mapping of PF IB device to PF netdev, ignore NETDEV_REGISTER events if the ib device has already been mapped to a netdev. Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/core/device.c | 4 + drivers/infiniband/hw/mlx5/ib_rep.c | 23 +-- drivers/infiniband/hw/mlx5/main.c | 183 ++++++++++++------ drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 +- .../net/ethernet/mellanox/mlx5/core/lag/lag.c | 76 ++++---- include/linux/mlx5/device.h | 1 + include/linux/mlx5/driver.h | 2 +- include/rdma/ib_verbs.h | 2 + 8 files changed, 191 insertions(+), 103 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 583047457de2..b2fc5a13577c 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -2236,6 +2236,9 @@ struct net_device *ib_device_get_netdev(struct ib_device *ib_dev, if (!rdma_is_port_valid(ib_dev, port)) return NULL; + if (!ib_dev->port_data) + return NULL; + pdata = &ib_dev->port_data[port]; /* @@ -2264,6 +2267,7 @@ struct net_device *ib_device_get_netdev(struct ib_device *ib_dev, return res; } +EXPORT_SYMBOL(ib_device_get_netdev); /** * ib_device_get_by_netdev - Find an IB device associated with a netdev diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c index 1ad934685d80..49af1cfbe6d1 100644 --- a/drivers/infiniband/hw/mlx5/ib_rep.c +++ b/drivers/infiniband/hw/mlx5/ib_rep.c @@ -13,6 +13,7 @@ mlx5_ib_set_vport_rep(struct mlx5_core_dev *dev, int vport_index) { struct mlx5_ib_dev *ibdev; + struct net_device *ndev; ibdev = mlx5_eswitch_uplink_get_proto_dev(dev->priv.eswitch, REP_IB); if (!ibdev) @@ -20,12 +21,9 @@ mlx5_ib_set_vport_rep(struct mlx5_core_dev *dev, ibdev->port[vport_index].rep = rep; rep->rep_data[REP_IB].priv = ibdev; - write_lock(&ibdev->port[vport_index].roce.netdev_lock); - ibdev->port[vport_index].roce.netdev = - mlx5_ib_get_rep_netdev(rep->esw, rep->vport); - write_unlock(&ibdev->port[vport_index].roce.netdev_lock); + ndev = mlx5_ib_get_rep_netdev(rep->esw, rep->vport); - return 0; + return ib_device_set_netdev(&ibdev->ib_dev, ndev, vport_index + 1); } static void mlx5_ib_register_peer_vport_reps(struct mlx5_core_dev *mdev); @@ -104,11 +102,15 @@ mlx5_ib_vport_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep) ibdev->is_rep = true; vport_index = rep->vport_index; ibdev->port[vport_index].rep = rep; - ibdev->ib_dev.phys_port_cnt = num_ports; - ibdev->port[vport_index].roce.netdev = - mlx5_ib_get_rep_netdev(lag_master->priv.eswitch, rep->vport); ibdev->mdev = lag_master; ibdev->num_ports = num_ports; + ibdev->ib_dev.phys_port_cnt = num_ports; + ret = ib_device_set_netdev(&ibdev->ib_dev, + mlx5_ib_get_rep_netdev(lag_master->priv.eswitch, + rep->vport), + vport_index + 1); + if (ret) + goto fail_add; ret = __mlx5_ib_add(ibdev, profile); if (ret) @@ -161,9 +163,8 @@ mlx5_ib_vport_rep_unload(struct mlx5_eswitch_rep *rep) } port = &dev->port[vport_index]; - write_lock(&port->roce.netdev_lock); - port->roce.netdev = NULL; - write_unlock(&port->roce.netdev_lock); + + ib_device_set_netdev(&dev->ib_dev, NULL, vport_index + 1); rep->rep_data[REP_IB].priv = NULL; port->rep = NULL; diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index 1046c92212c7..a750f61513d4 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -147,16 +147,52 @@ static struct mlx5_roce *mlx5_get_rep_roce(struct mlx5_ib_dev *dev, if (upper && port->rep->vport == MLX5_VPORT_UPLINK) continue; - - read_lock(&port->roce.netdev_lock); - rep_ndev = mlx5_ib_get_rep_netdev(port->rep->esw, - port->rep->vport); - if (rep_ndev == ndev) { - read_unlock(&port->roce.netdev_lock); + rep_ndev = ib_device_get_netdev(&dev->ib_dev, i + 1); + if (rep_ndev && rep_ndev == ndev) { + dev_put(rep_ndev); *port_num = i + 1; return &port->roce; } - read_unlock(&port->roce.netdev_lock); + + dev_put(rep_ndev); + } + + return NULL; +} + +static bool mlx5_netdev_send_event(struct mlx5_ib_dev *dev, + struct net_device *ndev, + struct net_device *upper, + struct net_device *ib_ndev) +{ + if (!dev->ib_active) + return false; + + /* Event is about our upper device */ + if (upper == ndev) + return true; + + /* RDMA device is not in lag and not in switchdev */ + if (!dev->is_rep && !upper && ndev == ib_ndev) + return true; + + /* RDMA devie is in switchdev */ + if (dev->is_rep && ndev == ib_ndev) + return true; + + return false; +} + +static struct net_device *mlx5_ib_get_rep_uplink_netdev(struct mlx5_ib_dev *ibdev) +{ + struct mlx5_ib_port *port; + int i; + + for (i = 0; i < ibdev->num_ports; i++) { + port = &ibdev->port[i]; + if (port->rep && port->rep->vport == MLX5_VPORT_UPLINK) { + return ib_device_get_netdev(&ibdev->ib_dev, i + 1); + } } return NULL; @@ -168,6 +204,7 @@ static int mlx5_netdev_event(struct notifier_block *this, struct mlx5_roce *roce = container_of(this, struct mlx5_roce, nb); struct net_device *ndev = netdev_notifier_info_to_dev(ptr); u32 port_num = roce->native_port_num; + struct net_device *ib_ndev = NULL; struct mlx5_core_dev *mdev; struct mlx5_ib_dev *ibdev; @@ -181,29 +218,38 @@ static int mlx5_netdev_event(struct notifier_block *this, /* Should already be registered during the load */ if (ibdev->is_rep) break; - write_lock(&roce->netdev_lock); + + ib_ndev = ib_device_get_netdev(&ibdev->ib_dev, port_num); + /* Exit if already registered */ + if (ib_ndev) + goto put_ndev; + if (ndev->dev.parent == mdev->device) - roce->netdev = ndev; - write_unlock(&roce->netdev_lock); + ib_device_set_netdev(&ibdev->ib_dev, ndev, port_num); break; case NETDEV_UNREGISTER: /* In case of reps, ib device goes away before the netdevs */ - write_lock(&roce->netdev_lock); - if (roce->netdev == ndev) - roce->netdev = NULL; - write_unlock(&roce->netdev_lock); - break; + if (ibdev->is_rep) + break; + ib_ndev = ib_device_get_netdev(&ibdev->ib_dev, port_num); + if (ib_ndev == ndev) + ib_device_set_netdev(&ibdev->ib_dev, NULL, port_num); + goto put_ndev; case NETDEV_CHANGE: case NETDEV_UP: case NETDEV_DOWN: { struct net_device *upper = NULL; - if (mlx5_lag_is_roce(mdev)) { + if (mlx5_lag_is_roce(mdev) || mlx5_lag_is_sriov(mdev)) { struct net_device *lag_ndev; - lag_ndev = mlx5_lag_get_roce_netdev(mdev); + if(mlx5_lag_is_roce(mdev)) + lag_ndev = ib_device_get_netdev(&ibdev->ib_dev, 1); + else /* sriov lag */ + lag_ndev = mlx5_ib_get_rep_uplink_netdev(ibdev); + if (lag_ndev) { upper = netdev_master_upper_dev_get(lag_ndev); dev_put(lag_ndev); @@ -216,18 +262,19 @@ static int mlx5_netdev_event(struct notifier_block *this, roce = mlx5_get_rep_roce(ibdev, ndev, upper, &port_num); if (!roce) return NOTIFY_DONE; - if ((upper == ndev || - ((!upper || ibdev->is_rep) && ndev == roce->netdev)) && - ibdev->ib_active) { + + ib_ndev = ib_device_get_netdev(&ibdev->ib_dev, port_num); + + if (mlx5_netdev_send_event(ibdev, ndev, upper, ib_ndev)) { struct ib_event ibev = { }; enum ib_port_state port_state; if (get_port_state(&ibdev->ib_dev, port_num, &port_state)) - goto done; + goto put_ndev; if (roce->last_port_state == port_state) - goto done; + goto put_ndev; roce->last_port_state = port_state; ibev.device = &ibdev->ib_dev; @@ -236,7 +283,7 @@ static int mlx5_netdev_event(struct notifier_block *this, else if (port_state == IB_PORT_ACTIVE) ibev.event = IB_EVENT_PORT_ACTIVE; else - goto done; + goto put_ndev; ibev.element.port_num = port_num; ib_dispatch_event(&ibev); @@ -247,39 +294,13 @@ static int mlx5_netdev_event(struct notifier_block *this, default: break; } +put_ndev: + dev_put(ib_ndev); done: mlx5_ib_put_native_port_mdev(ibdev, port_num); return NOTIFY_DONE; } -static struct net_device *mlx5_ib_get_netdev(struct ib_device *device, - u32 port_num) -{ - struct mlx5_ib_dev *ibdev = to_mdev(device); - struct net_device *ndev; - struct mlx5_core_dev *mdev; - - mdev = mlx5_ib_get_native_port_mdev(ibdev, port_num, NULL); - if (!mdev) - return NULL; - - if (mlx5_lag_is_roce(mdev)) { - ndev = mlx5_lag_get_roce_netdev(mdev); - goto out; - } - - /* Ensure ndev does not disappear before we invoke dev_hold() - */ - read_lock(&ibdev->port[port_num - 1].roce.netdev_lock); - ndev = ibdev->port[port_num - 1].roce.netdev; - dev_hold(ndev); - read_unlock(&ibdev->port[port_num - 1].roce.netdev_lock); - -out: - mlx5_ib_put_native_port_mdev(ibdev, port_num); - return ndev; -} - struct mlx5_core_dev *mlx5_ib_get_native_port_mdev(struct mlx5_ib_dev *ibdev, u32 ib_port_num, u32 *native_port_num) @@ -554,7 +575,7 @@ static int mlx5_query_port_roce(struct ib_device *device, u32 port_num, if (!put_mdev) goto out; - ndev = mlx5_ib_get_netdev(device, port_num); + ndev = ib_device_get_netdev(device, port_num); if (!ndev) goto out; @@ -3185,6 +3206,60 @@ static void get_dev_fw_str(struct ib_device *ibdev, char *str) fw_rev_sub(dev->mdev)); } +static int lag_event(struct notifier_block *nb, unsigned long event, void *data) +{ + struct mlx5_ib_dev *dev = container_of(nb, struct mlx5_ib_dev, + lag_events); + struct mlx5_core_dev *mdev = dev->mdev; + struct mlx5_ib_port *port; + struct net_device *ndev; + int i, err; + int portnum; + + portnum = 0; + switch (event) { + case MLX5_DRIVER_EVENT_ACTIVE_BACKUP_LAG_CHANGE_LOWERSTATE: + ndev = data; + if (ndev) { + if (!mlx5_lag_is_roce(mdev)) { + // sriov lag + for (i = 0; i < dev->num_ports; i++) { + port = &dev->port[i]; + if (port->rep && port->rep->vport == + MLX5_VPORT_UPLINK) { + portnum = i; + break; + } + } + } + err = ib_device_set_netdev(&dev->ib_dev, ndev, + portnum + 1); + dev_put(ndev); + if (err) + return err; + /* Rescan gids after new netdev assignment */ + rdma_roce_rescan_device(&dev->ib_dev); + } + break; + default: + return NOTIFY_DONE; + } + return NOTIFY_OK; +} + +static void mlx5e_lag_event_register(struct mlx5_ib_dev *dev) +{ + dev->lag_events.notifier_call = lag_event; + blocking_notifier_chain_register(&dev->mdev->priv.lag_nh, + &dev->lag_events); +} + +static void mlx5e_lag_event_unregister(struct mlx5_ib_dev *dev) +{ + blocking_notifier_chain_unregister(&dev->mdev->priv.lag_nh, + &dev->lag_events); +} + static int mlx5_eth_lag_init(struct mlx5_ib_dev *dev) { struct mlx5_core_dev *mdev = dev->mdev; @@ -3206,6 +3281,7 @@ static int mlx5_eth_lag_init(struct mlx5_ib_dev *dev) goto err_destroy_vport_lag; } + mlx5e_lag_event_register(dev); dev->flow_db->lag_demux_ft = ft; dev->lag_ports = mlx5_lag_get_num_ports(mdev); dev->lag_active = true; @@ -3223,6 +3299,7 @@ static void mlx5_eth_lag_cleanup(struct mlx5_ib_dev *dev) if (dev->lag_active) { dev->lag_active = false; + mlx5e_lag_event_unregister(dev); mlx5_destroy_flow_table(dev->flow_db->lag_demux_ft); dev->flow_db->lag_demux_ft = NULL; @@ -3937,7 +4014,6 @@ static int mlx5_ib_stage_init_init(struct mlx5_ib_dev *dev) for (i = 0; i < dev->num_ports; i++) { spin_lock_init(&dev->port[i].mp.mpi_lock); - rwlock_init(&dev->port[i].roce.netdev_lock); dev->port[i].roce.dev = dev; dev->port[i].roce.native_port_num = i + 1; dev->port[i].roce.last_port_state = IB_PORT_DOWN; @@ -4202,7 +4278,6 @@ static const struct ib_device_ops mlx5_ib_dev_common_roce_ops = { .create_wq = mlx5_ib_create_wq, .destroy_rwq_ind_table = mlx5_ib_destroy_rwq_ind_table, .destroy_wq = mlx5_ib_destroy_wq, - .get_netdev = mlx5_ib_get_netdev, .modify_wq = mlx5_ib_modify_wq, INIT_RDMA_OBJ_SIZE(ib_rwq_ind_table, mlx5_ib_rwq_ind_table, diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index be83a4d91a34..fd06d0d27a72 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -896,8 +896,6 @@ struct mlx5_roce { /* Protect mlx5_ib_get_netdev from invoking dev_hold() with a NULL * netdev pointer */ - rwlock_t netdev_lock; - struct net_device *netdev; struct notifier_block nb; struct netdev_net_notifier nn; struct notifier_block mdev_nb; @@ -1146,6 +1144,7 @@ struct mlx5_ib_dev { /* protect accessing data_direct_dev */ struct mutex data_direct_lock; struct notifier_block mdev_events; + struct notifier_block lag_events; int num_ports; /* serialize update of capability mask */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c index cf8045b92689..8577db3308cc 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c @@ -445,6 +445,34 @@ static int _mlx5_modify_lag(struct mlx5_lag *ldev, u8 *ports) return mlx5_cmd_modify_lag(dev0, ldev->ports, ports); } +static struct net_device *mlx5_lag_active_backup_get_netdev(struct mlx5_core_dev *dev) +{ + struct net_device *ndev = NULL; + struct mlx5_lag *ldev; + unsigned long flags; + int i; + + spin_lock_irqsave(&lag_lock, flags); + ldev = mlx5_lag_dev(dev); + + if (!ldev) + goto unlock; + + for (i = 0; i < ldev->ports; i++) + if (ldev->tracker.netdev_state[i].tx_enabled) + ndev = ldev->pf[i].netdev; + if (!ndev) + ndev = ldev->pf[ldev->ports - 1].netdev; + + if (ndev) + dev_hold(ndev); + +unlock: + spin_unlock_irqrestore(&lag_lock, flags); + + return ndev; +} + void mlx5_modify_lag(struct mlx5_lag *ldev, struct lag_tracker *tracker) { @@ -477,9 +505,18 @@ void mlx5_modify_lag(struct mlx5_lag *ldev, } } - if (tracker->tx_type == NETDEV_LAG_TX_TYPE_ACTIVEBACKUP && - !(ldev->mode == MLX5_LAG_MODE_ROCE)) - mlx5_lag_drop_rule_setup(ldev, tracker); + if (tracker->tx_type == NETDEV_LAG_TX_TYPE_ACTIVEBACKUP) { + struct net_device *ndev = mlx5_lag_active_backup_get_netdev(dev0); + + if(!(ldev->mode == MLX5_LAG_MODE_ROCE)) + mlx5_lag_drop_rule_setup(ldev, tracker); + /** Only sriov and roce lag should have tracker->tx_type set so + * no need to check the mode + */ + blocking_notifier_call_chain(&dev0->priv.lag_nh, + MLX5_DRIVER_EVENT_ACTIVE_BACKUP_LAG_CHANGE_LOWERSTATE, + ndev); + } } static int mlx5_lag_set_port_sel_mode_roce(struct mlx5_lag *ldev, @@ -613,6 +650,7 @@ static int mlx5_create_lag(struct mlx5_lag *ldev, mlx5_core_err(dev0, "Failed to deactivate RoCE LAG; driver restart required\n"); } + BLOCKING_INIT_NOTIFIER_HEAD(&dev0->priv.lag_nh); return err; } @@ -1492,38 +1530,6 @@ void mlx5_lag_enable_change(struct mlx5_core_dev *dev) mlx5_queue_bond_work(ldev, 0); } -struct net_device *mlx5_lag_get_roce_netdev(struct mlx5_core_dev *dev) -{ - struct net_device *ndev = NULL; - struct mlx5_lag *ldev; - unsigned long flags; - int i; - - spin_lock_irqsave(&lag_lock, flags); - ldev = mlx5_lag_dev(dev); - - if (!(ldev && __mlx5_lag_is_roce(ldev))) - goto unlock; - - if (ldev->tracker.tx_type == NETDEV_LAG_TX_TYPE_ACTIVEBACKUP) { - for (i = 0; i < ldev->ports; i++) - if (ldev->tracker.netdev_state[i].tx_enabled) - ndev = ldev->pf[i].netdev; - if (!ndev) - ndev = ldev->pf[ldev->ports - 1].netdev; - } else { - ndev = ldev->pf[MLX5_LAG_P1].netdev; - } - if (ndev) - dev_hold(ndev); - -unlock: - spin_unlock_irqrestore(&lag_lock, flags); - - return ndev; -} -EXPORT_SYMBOL(mlx5_lag_get_roce_netdev); - u8 mlx5_lag_get_slave_port(struct mlx5_core_dev *dev, struct net_device *slave) { diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h index ba875a619b97..a38a17f8c12c 100644 --- a/include/linux/mlx5/device.h +++ b/include/linux/mlx5/device.h @@ -370,6 +370,7 @@ enum mlx5_driver_event { MLX5_DRIVER_EVENT_SF_PEER_DEVLINK, MLX5_DRIVER_EVENT_AFFILIATION_DONE, MLX5_DRIVER_EVENT_AFFILIATION_REMOVED, + MLX5_DRIVER_EVENT_ACTIVE_BACKUP_LAG_CHANGE_LOWERSTATE, }; enum { diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index a96438ded15f..46a7a3d11048 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -643,6 +643,7 @@ struct mlx5_priv { struct mlx5_sf_hw_table *sf_hw_table; struct mlx5_sf_table *sf_table; #endif + struct blocking_notifier_head lag_nh; }; enum mlx5_device_state { @@ -1181,7 +1182,6 @@ bool mlx5_lag_mode_is_hash(struct mlx5_core_dev *dev); bool mlx5_lag_is_master(struct mlx5_core_dev *dev); bool mlx5_lag_is_shared_fdb(struct mlx5_core_dev *dev); bool mlx5_lag_is_mpesw(struct mlx5_core_dev *dev); -struct net_device *mlx5_lag_get_roce_netdev(struct mlx5_core_dev *dev); u8 mlx5_lag_get_slave_port(struct mlx5_core_dev *dev, struct net_device *slave); int mlx5_lag_query_cong_counters(struct mlx5_core_dev *dev, diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index a1dcf812d787..aa8ede439905 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -4453,6 +4453,8 @@ struct net_device *ib_get_net_dev_by_params(struct ib_device *dev, u32 port, const struct sockaddr *addr); int ib_device_set_netdev(struct ib_device *ib_dev, struct net_device *ndev, unsigned int port); +struct net_device *ib_device_get_netdev(struct ib_device *ib_dev, + u32 port); struct ib_wq *ib_create_wq(struct ib_pd *pd, struct ib_wq_init_attr *init_attr); int ib_destroy_wq_user(struct ib_wq *wq, struct ib_udata *udata); From patchwork Wed Aug 21 05:10:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13770789 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2042.outbound.protection.outlook.com [40.107.92.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1A8A155A32 for ; Wed, 21 Aug 2024 05:10:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.42 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217056; cv=fail; b=P5zdudTUhluQFG6QG886J41cH8jVgJp14uJX815CURlv+7b4jO4PyYPTq8d58VwyJgQrzuzsZY4oJAdzKy7R3zbF6OvT6Vp/b0EpoYqv1/m/94vm2d8jKfeHfhcFYczuq2ngRhY/027f9JEu2lS3WBjvjpXHypXZtqIsnc/3+Co= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217056; c=relaxed/simple; bh=0WKga267J6OZR5L3arpGAkNxNVESozhB9ujB1dXARIM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WjtQY+FOIZLdp71FAGJWyt8ElJxaS+nWUTZ2ew7Bz11KpFcvMjgB0tk8a43ttdsUBShedpQVoDsBPwADZxqLShH4DM2B1N2CIEtyOmUiVfZy5Jn6WFa7e763R6ZN6Loefz4rIBSzSni3CId0s0RBF3jYIhdBG1nWbizYG5W18Dk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=GlZmqq5i; arc=fail smtp.client-ip=40.107.92.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="GlZmqq5i" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=edrR0irhTXcPtSJ62sZvtgpZ15lVHk6KzvqU2xNXheDptJNHhA9Wds3emAKylArgiNVkLEFhf2wTZG5E9NEhBf8/oU3IICwjR8sTIgPeTWVjjtnaci5rMLcs9V16FY0WwxoXnFgB3gVCqxJFNoh+GORe+zlBVwPs0OimV43J48sQKbtK27RF3dXgFdSuMD2+yj1wxwRM0XJ7INt5LFTVkwITCvOJsJQZ5B6ByO2+OFe9+FAcvPG5/tAPMge8+oqYyYWZMHJpIgEksjLbAiHcDOBgIH0FetrzMSX4fdXwQoHk7G+YYKEtF6TfvnZ1pdDw+6gfgbdgk8S+yAoh1Kypyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0HaH+38pdKn7KLTcvy75vWdVLaB2VYrxafhkF16g/lQ=; b=LMXqnlo0AOs2p9aTSBn7DQsxbh/JSnXVeFhK/x2zC4gPk2H5d1U5uH390rZjeoq1iDcO3x/3W5roO45OWuPSmgomL0E9ThNxDBfNCmz1PKOI6ZZ6GpOGJaXwJVpMCgGDxfLtw9Z0x9NqrH9/EnMnEndKguQX9I9WJ5MNzAhKIvAEfCi+b2rQq7aIemm7PkHbWO90yi++ZkX6LUBwm5unDsk4DY1A6hnMTB2zuzawh9H0FjUpgAG1BeUnsRUVOl/nvl3B2xs/AkC2/sCOBCQ7goI1DnzbT8Xdv1r5NOIuk+a0hcUCxQCRVyGj4p0LiX0jRb4YAeGIifSfEQ26NYiUdw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0HaH+38pdKn7KLTcvy75vWdVLaB2VYrxafhkF16g/lQ=; b=GlZmqq5iYNchZ3pl4p7r4RQfMlNQuziCBUx9tGNQd44O5GyUb//BfbaG9j4pwolOc/qyz47HmwutjwHqEVkmSJYPi1yi+wDHoFgB+ENFVySFaIHTbrE7Tvhn2qn9mHg0pVs4jdBJOif+mvhGr4Dn4/IATpG0zS7A0GmCb9jNPoPEtUVR3RSfS8HQfRtZsBbjT5+2fE4ATq02Q5uVEkhVOnCPRMczSC0dvNp/MOUOBOmE5V0ovsRcXAgMCFZK2B/W24TmoQ+Vt9zzx51bZaciVFASoZhqWge0GkOUHtLAbAplRnSPRiariVMAhIEfmP6WtcjMJEU17RSUclqwb/1Nbg== Received: from BY5PR20CA0010.namprd20.prod.outlook.com (2603:10b6:a03:1f4::23) by MN6PR12MB8565.namprd12.prod.outlook.com (2603:10b6:208:47d::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.21; Wed, 21 Aug 2024 05:10:51 +0000 Received: from SJ5PEPF000001EA.namprd05.prod.outlook.com (2603:10b6:a03:1f4:cafe::83) by BY5PR20CA0010.outlook.office365.com (2603:10b6:a03:1f4::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.21 via Frontend Transport; Wed, 21 Aug 2024 05:10:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ5PEPF000001EA.mail.protection.outlook.com (10.167.242.198) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.11 via Frontend Transport; Wed, 21 Aug 2024 05:10:50 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:40 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:39 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 20 Aug 2024 22:10:37 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next 6/7] RDMA/nldev: Add support for RDMA monitoring Date: Wed, 21 Aug 2024 08:10:16 +0300 Message-ID: <20240821051017.7730-7-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240821051017.7730-1-michaelgur@nvidia.com> References: <20240821051017.7730-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001EA:EE_|MN6PR12MB8565:EE_ X-MS-Office365-Filtering-Correlation-Id: b6e74086-c724-4d49-16a9-08dcc19f9fcf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: ql4iecuGybHuBO1ioIUPndIVayR93hBojaJZX26hVYGKKt4EEFnFE3rrYAjuNPWptYE6+/KM9ReEnL43IN/wKkv1hKVf7iUixi6K8iEU0GkqfWjzaNqNsngkRy1n6WEZdlwi5EJEIInJ6zsvmnV8GlNjaKroOAcQnBlKRAU0cBHpMF1fulK+4Ccxa8i6m8KqU9N881b3Uy8Ow+B03calQvYAedehRCpR4lQ2g+/lSpNV16cP+bvGK6R7oUJX60zgTIuSL4fsz8+j0N/oQ5RiUYNgIX0pcdY/S+GtXqZxIxYfqxUpWNB9VGMlU1nlUXsTxE53AyhKDLxU6d+7AqC9wONtxOSR3urjuaczb+8aLSHMjPG6MNt48s3WbcWjcMowQCPQXj3jUhDgzYIOCdt6lwlNHYVqvysQazTIznBnVGtpvXRxS9vZ/xkjm1u7DO3ZCDThoKZYJXAVWQbTEIt8V2kTWI1ii0h6JKRs97Wyq21WUEDXPabQ0EzAnA2rKYNzLMssrHgQ8fWPZXN+Gfnq45Ej3RNTreygNEol/8Z/FhWS1YcgKkDx22Bq7YFm9qqj8QJ2L0x76BZtqP59A0HoRhfFUSAtEEfaXnP+p9dDn0szDCF0KctMGhBNeFrshyeMhh2NHhFAk3l1EzzqanOCZPpjGaE40EPRP577wJjJJ6jEV/4jJEva+8RZujuFOwiphQM/Fi0DMkkKeye/h1xPskf/3FWFKD0g8xgAj+C7tMD7WTa3j3e9terb1Cb3eNlLBlXHd9ARP7ypNh0gGKvQXDZHxQGsYhD8U0trBvooFrmYn5+mZR6dZkODeIlC/g4sgRNUGUZg6Gpx4Amo7N2i3s500FwHFfAeSli1iBMxZeIQqDgBeq9E+cZyoRS1IYyTVGJ4B4UTdUBzSCsTIOElU/jEJTKYyzqu/WHq+ay91YbVcUqGBKHc4pXdMH+bVSaPCWk5rcBycxuolQdb6t+g0PJ7vZ8UDbG1QQVMGnZbMixn5agB0NtJhDrMtqj5wM1I1lyF21PLYXiN9ZQDOYMmU5czmDKeMoxaZtgEeeqRc0pO9gysgDvG4IOCBvIczsy+P4vcIr97N4prFdxYaY1LaGszywDnWQD8/K/x2GqBwHVwL+Wgimd9BPwL3gS5VcYSrBEJj25VnK3dASSl1o2G14XFcUWONVEWlkD2xwtFGI6M75oRT+qzWfpra57qqYEFcqOrNl5+Wo04pKLsGS0098eMVc5VR4bZh0BLXRd1MhwvYeDpPQryC0RMy8e1ITJAZylYGI3l7S6Uz5f+GDfCmxwgOTjfy8Qk3b9scqh9xdflAYTU+wFnulpBvlRM9BLWjd/7FHoGSyQSx05HGUu84XyPOP3azqEQ36LsMQhz1PkdZvsvRo7pDULxRbY39Y/CccWCP7C5nJE0DNw2q8MCCNSYXC8uBMH7uxtCxPdlTgTh/G9YW3rI0NUwrqYPNQwu X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 05:10:50.4565 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b6e74086-c724-4d49-16a9-08dcc19f9fcf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001EA.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN6PR12MB8565 From: Chiara Meiohas Introduce a new netlink command to allow rdma event monitoring. The rdma events supported now are IB device registration/unregistration and net device attachment/detachment. Example output of rdma monitor and the commands which trigger the events: $ rdma monitor $ rmmod mlx5_ib [UNREGISTER] dev 3 [UNREGISTER] dev 0 $modprobe mlx5_ib [REGISTER] dev 4 [NETDEV_ATTACH] dev 4 port 1 netdev 4 [REGISTER] dev 5 [NETDEV_ATTACH] dev 5 port 1 netdev 5 $ devlink dev eswitch set pci/0000:08:00.0 mode switchdev [UNREGISTER] dev 4 [REGISTER] dev 6 [NETDEV_ATTACH] dev 6 port 6 netdev 4 $ echo 4 > /sys/class/net/eth2/device/sriov_numvfs [NETDEV_ATTACH] dev 6 port 2 netdev 7 [NETDEV_ATTACH] dev 6 port 3 netdev 8 [NETDEV_ATTACH] dev 6 port 4 netdev 9 [NETDEV_ATTACH] dev 6 port 5 netdev 10 [REGISTER] dev 7 [NETDEV_ATTACH] dev 7 port 1 netdev 11 [REGISTER] dev 8 [NETDEV_ATTACH] dev 8 port 1 netdev 12 [REGISTER] dev 9 [NETDEV_ATTACH] dev 9 port 1 netdev 13 [REGISTER] dev 10 [NETDEV_ATTACH] dev 10 port 1 netdev 14 $ echo 0 > /sys/class/net/eth2/device/sriov_numvfs [UNREGISTER] dev 7 [UNREGISTER] dev 8 [UNREGISTER] dev 9 [UNREGISTER] dev 10 [NETDEV_DETACH] dev 6 port 2 [NETDEV_DETACH] dev 6 port 3 [NETDEV_DETACH] dev 6 port 4 [NETDEV_DETACH] dev 6 port 5 Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/core/device.c | 38 ++++++++++ drivers/infiniband/core/netlink.c | 1 + drivers/infiniband/core/nldev.c | 117 ++++++++++++++++++++++++++++++ include/rdma/rdma_netlink.h | 10 +++ include/uapi/rdma/rdma_netlink.h | 15 ++++ 5 files changed, 181 insertions(+) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index b2fc5a13577c..2113eb7c7573 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -1351,6 +1351,30 @@ static void prevent_dealloc_device(struct ib_device *ib_dev) { } +static void ib_device_notify_register(struct ib_device *device) +{ + struct net_device *netdev; + u32 port; + int ret; + + ret = rdma_nl_notify_event(device, 0, RDMA_REGISTER_EVENT); + if (ret) + return; + + rdma_for_each_port(device, port) { + netdev = ib_device_get_netdev(device, port); + if (!netdev) + continue; + + ret = rdma_nl_notify_event(device, port, + RDMA_NETDEV_ATTACH_EVENT); + dev_put(netdev); + if (ret) + return; + } + return; +} + /** * ib_register_device - Register an IB device with IB core * @device: Device to register @@ -1449,6 +1473,8 @@ int ib_register_device(struct ib_device *device, const char *name, dev_set_uevent_suppress(&device->dev, false); /* Mark for userspace that device is ready */ kobject_uevent(&device->dev.kobj, KOBJ_ADD); + + ib_device_notify_register(device); ib_device_put(device); return 0; @@ -1491,6 +1517,7 @@ static void __ib_unregister_device(struct ib_device *ib_dev) goto out; disable_device(ib_dev); + rdma_nl_notify_event(ib_dev, 0, RDMA_UNREGISTER_EVENT); /* Expedite removing unregistered pointers from the hash table */ free_netdevs(ib_dev); @@ -2159,6 +2186,7 @@ static void add_ndev_hash(struct ib_port_data *pdata) int ib_device_set_netdev(struct ib_device *ib_dev, struct net_device *ndev, u32 port) { + enum rdma_nl_notify_event_type etype; struct net_device *old_ndev; struct ib_port_data *pdata; unsigned long flags; @@ -2190,6 +2218,16 @@ int ib_device_set_netdev(struct ib_device *ib_dev, struct net_device *ndev, spin_unlock_irqrestore(&pdata->netdev_lock, flags); add_ndev_hash(pdata); + + down_read(&devices_rwsem); + if (xa_get_mark(&devices, ib_dev->index, DEVICE_REGISTERED) && + xa_load(&devices, ib_dev->index) == ib_dev) { + etype = ndev ? + RDMA_NETDEV_ATTACH_EVENT : RDMA_NETDEV_DETACH_EVENT; + rdma_nl_notify_event(ib_dev, port, etype); + } + up_read(&devices_rwsem); + return 0; } EXPORT_SYMBOL(ib_device_set_netdev); diff --git a/drivers/infiniband/core/netlink.c b/drivers/infiniband/core/netlink.c index ae2db0c70788..def14c54b648 100644 --- a/drivers/infiniband/core/netlink.c +++ b/drivers/infiniband/core/netlink.c @@ -311,6 +311,7 @@ int rdma_nl_net_init(struct rdma_dev_net *rnet) struct net *net = read_pnet(&rnet->net); struct netlink_kernel_cfg cfg = { .input = rdma_nl_rcv, + .flags = NL_CFG_F_NONROOT_RECV, }; struct sock *nls; diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index 4d4a1f90e484..5acbde242b97 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -170,6 +170,7 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = { [RDMA_NLDEV_ATTR_DEV_TYPE] = { .type = NLA_U8 }, [RDMA_NLDEV_ATTR_PARENT_NAME] = { .type = NLA_NUL_STRING }, [RDMA_NLDEV_ATTR_NAME_ASSIGN_TYPE] = { .type = NLA_U8 }, + [RDMA_NLDEV_ATTR_EVENT_TYPE] = { .type = NLA_U8 }, }; static int put_driver_name_print_type(struct sk_buff *msg, const char *name, @@ -2722,6 +2723,122 @@ static const struct rdma_nl_cbs nldev_cb_table[RDMA_NLDEV_NUM_OPS] = { }, }; +static int fill_mon_netdev_association(struct sk_buff *msg, + struct ib_device *device, u32 port, + const struct net *net) +{ + struct net_device *netdev = ib_device_get_netdev(device, port); + int ret = 0; + + if (netdev && !net_eq(dev_net(netdev), net)) + goto out; + + ret = nla_put_u32(msg, RDMA_NLDEV_ATTR_DEV_INDEX, device->index); + if (ret) + goto out; + ret = nla_put_u32(msg, RDMA_NLDEV_ATTR_PORT_INDEX, port); + if (ret) + goto out; + if (netdev) + ret = nla_put_u32(msg, RDMA_NLDEV_ATTR_NDEV_INDEX, + netdev->ifindex); + +out: + dev_put(netdev); + return ret; +} + +static int fill_mon_register(struct sk_buff *msg, struct ib_device *device, + const struct net *net) +{ + return nla_put_u32(msg, RDMA_NLDEV_ATTR_DEV_INDEX, device->index); +} + +static void rdma_nl_notify_err_msg(struct ib_device *device, u32 port_num, + enum rdma_nl_notify_event_type type) +{ + struct net_device *netdev; + + switch (type) { + case RDMA_REGISTER_EVENT: + dev_warn_ratelimited(&device->dev, + "Failed to send RDMA monitor register device event\n"); + break; + case RDMA_UNREGISTER_EVENT: + dev_warn_ratelimited(&device->dev, + "Failed to send RDMA monitor unregister device event\n"); + break; + case RDMA_NETDEV_ATTACH_EVENT: + netdev = ib_device_get_netdev(device, port_num); + dev_warn_ratelimited(&device->dev, + "Failed to send RDMA monitor netdev attach event: port %d netdev %d\n", + port_num, netdev->ifindex); + dev_put(netdev); + break; + case RDMA_NETDEV_DETACH_EVENT: + dev_warn_ratelimited(&device->dev, + "Failed to send RDMA monitor netdev detach event: port %d\n", + port_num); + default: + break; + }; +} + +int rdma_nl_notify_event(struct ib_device *device, u32 port_num, + enum rdma_nl_notify_event_type type) +{ + struct sk_buff *skb; + struct net *net; + int ret = 0; + void *nlh; + + net = read_pnet(&device->coredev.rdma_net); + if (!net) + return -EINVAL; + + skb = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL); + if (!skb) + return -ENOMEM; + nlh = nlmsg_put(skb, 0, 0, + RDMA_NL_GET_TYPE(RDMA_NL_NLDEV, RDMA_NLDEV_CMD_MONITOR), + 0, 0); + + switch (type) { + case RDMA_REGISTER_EVENT: + case RDMA_UNREGISTER_EVENT: + ret = fill_mon_register(skb, device, net); + if (ret) + goto err_free; + break; + case RDMA_NETDEV_ATTACH_EVENT: + case RDMA_NETDEV_DETACH_EVENT: + ret = fill_mon_netdev_association(skb, device, + port_num, net); + if (ret) + goto err_free; + break; + default: + goto err_free; + } + + ret = nla_put_u8(skb, RDMA_NLDEV_ATTR_EVENT_TYPE, type); + if (ret) + goto err_free; + + nlmsg_end(skb, nlh); + ret = rdma_nl_multicast(net, skb, RDMA_NL_GROUP_NOTIFY, GFP_KERNEL); + if (ret && ret != -ESRCH) { + skb = NULL; /* skb is freed in the netlink send-op handling */ + goto err_free; + } + return 0; + +err_free: + rdma_nl_notify_err_msg(device, port_num, type); + nlmsg_free(skb); + return ret; +} + void __init nldev_init(void) { rdma_nl_register(RDMA_NL_NLDEV, nldev_cb_table); diff --git a/include/rdma/rdma_netlink.h b/include/rdma/rdma_netlink.h index c2a79aeee113..a5b47ca4cd66 100644 --- a/include/rdma/rdma_netlink.h +++ b/include/rdma/rdma_netlink.h @@ -110,6 +110,16 @@ int rdma_nl_multicast(struct net *net, struct sk_buff *skb, */ bool rdma_nl_chk_listeners(unsigned int group); +/** + * Prepare and send an event message + * @ib: the IB device which triggered the event + * @port_num: the port number which triggered the event - 0 if unused + * @type: the event type + * Returns 0 on success or a negative error code + */ +int rdma_nl_notify_event(struct ib_device *ib, u32 port_num, + enum rdma_nl_notify_event_type type); + struct rdma_link_ops { struct list_head list; const char *type; diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h index 2f37568f5556..5f9636d26050 100644 --- a/include/uapi/rdma/rdma_netlink.h +++ b/include/uapi/rdma/rdma_netlink.h @@ -15,6 +15,7 @@ enum { enum { RDMA_NL_GROUP_IWPM = 2, RDMA_NL_GROUP_LS, + RDMA_NL_GROUP_NOTIFY, RDMA_NL_NUM_GROUPS }; @@ -305,6 +306,8 @@ enum rdma_nldev_command { RDMA_NLDEV_CMD_DELDEV, + RDMA_NLDEV_CMD_MONITOR, + RDMA_NLDEV_NUM_OPS }; @@ -574,6 +577,8 @@ enum rdma_nldev_attr { RDMA_NLDEV_ATTR_NAME_ASSIGN_TYPE, /* u8 */ + RDMA_NLDEV_ATTR_EVENT_TYPE, /* u8 */ + /* * Always the end */ @@ -624,4 +629,14 @@ enum rdma_nl_name_assign_type { RDMA_NAME_ASSIGN_TYPE_USER = 1, /* Provided by user-space */ }; +/* + * Supported rdma monitoring event types. + */ +enum rdma_nl_notify_event_type { + RDMA_REGISTER_EVENT, + RDMA_UNREGISTER_EVENT, + RDMA_NETDEV_ATTACH_EVENT, + RDMA_NETDEV_DETACH_EVENT, +}; + #endif /* _UAPI_RDMA_NETLINK_H */ From patchwork Wed Aug 21 05:10:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Guralnik X-Patchwork-Id: 13770790 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2052.outbound.protection.outlook.com [40.107.96.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28A2B15C147 for ; Wed, 21 Aug 2024 05:11:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.96.52 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217062; cv=fail; b=Kfe0JMBRzhi9IcGGY5pc6b145vEG4GRaznU9X0Xr8KqexE1WtU8kSJWYAGe2I09NWMwJ2mzvehs10UlwTxMe7zfeXod3vYIa5JK4CYLbglZHJWiKwqbqwut9Od8e5pmKP85Y9U4yxzlleklBJxL2gAcZjwksQelpSUwidVNeogg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724217062; c=relaxed/simple; bh=N5N9RtHvWbeo6ygR5laGlujSR+F9dMK8fFaVo/VU7vU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FdCnoh88M/oztPURIeK9aFrvvb621JJcwpnezgvS7rwi/d10XZe3qIFbSUWzBWx1J2OGqkvt79XuSIWTy7DK5tRDr6wOqcWQBEI8Fhtdzk/il0ZgyERXx1fUuoyaFcX7RWYFeC1fZHqhndB1oU2LORkCvltq7VJPDWBAKj1omU8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=pGuObAWW; arc=fail smtp.client-ip=40.107.96.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="pGuObAWW" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=SS2dclc78I69N+bx0m0KV4Bi/5X0Vdt3mSeBR2GblaUReGJkHaavXpDIPji8GdJfRnv5/d3DDaiusBY0mBjZQy24p5gj0gAnbyB4O1vdxdpyHzRytBWnTUnwAL0E17giGh+glXYprQivW6V6wbm8ZxCW6Xpf4vWE8dIoOniKGNAaiNsdZ2ep5XAyR/+kMhPobuKhES+WUG1ARXwLJKgkTbUWqMMheSrmhsc4hNK2rWMr+UNs2+K91bTSXd4L6OpBofYer8+qCToRjIyhkye7DSWklnhfQwrJLx+kkCiDNt9+fqvvmpM54pX61WaoYpIz47/noybCzXoKIelzNmxVwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XQ25i+eb9lo7wV0+iYrzEvPiqzOfZKgJGs+Nv1lyJ/A=; b=TP4jMw0iPj1IbY0E4TuRZRP6o20tc2P1ZMKKD9nrRccT5YLYQrGXQlqvZSbc1IR/PvBYiyrf1pfO1UIY/QbXvsIj3jME2gWFJhMsY08D33LEaEpYOi37S2t2BEagYTSSUxC/HgXuKzEfSPDVXZ8r7VKYb6vPkVMCqflOOqpIm8uuh6rYBYAhhO4a2vY0QR7bL8vVBrsEJq8c2mh1l9egcNaecnjSB6i5h3mBsrz1Q7dfbvBf4xbYlN1oMC+D6i6JTnWZ7XOnbOwNEUEldYCwk4LjBq23Fui5FpIB+kfWmAXDqz7Zka/ibBu+9LmS0j3C9Ixd78havbakobuXOt5gsA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XQ25i+eb9lo7wV0+iYrzEvPiqzOfZKgJGs+Nv1lyJ/A=; b=pGuObAWWyG26xM+wh94CjO0DZsAUTwQDb2SzBhPrL3XPz95bidlp8CqJATiwTeUywRJr5xy0ktlt8+Zkv10t57Qtaj35aGX9vb6SYbItuqmJzk7JIOZLngKf7Sg6qH7UnYOX+/5fV9TqROUQyHLvE8Hp55GzVBy9yWAHhXgl/85AjsWvuRx9UPViLFINpOxGP/zmK9RvSq0XKoTo0w8ELD8xNF9T8q7846QuI28eFma2S/aH72+sUN4Tsp/BuscEvZlYLTOjW8JgEmRXSpX4Ff1eyEyOs3wbckGITky4315UiPcRVB3ciq2npChfmZWoAEA8EtQSZhoGp6tKYHRGJg== Received: from BYAPR01CA0071.prod.exchangelabs.com (2603:10b6:a03:94::48) by CH2PR12MB4280.namprd12.prod.outlook.com (2603:10b6:610:ac::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.18; Wed, 21 Aug 2024 05:10:57 +0000 Received: from SJ5PEPF000001E9.namprd05.prod.outlook.com (2603:10b6:a03:94:cafe::e5) by BYAPR01CA0071.outlook.office365.com (2603:10b6:a03:94::48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.21 via Frontend Transport; Wed, 21 Aug 2024 05:10:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ5PEPF000001E9.mail.protection.outlook.com (10.167.242.197) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.11 via Frontend Transport; Wed, 21 Aug 2024 05:10:57 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:43 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 20 Aug 2024 22:10:42 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 20 Aug 2024 22:10:40 -0700 From: Michael Guralnik To: CC: , , , , , Michael Guralnik Subject: [PATCH rdma-next 7/7] RDMA/nldev: Expose whether RDMA monitoring is supported Date: Wed, 21 Aug 2024 08:10:17 +0300 Message-ID: <20240821051017.7730-8-michaelgur@nvidia.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20240821051017.7730-1-michaelgur@nvidia.com> References: <20240821051017.7730-1-michaelgur@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001E9:EE_|CH2PR12MB4280:EE_ X-MS-Office365-Filtering-Correlation-Id: fa6778c1-ed21-4bdd-0836-08dcc19fa3ee X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: FB60v1aIGF8SMw2IRnVxmtUVChrtYPdEOJSnFXp3m83EZoIc0IdG1r+IR4sMhL76kWHnn3ij9cAsuEQHrWG31Spg0EmKMOuWcoxttCRkkAGQbo6U5DAuykGLtXQRbUgx1z1VjrQiKZdbGbf0NX7bO5LWpOYnaQ3/HDZb1C3ZdyjMz4CqdEl20Js0+OROur1LmmNPi8wZI0yilgUI1l6CgtLnJ3bJqAMupm7Mr3AMyD1BVuP6KzLahak6xQaWlNI3+y0nBdprMbb6o0TBczrLi2lRSVL7PPg9SvPs68DmnhhulbNvureuT9i+AUQB3B/MTRtjofB+9l8bSVa4s1p6MjSmvCD9tSe1SckcGNhC7jQ5EPg1bt75qMIPc3+smRx2cxHezqVpwVT7rGV8DctiFijVgCHPR+p+L9y+yXymyPYp+VdDShKbzv9U4JySbqdKjSXdc0xMCCc8skZK5MSrT6cqLQwZGXFvDeyXOdDeSeDGY0/C0Wwn2gFXXP28epN1hnT48ZKj432H3qbFKripf6Eoe2XwqUlXQZO+7WhIxsJ0lHjO5LN2UgSOAWRl6i1+ADOyvlGHd1o5z8baNkY66CsBMqsq9lo5Fz6sgjiGcJbRF3a0AupdY0dIQjct2QjIJ61hl/qu+14rk0ybDK/54qWoAR93SiQLwTVVqCxXxQA+pPhFltKwxwmNlUTfcr6YFamXrFYJD+TphIj09SaojzkSbOwalbqyfDHFQDQvSlO7FLLImQIXB5TLMpq+1hgGwANN63fzcXvkC2DQeQg1SKF5SDGyCDalGLpPuQaUMOdcqVWjm8EAN5YTs/v+BMKEWARd6EuqH8srC6/A8evX4F4VN28rsztQwIlVfbMcxvyLCtxg1h2Hj6yciLCZzPPYw377ZZuoO3pxWj4Izm7lfBeiiY+akJtKfiNPth8nTGHF9QN9+fcLODoxVnaekOYtzyY2JpkpmORc2/We0j+GSXiieHw7uZyHt3ssyY3c2cfAvp8JvcCdby4x22yLRw9wNVf+7OlkO7Z1dPnLkW9OT6ORnBWn4qGZCmv2qMrZJmnWF4YoC3Zsp5hs5el7izcQdCzpudJoGSBlLSleM7hhz6mGL40kApJvtwvmndY2Att/7BIvE2+NeK+N5fggV+oVrZ1/wBkh6pbsbAQkkeggnFDEuJVvBOQesGXZ8YNIASXNXRykcCl7haRxYXgnjq1Wb6isfMKfTHX+XH7kqmliDH1WbdS7Lyxu/tqNLCygV6yW6zj22sGoV9nnYyIBIXDS+43CvxCyOs/fHkUyyyKMhMh0g3GGDhGNfhxpybJmhwodDinbvUDv5Uwg9h1EWglIXH8+eeo9nkbKS+tNCACTOVZQUPnEnjJM/k3+yYLsIhfKBBancoBI29Lvye8WhwdyBdxWJSJ91v8jnbT/vwCOYBZCSSBPcYJx6Pnh4tw/OvPj+8lLZQjiT5sYwwb6sSfh X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2024 05:10:57.3702 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fa6778c1-ed21-4bdd-0836-08dcc19fa3ee X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001E9.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4280 From: Chiara Meiohas Extend the "rdma sys" command to display whether RDMA monitoring is supported. RDMA monitoring is not supported in mlx4 because it does not use the ib_device_set_netdev() API, which sends the RDMA events. Example output for kernel where monitoring is supported: $ rdma sys show netns shared privileged-qkey off monitor on copy-on-fork on Example output for kernel where monitoring is not supported: $ rdma sys show netns shared privileged-qkey off monitor off copy-on-fork on Signed-off-by: Chiara Meiohas Signed-off-by: Michael Guralnik Reviewed-by: Leon Romanovsky --- drivers/infiniband/core/nldev.c | 6 ++++++ include/uapi/rdma/rdma_netlink.h | 1 + 2 files changed, 7 insertions(+) diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index 5acbde242b97..f6eaf9c1fc05 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -1952,6 +1952,12 @@ static int nldev_sys_get_doit(struct sk_buff *skb, struct nlmsghdr *nlh, nlmsg_free(msg); return err; } + + err = nla_put_u8(msg, RDMA_NLDEV_SYS_ATTR_MONITOR_MODE, 1); + if (err) { + nlmsg_free(msg); + return err; + } /* * Copy-on-fork is supported. * See commits: diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h index 5f9636d26050..39be09c0ffbb 100644 --- a/include/uapi/rdma/rdma_netlink.h +++ b/include/uapi/rdma/rdma_netlink.h @@ -579,6 +579,7 @@ enum rdma_nldev_attr { RDMA_NLDEV_ATTR_EVENT_TYPE, /* u8 */ + RDMA_NLDEV_SYS_ATTR_MONITOR_MODE, /* u8 */ /* * Always the end */