From patchwork Wed Nov 30 11:52:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shay Drori X-Patchwork-Id: 13059742 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8575EC352A1 for ; Wed, 30 Nov 2022 11:53:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234839AbiK3Lxa (ORCPT ); Wed, 30 Nov 2022 06:53:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235013AbiK3Lw7 (ORCPT ); Wed, 30 Nov 2022 06:52:59 -0500 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2089.outbound.protection.outlook.com [40.107.101.89]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BC3E4731B for ; Wed, 30 Nov 2022 03:52:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PkRdhRu+ShDNCPIhCKl+CDgR60N9fowJIUSgllmOFuEaOiuA0xZl7K5ptmOnb146rPygQeLQOOp7tNXecQ3qmgTQlhb3iQ91ofTptlo1WVaODbSU7GBYd67vsYQuJhaykvXkpbNhcci2fRoOb3MR2Y4zSkL2YalAQtIi8D9M9Msh+KG91J20njpRoi9rGcHiAatax6qRLZ7BaAfiz6uTZeXNSXFORTq0vEMNmEHs71o0wcUQaCXg7svcUwqnV946mbgBK55XL+BEbbG8flzvzHzUjRlWiZwLOJ3AyotqgNJXWkBEhk4SQ2M6WD14+ycXY0ltrKW1MDjpvTdlwIJviw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UdYWbJu/O4vexc1lw1wYV1Uu1YQuuFUiLhiW3wfBEOQ=; b=N4VH475MUhONYHLz3LnvpAgmqMbcupT/jxcOthL+u/tra9AY8+AfC3cghmp8ZzU8OCEfUmq53bpzTzREwj7XCk12JlXFBijzdsgzgyxGI7moEJe6fOpZSAIN13ijRPixgnCTfo0ukIxM3nIm4Vy6p5RomKiVYHWBrx0+XfK8k/SYUzNOaViZMUd+oRnXrTo2HKw7Cc88K9vc3BY+alZW90vM8yTbyhmOPMePWgt6qoeftfVNPBg/w5nKegpdT0PvJ6dR5RblQr6uOPjJbhXZ5VM2I7RFRV1Q2liZHXqVz7UPbyxutRKlYdoE+h6TYfGbS5T9QN9tBQr/ZsNF9IazNg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UdYWbJu/O4vexc1lw1wYV1Uu1YQuuFUiLhiW3wfBEOQ=; b=LvgOVig9UCgB5bDPX4B+Aes0k4icgP326fdXJBlwDyJ6jUs9b71aN9eTkcW9JGcLoBnIajEtA1VyHjtelycohNwet05rDytckl0qb90Ezign8n2ZinNFoq0aZuocFjw40RGIT1u2TYCjrCPxGCinYz+oJIwfziZa/R6RX8r3yJ2Mq9QTDaN6abUhxcYZbtD4mPrqsCfjGgtzNd4EtQefBysqrl9okETkSB+lbUCnszZlqwUb1IiQU8+GvGPhh3F+rCTYcOOerCKhmElhBj5voJk2CGJgcCL7cjGLeRG2BAenbRkBz0MqmHq8EZkc152u+zSrIS2DC0sE2jlXZPD/+g== Received: from BN1PR13CA0010.namprd13.prod.outlook.com (2603:10b6:408:e2::15) by CY5PR12MB6621.namprd12.prod.outlook.com (2603:10b6:930:43::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23; Wed, 30 Nov 2022 11:52:47 +0000 Received: from BN8NAM11FT106.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e2:cafe::bb) by BN1PR13CA0010.outlook.office365.com (2603:10b6:408:e2::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5880.8 via Frontend Transport; Wed, 30 Nov 2022 11:52:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT106.mail.protection.outlook.com (10.13.177.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.22 via Frontend Transport; Wed, 30 Nov 2022 11:52:47 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:36 -0800 Received: from nps-server-23.mtl.labs.mlnx (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:31 -0800 From: Shay Drory To: , , CC: , , , , Subject: [PATCH net-next 1/8] net/mlx5: Introduce IFC bits for migratable Date: Wed, 30 Nov 2022 13:52:10 +0200 Message-ID: <20221130115217.7171-2-shayd@nvidia.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221130115217.7171-1-shayd@nvidia.com> References: <20221130115217.7171-1-shayd@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT106:EE_|CY5PR12MB6621:EE_ X-MS-Office365-Filtering-Correlation-Id: 2f5718d7-1a0f-4f21-486a-08dad2c96693 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xRq/TJDl6TVV4/gJaPHJlhbtetB7YfGnMC2zx4URj9ich3lXf/vcNRm4lazZsNhv+8NopPzu3gOvQpLLfjngpagT9jEfp1AJbyUlOfwtvBsCxdY1mXLnE23f1SIUD5X0S9/S37ZXvDEecSdthlfYuGTRnKL4kqA2eDG/19Ja5KLeiLgQOhDr+7oLjR/yuDRnWBviYBW1Zz6g16mRaF/9c1XsAwOCDmaWItT2+Ea3NITLE/FHHZVF1sL0wsU54sf5cftqZ6rvyrKtR0UCxdjlnX5aGD/nnZZlnSSJkU3hR1UdaV9/7/GE83M+y58RGaV38nEBrrcOqfZRQljxqjZX8i6RuqVpzynkvo220Wu5oaJ4ktnZuTgdFbWRbb03qW5WyQim6ouq+9z8jlvZ+VNoDVO+0ZKIbeSJdR3gH4L29u2/ilJy1228f3adLWBgxNZZQ4q08omF2krAymvB/ip2gYxqimxRfTzkiJUIQB1fsoSHeRSZ8Gne7CrEKWEE3c5nEieoB6igrYRUJUkagIxLXu/OObA6uxpTgIWcFJcj/qkIFt5EuJtjbXCiph6xYbDuSv9F1CKCeIUdC+MFvpMlhg/Ly6kIcGPjYzhWAgVgu/NrlghhPG8TT77/cXmFZ2ZfR0R3AmZ8BytDLYpZKEJ1x9rOUtJI74d7Dy1+/cY2/zvWHpolR1Wb7D2amJVsWzlZbS1JavnxvUrQ/UHlDdr/yg== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(346002)(39860400002)(136003)(396003)(376002)(451199015)(40470700004)(46966006)(36840700001)(356005)(40480700001)(7636003)(82740400003)(336012)(36756003)(82310400005)(26005)(41300700001)(478600001)(6666004)(8676002)(70586007)(40460700003)(70206006)(86362001)(2616005)(4326008)(54906003)(110136005)(316002)(5660300002)(186003)(8936002)(1076003)(47076005)(426003)(16526019)(83380400001)(107886003)(2906002)(36860700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2022 11:52:47.5503 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2f5718d7-1a0f-4f21-486a-08dad2c96693 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT106.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6621 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Yishai Hadas Introduce IFC related capabilities to enable setting VF to be able to perform live migration. e.g.: to be migratable. Signed-off-by: Yishai Hadas Reviewed-by: Mark Bloch Ack-by: Saeed Mahameed --- include/linux/mlx5/mlx5_ifc.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index 5a4e914e2a6f..2093131483c7 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -68,6 +68,7 @@ enum { MLX5_SET_HCA_CAP_OP_MOD_ODP = 0x2, MLX5_SET_HCA_CAP_OP_MOD_ATOMIC = 0x3, MLX5_SET_HCA_CAP_OP_MOD_ROCE = 0x4, + MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE2 = 0x20, MLX5_SET_HCA_CAP_OP_MODE_PORT_SELECTION = 0x25, }; @@ -1875,7 +1876,10 @@ struct mlx5_ifc_cmd_hca_cap_bits { }; struct mlx5_ifc_cmd_hca_cap_2_bits { - u8 reserved_at_0[0xa0]; + u8 reserved_at_0[0x80]; + + u8 migratable[0x1]; + u8 reserved_at_81[0x1f]; u8 max_reformat_insert_size[0x8]; u8 max_reformat_insert_offset[0x8]; From patchwork Wed Nov 30 11:52:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shay Drori X-Patchwork-Id: 13059743 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA75AC352A1 for ; Wed, 30 Nov 2022 11:53:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234857AbiK3Lxj (ORCPT ); Wed, 30 Nov 2022 06:53:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44622 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235050AbiK3LxJ (ORCPT ); Wed, 30 Nov 2022 06:53:09 -0500 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on20610.outbound.protection.outlook.com [IPv6:2a01:111:f400:7e89::610]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4293D4384F for ; Wed, 30 Nov 2022 03:52:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TnW6RJ+ZGtfORZSPHykJbnhhL1JsuTBSg8gRe1+LBACP9dZf3t5xDS/9EXVfziaS7cfKM2jvLO6B/G1/x+7P4akL+NftTepRmjbweeubAjxRHlf/L3uYZxJnI8BcROippm8H8YVCMGi1Mz5D0dgO9n0pxEkX8gX2T2mTZn2fqWmBpcME5jXUwadgtcBchSUIeu/HqzrMG/mPOQYfVS4HEzAomutwVhCHaV2r0p53Es5BBz0azTWjAWexD2Imyfoxg0sUrOmSiUSh1e3x41etRor8aSAcgj43UHSa+tzewbatXJnHT3mfKsxD7LRLLXFZBemZ2ZHMPGg+XX7iRURuxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+8tnz/fAIpB6X4FwfAExW1GLX/Kd3rHDtlo5rFKlJ40=; b=X/tlbAeqboYqVJPgF1UrMDd1aJaEPhqt8H3DsVxv2QZzrgTHnuCVcc8D05eLf3H/hrk893nYvD2P7rOUQEGJR79TUb43t/j01NLjiiHtpqnZJ7STdKWCcgU6o5pJIHwNX8YhJ8BuW66GU67sRCnOG9SLiNlU79rkzbyH1IyBFGDIN4AqJy1WqyktgSo1JjKq31UHFbLN5wvEe751HpTyHyTEtQQl+NhAyA26YAco1fzOu6yAfFbw1gQ4F8mBH1X2eFU+PcbrFpLZ2tYCo9VRAD1FCgpWEC9Eqf7FF67VMHaaMcG3mgQvN0qAVyH/ZDqxx582nmJUmiepo0sZsrO2ew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+8tnz/fAIpB6X4FwfAExW1GLX/Kd3rHDtlo5rFKlJ40=; b=gY71+n50ebDdbaBin7YJCobU4+fD9zI4aTvXYZxM8ITWY7+ycp93QQHJvuB7YeNZWKByXrgFTRd7GAbZqOjcmGPdgRdT2Rm5NWE7sDWkPkXBDZ1NtaTK8GKzfdX245X05Wqy9TLn9LZ5jC4RQgbHzLCEUwx6+ZAO/pOX7CtzcB8v1aggdXhqfbFsHKS5RBOuKsPmjBVAqEnOd+u9xmew2Kx+hrExr8Ys/uyZ8fWXgmyRFhCb/7dIsijRWxbY3OJ7eA0bA7ufLG0EIK7oxgDAVsZgsY5RaAvgr5xvWrLMr9g+NsyNHi8qI3Yz/jMMMC+CghfafWZdpkp6jZgBN8q0aA== Received: from BN1PR13CA0015.namprd13.prod.outlook.com (2603:10b6:408:e2::20) by MN0PR12MB5979.namprd12.prod.outlook.com (2603:10b6:208:37e::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23; Wed, 30 Nov 2022 11:52:51 +0000 Received: from BN8NAM11FT106.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e2:cafe::81) by BN1PR13CA0015.outlook.office365.com (2603:10b6:408:e2::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5880.8 via Frontend Transport; Wed, 30 Nov 2022 11:52:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT106.mail.protection.outlook.com (10.13.177.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.22 via Frontend Transport; Wed, 30 Nov 2022 11:52:50 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:40 -0800 Received: from nps-server-23.mtl.labs.mlnx (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:36 -0800 From: Shay Drory To: , , CC: , , , , Subject: [PATCH net-next 2/8] devlink: Validate port function request Date: Wed, 30 Nov 2022 13:52:11 +0200 Message-ID: <20221130115217.7171-3-shayd@nvidia.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221130115217.7171-1-shayd@nvidia.com> References: <20221130115217.7171-1-shayd@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT106:EE_|MN0PR12MB5979:EE_ X-MS-Office365-Filtering-Correlation-Id: 227d07c0-3560-461c-7795-08dad2c96872 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LJLF6ADR1fotEIfV3Yhv/qQSAn4HoNrlMDy3tgZEv2objYvyh//O6mu+IpykOQm+KtXxuJ/XctOQcZIwNiFK3GmDmpREN29Qxc0ojTJO/OcXi5b7gSVNh5OgNHKtwjnflwnyr4N6V7T9/fRrE3weG6+uq4uRn6kWLg6tvugERVIpVlR+q2S7M/SMmdcGDJ+V8GuwDcpPP4ydb0pTDGrzXAS982m7YqX3/yET3ieGhfMyR4Lk+yxq0xBemvT3yMky/JHrzk75mGejwd6pHSheB0kSyr7zieZe9TdBBGl6pAIpXEEe5AGEcc6AuMwZf83w5XP3/kRChvx2N05RNTgmuGZesHdLFTAH2IcNEahs+YFnlQVMxWpeoghvWUNKBuqX96fkmMsJPQtx+xxmT6r88R1wsclqw725JoR+zgffvsdGDQNqgNFcj36/KK78ngB8U0pDkI7VUsHtxWDvk2TsX/codR5Sl+5ItdQ1JhoAMLExjf6wsKXT3IwdYfXkM1waXMWiHKwuUB0HoQ7ul9joGplUCEy/kaQsngjAv2KwltynnnaQKSzsa6/8QULc/anKMHL/hSZzOWaJ2X2zYSDj3ZJNt2bzZ2ek10CKzM4huO44DXwVXgt1VCYY9e/GFiag8ZkpMVCQL8uhHnbfufMLjIFIS7rahbN93TnQhu747cF6BkiydljYqaOWYM3kjnW7vzM90n/2qhXAsWEou9dk4w== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(396003)(39860400002)(136003)(346002)(376002)(451199015)(46966006)(40470700004)(36840700001)(186003)(1076003)(2616005)(107886003)(336012)(426003)(36860700001)(47076005)(16526019)(5660300002)(6666004)(41300700001)(36756003)(82310400005)(70206006)(86362001)(26005)(8676002)(82740400003)(8936002)(478600001)(4326008)(40480700001)(54906003)(83380400001)(70586007)(2906002)(40460700003)(316002)(356005)(7636003)(15650500001)(110136005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2022 11:52:50.6907 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 227d07c0-3560-461c-7795-08dad2c96872 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT106.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB5979 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org In order to avoid partial request processing, validate the request before processing it. Signed-off-by: Shay Drory Reviewed-by: Jiri Pirko --- net/core/devlink.c | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/net/core/devlink.c b/net/core/devlink.c index cea154ddce7a..485348697290 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -1644,11 +1644,6 @@ static int devlink_port_function_hw_addr_set(struct devlink_port *port, } } - if (!ops->port_function_hw_addr_set) { - NL_SET_ERR_MSG_MOD(extack, "Port doesn't support function attributes"); - return -EOPNOTSUPP; - } - return ops->port_function_hw_addr_set(port, hw_addr, hw_addr_len, extack); } @@ -1662,12 +1657,26 @@ static int devlink_port_fn_state_set(struct devlink_port *port, state = nla_get_u8(attr); ops = port->devlink->ops; - if (!ops->port_fn_state_set) { + return ops->port_fn_state_set(port, state, extack); +} + +static int devlink_port_function_validate(struct devlink_port *devlink_port, + struct nlattr **tb, + struct netlink_ext_ack *extack) +{ + const struct devlink_ops *ops = devlink_port->devlink->ops; + + if (tb[DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR] && + !ops->port_function_hw_addr_set) { + NL_SET_ERR_MSG_MOD(extack, "Port doesn't support function attributes"); + return -EOPNOTSUPP; + } + if (tb[DEVLINK_PORT_FN_ATTR_STATE] && !ops->port_fn_state_set) { NL_SET_ERR_MSG_MOD(extack, "Function does not support state setting"); return -EOPNOTSUPP; } - return ops->port_fn_state_set(port, state, extack); + return 0; } static int devlink_port_function_set(struct devlink_port *port, @@ -1684,6 +1693,10 @@ static int devlink_port_function_set(struct devlink_port *port, return err; } + err = devlink_port_function_validate(port, tb, extack); + if (err) + return err; + attr = tb[DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR]; if (attr) { err = devlink_port_function_hw_addr_set(port, attr, extack); From patchwork Wed Nov 30 11:52:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shay Drori X-Patchwork-Id: 13059744 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FACDC433FE for ; Wed, 30 Nov 2022 11:53:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234475AbiK3Lxu (ORCPT ); Wed, 30 Nov 2022 06:53:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234023AbiK3LxM (ORCPT ); Wed, 30 Nov 2022 06:53:12 -0500 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2054.outbound.protection.outlook.com [40.107.92.54]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 191816395 for ; Wed, 30 Nov 2022 03:52:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Nud7dxsny+t50OYD3f6nN2lbzelQjLE+FiyzRtXRnQFCrEZMAnXqcBhDFZMnTmCITCMFsOYmanenrS87fRPaISXsaihZwEVLR0tZjWogMMIBiUeKwlOJ+yZbI4RjQD59sqfy1fT09y3kO2BlcOEqKXIxZXHH9T5IzJCEPWQphE+nUEZdWBlEKM56LYnLtzSdEkLAOXA8v6ifRIrqFqd9xUWkgyRARknOHjHJN2H2iPNnYjr8+I1RYGGULdgs+wER3X3/Wka7FerLhByXQWC6b2K0fxI4b2zCK19AI5CS4UuxXWxDWyw4GI3dYOHwTugg4yPjAlUJmKp4gnbGJf9lBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uxbjXPUmyVu7jtPYS4lTU8jCr6P3PdNRtmwYwSWthpk=; b=C3SnvE3iuGbOtaS9r6H7grMI1h8EULelrUcg6kMQCvmHGDof6C3OGob1xtRxH43I6AJrjJxz3onIYAk2ODiIl/BliRgRabxKmq02Fh2b56Anl6XXu7UJkyl2HDfoEK7SFfwYoLhUssouplJRdwOuc32DrWNAS8DTS9glVBP7bnVSnqCRmAeNutZCjZY2PllWtO/maN2MJfo2s3/XnO7zK8LjmLzMo1necBODj2sSo9IxN1gf1XeZHmmEo+OWmdtk0c+EwcYVLEf0uS09Tri6IFVFjWmRInlALpQRbRLFJ5pYleXxvad9MCqWEfC5ich48nKKLoOH3SFxG8LvE0TF9g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uxbjXPUmyVu7jtPYS4lTU8jCr6P3PdNRtmwYwSWthpk=; b=oCRMbwqcW2CfOQdozEhCz8Y1+xQIZurqaYeyMsPy7muVaxMjC4vYWx7hXvrR3gjRtF9S/IOyDxwgfZF3pR6Oad/XIuOYDD+R8Kuu8xZ4dN5YR5tKcP/hPJBMlMZ/4DkAUqN/+9Y0XCxR6gLGZSjVpoFYVRqyNjLgi/F8n16ZKYdRWSL2zFFjk0gQdTUNKw2NnOiU1nWlB3nbKtnKnP8CxNqvdN1OJ1lsRBRk798/10gKyaZtLgUo484t3PIP79+c0RYWQRUBlKAL/OdeYqmU7R55eeSY3o5UH1hIO5ckCj+gQHY/xUWP7rBz5Xj1Sza9NsRuipfv2/aLQsdO1rM5XQ== Received: from BN9PR03CA0588.namprd03.prod.outlook.com (2603:10b6:408:10d::23) by SA1PR12MB6702.namprd12.prod.outlook.com (2603:10b6:806:252::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23; Wed, 30 Nov 2022 11:52:57 +0000 Received: from BN8NAM11FT051.eop-nam11.prod.protection.outlook.com (2603:10b6:408:10d:cafe::70) by BN9PR03CA0588.outlook.office365.com (2603:10b6:408:10d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23 via Frontend Transport; Wed, 30 Nov 2022 11:52:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT051.mail.protection.outlook.com (10.13.177.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.22 via Frontend Transport; Wed, 30 Nov 2022 11:52:57 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:44 -0800 Received: from nps-server-23.mtl.labs.mlnx (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:40 -0800 From: Shay Drory To: , , CC: , , , , Subject: [PATCH net-next 3/8] devlink: Move devlink port function hw_addr attr documentation Date: Wed, 30 Nov 2022 13:52:12 +0200 Message-ID: <20221130115217.7171-4-shayd@nvidia.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221130115217.7171-1-shayd@nvidia.com> References: <20221130115217.7171-1-shayd@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT051:EE_|SA1PR12MB6702:EE_ X-MS-Office365-Filtering-Correlation-Id: 3e28c3e5-de01-470d-688e-08dad2c96c44 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IxExsXfjKXf8tWoP78lTBBcD0XQOJ+94m0of38znyMhyL6titvpk7cCx23Urzm8MjbENfB+JkGtF5rYMXFOO70mH6DTcb3WCHYY35tL6imWZ34Afd0+cLWWaUImkSaksFKt5ZuRqIsd0+j8ZdB0dGa3N/Tb2fdjLnrJ62HQ8smDO61PzYdM1PZTes/aVej70dNV5xnwJUiOYgc70Hno7TblctdEgF1DKtPy4lUFDS3OL0GJRQFqynki9dwECr1K3mrrorsaXapN01ShcwaqfLG6pDKZTMMUCR+uiQB242mgNvqhC3eXFM+iixWYWwwWpPsic6E3B1anes1kMXR4Ap0e1cNX5Klswl7ZtxiPmtFqaamAFKZUmUh+j8f9Xtk3gtmnAICl/ltWnqD6Se5KUJPlAwu9/OM+G0bkrI7pJKjX9Dqnwrup2wojs95aLK3wGGnywCwjtJL25UkslHqF8CA1a9965tOhz8XHP/8PVliVMyHiYAWtIONjGHf+ghyqdPKkj31BVmErBVIT6BTcvkOwr0gHLCaeh7Vgv10V+8DpPkPMKVG6T0kQ/UrvbfEckuaYMTLghDVFxPneSE6Tzjn3iCJD8XAdsQMRLMHuOjS7jaBbqredPOKKmD8LjXUP6ot2cee+iCuQVJsq/cktWiJEQZv32SknnQTHfu6IGddihIHpPNaTJmX1H9U5MqkNoUtrB5pL7E/lKIpZo28o07w== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(396003)(39860400002)(136003)(376002)(346002)(451199015)(46966006)(36840700001)(40470700004)(8676002)(41300700001)(70206006)(316002)(110136005)(54906003)(4326008)(5660300002)(8936002)(478600001)(426003)(40460700003)(336012)(26005)(186003)(107886003)(1076003)(2906002)(16526019)(82310400005)(6666004)(47076005)(70586007)(36860700001)(36756003)(82740400003)(2616005)(40480700001)(7636003)(86362001)(83380400001)(356005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2022 11:52:57.0997 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3e28c3e5-de01-470d-688e-08dad2c96c44 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT051.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6702 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org devlink port function hw_addr attr documentation is in mlx5 specific file while there is nothing mlx5 specific about it. Move it to devlink-port.rst. Signed-off-by: Shay Drory Reviewed-by: Jiri Pirko --- .../device_drivers/ethernet/mellanox/mlx5.rst | 38 +----------------- .../networking/devlink/devlink-port.rst | 40 +++++++++++++++++++ 2 files changed, 42 insertions(+), 36 deletions(-) diff --git a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5.rst b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5.rst index 5edf50d7dbd5..6ae3b35a17d5 100644 --- a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5.rst +++ b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5.rst @@ -351,42 +351,8 @@ driver. MAC address setup ----------------- -mlx5 driver provides mechanism to setup the MAC address of the PCI VF/SF. - -The configured MAC address of the PCI VF/SF will be used by netdevice and rdma -device created for the PCI VF/SF. - -- Get the MAC address of the VF identified by its unique devlink port index:: - - $ devlink port show pci/0000:06:00.0/2 - pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 - function: - hw_addr 00:00:00:00:00:00 - -- Set the MAC address of the VF identified by its unique devlink port index:: - - $ devlink port function set pci/0000:06:00.0/2 hw_addr 00:11:22:33:44:55 - - $ devlink port show pci/0000:06:00.0/2 - pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 - function: - hw_addr 00:11:22:33:44:55 - -- Get the MAC address of the SF identified by its unique devlink port index:: - - $ devlink port show pci/0000:06:00.0/32768 - pci/0000:06:00.0/32768: type eth netdev enp6s0pf0sf88 flavour pcisf pfnum 0 sfnum 88 - function: - hw_addr 00:00:00:00:00:00 - -- Set the MAC address of the VF identified by its unique devlink port index:: - - $ devlink port function set pci/0000:06:00.0/32768 hw_addr 00:00:00:00:88:88 - - $ devlink port show pci/0000:06:00.0/32768 - pci/0000:06:00.0/32768: type eth netdev enp6s0pf0sf88 flavour pcivf pfnum 0 sfnum 88 - function: - hw_addr 00:00:00:00:88:88 +mlx5 driver support devlink port function attr mechanism to setup MAC +address. (refer to Documentation/networking/devlink/devlink-port.rst) SF state setup -------------- diff --git a/Documentation/networking/devlink/devlink-port.rst b/Documentation/networking/devlink/devlink-port.rst index 98557c2ab1c1..0b520363c6af 100644 --- a/Documentation/networking/devlink/devlink-port.rst +++ b/Documentation/networking/devlink/devlink-port.rst @@ -122,6 +122,46 @@ A user may set the hardware address of the function using 'devlink port function set hw_addr' command. For Ethernet port function this means a MAC address. +Function attributes +=================== + +MAC address setup +----------------- +The configured MAC address of the PCI VF/SF will be used by netdevice and rdma +device created for the PCI VF/SF. + +- Get the MAC address of the VF identified by its unique devlink port index:: + + $ devlink port show pci/0000:06:00.0/2 + pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 + function: + hw_addr 00:00:00:00:00:00 + +- Set the MAC address of the VF identified by its unique devlink port index:: + + $ devlink port function set pci/0000:06:00.0/2 hw_addr 00:11:22:33:44:55 + + $ devlink port show pci/0000:06:00.0/2 + pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 + function: + hw_addr 00:11:22:33:44:55 + +- Get the MAC address of the SF identified by its unique devlink port index:: + + $ devlink port show pci/0000:06:00.0/32768 + pci/0000:06:00.0/32768: type eth netdev enp6s0pf0sf88 flavour pcisf pfnum 0 sfnum 88 + function: + hw_addr 00:00:00:00:00:00 + +- Set the MAC address of the VF identified by its unique devlink port index:: + + $ devlink port function set pci/0000:06:00.0/32768 hw_addr 00:00:00:00:88:88 + + $ devlink port show pci/0000:06:00.0/32768 + pci/0000:06:00.0/32768: type eth netdev enp6s0pf0sf88 flavour pcivf pfnum 0 sfnum 88 + function: + hw_addr 00:00:00:00:88:88 + Subfunction ============ From patchwork Wed Nov 30 11:52:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shay Drori X-Patchwork-Id: 13059745 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9A66C352A1 for ; Wed, 30 Nov 2022 11:54:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234371AbiK3LyA (ORCPT ); Wed, 30 Nov 2022 06:54:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43904 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235223AbiK3LxP (ORCPT ); Wed, 30 Nov 2022 06:53:15 -0500 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2052.outbound.protection.outlook.com [40.107.93.52]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 312A742983 for ; Wed, 30 Nov 2022 03:53:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mb25SWgjbl6bsXjpe9kuZU/4GsBo/zv111FOK1bW+Z4Y8K7IUKt81+eT+2qqtfMdXu8urmPyJfu+dYVgHYzFj+FCOVcYhckUB3vIe4LHlKPNGAInnYqxFsbN5ZTYy6l/GPxHXozXvHqBQ+kSelvTBNthfKIoRLUElXfwBZCiicAS4j8Kh7Xlb46qOEZy+OT+OvfHyaSQxL+qI7R1a6nvxQTw6OaxNuKg6xAhB1WWSIBphuWfzD3xlzDJQzPltnbXkT4mUHt0wczzTecdOy4+eaiP7sc7msrH/WW0Z8P9Y6AHdAalJo/mKL9NufGlO/0XTnDYmd2HlybLAczhsK+j+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iTK0rw6/TyJv2+ihI9Aw/f1x/BS+vQNhqWpeJCpL33o=; b=IxbvP8u15dA/6pU110L7dI6CNx7z2PMpvjBUmHSAm5CJbDx5/kdxV7BpsAsoZu6VIDYH9w36QkQx6iC8sT+TzcKeIWUBqPEJcJJJUiQ5ygq8fmpQAcOIkPmfnOBI4002hdCRv5XC4jM8IvkyIMyC7USruXtYKa2vLY+R+0Q8aZXcYiIW93D1qOTF4KKT3ybYKXX19+Tn4d9EZ+cglU4AtuKWmR/g3r/8lRvXNNtvXkvQ8UxRyhXuCSoc6LWoETu8TxHWD4uD01ZlrHQMucraD5ml4ae28SwJijCAiKNmpfuYLauQs4EDg8q5q2kaq/Z2zQgoFgR0a2qQLgg8+5SdGw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iTK0rw6/TyJv2+ihI9Aw/f1x/BS+vQNhqWpeJCpL33o=; b=ElbSwY9ZTosd181GGa4xCerGJsqK6asfMLzqzz+9eFGnyZY3E6y/14Z/w2PVzvQHpoac+SUtF5BBs8pnflSR5X9lQMY+SJN1ULlNCz6fNanciMJ59wPYcAOLKZz81cvErPZYlZqgbrVuqNdIGtJwyPQ0E+AMDjjZ2DueqPn9DzC4KSGijlnMy/4ls7D209UkAeOo3gmWf8VNPAoyOLXUmxd9uSPCNZ5q03AtkrHDlij0XiTqFphxhmeE1Ay4K038B+LjhGJYoILr2PPikJ8JNinlwKp/gVbArAdoxLgC0bhq7vhgqHvfbA8pX/Kp0QiZOeFU0cXL9zz2ar/E2dQ9yw== Received: from BN1PR13CA0025.namprd13.prod.outlook.com (2603:10b6:408:e2::30) by IA1PR12MB6556.namprd12.prod.outlook.com (2603:10b6:208:3a0::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23; Wed, 30 Nov 2022 11:53:00 +0000 Received: from BN8NAM11FT106.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e2:cafe::c0) by BN1PR13CA0025.outlook.office365.com (2603:10b6:408:e2::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5880.8 via Frontend Transport; Wed, 30 Nov 2022 11:53:00 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT106.mail.protection.outlook.com (10.13.177.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.22 via Frontend Transport; Wed, 30 Nov 2022 11:53:00 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:49 -0800 Received: from nps-server-23.mtl.labs.mlnx (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:45 -0800 From: Shay Drory To: , , CC: , , , , Subject: [PATCH net-next 4/8] devlink: Expose port function commands to control RoCE Date: Wed, 30 Nov 2022 13:52:13 +0200 Message-ID: <20221130115217.7171-5-shayd@nvidia.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221130115217.7171-1-shayd@nvidia.com> References: <20221130115217.7171-1-shayd@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT106:EE_|IA1PR12MB6556:EE_ X-MS-Office365-Filtering-Correlation-Id: c3614b9c-354a-441d-f060-08dad2c96e1b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: T9vUxto5JJ6cOoplSYMMWTnLi5RA93misNXWkjSfeXzIykxb3/v0hQnqxfNxOltGn00WWi0Qv6XusEsTCy2pux7OBeEsypzH4rdJ18ApgKPWOfMGsHVDUz/lmPnQeP2LTxrL7v8bUaXWdLn05UIMVES4+mAZ4lG1hasO6UV+s/R8xAtqTgNGinwoSHJX0TQSuqEMq4v3Qm7jpLeGuMzg+EjoXwBEggrIg+Lrn5W8QM48CVFBaiwW7FIhgTo9beHjGvshgPWohJgWYNOSo1BbcBJSYx80Ko1XlMGvTKaryiRQ4hGjhF2wM+hMJW1ekPsemMhI1wiBvwL4t2bPOqnaBuhh+2MTavsp0dL5HZMLt5V3sQ41zBc31mddGidmvOmIw5k8D4RThdSnifViumd/WW9UUBl9DR6cTDJz0sFkp1DvRuu365i3gQOXc2Bo3gzBeaauFkaCOcXHFQSUJhMRuEicQAwFIRYk/LFMm6FHuy48iq0DUj3NDWL5TLO1pu5egswJif6p9IfbTtxYH+60G+QSRNg/KxzG5oBJIJA4RQtruS6b5QW1m6PTTfYRNQ0F3fC04HMdzZHJOUuoXJZvWOoapYa3hqLkSwf+x6D+RUPOSVRs3Wwx7QQ0iI22TVrWANSdX4FJWcTPvvgEiuDsX9iP5lkQ7hC7DSsPSB+gXKJRdILXgdpjbGuR5MS5so8tLwGk1NAuc06hPh9Uo4UEkg== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(346002)(39860400002)(376002)(396003)(136003)(451199015)(46966006)(40470700004)(36840700001)(107886003)(336012)(2906002)(426003)(26005)(6666004)(478600001)(40460700003)(1076003)(82740400003)(86362001)(40480700001)(7636003)(356005)(83380400001)(186003)(16526019)(47076005)(2616005)(36756003)(36860700001)(82310400005)(41300700001)(8936002)(8676002)(70586007)(4326008)(5660300002)(110136005)(70206006)(54906003)(316002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2022 11:53:00.1899 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c3614b9c-354a-441d-f060-08dad2c96e1b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT106.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6556 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Expose port function commands to enable / disable RoCE, this is used to control the port RoCE device capabilities. When RoCE is disabled for a function of the port, function cannot create any RoCE specific resources (e.g GID table). It also saves system memory utilization. For example disabling RoCE enable a VF/SF saves 1 Mbytes of system memory per function. Example of a PCI VF port which supports function configuration: Set RoCE of the VF's port function. $ devlink port show pci/0000:06:00.0/2 pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 function: hw_addr 00:00:00:00:00:00 roce enable $ devlink port function set pci/0000:06:00.0/2 roce disable $ devlink port show pci/0000:06:00.0/2 pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 function: hw_addr 00:00:00:00:00:00 roce disable Signed-off-by: Shay Drory Reviewed-by: Jiri Pirko --- .../networking/devlink/devlink-port.rst | 34 +++++- include/net/devlink.h | 19 +++ include/uapi/linux/devlink.h | 12 ++ net/core/devlink.c | 114 ++++++++++++++++++ 4 files changed, 178 insertions(+), 1 deletion(-) diff --git a/Documentation/networking/devlink/devlink-port.rst b/Documentation/networking/devlink/devlink-port.rst index 0b520363c6af..79f9c0390b47 100644 --- a/Documentation/networking/devlink/devlink-port.rst +++ b/Documentation/networking/devlink/devlink-port.rst @@ -110,7 +110,7 @@ devlink ports for both the controllers. Function configuration ====================== -A user can configure the function attribute before enumerating the PCI +Users can configure one or more function attributes before enumerating the PCI function. Usually it means, user should configure function attribute before a bus specific device for the function is created. However, when SRIOV is enabled, virtual function devices are created on the PCI bus. @@ -122,6 +122,9 @@ A user may set the hardware address of the function using 'devlink port function set hw_addr' command. For Ethernet port function this means a MAC address. +Users may also set the RoCE capability of the function using +'devlink port function set roce' command. + Function attributes =================== @@ -162,6 +165,35 @@ device created for the PCI VF/SF. function: hw_addr 00:00:00:00:88:88 +RoCE capability setup +--------------------- +Not all PCI VFs/SFs require RoCE capability. + +When RoCE capability is disabled, it saves system memory per PCI VF/SF. + +When user disables RoCE capability for a VF/SF, user application cannot send or +receive any RoCE packets through this VF/SF and RoCE GID table for this PCI +will be empty. + +When RoCE capability is disabled in the device using port function attribute, +VF/SF driver cannot override it. + +- Get RoCE capability of the VF device:: + + $ devlink port show pci/0000:06:00.0/2 + pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 + function: + hw_addr 00:00:00:00:00:00 roce enable + +- Set RoCE capability of the VF device:: + + $ devlink port function set pci/0000:06:00.0/2 roce disable + + $ devlink port show pci/0000:06:00.0/2 + pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 + function: + hw_addr 00:00:00:00:00:00 roce disable + Subfunction ============ diff --git a/include/net/devlink.h b/include/net/devlink.h index 074a79b8933f..7f75100e8b26 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -1435,6 +1435,25 @@ struct devlink_ops { int (*port_function_hw_addr_set)(struct devlink_port *port, const u8 *hw_addr, int hw_addr_len, struct netlink_ext_ack *extack); + /** + * @port_function_roce_get: Port function's roce get function. + * + * Query RoCE state of a function managed by the devlink port. + * Return -EOPNOTSUPP if port function RoCE handling is not supported. + */ + int (*port_function_roce_get)(struct devlink_port *devlink_port, + bool *is_enable, + struct netlink_ext_ack *extack); + /** + * @port_function_roce_set: Port function's roce set function. + * + * Enable/Disable the RoCE state of a function managed by the devlink + * port. + * Return -EOPNOTSUPP if port function RoCE handling is not supported. + */ + int (*port_function_roce_set)(struct devlink_port *devlink_port, + bool enable, + struct netlink_ext_ack *extack); /** * port_new() - Add a new port function of a specified flavor * @devlink: Devlink instance diff --git a/include/uapi/linux/devlink.h b/include/uapi/linux/devlink.h index 498d0d5d0957..c6f1fbe54095 100644 --- a/include/uapi/linux/devlink.h +++ b/include/uapi/linux/devlink.h @@ -656,11 +656,23 @@ enum devlink_resource_unit { DEVLINK_RESOURCE_UNIT_ENTRY, }; +enum devlink_port_fn_attr_cap { + DEVLINK_PORT_FN_ATTR_CAP_ROCE, + + /* Add new caps above */ + __DEVLINK_PORT_FN_ATTR_CAPS_MAX, + DEVLINK_PORT_FN_ATTR_CAPS_MAX = __DEVLINK_PORT_FN_ATTR_CAPS_MAX - 1 +}; + +#define DEVLINK_PORT_FN_ATTR_CAPS_VALID_MASK \ + (_BITUL(__DEVLINK_PORT_FN_ATTR_CAPS_MAX) - 1) + enum devlink_port_function_attr { DEVLINK_PORT_FUNCTION_ATTR_UNSPEC, DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR, /* binary */ DEVLINK_PORT_FN_ATTR_STATE, /* u8 */ DEVLINK_PORT_FN_ATTR_OPSTATE, /* u8 */ + DEVLINK_PORT_FN_ATTR_CAPS, /* bitfield32 */ __DEVLINK_PORT_FUNCTION_ATTR_MAX, DEVLINK_PORT_FUNCTION_ATTR_MAX = __DEVLINK_PORT_FUNCTION_ATTR_MAX - 1 diff --git a/net/core/devlink.c b/net/core/devlink.c index 485348697290..88846ad635a0 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -200,6 +200,8 @@ static const struct nla_policy devlink_function_nl_policy[DEVLINK_PORT_FUNCTION_ [DEVLINK_PORT_FN_ATTR_STATE] = NLA_POLICY_RANGE(NLA_U8, DEVLINK_PORT_FN_STATE_INACTIVE, DEVLINK_PORT_FN_STATE_ACTIVE), + [DEVLINK_PORT_FN_ATTR_CAPS] = + NLA_POLICY_BITFIELD32(DEVLINK_PORT_FN_ATTR_CAPS_VALID_MASK), }; static const struct nla_policy devlink_selftest_nl_policy[DEVLINK_ATTR_SELFTEST_ID_MAX + 1] = { @@ -692,6 +694,64 @@ devlink_sb_tc_index_get_from_attrs(struct devlink_sb *devlink_sb, return 0; } +#define DEVLINK_PORT_FN_CAP(_name) \ + BIT(DEVLINK_PORT_FN_ATTR_CAP_##_name) + +#define DEVLINK_PORT_FN_SET_CAP(caps, cap, enable) \ + do { \ + typeof(cap) cap_ = (cap); \ + typeof(caps) caps_ = (caps); \ + (caps_)->selector |= cap_; \ + if (enable) \ + (caps_)->value |= cap_; \ + } while (0) + +static int devlink_port_fn_roce_fill(const struct devlink_ops *ops, + struct devlink_port *devlink_port, + struct nla_bitfield32 *caps, + struct netlink_ext_ack *extack) +{ + bool is_enable; + int err; + + if (!ops->port_function_roce_get) + return 0; + + err = ops->port_function_roce_get(devlink_port, &is_enable, extack); + if (err) { + if (err == -EOPNOTSUPP) + return 0; + return err; + } + + DEVLINK_PORT_FN_SET_CAP(caps, DEVLINK_PORT_FN_CAP(ROCE), is_enable); + return 0; +} + +static int devlink_port_fn_caps_fill(const struct devlink_ops *ops, + struct devlink_port *devlink_port, + struct sk_buff *msg, + struct netlink_ext_ack *extack, + bool *msg_updated) +{ + struct nla_bitfield32 caps = {}; + int err; + + err = devlink_port_fn_roce_fill(ops, devlink_port, &caps, extack); + if (err) + return err; + + if (!caps.selector) + return 0; + err = nla_put_bitfield32(msg, DEVLINK_PORT_FN_ATTR_CAPS, caps.value, + caps.selector); + if (err) + return err; + + *msg_updated = true; + return 0; +} + static int devlink_sb_tc_index_get_from_info(struct devlink_sb *devlink_sb, struct genl_info *info, @@ -1275,6 +1335,35 @@ static int devlink_port_fn_state_fill(const struct devlink_ops *ops, return 0; } +static int +devlink_port_fn_roce_set(struct devlink_port *devlink_port, bool enable, + struct netlink_ext_ack *extack) +{ + const struct devlink_ops *ops = devlink_port->devlink->ops; + + return ops->port_function_roce_set(devlink_port, enable, extack); +} + +static int devlink_port_fn_caps_set(struct devlink_port *devlink_port, + const struct nlattr *attr, + struct netlink_ext_ack *extack) +{ + struct nla_bitfield32 caps; + u32 caps_value; + int err; + + caps = nla_get_bitfield32(attr); + caps_value = caps.value & caps.selector; + if (caps.selector & DEVLINK_PORT_FN_CAP(ROCE)) { + err = devlink_port_fn_roce_set(devlink_port, + caps_value & DEVLINK_PORT_FN_CAP(ROCE), + extack); + if (err) + return err; + } + return 0; +} + static int devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port *port, struct netlink_ext_ack *extack) @@ -1293,6 +1382,10 @@ devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port *por &msg_updated); if (err) goto out; + err = devlink_port_fn_caps_fill(ops, port, msg, extack, + &msg_updated); + if (err) + goto out; err = devlink_port_fn_state_fill(ops, port, msg, extack, &msg_updated); out: if (err || !msg_updated) @@ -1665,6 +1758,7 @@ static int devlink_port_function_validate(struct devlink_port *devlink_port, struct netlink_ext_ack *extack) { const struct devlink_ops *ops = devlink_port->devlink->ops; + struct nlattr *attr; if (tb[DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR] && !ops->port_function_hw_addr_set) { @@ -1676,6 +1770,18 @@ static int devlink_port_function_validate(struct devlink_port *devlink_port, "Function does not support state setting"); return -EOPNOTSUPP; } + attr = tb[DEVLINK_PORT_FN_ATTR_CAPS]; + if (attr) { + struct nla_bitfield32 caps; + + caps = nla_get_bitfield32(attr); + if (caps.selector & DEVLINK_PORT_FN_CAP(ROCE) && + !ops->port_function_roce_set) { + NL_SET_ERR_MSG_ATTR(extack, attr, + "Port doesn't support RoCE function attribute"); + return -EOPNOTSUPP; + } + } return 0; } @@ -1703,6 +1809,14 @@ static int devlink_port_function_set(struct devlink_port *port, if (err) return err; } + + attr = tb[DEVLINK_PORT_FN_ATTR_CAPS]; + if (attr) { + err = devlink_port_fn_caps_set(port, attr, extack); + if (err) + return err; + } + /* Keep this as the last function attribute set, so that when * multiple port function attributes are set along with state, * Those can be applied first before activating the state. From patchwork Wed Nov 30 11:52:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shay Drori X-Patchwork-Id: 13059748 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D393FC433FE for ; Wed, 30 Nov 2022 11:54:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234741AbiK3LyM (ORCPT ); Wed, 30 Nov 2022 06:54:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235255AbiK3LxV (ORCPT ); Wed, 30 Nov 2022 06:53:21 -0500 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2050.outbound.protection.outlook.com [40.107.96.50]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 59674442DF for ; Wed, 30 Nov 2022 03:53:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Yy7w12bDQ8Hbgi4L8s2SpbnhHEHWSlY8ERQg8Y/xnbCGmYIexCiiXiobVMIfdSYuaCPqGd8JZEFeNua+WSvESIGC7hRfB8ziPkz23KZLJLRnYitSoDgyO95uxshv175HOboHYx2qdtcd1f9X3w5/2GTsriMue14rIfkXUXQrgX2mB+1ULgI8oswZhbIzZd7nsGGfFcW7woyZ4CYHpHxRnPiYzKaZPQRJaVFchZx+0u9ZtBmC30Icepf4yLI7m10SY6IIexM7XKf/YFRefHuXzNAEQA7XM44bzkwVW+zbCdSJ91946mIZq5I59qbCFk3lW7T6vyZ5pmpgKQsin9+6Tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NCwTM8oskniKNVR3KEcszO+njYiz0yJPt0QZhnV1sWA=; b=QivaArRpUV5VpTKkhnOqCXnAgRVgIaVc8QOQeywh13id0tIT0HxVSJ1WfwtO/0QtNVip/UDdZq2IMQN8PZLymY77Ikkg3YssyGYth7dJJZ8SKumhto3feSVk4CZVwQhOW0RusVGU7fwY5rxv0/Z9CGTmarUG0CGVP9ad+QTWdKfpf123py6a3hSZuqnztm2LzUGZJa1VGZMP2FN6VMbhmeLZqlpFkoMJod6u2Onj1EIo1t2imdbACLRNaPGQF9EzP0Bu1qTlniTBJmJswPahop4UfwG/gmdhIjNeLmzYBQJe+pikg6EaSEs+3/4DSjhM558bZ4P2CxoIVDzu9JmlGQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NCwTM8oskniKNVR3KEcszO+njYiz0yJPt0QZhnV1sWA=; b=AofmCNDMvXojHtP5eHXBxodDbYtLAXmAfMwren070lJGd/pmx+J775TiuZPWOqQg4gIzTqYmZOQZ4ERIanwbdmCzdrMuUbvrN/2Cly6SSQXw7KRJn4lT8HGcwDRrKEbgxJlMDQ11V4oZvQmCtJjWEPOtctx6xCzVXlgl7ycxc1X4kSxoAJr90QbkonuoJQ1MrDmUA4kQ9oEbG5A5xF3eI2GjpnxDnwkdnOnSqSF+y1YyywYl8o9gQVEsAeYAypePxS7R9Wm2guOEx9+wtmpmjjVVYaZp1AcG64N/2MDlH0rBPnj2XmguT0Tk1orLsiOz312+P9JeK1QFF+Yc8dE1zQ== Received: from BN9PR03CA0436.namprd03.prod.outlook.com (2603:10b6:408:113::21) by PH7PR12MB7380.namprd12.prod.outlook.com (2603:10b6:510:20f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23; Wed, 30 Nov 2022 11:53:09 +0000 Received: from BN8NAM11FT085.eop-nam11.prod.protection.outlook.com (2603:10b6:408:113:cafe::35) by BN9PR03CA0436.outlook.office365.com (2603:10b6:408:113::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23 via Frontend Transport; Wed, 30 Nov 2022 11:53:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT085.mail.protection.outlook.com (10.13.176.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.17 via Frontend Transport; Wed, 30 Nov 2022 11:53:08 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:53 -0800 Received: from nps-server-23.mtl.labs.mlnx (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:49 -0800 From: Shay Drory To: , , CC: , , , , Subject: [PATCH net-next 5/8] net/mlx5: Add generic getters for other functions caps Date: Wed, 30 Nov 2022 13:52:14 +0200 Message-ID: <20221130115217.7171-6-shayd@nvidia.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221130115217.7171-1-shayd@nvidia.com> References: <20221130115217.7171-1-shayd@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT085:EE_|PH7PR12MB7380:EE_ X-MS-Office365-Filtering-Correlation-Id: ef387fa4-4687-4f95-dd7c-08dad2c97349 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yuAGMRS35CrESRoIJX1MvuSK6oIX9sVRUJlre2joM3ievAqxyYlV69zsTA7v13TyvizpbgLdVrjdqY6AF7eF1FEbEot591lsOckmn8VNPUZRgCoHBaR9fcBNQgus2OVLL8hCfH5OPsli1dn4XlUm7v4UFYNZ4cNsVujCwg5tl0G19mS4mdnAI7HZFiL1qkA0lHn7YzM02gSMcbcVOJBrXCTlxkyrpNFqKdL7OYJgZQD7ir/gI3qp0x1Gzqp+xo/AYN/RMkdM2EelStuu+AkCo8zc2qbpuqSytv6K0F5ucpJ92iIyacWBPCnnwKHtWZUIjfwDWTQqDzj+fG1I2DSKJsPravIARot7VQanOizwnC7CBi+tYumAnVk+QP8Ct3LWWbmM1DB14VaIgi9GU3xO4GCavUv1yZNkfSqaOsafhSBy8XIvOJf71gUVmkT601sThY9eoiS4bQV613kU3KYR7RlZD8R7htoJ/+h1vCDG+Xj6mvF7kQ8omnLnZxZo1f7t5dcDSlTrWyBckNgq86wxelfCtD5lHMpNhW0K+KCOXeUcQFP7L4giQi2DUOprSbvzHT4bbixW9+t5pem7NcaKbyi3HXiAJtsduR87V9z9CqAgOxlvKgteDwn9/wvjYEtoK3ELjAiKtClaQHHq8sGYAQX9mRU1xcWKIcaqbBWp9hbl33VxbuzMy8TcmNvEnaSfmNBbm0PCKgKeQdwB2gYgUbe8+mCIlW1bfw9tNKL356o= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(376002)(136003)(396003)(39860400002)(346002)(451199015)(46966006)(36840700001)(40470700004)(5660300002)(8936002)(40460700003)(83380400001)(26005)(41300700001)(2906002)(4326008)(8676002)(70586007)(478600001)(54906003)(316002)(82310400005)(36756003)(6666004)(107886003)(86362001)(186003)(16526019)(47076005)(40480700001)(7636003)(336012)(82740400003)(356005)(110136005)(1076003)(2616005)(426003)(70206006)(36860700001)(41533002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2022 11:53:08.8639 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ef387fa4-4687-4f95-dd7c-08dad2c97349 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT085.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7380 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Downstream patch requires to get other function GENERAL2 caps while mlx5_vport_get_other_func_cap() gets only one type of caps (general). Rename it to represent this and introduce a generic implementation of mlx5_vport_get_other_func_cap(). Signed-off-by: Shay Drory Reviewed-by: Mark Bloch Ack-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c | 2 +- drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h | 3 ++- drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c | 3 ++- drivers/net/ethernet/mellanox/mlx5/core/vport.c | 6 ++++-- include/linux/mlx5/vport.h | 2 ++ 5 files changed, 11 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index 728ca9f2bb9d..40af64d31f65 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -3879,7 +3879,7 @@ static int mlx5_esw_query_vport_vhca_id(struct mlx5_eswitch *esw, u16 vport_num, if (!query_ctx) return -ENOMEM; - err = mlx5_vport_get_other_func_cap(esw->dev, vport_num, query_ctx); + err = mlx5_vport_get_other_func_general_cap(esw->dev, vport_num, query_ctx); if (err) goto out_free; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h index a806e3de7b7c..09473983778f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h @@ -324,7 +324,8 @@ void mlx5_unload_one_devl_locked(struct mlx5_core_dev *dev); int mlx5_load_one(struct mlx5_core_dev *dev, bool recovery); int mlx5_load_one_devl_locked(struct mlx5_core_dev *dev, bool recovery); -int mlx5_vport_get_other_func_cap(struct mlx5_core_dev *dev, u16 function_id, void *out); +#define mlx5_vport_get_other_func_general_cap(dev, fid, out) \ + mlx5_vport_get_other_func_cap(dev, fid, out, MLX5_CAP_GENERAL) void mlx5_events_work_enqueue(struct mlx5_core_dev *dev, struct work_struct *work); static inline u32 mlx5_sriov_get_vf_total_msix(struct pci_dev *pdev) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c index 662f1d55e30e..6bde18bcd42f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c @@ -4,6 +4,7 @@ #include #include #include +#include #include "mlx5_core.h" #include "mlx5_irq.h" #include "pci_irq.h" @@ -101,7 +102,7 @@ int mlx5_set_msix_vec_count(struct mlx5_core_dev *dev, int function_id, goto out; } - ret = mlx5_vport_get_other_func_cap(dev, function_id, query_cap); + ret = mlx5_vport_get_other_func_general_cap(dev, function_id, query_cap); if (ret) goto out; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vport.c b/drivers/net/ethernet/mellanox/mlx5/core/vport.c index d5c317325030..7eca7582f243 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/vport.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/vport.c @@ -1160,14 +1160,16 @@ u64 mlx5_query_nic_system_image_guid(struct mlx5_core_dev *mdev) } EXPORT_SYMBOL_GPL(mlx5_query_nic_system_image_guid); -int mlx5_vport_get_other_func_cap(struct mlx5_core_dev *dev, u16 function_id, void *out) +int mlx5_vport_get_other_func_cap(struct mlx5_core_dev *dev, u16 function_id, void *out, + u16 opmod) { - u16 opmod = (MLX5_CAP_GENERAL << 1) | (HCA_CAP_OPMOD_GET_MAX & 0x01); u8 in[MLX5_ST_SZ_BYTES(query_hca_cap_in)] = {}; + opmod = (opmod << 1) | (HCA_CAP_OPMOD_GET_MAX & 0x01); MLX5_SET(query_hca_cap_in, in, opcode, MLX5_CMD_OP_QUERY_HCA_CAP); MLX5_SET(query_hca_cap_in, in, op_mod, opmod); MLX5_SET(query_hca_cap_in, in, function_id, function_id); MLX5_SET(query_hca_cap_in, in, other_function, true); return mlx5_cmd_exec_inout(dev, query_hca_cap, in, out); } +EXPORT_SYMBOL_GPL(mlx5_vport_get_other_func_cap); diff --git a/include/linux/mlx5/vport.h b/include/linux/mlx5/vport.h index aad53cb72f17..7f31432f44c2 100644 --- a/include/linux/mlx5/vport.h +++ b/include/linux/mlx5/vport.h @@ -132,4 +132,6 @@ int mlx5_nic_vport_affiliate_multiport(struct mlx5_core_dev *master_mdev, int mlx5_nic_vport_unaffiliate_multiport(struct mlx5_core_dev *port_mdev); u64 mlx5_query_nic_system_image_guid(struct mlx5_core_dev *mdev); +int mlx5_vport_get_other_func_cap(struct mlx5_core_dev *dev, u16 function_id, void *out, + u16 opmod); #endif /* __MLX5_VPORT_H__ */ From patchwork Wed Nov 30 11:52:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shay Drori X-Patchwork-Id: 13059747 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33699C433FE for ; Wed, 30 Nov 2022 11:54:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234450AbiK3LyK (ORCPT ); Wed, 30 Nov 2022 06:54:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43946 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234801AbiK3LxV (ORCPT ); Wed, 30 Nov 2022 06:53:21 -0500 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2071.outbound.protection.outlook.com [40.107.92.71]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE4B4C1B for ; Wed, 30 Nov 2022 03:53:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=obWTmNDtbHPwxPidmJ2U5FPCkTbjPbiMM8S0iYASfNyM/UijcWxuuUjf1KMhWHEHyBX4eB2/IYJnrXZY9Iy6olZqyezQPMM3jezbPJqdEWkhSheyUMZCXFOWFWXHy8nVUK2AIpxIdiU9LSeHxRid0xb2RgzXxsnp1K17TU69UhgTzI4qcdt8W1fZYm2AlTtlIIMx3h9uWi46RIKsN63AL7TmWQL3Q7EQoOWHqw/Slqi6gf81bL/LNb1D06c9ahQYyleTqNGxnpYs36mrP4q8zkoAvwQl/7Zjy+451GPJaIIasKDKF6w///ZWGJcMbTD9zBH+9/2zc5hRUylbQrMwqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9tGw/3T0iRwb0mhdH8pqrUCDBCaO887x14KljyHjZnE=; b=kAUgFP2ZRWDcL/LDTD+6G9GRt8DWaHgOWh9uZSaX2KAJwopazFzL/y6mMZ/C+0S6izL1OMomsPvpHTUKO459hPHbQdAjCt7/yXAyFjEQM8+pRGkXBktjBSz195ZTjgl/zYPg3SCJbVMDC1QQZ5OvTNpYpVYOlDZDFqm6jnKpGOYDeqgTP2pEY62LPKWCaDdDmbW9IVhqWBIvg2d2/cGNWWzvQTUvo02kVVszhTFvOv4RgNgTSi0xBrtvC9t6O/YIyc8sgVebu6tW4fYpr9GIjv7AUrkMKHeHTJgxEL1LGs3xfhcI1WTR6iU8Y709WtEgCuZNIx+8dqiFMN9Fwa+K3g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9tGw/3T0iRwb0mhdH8pqrUCDBCaO887x14KljyHjZnE=; b=bwopKPN6DieSS/Yhhp3pnKfHnIhRDvGjjROXPLDqWpwTdYu9TLpCV0oCodW3tmJH/fQKskiV32IVhkKeWaaoSDv/oGpW1ziThTNz4qQCFy9IhAWaJmgdSayuy9au60sE2NG05SrlJa7XBWI75TSD6uDVTz+rL2lGhgc/qzNxo7aXu8gKn8yZUmiAsHItr59gygnl7Mm5aYtfyKTQusykKdfjCF1JAXYWPbWvJk3tQimLBc0+7fkc6sct5FxL5FeifH4+ponzJNN5lKPuLNq8nF6PFbOWGLYYLyk5ZeCQUbA5SQm4yq3z+cwsAANW4zGVJthSPtJO768BYIdI00UPdw== Received: from BN0PR04CA0119.namprd04.prod.outlook.com (2603:10b6:408:ec::34) by DM4PR12MB5987.namprd12.prod.outlook.com (2603:10b6:8:6a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23; Wed, 30 Nov 2022 11:53:11 +0000 Received: from BN8NAM11FT074.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ec:cafe::71) by BN0PR04CA0119.outlook.office365.com (2603:10b6:408:ec::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23 via Frontend Transport; Wed, 30 Nov 2022 11:53:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN8NAM11FT074.mail.protection.outlook.com (10.13.176.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.8 via Frontend Transport; Wed, 30 Nov 2022 11:53:10 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:57 -0800 Received: from nps-server-23.mtl.labs.mlnx (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:53 -0800 From: Shay Drory To: , , CC: , , , , Subject: [PATCH net-next 6/8] net/mlx5: E-Switch, Implement devlink port function cmds to control RoCE Date: Wed, 30 Nov 2022 13:52:15 +0200 Message-ID: <20221130115217.7171-7-shayd@nvidia.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221130115217.7171-1-shayd@nvidia.com> References: <20221130115217.7171-1-shayd@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT074:EE_|DM4PR12MB5987:EE_ X-MS-Office365-Filtering-Correlation-Id: 2f32b3a7-bf5b-4a89-1321-08dad2c9743d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: C3z/VcLYOuTNPIFzIor0hkFlXTxJs2ANVcmDk+leFTRWcMvMC3YTPqq6f9rxp73BNAJH4aLve2B83ua0i6CjQyBi/Xc/XXmnRStH3gEfLuvVatXYrFAFeN66wUN1hWNAS25elciEdiBTS6myUeRrcThpZsy0JyQw8PHOx9Jyew0gl87pYNFVrtxyNS3BWQUPUAACBTSFH38WLwlt5Z35urL8JTdwgGSSsbnQ/6Jf9Xnk6dTyUzAPE3QF7xTnDUWw7Xwq7HNq1tCNZxJE4p3ru51rYSpdKHScWDPyY92fvG4DhLR86eokzgjzWzStHifJqDuMVYLXM28JoS64o9jto/4emvBobmBMmXM19+XlqjbD2JJjayjg+/pETm1DdLJ+nrabw3Br8bM3i0uKtXjzTa1RHAiERwAYHs7cslJAAhYLNGYiohkt2O0DgvLe616/4pMusfhzN9P0sBLV5qU4LjqDcjaJ3gE0ZqecmCKxkmlQmBUyHZxdDQQFtGkj9RoofJCTk+HbpEwNQmZnnSm41//aTP3YFB7+Ls+FmJmLAS+8SoieMaaW0dEL7R9iCcxB+ocThgwm3LpTnau56yn5DnvLfj84RF+0uRXcNmRaOHlsgivjvDGyRn+wot1znaYXzniMqieRUE2V0PMm936Hbk5TQxLZbw7shnEflIEmMxiZsLqPdjiEPlBLhDo7H2aoyEbAuQXJZCvVdZRM9UQC+A== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(396003)(39860400002)(136003)(376002)(346002)(451199015)(36840700001)(40470700004)(46966006)(2906002)(107886003)(2616005)(186003)(1076003)(16526019)(426003)(26005)(478600001)(6666004)(336012)(36756003)(40460700003)(82740400003)(40480700001)(7636003)(356005)(86362001)(83380400001)(47076005)(82310400005)(36860700001)(5660300002)(70206006)(70586007)(30864003)(8936002)(110136005)(316002)(8676002)(54906003)(41300700001)(4326008);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2022 11:53:10.4734 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2f32b3a7-bf5b-4a89-1321-08dad2c9743d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT074.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5987 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Yishai Hadas Implement devlink port function commands to enable / disable RoCE. This is used to control the RoCE device capabilities. This patch implement infrastructure which will be used by downstream patches that will add additional capabilities. Signed-off-by: Yishai Hadas Signed-off-by: Daniel Jurgens Signed-off-by: Shay Drory Reviewed-by: Parav Pandit Reviewed-by: Mark Bloch Ack-by: Saeed Mahameed --- .../device_drivers/ethernet/mellanox/mlx5.rst | 10 ++ .../net/ethernet/mellanox/mlx5/core/devlink.c | 2 + .../net/ethernet/mellanox/mlx5/core/eswitch.c | 35 ++++++ .../net/ethernet/mellanox/mlx5/core/eswitch.h | 6 +- .../mellanox/mlx5/core/eswitch_offloads.c | 108 ++++++++++++++++++ .../ethernet/mellanox/mlx5/core/mlx5_core.h | 2 + .../net/ethernet/mellanox/mlx5/core/vport.c | 24 ++++ 7 files changed, 186 insertions(+), 1 deletion(-) diff --git a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5.rst b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5.rst index 6ae3b35a17d5..992e3d2830ad 100644 --- a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5.rst +++ b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5.rst @@ -354,6 +354,16 @@ MAC address setup mlx5 driver support devlink port function attr mechanism to setup MAC address. (refer to Documentation/networking/devlink/devlink-port.rst) +RoCE capability setup +--------------------- +Not all mlx5 PCI devices/SFs require RoCE capability. + +When RoCE capability is disabled, it saves 1 Mbytes worth of system memory per +PCI devices/SF. + +mlx5 driver support devlink port function attr mechanism to setup RoCE +capability. (refer to Documentation/networking/devlink/devlink-port.rst) + SF state setup -------------- To use the SF, the user must active the SF using the SF function state diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index cc2ae427dcb0..6b0685d9260b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -318,6 +318,8 @@ static const struct devlink_ops mlx5_devlink_ops = { .rate_node_new = mlx5_esw_devlink_rate_node_new, .rate_node_del = mlx5_esw_devlink_rate_node_del, .rate_leaf_parent_set = mlx5_esw_devlink_rate_parent_set, + .port_function_roce_get = mlx5_devlink_port_function_roce_get, + .port_function_roce_set = mlx5_devlink_port_function_roce_set, #endif #ifdef CONFIG_MLX5_SF_MANAGER .port_new = mlx5_devlink_sf_port_new, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c index 2169486c4bfb..66d434dafb0b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c @@ -772,6 +772,33 @@ static void esw_vport_cleanup_acl(struct mlx5_eswitch *esw, esw_vport_destroy_offloads_acl_tables(esw, vport); } +static int mlx5_esw_vport_caps_get(struct mlx5_eswitch *esw, struct mlx5_vport *vport) +{ + int query_out_sz = MLX5_ST_SZ_BYTES(query_hca_cap_out); + void *query_ctx; + void *hca_caps; + int err; + + if (!MLX5_CAP_GEN(esw->dev, vhca_resource_manager)) + return 0; + + query_ctx = kzalloc(query_out_sz, GFP_KERNEL); + if (!query_ctx) + return -ENOMEM; + + err = mlx5_vport_get_other_func_cap(esw->dev, vport->vport, query_ctx, + MLX5_CAP_GENERAL); + if (err) + goto out_free; + + hca_caps = MLX5_ADDR_OF(query_hca_cap_out, query_ctx, capability); + vport->info.roce_enabled = MLX5_GET(cmd_hca_cap, hca_caps, roce); + +out_free: + kfree(query_ctx); + return err; +} + static int esw_vport_setup(struct mlx5_eswitch *esw, struct mlx5_vport *vport) { u16 vport_num = vport->vport; @@ -785,6 +812,10 @@ static int esw_vport_setup(struct mlx5_eswitch *esw, struct mlx5_vport *vport) if (mlx5_esw_is_manager_vport(esw, vport_num)) return 0; + err = mlx5_esw_vport_caps_get(esw, vport); + if (err) + goto err_caps; + mlx5_modify_vport_admin_state(esw->dev, MLX5_VPORT_STATE_OP_MOD_ESW_VPORT, vport_num, 1, @@ -804,6 +835,10 @@ static int esw_vport_setup(struct mlx5_eswitch *esw, struct mlx5_vport *vport) vport->info.qos, flags); return 0; + +err_caps: + esw_vport_cleanup_acl(esw, vport); + return err; } /* Don't cleanup vport->info, it's needed to restore vport configuration */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h index f68dc2d0dbe6..ef0dfe8e55d6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h @@ -153,6 +153,7 @@ struct mlx5_vport_info { u8 qos; u8 spoofchk: 1; u8 trusted: 1; + u8 roce_enabled: 1; }; /* Vport context events */ @@ -508,7 +509,10 @@ int mlx5_devlink_port_function_hw_addr_get(struct devlink_port *port, int mlx5_devlink_port_function_hw_addr_set(struct devlink_port *port, const u8 *hw_addr, int hw_addr_len, struct netlink_ext_ack *extack); - +int mlx5_devlink_port_function_roce_get(struct devlink_port *port, bool *is_enabled, + struct netlink_ext_ack *extack); +int mlx5_devlink_port_function_roce_set(struct devlink_port *port, bool enable, + struct netlink_ext_ack *extack); void *mlx5_eswitch_get_uplink_priv(struct mlx5_eswitch *esw, u8 rep_type); int mlx5_eswitch_add_vlan_action(struct mlx5_eswitch *esw, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index 40af64d31f65..fe1d0f9e723e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -4012,3 +4012,111 @@ int mlx5_devlink_port_function_hw_addr_set(struct devlink_port *port, return mlx5_eswitch_set_vport_mac(esw, vport_num, hw_addr); } + +static struct mlx5_vport * +mlx5_devlink_port_function_get_vport(struct devlink_port *port, struct mlx5_eswitch *esw) +{ + u16 vport_num; + + if (!MLX5_CAP_GEN(esw->dev, vhca_resource_manager)) + return ERR_PTR(-EOPNOTSUPP); + + vport_num = mlx5_esw_devlink_port_index_to_vport_num(port->index); + if (!is_port_function_supported(esw, vport_num)) + return ERR_PTR(-EOPNOTSUPP); + + return mlx5_eswitch_get_vport(esw, vport_num); +} + +int mlx5_devlink_port_function_roce_get(struct devlink_port *port, bool *is_enabled, + struct netlink_ext_ack *extack) +{ + struct mlx5_eswitch *esw; + struct mlx5_vport *vport; + int err = -EOPNOTSUPP; + + esw = mlx5_devlink_eswitch_get(port->devlink); + if (IS_ERR(esw)) + return PTR_ERR(esw); + + vport = mlx5_devlink_port_function_get_vport(port, esw); + if (IS_ERR(vport)) { + NL_SET_ERR_MSG_MOD(extack, "Invalid port"); + return PTR_ERR(vport); + } + + mutex_lock(&esw->state_lock); + if (vport->enabled) { + *is_enabled = vport->info.roce_enabled; + err = 0; + } + mutex_unlock(&esw->state_lock); + return err; +} + +int mlx5_devlink_port_function_roce_set(struct devlink_port *port, bool enable, + struct netlink_ext_ack *extack) +{ + int query_out_sz = MLX5_ST_SZ_BYTES(query_hca_cap_out); + struct mlx5_eswitch *esw; + struct mlx5_vport *vport; + int err = -EOPNOTSUPP; + void *query_ctx; + void *hca_caps; + u16 vport_num; + + esw = mlx5_devlink_eswitch_get(port->devlink); + if (IS_ERR(esw)) + return PTR_ERR(esw); + + vport = mlx5_devlink_port_function_get_vport(port, esw); + if (IS_ERR(vport)) { + NL_SET_ERR_MSG_MOD(extack, "Invalid port"); + return PTR_ERR(vport); + } + vport_num = vport->vport; + + mutex_lock(&esw->state_lock); + if (!vport->enabled) { + NL_SET_ERR_MSG_MOD(extack, "Eswitch vport is disabled"); + goto out; + } + + if (vport->info.roce_enabled == enable) { + err = 0; + goto out; + } + + query_ctx = kzalloc(query_out_sz, GFP_KERNEL); + if (!query_ctx) { + err = -ENOMEM; + goto out; + } + + err = mlx5_vport_get_other_func_cap(esw->dev, vport_num, query_ctx, + MLX5_CAP_GENERAL); + if (err) { + NL_SET_ERR_MSG_MOD(extack, "Failed getting HCA caps"); + goto out_free; + } + + hca_caps = MLX5_ADDR_OF(query_hca_cap_out, query_ctx, capability); + memcpy(hca_caps, MLX5_ADDR_OF(query_hca_cap_out, query_ctx, capability), + MLX5_UN_SZ_BYTES(hca_cap_union)); + MLX5_SET(cmd_hca_cap, hca_caps, roce, enable); + + err = mlx5_vport_set_other_func_cap(esw->dev, hca_caps, vport_num, + MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE); + if (err) { + NL_SET_ERR_MSG_MOD(extack, "Failed setting HCA roce cap"); + goto out_free; + } + + vport->info.roce_enabled = enable; + +out_free: + kfree(query_ctx); +out: + mutex_unlock(&esw->state_lock); + return err; +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h index 09473983778f..029305a8b80a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h @@ -324,6 +324,8 @@ void mlx5_unload_one_devl_locked(struct mlx5_core_dev *dev); int mlx5_load_one(struct mlx5_core_dev *dev, bool recovery); int mlx5_load_one_devl_locked(struct mlx5_core_dev *dev, bool recovery); +int mlx5_vport_set_other_func_cap(struct mlx5_core_dev *dev, const void *hca_cap, u16 function_id, + u16 opmod); #define mlx5_vport_get_other_func_general_cap(dev, fid, out) \ mlx5_vport_get_other_func_cap(dev, fid, out, MLX5_CAP_GENERAL) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vport.c b/drivers/net/ethernet/mellanox/mlx5/core/vport.c index 7eca7582f243..ba7e3df22413 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/vport.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/vport.c @@ -1173,3 +1173,27 @@ int mlx5_vport_get_other_func_cap(struct mlx5_core_dev *dev, u16 function_id, vo return mlx5_cmd_exec_inout(dev, query_hca_cap, in, out); } EXPORT_SYMBOL_GPL(mlx5_vport_get_other_func_cap); + +int mlx5_vport_set_other_func_cap(struct mlx5_core_dev *dev, const void *hca_cap, + u16 function_id, u16 opmod) +{ + int set_sz = MLX5_ST_SZ_BYTES(set_hca_cap_in); + void *set_hca_cap; + void *set_ctx; + int ret; + + set_ctx = kzalloc(set_sz, GFP_KERNEL); + if (!set_ctx) + return -ENOMEM; + + MLX5_SET(set_hca_cap_in, set_ctx, opcode, MLX5_CMD_OP_SET_HCA_CAP); + MLX5_SET(set_hca_cap_in, set_ctx, op_mod, opmod << 1); + set_hca_cap = MLX5_ADDR_OF(set_hca_cap_in, set_ctx, capability); + memcpy(set_hca_cap, hca_cap, MLX5_ST_SZ_BYTES(cmd_hca_cap)); + MLX5_SET(set_hca_cap_in, set_ctx, function_id, function_id); + MLX5_SET(set_hca_cap_in, set_ctx, other_function, true); + ret = mlx5_cmd_exec_in(dev, set_hca_cap, set_ctx); + + kfree(set_ctx); + return ret; +} From patchwork Wed Nov 30 11:52:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shay Drori X-Patchwork-Id: 13059746 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A4A1C352A1 for ; Wed, 30 Nov 2022 11:54:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234230AbiK3LyJ (ORCPT ); Wed, 30 Nov 2022 06:54:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234800AbiK3LxV (ORCPT ); Wed, 30 Nov 2022 06:53:21 -0500 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2054.outbound.protection.outlook.com [40.107.100.54]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13DFBD54 for ; Wed, 30 Nov 2022 03:53:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jLWyyblvvhHnOzRILz+8O6Zsojg0+lZxlszvqnrvCubPfwWtZj5bySFkTmLaTCztg3TsUbjwYwNe4Jyqmz3hLgVDo+6lGApMOReUGjUmKFbMu8yFEMsqJWEGkPqLP+736mA0ZkNQOTrwShZA49Z4DX9+D4OaZCG7BA1XmHLxjO87gl8fzxJ8RNfJ54rcY3xWifyGnRqowlFvq7N381hSFc7ubsss7kIqa2DkNI5Kb0l/hvxhz7s8LSq9OHDut3WDnOB7aWF/eDu7n4/lGYvmuYSkORtw5IZkHFb5xlW2s4oXavFdcz4mY8Lnm68WSpONuI6qjCJ/A7ZK8xdCscKEyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FDJozuQHCSr3ZMNjAnIaJoaAehIXma1I+dSjgGlqGpU=; b=HFqoAnS4YenKgUDivwAEn0P8V52K+bW9cEiZgiU4mK79HvMeOJl0HOdp1s9PuKxdJY8dBzb/TXgrI8c3en94g2VGaMwlEqO3zQk8RH0C1xQ0BfsrXq5/ntRzEmhlMhFP9ua6erPxEkkEQMH91AdcQ3cbfgzpP+i+lJLdrvdT1unWPbxSOi3h4e84MpbCoDvEi9ILV9jSrzeI+1E0k9L9SZx9SxQc4L6VDL78/fdHmgZwYC/WVjo6y7AeCWTezLp+vxDHnXAU0nAmQ/aEAxolcewjAnkbFQIEj2E5OhM4div0KSBmMdjJFzhhJJi7polSTGzbvUEI0J467VpWbat9zA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FDJozuQHCSr3ZMNjAnIaJoaAehIXma1I+dSjgGlqGpU=; b=GuX984lnYDw4W3YMb1yfcPl09Jb9JIMr0H2YEsYFqvbRgu3divBiX6Tsl9i0Hk6csN9WtOCLuYblv7jy/ag9h8JLW06lxxlMCBGl9MdWmgY4iSoQgMEC7sOQuUqX/tZtuL9u3w7GpBOslTGeDYKev14r7VFUm1ZSOYnCvdxVo2nhUXMAj9w6DnXmz7dKVOVqCxpq0A0GpvS5qI8o5txHkMpAENzcOuX14e/W5JveoeFSsiwxjAePrkGCH5Fj6BNJC1hwqAe5DSke4Tj0J04NZTUh1MAp06ASdPpantmUAVoXw7jr3FEg5AWep6GeTIwTXvtlQwjIZVIR5RjnqC0lmA== Received: from BN0PR08CA0014.namprd08.prod.outlook.com (2603:10b6:408:142::23) by DS0PR12MB7654.namprd12.prod.outlook.com (2603:10b6:8:11d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23; Wed, 30 Nov 2022 11:53:16 +0000 Received: from BN8NAM11FT032.eop-nam11.prod.protection.outlook.com (2603:10b6:408:142:cafe::5a) by BN0PR08CA0014.outlook.office365.com (2603:10b6:408:142::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23 via Frontend Transport; Wed, 30 Nov 2022 11:53:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT032.mail.protection.outlook.com (10.13.177.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.17 via Frontend Transport; Wed, 30 Nov 2022 11:53:16 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:53:02 -0800 Received: from nps-server-23.mtl.labs.mlnx (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:52:57 -0800 From: Shay Drory To: , , CC: , , , , Subject: [PATCH net-next 7/8] devlink: Expose port function commands to control migratable Date: Wed, 30 Nov 2022 13:52:16 +0200 Message-ID: <20221130115217.7171-8-shayd@nvidia.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221130115217.7171-1-shayd@nvidia.com> References: <20221130115217.7171-1-shayd@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT032:EE_|DS0PR12MB7654:EE_ X-MS-Office365-Filtering-Correlation-Id: acc0c921-c561-4983-ece2-08dad2c977a0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CI+zvDAsL5lZiOK6y7/pDXmmFVqQcd4RGfnyC3rdoVzaXOxtXXnCjsT9uRjJGHQd/QpOIKnxx1WKZRLFdOKc6ruUMVpCH4eOEIW+clxYJPqPwB5K5vApRCc2WbKzyImy7uhA9f6iDkOOyylC3scq17/KJPA+RHwr+nBWGXhDfXH9nRJaKXmha3NQ0jW+djksObwtuxecd6I8os2MpmR6I1jZMXQzvIRKAazjrEJaXqhnWh/NYwKDEhcQITwCSJ8ov6saFUvw9prcVML8gDu1Ag8J3KIetgMju/M4tbXT8l6Pe7wlEwUEW17PVx8IcL+P30n9SkomCwYF7NytObJJlEM7x3EoUEvlixefzi1l8dB7JBF0ZUeKBPPxcKn1IFir4gjR9ztLRXzJipATLhtiRCHmPhh03put843FsAnYsbxARxVpuxgONU1oCg1dBTIFEsznOu9H3438+sYN2Pg7qLsfOQ8e2Jj2bgysaQfeoM0BcJnRNGkmGc6KZGdDaJC5489n0FXVC09Ex3E+AHqTWWp7KiMbS64QPEhTZnJojPKoSMuqz/58OsCGGNm4gLYHmzJsE8MOl5kfh9Z8D985pOyVxLPxgJK9jO5LZYuIZ/wvFQ3SBXOC12ksEBgQfnNgEUkWrrfFePJ+GSA92222U+f+Dy4d/VeVakFMB2C42bXA8WJf4yMQs3B+C+AgENEHiTo1PXLKHXFiUoi2G9wQwA== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(346002)(136003)(376002)(396003)(39860400002)(451199015)(40470700004)(36840700001)(46966006)(2906002)(7636003)(41300700001)(36756003)(54906003)(70206006)(8676002)(36860700001)(40460700003)(110136005)(8936002)(70586007)(5660300002)(2616005)(26005)(16526019)(82310400005)(1076003)(86362001)(316002)(107886003)(6666004)(426003)(336012)(4326008)(186003)(356005)(83380400001)(40480700001)(82740400003)(478600001)(47076005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2022 11:53:16.1453 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: acc0c921-c561-4983-ece2-08dad2c977a0 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT032.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7654 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Expose port function commands to enable / disable migratable capability, this is used to set the port function as migratable. Live migration is the process of transferring a live virtual machine from one physical host to another without disrupting its normal operation. In order for a VM to be able to perform LM, all the VM components must be able to perform migration. e.g.: to be migratable. In order for VF to be migratable, VF must be bound to VFIO driver with migration support. When migratable capability is enable for a function of the port, the device is making the necessary preparations for the function to be migratable, which might include disabling features which cannot be migrated. Example of LM with migratable function configuration: Set migratable of the VF's port function. $ devlink port show pci/0000:06:00.0/2 pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 function: hw_addr 00:00:00:00:00:00 migratable disable $ devlink port function set pci/0000:06:00.0/2 migratable enable $ devlink port show pci/0000:06:00.0/2 pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 function: hw_addr 00:00:00:00:00:00 migratable enable Bind VF to VFIO driver with migration support: $ echo > /sys/bus/pci/devices/0000:08:00.0/driver/unbind $ echo mlx5_vfio_pci > /sys/bus/pci/devices/0000:08:00.0/driver_override $ echo > /sys/bus/pci/devices/0000:08:00.0/driver/bind Attach VF to the VM. Start the VM. Perform LM. Signed-off-by: Shay Drory Reviewed-by: Jiri Pirko --- .../networking/devlink/devlink-port.rst | 45 ++++++++++++++++ include/net/devlink.h | 21 ++++++++ include/uapi/linux/devlink.h | 1 + net/core/devlink.c | 54 +++++++++++++++++++ 4 files changed, 121 insertions(+) diff --git a/Documentation/networking/devlink/devlink-port.rst b/Documentation/networking/devlink/devlink-port.rst index 79f9c0390b47..b1b0ce50d9f1 100644 --- a/Documentation/networking/devlink/devlink-port.rst +++ b/Documentation/networking/devlink/devlink-port.rst @@ -125,6 +125,9 @@ this means a MAC address. Users may also set the RoCE capability of the function using 'devlink port function set roce' command. +Users may also set the function as migratable using +'devlink port function set migratable' command. + Function attributes =================== @@ -194,6 +197,48 @@ VF/SF driver cannot override it. function: hw_addr 00:00:00:00:00:00 roce disable +migratable capability setup +--------------------------- +Live migration is the process of transferring a live virtual machine +from one physical host to another without disrupting its normal +operation. + +User who want PCI VFs to be able to perform live migration need to +explicitly enable the VF migratable capability. + +When user enables migratable capability for a VF, and the HV binds the VF to VFIO driver +with migration support, the user can migrate the VM with this VF from one HV to a +different one. + +However, when migratable capability is enable, device will disable features which cannot +be migrated. Thus migratable cap can impose limitations on a VF so let the user decide. + +Example of LM with migratable function configuration: +- Get migratable capability of the VF device:: + + $ devlink port show pci/0000:06:00.0/2 + pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 + function: + hw_addr 00:00:00:00:00:00 migratable disable + +- Set migratable capability of the VF device:: + + $ devlink port function set pci/0000:06:00.0/2 migratable enable + + $ devlink port show pci/0000:06:00.0/2 + pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 + function: + hw_addr 00:00:00:00:00:00 migratable enable + +- Bind VF to VFIO driver with migration support: +$ echo > /sys/bus/pci/devices/0000:08:00.0/driver/unbind +$ echo mlx5_vfio_pci > /sys/bus/pci/devices/0000:08:00.0/driver_override +$ echo > /sys/bus/pci/devices/0000:08:00.0/driver/bind + +Attach VF to the VM. +Start the VM. +Perform live migration. + Subfunction ============ diff --git a/include/net/devlink.h b/include/net/devlink.h index 7f75100e8b26..8abffd9201d7 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -1454,6 +1454,27 @@ struct devlink_ops { int (*port_function_roce_set)(struct devlink_port *devlink_port, bool enable, struct netlink_ext_ack *extack); + /** + * @port_function_mig_get: Port function's migratable get function. + * + * Query migratable state of a function managed by the devlink port. + * Return -EOPNOTSUPP if port function migratable handling is not + * supported. + */ + int (*port_function_mig_get)(struct devlink_port *devlink_port, + bool *is_enable, + struct netlink_ext_ack *extack); + /** + * @port_function_mig_set: Port function's migratable set function. + * + * Enable/Disable migratable state of a function managed by the devlink + * port. + * Return -EOPNOTSUPP if port function migratable handling is not + * supported. + */ + int (*port_function_mig_set)(struct devlink_port *devlink_port, + bool enable, + struct netlink_ext_ack *extack); /** * port_new() - Add a new port function of a specified flavor * @devlink: Devlink instance diff --git a/include/uapi/linux/devlink.h b/include/uapi/linux/devlink.h index c6f1fbe54095..f70201be0479 100644 --- a/include/uapi/linux/devlink.h +++ b/include/uapi/linux/devlink.h @@ -658,6 +658,7 @@ enum devlink_resource_unit { enum devlink_port_fn_attr_cap { DEVLINK_PORT_FN_ATTR_CAP_ROCE, + DEVLINK_PORT_FN_ATTR_CAP_MIGRATABLE, /* Add new caps above */ __DEVLINK_PORT_FN_ATTR_CAPS_MAX, diff --git a/net/core/devlink.c b/net/core/devlink.c index 88846ad635a0..3fd47bbf891b 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -728,6 +728,29 @@ static int devlink_port_fn_roce_fill(const struct devlink_ops *ops, return 0; } +static int devlink_port_function_mig_fill(const struct devlink_ops *ops, + struct devlink_port *devlink_port, + struct nla_bitfield32 *caps, + struct netlink_ext_ack *extack) +{ + bool is_enable; + int err; + + if (!ops->port_function_mig_get || + devlink_port->attrs.flavour != DEVLINK_PORT_FLAVOUR_PCI_VF) + return 0; + + err = ops->port_function_mig_get(devlink_port, &is_enable, extack); + if (err) { + if (err == -EOPNOTSUPP) + return 0; + return err; + } + + DEVLINK_PORT_FN_SET_CAP(caps, DEVLINK_PORT_FN_CAP(MIGRATABLE), is_enable); + return 0; +} + static int devlink_port_fn_caps_fill(const struct devlink_ops *ops, struct devlink_port *devlink_port, struct sk_buff *msg, @@ -741,6 +764,10 @@ static int devlink_port_fn_caps_fill(const struct devlink_ops *ops, if (err) return err; + err = devlink_port_function_mig_fill(ops, devlink_port, &caps, extack); + if (err) + return err; + if (!caps.selector) return 0; err = nla_put_bitfield32(msg, DEVLINK_PORT_FN_ATTR_CAPS, caps.value, @@ -1335,6 +1362,15 @@ static int devlink_port_fn_state_fill(const struct devlink_ops *ops, return 0; } +static int +devlink_port_fn_mig_set(struct devlink_port *devlink_port, bool enable, + struct netlink_ext_ack *extack) +{ + const struct devlink_ops *ops = devlink_port->devlink->ops; + + return ops->port_function_mig_set(devlink_port, enable, extack); +} + static int devlink_port_fn_roce_set(struct devlink_port *devlink_port, bool enable, struct netlink_ext_ack *extack) @@ -1361,6 +1397,13 @@ static int devlink_port_fn_caps_set(struct devlink_port *devlink_port, if (err) return err; } + if (caps.selector & DEVLINK_PORT_FN_CAP(MIGRATABLE)) { + err = devlink_port_fn_mig_set(devlink_port, caps_value & + DEVLINK_PORT_FN_CAP(MIGRATABLE), + extack); + if (err) + return err; + } return 0; } @@ -1781,6 +1824,17 @@ static int devlink_port_function_validate(struct devlink_port *devlink_port, "Port doesn't support RoCE function attribute"); return -EOPNOTSUPP; } + if (caps.selector & DEVLINK_PORT_FN_CAP(MIGRATABLE)) { + if (!ops->port_function_mig_set) { + NL_SET_ERR_MSG_ATTR(extack, attr, + "Port doesn't support migratable function attribute"); + } + if (devlink_port->attrs.flavour != DEVLINK_PORT_FLAVOUR_PCI_VF) { + NL_SET_ERR_MSG_ATTR(extack, attr, + "migratable function attribute supported for VFs only"); + return -EOPNOTSUPP; + } + } } return 0; } From patchwork Wed Nov 30 11:52:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shay Drori X-Patchwork-Id: 13059749 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C663C352A1 for ; Wed, 30 Nov 2022 11:54:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235050AbiK3LyO (ORCPT ); Wed, 30 Nov 2022 06:54:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234305AbiK3LxZ (ORCPT ); Wed, 30 Nov 2022 06:53:25 -0500 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2070.outbound.protection.outlook.com [40.107.220.70]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53F461D0EE for ; Wed, 30 Nov 2022 03:53:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VkxkE5g2+00BhiDza5Ho+kAYg6FOH62cuLfIiWta2W0RmpPB0vFLDTPDK7AUn+o3cMHV3hzws2pRdrveI6SaNW0WLwnkKycmJhWs5XmJKLtJhC72mJ1Pf85iZtO3p1peetzvCdiZ/qGUX6sxOYRTfpyb5a3VJGZcPW77vRJf17xVthboKfguqUyB9K69TQWEH2oUrKh956bZNlNkW/HRtqGvQyc/hlIlZ6KcN8ZIjWMw9w293Jh+SXh4MzYVQTE2LwFgHDyA+xoES8hFKCmqOLH2GC1Zq/gnTN4Td5zaYTMWc7Tz4IqOS6659+qwNUzXOCYiGclyj9O67Dt5hOfkQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FPwciwfQWYkWUh0pBwocHbSeOOWxbtoNNx5KJ+Cy4as=; b=iTImSfcRjKv6wLIEoInearrvkNCmiswzwnK+DO0X48fc0R50g8bYedD7it53GhjJ59Y6kdqtn5IsRk3bNMtzqw0nszFQLIaxCJmoU7f2ReM4+9MQr7EU45pX28nVW1EkPSxQcxqAzaBOSGwkajg/7C3lOtbUL+0rqcCVPmnfMhmtfDABnvifezHkAVsFNY7PurgNl8Vn6TUQzXfM88Os0Ut3Gdguvg6yvl2JwDfC5WAzq5TfjTubs/f0Tb3vxY4KcuqLpZGY5nAe+GAXIN2CVytlQvAWGGKxVut/W/hKkidL67h1ypQT9TD0On4Vnv6GV63XzkVhQcrqfvDOhTyQ3g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FPwciwfQWYkWUh0pBwocHbSeOOWxbtoNNx5KJ+Cy4as=; b=dOw9C1OrNJCSO+GthyMuqzO0TTA05NJikIK5m1nt0jxFxBxiTRBCHyPkxQvbE1ziN0QuUDCudCcD0fXybpbu9Zi78Z8TA6C2GRg70gW4KbgnkJEUgXUyVh8g/hw+kP5bjF5ZWJoAyzP+b0ijWWXhqbDZ0aDSXaY+sbTASir/xAeb9M0bFo0X/Ig24ldBvM/j2GZ9a+Hp73Qjxpns2ClaHRCQeEYRPiNnZ8e6Kx9OKHgO+5QVKKaEctwA4fsA+NypnqjbMabos4ivMtPsDzCqzXMs11wzdR4Caev1f5+NspUL+UeaZzPeTdSDlZ9GEbfUCyM5hn8qS85s5vLrwdOv/w== Received: from BN8PR07CA0023.namprd07.prod.outlook.com (2603:10b6:408:ac::36) by DM4PR12MB7622.namprd12.prod.outlook.com (2603:10b6:8:109::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23; Wed, 30 Nov 2022 11:53:22 +0000 Received: from BN8NAM11FT008.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ac:cafe::a1) by BN8PR07CA0023.outlook.office365.com (2603:10b6:408:ac::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.23 via Frontend Transport; Wed, 30 Nov 2022 11:53:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT008.mail.protection.outlook.com (10.13.177.95) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.21 via Frontend Transport; Wed, 30 Nov 2022 11:53:22 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:53:07 -0800 Received: from nps-server-23.mtl.labs.mlnx (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 30 Nov 2022 03:53:02 -0800 From: Shay Drory To: , , CC: , , , , Subject: [PATCH net-next 8/8] net/mlx5: E-Switch, Implement devlink port function cmds to control migratable Date: Wed, 30 Nov 2022 13:52:17 +0200 Message-ID: <20221130115217.7171-9-shayd@nvidia.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221130115217.7171-1-shayd@nvidia.com> References: <20221130115217.7171-1-shayd@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT008:EE_|DM4PR12MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: 09e82d43-f8bb-4591-82b3-08dad2c97b46 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iCNgfeW7YQyomTWs9x2JYI17VEkOaTgR3SXc89as/lSVs0Lofu+uHYHU08eWRqKF7CFYLlxLj49j31NJUXKp0D5d2X/aGwy+z8dgdAYB7UXl28BjsoyffMSCBf3Ak0NIVNyH+UAV7cE4KTNykMRlKWKjTBz+CZ3Wa/SssPCJ/DWbbe18a5XxnmDgG9jeiUnKDXmp0BcQ91cbGuLWP5/BV24wnz7ZbKpdeLvwah8SYVxw7Y7wrEBujcQGvVxklMmr98GFZ/9G+D2G9DC/oKHLjX0UvOQ34uCHDLc2BWybgjabDXPn68dJHrcveSG1YmTsHK3yxDyme1rLFqHzPW3Rt03W3sXKVDAs0jnkpQONC3U+2aaHwXkc9V90tgoOc32E/vgyIwDC9RXS21iA5goKuEzf47QQwQZckKj1bXM9kqU1v4FftHDigDERCOPmzMzjpqdr6gMz+vbcd4m9cnEPHHAFnbVJ3V/9+f7AS0j4d0zQ36SjZjnHGBmVrryqhKF2BuTnD2gO25TliKR0P+NyrOZnjsIG5s6vsZjNrwbWBEdsPHvb3GSpsX3jW/SvwHl0kfpU8PFdF3z1QEX4NCic6bZwomPnV+BfZC+XdWoBohHtqNy1BVUpQSpgClnjWgo6nu5/ddeZEMStqY561d+AzK/D2zhb3SyIRN9jq35dLATDec+nDL4l9Z/i+lTmPOuldutdm3HrQRgbJp0izU+X7g== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(376002)(136003)(346002)(396003)(39860400002)(451199015)(46966006)(36840700001)(40470700004)(16526019)(356005)(7636003)(110136005)(316002)(2906002)(54906003)(36860700001)(8936002)(40460700003)(41300700001)(70206006)(5660300002)(83380400001)(70586007)(4326008)(8676002)(86362001)(47076005)(426003)(336012)(26005)(186003)(1076003)(2616005)(82310400005)(107886003)(36756003)(6666004)(478600001)(82740400003)(40480700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2022 11:53:22.2637 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 09e82d43-f8bb-4591-82b3-08dad2c97b46 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT008.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7622 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Implement devlink port function commands to enable / disable migratable. This is used to control the migratable capability of the device. Signed-off-by: Shay Drory Reviewed-by: Mark Bloch Ack-by: Saeed Mahameed --- .../device_drivers/ethernet/mellanox/mlx5.rst | 8 ++ .../net/ethernet/mellanox/mlx5/core/devlink.c | 2 + .../net/ethernet/mellanox/mlx5/core/eswitch.c | 8 ++ .../net/ethernet/mellanox/mlx5/core/eswitch.h | 5 + .../mellanox/mlx5/core/eswitch_offloads.c | 100 ++++++++++++++++++ 5 files changed, 123 insertions(+) diff --git a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5.rst b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5.rst index 992e3d2830ad..7825dcb4922e 100644 --- a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5.rst +++ b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5.rst @@ -364,6 +364,14 @@ PCI devices/SF. mlx5 driver support devlink port function attr mechanism to setup RoCE capability. (refer to Documentation/networking/devlink/devlink-port.rst) +migratable capability setup +--------------------------- +User who wants mlx5 PCI VFs to be able to perform live migration need to +explicitly enable the VF migratable capability. + +mlx5 driver support devlink port function attr mechanism to setup migratable +capability. (refer to Documentation/networking/devlink/devlink-port.rst) + SF state setup -------------- To use the SF, the user must active the SF using the SF function state diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index 6b0685d9260b..9a70a541215b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -320,6 +320,8 @@ static const struct devlink_ops mlx5_devlink_ops = { .rate_leaf_parent_set = mlx5_esw_devlink_rate_parent_set, .port_function_roce_get = mlx5_devlink_port_function_roce_get, .port_function_roce_set = mlx5_devlink_port_function_roce_set, + .port_function_mig_get = mlx5_devlink_port_function_mig_get, + .port_function_mig_set = mlx5_devlink_port_function_mig_set, #endif #ifdef CONFIG_MLX5_SF_MANAGER .port_new = mlx5_devlink_sf_port_new, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c index 66d434dafb0b..fcf5afc5886f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c @@ -794,6 +794,14 @@ static int mlx5_esw_vport_caps_get(struct mlx5_eswitch *esw, struct mlx5_vport * hca_caps = MLX5_ADDR_OF(query_hca_cap_out, query_ctx, capability); vport->info.roce_enabled = MLX5_GET(cmd_hca_cap, hca_caps, roce); + memset(query_ctx, 0, query_out_sz); + err = mlx5_vport_get_other_func_cap(esw->dev, vport->vport, query_ctx, + MLX5_CAP_GENERAL_2); + if (err) + goto out_free; + + hca_caps = MLX5_ADDR_OF(query_hca_cap_out, query_ctx, capability); + vport->info.mig_enabled = MLX5_GET(cmd_hca_cap_2, hca_caps, migratable); out_free: kfree(query_ctx); return err; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h index ef0dfe8e55d6..1d8eda0bbd6f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h @@ -154,6 +154,7 @@ struct mlx5_vport_info { u8 spoofchk: 1; u8 trusted: 1; u8 roce_enabled: 1; + u8 mig_enabled: 1; }; /* Vport context events */ @@ -509,6 +510,10 @@ int mlx5_devlink_port_function_hw_addr_get(struct devlink_port *port, int mlx5_devlink_port_function_hw_addr_set(struct devlink_port *port, const u8 *hw_addr, int hw_addr_len, struct netlink_ext_ack *extack); +int mlx5_devlink_port_function_mig_get(struct devlink_port *port, bool *is_enabled, + struct netlink_ext_ack *extack); +int mlx5_devlink_port_function_mig_set(struct devlink_port *port, bool enable, + struct netlink_ext_ack *extack); int mlx5_devlink_port_function_roce_get(struct devlink_port *port, bool *is_enabled, struct netlink_ext_ack *extack); int mlx5_devlink_port_function_roce_set(struct devlink_port *port, bool enable, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index fe1d0f9e723e..622dbf9e41dc 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -4028,6 +4028,106 @@ mlx5_devlink_port_function_get_vport(struct devlink_port *port, struct mlx5_eswi return mlx5_eswitch_get_vport(esw, vport_num); } +int mlx5_devlink_port_function_mig_get(struct devlink_port *port, bool *is_enabled, + struct netlink_ext_ack *extack) +{ + struct mlx5_eswitch *esw; + struct mlx5_vport *vport; + int err = -EOPNOTSUPP; + + esw = mlx5_devlink_eswitch_get(port->devlink); + if (IS_ERR(esw)) + return PTR_ERR(esw); + + if (!MLX5_CAP_GEN(esw->dev, migration)) { + NL_SET_ERR_MSG_MOD(extack, "Device doesn't support migration"); + return err; + } + + vport = mlx5_devlink_port_function_get_vport(port, esw); + if (IS_ERR(vport)) { + NL_SET_ERR_MSG_MOD(extack, "Invalid port"); + return PTR_ERR(vport); + } + + mutex_lock(&esw->state_lock); + if (vport->enabled) { + *is_enabled = vport->info.mig_enabled; + err = 0; + } + mutex_unlock(&esw->state_lock); + return err; +} + +int mlx5_devlink_port_function_mig_set(struct devlink_port *port, bool enable, + struct netlink_ext_ack *extack) +{ + int query_out_sz = MLX5_ST_SZ_BYTES(query_hca_cap_out); + struct mlx5_eswitch *esw; + struct mlx5_vport *vport; + void *query_ctx; + void *hca_caps; + int err = -EOPNOTSUPP; + + esw = mlx5_devlink_eswitch_get(port->devlink); + if (IS_ERR(esw)) + return PTR_ERR(esw); + + if (!MLX5_CAP_GEN(esw->dev, migration)) { + NL_SET_ERR_MSG_MOD(extack, "Device doesn't support migration"); + return err; + } + + vport = mlx5_devlink_port_function_get_vport(port, esw); + if (IS_ERR(vport)) { + NL_SET_ERR_MSG_MOD(extack, "Invalid port"); + return PTR_ERR(vport); + } + + mutex_lock(&esw->state_lock); + if (!vport->enabled) { + NL_SET_ERR_MSG_MOD(extack, "Eswitch vport is disabled"); + goto out; + } + + if (vport->info.mig_enabled == enable) { + err = 0; + goto out; + } + + query_ctx = kzalloc(query_out_sz, GFP_KERNEL); + if (!query_ctx) { + err = -ENOMEM; + goto out; + } + + err = mlx5_vport_get_other_func_cap(esw->dev, vport->vport, query_ctx, + MLX5_CAP_GENERAL_2); + if (err) { + NL_SET_ERR_MSG_MOD(extack, "Failed getting HCA caps"); + goto out_free; + } + + hca_caps = MLX5_ADDR_OF(query_hca_cap_out, query_ctx, capability); + memcpy(hca_caps, MLX5_ADDR_OF(query_hca_cap_out, query_ctx, capability), + MLX5_UN_SZ_BYTES(hca_cap_union)); + MLX5_SET(cmd_hca_cap_2, hca_caps, migratable, 1); + + err = mlx5_vport_set_other_func_cap(esw->dev, hca_caps, vport->vport, + MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE2); + if (err) { + NL_SET_ERR_MSG_MOD(extack, "Failed setting HCA migratable cap"); + goto out_free; + } + + vport->info.mig_enabled = enable; + +out_free: + kfree(query_ctx); +out: + mutex_unlock(&esw->state_lock); + return err; +} int mlx5_devlink_port_function_roce_get(struct devlink_port *port, bool *is_enabled, struct netlink_ext_ack *extack) {