From patchwork Thu Mar 23 11:10:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Chumak X-Patchwork-Id: 13185521 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1A53C6FD1D for ; Thu, 23 Mar 2023 11:12:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229993AbjCWLL7 (ORCPT ); Thu, 23 Mar 2023 07:11:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229849AbjCWLL5 (ORCPT ); Thu, 23 Mar 2023 07:11:57 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2059.outbound.protection.outlook.com [40.107.220.59]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DE7162D168 for ; Thu, 23 Mar 2023 04:11:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PncVfewLHmytK/rbqqRy+ET+BRoXSfdlKZwoNGiFwiNdYcPLSz1EQTv8t/uZ/SmWyq8/yt6yliJ0DAHUTGYRnuH1oOsSAx5HBzlRmu4QjDl7cxf/U9nlCO7GMV0JRRGKs0ygD3afJh5DrVFJYns6rp7g4iQMZ9caTu5CVBXh6NSfqiq+PSvGVRugsepSnRhKukGKt02okTSf63sgGbTw2PLc3c5Sr52xhGIKDfJl9LRqm6s77dMGsb7suj1EC4Nsj0Fpoj9fzZbS4uNDGs4HpkUWbUTo2e2TK6o/QAO1s/Zw5UqU51GUEUmcH5D3f6GVmmZAjrxfBE/fcjXKvHiRbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/dTNXqjsDAOycLUOm5ZKMKDQ3BK2Lq+cwe07jpwPyys=; b=Y4e61QOmI0WkjES4cZ/5X+VfGWTekN1sO2eDDs+kVJcxiD/tl/jRGbCCZrvtc6ZXVDapefl1SKXONpu50HnoMDi+xSSeNJp78amFN06+mxEsfuHTI2LERx8MikflyZzn9Jl0Zhli6XEF+XS/6Pr1XbqQ3juzzoa9QmmocDzwt/W7ylVvR3rsyK+sebYw+O1sDCnhQAAECEr/hMP+SF76RcSU/7CEHV2gyErLaYhqHJ2U8FMVKQ26UE2SQKOA5kzW2s20saIApqwHo5pAZ+YPFA9VL2ZdwFbe9y3fX1+t6ONXw1DgHxH6r8AZ0xXFbiJozmpSv7XnLRdkPga6HqVZgw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/dTNXqjsDAOycLUOm5ZKMKDQ3BK2Lq+cwe07jpwPyys=; b=QDZhjZb3MAnmQAx/B2u78G3JhGDQAYN+NZ4JHcNIGIRmAqqucCfsB91QAZGq1SNezWEuxowjyiXzs8iLaEMC1bUDWTHLp08+ccKSzTdkmmpI5QWR1WJ6ij3TYHchlaNouqqduW1TK41CjDktMzMvXhuLlit1ZCDKjs+BacuFTTLOIsdSYaJe10I2k5C9PGTo8qecVtts5a9n9x0CzAvuv4KGBuEtaLWNi6cUqH5oTAPRgorgh7efgPS8pawaP1PtpzjKuHMeDOGmFdqoSTltbV1mJlN6k9WNnc1vdWol5JTRNyrGPbFWP3Iagd2KmVw1lxkgURD1u4IsjQ9qAvavSw== Received: from DS7P222CA0015.NAMP222.PROD.OUTLOOK.COM (2603:10b6:8:2e::27) by DM4PR12MB6661.namprd12.prod.outlook.com (2603:10b6:8:bb::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.38; Thu, 23 Mar 2023 11:11:52 +0000 Received: from DS1PEPF0000E654.namprd02.prod.outlook.com (2603:10b6:8:2e:cafe::c5) by DS7P222CA0015.outlook.office365.com (2603:10b6:8:2e::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.38 via Frontend Transport; Thu, 23 Mar 2023 11:11:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS1PEPF0000E654.mail.protection.outlook.com (10.167.18.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.30 via Frontend Transport; Thu, 23 Mar 2023 11:11:52 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Thu, 23 Mar 2023 04:11:42 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Thu, 23 Mar 2023 04:11:42 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.986.5 via Frontend Transport; Thu, 23 Mar 2023 04:11:39 -0700 From: Dima Chumak To: Jakub Kicinski CC: "David S. Miller" , Eric Dumazet , Paolo Abeni , Jiri Pirko , Leon Romanovsky , Saeed Mahameed , , Dima Chumak , Jiri Pirko Subject: [PATCH net-next 1/4] devlink: Expose port function commands to control IPsec crypto offloads Date: Thu, 23 Mar 2023 13:10:56 +0200 Message-ID: <20230323111059.210634-2-dchumak@nvidia.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230323111059.210634-1-dchumak@nvidia.com> References: <20230323111059.210634-1-dchumak@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF0000E654:EE_|DM4PR12MB6661:EE_ X-MS-Office365-Filtering-Correlation-Id: 5ba519e2-1f6f-4bab-0510-08db2b8f67e1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +9/4YRvFS9G80uHv2bKcDgxsq2ogMpinpBi63O9sEiAsXcsw7PeQYbiP4AuhI/TFmN2UVxcuDQd3h7FaIkEPZ8Bl/p0jnMQH3vKtLvtZgD0eQyr3NablQpgzOXc2pl3LbOLhZIewEw/bYysA+nVhKdVlPblQenuIyZEqlbx1o7dtPATOxtCxMgPzIaVVpkvvM+pxGA4HbaEeE/U8cIGwy6iABLzQ8WoG/3d0n0UU4CIRbO/pg1wjQb25BhGBI1kOkklbLOYSp6zCyUgjGSyOS8f8tj8eLEo6ANzJU3gJ5mR49BEu8slqC0vx4pvaPwKm+VwlT6PJ+wx+AVKKDe+RaDkz+hn3ISZcPHf2vG3/UKc3r9v8UfQxQ75px2ulTp9ubMdTy/4z75LxHjuQLX27vXIVI0odzebjcd8XGN+G2+BE6mWlhOvax8gztUpu7tKGkLhI/bKdY95UfC7nU4uI+sxYhsEvGO6x6QxsN43Q82sHOSbFokMizqXWD1iJWrXD8IkAZEL0bGR/yyPhh2ZbSfwtBtJmRL/DbAVwHnNT8icOWMN/zhFs8S9UPOMs0tUj2DqTfs4dee4wrXxoV9mdUJHgcxQm4b9UO8D6xnzOd+VCuU8qrZPay40P584nASkHs6gcomz/ohalUn2C6HGk/JSkFcWNFq6OH73AgM4okYJZvWPflHBH9nRWEZbbhfFq8YDaZJUjYu1g8sqbJDEla+Ez7e33ETB0r5Ys4Q2lWMVoC0EXIajuPI6tJ3gpNx0q X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230025)(4636009)(396003)(136003)(376002)(346002)(39860400002)(451199018)(36840700001)(46966006)(40470700004)(41300700001)(5660300002)(8676002)(6916009)(4326008)(2906002)(40460700003)(36860700001)(82740400003)(356005)(36756003)(86362001)(7636003)(7696005)(107886003)(6666004)(26005)(478600001)(70586007)(70206006)(316002)(8936002)(40480700001)(82310400005)(1076003)(54906003)(47076005)(426003)(83380400001)(336012)(2616005)(186003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2023 11:11:52.4612 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5ba519e2-1f6f-4bab-0510-08db2b8f67e1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF0000E654.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6661 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Expose port function commands to enable / disable IPsec crypto offloads, this is used to control the port IPsec capabilities. When IPsec is disabled for a function of the port (default), function cannot offload any IPsec crypto operations. When enabled, IPsec crypto operations can be offloaded by the function of the port. Example of a PCI VF port which supports IPsec crypto offloads: $ devlink port show pci/0000:06:00.0/1 pci/0000:06:00.0/1: type eth netdev enp6s0pf0vf0 flavour pcivf pfnum 0 vfnum 0 function: hw_addr 00:00:00:00:00:00 roce enable ipsec_crypto disable $ devlink port function set pci/0000:06:00.0/1 ipsec_crypto enable $ devlink port show pci/0000:06:00.0/1 pci/0000:06:00.0/1: type eth netdev enp6s0pf0vf0 flavour pcivf pfnum 0 vfnum 0 function: hw_addr 00:00:00:00:00:00 roce enable ipsec_crypto enable Signed-off-by: Dima Chumak Reviewed-by: Jiri Pirko --- .../networking/devlink/devlink-port.rst | 27 +++++++++ include/net/devlink.h | 21 +++++++ include/uapi/linux/devlink.h | 2 + net/devlink/leftover.c | 55 +++++++++++++++++++ 4 files changed, 105 insertions(+) diff --git a/Documentation/networking/devlink/devlink-port.rst b/Documentation/networking/devlink/devlink-port.rst index 3da590953ce8..e7c7482714d7 100644 --- a/Documentation/networking/devlink/devlink-port.rst +++ b/Documentation/networking/devlink/devlink-port.rst @@ -128,6 +128,9 @@ Users may also set the RoCE capability of the function using Users may also set the function as migratable using 'devlink port function set migratable' command. +Users may also set the IPsec crypto capability of the function using +`devlink port function set ipsec_crypto` command. + Function attributes =================== @@ -240,6 +243,30 @@ Attach VF to the VM. Start the VM. Perform live migration. +IPsec crypto capability setup +----------------------------- +When user enables IPsec crypto capability for a VF, user application can offload +XFRM state to this VF. + +When IPsec crypto capability is disabled (default) for a VF, the XFRM state is +processed in software by the kernel. + +- Get IPsec crypto capability of the VF device:: + + $ devlink port show pci/0000:06:00.0/2 + pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 + function: + hw_addr 00:00:00:00:00:00 ipsec_crypto disabled + +- Set IPsec crypto capability of the VF device:: + + $ devlink port function set pci/0000:06:00.0/2 ipsec_crypto enable + + $ devlink port show pci/0000:06:00.0/2 + pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 + function: + hw_addr 00:00:00:00:00:00 ipsec_crypto enabled + Subfunction ============ diff --git a/include/net/devlink.h b/include/net/devlink.h index 6a942e70e451..4e5f4aeca29d 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -1495,6 +1495,27 @@ struct devlink_ops { int (*port_fn_migratable_set)(struct devlink_port *devlink_port, bool enable, struct netlink_ext_ack *extack); + /** + * @port_fn_ipsec_crypto_get: Port function's ipsec_crypto get function. + * + * Query ipsec_crypto state of a function managed by the devlink port. + * Return -EOPNOTSUPP if port function IPsec crypto offload is not + * supported. + */ + int (*port_fn_ipsec_crypto_get)(struct devlink_port *devlink_port, + bool *is_enable, + struct netlink_ext_ack *extack); + /** + * @port_fn_ipsec_crypto_set: Port function's ipsec_crypto set function. + * + * Enable/Disable ipsec_crypto state of a function managed by the devlink + * port. + * Return -EOPNOTSUPP if port function IPsec crypto offload is not + * supported. + */ + int (*port_fn_ipsec_crypto_set)(struct devlink_port *devlink_port, + bool enable, + struct netlink_ext_ack *extack); /** * port_new() - Add a new port function of a specified flavor * @devlink: Devlink instance diff --git a/include/uapi/linux/devlink.h b/include/uapi/linux/devlink.h index 3782d4219ac9..f9ae9a058ad2 100644 --- a/include/uapi/linux/devlink.h +++ b/include/uapi/linux/devlink.h @@ -661,6 +661,7 @@ enum devlink_resource_unit { enum devlink_port_fn_attr_cap { DEVLINK_PORT_FN_ATTR_CAP_ROCE_BIT, DEVLINK_PORT_FN_ATTR_CAP_MIGRATABLE_BIT, + DEVLINK_PORT_FN_ATTR_CAP_IPSEC_CRYPTO_BIT, /* Add new caps above */ __DEVLINK_PORT_FN_ATTR_CAPS_MAX, @@ -669,6 +670,7 @@ enum devlink_port_fn_attr_cap { #define DEVLINK_PORT_FN_CAP_ROCE _BITUL(DEVLINK_PORT_FN_ATTR_CAP_ROCE_BIT) #define DEVLINK_PORT_FN_CAP_MIGRATABLE \ _BITUL(DEVLINK_PORT_FN_ATTR_CAP_MIGRATABLE_BIT) +#define DEVLINK_PORT_FN_CAP_IPSEC_CRYPTO _BITUL(DEVLINK_PORT_FN_ATTR_CAP_IPSEC_CRYPTO_BIT) enum devlink_port_function_attr { DEVLINK_PORT_FUNCTION_ATTR_UNSPEC, diff --git a/net/devlink/leftover.c b/net/devlink/leftover.c index dffca2f9bfa7..07761df2471d 100644 --- a/net/devlink/leftover.c +++ b/net/devlink/leftover.c @@ -492,6 +492,29 @@ static int devlink_port_fn_migratable_fill(const struct devlink_ops *ops, return 0; } +static int devlink_port_fn_ipsec_crypto_fill(const struct devlink_ops *ops, + struct devlink_port *devlink_port, + struct nla_bitfield32 *caps, + struct netlink_ext_ack *extack) +{ + bool is_enable; + int err; + + if (!ops->port_fn_ipsec_crypto_get || + devlink_port->attrs.flavour != DEVLINK_PORT_FLAVOUR_PCI_VF) + return 0; + + err = ops->port_fn_ipsec_crypto_get(devlink_port, &is_enable, extack); + if (err) { + if (err == -EOPNOTSUPP) + return 0; + return err; + } + + devlink_port_fn_cap_fill(caps, DEVLINK_PORT_FN_CAP_IPSEC_CRYPTO, is_enable); + return 0; +} + static int devlink_port_fn_caps_fill(const struct devlink_ops *ops, struct devlink_port *devlink_port, struct sk_buff *msg, @@ -509,6 +532,10 @@ static int devlink_port_fn_caps_fill(const struct devlink_ops *ops, if (err) return err; + err = devlink_port_fn_ipsec_crypto_fill(ops, devlink_port, &caps, extack); + if (err) + return err; + if (!caps.selector) return 0; err = nla_put_bitfield32(msg, DEVLINK_PORT_FN_ATTR_CAPS, caps.value, @@ -843,6 +870,15 @@ devlink_port_fn_roce_set(struct devlink_port *devlink_port, bool enable, return ops->port_fn_roce_set(devlink_port, enable, extack); } +static int +devlink_port_fn_ipsec_crypto_set(struct devlink_port *devlink_port, bool enable, + struct netlink_ext_ack *extack) +{ + const struct devlink_ops *ops = devlink_port->devlink->ops; + + return ops->port_fn_ipsec_crypto_set(devlink_port, enable, extack); +} + static int devlink_port_fn_caps_set(struct devlink_port *devlink_port, const struct nlattr *attr, struct netlink_ext_ack *extack) @@ -867,6 +903,13 @@ static int devlink_port_fn_caps_set(struct devlink_port *devlink_port, if (err) return err; } + if (caps.selector & DEVLINK_PORT_FN_CAP_IPSEC_CRYPTO) { + err = devlink_port_fn_ipsec_crypto_set(devlink_port, caps_value & + DEVLINK_PORT_FN_CAP_IPSEC_CRYPTO, + extack); + if (err) + return err; + } return 0; } @@ -1235,6 +1278,18 @@ static int devlink_port_function_validate(struct devlink_port *devlink_port, return -EOPNOTSUPP; } } + if (caps.selector & DEVLINK_PORT_FN_CAP_IPSEC_CRYPTO) { + if (!ops->port_fn_ipsec_crypto_set) { + NL_SET_ERR_MSG_ATTR(extack, attr, + "Port doesn't support ipsec_crypto function attribute"); + return -EOPNOTSUPP; + } + if (devlink_port->attrs.flavour != DEVLINK_PORT_FLAVOUR_PCI_VF) { + NL_SET_ERR_MSG_ATTR(extack, attr, + "ipsec_crypto function attribute supported for VFs only"); + return -EOPNOTSUPP; + } + } } return 0; } From patchwork Thu Mar 23 11:10:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Chumak X-Patchwork-Id: 13185522 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0498C76195 for ; Thu, 23 Mar 2023 11:12:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229893AbjCWLMA (ORCPT ); Thu, 23 Mar 2023 07:12:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52358 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229800AbjCWLL5 (ORCPT ); Thu, 23 Mar 2023 07:11:57 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 106532CFD4 for ; Thu, 23 Mar 2023 04:11:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZrIyzpIyEF+mavFo65vmFz7oUlMUcJQZ6rjgSWwpo51SZ3qEemSAPxU79KRnE1fyG7FxN0u33VgFY640MkyDFeM6vTqwRtRG20NSU/tNg/Fh4i/SzO21wRp6JvkDmhpIcf0blofJbrxuTKbgLQYy8qGfLWzDXQO329WewVHHa3SO2sqzYQEW8V3WybB5fLJ37ytnu/cvcKiiwdSBWl7y4ApHiIg+Qs+FmrzFMxrshbdTqTlHViLxSj0R+9xmw4wSvs5BsWYcOCa2PS7rvsDHkYz/AECPvW6aur5EAk12LhHI96h2NklKtJLeaqyxRzKAKLmSD2mnV9RCekHvLdfuuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=McCag0KGpa3dRuv6lc7U5CPZYXflkVHZ2NahGFznI4Y=; b=BoHX39w41tsN+6FhZgLlkxWpxJ6+4PZBcwMfr4oiB+ZD1JYwQQD6dWZlShAs5FaNDLVX0XRWwx3FbZiL4fTQxDRJOKFV6GniiIZU6tYGv7ZaHPZG57X8UiBO3wGn8IAbA/0gqjhT1PBMEBFkOPcIrYoQ8ha3g0+izkIc5un9hDzkJ0sL8yy4xXplVGlLSi2jI0zYfQFPnmCYcFuSy7TdUZ9CyIUQzAJgIqGZNwYsA+MJ9dbqZtpUANXvRCaZavZtE9Sr0qJsMJ4FID9xs0mTUwaEbN6RL1Y0nDI5jCvZogwonE5BdUfNVjYROsbCHq5dyEPP9VTG1LzPWzWXLxLvnw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=McCag0KGpa3dRuv6lc7U5CPZYXflkVHZ2NahGFznI4Y=; b=aP/7OKS6+UPeP/XcoEOMLP/68L+Oixo5EuvxSmlDBbfKFvasSBiytY8+zQzJPNn02rNr4UklwYacuuLh8wGtsMLOCyjx8LKvLcd3tJgF3smlC8V6yN6zmTx+aeSdvD+ooT1IYdajWVuefT/UsU++E+N8pAJ8u5xXCsE45V8eDqCKLjwX56JT07aBwbyRB9+mspVKQJk7Zfu7eizfmo57zFgxbcZX6ldN0G0YOGaoj1JqGMcScNgYPnc8M19T+eFgp2jo7UBfjQGGAB8ZY/udtXJv7FvBPZY5R6QXCGZ8RdRKZrIH54U4GbFMKbHxxz0DY5VkWX6JApfkX6+cm8pTfQ== Received: from DS7PR03CA0289.namprd03.prod.outlook.com (2603:10b6:5:3ad::24) by CH2PR12MB5017.namprd12.prod.outlook.com (2603:10b6:610:36::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.37; Thu, 23 Mar 2023 11:11:51 +0000 Received: from DS1PEPF0000E648.namprd02.prod.outlook.com (2603:10b6:5:3ad:cafe::a) by DS7PR03CA0289.outlook.office365.com (2603:10b6:5:3ad::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.38 via Frontend Transport; Thu, 23 Mar 2023 11:11:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS1PEPF0000E648.mail.protection.outlook.com (10.167.18.38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.30 via Frontend Transport; Thu, 23 Mar 2023 11:11:50 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Thu, 23 Mar 2023 04:11:45 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Thu, 23 Mar 2023 04:11:45 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.986.5 via Frontend Transport; Thu, 23 Mar 2023 04:11:42 -0700 From: Dima Chumak To: Jakub Kicinski CC: "David S. Miller" , Eric Dumazet , Paolo Abeni , Jiri Pirko , Leon Romanovsky , Saeed Mahameed , , Dima Chumak , Jiri Pirko Subject: [PATCH net-next 2/4] net/mlx5: Implement devlink port function cmds to control ipsec_crypto Date: Thu, 23 Mar 2023 13:10:57 +0200 Message-ID: <20230323111059.210634-3-dchumak@nvidia.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230323111059.210634-1-dchumak@nvidia.com> References: <20230323111059.210634-1-dchumak@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF0000E648:EE_|CH2PR12MB5017:EE_ X-MS-Office365-Filtering-Correlation-Id: 24237b52-4feb-4898-df38-08db2b8f66ee X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GkGLNoOKGdOfeq3Rrmqg6TKNDJwhVU7JYSPXJz+JIL0jJzR4I4MVJf0dhr3bzuCnTIJgXcPy+71tC/B3IQnCrZ8efRuZjC36C+tWQlLoGKfpwpuSd2gFMxCSDufKIP4ntjlagi2DsKyF40XBSP0wcXP2/0JYJrvRZQ7+faqpIpG1WczZf+i6+Nl21vejJMAQ5WIh5k5xeB0nmHdTvFz41AScaFFyRLJ8EmxJi1ySpQhNy0H0OqAJa9ltjX8RPt15m4i+IpA+bPtOKnNJOGwlUt9sIjY2PkP1h9jCQAmPqJbzvtftO0e7huulz9juZHVLtdFM0QciWCxFT7RlUPEAxEaX/N7h5ONcGEntZr1jlH+ivudxiLHMBtMj6eo5Rba+tjlqo/BWBSwnI9tPwCK95BgYEyCq3NZTG8x8FcdXn7qiJb+PUnv072QDxHADfxwI61xq3QQVeEnD+JYJWzUIq84jTi3Q0XtfpL8XWtSVPkv4iXHFo5Af7fY3cpEtZwWx0sAblOC/V4h/cGSaSl/Svx+Slo1RGW2NJh/8XoYuPyqPMTxvaytzosnKmoQA8tvlRgR6olzRgqxhIn56mTP2oYZezehYrwqRvBcjv2H4X3DXAYDL/SMbfy///63XLyQY4GoPb+t0GSJRdEHU9w9Unj3LB7LU7qL3BeMcmsVKuc6jr4SqUBaA79kC/BnsDLerQv4fNjYxrgLJtr5iTibG3YNhpBIg+JqPgx1PmHRFw8wd2J49ics2bDfHGZj2WJKdTUcW91b8qoNvVBZa6U6bhA== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230025)(4636009)(346002)(376002)(39860400002)(396003)(136003)(451199018)(46966006)(40470700004)(36840700001)(336012)(107886003)(26005)(1076003)(47076005)(6666004)(426003)(8676002)(70206006)(316002)(4326008)(7696005)(2616005)(54906003)(70586007)(6916009)(40480700001)(478600001)(8936002)(2906002)(41300700001)(36860700001)(5660300002)(30864003)(7636003)(82740400003)(356005)(83380400001)(186003)(86362001)(82310400005)(36756003)(40460700003)(309714004);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2023 11:11:50.8743 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 24237b52-4feb-4898-df38-08db2b8f66ee X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF0000E648.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB5017 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Implement devlink port function commands to enable / disable IPsec crypto offloads. This is used to control the IPsec capability of the device. When ipsec_crypto is enabled for a VF, it prevents adding IPsec crypto offloads on the PF, because the two cannot be active simultaneously due to HW constraints. Conversely, if there are any active IPsec crypto offloads on the PF, it's not allowed to enable ipsec_crypto on a VF, until PF IPsec offloads are cleared. Signed-off-by: Dima Chumak Reviewed-by: Jiri Pirko --- .../ethernet/mellanox/mlx5/switchdev.rst | 8 + .../net/ethernet/mellanox/mlx5/core/Makefile | 2 +- .../net/ethernet/mellanox/mlx5/core/devlink.c | 2 + .../mellanox/mlx5/core/en_accel/ipsec.c | 18 ++ .../ethernet/mellanox/mlx5/core/esw/ipsec.c | 271 ++++++++++++++++++ .../net/ethernet/mellanox/mlx5/core/eswitch.c | 29 ++ .../net/ethernet/mellanox/mlx5/core/eswitch.h | 20 ++ .../mellanox/mlx5/core/eswitch_offloads.c | 100 +++++++ .../ethernet/mellanox/mlx5/core/lib/ipsec.h | 41 +++ include/linux/mlx5/driver.h | 1 + include/linux/mlx5/mlx5_ifc.h | 3 + 11 files changed, 494 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/esw/ipsec.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/lib/ipsec.h diff --git a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/switchdev.rst b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/switchdev.rst index 01deedb71597..9a41da6b33ff 100644 --- a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/switchdev.rst +++ b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/switchdev.rst @@ -168,6 +168,14 @@ explicitly enable the VF migratable capability. mlx5 driver support devlink port function attr mechanism to setup migratable capability. (refer to Documentation/networking/devlink/devlink-port.rst) +IPsec crypto capability setup +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +User who wants mlx5 PCI VFs to be able to perform IPsec crypto offloading need +to explicitly enable the VF ipsec_crypto capability. + +mlx5 driver support devlink port function attr mechanism to setup ipsec_crypto +capability. (refer to Documentation/networking/devlink/devlink-port.rst) + SF state setup -------------- diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index 6c2f1d4a58ab..02ccf440a09f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -69,7 +69,7 @@ mlx5_core-$(CONFIG_MLX5_TC_SAMPLE) += en/tc/sample.o # mlx5_core-$(CONFIG_MLX5_ESWITCH) += eswitch.o eswitch_offloads.o eswitch_offloads_termtbl.o \ ecpf.o rdma.o esw/legacy.o \ - esw/debugfs.o esw/devlink_port.o esw/vporttbl.o esw/qos.o + esw/debugfs.o esw/devlink_port.o esw/vporttbl.o esw/qos.o esw/ipsec.o mlx5_core-$(CONFIG_MLX5_ESWITCH) += esw/acl/helper.o \ esw/acl/egress_lgcy.o esw/acl/egress_ofld.o \ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index 1ee2a472e1d2..6beea396401a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -324,6 +324,8 @@ static const struct devlink_ops mlx5_devlink_ops = { .port_fn_roce_set = mlx5_devlink_port_fn_roce_set, .port_fn_migratable_get = mlx5_devlink_port_fn_migratable_get, .port_fn_migratable_set = mlx5_devlink_port_fn_migratable_set, + .port_fn_ipsec_crypto_get = mlx5_devlink_port_fn_ipsec_crypto_get, + .port_fn_ipsec_crypto_set = mlx5_devlink_port_fn_ipsec_crypto_set, #endif #ifdef CONFIG_MLX5_SF_MANAGER .port_new = mlx5_devlink_sf_port_new, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c index 7b0d3de0ec6c..573769a6b002 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c @@ -37,6 +37,8 @@ #include #include "en.h" +#include "eswitch.h" +#include "lib/ipsec.h" #include "ipsec.h" #include "ipsec_rxtx.h" @@ -307,6 +309,7 @@ static int mlx5e_xfrm_add_state(struct xfrm_state *x, struct mlx5e_ipsec_sa_entry *sa_entry = NULL; struct net_device *netdev = x->xso.real_dev; struct mlx5e_ipsec *ipsec; + struct mlx5_eswitch *esw; struct mlx5e_priv *priv; int err; @@ -326,6 +329,11 @@ static int mlx5e_xfrm_add_state(struct xfrm_state *x, sa_entry->x = x; sa_entry->ipsec = ipsec; + esw = priv->mdev->priv.eswitch; + if (esw && mlx5_esw_vport_ipsec_offload_enabled(esw)) + return -EBUSY; + mlx5_eswitch_ipsec_offloads_count_inc(priv->mdev); + /* check esn */ mlx5e_ipsec_update_esn_state(sa_entry); @@ -361,6 +369,7 @@ static int mlx5e_xfrm_add_state(struct xfrm_state *x, err_hw_ctx: mlx5_ipsec_free_sa_ctx(sa_entry); err_xfrm: + mlx5_eswitch_ipsec_offloads_count_dec(priv->mdev); kfree(sa_entry); NL_SET_ERR_MSG_MOD(extack, "Device failed to offload this policy"); return err; @@ -374,6 +383,7 @@ static void mlx5e_xfrm_del_state(struct xfrm_state *x) old = xa_erase_bh(&ipsec->sadb, sa_entry->ipsec_obj_id); WARN_ON(old != sa_entry); + mlx5_eswitch_ipsec_offloads_count_dec(ipsec->mdev); } static void mlx5e_xfrm_free_state(struct xfrm_state *x) @@ -567,6 +577,7 @@ static int mlx5e_xfrm_add_policy(struct xfrm_policy *x, { struct net_device *netdev = x->xdo.real_dev; struct mlx5e_ipsec_pol_entry *pol_entry; + struct mlx5_eswitch *esw; struct mlx5e_priv *priv; int err; @@ -587,6 +598,11 @@ static int mlx5e_xfrm_add_policy(struct xfrm_policy *x, pol_entry->x = x; pol_entry->ipsec = priv->ipsec; + esw = priv->mdev->priv.eswitch; + if (esw && mlx5_esw_vport_ipsec_offload_enabled(esw)) + return -EBUSY; + mlx5_eswitch_ipsec_offloads_count_inc(priv->mdev); + mlx5e_ipsec_build_accel_pol_attrs(pol_entry, &pol_entry->attrs); err = mlx5e_accel_ipsec_fs_add_pol(pol_entry); if (err) @@ -596,6 +612,7 @@ static int mlx5e_xfrm_add_policy(struct xfrm_policy *x, return 0; err_fs: + mlx5_eswitch_ipsec_offloads_count_dec(priv->mdev); kfree(pol_entry); NL_SET_ERR_MSG_MOD(extack, "Device failed to offload this policy"); return err; @@ -605,6 +622,7 @@ static void mlx5e_xfrm_free_policy(struct xfrm_policy *x) { struct mlx5e_ipsec_pol_entry *pol_entry = to_ipsec_pol_entry(x); + mlx5_eswitch_ipsec_offloads_count_dec(pol_entry->ipsec->mdev); mlx5e_accel_ipsec_fs_del_pol(pol_entry); kfree(pol_entry); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/ipsec.c new file mode 100644 index 000000000000..ab67e375c87b --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/ipsec.c @@ -0,0 +1,271 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +// Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. + +#include +#include +#include "mlx5_core.h" +#include "eswitch.h" +#include "lib/ipsec.h" + +static int esw_ipsec_vf_query_generic(struct mlx5_core_dev *dev, u16 vport_num, bool *result) +{ + int query_sz = MLX5_ST_SZ_BYTES(query_hca_cap_out); + void *hca_cap = NULL, *query_cap = NULL; + int err; + + if (!MLX5_CAP_GEN(dev, vhca_resource_manager)) + return -EOPNOTSUPP; + + if (!mlx5_esw_ipsec_vf_offload_supported(dev)) + return 0; + + query_cap = kvzalloc(query_sz, GFP_KERNEL); + if (!query_cap) + return -ENOMEM; + + err = mlx5_vport_get_other_func_general_cap(dev, vport_num, query_cap); + if (err) + goto out; + + hca_cap = MLX5_ADDR_OF(query_hca_cap_out, query_cap, capability); + *result = MLX5_GET(cmd_hca_cap, hca_cap, ipsec_offload); +out: + kvfree(query_cap); + return err; +} + +enum esw_vport_ipsec_offload { + MLX5_ESW_VPORT_IPSEC_CRYPTO_OFFLOAD, +}; + +static int esw_ipsec_vf_query(struct mlx5_core_dev *dev, struct mlx5_vport *vport, bool *crypto) +{ + int query_sz = MLX5_ST_SZ_BYTES(query_hca_cap_out); + void *hca_cap = NULL, *query_cap = NULL; + bool ipsec_enabled; + int err; + + /* Querying IPsec caps only makes sense when generic ipsec_offload + * HCA cap is enabled + */ + err = esw_ipsec_vf_query_generic(dev, vport->index, &ipsec_enabled); + if (err) + return err; + if (!ipsec_enabled) { + *crypto = false; + return 0; + } + + query_cap = kvzalloc(query_sz, GFP_KERNEL); + if (!query_cap) + return -ENOMEM; + + err = mlx5_vport_get_other_func_cap(dev, vport->index, query_cap, MLX5_CAP_IPSEC); + if (err) + goto out; + + hca_cap = MLX5_ADDR_OF(query_hca_cap_out, query_cap, capability); + *crypto = MLX5_GET(ipsec_cap, hca_cap, ipsec_crypto_offload); +out: + kvfree(query_cap); + return err; +} + +static int esw_ipsec_vf_set_generic(struct mlx5_core_dev *dev, u16 vport_num, bool ipsec_ofld) +{ + int query_sz = MLX5_ST_SZ_BYTES(query_hca_cap_out); + int set_sz = MLX5_ST_SZ_BYTES(set_hca_cap_in); + void *hca_cap = NULL, *query_cap = NULL, *cap; + int ret; + + if (!MLX5_CAP_GEN(dev, vhca_resource_manager)) + return -EOPNOTSUPP; + + query_cap = kvzalloc(query_sz, GFP_KERNEL); + hca_cap = kvzalloc(set_sz, GFP_KERNEL); + if (!hca_cap || !query_cap) { + ret = -ENOMEM; + goto out; + } + + ret = mlx5_vport_get_other_func_general_cap(dev, vport_num, query_cap); + if (ret) + goto out; + + cap = MLX5_ADDR_OF(set_hca_cap_in, hca_cap, capability); + memcpy(cap, MLX5_ADDR_OF(query_hca_cap_out, query_cap, capability), + MLX5_UN_SZ_BYTES(hca_cap_union)); + MLX5_SET(cmd_hca_cap, cap, ipsec_offload, ipsec_ofld); + + MLX5_SET(set_hca_cap_in, hca_cap, opcode, MLX5_CMD_OP_SET_HCA_CAP); + MLX5_SET(set_hca_cap_in, hca_cap, other_function, 1); + MLX5_SET(set_hca_cap_in, hca_cap, function_id, vport_num); + + MLX5_SET(set_hca_cap_in, hca_cap, op_mod, + MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE << 1); + ret = mlx5_cmd_exec_in(dev, set_hca_cap, hca_cap); +out: + kvfree(hca_cap); + kvfree(query_cap); + return ret; +} + +static int esw_ipsec_vf_set_bytype(struct mlx5_core_dev *dev, struct mlx5_vport *vport, + bool enable, enum esw_vport_ipsec_offload type) +{ + int query_sz = MLX5_ST_SZ_BYTES(query_hca_cap_out); + int set_sz = MLX5_ST_SZ_BYTES(set_hca_cap_in); + void *hca_cap = NULL, *query_cap = NULL, *cap; + int ret; + + if (!MLX5_CAP_GEN(dev, vhca_resource_manager)) + return -EOPNOTSUPP; + + query_cap = kvzalloc(query_sz, GFP_KERNEL); + hca_cap = kvzalloc(set_sz, GFP_KERNEL); + if (!hca_cap || !query_cap) { + ret = -ENOMEM; + goto out; + } + + ret = mlx5_vport_get_other_func_cap(dev, vport->index, query_cap, MLX5_CAP_IPSEC); + if (ret) + goto out; + + cap = MLX5_ADDR_OF(set_hca_cap_in, hca_cap, capability); + memcpy(cap, MLX5_ADDR_OF(query_hca_cap_out, query_cap, capability), + MLX5_UN_SZ_BYTES(hca_cap_union)); + + switch (type) { + case MLX5_ESW_VPORT_IPSEC_CRYPTO_OFFLOAD: + MLX5_SET(ipsec_cap, cap, ipsec_crypto_offload, enable); + break; + default: + ret = -EOPNOTSUPP; + goto out; + } + + MLX5_SET(set_hca_cap_in, hca_cap, opcode, MLX5_CMD_OP_SET_HCA_CAP); + MLX5_SET(set_hca_cap_in, hca_cap, other_function, 1); + MLX5_SET(set_hca_cap_in, hca_cap, function_id, vport->index); + + MLX5_SET(set_hca_cap_in, hca_cap, op_mod, + MLX5_SET_HCA_CAP_OP_MOD_IPSEC << 1); + ret = mlx5_cmd_exec_in(dev, set_hca_cap, hca_cap); +out: + kvfree(hca_cap); + kvfree(query_cap); + return ret; +} + +static int esw_ipsec_vf_crypto_aux_caps_set(struct mlx5_core_dev *dev, u16 vport_num, bool enable) +{ + int query_sz = MLX5_ST_SZ_BYTES(query_hca_cap_out); + int set_sz = MLX5_ST_SZ_BYTES(set_hca_cap_in); + void *hca_cap = NULL, *query_cap = NULL, *cap; + struct mlx5_eswitch *esw = dev->priv.eswitch; + int ret; + + query_cap = kvzalloc(query_sz, GFP_KERNEL); + hca_cap = kvzalloc(set_sz, GFP_KERNEL); + if (!hca_cap || !query_cap) { + ret = -ENOMEM; + goto out; + } + + ret = mlx5_vport_get_other_func_cap(dev, vport_num, query_cap, MLX5_CAP_ETHERNET_OFFLOADS); + if (ret) + goto out; + + cap = MLX5_ADDR_OF(set_hca_cap_in, hca_cap, capability); + memcpy(cap, MLX5_ADDR_OF(query_hca_cap_out, query_cap, capability), + MLX5_UN_SZ_BYTES(hca_cap_union)); + MLX5_SET(per_protocol_networking_offload_caps, cap, insert_trailer, enable); + MLX5_SET(set_hca_cap_in, hca_cap, opcode, MLX5_CMD_OP_SET_HCA_CAP); + MLX5_SET(set_hca_cap_in, hca_cap, other_function, 1); + MLX5_SET(set_hca_cap_in, hca_cap, function_id, vport_num); + MLX5_SET(set_hca_cap_in, hca_cap, op_mod, + MLX5_SET_HCA_CAP_OP_MOD_ETHERNET_OFFLOADS << 1); + ret = mlx5_cmd_exec_in(esw->dev, set_hca_cap, hca_cap); +out: + kvfree(hca_cap); + kvfree(query_cap); + return ret; +} + +static int esw_ipsec_vf_offload_set_bytype(struct mlx5_eswitch *esw, struct mlx5_vport *vport, + bool enable, enum esw_vport_ipsec_offload type) +{ + struct mlx5_core_dev *dev = esw->dev; + int err = 0; + + if (vport->index == MLX5_VPORT_PF) + return -EOPNOTSUPP; + + if (!mlx5_esw_vport_ipsec_offload_enabled(esw) && mlx5_eswitch_ipsec_offloads_enabled(dev)) + return -EBUSY; + + if (type == MLX5_ESW_VPORT_IPSEC_CRYPTO_OFFLOAD) { + err = esw_ipsec_vf_crypto_aux_caps_set(dev, vport->index, enable); + if (err) { + mlx5_core_dbg(dev, + "Failed to set auxiliary caps for ipsec_crypto_offload: %d\n", + err); + return err; + } + } + + if (enable) { + err = esw_ipsec_vf_set_generic(dev, vport->index, enable); + if (err) { + mlx5_core_dbg(dev, "Failed to enable generic ipsec_offload: %d\n", err); + return err; + } + err = esw_ipsec_vf_set_bytype(dev, vport, enable, type); + if (err) { + mlx5_core_dbg(dev, "Failed to enable ipsec_offload type %d: %d\n", type, + err); + return err; + } + } else { + err = esw_ipsec_vf_set_bytype(dev, vport, enable, type); + if (err) { + mlx5_core_dbg(dev, "Failed to disable ipsec_offload type %d: %d\n", type, + err); + return err; + } + err = esw_ipsec_vf_set_generic(dev, vport->index, enable); + if (err) { + mlx5_core_dbg(dev, "Failed to disable generic ipsec_offload: %d\n", + err); + return err; + } + } + + if (type == MLX5_ESW_VPORT_IPSEC_CRYPTO_OFFLOAD) + vport->info.ipsec_crypto_enabled = enable; + + return err; +} + +bool mlx5_esw_ipsec_vf_offload_supported(struct mlx5_core_dev *dev) +{ + /* Old firmware doesn't support ipsec_offload capability for VFs. This + * can be detected by checking reformat_add_esp_trasport capability - + * when this cap isn't supported it means firmware cannot be trusted + * about what it reports for ipsec_offload cap. + */ + return MLX5_CAP_FLOWTABLE_NIC_TX(dev, reformat_add_esp_trasport); +} + +int mlx5_esw_ipsec_vf_offload_get(struct mlx5_core_dev *dev, struct mlx5_vport *vport, bool *crypto) +{ + return esw_ipsec_vf_query(dev, vport, crypto); +} + +int mlx5_esw_ipsec_vf_crypto_offload_set(struct mlx5_eswitch *esw, struct mlx5_vport *vport, + bool enable) +{ + return esw_ipsec_vf_offload_set_bytype(esw, vport, enable, + MLX5_ESW_VPORT_IPSEC_CRYPTO_OFFLOAD); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c index 0f052513fefa..7d4f19c21f48 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c @@ -47,6 +47,7 @@ #include "devlink.h" #include "ecpf.h" #include "en/mod_hdr.h" +#include "en_accel/ipsec.h" enum { MLX5_ACTION_NONE = 0, @@ -782,6 +783,7 @@ static void esw_vport_cleanup_acl(struct mlx5_eswitch *esw, static int mlx5_esw_vport_caps_get(struct mlx5_eswitch *esw, struct mlx5_vport *vport) { int query_out_sz = MLX5_ST_SZ_BYTES(query_hca_cap_out); + bool ipsec_crypto_enabled; void *query_ctx; void *hca_caps; int err; @@ -809,6 +811,11 @@ static int mlx5_esw_vport_caps_get(struct mlx5_eswitch *esw, struct mlx5_vport * hca_caps = MLX5_ADDR_OF(query_hca_cap_out, query_ctx, capability); vport->info.mig_enabled = MLX5_GET(cmd_hca_cap_2, hca_caps, migratable); + + err = mlx5_esw_ipsec_vf_offload_get(esw->dev, vport, &ipsec_crypto_enabled); + if (err) + goto out_free; + vport->info.ipsec_crypto_enabled = ipsec_crypto_enabled; out_free: kfree(query_ctx); return err; @@ -873,6 +880,23 @@ static void esw_vport_cleanup(struct mlx5_eswitch *esw, struct mlx5_vport *vport esw_vport_cleanup_acl(esw, vport); } +void mlx5_esw_vport_ipsec_offload_enable(struct mlx5_eswitch *esw) +{ + esw->enabled_ipsec_vf_count++; + WARN_ON(!esw->enabled_ipsec_vf_count); +} + +void mlx5_esw_vport_ipsec_offload_disable(struct mlx5_eswitch *esw) +{ + esw->enabled_ipsec_vf_count--; + WARN_ON(esw->enabled_ipsec_vf_count == U16_MAX); +} + +bool mlx5_esw_vport_ipsec_offload_enabled(struct mlx5_eswitch *esw) +{ + return !!esw->enabled_ipsec_vf_count; +} + int mlx5_esw_vport_enable(struct mlx5_eswitch *esw, u16 vport_num, enum mlx5_eswitch_vport_event enabled_events) { @@ -895,6 +919,8 @@ int mlx5_esw_vport_enable(struct mlx5_eswitch *esw, u16 vport_num, /* Sync with current vport context */ vport->enabled_events = enabled_events; vport->enabled = true; + if (vport->vport != MLX5_VPORT_PF && vport->info.ipsec_crypto_enabled) + mlx5_esw_vport_ipsec_offload_enable(esw); /* Esw manager is trusted by default. Host PF (vport 0) is trusted as well * in smartNIC as it's a vport group manager. @@ -953,6 +979,9 @@ void mlx5_esw_vport_disable(struct mlx5_eswitch *esw, u16 vport_num) MLX5_CAP_GEN(esw->dev, vhca_resource_manager)) mlx5_esw_vport_vhca_id_clear(esw, vport_num); + if (vport->vport != MLX5_VPORT_PF && vport->info.ipsec_crypto_enabled) + mlx5_esw_vport_ipsec_offload_disable(esw); + /* We don't assume VFs will cleanup after themselves. * Calling vport change handler while vport is disabled will cleanup * the vport resources. diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h index 19e9a77c4633..dc7949814b91 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h @@ -157,6 +157,7 @@ struct mlx5_vport_info { u8 trusted: 1; u8 roce_enabled: 1; u8 mig_enabled: 1; + u8 ipsec_crypto_enabled: 1; }; /* Vport context events */ @@ -343,6 +344,7 @@ struct mlx5_eswitch { } params; struct blocking_notifier_head n_head; struct dentry *dbgfs; + u16 enabled_ipsec_vf_count; }; void esw_offloads_disable(struct mlx5_eswitch *esw); @@ -519,6 +521,10 @@ int mlx5_devlink_port_fn_migratable_get(struct devlink_port *port, bool *is_enab struct netlink_ext_ack *extack); int mlx5_devlink_port_fn_migratable_set(struct devlink_port *port, bool enable, struct netlink_ext_ack *extack); +int mlx5_devlink_port_fn_ipsec_crypto_get(struct devlink_port *port, bool *is_enabled, + struct netlink_ext_ack *extack); +int mlx5_devlink_port_fn_ipsec_crypto_set(struct devlink_port *port, bool enable, + struct netlink_ext_ack *extack); void *mlx5_eswitch_get_uplink_priv(struct mlx5_eswitch *esw, u8 rep_type); int __mlx5_eswitch_set_vport_vlan(struct mlx5_eswitch *esw, @@ -653,6 +659,15 @@ mlx5_eswitch_enable_pf_vf_vports(struct mlx5_eswitch *esw, enum mlx5_eswitch_vport_event enabled_events); void mlx5_eswitch_disable_pf_vf_vports(struct mlx5_eswitch *esw); +bool mlx5_esw_ipsec_vf_offload_supported(struct mlx5_core_dev *dev); +int mlx5_esw_ipsec_vf_offload_get(struct mlx5_core_dev *dev, struct mlx5_vport *vport, + bool *crypto); +int mlx5_esw_ipsec_vf_crypto_offload_set(struct mlx5_eswitch *esw, struct mlx5_vport *vport, + bool enable); +void mlx5_esw_vport_ipsec_offload_enable(struct mlx5_eswitch *esw); +void mlx5_esw_vport_ipsec_offload_disable(struct mlx5_eswitch *esw); +bool mlx5_esw_vport_ipsec_offload_enabled(struct mlx5_eswitch *esw); + int mlx5_esw_vport_enable(struct mlx5_eswitch *esw, u16 vport_num, enum mlx5_eswitch_vport_event enabled_events); void mlx5_esw_vport_disable(struct mlx5_eswitch *esw, u16 vport_num); @@ -805,6 +820,11 @@ mlx5_eswitch_reload_reps(struct mlx5_eswitch *esw) { return 0; } + +static inline bool mlx5_esw_vport_ipsec_offload_enabled(struct mlx5_eswitch *esw) +{ + return false; +} #endif /* CONFIG_MLX5_ESWITCH */ #endif /* __MLX5_ESWITCH_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index 22075943bb58..fd546dd0a481 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -4139,3 +4139,103 @@ int mlx5_devlink_port_fn_roce_set(struct devlink_port *port, bool enable, mutex_unlock(&esw->state_lock); return err; } + +int mlx5_devlink_port_fn_ipsec_crypto_get(struct devlink_port *port, bool *is_enabled, + struct netlink_ext_ack *extack) +{ + struct mlx5_eswitch *esw; + struct mlx5_vport *vport; + int err = -EOPNOTSUPP; + + esw = mlx5_devlink_eswitch_get(port->devlink); + if (IS_ERR(esw)) + return PTR_ERR(esw); + + if (!mlx5_esw_ipsec_vf_offload_supported(esw->dev)) { + NL_SET_ERR_MSG_MOD(extack, "Device doesn't support ipsec_crypto"); + return err; + } + + vport = mlx5_devlink_port_fn_get_vport(port, esw); + if (IS_ERR(vport)) { + NL_SET_ERR_MSG_MOD(extack, "Invalid port"); + return PTR_ERR(vport); + } + + mutex_lock(&esw->state_lock); + if (vport->enabled) { + *is_enabled = vport->info.ipsec_crypto_enabled; + err = 0; + } + mutex_unlock(&esw->state_lock); + return err; +} + +int mlx5_devlink_port_fn_ipsec_crypto_set(struct devlink_port *port, bool enable, + struct netlink_ext_ack *extack) +{ + struct mlx5_eswitch *esw; + struct mlx5_vport *vport; + int err = -EOPNOTSUPP; + struct net *net; + + esw = mlx5_devlink_eswitch_get(port->devlink); + if (IS_ERR(esw)) + return PTR_ERR(esw); + + if (!mlx5_esw_ipsec_vf_offload_supported(esw->dev)) { + NL_SET_ERR_MSG_MOD(extack, "Device doesn't support ipsec_crypto"); + return err; + } + + vport = mlx5_devlink_port_fn_get_vport(port, esw); + if (IS_ERR(vport)) { + NL_SET_ERR_MSG_MOD(extack, "Invalid port"); + return PTR_ERR(vport); + } + + /* xfrm_cfg lock is needed to avoid races with XFRM state being added to + * the PF net device. Netlink stack takes this lock for `ip xfrm` user + * commands, so here we need to take it before esw->state_lock to + * preserve the order. + */ + net = dev_net(esw->dev->mlx5e_res.uplink_netdev); + mutex_lock(&net->xfrm.xfrm_cfg_mutex); + + mutex_lock(&esw->state_lock); + if (!vport->enabled) { + NL_SET_ERR_MSG_MOD(extack, "Eswitch vport is disabled"); + goto out; + } + if (vport->info.ipsec_crypto_enabled == enable) { + err = 0; + goto out; + } + + err = mlx5_esw_ipsec_vf_crypto_offload_set(esw, vport, enable); + switch (err) { + case 0: + break; + case -EBUSY: + NL_SET_ERR_MSG_MOD(extack, + "Failed setting ipsec_crypto. Make sure ip xfrm state/policy is cleared on the PF."); + goto out; + case -EINVAL: + NL_SET_ERR_MSG_MOD(extack, + "Failed setting ipsec_crypto. Make sure to unbind the VF first"); + goto out; + default: + NL_SET_ERR_MSG_MOD(extack, "Failed setting HCA ipsec_crypto_offload cap."); + goto out; + } + + vport->info.ipsec_crypto_enabled = enable; + if (enable) + mlx5_esw_vport_ipsec_offload_enable(esw); + else + mlx5_esw_vport_ipsec_offload_disable(esw); +out: + mutex_unlock(&esw->state_lock); + mutex_unlock(&net->xfrm.xfrm_cfg_mutex); + return err; +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/ipsec.h b/drivers/net/ethernet/mellanox/mlx5/core/lib/ipsec.h new file mode 100644 index 000000000000..cf0bca6d5f3e --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/ipsec.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef __MLX5_LIB_IPSEC_H__ +#define __MLX5_LIB_IPSEC_H__ + +#include + +#ifdef CONFIG_MLX5_EN_IPSEC + +/* The caller must hold mlx5_eswitch->state_lock */ +static inline void mlx5_eswitch_ipsec_offloads_count_inc(struct mlx5_core_dev *mdev) +{ + WARN_ON(mdev->ipsec_offloads_count == U64_MAX); + mdev->ipsec_offloads_count++; +} + +/* The caller must hold mlx5_eswitch->state_lock */ +static inline void mlx5_eswitch_ipsec_offloads_count_dec(struct mlx5_core_dev *mdev) +{ + WARN_ON(mdev->ipsec_offloads_count == 0); + mdev->ipsec_offloads_count--; +} + +/* The caller must hold mlx5_eswitch->state_lock */ +static inline bool mlx5_eswitch_ipsec_offloads_enabled(struct mlx5_core_dev *mdev) +{ + return !!mdev->ipsec_offloads_count; +} +#else +static inline void mlx5_eswitch_ipsec_offloads_count_inc(struct mlx5_core_dev *mdev) { } + +static inline void mlx5_eswitch_ipsec_offloads_count_dec(struct mlx5_core_dev *mdev) { } + +static inline bool mlx5_eswitch_ipsec_offloads_enabled(struct mlx5_core_dev *mdev) +{ + return false; +} +#endif /* CONFIG_MLX5_EN_IPSEC */ + +#endif /* __MLX5_LIB_IPSEC_H__ */ diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index 7a898113b6b7..a139c9a8ddb5 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -811,6 +811,7 @@ struct mlx5_core_dev { u32 vsc_addr; struct mlx5_hv_vhca *hv_vhca; struct mlx5_thermal *thermal; + u64 ipsec_offloads_count; }; struct mlx5_db { diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index e47d6c58da35..6e4a013b36ed 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -65,9 +65,11 @@ enum { enum { MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE = 0x0, + MLX5_SET_HCA_CAP_OP_MOD_ETHERNET_OFFLOADS = 0x1, MLX5_SET_HCA_CAP_OP_MOD_ODP = 0x2, MLX5_SET_HCA_CAP_OP_MOD_ATOMIC = 0x3, MLX5_SET_HCA_CAP_OP_MOD_ROCE = 0x4, + MLX5_SET_HCA_CAP_OP_MOD_IPSEC = 0x15, MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE2 = 0x20, MLX5_SET_HCA_CAP_OP_MODE_PORT_SELECTION = 0x25, }; @@ -3456,6 +3458,7 @@ union mlx5_ifc_hca_cap_union_bits { struct mlx5_ifc_shampo_cap_bits shampo_cap; struct mlx5_ifc_macsec_cap_bits macsec_cap; struct mlx5_ifc_crypto_cap_bits crypto_cap; + struct mlx5_ifc_ipsec_cap_bits ipsec_cap; u8 reserved_at_0[0x8000]; }; From patchwork Thu Mar 23 11:10:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Chumak X-Patchwork-Id: 13185523 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1992AC76195 for ; Thu, 23 Mar 2023 11:12:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231335AbjCWLML (ORCPT ); Thu, 23 Mar 2023 07:12:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229800AbjCWLMB (ORCPT ); Thu, 23 Mar 2023 07:12:01 -0400 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2049.outbound.protection.outlook.com [40.107.92.49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 657B11C5BB for ; Thu, 23 Mar 2023 04:11:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fuIvV8JpYjHmEey8OM7Yknu5Ih4C6rJDTsewwl47zaIWo5FKO37/jaFppFx4L6tmlxHX474eDrbMrWAqXdEini1d3g4Zjog/Zrigl7Bc58Phq0xxqJMg55z5K+hWeSOmoklsIdbkvS9oPpMBiDKLK3mkygh8JZMosVjd6odjR1kE+REhIo/Ag9Cgrrl/N/y4NHxykMbUAOtmFSojDFLSdkOgyMo//mm5n9JOoeRMp+IjVbvaeUNoGqAdfQ8a/XrIqek2yEdgwbwvEbyETX64OZAVWXaDPnryDNWwFYUFuJyqZsMfzOryPJEfLs3GXCwGsiNcXGv56OjW0q5+Ms9TiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eswZTtlcQgQCRXR85SavX8OTDA8g1VN/gIkigQkvsyM=; b=PfbXPpTTHqqde/+L3NJ2ypW8ckJ7wjtFl3hcWTqLZVUTtnxGddk78Z6pwROdAU129PGYGQnUg6BT0BrqxBgJCOlNBlqrQs7FWh994oyIBDjpyD9n6hp1ozJuxTIZw35OEy26PlU2Fj655Np9vQ7iwrKg+DWuT34ZGQqrk7pvMNKKfmJV6jZ8Qwc6xtMORD8VhQZjgvmIamun0kIxAkHGGXzpp3A8uYdmJ1uMf5oT/+OS2RrjyNYXfzmDL1sPxxuPWdyEvNAe7mLJkMzbs+k/2zKqwvEFxHK4laLR9Bl+XVzf7QEsb/IsVtDobPyj1hAJHvBH2sJMI1VAiz6KvZgYsg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eswZTtlcQgQCRXR85SavX8OTDA8g1VN/gIkigQkvsyM=; b=Z65JJhpoj/ij3DY7oAkrUAnhGRWVNAl0dfr4cknFa8htVpxhUhbnqIJnMI5ZIp9tzDjapUfDicltaR6kvA8vUfJWMftZv9lFZ7qRMC0ba584s7JOXsvV8+PnGFvKLIN5rWDBEe9HkWGPs1lbhPf4v6DYZsoznEkKdRuq/+6d/VTpHoHtwuTFXGwH/KpnPsg80wa1lJhNxgYtCl6am/qGIiFkvrYB4y8x1IfL9liDdixPSTu9vPw2mT3btF2v2PH+EXQ8vChiB7BMUOTuRj+ydjj9VuxBn3HRocpVzISJpcX97QewsHbU8ybz+htfjjRqLsRq1w0pO0JEPHe/G3PWsA== Received: from DM6PR14CA0046.namprd14.prod.outlook.com (2603:10b6:5:18f::23) by CH0PR12MB5107.namprd12.prod.outlook.com (2603:10b6:610:be::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.38; Thu, 23 Mar 2023 11:11:56 +0000 Received: from DS1PEPF0000E64C.namprd02.prod.outlook.com (2603:10b6:5:18f:cafe::48) by DM6PR14CA0046.outlook.office365.com (2603:10b6:5:18f::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.38 via Frontend Transport; Thu, 23 Mar 2023 11:11:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS1PEPF0000E64C.mail.protection.outlook.com (10.167.18.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.30 via Frontend Transport; Thu, 23 Mar 2023 11:11:56 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Thu, 23 Mar 2023 04:11:48 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Thu, 23 Mar 2023 04:11:48 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.986.5 via Frontend Transport; Thu, 23 Mar 2023 04:11:45 -0700 From: Dima Chumak To: Jakub Kicinski CC: "David S. Miller" , Eric Dumazet , Paolo Abeni , Jiri Pirko , Leon Romanovsky , Saeed Mahameed , , Dima Chumak , Jiri Pirko Subject: [PATCH net-next 3/4] devlink: Expose port function commands to control IPsec packet offloads Date: Thu, 23 Mar 2023 13:10:58 +0200 Message-ID: <20230323111059.210634-4-dchumak@nvidia.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230323111059.210634-1-dchumak@nvidia.com> References: <20230323111059.210634-1-dchumak@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF0000E64C:EE_|CH0PR12MB5107:EE_ X-MS-Office365-Filtering-Correlation-Id: 70459d81-0ab2-49b4-0633-08db2b8f6a41 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BLUwfF/nAFGP27gpBQlWKhFZeB6kw6a/aObolj2sHdgrAdv/YjZidXgJyaj3ja2YXzagsFtR8BBKJ/q44fJquOaAZ6gITbixqydZ1pb+pJ85aJ8ZIG4ohSwYOp1+y8c0X0K4ORraJTgL3vczryK7QFbLpCXDjQIOnAUsB0uWhX0hjKHq9bzcDPcK2mG4AuwLmSIjTWgPc/yGg/HwcGBbt+kG0DaEO0bHMJZGdoHPDPKxW8Jm7dZqUHAuxoyspRtKv3jVX6g0ceeA1HXVPGxlPoaAi6JrMad1Usld5NTQU1rlFrvYt8CzBRhXgrQnUqbu3SwDGyr9bnfyPEQoYJFRUzHI2gz7+3+VjRlEtYSuvFS2Ny6IBzOP8zsi88MDrFksV7GzBmZI+k6E0fVnigH67mi/UeMU/5Tjmc6w8abgH+1a29If7/gLFlsJVLnB71DKarDOXmajFprgzwnA0mffYvNLMqiY/ylszo7KVfjlqZ/hiavDHFw/jWn/+UonIIPAgoe/Q7dgdSAZG04c61373CxlIT3a+tRREDu73eHdwPWc7OektxKL68CofE454grV7Wt+JIMY7GTsrIzNBxAn7lDNbw+Ie4thf5ZewZCDxw6nAHwL3ZeZuFs+b0ucsnooBAD403yd7PWwwQPp/gI2j3kxmOjfTOG2ozcXH6QQYIKRwiVu7H1Z4u0h6ZruWsCcTD4vNcYkYhAwVopB4M7f+dDUy7Hfjxjn2srV1K1mMzz7wTGqYKWdf5hmACrUynLA X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230025)(4636009)(39860400002)(376002)(136003)(346002)(396003)(451199018)(40470700004)(46966006)(36840700001)(40460700003)(4326008)(6916009)(70206006)(70586007)(8676002)(36860700001)(316002)(54906003)(5660300002)(7636003)(82740400003)(41300700001)(8936002)(1076003)(26005)(426003)(47076005)(107886003)(6666004)(2616005)(336012)(186003)(83380400001)(478600001)(7696005)(86362001)(82310400005)(36756003)(40480700001)(356005)(2906002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2023 11:11:56.4458 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 70459d81-0ab2-49b4-0633-08db2b8f6a41 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF0000E64C.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5107 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Expose port function commands to enable / disable IPsec packet offloads, this is used to control the port IPsec capabilities. When IPsec is disabled for a function of the port (default), function cannot offload any IPsec packet operations. When enabled, IPsec packet operations can be offloaded by the function of the port. Example of a PCI VF port which supports IPsec packet offloads: $ devlink port show pci/0000:06:00.0/1 pci/0000:06:00.0/1: type eth netdev enp6s0pf0vf0 flavour pcivf pfnum 0 vfnum 0 function: hw_addr 00:00:00:00:00:00 roce enable ipsec_packet disable $ devlink port function set pci/0000:06:00.0/1 ipsec_packet enable $ devlink port show pci/0000:06:00.0/1 pci/0000:06:00.0/1: type eth netdev enp6s0pf0vf0 flavour pcivf pfnum 0 vfnum 0 function: hw_addr 00:00:00:00:00:00 roce enable ipsec_packet enable Signed-off-by: Dima Chumak Reviewed-by: Jiri Pirko --- .../networking/devlink/devlink-port.rst | 27 +++++++++ include/net/devlink.h | 21 +++++++ include/uapi/linux/devlink.h | 2 + net/devlink/leftover.c | 55 +++++++++++++++++++ 4 files changed, 105 insertions(+) diff --git a/Documentation/networking/devlink/devlink-port.rst b/Documentation/networking/devlink/devlink-port.rst index e7c7482714d7..05d95cf95284 100644 --- a/Documentation/networking/devlink/devlink-port.rst +++ b/Documentation/networking/devlink/devlink-port.rst @@ -131,6 +131,9 @@ Users may also set the function as migratable using Users may also set the IPsec crypto capability of the function using `devlink port function set ipsec_crypto` command. +Users may also set the IPsec packet capability of the function using +`devlink port function set ipsec_packet` command. + Function attributes =================== @@ -267,6 +270,30 @@ processed in software by the kernel. function: hw_addr 00:00:00:00:00:00 ipsec_crypto enabled +IPsec packet capability setup +----------------------------- +When user enables IPsec packet capability for a VF, user application can offload +XFRM state to this VF. + +When IPsec packet capability is disabled (default) for a VF, the XFRM state is +processed in software by the kernel. + +- Get IPsec packet capability of the VF device:: + + $ devlink port show pci/0000:06:00.0/2 + pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 + function: + hw_addr 00:00:00:00:00:00 ipsec_packet disabled + +- Set IPsec packet capability of the VF device:: + + $ devlink port function set pci/0000:06:00.0/2 ipsec_packet enable + + $ devlink port show pci/0000:06:00.0/2 + pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1 + function: + hw_addr 00:00:00:00:00:00 ipsec_packet enabled + Subfunction ============ diff --git a/include/net/devlink.h b/include/net/devlink.h index 4e5f4aeca29d..772453b36c20 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -1516,6 +1516,27 @@ struct devlink_ops { int (*port_fn_ipsec_crypto_set)(struct devlink_port *devlink_port, bool enable, struct netlink_ext_ack *extack); + /** + * @port_fn_ipsec_packet_get: Port function's ipsec_packet get function. + * + * Query ipsec_packet state of a function managed by the devlink port. + * Return -EOPNOTSUPP if port function IPsec packet offload is not + * supported. + */ + int (*port_fn_ipsec_packet_get)(struct devlink_port *devlink_port, + bool *is_enable, + struct netlink_ext_ack *extack); + /** + * @port_fn_ipsec_packet_set: Port function's ipsec_packet set function. + * + * Enable/Disable ipsec_packet state of a function managed by the devlink + * port. + * Return -EOPNOTSUPP if port function IPsec packet offload is not + * supported. + */ + int (*port_fn_ipsec_packet_set)(struct devlink_port *devlink_port, + bool enable, + struct netlink_ext_ack *extack); /** * port_new() - Add a new port function of a specified flavor * @devlink: Devlink instance diff --git a/include/uapi/linux/devlink.h b/include/uapi/linux/devlink.h index f9ae9a058ad2..03875e078be8 100644 --- a/include/uapi/linux/devlink.h +++ b/include/uapi/linux/devlink.h @@ -662,6 +662,7 @@ enum devlink_port_fn_attr_cap { DEVLINK_PORT_FN_ATTR_CAP_ROCE_BIT, DEVLINK_PORT_FN_ATTR_CAP_MIGRATABLE_BIT, DEVLINK_PORT_FN_ATTR_CAP_IPSEC_CRYPTO_BIT, + DEVLINK_PORT_FN_ATTR_CAP_IPSEC_PACKET_BIT, /* Add new caps above */ __DEVLINK_PORT_FN_ATTR_CAPS_MAX, @@ -671,6 +672,7 @@ enum devlink_port_fn_attr_cap { #define DEVLINK_PORT_FN_CAP_MIGRATABLE \ _BITUL(DEVLINK_PORT_FN_ATTR_CAP_MIGRATABLE_BIT) #define DEVLINK_PORT_FN_CAP_IPSEC_CRYPTO _BITUL(DEVLINK_PORT_FN_ATTR_CAP_IPSEC_CRYPTO_BIT) +#define DEVLINK_PORT_FN_CAP_IPSEC_PACKET _BITUL(DEVLINK_PORT_FN_ATTR_CAP_IPSEC_PACKET_BIT) enum devlink_port_function_attr { DEVLINK_PORT_FUNCTION_ATTR_UNSPEC, diff --git a/net/devlink/leftover.c b/net/devlink/leftover.c index 07761df2471d..8cadfeb285a9 100644 --- a/net/devlink/leftover.c +++ b/net/devlink/leftover.c @@ -515,6 +515,29 @@ static int devlink_port_fn_ipsec_crypto_fill(const struct devlink_ops *ops, return 0; } +static int devlink_port_fn_ipsec_packet_fill(const struct devlink_ops *ops, + struct devlink_port *devlink_port, + struct nla_bitfield32 *caps, + struct netlink_ext_ack *extack) +{ + bool is_enable; + int err; + + if (!ops->port_fn_ipsec_packet_get || + devlink_port->attrs.flavour != DEVLINK_PORT_FLAVOUR_PCI_VF) + return 0; + + err = ops->port_fn_ipsec_packet_get(devlink_port, &is_enable, extack); + if (err) { + if (err == -EOPNOTSUPP) + return 0; + return err; + } + + devlink_port_fn_cap_fill(caps, DEVLINK_PORT_FN_CAP_IPSEC_PACKET, is_enable); + return 0; +} + static int devlink_port_fn_caps_fill(const struct devlink_ops *ops, struct devlink_port *devlink_port, struct sk_buff *msg, @@ -536,6 +559,10 @@ static int devlink_port_fn_caps_fill(const struct devlink_ops *ops, if (err) return err; + err = devlink_port_fn_ipsec_packet_fill(ops, devlink_port, &caps, extack); + if (err) + return err; + if (!caps.selector) return 0; err = nla_put_bitfield32(msg, DEVLINK_PORT_FN_ATTR_CAPS, caps.value, @@ -879,6 +906,15 @@ devlink_port_fn_ipsec_crypto_set(struct devlink_port *devlink_port, bool enable, return ops->port_fn_ipsec_crypto_set(devlink_port, enable, extack); } +static int +devlink_port_fn_ipsec_packet_set(struct devlink_port *devlink_port, bool enable, + struct netlink_ext_ack *extack) +{ + const struct devlink_ops *ops = devlink_port->devlink->ops; + + return ops->port_fn_ipsec_packet_set(devlink_port, enable, extack); +} + static int devlink_port_fn_caps_set(struct devlink_port *devlink_port, const struct nlattr *attr, struct netlink_ext_ack *extack) @@ -910,6 +946,13 @@ static int devlink_port_fn_caps_set(struct devlink_port *devlink_port, if (err) return err; } + if (caps.selector & DEVLINK_PORT_FN_CAP_IPSEC_PACKET) { + err = devlink_port_fn_ipsec_packet_set(devlink_port, caps_value & + DEVLINK_PORT_FN_CAP_IPSEC_PACKET, + extack); + if (err) + return err; + } return 0; } @@ -1290,6 +1333,18 @@ static int devlink_port_function_validate(struct devlink_port *devlink_port, return -EOPNOTSUPP; } } + if (caps.selector & DEVLINK_PORT_FN_CAP_IPSEC_PACKET) { + if (!ops->port_fn_ipsec_packet_set) { + NL_SET_ERR_MSG_ATTR(extack, attr, + "Port doesn't support ipsec_packet function attribute"); + return -EOPNOTSUPP; + } + if (devlink_port->attrs.flavour != DEVLINK_PORT_FLAVOUR_PCI_VF) { + NL_SET_ERR_MSG_ATTR(extack, attr, + "ipsec_packet function attribute supported for VFs only"); + return -EOPNOTSUPP; + } + } } return 0; } From patchwork Thu Mar 23 11:10:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Chumak X-Patchwork-Id: 13185524 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9FBEC6FD1C for ; Thu, 23 Mar 2023 11:12:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231548AbjCWLMY (ORCPT ); Thu, 23 Mar 2023 07:12:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229849AbjCWLMK (ORCPT ); Thu, 23 Mar 2023 07:12:10 -0400 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on20600.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe59::600]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01F992E0F1 for ; Thu, 23 Mar 2023 04:12:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Fdi6JTOxI1itR7r7Nb4vyrh0OUiyDcQskHCWCVvWtjhcIbMQuor5JJcpqHZliD/ufeSsDBAme8CD9rhWQkapFOVsdPeFQtbLGoioB8M11qQkB54BbFy+FXN+knmN5bkCvJTthENIUuO1HRNBfKmM0KRxQNhKA/c6pwMrGfx2BwthKaLsKjfIp7rF6IGMZpUwgmSaK3b6cK1HnAbWzhliLOk1eXMi4LfdVsBkp2eVabt0Vp6a0UeazW0dfvm0vv9yOhXKBGUep9qV00DgePRxVe09eG/KGtxCtl38pllyv+CAeHwFJ7VPJIgQN9Cqgg2dsEsEY9VsXtyPVR0LWB5c0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XAr94i1vxEPdl7xyvwcVcFGD1Z8ewH5tyBnU7gnW1L4=; b=I8HMG9+lCACajUERkaMU/GV/52OqQQV4wCN6thhLCAgVH7eUMGtusBpkjrEmxqKpDISgvNpREaPI4+gztyCKrxvGFPIO4YxBumPEnxhYILMqkUW6E8o7LipkUAyQG7OI6Pr5lVgT3tx3OIlKIGQOGbHFrmdUHSEJYytIx043ghfB53zDEYR8q9P0FSXX/+/6qsj9Dbx8oA4DLNDntN3/Ii2S8HEnbFR5HHny+ycyvkutm5MNhX1U678odJCneeuAxlATjBjMfBjg8iP5AybhfH2wxA9E+qFEUaDmAuZ0y328zJTBnYEo3ooSMgOkWOBtO5gp+1g/ekVrHBwv6DZjbw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XAr94i1vxEPdl7xyvwcVcFGD1Z8ewH5tyBnU7gnW1L4=; b=pt5PqkZSP0r/2eMJlsZtg00e5XHcJILfDtkKuAhqpm8Zi1LbNaK4xowH/6b9TFQDx6gRwnkLv+uj3/9cgZR/JPTcJwOP8ps3lV8C62I97HGTwGnKcrJ53YV5juMeC1NZbi1okOkJ5JmUDhWmQG76MXpsJ7p4AhshcVhwOVZIJDuF+ciKZIQM3jZGPwK4//HBJg3EfueKTN0yg70GPv890LpyqRr6elKodOtmsWr5zT7O172Q8fXdy2rdzhOUru0X6K7Uc636qk3HOMINUs9q8onaQOuIYJTrmeR5UCB+qkmtzCxngiU1EjY6qlkJRItGQ6lft5KIqA0zMHyS0bQbcQ== Received: from DM6PR08CA0029.namprd08.prod.outlook.com (2603:10b6:5:80::42) by DM4PR12MB6613.namprd12.prod.outlook.com (2603:10b6:8:b8::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.37; Thu, 23 Mar 2023 11:11:59 +0000 Received: from DS1PEPF0000E650.namprd02.prod.outlook.com (2603:10b6:5:80:cafe::71) by DM6PR08CA0029.outlook.office365.com (2603:10b6:5:80::42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.38 via Frontend Transport; Thu, 23 Mar 2023 11:11:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS1PEPF0000E650.mail.protection.outlook.com (10.167.18.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.30 via Frontend Transport; Thu, 23 Mar 2023 11:11:59 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Thu, 23 Mar 2023 04:11:52 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Thu, 23 Mar 2023 04:11:51 -0700 Received: from vdi.nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.986.5 via Frontend Transport; Thu, 23 Mar 2023 04:11:49 -0700 From: Dima Chumak To: Jakub Kicinski CC: "David S. Miller" , Eric Dumazet , Paolo Abeni , Jiri Pirko , Leon Romanovsky , Saeed Mahameed , , Dima Chumak , Jiri Pirko Subject: [PATCH net-next 4/4] net/mlx5: Implement devlink port function cmds to control ipsec_packet Date: Thu, 23 Mar 2023 13:10:59 +0200 Message-ID: <20230323111059.210634-5-dchumak@nvidia.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230323111059.210634-1-dchumak@nvidia.com> References: <20230323111059.210634-1-dchumak@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF0000E650:EE_|DM4PR12MB6613:EE_ X-MS-Office365-Filtering-Correlation-Id: 49092fde-1184-4b68-e4fc-08db2b8f6c2e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xF+jEhSDqvbqq7Dzm813A/UudM41KM1yKgMOtmC1ZybjhU6ppZW9g1SugG5ce8oK4zEFWuyQWuJoVlcXJqWZLsyP6NaOw2K4cF+oZxO5avUKxaLX2tUjI2DvBlrhCaISfknUlL1BU3WJTcr6pRuCGWnQnzTBd7LqkgkZ4QuhMwKXRVZrSmXDgJbuVrRTuJcEPNAQPA+sAjMtVrgEZDrZ665+vC92J2mMfGfeJl+JMwRkdZpk9L2mrjtH1mnsWwyiLji5b0AfCrfYzTgZ/EhljF5c+kiTnPByt0i4vz/s3pjKuPLExhb2HqpGrkubE6rC2kJeJgn8yaYiRH/C8zlACq1cpRGmWy0B5tuwf9ovpHaSTUm4JDWxODpHR1ufnK93JN7wj+rwsbDGtg2EDPToIrPBPMQrJNHbhUOjhxwnGKXf/KwBg2PyJyphOtUSP9ujcUrlNNJ86D08oRN2P03QKG5pDt1tdw4n8pUnChWeMoQ1QW5aPyZjFUPjG1Yf0MaRNaFcyIzJ2Pw5SZnYS9XAdIgC7j1CtdW6bPohy10OsRIX0Ae4w/gvr5j8Co1KLuDg0pkPlUvuaBStZR+9O0bLC3fPoXy5OcS7gB7uYlX8nwFJJUSwS+sk8i8hNCLsw9qRj1K3uK49ORc29s0NQtms9R7ha/Q8Poim7lsTa1lYxpw4IpolhhUUXxGdwuDyP5L5KC0Ef5ERs93C4iQWR5tD+Qn91rhU/YbHtiln+uUBNQosA87J/7ZC354wb5lIK2hM X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230025)(4636009)(39860400002)(346002)(396003)(376002)(136003)(451199018)(40470700004)(46966006)(36840700001)(4326008)(70206006)(6916009)(8676002)(41300700001)(70586007)(5660300002)(30864003)(8936002)(40460700003)(2906002)(7636003)(86362001)(356005)(36756003)(82740400003)(7696005)(316002)(6666004)(1076003)(186003)(26005)(54906003)(82310400005)(40480700001)(36860700001)(107886003)(478600001)(83380400001)(2616005)(47076005)(426003)(336012);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2023 11:11:59.6981 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 49092fde-1184-4b68-e4fc-08db2b8f6c2e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF0000E650.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6613 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Implement devlink port function commands to enable / disable IPsec packet offloads. This is used to control the IPsec capability of the device. When ipsec_offload is enabled for a VF, it prevents adding IPsec packet offloads on the PF, because the two cannot be active simultaneously due to HW constraints. Conversely, if there are any active IPsec packet offloads on the PF, it's not allowed to enable ipsec_packet on a VF, until PF IPsec offloads are cleared. Signed-off-by: Dima Chumak Reviewed-by: Jiri Pirko --- .../ethernet/mellanox/mlx5/switchdev.rst | 8 ++ .../net/ethernet/mellanox/mlx5/core/devlink.c | 2 + .../ethernet/mellanox/mlx5/core/esw/ipsec.c | 40 +++++-- .../net/ethernet/mellanox/mlx5/core/eswitch.c | 11 +- .../net/ethernet/mellanox/mlx5/core/eswitch.h | 9 +- .../mellanox/mlx5/core/eswitch_offloads.c | 100 ++++++++++++++++++ 6 files changed, 160 insertions(+), 10 deletions(-) diff --git a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/switchdev.rst b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/switchdev.rst index 9a41da6b33ff..ccfb02e7c2ad 100644 --- a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/switchdev.rst +++ b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/switchdev.rst @@ -176,6 +176,14 @@ to explicitly enable the VF ipsec_crypto capability. mlx5 driver support devlink port function attr mechanism to setup ipsec_crypto capability. (refer to Documentation/networking/devlink/devlink-port.rst) +IPsec packet capability setup +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +User who wants mlx5 PCI VFs to be able to perform IPsec packet offloading need +to explicitly enable the VF ipsec_packet capability. + +mlx5 driver support devlink port function attr mechanism to setup ipsec_packet +capability. (refer to Documentation/networking/devlink/devlink-port.rst) + SF state setup -------------- diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index 6beea396401a..36b7bb528d09 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -326,6 +326,8 @@ static const struct devlink_ops mlx5_devlink_ops = { .port_fn_migratable_set = mlx5_devlink_port_fn_migratable_set, .port_fn_ipsec_crypto_get = mlx5_devlink_port_fn_ipsec_crypto_get, .port_fn_ipsec_crypto_set = mlx5_devlink_port_fn_ipsec_crypto_set, + .port_fn_ipsec_packet_get = mlx5_devlink_port_fn_ipsec_packet_get, + .port_fn_ipsec_packet_set = mlx5_devlink_port_fn_ipsec_packet_set, #endif #ifdef CONFIG_MLX5_SF_MANAGER .port_new = mlx5_devlink_sf_port_new, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/ipsec.c index ab67e375c87b..af653bcadbb4 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/esw/ipsec.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/ipsec.c @@ -36,9 +36,11 @@ static int esw_ipsec_vf_query_generic(struct mlx5_core_dev *dev, u16 vport_num, enum esw_vport_ipsec_offload { MLX5_ESW_VPORT_IPSEC_CRYPTO_OFFLOAD, + MLX5_ESW_VPORT_IPSEC_PACKET_OFFLOAD, }; -static int esw_ipsec_vf_query(struct mlx5_core_dev *dev, struct mlx5_vport *vport, bool *crypto) +static int esw_ipsec_vf_query(struct mlx5_core_dev *dev, struct mlx5_vport *vport, + bool *crypto, bool *packet) { int query_sz = MLX5_ST_SZ_BYTES(query_hca_cap_out); void *hca_cap = NULL, *query_cap = NULL; @@ -53,6 +55,7 @@ static int esw_ipsec_vf_query(struct mlx5_core_dev *dev, struct mlx5_vport *vpor return err; if (!ipsec_enabled) { *crypto = false; + *packet = false; return 0; } @@ -66,6 +69,7 @@ static int esw_ipsec_vf_query(struct mlx5_core_dev *dev, struct mlx5_vport *vpor hca_cap = MLX5_ADDR_OF(query_hca_cap_out, query_cap, capability); *crypto = MLX5_GET(ipsec_cap, hca_cap, ipsec_crypto_offload); + *packet = MLX5_GET(ipsec_cap, hca_cap, ipsec_full_offload); out: kvfree(query_cap); return err; @@ -140,6 +144,9 @@ static int esw_ipsec_vf_set_bytype(struct mlx5_core_dev *dev, struct mlx5_vport case MLX5_ESW_VPORT_IPSEC_CRYPTO_OFFLOAD: MLX5_SET(ipsec_cap, cap, ipsec_crypto_offload, enable); break; + case MLX5_ESW_VPORT_IPSEC_PACKET_OFFLOAD: + MLX5_SET(ipsec_cap, cap, ipsec_full_offload, enable); + break; default: ret = -EOPNOTSUPP; goto out; @@ -197,6 +204,7 @@ static int esw_ipsec_vf_offload_set_bytype(struct mlx5_eswitch *esw, struct mlx5 bool enable, enum esw_vport_ipsec_offload type) { struct mlx5_core_dev *dev = esw->dev; + bool crypto_enabled, packet_enabled; int err = 0; if (vport->index == MLX5_VPORT_PF) @@ -234,16 +242,28 @@ static int esw_ipsec_vf_offload_set_bytype(struct mlx5_eswitch *esw, struct mlx5 err); return err; } - err = esw_ipsec_vf_set_generic(dev, vport->index, enable); + err = mlx5_esw_ipsec_vf_offload_get(dev, vport, &crypto_enabled, &packet_enabled); if (err) { - mlx5_core_dbg(dev, "Failed to disable generic ipsec_offload: %d\n", - err); + mlx5_core_dbg(dev, "Failed to get ipsec_offload caps: %d\n", err); return err; } + /* The generic ipsec_offload cap can be disabled only if both + * ipsec_crypto_offload and ipsec_full_offload aren't enabled. + */ + if (!crypto_enabled && !packet_enabled) { + err = esw_ipsec_vf_set_generic(dev, vport->index, enable); + if (err) { + mlx5_core_dbg(dev, "Failed to disable generic ipsec_offload: %d\n", + err); + return err; + } + } } if (type == MLX5_ESW_VPORT_IPSEC_CRYPTO_OFFLOAD) vport->info.ipsec_crypto_enabled = enable; + else if (type == MLX5_ESW_VPORT_IPSEC_PACKET_OFFLOAD) + vport->info.ipsec_packet_enabled = enable; return err; } @@ -258,9 +278,10 @@ bool mlx5_esw_ipsec_vf_offload_supported(struct mlx5_core_dev *dev) return MLX5_CAP_FLOWTABLE_NIC_TX(dev, reformat_add_esp_trasport); } -int mlx5_esw_ipsec_vf_offload_get(struct mlx5_core_dev *dev, struct mlx5_vport *vport, bool *crypto) +int mlx5_esw_ipsec_vf_offload_get(struct mlx5_core_dev *dev, struct mlx5_vport *vport, + bool *crypto, bool *packet) { - return esw_ipsec_vf_query(dev, vport, crypto); + return esw_ipsec_vf_query(dev, vport, crypto, packet); } int mlx5_esw_ipsec_vf_crypto_offload_set(struct mlx5_eswitch *esw, struct mlx5_vport *vport, @@ -269,3 +290,10 @@ int mlx5_esw_ipsec_vf_crypto_offload_set(struct mlx5_eswitch *esw, struct mlx5_v return esw_ipsec_vf_offload_set_bytype(esw, vport, enable, MLX5_ESW_VPORT_IPSEC_CRYPTO_OFFLOAD); } + +int mlx5_esw_ipsec_vf_packet_offload_set(struct mlx5_eswitch *esw, struct mlx5_vport *vport, + bool enable) +{ + return esw_ipsec_vf_offload_set_bytype(esw, vport, enable, + MLX5_ESW_VPORT_IPSEC_PACKET_OFFLOAD); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c index 7d4f19c21f48..65d52bba1b60 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c @@ -784,6 +784,7 @@ static int mlx5_esw_vport_caps_get(struct mlx5_eswitch *esw, struct mlx5_vport * { int query_out_sz = MLX5_ST_SZ_BYTES(query_hca_cap_out); bool ipsec_crypto_enabled; + bool ipsec_packet_enabled; void *query_ctx; void *hca_caps; int err; @@ -812,10 +813,12 @@ static int mlx5_esw_vport_caps_get(struct mlx5_eswitch *esw, struct mlx5_vport * hca_caps = MLX5_ADDR_OF(query_hca_cap_out, query_ctx, capability); vport->info.mig_enabled = MLX5_GET(cmd_hca_cap_2, hca_caps, migratable); - err = mlx5_esw_ipsec_vf_offload_get(esw->dev, vport, &ipsec_crypto_enabled); + err = mlx5_esw_ipsec_vf_offload_get(esw->dev, vport, &ipsec_crypto_enabled, + &ipsec_packet_enabled); if (err) goto out_free; vport->info.ipsec_crypto_enabled = ipsec_crypto_enabled; + vport->info.ipsec_packet_enabled = ipsec_packet_enabled; out_free: kfree(query_ctx); return err; @@ -919,7 +922,8 @@ int mlx5_esw_vport_enable(struct mlx5_eswitch *esw, u16 vport_num, /* Sync with current vport context */ vport->enabled_events = enabled_events; vport->enabled = true; - if (vport->vport != MLX5_VPORT_PF && vport->info.ipsec_crypto_enabled) + if (vport->vport != MLX5_VPORT_PF && + (vport->info.ipsec_crypto_enabled || vport->info.ipsec_packet_enabled)) mlx5_esw_vport_ipsec_offload_enable(esw); /* Esw manager is trusted by default. Host PF (vport 0) is trusted as well @@ -979,7 +983,8 @@ void mlx5_esw_vport_disable(struct mlx5_eswitch *esw, u16 vport_num) MLX5_CAP_GEN(esw->dev, vhca_resource_manager)) mlx5_esw_vport_vhca_id_clear(esw, vport_num); - if (vport->vport != MLX5_VPORT_PF && vport->info.ipsec_crypto_enabled) + if (vport->vport != MLX5_VPORT_PF && + (vport->info.ipsec_crypto_enabled || vport->info.ipsec_packet_enabled)) mlx5_esw_vport_ipsec_offload_disable(esw); /* We don't assume VFs will cleanup after themselves. diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h index dc7949814b91..43996101d784 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h @@ -158,6 +158,7 @@ struct mlx5_vport_info { u8 roce_enabled: 1; u8 mig_enabled: 1; u8 ipsec_crypto_enabled: 1; + u8 ipsec_packet_enabled: 1; }; /* Vport context events */ @@ -525,6 +526,10 @@ int mlx5_devlink_port_fn_ipsec_crypto_get(struct devlink_port *port, bool *is_en struct netlink_ext_ack *extack); int mlx5_devlink_port_fn_ipsec_crypto_set(struct devlink_port *port, bool enable, struct netlink_ext_ack *extack); +int mlx5_devlink_port_fn_ipsec_packet_get(struct devlink_port *port, bool *is_enabled, + struct netlink_ext_ack *extack); +int mlx5_devlink_port_fn_ipsec_packet_set(struct devlink_port *port, bool enable, + struct netlink_ext_ack *extack); void *mlx5_eswitch_get_uplink_priv(struct mlx5_eswitch *esw, u8 rep_type); int __mlx5_eswitch_set_vport_vlan(struct mlx5_eswitch *esw, @@ -661,9 +666,11 @@ void mlx5_eswitch_disable_pf_vf_vports(struct mlx5_eswitch *esw); bool mlx5_esw_ipsec_vf_offload_supported(struct mlx5_core_dev *dev); int mlx5_esw_ipsec_vf_offload_get(struct mlx5_core_dev *dev, struct mlx5_vport *vport, - bool *crypto); + bool *crypto, bool *packet); int mlx5_esw_ipsec_vf_crypto_offload_set(struct mlx5_eswitch *esw, struct mlx5_vport *vport, bool enable); +int mlx5_esw_ipsec_vf_packet_offload_set(struct mlx5_eswitch *esw, struct mlx5_vport *vport, + bool enable); void mlx5_esw_vport_ipsec_offload_enable(struct mlx5_eswitch *esw); void mlx5_esw_vport_ipsec_offload_disable(struct mlx5_eswitch *esw); bool mlx5_esw_vport_ipsec_offload_enabled(struct mlx5_eswitch *esw); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index fd546dd0a481..444ee8712584 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -4239,3 +4239,103 @@ int mlx5_devlink_port_fn_ipsec_crypto_set(struct devlink_port *port, bool enable mutex_unlock(&net->xfrm.xfrm_cfg_mutex); return err; } + +int mlx5_devlink_port_fn_ipsec_packet_get(struct devlink_port *port, bool *is_enabled, + struct netlink_ext_ack *extack) +{ + struct mlx5_eswitch *esw; + struct mlx5_vport *vport; + int err = -EOPNOTSUPP; + + esw = mlx5_devlink_eswitch_get(port->devlink); + if (IS_ERR(esw)) + return PTR_ERR(esw); + + if (!mlx5_esw_ipsec_vf_offload_supported(esw->dev)) { + NL_SET_ERR_MSG_MOD(extack, "Device doesn't support ipsec_packet"); + return err; + } + + vport = mlx5_devlink_port_fn_get_vport(port, esw); + if (IS_ERR(vport)) { + NL_SET_ERR_MSG_MOD(extack, "Invalid port"); + return PTR_ERR(vport); + } + + mutex_lock(&esw->state_lock); + if (vport->enabled) { + *is_enabled = vport->info.ipsec_packet_enabled; + err = 0; + } + mutex_unlock(&esw->state_lock); + return err; +} + +int mlx5_devlink_port_fn_ipsec_packet_set(struct devlink_port *port, bool enable, + struct netlink_ext_ack *extack) +{ + struct mlx5_eswitch *esw; + struct mlx5_vport *vport; + int err = -EOPNOTSUPP; + struct net *net; + + esw = mlx5_devlink_eswitch_get(port->devlink); + if (IS_ERR(esw)) + return PTR_ERR(esw); + + if (!mlx5_esw_ipsec_vf_offload_supported(esw->dev)) { + NL_SET_ERR_MSG_MOD(extack, "Device doesn't support ipsec_packet"); + return err; + } + + vport = mlx5_devlink_port_fn_get_vport(port, esw); + if (IS_ERR(vport)) { + NL_SET_ERR_MSG_MOD(extack, "Invalid port"); + return PTR_ERR(vport); + } + + /* xfrm_cfg lock is needed to avoid races with XFRM state being added to + * the PF net device. Netlink stack takes this lock for `ip xfrm` user + * commands, so here we need to take it before esw->state_lock to + * preserve the order. + */ + net = dev_net(esw->dev->mlx5e_res.uplink_netdev); + mutex_lock(&net->xfrm.xfrm_cfg_mutex); + + mutex_lock(&esw->state_lock); + if (!vport->enabled) { + NL_SET_ERR_MSG_MOD(extack, "Eswitch vport is disabled"); + goto out; + } + if (vport->info.ipsec_packet_enabled == enable) { + err = 0; + goto out; + } + + err = mlx5_esw_ipsec_vf_packet_offload_set(esw, vport, enable); + switch (err) { + case 0: + break; + case -EBUSY: + NL_SET_ERR_MSG_MOD(extack, + "Failed setting ipsec_packet. Make sure ip xfrm state/policy is cleared on the PF."); + goto out; + case -EINVAL: + NL_SET_ERR_MSG_MOD(extack, + "Failed setting ipsec_packet. Make sure to unbind the VF first"); + goto out; + default: + NL_SET_ERR_MSG_MOD(extack, "Failed setting HCA ipsec_full_offload cap."); + goto out; + } + + vport->info.ipsec_packet_enabled = enable; + if (enable) + mlx5_esw_vport_ipsec_offload_enable(esw); + else + mlx5_esw_vport_ipsec_offload_disable(esw); +out: + mutex_unlock(&esw->state_lock); + mutex_unlock(&net->xfrm.xfrm_cfg_mutex); + return err; +}