From patchwork Sun May 8 13:10:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 12842373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F8DFC433EF for ; Sun, 8 May 2022 13:11:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233280AbiEHNPk (ORCPT ); Sun, 8 May 2022 09:15:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51844 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232955AbiEHNPf (ORCPT ); Sun, 8 May 2022 09:15:35 -0400 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam08on2082.outbound.protection.outlook.com [40.107.100.82]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6526DE0CC; Sun, 8 May 2022 06:11:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EdQ9iYMaZMyTA5qgYCVEqityRoz5lH7CApvMJMl08kfIZjKc13pI5wa7naxHgnmM12ctgkQLj8vk0IK/cm2n2753RCHvXREAa1Rk8sfzUbz099yCxohiNWOcdkWz8bpJFEm2GkJB7BXTd+UBgEGBx3w4jl+9fPrY+WQUsQPvBvnG2B55ZT6y1TRaiqhuhvEZ3d6XBc/aAjJtqxOmqz1lrgKk+5OE7ucKeLAFM9K4Ub3RFShOf5YqyITFa7O4lKKxEaK15ojMzPPMuXX4FOQgOSsBzmHBccBMmbi7zoUL5zCSy/Ca4RSudasKxyKa+n3fU3EG5k7cImUk5YmJxMPI4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=S8SrpIf/Lat+BhZRQKoiLgahfmAYBon0yfVzzuEHEvc=; b=lmuxMQN08vZY4ki7FJe98+tbOtnvpa+3Fn2Wakm9LRFIbplREhuOQpNKYfPpXBjVRVICOS06rnIed4N0W2I08WuvDAZf0nixNp1V+pueMEgdbRwMh/ULriZyyk0uRFVqHCcAJ2wjlDlaDs1oXK6mRm9YokivqBHhQAGIF/y/Zpgf5dei0dRu0FfxJAfXE7fUu6Zqxl83rJc9swCoRIgOrGhlyb119ImqoZ6/EBOD2Xpo2+I1vIxbSBps0eNIehNsk6MVvofU4ygb5KBYzq24xijCwLiUulHtR/FfbPDiY9nF7I2Mk7Xjg026qARZQJSzOOhXyHdKtziQo6VfHHiNjQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S8SrpIf/Lat+BhZRQKoiLgahfmAYBon0yfVzzuEHEvc=; b=pz6YpdTdIGhQ6kFvSCu8cDGWC90M0BEJpxHt0EE8NL2q2gSaihSjwYYbnLNyccM/2Z4P6Olfgjj0hCNXrDH3EjEz6tRvalS23ky8uFWc3dpMj9sF7RUF8lfh54BEHyIl6zATZL8Q+pC2aSHtj2DCJl4oRt2ccgCDy5ONbam3PlNhE4w6joNUVsbolWbO2+IsKOkoMx+/MwWcs4pYtrGBWBDLorjXUt1QqjyT2Fxvh19n+tgOCmy8I/vKra72aC9Vo1uWrsEQJvmEyXgYIvzEt+zBtjrWNIvqOinlj/m/tzY+UGTlh4z0QGaZqHRfhfaAQd6FZNqf11J68zZL92+yGg== Received: from MW3PR05CA0006.namprd05.prod.outlook.com (2603:10b6:303:2b::11) by BL0PR12MB2386.namprd12.prod.outlook.com (2603:10b6:207:47::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.21; Sun, 8 May 2022 13:11:41 +0000 Received: from CO1NAM11FT018.eop-nam11.prod.protection.outlook.com (2603:10b6:303:2b:cafe::7c) by MW3PR05CA0006.outlook.office365.com (2603:10b6:303:2b::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.13 via Frontend Transport; Sun, 8 May 2022 13:11:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by CO1NAM11FT018.mail.protection.outlook.com (10.13.175.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5227.15 via Frontend Transport; Sun, 8 May 2022 13:11:41 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Sun, 8 May 2022 13:11:40 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Sun, 8 May 2022 06:11:40 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Sun, 8 May 2022 06:11:37 -0700 From: Yishai Hadas To: , , CC: , , , , , , Subject: [PATCH V1 mlx5-next 1/4] net/mlx5: Expose mlx5_sriov_blocking_notifier_register / unregister APIs Date: Sun, 8 May 2022 16:10:50 +0300 Message-ID: <20220508131053.241347-2-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20220508131053.241347-1-yishaih@nvidia.com> References: <20220508131053.241347-1-yishaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8c302453-9ccb-458a-a613-08da30f44b04 X-MS-TrafficTypeDiagnostic: BL0PR12MB2386:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KFSJkBf2bi9KqvNCn0LfU6wfRpQlFjk1Wpcmed1o4U3e6gFFWazdfILlhIzsQKm0aAvU31OHLP3TCR0yaidHJhalWeWQvdX3AZbP/UIVDnJpZnNlAuF6zjYiANIqzyIqn1cQUsNVwdl2Hl1yII66/8h/9Ru9pFf+vAJ4bb2w8qPqbwkCiFr6zXU2bn6jX2j6UU4sNNada9GYiIOmYHTyWzmLdKMR7HjR3Ya/Jh7Qnf5Z9ezPu8Z9vpGk+NmuNKFQnxc3lgAEF5mrYr9oTi3nq1FRT/iVhsd0GpUuRbUw3Ly00fbtKange9Gaqz3Sb2RwiEd/lZDFU1JgC6QDomJ2Jh8AAmxUJ7r1+9rYVF3+lZa0tn5lfDLxXw3rtT0Ec7IOFNWwz3eA5XNaC9+w1MLkClia9EoRqLCnh9g5tARz83LqNHAa9CeSOYnwR8KE/bXvbRlhX5GD9tazPonm4cqoiPGIWagAjycZi8Kw4kdDdtP81V0Hd9zZX/kPTg8QZ4TWrvAKat2pmPfFriSSVzo60gBKYvIKzAIo2GpMIsGPZP9SbYyBtdSYN/YGA8V4x2jqBfu91R/Ln036N3rzM37Oj1UiPToqvaZDWkv10lwJxrDb5dR4BQbtSYDMA6RD3ddcETa3vqh4Z7H4mzlI7SE2rn9u5ZTWR/k61FimnBQNCPqmn/hYoN/Vy3feQ9KzKnHnemaJ+wJLBRe/lkpdlDwvxA== X-Forefront-Antispam-Report: CIP:12.22.5.234;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(1076003)(8936002)(6636002)(316002)(5660300002)(47076005)(508600001)(82310400005)(426003)(54906003)(110136005)(336012)(36756003)(186003)(70206006)(70586007)(40460700003)(36860700001)(4326008)(86362001)(26005)(2616005)(8676002)(81166007)(6666004)(83380400001)(356005)(2906002)(7696005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2022 13:11:41.4002 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8c302453-9ccb-458a-a613-08da30f44b04 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.234];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT018.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB2386 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Expose mlx5_sriov_blocking_notifier_register / unregister APIs to let a VF register to be notified for its enablement / disablement by the PF. Upon VF probe it will call mlx5_sriov_blocking_notifier_register() with its notifier block and upon VF remove it will call mlx5_sriov_blocking_notifier_unregister() to drop its registration. This can give a VF the ability to clean some resources upon disable before that the command interface goes down and on the other hand sets some stuff before that it's enabled. This may be used by a VF which is migration capable in few cases.(e.g. PF load/unload upon an health recovery). Signed-off-by: Yishai Hadas Signed-off-by: Saeed Mahameed --- .../net/ethernet/mellanox/mlx5/core/sriov.c | 65 ++++++++++++++++++- include/linux/mlx5/driver.h | 12 ++++ 2 files changed, 76 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sriov.c b/drivers/net/ethernet/mellanox/mlx5/core/sriov.c index 887ee0f729d1..2935614f6fa9 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sriov.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sriov.c @@ -87,6 +87,11 @@ static int mlx5_device_enable_sriov(struct mlx5_core_dev *dev, int num_vfs) enable_vfs_hca: num_msix_count = mlx5_get_default_msix_vec_count(dev, num_vfs); for (vf = 0; vf < num_vfs; vf++) { + /* Notify the VF before its enablement to let it set + * some stuff. + */ + blocking_notifier_call_chain(&sriov->vfs_ctx[vf].notifier, + MLX5_PF_NOTIFY_ENABLE_VF, dev); err = mlx5_core_enable_hca(dev, vf + 1); if (err) { mlx5_core_warn(dev, "failed to enable VF %d (%d)\n", vf, err); @@ -127,6 +132,11 @@ mlx5_device_disable_sriov(struct mlx5_core_dev *dev, int num_vfs, bool clear_vf) for (vf = num_vfs - 1; vf >= 0; vf--) { if (!sriov->vfs_ctx[vf].enabled) continue; + /* Notify the VF before its disablement to let it clean + * some resources. + */ + blocking_notifier_call_chain(&sriov->vfs_ctx[vf].notifier, + MLX5_PF_NOTIFY_DISABLE_VF, dev); err = mlx5_core_disable_hca(dev, vf + 1); if (err) { mlx5_core_warn(dev, "failed to disable VF %d\n", vf); @@ -257,7 +267,7 @@ int mlx5_sriov_init(struct mlx5_core_dev *dev) { struct mlx5_core_sriov *sriov = &dev->priv.sriov; struct pci_dev *pdev = dev->pdev; - int total_vfs; + int total_vfs, i; if (!mlx5_core_is_pf(dev)) return 0; @@ -269,6 +279,9 @@ int mlx5_sriov_init(struct mlx5_core_dev *dev) if (!sriov->vfs_ctx) return -ENOMEM; + for (i = 0; i < total_vfs; i++) + BLOCKING_INIT_NOTIFIER_HEAD(&sriov->vfs_ctx[i].notifier); + return 0; } @@ -281,3 +294,53 @@ void mlx5_sriov_cleanup(struct mlx5_core_dev *dev) kfree(sriov->vfs_ctx); } + +/** + * mlx5_sriov_blocking_notifier_unregister - Unregister a VF from + * a notification block chain. + * + * @mdev: The mlx5 core device. + * @vf_id: The VF id. + * @nb: The notifier block to be unregistered. + */ +void mlx5_sriov_blocking_notifier_unregister(struct mlx5_core_dev *mdev, + int vf_id, + struct notifier_block *nb) +{ + struct mlx5_vf_context *vfs_ctx; + struct mlx5_core_sriov *sriov; + + sriov = &mdev->priv.sriov; + if (WARN_ON(vf_id < 0 || vf_id >= sriov->num_vfs)) + return; + + vfs_ctx = &sriov->vfs_ctx[vf_id]; + blocking_notifier_chain_unregister(&vfs_ctx->notifier, nb); +} +EXPORT_SYMBOL(mlx5_sriov_blocking_notifier_unregister); + +/** + * mlx5_sriov_blocking_notifier_register - Register a VF notification + * block chain. + * + * @mdev: The mlx5 core device. + * @vf_id: The VF id. + * @nb: The notifier block to be called upon the VF events. + * + * Returns 0 on success or an error code. + */ +int mlx5_sriov_blocking_notifier_register(struct mlx5_core_dev *mdev, + int vf_id, + struct notifier_block *nb) +{ + struct mlx5_vf_context *vfs_ctx; + struct mlx5_core_sriov *sriov; + + sriov = &mdev->priv.sriov; + if (vf_id < 0 || vf_id >= sriov->num_vfs) + return -EINVAL; + + vfs_ctx = &sriov->vfs_ctx[vf_id]; + return blocking_notifier_chain_register(&vfs_ctx->notifier, nb); +} +EXPORT_SYMBOL(mlx5_sriov_blocking_notifier_register); diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index 9424503eb8d3..3d1594bad4ec 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -445,6 +445,11 @@ struct mlx5_qp_table { struct radix_tree_root tree; }; +enum { + MLX5_PF_NOTIFY_DISABLE_VF, + MLX5_PF_NOTIFY_ENABLE_VF, +}; + struct mlx5_vf_context { int enabled; u64 port_guid; @@ -455,6 +460,7 @@ struct mlx5_vf_context { u8 port_guid_valid:1; u8 node_guid_valid:1; enum port_state_policy policy; + struct blocking_notifier_head notifier; }; struct mlx5_core_sriov { @@ -1155,6 +1161,12 @@ int mlx5_dm_sw_icm_dealloc(struct mlx5_core_dev *dev, enum mlx5_sw_icm_type type struct mlx5_core_dev *mlx5_vf_get_core_dev(struct pci_dev *pdev); void mlx5_vf_put_core_dev(struct mlx5_core_dev *mdev); +int mlx5_sriov_blocking_notifier_register(struct mlx5_core_dev *mdev, + int vf_id, + struct notifier_block *nb); +void mlx5_sriov_blocking_notifier_unregister(struct mlx5_core_dev *mdev, + int vf_id, + struct notifier_block *nb); #ifdef CONFIG_MLX5_CORE_IPOIB struct net_device *mlx5_rdma_netdev_alloc(struct mlx5_core_dev *mdev, struct ib_device *ibdev, From patchwork Sun May 8 13:10:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 12842374 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43C49C433F5 for ; Sun, 8 May 2022 13:11:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233348AbiEHNPn (ORCPT ); Sun, 8 May 2022 09:15:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51952 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233288AbiEHNPk (ORCPT ); Sun, 8 May 2022 09:15:40 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2050.outbound.protection.outlook.com [40.107.93.50]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09AD3FD33; Sun, 8 May 2022 06:11:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JTVDIyJKOEHXSERq33cqs9BLSXZnhtknk8g5nSLe4+602CK1opYAsAA36amK72LH9kwhL3Pycnqp2a8TCkpZWm1VLZCBpZXI4W3GwLhLAHRh5KsEq8sQNjMsph4qRTPoAFeF5+iJ7WxN3YZSATG+e6U5UIpyb6EytyBOcPSlJcjWxPX99EiF41jVEl2xoMr21jlmEfzS9YpMklkNvF1qJ6x5Ojkp8nGuqSbZywHrtNQkcdmBeZtuXbSAEolvEacJwYcgBmXtoRz1NN4FTZQXzrba8ihlzhicCgpzXHZQvTqOqeNZohdwPtAB0n0J/xjBxeOxCPBfMevFkdauEBiv7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=C3jSgGjz2oCGbm+Gy/f4YsFFHVQ6CJcrnDfk9DeehoA=; b=lBU+8YAmaH26k6QKrC7d9bT2FSAxKSTACNNpSsXafwdzut5722l9osajSna/hBDRkYN7Oq/VLU1sJAjscZoT2hV64AcSswxIsqL+bq49rP9ZYImy7uxGxuR/1MUfnbHGIL1ROaktwPuEYGuxymsbi4Hbq5N7x6TwyWVFW5lSpKueIPQuZFoYSCEmahw5G0UcNhVRt9imCEnPSp4+ZTSY8t7r+mvh5zAJBDz/A3GDci+f8MBkhF1b50S4GTfupRM5pMsLQaXvnSZb9hmjatwg+JTMxiy0IEtI7fi7O4yhu3Cj7Si6yFKKEJhWaIs/FbgTain6QnlEy/n6TcNt8CMFxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C3jSgGjz2oCGbm+Gy/f4YsFFHVQ6CJcrnDfk9DeehoA=; b=nFgkLHboWOfCDiL835rcMVUY+MiLA33aWhCCCyHjDNmN9T8jJAM82KBvzEOPE8BmPe2v9gMNdfKUJEmg1a0OWpZcXi2p/yPtyBLYVzNWwrAE5c2cQgRaP6OM2YXF8e1XYbL24EXmfw58Pd48UPfDNTDdvCUhQXU0gMZ/7yM8tYR/Ez/G+YGF6IXxTzknwqCCkuID0zfQAg7eNwFHgz3n9FsOAXwrTPQeHOlRoUSPawUjshMuSH3yDFjxK8Itzo1IoApAagv1T6yswSj9pzo8Yw/9d9B6I/SVW92hpkm8YYuZjclCqNJ43rKgeDd7h4m0npKHma+edefexBB16DFC4Q== Received: from MW4PR04CA0376.namprd04.prod.outlook.com (2603:10b6:303:81::21) by BN6PR12MB1571.namprd12.prod.outlook.com (2603:10b6:405:4::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.21; Sun, 8 May 2022 13:11:45 +0000 Received: from CO1NAM11FT042.eop-nam11.prod.protection.outlook.com (2603:10b6:303:81:cafe::2a) by MW4PR04CA0376.outlook.office365.com (2603:10b6:303:81::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.20 via Frontend Transport; Sun, 8 May 2022 13:11:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT042.mail.protection.outlook.com (10.13.174.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5227.15 via Frontend Transport; Sun, 8 May 2022 13:11:44 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Sun, 8 May 2022 13:11:44 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Sun, 8 May 2022 06:11:43 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Sun, 8 May 2022 06:11:40 -0700 From: Yishai Hadas To: , , CC: , , , , , , Subject: [PATCH V1 mlx5-next 2/4] vfio/mlx5: Manage the VF attach/detach callback from the PF Date: Sun, 8 May 2022 16:10:51 +0300 Message-ID: <20220508131053.241347-3-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20220508131053.241347-1-yishaih@nvidia.com> References: <20220508131053.241347-1-yishaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6179cd2e-39b8-4506-8751-08da30f44cf6 X-MS-TrafficTypeDiagnostic: BN6PR12MB1571:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 47YVnjXzZxhjqFhq7g1C2phWgRXU3Zas94QNjE76ptTh4eYzlSRk1A3JTdh3W8G8vCutUSPZw0cvFe2CEyZRXOMs/PPQ6IOydAuGUdE8fs4mrhmOit9xtDMozEJTtRfV/fUXThqMaymvWiEJ/vAJK44nuCVv+LvEmi7RAN4uIxfsWql+L9NF0EZlNzfYxzegB1z9vHhHBbm7F8or6EjSAL6rNaapMFSk2WxVtV5nQW0wQJnR2AZBONFgpLKATxpOBAacSx4rBc5+LIedWllMNB8HmPokBycSakgyOaNTPAt6yNtTydnL+d2crIoaFzPP5T2FeVO53UbzwFHzdDqAWKOnpM0yABQKa7jNJkktFpgVMRuJdOvO5vq77ZrSav6suAwvU7Tdju5fwf2TKQnGC84WtxwnQjauZm99k+lz/ixasroCoby0cNY98lwAUDG50RFlTr0xjZ/cX3XeAB7Vzt7dg5D76jMVqmkYOOihXtCZleUKi43GIJqWwlz1IBw+WSS5Pus2ugbQHl0Ov8Gq16PxYX6o/vyaea4TQHZlnmNykQPG/PiV4NaxD5tNDwbFCHnSqZywTgOKPfY3QV2GO7lLZd/Wir8EtPYIuioXaFXT0eZBRb3kAVnGEiEw4NlfKWLwhWLIICA7JUeoEnC6wHbmCtKTeT3sLxbV8llJWTPpV0lIVgFaa4QGFm1O/KQu7kCQY4h9u0Us8cBMunbFsw== X-Forefront-Antispam-Report: CIP:12.22.5.236;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(2906002)(36860700001)(36756003)(82310400005)(5660300002)(8676002)(70206006)(4326008)(70586007)(1076003)(110136005)(54906003)(83380400001)(86362001)(40460700003)(186003)(6636002)(2616005)(26005)(336012)(7696005)(316002)(47076005)(6666004)(426003)(81166007)(356005)(508600001)(8936002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2022 13:11:44.6631 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6179cd2e-39b8-4506-8751-08da30f44cf6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.236];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT042.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1571 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Manage the VF attach/detach callback from the PF. This lets the driver to enable parallel VFs migration as will be introduced in the next patch. As part of this, reorganize the VF is migratable code to be in a separate function and rename it to be set_migratable() to match its functionality. Signed-off-by: Yishai Hadas Signed-off-by: Leon Romanovsky --- drivers/vfio/pci/mlx5/cmd.c | 63 ++++++++++++++++++++++++++++++++++++ drivers/vfio/pci/mlx5/cmd.h | 22 +++++++++++++ drivers/vfio/pci/mlx5/main.c | 40 ++++------------------- 3 files changed, 91 insertions(+), 34 deletions(-) diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c index 5c9f9218cc1d..5031978ae63a 100644 --- a/drivers/vfio/pci/mlx5/cmd.c +++ b/drivers/vfio/pci/mlx5/cmd.c @@ -71,6 +71,69 @@ int mlx5vf_cmd_query_vhca_migration_state(struct pci_dev *pdev, u16 vhca_id, return ret; } +static int mlx5fv_vf_event(struct notifier_block *nb, + unsigned long event, void *data) +{ + struct mlx5vf_pci_core_device *mvdev = + container_of(nb, struct mlx5vf_pci_core_device, nb); + + mutex_lock(&mvdev->state_mutex); + switch (event) { + case MLX5_PF_NOTIFY_ENABLE_VF: + mvdev->mdev_detach = false; + break; + case MLX5_PF_NOTIFY_DISABLE_VF: + mvdev->mdev_detach = true; + break; + default: + break; + } + mlx5vf_state_mutex_unlock(mvdev); + return 0; +} + +void mlx5vf_cmd_remove_migratable(struct mlx5vf_pci_core_device *mvdev) +{ + mlx5_sriov_blocking_notifier_unregister(mvdev->mdev, mvdev->vf_id, + &mvdev->nb); +} + +void mlx5vf_cmd_set_migratable(struct mlx5vf_pci_core_device *mvdev) +{ + struct pci_dev *pdev = mvdev->core_device.pdev; + int ret; + + if (!pdev->is_virtfn) + return; + + mvdev->mdev = mlx5_vf_get_core_dev(pdev); + if (!mvdev->mdev) + return; + + if (!MLX5_CAP_GEN(mvdev->mdev, migration)) + goto end; + + mvdev->vf_id = pci_iov_vf_id(pdev); + if (mvdev->vf_id < 0) + goto end; + + mutex_init(&mvdev->state_mutex); + spin_lock_init(&mvdev->reset_lock); + mvdev->nb.notifier_call = mlx5fv_vf_event; + ret = mlx5_sriov_blocking_notifier_register(mvdev->mdev, mvdev->vf_id, + &mvdev->nb); + if (ret) + goto end; + + mvdev->migrate_cap = 1; + mvdev->core_device.vdev.migration_flags = + VFIO_MIGRATION_STOP_COPY | + VFIO_MIGRATION_P2P; + +end: + mlx5_vf_put_core_dev(mvdev->mdev); +} + int mlx5vf_cmd_get_vhca_id(struct pci_dev *pdev, u16 function_id, u16 *vhca_id) { struct mlx5_core_dev *mdev = mlx5_vf_get_core_dev(pdev); diff --git a/drivers/vfio/pci/mlx5/cmd.h b/drivers/vfio/pci/mlx5/cmd.h index 1392a11a9cc0..340a06b98007 100644 --- a/drivers/vfio/pci/mlx5/cmd.h +++ b/drivers/vfio/pci/mlx5/cmd.h @@ -7,6 +7,7 @@ #define MLX5_VFIO_CMD_H #include +#include #include struct mlx5_vf_migration_file { @@ -24,13 +25,34 @@ struct mlx5_vf_migration_file { unsigned long last_offset; }; +struct mlx5vf_pci_core_device { + struct vfio_pci_core_device core_device; + int vf_id; + u16 vhca_id; + u8 migrate_cap:1; + u8 deferred_reset:1; + /* protect migration state */ + struct mutex state_mutex; + enum vfio_device_mig_state mig_state; + /* protect the reset_done flow */ + spinlock_t reset_lock; + struct mlx5_vf_migration_file *resuming_migf; + struct mlx5_vf_migration_file *saving_migf; + struct notifier_block nb; + struct mlx5_core_dev *mdev; + u8 mdev_detach:1; +}; + int mlx5vf_cmd_suspend_vhca(struct pci_dev *pdev, u16 vhca_id, u16 op_mod); int mlx5vf_cmd_resume_vhca(struct pci_dev *pdev, u16 vhca_id, u16 op_mod); int mlx5vf_cmd_query_vhca_migration_state(struct pci_dev *pdev, u16 vhca_id, size_t *state_size); int mlx5vf_cmd_get_vhca_id(struct pci_dev *pdev, u16 function_id, u16 *vhca_id); +void mlx5vf_cmd_set_migratable(struct mlx5vf_pci_core_device *mvdev); +void mlx5vf_cmd_remove_migratable(struct mlx5vf_pci_core_device *mvdev); int mlx5vf_cmd_save_vhca_state(struct pci_dev *pdev, u16 vhca_id, struct mlx5_vf_migration_file *migf); int mlx5vf_cmd_load_vhca_state(struct pci_dev *pdev, u16 vhca_id, struct mlx5_vf_migration_file *migf); +void mlx5vf_state_mutex_unlock(struct mlx5vf_pci_core_device *mvdev); #endif /* MLX5_VFIO_CMD_H */ diff --git a/drivers/vfio/pci/mlx5/main.c b/drivers/vfio/pci/mlx5/main.c index bbec5d288fee..9716c87e31f9 100644 --- a/drivers/vfio/pci/mlx5/main.c +++ b/drivers/vfio/pci/mlx5/main.c @@ -17,7 +17,6 @@ #include #include #include -#include #include #include "cmd.h" @@ -25,20 +24,6 @@ /* Arbitrary to prevent userspace from consuming endless memory */ #define MAX_MIGRATION_SIZE (512*1024*1024) -struct mlx5vf_pci_core_device { - struct vfio_pci_core_device core_device; - u16 vhca_id; - u8 migrate_cap:1; - u8 deferred_reset:1; - /* protect migration state */ - struct mutex state_mutex; - enum vfio_device_mig_state mig_state; - /* protect the reset_done flow */ - spinlock_t reset_lock; - struct mlx5_vf_migration_file *resuming_migf; - struct mlx5_vf_migration_file *saving_migf; -}; - static struct page * mlx5vf_get_migration_page(struct mlx5_vf_migration_file *migf, unsigned long offset) @@ -444,7 +429,7 @@ mlx5vf_pci_step_device_state_locked(struct mlx5vf_pci_core_device *mvdev, * This function is called in all state_mutex unlock cases to * handle a 'deferred_reset' if exists. */ -static void mlx5vf_state_mutex_unlock(struct mlx5vf_pci_core_device *mvdev) +void mlx5vf_state_mutex_unlock(struct mlx5vf_pci_core_device *mvdev) { again: spin_lock(&mvdev->reset_lock); @@ -596,24 +581,7 @@ static int mlx5vf_pci_probe(struct pci_dev *pdev, if (!mvdev) return -ENOMEM; vfio_pci_core_init_device(&mvdev->core_device, pdev, &mlx5vf_pci_ops); - - if (pdev->is_virtfn) { - struct mlx5_core_dev *mdev = - mlx5_vf_get_core_dev(pdev); - - if (mdev) { - if (MLX5_CAP_GEN(mdev, migration)) { - mvdev->migrate_cap = 1; - mvdev->core_device.vdev.migration_flags = - VFIO_MIGRATION_STOP_COPY | - VFIO_MIGRATION_P2P; - mutex_init(&mvdev->state_mutex); - spin_lock_init(&mvdev->reset_lock); - } - mlx5_vf_put_core_dev(mdev); - } - } - + mlx5vf_cmd_set_migratable(mvdev); ret = vfio_pci_core_register_device(&mvdev->core_device); if (ret) goto out_free; @@ -622,6 +590,8 @@ static int mlx5vf_pci_probe(struct pci_dev *pdev, return 0; out_free: + if (mvdev->migrate_cap) + mlx5vf_cmd_remove_migratable(mvdev); vfio_pci_core_uninit_device(&mvdev->core_device); kfree(mvdev); return ret; @@ -632,6 +602,8 @@ static void mlx5vf_pci_remove(struct pci_dev *pdev) struct mlx5vf_pci_core_device *mvdev = dev_get_drvdata(&pdev->dev); vfio_pci_core_unregister_device(&mvdev->core_device); + if (mvdev->migrate_cap) + mlx5vf_cmd_remove_migratable(mvdev); vfio_pci_core_uninit_device(&mvdev->core_device); kfree(mvdev); } From patchwork Sun May 8 13:10:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 12842375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75FD3C433FE for ; Sun, 8 May 2022 13:12:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233370AbiEHNPt (ORCPT ); Sun, 8 May 2022 09:15:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233288AbiEHNPp (ORCPT ); Sun, 8 May 2022 09:15:45 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2066.outbound.protection.outlook.com [40.107.93.66]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0EFD2E0CC; Sun, 8 May 2022 06:11:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=na2MC1qB+S46MtAW3syY4OmO0xlXBUsrdQbIPIHwo4X4uYGE/hGymyUO57ZVURH5UC40u7Rmi5tZs9JHW5Ws/diXTIZ+fNU7O/zFs1QMRxHRVYsTQfAGWwFPl5/5LCIjHtXnJHYLhpU75eFOQy7ptcXXO09ZQRIDLXYK+7htP9z9muZWi2QhohF6QDS2FCiqOAjGvlEFBa1hOaPWIOFyD0UAJWTl3rdv9KYpBhB3zhLkJEDcVyKhMkjnOAFcTdBCys4hB4kdUGGQkjIRs/sBan8CpebjHwx83OgNNSW5jtu4r2Tx9VK3TNjGARUQhqaa7Y+Oh3lRp7RtQbX44X6J5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=414Lu1qD+Ha+GhDwTsSrTf4FyCjW5r34+jzbgahNiME=; b=DetTQS2od+KmqtbvpfMw8owEhxbS/4o923rAerLDQCCoBZ2eGS8//HKvQGH8UZBmDxVrHZutZvpGxwbZnJzqMIBM67wN3N82HGBgwLN8cuw224ZGtlIVraxXjwDiGUwDg1SYc0LDBmfCWGACrbLf7QkYZwDASarwZyZeIrsgkK0kf4b9yyAWe2KiGGREvv3W1BvOrOd1xoJCKYzCnqay86w7n4Z394Cb2IVWIPbXpuLdd71PEUC8uTiYQSzNPx067zBCmAV5GAQn5GjOzT3ckvq1N0Zi44ZXxewvwgaOYPTQCQbV6jhLLkAbR7SgHdh45Ge70oyieKp8N2vu90LnlQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=414Lu1qD+Ha+GhDwTsSrTf4FyCjW5r34+jzbgahNiME=; b=E7VAEjYQbEtN8OofRur3HpKDoUQkaw0zSu2+prVUu8rEtS7SAqgiaug4qU9TIoogFvWh+QcMa71y8RYyeClLBEfl6iGB2fG+kfaWMXVAM7u8NEGZNbAhSZrAM8idYwqP+AEsm6hg+PJJa2f8+a5V2s4ORCeAtohL+lceYV2aE+9X88TC+Mms3TLV1QBW3y/ZJ2o1ffsW/zHIW4sPEfziwoIVzGgAKvch0vYVe0uwEFzteyIV2vnzh4I6DmRSW7WwOHeKAVO/39nfxOMXyYnVlArWF0vJG0Cp/79qmjPRWQhZJ4IbTYDm/418XdmeZM96IvV9QHB3H707zE4V/ku9LA== Received: from MW4PR03CA0176.namprd03.prod.outlook.com (2603:10b6:303:8d::31) by MN2PR12MB3422.namprd12.prod.outlook.com (2603:10b6:208:ce::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.22; Sun, 8 May 2022 13:11:48 +0000 Received: from CO1NAM11FT041.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8d:cafe::85) by MW4PR03CA0176.outlook.office365.com (2603:10b6:303:8d::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.24 via Frontend Transport; Sun, 8 May 2022 13:11:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT041.mail.protection.outlook.com (10.13.174.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5227.15 via Frontend Transport; Sun, 8 May 2022 13:11:48 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Sun, 8 May 2022 13:11:47 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Sun, 8 May 2022 06:11:46 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Sun, 8 May 2022 06:11:44 -0700 From: Yishai Hadas To: , , CC: , , , , , , Subject: [PATCH V1 mlx5-next 3/4] vfio/mlx5: Refactor to enable VFs migration in parallel Date: Sun, 8 May 2022 16:10:52 +0300 Message-ID: <20220508131053.241347-4-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20220508131053.241347-1-yishaih@nvidia.com> References: <20220508131053.241347-1-yishaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 43c9c98e-9f37-431d-ccbb-08da30f44f01 X-MS-TrafficTypeDiagnostic: MN2PR12MB3422:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Oy5h6R7poIRBNT5FjDreb+IViSLPfdmP157axfc0Brjfv3OpnAYzTmXGpATW/UO42Pm0Hm2Yl4a46Vk+1d6yan1WwDcrcflpPc4mvxy69A93JxGkKa5krzfcGD4F0fa/TJoJBUBXFB12RWLtnROdipIFlLxy7C2XP9+UL+T6BeV4cWs+Ip5xyCNV0660z16Eob0OH/b+bjxobuGIblvD2VOVql0JFHcjnpOsNWJfcRaGoglgRC623x1BovVm06zdvW/vEmqgigmkIY2Jcb5sNiXYwB5bT4aZlbsfhhQL7R9DhV73jIiE6RhSn5gj+hJOsAmzxFVru78uUtYwvMIChYz6ANzOqw2JZdSYha3CgBfqoGHJvfwTsX3RXM0lVOPmFfzt6nMAurvViy2otYhMyEdxIloF7xq3h/R8ttLLaEBixQzl+/8kx84+ICS45CQb3xQ4OzmNw+bwY+V+qUrSlO9asoFru/B/748ShDSwqd54ljDCmJ5veNK0nn9JNRb6HK8ryP3n2PB2U7y8gZAFOKhdkCZdjahbK/BEVuhhv6hZ5P9k/FWO+1JOA9tuNcst5t3DXcNNZx5A/W8Hp+KRTUCSbMlfBjPXF+ilcbb2roPpCwjMSL/FUs7JOfDGB+9b9Q1YtFdOMZjwY9UkYXXymd7L1talQT5Y3tMZlwZGCKC9IvBJvttgQzUt7pqRZvedSegFREUbbSGWMfzupnT6sw== X-Forefront-Antispam-Report: CIP:12.22.5.235;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(2906002)(36860700001)(82310400005)(36756003)(5660300002)(30864003)(8676002)(70206006)(4326008)(70586007)(8936002)(1076003)(110136005)(83380400001)(54906003)(86362001)(40460700003)(186003)(2616005)(6636002)(26005)(336012)(7696005)(47076005)(316002)(81166007)(6666004)(426003)(356005)(508600001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2022 13:11:48.0768 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 43c9c98e-9f37-431d-ccbb-08da30f44f01 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT041.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3422 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Refactor to enable different VFs to run their commands over the PF command interface in parallel and to not block one each other. This is done by not using the global PF lock that was used before but relying on the VF attach/detach mechanism to sync. Signed-off-by: Yishai Hadas Signed-off-by: Leon Romanovsky --- drivers/vfio/pci/mlx5/cmd.c | 103 +++++++++++++++-------------------- drivers/vfio/pci/mlx5/cmd.h | 11 ++-- drivers/vfio/pci/mlx5/main.c | 44 ++++----------- 3 files changed, 59 insertions(+), 99 deletions(-) diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c index 5031978ae63a..9a6e3d3e0d44 100644 --- a/drivers/vfio/pci/mlx5/cmd.c +++ b/drivers/vfio/pci/mlx5/cmd.c @@ -5,70 +5,65 @@ #include "cmd.h" -int mlx5vf_cmd_suspend_vhca(struct pci_dev *pdev, u16 vhca_id, u16 op_mod) +static int mlx5vf_cmd_get_vhca_id(struct mlx5_core_dev *mdev, u16 function_id, + u16 *vhca_id); + +int mlx5vf_cmd_suspend_vhca(struct mlx5vf_pci_core_device *mvdev, u16 op_mod) { - struct mlx5_core_dev *mdev = mlx5_vf_get_core_dev(pdev); u32 out[MLX5_ST_SZ_DW(suspend_vhca_out)] = {}; u32 in[MLX5_ST_SZ_DW(suspend_vhca_in)] = {}; - int ret; - if (!mdev) + lockdep_assert_held(&mvdev->state_mutex); + if (mvdev->mdev_detach) return -ENOTCONN; MLX5_SET(suspend_vhca_in, in, opcode, MLX5_CMD_OP_SUSPEND_VHCA); - MLX5_SET(suspend_vhca_in, in, vhca_id, vhca_id); + MLX5_SET(suspend_vhca_in, in, vhca_id, mvdev->vhca_id); MLX5_SET(suspend_vhca_in, in, op_mod, op_mod); - ret = mlx5_cmd_exec_inout(mdev, suspend_vhca, in, out); - mlx5_vf_put_core_dev(mdev); - return ret; + return mlx5_cmd_exec_inout(mvdev->mdev, suspend_vhca, in, out); } -int mlx5vf_cmd_resume_vhca(struct pci_dev *pdev, u16 vhca_id, u16 op_mod) +int mlx5vf_cmd_resume_vhca(struct mlx5vf_pci_core_device *mvdev, u16 op_mod) { - struct mlx5_core_dev *mdev = mlx5_vf_get_core_dev(pdev); u32 out[MLX5_ST_SZ_DW(resume_vhca_out)] = {}; u32 in[MLX5_ST_SZ_DW(resume_vhca_in)] = {}; - int ret; - if (!mdev) + lockdep_assert_held(&mvdev->state_mutex); + if (mvdev->mdev_detach) return -ENOTCONN; MLX5_SET(resume_vhca_in, in, opcode, MLX5_CMD_OP_RESUME_VHCA); - MLX5_SET(resume_vhca_in, in, vhca_id, vhca_id); + MLX5_SET(resume_vhca_in, in, vhca_id, mvdev->vhca_id); MLX5_SET(resume_vhca_in, in, op_mod, op_mod); - ret = mlx5_cmd_exec_inout(mdev, resume_vhca, in, out); - mlx5_vf_put_core_dev(mdev); - return ret; + return mlx5_cmd_exec_inout(mvdev->mdev, resume_vhca, in, out); } -int mlx5vf_cmd_query_vhca_migration_state(struct pci_dev *pdev, u16 vhca_id, +int mlx5vf_cmd_query_vhca_migration_state(struct mlx5vf_pci_core_device *mvdev, size_t *state_size) { - struct mlx5_core_dev *mdev = mlx5_vf_get_core_dev(pdev); u32 out[MLX5_ST_SZ_DW(query_vhca_migration_state_out)] = {}; u32 in[MLX5_ST_SZ_DW(query_vhca_migration_state_in)] = {}; int ret; - if (!mdev) + lockdep_assert_held(&mvdev->state_mutex); + if (mvdev->mdev_detach) return -ENOTCONN; MLX5_SET(query_vhca_migration_state_in, in, opcode, MLX5_CMD_OP_QUERY_VHCA_MIGRATION_STATE); - MLX5_SET(query_vhca_migration_state_in, in, vhca_id, vhca_id); + MLX5_SET(query_vhca_migration_state_in, in, vhca_id, mvdev->vhca_id); MLX5_SET(query_vhca_migration_state_in, in, op_mod, 0); - ret = mlx5_cmd_exec_inout(mdev, query_vhca_migration_state, in, out); + ret = mlx5_cmd_exec_inout(mvdev->mdev, query_vhca_migration_state, in, + out); if (ret) - goto end; + return ret; *state_size = MLX5_GET(query_vhca_migration_state_out, out, required_umem_size); - -end: - mlx5_vf_put_core_dev(mdev); - return ret; + return 0; } static int mlx5fv_vf_event(struct notifier_block *nb, @@ -117,6 +112,10 @@ void mlx5vf_cmd_set_migratable(struct mlx5vf_pci_core_device *mvdev) if (mvdev->vf_id < 0) goto end; + if (mlx5vf_cmd_get_vhca_id(mvdev->mdev, mvdev->vf_id + 1, + &mvdev->vhca_id)) + goto end; + mutex_init(&mvdev->state_mutex); spin_lock_init(&mvdev->reset_lock); mvdev->nb.notifier_call = mlx5fv_vf_event; @@ -134,23 +133,18 @@ void mlx5vf_cmd_set_migratable(struct mlx5vf_pci_core_device *mvdev) mlx5_vf_put_core_dev(mvdev->mdev); } -int mlx5vf_cmd_get_vhca_id(struct pci_dev *pdev, u16 function_id, u16 *vhca_id) +static int mlx5vf_cmd_get_vhca_id(struct mlx5_core_dev *mdev, u16 function_id, + u16 *vhca_id) { - struct mlx5_core_dev *mdev = mlx5_vf_get_core_dev(pdev); u32 in[MLX5_ST_SZ_DW(query_hca_cap_in)] = {}; int out_size; void *out; int ret; - if (!mdev) - return -ENOTCONN; - out_size = MLX5_ST_SZ_BYTES(query_hca_cap_out); out = kzalloc(out_size, GFP_KERNEL); - if (!out) { - ret = -ENOMEM; - goto end; - } + if (!out) + return -ENOMEM; MLX5_SET(query_hca_cap_in, in, opcode, MLX5_CMD_OP_QUERY_HCA_CAP); MLX5_SET(query_hca_cap_in, in, other_function, 1); @@ -168,8 +162,6 @@ int mlx5vf_cmd_get_vhca_id(struct pci_dev *pdev, u16 function_id, u16 *vhca_id) err_exec: kfree(out); -end: - mlx5_vf_put_core_dev(mdev); return ret; } @@ -214,21 +206,23 @@ static int _create_state_mkey(struct mlx5_core_dev *mdev, u32 pdn, return err; } -int mlx5vf_cmd_save_vhca_state(struct pci_dev *pdev, u16 vhca_id, +int mlx5vf_cmd_save_vhca_state(struct mlx5vf_pci_core_device *mvdev, struct mlx5_vf_migration_file *migf) { - struct mlx5_core_dev *mdev = mlx5_vf_get_core_dev(pdev); u32 out[MLX5_ST_SZ_DW(save_vhca_state_out)] = {}; u32 in[MLX5_ST_SZ_DW(save_vhca_state_in)] = {}; + struct mlx5_core_dev *mdev; u32 pdn, mkey; int err; - if (!mdev) + lockdep_assert_held(&mvdev->state_mutex); + if (mvdev->mdev_detach) return -ENOTCONN; + mdev = mvdev->mdev; err = mlx5_core_alloc_pd(mdev, &pdn); if (err) - goto end; + return err; err = dma_map_sgtable(mdev->device, &migf->table.sgt, DMA_FROM_DEVICE, 0); @@ -242,7 +236,7 @@ int mlx5vf_cmd_save_vhca_state(struct pci_dev *pdev, u16 vhca_id, MLX5_SET(save_vhca_state_in, in, opcode, MLX5_CMD_OP_SAVE_VHCA_STATE); MLX5_SET(save_vhca_state_in, in, op_mod, 0); - MLX5_SET(save_vhca_state_in, in, vhca_id, vhca_id); + MLX5_SET(save_vhca_state_in, in, vhca_id, mvdev->vhca_id); MLX5_SET(save_vhca_state_in, in, mkey, mkey); MLX5_SET(save_vhca_state_in, in, size, migf->total_length); @@ -250,37 +244,28 @@ int mlx5vf_cmd_save_vhca_state(struct pci_dev *pdev, u16 vhca_id, if (err) goto err_exec; - migf->total_length = - MLX5_GET(save_vhca_state_out, out, actual_image_size); - - mlx5_core_destroy_mkey(mdev, mkey); - mlx5_core_dealloc_pd(mdev, pdn); - dma_unmap_sgtable(mdev->device, &migf->table.sgt, DMA_FROM_DEVICE, 0); - mlx5_vf_put_core_dev(mdev); - - return 0; - + migf->total_length = MLX5_GET(save_vhca_state_out, out, + actual_image_size); err_exec: mlx5_core_destroy_mkey(mdev, mkey); err_create_mkey: dma_unmap_sgtable(mdev->device, &migf->table.sgt, DMA_FROM_DEVICE, 0); err_dma_map: mlx5_core_dealloc_pd(mdev, pdn); -end: - mlx5_vf_put_core_dev(mdev); return err; } -int mlx5vf_cmd_load_vhca_state(struct pci_dev *pdev, u16 vhca_id, +int mlx5vf_cmd_load_vhca_state(struct mlx5vf_pci_core_device *mvdev, struct mlx5_vf_migration_file *migf) { - struct mlx5_core_dev *mdev = mlx5_vf_get_core_dev(pdev); + struct mlx5_core_dev *mdev; u32 out[MLX5_ST_SZ_DW(save_vhca_state_out)] = {}; u32 in[MLX5_ST_SZ_DW(save_vhca_state_in)] = {}; u32 pdn, mkey; int err; - if (!mdev) + lockdep_assert_held(&mvdev->state_mutex); + if (mvdev->mdev_detach) return -ENOTCONN; mutex_lock(&migf->lock); @@ -289,6 +274,7 @@ int mlx5vf_cmd_load_vhca_state(struct pci_dev *pdev, u16 vhca_id, goto end; } + mdev = mvdev->mdev; err = mlx5_core_alloc_pd(mdev, &pdn); if (err) goto end; @@ -304,7 +290,7 @@ int mlx5vf_cmd_load_vhca_state(struct pci_dev *pdev, u16 vhca_id, MLX5_SET(load_vhca_state_in, in, opcode, MLX5_CMD_OP_LOAD_VHCA_STATE); MLX5_SET(load_vhca_state_in, in, op_mod, 0); - MLX5_SET(load_vhca_state_in, in, vhca_id, vhca_id); + MLX5_SET(load_vhca_state_in, in, vhca_id, mvdev->vhca_id); MLX5_SET(load_vhca_state_in, in, mkey, mkey); MLX5_SET(load_vhca_state_in, in, size, migf->total_length); @@ -316,7 +302,6 @@ int mlx5vf_cmd_load_vhca_state(struct pci_dev *pdev, u16 vhca_id, err_reg: mlx5_core_dealloc_pd(mdev, pdn); end: - mlx5_vf_put_core_dev(mdev); mutex_unlock(&migf->lock); return err; } diff --git a/drivers/vfio/pci/mlx5/cmd.h b/drivers/vfio/pci/mlx5/cmd.h index 340a06b98007..2a20b7435393 100644 --- a/drivers/vfio/pci/mlx5/cmd.h +++ b/drivers/vfio/pci/mlx5/cmd.h @@ -43,16 +43,15 @@ struct mlx5vf_pci_core_device { u8 mdev_detach:1; }; -int mlx5vf_cmd_suspend_vhca(struct pci_dev *pdev, u16 vhca_id, u16 op_mod); -int mlx5vf_cmd_resume_vhca(struct pci_dev *pdev, u16 vhca_id, u16 op_mod); -int mlx5vf_cmd_query_vhca_migration_state(struct pci_dev *pdev, u16 vhca_id, +int mlx5vf_cmd_suspend_vhca(struct mlx5vf_pci_core_device *mvdev, u16 op_mod); +int mlx5vf_cmd_resume_vhca(struct mlx5vf_pci_core_device *mvdev, u16 op_mod); +int mlx5vf_cmd_query_vhca_migration_state(struct mlx5vf_pci_core_device *mvdev, size_t *state_size); -int mlx5vf_cmd_get_vhca_id(struct pci_dev *pdev, u16 function_id, u16 *vhca_id); void mlx5vf_cmd_set_migratable(struct mlx5vf_pci_core_device *mvdev); void mlx5vf_cmd_remove_migratable(struct mlx5vf_pci_core_device *mvdev); -int mlx5vf_cmd_save_vhca_state(struct pci_dev *pdev, u16 vhca_id, +int mlx5vf_cmd_save_vhca_state(struct mlx5vf_pci_core_device *mvdev, struct mlx5_vf_migration_file *migf); -int mlx5vf_cmd_load_vhca_state(struct pci_dev *pdev, u16 vhca_id, +int mlx5vf_cmd_load_vhca_state(struct mlx5vf_pci_core_device *mvdev, struct mlx5_vf_migration_file *migf); void mlx5vf_state_mutex_unlock(struct mlx5vf_pci_core_device *mvdev); #endif /* MLX5_VFIO_CMD_H */ diff --git a/drivers/vfio/pci/mlx5/main.c b/drivers/vfio/pci/mlx5/main.c index 9716c87e31f9..5bda6c0e194c 100644 --- a/drivers/vfio/pci/mlx5/main.c +++ b/drivers/vfio/pci/mlx5/main.c @@ -208,8 +208,8 @@ mlx5vf_pci_save_device_data(struct mlx5vf_pci_core_device *mvdev) stream_open(migf->filp->f_inode, migf->filp); mutex_init(&migf->lock); - ret = mlx5vf_cmd_query_vhca_migration_state( - mvdev->core_device.pdev, mvdev->vhca_id, &migf->total_length); + ret = mlx5vf_cmd_query_vhca_migration_state(mvdev, + &migf->total_length); if (ret) goto out_free; @@ -218,8 +218,7 @@ mlx5vf_pci_save_device_data(struct mlx5vf_pci_core_device *mvdev) if (ret) goto out_free; - ret = mlx5vf_cmd_save_vhca_state(mvdev->core_device.pdev, - mvdev->vhca_id, migf); + ret = mlx5vf_cmd_save_vhca_state(mvdev, migf); if (ret) goto out_free; return migf; @@ -346,8 +345,7 @@ mlx5vf_pci_step_device_state_locked(struct mlx5vf_pci_core_device *mvdev, int ret; if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && new == VFIO_DEVICE_STATE_STOP) { - ret = mlx5vf_cmd_suspend_vhca( - mvdev->core_device.pdev, mvdev->vhca_id, + ret = mlx5vf_cmd_suspend_vhca(mvdev, MLX5_SUSPEND_VHCA_IN_OP_MOD_SUSPEND_RESPONDER); if (ret) return ERR_PTR(ret); @@ -355,8 +353,7 @@ mlx5vf_pci_step_device_state_locked(struct mlx5vf_pci_core_device *mvdev, } if (cur == VFIO_DEVICE_STATE_STOP && new == VFIO_DEVICE_STATE_RUNNING_P2P) { - ret = mlx5vf_cmd_resume_vhca( - mvdev->core_device.pdev, mvdev->vhca_id, + ret = mlx5vf_cmd_resume_vhca(mvdev, MLX5_RESUME_VHCA_IN_OP_MOD_RESUME_RESPONDER); if (ret) return ERR_PTR(ret); @@ -364,8 +361,7 @@ mlx5vf_pci_step_device_state_locked(struct mlx5vf_pci_core_device *mvdev, } if (cur == VFIO_DEVICE_STATE_RUNNING && new == VFIO_DEVICE_STATE_RUNNING_P2P) { - ret = mlx5vf_cmd_suspend_vhca( - mvdev->core_device.pdev, mvdev->vhca_id, + ret = mlx5vf_cmd_suspend_vhca(mvdev, MLX5_SUSPEND_VHCA_IN_OP_MOD_SUSPEND_INITIATOR); if (ret) return ERR_PTR(ret); @@ -373,8 +369,7 @@ mlx5vf_pci_step_device_state_locked(struct mlx5vf_pci_core_device *mvdev, } if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && new == VFIO_DEVICE_STATE_RUNNING) { - ret = mlx5vf_cmd_resume_vhca( - mvdev->core_device.pdev, mvdev->vhca_id, + ret = mlx5vf_cmd_resume_vhca(mvdev, MLX5_RESUME_VHCA_IN_OP_MOD_RESUME_INITIATOR); if (ret) return ERR_PTR(ret); @@ -409,8 +404,7 @@ mlx5vf_pci_step_device_state_locked(struct mlx5vf_pci_core_device *mvdev, } if (cur == VFIO_DEVICE_STATE_RESUMING && new == VFIO_DEVICE_STATE_STOP) { - ret = mlx5vf_cmd_load_vhca_state(mvdev->core_device.pdev, - mvdev->vhca_id, + ret = mlx5vf_cmd_load_vhca_state(mvdev, mvdev->resuming_migf); if (ret) return ERR_PTR(ret); @@ -517,34 +511,16 @@ static int mlx5vf_pci_open_device(struct vfio_device *core_vdev) struct mlx5vf_pci_core_device *mvdev = container_of( core_vdev, struct mlx5vf_pci_core_device, core_device.vdev); struct vfio_pci_core_device *vdev = &mvdev->core_device; - int vf_id; int ret; ret = vfio_pci_core_enable(vdev); if (ret) return ret; - if (!mvdev->migrate_cap) { - vfio_pci_core_finish_enable(vdev); - return 0; - } - - vf_id = pci_iov_vf_id(vdev->pdev); - if (vf_id < 0) { - ret = vf_id; - goto out_disable; - } - - ret = mlx5vf_cmd_get_vhca_id(vdev->pdev, vf_id + 1, &mvdev->vhca_id); - if (ret) - goto out_disable; - - mvdev->mig_state = VFIO_DEVICE_STATE_RUNNING; + if (mvdev->migrate_cap) + mvdev->mig_state = VFIO_DEVICE_STATE_RUNNING; vfio_pci_core_finish_enable(vdev); return 0; -out_disable: - vfio_pci_core_disable(vdev); - return ret; } static void mlx5vf_pci_close_device(struct vfio_device *core_vdev) From patchwork Sun May 8 13:10:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 12842376 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9748C4332F for ; Sun, 8 May 2022 13:12:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233381AbiEHNPu (ORCPT ); Sun, 8 May 2022 09:15:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233354AbiEHNPr (ORCPT ); Sun, 8 May 2022 09:15:47 -0400 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam08on2066.outbound.protection.outlook.com [40.107.101.66]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 002B9101CB; Sun, 8 May 2022 06:11:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jGG3GgE0KSwkQ2TSriWwVOETqXNx7MVleXKFSAdf0ysyUYL86LOeskynWVHDrhEl1ZMVrOEFb7r7OM/4Bk3zzUuT3G9wNeFCoRCcaEjjfKLfApM6UuTWjHc/L2ByNh+C4NWADHcw/hBWhpSJb5VribJgCGdA9Aao40E8ILWinXGPXMgmBdvhihbEHt/GaPmW3R2SDQSYEr1NKoOPPRqlmUrmUuI0NwBNfGHfD5b0qnveDxRwCgDIkx5hzZ2c4RllBu9Q/p/7NF6igs0w+fzRHdz7Wrmf0A07tQvImQwWPNwIzFWbBjZTvVpj8OfWr5YJ0hfkoDrHr/HKKhvECx+jZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KTmwMp+3S+/JbNelQ2DrTFXxzSNDZoLExaYTZ4F2uq0=; b=AUUUiIoVPMBB3rNTFgxIp+LmC+Plj711wEf0xEj9nq9aUGtGw73FLjTcxfAEis+bLbeRC0JABCSctBnEUZ1jb+o89NOmevR8rB3VddWCGVy9afr7AN76xpe1QenvAS3xoo2DO1npNiH7Nuk9bfL+45DYdofYwyWXITe6ZKH7b2K0vQeYlXcvPV3YEW9OwdbvfFNrKw3Ewt1ByZ0OY2gx85/pj9bJP5nUQtAoNz+dvjsTPd63U79LhUevWXYFDFfreS8cVun8nWaeOMDdYIEejmUffz2MHjVNlXIqP5P1LqpFbOZbtWrO2JuG26FohUjloklBsaYbsRj3Pky7hIw9uA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KTmwMp+3S+/JbNelQ2DrTFXxzSNDZoLExaYTZ4F2uq0=; b=LHYJg18tEVx3lx2xv85DV7+MNmcftttLu3LwnFZFQYdLUO26kgOMjVJbzg6u3n+Pi3C2R+8ZViHgAzZHLlgQFMF3lzrmYtGuwTNMPmXxL4nSvT8JAFyL29Yfxz+fgOR5Lg8KGbmbt8XhaohMz66SmSl8J+LKWcLMZCDRinJE73FEar9WyEExTxzM0CZv+d2K+QbeaeiNJpIP2bujq5l1hPzMrRPTluycuhWP5NBHDZdjXbvvh12RWb9LwJf77gLzCixSA2jaiNQr3DvPfPaA1kMWpg+rsyDBj6JHWgANOjXnQCqVfGUNvqh0hcMWn3U2tmbT9zClhazMmiKHKWhPOg== Received: from MW4P221CA0002.NAMP221.PROD.OUTLOOK.COM (2603:10b6:303:8b::7) by DM8PR12MB5414.namprd12.prod.outlook.com (2603:10b6:8:3e::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.21; Sun, 8 May 2022 13:11:51 +0000 Received: from CO1NAM11FT010.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8b:cafe::87) by MW4P221CA0002.outlook.office365.com (2603:10b6:303:8b::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.20 via Frontend Transport; Sun, 8 May 2022 13:11:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.238) by CO1NAM11FT010.mail.protection.outlook.com (10.13.175.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5227.15 via Frontend Transport; Sun, 8 May 2022 13:11:51 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Sun, 8 May 2022 13:11:50 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Sun, 8 May 2022 06:11:50 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Sun, 8 May 2022 06:11:47 -0700 From: Yishai Hadas To: , , CC: , , , , , , Subject: [PATCH V1 mlx5-next 4/4] vfio/mlx5: Run the SAVE state command in an async mode Date: Sun, 8 May 2022 16:10:53 +0300 Message-ID: <20220508131053.241347-5-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20220508131053.241347-1-yishaih@nvidia.com> References: <20220508131053.241347-1-yishaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 10ee6070-426b-4c54-ff34-08da30f450f2 X-MS-TrafficTypeDiagnostic: DM8PR12MB5414:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Hmgap/qaf4sILhD2Zzm+4KcPOdqvsFFra6bYsumUt2/jnI0OsiUP2akCkLNBiy0GlOO7TXTRH0HdAj298fu/8YbSiiiPZxzs7oUAGV0N8JPd/Sw0Bi+RW/16lrzQD8CSoudmKGv4w0tg2qD+a3lmbAn0pro7l1qX1sUkJoMjzSKLyxrajxTbfSJw/0AM9vScwcO0owJoBBhrZDz1+6czUtZfsURqGj4h+F/g426oWOF3tv5JZAmGACqeFR+So9VmPgnrrjtm5muO0GgiLV1j8kDYsWXhypWdGy7zhLpDmXTXTYFhLMeXXJ/sWk0Wcs0gPsfJ2ywbAxCNGmbsi3ULgbW5F6+VAgNdT/jmG1My2QqPDQp/tq6oJsdaiqE7sIHOFaj0UKfvZpURQQD0hWyAvE627Q+ISKMKVbi+/WnJ6UitZ9aJE7vOQvHnjmmxDptipY7vcX3HXII9wp4iFYoGNwIjM0fTSKGkgUVkuD6Hau962RCXYwXhButHcUHSJY3TPr6pAVKThzp+WrRbBBN7vHIkJ+RGHFgMp48TXT56/VFd8cRcsMWQCZrOyDexlzeI5X5pvwVnlE8d8EjgorVR9dbS6HrSoejYsW67YKNj/79+wutR/rgseYlNIYMYJyy9yGl5fBA5AN6NVWbk7oJ8A901tXBbrI84zPiqXGlDvgM1lVV2jqm89CW0DDi5x6/9UEm0jFl3bzLfdwE7+4U/6Q== X-Forefront-Antispam-Report: CIP:12.22.5.238;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(1076003)(2616005)(8676002)(70586007)(4326008)(83380400001)(186003)(47076005)(426003)(336012)(70206006)(508600001)(6666004)(86362001)(26005)(316002)(82310400005)(54906003)(7696005)(110136005)(6636002)(81166007)(8936002)(36860700001)(356005)(40460700003)(36756003)(2906002)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2022 13:11:51.3039 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 10ee6070-426b-4c54-ff34-08da30f450f2 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.238];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT010.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5414 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use the PF asynchronous command mode for the SAVE state command. This enables returning earlier to user space upon issuing successfully the command and improve latency by let things run in parallel. Signed-off-by: Yishai Hadas --- drivers/vfio/pci/mlx5/cmd.c | 72 ++++++++++++++++++++++++++++++++++-- drivers/vfio/pci/mlx5/cmd.h | 17 +++++++++ drivers/vfio/pci/mlx5/main.c | 56 +++++++++++++++++++++++++--- 3 files changed, 136 insertions(+), 9 deletions(-) diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c index 9a6e3d3e0d44..0538e44e7eac 100644 --- a/drivers/vfio/pci/mlx5/cmd.c +++ b/drivers/vfio/pci/mlx5/cmd.c @@ -78,6 +78,7 @@ static int mlx5fv_vf_event(struct notifier_block *nb, mvdev->mdev_detach = false; break; case MLX5_PF_NOTIFY_DISABLE_VF: + mlx5vf_disable_fds(mvdev); mvdev->mdev_detach = true; break; default: @@ -206,11 +207,56 @@ static int _create_state_mkey(struct mlx5_core_dev *mdev, u32 pdn, return err; } +void mlx5vf_mig_file_cleanup_cb(struct work_struct *_work) +{ + struct mlx5vf_async_data *async_data = container_of(_work, + struct mlx5vf_async_data, work); + struct mlx5_vf_migration_file *migf = container_of(async_data, + struct mlx5_vf_migration_file, async_data); + struct mlx5_core_dev *mdev = migf->mvdev->mdev; + + mutex_lock(&migf->lock); + if (async_data->status) { + migf->is_err = true; + wake_up_interruptible(&migf->poll_wait); + } + mutex_unlock(&migf->lock); + + mlx5_core_destroy_mkey(mdev, async_data->mkey); + dma_unmap_sgtable(mdev->device, &migf->table.sgt, DMA_FROM_DEVICE, 0); + mlx5_core_dealloc_pd(mdev, async_data->pdn); + kvfree(async_data->out); + fput(migf->filp); +} + +static void mlx5vf_save_callback(int status, struct mlx5_async_work *context) +{ + struct mlx5vf_async_data *async_data = container_of(context, + struct mlx5vf_async_data, cb_work); + struct mlx5_vf_migration_file *migf = container_of(async_data, + struct mlx5_vf_migration_file, async_data); + + if (!status) { + WRITE_ONCE(migf->total_length, + MLX5_GET(save_vhca_state_out, async_data->out, + actual_image_size)); + wake_up_interruptible(&migf->poll_wait); + } + + /* + * The error and the cleanup flows can't run from an + * interrupt context + */ + async_data->status = status; + queue_work(migf->mvdev->cb_wq, &async_data->work); +} + int mlx5vf_cmd_save_vhca_state(struct mlx5vf_pci_core_device *mvdev, struct mlx5_vf_migration_file *migf) { - u32 out[MLX5_ST_SZ_DW(save_vhca_state_out)] = {}; + u32 out_size = MLX5_ST_SZ_BYTES(save_vhca_state_out); u32 in[MLX5_ST_SZ_DW(save_vhca_state_in)] = {}; + struct mlx5vf_async_data *async_data; struct mlx5_core_dev *mdev; u32 pdn, mkey; int err; @@ -240,13 +286,31 @@ int mlx5vf_cmd_save_vhca_state(struct mlx5vf_pci_core_device *mvdev, MLX5_SET(save_vhca_state_in, in, mkey, mkey); MLX5_SET(save_vhca_state_in, in, size, migf->total_length); - err = mlx5_cmd_exec_inout(mdev, save_vhca_state, in, out); + async_data = &migf->async_data; + async_data->out = kvzalloc(out_size, GFP_KERNEL); + if (!async_data->out) { + err = -ENOMEM; + goto err_out; + } + + /* no data exists till the callback comes back */ + migf->total_length = 0; + get_file(migf->filp); + async_data->mkey = mkey; + async_data->pdn = pdn; + err = mlx5_cmd_exec_cb(&migf->async_ctx, in, sizeof(in), + async_data->out, + out_size, mlx5vf_save_callback, + &async_data->cb_work); if (err) goto err_exec; - migf->total_length = MLX5_GET(save_vhca_state_out, out, - actual_image_size); + return 0; + err_exec: + fput(migf->filp); + kvfree(async_data->out); +err_out: mlx5_core_destroy_mkey(mdev, mkey); err_create_mkey: dma_unmap_sgtable(mdev->device, &migf->table.sgt, DMA_FROM_DEVICE, 0); diff --git a/drivers/vfio/pci/mlx5/cmd.h b/drivers/vfio/pci/mlx5/cmd.h index 2a20b7435393..d053d314b745 100644 --- a/drivers/vfio/pci/mlx5/cmd.h +++ b/drivers/vfio/pci/mlx5/cmd.h @@ -10,10 +10,20 @@ #include #include +struct mlx5vf_async_data { + struct mlx5_async_work cb_work; + struct work_struct work; + int status; + u32 pdn; + u32 mkey; + void *out; +}; + struct mlx5_vf_migration_file { struct file *filp; struct mutex lock; bool disabled; + u8 is_err:1; struct sg_append_table table; size_t total_length; @@ -23,6 +33,10 @@ struct mlx5_vf_migration_file { struct scatterlist *last_offset_sg; unsigned int sg_last_entry; unsigned long last_offset; + struct mlx5vf_pci_core_device *mvdev; + wait_queue_head_t poll_wait; + struct mlx5_async_ctx async_ctx; + struct mlx5vf_async_data async_data; }; struct mlx5vf_pci_core_device { @@ -38,6 +52,7 @@ struct mlx5vf_pci_core_device { spinlock_t reset_lock; struct mlx5_vf_migration_file *resuming_migf; struct mlx5_vf_migration_file *saving_migf; + struct workqueue_struct *cb_wq; struct notifier_block nb; struct mlx5_core_dev *mdev; u8 mdev_detach:1; @@ -54,4 +69,6 @@ int mlx5vf_cmd_save_vhca_state(struct mlx5vf_pci_core_device *mvdev, int mlx5vf_cmd_load_vhca_state(struct mlx5vf_pci_core_device *mvdev, struct mlx5_vf_migration_file *migf); void mlx5vf_state_mutex_unlock(struct mlx5vf_pci_core_device *mvdev); +void mlx5vf_disable_fds(struct mlx5vf_pci_core_device *mvdev); +void mlx5vf_mig_file_cleanup_cb(struct work_struct *_work); #endif /* MLX5_VFIO_CMD_H */ diff --git a/drivers/vfio/pci/mlx5/main.c b/drivers/vfio/pci/mlx5/main.c index 5bda6c0e194c..625ed5743f26 100644 --- a/drivers/vfio/pci/mlx5/main.c +++ b/drivers/vfio/pci/mlx5/main.c @@ -134,12 +134,22 @@ static ssize_t mlx5vf_save_read(struct file *filp, char __user *buf, size_t len, return -ESPIPE; pos = &filp->f_pos; + if (!(filp->f_flags & O_NONBLOCK)) { + if (wait_event_interruptible(migf->poll_wait, + READ_ONCE(migf->total_length) || migf->is_err)) + return -ERESTARTSYS; + } + mutex_lock(&migf->lock); + if ((filp->f_flags & O_NONBLOCK) && !READ_ONCE(migf->total_length)) { + done = -EAGAIN; + goto out_unlock; + } if (*pos > migf->total_length) { done = -EINVAL; goto out_unlock; } - if (migf->disabled) { + if (migf->disabled || migf->is_err) { done = -ENODEV; goto out_unlock; } @@ -179,9 +189,28 @@ static ssize_t mlx5vf_save_read(struct file *filp, char __user *buf, size_t len, return done; } +static __poll_t mlx5vf_save_poll(struct file *filp, + struct poll_table_struct *wait) +{ + struct mlx5_vf_migration_file *migf = filp->private_data; + __poll_t pollflags = 0; + + poll_wait(filp, &migf->poll_wait, wait); + + mutex_lock(&migf->lock); + if (migf->disabled || migf->is_err) + pollflags = EPOLLIN | EPOLLRDNORM | EPOLLRDHUP; + else if (READ_ONCE(migf->total_length)) + pollflags = EPOLLIN | EPOLLRDNORM; + mutex_unlock(&migf->lock); + + return pollflags; +} + static const struct file_operations mlx5vf_save_fops = { .owner = THIS_MODULE, .read = mlx5vf_save_read, + .poll = mlx5vf_save_poll, .release = mlx5vf_release_file, .llseek = no_llseek, }; @@ -207,7 +236,9 @@ mlx5vf_pci_save_device_data(struct mlx5vf_pci_core_device *mvdev) stream_open(migf->filp->f_inode, migf->filp); mutex_init(&migf->lock); - + init_waitqueue_head(&migf->poll_wait); + mlx5_cmd_init_async_ctx(mvdev->mdev, &migf->async_ctx); + INIT_WORK(&migf->async_data.work, mlx5vf_mig_file_cleanup_cb); ret = mlx5vf_cmd_query_vhca_migration_state(mvdev, &migf->total_length); if (ret) @@ -218,6 +249,7 @@ mlx5vf_pci_save_device_data(struct mlx5vf_pci_core_device *mvdev) if (ret) goto out_free; + migf->mvdev = mvdev; ret = mlx5vf_cmd_save_vhca_state(mvdev, migf); if (ret) goto out_free; @@ -323,7 +355,7 @@ mlx5vf_pci_resume_device_data(struct mlx5vf_pci_core_device *mvdev) return migf; } -static void mlx5vf_disable_fds(struct mlx5vf_pci_core_device *mvdev) +void mlx5vf_disable_fds(struct mlx5vf_pci_core_device *mvdev) { if (mvdev->resuming_migf) { mlx5vf_disable_fd(mvdev->resuming_migf); @@ -331,6 +363,8 @@ static void mlx5vf_disable_fds(struct mlx5vf_pci_core_device *mvdev) mvdev->resuming_migf = NULL; } if (mvdev->saving_migf) { + mlx5_cmd_cleanup_async_ctx(&mvdev->saving_migf->async_ctx); + cancel_work_sync(&mvdev->saving_migf->async_data.work); mlx5vf_disable_fd(mvdev->saving_migf); fput(mvdev->saving_migf->filp); mvdev->saving_migf = NULL; @@ -558,6 +592,13 @@ static int mlx5vf_pci_probe(struct pci_dev *pdev, return -ENOMEM; vfio_pci_core_init_device(&mvdev->core_device, pdev, &mlx5vf_pci_ops); mlx5vf_cmd_set_migratable(mvdev); + if (mvdev->migrate_cap) { + mvdev->cb_wq = alloc_ordered_workqueue("mlx5vf_wq", 0); + if (!mvdev->cb_wq) { + ret = -ENOMEM; + goto out_free; + } + } ret = vfio_pci_core_register_device(&mvdev->core_device); if (ret) goto out_free; @@ -566,8 +607,11 @@ static int mlx5vf_pci_probe(struct pci_dev *pdev, return 0; out_free: - if (mvdev->migrate_cap) + if (mvdev->migrate_cap) { mlx5vf_cmd_remove_migratable(mvdev); + if (mvdev->cb_wq) + destroy_workqueue(mvdev->cb_wq); + } vfio_pci_core_uninit_device(&mvdev->core_device); kfree(mvdev); return ret; @@ -578,8 +622,10 @@ static void mlx5vf_pci_remove(struct pci_dev *pdev) struct mlx5vf_pci_core_device *mvdev = dev_get_drvdata(&pdev->dev); vfio_pci_core_unregister_device(&mvdev->core_device); - if (mvdev->migrate_cap) + if (mvdev->migrate_cap) { mlx5vf_cmd_remove_migratable(mvdev); + destroy_workqueue(mvdev->cb_wq); + } vfio_pci_core_uninit_device(&mvdev->core_device); kfree(mvdev); }