From patchwork Tue Jan 30 17:02:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 13537792 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C18D712838F for ; Tue, 30 Jan 2024 17:03:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.41 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706634209; cv=fail; b=MWi54qsNi7ajKNnU/t9nvPseAUnHNDRoW7UJp50vQJBei1P+Ye06rnaQDQZByUeBm/fGQo3IlUgV0p0heiQZ0Y5XSzT77G3FPRyCmd3cQiHi1pp4Az5ds7Yz3jBIeTVejZcwo1Wl/JPhARqI1vFiP9me+ftxINU2dCdUvT0UMFs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706634209; c=relaxed/simple; bh=bEStpGhp/jxgcocrPeRlB2Z2Aq4KVLJWW81SgF4OK+s=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=fEMjg6bAqDxpQ3skVfgEVI3ekmSMHLT5AaSgqAS5JZt4O4VwKeU4vs5Za+XTU+CGWJa51MKjjVgcycu8ZJZQSb1FnY2wkmI68ZOK80/eu9qruFEQWpNPro8phiLz2CctMsW5bwiiP/eqcA8Ra11n096hxqTwlAb+rt9qBQB/BE0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=dWcPmKxQ; arc=fail smtp.client-ip=40.107.236.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="dWcPmKxQ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ly05MkkNnsXiBnVfKDjVFlPiC70qP7TER5yoemvlsjU99T3mcSDI1r97Co4JLj7/J3ex1rPlF8c79DIdpFgm4S2YlNW333b+5cLCtbWDWOuRnv+eBntyHv2KwjhG+IJJRocHynPzcNp/vA7Gg5JoqwPDRKrAXuLvg6UZjd1zKvWvtyBoOsa00yFIZZTLAL7OlrA8Zk+3S5ElgcY9pGIkRDbsk2TMpEQig8Svp7n3jQi7jz8UXAbxnCizovvf+KQ7vqUgkEnn7RmHYIcel6i3gKDU/CGy2BpBjZoG9/RrDjsZLKB21gCH6P9nWWkHdQoya3YZeADkSU/hg9CDXZ354g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9m5z4j22vPdJjJPs+MBTWAJbpLlAEXZmcstO+GhBDys=; b=NQIxWAXVMzvSZvn0PPvoNro0jWOMYZAmJoa2St0gTsP0mZMqsS94RlUvprcl3Q+kRNwKQiRm/7neSJ70B8rvS0P8gNwzeghEkaEVchNWhHu4Y5q73NvtaRA2IdUwmBtZ+UZrtp9kcgWsJ5m2LJR77+s1R81E4KLKqfIk3VZ1qyhXUzfhH9DZPGnmCDhvYx1LxRZQAzQI75u2MC4B/oyODNt1iCSCcuwvj0Er37Gyqkhc6BBOmFSWVWMylIN1gRvYO1JVBZYe9zE7i/J1i1rmSz649zqr3DwSwkvZ9P0RJPfsg15p5YRddLMBGT0WIJdNcHfWrHxaV/vwnrxd9jJtzA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9m5z4j22vPdJjJPs+MBTWAJbpLlAEXZmcstO+GhBDys=; b=dWcPmKxQk0Ifw/oYmaoEtZ7miy+EN9WVoisuwA1/qv9N6gw+Fmcl6QGQjClpY237xc19Hf+MeT47uekeCn25DieoHdqBd4fpwr9h9yzj3qfMQHJTlghkKxT/VOZZC7j+u/3MYEmWDSWi2A20x4V4ZVmgK0DWKqFnA/fbP/CyL1k19TJBn5naVcEf9ydaL6E94hqmdF5Lkbd8zNv1fZTKmsPVUdzxuktThrUrS9m93sqeNn/8ZeuCun8FNVJTo9+NSNEYluDWYjbCtERr3Z3e6iYI5mYgekiTn8wra2kX6dZ4foUmM2KeAWEUtaXUkq53Y4rftLSCrLfNw0jo4AR+vQ== Received: from MW4PR04CA0322.namprd04.prod.outlook.com (2603:10b6:303:82::27) by IA0PR12MB7532.namprd12.prod.outlook.com (2603:10b6:208:43e::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34; Tue, 30 Jan 2024 17:03:24 +0000 Received: from MWH0EPF000971E3.namprd02.prod.outlook.com (2603:10b6:303:82:cafe::eb) by MW4PR04CA0322.outlook.office365.com (2603:10b6:303:82::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.22 via Frontend Transport; Tue, 30 Jan 2024 17:03:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by MWH0EPF000971E3.mail.protection.outlook.com (10.167.243.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 17:03:24 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 30 Jan 2024 09:03:07 -0800 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 30 Jan 2024 09:03:06 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Tue, 30 Jan 2024 09:03:04 -0800 From: Yishai Hadas To: , CC: , , , , , Subject: [PATCH vfio 1/5] net/mlx5: Add the IFC related bits for query tracker Date: Tue, 30 Jan 2024 19:02:23 +0200 Message-ID: <20240130170227.153464-2-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20240130170227.153464-1-yishaih@nvidia.com> References: <20240130170227.153464-1-yishaih@nvidia.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000971E3:EE_|IA0PR12MB7532:EE_ X-MS-Office365-Filtering-Correlation-Id: 84ee3e33-e7c8-4f1a-959f-08dc21b55eb6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: A9W5CBP5Hr6Pik1Wa3FqFFGOzRLn///5l4U/1zbU4aTW1Uf86Zbf5iLstGYCwP5aeDbd30G+hFXnTCEatBxWEHpl1YfoZ9T+J7GoG43U11DDYSGGZl26NNOA+Z5ktPhGFQRfHHVSOeQQNPlvcUpz+HirhphzyU9VAyaFxslnON5ZCN+Jkhn9Ex4YlyExePx/FeEmon7+uTpmBiPxhyiS5ZRXUMuWmD/2Euuwdr0b5MtGbEZBWMHdp9CSbJ9yJNLAp9KCNhOdn8B5QsrMa/ID4V1O+zzH9W7n+0PE2KFIpUUQo7BlfzzNayj0zpNURth72tLQTBZlGnuhwjggmzPkKbkHNqdS/ph/U/OEF41X8SkV6swmp4Y6zZic7k4+zqyze0oAVB41+xXKQxgJiMxD0r174NsWGxv5f9I6vhkrLWph1awOQbUr46QBn5uZ5iyCBkiGAp//TkQ5xy4IpMnjx1tm62kV6rT/SdXCWKVwbEMda5anJ5kfo8nnUKhQHdmBPHZNVm5s8e0cZyVh+76yGWX/WsWjNzDcodnybwZ31/9vt82UWvkVohzjKQ6UYrSLnXNvS6z0VovDGiMq971ocf/ykx7QH5v7epE6uP8mbBRnCRbEej9HmMlA9DpGAiUkCEfeaJyG5T5NnpXKD/m5OP3+01nFvv+B/N9x3kcR8PuoNPo8KKn+W85L1tRzM1O/vCMyN6QTW8JWFxKpl4Mr/TBzZSvgGvg7uzsYO9Llps7yxLcmlJq2PH6D4CDiGUnK X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(376002)(346002)(396003)(136003)(39860400002)(230922051799003)(1800799012)(82310400011)(451199024)(64100799003)(186009)(36840700001)(40470700004)(46966006)(47076005)(2616005)(426003)(1076003)(107886003)(336012)(26005)(110136005)(7636003)(4326008)(8676002)(5660300002)(8936002)(478600001)(4744005)(36860700001)(7696005)(2906002)(6666004)(316002)(70586007)(70206006)(6636002)(82740400003)(54906003)(41300700001)(86362001)(356005)(36756003)(40460700003)(40480700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 17:03:24.0651 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 84ee3e33-e7c8-4f1a-959f-08dc21b55eb6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000971E3.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB7532 Add the IFC related bits for query tracker. Signed-off-by: Yishai Hadas --- include/linux/mlx5/mlx5_ifc.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index 6f3631425f38..cb08b5e36c21 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -12619,6 +12619,11 @@ struct mlx5_ifc_modify_page_track_obj_in_bits { struct mlx5_ifc_page_track_bits obj_context; }; +struct mlx5_ifc_query_page_track_obj_out_bits { + struct mlx5_ifc_general_obj_out_cmd_hdr_bits general_obj_out_cmd_hdr; + struct mlx5_ifc_page_track_bits obj_context; +}; + struct mlx5_ifc_msecq_reg_bits { u8 reserved_at_0[0x20]; From patchwork Tue Jan 30 17:02:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 13537790 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2085.outbound.protection.outlook.com [40.107.244.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44B7012838F for ; Tue, 30 Jan 2024 17:03:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.85 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706634206; cv=fail; b=cFc36pIp257xeOvaLR2uNU3JFDlb1R+yFEd5tRuTsy7e0tz283hH0iAaUrLs8Da0sMupBfiPmu3YUC0KCTylNEQIkZL8jmlfgVNViDoyI0cCv/aZa5NROao3gLOpaWwQXk/C0RJVky/d9L9wx5/HmKMhDsFkY7muqTOYXPg/CyY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706634206; c=relaxed/simple; bh=yMNwWfAW/xkU50N7hoNnye53o5Xxn8/F1GOKB8r5XiI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=daspAewAkoxKuMSRkv/jsR8TLZql1URXea8fhGhKRW0fOb62JvLjqu4TUwGvrcNtEObdX0K0UcuDMtbL3aKcBVh20D3sD7iOejcv8lACTBkWQB6CR3vZNvZvbm3Yp7ccql30ASoQlUY0ZARRpNEzY7j7BFbU2qlnAGqwwzAjAHs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=QarO7WYP; arc=fail smtp.client-ip=40.107.244.85 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="QarO7WYP" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=B4a1eciaen9vO0T5m3BW0su31vRNuOrxRzibdqJV4X2sL7LtF5tx/ndQVRQWBUo2+q6DFWyFjehcPRqjPBWEjI5gHynGZrDH9OgmzRriQ96ZaEAwoqjdZ2relt4pZPxDA/KD4iAP4MrKIsJlLxmwv01bWRddiifV4b9qvlsVhUuxaXosPiF7da4Ez5pPMklPFl5STkWjGkfQzTufy85vuWRE+q297RHXBQNehPZ3KHLUC1lPpi4qq8alH1rXOeMiu5K2A8LJG0FLd+BPjg5KpOAsv8PGQOC4xCkqTla4jiWLd11Mx/Y9V4iDEwRMH40PHm5IgCl6pOb0wCK2brklxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=POUp2n4MhdH2g+sSK8Guq4G0ih3QaEMbS9ERwNXrO7g=; b=DpRi18XzueCcPsyILbkW9IF5tOkSqUpIKrsB6ds6xo+bG45yy82kb8mPdCoqggB2vD7AIRhuhK7kjSsVj72hkv0zTEOXvokC+z7cV6vccOpXZr54jqcSLkMUMMgpwehroYf2lnccp5dyhPuBXx/Wke//pFT9++cd1HAHmpzJ7HqihudNkYk8GYoEeTAQme0EnBJXZjNwkmXDDHRmoE1P5FgfWDK0tiqS5iYfNJfCZBEQKqFiTLiEcRxEHwf12GVShT0QDS1RoRNsvuw4qcLQI5JOiNGxQsdOABDQdvmQJLxmHmO8hCew78wdhe7uePeuZu6KFDKL2xLlcwhc1I6AVw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=POUp2n4MhdH2g+sSK8Guq4G0ih3QaEMbS9ERwNXrO7g=; b=QarO7WYPQbmy/KlaW3Spa5yXhHqc7MiL8KJ4DbEvaYrEfluGes/9wuQdzwQ2sarP6d/nCtcFjZc5ADDkv1eXnMkms1oAE2nUZIb3KxAePypOc/URRZKWbOjgf6w/2mu60MOqkCEZLS6QhaQBjbbmDuVZN7UISXWtlPjBpI/H1kOWPSdtCzdF7y+WmY0QHRQPJ7Tu7+ot5RT4N8Xlsz62d438aKxGqVGNG6Ari2k1V1TpcBJBnFeHqKSAr4VcQuLglyD9+4bde2h/qVL3z4gFgqpLhloDRCvJzxlo6u62rFSEmqNFcmFT6hVc9ZcCGJdTk2oKJXir3w1pCcMQjABiyQ== Received: from DS7P222CA0016.NAMP222.PROD.OUTLOOK.COM (2603:10b6:8:2e::8) by IA1PR12MB7709.namprd12.prod.outlook.com (2603:10b6:208:423::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.32; Tue, 30 Jan 2024 17:03:21 +0000 Received: from DS3PEPF000099D4.namprd04.prod.outlook.com (2603:10b6:8:2e:cafe::8) by DS7P222CA0016.outlook.office365.com (2603:10b6:8:2e::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend Transport; Tue, 30 Jan 2024 17:03:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by DS3PEPF000099D4.mail.protection.outlook.com (10.167.17.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 17:03:20 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 30 Jan 2024 09:03:09 -0800 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 30 Jan 2024 09:03:09 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Tue, 30 Jan 2024 09:03:07 -0800 From: Yishai Hadas To: , CC: , , , , , Subject: [PATCH vfio 2/5] vfio/mlx5: Add support for tracker object events Date: Tue, 30 Jan 2024 19:02:24 +0200 Message-ID: <20240130170227.153464-3-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20240130170227.153464-1-yishaih@nvidia.com> References: <20240130170227.153464-1-yishaih@nvidia.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099D4:EE_|IA1PR12MB7709:EE_ X-MS-Office365-Filtering-Correlation-Id: dbc4fb85-6f61-4c9c-28f9-08dc21b55cac X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OFXuTjOwsU+4sjEq9Xv+ilTTHwuVb4x/4RmmJpT74dvARI+HnDTuD9fVA7p3loq52rpMLoSEmAE35zCjZaqMpu3/GoxU7Ohd7kFekXsoqQWPKrVfBnTZgdM7I9OJ/86UB4GfMYA8xnRyeS5dMSNKYQTuGPR9NOiUwRYOMyWDBMmAtr1j+gJZoITk28YfmzOBeIv7j0AmSGgRV3qDA85MVVAi2Wn+d8ABita9F/lNeXvrjyiWLthqKkVGLLqC0UMCLDfq1kn1j4m+yOcVXhx28CUqwHaJ6k8Fd0jdByDMgTWvK3Vx1xW7wXS+IIJEV+9svOCxsvJ0k6stYozcXYgQNA4cqdc3d23oZo/uq+Wj1TR+TMU4WEAt8emAEpSmLEQa4fTx3rVOddw8+sK1uR+xjSye1i3oax/Hru+q+pe4iYwhOQoeOvNjkbtyZNTlZ9NvZlkjfFOIK+pHkTyvYkXYEcbzPn0sblOGoFXZKnJWflFASUAd/a/YZKRV/jUaM43YDc43umcSocKjELnpQznMw+7hAFiOYZRw6+UwioMcHJL0dAf7EQJWiW4iDOVGjQcnRnanQYIlBGDP630ZFNjbCWdhnTzI7r2HgKXX17EupirM4hWFysuncsr6vc0AWq1SBgiuF4E808yrL8hrE2VRazPLGV/kxyAm9QBwHfBywneWAUv89OoPtXwsexBWHd9iQ7qKPF51eT7RzN99pVVV5HBOZqoZcL8AZ+YDhbEC6XbPPvEUlJVerNOQkGQwlLUT X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(376002)(396003)(136003)(39860400002)(346002)(230922051799003)(451199024)(1800799012)(186009)(82310400011)(64100799003)(40470700004)(46966006)(36840700001)(83380400001)(426003)(1076003)(6666004)(7696005)(336012)(478600001)(36860700001)(8936002)(47076005)(316002)(107886003)(2616005)(41300700001)(5660300002)(4326008)(2906002)(8676002)(70586007)(6636002)(54906003)(110136005)(70206006)(82740400003)(26005)(36756003)(86362001)(356005)(7636003)(40480700001)(40460700003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 17:03:20.5945 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dbc4fb85-6f61-4c9c-28f9-08dc21b55cac X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099D4.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7709 Add support for tracker object events by referring to its MLX5_EVENT_TYPE_OBJECT_CHANGE event when occurs. This lets the driver recognize whether the firmware moved the tracker object to an error state. Signed-off-by: Yishai Hadas --- drivers/vfio/pci/mlx5/cmd.c | 48 +++++++++++++++++++++++++++++++++++++ drivers/vfio/pci/mlx5/cmd.h | 1 + 2 files changed, 49 insertions(+) diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c index efd1d252cdc9..55ba02c70093 100644 --- a/drivers/vfio/pci/mlx5/cmd.c +++ b/drivers/vfio/pci/mlx5/cmd.c @@ -149,6 +149,12 @@ int mlx5vf_cmd_query_vhca_migration_state(struct mlx5vf_pci_core_device *mvdev, return 0; } +static void set_tracker_event(struct mlx5vf_pci_core_device *mvdev) +{ + mvdev->tracker.event_occur = true; + complete(&mvdev->tracker_comp); +} + static void set_tracker_error(struct mlx5vf_pci_core_device *mvdev) { /* Mark the tracker under an error and wake it up if it's running */ @@ -900,6 +906,29 @@ static int mlx5vf_cmd_modify_tracker(struct mlx5_core_dev *mdev, return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out)); } +static int mlx5vf_cmd_query_tracker(struct mlx5_core_dev *mdev, + struct mlx5_vhca_page_tracker *tracker) +{ + u32 out[MLX5_ST_SZ_DW(query_page_track_obj_out)] = {}; + u32 in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {}; + void *obj_context; + void *cmd_hdr; + int err; + + cmd_hdr = MLX5_ADDR_OF(modify_page_track_obj_in, in, general_obj_in_cmd_hdr); + MLX5_SET(general_obj_in_cmd_hdr, cmd_hdr, opcode, MLX5_CMD_OP_QUERY_GENERAL_OBJECT); + MLX5_SET(general_obj_in_cmd_hdr, cmd_hdr, obj_type, MLX5_OBJ_TYPE_PAGE_TRACK); + MLX5_SET(general_obj_in_cmd_hdr, cmd_hdr, obj_id, tracker->id); + + err = mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out)); + if (err) + return err; + + obj_context = MLX5_ADDR_OF(query_page_track_obj_out, out, obj_context); + tracker->status = MLX5_GET(page_track, obj_context, state); + return 0; +} + static int alloc_cq_frag_buf(struct mlx5_core_dev *mdev, struct mlx5_vhca_cq_buf *buf, int nent, int cqe_size) @@ -957,9 +986,11 @@ static int mlx5vf_event_notifier(struct notifier_block *nb, unsigned long type, mlx5_nb_cof(nb, struct mlx5_vhca_page_tracker, nb); struct mlx5vf_pci_core_device *mvdev = container_of( tracker, struct mlx5vf_pci_core_device, tracker); + struct mlx5_eqe_obj_change *object; struct mlx5_eqe *eqe = data; u8 event_type = (u8)type; u8 queue_type; + u32 obj_id; int qp_num; switch (event_type) { @@ -975,6 +1006,12 @@ static int mlx5vf_event_notifier(struct notifier_block *nb, unsigned long type, break; set_tracker_error(mvdev); break; + case MLX5_EVENT_TYPE_OBJECT_CHANGE: + object = &eqe->data.obj_change; + obj_id = be32_to_cpu(object->obj_id); + if (obj_id == tracker->id) + set_tracker_event(mvdev); + break; default: break; } @@ -1634,6 +1671,11 @@ int mlx5vf_tracker_read_and_clear(struct vfio_device *vdev, unsigned long iova, goto end; } + if (tracker->is_err) { + err = -EIO; + goto end; + } + mdev = mvdev->mdev; err = mlx5vf_cmd_modify_tracker(mdev, tracker->id, iova, length, MLX5_PAGE_TRACK_STATE_REPORTING); @@ -1652,6 +1694,12 @@ int mlx5vf_tracker_read_and_clear(struct vfio_device *vdev, unsigned long iova, dirty, &tracker->status); if (poll_err == CQ_EMPTY) { wait_for_completion(&mvdev->tracker_comp); + if (tracker->event_occur) { + tracker->event_occur = false; + err = mlx5vf_cmd_query_tracker(mdev, tracker); + if (err) + goto end; + } continue; } } diff --git a/drivers/vfio/pci/mlx5/cmd.h b/drivers/vfio/pci/mlx5/cmd.h index f2c7227fa683..09d0ed6e2345 100644 --- a/drivers/vfio/pci/mlx5/cmd.h +++ b/drivers/vfio/pci/mlx5/cmd.h @@ -162,6 +162,7 @@ struct mlx5_vhca_page_tracker { u32 id; u32 pdn; u8 is_err:1; + u8 event_occur:1; struct mlx5_uars_page *uar; struct mlx5_vhca_cq cq; struct mlx5_vhca_qp *host_qp; From patchwork Tue Jan 30 17:02:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 13537793 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B1EB91292DE for ; Tue, 30 Jan 2024 17:03:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.41 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706634210; cv=fail; b=o/3lX6LOIrNaLYIAzM4vmealBMFXsQer+hy2YU++jmmAVxCfQZcM87sBXSiO6pZzRAHVXBIR4Zh3l85B/CX6IiumQwZudWF6cx9oN5kSWxPhFOMZt0Y+CaXQxeiVNLluoPg398J336pyj/bGYOsJU6ueaKoLEmIEYAMg6cKLNvQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706634210; c=relaxed/simple; bh=fmDV6O0A8bSm91lUtmtYy9SJDYRqtSEejmfL1OO65GE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=m89emeps7lqWQRjN41qcNdibv6b+NkoY8xJ7s3qYfUV6B5xIXq6FbWYBDa7IM51Ee3dzdqin2Y/GQS1Td69uYhx4RsF9rxgRoyWk/tDgryDA7Ee/sphRWIhCsgDo9Rbmgb+AjAry/1r2HG1GLmo4G2Wq4qvq+oSikw638IKbLkw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=JKWTv5oQ; arc=fail smtp.client-ip=40.107.236.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="JKWTv5oQ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M4sV84aAw9Tv/P4VRPJo/9TK6W8NwEW7OBY0hVwSs3zXzV6w74ORgWIW11Gg86qAxyPPCxdlB74NSmUkvqIpDXFTosUk9wr/QYl9Qfm4B+HZiYf1kkncgh8DDAacb6A8xxs8GJEzjbazDr5JkKBOAmsdY2BKjC7E+oZJWLbPvz5V1woqrDhKrJf8yvK1J8jf+qfl8AJnCZbqfVwyO1Y+TOiX61UHSotK0bMwg6fyPjHRF694kzSBsS0j+bnuOYMN/REGTF3rR5giXGL4Mg7r1TDcrb/sl4yHuR3KAl8jLKbpZQ3oB4br7mYgKnDYhr4FiXCYmrzhjRCurnaDAajZUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=P/sEk1k9YtRPQfalPnXRXOZn57RWQOeRsPP5KWqcmeA=; b=lY4dZVI6GLVS2XDjXpS1poXT5XPHkKee4cs/TaoTAk0LKOkhiUd9zVkOUCGkb/LARtkB1LfbuE8oJNwhuDosqOQOFIvoCO48zK9l45lec4suKsIMboMwJXsNmKP+Ro/H1C2CMgVzus01ipNJk//DVdXGS/HOMsUwjZfucBTg3K2naKLTRCN2Ha/IT1JZPcHX/yRPu5JLzkWqt0LmGlcP+5Qj61hhVRUTAIDCzG4EadGxuWZ/EG7gMFaXIJd5n2C0nnS/8UuAdpRQfDviPEWlpuck0oJ+ZZNqMnZF+gpvM2mzrOj7/qOjLO0VDMTGbqj4sYMUgAe0SOit1H8KGN5f3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=P/sEk1k9YtRPQfalPnXRXOZn57RWQOeRsPP5KWqcmeA=; b=JKWTv5oQSLkxO6NYE9QsVvPQUW/obE/2wPmAj0+X/WROHO0KtoUv6gzywmP2PsrSmZ87BXypWu3eeNoHeNwSnBdnZXirlrmfDO14sPH9RJ/p8REnCLJx5riZHILJSYCRQLfKZ/fnavfCTtoGWCPt7kpUhVWHLq6JAN7y8QxzSUrenU5PDTpdxZJP7XS9ogMU0ZhWfBM7/x2XnGsZrT330Mz1l4WN+WlVjBIP/6rZkBo2MS17fwbizpkyy9x9VGi4H3snAqbKJ/mJGucMEEzXiWXfIfW/2KRwMkSrUBhKW1s0HOFg1g99YyegEa7J7/M+XEHfgOnclZQ/t71K6v8ilQ== Received: from DS7P222CA0024.NAMP222.PROD.OUTLOOK.COM (2603:10b6:8:2e::13) by BL1PR12MB5239.namprd12.prod.outlook.com (2603:10b6:208:315::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.35; Tue, 30 Jan 2024 17:03:24 +0000 Received: from DS3PEPF000099D4.namprd04.prod.outlook.com (2603:10b6:8:2e:cafe::f4) by DS7P222CA0024.outlook.office365.com (2603:10b6:8:2e::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend Transport; Tue, 30 Jan 2024 17:03:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by DS3PEPF000099D4.mail.protection.outlook.com (10.167.17.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 17:03:23 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 30 Jan 2024 09:03:12 -0800 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 30 Jan 2024 09:03:11 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Tue, 30 Jan 2024 09:03:09 -0800 From: Yishai Hadas To: , CC: , , , , , Subject: [PATCH vfio 3/5] vfio/mlx5: Handle the EREMOTEIO error upon the SAVE command Date: Tue, 30 Jan 2024 19:02:25 +0200 Message-ID: <20240130170227.153464-4-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20240130170227.153464-1-yishaih@nvidia.com> References: <20240130170227.153464-1-yishaih@nvidia.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099D4:EE_|BL1PR12MB5239:EE_ X-MS-Office365-Filtering-Correlation-Id: 4abdcf87-3387-409a-d72e-08dc21b55e81 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Vdt8ZNNPMy4Py0O5XykIdi/12YviD8VU8q3RLTfwudZRONhH06RNdLNL8TElLe2CLdF2EQX5F1DTfK2sqZEitXiueQK/IHJ0qkeIW0RoPMuHlanwtaD0zPqfg8c9/57hXfvcr2tB7DK7p6ZTzv+EhR+owIm4b7tB26cjhljVwWj1efRnfGfvb0y8w5+yLJKmt+GJNnMmvclaqpKq7hFij2zlmWuqGUFQ4BQJOYLRj7xgb0hHFVcLky5K61niR70Wr2OGk6egVOSMD2Kyl4iZu5duF9TAq/ofmHU7/sClXrvxjNjPc3f5fb6gaYm3DqRi5MxTb3rBE8NR13+xaK6kneSJvxu+g6JygjkQ2swIcNSBIb/i6Aw1k6ptxD1RhiBsatQt8FJwM4ToInONQEzgCwXg4BvWLVzn1hUPGVtryboaGqi2Mye4dLBDoUXyQsNKIHeThkrT01frneMRd+9nnwIuFqiyKBYmYsWcW+QaG+bmah3nZPJuDsIc7Q/yYVjNLf2azGJw9/oV9bZTCiAVfkabwuly+JrzoF6Atrk/vouUTRa/4EIKx+/TijR9VGm3gXQ0ICEN6eJ8HxoatXcxLAFsu7nH5kbg50UeBnSz1KO4CtmoWegMlAiEP8ZtHVSoEQqNMW/gIWNFtpnP5sHaX3yEtFRCflhXA+sdDs93L3USn1r9zqRkXlNENT0AYe6yVIcUeKGdfV/ThezrDC8n9lbD6Svl750koV4Y4dshYzRP3l94rORYoodvTv/VeIJp X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(39860400002)(376002)(136003)(396003)(230922051799003)(186009)(82310400011)(64100799003)(451199024)(1800799012)(40470700004)(46966006)(36840700001)(8936002)(4326008)(8676002)(5660300002)(86362001)(70586007)(110136005)(2906002)(70206006)(316002)(6636002)(26005)(7636003)(36756003)(47076005)(54906003)(36860700001)(82740400003)(356005)(478600001)(6666004)(7696005)(83380400001)(1076003)(107886003)(336012)(426003)(41300700001)(2616005)(40480700001)(40460700003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 17:03:23.6726 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4abdcf87-3387-409a-d72e-08dc21b55e81 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099D4.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5239 The SAVE command uses the async command interface over the PF. Upon a failure in the firmware -EREMOTEIO is returned. In that case call mlx5_cmd_out_err() to let it print the command failure details including the firmware syndrome. Note: The other commands in the driver use the sync command interface in a way that a firmware syndrome is printed upon an error inside mlx5_core. Signed-off-by: Yishai Hadas --- drivers/vfio/pci/mlx5/cmd.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c index 55ba02c70093..bed1ba9f1bf5 100644 --- a/drivers/vfio/pci/mlx5/cmd.c +++ b/drivers/vfio/pci/mlx5/cmd.c @@ -614,8 +614,13 @@ static void mlx5vf_save_callback(int status, struct mlx5_async_work *context) err: /* The error flow can't run from an interrupt context */ - if (status == -EREMOTEIO) + if (status == -EREMOTEIO) { status = MLX5_GET(save_vhca_state_out, async_data->out, status); + /* Failed in FW, print cmd out failure details */ + mlx5_cmd_out_err(migf->mvdev->mdev, MLX5_CMD_OP_SAVE_VHCA_STATE, 0, + async_data->out); + } + async_data->status = status; queue_work(migf->mvdev->cb_wq, &async_data->work); } From patchwork Tue Jan 30 17:02:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 13537794 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2089.outbound.protection.outlook.com [40.107.93.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 46B4F1947E for ; Tue, 30 Jan 2024 17:03:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.89 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706634212; cv=fail; b=cVHLfBf7RZLiNPImERK6dch5wxurmFdKJ/pLFhTHbLZwI1PIlPdgPolE2ZCpJ9Vj52PgYYBou15Kp+IjeiZhUo0h9S1J3eUR2+p54MnlPDlkW+bitrHiRitUaXLucpmMsJLjum32LXJ9GUqrLIptgH7JhOKssZBKcw+5ihE1FM4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706634212; c=relaxed/simple; bh=itpqtQiR+i4bIu8JvhYCCEJZNENJMrkgDe41Hjy9RMA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=OSlv28mYZ4wQcCzhBfvmkaX03yXzj2dyUpt8ywF75K/qD5f/AutfKCWMQp5LOi8B6K/zmMOz1wigGUDCoDefAyISNELaFtC7L3YaNjs4x7YJfXn9TZubr0sTxG8czEv9KqRMURoloeYNqmps1fxTp2dM19kvYB7bbNxvjTIngcY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ROcY+PjY; arc=fail smtp.client-ip=40.107.93.89 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ROcY+PjY" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FhhVLzUylsF0UHoIO7MCsuKr1RyFIrWXwah0iXh/QioNO0x3U0GB/05QJFIxo8i/v+YSvgIDTmbDAinUHI3mxSdQ84ysbz8fU7LddkCt9ufPFqic5WMYP8rj02s/oxXj4dJy72DtwwaGMrXT1Mm3uNN3rPhm7D91E7bW+nFekpQPd5jZqViKw0uBPkVo20S+InERUr7/1iMraYsheuQ3vRzkvbIxA4orw7RJ/ezi3/16QlhfBdg1wSOQiucWiADSk4a5D+5tteclYN15+AE87lq9TwypQmtw0sp4ooGsh6cVevYl9ApMbU8ErEYA5LKK3iw0wVxk1mfVvyIKM+LdbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WrZQZdKGxXTkM9EmnvPmx5VVc+WnnBBRaMEFHfSqs0Q=; b=PalheGOamdYhRKL8n4NbsQftLPM1KW9dit3E5KhOqo9Rcbqrkl+lFh0j60NzUncmzoalovrXW5OQyuluAscJhBhe+vDW1JlmvUtzE/aZDCEKtVIVDE0VkMU5yJSSjqfjBuZTT/FzAKDLU9dVLmGmQoVRLQ0e7KEx/aZ6ImUuQBdVtI2X2y701xXMzsP4GYI1h8co9kb4kaSUL7iVUzdY126Hij35wAnSjXeoXVtp7flpB6jeGVgEBihXGAZHwIYINL957+C6i8CWeWUreHKD0pm0y08MpwS0ZQBrXQxHsTZdn4xVL0MYDJYEF5aaMftSnkgLRXO63lPliNhA3lnWZw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WrZQZdKGxXTkM9EmnvPmx5VVc+WnnBBRaMEFHfSqs0Q=; b=ROcY+PjYZlxBU/dkI0dVpN3f20QSLR3MTRl3+drOOkMADwp+ccb/i/prpzgDh8Qx5jd8Xv9+HVtbgzdwHN9ZjWs85g7z/3vQQVPvJoU+s2wZIMpgWHYCFklarl/JE7rkI5OgRV256+WqQaKNCoOGZjts1rIN3prOQTYahXFrLswSos7zXqWIQpMmtWhzQ5WxxlgU0K1NiAh6jPOUjaoSxa+yRJikKB1dGw0C8s7xPLe8ZgKOxq6Y/04NlaZXVdu3v5x9w/M/ROBIc4iqAL1U1DNfZqzRo/YQVZjbjxb5RhDctwGrPSdSJ5/X5PYNbwYZj/2KJlR3k/aWgJBlwG+DcA== Received: from CH0PR03CA0389.namprd03.prod.outlook.com (2603:10b6:610:119::28) by DM4PR12MB6061.namprd12.prod.outlook.com (2603:10b6:8:b3::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.32; Tue, 30 Jan 2024 17:03:25 +0000 Received: from DS3PEPF000099DA.namprd04.prod.outlook.com (2603:10b6:610:119:cafe::9d) by CH0PR03CA0389.outlook.office365.com (2603:10b6:610:119::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend Transport; Tue, 30 Jan 2024 17:03:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by DS3PEPF000099DA.mail.protection.outlook.com (10.167.17.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 17:03:25 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 30 Jan 2024 09:03:14 -0800 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 30 Jan 2024 09:03:14 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Tue, 30 Jan 2024 09:03:12 -0800 From: Yishai Hadas To: , CC: , , , , , Subject: [PATCH vfio 4/5] vfio/mlx5: Block incremental query upon migf state error Date: Tue, 30 Jan 2024 19:02:26 +0200 Message-ID: <20240130170227.153464-5-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20240130170227.153464-1-yishaih@nvidia.com> References: <20240130170227.153464-1-yishaih@nvidia.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099DA:EE_|DM4PR12MB6061:EE_ X-MS-Office365-Filtering-Correlation-Id: 42d848e5-a048-4e42-5ece-08dc21b55f95 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7MN4Pic8kzgIdlEQEdZWKdDFYV59+f04WphiBNQr1cSK3O4z6+ljjoHAmzA7hlBMTJHUvjOhooozkco3carjMs6Jc5dPNhHEzFtEt4JB9E8GhZ3csqacriCMOqbcSe1ADHYHSxNAqlztJkCiHgVohg/oN+KkwDrLY6gw6yJNK58sHQa7l7y/zv3pki8wj2ZC2Dg3ut4cS+6VdfJb0QJyXy7sv1NEsWtEwr0fUrqnSOGUVM0+c0XSx//XLMKgl89KplXybgoNS9b1swXWwSGILqSLDojlannxwyUKilihXEvrnXGDRJ8c21CVW0V7u8kp8204mjRMS7XYPNYj72c1aFImmUDj1Kkt7aMi7zziSPUMESo9UO3K7j1IwK0PFbLNXlTPakYcC0UnXmEf4yBrJ+WhGc1TncL3D9Nasjbuq3lEyn/RxP5Yra+Y9HxXmvmSfnOn84WPhK/nF5t0V25bnFFnAPNECmUE28QvFqscRBuRiXQEs5WL1zEtF3/D8XrI0Dk2yLvR66T404rECdR3Qdr+6mpdQcXS6x7I/I4dtOeYyf7zZ7NUEKsBuS8v4pSYK+vRbIBApQNV2XsxQRx54mYdEzCxYth4QFqLdLvpKeyHv/Xtyc5vWjujMZ5GCusA7gVJW+2o1ZQBD/QUn7gXdPzJoFVwRrV4pWhUIBWEwbpEwEtfUFonLp6dykN0CA0BPKpEnJeR4gNPiuB0q50l+ErzewwiWRlT321pH9J9Etk/XTtvN8WuqC5c71vcAsnH X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(396003)(136003)(346002)(376002)(230922051799003)(186009)(451199024)(1800799012)(82310400011)(64100799003)(46966006)(36840700001)(40470700004)(36860700001)(47076005)(110136005)(316002)(83380400001)(6636002)(54906003)(7696005)(70586007)(70206006)(8936002)(8676002)(26005)(4326008)(478600001)(2906002)(36756003)(1076003)(6666004)(4744005)(107886003)(336012)(426003)(86362001)(5660300002)(2616005)(41300700001)(40480700001)(40460700003)(82740400003)(7636003)(356005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 17:03:25.4939 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 42d848e5-a048-4e42-5ece-08dc21b55f95 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099DA.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6061 Block incremental query which is state-dependent once the migration file was previously marked with state error. This may prevent redundant calls to firmware upon PRE_COPY which will end-up with a failure and a syndrome printed in dmesg. Signed-off-by: Yishai Hadas --- drivers/vfio/pci/mlx5/cmd.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c index bed1ba9f1bf5..65135c614b5e 100644 --- a/drivers/vfio/pci/mlx5/cmd.c +++ b/drivers/vfio/pci/mlx5/cmd.c @@ -121,6 +121,11 @@ int mlx5vf_cmd_query_vhca_migration_state(struct mlx5vf_pci_core_device *mvdev, } query_flags &= ~MLX5VF_QUERY_INC; } + /* Block incremental query which is state-dependent */ + if (mvdev->saving_migf->state == MLX5_MIGF_STATE_ERROR) { + complete(&mvdev->saving_migf->save_comp); + return -ENODEV; + } } MLX5_SET(query_vhca_migration_state_in, in, opcode, From patchwork Tue Jan 30 17:02:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 13537795 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2050.outbound.protection.outlook.com [40.107.223.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8C791272C7 for ; Tue, 30 Jan 2024 17:03:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.50 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706634216; cv=fail; b=QEWlTxjC0zP0soQmMNqKkMssGlYaazqKpxJKNSNJnJTFOrpXBGYRbFKLkh4YyFO9n8Qy3LK/gPTJb4kIRgkgVyEI9XjAffREhF9PPlLBZCMWNB2iqJ7sWHkiWqvWf4qr+EK7KKuNqQSIBOqoLqaMofUwiUXkAhAFsa0mlh2bKRM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706634216; c=relaxed/simple; bh=y9EwWf5kgR/4yLBXOZZRAdIv2xsgii7/PgIRfHPc9Fw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=CQ1emDdtQiKUM9CoeVVEE1AcGvopLo95jrOm216N0NFaRGHDDfSgQqWQlaqD5OvE+277FwqkUdNfbaYzd/dCbONxFRmeVcleJZFEFct3PkiTZ0R+uMGQ/nj1wEfaCMNhIgi+8rMtCoxSpebySrGthjPAmTQN2iJrGtaaLdniZhM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=PkABVf/6; arc=fail smtp.client-ip=40.107.223.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="PkABVf/6" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Byh7hEQfA5Qp+Z+JVKQiEcvl8N2zbzPWxf5EkzOgl3z39iUHlDob8O1bHqg8FR9k0YC5favtTjv8RyqvPsq9ESO8vbeiMgmuV+laCRINWiS9ERp+LSyjMcYZ9dZYcxAGPRQFO9rWP0PHuDc0qEHzgPGNQA5Iq5gLxVJhIfUWuZ+2V+DoLE7eIy0IGeqDfNN3vW34aprgdUGGZfi+tebsOg+1O2oi7YL3g4ql+AVjGuvXhtRCZwma/I1fx/7v44PoSZlQ0up0O5b2IlLVVrZOI44pO42j/VA0peh2o+wpeKxeqaCPhaoxOW6wdASlJC6LesSX/drRVaKD3Skm8rxBLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZadIp9AFi7chsGWgnW+qi8pMW4y58mfD5nYvUiS5fM8=; b=MMF6HRtQ5ceZ+RVAsb3hktN6vmgExtIpqG57184cDvLb6y30r4Tm4gASblmcjJ7voeuZ2OFJ7ZnaKlFZq8XJo8ZIlzgIhjT/1mmRYHwcXOOHre0sPwv5JUSrsqj3Gk2QQN3g7tL0f9JeuvWI8MF3UkzExshRnH41Z4onU5juEaFpIklLoOPe1qeSqvpMmN1wsep1NCJLwf0JrBIZ0/kOUrB+4j8pCUP9rIuRZVwIQf5y00N++knOCWasPwdeN4OxLhiHSXM4NoXjW9jqHQw5cUeh3+rUjpNAA5KiTxwuiBjbvVlN/yhdmyk8zQnYmRp6h3vZpywmbnlYPe0CEKrKgw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZadIp9AFi7chsGWgnW+qi8pMW4y58mfD5nYvUiS5fM8=; b=PkABVf/6P/2fO8F8M5lVRavAGNvZqSc8e6NDZEYUfJJspa51lWsHwOjfZzXH28n6leA8BCsc/kzWUIPwinL3rmIGFo2uTrwmB+/6UIUgVl0xv7HV26otJLCnJlIQ0LxzfZoROc96GMxstBbWFBtnzJtdJet0w0o5szTCCygG5ZKrDfUdsDtcaC0bA6sCdL6JYWzZah2ScCOvMpPOay9gDFZ115vV0LIGdFI7DdxYE4OB50AGzDvVZl3Vd/q+qKQLaqLHmrBUxf+r1T0ppQhZTKHumjV/VMbUulvtNJI38uPiIbesR6U65R49heqVRmtcz4vyqzLfjLXJVcLdNIOdaw== Received: from DS7P222CA0016.NAMP222.PROD.OUTLOOK.COM (2603:10b6:8:2e::8) by SJ1PR12MB6124.namprd12.prod.outlook.com (2603:10b6:a03:459::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34; Tue, 30 Jan 2024 17:03:27 +0000 Received: from DS3PEPF000099D4.namprd04.prod.outlook.com (2603:10b6:8:2e:cafe::f3) by DS7P222CA0016.outlook.office365.com (2603:10b6:8:2e::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend Transport; Tue, 30 Jan 2024 17:03:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by DS3PEPF000099D4.mail.protection.outlook.com (10.167.17.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 17:03:27 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 30 Jan 2024 09:03:17 -0800 Received: from drhqmail201.nvidia.com (10.126.190.180) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Tue, 30 Jan 2024 09:03:16 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.126.190.180) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Tue, 30 Jan 2024 09:03:14 -0800 From: Yishai Hadas To: , CC: , , , , , Subject: [PATCH vfio 5/5] vfio/mlx5: Let firmware knows upon leaving PRE_COPY back to RUNNING Date: Tue, 30 Jan 2024 19:02:27 +0200 Message-ID: <20240130170227.153464-6-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20240130170227.153464-1-yishaih@nvidia.com> References: <20240130170227.153464-1-yishaih@nvidia.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099D4:EE_|SJ1PR12MB6124:EE_ X-MS-Office365-Filtering-Correlation-Id: ba234ce5-118d-47ec-2f93-08dc21b560e4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: N4IaV6PxjzfdTh55xW7HmzxtCAcxrGXifhzFxy/x+rmNG+//yHA8tatfWMbA9SPP6cg9uZ6+ePSi0Z5N3nYCMA0nuejQyM/R6sZ/XhOQU/A8hyN/QIjqUGHAvFOx3AryZ6VmqEE733CrZeclMhMLaJUUcCwqmRZFbLJAf+MuIIRv4TJicOJbCDG9Y72w/v610bnl+8E7H54G0MIr62mNbD0HOmlNHxL2NJrQbOoERxdhDDeooCJrlFjW4/ET3e/TcC9eQukTdtI7/+06PcJGYQ4yzmM4Tak6BGGJdBVz3f+3DSAS0+JgrcpE36mOSSeVjgk8EMgjhI+d+5b2pXikEReUGp+HrFUCI0cx3PyhY3B+bTeVt6JcXKPSfJAjes6A4hXLUSikqmah3Mm8+lbxukjZzJ097G3kwiIpSj4cxGq1tY+HMYQZjL4WyrO8W11IktUGr9IEG4M1bL4AOKtaGHdGSZCKZDZ3vuqJWZElIh8040lkf79hZrdcGCqzTvsJSC0AwtnPtL3oG6steYve2mqF633ifOPO9aLmcAnOFg5c7qX0iqa+hgsHP4AesZetQ/DXR4znqjf2yV/8XHn1EN089sF4bjkxigMKwuG4iRCPJwZXvhvQNhF999UJqyRTF34r/X0CVXpLSlU9wenpbds9vSdsE1IDogHAhJrHcESN9rMeR8sZIsidIHgaF+ikeDLSjFSluvuWrVpKtSJcNkWc5JDV9C9M0JvXHjO6D7F91S3fy+StgA05NJxdcBJU X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(396003)(39860400002)(376002)(136003)(230922051799003)(451199024)(64100799003)(1800799012)(186009)(82310400011)(36840700001)(40470700004)(46966006)(316002)(6666004)(7696005)(83380400001)(7636003)(1076003)(107886003)(2616005)(5660300002)(8676002)(426003)(6636002)(336012)(54906003)(8936002)(26005)(70206006)(70586007)(356005)(110136005)(86362001)(478600001)(36860700001)(82740400003)(4326008)(47076005)(41300700001)(36756003)(40460700003)(40480700001)(2906002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 17:03:27.6569 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ba234ce5-118d-47ec-2f93-08dc21b560e4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099D4.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6124 Let firmware knows upon leaving PRE_COPY back to RUNNING as of some error in the target/migration cancellation. This will let firmware cleaning its internal resources that were turned on upon PRE_COPY. The flow is based on the device specification in this area. Signed-off-by: Yishai Hadas --- drivers/vfio/pci/mlx5/cmd.c | 14 +++++++++---- drivers/vfio/pci/mlx5/cmd.h | 4 +++- drivers/vfio/pci/mlx5/main.c | 39 +++++++++++++++++++++++++++++------- 3 files changed, 45 insertions(+), 12 deletions(-) diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c index 65135c614b5e..320751dac377 100644 --- a/drivers/vfio/pci/mlx5/cmd.c +++ b/drivers/vfio/pci/mlx5/cmd.c @@ -108,8 +108,9 @@ int mlx5vf_cmd_query_vhca_migration_state(struct mlx5vf_pci_core_device *mvdev, ret = wait_for_completion_interruptible(&mvdev->saving_migf->save_comp); if (ret) return ret; - if (mvdev->saving_migf->state == - MLX5_MIGF_STATE_PRE_COPY_ERROR) { + /* Upon cleanup, ignore previous pre_copy error state */ + if (mvdev->saving_migf->state == MLX5_MIGF_STATE_PRE_COPY_ERROR && + !(query_flags & MLX5VF_QUERY_CLEANUP)) { /* * In case we had a PRE_COPY error, only query full * image for final image @@ -200,7 +201,7 @@ void mlx5vf_cmd_close_migratable(struct mlx5vf_pci_core_device *mvdev) /* Must be done outside the lock to let it progress */ set_tracker_error(mvdev); mutex_lock(&mvdev->state_mutex); - mlx5vf_disable_fds(mvdev); + mlx5vf_disable_fds(mvdev, NULL); _mlx5vf_free_page_tracker_resources(mvdev); mlx5vf_state_mutex_unlock(mvdev); } @@ -639,6 +640,7 @@ int mlx5vf_cmd_save_vhca_state(struct mlx5vf_pci_core_device *mvdev, u32 in[MLX5_ST_SZ_DW(save_vhca_state_in)] = {}; struct mlx5_vhca_data_buffer *header_buf = NULL; struct mlx5vf_async_data *async_data; + bool pre_copy_cleanup = false; int err; lockdep_assert_held(&mvdev->state_mutex); @@ -649,6 +651,10 @@ int mlx5vf_cmd_save_vhca_state(struct mlx5vf_pci_core_device *mvdev, if (err) return err; + if ((migf->state == MLX5_MIGF_STATE_PRE_COPY || + migf->state == MLX5_MIGF_STATE_PRE_COPY_ERROR) && !track && !inc) + pre_copy_cleanup = true; + if (migf->state == MLX5_MIGF_STATE_PRE_COPY_ERROR) /* * In case we had a PRE_COPY error, SAVE is triggered only for @@ -667,7 +673,7 @@ int mlx5vf_cmd_save_vhca_state(struct mlx5vf_pci_core_device *mvdev, async_data = &migf->async_data; async_data->buf = buf; - async_data->stop_copy_chunk = !track; + async_data->stop_copy_chunk = (!track && !pre_copy_cleanup); async_data->out = kvzalloc(out_size, GFP_KERNEL); if (!async_data->out) { err = -ENOMEM; diff --git a/drivers/vfio/pci/mlx5/cmd.h b/drivers/vfio/pci/mlx5/cmd.h index 09d0ed6e2345..116ed9326421 100644 --- a/drivers/vfio/pci/mlx5/cmd.h +++ b/drivers/vfio/pci/mlx5/cmd.h @@ -197,6 +197,7 @@ struct mlx5vf_pci_core_device { enum { MLX5VF_QUERY_INC = (1UL << 0), MLX5VF_QUERY_FINAL = (1UL << 1), + MLX5VF_QUERY_CLEANUP = (1UL << 2), }; int mlx5vf_cmd_suspend_vhca(struct mlx5vf_pci_core_device *mvdev, u16 op_mod); @@ -232,7 +233,8 @@ int mlx5vf_add_migration_pages(struct mlx5_vhca_data_buffer *buf, struct page *mlx5vf_get_migration_page(struct mlx5_vhca_data_buffer *buf, unsigned long offset); void mlx5vf_state_mutex_unlock(struct mlx5vf_pci_core_device *mvdev); -void mlx5vf_disable_fds(struct mlx5vf_pci_core_device *mvdev); +void mlx5vf_disable_fds(struct mlx5vf_pci_core_device *mvdev, + enum mlx5_vf_migf_state *last_save_state); void mlx5vf_mig_file_cleanup_cb(struct work_struct *_work); void mlx5vf_mig_file_set_save_work(struct mlx5_vf_migration_file *migf, u8 chunk_num, size_t next_required_umem_size); diff --git a/drivers/vfio/pci/mlx5/main.c b/drivers/vfio/pci/mlx5/main.c index fe09a8c8af95..3982fcf60cf2 100644 --- a/drivers/vfio/pci/mlx5/main.c +++ b/drivers/vfio/pci/mlx5/main.c @@ -1146,7 +1146,8 @@ mlx5vf_pci_resume_device_data(struct mlx5vf_pci_core_device *mvdev) return ERR_PTR(ret); } -void mlx5vf_disable_fds(struct mlx5vf_pci_core_device *mvdev) +void mlx5vf_disable_fds(struct mlx5vf_pci_core_device *mvdev, + enum mlx5_vf_migf_state *last_save_state) { if (mvdev->resuming_migf) { mlx5vf_disable_fd(mvdev->resuming_migf); @@ -1157,6 +1158,8 @@ void mlx5vf_disable_fds(struct mlx5vf_pci_core_device *mvdev) if (mvdev->saving_migf) { mlx5_cmd_cleanup_async_ctx(&mvdev->saving_migf->async_ctx); cancel_work_sync(&mvdev->saving_migf->async_data.work); + if (last_save_state) + *last_save_state = mvdev->saving_migf->state; mlx5vf_disable_fd(mvdev->saving_migf); wake_up_interruptible(&mvdev->saving_migf->poll_wait); mlx5fv_cmd_clean_migf_resources(mvdev->saving_migf); @@ -1217,12 +1220,34 @@ mlx5vf_pci_step_device_state_locked(struct mlx5vf_pci_core_device *mvdev, return migf->filp; } - if ((cur == VFIO_DEVICE_STATE_STOP_COPY && new == VFIO_DEVICE_STATE_STOP) || - (cur == VFIO_DEVICE_STATE_PRE_COPY && new == VFIO_DEVICE_STATE_RUNNING) || + if (cur == VFIO_DEVICE_STATE_STOP_COPY && new == VFIO_DEVICE_STATE_STOP) { + mlx5vf_disable_fds(mvdev, NULL); + return NULL; + } + + if ((cur == VFIO_DEVICE_STATE_PRE_COPY && new == VFIO_DEVICE_STATE_RUNNING) || (cur == VFIO_DEVICE_STATE_PRE_COPY_P2P && new == VFIO_DEVICE_STATE_RUNNING_P2P)) { - mlx5vf_disable_fds(mvdev); - return NULL; + struct mlx5_vf_migration_file *migf = mvdev->saving_migf; + struct mlx5_vhca_data_buffer *buf; + enum mlx5_vf_migf_state state; + size_t size; + + ret = mlx5vf_cmd_query_vhca_migration_state(mvdev, &size, NULL, + MLX5VF_QUERY_INC | MLX5VF_QUERY_CLEANUP); + if (ret) + return ERR_PTR(ret); + buf = mlx5vf_get_data_buffer(migf, size, DMA_FROM_DEVICE); + if (IS_ERR(buf)) + return ERR_CAST(buf); + /* pre_copy cleanup */ + ret = mlx5vf_cmd_save_vhca_state(mvdev, migf, buf, false, false); + if (ret) { + mlx5vf_put_data_buffer(buf); + return ERR_PTR(ret); + } + mlx5vf_disable_fds(mvdev, &state); + return (state != MLX5_MIGF_STATE_ERROR) ? NULL : ERR_PTR(-EIO); } if (cur == VFIO_DEVICE_STATE_STOP && new == VFIO_DEVICE_STATE_RESUMING) { @@ -1244,7 +1269,7 @@ mlx5vf_pci_step_device_state_locked(struct mlx5vf_pci_core_device *mvdev, if (ret) return ERR_PTR(ret); } - mlx5vf_disable_fds(mvdev); + mlx5vf_disable_fds(mvdev, NULL); return NULL; } @@ -1289,7 +1314,7 @@ void mlx5vf_state_mutex_unlock(struct mlx5vf_pci_core_device *mvdev) mvdev->deferred_reset = false; spin_unlock(&mvdev->reset_lock); mvdev->mig_state = VFIO_DEVICE_STATE_RUNNING; - mlx5vf_disable_fds(mvdev); + mlx5vf_disable_fds(mvdev, NULL); goto again; } mutex_unlock(&mvdev->state_mutex);