From patchwork Tue Nov 12 08:37:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 13871909 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2064.outbound.protection.outlook.com [40.107.94.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F59320C009 for ; Tue, 12 Nov 2024 08:38:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.64 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400724; cv=fail; b=YFDeBJYyjR6gz4KQ5kcCS7a7UkhGngMl0EUPDTUeHSyOEbEVk6FFz4Mk8YH9P0xG7voAeFNthgvUgBnLdJ9evQIVCcq8Qtr27fczyLcWv4lkGfLDoST0MFfGBGCWRFz14cCGcYlk7RdJItYewnsouvXF4iFf0UKDwUt/0pyq8M8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400724; c=relaxed/simple; bh=pe47dxh73UgEBY28lJctp6TIcjx0lc6PM+qY4LVDV6I=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hk6j+bUOg5ps3MbWJcgfu8jniRh0vx2DGTGda1w8p1kSyz1CHXRZ7rxdcfgfNAlmGAld/hxxXgdwuc6U5OyZGHKRX8wcYLnaL0iU56ARUu06OmfPipfBeJnv3268iaPQYnC2sTfzTxiZJMJXLuBt5Zqu+wARjmoKximyvXdRH8I= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=sxqOZWoL; arc=fail smtp.client-ip=40.107.94.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="sxqOZWoL" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aUX0v5smseYIAmNVfDsB/dJbflSTELgpEim84w+8TipXBUvdL7k7J0eoGzNjf67M02B9HLrdcHkKqpcUeSN6dEyzgHg26mQkIqfMTDFnHSN3PuW/Ypc5amoKjNSPMwKw4vpI/oxHJ/LSm2fj0vtfT58KvYpRGs71rlCCsGYB7Z11y1/meS6Y+w139MpDRjUvtZ73Iq/LdZhfKiYb4tqpuyzHgE5F84NvxPQc0Rqkn5T8EYc8c/7v1AYwVeI9ZHaN2Y3sajRmB7CXCldZPI+PT3A/PZWxbPBVTySM/uG875lXKAeBTjmtIDZeG/wBVDUMH+vbxIz6GDsJiYzc1xLc7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6UH2a9AejqPu8XRAVzr+yVLeBfNdCRpTJ5IZVd8H4IU=; b=cLhIgw9Z+FETEqlOJphhTY/6jLn2auMpxIXX8Abzyny/fszJybpLEOXBDwbjda8MhVTi671GYoEmTe7HxAT/GyPel8iKKYxIgeTGiXS7IWMjrK/ntyRFx/aI0odbj8mNqyxEayCh/7UnDiQUc/zXL9QbXsvwQhgI7+C+GGHmrdbwvJeXOIdi9oLUk2twHwBOqeLc4cN0WUgytTwxRZn5Ue22blKuh8jvJmMuHUlZSCQQuyrRdcLISnJ61CLOwN0F8DS6ed/kmXalj7xss+Sm1OkUk1+FL2TmLY0YjStjniGqAtWT0xB0QdlQmdtYXV9hecIooTBXWawOR3Uu89Znlw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6UH2a9AejqPu8XRAVzr+yVLeBfNdCRpTJ5IZVd8H4IU=; b=sxqOZWoLqXdXvdCvfQXo84X+Cs+rP17pJ8CRASRVPiRWsZixLfUJ79UFrBF25jPFrMH/xtnHaNJcrl8aMwsn4um/p48oQaMawSHl0Dws9j2v7DCzu9sX1XD2SM1NOLq+ouVCA3qdwQAiWp9y/kX1ctDMMsIJ7qi4q0s5aIPI4BpKppp8EgvlHQZFUec5lBuxptIuZpE7W8v2ZHt7/k/Q2djy8Hdq/+VzLik4MVsRDosUlXnKotAy923hLPO0cgXF69aPLJ7Ru5VTcHskJ+602i0GF/NYTWLKaYCviSrwdAiaGZ8tMw+dWA6+wMBSGzPziVMR/wNv8D2835O/bSfj2w== Received: from DS7PR03CA0249.namprd03.prod.outlook.com (2603:10b6:5:3b3::14) by SJ0PR12MB6829.namprd12.prod.outlook.com (2603:10b6:a03:47b::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8137.27; Tue, 12 Nov 2024 08:38:35 +0000 Received: from DS1PEPF0001708E.namprd03.prod.outlook.com (2603:10b6:5:3b3:cafe::51) by DS7PR03CA0249.outlook.office365.com (2603:10b6:5:3b3::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.16 via Frontend Transport; Tue, 12 Nov 2024 08:38:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS1PEPF0001708E.mail.protection.outlook.com (10.167.17.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.14 via Frontend Transport; Tue, 12 Nov 2024 08:38:34 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:21 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:20 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 12 Nov 2024 00:38:17 -0800 From: Yishai Hadas To: , , , CC: , , , , , , , , Subject: [PATCH V3 vfio 1/7] virtio_pci: Introduce device parts access commands Date: Tue, 12 Nov 2024 10:37:23 +0200 Message-ID: <20241112083729.145005-2-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20241112083729.145005-1-yishaih@nvidia.com> References: <20241112083729.145005-1-yishaih@nvidia.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF0001708E:EE_|SJ0PR12MB6829:EE_ X-MS-Office365-Filtering-Correlation-Id: 691ea1d8-dceb-4ce3-4428-08dd02f5656f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: GT4njXUKuonumrAKmXkwBusqUMt4lbd7flTQvLJWFCjSf4MknGMYUvK/fXCB/fHZmgboQkr7xHDAQJZrb+KSu4PiZ/f2kfUQcTvbjup4afjSVBvNwn/VFn2Sw5GDbScvf/QgYcbwdPqR9b9quMyOA9rMT8+H6nTcdj8E1UD6MeFgeFFEjAmtA0iHlCKuPqg93rNdVTz3AX2sOM+MFlkOf9eG0My+K7vaJX0lxjLw5/RTQ0z0++aFLCa3dtlt5rmV1k6QXm6av/Q872GLivq3YtAjmVjwT49XFZ9GxVJnAp/QU8hEVM+QsBfxIWF+Mu07XN50rIEnkUs1jF6vMQ+kj9fnr9k36eDk8U2BClUVywamCm2Eb6ihHtqb8JT7WyNaGU2Ef4Op+K5oDa+BqRCelNGkHJGOnLyDVJ/9iifNJo3z0yq2/aK1JYoq1eu8d6U+0N+zh6lGlZDc87ZZYxVT3bxOPNpFIXTNpWfNroIIIwtyAZrMSfXzesOYGN9xZd/buUlYugIPvCTNiINtOjWyI0zsmtN9Ggzk02QBhmnjbSalNnXHn4WI+cNz4XwHB/H4pW0sDcxoOHdEUgwIdG82rm8XgQX3zf7TLCvVvWmvJvfotxQbsxiEQ11QrlRgNG8FcI1jJbc+c44/mZ74ov3YaVgUix8CK7isUi5NX16CPfF4Gy8XkHFWKl5quCRQVGneqVsfrjTSy6PJ+DxR0TzUZRSOFa5dfykEkjsbQE11Qv6umDPbqYEJckOl9OiadM0Eq8KCN6418UYGsCMseUy8haemGggbbAjLyQLRDK6NMoJ1z7zRVPxFgshnxiimQKhw8i23r0gq6qVzvOgDHFuAhAyHGnv6GbczuePytKZck/b3i6p2+kAaGjkiEZUHEriN5TWwE0EibYTZNFGWLEuTLJyr8LnfG+yjKCKim9RXR3+K3dsZFaqUJqUnEWp4fB9jOOrbxVy2L8veMy3Bp2dVg4MnVlqIObGKJx2HYBZ9+6CCRwhnMm6ZwYmrYybssfW0tNOAJQoBdVCdc0DYlTilDgrL7X+zfdvEar2r7RseQP4HMZDCJKAtyCZyH8C9pdojU5FcSkxl8jL7h45KAdiOEzY5m58MsQEqXToRHVqCqpkpe+0nGjnF7MEgcPYaG1TJRP4ZCIt3YQpluwR1Yf+aRZZJfFJprjXKrmK2CIUarATCJNaB3X20SCruRm12z8ilJycztyc0szllhSLZa4DjT0SOoRFFtVXOTvJpvFv4KTrqhFQtz2aBDLCfQgf8nrlv758c7haUfgXc0LukZ2+meoFevnb/aYZBIAGRDn0UcwziTFmhc7/Hql552csfgIjoNixIAFn4KIvuTBr56l6eDj2j8xlF3i5Pvf21/X6XDCOUUio49hfLGASvWyXRLNBjc8yZMGcKlFCw3I9JxNHugQ== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2024 08:38:34.6949 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 691ea1d8-dceb-4ce3-4428-08dd02f5656f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF0001708E.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6829 Introduce device parts access commands via the admin queue. These commands and their structure adhere to the Virtio 1.4 specification. Acked-by: Michael S. Tsirkin Signed-off-by: Yishai Hadas --- include/uapi/linux/virtio_pci.h | 131 ++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h index a8208492e822..1beb317df1b9 100644 --- a/include/uapi/linux/virtio_pci.h +++ b/include/uapi/linux/virtio_pci.h @@ -40,6 +40,7 @@ #define _LINUX_VIRTIO_PCI_H #include +#include #ifndef VIRTIO_PCI_NO_LEGACY @@ -240,6 +241,17 @@ struct virtio_pci_cfg_cap { #define VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ 0x5 #define VIRTIO_ADMIN_CMD_LEGACY_NOTIFY_INFO 0x6 +/* Device parts access commands. */ +#define VIRTIO_ADMIN_CMD_CAP_ID_LIST_QUERY 0x7 +#define VIRTIO_ADMIN_CMD_DEVICE_CAP_GET 0x8 +#define VIRTIO_ADMIN_CMD_DRIVER_CAP_SET 0x9 +#define VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE 0xa +#define VIRTIO_ADMIN_CMD_RESOURCE_OBJ_DESTROY 0xd +#define VIRTIO_ADMIN_CMD_DEV_PARTS_METADATA_GET 0xe +#define VIRTIO_ADMIN_CMD_DEV_PARTS_GET 0xf +#define VIRTIO_ADMIN_CMD_DEV_PARTS_SET 0x10 +#define VIRTIO_ADMIN_CMD_DEV_MODE_SET 0x11 + struct virtio_admin_cmd_hdr { __le16 opcode; /* @@ -286,4 +298,123 @@ struct virtio_admin_cmd_notify_info_result { struct virtio_admin_cmd_notify_info_data entries[VIRTIO_ADMIN_CMD_MAX_NOTIFY_INFO]; }; +#define VIRTIO_DEV_PARTS_CAP 0x0000 + +struct virtio_dev_parts_cap { + __u8 get_parts_resource_objects_limit; + __u8 set_parts_resource_objects_limit; +}; + +#define MAX_CAP_ID __KERNEL_DIV_ROUND_UP(VIRTIO_DEV_PARTS_CAP + 1, 64) + +struct virtio_admin_cmd_query_cap_id_result { + __le64 supported_caps[MAX_CAP_ID]; +}; + +struct virtio_admin_cmd_cap_get_data { + __le16 id; + __u8 reserved[6]; +}; + +struct virtio_admin_cmd_cap_set_data { + __le16 id; + __u8 reserved[6]; + __u8 cap_specific_data[]; +}; + +struct virtio_admin_cmd_resource_obj_cmd_hdr { + __le16 type; + __u8 reserved[2]; + __le32 id; /* Indicates unique resource object id per resource object type */ +}; + +struct virtio_admin_cmd_resource_obj_create_data { + struct virtio_admin_cmd_resource_obj_cmd_hdr hdr; + __le64 flags; + __u8 resource_obj_specific_data[]; +}; + +#define VIRTIO_RESOURCE_OBJ_DEV_PARTS 0 + +#define VIRTIO_RESOURCE_OBJ_DEV_PARTS_TYPE_GET 0 +#define VIRTIO_RESOURCE_OBJ_DEV_PARTS_TYPE_SET 1 + +struct virtio_resource_obj_dev_parts { + __u8 type; + __u8 reserved[7]; +}; + +#define VIRTIO_ADMIN_CMD_DEV_PARTS_METADATA_TYPE_SIZE 0 +#define VIRTIO_ADMIN_CMD_DEV_PARTS_METADATA_TYPE_COUNT 1 +#define VIRTIO_ADMIN_CMD_DEV_PARTS_METADATA_TYPE_LIST 2 + +struct virtio_admin_cmd_dev_parts_metadata_data { + struct virtio_admin_cmd_resource_obj_cmd_hdr hdr; + __u8 type; + __u8 reserved[7]; +}; + +#define VIRTIO_DEV_PART_F_OPTIONAL 0 + +struct virtio_dev_part_hdr { + __le16 part_type; + __u8 flags; + __u8 reserved; + union { + struct { + __le32 offset; + __le32 reserved; + } pci_common_cfg; + struct { + __le16 index; + __u8 reserved[6]; + } vq_index; + } selector; + __le32 length; +}; + +struct virtio_dev_part { + struct virtio_dev_part_hdr hdr; + __u8 value[]; +}; + +struct virtio_admin_cmd_dev_parts_metadata_result { + union { + struct { + __le32 size; + __le32 reserved; + } parts_size; + struct { + __le32 count; + __le32 reserved; + } hdr_list_count; + struct { + __le32 count; + __le32 reserved; + struct virtio_dev_part_hdr hdrs[]; + } hdr_list; + }; +}; + +#define VIRTIO_ADMIN_CMD_DEV_PARTS_GET_TYPE_SELECTED 0 +#define VIRTIO_ADMIN_CMD_DEV_PARTS_GET_TYPE_ALL 1 + +struct virtio_admin_cmd_dev_parts_get_data { + struct virtio_admin_cmd_resource_obj_cmd_hdr hdr; + __u8 type; + __u8 reserved[7]; + struct virtio_dev_part_hdr hdr_list[]; +}; + +struct virtio_admin_cmd_dev_parts_set_data { + struct virtio_admin_cmd_resource_obj_cmd_hdr hdr; + struct virtio_dev_part parts[]; +}; + +#define VIRTIO_ADMIN_CMD_DEV_MODE_F_STOPPED 0 + +struct virtio_admin_cmd_dev_mode_set_data { + __u8 flags; +}; + #endif From patchwork Tue Nov 12 08:37:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 13871910 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2082.outbound.protection.outlook.com [40.107.237.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4906020CCDD for ; Tue, 12 Nov 2024 08:38:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.82 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400727; cv=fail; b=PhtRvDbN0EHCMI6Yn2dQU3gTkU32WNx7uK9vKU9HFHmD2aUcQv3G/fn7PbpBvsPNssxu2m37XA0M8TNPouqVOwMIO4nTbbl4P3lrUZquLCmDtUsE21+jwVvFqu2dkOY57TPL2beSATZ/mNZ7PCOkR+P9esarMTD3NQUBOeLN1HE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400727; c=relaxed/simple; bh=E60yR9n+W84OfwsNUp7Ia+f9UsEC9YhP31DXQSszqEU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=OFBv8UVR2YM0Qb4QcqHtLz40t+TYe5ET0yjpuYLs80oXHtn29OK8D5iv+9THN9tP+jhh598iz6XrK17HehOV6/rDgfxF8ilqkYIsjgTW+gLolJ7GrWNlAfru8uWPK1jQ+Ysu4QEuG/iSocbxSDryvpCEd5rmJw/IYXP5ybgnsXY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Aov5opm9; arc=fail smtp.client-ip=40.107.237.82 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Aov5opm9" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=VIHyaGTaXL4mUtnVhfXv/oZ5NNK7XJ/ffeU+BYi4K7Nvt5BvGWsXHDZ9Pi7WmuPAZzSM3WLLKxExECD9/p6+Vh7+SYdg9mCvjXBa8x7+eB/r1sGN7CPBNfo4Xv00deXdpRHA4f+tmschdS0wi0+v79QBgpJMaX4Hf4I22JDemltwmruTdFvNHBsFCRMWqvaWeJFhkFhong1mp+lLIrD/thYJuWJJ81y4b5ESMSYdIXE60JKypJi3136QVCqomE5jQu6yd7ypsn9Ova4ptEXDCpZKnbC2raTFH70ZqEvPHtZfRdcY03QJwnYfw1T83ZryUUaKOF+dde1aHpBYnl122A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zTMLxdx8o35fTyUbpLODHFggmpddmQFwmFYqLhdwaYQ=; b=fCvVdJHvj5fvNKXl7eTqCS+WQ+F91/l1UZ22QKcPM5AQbfvAlygNElypKqaiQNk8gZIWZlniw1/QUDW1ZmPqBbhUiuMEOBY8fQiNQm2rd3A8lhGMq9rhgu1N2h9LNqSYUpwWZk4IIRjPGUYp8z16Diq1iAWW8S5ngAeUx03NwlWrHi8gl2eY4X8Mt2IjcGjeZ9hJjnGdPvzL4Ajb/Qa2t9Ng+TX+S/84p3fpOtus+RCkF9KY1idDhZ86EMsgNf6HxffGqZ8dthT/udLmnsISQod5xhBm6OhTNNUEdPQgvWEpg4tRtIaVJFEHvFvEAtNkjV+02FF58+mGQ5HG3RMWgQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zTMLxdx8o35fTyUbpLODHFggmpddmQFwmFYqLhdwaYQ=; b=Aov5opm9Z2CQPGXXNyH0AmK7CgCiiBpjW2sH6fV6RmLrbu5ki1MJA5ZxrfOxv0mGcYFBzbYC2Zy47kQ6xPoC14yf7KAmnv+DzmzczJyvPTOj3HvItUj3CxaTlkG/PTDysYLfhqIFGZsigRSvXu+AaWzDY2GNetEnkq5GzVp3oF7F0RtqqiuVZCxrUGFezGbR0B6syEhGT7TrDG09SRaVabHq/jRXAYGxJdp7rhQLurcKxnxeBIpTNqCrzPezfE3GZJNKzKkVzgMfWj+V0kictHfX24abF7WU4RKuxJx/3OI1Cy8J+PguWDUfmqJWnpLPpMdxKJSuUJOvNYa6PPuIug== Received: from BN9PR03CA0774.namprd03.prod.outlook.com (2603:10b6:408:13a::29) by DM4PR12MB7574.namprd12.prod.outlook.com (2603:10b6:8:10e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8137.28; Tue, 12 Nov 2024 08:38:42 +0000 Received: from MN1PEPF0000F0E3.namprd04.prod.outlook.com (2603:10b6:408:13a:cafe::33) by BN9PR03CA0774.outlook.office365.com (2603:10b6:408:13a::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.16 via Frontend Transport; Tue, 12 Nov 2024 08:38:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by MN1PEPF0000F0E3.mail.protection.outlook.com (10.167.242.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.14 via Frontend Transport; Tue, 12 Nov 2024 08:38:41 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:25 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:24 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 12 Nov 2024 00:38:21 -0800 From: Yishai Hadas To: , , , CC: , , , , , , , , Subject: [PATCH V3 vfio 2/7] virtio: Extend the admin command to include the result size Date: Tue, 12 Nov 2024 10:37:24 +0200 Message-ID: <20241112083729.145005-3-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20241112083729.145005-1-yishaih@nvidia.com> References: <20241112083729.145005-1-yishaih@nvidia.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E3:EE_|DM4PR12MB7574:EE_ X-MS-Office365-Filtering-Correlation-Id: 84a64bb1-507a-4f57-778a-08dd02f5697a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: GjSqTqGNCVT2a2H2LZrjy2WG+rIGMyDFV2g6zH3aDlb8ee4jhfz+/nYJrl3sHErKT5qiLhHlo0dlu8+XYcJL+JUPwa9wVvCxmA8QH8ICHwgHOotWJkSh6g1QsX3NyyZ9P2bEaO0tVYOKx9sRXUnjr8cmRuEonLkWyp/IiFpzBvRbLpzY+ZmXck/sjoEJYSiH3xC7VSm7DN5sDSuQGuTSHTyU6P5/NrDCQu2VhgjE+Yk3cLy3jwX5X9zToHpKBeqrUHjzYnwE9G2NwGqKnJbbfOIi8nXZ62LXuHIPvQh+8oXX7Ac2FWZB5tfH7G0pdjEytZJwmxg4Rw6j4nWY90eFceAuOlA5zAt3XcPVslXSw8iDtdLEEKLQr1CzaiJvmb870fDWjgjLL0cKNCJVUDbHMH6EkIm5tff/vQyXaenG5AYyxh9Z2NqdMlNFgZ4LVkVjUU+B8RMkg276Y4lChdG5WBsWkLVkT4CS5Sl7Ae2I298k9JOwxTATos/tWURdPnp2ZecN0uPoevYR7JQ3ua0iOacDEwSfb2Qf+RFBOABb17/6qLGGoUs1sxmMxGCmcfVmskGPasYQrYWklIgq8v0VmJ194AS8elwTgI4pKGrAyVmf9DSgd8gAdB7qXrBGy4D5aZZleo5ElCunKqPeEWV43t3B6GtjLyqERZabP5A235YTFbrTbezr0yZ158nHwksipHhJtjigaCs14fWJ4Zf//rvK+PKzgnlruwzv5aX6RLssDNiR9cO368ZwnVpCaitGrvmgsYC5TzotsASPIVDOM/3ZeI2fhpue4JemydwUcBY3QT0yubAuU5Puiq1afKRyLa9h8jiUxT5Yf9bqOM92zqFTHsp/s9BL88Yx+9vJaHGEN7ZfoN8KquXUXaXlqRzgHvOJjl1NGLCmFDSzJ94v5/bLNXjHOfyTsjhwjsSdbcGH777YLM13+LLTS/b4y7QKxers6bZOxx18jqZN+U13hTJasFFgZR6jmy6UDL1mzrQWNI7FmgflOZO1qt4vaKZTHGGlVwRoA/T994aDtz17OCRU8wSshL/Vddmr7HWZtChaKMSz8NIc9M5POk1FPFN1mOoxCJM2Zsf+1iNS/4ZxY/uq5c0EFWwumspTP7kjCTGLIXb02p49+qpeHisK7gnVku0jdy6G86jrWhv1CgTspIM20/pI8t3lxKOXwCqDbSpMubsUhN95NkoVSmAx/Ztp6cccIPIfHZQ0ZIwBFdXhFUI5tBErVlcamjT8hY/7sXw89n4NTXNARMIyVz4Ie/EwOoubclzWJ9PDxoBcYRv2c89rSoq81LDeOtlAyWj5PfzSWAL3PDDpXlS1WBlpHACrKatBuRbfqBKIbpXZY2jw2oAJ8fPA9/te+FfoA3Stp6ZjntDVQadZR7etn/OPkTYjdogsJsBQ9aUeh3i5L78OaA== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2024 08:38:41.4167 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 84a64bb1-507a-4f57-778a-08dd02f5697a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E3.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7574 Extend the admin command by incorporating a result size field. This allows higher layers to determine the actual result size from the backend when this information is not included in the result_sg. The additional information introduced here will be used in subsequent patches of this series. Acked-by: Michael S. Tsirkin Signed-off-by: Yishai Hadas --- drivers/virtio/virtio_pci_modern.c | 4 +++- include/linux/virtio.h | 1 + 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c index 9193c30d640a..487d04610ecb 100644 --- a/drivers/virtio/virtio_pci_modern.c +++ b/drivers/virtio/virtio_pci_modern.c @@ -64,8 +64,10 @@ void vp_modern_avq_done(struct virtqueue *vq) spin_lock_irqsave(&admin_vq->lock, flags); do { virtqueue_disable_cb(vq); - while ((cmd = virtqueue_get_buf(vq, &len))) + while ((cmd = virtqueue_get_buf(vq, &len))) { + cmd->result_sg_size = len; complete(&cmd->completion); + } } while (!virtqueue_enable_cb(vq)); spin_unlock_irqrestore(&admin_vq->lock, flags); } diff --git a/include/linux/virtio.h b/include/linux/virtio.h index 306137a15d07..b5f7a611715a 100644 --- a/include/linux/virtio.h +++ b/include/linux/virtio.h @@ -111,6 +111,7 @@ struct virtio_admin_cmd { struct scatterlist *data_sg; struct scatterlist *result_sg; struct completion completion; + u32 result_sg_size; int ret; }; From patchwork Tue Nov 12 08:37:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 13871911 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2065.outbound.protection.outlook.com [40.107.244.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 555BB20C03B for ; Tue, 12 Nov 2024 08:38:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.65 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400729; cv=fail; b=ZcoP9Hqw8aZSTUQAFXcgLJAnY5mHdOQbnp9DHUGe1vCVf219n7w7ZBVlbelyOLBVVqCESUmBPaIpPvz+su179jNlFucyR6EZuiCd2bI10QCxE6aXDPMHbci8q4zlQqwyMYUyYzWDP6BhF5pfu0jDC65OItnWo3/DTcv0J6/x0I0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400729; c=relaxed/simple; bh=0UvZ+SfETqUM8evMg8qrOMY1/59yOJAFQwWInV2JgdE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=eaTFE2r1aB+69Q7hIGkTF21QzVcdi511bJ6xG/Lrcj/OjZOZYszeyPGEUyEnLJClB9PedKnR27jeN5Ch8Z9hDwZz/fFlMcVERwjh7X8Auv5ntYLrf4HQp2yxFOgOj3CbMOUYM8lhJQeX+MkE5BC+JdGeHjP9Jc9tzlyLDnLQfUM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=DK3pjwr/; arc=fail smtp.client-ip=40.107.244.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="DK3pjwr/" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Sd0Sg05KxYN4OVYGiSm8cJX+WoIMt//DPrQlmRGpM+ZXrmCHEIjClRGlp2+ihMPoUcP82oAkSdCUVpDsOadDzkvpCvB7fA62FcSa5abvxCUeVFhjmF7QNWMfDPLd/rkBR9RA8HeA1d2002qsblzneOW1MBZXwLLXoGTJGAYdeCA7gdrPD+zBt9buoZhJQvwbRX+fMcq5+YTrW6qYOrMLDu8jPgkvz3FTlwo7ZHFJNMkm+BNuIELpZ0luZub2fOsU5ZN5Rqi+LTEXG0epiJriYgYjdFwnTwOgtTdlho43ZjUTIu8c1Wmy2HVSSlbWDwFUxNN/WtQn2sCFzXyiTdHzVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FmtJiXU1fa9KsBEp20lkRQC5T0UJXnWKwHLeQei9VfU=; b=pinkCz/kJTrwvi/DcHZLteEiHvLg78MBDH4jGTLlnC2Jv4Lpdxt4JRrklFWmbHl3T1IsrD3e0nIos0PubrmeXyZZekplQII7/LrE0ZMb/QzcDDBvYhago7BLCM8YZCiO266D1I7Xb2T5IFUihbuJpXwsNqQEncv4+5vZzGZpmRZ9FvLPXTAY339EWDJ/T7Hq2xD9lYQiCluEJad5QHjJ/DYXliXhALZ8hD9Z6Lfs+XqJXNsl+uGKvmFayqTKVpML+N0KJqwKgcBqrEo4MlGV6hzN7P3wV2NDnucQ3vANB2FJupxTSlEYwRdI8kbU7fYkLRh8HOeWVv6LWLIJuPl54Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FmtJiXU1fa9KsBEp20lkRQC5T0UJXnWKwHLeQei9VfU=; b=DK3pjwr/e9fAyVqri7kBxxSG/qsSFXuVqxMHCMHKPrV3vEl/tY/9+b1Gd5TDi9ndHnlnMl4wjfM9jcW0q58LEie5z+AEIzHY+vTzcOh7catUSHTHAbv7Podvd/HFvXwTQzS0AP4UTT8ttEoZJ2JkPfHceQ212W+DXW502xy+f4UdcS5SNm62YCV8jkUXrNr5DeSkfudKLwV7kjJspSxnWmDrN5tK8wY0OsaqbRWcjKvSvYQ+qSx4jdrqVWzAhMIXXPEQDmRmZ/drwFa9qwefz3/V8+k0RFSRBZ4TDvz5/ZFGVpKSdsEJ4Zt24lZ4plqVF+wu6nle/Byfl6qThdT2Bw== Received: from BN9PR03CA0106.namprd03.prod.outlook.com (2603:10b6:408:fd::21) by PH7PR12MB5807.namprd12.prod.outlook.com (2603:10b6:510:1d3::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.16; Tue, 12 Nov 2024 08:38:44 +0000 Received: from MN1PEPF0000F0E5.namprd04.prod.outlook.com (2603:10b6:408:fd:cafe::d5) by BN9PR03CA0106.outlook.office365.com (2603:10b6:408:fd::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8137.28 via Frontend Transport; Tue, 12 Nov 2024 08:38:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by MN1PEPF0000F0E5.mail.protection.outlook.com (10.167.242.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.14 via Frontend Transport; Tue, 12 Nov 2024 08:38:43 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:28 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:28 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 12 Nov 2024 00:38:25 -0800 From: Yishai Hadas To: , , , CC: , , , , , , , , Subject: [PATCH V3 vfio 3/7] virtio: Manage device and driver capabilities via the admin commands Date: Tue, 12 Nov 2024 10:37:25 +0200 Message-ID: <20241112083729.145005-4-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20241112083729.145005-1-yishaih@nvidia.com> References: <20241112083729.145005-1-yishaih@nvidia.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E5:EE_|PH7PR12MB5807:EE_ X-MS-Office365-Filtering-Correlation-Id: f2c7ced4-dc9d-4d55-df41-08dd02f56af7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|82310400026|376014; X-Microsoft-Antispam-Message-Info: gFxTB/Sm+xB6J0dS7AQaJZ9y50TfcOzIyisqZFIFA8GYEBnJq85QH4lTFtHs8CE0kXMAKyG9N723bll7JrtrjtFKNyl1uvRiWgQou9NTO/Zh2Y96kS7hfnZUkvL2idX3ifeJjYWF7zN/SuLWVkM/ACMIkT2yPYbG1fmfc8Z6OEZhiTszUZ8BOf+p+QtDy6VqNtEq2GPXG76HLxGcCjxwMgMgF4xtfjM5yQz0aESQ9xjIDDaNCbsokg/kGYcxBafPnb/pOB1UDQMGl5XKMZPe4WqSsRGURs4m/znNIvSmcpAI1aEswHz+ZTN/WTPWu2ZY75desh1shu/p9aVjDKyMRJIyQ1Bn6gavvBHGb9a8JD6F4Gwex4dm6ZR9KgtT0466a8k5B93+PKM6XqKSPq5k1r5jj71b6LoV1zN1R2FD8iRLNbY/Fsun7IHbPXu5B9ZSygdbtki4eBUWCpp5rokeSpplCHqpd9Bs39/erBD0zbl0Lvqm9L3HjZkV32m1AAcPVdEbeYq+D1iLpcxQNMZOjFrmF3h14J8UYLvZ/ZOIPxNkxtuOmZjbRa8Kp27z6TxVk+sq2mIg71lt2DRYT7Y8q7aDEPO/5bsk+19gIIVS8P9P1CU7N4iIK3BFBuph0zoqV2M3AWESFARngVL+0i/hdKerBO7fHEgNAb3P8cgYTiccvYACHYhTHp7WKcF1usLLQBIMo/NrmXDBNi7Vxd8OQ/jxtSF4azzBirDeR7s0eCGCukDtqU9sS1ETZGNZX3OsHbevn52VLhKY5dYJP5E66wbSxpzdQGeeDSqvQ+2JOjtXklOmQpKuym0h+XqvSl7P+fVn+ZHCp1nO+bvYdiksZFp5M5Edr6pMfGGFc6HPPAY9N3+5L1yILKSOXVGiEJK7EGI+UTSS5GI7VEp3rFVyWJ2Q/F05vnVRTUrsiQy0cZT1nrWnl9DWoersnDFUiXh1ue9Sr5BHCv2dRPDt5aPnQG6vTODkHR+CZ8lsnVOBnIoX2f0JdL8iLJvxtNic57qiGnjxLJz+acMahLKQFgtrz7ILHW/FC/9Mve4C5AUX7Yoi83sLPPaWSF4Uv+VpEeghRyqfCRqEhrmUEnJQ74PK55hlbUs12a2Q2Jsn5AmjaBKAGVMqf/ySUD9dnZvnYFcg4HwIEiiJug/J+7f+jBO5NyfJ6dpFzgU2nKb/SLYp3x6gNFEWzuQOHoJ2SRZH+K22Ocku69CZHR7qMngo91Y7ZeOtwdhYhvjwjzVlk25zI5DYh2B8QewPASFGrMNRUHxz831fYcAv0O+Y+7MHlvsXsnB7KKPgr5glTtlTTTrwpcjNSR7p/v/wP2Ane1058fALLH2M+l+6KkGkegGZ2VQWdwEs4c62c29i/sViDpd5BxVflKErlP0fEI/y3vZcFxOqmYjHquUXxdIyMWraTr/ayg== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(82310400026)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2024 08:38:43.9163 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f2c7ced4-dc9d-4d55-df41-08dd02f56af7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5807 Manage device and driver capabilities via the admin commands. The device exposes its supported features and resource object limits via an administrative command called VIRTIO_ADMIN_CMD_CAP_ID_LIST_QUERY, using the 'self group type.' Each capability is identified by a unique ID, and the driver communicates the functionality and resource limits it plans to utilize. The capability VIRTIO_DEV_PARTS_CAP specifically represents the device's parts resource object limit. Manage the device's parts resource object ID using a common IDA for both get and set operations. Acked-by: Michael S. Tsirkin Signed-off-by: Yishai Hadas --- drivers/virtio/virtio_pci_common.h | 13 +++- drivers/virtio/virtio_pci_modern.c | 105 +++++++++++++++++++++++++++++ 2 files changed, 116 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h index 1d9c49947f52..04b1d17663b3 100644 --- a/drivers/virtio/virtio_pci_common.h +++ b/drivers/virtio/virtio_pci_common.h @@ -48,6 +48,9 @@ struct virtio_pci_admin_vq { /* Protects virtqueue access. */ spinlock_t lock; u64 supported_cmds; + u64 supported_caps; + u8 max_dev_parts_objects; + struct ida dev_parts_ida; /* Name of the admin queue: avq.$vq_index. */ char name[10]; u16 vq_index; @@ -167,15 +170,21 @@ struct virtio_device *virtio_pci_vf_get_pf_dev(struct pci_dev *pdev); BIT_ULL(VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ) | \ BIT_ULL(VIRTIO_ADMIN_CMD_LEGACY_NOTIFY_INFO)) +#define VIRTIO_DEV_PARTS_ADMIN_CMD_BITMAP \ + (BIT_ULL(VIRTIO_ADMIN_CMD_CAP_ID_LIST_QUERY) | \ + BIT_ULL(VIRTIO_ADMIN_CMD_DRIVER_CAP_SET) | \ + BIT_ULL(VIRTIO_ADMIN_CMD_DEVICE_CAP_GET)) + /* Unlike modern drivers which support hardware virtio devices, legacy drivers * assume software-based devices: e.g. they don't use proper memory barriers * on ARM, use big endian on PPC, etc. X86 drivers are mostly ok though, more * or less by chance. For now, only support legacy IO on X86. */ #ifdef CONFIG_VIRTIO_PCI_ADMIN_LEGACY -#define VIRTIO_ADMIN_CMD_BITMAP VIRTIO_LEGACY_ADMIN_CMD_BITMAP +#define VIRTIO_ADMIN_CMD_BITMAP (VIRTIO_LEGACY_ADMIN_CMD_BITMAP | \ + VIRTIO_DEV_PARTS_ADMIN_CMD_BITMAP) #else -#define VIRTIO_ADMIN_CMD_BITMAP 0 +#define VIRTIO_ADMIN_CMD_BITMAP VIRTIO_DEV_PARTS_ADMIN_CMD_BITMAP #endif void vp_modern_avq_done(struct virtqueue *vq); diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c index 487d04610ecb..8ddac2829bc8 100644 --- a/drivers/virtio/virtio_pci_modern.c +++ b/drivers/virtio/virtio_pci_modern.c @@ -230,12 +230,117 @@ static void virtio_pci_admin_cmd_list_init(struct virtio_device *virtio_dev) kfree(data); } +static void +virtio_pci_admin_cmd_dev_parts_objects_enable(struct virtio_device *virtio_dev) +{ + struct virtio_pci_device *vp_dev = to_vp_device(virtio_dev); + struct virtio_admin_cmd_cap_get_data *get_data; + struct virtio_admin_cmd_cap_set_data *set_data; + struct virtio_dev_parts_cap *result; + struct virtio_admin_cmd cmd = {}; + struct scatterlist result_sg; + struct scatterlist data_sg; + u8 resource_objects_limit; + u16 set_data_size; + int ret; + + get_data = kzalloc(sizeof(*get_data), GFP_KERNEL); + if (!get_data) + return; + + result = kzalloc(sizeof(*result), GFP_KERNEL); + if (!result) + goto end; + + get_data->id = cpu_to_le16(VIRTIO_DEV_PARTS_CAP); + sg_init_one(&data_sg, get_data, sizeof(*get_data)); + sg_init_one(&result_sg, result, sizeof(*result)); + cmd.opcode = cpu_to_le16(VIRTIO_ADMIN_CMD_DEVICE_CAP_GET); + cmd.group_type = cpu_to_le16(VIRTIO_ADMIN_GROUP_TYPE_SRIOV); + cmd.data_sg = &data_sg; + cmd.result_sg = &result_sg; + ret = vp_modern_admin_cmd_exec(virtio_dev, &cmd); + if (ret) + goto err_get; + + set_data_size = sizeof(*set_data) + sizeof(*result); + set_data = kzalloc(set_data_size, GFP_KERNEL); + if (!set_data) + goto err_get; + + set_data->id = cpu_to_le16(VIRTIO_DEV_PARTS_CAP); + + /* Set the limit to the minimum value between the GET and SET values + * supported by the device. Since the obj_id for VIRTIO_DEV_PARTS_CAP + * is a globally unique value per PF, there is no possibility of + * overlap between GET and SET operations. + */ + resource_objects_limit = min(result->get_parts_resource_objects_limit, + result->set_parts_resource_objects_limit); + result->get_parts_resource_objects_limit = resource_objects_limit; + result->set_parts_resource_objects_limit = resource_objects_limit; + memcpy(set_data->cap_specific_data, result, sizeof(*result)); + sg_init_one(&data_sg, set_data, set_data_size); + cmd.data_sg = &data_sg; + cmd.result_sg = NULL; + cmd.opcode = cpu_to_le16(VIRTIO_ADMIN_CMD_DRIVER_CAP_SET); + ret = vp_modern_admin_cmd_exec(virtio_dev, &cmd); + if (ret) + goto err_set; + + /* Allocate IDR to manage the dev caps objects */ + ida_init(&vp_dev->admin_vq.dev_parts_ida); + vp_dev->admin_vq.max_dev_parts_objects = resource_objects_limit; + +err_set: + kfree(set_data); +err_get: + kfree(result); +end: + kfree(get_data); +} + +static void virtio_pci_admin_cmd_cap_init(struct virtio_device *virtio_dev) +{ + struct virtio_pci_device *vp_dev = to_vp_device(virtio_dev); + struct virtio_admin_cmd_query_cap_id_result *data; + struct virtio_admin_cmd cmd = {}; + struct scatterlist result_sg; + int ret; + + data = kzalloc(sizeof(*data), GFP_KERNEL); + if (!data) + return; + + sg_init_one(&result_sg, data, sizeof(*data)); + cmd.opcode = cpu_to_le16(VIRTIO_ADMIN_CMD_CAP_ID_LIST_QUERY); + cmd.group_type = cpu_to_le16(VIRTIO_ADMIN_GROUP_TYPE_SRIOV); + cmd.result_sg = &result_sg; + + ret = vp_modern_admin_cmd_exec(virtio_dev, &cmd); + if (ret) + goto end; + + /* Max number of caps fits into a single u64 */ + BUILD_BUG_ON(sizeof(data->supported_caps) > sizeof(u64)); + + vp_dev->admin_vq.supported_caps = le64_to_cpu(data->supported_caps[0]); + + if (!(vp_dev->admin_vq.supported_caps & (1 << VIRTIO_DEV_PARTS_CAP))) + goto end; + + virtio_pci_admin_cmd_dev_parts_objects_enable(virtio_dev); +end: + kfree(data); +} + static void vp_modern_avq_activate(struct virtio_device *vdev) { if (!virtio_has_feature(vdev, VIRTIO_F_ADMIN_VQ)) return; virtio_pci_admin_cmd_list_init(vdev); + virtio_pci_admin_cmd_cap_init(vdev); } static void vp_modern_avq_cleanup(struct virtio_device *vdev) From patchwork Tue Nov 12 08:37:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 13871912 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2067.outbound.protection.outlook.com [40.107.223.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40E9920DD40 for ; Tue, 12 Nov 2024 08:38:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.67 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400732; cv=fail; b=uzwGw3tRB1Qf/noBMauKiMefz3Va9YllMl2NqJ/b5kTcd6Qs5zbRrcrL9VOH9DCy7PJ5zmxxEEhB7rFu9cl21Ezl1cmX3DnMWyyhFxDf6wn+DLe7OKVNBXJDKeUcUa58JewQKKnVZVXyYZ9zzFqQ4/Tx7mGT+jFHCwkGfsuvrDQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400732; c=relaxed/simple; bh=QYGMKAMl8mTsCS6jHu129ITEZw1xUMTxBUAweAW3Gqs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hFjVee5TenDWiKPOU1yxAdm2TNGhoAfxRHY5BzRpyzbKU4XFK9FpqD9wy9zjTYuexSvf+wDH5dt7KmeIwFrzXbRJs2rn9UsRT1gN0ZP9BWAY9oMN5y3cpTvvVW1cchJu5mUtTrzAvQa60+FTttDQ2znSpOVab7PWFuHOfE6Gkic= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=C9wM2+75; arc=fail smtp.client-ip=40.107.223.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="C9wM2+75" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=t0s9TzNnFt8K505mA6aBD/Rs0qFlUWODKXe5CEufq8oTCZkzrC6LzgY8cV11snGXsub1LmAoFGJ0JtLap5f78CocimKntOMEJ5Rsp6J25jr+PLag+8gm3dAvMzVQ1AUOn1DMgfKqsFrn05niIAbilvlWvnRl/+9L3GFM/ixtpLdUTP+SnDg6Mm2V3U8A3dhilEgrq0CsAdZjAZ67Avhb2ANgVPYmGIa1ooLHQEqCgenAHEtlneQoMDUYXIiwOCgf8QQzdCFYaz8ic2IgSgvNl3bcPRva4rIkgpQF0P+KCIJ9nbYJ7ED1RT17dTsNw0088INnq+eg95FnkTr3wKot5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AnmjS5Ws8pOTnu3tw1SCuin/i2X1p8HKm4b/6Fk3VPM=; b=e6OUr6eYBq9mhfKXDyGg3a/aBr4XFORu8aIaFnz1sC179fSg5JB849+cq9Mtm4Kj1nAM1FUE32IROHmFhuXYIXpIJ9Ruoqh/h2wlnU/MauXlNiRJeTnNFK+nPJaFWR+U/MGhKid6dWff9c8mEhaG4bW/IZnq/5iINJCoftxWJmBaEO5dEV8HsFOLyCiLk/Dz4DXPqywUtslD+iOTM30tMyHtru/Z4yx0oMlHe9Ddnu6A1lm73gXKJRZ1uuNIpJAvoYciJ8fjR7aYPYNojHemt5y1RRhq9tTfR8J/rZ/Np8ZXcu7WJs2h1s4pprB+5oBFceZwoLGhI4sK1SGJUx7AsQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AnmjS5Ws8pOTnu3tw1SCuin/i2X1p8HKm4b/6Fk3VPM=; b=C9wM2+75Mr0HBouy5dKZqLstll71qBTAu9p7zIL8Ar75m5HRdBBh8M+lv5Ra6ONrwmojlbvHdRy4MxafDLPUc/QFo1QZIryrKkCl9+l7eVQjL6/XH4niQD5DRiBXQRAH0SX3ui4uTkK7KIP0o4b5jP1Fqzo6J0bkPd2PN2x+AqrOJdLuBuGzS8Kf1xO4faBn6sliBx0NRl+2G0aADR5qq+jbBWWeAgY5vZOnxIr9W+2PDvLRyiAjNLcobcs7fWHNOWxFnmNElZPgcgpkz44SDh3AGakzGF3oui8LqK/GhoHeaApyxlr0vZz1bGHM6WQyVnYSkWA2RhLnvVuklGmxgQ== Received: from DM6PR06CA0060.namprd06.prod.outlook.com (2603:10b6:5:54::37) by SJ2PR12MB7963.namprd12.prod.outlook.com (2603:10b6:a03:4c1::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8137.30; Tue, 12 Nov 2024 08:38:47 +0000 Received: from DS1PEPF00017095.namprd03.prod.outlook.com (2603:10b6:5:54:cafe::a1) by DM6PR06CA0060.outlook.office365.com (2603:10b6:5:54::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8137.28 via Frontend Transport; Tue, 12 Nov 2024 08:38:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS1PEPF00017095.mail.protection.outlook.com (10.167.17.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.14 via Frontend Transport; Tue, 12 Nov 2024 08:38:46 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:32 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:32 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 12 Nov 2024 00:38:28 -0800 From: Yishai Hadas To: , , , CC: , , , , , , , , Subject: [PATCH V3 vfio 4/7] virtio-pci: Introduce APIs to execute device parts admin commands Date: Tue, 12 Nov 2024 10:37:26 +0200 Message-ID: <20241112083729.145005-5-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20241112083729.145005-1-yishaih@nvidia.com> References: <20241112083729.145005-1-yishaih@nvidia.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF00017095:EE_|SJ2PR12MB7963:EE_ X-MS-Office365-Filtering-Correlation-Id: bd287d4a-dd3e-499c-6040-08dd02f56c73 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|376014|82310400026|1800799024; X-Microsoft-Antispam-Message-Info: sITf8lErUIQqSgt0UKe+FBt6AlEM67Jha8h8BOwc1bk21rExQNbwel+L0O0EvK3p92EIOFVG0JtnpSqc28l2smdPiAOzkhDJSkkCHMpDFT3QUm5DYnw+WdZdkwrW/fpph6M9kVU31SerZ3tCqPuHdQyfLusJBbCzyglsiZo0v4NR2hsxcWgxFpFFQf2urVODNUIIfCtHB7W/I75YdsTfLviZKybqybKruUwbGouDGZFAQUv+6W0bt+Y2pmO/eBba5dKRDS/kCDh/UC0bhIv7GOGazDORJ9gqp+LejMXLdq6PkSgIzBkT1jDpWQujLGPZqh7bwMy+DuOrHTkxOSD9QnL2EXM+4zRBarFg7VhqsoI6JUM27LFE7izyF4ppEb+bzMXnASrFM+n6tP4yFb8WwasQJC6DvoCm51fThs7JDdzdzixVaRRKZETGKrNItsuC7R95YEv+eNl2UTDq4CNFNJcPsweqBr7hUV2y3ZvW/XBcGRqmwlvJ5/gRPYWbTNOoMmNHXxZHTMbjZelDqEzy55zGRrkOUOBr56VqgneGbdQwlqFq14on3TvuAvcNytvi1A1M/XCeKLDwmF0OUxpr4AKBDwZvUDBDUX+QkNX5paV2GQ9MxNhK6cA4tKqOjZmfpe/+1P74qY3fCZM3DvUE2HRd9bnRV80qu7JFWDuXV/SR8Ewnw9hEFym7JmxqHYXsAIio+/7vrL9OGmKUQBhDhA+z0skLiNRJWvEv3ruaHA35N77YQoeX1SOySAakyW2mmmGnlomid4kJUqClaVBQyiLGCpGkby9GusAHcdM5cpTkhNr566Jh+XLAnwq3hlFyYF5ULC5WAeL7MFsqt411rA/rxZI9WtaAtJcg0iw239/fFu5jctSmmr9XX79FvIG5tJ1hmccbrwnscsasIvp1eAaIyp5nlHH2j7Lbiw54L6HMs4hF4ESTNArKJQg/Pb405gMrLB5S4uZL5+hTw4xgHLNP0+E7Vo8tkx30G0jcHUBowT+yKIqHJu+RYFzZ/OfvmT+ZUDYhlugDeC8OGQbu08tVSvzv0Gn99zGQKbVmVonTFvS8dol5mwaVv3X+tN1usdq0Hnm1Hqg4v5sVEwk8FlhdtyOnXZBhlgI3OxbwBeyhXR1NJFUU7EKJW0FFdo5CIsfVAu36RiKDMevZiWw8w29ziS+VJryxwowTwFjRLHdCcstHY2lAstaQBksaMu7n9LiYZ3qrevJSW/yVVwiIw5PTyD1lONQS11GzQS2WhOlb1PumO9SjXoOCJdv3GSd2+jsOvJDh0Uzpa9AMMtklX6/P0v7tN87qrC39GoNcpv9Dumwi5ziX6w9jWfkxTK1VbvFDmcvFl67UhEKN0vmgTB2sqtHlehjvi4ClM+WdEv3ksZxE/XaYv9LmfgUmR5zH9Xcf6QfDtmW0ZVchVgVTxQ== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(376014)(82310400026)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2024 08:38:46.4659 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bd287d4a-dd3e-499c-6040-08dd02f56c73 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF00017095.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7963 Introduce APIs to handle the execution of device parts admin commands. These APIs cover functionalities such as mode setting, object creation and destruction, and operations like parts get/set and metadata retrieval. These APIs will be utilized in upcoming patches within this series. Acked-by: Michael S. Tsirkin Signed-off-by: Yishai Hadas --- drivers/virtio/virtio_pci_common.h | 8 +- drivers/virtio/virtio_pci_modern.c | 348 +++++++++++++++++++++++++++++ include/linux/virtio_pci_admin.h | 11 + 3 files changed, 366 insertions(+), 1 deletion(-) diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h index 04b1d17663b3..0d00740cca07 100644 --- a/drivers/virtio/virtio_pci_common.h +++ b/drivers/virtio/virtio_pci_common.h @@ -173,7 +173,13 @@ struct virtio_device *virtio_pci_vf_get_pf_dev(struct pci_dev *pdev); #define VIRTIO_DEV_PARTS_ADMIN_CMD_BITMAP \ (BIT_ULL(VIRTIO_ADMIN_CMD_CAP_ID_LIST_QUERY) | \ BIT_ULL(VIRTIO_ADMIN_CMD_DRIVER_CAP_SET) | \ - BIT_ULL(VIRTIO_ADMIN_CMD_DEVICE_CAP_GET)) + BIT_ULL(VIRTIO_ADMIN_CMD_DEVICE_CAP_GET) | \ + BIT_ULL(VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE) | \ + BIT_ULL(VIRTIO_ADMIN_CMD_RESOURCE_OBJ_DESTROY) | \ + BIT_ULL(VIRTIO_ADMIN_CMD_DEV_PARTS_METADATA_GET) | \ + BIT_ULL(VIRTIO_ADMIN_CMD_DEV_PARTS_GET) | \ + BIT_ULL(VIRTIO_ADMIN_CMD_DEV_PARTS_SET) | \ + BIT_ULL(VIRTIO_ADMIN_CMD_DEV_MODE_SET)) /* Unlike modern drivers which support hardware virtio devices, legacy drivers * assume software-based devices: e.g. they don't use proper memory barriers diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c index 8ddac2829bc8..3f5aba71cfde 100644 --- a/drivers/virtio/virtio_pci_modern.c +++ b/drivers/virtio/virtio_pci_modern.c @@ -15,6 +15,7 @@ */ #include +#include #define VIRTIO_PCI_NO_LEGACY #define VIRTIO_RING_NO_LEGACY #include "virtio_pci_common.h" @@ -875,6 +876,353 @@ static bool vp_get_shm_region(struct virtio_device *vdev, return true; } +/* + * virtio_pci_admin_has_dev_parts - Checks whether the device parts + * functionality is supported + * @pdev: VF pci_dev + * + * Returns true on success. + */ +bool virtio_pci_admin_has_dev_parts(struct pci_dev *pdev) +{ + struct virtio_device *virtio_dev = virtio_pci_vf_get_pf_dev(pdev); + struct virtio_pci_device *vp_dev; + + if (!virtio_dev) + return false; + + if (!virtio_has_feature(virtio_dev, VIRTIO_F_ADMIN_VQ)) + return false; + + vp_dev = to_vp_device(virtio_dev); + + if (!((vp_dev->admin_vq.supported_cmds & VIRTIO_DEV_PARTS_ADMIN_CMD_BITMAP) == + VIRTIO_DEV_PARTS_ADMIN_CMD_BITMAP)) + return false; + + return vp_dev->admin_vq.max_dev_parts_objects; +} +EXPORT_SYMBOL_GPL(virtio_pci_admin_has_dev_parts); + +/* + * virtio_pci_admin_mode_set - Sets the mode of a member device + * @pdev: VF pci_dev + * @flags: device mode's flags + * + * Note: caller must serialize access for the given device. + * Returns 0 on success, or negative on failure. + */ +int virtio_pci_admin_mode_set(struct pci_dev *pdev, u8 flags) +{ + struct virtio_device *virtio_dev = virtio_pci_vf_get_pf_dev(pdev); + struct virtio_admin_cmd_dev_mode_set_data *data; + struct virtio_admin_cmd cmd = {}; + struct scatterlist data_sg; + int vf_id; + int ret; + + if (!virtio_dev) + return -ENODEV; + + vf_id = pci_iov_vf_id(pdev); + if (vf_id < 0) + return vf_id; + + data = kzalloc(sizeof(*data), GFP_KERNEL); + if (!data) + return -ENOMEM; + + data->flags = flags; + sg_init_one(&data_sg, data, sizeof(*data)); + cmd.opcode = cpu_to_le16(VIRTIO_ADMIN_CMD_DEV_MODE_SET); + cmd.group_type = cpu_to_le16(VIRTIO_ADMIN_GROUP_TYPE_SRIOV); + cmd.group_member_id = cpu_to_le64(vf_id + 1); + cmd.data_sg = &data_sg; + ret = vp_modern_admin_cmd_exec(virtio_dev, &cmd); + + kfree(data); + return ret; +} +EXPORT_SYMBOL_GPL(virtio_pci_admin_mode_set); + +/* + * virtio_pci_admin_obj_create - Creates an object for a given type and operation, + * following the max objects that can be created for that request. + * @pdev: VF pci_dev + * @obj_type: Object type + * @operation_type: Operation type + * @obj_id: Output unique object id + * + * Note: caller must serialize access for the given device. + * Returns 0 on success, or negative on failure. + */ +int virtio_pci_admin_obj_create(struct pci_dev *pdev, u16 obj_type, u8 operation_type, + u32 *obj_id) +{ + struct virtio_device *virtio_dev = virtio_pci_vf_get_pf_dev(pdev); + u16 data_size = sizeof(struct virtio_admin_cmd_resource_obj_create_data); + struct virtio_admin_cmd_resource_obj_create_data *obj_create_data; + struct virtio_resource_obj_dev_parts obj_dev_parts = {}; + struct virtio_pci_admin_vq *avq; + struct virtio_admin_cmd cmd = {}; + struct scatterlist data_sg; + void *data; + int id = -1; + int vf_id; + int ret; + + if (!virtio_dev) + return -ENODEV; + + vf_id = pci_iov_vf_id(pdev); + if (vf_id < 0) + return vf_id; + + if (obj_type != VIRTIO_RESOURCE_OBJ_DEV_PARTS) + return -EOPNOTSUPP; + + if (operation_type != VIRTIO_RESOURCE_OBJ_DEV_PARTS_TYPE_GET && + operation_type != VIRTIO_RESOURCE_OBJ_DEV_PARTS_TYPE_SET) + return -EINVAL; + + avq = &to_vp_device(virtio_dev)->admin_vq; + if (!avq->max_dev_parts_objects) + return -EOPNOTSUPP; + + id = ida_alloc_range(&avq->dev_parts_ida, 0, + avq->max_dev_parts_objects - 1, GFP_KERNEL); + if (id < 0) + return id; + + *obj_id = id; + data_size += sizeof(obj_dev_parts); + data = kzalloc(data_size, GFP_KERNEL); + if (!data) { + ret = -ENOMEM; + goto end; + } + + obj_create_data = data; + obj_create_data->hdr.type = cpu_to_le16(obj_type); + obj_create_data->hdr.id = cpu_to_le32(*obj_id); + obj_dev_parts.type = operation_type; + memcpy(obj_create_data->resource_obj_specific_data, &obj_dev_parts, + sizeof(obj_dev_parts)); + sg_init_one(&data_sg, data, data_size); + cmd.opcode = cpu_to_le16(VIRTIO_ADMIN_CMD_RESOURCE_OBJ_CREATE); + cmd.group_type = cpu_to_le16(VIRTIO_ADMIN_GROUP_TYPE_SRIOV); + cmd.group_member_id = cpu_to_le64(vf_id + 1); + cmd.data_sg = &data_sg; + ret = vp_modern_admin_cmd_exec(virtio_dev, &cmd); + + kfree(data); +end: + if (ret) + ida_free(&avq->dev_parts_ida, id); + + return ret; +} +EXPORT_SYMBOL_GPL(virtio_pci_admin_obj_create); + +/* + * virtio_pci_admin_obj_destroy - Destroys an object of a given type and id + * @pdev: VF pci_dev + * @obj_type: Object type + * @id: Object id + * + * Note: caller must serialize access for the given device. + * Returns 0 on success, or negative on failure. + */ +int virtio_pci_admin_obj_destroy(struct pci_dev *pdev, u16 obj_type, u32 id) +{ + struct virtio_device *virtio_dev = virtio_pci_vf_get_pf_dev(pdev); + struct virtio_admin_cmd_resource_obj_cmd_hdr *data; + struct virtio_pci_device *vp_dev; + struct virtio_admin_cmd cmd = {}; + struct scatterlist data_sg; + int vf_id; + int ret; + + if (!virtio_dev) + return -ENODEV; + + vf_id = pci_iov_vf_id(pdev); + if (vf_id < 0) + return vf_id; + + if (obj_type != VIRTIO_RESOURCE_OBJ_DEV_PARTS) + return -EINVAL; + + data = kzalloc(sizeof(*data), GFP_KERNEL); + if (!data) + return -ENOMEM; + + data->type = cpu_to_le16(obj_type); + data->id = cpu_to_le32(id); + sg_init_one(&data_sg, data, sizeof(*data)); + cmd.opcode = cpu_to_le16(VIRTIO_ADMIN_CMD_RESOURCE_OBJ_DESTROY); + cmd.group_type = cpu_to_le16(VIRTIO_ADMIN_GROUP_TYPE_SRIOV); + cmd.group_member_id = cpu_to_le64(vf_id + 1); + cmd.data_sg = &data_sg; + ret = vp_modern_admin_cmd_exec(virtio_dev, &cmd); + if (!ret) { + vp_dev = to_vp_device(virtio_dev); + ida_free(&vp_dev->admin_vq.dev_parts_ida, id); + } + + kfree(data); + return ret; +} +EXPORT_SYMBOL_GPL(virtio_pci_admin_obj_destroy); + +/* + * virtio_pci_admin_dev_parts_metadata_get - Gets the metadata of the device parts + * identified by the below attributes. + * @pdev: VF pci_dev + * @obj_type: Object type + * @id: Object id + * @metadata_type: Metadata type + * @out: Upon success holds the output for 'metadata type size' + * + * Note: caller must serialize access for the given device. + * Returns 0 on success, or negative on failure. + */ +int virtio_pci_admin_dev_parts_metadata_get(struct pci_dev *pdev, u16 obj_type, + u32 id, u8 metadata_type, u32 *out) +{ + struct virtio_device *virtio_dev = virtio_pci_vf_get_pf_dev(pdev); + struct virtio_admin_cmd_dev_parts_metadata_result *result; + struct virtio_admin_cmd_dev_parts_metadata_data *data; + struct scatterlist data_sg, result_sg; + struct virtio_admin_cmd cmd = {}; + int vf_id; + int ret; + + if (!virtio_dev) + return -ENODEV; + + if (metadata_type != VIRTIO_ADMIN_CMD_DEV_PARTS_METADATA_TYPE_SIZE) + return -EOPNOTSUPP; + + vf_id = pci_iov_vf_id(pdev); + if (vf_id < 0) + return vf_id; + + data = kzalloc(sizeof(*data), GFP_KERNEL); + if (!data) + return -ENOMEM; + + result = kzalloc(sizeof(*result), GFP_KERNEL); + if (!result) { + ret = -ENOMEM; + goto end; + } + + data->hdr.type = cpu_to_le16(obj_type); + data->hdr.id = cpu_to_le32(id); + data->type = metadata_type; + sg_init_one(&data_sg, data, sizeof(*data)); + sg_init_one(&result_sg, result, sizeof(*result)); + cmd.opcode = cpu_to_le16(VIRTIO_ADMIN_CMD_DEV_PARTS_METADATA_GET); + cmd.group_type = cpu_to_le16(VIRTIO_ADMIN_GROUP_TYPE_SRIOV); + cmd.group_member_id = cpu_to_le64(vf_id + 1); + cmd.data_sg = &data_sg; + cmd.result_sg = &result_sg; + ret = vp_modern_admin_cmd_exec(virtio_dev, &cmd); + if (!ret) + *out = le32_to_cpu(result->parts_size.size); + + kfree(result); +end: + kfree(data); + return ret; +} +EXPORT_SYMBOL_GPL(virtio_pci_admin_dev_parts_metadata_get); + +/* + * virtio_pci_admin_dev_parts_get - Gets the device parts identified by the below attributes. + * @pdev: VF pci_dev + * @obj_type: Object type + * @id: Object id + * @get_type: Get type + * @res_sg: Upon success holds the output result data + * @res_size: Upon success holds the output result size + * + * Note: caller must serialize access for the given device. + * Returns 0 on success, or negative on failure. + */ +int virtio_pci_admin_dev_parts_get(struct pci_dev *pdev, u16 obj_type, u32 id, + u8 get_type, struct scatterlist *res_sg, + u32 *res_size) +{ + struct virtio_device *virtio_dev = virtio_pci_vf_get_pf_dev(pdev); + struct virtio_admin_cmd_dev_parts_get_data *data; + struct scatterlist data_sg; + struct virtio_admin_cmd cmd = {}; + int vf_id; + int ret; + + if (!virtio_dev) + return -ENODEV; + + if (get_type != VIRTIO_ADMIN_CMD_DEV_PARTS_GET_TYPE_ALL) + return -EOPNOTSUPP; + + vf_id = pci_iov_vf_id(pdev); + if (vf_id < 0) + return vf_id; + + data = kzalloc(sizeof(*data), GFP_KERNEL); + if (!data) + return -ENOMEM; + + data->hdr.type = cpu_to_le16(obj_type); + data->hdr.id = cpu_to_le32(id); + data->type = get_type; + sg_init_one(&data_sg, data, sizeof(*data)); + cmd.opcode = cpu_to_le16(VIRTIO_ADMIN_CMD_DEV_PARTS_GET); + cmd.group_type = cpu_to_le16(VIRTIO_ADMIN_GROUP_TYPE_SRIOV); + cmd.group_member_id = cpu_to_le64(vf_id + 1); + cmd.data_sg = &data_sg; + cmd.result_sg = res_sg; + ret = vp_modern_admin_cmd_exec(virtio_dev, &cmd); + if (!ret) + *res_size = cmd.result_sg_size; + + kfree(data); + return ret; +} +EXPORT_SYMBOL_GPL(virtio_pci_admin_dev_parts_get); + +/* + * virtio_pci_admin_dev_parts_set - Sets the device parts identified by the below attributes. + * @pdev: VF pci_dev + * @data_sg: The device parts data, its layout follows struct virtio_admin_cmd_dev_parts_set_data + * + * Note: caller must serialize access for the given device. + * Returns 0 on success, or negative on failure. + */ +int virtio_pci_admin_dev_parts_set(struct pci_dev *pdev, struct scatterlist *data_sg) +{ + struct virtio_device *virtio_dev = virtio_pci_vf_get_pf_dev(pdev); + struct virtio_admin_cmd cmd = {}; + int vf_id; + + if (!virtio_dev) + return -ENODEV; + + vf_id = pci_iov_vf_id(pdev); + if (vf_id < 0) + return vf_id; + + cmd.opcode = cpu_to_le16(VIRTIO_ADMIN_CMD_DEV_PARTS_SET); + cmd.group_type = cpu_to_le16(VIRTIO_ADMIN_GROUP_TYPE_SRIOV); + cmd.group_member_id = cpu_to_le64(vf_id + 1); + cmd.data_sg = data_sg; + return vp_modern_admin_cmd_exec(virtio_dev, &cmd); +} +EXPORT_SYMBOL_GPL(virtio_pci_admin_dev_parts_set); + static const struct virtio_config_ops virtio_pci_config_nodev_ops = { .get = NULL, .set = NULL, diff --git a/include/linux/virtio_pci_admin.h b/include/linux/virtio_pci_admin.h index f4a100a0fe2e..dffc92c17ad2 100644 --- a/include/linux/virtio_pci_admin.h +++ b/include/linux/virtio_pci_admin.h @@ -20,4 +20,15 @@ int virtio_pci_admin_legacy_io_notify_info(struct pci_dev *pdev, u64 *bar_offset); #endif +bool virtio_pci_admin_has_dev_parts(struct pci_dev *pdev); +int virtio_pci_admin_mode_set(struct pci_dev *pdev, u8 mode); +int virtio_pci_admin_obj_create(struct pci_dev *pdev, u16 obj_type, u8 operation_type, + u32 *obj_id); +int virtio_pci_admin_obj_destroy(struct pci_dev *pdev, u16 obj_type, u32 id); +int virtio_pci_admin_dev_parts_metadata_get(struct pci_dev *pdev, u16 obj_type, + u32 id, u8 metadata_type, u32 *out); +int virtio_pci_admin_dev_parts_get(struct pci_dev *pdev, u16 obj_type, u32 id, + u8 get_type, struct scatterlist *res_sg, u32 *res_size); +int virtio_pci_admin_dev_parts_set(struct pci_dev *pdev, struct scatterlist *data_sg); + #endif /* _LINUX_VIRTIO_PCI_ADMIN_H */ From patchwork Tue Nov 12 08:37:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 13871913 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2042.outbound.protection.outlook.com [40.107.237.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0EDC220B7FC for ; Tue, 12 Nov 2024 08:38:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.42 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400740; cv=fail; b=sfpQHgRRmIschDbQUGyHnERIwuuapYCWElLb8M2DaPN/5nvcSJl5SbG8V6qRIEGHM+SGxiAe9r2ZxoGP47zLhd0hpeJ/diq7Y2ZhYlqbKtubh80PrKUpDmuWHHOJrWtyrXCwZizWRTWpRNaelGkEumZIGoCxro7T6dWPteDaxOM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400740; c=relaxed/simple; bh=cekZlI8AnETTnU/lbaDgI6a9W2I1H6c+IzcTkv+MYto=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=mIpaht/4ROGQAiTSFFduxTN8qv+WsMk3/VGB+fUuG+Y4AzK3MsLzg+FBmRMeNM4aex4oUYqxHV/KMrEEg4QiL/Lr98AZOqbSscELdYukmN76sWQJlWkRNbPIy/RR4wfD7HfeUc0Y7QYI/+kLhaylvm3Vvck/+N1liYQRxYcGYBs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=jJZKJkMO; arc=fail smtp.client-ip=40.107.237.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="jJZKJkMO" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LeZvWLpJZoXFITB2iSS5DAJRcJUSOYyBBJLPenYZ6ysTsL+m0ySbtsa09M0uEthhtpEVaC7wgMiW7w7SZSMTEsQkYmhbH17le56wL0QUKlIaug4xzDghqSNcUpzyYWhhsakQe5/rzjJvpebp52jXNMuvfIXw4e9WmadCa8s3f0QljV4/V+/tZkpE4u0wnqtdPDpfmvyKblHclaFmg28X5to2GCcTovyaYJFQ4fbzEPHVF96o9VdrQyfgoU3nQefgM/5eFypy8mnsxeEE1nWwPZ998oGq341S0gcpGbyWFi9Im5rqyE6qH0tnrIgFzkf4PnWkSmDUkX8tBR0LjJIbOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6w6pQyCB9/4/EDNabHO1rfoHnVpx/taWYIbnIvQNUOE=; b=gxR0chpqb+45fZNrJZxa0gR15Pejg6Q667RjXoCEJXSGZWfFzJ3gkZXV9nK71dyOW/FYG73Mud+RaTFTN1m2LqnNqKUpvPMw5E1polZR/pbgrtNbhbJJnK95zGFeyeUrjMXDOxVvW3sKgHJbiBSmewTTJb2ccto71EeDRcsDV/NtdNqkWOzi9KTr2gdLNyw2rctUF4r5d3wjlXEQGt7MWzlEF9/ySsInSCe7pCZI7W8tlbrpoazHuOp+xYJqP6+KlY/ocWUoBJnMdW8nHMggs+w/XSguMhSbN5HeDx8E77afURw7eRbKSef8EzWpkkwwTqdeVkiOUDmYrR69cbvucg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6w6pQyCB9/4/EDNabHO1rfoHnVpx/taWYIbnIvQNUOE=; b=jJZKJkMOuQ4Cx/18jgzGJQWsXDkyeYxkzz1IiOX4K2w9pv/A5ulMxSHd2SsRAjcFXRGA80mTAXfW6JSuKEFEAKiKYaIXHy8/WNa07MLhsEEeuuTFFIAhWKMLbYruNTYK8cDPTcOA+PCeIT7NUX9TIrNazvurf9RqBlDzjHsluv5DlfmZkx0HnP+Rxqlg7XVcN/9uJbb3IxWANrc22FyJrfX0S2JaDYDmPVq1nbQpLAkNqgXiPgeul0tGEOshD1GsD9o0S00oTglwrGIZGwN6t4n6Vyi8GyhLwLdKj7N8CDowlz46gDKiXdVVEjLmHDxF66nbLV8dE3qcQFOvKC/hvA== Received: from DS7PR05CA0060.namprd05.prod.outlook.com (2603:10b6:8:2f::13) by BY5PR12MB4306.namprd12.prod.outlook.com (2603:10b6:a03:206::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8137.28; Tue, 12 Nov 2024 08:38:51 +0000 Received: from DS1PEPF00017093.namprd03.prod.outlook.com (2603:10b6:8:2f:cafe::a3) by DS7PR05CA0060.outlook.office365.com (2603:10b6:8:2f::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.15 via Frontend Transport; Tue, 12 Nov 2024 08:38:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS1PEPF00017093.mail.protection.outlook.com (10.167.17.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.14 via Frontend Transport; Tue, 12 Nov 2024 08:38:50 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:36 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:35 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 12 Nov 2024 00:38:32 -0800 From: Yishai Hadas To: , , , CC: , , , , , , , , Subject: [PATCH V3 vfio 5/7] vfio/virtio: Add support for the basic live migration functionality Date: Tue, 12 Nov 2024 10:37:27 +0200 Message-ID: <20241112083729.145005-6-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20241112083729.145005-1-yishaih@nvidia.com> References: <20241112083729.145005-1-yishaih@nvidia.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF00017093:EE_|BY5PR12MB4306:EE_ X-MS-Office365-Filtering-Correlation-Id: 2d0f6e0b-9a27-48bb-05cc-08dd02f56f1b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|36860700013|376014; X-Microsoft-Antispam-Message-Info: tUswExgxVck+/wB+pdrlyKcKRMa4ezWS8rD1YKdDoVBtDl7YsiOzo8ifSYRYi22gxOQ18j9Qb2V2JSB9Gbl3hNhvd/RoSZ0d67+milrR8NUCBmyvDPgAQv2wmiXcSsT6+Ne+NYrdsGjM+zUYYGJrcFzGQ9EyRW6kFDVxpnDYmUS+kQVBQKvifAI7GM7M+wvbHTzE/UbM/M2dUTTmtMG8CcBsTLqyfeBQz4w+CVt4GzX7xGgZRnjepGGKAbKnv46NAov97H1N1KRwS49SvrV2sCgbii854mf9zkAfV7YX6Y1z1z+EP68H51baOt43iD2lr4TBaXhWCCdkanaAxiHYNUP0i47Y2Hrg+5hGxHXdvCEZbJ0ueQykOuOQwINOlIS9S3JmMJ9tKcT70Eyg5bKkT+3feei52UF5hffe0IKEBQh2rD3olXtaAhQ3AK/uw/Kx1JTQz7leid3OTLP1JjVxKeYZBAp50tGEfyQZqAtSDi0I+AeUG8OXvgPHRHYEegesGkBup6qRXFCinaucvunIAX4GxtEmosDN05cNisLxeX1iHTkFR5Ljo5PP7Cg1Z3KGSTcutI6rB2kXiQuKBEEaDh879Ov1pqSstztxHs3V+AJj0L0TvWYKgf0RyeO/ZpXfq+kDFeLX6M8qaa6b9/Q1q0LuFcIaTPHECpN4FzxQKO6nwRSzU+EpfeTdAK+PGI1Xi8q6LHms0J3ardFZYi8IPJOnbsXpTrvRi3JSYkgaXTUiiHsS1UO0hWOdE4ry2Wqub6DNG5He/BtoQn7mzVJuXYFPgnX4yTWA5AdAokzzLcMRFpurlmJgTwYZA45Jp469MAeiq1TCGCnBYqAW9ol1UTBqiYrV0pIYkM6gdY06cVXM4zA2rito0wap5kjbohulUIjNFQQpP4bBf6dWCsTZ1LXacQOnPisA2t/gUXtVXtczUr7FNnx6ApXibKTZMakscF706x+URG/YExn41T9N6d59/B861vjVKZkxD62dC8uPXl7S5TeWC/iYfluN3HDzpNaQFJ7+YxgclhvZZ6qyfrVR+/728czkVSACkTnd72UQljGHx+1eF9nwXNEA7SpfIA4ybYKMaKXXfodIC7GDqs/LxoyT84HpXJNinW9FOwCZPBhd0x1k4zTCbH56JXzijGGktOv5IgS7swM3aEodqHXY0kBHjbCb7UiNgKh8FKaSf6zWHAv562g+xjehYkQfKXtcLfJV6FkGl1+fMjGtb8+erELBTJDpULTalXKOXCfOFNjzl2cs90C/RFqKtKOczaVCMtn9y0HPnGpRYYIJ958B79RqGMfIIsoJ5X9AvJgBmjMZEqpKrOsByn7TInaWfQzJ18VlhTaLBpzFWeL71DU3/hhjS0NV3pP6zYpeOulTO9H1JdKW6x/8tFIvLA+aNZuPP9UF3OuMHMW4pQ67Lw== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(36860700013)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2024 08:38:50.9200 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2d0f6e0b-9a27-48bb-05cc-08dd02f56f1b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF00017093.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4306 Add support for basic live migration functionality in VFIO over virtio-net devices, aligned with the virtio device specification 1.4. This includes the following VFIO features: VFIO_MIGRATION_STOP_COPY, VFIO_MIGRATION_P2P. The implementation registers with the VFIO subsystem using vfio_pci_core and then incorporates the virtio-specific logic for the migration process. The migration follows the definitions in uapi/vfio.h and leverages the virtio VF-to-PF admin queue command channel for execution device parts related commands. Additional Notes: ----------------- The kernel protocol between the source and target devices contains a header with metadata, including record size, tag, and flags. The record size allows the target to recognize and read a complete image from the source before passing the device part data. This adheres to the virtio device specification, which mandates that partial device parts cannot be supplied. The tag and flags serve as placeholders for future extensions of the kernel protocol between the source and target, ensuring backward and forward compatibility. Both the source and target comply with the virtio device specification by using a device part object with a unique ID as part of the migration process. Since this resource is limited to a maximum of 255, its lifecycle is confined to periods with an active live migration flow. According to the virtio specification, a device has only two modes: RUNNING and STOPPED. As a result, certain VFIO transitions (i.e., RUNNING_P2P->STOP, STOP->RUNNING_P2P) are treated as no-ops. When transitioning to RUNNING_P2P, the device state is set to STOP, and it will remain STOPPED until the transition out of RUNNING_P2P->RUNNING, at which point it returns to RUNNING. During transition to STOP, the virtio device only stops initiating outgoing requests(e.g. DMA, MSIx, etc.) but still must accept incoming operations. Signed-off-by: Yishai Hadas --- drivers/vfio/pci/virtio/Makefile | 2 +- drivers/vfio/pci/virtio/common.h | 104 +++ drivers/vfio/pci/virtio/main.c | 82 ++- drivers/vfio/pci/virtio/migrate.c | 1120 +++++++++++++++++++++++++++++ 4 files changed, 1283 insertions(+), 25 deletions(-) create mode 100644 drivers/vfio/pci/virtio/common.h create mode 100644 drivers/vfio/pci/virtio/migrate.c diff --git a/drivers/vfio/pci/virtio/Makefile b/drivers/vfio/pci/virtio/Makefile index 7171105baf33..bf0ccde6a91a 100644 --- a/drivers/vfio/pci/virtio/Makefile +++ b/drivers/vfio/pci/virtio/Makefile @@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_VIRTIO_VFIO_PCI) += virtio-vfio-pci.o -virtio-vfio-pci-y := main.o +virtio-vfio-pci-y := main.o migrate.o diff --git a/drivers/vfio/pci/virtio/common.h b/drivers/vfio/pci/virtio/common.h new file mode 100644 index 000000000000..3bdfb3ea1174 --- /dev/null +++ b/drivers/vfio/pci/virtio/common.h @@ -0,0 +1,104 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef VIRTIO_VFIO_COMMON_H +#define VIRTIO_VFIO_COMMON_H + +#include +#include +#include +#include + +enum virtiovf_migf_state { + VIRTIOVF_MIGF_STATE_ERROR = 1, +}; + +enum virtiovf_load_state { + VIRTIOVF_LOAD_STATE_READ_HEADER, + VIRTIOVF_LOAD_STATE_PREP_HEADER_DATA, + VIRTIOVF_LOAD_STATE_READ_HEADER_DATA, + VIRTIOVF_LOAD_STATE_PREP_CHUNK, + VIRTIOVF_LOAD_STATE_READ_CHUNK, + VIRTIOVF_LOAD_STATE_LOAD_CHUNK, +}; + +struct virtiovf_data_buffer { + struct sg_append_table table; + loff_t start_pos; + u64 length; + u64 allocated_length; + struct list_head buf_elm; + u8 include_header_object:1; + struct virtiovf_migration_file *migf; + /* Optimize virtiovf_get_migration_page() for sequential access */ + struct scatterlist *last_offset_sg; + unsigned int sg_last_entry; + unsigned long last_offset; +}; + +enum virtiovf_migf_header_flags { + VIRTIOVF_MIGF_HEADER_FLAGS_TAG_MANDATORY = 0, + VIRTIOVF_MIGF_HEADER_FLAGS_TAG_OPTIONAL = 1 << 0, +}; + +enum virtiovf_migf_header_tag { + VIRTIOVF_MIGF_HEADER_TAG_DEVICE_DATA = 0, +}; + +struct virtiovf_migration_header { + __le64 record_size; + /* For future use in case we may need to change the kernel protocol */ + __le32 flags; /* Use virtiovf_migf_header_flags */ + __le32 tag; /* Use virtiovf_migf_header_tag */ + __u8 data[]; /* Its size is given in the record_size */ +}; + +struct virtiovf_migration_file { + struct file *filp; + /* synchronize access to the file state */ + struct mutex lock; + loff_t max_pos; + u64 record_size; + u32 record_tag; + u8 has_obj_id:1; + u32 obj_id; + enum virtiovf_migf_state state; + enum virtiovf_load_state load_state; + /* synchronize access to the lists */ + spinlock_t list_lock; + struct list_head buf_list; + struct list_head avail_list; + struct virtiovf_data_buffer *buf; + struct virtiovf_data_buffer *buf_header; + struct virtiovf_pci_core_device *virtvdev; +}; + +struct virtiovf_pci_core_device { + struct vfio_pci_core_device core_device; + u8 *bar0_virtual_buf; + /* synchronize access to the virtual buf */ + struct mutex bar_mutex; + void __iomem *notify_addr; + u64 notify_offset; + __le32 pci_base_addr_0; + __le16 pci_cmd; + u8 bar0_virtual_buf_size; + u8 notify_bar; + + /* LM related */ + u8 migrate_cap:1; + u8 deferred_reset:1; + /* protect migration state */ + struct mutex state_mutex; + enum vfio_device_mig_state mig_state; + /* protect the reset_done flow */ + spinlock_t reset_lock; + struct virtiovf_migration_file *resuming_migf; + struct virtiovf_migration_file *saving_migf; +}; + +void virtiovf_set_migratable(struct virtiovf_pci_core_device *virtvdev); +void virtiovf_open_migration(struct virtiovf_pci_core_device *virtvdev); +void virtiovf_close_migration(struct virtiovf_pci_core_device *virtvdev); +void virtiovf_migration_reset_done(struct pci_dev *pdev); + +#endif /* VIRTIO_VFIO_COMMON_H */ diff --git a/drivers/vfio/pci/virtio/main.c b/drivers/vfio/pci/virtio/main.c index b5d3a8c5bbc9..e9ae17209026 100644 --- a/drivers/vfio/pci/virtio/main.c +++ b/drivers/vfio/pci/virtio/main.c @@ -16,18 +16,7 @@ #include #include -struct virtiovf_pci_core_device { - struct vfio_pci_core_device core_device; - u8 *bar0_virtual_buf; - /* synchronize access to the virtual buf */ - struct mutex bar_mutex; - void __iomem *notify_addr; - u64 notify_offset; - __le32 pci_base_addr_0; - __le16 pci_cmd; - u8 bar0_virtual_buf_size; - u8 notify_bar; -}; +#include "common.h" static int virtiovf_issue_legacy_rw_cmd(struct virtiovf_pci_core_device *virtvdev, @@ -355,8 +344,8 @@ virtiovf_set_notify_addr(struct virtiovf_pci_core_device *virtvdev) static int virtiovf_pci_open_device(struct vfio_device *core_vdev) { - struct virtiovf_pci_core_device *virtvdev = container_of( - core_vdev, struct virtiovf_pci_core_device, core_device.vdev); + struct virtiovf_pci_core_device *virtvdev = container_of(core_vdev, + struct virtiovf_pci_core_device, core_device.vdev); struct vfio_pci_core_device *vdev = &virtvdev->core_device; int ret; @@ -377,10 +366,20 @@ static int virtiovf_pci_open_device(struct vfio_device *core_vdev) } } + virtiovf_open_migration(virtvdev); vfio_pci_core_finish_enable(vdev); return 0; } +static void virtiovf_pci_close_device(struct vfio_device *core_vdev) +{ + struct virtiovf_pci_core_device *virtvdev = container_of(core_vdev, + struct virtiovf_pci_core_device, core_device.vdev); + + virtiovf_close_migration(virtvdev); + vfio_pci_core_close_device(core_vdev); +} + static int virtiovf_get_device_config_size(unsigned short device) { /* Network card */ @@ -406,8 +405,8 @@ static int virtiovf_read_notify_info(struct virtiovf_pci_core_device *virtvdev) static int virtiovf_pci_init_device(struct vfio_device *core_vdev) { - struct virtiovf_pci_core_device *virtvdev = container_of( - core_vdev, struct virtiovf_pci_core_device, core_device.vdev); + struct virtiovf_pci_core_device *virtvdev = container_of(core_vdev, + struct virtiovf_pci_core_device, core_device.vdev); struct pci_dev *pdev; int ret; @@ -416,6 +415,10 @@ static int virtiovf_pci_init_device(struct vfio_device *core_vdev) return ret; pdev = virtvdev->core_device.pdev; + /* + * The vfio_device_ops.init() callback is set to virtiovf_pci_init_device() + * only when legacy I/O is supported. Now, let's initialize it. + */ ret = virtiovf_read_notify_info(virtvdev); if (ret) return ret; @@ -433,19 +436,38 @@ static int virtiovf_pci_init_device(struct vfio_device *core_vdev) static void virtiovf_pci_core_release_dev(struct vfio_device *core_vdev) { - struct virtiovf_pci_core_device *virtvdev = container_of( - core_vdev, struct virtiovf_pci_core_device, core_device.vdev); + struct virtiovf_pci_core_device *virtvdev = container_of(core_vdev, + struct virtiovf_pci_core_device, core_device.vdev); kfree(virtvdev->bar0_virtual_buf); vfio_pci_core_release_dev(core_vdev); } -static const struct vfio_device_ops virtiovf_vfio_pci_tran_ops = { - .name = "virtio-vfio-pci-trans", +static const struct vfio_device_ops virtiovf_vfio_pci_lm_ops = { + .name = "virtio-vfio-pci-lm", + .init = vfio_pci_core_init_dev, + .release = virtiovf_pci_core_release_dev, + .open_device = virtiovf_pci_open_device, + .close_device = virtiovf_pci_close_device, + .ioctl = vfio_pci_core_ioctl, + .device_feature = vfio_pci_core_ioctl_feature, + .read = vfio_pci_core_read, + .write = vfio_pci_core_write, + .mmap = vfio_pci_core_mmap, + .request = vfio_pci_core_request, + .match = vfio_pci_core_match, + .bind_iommufd = vfio_iommufd_physical_bind, + .unbind_iommufd = vfio_iommufd_physical_unbind, + .attach_ioas = vfio_iommufd_physical_attach_ioas, + .detach_ioas = vfio_iommufd_physical_detach_ioas, +}; + +static const struct vfio_device_ops virtiovf_vfio_pci_tran_lm_ops = { + .name = "virtio-vfio-pci-trans-lm", .init = virtiovf_pci_init_device, .release = virtiovf_pci_core_release_dev, .open_device = virtiovf_pci_open_device, - .close_device = vfio_pci_core_close_device, + .close_device = virtiovf_pci_close_device, .ioctl = virtiovf_vfio_pci_core_ioctl, .device_feature = vfio_pci_core_ioctl_feature, .read = virtiovf_pci_core_read, @@ -490,17 +512,28 @@ static int virtiovf_pci_probe(struct pci_dev *pdev, { const struct vfio_device_ops *ops = &virtiovf_vfio_pci_ops; struct virtiovf_pci_core_device *virtvdev; + bool sup_legacy_io = false; + bool sup_lm = false; int ret; - if (pdev->is_virtfn && virtio_pci_admin_has_legacy_io(pdev) && - !virtiovf_bar0_exists(pdev)) - ops = &virtiovf_vfio_pci_tran_ops; + if (pdev->is_virtfn) { + sup_legacy_io = virtio_pci_admin_has_legacy_io(pdev) && + !virtiovf_bar0_exists(pdev); + sup_lm = virtio_pci_admin_has_dev_parts(pdev); + if (sup_legacy_io) + ops = &virtiovf_vfio_pci_tran_lm_ops; + else if (sup_lm) + ops = &virtiovf_vfio_pci_lm_ops; + } virtvdev = vfio_alloc_device(virtiovf_pci_core_device, core_device.vdev, &pdev->dev, ops); if (IS_ERR(virtvdev)) return PTR_ERR(virtvdev); + if (sup_lm) + virtiovf_set_migratable(virtvdev); + dev_set_drvdata(&pdev->dev, &virtvdev->core_device); ret = vfio_pci_core_register_device(&virtvdev->core_device); if (ret) @@ -532,6 +565,7 @@ static void virtiovf_pci_aer_reset_done(struct pci_dev *pdev) struct virtiovf_pci_core_device *virtvdev = dev_get_drvdata(&pdev->dev); virtvdev->pci_cmd = 0; + virtiovf_migration_reset_done(pdev); } static const struct pci_error_handlers virtiovf_err_handlers = { diff --git a/drivers/vfio/pci/virtio/migrate.c b/drivers/vfio/pci/virtio/migrate.c new file mode 100644 index 000000000000..3d5eaa1cbcdb --- /dev/null +++ b/drivers/vfio/pci/virtio/migrate.c @@ -0,0 +1,1120 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES. All rights reserved + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "common.h" + +/* Device specification max parts size */ +#define MAX_LOAD_SIZE (BIT_ULL(BITS_PER_TYPE \ + (((struct virtio_admin_cmd_dev_parts_metadata_result *)0)->parts_size.size)) - 1) + +/* Initial target buffer size */ +#define VIRTIOVF_TARGET_INITIAL_BUF_SIZE SZ_1M + +static struct page * +virtiovf_get_migration_page(struct virtiovf_data_buffer *buf, + unsigned long offset) +{ + unsigned long cur_offset = 0; + struct scatterlist *sg; + unsigned int i; + + /* All accesses are sequential */ + if (offset < buf->last_offset || !buf->last_offset_sg) { + buf->last_offset = 0; + buf->last_offset_sg = buf->table.sgt.sgl; + buf->sg_last_entry = 0; + } + + cur_offset = buf->last_offset; + + for_each_sg(buf->last_offset_sg, sg, + buf->table.sgt.orig_nents - buf->sg_last_entry, i) { + if (offset < sg->length + cur_offset) { + buf->last_offset_sg = sg; + buf->sg_last_entry += i; + buf->last_offset = cur_offset; + return nth_page(sg_page(sg), + (offset - cur_offset) / PAGE_SIZE); + } + cur_offset += sg->length; + } + return NULL; +} + +static int virtiovf_add_migration_pages(struct virtiovf_data_buffer *buf, + unsigned int npages) +{ + unsigned int to_alloc = npages; + struct page **page_list; + unsigned long filled; + unsigned int to_fill; + int ret; + + to_fill = min_t(unsigned int, npages, PAGE_SIZE / sizeof(*page_list)); + page_list = kvzalloc(to_fill * sizeof(*page_list), GFP_KERNEL_ACCOUNT); + if (!page_list) + return -ENOMEM; + + do { + filled = alloc_pages_bulk_array(GFP_KERNEL_ACCOUNT, to_fill, + page_list); + if (!filled) { + ret = -ENOMEM; + goto err; + } + to_alloc -= filled; + ret = sg_alloc_append_table_from_pages(&buf->table, page_list, + filled, 0, filled << PAGE_SHIFT, UINT_MAX, + SG_MAX_SINGLE_ALLOC, GFP_KERNEL_ACCOUNT); + + if (ret) + goto err; + buf->allocated_length += filled * PAGE_SIZE; + /* clean input for another bulk allocation */ + memset(page_list, 0, filled * sizeof(*page_list)); + to_fill = min_t(unsigned int, to_alloc, + PAGE_SIZE / sizeof(*page_list)); + } while (to_alloc > 0); + + kvfree(page_list); + return 0; + +err: + kvfree(page_list); + return ret; +} + +static void virtiovf_free_data_buffer(struct virtiovf_data_buffer *buf) +{ + struct sg_page_iter sg_iter; + + /* Undo alloc_pages_bulk_array() */ + for_each_sgtable_page(&buf->table.sgt, &sg_iter, 0) + __free_page(sg_page_iter_page(&sg_iter)); + sg_free_append_table(&buf->table); + kfree(buf); +} + +static struct virtiovf_data_buffer * +virtiovf_alloc_data_buffer(struct virtiovf_migration_file *migf, size_t length) +{ + struct virtiovf_data_buffer *buf; + int ret; + + buf = kzalloc(sizeof(*buf), GFP_KERNEL_ACCOUNT); + if (!buf) + return ERR_PTR(-ENOMEM); + + ret = virtiovf_add_migration_pages(buf, + DIV_ROUND_UP_ULL(length, PAGE_SIZE)); + if (ret) + goto end; + + buf->migf = migf; + return buf; +end: + virtiovf_free_data_buffer(buf); + return ERR_PTR(ret); +} + +static void virtiovf_put_data_buffer(struct virtiovf_data_buffer *buf) +{ + spin_lock_irq(&buf->migf->list_lock); + list_add_tail(&buf->buf_elm, &buf->migf->avail_list); + spin_unlock_irq(&buf->migf->list_lock); +} + +static int +virtiovf_pci_alloc_obj_id(struct virtiovf_pci_core_device *virtvdev, u8 type, + u32 *obj_id) +{ + return virtio_pci_admin_obj_create(virtvdev->core_device.pdev, + VIRTIO_RESOURCE_OBJ_DEV_PARTS, type, obj_id); +} + +static void +virtiovf_pci_free_obj_id(struct virtiovf_pci_core_device *virtvdev, u32 obj_id) +{ + virtio_pci_admin_obj_destroy(virtvdev->core_device.pdev, + VIRTIO_RESOURCE_OBJ_DEV_PARTS, obj_id); +} + +static void virtiovf_clean_migf_resources(struct virtiovf_migration_file *migf) +{ + struct virtiovf_data_buffer *entry; + + if (migf->buf) { + virtiovf_free_data_buffer(migf->buf); + migf->buf = NULL; + } + + if (migf->buf_header) { + virtiovf_free_data_buffer(migf->buf_header); + migf->buf_header = NULL; + } + + list_splice(&migf->avail_list, &migf->buf_list); + + while ((entry = list_first_entry_or_null(&migf->buf_list, + struct virtiovf_data_buffer, buf_elm))) { + list_del(&entry->buf_elm); + virtiovf_free_data_buffer(entry); + } + + if (migf->has_obj_id) + virtiovf_pci_free_obj_id(migf->virtvdev, migf->obj_id); +} + +static void virtiovf_disable_fd(struct virtiovf_migration_file *migf) +{ + mutex_lock(&migf->lock); + migf->state = VIRTIOVF_MIGF_STATE_ERROR; + migf->filp->f_pos = 0; + mutex_unlock(&migf->lock); +} + +static void virtiovf_disable_fds(struct virtiovf_pci_core_device *virtvdev) +{ + if (virtvdev->resuming_migf) { + virtiovf_disable_fd(virtvdev->resuming_migf); + virtiovf_clean_migf_resources(virtvdev->resuming_migf); + fput(virtvdev->resuming_migf->filp); + virtvdev->resuming_migf = NULL; + } + if (virtvdev->saving_migf) { + virtiovf_disable_fd(virtvdev->saving_migf); + virtiovf_clean_migf_resources(virtvdev->saving_migf); + fput(virtvdev->saving_migf->filp); + virtvdev->saving_migf = NULL; + } +} + +/* + * This function is called in all state_mutex unlock cases to + * handle a 'deferred_reset' if exists. + */ +static void virtiovf_state_mutex_unlock(struct virtiovf_pci_core_device *virtvdev) +{ +again: + spin_lock(&virtvdev->reset_lock); + if (virtvdev->deferred_reset) { + virtvdev->deferred_reset = false; + spin_unlock(&virtvdev->reset_lock); + virtvdev->mig_state = VFIO_DEVICE_STATE_RUNNING; + virtiovf_disable_fds(virtvdev); + goto again; + } + mutex_unlock(&virtvdev->state_mutex); + spin_unlock(&virtvdev->reset_lock); +} + +void virtiovf_migration_reset_done(struct pci_dev *pdev) +{ + struct virtiovf_pci_core_device *virtvdev = dev_get_drvdata(&pdev->dev); + + if (!virtvdev->migrate_cap) + return; + + /* + * As the higher VFIO layers are holding locks across reset and using + * those same locks with the mm_lock we need to prevent ABBA deadlock + * with the state_mutex and mm_lock. + * In case the state_mutex was taken already we defer the cleanup work + * to the unlock flow of the other running context. + */ + spin_lock(&virtvdev->reset_lock); + virtvdev->deferred_reset = true; + if (!mutex_trylock(&virtvdev->state_mutex)) { + spin_unlock(&virtvdev->reset_lock); + return; + } + spin_unlock(&virtvdev->reset_lock); + virtiovf_state_mutex_unlock(virtvdev); +} + +static int virtiovf_release_file(struct inode *inode, struct file *filp) +{ + struct virtiovf_migration_file *migf = filp->private_data; + + virtiovf_disable_fd(migf); + mutex_destroy(&migf->lock); + kfree(migf); + return 0; +} + +static struct virtiovf_data_buffer * +virtiovf_get_data_buff_from_pos(struct virtiovf_migration_file *migf, + loff_t pos, bool *end_of_data) +{ + struct virtiovf_data_buffer *buf; + bool found = false; + + *end_of_data = false; + spin_lock_irq(&migf->list_lock); + if (list_empty(&migf->buf_list)) { + *end_of_data = true; + goto end; + } + + buf = list_first_entry(&migf->buf_list, struct virtiovf_data_buffer, + buf_elm); + if (pos >= buf->start_pos && + pos < buf->start_pos + buf->length) { + found = true; + goto end; + } + + /* + * As we use a stream based FD we may expect having the data always + * on first chunk + */ + migf->state = VIRTIOVF_MIGF_STATE_ERROR; + +end: + spin_unlock_irq(&migf->list_lock); + return found ? buf : NULL; +} + +static ssize_t virtiovf_buf_read(struct virtiovf_data_buffer *vhca_buf, + char __user **buf, size_t *len, loff_t *pos) +{ + unsigned long offset; + ssize_t done = 0; + size_t copy_len; + + copy_len = min_t(size_t, + vhca_buf->start_pos + vhca_buf->length - *pos, *len); + while (copy_len) { + size_t page_offset; + struct page *page; + size_t page_len; + u8 *from_buff; + int ret; + + offset = *pos - vhca_buf->start_pos; + page_offset = offset % PAGE_SIZE; + offset -= page_offset; + page = virtiovf_get_migration_page(vhca_buf, offset); + if (!page) + return -EINVAL; + page_len = min_t(size_t, copy_len, PAGE_SIZE - page_offset); + from_buff = kmap_local_page(page); + ret = copy_to_user(*buf, from_buff + page_offset, page_len); + kunmap_local(from_buff); + if (ret) + return -EFAULT; + *pos += page_len; + *len -= page_len; + *buf += page_len; + done += page_len; + copy_len -= page_len; + } + + if (*pos >= vhca_buf->start_pos + vhca_buf->length) { + spin_lock_irq(&vhca_buf->migf->list_lock); + list_del_init(&vhca_buf->buf_elm); + list_add_tail(&vhca_buf->buf_elm, &vhca_buf->migf->avail_list); + spin_unlock_irq(&vhca_buf->migf->list_lock); + } + + return done; +} + +static ssize_t virtiovf_save_read(struct file *filp, char __user *buf, size_t len, + loff_t *pos) +{ + struct virtiovf_migration_file *migf = filp->private_data; + struct virtiovf_data_buffer *vhca_buf; + bool end_of_data; + ssize_t done = 0; + + if (pos) + return -ESPIPE; + pos = &filp->f_pos; + + mutex_lock(&migf->lock); + if (migf->state == VIRTIOVF_MIGF_STATE_ERROR) { + done = -ENODEV; + goto out_unlock; + } + + while (len) { + ssize_t count; + + vhca_buf = virtiovf_get_data_buff_from_pos(migf, *pos, &end_of_data); + if (end_of_data) + goto out_unlock; + + if (!vhca_buf) { + done = -EINVAL; + goto out_unlock; + } + + count = virtiovf_buf_read(vhca_buf, &buf, &len, pos); + if (count < 0) { + done = count; + goto out_unlock; + } + done += count; + } + +out_unlock: + mutex_unlock(&migf->lock); + return done; +} + +static const struct file_operations virtiovf_save_fops = { + .owner = THIS_MODULE, + .read = virtiovf_save_read, + .release = virtiovf_release_file, +}; + +static int +virtiovf_add_buf_header(struct virtiovf_data_buffer *header_buf, + u32 data_size) +{ + struct virtiovf_migration_file *migf = header_buf->migf; + struct virtiovf_migration_header header = {}; + struct page *page; + u8 *to_buff; + + header.record_size = cpu_to_le64(data_size); + header.flags = cpu_to_le32(VIRTIOVF_MIGF_HEADER_FLAGS_TAG_MANDATORY); + header.tag = cpu_to_le32(VIRTIOVF_MIGF_HEADER_TAG_DEVICE_DATA); + page = virtiovf_get_migration_page(header_buf, 0); + if (!page) + return -EINVAL; + to_buff = kmap_local_page(page); + memcpy(to_buff, &header, sizeof(header)); + kunmap_local(to_buff); + header_buf->length = sizeof(header); + header_buf->start_pos = header_buf->migf->max_pos; + migf->max_pos += header_buf->length; + spin_lock_irq(&migf->list_lock); + list_add_tail(&header_buf->buf_elm, &migf->buf_list); + spin_unlock_irq(&migf->list_lock); + return 0; +} + +static int +virtiovf_read_device_context_chunk(struct virtiovf_migration_file *migf, + u32 ctx_size) +{ + struct virtiovf_data_buffer *header_buf; + struct virtiovf_data_buffer *buf; + bool unmark_end = false; + struct scatterlist *sg; + unsigned int i; + u32 res_size; + int nent; + int ret; + + buf = virtiovf_alloc_data_buffer(migf, ctx_size); + if (IS_ERR(buf)) + return PTR_ERR(buf); + + /* Find the total count of SG entries which satisfies the size */ + nent = sg_nents_for_len(buf->table.sgt.sgl, ctx_size); + if (nent <= 0) { + ret = -EINVAL; + goto out; + } + + /* + * Iterate to that SG entry and mark it as last (if it's not already) + * to let underlay layers iterate only till that entry. + */ + for_each_sg(buf->table.sgt.sgl, sg, nent - 1, i) + ; + + if (!sg_is_last(sg)) { + unmark_end = true; + sg_mark_end(sg); + } + + ret = virtio_pci_admin_dev_parts_get(migf->virtvdev->core_device.pdev, + VIRTIO_RESOURCE_OBJ_DEV_PARTS, + migf->obj_id, + VIRTIO_ADMIN_CMD_DEV_PARTS_GET_TYPE_ALL, + buf->table.sgt.sgl, &res_size); + /* Restore the original SG mark end */ + if (unmark_end) + sg_unmark_end(sg); + if (ret) + goto out; + + buf->length = res_size; + header_buf = virtiovf_alloc_data_buffer(migf, + sizeof(struct virtiovf_migration_header)); + if (IS_ERR(header_buf)) { + ret = PTR_ERR(header_buf); + goto out; + } + + ret = virtiovf_add_buf_header(header_buf, res_size); + if (ret) + goto out_header; + + buf->start_pos = buf->migf->max_pos; + migf->max_pos += buf->length; + spin_lock(&migf->list_lock); + list_add_tail(&buf->buf_elm, &migf->buf_list); + spin_unlock_irq(&migf->list_lock); + return 0; + +out_header: + virtiovf_put_data_buffer(header_buf); +out: + virtiovf_put_data_buffer(buf); + return ret; +} + +static struct virtiovf_migration_file * +virtiovf_pci_save_device_data(struct virtiovf_pci_core_device *virtvdev) +{ + struct virtiovf_migration_file *migf; + u32 ctx_size; + u32 obj_id; + int ret; + + migf = kzalloc(sizeof(*migf), GFP_KERNEL_ACCOUNT); + if (!migf) + return ERR_PTR(-ENOMEM); + + migf->filp = anon_inode_getfile("virtiovf_mig", &virtiovf_save_fops, migf, + O_RDONLY); + if (IS_ERR(migf->filp)) { + ret = PTR_ERR(migf->filp); + goto end; + } + + stream_open(migf->filp->f_inode, migf->filp); + mutex_init(&migf->lock); + INIT_LIST_HEAD(&migf->buf_list); + INIT_LIST_HEAD(&migf->avail_list); + spin_lock_init(&migf->list_lock); + migf->virtvdev = virtvdev; + + lockdep_assert_held(&virtvdev->state_mutex); + ret = virtiovf_pci_alloc_obj_id(virtvdev, VIRTIO_RESOURCE_OBJ_DEV_PARTS_TYPE_GET, + &obj_id); + if (ret) + goto out; + + migf->obj_id = obj_id; + /* Mark as having a valid obj id which can be even 0 */ + migf->has_obj_id = true; + ret = virtio_pci_admin_dev_parts_metadata_get(virtvdev->core_device.pdev, + VIRTIO_RESOURCE_OBJ_DEV_PARTS, obj_id, + VIRTIO_ADMIN_CMD_DEV_PARTS_METADATA_TYPE_SIZE, + &ctx_size); + if (ret) + goto out_clean; + + if (!ctx_size) { + ret = -EINVAL; + goto out_clean; + } + + ret = virtiovf_read_device_context_chunk(migf, ctx_size); + if (ret) + goto out_clean; + + return migf; + +out_clean: + virtiovf_clean_migf_resources(migf); +out: + fput(migf->filp); +end: + kfree(migf); + return ERR_PTR(ret); +} + +/* + * Set the required object header at the beginning of the buffer. + * The actual device parts data will be written post of the header offset. + */ +static int virtiovf_set_obj_cmd_header(struct virtiovf_data_buffer *vhca_buf) +{ + struct virtio_admin_cmd_resource_obj_cmd_hdr obj_hdr = {}; + struct page *page; + u8 *to_buff; + + obj_hdr.type = cpu_to_le16(VIRTIO_RESOURCE_OBJ_DEV_PARTS); + obj_hdr.id = cpu_to_le32(vhca_buf->migf->obj_id); + page = virtiovf_get_migration_page(vhca_buf, 0); + if (!page) + return -EINVAL; + to_buff = kmap_local_page(page); + memcpy(to_buff, &obj_hdr, sizeof(obj_hdr)); + kunmap_local(to_buff); + + /* Mark the buffer as including the header object data */ + vhca_buf->include_header_object = 1; + return 0; +} + +static int +virtiovf_append_page_to_mig_buf(struct virtiovf_data_buffer *vhca_buf, + const char __user **buf, size_t *len, + loff_t *pos, ssize_t *done) +{ + unsigned long offset; + size_t page_offset; + struct page *page; + size_t page_len; + u8 *to_buff; + int ret; + + offset = *pos - vhca_buf->start_pos; + + if (vhca_buf->include_header_object) + /* The buffer holds the object header, update the offset accordingly */ + offset += sizeof(struct virtio_admin_cmd_resource_obj_cmd_hdr); + + page_offset = offset % PAGE_SIZE; + + page = virtiovf_get_migration_page(vhca_buf, offset - page_offset); + if (!page) + return -EINVAL; + + page_len = min_t(size_t, *len, PAGE_SIZE - page_offset); + to_buff = kmap_local_page(page); + ret = copy_from_user(to_buff + page_offset, *buf, page_len); + kunmap_local(to_buff); + if (ret) + return -EFAULT; + + *pos += page_len; + *done += page_len; + *buf += page_len; + *len -= page_len; + vhca_buf->length += page_len; + return 0; +} + +static ssize_t +virtiovf_resume_read_chunk(struct virtiovf_migration_file *migf, + struct virtiovf_data_buffer *vhca_buf, + size_t chunk_size, const char __user **buf, + size_t *len, loff_t *pos, ssize_t *done, + bool *has_work) +{ + size_t copy_len, to_copy; + int ret; + + to_copy = min_t(size_t, *len, chunk_size - vhca_buf->length); + copy_len = to_copy; + while (to_copy) { + ret = virtiovf_append_page_to_mig_buf(vhca_buf, buf, &to_copy, + pos, done); + if (ret) + return ret; + } + + *len -= copy_len; + if (vhca_buf->length == chunk_size) { + migf->load_state = VIRTIOVF_LOAD_STATE_LOAD_CHUNK; + migf->max_pos += chunk_size; + *has_work = true; + } + + return 0; +} + +static int +virtiovf_resume_read_header_data(struct virtiovf_migration_file *migf, + struct virtiovf_data_buffer *vhca_buf, + const char __user **buf, size_t *len, + loff_t *pos, ssize_t *done) +{ + size_t copy_len, to_copy; + size_t required_data; + int ret; + + required_data = migf->record_size - vhca_buf->length; + to_copy = min_t(size_t, *len, required_data); + copy_len = to_copy; + while (to_copy) { + ret = virtiovf_append_page_to_mig_buf(vhca_buf, buf, &to_copy, + pos, done); + if (ret) + return ret; + } + + *len -= copy_len; + if (vhca_buf->length == migf->record_size) { + switch (migf->record_tag) { + default: + /* Optional tag */ + break; + } + + migf->load_state = VIRTIOVF_LOAD_STATE_READ_HEADER; + migf->max_pos += migf->record_size; + vhca_buf->length = 0; + } + + return 0; +} + +static int +virtiovf_resume_read_header(struct virtiovf_migration_file *migf, + struct virtiovf_data_buffer *vhca_buf, + const char __user **buf, + size_t *len, loff_t *pos, + ssize_t *done, bool *has_work) +{ + struct page *page; + size_t copy_len; + u8 *to_buff; + int ret; + + copy_len = min_t(size_t, *len, + sizeof(struct virtiovf_migration_header) - vhca_buf->length); + page = virtiovf_get_migration_page(vhca_buf, 0); + if (!page) + return -EINVAL; + to_buff = kmap_local_page(page); + ret = copy_from_user(to_buff + vhca_buf->length, *buf, copy_len); + if (ret) { + ret = -EFAULT; + goto end; + } + + *buf += copy_len; + *pos += copy_len; + *done += copy_len; + *len -= copy_len; + vhca_buf->length += copy_len; + if (vhca_buf->length == sizeof(struct virtiovf_migration_header)) { + u64 record_size; + u32 flags; + + record_size = le64_to_cpup((__le64 *)to_buff); + if (record_size > MAX_LOAD_SIZE) { + ret = -ENOMEM; + goto end; + } + + migf->record_size = record_size; + flags = le32_to_cpup((__le32 *)(to_buff + + offsetof(struct virtiovf_migration_header, flags))); + migf->record_tag = le32_to_cpup((__le32 *)(to_buff + + offsetof(struct virtiovf_migration_header, tag))); + switch (migf->record_tag) { + case VIRTIOVF_MIGF_HEADER_TAG_DEVICE_DATA: + migf->load_state = VIRTIOVF_LOAD_STATE_PREP_CHUNK; + break; + default: + if (!(flags & VIRTIOVF_MIGF_HEADER_FLAGS_TAG_OPTIONAL)) { + ret = -EOPNOTSUPP; + goto end; + } + /* We may read and skip this optional record data */ + migf->load_state = VIRTIOVF_LOAD_STATE_PREP_HEADER_DATA; + } + + migf->max_pos += vhca_buf->length; + vhca_buf->length = 0; + *has_work = true; + } +end: + kunmap_local(to_buff); + return ret; +} + +static ssize_t virtiovf_resume_write(struct file *filp, const char __user *buf, + size_t len, loff_t *pos) +{ + struct virtiovf_migration_file *migf = filp->private_data; + struct virtiovf_data_buffer *vhca_buf = migf->buf; + struct virtiovf_data_buffer *vhca_buf_header = migf->buf_header; + unsigned int orig_length; + bool has_work = false; + ssize_t done = 0; + int ret = 0; + + if (pos) + return -ESPIPE; + + pos = &filp->f_pos; + if (*pos < vhca_buf->start_pos) + return -EINVAL; + + mutex_lock(&migf->virtvdev->state_mutex); + mutex_lock(&migf->lock); + if (migf->state == VIRTIOVF_MIGF_STATE_ERROR) { + done = -ENODEV; + goto out_unlock; + } + + while (len || has_work) { + has_work = false; + switch (migf->load_state) { + case VIRTIOVF_LOAD_STATE_READ_HEADER: + ret = virtiovf_resume_read_header(migf, vhca_buf_header, &buf, + &len, pos, &done, &has_work); + if (ret) + goto out_unlock; + break; + case VIRTIOVF_LOAD_STATE_PREP_HEADER_DATA: + if (vhca_buf_header->allocated_length < migf->record_size) { + virtiovf_free_data_buffer(vhca_buf_header); + + migf->buf_header = virtiovf_alloc_data_buffer(migf, + migf->record_size); + if (IS_ERR(migf->buf_header)) { + ret = PTR_ERR(migf->buf_header); + migf->buf_header = NULL; + goto out_unlock; + } + + vhca_buf_header = migf->buf_header; + } + + vhca_buf_header->start_pos = migf->max_pos; + migf->load_state = VIRTIOVF_LOAD_STATE_READ_HEADER_DATA; + break; + case VIRTIOVF_LOAD_STATE_READ_HEADER_DATA: + ret = virtiovf_resume_read_header_data(migf, vhca_buf_header, + &buf, &len, pos, &done); + if (ret) + goto out_unlock; + break; + case VIRTIOVF_LOAD_STATE_PREP_CHUNK: + { + u32 cmd_size = migf->record_size + + sizeof(struct virtio_admin_cmd_resource_obj_cmd_hdr); + + /* + * The DMA map/unmap is managed in virtio layer, we just need to extend + * the SG pages to hold the extra required chunk data. + */ + if (vhca_buf->allocated_length < cmd_size) { + ret = virtiovf_add_migration_pages(vhca_buf, + DIV_ROUND_UP_ULL(cmd_size - vhca_buf->allocated_length, + PAGE_SIZE)); + if (ret) + goto out_unlock; + } + + vhca_buf->start_pos = migf->max_pos; + migf->load_state = VIRTIOVF_LOAD_STATE_READ_CHUNK; + break; + } + case VIRTIOVF_LOAD_STATE_READ_CHUNK: + ret = virtiovf_resume_read_chunk(migf, vhca_buf, migf->record_size, + &buf, &len, pos, &done, &has_work); + if (ret) + goto out_unlock; + break; + case VIRTIOVF_LOAD_STATE_LOAD_CHUNK: + /* Mark the last SG entry and set its length */ + sg_mark_end(vhca_buf->last_offset_sg); + orig_length = vhca_buf->last_offset_sg->length; + /* Length should include the resource object command header */ + vhca_buf->last_offset_sg->length = vhca_buf->length + + sizeof(struct virtio_admin_cmd_resource_obj_cmd_hdr) - + vhca_buf->last_offset; + ret = virtio_pci_admin_dev_parts_set(migf->virtvdev->core_device.pdev, + vhca_buf->table.sgt.sgl); + /* Restore the original SG data */ + vhca_buf->last_offset_sg->length = orig_length; + sg_unmark_end(vhca_buf->last_offset_sg); + if (ret) + goto out_unlock; + migf->load_state = VIRTIOVF_LOAD_STATE_READ_HEADER; + /* be ready for reading the next chunk */ + vhca_buf->length = 0; + break; + default: + break; + } + } + +out_unlock: + if (ret) + migf->state = VIRTIOVF_MIGF_STATE_ERROR; + mutex_unlock(&migf->lock); + virtiovf_state_mutex_unlock(migf->virtvdev); + return ret ? ret : done; +} + +static const struct file_operations virtiovf_resume_fops = { + .owner = THIS_MODULE, + .write = virtiovf_resume_write, + .release = virtiovf_release_file, +}; + +static struct virtiovf_migration_file * +virtiovf_pci_resume_device_data(struct virtiovf_pci_core_device *virtvdev) +{ + struct virtiovf_migration_file *migf; + struct virtiovf_data_buffer *buf; + u32 obj_id; + int ret; + + migf = kzalloc(sizeof(*migf), GFP_KERNEL_ACCOUNT); + if (!migf) + return ERR_PTR(-ENOMEM); + + migf->filp = anon_inode_getfile("virtiovf_mig", &virtiovf_resume_fops, migf, + O_WRONLY); + if (IS_ERR(migf->filp)) { + ret = PTR_ERR(migf->filp); + goto end; + } + + stream_open(migf->filp->f_inode, migf->filp); + mutex_init(&migf->lock); + INIT_LIST_HEAD(&migf->buf_list); + INIT_LIST_HEAD(&migf->avail_list); + spin_lock_init(&migf->list_lock); + + buf = virtiovf_alloc_data_buffer(migf, VIRTIOVF_TARGET_INITIAL_BUF_SIZE); + if (IS_ERR(buf)) { + ret = PTR_ERR(buf); + goto out_free; + } + + migf->buf = buf; + + buf = virtiovf_alloc_data_buffer(migf, + sizeof(struct virtiovf_migration_header)); + if (IS_ERR(buf)) { + ret = PTR_ERR(buf); + goto out_clean; + } + + migf->buf_header = buf; + migf->load_state = VIRTIOVF_LOAD_STATE_READ_HEADER; + + migf->virtvdev = virtvdev; + ret = virtiovf_pci_alloc_obj_id(virtvdev, VIRTIO_RESOURCE_OBJ_DEV_PARTS_TYPE_SET, + &obj_id); + if (ret) + goto out_clean; + + migf->obj_id = obj_id; + /* Mark as having a valid obj id which can be even 0 */ + migf->has_obj_id = true; + ret = virtiovf_set_obj_cmd_header(migf->buf); + if (ret) + goto out_clean; + + return migf; + +out_clean: + virtiovf_clean_migf_resources(migf); +out_free: + fput(migf->filp); +end: + kfree(migf); + return ERR_PTR(ret); +} + +static struct file * +virtiovf_pci_step_device_state_locked(struct virtiovf_pci_core_device *virtvdev, + u32 new) +{ + u32 cur = virtvdev->mig_state; + int ret; + + if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && new == VFIO_DEVICE_STATE_STOP) { + /* NOP */ + return NULL; + } + + if (cur == VFIO_DEVICE_STATE_STOP && new == VFIO_DEVICE_STATE_RUNNING_P2P) { + /* NOP */ + return NULL; + } + + if (cur == VFIO_DEVICE_STATE_RUNNING && new == VFIO_DEVICE_STATE_RUNNING_P2P) { + ret = virtio_pci_admin_mode_set(virtvdev->core_device.pdev, + BIT(VIRTIO_ADMIN_CMD_DEV_MODE_F_STOPPED)); + if (ret) + return ERR_PTR(ret); + return NULL; + } + + if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && new == VFIO_DEVICE_STATE_RUNNING) { + ret = virtio_pci_admin_mode_set(virtvdev->core_device.pdev, 0); + if (ret) + return ERR_PTR(ret); + return NULL; + } + + if (cur == VFIO_DEVICE_STATE_STOP && new == VFIO_DEVICE_STATE_STOP_COPY) { + struct virtiovf_migration_file *migf; + + migf = virtiovf_pci_save_device_data(virtvdev); + if (IS_ERR(migf)) + return ERR_CAST(migf); + get_file(migf->filp); + virtvdev->saving_migf = migf; + return migf->filp; + } + + if (cur == VFIO_DEVICE_STATE_STOP_COPY && new == VFIO_DEVICE_STATE_STOP) { + virtiovf_disable_fds(virtvdev); + return NULL; + } + + if (cur == VFIO_DEVICE_STATE_STOP && new == VFIO_DEVICE_STATE_RESUMING) { + struct virtiovf_migration_file *migf; + + migf = virtiovf_pci_resume_device_data(virtvdev); + if (IS_ERR(migf)) + return ERR_CAST(migf); + get_file(migf->filp); + virtvdev->resuming_migf = migf; + return migf->filp; + } + + if (cur == VFIO_DEVICE_STATE_RESUMING && new == VFIO_DEVICE_STATE_STOP) { + virtiovf_disable_fds(virtvdev); + return NULL; + } + + /* + * vfio_mig_get_next_state() does not use arcs other than the above + */ + WARN_ON(true); + return ERR_PTR(-EINVAL); +} + +static struct file * +virtiovf_pci_set_device_state(struct vfio_device *vdev, + enum vfio_device_mig_state new_state) +{ + struct virtiovf_pci_core_device *virtvdev = container_of( + vdev, struct virtiovf_pci_core_device, core_device.vdev); + enum vfio_device_mig_state next_state; + struct file *res = NULL; + int ret; + + mutex_lock(&virtvdev->state_mutex); + while (new_state != virtvdev->mig_state) { + ret = vfio_mig_get_next_state(vdev, virtvdev->mig_state, + new_state, &next_state); + if (ret) { + res = ERR_PTR(ret); + break; + } + res = virtiovf_pci_step_device_state_locked(virtvdev, next_state); + if (IS_ERR(res)) + break; + virtvdev->mig_state = next_state; + if (WARN_ON(res && new_state != virtvdev->mig_state)) { + fput(res); + res = ERR_PTR(-EINVAL); + break; + } + } + virtiovf_state_mutex_unlock(virtvdev); + return res; +} + +static int virtiovf_pci_get_device_state(struct vfio_device *vdev, + enum vfio_device_mig_state *curr_state) +{ + struct virtiovf_pci_core_device *virtvdev = container_of( + vdev, struct virtiovf_pci_core_device, core_device.vdev); + + mutex_lock(&virtvdev->state_mutex); + *curr_state = virtvdev->mig_state; + virtiovf_state_mutex_unlock(virtvdev); + return 0; +} + +static int virtiovf_pci_get_data_size(struct vfio_device *vdev, + unsigned long *stop_copy_length) +{ + struct virtiovf_pci_core_device *virtvdev = container_of( + vdev, struct virtiovf_pci_core_device, core_device.vdev); + bool obj_id_exists; + u32 res_size; + u32 obj_id; + int ret; + + mutex_lock(&virtvdev->state_mutex); + obj_id_exists = virtvdev->saving_migf && virtvdev->saving_migf->has_obj_id; + if (!obj_id_exists) { + ret = virtiovf_pci_alloc_obj_id(virtvdev, + VIRTIO_RESOURCE_OBJ_DEV_PARTS_TYPE_GET, + &obj_id); + if (ret) + goto end; + } else { + obj_id = virtvdev->saving_migf->obj_id; + } + + ret = virtio_pci_admin_dev_parts_metadata_get(virtvdev->core_device.pdev, + VIRTIO_RESOURCE_OBJ_DEV_PARTS, obj_id, + VIRTIO_ADMIN_CMD_DEV_PARTS_METADATA_TYPE_SIZE, + &res_size); + if (!ret) + *stop_copy_length = res_size; + + /* + * We can't leave this obj_id alive if didn't exist before, otherwise, it might + * stay alive, even without an active migration flow (e.g. migration was cancelled) + */ + if (!obj_id_exists) + virtiovf_pci_free_obj_id(virtvdev, obj_id); +end: + virtiovf_state_mutex_unlock(virtvdev); + return ret; +} + +static const struct vfio_migration_ops virtvdev_pci_mig_ops = { + .migration_set_state = virtiovf_pci_set_device_state, + .migration_get_state = virtiovf_pci_get_device_state, + .migration_get_data_size = virtiovf_pci_get_data_size, +}; + +void virtiovf_set_migratable(struct virtiovf_pci_core_device *virtvdev) +{ + virtvdev->migrate_cap = 1; + mutex_init(&virtvdev->state_mutex); + spin_lock_init(&virtvdev->reset_lock); + virtvdev->core_device.vdev.migration_flags = + VFIO_MIGRATION_STOP_COPY | + VFIO_MIGRATION_P2P; + virtvdev->core_device.vdev.mig_ops = &virtvdev_pci_mig_ops; +} + +void virtiovf_open_migration(struct virtiovf_pci_core_device *virtvdev) +{ + if (!virtvdev->migrate_cap) + return; + + virtvdev->mig_state = VFIO_DEVICE_STATE_RUNNING; +} + +void virtiovf_close_migration(struct virtiovf_pci_core_device *virtvdev) +{ + if (!virtvdev->migrate_cap) + return; + + virtiovf_disable_fds(virtvdev); +} From patchwork Tue Nov 12 08:37:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 13871914 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2082.outbound.protection.outlook.com [40.107.236.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7926A20DD4F for ; Tue, 12 Nov 2024 08:39:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.82 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400744; cv=fail; b=P/pXoWWnis5p4M30pnZpJ8s58qXiPwLJkt3UCnIWgkpqxw+728jp9G+jUPVnmMoNKvLhWlcLUvaL8mqGOO5EVilUivt3kaExqNWaNqRFn1DZzR8wFTgrFHCtaPg11wHBxZJQ/7gc6t/vITyGw98NM2EPPR8Wt1JMcPWQTBl55BY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400744; c=relaxed/simple; bh=sJuOYMXDIuSbq9w5xKvDGObmCVc6TuLJzcQfiZLPFq0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=sYNWbyNOQewY/7fg0zZVFa835R+Zcl7Ix+DVzuBoLms+beyebBydX9GZHBlQwy1UAIJcds1NCki2V2ekZXO54azv2dmWPHFlEel8h7h6lZyRg6w1vSO37ZSVZ1P5DajOkd8MdLJskhRLUoSCmw+nN971QUYPTgx5wFThogMPGPM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=rvjfPYEh; arc=fail smtp.client-ip=40.107.236.82 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="rvjfPYEh" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fErCKTfbUpFh+8jO6kS0UdhDX49NHBhpX2AKMCorZJq71Wtwn9vLmE+3whvTBESXR289vk3iW/fwUsF5TSJdPG8kB63ZpviGOKzwoaqcGbxe8sNcWUrHU2GApOM5aSl4YqQX4ureNC6uw8+YFXY7tuJxvCPDYfPvUTxVdkxCyUP7oMX/5Bpad+A+jC9xiWEzjm2C5JnQPYVf+XF7NtSjpEY06FuUttsybGlCCx06nZohNm0tOPoIg8ZpgRUGvYKH3OJ+fZUS20w1YUPzQThjlByaGTkOadvUuyPh6uI3VgDSMEOlG+5m9Szh20nPRIrmQJjiHzIfBdpT9fkJNVjmfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nUgMiQEwGrniVi1wVBLHMqNuqqEeCW5GdVnih0vXrI0=; b=lgI6pv6PbK7y8QtnVFHJYmZAME21K5sQDRR7SzXoAkA8gIjP6nuhqgYPV9MiNGYoADMFXQQ/WmeMMM16TpcW1VQUGpIBv0FYJZzJJoFoqYt3Tl8SGiT1CRvLPlN+OVIUf3ZoCjNN8I5PhWBdZaPRpCy8Qjn+/VcwaQLPj69s3GViYcygTc7Pbmdy5sN3eoCKCmiZJRQVDQy71aQoy3U8Oo639TlJQp4Zat3IMc1fSIDO+8t7jgKWhjmRwfb6OrrsyEX9IXDqx5KTPFQeAWiqoisMGjOLUvxfRhE2D6m4qXsqHXkY2N1M+GkSRyjbsY/aJixUu3vDJO6vIT7m3WYUZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nUgMiQEwGrniVi1wVBLHMqNuqqEeCW5GdVnih0vXrI0=; b=rvjfPYEhoGKprehzhGTTujtn2kw/QVuEAmlf14Ysb1ZafaKz/3i+PFyclLtVkvvzWHiYazO5BsSRgieDTWzvay7vSmOtk1eufO5JlSj/aZ+MVZIVIur4hQ7Cs12mAjEZ8kmlJ074y29ZPXazWSbjIUXk9q9wLUfj5Pt4BPsB3UdvbnWl+X4yQ7KVHk4e98PP+nCvJvJyEqLfPWrDhjGD2HtxeDgcQv2pgfjx6Zi98gkbl8maxAorJsXpi00PjJQyG0x0uJNQsK87QHq90y6kys87tGu9ouaPzyl0aAdfTgvEEugASlyDn6Iwgm/geH91PUo91u8g0fvsizCl2kwITQ== Received: from MN2PR03CA0003.namprd03.prod.outlook.com (2603:10b6:208:23a::8) by IA1PR12MB7734.namprd12.prod.outlook.com (2603:10b6:208:422::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.17; Tue, 12 Nov 2024 08:38:56 +0000 Received: from MN1PEPF0000F0E1.namprd04.prod.outlook.com (2603:10b6:208:23a:cafe::81) by MN2PR03CA0003.outlook.office365.com (2603:10b6:208:23a::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8137.28 via Frontend Transport; Tue, 12 Nov 2024 08:38:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by MN1PEPF0000F0E1.mail.protection.outlook.com (10.167.242.39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.14 via Frontend Transport; Tue, 12 Nov 2024 08:38:55 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:40 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:39 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 12 Nov 2024 00:38:36 -0800 From: Yishai Hadas To: , , , CC: , , , , , , , , Subject: [PATCH V3 vfio 6/7] vfio/virtio: Add PRE_COPY support for live migration Date: Tue, 12 Nov 2024 10:37:28 +0200 Message-ID: <20241112083729.145005-7-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20241112083729.145005-1-yishaih@nvidia.com> References: <20241112083729.145005-1-yishaih@nvidia.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E1:EE_|IA1PR12MB7734:EE_ X-MS-Office365-Filtering-Correlation-Id: 28b5747f-aa6f-48ae-9d6a-08dd02f5720b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|376014|82310400026|1800799024; X-Microsoft-Antispam-Message-Info: zT2Q1QBCQzkpvhAIHQYXb5rinp9IJxcok0kGEcIO5klbAmwZWiGC3zRCe2AuF3OTQ3r1s2sGmWemkFnHeqk/ZBjiFNH/p4MbAVyDK+nRVwpaxS5P6MvymbKjhTRX2fGkF3v1nVQql+BcDO4NtC7Sc7dGknbh1IUbflKKpZDfiC5Y/DEzxrTwqgvShVSyF39BVENtqq4n/ocYclJqB9WQD5j9WDHF/h3CVQUcZW3HZ6PnPHdMuBIyD5PgF8nauCoL560hPHBJ7I3eeLlA01qdmYFLrKhWyRu+pyLh767CfzKP7ux9vk0VmRo78srywGLadfyMuCpAbKw7yb9koq9+TLSER+anh0n403YySDxLYdAYW9auwjYvw0ZLRAiglgj890MmaxeToz8qB5hb6eAfHAE0OA4XvBtsdkEs72cVFTS+xr+2n1IRfuLEVF/lhJQ0Q/6ffpWuAse0NdlksGk1HDJYoM/PYedA9XDa1TulfNAVH6chKhdhugcIk8nLFPl+9zMUDjn/E+gJ5txQZKtz42MqlpwTlfFr3c+/X9zk9WiD8qDRJsAKhL5ipw4QQP4JzMcochp0Nt8DhA0Z0mtrPpopCo5tBWQ54PG/KaRpFeAuOdg5h7dbYShupYIIMtApihjy3tY7n3+kOdxbwV9RwvxCHWkMTHhmn9fIiSfZvjRC6gYcZtkS5RLbIPQLIrWhHgD19z/c+RYH9av9l10adJEmudIZvz9F3VMEbASwXVXTCoHDtSXpsQFQ5kjXJEa9AhHM70FiJXhwu1ExMvHCdo/gdusqlBljuUL3CxUXQo/u2q7sgKgkYfzjOjhcxDwx0fTjRkUBGH68oQNqip99VgEFtyanOwwfYIqrWBz/J/1ZjI8eGW/5dfk4bZXB51wA46wdCtckLplo1NwgA2QeI5pzyzpYuRDXbXXg5j9ZBmqLjVIushqdnSeyrMJcVvf/WnM+dZ8RNqbR9zxNs5M7V3RNmUaShKahWI2fIbauO1rf7OWDN4B7Iy1jgmhQavMutBnCwcliKPgMUUGAlAiTlThJEIFTljuootYHukscGBKL/5NI1f/gO8bZsFWL5nEmNXQk/PqIYFzckSU4sJMFqKiImqSuMmCDQn5lmcv8wzbI5SV0NP4R/5notfwdGIhZOq6HlP77TB6oJGDK3tXYQeYNJKb8HjmhEDohIF5ojdoJIgDb0BkqgpEqZ4TGhEm7aF3LaF/VIRz4DCHxyg/YMa14a9zstA5XxuRfCZEdpNyS3a3x3sNd06+ay7tsIBcnlZKBrnAoExjqmpV5JTdiOxcAvdvW+536/CWMjS1b08AJkRAttkL3Dn0iww6O4szwjE6r+kETeO6Ixo0YGPqOpipCEwxQ5F1H+gPhmAz1Y5JqMR3NQ3llGdMHu/WqLcHzi0rIUkArx6BqwX8suVofhA== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(376014)(82310400026)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2024 08:38:55.8054 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 28b5747f-aa6f-48ae-9d6a-08dd02f5720b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E1.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7734 Add PRE_COPY support for live migration. This functionality may reduce the downtime upon STOP_COPY as of letting the target machine to get some 'initial data' from the source once the machine is still in its RUNNING state and let it prepares itself pre-ahead to get the final STOP_COPY data. As the Virtio specification does not support reading partial or incremental device contexts. This means that during the PRE_COPY state, the vfio-virtio driver reads the full device state. As the device state can be changed and the benefit is highest when the pre copy data closely matches the final data we read it in a rate limiter mode. This means we avoid reading new data from the device for a specified time interval after the last read. With PRE_COPY enabled, we observed a downtime reduction of approximately 70-75% in various scenarios compared to when PRE_COPY was disabled, while keeping the total migration time nearly the same. Signed-off-by: Yishai Hadas --- drivers/vfio/pci/virtio/common.h | 4 + drivers/vfio/pci/virtio/migrate.c | 231 ++++++++++++++++++++++++++++-- 2 files changed, 227 insertions(+), 8 deletions(-) diff --git a/drivers/vfio/pci/virtio/common.h b/drivers/vfio/pci/virtio/common.h index 3bdfb3ea1174..5704603f0f9d 100644 --- a/drivers/vfio/pci/virtio/common.h +++ b/drivers/vfio/pci/virtio/common.h @@ -10,6 +10,8 @@ enum virtiovf_migf_state { VIRTIOVF_MIGF_STATE_ERROR = 1, + VIRTIOVF_MIGF_STATE_PRECOPY = 2, + VIRTIOVF_MIGF_STATE_COMPLETE = 3, }; enum virtiovf_load_state { @@ -57,6 +59,8 @@ struct virtiovf_migration_file { /* synchronize access to the file state */ struct mutex lock; loff_t max_pos; + u64 pre_copy_initial_bytes; + struct ratelimit_state pre_copy_rl_state; u64 record_size; u32 record_tag; u8 has_obj_id:1; diff --git a/drivers/vfio/pci/virtio/migrate.c b/drivers/vfio/pci/virtio/migrate.c index 3d5eaa1cbcdb..70fae7de11e9 100644 --- a/drivers/vfio/pci/virtio/migrate.c +++ b/drivers/vfio/pci/virtio/migrate.c @@ -26,6 +26,10 @@ /* Initial target buffer size */ #define VIRTIOVF_TARGET_INITIAL_BUF_SIZE SZ_1M +static int +virtiovf_read_device_context_chunk(struct virtiovf_migration_file *migf, + u32 ctx_size); + static struct page * virtiovf_get_migration_page(struct virtiovf_data_buffer *buf, unsigned long offset) @@ -155,6 +159,41 @@ virtiovf_pci_free_obj_id(struct virtiovf_pci_core_device *virtvdev, u32 obj_id) VIRTIO_RESOURCE_OBJ_DEV_PARTS, obj_id); } +static struct virtiovf_data_buffer * +virtiovf_get_data_buffer(struct virtiovf_migration_file *migf, size_t length) +{ + struct virtiovf_data_buffer *buf, *temp_buf; + struct list_head free_list; + + INIT_LIST_HEAD(&free_list); + + spin_lock_irq(&migf->list_lock); + list_for_each_entry_safe(buf, temp_buf, &migf->avail_list, buf_elm) { + list_del_init(&buf->buf_elm); + if (buf->allocated_length >= length) { + spin_unlock_irq(&migf->list_lock); + goto found; + } + /* + * Prevent holding redundant buffers. Put in a free + * list and call at the end not under the spin lock + * (&migf->list_lock) to minimize its scope usage. + */ + list_add(&buf->buf_elm, &free_list); + } + spin_unlock_irq(&migf->list_lock); + buf = virtiovf_alloc_data_buffer(migf, length); + +found: + while ((temp_buf = list_first_entry_or_null(&free_list, + struct virtiovf_data_buffer, buf_elm))) { + list_del(&temp_buf->buf_elm); + virtiovf_free_data_buffer(temp_buf); + } + + return buf; +} + static void virtiovf_clean_migf_resources(struct virtiovf_migration_file *migf) { struct virtiovf_data_buffer *entry; @@ -341,6 +380,7 @@ static ssize_t virtiovf_save_read(struct file *filp, char __user *buf, size_t le { struct virtiovf_migration_file *migf = filp->private_data; struct virtiovf_data_buffer *vhca_buf; + bool first_loop_call = true; bool end_of_data; ssize_t done = 0; @@ -358,6 +398,19 @@ static ssize_t virtiovf_save_read(struct file *filp, char __user *buf, size_t le ssize_t count; vhca_buf = virtiovf_get_data_buff_from_pos(migf, *pos, &end_of_data); + if (first_loop_call) { + first_loop_call = false; + /* Temporary end of file as part of PRE_COPY */ + if (end_of_data && migf->state == VIRTIOVF_MIGF_STATE_PRECOPY) { + done = -ENOMSG; + goto out_unlock; + } + if (end_of_data && migf->state != VIRTIOVF_MIGF_STATE_COMPLETE) { + done = -EINVAL; + goto out_unlock; + } + } + if (end_of_data) goto out_unlock; @@ -379,9 +432,101 @@ static ssize_t virtiovf_save_read(struct file *filp, char __user *buf, size_t le return done; } +static long virtiovf_precopy_ioctl(struct file *filp, unsigned int cmd, + unsigned long arg) +{ + struct virtiovf_migration_file *migf = filp->private_data; + struct virtiovf_pci_core_device *virtvdev = migf->virtvdev; + struct vfio_precopy_info info = {}; + loff_t *pos = &filp->f_pos; + bool end_of_data = false; + unsigned long minsz; + u32 ctx_size = 0; + int ret; + + if (cmd != VFIO_MIG_GET_PRECOPY_INFO) + return -ENOTTY; + + minsz = offsetofend(struct vfio_precopy_info, dirty_bytes); + if (copy_from_user(&info, (void __user *)arg, minsz)) + return -EFAULT; + + if (info.argsz < minsz) + return -EINVAL; + + mutex_lock(&virtvdev->state_mutex); + if (virtvdev->mig_state != VFIO_DEVICE_STATE_PRE_COPY && + virtvdev->mig_state != VFIO_DEVICE_STATE_PRE_COPY_P2P) { + ret = -EINVAL; + goto err_state_unlock; + } + + /* + * The virtio specification does not include a PRE_COPY concept. + * Since we can expect the data to remain the same for a certain period, + * we use a rate limiter mechanism before making a call to the device. + */ + if (__ratelimit(&migf->pre_copy_rl_state)) { + + ret = virtio_pci_admin_dev_parts_metadata_get(virtvdev->core_device.pdev, + VIRTIO_RESOURCE_OBJ_DEV_PARTS, migf->obj_id, + VIRTIO_ADMIN_CMD_DEV_PARTS_METADATA_TYPE_SIZE, + &ctx_size); + if (ret) + goto err_state_unlock; + } + + mutex_lock(&migf->lock); + if (migf->state == VIRTIOVF_MIGF_STATE_ERROR) { + ret = -ENODEV; + goto err_migf_unlock; + } + + if (migf->pre_copy_initial_bytes > *pos) { + info.initial_bytes = migf->pre_copy_initial_bytes - *pos; + } else { + info.dirty_bytes = migf->max_pos - *pos; + if (!info.dirty_bytes) + end_of_data = true; + info.dirty_bytes += ctx_size; + } + + if (!end_of_data || !ctx_size) { + mutex_unlock(&migf->lock); + goto done; + } + + mutex_unlock(&migf->lock); + /* + * We finished transferring the current state and the device has a + * dirty state, read a new state. + */ + ret = virtiovf_read_device_context_chunk(migf, ctx_size); + if (ret) + /* + * The machine is running, and context size could be grow, so no reason to mark + * the device state as VIRTIOVF_MIGF_STATE_ERROR. + */ + goto err_state_unlock; + +done: + virtiovf_state_mutex_unlock(virtvdev); + if (copy_to_user((void __user *)arg, &info, minsz)) + return -EFAULT; + return 0; + +err_migf_unlock: + mutex_unlock(&migf->lock); +err_state_unlock: + virtiovf_state_mutex_unlock(virtvdev); + return ret; +} + static const struct file_operations virtiovf_save_fops = { .owner = THIS_MODULE, .read = virtiovf_save_read, + .unlocked_ioctl = virtiovf_precopy_ioctl, + .compat_ioctl = compat_ptr_ioctl, .release = virtiovf_release_file, }; @@ -425,7 +570,7 @@ virtiovf_read_device_context_chunk(struct virtiovf_migration_file *migf, int nent; int ret; - buf = virtiovf_alloc_data_buffer(migf, ctx_size); + buf = virtiovf_get_data_buffer(migf, ctx_size); if (IS_ERR(buf)) return PTR_ERR(buf); @@ -460,7 +605,7 @@ virtiovf_read_device_context_chunk(struct virtiovf_migration_file *migf, goto out; buf->length = res_size; - header_buf = virtiovf_alloc_data_buffer(migf, + header_buf = virtiovf_get_data_buffer(migf, sizeof(struct virtiovf_migration_header)); if (IS_ERR(header_buf)) { ret = PTR_ERR(header_buf); @@ -485,8 +630,43 @@ virtiovf_read_device_context_chunk(struct virtiovf_migration_file *migf, return ret; } +static int +virtiovf_pci_save_device_final_data(struct virtiovf_pci_core_device *virtvdev) +{ + struct virtiovf_migration_file *migf = virtvdev->saving_migf; + u32 ctx_size; + int ret; + + if (migf->state == VIRTIOVF_MIGF_STATE_ERROR) + return -ENODEV; + + ret = virtio_pci_admin_dev_parts_metadata_get(virtvdev->core_device.pdev, + VIRTIO_RESOURCE_OBJ_DEV_PARTS, migf->obj_id, + VIRTIO_ADMIN_CMD_DEV_PARTS_METADATA_TYPE_SIZE, + &ctx_size); + if (ret) + goto err; + + if (!ctx_size) { + ret = -EINVAL; + goto err; + } + + ret = virtiovf_read_device_context_chunk(migf, ctx_size); + if (ret) + goto err; + + migf->state = VIRTIOVF_MIGF_STATE_COMPLETE; + return 0; + +err: + migf->state = VIRTIOVF_MIGF_STATE_ERROR; + return ret; +} + static struct virtiovf_migration_file * -virtiovf_pci_save_device_data(struct virtiovf_pci_core_device *virtvdev) +virtiovf_pci_save_device_data(struct virtiovf_pci_core_device *virtvdev, + bool pre_copy) { struct virtiovf_migration_file *migf; u32 ctx_size; @@ -536,6 +716,18 @@ virtiovf_pci_save_device_data(struct virtiovf_pci_core_device *virtvdev) if (ret) goto out_clean; + if (pre_copy) { + migf->pre_copy_initial_bytes = migf->max_pos; + /* Arbitrarily set the pre-copy rate limit to 1-second intervals */ + ratelimit_state_init(&migf->pre_copy_rl_state, 1 * HZ, 1); + /* Prevent any rate messages upon its usage */ + ratelimit_set_flags(&migf->pre_copy_rl_state, + RATELIMIT_MSG_ON_RELEASE); + migf->state = VIRTIOVF_MIGF_STATE_PRECOPY; + } else { + migf->state = VIRTIOVF_MIGF_STATE_COMPLETE; + } + return migf; out_clean: @@ -948,7 +1140,8 @@ virtiovf_pci_step_device_state_locked(struct virtiovf_pci_core_device *virtvdev, return NULL; } - if (cur == VFIO_DEVICE_STATE_RUNNING && new == VFIO_DEVICE_STATE_RUNNING_P2P) { + if ((cur == VFIO_DEVICE_STATE_RUNNING && new == VFIO_DEVICE_STATE_RUNNING_P2P) || + (cur == VFIO_DEVICE_STATE_PRE_COPY && new == VFIO_DEVICE_STATE_PRE_COPY_P2P)) { ret = virtio_pci_admin_mode_set(virtvdev->core_device.pdev, BIT(VIRTIO_ADMIN_CMD_DEV_MODE_F_STOPPED)); if (ret) @@ -956,7 +1149,8 @@ virtiovf_pci_step_device_state_locked(struct virtiovf_pci_core_device *virtvdev, return NULL; } - if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && new == VFIO_DEVICE_STATE_RUNNING) { + if ((cur == VFIO_DEVICE_STATE_RUNNING_P2P && new == VFIO_DEVICE_STATE_RUNNING) || + (cur == VFIO_DEVICE_STATE_PRE_COPY_P2P && new == VFIO_DEVICE_STATE_PRE_COPY)) { ret = virtio_pci_admin_mode_set(virtvdev->core_device.pdev, 0); if (ret) return ERR_PTR(ret); @@ -966,7 +1160,7 @@ virtiovf_pci_step_device_state_locked(struct virtiovf_pci_core_device *virtvdev, if (cur == VFIO_DEVICE_STATE_STOP && new == VFIO_DEVICE_STATE_STOP_COPY) { struct virtiovf_migration_file *migf; - migf = virtiovf_pci_save_device_data(virtvdev); + migf = virtiovf_pci_save_device_data(virtvdev, false); if (IS_ERR(migf)) return ERR_CAST(migf); get_file(migf->filp); @@ -974,7 +1168,9 @@ virtiovf_pci_step_device_state_locked(struct virtiovf_pci_core_device *virtvdev, return migf->filp; } - if (cur == VFIO_DEVICE_STATE_STOP_COPY && new == VFIO_DEVICE_STATE_STOP) { + if ((cur == VFIO_DEVICE_STATE_STOP_COPY && new == VFIO_DEVICE_STATE_STOP) || + (cur == VFIO_DEVICE_STATE_PRE_COPY && new == VFIO_DEVICE_STATE_RUNNING) || + (cur == VFIO_DEVICE_STATE_PRE_COPY_P2P && new == VFIO_DEVICE_STATE_RUNNING_P2P)) { virtiovf_disable_fds(virtvdev); return NULL; } @@ -995,6 +1191,24 @@ virtiovf_pci_step_device_state_locked(struct virtiovf_pci_core_device *virtvdev, return NULL; } + if ((cur == VFIO_DEVICE_STATE_RUNNING && new == VFIO_DEVICE_STATE_PRE_COPY) || + (cur == VFIO_DEVICE_STATE_RUNNING_P2P && + new == VFIO_DEVICE_STATE_PRE_COPY_P2P)) { + struct virtiovf_migration_file *migf; + + migf = virtiovf_pci_save_device_data(virtvdev, true); + if (IS_ERR(migf)) + return ERR_CAST(migf); + get_file(migf->filp); + virtvdev->saving_migf = migf; + return migf->filp; + } + + if (cur == VFIO_DEVICE_STATE_PRE_COPY_P2P && new == VFIO_DEVICE_STATE_STOP_COPY) { + ret = virtiovf_pci_save_device_final_data(virtvdev); + return ret ? ERR_PTR(ret) : NULL; + } + /* * vfio_mig_get_next_state() does not use arcs other than the above */ @@ -1099,7 +1313,8 @@ void virtiovf_set_migratable(struct virtiovf_pci_core_device *virtvdev) spin_lock_init(&virtvdev->reset_lock); virtvdev->core_device.vdev.migration_flags = VFIO_MIGRATION_STOP_COPY | - VFIO_MIGRATION_P2P; + VFIO_MIGRATION_P2P | + VFIO_MIGRATION_PRE_COPY; virtvdev->core_device.vdev.mig_ops = &virtvdev_pci_mig_ops; } From patchwork Tue Nov 12 08:37:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 13871915 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2049.outbound.protection.outlook.com [40.107.237.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E692120E318 for ; Tue, 12 Nov 2024 08:39:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.49 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400752; cv=fail; b=ot6ivKvF7Tizee48xsmmVmWzrKUHrd5ZP0+LAAzoh6xpFoHyg+sO3ZVBNzo5F8sxe5Q7BNS/X2iHXdAAm18Muk9Uv0Fi1QhOlSrD979V2VFIEwvggegp/bjUtpGthltnNZOgqJzA/CQsER+rIAsqaaBBttBzLwKp0isNhhJ+KZQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400752; c=relaxed/simple; bh=xbi/rAN564yv73rDVjPnGYviuv/7tzCL2povQWhBOqI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=IZqjRf26RmblxwD4nt0vj5eqVq2oPYsSCsd/gImER+aJEoDTKudkmCNOEPQk6lAa4l+diQuoCCkUhxJHP6reEjGU2In+Enh4mB5DUkfM0PoAMhsYeIL/VSJWRROjCNIQD63JEx1eVaBWfG8XEViScft5d8/rOdJHk5eg7E8JXwU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=DWggzu9H; arc=fail smtp.client-ip=40.107.237.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="DWggzu9H" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JkN7nnfP28uS0J/FaH+AE3/nRuhYI85PfrXpwWn/opVkswg6BFyT0qbUHfcXaa2tZiO4ATVwcdI/QRAPdICWB8/X57PoZY5fV+dMxbo7+S2Ua/k8tqBIbBODvAnZtbcSaYlWmTKmU68/frTIvPZCxrYNz3HPTR99x1HNnKO07MkPPFzYxk0fZ1fJHrBRxDSg302S3G89ubP3hCQw3MeNav/+dywTktCYxBrFM8nLX5zeW9sXgLOLacW3PVvpAwHy/ZmCrXcvm372HXGZbIGwPxmv8Meu+Q+FB+k0UGv41JDiTRipAT2JFH2YmO3TRZ00eOyj3+wGmMKLw6dnS/ljlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tc35ZbhiG9XbcVesmk5U7VillT2nU/txSmLz9VsakcI=; b=GegR6GifzxmE/49o4Sz3wCZLCIDGejM9Kwn2OwLSMvQX3TZj63b2Mj7ZDZ3finbL91/F8Abn4Nv6s/tTruLme3xS+etZS0K8N1Zg0oMyy+HXy/ZiFZa3eA+PWcICuwBRoHb8aBpjdzavx0zhjMpfNB/4M1q79v9FLcp4A6F3gVYPVXhQmoSgzyUo2SvMfJpV2FBqCPJTJZ9EO3k4D836j5JpwMpxhpdT5UkuLOmkMRiYY2VNgW4ZkdbFSPDzH9jvJNFQbY45Jj7Ux+SpE9Sg4Bw8wPrNiYauN1nZvVv2qWbYgTELbtuYAaORxrnEg0wrkJaTABfTxexAKS+OML7JJw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tc35ZbhiG9XbcVesmk5U7VillT2nU/txSmLz9VsakcI=; b=DWggzu9HSKKXTil47MyTKOpg8ARglhKpy8JIjSCJ2R5SSIITUg6jfcuGjW8WmJxodbGuE01UewX4eUM5ryPEo1hEjX633X8fdm0t/4txnumgnxaNcDbeUzABM9Of1eJ4JOzh7yccM0TNKNUhiPzMjCXUNR5kWwjmkHQUeKbkWrJz7dEFn/wWG22W08FQtZHvWG/oGpm58NCBVjrVQ347J5f0gfV3vXNsoWqqG+OzghE9pKTaNN19pmXKETs9iYLeJeDoWqPw2t7W23WemkhvzqoBQ3LKxLQx+pt5ZGQDwBEvzhI8AzV9+MaBUg91PRq9QOTQULveKpcvMIwjHLiu7Q== Received: from MN2PR15CA0065.namprd15.prod.outlook.com (2603:10b6:208:237::34) by MN2PR12MB4270.namprd12.prod.outlook.com (2603:10b6:208:1d9::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.17; Tue, 12 Nov 2024 08:39:01 +0000 Received: from MN1PEPF0000F0E2.namprd04.prod.outlook.com (2603:10b6:208:237:cafe::17) by MN2PR15CA0065.outlook.office365.com (2603:10b6:208:237::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8137.29 via Frontend Transport; Tue, 12 Nov 2024 08:39:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by MN1PEPF0000F0E2.mail.protection.outlook.com (10.167.242.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.14 via Frontend Transport; Tue, 12 Nov 2024 08:39:00 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:43 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 12 Nov 2024 00:38:43 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 12 Nov 2024 00:38:40 -0800 From: Yishai Hadas To: , , , CC: , , , , , , , , Subject: [PATCH V3 vfio 7/7] vfio/virtio: Enable live migration once VIRTIO_PCI was configured Date: Tue, 12 Nov 2024 10:37:29 +0200 Message-ID: <20241112083729.145005-8-yishaih@nvidia.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20241112083729.145005-1-yishaih@nvidia.com> References: <20241112083729.145005-1-yishaih@nvidia.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E2:EE_|MN2PR12MB4270:EE_ X-MS-Office365-Filtering-Correlation-Id: 92e7f07d-ca08-4113-592b-08dd02f574ff X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: JwGhObOtlro3Zpc7TQeCHY9WNyaGJYaAKrWelVfvtcvZEczggYJNsVa4tpbfjhtvSjjJzDcGO2CeKMX/geG34ZJc4gT5Hb1AVj6dAWPwDzfiPgl71rVPvU+vm5dUEtjI+rWzq9QSqFyJFcT6NeswaIK9BczlV1p16h9wRMKOQK0S9HnNg3j8Q97TyVMJwilvJEIMmH7X2A7EZY2m5i9mDefnR8nyvTQqj9RWz2TXYTGKvQPKKz1be6qqCRZa6pGgbY1jMAGPgIIxZyaubIIgBVpIUylWiWqNdEMpCC0igVxxVOQwdOYaj89mn36G9Y4ZYPmaGMYaejd4pkc3Ufu5s+I/p4XGEwHiQg96CjD1giItptbR5uK4kV0GAHZZEZHYwBoE4qiV17UGd9octV7PGqLyfCi8bkqFv5ILynRFxdO7QbaO1+XHLPYpWEwoKloON8SlrYt2Q015qDW70m4RbisAVrWh6u9GJ797eHetuHBgKvSt6zAbRP/ZFHKjyXikEEQ1yf6rYgOPo2UChk+LZSMeEmS4LpEj/KHp+FUxdwW+ivFLVA3KA7qqHwwRQP956N0ojnSMKUPqDE1Qqyfh6QjSGNu8MrYLOhuh0wVhq/8EJsBXCdybG4PZBRoU+LH89vivn+ikC/WGkNN2kUsvE7JY+JX0SpjbJcy9H8aZ1Dbfn73xvMUA0VnMP6R7yD7C0axQx856cX2WShoWp0ISc/kqWlGaGAaF4DdQHDaElkhCWF6wwb29eddmXq4HwMCwhvbrAEcP4SrbqGNVCnK03t0I23Tv+ZkBmwGUSOhsDrde4unMTCVxdo/1qDdpPZi4wpuKHfP6oypQHwAXhWx/rplLiCO6c9v7i02SXwIxntqtFUk0ZO0EzToznJ8XjwO8Q01ltalPni1z9tNkbsyrgEHpkmNSgaTfb4HtSvLL9zbwY/SOWlRe4rHiNGmCP+o+kUEeo8e+Ms3Hm45WcM+erY6Sg3BjmG+8KVkidw0KA8Ylo1SV82pOLQqIXJano8gvj+lRO31/jIUikr+vF3O37FExo9GIQqPx47ZQUOwfj8tFiVmf480Rs5f028wD1oN0BG4HznY7+DnFjz6fNpjpMOeA/GlqRlnksrJ4ix4J8KW+b9KAGajpLSoI2k0a+/J8mfHVRsF8cu19HKxn86iC9jlJpYPfSFShneLDd7ZCEJcwqiW41si40lvRCjptlyRAjf4sIbSnrLRYfCtPAMawdAvaTE8fFMdSFaGJNx/bqYE2sJXsGOaNpcoI9S6vEHiqAyjLytDq3UU09PVGaAEZgCN/AeXsKcwEopwRX2JWn1DJOB0yVXqRReaNk1PWvGHZKlzwbG0YJOX5SYgIRsxkQUCUu1y3+vkTU3HuEsQTbe+FTXtuI1pdRe9kUEvVQnZioxsNmy44OePDHdcvS2n8vg== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2024 08:39:00.7581 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 92e7f07d-ca08-4113-592b-08dd02f574ff X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E2.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4270 Now that the driver supports live migration, only the legacy IO functionality depends on config VIRTIO_PCI_ADMIN_LEGACY. As part of that we introduce a bool configuration option as a sub menu under the driver's main live migration feature named VIRTIO_VFIO_PCI_ADMIN_LEGACY, to control the legacy IO functionality. This will let users configuring the kernel, know which features from the description might be available in the resulting driver. As of that, move the legacy IO into a separate file to be compiled only once CONFIG_VIRTIO_VFIO_PCI_ADMIN_LEGACY was configured and let the live migration depends only on VIRTIO_PCI. Signed-off-by: Yishai Hadas --- drivers/vfio/pci/virtio/Kconfig | 42 ++- drivers/vfio/pci/virtio/Makefile | 1 + drivers/vfio/pci/virtio/common.h | 19 ++ drivers/vfio/pci/virtio/legacy_io.c | 418 ++++++++++++++++++++++++++++ drivers/vfio/pci/virtio/main.c | 410 ++------------------------- 5 files changed, 489 insertions(+), 401 deletions(-) create mode 100644 drivers/vfio/pci/virtio/legacy_io.c diff --git a/drivers/vfio/pci/virtio/Kconfig b/drivers/vfio/pci/virtio/Kconfig index bd80eca4a196..2770f7eb702c 100644 --- a/drivers/vfio/pci/virtio/Kconfig +++ b/drivers/vfio/pci/virtio/Kconfig @@ -1,15 +1,31 @@ # SPDX-License-Identifier: GPL-2.0-only config VIRTIO_VFIO_PCI - tristate "VFIO support for VIRTIO NET PCI devices" - depends on VIRTIO_PCI && VIRTIO_PCI_ADMIN_LEGACY - select VFIO_PCI_CORE - help - This provides support for exposing VIRTIO NET VF devices which support - legacy IO access, using the VFIO framework that can work with a legacy - virtio driver in the guest. - Based on PCIe spec, VFs do not support I/O Space. - As of that this driver emulates I/O BAR in software to let a VF be - seen as a transitional device by its users and let it work with - a legacy driver. - - If you don't know what to do here, say N. + tristate "VFIO support for VIRTIO NET PCI VF devices" + depends on VIRTIO_PCI + select VFIO_PCI_CORE + help + This provides migration support for VIRTIO NET PCI VF devices + using the VFIO framework. Migration support requires the + SR-IOV PF device to support specific VIRTIO extensions, + otherwise this driver provides no additional functionality + beyond vfio-pci. + + Migration support in this driver relies on dirty page tracking + provided by the IOMMU hardware and exposed through IOMMUFD, any + other use cases are dis-recommended. + + If you don't know what to do here, say N. + +config VIRTIO_VFIO_PCI_ADMIN_LEGACY + bool "Legacy I/O support for VIRTIO NET PCI VF devices" + depends on VIRTIO_VFIO_PCI && VIRTIO_PCI_ADMIN_LEGACY + default y + help + This extends the virtio-vfio-pci driver to support legacy I/O + access, allowing use of legacy virtio drivers with VIRTIO NET + PCI VF devices. Legacy I/O support requires the SR-IOV PF + device to support and enable specific VIRTIO extensions, + otherwise this driver provides no additional functionality + beyond vfio-pci. + + If you don't know what to do here, say N. diff --git a/drivers/vfio/pci/virtio/Makefile b/drivers/vfio/pci/virtio/Makefile index bf0ccde6a91a..d9b0bb40d6b3 100644 --- a/drivers/vfio/pci/virtio/Makefile +++ b/drivers/vfio/pci/virtio/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_VIRTIO_VFIO_PCI) += virtio-vfio-pci.o virtio-vfio-pci-y := main.o migrate.o +virtio-vfio-pci-$(CONFIG_VIRTIO_VFIO_PCI_ADMIN_LEGACY) += legacy_io.o diff --git a/drivers/vfio/pci/virtio/common.h b/drivers/vfio/pci/virtio/common.h index 5704603f0f9d..c7d7e27af386 100644 --- a/drivers/vfio/pci/virtio/common.h +++ b/drivers/vfio/pci/virtio/common.h @@ -78,6 +78,7 @@ struct virtiovf_migration_file { struct virtiovf_pci_core_device { struct vfio_pci_core_device core_device; +#ifdef CONFIG_VIRTIO_VFIO_PCI_ADMIN_LEGACY u8 *bar0_virtual_buf; /* synchronize access to the virtual buf */ struct mutex bar_mutex; @@ -87,6 +88,7 @@ struct virtiovf_pci_core_device { __le16 pci_cmd; u8 bar0_virtual_buf_size; u8 notify_bar; +#endif /* LM related */ u8 migrate_cap:1; @@ -105,4 +107,21 @@ void virtiovf_open_migration(struct virtiovf_pci_core_device *virtvdev); void virtiovf_close_migration(struct virtiovf_pci_core_device *virtvdev); void virtiovf_migration_reset_done(struct pci_dev *pdev); +#ifdef CONFIG_VIRTIO_VFIO_PCI_ADMIN_LEGACY +int virtiovf_open_legacy_io(struct virtiovf_pci_core_device *virtvdev); +long virtiovf_vfio_pci_core_ioctl(struct vfio_device *core_vdev, + unsigned int cmd, unsigned long arg); +int virtiovf_pci_ioctl_get_region_info(struct vfio_device *core_vdev, + unsigned int cmd, unsigned long arg); +ssize_t virtiovf_pci_core_write(struct vfio_device *core_vdev, + const char __user *buf, size_t count, + loff_t *ppos); +ssize_t virtiovf_pci_core_read(struct vfio_device *core_vdev, char __user *buf, + size_t count, loff_t *ppos); +bool virtiovf_support_legacy_io(struct pci_dev *pdev); +int virtiovf_init_legacy_io(struct virtiovf_pci_core_device *virtvdev); +void virtiovf_release_legacy_io(struct virtiovf_pci_core_device *virtvdev); +void virtiovf_legacy_io_reset_done(struct pci_dev *pdev); +#endif + #endif /* VIRTIO_VFIO_COMMON_H */ diff --git a/drivers/vfio/pci/virtio/legacy_io.c b/drivers/vfio/pci/virtio/legacy_io.c new file mode 100644 index 000000000000..20382ee15fac --- /dev/null +++ b/drivers/vfio/pci/virtio/legacy_io.c @@ -0,0 +1,418 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "common.h" + +static int +virtiovf_issue_legacy_rw_cmd(struct virtiovf_pci_core_device *virtvdev, + loff_t pos, char __user *buf, + size_t count, bool read) +{ + bool msix_enabled = + (virtvdev->core_device.irq_type == VFIO_PCI_MSIX_IRQ_INDEX); + struct pci_dev *pdev = virtvdev->core_device.pdev; + u8 *bar0_buf = virtvdev->bar0_virtual_buf; + bool common; + u8 offset; + int ret; + + common = pos < VIRTIO_PCI_CONFIG_OFF(msix_enabled); + /* offset within the relevant configuration area */ + offset = common ? pos : pos - VIRTIO_PCI_CONFIG_OFF(msix_enabled); + mutex_lock(&virtvdev->bar_mutex); + if (read) { + if (common) + ret = virtio_pci_admin_legacy_common_io_read(pdev, offset, + count, bar0_buf + pos); + else + ret = virtio_pci_admin_legacy_device_io_read(pdev, offset, + count, bar0_buf + pos); + if (ret) + goto out; + if (copy_to_user(buf, bar0_buf + pos, count)) + ret = -EFAULT; + } else { + if (copy_from_user(bar0_buf + pos, buf, count)) { + ret = -EFAULT; + goto out; + } + + if (common) + ret = virtio_pci_admin_legacy_common_io_write(pdev, offset, + count, bar0_buf + pos); + else + ret = virtio_pci_admin_legacy_device_io_write(pdev, offset, + count, bar0_buf + pos); + } +out: + mutex_unlock(&virtvdev->bar_mutex); + return ret; +} + +static int +virtiovf_pci_bar0_rw(struct virtiovf_pci_core_device *virtvdev, + loff_t pos, char __user *buf, + size_t count, bool read) +{ + struct vfio_pci_core_device *core_device = &virtvdev->core_device; + struct pci_dev *pdev = core_device->pdev; + u16 queue_notify; + int ret; + + if (!(le16_to_cpu(virtvdev->pci_cmd) & PCI_COMMAND_IO)) + return -EIO; + + if (pos + count > virtvdev->bar0_virtual_buf_size) + return -EINVAL; + + ret = pm_runtime_resume_and_get(&pdev->dev); + if (ret) { + pci_info_ratelimited(pdev, "runtime resume failed %d\n", ret); + return -EIO; + } + + switch (pos) { + case VIRTIO_PCI_QUEUE_NOTIFY: + if (count != sizeof(queue_notify)) { + ret = -EINVAL; + goto end; + } + if (read) { + ret = vfio_pci_core_ioread16(core_device, true, &queue_notify, + virtvdev->notify_addr); + if (ret) + goto end; + if (copy_to_user(buf, &queue_notify, + sizeof(queue_notify))) { + ret = -EFAULT; + goto end; + } + } else { + if (copy_from_user(&queue_notify, buf, count)) { + ret = -EFAULT; + goto end; + } + ret = vfio_pci_core_iowrite16(core_device, true, queue_notify, + virtvdev->notify_addr); + } + break; + default: + ret = virtiovf_issue_legacy_rw_cmd(virtvdev, pos, buf, count, + read); + } + +end: + pm_runtime_put(&pdev->dev); + return ret ? ret : count; +} + +static ssize_t virtiovf_pci_read_config(struct vfio_device *core_vdev, + char __user *buf, size_t count, + loff_t *ppos) +{ + struct virtiovf_pci_core_device *virtvdev = container_of( + core_vdev, struct virtiovf_pci_core_device, core_device.vdev); + loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK; + size_t register_offset; + loff_t copy_offset; + size_t copy_count; + __le32 val32; + __le16 val16; + u8 val8; + int ret; + + ret = vfio_pci_core_read(core_vdev, buf, count, ppos); + if (ret < 0) + return ret; + + if (vfio_pci_core_range_intersect_range(pos, count, PCI_DEVICE_ID, + sizeof(val16), ©_offset, + ©_count, ®ister_offset)) { + val16 = cpu_to_le16(VIRTIO_TRANS_ID_NET); + if (copy_to_user(buf + copy_offset, (void *)&val16 + register_offset, copy_count)) + return -EFAULT; + } + + if ((le16_to_cpu(virtvdev->pci_cmd) & PCI_COMMAND_IO) && + vfio_pci_core_range_intersect_range(pos, count, PCI_COMMAND, + sizeof(val16), ©_offset, + ©_count, ®ister_offset)) { + if (copy_from_user((void *)&val16 + register_offset, buf + copy_offset, + copy_count)) + return -EFAULT; + val16 |= cpu_to_le16(PCI_COMMAND_IO); + if (copy_to_user(buf + copy_offset, (void *)&val16 + register_offset, + copy_count)) + return -EFAULT; + } + + if (vfio_pci_core_range_intersect_range(pos, count, PCI_REVISION_ID, + sizeof(val8), ©_offset, + ©_count, ®ister_offset)) { + /* Transional needs to have revision 0 */ + val8 = 0; + if (copy_to_user(buf + copy_offset, &val8, copy_count)) + return -EFAULT; + } + + if (vfio_pci_core_range_intersect_range(pos, count, PCI_BASE_ADDRESS_0, + sizeof(val32), ©_offset, + ©_count, ®ister_offset)) { + u32 bar_mask = ~(virtvdev->bar0_virtual_buf_size - 1); + u32 pci_base_addr_0 = le32_to_cpu(virtvdev->pci_base_addr_0); + + val32 = cpu_to_le32((pci_base_addr_0 & bar_mask) | PCI_BASE_ADDRESS_SPACE_IO); + if (copy_to_user(buf + copy_offset, (void *)&val32 + register_offset, copy_count)) + return -EFAULT; + } + + if (vfio_pci_core_range_intersect_range(pos, count, PCI_SUBSYSTEM_ID, + sizeof(val16), ©_offset, + ©_count, ®ister_offset)) { + /* + * Transitional devices use the PCI subsystem device id as + * virtio device id, same as legacy driver always did. + */ + val16 = cpu_to_le16(VIRTIO_ID_NET); + if (copy_to_user(buf + copy_offset, (void *)&val16 + register_offset, + copy_count)) + return -EFAULT; + } + + if (vfio_pci_core_range_intersect_range(pos, count, PCI_SUBSYSTEM_VENDOR_ID, + sizeof(val16), ©_offset, + ©_count, ®ister_offset)) { + val16 = cpu_to_le16(PCI_VENDOR_ID_REDHAT_QUMRANET); + if (copy_to_user(buf + copy_offset, (void *)&val16 + register_offset, + copy_count)) + return -EFAULT; + } + + return count; +} + +ssize_t virtiovf_pci_core_read(struct vfio_device *core_vdev, char __user *buf, + size_t count, loff_t *ppos) +{ + struct virtiovf_pci_core_device *virtvdev = container_of( + core_vdev, struct virtiovf_pci_core_device, core_device.vdev); + unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos); + loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK; + + if (!count) + return 0; + + if (index == VFIO_PCI_CONFIG_REGION_INDEX) + return virtiovf_pci_read_config(core_vdev, buf, count, ppos); + + if (index == VFIO_PCI_BAR0_REGION_INDEX) + return virtiovf_pci_bar0_rw(virtvdev, pos, buf, count, true); + + return vfio_pci_core_read(core_vdev, buf, count, ppos); +} + +static ssize_t virtiovf_pci_write_config(struct vfio_device *core_vdev, + const char __user *buf, size_t count, + loff_t *ppos) +{ + struct virtiovf_pci_core_device *virtvdev = container_of( + core_vdev, struct virtiovf_pci_core_device, core_device.vdev); + loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK; + size_t register_offset; + loff_t copy_offset; + size_t copy_count; + + if (vfio_pci_core_range_intersect_range(pos, count, PCI_COMMAND, + sizeof(virtvdev->pci_cmd), + ©_offset, ©_count, + ®ister_offset)) { + if (copy_from_user((void *)&virtvdev->pci_cmd + register_offset, + buf + copy_offset, + copy_count)) + return -EFAULT; + } + + if (vfio_pci_core_range_intersect_range(pos, count, PCI_BASE_ADDRESS_0, + sizeof(virtvdev->pci_base_addr_0), + ©_offset, ©_count, + ®ister_offset)) { + if (copy_from_user((void *)&virtvdev->pci_base_addr_0 + register_offset, + buf + copy_offset, + copy_count)) + return -EFAULT; + } + + return vfio_pci_core_write(core_vdev, buf, count, ppos); +} + +ssize_t virtiovf_pci_core_write(struct vfio_device *core_vdev, const char __user *buf, + size_t count, loff_t *ppos) +{ + struct virtiovf_pci_core_device *virtvdev = container_of( + core_vdev, struct virtiovf_pci_core_device, core_device.vdev); + unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos); + loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK; + + if (!count) + return 0; + + if (index == VFIO_PCI_CONFIG_REGION_INDEX) + return virtiovf_pci_write_config(core_vdev, buf, count, ppos); + + if (index == VFIO_PCI_BAR0_REGION_INDEX) + return virtiovf_pci_bar0_rw(virtvdev, pos, (char __user *)buf, count, false); + + return vfio_pci_core_write(core_vdev, buf, count, ppos); +} + +int virtiovf_pci_ioctl_get_region_info(struct vfio_device *core_vdev, + unsigned int cmd, unsigned long arg) +{ + struct virtiovf_pci_core_device *virtvdev = container_of( + core_vdev, struct virtiovf_pci_core_device, core_device.vdev); + unsigned long minsz = offsetofend(struct vfio_region_info, offset); + void __user *uarg = (void __user *)arg; + struct vfio_region_info info = {}; + + if (copy_from_user(&info, uarg, minsz)) + return -EFAULT; + + if (info.argsz < minsz) + return -EINVAL; + + switch (info.index) { + case VFIO_PCI_BAR0_REGION_INDEX: + info.offset = VFIO_PCI_INDEX_TO_OFFSET(info.index); + info.size = virtvdev->bar0_virtual_buf_size; + info.flags = VFIO_REGION_INFO_FLAG_READ | + VFIO_REGION_INFO_FLAG_WRITE; + return copy_to_user(uarg, &info, minsz) ? -EFAULT : 0; + default: + return vfio_pci_core_ioctl(core_vdev, cmd, arg); + } +} + +long virtiovf_vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, + unsigned long arg) +{ + switch (cmd) { + case VFIO_DEVICE_GET_REGION_INFO: + return virtiovf_pci_ioctl_get_region_info(core_vdev, cmd, arg); + default: + return vfio_pci_core_ioctl(core_vdev, cmd, arg); + } +} + +static int virtiovf_set_notify_addr(struct virtiovf_pci_core_device *virtvdev) +{ + struct vfio_pci_core_device *core_device = &virtvdev->core_device; + int ret; + + /* + * Setup the BAR where the 'notify' exists to be used by vfio as well + * This will let us mmap it only once and use it when needed. + */ + ret = vfio_pci_core_setup_barmap(core_device, + virtvdev->notify_bar); + if (ret) + return ret; + + virtvdev->notify_addr = core_device->barmap[virtvdev->notify_bar] + + virtvdev->notify_offset; + return 0; +} + +int virtiovf_open_legacy_io(struct virtiovf_pci_core_device *virtvdev) +{ + if (!virtvdev->bar0_virtual_buf) + return 0; + + /* + * Upon close_device() the vfio_pci_core_disable() is called + * and will close all the previous mmaps, so it seems that the + * valid life cycle for the 'notify' addr is per open/close. + */ + return virtiovf_set_notify_addr(virtvdev); +} + +static int virtiovf_get_device_config_size(unsigned short device) +{ + /* Network card */ + return offsetofend(struct virtio_net_config, status); +} + +static int virtiovf_read_notify_info(struct virtiovf_pci_core_device *virtvdev) +{ + u64 offset; + int ret; + u8 bar; + + ret = virtio_pci_admin_legacy_io_notify_info(virtvdev->core_device.pdev, + VIRTIO_ADMIN_CMD_NOTIFY_INFO_FLAGS_OWNER_MEM, + &bar, &offset); + if (ret) + return ret; + + virtvdev->notify_bar = bar; + virtvdev->notify_offset = offset; + return 0; +} + +static bool virtiovf_bar0_exists(struct pci_dev *pdev) +{ + struct resource *res = pdev->resource; + + return res->flags; +} + +bool virtiovf_support_legacy_io(struct pci_dev *pdev) +{ + return virtio_pci_admin_has_legacy_io(pdev) && !virtiovf_bar0_exists(pdev); +} + +int virtiovf_init_legacy_io(struct virtiovf_pci_core_device *virtvdev) +{ + struct pci_dev *pdev = virtvdev->core_device.pdev; + int ret; + + ret = virtiovf_read_notify_info(virtvdev); + if (ret) + return ret; + + virtvdev->bar0_virtual_buf_size = VIRTIO_PCI_CONFIG_OFF(true) + + virtiovf_get_device_config_size(pdev->device); + BUILD_BUG_ON(!is_power_of_2(virtvdev->bar0_virtual_buf_size)); + virtvdev->bar0_virtual_buf = kzalloc(virtvdev->bar0_virtual_buf_size, + GFP_KERNEL); + if (!virtvdev->bar0_virtual_buf) + return -ENOMEM; + mutex_init(&virtvdev->bar_mutex); + return 0; +} + +void virtiovf_release_legacy_io(struct virtiovf_pci_core_device *virtvdev) +{ + kfree(virtvdev->bar0_virtual_buf); +} + +void virtiovf_legacy_io_reset_done(struct pci_dev *pdev) +{ + struct virtiovf_pci_core_device *virtvdev = dev_get_drvdata(&pdev->dev); + + virtvdev->pci_cmd = 0; +} diff --git a/drivers/vfio/pci/virtio/main.c b/drivers/vfio/pci/virtio/main.c index e9ae17209026..d534d48c4163 100644 --- a/drivers/vfio/pci/virtio/main.c +++ b/drivers/vfio/pci/virtio/main.c @@ -18,330 +18,6 @@ #include "common.h" -static int -virtiovf_issue_legacy_rw_cmd(struct virtiovf_pci_core_device *virtvdev, - loff_t pos, char __user *buf, - size_t count, bool read) -{ - bool msix_enabled = - (virtvdev->core_device.irq_type == VFIO_PCI_MSIX_IRQ_INDEX); - struct pci_dev *pdev = virtvdev->core_device.pdev; - u8 *bar0_buf = virtvdev->bar0_virtual_buf; - bool common; - u8 offset; - int ret; - - common = pos < VIRTIO_PCI_CONFIG_OFF(msix_enabled); - /* offset within the relevant configuration area */ - offset = common ? pos : pos - VIRTIO_PCI_CONFIG_OFF(msix_enabled); - mutex_lock(&virtvdev->bar_mutex); - if (read) { - if (common) - ret = virtio_pci_admin_legacy_common_io_read(pdev, offset, - count, bar0_buf + pos); - else - ret = virtio_pci_admin_legacy_device_io_read(pdev, offset, - count, bar0_buf + pos); - if (ret) - goto out; - if (copy_to_user(buf, bar0_buf + pos, count)) - ret = -EFAULT; - } else { - if (copy_from_user(bar0_buf + pos, buf, count)) { - ret = -EFAULT; - goto out; - } - - if (common) - ret = virtio_pci_admin_legacy_common_io_write(pdev, offset, - count, bar0_buf + pos); - else - ret = virtio_pci_admin_legacy_device_io_write(pdev, offset, - count, bar0_buf + pos); - } -out: - mutex_unlock(&virtvdev->bar_mutex); - return ret; -} - -static int -virtiovf_pci_bar0_rw(struct virtiovf_pci_core_device *virtvdev, - loff_t pos, char __user *buf, - size_t count, bool read) -{ - struct vfio_pci_core_device *core_device = &virtvdev->core_device; - struct pci_dev *pdev = core_device->pdev; - u16 queue_notify; - int ret; - - if (!(le16_to_cpu(virtvdev->pci_cmd) & PCI_COMMAND_IO)) - return -EIO; - - if (pos + count > virtvdev->bar0_virtual_buf_size) - return -EINVAL; - - ret = pm_runtime_resume_and_get(&pdev->dev); - if (ret) { - pci_info_ratelimited(pdev, "runtime resume failed %d\n", ret); - return -EIO; - } - - switch (pos) { - case VIRTIO_PCI_QUEUE_NOTIFY: - if (count != sizeof(queue_notify)) { - ret = -EINVAL; - goto end; - } - if (read) { - ret = vfio_pci_core_ioread16(core_device, true, &queue_notify, - virtvdev->notify_addr); - if (ret) - goto end; - if (copy_to_user(buf, &queue_notify, - sizeof(queue_notify))) { - ret = -EFAULT; - goto end; - } - } else { - if (copy_from_user(&queue_notify, buf, count)) { - ret = -EFAULT; - goto end; - } - ret = vfio_pci_core_iowrite16(core_device, true, queue_notify, - virtvdev->notify_addr); - } - break; - default: - ret = virtiovf_issue_legacy_rw_cmd(virtvdev, pos, buf, count, - read); - } - -end: - pm_runtime_put(&pdev->dev); - return ret ? ret : count; -} - -static ssize_t virtiovf_pci_read_config(struct vfio_device *core_vdev, - char __user *buf, size_t count, - loff_t *ppos) -{ - struct virtiovf_pci_core_device *virtvdev = container_of( - core_vdev, struct virtiovf_pci_core_device, core_device.vdev); - loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK; - size_t register_offset; - loff_t copy_offset; - size_t copy_count; - __le32 val32; - __le16 val16; - u8 val8; - int ret; - - ret = vfio_pci_core_read(core_vdev, buf, count, ppos); - if (ret < 0) - return ret; - - if (vfio_pci_core_range_intersect_range(pos, count, PCI_DEVICE_ID, - sizeof(val16), ©_offset, - ©_count, ®ister_offset)) { - val16 = cpu_to_le16(VIRTIO_TRANS_ID_NET); - if (copy_to_user(buf + copy_offset, (void *)&val16 + register_offset, copy_count)) - return -EFAULT; - } - - if ((le16_to_cpu(virtvdev->pci_cmd) & PCI_COMMAND_IO) && - vfio_pci_core_range_intersect_range(pos, count, PCI_COMMAND, - sizeof(val16), ©_offset, - ©_count, ®ister_offset)) { - if (copy_from_user((void *)&val16 + register_offset, buf + copy_offset, - copy_count)) - return -EFAULT; - val16 |= cpu_to_le16(PCI_COMMAND_IO); - if (copy_to_user(buf + copy_offset, (void *)&val16 + register_offset, - copy_count)) - return -EFAULT; - } - - if (vfio_pci_core_range_intersect_range(pos, count, PCI_REVISION_ID, - sizeof(val8), ©_offset, - ©_count, ®ister_offset)) { - /* Transional needs to have revision 0 */ - val8 = 0; - if (copy_to_user(buf + copy_offset, &val8, copy_count)) - return -EFAULT; - } - - if (vfio_pci_core_range_intersect_range(pos, count, PCI_BASE_ADDRESS_0, - sizeof(val32), ©_offset, - ©_count, ®ister_offset)) { - u32 bar_mask = ~(virtvdev->bar0_virtual_buf_size - 1); - u32 pci_base_addr_0 = le32_to_cpu(virtvdev->pci_base_addr_0); - - val32 = cpu_to_le32((pci_base_addr_0 & bar_mask) | PCI_BASE_ADDRESS_SPACE_IO); - if (copy_to_user(buf + copy_offset, (void *)&val32 + register_offset, copy_count)) - return -EFAULT; - } - - if (vfio_pci_core_range_intersect_range(pos, count, PCI_SUBSYSTEM_ID, - sizeof(val16), ©_offset, - ©_count, ®ister_offset)) { - /* - * Transitional devices use the PCI subsystem device id as - * virtio device id, same as legacy driver always did. - */ - val16 = cpu_to_le16(VIRTIO_ID_NET); - if (copy_to_user(buf + copy_offset, (void *)&val16 + register_offset, - copy_count)) - return -EFAULT; - } - - if (vfio_pci_core_range_intersect_range(pos, count, PCI_SUBSYSTEM_VENDOR_ID, - sizeof(val16), ©_offset, - ©_count, ®ister_offset)) { - val16 = cpu_to_le16(PCI_VENDOR_ID_REDHAT_QUMRANET); - if (copy_to_user(buf + copy_offset, (void *)&val16 + register_offset, - copy_count)) - return -EFAULT; - } - - return count; -} - -static ssize_t -virtiovf_pci_core_read(struct vfio_device *core_vdev, char __user *buf, - size_t count, loff_t *ppos) -{ - struct virtiovf_pci_core_device *virtvdev = container_of( - core_vdev, struct virtiovf_pci_core_device, core_device.vdev); - unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos); - loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK; - - if (!count) - return 0; - - if (index == VFIO_PCI_CONFIG_REGION_INDEX) - return virtiovf_pci_read_config(core_vdev, buf, count, ppos); - - if (index == VFIO_PCI_BAR0_REGION_INDEX) - return virtiovf_pci_bar0_rw(virtvdev, pos, buf, count, true); - - return vfio_pci_core_read(core_vdev, buf, count, ppos); -} - -static ssize_t virtiovf_pci_write_config(struct vfio_device *core_vdev, - const char __user *buf, size_t count, - loff_t *ppos) -{ - struct virtiovf_pci_core_device *virtvdev = container_of( - core_vdev, struct virtiovf_pci_core_device, core_device.vdev); - loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK; - size_t register_offset; - loff_t copy_offset; - size_t copy_count; - - if (vfio_pci_core_range_intersect_range(pos, count, PCI_COMMAND, - sizeof(virtvdev->pci_cmd), - ©_offset, ©_count, - ®ister_offset)) { - if (copy_from_user((void *)&virtvdev->pci_cmd + register_offset, - buf + copy_offset, - copy_count)) - return -EFAULT; - } - - if (vfio_pci_core_range_intersect_range(pos, count, PCI_BASE_ADDRESS_0, - sizeof(virtvdev->pci_base_addr_0), - ©_offset, ©_count, - ®ister_offset)) { - if (copy_from_user((void *)&virtvdev->pci_base_addr_0 + register_offset, - buf + copy_offset, - copy_count)) - return -EFAULT; - } - - return vfio_pci_core_write(core_vdev, buf, count, ppos); -} - -static ssize_t -virtiovf_pci_core_write(struct vfio_device *core_vdev, const char __user *buf, - size_t count, loff_t *ppos) -{ - struct virtiovf_pci_core_device *virtvdev = container_of( - core_vdev, struct virtiovf_pci_core_device, core_device.vdev); - unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos); - loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK; - - if (!count) - return 0; - - if (index == VFIO_PCI_CONFIG_REGION_INDEX) - return virtiovf_pci_write_config(core_vdev, buf, count, ppos); - - if (index == VFIO_PCI_BAR0_REGION_INDEX) - return virtiovf_pci_bar0_rw(virtvdev, pos, (char __user *)buf, count, false); - - return vfio_pci_core_write(core_vdev, buf, count, ppos); -} - -static int -virtiovf_pci_ioctl_get_region_info(struct vfio_device *core_vdev, - unsigned int cmd, unsigned long arg) -{ - struct virtiovf_pci_core_device *virtvdev = container_of( - core_vdev, struct virtiovf_pci_core_device, core_device.vdev); - unsigned long minsz = offsetofend(struct vfio_region_info, offset); - void __user *uarg = (void __user *)arg; - struct vfio_region_info info = {}; - - if (copy_from_user(&info, uarg, minsz)) - return -EFAULT; - - if (info.argsz < minsz) - return -EINVAL; - - switch (info.index) { - case VFIO_PCI_BAR0_REGION_INDEX: - info.offset = VFIO_PCI_INDEX_TO_OFFSET(info.index); - info.size = virtvdev->bar0_virtual_buf_size; - info.flags = VFIO_REGION_INFO_FLAG_READ | - VFIO_REGION_INFO_FLAG_WRITE; - return copy_to_user(uarg, &info, minsz) ? -EFAULT : 0; - default: - return vfio_pci_core_ioctl(core_vdev, cmd, arg); - } -} - -static long -virtiovf_vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, - unsigned long arg) -{ - switch (cmd) { - case VFIO_DEVICE_GET_REGION_INFO: - return virtiovf_pci_ioctl_get_region_info(core_vdev, cmd, arg); - default: - return vfio_pci_core_ioctl(core_vdev, cmd, arg); - } -} - -static int -virtiovf_set_notify_addr(struct virtiovf_pci_core_device *virtvdev) -{ - struct vfio_pci_core_device *core_device = &virtvdev->core_device; - int ret; - - /* - * Setup the BAR where the 'notify' exists to be used by vfio as well - * This will let us mmap it only once and use it when needed. - */ - ret = vfio_pci_core_setup_barmap(core_device, - virtvdev->notify_bar); - if (ret) - return ret; - - virtvdev->notify_addr = core_device->barmap[virtvdev->notify_bar] + - virtvdev->notify_offset; - return 0; -} - static int virtiovf_pci_open_device(struct vfio_device *core_vdev) { struct virtiovf_pci_core_device *virtvdev = container_of(core_vdev, @@ -353,18 +29,13 @@ static int virtiovf_pci_open_device(struct vfio_device *core_vdev) if (ret) return ret; - if (virtvdev->bar0_virtual_buf) { - /* - * Upon close_device() the vfio_pci_core_disable() is called - * and will close all the previous mmaps, so it seems that the - * valid life cycle for the 'notify' addr is per open/close. - */ - ret = virtiovf_set_notify_addr(virtvdev); - if (ret) { - vfio_pci_core_disable(vdev); - return ret; - } +#ifdef CONFIG_VIRTIO_VFIO_PCI_ADMIN_LEGACY + ret = virtiovf_open_legacy_io(virtvdev); + if (ret) { + vfio_pci_core_disable(vdev); + return ret; } +#endif virtiovf_open_migration(virtvdev); vfio_pci_core_finish_enable(vdev); @@ -380,66 +51,33 @@ static void virtiovf_pci_close_device(struct vfio_device *core_vdev) vfio_pci_core_close_device(core_vdev); } -static int virtiovf_get_device_config_size(unsigned short device) -{ - /* Network card */ - return offsetofend(struct virtio_net_config, status); -} - -static int virtiovf_read_notify_info(struct virtiovf_pci_core_device *virtvdev) -{ - u64 offset; - int ret; - u8 bar; - - ret = virtio_pci_admin_legacy_io_notify_info(virtvdev->core_device.pdev, - VIRTIO_ADMIN_CMD_NOTIFY_INFO_FLAGS_OWNER_MEM, - &bar, &offset); - if (ret) - return ret; - - virtvdev->notify_bar = bar; - virtvdev->notify_offset = offset; - return 0; -} - +#ifdef CONFIG_VIRTIO_VFIO_PCI_ADMIN_LEGACY static int virtiovf_pci_init_device(struct vfio_device *core_vdev) { struct virtiovf_pci_core_device *virtvdev = container_of(core_vdev, struct virtiovf_pci_core_device, core_device.vdev); - struct pci_dev *pdev; int ret; ret = vfio_pci_core_init_dev(core_vdev); if (ret) return ret; - pdev = virtvdev->core_device.pdev; /* * The vfio_device_ops.init() callback is set to virtiovf_pci_init_device() * only when legacy I/O is supported. Now, let's initialize it. */ - ret = virtiovf_read_notify_info(virtvdev); - if (ret) - return ret; - - virtvdev->bar0_virtual_buf_size = VIRTIO_PCI_CONFIG_OFF(true) + - virtiovf_get_device_config_size(pdev->device); - BUILD_BUG_ON(!is_power_of_2(virtvdev->bar0_virtual_buf_size)); - virtvdev->bar0_virtual_buf = kzalloc(virtvdev->bar0_virtual_buf_size, - GFP_KERNEL); - if (!virtvdev->bar0_virtual_buf) - return -ENOMEM; - mutex_init(&virtvdev->bar_mutex); - return 0; + return virtiovf_init_legacy_io(virtvdev); } +#endif static void virtiovf_pci_core_release_dev(struct vfio_device *core_vdev) { +#ifdef CONFIG_VIRTIO_VFIO_PCI_ADMIN_LEGACY struct virtiovf_pci_core_device *virtvdev = container_of(core_vdev, struct virtiovf_pci_core_device, core_device.vdev); - kfree(virtvdev->bar0_virtual_buf); + virtiovf_release_legacy_io(virtvdev); +#endif vfio_pci_core_release_dev(core_vdev); } @@ -462,6 +100,7 @@ static const struct vfio_device_ops virtiovf_vfio_pci_lm_ops = { .detach_ioas = vfio_iommufd_physical_detach_ioas, }; +#ifdef CONFIG_VIRTIO_VFIO_PCI_ADMIN_LEGACY static const struct vfio_device_ops virtiovf_vfio_pci_tran_lm_ops = { .name = "virtio-vfio-pci-trans-lm", .init = virtiovf_pci_init_device, @@ -480,6 +119,7 @@ static const struct vfio_device_ops virtiovf_vfio_pci_tran_lm_ops = { .attach_ioas = vfio_iommufd_physical_attach_ioas, .detach_ioas = vfio_iommufd_physical_detach_ioas, }; +#endif static const struct vfio_device_ops virtiovf_vfio_pci_ops = { .name = "virtio-vfio-pci", @@ -500,13 +140,6 @@ static const struct vfio_device_ops virtiovf_vfio_pci_ops = { .detach_ioas = vfio_iommufd_physical_detach_ioas, }; -static bool virtiovf_bar0_exists(struct pci_dev *pdev) -{ - struct resource *res = pdev->resource; - - return res->flags; -} - static int virtiovf_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { @@ -517,12 +150,13 @@ static int virtiovf_pci_probe(struct pci_dev *pdev, int ret; if (pdev->is_virtfn) { - sup_legacy_io = virtio_pci_admin_has_legacy_io(pdev) && - !virtiovf_bar0_exists(pdev); - sup_lm = virtio_pci_admin_has_dev_parts(pdev); +#ifdef CONFIG_VIRTIO_VFIO_PCI_ADMIN_LEGACY + sup_legacy_io = virtiovf_support_legacy_io(pdev); if (sup_legacy_io) ops = &virtiovf_vfio_pci_tran_lm_ops; - else if (sup_lm) +#endif + sup_lm = virtio_pci_admin_has_dev_parts(pdev); + if (sup_lm && !sup_legacy_io) ops = &virtiovf_vfio_pci_lm_ops; } @@ -562,9 +196,9 @@ MODULE_DEVICE_TABLE(pci, virtiovf_pci_table); static void virtiovf_pci_aer_reset_done(struct pci_dev *pdev) { - struct virtiovf_pci_core_device *virtvdev = dev_get_drvdata(&pdev->dev); - - virtvdev->pci_cmd = 0; +#ifdef CONFIG_VIRTIO_VFIO_PCI_ADMIN_LEGACY + virtiovf_legacy_io_reset_done(pdev); +#endif virtiovf_migration_reset_done(pdev); }