From patchwork Thu May 12 15:43:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 12847856 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 74296C433F5 for ; Thu, 12 May 2022 15:58:54 +0000 (UTC) Received: from localhost ([::1]:47850 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1npBDJ-0004jP-Gg for qemu-devel@archiver.kernel.org; Thu, 12 May 2022 11:58:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41898) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyZ-0006Ho-ML for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:44 -0400 Received: from mail-co1nam11on20616.outbound.protection.outlook.com ([2a01:111:f400:7eab::616]:48960 helo=NAM11-CO1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyV-0005J3-HT for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:39 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LEToiR/CFHFtMYFPfUw7PqP+VZ16MpFyAFN4OvGPAxYhK/SIO/ahBreUIgTzrm0U434huFLlh052t+6K+pWkZ7R9TIHfeaYelZvRguOQuSV6jAhgr6TcHaU1XMALNsFlC3CIL3zqi8YtJEHNjGN+Ws/2euNVodYurBi948MewhoQr/5+36XbURchSS4RyL5wXOFKTDuHIYJpd63k7MNHp0cEPrP/eRAp6QyUbOuGQr1B5W3RCYppz2vxeHjAQ79vVIP2E7uo2zWXCpfrJOEGA4LMyQtp9t2TBDdLVrNOf0ffQSwTSDAcQCZHwl7xb/Vm7JPlET53Lmbxk/Q1ABZRGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=o4tDCK35xj4UeJHjTZk7zUovGlNatgIvioisl45+zp8=; b=dBLZvkkFKK7Jbsd3z04iPuaXYPgIPrZ/9ivxSFoMSViFcyhI5SDHeCDPjH5HV9qP/jnpBuNLkkCX/qbvvCTKsKKoQhS4jUpdFF/R4Cpj3JO19qNMiGhrVcrIcT99BGjHcOAI0OyIqLnL+bUjaNPsQlzVVJYkVEy/NibCCAYc3F5aInVlPfr45PPBYoEuYmfwqfWKZrLQtC9UN+mA5wvHDvbUJkUi9rAnR1PyV5VsdFj1jQoXq3FoF/OBPLDBSWDAAHzXWiPRkVM0hqTkS4TKD3QhKsIicJppaJHrJG+FGfYnMSH6WSt9mv4y4dZHIbdP8SvcUxB+n/32odLBbh3umQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=o4tDCK35xj4UeJHjTZk7zUovGlNatgIvioisl45+zp8=; b=cgSfnwS7iyEwBK9NM411HLl4Ah/kpxFUkMPWfkF+ki9NXFXhxqSWT+RTlb9oSmBXJtYDRHjJLMHsXsKSSOdhVuEqFadeAOvlXC+Z58OeozUA2NeQViiLzEGcKc+8W20f7nBPzp6lic/Ck456tSRP0Y9rTbYdCphBnwajfYOdUP5UoU7BEmhUbb640+9tJpY8jfrCHdA4BgNlPa1qQmc4SNFodPxlcspMvQG8t6JDb5sfQhXPaJVctiy3g0cpjkKN9PCZ0FDIAO0l4jfwm1x5AnY3oyc5nSmEyc8atZEYlmR1jBynna/ZbgyWtdynhKEw3cjQfgY5LW+LZyaNHd8ddw== Received: from MW4PR03CA0233.namprd03.prod.outlook.com (2603:10b6:303:b9::28) by BN8PR12MB3585.namprd12.prod.outlook.com (2603:10b6:408:49::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.14; Thu, 12 May 2022 15:43:30 +0000 Received: from CO1NAM11FT061.eop-nam11.prod.protection.outlook.com (2603:10b6:303:b9:cafe::9f) by MW4PR03CA0233.outlook.office365.com (2603:10b6:303:b9::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.23 via Frontend Transport; Thu, 12 May 2022 15:43:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.238) by CO1NAM11FT061.mail.protection.outlook.com (10.13.175.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5250.13 via Frontend Transport; Thu, 12 May 2022 15:43:29 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Thu, 12 May 2022 15:43:29 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Thu, 12 May 2022 08:43:28 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Thu, 12 May 2022 08:43:25 -0700 From: Avihai Horon To: , "Michael S . Tsirkin" , "Cornelia Huck" , Paolo Bonzini , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Yishai Hadas , Jason Gunthorpe , "Mark Bloch" , Maor Gottlieb , Kirti Wankhede , Tarun Gupta Subject: [PATCH 1/9] linux-headers: Update headers to v5.18-rc6 Date: Thu, 12 May 2022 18:43:12 +0300 Message-ID: <20220512154320.19697-2-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220512154320.19697-1-avihaih@nvidia.com> References: <20220512154320.19697-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8e1fd784-ac81-4c94-5ace-08da342e299f X-MS-TrafficTypeDiagnostic: BN8PR12MB3585:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mF6nRvXOnf5vqQExlmAPLqYi0/3h+hWl5X+YIQWDeqEaDd8loDZp1FXwt9AvWM5er1upX2YRQw0qdoMz+JqGmPFKVqpIYDub4xFmop6wcVo4Rqm+seUV0/XQisswv/W+hHKHFrnTRHntBdNju3SZSyFiE76nfMgcAZIF2OddZDvTtfzLZe+Cn9mwwo6/Yu+rE40r9buEyI0jXb5h8drLxoY1AALehPZk9V+q+mg7RwKB48Rq+OcemJHSiIXZTpiNzlJlF6zMwRVfBMwSNjcINno1omhc5WEVN2WdkUP5xsf9HCycuFWInBS72clRuiyHFP4LgmVWnMZQqlJFb9KCtd1l7FEh+97Jm6bsX+dmKkZHSXP9wSPEhMnYC4LfqoZQIQVEvTeXtRuICUMreNAQB7n2vnvNMAS9H2W2mr+wiS5Ib/DDcN/oDvlggQODs6zF2MwAwIIw9SiF7L1hlBRloR1NBhgiIzIXbaNCwtFMhrObDovJs61kMqj/dNrA7wmnTUgkSkpqR29yKN3OtuheQAgQUdr0RINpZJV37ehWE3VIrLDLRKucTOJ1z6sdsMtUhhq0YdcQYPHgeOgkSJnqwSw0T3h3+qk0UM38El1b7/FhHSwIEkLZiONN4rEZSdctCU+CiOkvYl8BjX/cwulH1/AYghqdZ+pHqVdvKa7hfO8wdebKBAQm0PkMXBJKotImjqlAm+cvtYoNEV0qNVCNXw== X-Forefront-Antispam-Report: CIP:12.22.5.238; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(83380400001)(15650500001)(81166007)(2906002)(356005)(36860700001)(336012)(508600001)(40460700003)(107886003)(86362001)(2616005)(186003)(66574015)(47076005)(426003)(1076003)(5660300002)(36756003)(316002)(54906003)(82310400005)(8676002)(4326008)(70586007)(110136005)(70206006)(30864003)(6666004)(7696005)(26005)(8936002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2022 15:43:29.6389 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8e1fd784-ac81-4c94-5ace-08da342e299f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.238]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT061.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR12MB3585 Received-SPF: softfail client-ip=2a01:111:f400:7eab::616; envelope-from=avihaih@nvidia.com; helo=NAM11-CO1-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Avihai Horon --- .../linux/input-event-codes.h | 25 +- .../standard-headers/linux/virtio_config.h | 6 + .../standard-headers/linux/virtio_crypto.h | 82 +++- linux-headers/asm-arm64/kvm.h | 16 + linux-headers/asm-generic/mman-common.h | 2 + linux-headers/asm-mips/mman.h | 2 + linux-headers/linux/kvm.h | 27 +- linux-headers/linux/psci.h | 4 + linux-headers/linux/userfaultfd.h | 8 +- linux-headers/linux/vfio.h | 406 +++++++++--------- linux-headers/linux/vhost.h | 7 + 11 files changed, 365 insertions(+), 220 deletions(-) diff --git a/include/standard-headers/linux/input-event-codes.h b/include/standard-headers/linux/input-event-codes.h index b5e86b40ab..50790aee5a 100644 --- a/include/standard-headers/linux/input-event-codes.h +++ b/include/standard-headers/linux/input-event-codes.h @@ -278,7 +278,8 @@ #define KEY_PAUSECD 201 #define KEY_PROG3 202 #define KEY_PROG4 203 -#define KEY_DASHBOARD 204 /* AL Dashboard */ +#define KEY_ALL_APPLICATIONS 204 /* AC Desktop Show All Applications */ +#define KEY_DASHBOARD KEY_ALL_APPLICATIONS #define KEY_SUSPEND 205 #define KEY_CLOSE 206 /* AC Close */ #define KEY_PLAY 207 @@ -612,6 +613,7 @@ #define KEY_ASSISTANT 0x247 /* AL Context-aware desktop assistant */ #define KEY_KBD_LAYOUT_NEXT 0x248 /* AC Next Keyboard Layout Select */ #define KEY_EMOJI_PICKER 0x249 /* Show/hide emoji picker (HUTRR101) */ +#define KEY_DICTATE 0x24a /* Start or Stop Voice Dictation Session (HUTRR99) */ #define KEY_BRIGHTNESS_MIN 0x250 /* Set Brightness to Minimum */ #define KEY_BRIGHTNESS_MAX 0x251 /* Set Brightness to Maximum */ @@ -660,6 +662,27 @@ /* Select an area of screen to be copied */ #define KEY_SELECTIVE_SCREENSHOT 0x27a +/* Move the focus to the next or previous user controllable element within a UI container */ +#define KEY_NEXT_ELEMENT 0x27b +#define KEY_PREVIOUS_ELEMENT 0x27c + +/* Toggle Autopilot engagement */ +#define KEY_AUTOPILOT_ENGAGE_TOGGLE 0x27d + +/* Shortcut Keys */ +#define KEY_MARK_WAYPOINT 0x27e +#define KEY_SOS 0x27f +#define KEY_NAV_CHART 0x280 +#define KEY_FISHING_CHART 0x281 +#define KEY_SINGLE_RANGE_RADAR 0x282 +#define KEY_DUAL_RANGE_RADAR 0x283 +#define KEY_RADAR_OVERLAY 0x284 +#define KEY_TRADITIONAL_SONAR 0x285 +#define KEY_CLEARVU_SONAR 0x286 +#define KEY_SIDEVU_SONAR 0x287 +#define KEY_NAV_INFO 0x288 +#define KEY_BRIGHTNESS_MENU 0x289 + /* * Some keyboards have keys which do not have a defined meaning, these keys * are intended to be programmed / bound to macros by the user. For most diff --git a/include/standard-headers/linux/virtio_config.h b/include/standard-headers/linux/virtio_config.h index 22e3a85f67..7acd8d4abc 100644 --- a/include/standard-headers/linux/virtio_config.h +++ b/include/standard-headers/linux/virtio_config.h @@ -80,6 +80,12 @@ /* This feature indicates support for the packed virtqueue layout. */ #define VIRTIO_F_RING_PACKED 34 +/* + * Inorder feature indicates that all buffers are used by the device + * in the same order in which they have been made available. + */ +#define VIRTIO_F_IN_ORDER 35 + /* * This feature indicates that memory accesses by the driver and the * device are ordered in a way described by the platform. diff --git a/include/standard-headers/linux/virtio_crypto.h b/include/standard-headers/linux/virtio_crypto.h index 5ff0b4ee59..68066dafb6 100644 --- a/include/standard-headers/linux/virtio_crypto.h +++ b/include/standard-headers/linux/virtio_crypto.h @@ -37,6 +37,7 @@ #define VIRTIO_CRYPTO_SERVICE_HASH 1 #define VIRTIO_CRYPTO_SERVICE_MAC 2 #define VIRTIO_CRYPTO_SERVICE_AEAD 3 +#define VIRTIO_CRYPTO_SERVICE_AKCIPHER 4 #define VIRTIO_CRYPTO_OPCODE(service, op) (((service) << 8) | (op)) @@ -57,6 +58,10 @@ struct virtio_crypto_ctrl_header { VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x02) #define VIRTIO_CRYPTO_AEAD_DESTROY_SESSION \ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x03) +#define VIRTIO_CRYPTO_AKCIPHER_CREATE_SESSION \ + VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AKCIPHER, 0x04) +#define VIRTIO_CRYPTO_AKCIPHER_DESTROY_SESSION \ + VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AKCIPHER, 0x05) uint32_t opcode; uint32_t algo; uint32_t flag; @@ -180,6 +185,58 @@ struct virtio_crypto_aead_create_session_req { uint8_t padding[32]; }; +struct virtio_crypto_rsa_session_para { +#define VIRTIO_CRYPTO_RSA_RAW_PADDING 0 +#define VIRTIO_CRYPTO_RSA_PKCS1_PADDING 1 + uint32_t padding_algo; + +#define VIRTIO_CRYPTO_RSA_NO_HASH 0 +#define VIRTIO_CRYPTO_RSA_MD2 1 +#define VIRTIO_CRYPTO_RSA_MD3 2 +#define VIRTIO_CRYPTO_RSA_MD4 3 +#define VIRTIO_CRYPTO_RSA_MD5 4 +#define VIRTIO_CRYPTO_RSA_SHA1 5 +#define VIRTIO_CRYPTO_RSA_SHA256 6 +#define VIRTIO_CRYPTO_RSA_SHA384 7 +#define VIRTIO_CRYPTO_RSA_SHA512 8 +#define VIRTIO_CRYPTO_RSA_SHA224 9 + uint32_t hash_algo; +}; + +struct virtio_crypto_ecdsa_session_para { +#define VIRTIO_CRYPTO_CURVE_UNKNOWN 0 +#define VIRTIO_CRYPTO_CURVE_NIST_P192 1 +#define VIRTIO_CRYPTO_CURVE_NIST_P224 2 +#define VIRTIO_CRYPTO_CURVE_NIST_P256 3 +#define VIRTIO_CRYPTO_CURVE_NIST_P384 4 +#define VIRTIO_CRYPTO_CURVE_NIST_P521 5 + uint32_t curve_id; + uint32_t padding; +}; + +struct virtio_crypto_akcipher_session_para { +#define VIRTIO_CRYPTO_NO_AKCIPHER 0 +#define VIRTIO_CRYPTO_AKCIPHER_RSA 1 +#define VIRTIO_CRYPTO_AKCIPHER_DSA 2 +#define VIRTIO_CRYPTO_AKCIPHER_ECDSA 3 + uint32_t algo; + +#define VIRTIO_CRYPTO_AKCIPHER_KEY_TYPE_PUBLIC 1 +#define VIRTIO_CRYPTO_AKCIPHER_KEY_TYPE_PRIVATE 2 + uint32_t keytype; + uint32_t keylen; + + union { + struct virtio_crypto_rsa_session_para rsa; + struct virtio_crypto_ecdsa_session_para ecdsa; + } u; +}; + +struct virtio_crypto_akcipher_create_session_req { + struct virtio_crypto_akcipher_session_para para; + uint8_t padding[36]; +}; + struct virtio_crypto_alg_chain_session_para { #define VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER 1 #define VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH 2 @@ -247,6 +304,8 @@ struct virtio_crypto_op_ctrl_req { mac_create_session; struct virtio_crypto_aead_create_session_req aead_create_session; + struct virtio_crypto_akcipher_create_session_req + akcipher_create_session; struct virtio_crypto_destroy_session_req destroy_session; uint8_t padding[56]; @@ -266,6 +325,14 @@ struct virtio_crypto_op_header { VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x00) #define VIRTIO_CRYPTO_AEAD_DECRYPT \ VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x01) +#define VIRTIO_CRYPTO_AKCIPHER_ENCRYPT \ + VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AKCIPHER, 0x00) +#define VIRTIO_CRYPTO_AKCIPHER_DECRYPT \ + VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AKCIPHER, 0x01) +#define VIRTIO_CRYPTO_AKCIPHER_SIGN \ + VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AKCIPHER, 0x02) +#define VIRTIO_CRYPTO_AKCIPHER_VERIFY \ + VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AKCIPHER, 0x03) uint32_t opcode; /* algo should be service-specific algorithms */ uint32_t algo; @@ -390,6 +457,16 @@ struct virtio_crypto_aead_data_req { uint8_t padding[32]; }; +struct virtio_crypto_akcipher_para { + uint32_t src_data_len; + uint32_t dst_data_len; +}; + +struct virtio_crypto_akcipher_data_req { + struct virtio_crypto_akcipher_para para; + uint8_t padding[40]; +}; + /* The request of the data virtqueue's packet */ struct virtio_crypto_op_data_req { struct virtio_crypto_op_header header; @@ -399,6 +476,7 @@ struct virtio_crypto_op_data_req { struct virtio_crypto_hash_data_req hash_req; struct virtio_crypto_mac_data_req mac_req; struct virtio_crypto_aead_data_req aead_req; + struct virtio_crypto_akcipher_data_req akcipher_req; uint8_t padding[48]; } u; }; @@ -408,6 +486,8 @@ struct virtio_crypto_op_data_req { #define VIRTIO_CRYPTO_BADMSG 2 #define VIRTIO_CRYPTO_NOTSUPP 3 #define VIRTIO_CRYPTO_INVSESS 4 /* Invalid session id */ +#define VIRTIO_CRYPTO_NOSPC 5 /* no free session ID */ +#define VIRTIO_CRYPTO_KEY_REJECTED 6 /* Signature verification failed */ /* The accelerator hardware is ready */ #define VIRTIO_CRYPTO_S_HW_READY (1 << 0) @@ -438,7 +518,7 @@ struct virtio_crypto_config { uint32_t max_cipher_key_len; /* Maximum length of authenticated key */ uint32_t max_auth_key_len; - uint32_t reserve; + uint32_t akcipher_algo; /* Maximum size of each crypto request's content */ uint64_t max_size; }; diff --git a/linux-headers/asm-arm64/kvm.h b/linux-headers/asm-arm64/kvm.h index 3d2ce9912d..5c28a9737a 100644 --- a/linux-headers/asm-arm64/kvm.h +++ b/linux-headers/asm-arm64/kvm.h @@ -281,6 +281,11 @@ struct kvm_arm_copy_mte_tags { #define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_NOT_REQUIRED 3 #define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_2_ENABLED (1U << 4) +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3 KVM_REG_ARM_FW_REG(3) +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_AVAIL 0 +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_AVAIL 1 +#define KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_3_NOT_REQUIRED 2 + /* SVE registers */ #define KVM_REG_ARM64_SVE (0x15 << KVM_REG_ARM_COPROC_SHIFT) @@ -362,6 +367,7 @@ struct kvm_arm_copy_mte_tags { #define KVM_ARM_VCPU_PMU_V3_IRQ 0 #define KVM_ARM_VCPU_PMU_V3_INIT 1 #define KVM_ARM_VCPU_PMU_V3_FILTER 2 +#define KVM_ARM_VCPU_PMU_V3_SET_PMU 3 #define KVM_ARM_VCPU_TIMER_CTRL 1 #define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0 #define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1 @@ -411,6 +417,16 @@ struct kvm_arm_copy_mte_tags { #define KVM_PSCI_RET_INVAL PSCI_RET_INVALID_PARAMS #define KVM_PSCI_RET_DENIED PSCI_RET_DENIED +/* arm64-specific kvm_run::system_event flags */ +/* + * Reset caused by a PSCI v1.1 SYSTEM_RESET2 call. + * Valid only when the system event has a type of KVM_SYSTEM_EVENT_RESET. + */ +#define KVM_SYSTEM_EVENT_RESET_FLAG_PSCI_RESET2 (1ULL << 0) + +/* run->fail_entry.hardware_entry_failure_reason codes. */ +#define KVM_EXIT_FAIL_ENTRY_CPU_UNSUPPORTED (1ULL << 0) + #endif #endif /* __ARM_KVM_H__ */ diff --git a/linux-headers/asm-generic/mman-common.h b/linux-headers/asm-generic/mman-common.h index 1567a3294c..6c1aa92a92 100644 --- a/linux-headers/asm-generic/mman-common.h +++ b/linux-headers/asm-generic/mman-common.h @@ -75,6 +75,8 @@ #define MADV_POPULATE_READ 22 /* populate (prefault) page tables readable */ #define MADV_POPULATE_WRITE 23 /* populate (prefault) page tables writable */ +#define MADV_DONTNEED_LOCKED 24 /* like DONTNEED, but drop locked pages too */ + /* compatibility flags */ #define MAP_FILE 0 diff --git a/linux-headers/asm-mips/mman.h b/linux-headers/asm-mips/mman.h index 40b210c65a..1be428663c 100644 --- a/linux-headers/asm-mips/mman.h +++ b/linux-headers/asm-mips/mman.h @@ -101,6 +101,8 @@ #define MADV_POPULATE_READ 22 /* populate (prefault) page tables readable */ #define MADV_POPULATE_WRITE 23 /* populate (prefault) page tables writable */ +#define MADV_DONTNEED_LOCKED 24 /* like DONTNEED, but drop locked pages too */ + /* compatibility flags */ #define MAP_FILE 0 diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h index d232feaae9..0d05d02ee4 100644 --- a/linux-headers/linux/kvm.h +++ b/linux-headers/linux/kvm.h @@ -445,7 +445,11 @@ struct kvm_run { #define KVM_SYSTEM_EVENT_RESET 2 #define KVM_SYSTEM_EVENT_CRASH 3 __u32 type; - __u64 flags; + __u32 ndata; + union { + __u64 flags; + __u64 data[16]; + }; } system_event; /* KVM_EXIT_S390_STSI */ struct { @@ -562,9 +566,12 @@ struct kvm_s390_mem_op { __u32 op; /* type of operation */ __u64 buf; /* buffer in userspace */ union { - __u8 ar; /* the access register number */ + struct { + __u8 ar; /* the access register number */ + __u8 key; /* access key, ignored if flag unset */ + }; __u32 sida_offset; /* offset into the sida */ - __u8 reserved[32]; /* should be set to 0 */ + __u8 reserved[32]; /* ignored */ }; }; /* types for kvm_s390_mem_op->op */ @@ -572,9 +579,12 @@ struct kvm_s390_mem_op { #define KVM_S390_MEMOP_LOGICAL_WRITE 1 #define KVM_S390_MEMOP_SIDA_READ 2 #define KVM_S390_MEMOP_SIDA_WRITE 3 +#define KVM_S390_MEMOP_ABSOLUTE_READ 4 +#define KVM_S390_MEMOP_ABSOLUTE_WRITE 5 /* flags for kvm_s390_mem_op->flags */ #define KVM_S390_MEMOP_F_CHECK_ONLY (1ULL << 0) #define KVM_S390_MEMOP_F_INJECT_EXCEPTION (1ULL << 1) +#define KVM_S390_MEMOP_F_SKEY_PROTECTION (1ULL << 2) /* for KVM_INTERRUPT */ struct kvm_interrupt { @@ -1134,6 +1144,12 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_VM_GPA_BITS 207 #define KVM_CAP_XSAVE2 208 #define KVM_CAP_SYS_ATTRIBUTES 209 +#define KVM_CAP_PPC_AIL_MODE_3 210 +#define KVM_CAP_S390_MEM_OP_EXTENSION 211 +#define KVM_CAP_PMU_CAPABILITY 212 +#define KVM_CAP_DISABLE_QUIRKS2 213 +/* #define KVM_CAP_VM_TSC_CONTROL 214 */ +#define KVM_CAP_SYSTEM_EVENT_DATA 215 #ifdef KVM_CAP_IRQ_ROUTING @@ -1624,9 +1640,6 @@ struct kvm_enc_region { #define KVM_S390_NORMAL_RESET _IO(KVMIO, 0xc3) #define KVM_S390_CLEAR_RESET _IO(KVMIO, 0xc4) -/* Available with KVM_CAP_XSAVE2 */ -#define KVM_GET_XSAVE2 _IOR(KVMIO, 0xcf, struct kvm_xsave) - struct kvm_s390_pv_sec_parm { __u64 origin; __u64 length; @@ -1973,6 +1986,8 @@ struct kvm_dirty_gfn { #define KVM_BUS_LOCK_DETECTION_OFF (1 << 0) #define KVM_BUS_LOCK_DETECTION_EXIT (1 << 1) +#define KVM_PMU_CAP_DISABLE (1 << 0) + /** * struct kvm_stats_header - Header of per vm/vcpu binary statistics data. * @flags: Some extra information for header, always 0 for now. diff --git a/linux-headers/linux/psci.h b/linux-headers/linux/psci.h index a6772d508b..213b2a0f70 100644 --- a/linux-headers/linux/psci.h +++ b/linux-headers/linux/psci.h @@ -82,6 +82,10 @@ #define PSCI_0_2_TOS_UP_NO_MIGRATE 1 #define PSCI_0_2_TOS_MP 2 +/* PSCI v1.1 reset type encoding for SYSTEM_RESET2 */ +#define PSCI_1_1_RESET_TYPE_SYSTEM_WARM_RESET 0 +#define PSCI_1_1_RESET_TYPE_VENDOR_START 0x80000000U + /* PSCI version decoding (independent of PSCI version) */ #define PSCI_VERSION_MAJOR_SHIFT 16 #define PSCI_VERSION_MINOR_MASK \ diff --git a/linux-headers/linux/userfaultfd.h b/linux-headers/linux/userfaultfd.h index 8479af5f4c..769b8379e4 100644 --- a/linux-headers/linux/userfaultfd.h +++ b/linux-headers/linux/userfaultfd.h @@ -32,7 +32,8 @@ UFFD_FEATURE_SIGBUS | \ UFFD_FEATURE_THREAD_ID | \ UFFD_FEATURE_MINOR_HUGETLBFS | \ - UFFD_FEATURE_MINOR_SHMEM) + UFFD_FEATURE_MINOR_SHMEM | \ + UFFD_FEATURE_EXACT_ADDRESS) #define UFFD_API_IOCTLS \ ((__u64)1 << _UFFDIO_REGISTER | \ (__u64)1 << _UFFDIO_UNREGISTER | \ @@ -189,6 +190,10 @@ struct uffdio_api { * * UFFD_FEATURE_MINOR_SHMEM indicates the same support as * UFFD_FEATURE_MINOR_HUGETLBFS, but for shmem-backed pages instead. + * + * UFFD_FEATURE_EXACT_ADDRESS indicates that the exact address of page + * faults would be provided and the offset within the page would not be + * masked. */ #define UFFD_FEATURE_PAGEFAULT_FLAG_WP (1<<0) #define UFFD_FEATURE_EVENT_FORK (1<<1) @@ -201,6 +206,7 @@ struct uffdio_api { #define UFFD_FEATURE_THREAD_ID (1<<8) #define UFFD_FEATURE_MINOR_HUGETLBFS (1<<9) #define UFFD_FEATURE_MINOR_SHMEM (1<<10) +#define UFFD_FEATURE_EXACT_ADDRESS (1<<11) __u64 features; __u64 ioctls; diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index e680594f27..e9f7795c39 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -323,7 +323,7 @@ struct vfio_region_info_cap_type { #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0xffff) #define VFIO_REGION_TYPE_GFX (1) #define VFIO_REGION_TYPE_CCW (2) -#define VFIO_REGION_TYPE_MIGRATION (3) +#define VFIO_REGION_TYPE_MIGRATION_DEPRECATED (3) /* sub-types for VFIO_REGION_TYPE_PCI_* */ @@ -405,225 +405,29 @@ struct vfio_region_gfx_edid { #define VFIO_REGION_SUBTYPE_CCW_CRW (3) /* sub-types for VFIO_REGION_TYPE_MIGRATION */ -#define VFIO_REGION_SUBTYPE_MIGRATION (1) - -/* - * The structure vfio_device_migration_info is placed at the 0th offset of - * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device related - * migration information. Field accesses from this structure are only supported - * at their native width and alignment. Otherwise, the result is undefined and - * vendor drivers should return an error. - * - * device_state: (read/write) - * - The user application writes to this field to inform the vendor driver - * about the device state to be transitioned to. - * - The vendor driver should take the necessary actions to change the - * device state. After successful transition to a given state, the - * vendor driver should return success on write(device_state, state) - * system call. If the device state transition fails, the vendor driver - * should return an appropriate -errno for the fault condition. - * - On the user application side, if the device state transition fails, - * that is, if write(device_state, state) returns an error, read - * device_state again to determine the current state of the device from - * the vendor driver. - * - The vendor driver should return previous state of the device unless - * the vendor driver has encountered an internal error, in which case - * the vendor driver may report the device_state VFIO_DEVICE_STATE_ERROR. - * - The user application must use the device reset ioctl to recover the - * device from VFIO_DEVICE_STATE_ERROR state. If the device is - * indicated to be in a valid device state by reading device_state, the - * user application may attempt to transition the device to any valid - * state reachable from the current state or terminate itself. - * - * device_state consists of 3 bits: - * - If bit 0 is set, it indicates the _RUNNING state. If bit 0 is clear, - * it indicates the _STOP state. When the device state is changed to - * _STOP, driver should stop the device before write() returns. - * - If bit 1 is set, it indicates the _SAVING state, which means that the - * driver should start gathering device state information that will be - * provided to the VFIO user application to save the device's state. - * - If bit 2 is set, it indicates the _RESUMING state, which means that - * the driver should prepare to resume the device. Data provided through - * the migration region should be used to resume the device. - * Bits 3 - 31 are reserved for future use. To preserve them, the user - * application should perform a read-modify-write operation on this - * field when modifying the specified bits. - * - * +------- _RESUMING - * |+------ _SAVING - * ||+----- _RUNNING - * ||| - * 000b => Device Stopped, not saving or resuming - * 001b => Device running, which is the default state - * 010b => Stop the device & save the device state, stop-and-copy state - * 011b => Device running and save the device state, pre-copy state - * 100b => Device stopped and the device state is resuming - * 101b => Invalid state - * 110b => Error state - * 111b => Invalid state - * - * State transitions: - * - * _RESUMING _RUNNING Pre-copy Stop-and-copy _STOP - * (100b) (001b) (011b) (010b) (000b) - * 0. Running or default state - * | - * - * 1. Normal Shutdown (optional) - * |------------------------------------->| - * - * 2. Save the state or suspend - * |------------------------->|---------->| - * - * 3. Save the state during live migration - * |----------->|------------>|---------->| - * - * 4. Resuming - * |<---------| - * - * 5. Resumed - * |--------->| - * - * 0. Default state of VFIO device is _RUNNING when the user application starts. - * 1. During normal shutdown of the user application, the user application may - * optionally change the VFIO device state from _RUNNING to _STOP. This - * transition is optional. The vendor driver must support this transition but - * must not require it. - * 2. When the user application saves state or suspends the application, the - * device state transitions from _RUNNING to stop-and-copy and then to _STOP. - * On state transition from _RUNNING to stop-and-copy, driver must stop the - * device, save the device state and send it to the application through the - * migration region. The sequence to be followed for such transition is given - * below. - * 3. In live migration of user application, the state transitions from _RUNNING - * to pre-copy, to stop-and-copy, and to _STOP. - * On state transition from _RUNNING to pre-copy, the driver should start - * gathering the device state while the application is still running and send - * the device state data to application through the migration region. - * On state transition from pre-copy to stop-and-copy, the driver must stop - * the device, save the device state and send it to the user application - * through the migration region. - * Vendor drivers must support the pre-copy state even for implementations - * where no data is provided to the user before the stop-and-copy state. The - * user must not be required to consume all migration data before the device - * transitions to a new state, including the stop-and-copy state. - * The sequence to be followed for above two transitions is given below. - * 4. To start the resuming phase, the device state should be transitioned from - * the _RUNNING to the _RESUMING state. - * In the _RESUMING state, the driver should use the device state data - * received through the migration region to resume the device. - * 5. After providing saved device data to the driver, the application should - * change the state from _RESUMING to _RUNNING. - * - * reserved: - * Reads on this field return zero and writes are ignored. - * - * pending_bytes: (read only) - * The number of pending bytes still to be migrated from the vendor driver. - * - * data_offset: (read only) - * The user application should read data_offset field from the migration - * region. The user application should read the device data from this - * offset within the migration region during the _SAVING state or write - * the device data during the _RESUMING state. See below for details of - * sequence to be followed. - * - * data_size: (read/write) - * The user application should read data_size to get the size in bytes of - * the data copied in the migration region during the _SAVING state and - * write the size in bytes of the data copied in the migration region - * during the _RESUMING state. - * - * The format of the migration region is as follows: - * ------------------------------------------------------------------ - * |vfio_device_migration_info| data section | - * | | /////////////////////////////// | - * ------------------------------------------------------------------ - * ^ ^ - * offset 0-trapped part data_offset - * - * The structure vfio_device_migration_info is always followed by the data - * section in the region, so data_offset will always be nonzero. The offset - * from where the data is copied is decided by the kernel driver. The data - * section can be trapped, mmapped, or partitioned, depending on how the kernel - * driver defines the data section. The data section partition can be defined - * as mapped by the sparse mmap capability. If mmapped, data_offset must be - * page aligned, whereas initial section which contains the - * vfio_device_migration_info structure, might not end at the offset, which is - * page aligned. The user is not required to access through mmap regardless - * of the capabilities of the region mmap. - * The vendor driver should determine whether and how to partition the data - * section. The vendor driver should return data_offset accordingly. - * - * The sequence to be followed while in pre-copy state and stop-and-copy state - * is as follows: - * a. Read pending_bytes, indicating the start of a new iteration to get device - * data. Repeated read on pending_bytes at this stage should have no side - * effects. - * If pending_bytes == 0, the user application should not iterate to get data - * for that device. - * If pending_bytes > 0, perform the following steps. - * b. Read data_offset, indicating that the vendor driver should make data - * available through the data section. The vendor driver should return this - * read operation only after data is available from (region + data_offset) - * to (region + data_offset + data_size). - * c. Read data_size, which is the amount of data in bytes available through - * the migration region. - * Read on data_offset and data_size should return the offset and size of - * the current buffer if the user application reads data_offset and - * data_size more than once here. - * d. Read data_size bytes of data from (region + data_offset) from the - * migration region. - * e. Process the data. - * f. Read pending_bytes, which indicates that the data from the previous - * iteration has been read. If pending_bytes > 0, go to step b. - * - * The user application can transition from the _SAVING|_RUNNING - * (pre-copy state) to the _SAVING (stop-and-copy) state regardless of the - * number of pending bytes. The user application should iterate in _SAVING - * (stop-and-copy) until pending_bytes is 0. - * - * The sequence to be followed while _RESUMING device state is as follows: - * While data for this device is available, repeat the following steps: - * a. Read data_offset from where the user application should write data. - * b. Write migration data starting at the migration region + data_offset for - * the length determined by data_size from the migration source. - * c. Write data_size, which indicates to the vendor driver that data is - * written in the migration region. Vendor driver must return this write - * operations on consuming data. Vendor driver should apply the - * user-provided migration region data to the device resume state. - * - * If an error occurs during the above sequences, the vendor driver can return - * an error code for next read() or write() operation, which will terminate the - * loop. The user application should then take the next necessary action, for - * example, failing migration or terminating the user application. - * - * For the user application, data is opaque. The user application should write - * data in the same order as the data is received and the data should be of - * same transaction size at the source. - */ +#define VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED (1) struct vfio_device_migration_info { __u32 device_state; /* VFIO device state */ -#define VFIO_DEVICE_STATE_STOP (0) -#define VFIO_DEVICE_STATE_RUNNING (1 << 0) -#define VFIO_DEVICE_STATE_SAVING (1 << 1) -#define VFIO_DEVICE_STATE_RESUMING (1 << 2) -#define VFIO_DEVICE_STATE_MASK (VFIO_DEVICE_STATE_RUNNING | \ - VFIO_DEVICE_STATE_SAVING | \ - VFIO_DEVICE_STATE_RESUMING) +#define VFIO_DEVICE_STATE_V1_STOP (0) +#define VFIO_DEVICE_STATE_V1_RUNNING (1 << 0) +#define VFIO_DEVICE_STATE_V1_SAVING (1 << 1) +#define VFIO_DEVICE_STATE_V1_RESUMING (1 << 2) +#define VFIO_DEVICE_STATE_MASK (VFIO_DEVICE_STATE_V1_RUNNING | \ + VFIO_DEVICE_STATE_V1_SAVING | \ + VFIO_DEVICE_STATE_V1_RESUMING) #define VFIO_DEVICE_STATE_VALID(state) \ - (state & VFIO_DEVICE_STATE_RESUMING ? \ - (state & VFIO_DEVICE_STATE_MASK) == VFIO_DEVICE_STATE_RESUMING : 1) + (state & VFIO_DEVICE_STATE_V1_RESUMING ? \ + (state & VFIO_DEVICE_STATE_MASK) == VFIO_DEVICE_STATE_V1_RESUMING : 1) #define VFIO_DEVICE_STATE_IS_ERROR(state) \ - ((state & VFIO_DEVICE_STATE_MASK) == (VFIO_DEVICE_STATE_SAVING | \ - VFIO_DEVICE_STATE_RESUMING)) + ((state & VFIO_DEVICE_STATE_MASK) == (VFIO_DEVICE_STATE_V1_SAVING | \ + VFIO_DEVICE_STATE_V1_RESUMING)) #define VFIO_DEVICE_STATE_SET_ERROR(state) \ - ((state & ~VFIO_DEVICE_STATE_MASK) | VFIO_DEVICE_SATE_SAVING | \ - VFIO_DEVICE_STATE_RESUMING) + ((state & ~VFIO_DEVICE_STATE_MASK) | VFIO_DEVICE_STATE_V1_SAVING | \ + VFIO_DEVICE_STATE_V1_RESUMING) __u32 reserved; __u64 pending_bytes; @@ -1002,6 +806,186 @@ struct vfio_device_feature { */ #define VFIO_DEVICE_FEATURE_PCI_VF_TOKEN (0) +/* + * Indicates the device can support the migration API through + * VFIO_DEVICE_FEATURE_MIG_DEVICE_STATE. If this GET succeeds, the RUNNING and + * ERROR states are always supported. Support for additional states is + * indicated via the flags field; at least VFIO_MIGRATION_STOP_COPY must be + * set. + * + * VFIO_MIGRATION_STOP_COPY means that STOP, STOP_COPY and + * RESUMING are supported. + * + * VFIO_MIGRATION_STOP_COPY | VFIO_MIGRATION_P2P means that RUNNING_P2P + * is supported in addition to the STOP_COPY states. + * + * Other combinations of flags have behavior to be defined in the future. + */ +struct vfio_device_feature_migration { + __aligned_u64 flags; +#define VFIO_MIGRATION_STOP_COPY (1 << 0) +#define VFIO_MIGRATION_P2P (1 << 1) +}; +#define VFIO_DEVICE_FEATURE_MIGRATION 1 + +/* + * Upon VFIO_DEVICE_FEATURE_SET, execute a migration state change on the VFIO + * device. The new state is supplied in device_state, see enum + * vfio_device_mig_state for details + * + * The kernel migration driver must fully transition the device to the new state + * value before the operation returns to the user. + * + * The kernel migration driver must not generate asynchronous device state + * transitions outside of manipulation by the user or the VFIO_DEVICE_RESET + * ioctl as described above. + * + * If this function fails then current device_state may be the original + * operating state or some other state along the combination transition path. + * The user can then decide if it should execute a VFIO_DEVICE_RESET, attempt + * to return to the original state, or attempt to return to some other state + * such as RUNNING or STOP. + * + * If the new_state starts a new data transfer session then the FD associated + * with that session is returned in data_fd. The user is responsible to close + * this FD when it is finished. The user must consider the migration data stream + * carried over the FD to be opaque and must preserve the byte order of the + * stream. The user is not required to preserve buffer segmentation when writing + * the data stream during the RESUMING operation. + * + * Upon VFIO_DEVICE_FEATURE_GET, get the current migration state of the VFIO + * device, data_fd will be -1. + */ +struct vfio_device_feature_mig_state { + __u32 device_state; /* From enum vfio_device_mig_state */ + __s32 data_fd; +}; +#define VFIO_DEVICE_FEATURE_MIG_DEVICE_STATE 2 + +/* + * The device migration Finite State Machine is described by the enum + * vfio_device_mig_state. Some of the FSM arcs will create a migration data + * transfer session by returning a FD, in this case the migration data will + * flow over the FD using read() and write() as discussed below. + * + * There are 5 states to support VFIO_MIGRATION_STOP_COPY: + * RUNNING - The device is running normally + * STOP - The device does not change the internal or external state + * STOP_COPY - The device internal state can be read out + * RESUMING - The device is stopped and is loading a new internal state + * ERROR - The device has failed and must be reset + * + * And 1 optional state to support VFIO_MIGRATION_P2P: + * RUNNING_P2P - RUNNING, except the device cannot do peer to peer DMA + * + * The FSM takes actions on the arcs between FSM states. The driver implements + * the following behavior for the FSM arcs: + * + * RUNNING_P2P -> STOP + * STOP_COPY -> STOP + * While in STOP the device must stop the operation of the device. The device + * must not generate interrupts, DMA, or any other change to external state. + * It must not change its internal state. When stopped the device and kernel + * migration driver must accept and respond to interaction to support external + * subsystems in the STOP state, for example PCI MSI-X and PCI config space. + * Failure by the user to restrict device access while in STOP must not result + * in error conditions outside the user context (ex. host system faults). + * + * The STOP_COPY arc will terminate a data transfer session. + * + * RESUMING -> STOP + * Leaving RESUMING terminates a data transfer session and indicates the + * device should complete processing of the data delivered by write(). The + * kernel migration driver should complete the incorporation of data written + * to the data transfer FD into the device internal state and perform + * final validity and consistency checking of the new device state. If the + * user provided data is found to be incomplete, inconsistent, or otherwise + * invalid, the migration driver must fail the SET_STATE ioctl and + * optionally go to the ERROR state as described below. + * + * While in STOP the device has the same behavior as other STOP states + * described above. + * + * To abort a RESUMING session the device must be reset. + * + * RUNNING_P2P -> RUNNING + * While in RUNNING the device is fully operational, the device may generate + * interrupts, DMA, respond to MMIO, all vfio device regions are functional, + * and the device may advance its internal state. + * + * RUNNING -> RUNNING_P2P + * STOP -> RUNNING_P2P + * While in RUNNING_P2P the device is partially running in the P2P quiescent + * state defined below. + * + * STOP -> STOP_COPY + * This arc begin the process of saving the device state and will return a + * new data_fd. + * + * While in the STOP_COPY state the device has the same behavior as STOP + * with the addition that the data transfers session continues to stream the + * migration state. End of stream on the FD indicates the entire device + * state has been transferred. + * + * The user should take steps to restrict access to vfio device regions while + * the device is in STOP_COPY or risk corruption of the device migration data + * stream. + * + * STOP -> RESUMING + * Entering the RESUMING state starts a process of restoring the device state + * and will return a new data_fd. The data stream fed into the data_fd should + * be taken from the data transfer output of a single FD during saving from + * a compatible device. The migration driver may alter/reset the internal + * device state for this arc if required to prepare the device to receive the + * migration data. + * + * any -> ERROR + * ERROR cannot be specified as a device state, however any transition request + * can be failed with an errno return and may then move the device_state into + * ERROR. In this case the device was unable to execute the requested arc and + * was also unable to restore the device to any valid device_state. + * To recover from ERROR VFIO_DEVICE_RESET must be used to return the + * device_state back to RUNNING. + * + * The optional peer to peer (P2P) quiescent state is intended to be a quiescent + * state for the device for the purposes of managing multiple devices within a + * user context where peer-to-peer DMA between devices may be active. The + * RUNNING_P2P states must prevent the device from initiating + * any new P2P DMA transactions. If the device can identify P2P transactions + * then it can stop only P2P DMA, otherwise it must stop all DMA. The migration + * driver must complete any such outstanding operations prior to completing the + * FSM arc into a P2P state. For the purpose of specification the states + * behave as though the device was fully running if not supported. Like while in + * STOP or STOP_COPY the user must not touch the device, otherwise the state + * can be exited. + * + * The remaining possible transitions are interpreted as combinations of the + * above FSM arcs. As there are multiple paths through the FSM arcs the path + * should be selected based on the following rules: + * - Select the shortest path. + * Refer to vfio_mig_get_next_state() for the result of the algorithm. + * + * The automatic transit through the FSM arcs that make up the combination + * transition is invisible to the user. When working with combination arcs the + * user may see any step along the path in the device_state if SET_STATE + * fails. When handling these types of errors users should anticipate future + * revisions of this protocol using new states and those states becoming + * visible in this case. + * + * The optional states cannot be used with SET_STATE if the device does not + * support them. The user can discover if these states are supported by using + * VFIO_DEVICE_FEATURE_MIGRATION. By using combination transitions the user can + * avoid knowing about these optional states if the kernel driver supports them. + */ +enum vfio_device_mig_state { + VFIO_DEVICE_STATE_ERROR = 0, + VFIO_DEVICE_STATE_STOP = 1, + VFIO_DEVICE_STATE_RUNNING = 2, + VFIO_DEVICE_STATE_STOP_COPY = 3, + VFIO_DEVICE_STATE_RESUMING = 4, + VFIO_DEVICE_STATE_RUNNING_P2P = 5, +}; + /* -------- API for Type1 VFIO IOMMU -------- */ /** diff --git a/linux-headers/linux/vhost.h b/linux-headers/linux/vhost.h index c998860d7b..5d99e7c242 100644 --- a/linux-headers/linux/vhost.h +++ b/linux-headers/linux/vhost.h @@ -150,4 +150,11 @@ /* Get the valid iova range */ #define VHOST_VDPA_GET_IOVA_RANGE _IOR(VHOST_VIRTIO, 0x78, \ struct vhost_vdpa_iova_range) + +/* Get the config size */ +#define VHOST_VDPA_GET_CONFIG_SIZE _IOR(VHOST_VIRTIO, 0x79, __u32) + +/* Get the count of all virtqueues */ +#define VHOST_VDPA_GET_VQS_COUNT _IOR(VHOST_VIRTIO, 0x80, __u32) + #endif From patchwork Thu May 12 15:43:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 12847859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DCCE0C433FE for ; Thu, 12 May 2022 16:03:30 +0000 (UTC) Received: from localhost ([::1]:55260 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1npBHi-0001Sh-3G for qemu-devel@archiver.kernel.org; Thu, 12 May 2022 12:03:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41908) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyb-0006Hq-PB for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:45 -0400 Received: from mail-dm6nam12on2062c.outbound.protection.outlook.com ([2a01:111:f400:fe59::62c]:5473 helo=NAM12-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyY-0005JD-NR for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:40 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XfBfRTN7owt0vWIe/jLZfexu2851iRQ/swqzG1iCOiLXjWvkS6vDf90B+kJnm34YEkCfxML9E/aKB4Vis2vD2t1D6/GlfaaHp5ZTy/iJXlAduA/QDI1uuqGTlS306E7i5qMfBD8Y2oLOF1u7HRaBGqoodkfuiujTikPmp3YRq3/C0gozWHFWtpE6R9b6wS63xeW4TEPw/bXb83adXcGs4KkayEqxhSSquaJ92nkjsq2mo0qkevWdeS4AzH/Ud9oC1EaBKQsgdMl7TsdMdWQK38nCYTIm65OiVtDLByctjzeCjmd4CB3dTYQllAOIhrB7pW+833/7wuvs200aXFJ0IQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GEMJNQvozy86c6TBdVjYWOJCXVPfzc4/yK6/X2bJC6E=; b=bpFccIBkGw475D+EvaC8YguCPhei3+eFGhH35omulvKUdD6M89MNOrcSjZzyVSSsT3JZ2NIAJGoFd6K3Q8NSYUeGTPWGLUgT/jZVqOyuC/vmVnweTmnnX5lUlfoCkZF6D81tq8SrB7guVWdXsVWttj/qbudNpOOkZjYrUI9O8E9qKEwVfRMgXHGGDLXmfPjKwn/mkYi7LgKA0YXI0YkWLy9PsFuipByxm0bBwBkgy6X+Syn77hFZ/KANKV9M6lbaAeZCWqMcqzumjTxGruNUKUPaDlA8UKfFwx+TNsSLx52XuTTX7HHg/lDbcRyliPzctk2m9JIBw7S2IKbY2SXUGw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GEMJNQvozy86c6TBdVjYWOJCXVPfzc4/yK6/X2bJC6E=; b=pR/vhxZHDWihIsphgORZa7NIu3JKCmhcQ1GojyRTFsexwX77KG/5TKEUJHo8/z7cQ8VmRR9ey16HQu7c54mxijCEQtCHaUZmHsaVfVQMhKJ3oJhjaOy8qSpO/hDGNPh5D2yW45VQCb0QNjDvUFcOQ8nAaAY+D+493//N9pk14D2aFfJoahLE8++ZqXUzd86nQm5kAmKPUUPBYLKLdAn9aO2Pi7phTC9YG3ZIJj3DXESOdbrfRJddUOblSyMP6Pz9kWRm/bLFIOfaNzGVS8p334FchkZTxGlX4Fy/oHVxgqt8AUK+uqWArJpI6U+C3NkYsZedJdBswWZY8zhMYdnJvQ== Received: from MW4P222CA0013.NAMP222.PROD.OUTLOOK.COM (2603:10b6:303:114::18) by DM6PR12MB3801.namprd12.prod.outlook.com (2603:10b6:5:1cc::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.14; Thu, 12 May 2022 15:43:34 +0000 Received: from CO1NAM11FT030.eop-nam11.prod.protection.outlook.com (2603:10b6:303:114:cafe::1) by MW4P222CA0013.outlook.office365.com (2603:10b6:303:114::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.24 via Frontend Transport; Thu, 12 May 2022 15:43:33 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by CO1NAM11FT030.mail.protection.outlook.com (10.13.174.125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5250.13 via Frontend Transport; Thu, 12 May 2022 15:43:33 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Thu, 12 May 2022 15:43:33 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Thu, 12 May 2022 08:43:32 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Thu, 12 May 2022 08:43:28 -0700 From: Avihai Horon To: , "Michael S . Tsirkin" , "Cornelia Huck" , Paolo Bonzini , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Yishai Hadas , Jason Gunthorpe , "Mark Bloch" , Maor Gottlieb , Kirti Wankhede , Tarun Gupta Subject: [PATCH 2/9] vfio: Fix compilation errors caused by VFIO migration v1 deprecation Date: Thu, 12 May 2022 18:43:13 +0300 Message-ID: <20220512154320.19697-3-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220512154320.19697-1-avihaih@nvidia.com> References: <20220512154320.19697-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 388844e3-c2fc-47d2-bc9f-08da342e2bfb X-MS-TrafficTypeDiagnostic: DM6PR12MB3801:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9FKiJDPz9gm6R3+nnS9Oyk4/aSVG0aHs70yaB034tOVO/I/OXUTvQbt9VH+f7B/M21cRgHlksTGE/vZwmA1uYo45TwdYiydd5V54IGo+fo2JeyOcnpEIULV81+JrHKpFHG3msu9RxZDK7iTxGbKaCwVuUUoaGqe/kHcQoX75ZAPetner4zcpXVgJmY1wA+39jG2komji+sXUY5Nz/m1n957/nCas4kIrQi+D3DlgaCGWWiQzwyGpcwakAwLYRro5S0SMHAMCb38qEr+zqS4Gd2ps+ANbRHuSYjjSHj02Z2RZx3FdLr/4s7AL185E1BeidTiiWnTMo0mpNXA6Ciu+FU0Np8q8pQj01AkM2lUv4jnMRrvo/GmnBHjTkkHGdR5It/2MjXjSJoz8/bV/d4J2PC4Vn4onBAVL6T4WVU2OoNtgyR5YB9TipTBs+DU6+wJTuNIRNOdXyUKCKnE7QTbrgiTFh1LwhNf9q2HlJkL/XgUXWAYAjSJ3912sInlRhBWAhDUL2lrNZy+nCIRfyJ+B/EFxmIUmisspIyl7VUfgxTuZWifESEpTkecWfNMsrRs2jJS7MvEXYvr4d0Mvdv4xk2WJ5sGi/R70og1QUW3O2VsSAZGao5VGn8xVtIoX05X0iiXvGS9MEZvaMeNSt//yoFfdcMIzkQBrAbogbtxExMr5F3tqbPsC+gtivbRHpBb4ckyqbZFsQ4DAyECbnEVJng== X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(40470700004)(36840700001)(86362001)(81166007)(47076005)(36860700001)(36756003)(6666004)(82310400005)(1076003)(5660300002)(83380400001)(186003)(8936002)(336012)(426003)(40460700003)(2616005)(4326008)(8676002)(70206006)(107886003)(70586007)(508600001)(54906003)(316002)(26005)(356005)(7696005)(110136005)(2906002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2022 15:43:33.6154 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 388844e3-c2fc-47d2-bc9f-08da342e2bfb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT030.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3801 Received-SPF: softfail client-ip=2a01:111:f400:fe59::62c; envelope-from=avihaih@nvidia.com; helo=NAM12-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" VFIO migration protocol v1 was deprecated and as part of it some of the uAPI definitions were renamed. This caused compilation errors. Fix them. Signed-off-by: Avihai Horon --- hw/vfio/common.c | 6 +++--- hw/vfio/migration.c | 29 ++++++++++++++++------------- 2 files changed, 19 insertions(+), 16 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 159f910421..29982c7af8 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -355,7 +355,7 @@ static bool vfio_devices_all_dirty_tracking(VFIOContainer *container) } if ((vbasedev->pre_copy_dirty_page_tracking == ON_OFF_AUTO_OFF) - && (migration->device_state & VFIO_DEVICE_STATE_RUNNING)) { + && (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING)) { return false; } } @@ -381,8 +381,8 @@ static bool vfio_devices_all_running_and_saving(VFIOContainer *container) return false; } - if ((migration->device_state & VFIO_DEVICE_STATE_SAVING) && - (migration->device_state & VFIO_DEVICE_STATE_RUNNING)) { + if ((migration->device_state & VFIO_DEVICE_STATE_V1_SAVING) && + (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING)) { continue; } else { return false; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index ff6b45de6b..835608cd23 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -432,7 +432,7 @@ static int vfio_save_setup(QEMUFile *f, void *opaque) } ret = vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_MASK, - VFIO_DEVICE_STATE_SAVING); + VFIO_DEVICE_STATE_V1_SAVING); if (ret) { error_report("%s: Failed to set state SAVING", vbasedev->name); return ret; @@ -531,8 +531,8 @@ static int vfio_save_complete_precopy(QEMUFile *f, void *opaque) uint64_t data_size; int ret; - ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_RUNNING, - VFIO_DEVICE_STATE_SAVING); + ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_RUNNING, + VFIO_DEVICE_STATE_V1_SAVING); if (ret) { error_report("%s: Failed to set state STOP and SAVING", vbasedev->name); @@ -569,7 +569,7 @@ static int vfio_save_complete_precopy(QEMUFile *f, void *opaque) return ret; } - ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_SAVING, 0); + ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_SAVING, 0); if (ret) { error_report("%s: Failed to set state STOPPED", vbasedev->name); return ret; @@ -609,7 +609,7 @@ static int vfio_load_setup(QEMUFile *f, void *opaque) } ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_MASK, - VFIO_DEVICE_STATE_RESUMING); + VFIO_DEVICE_STATE_V1_RESUMING); if (ret) { error_report("%s: Failed to set state RESUMING", vbasedev->name); if (migration->region.mmaps) { @@ -717,20 +717,20 @@ static void vfio_vmstate_change(void *opaque, bool running, RunState state) * In both the above cases, set _RUNNING bit. */ mask = ~VFIO_DEVICE_STATE_MASK; - value = VFIO_DEVICE_STATE_RUNNING; + value = VFIO_DEVICE_STATE_V1_RUNNING; } else { /* * Here device state could be either _RUNNING or _SAVING|_RUNNING. Reset * _RUNNING bit */ - mask = ~VFIO_DEVICE_STATE_RUNNING; + mask = ~VFIO_DEVICE_STATE_V1_RUNNING; /* * When VM state transition to stop for savevm command, device should * start saving data. */ if (state == RUN_STATE_SAVE_VM) { - value = VFIO_DEVICE_STATE_SAVING; + value = VFIO_DEVICE_STATE_V1_SAVING; } else { value = 0; } @@ -767,9 +767,10 @@ static void vfio_migration_state_notifier(Notifier *notifier, void *data) case MIGRATION_STATUS_CANCELLED: case MIGRATION_STATUS_FAILED: bytes_transferred = 0; - ret = vfio_migration_set_state(vbasedev, - ~(VFIO_DEVICE_STATE_SAVING | VFIO_DEVICE_STATE_RESUMING), - VFIO_DEVICE_STATE_RUNNING); + ret = vfio_migration_set_state( + vbasedev, + ~(VFIO_DEVICE_STATE_V1_SAVING | VFIO_DEVICE_STATE_V1_RESUMING), + VFIO_DEVICE_STATE_V1_RUNNING); if (ret) { error_report("%s: Failed to set state RUNNING", vbasedev->name); } @@ -864,8 +865,10 @@ int vfio_migration_probe(VFIODevice *vbasedev, Error **errp) goto add_blocker; } - ret = vfio_get_dev_region_info(vbasedev, VFIO_REGION_TYPE_MIGRATION, - VFIO_REGION_SUBTYPE_MIGRATION, &info); + ret = vfio_get_dev_region_info(vbasedev, + VFIO_REGION_TYPE_MIGRATION_DEPRECATED, + VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED, + &info); if (ret) { goto add_blocker; } From patchwork Thu May 12 15:43:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 12847846 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3F288C433EF for ; Thu, 12 May 2022 15:46:58 +0000 (UTC) Received: from localhost ([::1]:45434 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1npB1l-0000nM-De for qemu-devel@archiver.kernel.org; Thu, 12 May 2022 11:46:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41924) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyd-0006Ht-55 for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:46 -0400 Received: from mail-dm6nam10on2060d.outbound.protection.outlook.com ([2a01:111:f400:7e88::60d]:28609 helo=NAM10-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyb-0005JM-Bz for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:42 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=L/EiBSggeJS7OtHiv7Xo0Gn/mCQ3+WTKWMvaHNqk56rUH7GObDcDtpxbJS2yuqyIb2lMc8V8Uucy7/0CWtaXKY+KrrBbtBpKxrOHlwG2E8lvUd/xgcBLTgKMdac47PiHVVAUynglUlsiIhF8SG3yx7eNhg7Omi8Q/T7TnoP9+jB2917/8S0HNDT8LS0/HLKI5KBIztqzkShCIa0OkxqVP4ri6b4vV+RZzUGK22z3a8n6Ch0wYbdmDZlhg12/IoYv6ZJCfg33iLEE8HvLxKUoCMv09HEH/8S/0sA+XpQXdQWTNMkT1xh0sAjyKsRs9j0JwpUG+vbhJV4JbPB+iPjbOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PPkHnwWAu6iKElPml2VAVqhFysl0efHK1imZX3pHVg4=; b=GHl33JYfIOBmWEEz3bWZlWDVKlwKs8fStQLIaK3Uhhld8gCYlse8JoEZWy9Kphf/RTbMEysPVOitYJDffBdLWcT0v7wG5/FXrBgNkLQC7x5yVLtEn0b2zTfEaVx8KOStBXRn+fYfctUJsOu2b6+virQND66EP5knmj0gffEEhJDU5egnhb4UoqKzi0t6aX1zfuc9IWQfe8CPQ9NNyJaJBZs0sUOK4/9He1SlpZyRFNNEU9w0P3cMB1ZFIjzAdlgzdTxn03Gjdgjf5UvPIm6L9+L5eyMewzRix6dm2F6ROwtYsHLGfvNqsqqqZXdL/NOyB/o3A/g58HNUeFLhQdq80Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PPkHnwWAu6iKElPml2VAVqhFysl0efHK1imZX3pHVg4=; b=oe68BBN2itJDAUIBF7S0LoNhL02QzpPXYaqOWLMYJ+oRpQ7wTVcz314XVpHVXocg5xB50InlV21tHCakzolJu+4VheUX3EpGY5JgSer4FVKc3oYQOnMuMPh67HEU4P1KkDEhDrwwcqAzA52rVujPIQd739PwnYAFxIRcfau+tpm1Z64ow2uIIDCO+3B2K0JR0PnDAiEhiBCGzrPR314uUvJc4rLVlGGyiZ9El7OBJlifJhhQADYNMcdDWDz42F3F2Q9fv9tZz2GfWXog42yJ+IcTwLs3XeCsc2in80SRT1oG2qybBiONcX9adA42u0Yu8XrjtuN8TI/YOXFU6h4g/A== Received: from MW4P222CA0011.NAMP222.PROD.OUTLOOK.COM (2603:10b6:303:114::16) by DM6PR12MB4217.namprd12.prod.outlook.com (2603:10b6:5:219::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.14; Thu, 12 May 2022 15:43:37 +0000 Received: from CO1NAM11FT030.eop-nam11.prod.protection.outlook.com (2603:10b6:303:114:cafe::59) by MW4P222CA0011.outlook.office365.com (2603:10b6:303:114::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.22 via Frontend Transport; Thu, 12 May 2022 15:43:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT030.mail.protection.outlook.com (10.13.174.125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5250.13 via Frontend Transport; Thu, 12 May 2022 15:43:37 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Thu, 12 May 2022 15:43:37 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Thu, 12 May 2022 08:43:36 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Thu, 12 May 2022 08:43:32 -0700 From: Avihai Horon To: , "Michael S . Tsirkin" , "Cornelia Huck" , Paolo Bonzini , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Yishai Hadas , Jason Gunthorpe , "Mark Bloch" , Maor Gottlieb , Kirti Wankhede , Tarun Gupta Subject: [PATCH 3/9] vfio/migration: Fix NULL pointer dereference bug Date: Thu, 12 May 2022 18:43:14 +0300 Message-ID: <20220512154320.19697-4-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220512154320.19697-1-avihaih@nvidia.com> References: <20220512154320.19697-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6bcf9771-08ec-458f-5b1d-08da342e2e4d X-MS-TrafficTypeDiagnostic: DM6PR12MB4217:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aWbzYujY4VlvlL3T89uAxwyR6xCFOy/n2eFz0b9rPaJCfMyRpVZSBPQ1K0Mh8NND61N+RHie9jyzf3VCbJGGJib+TnePCjWdFOTYOOWvfX0lRcQM7uLuIE8qPn1wznQA1ifu/4X6bseDJ5NBIk5nOL2llfv9KuxOBgxD9v5dnWeElLS10T+9E8ek1CIcE1OU+I7vX0kHszAFJL8s/htb8xhaMiAwoJfzSY4NhbWMkv6z3l/09LpN97F99N6gzqe/MFn/ESgVbAVLaKxi6CKSRVy8EYB+QZbcnutNCvaBG7MeEtNZhYjBye5J9LTAzS1SVIwWRMhPcosEusH+0yVuQAxsjx/VUqyqs7sithwUKcKSkq6zFD1XmxMPvrcJ4KAlfA955HTrWADCN6HKLKFTgghhj9XPVMXu+bexTYt+JxgtIG/RNOqiyKqEq9t9lgsDowngDV/e7kXneaeGlQJGqkH4gM7fTIh/9FRaPWVAixwx7DJwB/P8EKicr/pTYTJAeVLjvYrQlaehkcsmkEvKRX9Nva73W5DV2sKQd4FbxzIUutkyxjEmHTM2JdDZMWh7f+KrnQPP6iagD974YisLyCYc7QS0pFWkUScG7Rstp4KVvnTlH4qWf0CMyhFAs8Hc9AtR5IQ2pdGI1Kxf3zzQYSyAl+kmFlP1VVcsCswAnTE6TUk4cvxclnutQUu5BnlN2pJLx62o+f0HKHcu5dXWEA== X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(186003)(2906002)(6666004)(508600001)(5660300002)(86362001)(2616005)(36860700001)(36756003)(40460700003)(336012)(426003)(47076005)(82310400005)(83380400001)(316002)(8676002)(4326008)(81166007)(70586007)(110136005)(70206006)(356005)(7696005)(8936002)(107886003)(1076003)(54906003)(26005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2022 15:43:37.5214 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6bcf9771-08ec-458f-5b1d-08da342e2e4d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT030.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4217 Received-SPF: softfail client-ip=2a01:111:f400:7e88::60d; envelope-from=avihaih@nvidia.com; helo=NAM10-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" As part of its error flow, vfio_vmstate_change() accesses MigrationState->to_dst_file without any checks. This can cause a NULL pointer dereference if the error flow is taken and MigrationState->to_dst_file is not set. For example, this can happen if VM is started or stopped not during migration and vfio_vmstate_change() error flow is taken, as MigrationState->to_dst_file is not set at that time. Fix it by checking that MigrationState->to_dst_file is set before using it. Fixes: 02a7e71b1e5b ("vfio: Add VM state change handler to know state of VM") Signed-off-by: Avihai Horon Reviewed-by: Juan Quintela --- hw/vfio/migration.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 835608cd23..21e8f9d4d4 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -744,7 +744,9 @@ static void vfio_vmstate_change(void *opaque, bool running, RunState state) */ error_report("%s: Failed to set device state 0x%x", vbasedev->name, (migration->device_state & mask) | value); - qemu_file_set_error(migrate_get_current()->to_dst_file, ret); + if (migrate_get_current()->to_dst_file) { + qemu_file_set_error(migrate_get_current()->to_dst_file, ret); + } } vbasedev->migration->vm_running = running; trace_vfio_vmstate_change(vbasedev->name, running, RunState_str(state), From patchwork Thu May 12 15:43:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 12847845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 68D1DC433F5 for ; Thu, 12 May 2022 15:46:56 +0000 (UTC) Received: from localhost ([::1]:45448 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1npB1j-0000nj-Ie for qemu-devel@archiver.kernel.org; Thu, 12 May 2022 11:46:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41960) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyg-0006Jr-Mj for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:47 -0400 Received: from mail-dm6nam10on20617.outbound.protection.outlook.com ([2a01:111:f400:7e88::617]:22401 helo=NAM10-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAye-0005KE-T9 for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:46 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fl6/ZMPef4EXxxf0tZi99awUK2/z5B1Gmrcabc4h9j4CSgV05f5KGu7E99SuWfwUcGjDPF1o/dEtM7JpmsMrVhdhGEv3pXrs8w6amkvQqQC2hjaaFet7AgJoIYzhooICTUTo5RyW05YVbC0q/cw0hFQc3QLz0O86NPZWjXKyWeESueA7WILSzCyPOj+4Ih6aMHO1PmK7L3eICfRo/ZfdFrSkmU40bG95AWYPVPdaSMR28ZHuuxDAd8EKQn3fVSbgN+ztaY1uqSzi3VX58cv1YEPruCcA98UUpZyQDqspwBsTyNsKQvFtQJUrdvCUkx3wX35tdIFGCbyWKLVAMNKCTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+Y8kNitKlbtxGQVO/kzT272waoEvNmR/0NCAs/q6IVE=; b=W/act7sXeuW5l2AIQ7QOktTP9gJNoPeKmiCwn5h2joLtnDzjFeoa/kCX9Sd1swns5YMpbVdrLduwBZejKlmWUhmUmNLy6PrEZzGg+QP6WxO66JOp8SjYtZTKcsJ7WXPYNhi2dUkfvhSqSUKltinkn3JYn+edWhEH+WqTSMCg4d1rL5z9PotJvVhY05eYTCiOOwj37q/6vk/KIhXwYmgejXPYiLMdHdyLWe+k2MzieF/R1E6iXd62GiuocqBUfp6Z0NC8Cqs98UNPIezG7UlKo/63nBd7JqL9iOd4leg/pRnFyeJETjpnxPj82lZ0NnOKqeDgXiusWDLJkk+kPo+9Mg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+Y8kNitKlbtxGQVO/kzT272waoEvNmR/0NCAs/q6IVE=; b=J6c6IqJDvhpy1KFoguL9qi+CYISEBA+yk8lGYrKbZpj0JWwA9P0TDbLwENTV3VevJ3soON44usl9GCyVXoJNpt7+zieiOYmRof0I71zwdj/PnmghIaQdQTBV7zekyW3xh3xyfys09MqYlMEqpKEhiAT1GPelnnOSBPjiEdYZHZL9V6R8WGdPJ0FLzeW/vyRx/UeHJYPhXOu/Gf/BrsGAMQMLSVVGNn18zm4XpWGEPLUle9D6/LpO9xxKjFNZAy9B5XNjt5WAMCdBfNgv0zj0kAWBijxaKE0yiG1hyMzk8nd6WFubGwtZ15mMYsYF1fv+63wa5beG4VWAyIITq0oVzA== Received: from MW4PR03CA0030.namprd03.prod.outlook.com (2603:10b6:303:8f::35) by MN2PR12MB3550.namprd12.prod.outlook.com (2603:10b6:208:108::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.14; Thu, 12 May 2022 15:43:41 +0000 Received: from CO1NAM11FT063.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8f:cafe::8f) by MW4PR03CA0030.outlook.office365.com (2603:10b6:303:8f::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.23 via Frontend Transport; Thu, 12 May 2022 15:43:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT063.mail.protection.outlook.com (10.13.175.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5250.13 via Frontend Transport; Thu, 12 May 2022 15:43:41 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Thu, 12 May 2022 15:43:40 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Thu, 12 May 2022 08:43:40 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Thu, 12 May 2022 08:43:36 -0700 From: Avihai Horon To: , "Michael S . Tsirkin" , "Cornelia Huck" , Paolo Bonzini , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Yishai Hadas , Jason Gunthorpe , "Mark Bloch" , Maor Gottlieb , Kirti Wankhede , Tarun Gupta Subject: [PATCH 4/9] vfio/migration: Skip pre-copy if dirty page tracking is not supported Date: Thu, 12 May 2022 18:43:15 +0300 Message-ID: <20220512154320.19697-5-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220512154320.19697-1-avihaih@nvidia.com> References: <20220512154320.19697-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d29cce10-dbbb-45bb-0d17-08da342e30b8 X-MS-TrafficTypeDiagnostic: MN2PR12MB3550:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NYI/D8hWMZ8/wEDMPDgGefGkyXr8zufYI/xwlH5pt9nAa0tF26upjLsj2EBSetVzLZScOKtdOC4MP6PReXNo1XdFx33l0NsOs7NqQLAj6YTaob9PNFfhdN9xDtHpPh8JKqgBkkbDoM54SxYWQG1/9VYhdonnll9jCwRguTSB7t+KvLykesiZl/9e+Eri6e3czcXdMdtDp6UaV9LmyixYwD4yIp8YZtxtY2Kjq3dxcGojbXq1BDg93YD2kIqjzsfyF9/15k4BZne4dv1cN0dv95u821CJN2F0RModMcSjDlZkoNfbs9BpJO3niN1ST0wjWr4aNgUlFMcTa+khEYZAYrBpEMdNWW2+tmlGMAZQXmP+n1fVfuX7v1hej9qx2m78oQNPMins/PJyhdH803D1Z4qb+86yrk9rnoqQYIvxXeoO+4kZ1FEWD9nL2bz604zryrZx1JNh8vHpeGbPrklme7DkTwIUmPDjG62VZcm+oREOS9iSli+BKI7NSIDO4Oz4JK1pllGDfczQkSUM75pCAaaiT6jgMpA3DON2XxOgYQgO8tMijXwLH2eOz+YpJ1rkYGcing7zCJWOBk5b0aaQXlim/7rcv7s/qG4t3X1raMWQ2h3PVCPdpTXsth3hwon55MSiYoyLMWggIFfkIJNrUENz+KEUYtByR0pi+Ufiwv9BgcVGz1xN92pecAx63duTapFSSi6RR2yK9nKwGdo7h/phCg/3WVjdXaEpGh4bn5U= X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(8936002)(508600001)(8676002)(2616005)(70586007)(4326008)(70206006)(86362001)(356005)(5660300002)(81166007)(1076003)(36860700001)(107886003)(36756003)(7696005)(40460700003)(426003)(336012)(316002)(6666004)(82310400005)(47076005)(2906002)(110136005)(54906003)(186003)(26005)(83380400001)(14143004)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2022 15:43:41.5761 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d29cce10-dbbb-45bb-0d17-08da342e30b8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT063.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3550 Received-SPF: softfail client-ip=2a01:111:f400:7e88::617; envelope-from=avihaih@nvidia.com; helo=NAM10-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Currently, if IOMMU of a VFIO container doesn't support dirty page tracking, migration is blocked completely. This is because a DMA-able VFIO device can dirty RAM pages without updating QEMU about it, thus breaking the migration. However, this doesn't mean that migration can't be done at all. If migration pre-copy phase is skipped, the VFIO device doesn't have a chance to dirty RAM pages that have been migrated already, thus eliminating the problem previously mentioned. Hence, in such case allow migration but skip pre-copy phase. Signed-off-by: Avihai Horon --- hw/vfio/migration.c | 9 ++++++++- migration/migration.c | 5 +++++ migration/migration.h | 3 +++ 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 21e8f9d4d4..d4b6653026 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -863,10 +863,17 @@ int vfio_migration_probe(VFIODevice *vbasedev, Error **errp) struct vfio_region_info *info = NULL; int ret = -ENOTSUP; - if (!vbasedev->enable_migration || !container->dirty_pages_supported) { + if (!vbasedev->enable_migration) { goto add_blocker; } + if (!container->dirty_pages_supported) { + warn_report( + "%s: IOMMU of the device's VFIO container doesn't support dirty page tracking, migration pre-copy phase will be skipped", + vbasedev->name); + migrate_get_current()->skip_precopy = true; + } + ret = vfio_get_dev_region_info(vbasedev, VFIO_REGION_TYPE_MIGRATION_DEPRECATED, VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED, diff --git a/migration/migration.c b/migration/migration.c index 5a31b23bd6..668343508d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3593,6 +3593,11 @@ static MigIterateState migration_iteration_run(MigrationState *s) uint64_t pending_size, pend_pre, pend_compat, pend_post; bool in_postcopy = s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE; + if (s->skip_precopy) { + migration_completion(s); + return MIG_ITERATE_BREAK; + } + qemu_savevm_state_pending(s->to_dst_file, s->threshold_size, &pend_pre, &pend_compat, &pend_post); pending_size = pend_pre + pend_compat + pend_post; diff --git a/migration/migration.h b/migration/migration.h index a863032b71..876713e7e1 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -332,6 +332,9 @@ struct MigrationState { * This save hostname when out-going migration starts */ char *hostname; + + /* Whether to skip pre-copy phase of migration or not */ + bool skip_precopy; }; void migrate_set_state(int *state, int old_state, int new_state); From patchwork Thu May 12 15:43:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 12847848 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 83D63C433F5 for ; Thu, 12 May 2022 15:50:02 +0000 (UTC) Received: from localhost ([::1]:53240 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1npB4j-00068W-KE for qemu-devel@archiver.kernel.org; Thu, 12 May 2022 11:50:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41986) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyk-0006LH-UO for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:50 -0400 Received: from mail-mw2nam12on20619.outbound.protection.outlook.com ([2a01:111:f400:fe5a::619]:51137 helo=NAM12-MW2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyj-0005LG-4R for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:50 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bSQehNh7zkOLkIeBNYVWDADZIDRKnl+oo60nMC2YqZ+JuUnRSkqPw/y0xt4LzCp6uOHLNNnORoxBXiNJKkA7aAtBVII0cnhywXw8NjPR+ga4NytddGBRJsrWufqa86JswRqVw3eL9LEipI2lkkszHn6yqt7N58xFNREkAFge+mC+ZhsLXkVDPffFLV4SHVaaMMPjEtBXwl3liuqpQIdW3F0aM3hPTxqDj3cBWF7/XrOq81bkkN/ECoOORoGm8ioxYlOkh5j95Alb3gmTmQnih5K179RFwaugPPqdgBuLbV5Z1+oyzM7OrtCcvtodV9nGt7LBaaYbObNQUezTtEtl9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=33dWet8F7c64nY1h8u8FSdJegji/Xrkz5DD1ILOqYGM=; b=HWeOWD4aZMf8OfDpNJ+XXKRqzEtK10G8xA09TUEgxY1z7vd/Np03Ph2n23767iEEzi9BZkLsd0H7x/QBIIhU68lYdHdG0fCk6HIMNIgBtbGlLQ1TJdQ0ap3/udzsLjVTk2f0nWkRSEaPC80upAT0Jv2HNrcMUNch6LDpgWK/FsGLixDKuD1tE37tJQL6uYbfJ8Uy5mnKaFf9EBBje+SVUWMjTDrVTfQf0yYJHD3h7FVqb3oD5+EqE+blZypBYXMhUk164LmZ3B6LztEMBzYaoHL15UlRtfgTwgDGYngAlLJv9f91WXYWU2b7nZ5S/eobCLg0iBCBnpKTGOMkW9K6gg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=33dWet8F7c64nY1h8u8FSdJegji/Xrkz5DD1ILOqYGM=; b=OtW3rX7LNYxwnni7nMkSBTv7/OBQxCZZRHnQUK0vgNJ6RvFEmQ5i+SCcWmi57gdZpHqyg3/tJQFeYXG3W1f/xC+o4xarDcJNymOwHOaUEGgzF2hjNQrLCMoLzHt8egQM4u2F9DEL+yX3WM1tT8exsp1YjJZ6/fuOjdeadpIQxxWGaSAYoYht1+MlOZ17fOtCNsem5s93DayP7cC84QTHlbuEOBofl7Tzm+5zWSbXh8w5DkwXZKcY5NEebixHu2uav6cfXf5g0T0+O6ekbrEqjx5UFJnF4YGl6p+0TZYHwrBbwLvAQgknQWDDxDm8S/NO/cXiwz3DKcoOzObZIKdD8w== Received: from MW4PR04CA0247.namprd04.prod.outlook.com (2603:10b6:303:88::12) by BL0PR12MB4962.namprd12.prod.outlook.com (2603:10b6:208:17e::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.13; Thu, 12 May 2022 15:43:46 +0000 Received: from CO1NAM11FT047.eop-nam11.prod.protection.outlook.com (2603:10b6:303:88:cafe::bf) by MW4PR04CA0247.outlook.office365.com (2603:10b6:303:88::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.22 via Frontend Transport; Thu, 12 May 2022 15:43:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT047.mail.protection.outlook.com (10.13.174.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5250.13 via Frontend Transport; Thu, 12 May 2022 15:43:45 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Thu, 12 May 2022 15:43:45 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Thu, 12 May 2022 08:43:44 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Thu, 12 May 2022 08:43:40 -0700 From: Avihai Horon To: , "Michael S . Tsirkin" , "Cornelia Huck" , Paolo Bonzini , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Yishai Hadas , Jason Gunthorpe , "Mark Bloch" , Maor Gottlieb , Kirti Wankhede , Tarun Gupta Subject: [PATCH 5/9] migration/qemu-file: Add qemu_file_get_to_fd() Date: Thu, 12 May 2022 18:43:16 +0300 Message-ID: <20220512154320.19697-6-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220512154320.19697-1-avihaih@nvidia.com> References: <20220512154320.19697-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 03e15a26-b992-469e-4d08-08da342e3326 X-MS-TrafficTypeDiagnostic: BL0PR12MB4962:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3ldAWDCA7Gcmzdiiz47fjc+xRxXBRLh8Sc5/vKPvbntd2AKYisW602nPEcjJSo4Mvh9ft0nIFvpP1mEAeVnjtr3ilhmxfATtnn6oyV24qaePMnl4aiyXIP149mDrMZEduEyBmtISC81SJRsWaXKowNR/zC4qRgwlQbcHGg7b4zuUIZf1R8pbb7bMG0HrQ0X3fVfLTLL+TSy1sXxWiyz003qomF09vVDqw4W9rDy4JpkDL6nZtA+QYtrNRhIt0o7t1OADZPexWUzO+JlK1H+4Qmq4jH1+Wluzqs3CMDxki5zB9UERsC+6Urnzjs9Mi9i7Prt1yE6XxMFNIB0Hb6D0XgVFeN0mXr1jjQTt4wg2p5ZAABR/OR3l+OYV3Iknk7KXpLIVtQtm3eOPOOoTx7FdBEFBrJMCQlGx/VmDKb6ZdJnpBPeFWO7+TW5V9hasFjijIcBsZ5A9bW9smEC8pXpz39m8BIpirbzYbMxTMnFANLT5mkQNod5Yy7B/caqH0Bds4d9UjVm7sEssDuEJvxgi8uAgfaXP717/5ON9ARtMzLqswyyM9Fiknoe24BtuCT/SfN8s1yMDLZ0NL/FWuPqA+kMXiF4F0sQhMyxlSb5gkY5s+tcC9fib899GCyMibibAdPHQ3bvRFleDPDSFBFJz87kkWHscq1vE8nN6TdlzGXOY2s9fbVTE92rnTuDr4LAX3MLVBL+7H9d9A3w+neG1OQ== X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(40470700004)(36840700001)(8676002)(36756003)(4326008)(8936002)(82310400005)(70586007)(70206006)(5660300002)(54906003)(316002)(2616005)(6666004)(110136005)(2906002)(508600001)(7696005)(356005)(81166007)(26005)(36860700001)(86362001)(186003)(107886003)(1076003)(47076005)(83380400001)(40460700003)(426003)(336012)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2022 15:43:45.6540 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 03e15a26-b992-469e-4d08-08da342e3326 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT047.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4962 Received-SPF: softfail client-ip=2a01:111:f400:fe5a::619; envelope-from=avihaih@nvidia.com; helo=NAM12-MW2-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add new function qemu_file_get_to_fd() that allows reading data from QEMUFile and writing it straight into a given fd. This will be used later in VFIO migration code. Signed-off-by: Avihai Horon --- migration/qemu-file.c | 34 ++++++++++++++++++++++++++++++++++ migration/qemu-file.h | 1 + 2 files changed, 35 insertions(+) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 1479cddad9..cad3d32eb3 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -867,3 +867,37 @@ QIOChannel *qemu_file_get_ioc(QEMUFile *file) { return file->has_ioc ? QIO_CHANNEL(file->opaque) : NULL; } + +/* + * Read size bytes from QEMUFile f and write them to fd. + */ +int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size) +{ + while (size) { + size_t pending = f->buf_size - f->buf_index; + ssize_t rc; + + if (!pending) { + rc = qemu_fill_buffer(f); + if (rc < 0) { + return rc; + } + if (rc == 0) { + return -1; + } + continue; + } + + rc = write(fd, f->buf + f->buf_index, MIN(pending, size)); + if (rc < 0) { + return rc; + } + if (rc == 0) { + return -1; + } + f->buf_index += rc; + size -= rc; + } + + return 0; +} diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 3f36d4dc8c..dd26037450 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -162,6 +162,7 @@ int qemu_file_shutdown(QEMUFile *f); QEMUFile *qemu_file_get_return_path(QEMUFile *f); void qemu_fflush(QEMUFile *f); void qemu_file_set_blocking(QEMUFile *f, bool block); +int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size); void ram_control_before_iterate(QEMUFile *f, uint64_t flags); void ram_control_after_iterate(QEMUFile *f, uint64_t flags); From patchwork Thu May 12 15:43:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 12847854 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C1125C433F5 for ; Thu, 12 May 2022 15:53:51 +0000 (UTC) Received: from localhost ([::1]:33976 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1npB8Q-0003ls-To for qemu-devel@archiver.kernel.org; Thu, 12 May 2022 11:53:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42024) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAys-0006SL-7n for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:58 -0400 Received: from mail-co1nam11on20607.outbound.protection.outlook.com ([2a01:111:f400:7eab::607]:15521 helo=NAM11-CO1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyo-0005LW-Fs for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:57 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MxxR6ASSrE4gUZ0en3KH/YYtF8Gh2vFOpvef6WhInOlwjxvsCHQdGZJRhFlD1p1Ao65gaWkLOe5b3DHvTUoQtuLb3ubZUdF13L/GW5cEmUFdgDjeROEkTY4zWpFvwhbf2DpyEpqAY3hsJfkPdHepCmxdzo7meDGBGyWKi3kZ0An9ryplc0KoQTMtBj1lvP393Q8s1o/Hry9yZvXEUyAT47xEN6lGS9Vc/JwxTT9wwuYBv+9kC675dA2rgr1c2ya2LMyt4BTwpW5fMOyMGKMWRqwfIxKN6/YHq3RMOglF9cfyVrkcqoMZWBfheGzwo12V+KYMd+vYlKBEEY+m8VCx5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NbWyXsElyZSxu9Fq9WDLufpfeC+qEKquVJCktKZnuzw=; b=czMzYO/PmRyOa9Hg3Mv7f4p7thcGJGY0mVUR2Id8TZQUqAlxMcN8xMPAz6tNC95P1h3/blZSuU5mg7ScZ5IJGTpzjZ5JHcEgu0xc6orYqeusEadVq5AAlg3Y48L4MFiJbXbXie7h+JiNzdnk74AfdGbYtKvpZYf2w/Seq/hXNsym2kH8R2NelgKXVyrAP3a/0PKvZuvUEH/oe73nTbM3iHkDkSVBqv/gSq2bD4tUtH0ALJQ5Msb2eSX0T+Pg73Dy//3wQToHVW1YCcA2HMRDTY1m5Ql0R6Ht9xo3BW0gb7pnU+DCti4auxYWzzhYXCLmxz8KnbteNbMwXpVGr8V6Rw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NbWyXsElyZSxu9Fq9WDLufpfeC+qEKquVJCktKZnuzw=; b=hJal3SoVESp8EhORUNC/JxnXAtEZJmsu/C8Ackvyc/P4TIO9AW9EmhsWMlPpp0q8+gPEs1+XQFvbxq3iqQ6XyHLxxIowMSd+ps7zwGD5A4v9USzeTkYntzThzT/8RcvcVPAHpNLFSJSJ/gcWRQhLYSmj8JKXcoM7ijTodt13p0FI5c1IGhFYtibAyEl0G1EU9ebiaowZosjB1bnrB5FZR2CvhXZmku3F69v30mcFN/hPZOSIf3/myuP1nUfoYBdxbzld6G089eSrTOp342+6hscHOJguLubn0bo5ikoUdmH7PAPL0+zzyGFLiL6hoo1R9ii4gvpeFQ5ArpzrHA5knw== Received: from MW4PR03CA0080.namprd03.prod.outlook.com (2603:10b6:303:b6::25) by BYAPR12MB5701.namprd12.prod.outlook.com (2603:10b6:a03:a6::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.13; Thu, 12 May 2022 15:43:50 +0000 Received: from CO1NAM11FT051.eop-nam11.prod.protection.outlook.com (2603:10b6:303:b6:cafe::67) by MW4PR03CA0080.outlook.office365.com (2603:10b6:303:b6::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.13 via Frontend Transport; Thu, 12 May 2022 15:43:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.238) by CO1NAM11FT051.mail.protection.outlook.com (10.13.174.114) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5250.13 via Frontend Transport; Thu, 12 May 2022 15:43:49 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Thu, 12 May 2022 15:43:49 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Thu, 12 May 2022 08:43:48 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Thu, 12 May 2022 08:43:45 -0700 From: Avihai Horon To: , "Michael S . Tsirkin" , "Cornelia Huck" , Paolo Bonzini , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Yishai Hadas , Jason Gunthorpe , "Mark Bloch" , Maor Gottlieb , Kirti Wankhede , Tarun Gupta Subject: [PATCH 6/9] vfio/migration: Implement VFIO migration protocol v2 Date: Thu, 12 May 2022 18:43:17 +0300 Message-ID: <20220512154320.19697-7-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220512154320.19697-1-avihaih@nvidia.com> References: <20220512154320.19697-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fc1853a8-f83a-462b-f5b4-08da342e35a5 X-MS-TrafficTypeDiagnostic: BYAPR12MB5701:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cQbnGQUDreJhgTd42rNNoXbrj80ZNFxoDa5iE7toBFV/xBqr3+dxAewJ5brWUMeJ4wSZS6bzbdntugdJZPCccn7DDKXxsJdicXqH1ntFZj3LNEz9/e1BJxzirjMgnbQ8eTzLovQ9nPieKYY3gqddC6dB0IV7V0Q6+1S5FVMXRTlntiUO87XCP4Am9LFRZziJpM092A8HcCVHpJnJ7psNXNFr0QgIfIjItg7ao0tLkFNj3AQdyxvBt0tR5MDW8Va6OLkm+wOB5V33LnX2ZMqAXW+v0FrVvtxcAFMVJZEgKo1mNq3uUoTReWMaItkiDd+04htjGFagCdDWLk1aCUpomPOic7AGgwHrlu6SyXAdy+9mZ8zYn44hOWkOZ0zuuD1nHM2FkbByfNfvfYSVfZAAaH9bzaeD739UCsLawDc8bm/gOq/LDaQOeRCiWi0ewngHh6famS0SzWm2shWXu0UKd0eCTN5npHI+ZLLrul0HNToVYNYqJCE2/kfYrSuqRNG+En5qpuO7E7Ozj7s/wq1d0Rk+q4JASumURfpeZMsGTIcz82yRs/XUHfjQIuBIDGNkyjOCkIMDxNpTg+A65HlQD9CWGuHgz8QoG07kq2FHpy5fsEOlgcPagt37PePoRGQqcGbwuAw75qGHvDAN4AAe8xI4pjddfjGy3381ZlMBO0WnjEcc/uUc9DxP2h0NuRf7jagVrTLOx/yBxiExN5aGqM61+zFjS6RKdzySKPKh5Joey4n6CrGbNyepTBJXcLLOEnYPryLzc/wB/LqWfCcDddRisfbbBtS8dykdX33BFoA= X-Forefront-Antispam-Report: CIP:12.22.5.238; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(36840700001)(46966006)(7696005)(81166007)(356005)(26005)(2906002)(8936002)(6666004)(508600001)(86362001)(5660300002)(966005)(30864003)(110136005)(54906003)(36860700001)(83380400001)(316002)(82310400005)(40460700003)(70586007)(47076005)(336012)(426003)(8676002)(4326008)(70206006)(2616005)(107886003)(1076003)(186003)(36756003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2022 15:43:49.8149 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fc1853a8-f83a-462b-f5b4-08da342e35a5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.238]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT051.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB5701 Received-SPF: softfail client-ip=2a01:111:f400:7eab::607; envelope-from=avihaih@nvidia.com; helo=NAM11-CO1-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Replace the current VFIO migration protocol v1 implementation with a new implementation corresponding to VFIO migration protocol v2. The main changes are: - VFIO device state is now represented as a finite state machine instead of a bitmap. - Migration interface with kernel is now done using VFIO_DEVICE_FEATURE ioctl and normal read() and write() instead of the migration region. - As VFIO migration protocol v2 currently doesn't support the pre-copy phase of migration, .save_live_pending and .save_live_iterate handlers plus pre-copy relevant code are removed. Detailed information about VFIO migration protocol v2 and difference compared to v1 can be found here [1]. [1] https://lore.kernel.org/all/20220224142024.147653-10-yishaih@nvidia.com/ Signed-off-by: Avihai Horon --- hw/vfio/common.c | 21 +- hw/vfio/migration.c | 628 +++++++--------------------------- hw/vfio/trace-events | 9 +- include/hw/vfio/vfio-common.h | 8 +- 4 files changed, 153 insertions(+), 513 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 29982c7af8..4c6baa5a79 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -40,6 +40,7 @@ #include "trace.h" #include "qapi/error.h" #include "migration/migration.h" +#include "migration/misc.h" #include "sysemu/tpm.h" VFIOGroupList vfio_group_list = @@ -354,8 +355,9 @@ static bool vfio_devices_all_dirty_tracking(VFIOContainer *container) return false; } - if ((vbasedev->pre_copy_dirty_page_tracking == ON_OFF_AUTO_OFF) - && (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING)) { + if ((vbasedev->pre_copy_dirty_page_tracking == ON_OFF_AUTO_OFF) && + (migration->device_state == VFIO_DEVICE_STATE_RUNNING || + migration->device_state == VFIO_DEVICE_STATE_RUNNING_P2P)) { return false; } } @@ -363,13 +365,16 @@ static bool vfio_devices_all_dirty_tracking(VFIOContainer *container) return true; } -static bool vfio_devices_all_running_and_saving(VFIOContainer *container) +/* + * Check if all VFIO devices are running and migration is active, which is + * essentially equivalent to the migration being in pre-copy phase. + */ +static bool vfio_devices_all_running_and_mig_active(VFIOContainer *container) { VFIOGroup *group; VFIODevice *vbasedev; - MigrationState *ms = migrate_get_current(); - if (!migration_is_setup_or_active(ms->state)) { + if (!migration_is_active(migrate_get_current())) { return false; } @@ -381,8 +386,8 @@ static bool vfio_devices_all_running_and_saving(VFIOContainer *container) return false; } - if ((migration->device_state & VFIO_DEVICE_STATE_V1_SAVING) && - (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING)) { + if (migration->device_state == VFIO_DEVICE_STATE_RUNNING || + migration->device_state == VFIO_DEVICE_STATE_RUNNING_P2P) { continue; } else { return false; @@ -461,7 +466,7 @@ static int vfio_dma_unmap(VFIOContainer *container, }; if (iotlb && container->dirty_pages_supported && - vfio_devices_all_running_and_saving(container)) { + vfio_devices_all_running_and_mig_active(container)) { return vfio_dma_unmap_bitmap(container, iova, size, iotlb); } diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index d4b6653026..8943ccbace 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -44,309 +44,96 @@ #define VFIO_MIG_FLAG_DEV_SETUP_STATE (0xffffffffef100003ULL) #define VFIO_MIG_FLAG_DEV_DATA_STATE (0xffffffffef100004ULL) +#define VFIO_MIG_DATA_BUFFER_SIZE (1024 * 1024) + static int64_t bytes_transferred; -static inline int vfio_mig_access(VFIODevice *vbasedev, void *val, int count, - off_t off, bool iswrite) +static const char *mig_state_to_str(enum vfio_device_mig_state state) { - int ret; - - ret = iswrite ? pwrite(vbasedev->fd, val, count, off) : - pread(vbasedev->fd, val, count, off); - if (ret < count) { - error_report("vfio_mig_%s %d byte %s: failed at offset 0x%" - HWADDR_PRIx", err: %s", iswrite ? "write" : "read", count, - vbasedev->name, off, strerror(errno)); - return (ret < 0) ? ret : -EINVAL; + switch (state) { + case VFIO_DEVICE_STATE_ERROR: + return "ERROR"; + case VFIO_DEVICE_STATE_STOP: + return "STOP"; + case VFIO_DEVICE_STATE_RUNNING: + return "RUNNING"; + case VFIO_DEVICE_STATE_STOP_COPY: + return "STOP_COPY"; + case VFIO_DEVICE_STATE_RESUMING: + return "RESUMING"; + case VFIO_DEVICE_STATE_RUNNING_P2P: + return "RUNNING_P2P"; + default: + return "UNKNOWN STATE"; } - return 0; } -static int vfio_mig_rw(VFIODevice *vbasedev, __u8 *buf, size_t count, - off_t off, bool iswrite) -{ - int ret, done = 0; - __u8 *tbuf = buf; - - while (count) { - int bytes = 0; - - if (count >= 8 && !(off % 8)) { - bytes = 8; - } else if (count >= 4 && !(off % 4)) { - bytes = 4; - } else if (count >= 2 && !(off % 2)) { - bytes = 2; - } else { - bytes = 1; - } - - ret = vfio_mig_access(vbasedev, tbuf, bytes, off, iswrite); - if (ret) { - return ret; - } - - count -= bytes; - done += bytes; - off += bytes; - tbuf += bytes; - } - return done; -} - -#define vfio_mig_read(f, v, c, o) vfio_mig_rw(f, (__u8 *)v, c, o, false) -#define vfio_mig_write(f, v, c, o) vfio_mig_rw(f, (__u8 *)v, c, o, true) - -#define VFIO_MIG_STRUCT_OFFSET(f) \ - offsetof(struct vfio_device_migration_info, f) -/* - * Change the device_state register for device @vbasedev. Bits set in @mask - * are preserved, bits set in @value are set, and bits not set in either @mask - * or @value are cleared in device_state. If the register cannot be accessed, - * the resulting state would be invalid, or the device enters an error state, - * an error is returned. - */ - -static int vfio_migration_set_state(VFIODevice *vbasedev, uint32_t mask, - uint32_t value) +static int vfio_migration_set_state(VFIODevice *vbasedev, + enum vfio_device_mig_state new_state, + enum vfio_device_mig_state recover_state) { VFIOMigration *migration = vbasedev->migration; - VFIORegion *region = &migration->region; - off_t dev_state_off = region->fd_offset + - VFIO_MIG_STRUCT_OFFSET(device_state); - uint32_t device_state; + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_mig_state), + sizeof(uint64_t))] = {}; + struct vfio_device_feature *feature = (void *)buf; + struct vfio_device_feature_mig_state *mig_state = (void *)feature->data; int ret; - ret = vfio_mig_read(vbasedev, &device_state, sizeof(device_state), - dev_state_off); - if (ret < 0) { - return ret; - } - - device_state = (device_state & mask) | value; - - if (!VFIO_DEVICE_STATE_VALID(device_state)) { - return -EINVAL; - } - - ret = vfio_mig_write(vbasedev, &device_state, sizeof(device_state), - dev_state_off); - if (ret < 0) { - int rret; - - rret = vfio_mig_read(vbasedev, &device_state, sizeof(device_state), - dev_state_off); - - if ((rret < 0) || (VFIO_DEVICE_STATE_IS_ERROR(device_state))) { - hw_error("%s: Device in error state 0x%x", vbasedev->name, - device_state); - return rret ? rret : -EIO; + feature->argsz = sizeof(buf); + feature->flags = + VFIO_DEVICE_FEATURE_SET | VFIO_DEVICE_FEATURE_MIG_DEVICE_STATE; + mig_state->device_state = new_state; + ret = ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature); + if (ret) { + /* Try to put the device in some good state */ + mig_state->device_state = recover_state; + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + hw_error("%s: Device in error state, can't recover", + vbasedev->name); } - return ret; - } - - migration->device_state = device_state; - trace_vfio_migration_set_state(vbasedev->name, device_state); - return 0; -} -static void *get_data_section_size(VFIORegion *region, uint64_t data_offset, - uint64_t data_size, uint64_t *size) -{ - void *ptr = NULL; - uint64_t limit = 0; - int i; + error_report("%s: Failed changing device state to %s", vbasedev->name, + mig_state_to_str(new_state)); + migration->device_state = recover_state; - if (!region->mmaps) { - if (size) { - *size = MIN(data_size, region->size - data_offset); - } - return ptr; + return -1; } - for (i = 0; i < region->nr_mmaps; i++) { - VFIOMmap *map = region->mmaps + i; - - if ((data_offset >= map->offset) && - (data_offset < map->offset + map->size)) { - - /* check if data_offset is within sparse mmap areas */ - ptr = map->mmap + data_offset - map->offset; - if (size) { - *size = MIN(data_size, map->offset + map->size - data_offset); - } - break; - } else if ((data_offset < map->offset) && - (!limit || limit > map->offset)) { + if (mig_state->data_fd != -1) { + if (migration->data_fd != -1) { /* - * data_offset is not within sparse mmap areas, find size of - * non-mapped area. Check through all list since region->mmaps list - * is not sorted. + * This can happen if the device is asynchronously reset and + * terminates a data transfer. */ - limit = map->offset; - } - } - - if (!ptr && size) { - *size = limit ? MIN(data_size, limit - data_offset) : data_size; - } - return ptr; -} - -static int vfio_save_buffer(QEMUFile *f, VFIODevice *vbasedev, uint64_t *size) -{ - VFIOMigration *migration = vbasedev->migration; - VFIORegion *region = &migration->region; - uint64_t data_offset = 0, data_size = 0, sz; - int ret; - - ret = vfio_mig_read(vbasedev, &data_offset, sizeof(data_offset), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_offset)); - if (ret < 0) { - return ret; - } - - ret = vfio_mig_read(vbasedev, &data_size, sizeof(data_size), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_size)); - if (ret < 0) { - return ret; - } + error_report("%s: data_fd out of sync", vbasedev->name); + close(mig_state->data_fd); - trace_vfio_save_buffer(vbasedev->name, data_offset, data_size, - migration->pending_bytes); - - qemu_put_be64(f, data_size); - sz = data_size; - - while (sz) { - void *buf; - uint64_t sec_size; - bool buf_allocated = false; - - buf = get_data_section_size(region, data_offset, sz, &sec_size); - - if (!buf) { - buf = g_try_malloc(sec_size); - if (!buf) { - error_report("%s: Error allocating buffer ", __func__); - return -ENOMEM; - } - buf_allocated = true; - - ret = vfio_mig_read(vbasedev, buf, sec_size, - region->fd_offset + data_offset); - if (ret < 0) { - g_free(buf); - return ret; - } + return -1; } - qemu_put_buffer(f, buf, sec_size); - - if (buf_allocated) { - g_free(buf); - } - sz -= sec_size; - data_offset += sec_size; + migration->data_fd = mig_state->data_fd; } + migration->device_state = new_state; - ret = qemu_file_get_error(f); + trace_vfio_migration_set_state(vbasedev->name, new_state); - if (!ret && size) { - *size = data_size; - } - - bytes_transferred += data_size; - return ret; + return 0; } static int vfio_load_buffer(QEMUFile *f, VFIODevice *vbasedev, uint64_t data_size) -{ - VFIORegion *region = &vbasedev->migration->region; - uint64_t data_offset = 0, size, report_size; - int ret; - - do { - ret = vfio_mig_read(vbasedev, &data_offset, sizeof(data_offset), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_offset)); - if (ret < 0) { - return ret; - } - - if (data_offset + data_size > region->size) { - /* - * If data_size is greater than the data section of migration region - * then iterate the write buffer operation. This case can occur if - * size of migration region at destination is smaller than size of - * migration region at source. - */ - report_size = size = region->size - data_offset; - data_size -= size; - } else { - report_size = size = data_size; - data_size = 0; - } - - trace_vfio_load_state_device_data(vbasedev->name, data_offset, size); - - while (size) { - void *buf; - uint64_t sec_size; - bool buf_alloc = false; - - buf = get_data_section_size(region, data_offset, size, &sec_size); - - if (!buf) { - buf = g_try_malloc(sec_size); - if (!buf) { - error_report("%s: Error allocating buffer ", __func__); - return -ENOMEM; - } - buf_alloc = true; - } - - qemu_get_buffer(f, buf, sec_size); - - if (buf_alloc) { - ret = vfio_mig_write(vbasedev, buf, sec_size, - region->fd_offset + data_offset); - g_free(buf); - - if (ret < 0) { - return ret; - } - } - size -= sec_size; - data_offset += sec_size; - } - - ret = vfio_mig_write(vbasedev, &report_size, sizeof(report_size), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_size)); - if (ret < 0) { - return ret; - } - } while (data_size); - - return 0; -} - -static int vfio_update_pending(VFIODevice *vbasedev) { VFIOMigration *migration = vbasedev->migration; - VFIORegion *region = &migration->region; - uint64_t pending_bytes = 0; int ret; - ret = vfio_mig_read(vbasedev, &pending_bytes, sizeof(pending_bytes), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(pending_bytes)); - if (ret < 0) { - migration->pending_bytes = 0; + ret = qemu_file_get_to_fd(f, migration->data_fd, data_size); + if (ret) { return ret; } - migration->pending_bytes = pending_bytes; - trace_vfio_update_pending(vbasedev->name, pending_bytes); + trace_vfio_load_state_device_data(vbasedev->name, data_size); + return 0; } @@ -398,9 +185,8 @@ static void vfio_migration_cleanup(VFIODevice *vbasedev) { VFIOMigration *migration = vbasedev->migration; - if (migration->region.mmaps) { - vfio_region_unmap(&migration->region); - } + close(migration->data_fd); + migration->data_fd = -1; } /* ---------------------------------------------------------------------- */ @@ -408,44 +194,13 @@ static void vfio_migration_cleanup(VFIODevice *vbasedev) static int vfio_save_setup(QEMUFile *f, void *opaque) { VFIODevice *vbasedev = opaque; - VFIOMigration *migration = vbasedev->migration; - int ret; trace_vfio_save_setup(vbasedev->name); qemu_put_be64(f, VFIO_MIG_FLAG_DEV_SETUP_STATE); - - if (migration->region.mmaps) { - /* - * Calling vfio_region_mmap() from migration thread. Memory API called - * from this function require locking the iothread when called from - * outside the main loop thread. - */ - qemu_mutex_lock_iothread(); - ret = vfio_region_mmap(&migration->region); - qemu_mutex_unlock_iothread(); - if (ret) { - error_report("%s: Failed to mmap VFIO migration region: %s", - vbasedev->name, strerror(-ret)); - error_report("%s: Falling back to slow path", vbasedev->name); - } - } - - ret = vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_MASK, - VFIO_DEVICE_STATE_V1_SAVING); - if (ret) { - error_report("%s: Failed to set state SAVING", vbasedev->name); - return ret; - } - qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); - ret = qemu_file_get_error(f); - if (ret) { - return ret; - } - - return 0; + return qemu_file_get_error(f); } static void vfio_save_cleanup(void *opaque) @@ -456,127 +211,67 @@ static void vfio_save_cleanup(void *opaque) trace_vfio_save_cleanup(vbasedev->name); } -static void vfio_save_pending(QEMUFile *f, void *opaque, - uint64_t threshold_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) -{ - VFIODevice *vbasedev = opaque; - VFIOMigration *migration = vbasedev->migration; - int ret; - - ret = vfio_update_pending(vbasedev); - if (ret) { - return; - } - - *res_precopy_only += migration->pending_bytes; - - trace_vfio_save_pending(vbasedev->name, *res_precopy_only, - *res_postcopy_only, *res_compatible); -} - -static int vfio_save_iterate(QEMUFile *f, void *opaque) +/* Returns 1 if end-of-stream is reached, 0 if more data and -1 if error */ +static int vfio_save_block(QEMUFile *f, VFIOMigration *migration) { - VFIODevice *vbasedev = opaque; - VFIOMigration *migration = vbasedev->migration; - uint64_t data_size; - int ret; - - qemu_put_be64(f, VFIO_MIG_FLAG_DEV_DATA_STATE); - - if (migration->pending_bytes == 0) { - ret = vfio_update_pending(vbasedev); - if (ret) { - return ret; - } + ssize_t data_size; - if (migration->pending_bytes == 0) { - qemu_put_be64(f, 0); - qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); - /* indicates data finished, goto complete phase */ - return 1; - } + data_size = read(migration->data_fd, migration->data_buffer, + migration->data_buffer_size); + if (data_size < 0) { + return -1; } - - ret = vfio_save_buffer(f, vbasedev, &data_size); - if (ret) { - error_report("%s: vfio_save_buffer failed %s", vbasedev->name, - strerror(errno)); - return ret; + if (data_size == 0) { + return 1; } - qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); + qemu_put_be64(f, VFIO_MIG_FLAG_DEV_DATA_STATE); + qemu_put_be64(f, data_size); + qemu_put_buffer_async(f, migration->data_buffer, data_size, false); + qemu_fflush(f); + bytes_transferred += data_size; - ret = qemu_file_get_error(f); - if (ret) { - return ret; - } + trace_vfio_save_block(migration->vbasedev->name, data_size); - /* - * Reset pending_bytes as .save_live_pending is not called during savevm or - * snapshot case, in such case vfio_update_pending() at the start of this - * function updates pending_bytes. - */ - migration->pending_bytes = 0; - trace_vfio_save_iterate(vbasedev->name, data_size); - return 0; + return qemu_file_get_error(f); } static int vfio_save_complete_precopy(QEMUFile *f, void *opaque) { VFIODevice *vbasedev = opaque; - VFIOMigration *migration = vbasedev->migration; - uint64_t data_size; + enum vfio_device_mig_state recover_state; int ret; - ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_RUNNING, - VFIO_DEVICE_STATE_V1_SAVING); + /* We reach here with device state STOP or STOP_COPY only */ + recover_state = VFIO_DEVICE_STATE_STOP; + ret = vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_STOP_COPY, + recover_state); if (ret) { - error_report("%s: Failed to set state STOP and SAVING", - vbasedev->name); return ret; } - ret = vfio_update_pending(vbasedev); - if (ret) { - return ret; - } - - while (migration->pending_bytes > 0) { - qemu_put_be64(f, VFIO_MIG_FLAG_DEV_DATA_STATE); - ret = vfio_save_buffer(f, vbasedev, &data_size); + do { + ret = vfio_save_block(f, vbasedev->migration); if (ret < 0) { - error_report("%s: Failed to save buffer", vbasedev->name); - return ret; - } - - if (data_size == 0) { - break; - } - - ret = vfio_update_pending(vbasedev); - if (ret) { return ret; } - } + } while (!ret); qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); - ret = qemu_file_get_error(f); if (ret) { return ret; } - ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_SAVING, 0); + ret = vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_STOP, + recover_state); if (ret) { - error_report("%s: Failed to set state STOPPED", vbasedev->name); return ret; } trace_vfio_save_complete_precopy(vbasedev->name); - return ret; + + return 0; } static void vfio_save_state(QEMUFile *f, void *opaque) @@ -595,28 +290,9 @@ static void vfio_save_state(QEMUFile *f, void *opaque) static int vfio_load_setup(QEMUFile *f, void *opaque) { VFIODevice *vbasedev = opaque; - VFIOMigration *migration = vbasedev->migration; - int ret = 0; - if (migration->region.mmaps) { - ret = vfio_region_mmap(&migration->region); - if (ret) { - error_report("%s: Failed to mmap VFIO migration region %d: %s", - vbasedev->name, migration->region.nr, - strerror(-ret)); - error_report("%s: Falling back to slow path", vbasedev->name); - } - } - - ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_MASK, - VFIO_DEVICE_STATE_V1_RESUMING); - if (ret) { - error_report("%s: Failed to set state RESUMING", vbasedev->name); - if (migration->region.mmaps) { - vfio_region_unmap(&migration->region); - } - } - return ret; + return vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_RESUMING, + vbasedev->migration->device_state); } static int vfio_load_cleanup(void *opaque) @@ -685,8 +361,6 @@ static int vfio_load_state(QEMUFile *f, void *opaque, int version_id) static SaveVMHandlers savevm_vfio_handlers = { .save_setup = vfio_save_setup, .save_cleanup = vfio_save_cleanup, - .save_live_pending = vfio_save_pending, - .save_live_iterate = vfio_save_iterate, .save_live_complete_precopy = vfio_save_complete_precopy, .save_state = vfio_save_state, .load_setup = vfio_load_setup, @@ -699,58 +373,28 @@ static SaveVMHandlers savevm_vfio_handlers = { static void vfio_vmstate_change(void *opaque, bool running, RunState state) { VFIODevice *vbasedev = opaque; - VFIOMigration *migration = vbasedev->migration; - uint32_t value, mask; + enum vfio_device_mig_state new_state; int ret; - if (vbasedev->migration->vm_running == running) { - return; - } - if (running) { - /* - * Here device state can have one of _SAVING, _RESUMING or _STOP bit. - * Transition from _SAVING to _RUNNING can happen if there is migration - * failure, in that case clear _SAVING bit. - * Transition from _RESUMING to _RUNNING occurs during resuming - * phase, in that case clear _RESUMING bit. - * In both the above cases, set _RUNNING bit. - */ - mask = ~VFIO_DEVICE_STATE_MASK; - value = VFIO_DEVICE_STATE_V1_RUNNING; + new_state = VFIO_DEVICE_STATE_RUNNING; } else { - /* - * Here device state could be either _RUNNING or _SAVING|_RUNNING. Reset - * _RUNNING bit - */ - mask = ~VFIO_DEVICE_STATE_V1_RUNNING; - - /* - * When VM state transition to stop for savevm command, device should - * start saving data. - */ - if (state == RUN_STATE_SAVE_VM) { - value = VFIO_DEVICE_STATE_V1_SAVING; - } else { - value = 0; - } + new_state = VFIO_DEVICE_STATE_STOP; } - ret = vfio_migration_set_state(vbasedev, mask, value); + ret = + vfio_migration_set_state(vbasedev, new_state, VFIO_DEVICE_STATE_ERROR); if (ret) { /* * Migration should be aborted in this case, but vm_state_notify() * currently does not support reporting failures. */ - error_report("%s: Failed to set device state 0x%x", vbasedev->name, - (migration->device_state & mask) | value); if (migrate_get_current()->to_dst_file) { qemu_file_set_error(migrate_get_current()->to_dst_file, ret); } } - vbasedev->migration->vm_running = running; trace_vfio_vmstate_change(vbasedev->name, running, RunState_str(state), - (migration->device_state & mask) | value); + new_state); } static void vfio_migration_state_notifier(Notifier *notifier, void *data) @@ -759,7 +403,6 @@ static void vfio_migration_state_notifier(Notifier *notifier, void *data) VFIOMigration *migration = container_of(notifier, VFIOMigration, migration_state); VFIODevice *vbasedev = migration->vbasedev; - int ret; trace_vfio_migration_state_notifier(vbasedev->name, MigrationStatus_str(s->state)); @@ -769,34 +412,45 @@ static void vfio_migration_state_notifier(Notifier *notifier, void *data) case MIGRATION_STATUS_CANCELLED: case MIGRATION_STATUS_FAILED: bytes_transferred = 0; - ret = vfio_migration_set_state( - vbasedev, - ~(VFIO_DEVICE_STATE_V1_SAVING | VFIO_DEVICE_STATE_V1_RESUMING), - VFIO_DEVICE_STATE_V1_RUNNING); - if (ret) { - error_report("%s: Failed to set state RUNNING", vbasedev->name); - } + vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_RUNNING, + VFIO_DEVICE_STATE_ERROR); } } static void vfio_migration_exit(VFIODevice *vbasedev) { - VFIOMigration *migration = vbasedev->migration; - - vfio_region_exit(&migration->region); - vfio_region_finalize(&migration->region); + g_free(vbasedev->migration->data_buffer); g_free(vbasedev->migration); vbasedev->migration = NULL; } -static int vfio_migration_init(VFIODevice *vbasedev, - struct vfio_region_info *info) +static int vfio_migration_query_flags(VFIODevice *vbasedev, uint64_t *mig_flags) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_migration), + sizeof(uint64_t))] = {}; + struct vfio_device_feature *feature = (void *)buf; + struct vfio_device_feature_migration *mig = (void *)feature->data; + + feature->argsz = sizeof(buf); + feature->flags = VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_MIGRATION; + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + return -EOPNOTSUPP; + } + + *mig_flags = mig->flags; + + return 0; +} + +static int vfio_migration_init(VFIODevice *vbasedev) { - int ret; Object *obj; VFIOMigration *migration; char id[256] = ""; g_autofree char *path = NULL, *oid = NULL; + uint64_t mig_flags; + int ret; if (!vbasedev->ops->vfio_get_object) { return -EINVAL; @@ -807,25 +461,23 @@ static int vfio_migration_init(VFIODevice *vbasedev, return -EINVAL; } - vbasedev->migration = g_new0(VFIOMigration, 1); - - ret = vfio_region_setup(obj, vbasedev, &vbasedev->migration->region, - info->index, "migration"); + ret = vfio_migration_query_flags(vbasedev, &mig_flags); if (ret) { - error_report("%s: Failed to setup VFIO migration region %d: %s", - vbasedev->name, info->index, strerror(-ret)); - goto err; + return ret; } - if (!vbasedev->migration->region.size) { - error_report("%s: Invalid zero-sized VFIO migration region %d", - vbasedev->name, info->index); - ret = -EINVAL; - goto err; + /* Basic migration functionality must be supported */ + if (!(mig_flags & VFIO_MIGRATION_STOP_COPY)) { + return -EOPNOTSUPP; } + vbasedev->migration = g_new0(VFIOMigration, 1); + vbasedev->migration->data_buffer_size = VFIO_MIG_DATA_BUFFER_SIZE; + vbasedev->migration->data_buffer = + g_malloc0(vbasedev->migration->data_buffer_size); migration = vbasedev->migration; migration->vbasedev = vbasedev; + migration->data_fd = -1; oid = vmstate_if_get_id(VMSTATE_IF(DEVICE(obj))); if (oid) { @@ -837,17 +489,13 @@ static int vfio_migration_init(VFIODevice *vbasedev, register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, &savevm_vfio_handlers, vbasedev); - migration->vm_state = qdev_add_vm_change_state_handler(vbasedev->dev, vfio_vmstate_change, vbasedev); migration->migration_state.notify = vfio_migration_state_notifier; add_migration_state_change_notifier(&migration->migration_state); - return 0; -err: - vfio_migration_exit(vbasedev); - return ret; + return 0; } /* ---------------------------------------------------------------------- */ @@ -860,7 +508,6 @@ int64_t vfio_mig_bytes_transferred(void) int vfio_migration_probe(VFIODevice *vbasedev, Error **errp) { VFIOContainer *container = vbasedev->group->container; - struct vfio_region_info *info = NULL; int ret = -ENOTSUP; if (!vbasedev->enable_migration) { @@ -874,27 +521,18 @@ int vfio_migration_probe(VFIODevice *vbasedev, Error **errp) migrate_get_current()->skip_precopy = true; } - ret = vfio_get_dev_region_info(vbasedev, - VFIO_REGION_TYPE_MIGRATION_DEPRECATED, - VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED, - &info); + ret = vfio_migration_init(vbasedev); if (ret) { goto add_blocker; } - ret = vfio_migration_init(vbasedev, info); - if (ret) { - goto add_blocker; - } + trace_vfio_migration_probe(vbasedev->name); - trace_vfio_migration_probe(vbasedev->name, info->index); - g_free(info); return 0; add_blocker: error_setg(&vbasedev->migration_blocker, "VFIO device doesn't support migration"); - g_free(info); ret = migrate_add_blocker(vbasedev->migration_blocker, errp); if (ret < 0) { diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 582882db91..a24ea7d8b0 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -148,21 +148,18 @@ vfio_display_edid_update(uint32_t prefx, uint32_t prefy) "%ux%u" vfio_display_edid_write_error(void) "" # migration.c -vfio_migration_probe(const char *name, uint32_t index) " (%s) Region %d" +vfio_migration_probe(const char *name) " (%s)" vfio_migration_set_state(const char *name, uint32_t state) " (%s) state %d" vfio_vmstate_change(const char *name, int running, const char *reason, uint32_t dev_state) " (%s) running %d reason %s device state %d" vfio_migration_state_notifier(const char *name, const char *state) " (%s) state %s" vfio_save_setup(const char *name) " (%s)" vfio_save_cleanup(const char *name) " (%s)" -vfio_save_buffer(const char *name, uint64_t data_offset, uint64_t data_size, uint64_t pending) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64" pending 0x%"PRIx64 -vfio_update_pending(const char *name, uint64_t pending) " (%s) pending 0x%"PRIx64 vfio_save_device_config_state(const char *name) " (%s)" -vfio_save_pending(const char *name, uint64_t precopy, uint64_t postcopy, uint64_t compatible) " (%s) precopy 0x%"PRIx64" postcopy 0x%"PRIx64" compatible 0x%"PRIx64 -vfio_save_iterate(const char *name, int data_size) " (%s) data_size %d" vfio_save_complete_precopy(const char *name) " (%s)" vfio_load_device_config_state(const char *name) " (%s)" vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 -vfio_load_state_device_data(const char *name, uint64_t data_offset, uint64_t data_size) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64 +vfio_load_state_device_data(const char *name, uint64_t data_size) " (%s) size 0x%"PRIx64 vfio_load_cleanup(const char *name) " (%s)" vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitmap_size, uint64_t start) "container fd=%d, iova=0x%"PRIx64" size= 0x%"PRIx64" bitmap_size=0x%"PRIx64" start=0x%"PRIx64 vfio_iommu_map_dirty_notify(uint64_t iova_start, uint64_t iova_end) "iommu dirty @ 0x%"PRIx64" - 0x%"PRIx64 +vfio_save_block(const char *name, int data_size) " (%s) data_size %d" diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index e573f5a9f1..09446a9082 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -61,11 +61,11 @@ typedef struct VFIORegion { typedef struct VFIOMigration { struct VFIODevice *vbasedev; VMChangeStateEntry *vm_state; - VFIORegion region; - uint32_t device_state; - int vm_running; + enum vfio_device_mig_state device_state; + int data_fd; Notifier migration_state; - uint64_t pending_bytes; + void *data_buffer; + size_t data_buffer_size; } VFIOMigration; typedef struct VFIOAddressSpace { From patchwork Thu May 12 15:43:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 12847855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1A9ACC433EF for ; Thu, 12 May 2022 15:56:43 +0000 (UTC) Received: from localhost ([::1]:42936 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1npBBC-0001Ln-5t for qemu-devel@archiver.kernel.org; Thu, 12 May 2022 11:56:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42034) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyw-0006UE-T2 for qemu-devel@nongnu.org; Thu, 12 May 2022 11:44:03 -0400 Received: from mail-bn1nam07on2060d.outbound.protection.outlook.com ([2a01:111:f400:7eb2::60d]:11353 helo=NAM02-BN1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyq-0005La-9U for qemu-devel@nongnu.org; Thu, 12 May 2022 11:43:59 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jY9Hd0LmjFKmgTxW58MFEy5gQDIU5M+Gx2S/vzlOKNLLANJtvYS1/A7JmCWP0SzUrp8fRbIdnq13YS+wlJ37HzTEa1UO0cOb2tYLsxhTaK8f1LqVvCO97C6FqtLtDILYXXfV4Hm9AOeqdEBOA0uccK4DKo+2gm9n+eUqmDSOfPza5djw5PLOyaTFk7ckUPAVvADD6GaUwME66guZ7MenQIOkyGfghJIIL+y9LRiEsM48lTJsi1Poq1hdrQp7GRWIgRBgUD/jpIDJ3PKvhwhXp6XOqOIz/bsx1J9VYsh+GXDb+VwlzVRTDf0jzCodTSAEydm3NDOBHIk2m3SRYjMOMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Cyw/PPr2SoPoAMYV02CPQlf9qOuzmF91duF0c1oGOAc=; b=g9Ojm6eNidYoQOEHD1xRmhfQNfGEXN9ohwXoDzVxtUgXr5tiwXdld32fuwSzQ8tNqRCkrXy7wCvHKhEs0Uj0y1i+2SQ2UHAWdZ+kpl5/efIUvAsSZj44M+7Y1LC3vgMJKTtUE1xqS6ZBO7AlBFwZ5nTM5piU5KvPlAdWzIPra4M5WhSCnUsDQAr7Fymsw6/U3iNcfeHmDBAmBwTL15ZjTHOhgUZIo0HMCwLwHK4bKzZxNFzDe/C6yM3t3e2sP1iw9WcREmipFBmXhExDhrOivE6olNxdd/eIcQXs5p7Calm8xHhRnd635Gg6sLyzLJHttQv2ubFqyUBhULGcatiujg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cyw/PPr2SoPoAMYV02CPQlf9qOuzmF91duF0c1oGOAc=; b=f5PaNd+QFd0hpCNksjNmECBD7KUCFvtlYo0C7/Nx4H0s6AXNP3xMdgMx2HYssyIswFvsOzUk8BhUVLEKG/NpooD6RvyOctyWfWvwZgp6z3JYnEZ5+cy7060yQlG7uJ8cD2CohLanL5HmnO8TGupf1EGl7yqf/55M2fMG08/51YkuN7aBbnKxKxTpN6W+zJP+aiALbp6aatzFFdqRxXOtL29/sxUhYMLtg2aFbpxfcNwk88eiHHQVKtVTVZ8eGwzp/QkyjSvN05ARutfR7BiVrTXWi33efUAxpU2GD+osoNEmdrtfWEVCDTywOsj63s57jmF/lXcYsWwSg6JCDshrZA== Received: from MW4PR04CA0265.namprd04.prod.outlook.com (2603:10b6:303:88::30) by PH7PR12MB5736.namprd12.prod.outlook.com (2603:10b6:510:1e3::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.23; Thu, 12 May 2022 15:43:53 +0000 Received: from CO1NAM11FT036.eop-nam11.prod.protection.outlook.com (2603:10b6:303:88:cafe::77) by MW4PR04CA0265.outlook.office365.com (2603:10b6:303:88::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.22 via Frontend Transport; Thu, 12 May 2022 15:43:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.238) by CO1NAM11FT036.mail.protection.outlook.com (10.13.174.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5250.13 via Frontend Transport; Thu, 12 May 2022 15:43:53 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Thu, 12 May 2022 15:43:53 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Thu, 12 May 2022 08:43:52 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Thu, 12 May 2022 08:43:49 -0700 From: Avihai Horon To: , "Michael S . Tsirkin" , "Cornelia Huck" , Paolo Bonzini , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Yishai Hadas , Jason Gunthorpe , "Mark Bloch" , Maor Gottlieb , Kirti Wankhede , Tarun Gupta Subject: [PATCH 7/9] vfio/migration: Reset device if setting recover state fails Date: Thu, 12 May 2022 18:43:18 +0300 Message-ID: <20220512154320.19697-8-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220512154320.19697-1-avihaih@nvidia.com> References: <20220512154320.19697-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 65832d28-31a7-46dd-3d58-08da342e37e5 X-MS-TrafficTypeDiagnostic: PH7PR12MB5736:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6Hf2jz8/9qDjgDBBPj3QDM/Wj33VrW4v0cx8jDF+h0v8pq4pfj1K1V5DSyKWl838V32/dLgDe6bQfyc7Ck26qmvxgZ4q0R4SVEbIL/O+sA3vaplN+7f3QR2sdK8vqYc9kuOQkqpMtWKfFsZ8ZnnSusWhdbH7e8OtRhsulUHkuVDEtJ5ozg+iagBI63C5NwZQogdJtAU1hl6UWZeJ5k/UTVmnATQn+X9nMGP5SlMZj6cX0iCKdZJrCfaysY2XFceIyBk+DTDdWfvXreic28KhyRu3Zzs+D1UJmEmOzsJvJZEjtrgyrqFTo5MKVzMkEz7LhVpiyh2hqE+sMy92v+NmzvHi/19T8eQWwEzKyErsmboT6q0jYKSddFucxBlTifjJ7N/3ILJUpu3JsOb/7wVn2X2X/JBoW460kboel/8mW+JAmR6UAS/ZNzB0Xg6bxZJisVZyFI2/iJFCuQNZaolNVtT7CiCxSZthEvvznVWpIkSA5R3y3YlOkg82umrg+nc+OVCypfO7KEOPmu6if94zClgkgSvQ3e57q9Q3Y4VIqcb0ZWYuk0ZyiGSUPUGj3VzndCNbPSzLv1lujk9Bb8CNA4VFH0wFiUOyQMFknkiT80+X7B36v2kG5OxUDlDhqoLkWVNwcsCMrD6o8N6hJoZ7X3fgyvgrM7UNMvit2dyfJvkiGZ0rkh5QY4fcyNZXV3BUD46T2kuW4EdP5x6MFscuyw== X-Forefront-Antispam-Report: CIP:12.22.5.238; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(40460700003)(6666004)(7696005)(2616005)(356005)(36860700001)(81166007)(2906002)(26005)(186003)(8936002)(8676002)(82310400005)(4326008)(70206006)(86362001)(70586007)(1076003)(336012)(47076005)(110136005)(107886003)(54906003)(36756003)(426003)(83380400001)(5660300002)(508600001)(316002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2022 15:43:53.6163 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 65832d28-31a7-46dd-3d58-08da342e37e5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.238]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT036.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5736 Received-SPF: softfail client-ip=2a01:111:f400:7eb2::60d; envelope-from=avihaih@nvidia.com; helo=NAM02-BN1-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" If vfio_migration_set_state() fails to set the device in the requested state it tries to put it in a recover state. If setting the device in the recover state fails as well, hw_error is triggered and the VM is aborted. To improve user experience and avoid VM data loss, reset the device with VFIO_RESET_DEVICE instead of aborting the VM. Signed-off-by: Avihai Horon --- hw/vfio/migration.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 8943ccbace..aec8afcd6c 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -89,8 +89,16 @@ static int vfio_migration_set_state(VFIODevice *vbasedev, /* Try to put the device in some good state */ mig_state->device_state = recover_state; if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { - hw_error("%s: Device in error state, can't recover", - vbasedev->name); + if (ioctl(vbasedev->fd, VFIO_DEVICE_RESET)) { + hw_error("%s: Device in error state, can't recover", + vbasedev->name); + } + + error_report( + "%s: Device was reset due to failure in changing device state to recover state %s", + vbasedev->name, mig_state_to_str(recover_state)); + + return -1; } error_report("%s: Failed changing device state to %s", vbasedev->name, From patchwork Thu May 12 15:43:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 12847857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96844C433F5 for ; Thu, 12 May 2022 15:59:37 +0000 (UTC) Received: from localhost ([::1]:50660 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1npBE0-0006bw-MR for qemu-devel@archiver.kernel.org; Thu, 12 May 2022 11:59:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42052) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyz-0006VA-42 for qemu-devel@nongnu.org; Thu, 12 May 2022 11:44:05 -0400 Received: from mail-dm6nam11on2061a.outbound.protection.outlook.com ([2a01:111:f400:7eaa::61a]:44641 helo=NAM11-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyw-0005Lr-Pi for qemu-devel@nongnu.org; Thu, 12 May 2022 11:44:04 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ENeNRvNUER9UTy4DHR2UtsLe2GQj5xx22e16lKNuBJo4gpaKKl2gLG5wMyndtjLhG4+pQtcMRbnjJzAjpcmMUx3fZBUT/sjlHmek/ul9Atfs0Vj8bYGF/Y9G4HQGCxtId39bMxbvsPJyHKPmBOzm6rQ8aV/touLTsXbdjNuIYEdnrjY3hj+TqGjCzgZw4sdV7ueD83ocGTG7ahfJh/+1YxJ/GUMrzcN8+QdTPkasKNfacKJwldAS7QPHmzr3cFjIv+08/I9sQcqNVFONrH/Q5qpW+Lrq6EyZvgP8B5aXt50xH8esOJ+UKJLB6GT06i2ePTbyT10iXxumNH0ozOEUzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=liYUGVmuX0kZmO+aQDDBrQ7rfJPkkF+ldMbOwennimQ=; b=DMcqX6UlvvG3MKnZPcwWHJKZ4RBI/H+FxtbN28MrknyTFNyB0RHTMmQ+9S8QgsKnihLi2x++HCYFxc4IKXb9HExPtNEJ1guk1/5YMm+HK+got4Ro5wIlqy411/5JrEDqN9Fx8kAse4c4ozkqL6ty+2y3hrcMGciFlKckiAV8Vhd1ve1sa+zEPYFwU7urHKDnT2TZ6DF8i80Upt3MIGt4nPVfKvY7tdS26wL7jG+Zxl001zekQNH7jYgCs4LjbFVxiFn73tbKTMvVbDClUX8ftKtfyRLvdKy6XqBdZYyJnYEOhW0TBKFnubKxkcwmNYGjg3T98qWUPwohxYCidebDUA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=liYUGVmuX0kZmO+aQDDBrQ7rfJPkkF+ldMbOwennimQ=; b=gi/4JxMXA4HHpdp6RwuocESgFIqgqx7X97A5PBDty/Oe5YsJkh6EsD/qGdtmaYhgWHZdYaS+i6PQ5CQgfdTVjVELNVvPnKeDoHbnQlRXlNsSpS0F7Q2H2c3CcgAHsQNmnH84BH6wKQXq2QtlpCvR8dWrACCg9Xj7tO6uSQ3+83ZTVWon9qiMVHoP3NV4vy2ACCjKimWx/Ain/AXH3MAI7m6oAA4kEqQCw+EDwWsMXGIrEF0SqdPAXoauKQ0OGGoFN0p22X913rqdAgaAu3e3hjsVZu2RgOfIYsfwVqOsh0z1xybx/zvrQtJwQYK1twx14HkIX7grK5qQrQZxgjbf4w== Received: from MW4PR04CA0081.namprd04.prod.outlook.com (2603:10b6:303:6b::26) by CY4PR1201MB0216.namprd12.prod.outlook.com (2603:10b6:910:18::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.23; Thu, 12 May 2022 15:43:57 +0000 Received: from CO1NAM11FT031.eop-nam11.prod.protection.outlook.com (2603:10b6:303:6b:cafe::77) by MW4PR04CA0081.outlook.office365.com (2603:10b6:303:6b::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.23 via Frontend Transport; Thu, 12 May 2022 15:43:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by CO1NAM11FT031.mail.protection.outlook.com (10.13.174.118) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5250.13 via Frontend Transport; Thu, 12 May 2022 15:43:57 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Thu, 12 May 2022 15:43:57 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Thu, 12 May 2022 08:43:56 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Thu, 12 May 2022 08:43:52 -0700 From: Avihai Horon To: , "Michael S . Tsirkin" , "Cornelia Huck" , Paolo Bonzini , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Yishai Hadas , Jason Gunthorpe , "Mark Bloch" , Maor Gottlieb , Kirti Wankhede , Tarun Gupta Subject: [PATCH 8/9] vfio: Alphabetize migration section of VFIO trace-events file Date: Thu, 12 May 2022 18:43:19 +0300 Message-ID: <20220512154320.19697-9-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220512154320.19697-1-avihaih@nvidia.com> References: <20220512154320.19697-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8b896f2a-8b6f-404a-c073-08da342e3a37 X-MS-TrafficTypeDiagnostic: CY4PR1201MB0216:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: RJ+dYY5o//q4IzE5USvNhJAFVQqo6zcaFBAYqnWCg3WP+sf+9ryqXeLBmfDeI3TYA7SBNdtOUprqBjWo6NKg/MGMohOiQq9YoImHK6s7Uxvv+Vs1NusJHokv+NaDu1SU1WrokEFKSVbT8/c/vW/CrIlfhLyasMS3nw4MULkRrEkBmRkPudCPu8QSWR6HmMiuzmSgl6umfxDioX7IU7bcmpxTO+KFJauaIseq+FgNB9hcoFtA0D6kXkM7gOUoj/tc+D4hfyAoZy2Z3pSTbgSX59uclZrN6mBSSfBSO4q8eLB7rtUi44rKCxBWvUl0+eIcO5rsDn90iSSo0HxAgK7G15Dxc0t46CJqEh5zdhikjnCy4I5brKttUrAGv71/W46ITXdEFWTDfZieGFjGB8wLGCMvcHERC9Ylpn0Cp4zTFa/+FqXXrD2mvLNa4Q66jlh8AQ4ztqgkliQaMzH3wCPyPzljO0J9jFHC6KrgXtl1Pa34VQXTt9lp3RwLEbWCkCLprqBuzirXCghA5O9Z8KEk3Z2f31SrMdp/apvFlkyRJQanwfCkH9dl0oWKNEidtJZgOVvWAj8KFryBuJIzmwdi9f+9/pOd/H7iyQo68WnKzqbm8Z1j8cKlRDwS/YUsdXuFOgHv3T6Bo/Cnjix6IenWRa9eLi2lS75bJQK7p5AxT7Bir25VM0qZKsxPCxspiiA9+d45RxlXF3FKdk0sFFRWNQ== X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(8936002)(86362001)(186003)(70586007)(70206006)(2616005)(40460700003)(36860700001)(4326008)(7696005)(8676002)(2906002)(316002)(82310400005)(26005)(54906003)(110136005)(6666004)(83380400001)(36756003)(356005)(1076003)(81166007)(47076005)(426003)(336012)(107886003)(5660300002)(508600001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2022 15:43:57.5125 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8b896f2a-8b6f-404a-c073-08da342e3a37 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT031.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1201MB0216 Received-SPF: softfail client-ip=2a01:111:f400:7eaa::61a; envelope-from=avihaih@nvidia.com; helo=NAM11-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Sort the migration section of VFIO trace events file alphabetically and move two misplaced traces to common.c section. Signed-off-by: Avihai Horon --- hw/vfio/trace-events | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index a24ea7d8b0..d3cba59bfd 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -119,6 +119,8 @@ vfio_region_sparse_mmap_header(const char *name, int index, int nr_areas) "Devic vfio_region_sparse_mmap_entry(int i, unsigned long start, unsigned long end) "sparse entry %d [0x%lx - 0x%lx]" vfio_get_dev_region(const char *name, int index, uint32_t type, uint32_t subtype) "%s index %d, %08x/%0x8" vfio_dma_unmap_overflow_workaround(void) "" +vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitmap_size, uint64_t start) "container fd=%d, iova=0x%"PRIx64" size= 0x%"PRIx64" bitmap_size=0x%"PRIx64" start=0x%"PRIx64 +vfio_iommu_map_dirty_notify(uint64_t iova_start, uint64_t iova_end) "iommu dirty @ 0x%"PRIx64" - 0x%"PRIx64 # platform.c vfio_platform_base_device_init(char *name, int groupid) "%s belongs to group #%d" @@ -148,18 +150,16 @@ vfio_display_edid_update(uint32_t prefx, uint32_t prefy) "%ux%u" vfio_display_edid_write_error(void) "" # migration.c +vfio_load_cleanup(const char *name) " (%s)" +vfio_load_device_config_state(const char *name) " (%s)" +vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 +vfio_load_state_device_data(const char *name, uint64_t data_size) " (%s) size 0x%"PRIx64 vfio_migration_probe(const char *name) " (%s)" vfio_migration_set_state(const char *name, uint32_t state) " (%s) state %d" -vfio_vmstate_change(const char *name, int running, const char *reason, uint32_t dev_state) " (%s) running %d reason %s device state %d" vfio_migration_state_notifier(const char *name, const char *state) " (%s) state %s" -vfio_save_setup(const char *name) " (%s)" +vfio_save_block(const char *name, int data_size) " (%s) data_size %d" vfio_save_cleanup(const char *name) " (%s)" -vfio_save_device_config_state(const char *name) " (%s)" vfio_save_complete_precopy(const char *name) " (%s)" -vfio_load_device_config_state(const char *name) " (%s)" -vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 -vfio_load_state_device_data(const char *name, uint64_t data_size) " (%s) size 0x%"PRIx64 -vfio_load_cleanup(const char *name) " (%s)" -vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitmap_size, uint64_t start) "container fd=%d, iova=0x%"PRIx64" size= 0x%"PRIx64" bitmap_size=0x%"PRIx64" start=0x%"PRIx64 -vfio_iommu_map_dirty_notify(uint64_t iova_start, uint64_t iova_end) "iommu dirty @ 0x%"PRIx64" - 0x%"PRIx64 -vfio_save_block(const char *name, int data_size) " (%s) data_size %d" +vfio_save_device_config_state(const char *name) " (%s)" +vfio_save_setup(const char *name) " (%s)" +vfio_vmstate_change(const char *name, int running, const char *reason, uint32_t dev_state) " (%s) running %d reason %s device state %d" From patchwork Thu May 12 15:43:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 12847849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8CBB0C433FE for ; Thu, 12 May 2022 15:50:02 +0000 (UTC) Received: from localhost ([::1]:53178 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1npB4j-00065n-Fa for qemu-devel@archiver.kernel.org; Thu, 12 May 2022 11:50:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42082) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAz2-0006Xx-Py for qemu-devel@nongnu.org; Thu, 12 May 2022 11:44:10 -0400 Received: from mail-dm6nam08on20611.outbound.protection.outlook.com ([2a01:111:f400:7e8b::611]:20288 helo=NAM04-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npAyz-0005M1-Dr for qemu-devel@nongnu.org; Thu, 12 May 2022 11:44:07 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SuINeZDsqizolUGGnboZt4m6I6SjNhVvjw2jMymB/3LE3NMSeMlIJ0+cVkRGtMIEN7US54zE8rOeUfNN0WT+GdOOTT+PG1RFBm0/qwQCr9dh/zazyTSUUanePE1oMtaFyWmdoo6spZI8ScdRzhDaAwZasNZgbjtjGtUsLq+wSMO1shesCUytUhlcuM1dpqn0zH+kmhHwmnIy9dmJTqMChLPo9ZO1W9AmuIM7ku3aNsxIBupmXvm1XUa7uriyTQ6S9NcgAVfFJD2a998qMCgCypKqEsqrkfxPZJndV/jIDcMnGZW+YXqjNKXUCd7MDySjKQ0QnFgCISAFCfKY4yw/TA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PSPaPjjDN9b8FTrPUpEHjY11iJitw0hUj9yIciMkLkE=; b=ToRq9+5e5Ya07B4ee0JSUMGoiY09MsGW+YhUwOzEsvh9iGKcuR/UGfKMyANeFOBjURLrf+aLtmz0BmeG942aQYvEkrXlifqIGaExYJ6aBrT4XO0ZJ4MyHWxOPVGgiztxfK7ZgqKEh4HMEFq9oa+HmvleeIlKC4M55iqy4DghI9QAk1tmF8craNp4xYByK9Rl/CH/wPjUpwvBn6LhStH1x8lrRXM881t4DxfXqnrBIyBwY1LLp5e3ai7tYyyu1YTzir0jqBBJ1wb+wIAvCuKdGOaZ7JbragJNixCA7DbbLKxI8AtL8YWstM69UCvbjg3aCyp3BTSTDxV+BfPKNu0d/w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PSPaPjjDN9b8FTrPUpEHjY11iJitw0hUj9yIciMkLkE=; b=Hcuw2iKMs5WeBgbuz7fSH9rq1WxZ+cfj/G9I9H53Ch/RpfOP/yoR77nsCmk5Loee7UvSdOTH+JEzaIUGq887Sx+bvTRXzpqRit0g9gGR/34TwPyn3N6vdnI8QjRAIYi80/kA3hgrLBhaMXx1yPg4ltZ9LmCMJGwNmE4WvUY43vmVPxx/bGTaZrXXCsThcrw3ISANwm8tSDZhQYfcKtRRkhUlaprUDU3rkbj/vL5gji0yecCGBQi0NbLl4mQIRHffrIWYEqh+NaX2NZOwWfcjJQswZmhBK1UvYvUt6JO2xavdpWHKgegA5Wqfk7GVknlk3igSrYvwb4+ik9VoWiKQeg== Received: from MW3PR06CA0029.namprd06.prod.outlook.com (2603:10b6:303:2a::34) by MWHPR12MB1248.namprd12.prod.outlook.com (2603:10b6:300:12::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.13; Thu, 12 May 2022 15:44:01 +0000 Received: from CO1NAM11FT010.eop-nam11.prod.protection.outlook.com (2603:10b6:303:2a:cafe::e0) by MW3PR06CA0029.outlook.office365.com (2603:10b6:303:2a::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.24 via Frontend Transport; Thu, 12 May 2022 15:44:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by CO1NAM11FT010.mail.protection.outlook.com (10.13.175.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5250.13 via Frontend Transport; Thu, 12 May 2022 15:44:01 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Thu, 12 May 2022 15:44:01 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Thu, 12 May 2022 08:44:00 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.22 via Frontend Transport; Thu, 12 May 2022 08:43:56 -0700 From: Avihai Horon To: , "Michael S . Tsirkin" , "Cornelia Huck" , Paolo Bonzini , "Alex Williamson" , Juan Quintela , "Dr . David Alan Gilbert" CC: Yishai Hadas , Jason Gunthorpe , "Mark Bloch" , Maor Gottlieb , Kirti Wankhede , Tarun Gupta Subject: [PATCH 9/9] docs/devel: Align vfio-migration docs to VFIO migration v2 Date: Thu, 12 May 2022 18:43:20 +0300 Message-ID: <20220512154320.19697-10-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220512154320.19697-1-avihaih@nvidia.com> References: <20220512154320.19697-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e6cb182f-2ea1-4907-940c-08da342e3cb9 X-MS-TrafficTypeDiagnostic: MWHPR12MB1248:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: A6b33mxuAQPa609N70d8YL/RGRHSG2Gj5b8SJlglZXV9RlgLv6NALocdZxK8+N5NQ2/8Q06fcmhliuYaEtHjM2NjZzpO1FeL8tanM9u0cXfGi9u4cbAkMZbpM7hAVDncNffN5pgpD1iZcMCymOHn0Cm8F3binUCrb1FCk1LK6zDhFEpc9hqUUDaD8lLgPSe191CdyzWdYzIZGNJ0chthic/gmg8BkUUnSffJJQ8U6P0z5hwbwpDSquC2y/rbrnfXylihQj9/QmwgZbb719ufEZBmfHoIHfJxG7rv8LfUh0WquBQK3+V66MrlE6/B+b14CWFlom7bI7qbSglI5c+cYH4pZq5WnGINegcCLDByig27A6OH/din11TPUS+hkcj9+H70Ed+GnyofCWgOjj5wkO8ZmJ0J4PsIdLccCmFai7pug6X0eu7qvCJYBMkqgXp9oxXS36czjccCsab6pav/391eAqwuOR6/6MmGv2W+JdEx2JUPayk88Vb5IL8n0zwZJFwx1TqTpo8sPdd3ictAeqAW9EuXGe2jvQYAzLdJcGWmZ/xba5XE09KplfLxcJa2gtHlGA2oaUPcmwVhmWg8Pgp8b7lYjBMUsxWxp7vOe14+hH4eFPMnrpTW2ziKjZBdFw2cwAQcFq69SMavWUbFfCqpvOBbgnE21dABOqJiwqvMOQ3o/PncMfbgAdzZzMASc8hkRp4P6KRpQxZSQ0TOTA== X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(8676002)(316002)(81166007)(4326008)(70586007)(356005)(508600001)(1076003)(5660300002)(186003)(54906003)(2616005)(110136005)(107886003)(36756003)(8936002)(82310400005)(336012)(47076005)(426003)(26005)(7696005)(83380400001)(2906002)(40460700003)(6666004)(86362001)(36860700001)(70206006)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2022 15:44:01.7187 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e6cb182f-2ea1-4907-940c-08da342e3cb9 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT010.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1248 Received-SPF: softfail client-ip=2a01:111:f400:7e8b::611; envelope-from=avihaih@nvidia.com; helo=NAM04-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Align the vfio-migration documentation to VFIO migration protocol v2. Signed-off-by: Avihai Horon --- docs/devel/vfio-migration.rst | 77 +++++++++++++++-------------------- 1 file changed, 33 insertions(+), 44 deletions(-) diff --git a/docs/devel/vfio-migration.rst b/docs/devel/vfio-migration.rst index 9ff6163c88..09744af5a6 100644 --- a/docs/devel/vfio-migration.rst +++ b/docs/devel/vfio-migration.rst @@ -7,46 +7,35 @@ the guest is running on source host and restoring this saved state on the destination host. This document details how saving and restoring of VFIO devices is done in QEMU. -Migration of VFIO devices consists of two phases: the optional pre-copy phase, -and the stop-and-copy phase. The pre-copy phase is iterative and allows to -accommodate VFIO devices that have a large amount of data that needs to be -transferred. The iterative pre-copy phase of migration allows for the guest to -continue whilst the VFIO device state is transferred to the destination, this -helps to reduce the total downtime of the VM. VFIO devices can choose to skip -the pre-copy phase of migration by returning pending_bytes as zero during the -pre-copy phase. +Migration of VFIO devices currently consists of a single stop-and-copy phase. +During the stop-and-copy phase the guest is stopped and the entire VFIO device +data is transferred to the destination. + +The pre-copy phase of migration is currently not supported for VFIO devices, +so VFIO device data is not transferred during pre-copy phase. A detailed description of the UAPI for VFIO device migration can be found in -the comment for the ``vfio_device_migration_info`` structure in the header -file linux-headers/linux/vfio.h. +the comment for the ``vfio_device_mig_state`` structure in the header file +linux-headers/linux/vfio.h. VFIO implements the device hooks for the iterative approach as follows: -* A ``save_setup`` function that sets up the migration region and sets _SAVING - flag in the VFIO device state. - -* A ``load_setup`` function that sets up the migration region on the - destination and sets _RESUMING flag in the VFIO device state. - -* A ``save_live_pending`` function that reads pending_bytes from the vendor - driver, which indicates the amount of data that the vendor driver has yet to - save for the VFIO device. +* A ``save_setup`` function that sets up migration on the source. -* A ``save_live_iterate`` function that reads the VFIO device's data from the - vendor driver through the migration region during iterative phase. +* A ``load_setup`` function that sets the VFIO device on the destination in + _RESUMING state. * A ``save_state`` function to save the device config space if it is present. -* A ``save_live_complete_precopy`` function that resets _RUNNING flag from the - VFIO device state and iteratively copies the remaining data for the VFIO - device until the vendor driver indicates that no data remains (pending bytes - is zero). +* A ``save_live_complete_precopy`` function that sets the VFIO device in + _STOP_COPY state and iteratively copies the data for the VFIO device until + the vendor driver indicates that no data remains. * A ``load_state`` function that loads the config section and the data - sections that are generated by the save functions above + sections that are generated by the save functions above. * ``cleanup`` functions for both save and load that perform any migration - related cleanup, including unmapping the migration region + related cleanup. The VFIO migration code uses a VM state change handler to change the VFIO @@ -71,13 +60,13 @@ tracking can identify dirtied pages, but any page pinned by the vendor driver can also be written by the device. There is currently no device or IOMMU support for dirty page tracking in hardware. -By default, dirty pages are tracked when the device is in pre-copy as well as -stop-and-copy phase. So, a page pinned by the vendor driver will be copied to -the destination in both phases. Copying dirty pages in pre-copy phase helps -QEMU to predict if it can achieve its downtime tolerances. If QEMU during -pre-copy phase keeps finding dirty pages continuously, then it understands -that even in stop-and-copy phase, it is likely to find dirty pages and can -predict the downtime accordingly. +By default, dirty pages are tracked during pre-copy as well as stop-and-copy +phase. So, a page pinned by the vendor driver will be copied to the destination +in both phases. Copying dirty pages in pre-copy phase helps QEMU to predict if +it can achieve its downtime tolerances. If QEMU during pre-copy phase keeps +finding dirty pages continuously, then it understands that even in stop-and-copy +phase, it is likely to find dirty pages and can predict the downtime +accordingly. QEMU also provides a per device opt-out option ``pre-copy-dirty-page-tracking`` which disables querying the dirty bitmap during pre-copy phase. If it is set to @@ -111,23 +100,23 @@ Live migration save path | migrate_init spawns migration_thread Migration thread then calls each device's .save_setup() - (RUNNING, _SETUP, _RUNNING|_SAVING) + (RUNNING, _SETUP, _RUNNING) | - (RUNNING, _ACTIVE, _RUNNING|_SAVING) - If device is active, get pending_bytes by .save_live_pending() - If total pending_bytes >= threshold_size, call .save_live_iterate() - Data of VFIO device for pre-copy phase is copied + (RUNNING, _ACTIVE, _RUNNING) + Migration thread calls each .save_live_pending() handler + If total pending_bytes >= threshold_size, call each .save_live_iterate() handler + Data of this iteration for pre-copy phase is copied Iterate till total pending bytes converge and are less than threshold | On migration completion, vCPU stops and calls .save_live_complete_precopy for - each active device. The VFIO device is then transitioned into _SAVING state - (FINISH_MIGRATE, _DEVICE, _SAVING) + each active device. The VFIO device is then transitioned into _STOP_COPY state + (FINISH_MIGRATE, _DEVICE, _STOP_COPY) | For the VFIO device, iterate in .save_live_complete_precopy until pending data is 0 - (FINISH_MIGRATE, _DEVICE, _STOPPED) + (FINISH_MIGRATE, _DEVICE, _STOP) | - (FINISH_MIGRATE, _COMPLETED, _STOPPED) + (FINISH_MIGRATE, _COMPLETED, _STOP) Migraton thread schedules cleanup bottom half and exits Live migration resume path @@ -136,7 +125,7 @@ Live migration resume path :: Incoming migration calls .load_setup for each device - (RESTORE_VM, _ACTIVE, _STOPPED) + (RESTORE_VM, _ACTIVE, _STOP) | For each device, .load_state is called for that device section data (RESTORE_VM, _ACTIVE, _RESUMING)