From patchwork Thu Nov 3 16:16:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 10931C4332F for ; Thu, 3 Nov 2022 16:18:33 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqctl-0000QT-PK; Thu, 03 Nov 2022 12:16:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqctg-0000F8-5H; Thu, 03 Nov 2022 12:16:55 -0400 Received: from mail-bn8nam12on2041.outbound.protection.outlook.com ([40.107.237.41] helo=NAM12-BN8-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqctd-00081M-B6; Thu, 03 Nov 2022 12:16:51 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XaIgGE1Y38GEaiWw04WNS2Hc6inUly/SiZ1/3uDkouxvfkoEOSCUoRI22hjssU/4fVVllfQFhbPkbh5z94EJZukUg7fUN0JXkCKqVE/DMly2T5d/kjfIJ8/kt8fM7JrsxHIL4Cd1imBZNVWNExVh+EmOSXF7CpKNUm+nZiLpLGTZq7wxaIgG3sNG0FbdJLyMmZrhjL6HR5zlo1azZB2Q2QKJb36oTkXfk1JnDhWI14TYlcVdEW8iw3PhoI3TuVyvDgvafXtSp+ykaBHZifL8J0OhDXOIqWiHQ2Hh8Ke45G/Aa7ZqozhNDrs7MNmSxRlnUfjmRZ9O2jR3bR4cObfvhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ojSbGep+UgJPkv1W2MpwnqwZEPV+cTBk7M4f4wct/Lo=; b=kN+dHolsFLgNvwGjyAR2A9YU5vwlaElVc3ycT7ljQhSLwwKc4Z3MEnxkofxBMZdymZKjgBYRozkSm6x4Vx+5PDF9QMFeplcWlqxKbKhDVdMarjB5ZroEg2xHvFCEA4UpjhSyLgAT73ZZ+frRNfDCg3ZO/koqEOSFe0LkgAyhDs9/qehdRhLp2Mzt6G4/GAH64kt+XNO5OqzJYFuovQXXzBAz8zavsCTa8W77gxtLA10xtqLCrgGxpVbEHSg7MRc+ATSIm6lGBSLUyWiG0DEIh1jyigQDh1KEQV2kKHrsBHuPPoZI1fPomMBhJxBP4iBkUWzh2JceveDF4Ow8jokJ1w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ojSbGep+UgJPkv1W2MpwnqwZEPV+cTBk7M4f4wct/Lo=; b=cb8JGgulhdiPSkI9hZtsFcvlR/oYsZgkeHxAEPtJYLaYfzyPMhsNKn/iRJWS96zpRKydNW82LD0dAE6cl3cK3Yn+3CJpk4CRh/7jWKRdnwAEX/SC+8Ji4VP1AVqbG9rpR+6hy6EOAw6c2hxFwADcGxCmQmNB/uJI+NXRXiF+BSJYxc0gtrvvCnPZJLLXEBsLXPB62r5UQfdZiAdXqvSkXEDDGT2/soq1I38FbsCF32CJBAzO4p4TkbbsSc9BC4kP4U+lcVutmW9UkWDY08Pk2w+rACndLULcXFX9p9KN99lyNjZM3JfFj/21tAajSj8TuYCQh1rK0yIyLnqNl07mgA== Received: from BN8PR03CA0012.namprd03.prod.outlook.com (2603:10b6:408:94::25) by DS7PR12MB6357.namprd12.prod.outlook.com (2603:10b6:8:96::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.15; Thu, 3 Nov 2022 16:16:45 +0000 Received: from BN8NAM11FT022.eop-nam11.prod.protection.outlook.com (2603:10b6:408:94:cafe::38) by BN8PR03CA0012.outlook.office365.com (2603:10b6:408:94::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22 via Frontend Transport; Thu, 3 Nov 2022 16:16:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by BN8NAM11FT022.mail.protection.outlook.com (10.13.176.112) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:16:44 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:16:36 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:16:36 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:16:29 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 01/17] migration: Remove res_compatible parameter Date: Thu, 3 Nov 2022 18:16:04 +0200 Message-ID: <20221103161620.13120-2-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT022:EE_|DS7PR12MB6357:EE_ X-MS-Office365-Filtering-Correlation-Id: b25c24e3-2262-47e5-daf1-08dabdb6ccec X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hwZCvR01G4VLxOD+4AnF1m09QisLjBRNxBSZ/NzbBb9hsof9VfnqRkeT7ZZLW8b3j32v5ovrTKJ0ymdswnn1JAY8qpX8rbTWAP5e6gweIEWMBdSHvVuFtfe7d5dJy2kKLom8x0EBX6t2UBe8vqQFqo0TVNUAgfnn966uthNbrd717hnn7/OLAPtup0lH3h0b15GuCKXphNRCqWX4ORr+q4Sjlba+hEp/9bYTh0HVMJD9hBzqwLjuK95puRhQiwPi2oiR4KZlls5lv2LNOlTpnF2Q+gfN+ItE43nKjsm9V572Ef1KvlODMcnVg8lErgiXLPGVSqX3Cp8JAMlfMkpf4Dqe+7QCApeFyyA7eFe4f6eEV1AChTdOBmquB4iJD6fCrOmUVa0G7NNX1QMlp3ZxqK8/0nfOEuFhH5o+yk3LGF/yJJyUZn42DDP0+QcoELdRC7s3TZ6BKnwy6XlY1+YabRR12ikEYcf6wRD7ijBVclrLNKOxm3IiL+7xrQL8Tb7MzIlTKVvY49d/uo3KdXCYzdlSst18yexuf3SUfwGDCloaMjuPUeI9Pfr09xZ5k66RIoS2H0FrAcEIq366GXBEnIZI8UrBzoDdoRcTLoeLRThfyT0/RmGhTPs96/nC0wI1aQ9utVR5tG4BqENyW+YDXE9dGnE7K6hByUKKNX7NMt+OGaIeJAC0Q4WDZhUPFf1JjAKFCfz0MyYKZ+prC8gIaHQPR1U6p10OS+zJYWtEzJQt7IbVz3BxGoBz13IFrgejgjUNBvlmZOdM0pG0jYn/qg== X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(396003)(39860400002)(376002)(136003)(346002)(451199015)(40470700004)(46966006)(36840700001)(356005)(40460700003)(336012)(1076003)(7636003)(2906002)(186003)(478600001)(82740400003)(41300700001)(8676002)(36756003)(36860700001)(426003)(7416002)(54906003)(6666004)(70586007)(70206006)(6916009)(7696005)(82310400005)(5660300002)(2616005)(316002)(47076005)(40480700001)(26005)(8936002)(30864003)(4326008)(83380400001)(86362001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:16:44.4243 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b25c24e3-2262-47e5-daf1-08dabdb6ccec X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT022.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6357 Received-SPF: softfail client-ip=40.107.237.41; envelope-from=avihaih@nvidia.com; helo=NAM12-BN8-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Juan Quintela It was only used for RAM, and in that case, it means that this amount of data was sent for memory. Just delete the field in all callers. Signed-off-by: Juan Quintela --- hw/s390x/s390-stattrib.c | 6 ++---- hw/vfio/migration.c | 10 ++++------ hw/vfio/trace-events | 2 +- include/migration/register.h | 20 ++++++++++---------- migration/block-dirty-bitmap.c | 7 +++---- migration/block.c | 7 +++---- migration/migration.c | 9 ++++----- migration/ram.c | 8 +++----- migration/savevm.c | 14 +++++--------- migration/savevm.h | 4 +--- migration/trace-events | 2 +- 11 files changed, 37 insertions(+), 52 deletions(-) diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index 9eda1c3b2a..ee60b53da4 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -183,16 +183,14 @@ static int cmma_save_setup(QEMUFile *f, void *opaque) } static void cmma_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) + uint64_t *res_precopy, uint64_t *res_postcopy) { S390StAttribState *sas = S390_STATTRIB(opaque); S390StAttribClass *sac = S390_STATTRIB_GET_CLASS(sas); long long res = sac->get_dirtycount(sas); if (res >= 0) { - *res_precopy_only += res; + *res_precopy += res; } } diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 3de4252111..3423f113f0 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -458,9 +458,8 @@ static void vfio_save_cleanup(void *opaque) static void vfio_save_pending(QEMUFile *f, void *opaque, uint64_t threshold_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) + uint64_t *res_precopy, + uint64_t *res_postcopy) { VFIODevice *vbasedev = opaque; VFIOMigration *migration = vbasedev->migration; @@ -471,10 +470,9 @@ static void vfio_save_pending(QEMUFile *f, void *opaque, return; } - *res_precopy_only += migration->pending_bytes; + *res_precopy += migration->pending_bytes; - trace_vfio_save_pending(vbasedev->name, *res_precopy_only, - *res_postcopy_only, *res_compatible); + trace_vfio_save_pending(vbasedev->name, *res_precopy, *res_postcopy); } static int vfio_save_iterate(QEMUFile *f, void *opaque) diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 73dffe9e00..a21cbd2a56 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -157,7 +157,7 @@ vfio_save_cleanup(const char *name) " (%s)" vfio_save_buffer(const char *name, uint64_t data_offset, uint64_t data_size, uint64_t pending) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64" pending 0x%"PRIx64 vfio_update_pending(const char *name, uint64_t pending) " (%s) pending 0x%"PRIx64 vfio_save_device_config_state(const char *name) " (%s)" -vfio_save_pending(const char *name, uint64_t precopy, uint64_t postcopy, uint64_t compatible) " (%s) precopy 0x%"PRIx64" postcopy 0x%"PRIx64" compatible 0x%"PRIx64 +vfio_save_pending(const char *name, uint64_t precopy, uint64_t postcopy) " (%s) precopy 0x%"PRIx64" postcopy 0x%"PRIx64 vfio_save_iterate(const char *name, int data_size) " (%s) data_size %d" vfio_save_complete_precopy(const char *name) " (%s)" vfio_load_device_config_state(const char *name) " (%s)" diff --git a/include/migration/register.h b/include/migration/register.h index c1dcff0f90..1950fee6a8 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -48,18 +48,18 @@ typedef struct SaveVMHandlers { int (*save_setup)(QEMUFile *f, void *opaque); void (*save_live_pending)(QEMUFile *f, void *opaque, uint64_t threshold_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only); + uint64_t *rest_precopy, + uint64_t *rest_postcopy); /* Note for save_live_pending: - * - res_precopy_only is for data which must be migrated in precopy phase - * or in stopped state, in other words - before target vm start - * - res_compatible is for data which may be migrated in any phase - * - res_postcopy_only is for data which must be migrated in postcopy phase - * or in stopped state, in other words - after source vm stop + * - res_precopy is for data which must be migrated in precopy + * phase or in stopped state, in other words - before target + * vm start + * - res_postcopy is for data which must be migrated in postcopy + * phase or in stopped state, in other words - after source vm + * stop * - * Sum of res_postcopy_only, res_compatible and res_postcopy_only is the - * whole amount of pending data. + * Sum of res_precopy and res_postcopy is the whole amount of + * pending data. */ diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 9aba7d9c22..dfea546330 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -763,9 +763,8 @@ static int dirty_bitmap_save_complete(QEMUFile *f, void *opaque) static void dirty_bitmap_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) + uint64_t *res_precopy, + uint64_t *res_postcopy) { DBMSaveState *s = &((DBMState *)opaque)->save; SaveBitmapState *dbms; @@ -785,7 +784,7 @@ static void dirty_bitmap_save_pending(QEMUFile *f, void *opaque, trace_dirty_bitmap_save_pending(pending, max_size); - *res_postcopy_only += pending; + *res_postcopy += pending; } /* First occurrence of this bitmap. It should be created if doesn't exist */ diff --git a/migration/block.c b/migration/block.c index 3577c815a9..4ae8f837b0 100644 --- a/migration/block.c +++ b/migration/block.c @@ -863,9 +863,8 @@ static int block_save_complete(QEMUFile *f, void *opaque) } static void block_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) + uint64_t *res_precopy, + uint64_t *res_postcopy) { /* Estimate pending number of bytes to send */ uint64_t pending; @@ -886,7 +885,7 @@ static void block_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, trace_migration_block_save_pending(pending); /* We don't do postcopy */ - *res_precopy_only += pending; + *res_precopy += pending; } static int block_load(QEMUFile *f, void *opaque, int version_id) diff --git a/migration/migration.c b/migration/migration.c index 739bb683f3..a4a18228c6 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3735,15 +3735,14 @@ typedef enum { */ static MigIterateState migration_iteration_run(MigrationState *s) { - uint64_t pending_size, pend_pre, pend_compat, pend_post; + uint64_t pending_size, pend_pre, pend_post; bool in_postcopy = s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE; qemu_savevm_state_pending(s->to_dst_file, s->threshold_size, &pend_pre, - &pend_compat, &pend_post); - pending_size = pend_pre + pend_compat + pend_post; + &pend_post); + pending_size = pend_pre + pend_post; - trace_migrate_pending(pending_size, s->threshold_size, - pend_pre, pend_compat, pend_post); + trace_migrate_pending(pending_size, s->threshold_size, pend_pre, pend_post); if (pending_size && pending_size >= s->threshold_size) { /* Still a significant amount to transfer */ diff --git a/migration/ram.c b/migration/ram.c index dc1de9ddbc..20167e1102 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3435,9 +3435,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) } static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) + uint64_t *res_precopy, uint64_t *res_postcopy) { RAMState **temp = opaque; RAMState *rs = *temp; @@ -3457,9 +3455,9 @@ static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, if (migrate_postcopy_ram()) { /* We can do postcopy, and all the data is postcopiable */ - *res_compatible += remaining_size; + *res_postcopy += remaining_size; } else { - *res_precopy_only += remaining_size; + *res_precopy += remaining_size; } } diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..4d02887f25 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1472,16 +1472,13 @@ flush: * for units that can't do postcopy. */ void qemu_savevm_state_pending(QEMUFile *f, uint64_t threshold_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) + uint64_t *res_precopy, + uint64_t *res_postcopy) { SaveStateEntry *se; - *res_precopy_only = 0; - *res_compatible = 0; - *res_postcopy_only = 0; - + *res_precopy = 0; + *res_postcopy = 0; QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { if (!se->ops || !se->ops->save_live_pending) { @@ -1493,8 +1490,7 @@ void qemu_savevm_state_pending(QEMUFile *f, uint64_t threshold_size, } } se->ops->save_live_pending(f, se->opaque, threshold_size, - res_precopy_only, res_compatible, - res_postcopy_only); + res_precopy, res_postcopy); } } diff --git a/migration/savevm.h b/migration/savevm.h index 6461342cb4..9bd55c336c 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -41,9 +41,7 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only, bool inactivate_disks); void qemu_savevm_state_pending(QEMUFile *f, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only); + uint64_t *res_precopy, uint64_t *res_postcopy); void qemu_savevm_send_ping(QEMUFile *f, uint32_t value); void qemu_savevm_send_open_return_path(QEMUFile *f); int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t *buf, size_t len); diff --git a/migration/trace-events b/migration/trace-events index 57003edcbd..f2a873fd6c 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -150,7 +150,7 @@ migrate_fd_cleanup(void) "" migrate_fd_error(const char *error_desc) "error=%s" migrate_fd_cancel(void) "" migrate_handle_rp_req_pages(const char *rbname, size_t start, size_t len) "in %s at 0x%zx len 0x%zx" -migrate_pending(uint64_t size, uint64_t max, uint64_t pre, uint64_t compat, uint64_t post) "pending size %" PRIu64 " max %" PRIu64 " (pre = %" PRIu64 " compat=%" PRIu64 " post=%" PRIu64 ")" +migrate_pending(uint64_t size, uint64_t max, uint64_t pre, uint64_t post) "pending size %" PRIu64 " max %" PRIu64 " (pre = %" PRIu64 " post=%" PRIu64 ")" migrate_send_rp_message(int msg_type, uint16_t len) "%d: len %d" migrate_send_rp_recv_bitmap(char *name, int64_t size) "block '%s' size 0x%"PRIi64 migration_completion_file_err(void) "" From patchwork Thu Nov 3 16:16:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030514 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 72EC8C43219 for ; Thu, 3 Nov 2022 16:18:18 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcuH-0000aY-1G; Thu, 03 Nov 2022 12:17:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqctp-0000VO-GZ; Thu, 03 Nov 2022 12:17:01 -0400 Received: from mail-dm6nam11on2068.outbound.protection.outlook.com ([40.107.223.68] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqctn-00082S-CH; Thu, 03 Nov 2022 12:17:01 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dYFgnANc9X74eFFpzZcDnHgIz0Yxr0OA9AyVJitKz1cAqj1uFKrPDRCws6/YMFsor1eDed7Ij26AwDwuBxndu0xj2A7OBKCBhUx8CXwM0ySV04w7H2Efc9SFZVbl7fqQEGbB2eSvjbhVZ0msVzWTY4L5A5ma9QIe7AvVzsgOy/ke06ziVfWwa271DOMUpQJWw7UoEw4BORyCnE1EC5iO7RnjfVOxCNXjId5VorMjW1zCzCyJCv/19bqPF0yBrTE7FiktttLWSwwxOlFfOX9lQObEfvbFnWr26GSmxYTx3EQTUFCsPN/PqStI7IuwcorJx0TSKAPEKn2lRxP+HYib4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nwIBRUPyKH6BIuLMLCzubPbprEaZWUD3nTtn07OAp4I=; b=XCWmZGQfKblHz5X/6gWexV7kp1svIK1MIVvNnzY9YuatRbTN0PBnt8Eh7l4L5WtC+6Ga7YxsiEzeHAcnsQ/eSMmmWL8NWq72KVLm8qkA3P//JrIH5WnYNLBzF/QFxWUBK3u+vbeHdhYM6XLmE9wOrtF/JosZRhNamgQ2eJksAJqBnH4bCTU/et2vFsr8mROei/9vVbgJ5wwdYQcf/PFdLlQSq+IKO7L5qRNkeN7hwbhD/OZYXJ5im2Jk0dE7eNEAcRRZ/CsW/U+1v5xBUyXFfAzez2H4jVGejwhyf4z1JzQ6d2TdwBWh1krAWhdP6COHE57my0GtzYMEVERBDh4nSg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nwIBRUPyKH6BIuLMLCzubPbprEaZWUD3nTtn07OAp4I=; b=T8b/rRc1uimkT3a05nortW7yXETTQ0bubA5xwD9F2AEnbSzgOqE4GQDmlZ5baIOIuRmES3SjPtDClyI+jIA4L4J0DJRQ/6QvCi0IUxYb8Fq6W/4iTX5HF9BOsAnrNtuh70TankUEspCYV+28xCsZ6UF0Bg6sPMlfbJOppltoBUrFffgVAUI8QfPTx3Ub/QgDRD0OxnciZbtOLSbOIPMkrBiLlciVCwfqSRyrYDKYycco4KRvn7PuIEZ7Bj8ZAYh78kEL3gSFX9VyQQGDDoto5tXyhZl5GwU49OBSwdnslrQ+BfA7Q4tfpRMSpTXbg1iinsFtz5UTBbNxs3cWFhmzBA== Received: from BN0PR07CA0009.namprd07.prod.outlook.com (2603:10b6:408:141::8) by IA0PR12MB7601.namprd12.prod.outlook.com (2603:10b6:208:43b::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22; Thu, 3 Nov 2022 16:16:55 +0000 Received: from BN8NAM11FT069.eop-nam11.prod.protection.outlook.com (2603:10b6:408:141:cafe::76) by BN0PR07CA0009.outlook.office365.com (2603:10b6:408:141::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.21 via Frontend Transport; Thu, 3 Nov 2022 16:16:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by BN8NAM11FT069.mail.protection.outlook.com (10.13.176.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:16:54 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:16:44 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:16:43 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:16:36 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 02/17] migration: No save_live_pending() method uses the QEMUFile parameter Date: Thu, 3 Nov 2022 18:16:05 +0200 Message-ID: <20221103161620.13120-3-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT069:EE_|IA0PR12MB7601:EE_ X-MS-Office365-Filtering-Correlation-Id: 577061a2-f56c-4611-ccce-08dabdb6d2e5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: C7P6RrYppJ5U2qf9n0X7cG1F4bstLjV96E5hDYIP+xWxDm3V4yde16tr1wzMpBBybwPVT9l9YetPrY9igu+xfXSfWVbn/517LPMt+Dzy2KQecH9luRvGe8QEOcO2Yj8csxM7rekBLGmDgy2SWBoLBa9Rl6mNuauX9EXMrhlS+cEqjDG2CB6fThT2Z3RNiEKRT6mANdU6uqZzgbUhSjaFyObBruW32CAMxk1vI6UMfMatU/lBSNPsEvQE9BJJnoJChhM5z2oiqieYMXu+ruYExBG9KQNBv+OiNQGjtJXJi9XU32m6iYhXLalCIlrSyehhP02BRhNNab40UxxO04+4IvzJY9AqA6DxiYmF5c36NrHh9VCyzzoc+SDEx/sm5WY+Bi6kfBW04BGsXmqP6TT7QvzuQKo+PfjwAJpsy/5zlhumYDi6aE49lWNcDoE1JKSOp6Rpq1xa4cud2rRrEqePGku5eFtOcJuieHxKQ5OH+PbhGbtgL1IV7Ci14nypTzmj4zK56tm9GcAs/ZDrZWIfA8PeKyYecuZtbTaq1plrdkKIVc1WBBwyv4SMct5J2Tplo1vY6kgZuriVhoS1W9gB94Cb8Sne7wQMLTHXvgzRgwGbgNnw5XS74z7GHfnMcjQqmggdNQJia80J11StuXjNNIjgEFGoaehGABQBZmha5vYEapljBJdFkxMrp1GgwMZT4AHGP+sM4ji9dCQMcDsNYkOhhIEcSC60bAkWRhZJDJjIDN5EWTAjyYTMPVdQ7RLf/ZlfEZa8Biyq9dqLAl5+yg== X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(136003)(376002)(396003)(346002)(451199015)(40470700004)(36840700001)(46966006)(36756003)(82740400003)(478600001)(6666004)(82310400005)(4326008)(2906002)(40460700003)(7416002)(70586007)(70206006)(36860700001)(5660300002)(86362001)(8936002)(41300700001)(356005)(316002)(54906003)(6916009)(40480700001)(7636003)(8676002)(83380400001)(186003)(336012)(2616005)(426003)(1076003)(7696005)(26005)(47076005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:16:54.4444 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 577061a2-f56c-4611-ccce-08dabdb6d2e5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT069.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB7601 Received-SPF: softfail client-ip=40.107.223.68; envelope-from=avihaih@nvidia.com; helo=NAM11-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Juan Quintela So remove it everywhere. Signed-off-by: Juan Quintela Reviewed-by: Vladimir Sementsov-Ogievskiy --- hw/s390x/s390-stattrib.c | 2 +- hw/vfio/migration.c | 6 ++---- include/migration/register.h | 6 ++---- migration/block-dirty-bitmap.c | 3 +-- migration/block.c | 2 +- migration/migration.c | 3 +-- migration/ram.c | 2 +- migration/savevm.c | 5 ++--- migration/savevm.h | 2 +- 9 files changed, 12 insertions(+), 19 deletions(-) diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index ee60b53da4..9b74eeadf3 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -182,7 +182,7 @@ static int cmma_save_setup(QEMUFile *f, void *opaque) return 0; } -static void cmma_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, +static void cmma_save_pending(void *opaque, uint64_t max_size, uint64_t *res_precopy, uint64_t *res_postcopy) { S390StAttribState *sas = S390_STATTRIB(opaque); diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 3423f113f0..760d5f3c5c 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -456,10 +456,8 @@ static void vfio_save_cleanup(void *opaque) trace_vfio_save_cleanup(vbasedev->name); } -static void vfio_save_pending(QEMUFile *f, void *opaque, - uint64_t threshold_size, - uint64_t *res_precopy, - uint64_t *res_postcopy) +static void vfio_save_pending(void *opaque, uint64_t threshold_size, + uint64_t *res_precopy, uint64_t *res_postcopy) { VFIODevice *vbasedev = opaque; VFIOMigration *migration = vbasedev->migration; diff --git a/include/migration/register.h b/include/migration/register.h index 1950fee6a8..5b5424ed8f 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -46,10 +46,8 @@ typedef struct SaveVMHandlers { /* This runs outside the iothread lock! */ int (*save_setup)(QEMUFile *f, void *opaque); - void (*save_live_pending)(QEMUFile *f, void *opaque, - uint64_t threshold_size, - uint64_t *rest_precopy, - uint64_t *rest_postcopy); + void (*save_live_pending)(void *opaque, uint64_t threshold_size, + uint64_t *rest_precopy, uint64_t *rest_postcopy); /* Note for save_live_pending: * - res_precopy is for data which must be migrated in precopy * phase or in stopped state, in other words - before target diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index dfea546330..9d4f56693f 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -761,8 +761,7 @@ static int dirty_bitmap_save_complete(QEMUFile *f, void *opaque) return 0; } -static void dirty_bitmap_save_pending(QEMUFile *f, void *opaque, - uint64_t max_size, +static void dirty_bitmap_save_pending(void *opaque, uint64_t max_size, uint64_t *res_precopy, uint64_t *res_postcopy) { diff --git a/migration/block.c b/migration/block.c index 4ae8f837b0..b3d680af75 100644 --- a/migration/block.c +++ b/migration/block.c @@ -862,7 +862,7 @@ static int block_save_complete(QEMUFile *f, void *opaque) return 0; } -static void block_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, +static void block_save_pending(void *opaque, uint64_t max_size, uint64_t *res_precopy, uint64_t *res_postcopy) { diff --git a/migration/migration.c b/migration/migration.c index a4a18228c6..ffe868b86f 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3738,8 +3738,7 @@ static MigIterateState migration_iteration_run(MigrationState *s) uint64_t pending_size, pend_pre, pend_post; bool in_postcopy = s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE; - qemu_savevm_state_pending(s->to_dst_file, s->threshold_size, &pend_pre, - &pend_post); + qemu_savevm_state_pending(s->threshold_size, &pend_pre, &pend_post); pending_size = pend_pre + pend_post; trace_migrate_pending(pending_size, s->threshold_size, pend_pre, pend_post); diff --git a/migration/ram.c b/migration/ram.c index 20167e1102..48a31b87c8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3434,7 +3434,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) return 0; } -static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, +static void ram_save_pending(void *opaque, uint64_t max_size, uint64_t *res_precopy, uint64_t *res_postcopy) { RAMState **temp = opaque; diff --git a/migration/savevm.c b/migration/savevm.c index 4d02887f25..9ddcbba4e3 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1471,8 +1471,7 @@ flush: * the result is split into the amount for units that can and * for units that can't do postcopy. */ -void qemu_savevm_state_pending(QEMUFile *f, uint64_t threshold_size, - uint64_t *res_precopy, +void qemu_savevm_state_pending(uint64_t threshold_size, uint64_t *res_precopy, uint64_t *res_postcopy) { SaveStateEntry *se; @@ -1489,7 +1488,7 @@ void qemu_savevm_state_pending(QEMUFile *f, uint64_t threshold_size, continue; } } - se->ops->save_live_pending(f, se->opaque, threshold_size, + se->ops->save_live_pending(se->opaque, threshold_size, res_precopy, res_postcopy); } } diff --git a/migration/savevm.h b/migration/savevm.h index 9bd55c336c..98fae6f9b3 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -40,7 +40,7 @@ void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only, bool inactivate_disks); -void qemu_savevm_state_pending(QEMUFile *f, uint64_t max_size, +void qemu_savevm_state_pending(uint64_t max_size, uint64_t *res_precopy, uint64_t *res_postcopy); void qemu_savevm_send_ping(QEMUFile *f, uint32_t value); void qemu_savevm_send_open_return_path(QEMUFile *f); From patchwork Thu Nov 3 16:16:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030516 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A20E3C4332F for ; Thu, 3 Nov 2022 16:18:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcuN-0000fM-Us; Thu, 03 Nov 2022 12:17:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcty-0000Y5-8V; Thu, 03 Nov 2022 12:17:27 -0400 Received: from mail-bn8nam11on2040.outbound.protection.outlook.com ([40.107.236.40] helo=NAM11-BN8-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqctw-00082x-6u; Thu, 03 Nov 2022 12:17:09 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kkc0+ofeamu5kvOiL7OEu+euxi1ipilyl5WAMIxFvyNZ5sUvMPBGpxwa8z5nK1QIrw1+7N1vkBZ9d7EctrswZh5oVEIfZabWMl4JTUVbgNzzM1G+lfKn9ggdJeMPgBUcg62B3fyDQts46RXOWcfYH+1EGXXNwxC3V53YIsXpV0/8eS/YTa+RFR7GXnRtrPzd95LR8vOYvcZ3NhTOQPTbvAwEGcHSfBz8AoRmQpllb/1/kTQ4N68J/nR21tgin+4R3RbOBdjbh2Csli36aTkEp19IU9qLo3ClnED51uTdTWUJcAbUGYm0RTqfb19S+zxVNZCdm4M96j0dVjxGhFiPYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bPdIohi9gxDAuW35g8Jox4V3qjITSs+uCSD4oJi0QDo=; b=Inzy5Uuc5fI2g7p0cTOgdpizGygc++1swC0Yz0t3MVFZXM/X7WPAJ5WHUJGrvvV5B5b+X3/N+rf+gOvjuVsK6ex8YkjZwC1qz6RE90Qqi5j08jQGgwm6BPHx8I37t0Nxop+ULpMPWd03kVmr91JT0KvVAviV2v04S0VwFfpwBloyvnuiomaGopyuvObPhGwlGmhmBHberlslKaRRn8TfekXzgz9LvxarUtWqM5Sa38E9NkrCZvscA101eSZgRk99mzcen34kF2sA9YoJqV4hJ4fbocCi3FApgrd0N4jxlhzbp3CYnYe9hXdGCI7Ugs6eTJhcSfVnzqIZ+IkjdW01Zw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bPdIohi9gxDAuW35g8Jox4V3qjITSs+uCSD4oJi0QDo=; b=S3l1lOm4ziijzoP7aNpDqq/i3pbLkhUOLU3+5/wl46BvzphQcY97dB4qxMndLcy/xDienNJouG4uciV7Wnlu+3H74sjWSF9pV5mKXXGRHe9QYzIGlJiXRbN73sDFsXsiOpVXPKYyGqiXpEcTIglnXUn35ttw/BDv3i/lXzrgsdTsTnnZmSBPQDLzZloS85SFInr34GFKX0KjOcPLpXm4zpL5cnLi/q85mKZf97prU5Yun9oRmHDZG6QE/EGoao+fSCdjLij8Do4TzH4Sgz+8M+MOD4mx4/iHdqf+nggPelXJZ9oiqIvmlcJXGs8Lo54U4l5+bCVs+5J6w4u6d6WQWg== Received: from MW4P222CA0024.NAMP222.PROD.OUTLOOK.COM (2603:10b6:303:114::29) by BY5PR12MB4097.namprd12.prod.outlook.com (2603:10b6:a03:213::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22; Thu, 3 Nov 2022 16:17:04 +0000 Received: from CO1NAM11FT098.eop-nam11.prod.protection.outlook.com (2603:10b6:303:114:cafe::78) by MW4P222CA0024.outlook.office365.com (2603:10b6:303:114::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.18 via Frontend Transport; Thu, 3 Nov 2022 16:17:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1NAM11FT098.mail.protection.outlook.com (10.13.174.207) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:17:03 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:16:53 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:16:52 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:16:44 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 03/17] migration: Block migration comment or code is wrong Date: Thu, 3 Nov 2022 18:16:06 +0200 Message-ID: <20221103161620.13120-4-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT098:EE_|BY5PR12MB4097:EE_ X-MS-Office365-Filtering-Correlation-Id: 253da080-8a79-4157-f802-08dabdb6d830 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ygYOKWfWIzFqn6f23qAk1ooZstPudZidfwPy/vvsIpZNl1JBedsIy9NXulNq3ijnk90K/QfPnLGfSBtHf3t5fE84la97aIrPAHqQOVGiD5V+8y4gBG7wr3lZ3HNXLtZaGSYUBizZ+FGmd7cDZdKralBTXAQ2nWUmYFHqegghElGkYlbe2FHOMLKyfC2ldbBA9sJfSPtybDvwz3Dn9wpWA6D0h6uoe45iB0f1V/hxo/E8Zp+Rjrza+pQ3MkLnGC408DA5ti1qxoOMcatPnwfyduHEyWIf+/hYWbtNfKCqpB6cjCGyWvWbMbVgdNzxxKL9YdPbz3sp68zze9tQyJx3EqGZtwbm+8lvVJPdwhBClM2Ah6pS/cx3kwyxxFuFCoD/8myD/VXqb+2rVdzCUn0K70xMxE348yAWBtN5brIY52M01vBfqN2Hn6+kR1D5Du9a2gJ/2BO+814x4a+ODp/pQJ5Ps+hUre7OFA2s63Q6ce0RH7qGndXttVj+7ft+XpRIO8e71HQbpmRpqWi6/duh43ZWmYT7xyWMVyZ0lXqTQ+e5kwONtRhkFcOB0nOWKziCWBCjiboFOn6mDlcu/QalrWQbJsh+uFEJVwPrSIvFzAH1xyzmNpJStm+ENpyTKTtv6GHxWCpFSyyfC+FNerImMM2Qy/KqWgJFmwaaSwN86/ZZR2t/bilnXtqyv9TRI/2ZODT1guGBHber53ecvjOwJZ1XFM3JIQTJr9bVFXk1m+qdkeayEKGWAYzhr9ukRbhtxDr8eqKAAnSSHv55sWnjqA== X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(346002)(376002)(39860400002)(136003)(396003)(451199015)(40470700004)(36840700001)(46966006)(4326008)(4744005)(336012)(6666004)(478600001)(70206006)(426003)(70586007)(47076005)(8676002)(186003)(1076003)(7416002)(2616005)(40460700003)(5660300002)(316002)(6916009)(2906002)(82310400005)(36860700001)(41300700001)(356005)(83380400001)(54906003)(7636003)(82740400003)(36756003)(7696005)(8936002)(86362001)(26005)(40480700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:17:03.4194 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 253da080-8a79-4157-f802-08dabdb6d830 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT098.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4097 Received-SPF: softfail client-ip=40.107.236.40; envelope-from=avihaih@nvidia.com; helo=NAM11-BN8-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Juan Quintela And it appears that what is wrong is the code. During bulk stage we need to make sure that some block is dirty, but no games with max_size at all. Signed-off-by: Juan Quintela Reviewed-by: Stefan Hajnoczi --- migration/block.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/migration/block.c b/migration/block.c index b3d680af75..39ce4003c6 100644 --- a/migration/block.c +++ b/migration/block.c @@ -879,8 +879,8 @@ static void block_save_pending(void *opaque, uint64_t max_size, blk_mig_unlock(); /* Report at least one block pending during bulk phase */ - if (pending <= max_size && !block_mig_state.bulk_completed) { - pending = max_size + BLK_MIG_BLOCK_SIZE; + if (!pending && !block_mig_state.bulk_completed) { + pending = BLK_MIG_BLOCK_SIZE; } trace_migration_block_save_pending(pending); From patchwork Thu Nov 3 16:16:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69925C4332F for ; Thu, 3 Nov 2022 16:19:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcvL-0001AO-J0; Thu, 03 Nov 2022 12:18:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcu4-0000Zh-2f; Thu, 03 Nov 2022 12:17:27 -0400 Received: from mail-bn8nam11on2040.outbound.protection.outlook.com ([40.107.236.40] helo=NAM11-BN8-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcu2-00083J-4C; Thu, 03 Nov 2022 12:17:15 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Dwso9fYtjcpQRU/3Ct+TZmfHgZ2UEaxuEvAWHwIFluPC8Zc9YptxPyGUfL6ju2Bej8tXTbcdFmVXGXTs9GHCmBcsbiwj7Wk3P16nKpL3hpJSutrYpTTNuViHrJCRRSvbiLNn+F/eZZsK3DMm7e39+JrGQGpFTOuHmIjhYMLU6vrQXj/GJvhqDgJM/MXVDIGN1BkSJfOz6E1nzCCnHx3FsRCcvZNuwscwjurHlv5Zz1yUMp0l0hGvbwOMtMBjmdiTa3u60L8Ym+kmb4aFN+mt2OxovR1MYH42JXWsROohjdubR92F/XHxOZAxnsMT8nyKbbBteMfK2cubSFc+r+y1KA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=l9mrF8KR6IRzvzJw8cj21mszNev3GcRuLJlTAUyzvPI=; b=I1QpSM2REbiLV4yj+cx4OP2ayavDXbRmrynwFWVY2SLi356zFKb/EW5wR51ocKQ4nzp5u7uYrzO1VnRW+uzYGBX0/tNB1rtopa7zDv0fJ0qdxaKnzVyr0B40pEh0tXmiZ3J5wV/lqM7W6iTK1ZAwVsbMuUlmIy4zcDBYIE59LyEZt1ue3cN5HgKcqIzeuw4RoUqzQVK4DGhXSoUP7cY5sF0Cv8ZNjajKI0bjFoky5AQiA8IWXqB43TVM6zWZHMMe52onmEZ+KIRIr8hW4k5bTgqAAwoiHhH+QCU+FPItvDmxjtx10FYJwyv9g+wT+eyCbosHd8aPt6qIdRmrXB1M+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=l9mrF8KR6IRzvzJw8cj21mszNev3GcRuLJlTAUyzvPI=; b=iROZiz0nsCEMhEmK3kHlB1PH7W/Ah4LgEdrrs/6W9HPNGlZlxZjje1lfgz8QZFOfFD5LvehExBz9ktj3dWLzmmHBpJ66ouHEZG5eOkD9Q/L6PxxDw5KJwu2/Nek0KnVTfqLO+5fEU+wTP64UOIMVjkvEKs7F4/1X9OTkmqrUS5mykLu96DsAbe3WMInE3KOMmE4UrySznyKQ6oS9Hy5Lm68Yee3EDYWZ9ICyvsDDk2Chwd579bwXPsj0VGkM+zDsWpfsphBSW+I5P1N3m3N+To+nsQ24V+xUBlHRqwYsNc6j+OLJStckrFp6ayVJrpxAJ7MhpT5vjVqoaZ55rfBvLQ== Received: from MW4PR04CA0169.namprd04.prod.outlook.com (2603:10b6:303:85::24) by SJ0PR12MB6943.namprd12.prod.outlook.com (2603:10b6:a03:44b::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22; Thu, 3 Nov 2022 16:17:10 +0000 Received: from CO1NAM11FT084.eop-nam11.prod.protection.outlook.com (2603:10b6:303:85:cafe::3f) by MW4PR04CA0169.outlook.office365.com (2603:10b6:303:85::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22 via Frontend Transport; Thu, 3 Nov 2022 16:17:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1NAM11FT084.mail.protection.outlook.com (10.13.174.194) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:17:09 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:17:00 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:16:59 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:16:52 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 04/17] migration: Simplify migration_iteration_run() Date: Thu, 3 Nov 2022 18:16:07 +0200 Message-ID: <20221103161620.13120-5-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT084:EE_|SJ0PR12MB6943:EE_ X-MS-Office365-Filtering-Correlation-Id: ce366fad-b094-43f3-1e3d-08dabdb6dbf7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ElJyjIbuoOCOlRi7GKPpmaeLAAbCH/AR+X3jzoHZear94hNqkLdRYMqCF+4hPAlLBlAf9hfxIv+8q3wW3WhxNowbxvzwFSM+8JJBnS/J+WMCgV6ZHGhmid2s9RrqKTrm9GCVfSzghuHqGgEXVuJdUMHImukr1D0dD3QWk0s+PulMdWmoaJ5DtA+JCqYxVi3SBgyN17TwwCFgj3w8UWnhHyoX3r8QQawV018P+UxwDHyQMMjpwnOMTHDPE6lu9IRfLTb+tDIKGW6NqJ3HjT5AscjjWWFF+0XRvv1xXsKNJulKpEWQWMz81i9OhRoP+0j5iDxnv3Vsa0IeWbkYe53mXzvvrBQwlcy01Lnu27QQQC+EtY/jb8qN3++AZ0K1RU8aHd7fkPH8RhtBvW9i3u/eOj5TX1jehKsRGsKMtByazIPMj9z5/CinFloASKTEgwQXbFiWYwY6xHRZKr7t7vCEr/Yt2iW701fbNl+hIvVASM2LgiHr3RtxWp/e/RpjH3DVojAdVm5J2Pk7PQtM/MOismT3WUvf8PmR9ZFnxL667hULnV+7o/bGtWojlkkmdlGjADyIeFPgIWDni8X52Bfk9Z1iF2Q2fcuIBlERYYecegTBKl5aRJ720JqA1YSNh3MRg9LDgMdw/sLA3Dw+A1RuA/WBxwLSg/CJR5uj68559XB8px5TSilA0s6/nc/HcRZxXFR8+K1a1qWHj3/+ZPf4G8tFk5ibMAscjnSuWN80Rfl55kN77OV5CY6iMO/sDyYnNy/O6M6gzpqRKzyO4y7UAQ== X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(396003)(39860400002)(346002)(136003)(376002)(451199015)(40470700004)(46966006)(36840700001)(86362001)(82740400003)(41300700001)(36860700001)(5660300002)(54906003)(8936002)(7416002)(316002)(47076005)(356005)(6916009)(7696005)(426003)(2616005)(26005)(186003)(1076003)(336012)(7636003)(6666004)(478600001)(82310400005)(40480700001)(4326008)(36756003)(8676002)(70586007)(2906002)(83380400001)(40460700003)(70206006); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:17:09.7512 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ce366fad-b094-43f3-1e3d-08dabdb6dbf7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT084.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6943 Received-SPF: softfail client-ip=40.107.236.40; envelope-from=avihaih@nvidia.com; helo=NAM11-BN8-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Avihai Horon --- migration/migration.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index ffe868b86f..59cc3c309b 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3743,23 +3743,24 @@ static MigIterateState migration_iteration_run(MigrationState *s) trace_migrate_pending(pending_size, s->threshold_size, pend_pre, pend_post); - if (pending_size && pending_size >= s->threshold_size) { - /* Still a significant amount to transfer */ - if (!in_postcopy && pend_pre <= s->threshold_size && - qatomic_read(&s->start_postcopy)) { - if (postcopy_start(s)) { - error_report("%s: postcopy failed to start", __func__); - } - return MIG_ITERATE_SKIP; - } - /* Just another iteration step */ - qemu_savevm_state_iterate(s->to_dst_file, in_postcopy); - } else { + + if (pending_size < s->threshold_size) { trace_migration_thread_low_pending(pending_size); migration_completion(s); return MIG_ITERATE_BREAK; } + /* Still a significant amount to transfer */ + if (!in_postcopy && pend_pre <= s->threshold_size && + qatomic_read(&s->start_postcopy)) { + if (postcopy_start(s)) { + error_report("%s: postcopy failed to start", __func__); + } + return MIG_ITERATE_SKIP; + } + + /* Just another iteration step */ + qemu_savevm_state_iterate(s->to_dst_file, in_postcopy); return MIG_ITERATE_RESUME; } From patchwork Thu Nov 3 16:16:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030512 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 607F8C433FE for ; Thu, 3 Nov 2022 16:18:18 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcuP-0000l5-V2; Thu, 03 Nov 2022 12:17:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcuD-0000aI-40; Thu, 03 Nov 2022 12:17:27 -0400 Received: from mail-dm6nam10on2040.outbound.protection.outlook.com ([40.107.93.40] helo=NAM10-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcuA-00083l-GT; Thu, 03 Nov 2022 12:17:24 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XQGMOL4iIBlUYM0u8h/arCY6bQOlzaZsjS6qoa//Xj86zlIrOUJ+kvZaRuepvVcygyOOrqKAGqNAY0AaeciWYs1A6VuX4NO8uLKzDrWUtOsmlThHVCs8ZqLK4jcwwVd4rUVwxwJoPk5I+8oeNfmDb9O0UPfh2giIGYflMdCHyT/NhCe60wHdcwhPi/USAGi9bf9ondifxZQpVfOW0/1pkvLu8cyouzq3VbSD4ryz0lELPMcU/SxbUdcup8XS3UKoGBfjUkSWyJ+0A+lIdW/DnMgBSafm5zzAKk/PW0upvboqxWxJ6M0K66kWZGPtoWXwiv1kBK+U/LXf/7tEVK0HRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EpUyHKAieomgHAQsQGREWdJpGdjrz4gLNv1Kubsbstk=; b=ZK0KY0/+m7eC6w5/MXG3nA2KVfeyaav7FlAMr/oWvdeckXjP9RQfdw/n6OM0zxssS/QPMwnfLPFK15fAmuPO19NDwv/9X2ZhnlvDau/Ijgpq8JoLErOpIEf2dMOVVJhvYxoOHbXBphsEsgKigpIG5j7y1JkMKpWXPH7/MHpoialfOTWlml5tPHVfZwsGCYEJFoJL3y93+TERoBwyJRw8GYKK9CNmXdXaPSGoTpe9C0Px9e9HYqImcz+S0bgj2GbrOj631lURFdsJmURUnYPFiXJwYey3Prfp7FAk9USmFjLM0M5Aq0EpEAPo6S01BTuIzeZm2PLXghCfWl5syODsqg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EpUyHKAieomgHAQsQGREWdJpGdjrz4gLNv1Kubsbstk=; b=U8iB6tV5e8x9LffQ9U/BggPNNDQbzcLBQsPwAWzcAPCjejBb4HkD5PomPey3t3IEyiCgSCepmAbq1DINcsSLV0JfYRB2iDiEDe8k+bllTEOZpg6PsUEIA5mxXN2fxfKKSFT5Q0bytu2u7ANMVEyq8PElDG3Rsi18yJpBtJ36Cl6HFyhYuPq/inb+vZ1B14Vam8DHbzTKUT1sz0EsJZAadJojVXqFLFV43+AIpEAV/JXVTnZyl4gR7hC855sGYVPMxCTJCkcrpWdzdiuH0kuPPYU3m98zU374nQq44qHTZj2WDc+h4OkOr6CdA3Gm4tyE9R7HhvygL0B3bPgCQT1rpA== Received: from MW4PR04CA0173.namprd04.prod.outlook.com (2603:10b6:303:85::28) by DM8PR12MB5399.namprd12.prod.outlook.com (2603:10b6:8:34::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22; Thu, 3 Nov 2022 16:17:17 +0000 Received: from CO1NAM11FT084.eop-nam11.prod.protection.outlook.com (2603:10b6:303:85:cafe::f6) by MW4PR04CA0173.outlook.office365.com (2603:10b6:303:85::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22 via Frontend Transport; Thu, 3 Nov 2022 16:17:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1NAM11FT084.mail.protection.outlook.com (10.13.174.194) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:17:17 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:17:07 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:17:07 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:17:00 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 05/17] vfio/migration: Fix wrong enum usage Date: Thu, 3 Nov 2022 18:16:08 +0200 Message-ID: <20221103161620.13120-6-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT084:EE_|DM8PR12MB5399:EE_ X-MS-Office365-Filtering-Correlation-Id: 60cbd5d3-728d-41d2-0864-08dabdb6e079 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fxwGJskxGWhC0gLTssfLZrV3nSAOY3um4VNxUxVbxDOpuQVAVN2h0TLZU2/zSGMXD1rU1Gzfl/C1sd57lctlcBAVx/o40FsZYvwlvnqUPrHBLY8PzZ9UMNJN0Xvpp5mEuECKdi1ingOEP2/4ahXSVn2DI3cJrCDQpkxwSpdDbNdpDDQfAMgCpcpwzV7/6LcC4f3py9e/CxgoP4qFxAk0+668URSaxQOASWzk2YLH1VhLKhA4IFf1caQIXWttRnndKlH2h7pTk5B5RZE8bvnNzwjPQvlbtkjiyUynbc70tcCjI+8Lz4U3BEiHLFOoG6eGIRPBT29Ot+/2dHjcseUfN9XVj6wqStGOnkoVdvr1xMMWPUZq2f0JgCntG9DDkEqui9AZFD8EGkljYuaCfJgT1C8jm/KqEEoAhg6ZbEfGYHIYKuaoSQ+jngfrFgCO+S4VxCRd/0BE/zKA7UvRNl6EnyqJB4Pi6EKVS7a69+NYsYrJ8eMHIgcugBSrzdRuAr2N7VXx/82JD9xEitPBEfGOD9v1a+sS7tJKj5gcamldbWvGWXNTsBHSd7JiTLmNS9fSrC+ktiJazXNi1teDZnIS4RvvoPrsbeRdMwPLIEyQgQxuVM7aVlLXiZ29jJYhflJaRkVmRf94MxXNTvQo9dOfBxzxy5ba14wIUzW8xXVBZue0+WmuOseOgmfEVYtGpA6TGu8BFFqEgRYcQ5OPS4RiW/1oZSWM3zMx1icOkTMZuuL8NWkTbjWOQViqGO9rHdjs/JKCOmKUgCpCdXgjISynWA== X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(376002)(396003)(136003)(346002)(39860400002)(451199015)(36840700001)(46966006)(40470700004)(36860700001)(7636003)(356005)(82310400005)(40480700001)(36756003)(86362001)(40460700003)(82740400003)(7696005)(316002)(70586007)(4326008)(8676002)(54906003)(478600001)(6916009)(1076003)(336012)(186003)(2906002)(5660300002)(2616005)(8936002)(7416002)(47076005)(4744005)(83380400001)(70206006)(426003)(41300700001)(26005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:17:17.3134 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 60cbd5d3-728d-41d2-0864-08dabdb6e079 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT084.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5399 Received-SPF: softfail client-ip=40.107.93.40; envelope-from=avihaih@nvidia.com; helo=NAM10-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org vfio_migration_init() initializes VFIOMigration->device_state using enum of VFIO migration protocol v2. Current implemented protocol is v1 so v1 enum should be used. Fix it. Fixes: 429c72800654 ("vfio/migration: Fix incorrect initialization value for parameters in VFIOMigration") Signed-off-by: Avihai Horon Reviewed-by: Zhang Chen --- hw/vfio/migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 760d5f3c5c..8ae1bd31a8 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -802,7 +802,7 @@ static int vfio_migration_init(VFIODevice *vbasedev, } vbasedev->migration = g_new0(VFIOMigration, 1); - vbasedev->migration->device_state = VFIO_DEVICE_STATE_RUNNING; + vbasedev->migration->device_state = VFIO_DEVICE_STATE_V1_RUNNING; vbasedev->migration->vm_running = runstate_is_running(); ret = vfio_region_setup(obj, vbasedev, &vbasedev->migration->region, From patchwork Thu Nov 3 16:16:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 76C68C433FE for ; Thu, 3 Nov 2022 16:19:43 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcw2-0002f4-Mb; Thu, 03 Nov 2022 12:19:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcuQ-0000mA-Lh; Thu, 03 Nov 2022 12:17:38 -0400 Received: from mail-bn7nam10on2058.outbound.protection.outlook.com ([40.107.92.58] helo=NAM10-BN7-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcuO-000857-O1; Thu, 03 Nov 2022 12:17:38 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UfqabK+HqaNEcXdYdWLyUhpd2Pb3KVATtJ+fcr+qXXrHT44M01Navl6FRE3TNIWlc5w9WlbyJ/rlhr/OEshyc7zglebZOj+bd36HuKeaq8f/2UQsdFs21xjdhfeIOoCd96INBRPwdjuFZNWVChw76JIiZPkOjt1oWMC3xBjsHHm7Hgl7ijrPjWsYVS4jh4xXO0CGkmwXs4o69Ri9KzHmQ+xy7rIJIvafW3CByCLwFse2u7BxMdRAmVFTjlApB1gg2gS+rPmEw6UYXXIQCJUIqnbZys3bq5tIBIw51mFcs3uK7jtsk/DOz1OflbbVZ6g//gL1Y/XnTf9VNdK2xgxbbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4Bjl1yHZbLc360zWMZVbBMvvvLaKTuzDAeCCfcJLGn8=; b=TGHhG3mTXIE+TN6eLpfo9JLYG5mK865ra/sga164vIhu1Mv6gmMXCeLxrZtC8T4xJfY5rXHEj42WwMjy2Gf0zmT03MYraYIb4PfWy2XKE0mw83vNmFkFj91BkUTYsLUtHnGLFj00eI0szkZYF0vVHWJjbzUNjl/+Jxkkfagn/pKEC2SnJttAGYVN19HXVSToYXo/ofEwaKvtU+JGql3BKiG2hsflij80gQihlptznFgOMFyhee6fVVwLROf2dewdTz03QYXG9VnwOk3MxoK95yx0Xt6h+9Yqdk/1wsX91XBlNocbjeqU4PhZ49I2hqopQanZ/aTTbF8KlDK7nYYoSQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4Bjl1yHZbLc360zWMZVbBMvvvLaKTuzDAeCCfcJLGn8=; b=T/xd+r1bNeCNo9iUVaV2d/UCqYOTfIocYjSbX6QrbVLB3/AILwYwUUBH2JTsW0oLf4oYQnS3OrHCmcZCNDlr35c+F273rKOAyVzAFiDJxdK93oYUzW33CQ6T5xNnfl65B2deO0zE0DAx4YNiJm3iyioTdgqCQnSdopdXhNUqibN8YcCgWZDL7N8c2vXft0rl0pbuPupnqIdyzCJFScmcdGgrayxTqknKe87wRGjWM71KX2Mygpem1QI9DL8ldUF2ciWkTNIjmj0ynp194LXDv/uSRXrEfP1gHyv43IARF6/OCPbnSX1rnJXcng75sJZNGljsupjJlHGXdG6o/BYDTw== Received: from BN9PR03CA0990.namprd03.prod.outlook.com (2603:10b6:408:109::35) by DM6PR12MB4353.namprd12.prod.outlook.com (2603:10b6:5:2a6::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22; Thu, 3 Nov 2022 16:17:30 +0000 Received: from BN8NAM11FT008.eop-nam11.prod.protection.outlook.com (2603:10b6:408:109:cafe::11) by BN9PR03CA0990.outlook.office365.com (2603:10b6:408:109::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22 via Frontend Transport; Thu, 3 Nov 2022 16:17:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by BN8NAM11FT008.mail.protection.outlook.com (10.13.177.95) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:17:30 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:17:15 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:17:14 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:17:07 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 06/17] vfio/migration: Fix NULL pointer dereference bug Date: Thu, 3 Nov 2022 18:16:09 +0200 Message-ID: <20221103161620.13120-7-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT008:EE_|DM6PR12MB4353:EE_ X-MS-Office365-Filtering-Correlation-Id: 0d020bef-70f6-4d5f-1d0c-08dabdb6e81e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dDetaVDLnCvsPUQSVeJi9E/7RbK4CPWBL5g+s/xJHNNU5DkKXX0o71mkNVPlIcLlSd6GsLSMzfZRaqc2EcVqfB47vMpB0vfRnI7R98mc8J2m32Sev1APLJAWF80AhwC6pWHNp9NOQjfvJ5wO7QsKfoQQyvOlPhX24NXdi8q26ZBuIfQD2YFc+bw3X+GfRDG4N2fpIH+oESQThfYdw6O78l/vVEAjaWhVzkHUO52zcER2kfbP9Yj322/13bpNoLkv3ZtHx53pO+1MtN1omzeY2Z7J4Y+VHeNKgxWdd8gzNFYmsq6Wm1uFwlYOVwCaGvux4rJHF60DzYIqlle8vc7hjtoexhvRGoKzq1JiLwSEP6juOVSq26CPju/sW22dhF/6R/M43s0Q/lJ1mqWhlj4tgODfpyfmDj9VJByFp7mCvmwIsahrUDp6gVjHyJKCQaYtRs/6a46sXzVzA2FBvdgz5T8jTbmcSxZxhCzuNWi32u4RSDWX4uVLRvg31ReJBgZTiH0Pj3YSG6K50g87oPSXUvHV7lO2/vsBPI3vcacinSVdtXHMOg5OG5J/EvAqVUTroVOfgzRNEF7TvVvGx2e9Xu1Ocx6MHQ7SutePav/ip0Y35JZCJ5EfQTq6ifue0pPDN/mP5lxcxDe6teEVRVLpHjiZaPkoSY10uC2xGodMUyy8VyntUlHh64AQEPVNAffmNVI80N0SQuZL3c3mNQiliR6FFI4F16EpE69uDTMjmpfKg7PQwRQ1agQpVPjktgdnFSV3aL83tgTSdbVLpESgLA== X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(346002)(136003)(376002)(396003)(39860400002)(451199015)(36840700001)(46966006)(40470700004)(336012)(8676002)(4326008)(47076005)(70586007)(70206006)(478600001)(186003)(2616005)(40460700003)(7416002)(1076003)(5660300002)(426003)(316002)(41300700001)(36860700001)(82310400005)(2906002)(6916009)(83380400001)(356005)(54906003)(82740400003)(36756003)(7636003)(8936002)(7696005)(86362001)(26005)(40480700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:17:30.0456 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0d020bef-70f6-4d5f-1d0c-08dabdb6e81e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT008.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4353 Received-SPF: softfail client-ip=40.107.92.58; envelope-from=avihaih@nvidia.com; helo=NAM10-BN7-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org As part of its error flow, vfio_vmstate_change() accesses MigrationState->to_dst_file without any checks. This can cause a NULL pointer dereference if the error flow is taken and MigrationState->to_dst_file is not set. For example, this can happen if VM is started or stopped not during migration and vfio_vmstate_change() error flow is taken, as MigrationState->to_dst_file is not set at that time. Fix it by checking that MigrationState->to_dst_file is set before using it. Fixes: 02a7e71b1e5b ("vfio: Add VM state change handler to know state of VM") Signed-off-by: Avihai Horon Reviewed-by: Juan Quintela Reviewed-by: Vladimir Sementsov-Ogievskiy --- hw/vfio/migration.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 8ae1bd31a8..f5e72c7ac1 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -740,7 +740,9 @@ static void vfio_vmstate_change(void *opaque, bool running, RunState state) */ error_report("%s: Failed to set device state 0x%x", vbasedev->name, (migration->device_state & mask) | value); - qemu_file_set_error(migrate_get_current()->to_dst_file, ret); + if (migrate_get_current()->to_dst_file) { + qemu_file_set_error(migrate_get_current()->to_dst_file, ret); + } } vbasedev->migration->vm_running = running; trace_vfio_vmstate_change(vbasedev->name, running, RunState_str(state), From patchwork Thu Nov 3 16:16:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030520 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B84E9C4332F for ; Thu, 3 Nov 2022 16:19:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcw1-0002VC-3h; Thu, 03 Nov 2022 12:19:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcuP-0000k9-5K; Thu, 03 Nov 2022 12:17:37 -0400 Received: from mail-bn8nam12on2068.outbound.protection.outlook.com ([40.107.237.68] helo=NAM12-BN8-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcuM-00084b-95; Thu, 03 Nov 2022 12:17:36 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JZ/rEKBe97sdBhFt6GG2g/pz2dOKeqmo8zk1xn2ILDAnu3xYOg11E72fq+HYNQn5zd+jS7oizrmLMYLQWjfi09mwgbZFrsG60OaoWDC/rusuZj152QSXFL3DDsedJG4Ek9Vot0/LJMZP45qwGmQ4+0QcMQs8A+mHHCLwwzaACncH6Jh3vCtw4AHBMbhedDnGcGRZLoWiS64AMRQYDvLrYyBCHmEJ12aYQokwu/oegF79KzqG+IUccCPQ0Z89CO5NEMfAkEfkc56PoRqBuR8ehk/6sABlmXm0iRs8XRCurrv1EhWa4Vc4pmLv7X82abz7Aa/Mu8Hk1N5gUie0BQX2mg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Z3kkFJovwULQFMdhS9M5tLi/WA21wjs74WNpMiNAozo=; b=YC7EFiNjCEl7EK6XTYCVlOwQfV1gSOjgkdiRdxJX6Rg2MwENH/4qF7m0Hyz2qWJkczzoumEGEKx6oY057OL+bZnncyQBPfnFF6s09tzffuDVX7LNeXItcveagHmn1mFAZThLBZm2MJGgwJRjyacvz5/TrkMcEb80Hcvvr63PnDlgqwApDC0uzznatM7cwl3swlfLRg4OOjWwOqEJUvEir5/q3GL/Hn4e9MSiaTjpvrd/qyqAbSRpsrwnAOR2vWoH8mPF2d1L0GtNfMyna2k/fauuTDpXJggOBmGdHJavBG0OECu90EqZz3jHAviXblEJi5VTTbGxa8zUKGR5N5GLDg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z3kkFJovwULQFMdhS9M5tLi/WA21wjs74WNpMiNAozo=; b=a6X6jYZk9IYbSjR0VoI7a5M8Ec90wbzv0gbL+y7uO+ggMxO8k/qS7uKmH+Zp4LsIHggE1G8VEn9bj4Kvko93xWTDLDRRgMmUqUSYbdVi1P3E1eBoKJSErsUhLTSRSgMaNGJ9maVBHBz+NeRndpp9w/UKg9LUiuvL7034lNu7Vj9xP00Vfhp+VUmKi2UfnVLUyTEyM1/9wRKQ3rnznKFhDv28KqqNN27jQi5I3xfnwa3+Ni04ZOAufQJZAPkdEQvphLL7I60/P85GAJgW0hi8Q9CpHNfY1sWdrf3kihGceH61weAmSGlVBaPwPbTzf/ieMNn2XEkWfxXFzvSN2faPOw== Received: from MW2PR16CA0029.namprd16.prod.outlook.com (2603:10b6:907::42) by SA3PR12MB7952.namprd12.prod.outlook.com (2603:10b6:806:316::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22; Thu, 3 Nov 2022 16:17:29 +0000 Received: from CO1NAM11FT065.eop-nam11.prod.protection.outlook.com (2603:10b6:907:0:cafe::d1) by MW2PR16CA0029.outlook.office365.com (2603:10b6:907::42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.19 via Frontend Transport; Thu, 3 Nov 2022 16:17:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1NAM11FT065.mail.protection.outlook.com (10.13.174.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:17:28 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:17:23 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:17:22 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:17:15 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 07/17] vfio/migration: Allow migration without VFIO IOMMU dirty tracking support Date: Thu, 3 Nov 2022 18:16:10 +0200 Message-ID: <20221103161620.13120-8-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT065:EE_|SA3PR12MB7952:EE_ X-MS-Office365-Filtering-Correlation-Id: 7d576ee0-d41b-43d7-4fa3-08dabdb6e721 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: y0i1LiCxflaUN/XrGuMZ5yk8Nweq2lVoQZ6nKhE9VJcW9vgJ8n1wtNlvRPkw+RUzoApqJY7zDcu/ShVqTxaO7NK1B4kLccTS9xbwetuhNK1hoiMuVUWrgAtHjDHcggWm8t9PYr87fYohjkD4u6TL5iavr73en15YXzaO9H0EHqQZ7efhafEcGEIgvZ1kp8X5PuN/VBCCyh5krkMROWnWniCwqpF3Lchj1JIkPn4hgNW6ej/QRnHp0j8nIKr4EkWpekme5aLr8MMBjDm/rR7IFzpamgcMdhbDwBzdiFwEAMneTH2pqGDccO2DCA1O8KmpOfQJgmYMjbLh7ibYCaVxrcj5wD2N05zkixdy03UzkpWwAD52S1aPIo9yHw86C4YLKolfMIaRIWYdZsnpablaBNNshnap0bmg8y+xB8F1OPw4eteykV7RIKEmVIeieNvrByng6mf2gKs8gtBqN+xhgaFni0+b3rmnmWFzxnMQnRF/7DinkTy1+w4SijvCH0AP6epbqulIcIjQbwakp66BAsdF5S/aY4FyO/LH4dgzyzvnR5rK6/ooCy8uEfgHrZcL+Q1Aig41RuRxRGiV0xxsy+JrvF8L1Je6F6kB7UFnpq+CAsfcyvupBomhM04twEeet4rHFoLDujUPyAhNPV7BQRVXyoNo3q2dEKoLsp3iSJ4n7W6zaYnZWAnR/rxmkuzknw5P2JnfyNkFFykhebOrk4varPMM/0KH6XxW/5zUoAGkEv0a2NRJhTQv2iar5lpaa23jLmMA08CauGiqgGqKsgnwcJjC8Vvdoq4p+wB1hom6/wb568/+Ed/4ZEYvWZharBEyHr9VefjqUwx9SjdwIiTCjTXawvB13LK6G7KN+HH3vjVovIifh3xzzvFIlPbu X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(396003)(136003)(346002)(376002)(39860400002)(451199015)(46966006)(36840700001)(40470700004)(2906002)(86362001)(70586007)(70206006)(40460700003)(26005)(7696005)(41300700001)(8676002)(6916009)(316002)(54906003)(8936002)(7416002)(4326008)(5660300002)(36860700001)(36756003)(82310400005)(966005)(478600001)(83380400001)(2616005)(426003)(47076005)(6666004)(356005)(336012)(40480700001)(7636003)(82740400003)(1076003)(186003)(14143004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:17:28.4514 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7d576ee0-d41b-43d7-4fa3-08dabdb6e721 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT065.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB7952 Received-SPF: softfail client-ip=40.107.237.68; envelope-from=avihaih@nvidia.com; helo=NAM12-BN8-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Currently, if IOMMU of a VFIO container doesn't support dirty page tracking, migration is blocked. This is because a DMA-able VFIO device can dirty RAM pages without updating QEMU about it, thus breaking the migration. However, this doesn't mean that migration can't be done at all. In such case, allow migration and let QEMU VFIO code mark the entire bitmap dirty. This guarantees that all pages that might have gotten dirty are reported back, and thus guarantees a valid migration even without VFIO IOMMU dirty tracking support. The motivation for this patch is the future introduction of iommufd [1]. iommufd will directly implement the /dev/vfio/vfio container IOCTLs by mapping them into its internal ops, allowing the usage of these IOCTLs over iommufd. However, VFIO IOMMU dirty tracking will not be supported by this VFIO compatibility API. This patch will allow migration by hosts that use the VFIO compatibility API and prevent migration regressions caused by the lack of VFIO IOMMU dirty tracking support. [1] https://lore.kernel.org/kvm/0-v2-f9436d0bde78+4bb-iommufd_jgg@nvidia.com/ Signed-off-by: Avihai Horon --- hw/vfio/common.c | 84 +++++++++++++++++++++++++++++++++++++-------- hw/vfio/migration.c | 3 +- 2 files changed, 70 insertions(+), 17 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 6b5d8c0bf6..5470dbcb04 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -392,6 +392,41 @@ static bool vfio_devices_all_running_and_saving(VFIOContainer *container) return true; } +static int vfio_dma_unmap_mark_dirty(VFIOContainer *container, hwaddr iova, + hwaddr size, IOMMUTLBEntry *iotlb) +{ + struct vfio_iommu_type1_dma_unmap unmap = { + .argsz = sizeof(unmap), + .flags = 0, + .iova = iova, + .size = size, + }; + unsigned long *bitmap; + uint64_t pages; + int ret; + + pages = REAL_HOST_PAGE_ALIGN(size) / qemu_real_host_page_size(); + bitmap = g_try_malloc0(ROUND_UP(pages, sizeof(__u64) * BITS_PER_BYTE) / + BITS_PER_BYTE); + if (!bitmap) { + return -ENOMEM; + } + + ret = ioctl(container->fd, VFIO_IOMMU_UNMAP_DMA, &unmap); + if (ret) { + error_report("VFIO_UNMAP_DMA failed : %m"); + g_free(bitmap); + return ret; + } + + bitmap_set(bitmap, 0, pages); + cpu_physical_memory_set_dirty_lebitmap(bitmap, iotlb->translated_addr, + pages); + g_free(bitmap); + + return 0; +} + static int vfio_dma_unmap_bitmap(VFIOContainer *container, hwaddr iova, ram_addr_t size, IOMMUTLBEntry *iotlb) @@ -401,6 +436,10 @@ static int vfio_dma_unmap_bitmap(VFIOContainer *container, uint64_t pages = REAL_HOST_PAGE_ALIGN(size) / qemu_real_host_page_size(); int ret; + if (!container->dirty_pages_supported) { + return vfio_dma_unmap_mark_dirty(container, iova, size, iotlb); + } + unmap = g_malloc0(sizeof(*unmap) + sizeof(*bitmap)); unmap->argsz = sizeof(*unmap) + sizeof(*bitmap); @@ -460,8 +499,7 @@ static int vfio_dma_unmap(VFIOContainer *container, .size = size, }; - if (iotlb && container->dirty_pages_supported && - vfio_devices_all_running_and_saving(container)) { + if (iotlb && vfio_devices_all_running_and_saving(container)) { return vfio_dma_unmap_bitmap(container, iova, size, iotlb); } @@ -1274,14 +1312,18 @@ static void vfio_listener_log_global_start(MemoryListener *listener) { VFIOContainer *container = container_of(listener, VFIOContainer, listener); - vfio_set_dirty_page_tracking(container, true); + if (container->dirty_pages_supported) { + vfio_set_dirty_page_tracking(container, true); + } } static void vfio_listener_log_global_stop(MemoryListener *listener) { VFIOContainer *container = container_of(listener, VFIOContainer, listener); - vfio_set_dirty_page_tracking(container, false); + if (container->dirty_pages_supported) { + vfio_set_dirty_page_tracking(container, false); + } } static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, @@ -1289,9 +1331,29 @@ static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, { struct vfio_iommu_type1_dirty_bitmap *dbitmap; struct vfio_iommu_type1_dirty_bitmap_get *range; + unsigned long *bitmap; + uint64_t bitmap_size; uint64_t pages; int ret; + pages = REAL_HOST_PAGE_ALIGN(size) / qemu_real_host_page_size(); + bitmap_size = ROUND_UP(pages, sizeof(__u64) * BITS_PER_BYTE) / + BITS_PER_BYTE; + bitmap = g_try_malloc0(bitmap_size); + if (!bitmap) { + return -ENOMEM; + } + + if (!container->dirty_pages_supported) { + bitmap_set(bitmap, 0, pages); + cpu_physical_memory_set_dirty_lebitmap(bitmap, ram_addr, pages); + trace_vfio_get_dirty_bitmap(container->fd, iova, size, bitmap_size, + ram_addr); + g_free(bitmap); + + return 0; + } + dbitmap = g_malloc0(sizeof(*dbitmap) + sizeof(*range)); dbitmap->argsz = sizeof(*dbitmap) + sizeof(*range); @@ -1306,15 +1368,8 @@ static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, * to qemu_real_host_page_size. */ range->bitmap.pgsize = qemu_real_host_page_size(); - - pages = REAL_HOST_PAGE_ALIGN(range->size) / qemu_real_host_page_size(); - range->bitmap.size = ROUND_UP(pages, sizeof(__u64) * BITS_PER_BYTE) / - BITS_PER_BYTE; - range->bitmap.data = g_try_malloc0(range->bitmap.size); - if (!range->bitmap.data) { - ret = -ENOMEM; - goto err_out; - } + range->bitmap.size = bitmap_size; + range->bitmap.data = (void *)bitmap; ret = ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, dbitmap); if (ret) { @@ -1465,8 +1520,7 @@ static void vfio_listener_log_sync(MemoryListener *listener, { VFIOContainer *container = container_of(listener, VFIOContainer, listener); - if (vfio_listener_skipped_section(section) || - !container->dirty_pages_supported) { + if (vfio_listener_skipped_section(section)) { return; } diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index f5e72c7ac1..99ffb75782 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -857,11 +857,10 @@ int64_t vfio_mig_bytes_transferred(void) int vfio_migration_probe(VFIODevice *vbasedev, Error **errp) { - VFIOContainer *container = vbasedev->group->container; struct vfio_region_info *info = NULL; int ret = -ENOTSUP; - if (!vbasedev->enable_migration || !container->dirty_pages_supported) { + if (!vbasedev->enable_migration) { goto add_blocker; } From patchwork Thu Nov 3 16:16:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7AD70C4332F for ; Thu, 3 Nov 2022 16:22:54 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcwZ-0004B1-2N; Thu, 03 Nov 2022 12:19:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcuj-0000vH-UD; Thu, 03 Nov 2022 12:18:01 -0400 Received: from mail-dm6nam10on2085.outbound.protection.outlook.com ([40.107.93.85] helo=NAM10-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcuf-00088V-Qp; Thu, 03 Nov 2022 12:17:56 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hrMNrL8wRUhZyQaVjq0PRVC2hKF3vk93ymo3hWyD9VBzhjqxvEaBCf0v1B8Zg/CkHuJRtGJ0Th+c0HGdzM3b91N5KI0ZQP4r1XSNV9iEkZGeUf9E1r+8DxyoGkNi3XoMU9GtdtQqGbXVlsf8GoB3tECGhv7yhV9+LFIDaUaQRx9SBMIeoCePEKtx7OKCmaYaGUJynj0Fsha2yQP9w3gfLl3/54ByYEo9ekexL0HmsB3O9QthB0F3bfl3JibRzEDcSgS1UvXVVPuzQQisWvKfj9R0oIzQvnR2odwkZu/UfObDK4kOyg7uO0qer/zeO1cElEwubt9NQvo5DzR3kNd4dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=z5ZDCaJEj7rGphFv5mwbs/C0kTqe7TbM9xM2F0vPEKM=; b=EWMnPQIuVdTCdBZtpxeIMGvA3VShF0C7THvZZ/UCiLmMfvBheqIKycPYCmxm9VVp1bRdSSJpqGJivifJlRowN9lup8iV8m5CgC/IUgPWpK+crYAjQYgAqvGatZ7O6JMdk6kMqJ+V2IYzYW2Rcpah8p75m4WeFLCjAhn6GSrB7nTC3j64rjxxnWARNNLX3DT5unroE7N4FAQVzYZZsAg+BK0stYtInbOjScP1MsItzGia/fm1uwIkYo2YfwhyTVimOHrlV+r4ydU/6XMkujly/RVeAS8aSvadfXDuScazc/3eyoN43Y6O+UajQ4tqOM3UK0rO94/rxBRD0ClEgHd1IA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z5ZDCaJEj7rGphFv5mwbs/C0kTqe7TbM9xM2F0vPEKM=; b=aUlR7suKMopKmYO+4QrhAWrA3HEh2+TJsldZONw+xWi77brSIm/dLLv6Dk8GcNPwG/GFS/O+BbqHdmQ0IYxUZ6Tfd53oQM5VBAEkUNXc6D4cokwPYfJ+n/Ecf3m5xrs2nL1NEblwmVwPiSPHqzleFWeQ7/eEYf5CpnJcaNayg+/6z0d43tf6Rek7i0u/0oMWWOJmiy7lf1Z7hniN3FAxbaDo+yDDUBZdjWtC2UEnFLFi4u4UIvC/hF7q+FN5iq4Hu9fZwdik6AiZeM8E5eGIMcV0B6zxuG35Hk5L83Pz5bGPo7pQvRTEa2Io5HLndPoy1KOmB/DoEffq6Lxsq94ggA== Received: from MW4PR04CA0132.namprd04.prod.outlook.com (2603:10b6:303:84::17) by DM6PR12MB4337.namprd12.prod.outlook.com (2603:10b6:5:2a9::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.21; Thu, 3 Nov 2022 16:17:46 +0000 Received: from CO1NAM11FT011.eop-nam11.prod.protection.outlook.com (2603:10b6:303:84:cafe::83) by MW4PR04CA0132.outlook.office365.com (2603:10b6:303:84::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22 via Frontend Transport; Thu, 3 Nov 2022 16:17:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1NAM11FT011.mail.protection.outlook.com (10.13.175.186) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:17:46 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:17:30 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:17:30 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:17:23 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 08/17] migration/qemu-file: Add qemu_file_get_to_fd() Date: Thu, 3 Nov 2022 18:16:11 +0200 Message-ID: <20221103161620.13120-9-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT011:EE_|DM6PR12MB4337:EE_ X-MS-Office365-Filtering-Correlation-Id: f1775c3c-19c1-4a09-59dd-08dabdb6f1ca X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dTMS6XWAUdNZa/b5aKyd9qy3xJpA26bEFz6UVhhrI3coQhdOlgbazjYsm5Qypria2KTEkYn6GV0MfN4I5sh8TyFjFXckX6+SraEtSlndQYwYf6uAEvI5zWA+2v5+XpQ02/R33KN+vtVn+UIfFffdLGBdA5AdT67qtYqoUGJVpjmyq0Tth99Yu19yCVvKiLGwcYEpUG+oyrKtO+j6KOwvlyQojEgsuAog2HnRFBa3XPdghEgt1zy8cxo09ub8CLCWryG5AzIEjmSNA90tjnVsHL2vGtGnI/cFiuBgu86hmDtOFVXsDAcQoLMQR11OBGJR9ZZ+HRUjmGiQRyGomnHL6/AwsM5Pova5CHzsg1Bo8g7NlYR8EPDveKCIMU0Mp9klLvISZmqIGXaGVEfQTmAbY8h+kl0tR0bgTtbTRVJHzGr3nBMI/acEj7rNDppJIlRUQvj2Clbwg9qLgQF+8AHNthdezS9Wg2dKuUAL+Kd/V6MCNRDY0syJO2MANMv/kKWMEcG04hmrwwdPImEoDRz8Ioa6BaB06h7xaxqFzFADDzLhteqmjd8L8J/xSrQ5uvxzTZcZD7xDiWfzrQhRMFJFRhZBkn6O1/FK7P37g6GGBtbyDLmj76Aa3jAJVmAIoDv0butKvXn/0Ihvpe5RP65ft+uHSyVOAYr8HS6Eq91hWWCrFTiR1wGyAC3mncnxG+MB43jd5pSt8L2RxaNTelkrxDYnzQgntGjE4laefvNxWoMpixgxklCHsGFnmH7PkruDI3iEVJIKgGD8gQkO1Ny2WA== X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(396003)(346002)(376002)(136003)(39860400002)(451199015)(46966006)(36840700001)(40470700004)(426003)(83380400001)(47076005)(6666004)(70586007)(478600001)(40480700001)(2906002)(356005)(7696005)(82740400003)(36756003)(7636003)(40460700003)(36860700001)(336012)(186003)(26005)(2616005)(1076003)(316002)(82310400005)(86362001)(8676002)(4326008)(70206006)(41300700001)(5660300002)(6916009)(7416002)(54906003)(8936002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:17:46.3676 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f1775c3c-19c1-4a09-59dd-08dabdb6f1ca X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT011.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4337 Received-SPF: softfail client-ip=40.107.93.85; envelope-from=avihaih@nvidia.com; helo=NAM10-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add new function qemu_file_get_to_fd() that allows reading data from QEMUFile and writing it straight into a given fd. This will be used later in VFIO migration code. Signed-off-by: Avihai Horon Reviewed-by: Vladimir Sementsov-Ogievskiy --- migration/qemu-file.c | 34 ++++++++++++++++++++++++++++++++++ migration/qemu-file.h | 1 + 2 files changed, 35 insertions(+) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 4f400c2e52..58cb0cd608 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -919,3 +919,37 @@ QIOChannel *qemu_file_get_ioc(QEMUFile *file) { return file->ioc; } + +/* + * Read size bytes from QEMUFile f and write them to fd. + */ +int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size) +{ + while (size) { + size_t pending = f->buf_size - f->buf_index; + ssize_t rc; + + if (!pending) { + rc = qemu_fill_buffer(f); + if (rc < 0) { + return rc; + } + if (rc == 0) { + return -1; + } + continue; + } + + rc = write(fd, f->buf + f->buf_index, MIN(pending, size)); + if (rc < 0) { + return rc; + } + if (rc == 0) { + return -1; + } + f->buf_index += rc; + size -= rc; + } + + return 0; +} diff --git a/migration/qemu-file.h b/migration/qemu-file.h index fa13d04d78..9d0155a2a1 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -148,6 +148,7 @@ int qemu_file_shutdown(QEMUFile *f); QEMUFile *qemu_file_get_return_path(QEMUFile *f); void qemu_fflush(QEMUFile *f); void qemu_file_set_blocking(QEMUFile *f, bool block); +int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size); void ram_control_before_iterate(QEMUFile *f, uint64_t flags); void ram_control_after_iterate(QEMUFile *f, uint64_t flags); From patchwork Thu Nov 3 16:16:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5ECB9C43219 for ; Thu, 3 Nov 2022 16:20:54 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcwa-0004RN-TY; Thu, 03 Nov 2022 12:19:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcuj-0000vI-VH; Thu, 03 Nov 2022 12:18:01 -0400 Received: from mail-dm6nam11on2062.outbound.protection.outlook.com ([40.107.223.62] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcuf-00088Y-Rv; Thu, 03 Nov 2022 12:17:56 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=L38ghwLVKP7Pb8fnqT9TbK/YZiQJnmB/zxm5QIrTvPngI8OASi+Fjbgkc3E5fX9vYOHJiE85E+HtZ+0UyMQAFw4RrhZe4pYXZdT3Tydd+BHd/LLG0mRxPjYhWOvoQX/kvfLPr5aB+l7a8KV/tR7b84vkQYOX3npbW2OLZRoaYiy+oAPGahP6w8YQQo9RRkHqtk7pGo7oDz2DdJpnrne1Yu3Mb5ZZWFAFx4x9cftOZyaUUuKgJxluMzQ7CAt08fQw07KSJRDgPeCiKGmYV7KO1JqmoP8RwafNBNBAUGOUlU9TsupHEk/vqyXVJJHMGdm+0myZJN6LZq5WlO6WlBNV3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1D7d3pOPMGY/tNHEs+LTS6neD46wPx5fhmpsdXfnFXg=; b=nlHGz062vUEwOJl8KcMnXF+FYvgW8zUVBURmPVQ/dlb0kLMYQgN+IE9U3WVOqiFJXGOTSGepb1r84+RGzrbK/y+vMVsA/2kc2I7LSPIcQZxl0IWf9DU/RU8J0+9TJ11iOvNrfOOS16R8sYflsBXaEG0niJEAV3KV1eKLyI5RvJo6xBZjuVopajiWzMt058VJKTw5eVN5UFU5OtodzBjOlCuYTB5JK/mb2vhQG9tNVIsPKtpZUVzY5px+lgLx4WmBAAFj7Mtjwlv97RUBx060+Zjn8xGc0hKI/eotWGJcIvXrHYYiC6F5di2itaCncssfK/KSAOR/ZQeE+pBZtOMRBQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1D7d3pOPMGY/tNHEs+LTS6neD46wPx5fhmpsdXfnFXg=; b=fZIL281It1dpBNeXSEApLYX+twrIY3Au23HntxM5k5xi9IFofGObArOoLPZRQ3MBmvK4sX/lv88VAF2hvTjwQgf7tmIn+Zjl++mbF4uXkAqstOhJrPtT6+zXb2F15OFP7tYIiYOfRvolYQ2wYLE/yK4L+oIRW6RaEUanTCamexa2tx6d4o2KNEHzqdvLLXN4olK5sXR3JPejsUz/ABNi1ixxR2QaoAn1+gHkrMfAB/ABRbo7+xPsr3h4UV58ApNIWGGqd9u8q0Hgg37iOECtYrSWBBOhucAXUpNYkahzjA6TMDgP9f9CE5wNCPYjsk7sXX1VsUmAvbQJGQab+85fPA== Received: from BN9PR03CA0895.namprd03.prod.outlook.com (2603:10b6:408:13c::30) by LV2PR12MB5989.namprd12.prod.outlook.com (2603:10b6:408:171::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.23; Thu, 3 Nov 2022 16:17:45 +0000 Received: from BN8NAM11FT031.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13c:cafe::e7) by BN9PR03CA0895.outlook.office365.com (2603:10b6:408:13c::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22 via Frontend Transport; Thu, 3 Nov 2022 16:17:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by BN8NAM11FT031.mail.protection.outlook.com (10.13.177.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:17:45 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:17:38 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:17:37 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:17:30 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 09/17] vfio/common: Change vfio_devices_all_running_and_saving() logic to equivalent one Date: Thu, 3 Nov 2022 18:16:12 +0200 Message-ID: <20221103161620.13120-10-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT031:EE_|LV2PR12MB5989:EE_ X-MS-Office365-Filtering-Correlation-Id: 09d7439c-9720-4e82-3b5c-08dabdb6f113 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZedL3krLN0ItulejuEM3P3xkjFVbJOpQBIehKRy9Jue2s7QFEItkDaFfLR+imkem63ax+ii6K8xBaOKTyJrwHIyBfl8lOXdxh1juKOD8e9YfbuwKU3b8XkDdaeUEEheVSeYC53ZH6gI2v/H21W4MNftpDhIcf943xtzkopj4r8fWGzsFVhLzxPcdSJUX8DfzT4T5Z3CjSWdFj24Dv481+FxtuuGjXKUOwOOelRCr56rqL8RaE3ThH8zdj4CmRtOEAsQ1nnUQMAXNssnCR4JyoaFNbsoG2FEy9GLx5vBlZ6Nikvs3DKVepIVk5KNzEp7kLWcuN/+2Fuf1uyJsSSrAjY34jdBuZQ9+MAVkHEOqsW1n791vG4t/9w4hwHFNfrl9eqYXCmRT/F/tBxbQFVLKABpZ3bCQXtHg1x1yzbhVts7QKMoSL0ieZilUY+MfPSPA5D6HlmUgR4fY6GsvZaaRLe9OkfA81qHNIyE9+LL2bdUPdhzHDthfAIn8rMAb4jjt5i4hUodeGYPapz6sNmF+o+Axmv1WUg0Wp8do//LxTE+9dydRX/rNFLyWDbCDc8rWzVQwn16AaK30lGvX9pNozsmpYqLKNqURtPwb/CcBtxF+lgaguQrSPI+ivRzpjNye+5O1yXTYCvcDmRUoo9LPOLaoM5Fs4+Lg1RSn+bBR8TJDteW7mStxlSDy1+bYwLg2oqAaML2DGLpssmfRlnUlK9PsuFMIJ7CilkqVYM1C15j6K+n2hrT4gr7We72N9JoQRwmtF4hkcTIEmjFOdEO+ow== X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(346002)(39860400002)(376002)(396003)(136003)(451199015)(40470700004)(46966006)(36840700001)(356005)(7636003)(83380400001)(26005)(8676002)(36860700001)(478600001)(86362001)(40480700001)(426003)(5660300002)(7416002)(2906002)(47076005)(1076003)(6666004)(336012)(186003)(82310400005)(7696005)(70206006)(54906003)(316002)(40460700003)(6916009)(2616005)(4326008)(41300700001)(8936002)(36756003)(70586007)(82740400003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:17:45.0738 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 09d7439c-9720-4e82-3b5c-08dabdb6f113 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT031.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5989 Received-SPF: softfail client-ip=40.107.223.62; envelope-from=avihaih@nvidia.com; helo=NAM11-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org vfio_devices_all_running_and_saving() is used to check if migration is in pre-copy phase. This is done by checking if migration is in setup or active states and if all VFIO devices are in pre-copy state, i.e. _SAVING | _RUNNING. VFIO migration v2 protocol currently doesn't support pre-copy, so it doesn't have an equivalent VFIO pre-copy state like v1 protocol. As preparation for adding v2 protocol, change vfio_devices_all_running_and_saving() logic such that it doesn't use the VFIO pre-copy state. The new equivalent logic checks if migration is in active state and if all VFIO devices are in running state [1]. No functional changes intended. [1] Note that checking if migration is in setup or active states and if all VFIO devices are in running state doesn't guarantee that we are in pre-copy phase, thus we check if migration is only in active state. Signed-off-by: Avihai Horon --- hw/vfio/common.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 5470dbcb04..47116ba668 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -40,6 +40,7 @@ #include "trace.h" #include "qapi/error.h" #include "migration/migration.h" +#include "migration/misc.h" #include "sysemu/tpm.h" VFIOGroupList vfio_group_list = @@ -363,13 +364,16 @@ static bool vfio_devices_all_dirty_tracking(VFIOContainer *container) return true; } -static bool vfio_devices_all_running_and_saving(VFIOContainer *container) +/* + * Check if all VFIO devices are running and migration is active, which is + * essentially equivalent to the migration being in pre-copy phase. + */ +static bool vfio_devices_all_running_and_mig_active(VFIOContainer *container) { VFIOGroup *group; VFIODevice *vbasedev; - MigrationState *ms = migrate_get_current(); - if (!migration_is_setup_or_active(ms->state)) { + if (!migration_is_active(migrate_get_current())) { return false; } @@ -381,8 +385,7 @@ static bool vfio_devices_all_running_and_saving(VFIOContainer *container) return false; } - if ((migration->device_state & VFIO_DEVICE_STATE_V1_SAVING) && - (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING)) { + if (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING) { continue; } else { return false; @@ -499,7 +502,7 @@ static int vfio_dma_unmap(VFIOContainer *container, .size = size, }; - if (iotlb && vfio_devices_all_running_and_saving(container)) { + if (iotlb && vfio_devices_all_running_and_mig_active(container)) { return vfio_dma_unmap_bitmap(container, iova, size, iotlb); } From patchwork Thu Nov 3 16:16:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030542 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B0E74C43219 for ; Thu, 3 Nov 2022 16:22:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcwb-0004bb-VX; Thu, 03 Nov 2022 12:19:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcun-0000vr-SO; Thu, 03 Nov 2022 12:18:04 -0400 Received: from mail-bn1nam07on2040.outbound.protection.outlook.com ([40.107.212.40] helo=NAM02-BN1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcuk-00089U-PZ; Thu, 03 Nov 2022 12:18:00 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mGboQ+GP6ZoPVW2RDkDcG4+2FB0tbtkQcfmRi9ShrajyapYHqwBUPdQDEkKd/sNPhJjbNgqvinjIp8CN9R3Vkf5ikHJUAITMGvhADxeXfhljw16g8kcj6gRmhxG9nz2z7Ui9ScXxzhuKbu4QIC6uJRMndiosWqV9eUJcQywN2Q9kmLor7ma0PV+NkGku2xTWPc9R8rLRwpb/kk69odAnMARofWf9WLv3vZkgDiHvc9SzTjaDZuMXjlt0uadGfxHwSnoOjCUuyzyPNvvMFpM9aDtiXNAI2PW4Ugaj125HbKuMbYE4dR2IN8hIYfbtzjiosqmi1YAQ/O03OOBeEaUE2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bcyCzUTe4XhO9Hun42CZqHXJkbhMwsSgb0e/nr2mE3k=; b=glpYw59bTw2B1Rx2aT5wPtxuMtNoQg9UDiHFSEGmdrzd0smW0S/EY4BZMphAx6bwd7XcHYQcQV7wJwaF68MqkoXXboCKfibW6cD53CAjJwxQup6BtGe1jQHF0Js+ITmrRpEwFSj3/BdvumpAgzRvncZqAFWc5okxuREU2qs1yrdGvhqVP6Grhkh+Xd8DBuavUFnDPcENvx7CjWT6NLF7Bc5MKu/dU2j551DJaZT3UzTAZuuJxXdUv5Rf9JqaniBxH6mRJRsMo5BoboN0rqHv65vMGk6r/qzBDWuEWDtqjF7yreps6afYxkTGkDDjpDUpmZvPSIn0Y6GBNaFa75ZbNQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bcyCzUTe4XhO9Hun42CZqHXJkbhMwsSgb0e/nr2mE3k=; b=b97wVkQtlEgetg7krEZ+pr4RCCx2qcqggXHMaG2FIkXrYwZbyw1HXJjJ7PtdYP5jLBjJExMunw0p8Y8dOCQDLZSW8bVUYvj7LMTMY6Ixy1O4t9K5SkdGcl3C3Ys+7Lwyc2n93t9JSQweuF7658V7bBwdPzDs+lAh7GnJbQbINJ9UgKxNJlHnW+CVuIqih+rs23f+DzqyulwpFv3nbaBRull9jNpO6MUfgWTQkRA9ui8nkUQhNgtc/zsx6vrV6zvuVuZcv07VB5bvwfjPXJPSQiPnyDx8ZmyEeEBOoxevz4lbg6yA+Gm+CgpDieNwWh++frBNoOuHZN4DbXo4B/kLtw== Received: from MW4PR04CA0150.namprd04.prod.outlook.com (2603:10b6:303:84::35) by PH8PR12MB7109.namprd12.prod.outlook.com (2603:10b6:510:22f::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.16; Thu, 3 Nov 2022 16:17:53 +0000 Received: from CO1NAM11FT011.eop-nam11.prod.protection.outlook.com (2603:10b6:303:84:cafe::81) by MW4PR04CA0150.outlook.office365.com (2603:10b6:303:84::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.21 via Frontend Transport; Thu, 3 Nov 2022 16:17:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1NAM11FT011.mail.protection.outlook.com (10.13.175.186) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:17:53 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:17:46 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:17:45 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:17:38 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 10/17] vfio/migration: Move migration v1 logic to vfio_migration_init() Date: Thu, 3 Nov 2022 18:16:13 +0200 Message-ID: <20221103161620.13120-11-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT011:EE_|PH8PR12MB7109:EE_ X-MS-Office365-Filtering-Correlation-Id: 8e9d69ca-5d3f-4e2a-de89-08dabdb6f5cd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: faqFH/Hrc158Pt1JxWCLAQO6mri0c7c16rnx3EqkAlLXREHALWrlCtLGrSadBkqplgByZ1C5tN3djaUL+SI7Yjc1dFttSpWhai3EEORhvibZYISLc2odv91INRzC/oeLAHxODtXq3ckxpXhzbgWnm0NC/n2+sxedYzVGmzJ5cSkWM7s5t3hi8ghW6+a8vRayOWe3Ac1UvxTQIOauuhEMo8/nr5nexlUvVlBcV4PWeSoEOuL6KFE4rbQHbCXO8U8BpxeO6JFNWs95v3LwdDh+02hwDYRj3vWTerA88jj9HyXARllXKwScn1d6Jyo6/hUDfz1rFUpZhn3BmzhSg3ZzOL0wRLagL1Ysda5ldkqQDPDahK/Q+Wi4/+Q+u3tYJzFirLER6dO+c7cOTB9d9gzrIgeNyK4nno0FrrQ8e2aCYRvpFgLrJV0lvWjpFaQnOWzASoFbUxXDitExaeohcfqhttSVx/AHdIfwVvxcSvVA0fYuHw/6PYQ407lilthP29HVUYxKPsxYpblBtr4LKroA0fd+2cZQZCAH6KPcQ+IGtnXeDCiO7ORFSd5tL6W8vNxz9jA/eT+7jmCrPAFz7o1BovzTkgJ0q8zEa0B/prDCVV81O3Qgaaapp7cdq7+dYLO/9/17pG/dBnVyE2nOHKhqwm/lY7vPn4AWyN0GrUKKNiPqWzRyUy0916GlSuucAmTDqXsNbHy+RBIvPaywqUGzXcw/UeolU0AzOHnb71dRYabvxT0zOj3hdXvwyY9nmMfmM0v7E7TTRZTYp9eyLhw2zA== X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(396003)(136003)(376002)(346002)(39860400002)(451199015)(36840700001)(40470700004)(46966006)(2906002)(336012)(83380400001)(86362001)(8676002)(36756003)(7696005)(40460700003)(4326008)(41300700001)(26005)(7636003)(82310400005)(54906003)(6666004)(82740400003)(36860700001)(6916009)(186003)(426003)(70206006)(47076005)(356005)(5660300002)(7416002)(70586007)(316002)(8936002)(2616005)(478600001)(40480700001)(1076003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:17:53.0860 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8e9d69ca-5d3f-4e2a-de89-08dabdb6f5cd X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT011.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7109 Received-SPF: softfail client-ip=40.107.212.40; envelope-from=avihaih@nvidia.com; helo=NAM02-BN1-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Move vfio_dev_get_region_info() logic from vfio_migration_probe() to vfio_migration_init(). This logic is specific to v1 protocol and moving it will make it easier to add the v2 protocol implementation later. No functional changes intended. Signed-off-by: Avihai Horon --- hw/vfio/migration.c | 30 +++++++++++++++--------------- hw/vfio/trace-events | 2 +- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 99ffb75782..0e3a950746 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -785,14 +785,14 @@ static void vfio_migration_exit(VFIODevice *vbasedev) vbasedev->migration = NULL; } -static int vfio_migration_init(VFIODevice *vbasedev, - struct vfio_region_info *info) +static int vfio_migration_init(VFIODevice *vbasedev) { int ret; Object *obj; VFIOMigration *migration; char id[256] = ""; g_autofree char *path = NULL, *oid = NULL; + struct vfio_region_info *info = NULL; if (!vbasedev->ops->vfio_get_object) { return -EINVAL; @@ -803,6 +803,14 @@ static int vfio_migration_init(VFIODevice *vbasedev, return -EINVAL; } + ret = vfio_get_dev_region_info(vbasedev, + VFIO_REGION_TYPE_MIGRATION_DEPRECATED, + VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED, + &info); + if (ret) { + return ret; + } + vbasedev->migration = g_new0(VFIOMigration, 1); vbasedev->migration->device_state = VFIO_DEVICE_STATE_V1_RUNNING; vbasedev->migration->vm_running = runstate_is_running(); @@ -822,6 +830,8 @@ static int vfio_migration_init(VFIODevice *vbasedev, goto err; } + g_free(info); + migration = vbasedev->migration; migration->vbasedev = vbasedev; @@ -844,6 +854,7 @@ static int vfio_migration_init(VFIODevice *vbasedev, return 0; err: + g_free(info); vfio_migration_exit(vbasedev); return ret; } @@ -857,34 +868,23 @@ int64_t vfio_mig_bytes_transferred(void) int vfio_migration_probe(VFIODevice *vbasedev, Error **errp) { - struct vfio_region_info *info = NULL; int ret = -ENOTSUP; if (!vbasedev->enable_migration) { goto add_blocker; } - ret = vfio_get_dev_region_info(vbasedev, - VFIO_REGION_TYPE_MIGRATION_DEPRECATED, - VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED, - &info); + ret = vfio_migration_init(vbasedev); if (ret) { goto add_blocker; } - ret = vfio_migration_init(vbasedev, info); - if (ret) { - goto add_blocker; - } - - trace_vfio_migration_probe(vbasedev->name, info->index); - g_free(info); + trace_vfio_migration_probe(vbasedev->name); return 0; add_blocker: error_setg(&vbasedev->migration_blocker, "VFIO device doesn't support migration"); - g_free(info); ret = migrate_add_blocker(vbasedev->migration_blocker, errp); if (ret < 0) { diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index a21cbd2a56..27c059f96e 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -148,7 +148,7 @@ vfio_display_edid_update(uint32_t prefx, uint32_t prefy) "%ux%u" vfio_display_edid_write_error(void) "" # migration.c -vfio_migration_probe(const char *name, uint32_t index) " (%s) Region %d" +vfio_migration_probe(const char *name) " (%s)" vfio_migration_set_state(const char *name, uint32_t state) " (%s) state %d" vfio_vmstate_change(const char *name, int running, const char *reason, uint32_t dev_state) " (%s) running %d reason %s device state %d" vfio_migration_state_notifier(const char *name, const char *state) " (%s) state %s" From patchwork Thu Nov 3 16:16:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 45EE5C433FE for ; Thu, 3 Nov 2022 16:22:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcxK-0006sD-Ms; Thu, 03 Nov 2022 12:20:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcv0-00015O-FO; Thu, 03 Nov 2022 12:18:17 -0400 Received: from mail-bn8nam12on2050.outbound.protection.outlook.com ([40.107.237.50] helo=NAM12-BN8-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcux-0008AY-UZ; Thu, 03 Nov 2022 12:18:14 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SOns+zwAw2cIAPL8koFrK5BhXIwpxHHT1yaH716s7xcTPn6JNyLPV6a8JEv9V7nwet1SUQM1DupQMJ+2Nksbh3sVjZ9MD+SMIlz6gXI/ReFJxRYBeVcElFt35SXToGQAZcYkwbyb7Wh6HlwK7JYuoUPYkFoviZoFxVPhPlhafZYgVOYzlwGifvrQy7pzAoJlFxHWkumzhRrNdVM1aB17Pjn/5E3eBCAYeFARs0lXiJdbjSBVWpKdTI5Fdbm68jdUaFi1E+OkCo0Md1L6qSrFY0atAopaJmL53sTF8o4IshHyARvOXryLeffw3mJovUHF0ipGkrq1U/eaUv3mTDLdyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=O2UZ9DEYNHlVw5/TWO2BqRFJRzL1v0+aRJv4nqEiOnU=; b=BGS+upuGVOFiAEvN25YwtunRVc0VTzHRa5cT/JHcWV6u1dC/xk1v3aBerFCAgvvkjeacnl3idX7fKBT8LzwSxomx6ww3bBEXWIqnOL0dA2aoFbZbLTCWEF+W9pFzGdInz850fq76Yl/eSYyZSmPDWihbbyE2dstxWKl+H4Nx8+6kmIIMyAD8OMSLiTbixM2oQzSXT8n3R7XgWgmT2zDsyIKas5C44XjGR7H6KPezQHgjAHMx/Vmk/kEtN6d3pzqoBd9PKIMZwQ26MQlWgCng+OxVnf01G+o+P1v6Vob4OeCxr3QqInQ8YRaXDcrEReTX27My6G+F5DzPCkX2dnoXHA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O2UZ9DEYNHlVw5/TWO2BqRFJRzL1v0+aRJv4nqEiOnU=; b=fm8hfR1f5MNvDmGpHdF4oJcepBJuLGl4rO9kWzL5jBUeyECdXSFbRik4o14QZleuCST5iv58WYsDy0Wri8M0PLeN1w7KAXdg3Lha1DGu0w6eslncis6MCtlOIzHbTuXqOFnyzy45Ncns1MFoHDGcwOUwue75ciA/lys+ynynpr05+veykbqYIKfq7MDRMHVdsvX5hFce9MQgRDiDI1579tETIjXAnf2nLl6Pj4h1zUvIawTuZUVuR2R3gTNq1J82y4B2m77Y7gknzGb1S/T9uozfOYajquJwlDI4u0h0Jnnsprm0OAHfLF4VjIQVn8tlIvOMm996IIVQT+6+gxJJsw== Received: from BN0PR04CA0190.namprd04.prod.outlook.com (2603:10b6:408:e9::15) by DS7PR12MB5765.namprd12.prod.outlook.com (2603:10b6:8:74::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.16; Thu, 3 Nov 2022 16:18:06 +0000 Received: from BN8NAM11FT067.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e9:cafe::42) by BN0PR04CA0190.outlook.office365.com (2603:10b6:408:e9::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22 via Frontend Transport; Thu, 3 Nov 2022 16:18:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by BN8NAM11FT067.mail.protection.outlook.com (10.13.177.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:18:05 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:17:54 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:17:53 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:17:46 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 11/17] vfio/migration: Rename functions/structs related to v1 protocol Date: Thu, 3 Nov 2022 18:16:14 +0200 Message-ID: <20221103161620.13120-12-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT067:EE_|DS7PR12MB5765:EE_ X-MS-Office365-Filtering-Correlation-Id: 1b8347bb-acfa-43ea-a536-08dabdb6fd8b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cYzXhJxgRhMkOTulUqfzMCUGMGSqmnqiA6kW5X1CuxcXweMxuhxIzP3DbWqxsZQF9QB3G/nlKyXHKJXGMnWerRgPfi/oGeCIbXMrLvJcPQseot+ykN1y/TjTcv5rHs0X2r5+NuVTrtNR/HFvvKDdOWBmZ4gwTJ3Oi7UqlQC7FdNVDuAB3eulq1kLS698aehxyPUCqpwcuMdY7PHElSHMS2ZYpWa/74UmEBQFbVBo2DT+5XiU/ulKXsBZxE+B/0A47iHd8gN50lw0NtpK2kQw+hnrmaEiX24ezwF5qVYnzqP56PIDaJp8KYwMLoW4kmdWu6VsmC2KVXsNmxM155Pjf0dXRx3QKopf3oWL5IbApANcvljNCufQt+HI2/j+4nq/LdqBv1BVYYOYm8n42Izl0tCxcf/C+wc5vVoG89xhp6Gnt/dc5NT7bZ0r9DxpvThtfvLIhOeV8m/0KAToukXWGIk2J2BBY1WgHATeofOwBCA6+ReHGXMijjPniANUKPlKArHjpyAFzK4/QFlJ0q6NszTaS7V3XwxrtR+l5XsOYal3O8yMSGbNU4o3ZzMxtSsJUodZKM3lFaXtRomNUyiOuetWjDaTJktDnvD4nkUGfjjanjpcIzF4qRbKtR45nCC27qNhWy6BmMWL9DPN5PfbGOZ+LtSuNR547zssyGFinm7ix049Lr8voY5w5Yl26KsRm/BKv5IjxUm5cZavv4RwhggrKi5f9GrnJwIRY8ERTCxSHVVmylloLbjKRnuRtKjfrfn/zG/ye5v+RdhH0xXz/Q== X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(376002)(136003)(396003)(39860400002)(346002)(451199015)(46966006)(40470700004)(36840700001)(8936002)(6916009)(8676002)(83380400001)(6666004)(40460700003)(186003)(36756003)(70586007)(30864003)(70206006)(316002)(5660300002)(4326008)(41300700001)(2906002)(336012)(2616005)(40480700001)(7636003)(7696005)(36860700001)(356005)(1076003)(47076005)(7416002)(82740400003)(478600001)(82310400005)(86362001)(54906003)(426003)(26005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:18:05.9800 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1b8347bb-acfa-43ea-a536-08dabdb6fd8b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT067.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB5765 Received-SPF: softfail client-ip=40.107.237.50; envelope-from=avihaih@nvidia.com; helo=NAM12-BN8-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org To avoid name collisions, rename functions and structs related to VFIO migration protocol v1. This will allow the two protocols to co-exist when v2 protocol is added, until v1 is removed. No functional changes intended. Signed-off-by: Avihai Horon --- hw/vfio/common.c | 6 +-- hw/vfio/migration.c | 94 +++++++++++++++++------------------ hw/vfio/trace-events | 6 +-- include/hw/vfio/vfio-common.h | 2 +- 4 files changed, 54 insertions(+), 54 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 47116ba668..617e6cd901 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -355,8 +355,8 @@ static bool vfio_devices_all_dirty_tracking(VFIOContainer *container) return false; } - if ((vbasedev->pre_copy_dirty_page_tracking == ON_OFF_AUTO_OFF) - && (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING)) { + if ((vbasedev->pre_copy_dirty_page_tracking == ON_OFF_AUTO_OFF) && + (migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING)) { return false; } } @@ -385,7 +385,7 @@ static bool vfio_devices_all_running_and_mig_active(VFIOContainer *container) return false; } - if (migration->device_state & VFIO_DEVICE_STATE_V1_RUNNING) { + if (migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING) { continue; } else { return false; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 0e3a950746..e784374453 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -107,8 +107,8 @@ static int vfio_mig_rw(VFIODevice *vbasedev, __u8 *buf, size_t count, * an error is returned. */ -static int vfio_migration_set_state(VFIODevice *vbasedev, uint32_t mask, - uint32_t value) +static int vfio_migration_v1_set_state(VFIODevice *vbasedev, uint32_t mask, + uint32_t value) { VFIOMigration *migration = vbasedev->migration; VFIORegion *region = &migration->region; @@ -145,8 +145,8 @@ static int vfio_migration_set_state(VFIODevice *vbasedev, uint32_t mask, return ret; } - migration->device_state = device_state; - trace_vfio_migration_set_state(vbasedev->name, device_state); + migration->device_state_v1 = device_state; + trace_vfio_migration_v1_set_state(vbasedev->name, device_state); return 0; } @@ -260,8 +260,8 @@ static int vfio_save_buffer(QEMUFile *f, VFIODevice *vbasedev, uint64_t *size) return ret; } -static int vfio_load_buffer(QEMUFile *f, VFIODevice *vbasedev, - uint64_t data_size) +static int vfio_v1_load_buffer(QEMUFile *f, VFIODevice *vbasedev, + uint64_t data_size) { VFIORegion *region = &vbasedev->migration->region; uint64_t data_offset = 0, size, report_size; @@ -288,7 +288,7 @@ static int vfio_load_buffer(QEMUFile *f, VFIODevice *vbasedev, data_size = 0; } - trace_vfio_load_state_device_data(vbasedev->name, data_offset, size); + trace_vfio_v1_load_state_device_data(vbasedev->name, data_offset, size); while (size) { void *buf; @@ -394,7 +394,7 @@ static int vfio_load_device_config_state(QEMUFile *f, void *opaque) return qemu_file_get_error(f); } -static void vfio_migration_cleanup(VFIODevice *vbasedev) +static void vfio_migration_v1_cleanup(VFIODevice *vbasedev) { VFIOMigration *migration = vbasedev->migration; @@ -405,7 +405,7 @@ static void vfio_migration_cleanup(VFIODevice *vbasedev) /* ---------------------------------------------------------------------- */ -static int vfio_save_setup(QEMUFile *f, void *opaque) +static int vfio_v1_save_setup(QEMUFile *f, void *opaque) { VFIODevice *vbasedev = opaque; VFIOMigration *migration = vbasedev->migration; @@ -431,8 +431,8 @@ static int vfio_save_setup(QEMUFile *f, void *opaque) } } - ret = vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_MASK, - VFIO_DEVICE_STATE_V1_SAVING); + ret = vfio_migration_v1_set_state(vbasedev, VFIO_DEVICE_STATE_MASK, + VFIO_DEVICE_STATE_V1_SAVING); if (ret) { error_report("%s: Failed to set state SAVING", vbasedev->name); return ret; @@ -448,16 +448,16 @@ static int vfio_save_setup(QEMUFile *f, void *opaque) return 0; } -static void vfio_save_cleanup(void *opaque) +static void vfio_v1_save_cleanup(void *opaque) { VFIODevice *vbasedev = opaque; - vfio_migration_cleanup(vbasedev); + vfio_migration_v1_cleanup(vbasedev); trace_vfio_save_cleanup(vbasedev->name); } -static void vfio_save_pending(void *opaque, uint64_t threshold_size, - uint64_t *res_precopy, uint64_t *res_postcopy) +static void vfio_v1_save_pending(void *opaque, uint64_t threshold_size, + uint64_t *res_precopy, uint64_t *res_postcopy) { VFIODevice *vbasedev = opaque; VFIOMigration *migration = vbasedev->migration; @@ -520,15 +520,15 @@ static int vfio_save_iterate(QEMUFile *f, void *opaque) return 0; } -static int vfio_save_complete_precopy(QEMUFile *f, void *opaque) +static int vfio_v1_save_complete_precopy(QEMUFile *f, void *opaque) { VFIODevice *vbasedev = opaque; VFIOMigration *migration = vbasedev->migration; uint64_t data_size; int ret; - ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_RUNNING, - VFIO_DEVICE_STATE_V1_SAVING); + ret = vfio_migration_v1_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_RUNNING, + VFIO_DEVICE_STATE_V1_SAVING); if (ret) { error_report("%s: Failed to set state STOP and SAVING", vbasedev->name); @@ -565,7 +565,8 @@ static int vfio_save_complete_precopy(QEMUFile *f, void *opaque) return ret; } - ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_SAVING, 0); + ret = vfio_migration_v1_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_SAVING, + 0); if (ret) { error_report("%s: Failed to set state STOPPED", vbasedev->name); return ret; @@ -588,7 +589,7 @@ static void vfio_save_state(QEMUFile *f, void *opaque) } } -static int vfio_load_setup(QEMUFile *f, void *opaque) +static int vfio_v1_load_setup(QEMUFile *f, void *opaque) { VFIODevice *vbasedev = opaque; VFIOMigration *migration = vbasedev->migration; @@ -604,8 +605,8 @@ static int vfio_load_setup(QEMUFile *f, void *opaque) } } - ret = vfio_migration_set_state(vbasedev, ~VFIO_DEVICE_STATE_MASK, - VFIO_DEVICE_STATE_V1_RESUMING); + ret = vfio_migration_v1_set_state(vbasedev, ~VFIO_DEVICE_STATE_MASK, + VFIO_DEVICE_STATE_V1_RESUMING); if (ret) { error_report("%s: Failed to set state RESUMING", vbasedev->name); if (migration->region.mmaps) { @@ -615,11 +616,11 @@ static int vfio_load_setup(QEMUFile *f, void *opaque) return ret; } -static int vfio_load_cleanup(void *opaque) +static int vfio_v1_load_cleanup(void *opaque) { VFIODevice *vbasedev = opaque; - vfio_migration_cleanup(vbasedev); + vfio_migration_v1_cleanup(vbasedev); trace_vfio_load_cleanup(vbasedev->name); return 0; } @@ -657,7 +658,7 @@ static int vfio_load_state(QEMUFile *f, void *opaque, int version_id) uint64_t data_size = qemu_get_be64(f); if (data_size) { - ret = vfio_load_buffer(f, vbasedev, data_size); + ret = vfio_v1_load_buffer(f, vbasedev, data_size); if (ret < 0) { return ret; } @@ -678,21 +679,21 @@ static int vfio_load_state(QEMUFile *f, void *opaque, int version_id) return ret; } -static SaveVMHandlers savevm_vfio_handlers = { - .save_setup = vfio_save_setup, - .save_cleanup = vfio_save_cleanup, - .save_live_pending = vfio_save_pending, +static SaveVMHandlers savevm_vfio_v1_handlers = { + .save_setup = vfio_v1_save_setup, + .save_cleanup = vfio_v1_save_cleanup, + .save_live_pending = vfio_v1_save_pending, .save_live_iterate = vfio_save_iterate, - .save_live_complete_precopy = vfio_save_complete_precopy, + .save_live_complete_precopy = vfio_v1_save_complete_precopy, .save_state = vfio_save_state, - .load_setup = vfio_load_setup, - .load_cleanup = vfio_load_cleanup, + .load_setup = vfio_v1_load_setup, + .load_cleanup = vfio_v1_load_cleanup, .load_state = vfio_load_state, }; /* ---------------------------------------------------------------------- */ -static void vfio_vmstate_change(void *opaque, bool running, RunState state) +static void vfio_v1_vmstate_change(void *opaque, bool running, RunState state) { VFIODevice *vbasedev = opaque; VFIOMigration *migration = vbasedev->migration; @@ -732,21 +733,21 @@ static void vfio_vmstate_change(void *opaque, bool running, RunState state) } } - ret = vfio_migration_set_state(vbasedev, mask, value); + ret = vfio_migration_v1_set_state(vbasedev, mask, value); if (ret) { /* * Migration should be aborted in this case, but vm_state_notify() * currently does not support reporting failures. */ error_report("%s: Failed to set device state 0x%x", vbasedev->name, - (migration->device_state & mask) | value); + (migration->device_state_v1 & mask) | value); if (migrate_get_current()->to_dst_file) { qemu_file_set_error(migrate_get_current()->to_dst_file, ret); } } vbasedev->migration->vm_running = running; - trace_vfio_vmstate_change(vbasedev->name, running, RunState_str(state), - (migration->device_state & mask) | value); + trace_vfio_v1_vmstate_change(vbasedev->name, running, RunState_str(state), + (migration->device_state_v1 & mask) | value); } static void vfio_migration_state_notifier(Notifier *notifier, void *data) @@ -765,10 +766,10 @@ static void vfio_migration_state_notifier(Notifier *notifier, void *data) case MIGRATION_STATUS_CANCELLED: case MIGRATION_STATUS_FAILED: bytes_transferred = 0; - ret = vfio_migration_set_state(vbasedev, - ~(VFIO_DEVICE_STATE_V1_SAVING | - VFIO_DEVICE_STATE_V1_RESUMING), - VFIO_DEVICE_STATE_V1_RUNNING); + ret = vfio_migration_v1_set_state(vbasedev, + ~(VFIO_DEVICE_STATE_V1_SAVING | + VFIO_DEVICE_STATE_V1_RESUMING), + VFIO_DEVICE_STATE_V1_RUNNING); if (ret) { error_report("%s: Failed to set state RUNNING", vbasedev->name); } @@ -812,7 +813,7 @@ static int vfio_migration_init(VFIODevice *vbasedev) } vbasedev->migration = g_new0(VFIOMigration, 1); - vbasedev->migration->device_state = VFIO_DEVICE_STATE_V1_RUNNING; + vbasedev->migration->device_state_v1 = VFIO_DEVICE_STATE_V1_RUNNING; vbasedev->migration->vm_running = runstate_is_running(); ret = vfio_region_setup(obj, vbasedev, &vbasedev->migration->region, @@ -843,12 +844,11 @@ static int vfio_migration_init(VFIODevice *vbasedev) } strpadcpy(id, sizeof(id), path, '\0'); - register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, &savevm_vfio_handlers, - vbasedev); + register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, + &savevm_vfio_v1_handlers, vbasedev); - migration->vm_state = qdev_add_vm_change_state_handler(vbasedev->dev, - vfio_vmstate_change, - vbasedev); + migration->vm_state = qdev_add_vm_change_state_handler( + vbasedev->dev, vfio_v1_vmstate_change, vbasedev); migration->migration_state.notify = vfio_migration_state_notifier; add_migration_state_change_notifier(&migration->migration_state); return 0; diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 27c059f96e..d88d2b4053 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -149,8 +149,8 @@ vfio_display_edid_write_error(void) "" # migration.c vfio_migration_probe(const char *name) " (%s)" -vfio_migration_set_state(const char *name, uint32_t state) " (%s) state %d" -vfio_vmstate_change(const char *name, int running, const char *reason, uint32_t dev_state) " (%s) running %d reason %s device state %d" +vfio_migration_v1_set_state(const char *name, uint32_t state) " (%s) state %d" +vfio_v1_vmstate_change(const char *name, int running, const char *reason, uint32_t dev_state) " (%s) running %d reason %s device state %d" vfio_migration_state_notifier(const char *name, const char *state) " (%s) state %s" vfio_save_setup(const char *name) " (%s)" vfio_save_cleanup(const char *name) " (%s)" @@ -162,7 +162,7 @@ vfio_save_iterate(const char *name, int data_size) " (%s) data_size %d" vfio_save_complete_precopy(const char *name) " (%s)" vfio_load_device_config_state(const char *name) " (%s)" vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 -vfio_load_state_device_data(const char *name, uint64_t data_offset, uint64_t data_size) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64 +vfio_v1_load_state_device_data(const char *name, uint64_t data_offset, uint64_t data_size) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64 vfio_load_cleanup(const char *name) " (%s)" vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitmap_size, uint64_t start) "container fd=%d, iova=0x%"PRIx64" size= 0x%"PRIx64" bitmap_size=0x%"PRIx64" start=0x%"PRIx64 vfio_iommu_map_dirty_notify(uint64_t iova_start, uint64_t iova_end) "iommu dirty @ 0x%"PRIx64" - 0x%"PRIx64 diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index e573f5a9f1..bbaf72ba00 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -62,7 +62,7 @@ typedef struct VFIOMigration { struct VFIODevice *vbasedev; VMChangeStateEntry *vm_state; VFIORegion region; - uint32_t device_state; + uint32_t device_state_v1; int vm_running; Notifier migration_state; uint64_t pending_bytes; From patchwork Thu Nov 3 16:16:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 58C0AC4332F for ; Thu, 3 Nov 2022 16:22:20 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcxU-0007JI-K0; Thu, 03 Nov 2022 12:20:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcvY-0001Ti-8i; Thu, 03 Nov 2022 12:18:49 -0400 Received: from mail-mw2nam12on2056.outbound.protection.outlook.com ([40.107.244.56] helo=NAM12-MW2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcv8-0008Ey-VM; Thu, 03 Nov 2022 12:18:25 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Xz5HRbhi6M6edniGUov01emIqRRm/J8Sx6Pa8LKyRVIMF+LdpYCyxcDeSXAlcyILQWjMZg3vr4Lwbfoh5drCqZJVaUAJ+ufL2slTCk+atSYe8k2Y2oRGvY9ds/BXhOKW76jkDkvPw3Uc5uKosf+aFcp0Jo7stkR5t1d9OAZaak3xZrsnS1EpZBn+gn+3VAnT8929jPQeuFH8jUeCj1shdQZRzNnxgVI7JbY/wsUJBkYbnXLycMXl6bB7DcM808/uWQ/YaEcCY6Cs3jzIhHEVkiaHXXR9wNLn10jN6Eo2G2fyi+9ZpnWBBxho4cUaLiGenNSdoABmwrFyKdpGS1/UcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NTl2x7sds8wk1P8U29wu7CrcwKwGwvaTAOiH0OtBdOk=; b=kWb02mIeZC8Yy23vMkz7E7sYXebXR2Ju/eGKXTOExppKUImiJsMV+4dhaISnE/YcXPOiIz/ZijboyralJa9csDony3IOg2aMh12GjwPp/TSz2TSTdb6mjSuT2vLcQYWCMZ0tfxWwOStuxelF/FNqv7Bol4C8UZicNUbSzTAsZHPn7/hFO/diT8QuCJ1qIHB35ZbT1WixDMaK4/tVQENv4ooB9oOt7SCWTLDEWpChShuQH1sAxnYgfaVQzXfFrZhrWvQ/oKek03koWhTkwVN7ZPQMAH2yRUnXEpwVHVDWKbcQ2GZM6bUJwOrq8TdctmUVdDeTyuVdm0wM1rn4GNJClw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NTl2x7sds8wk1P8U29wu7CrcwKwGwvaTAOiH0OtBdOk=; b=aEXsA3xq/+pShoBA28T77GBO0v74PS07d1CgLoRvSqNR9b4NFUxRKKBJlT5gOJaaz1T4aVhdDCsMiIZiglvOD4NO/Z9ZTX3H6oyGy4fcBx1+SBMz5r5tmeXIoLoAFjvFvhK3xndnFj7sUC6JGUTrXP7vyKIpYavggKGTkW39HpUit7E6qxg8ZHK0FY/2toRzf7RK7xEPncgT4YF0FxVe4c3wnXXstnBxbL7PxW9nvf/G4ZzndvhBbhob+vTyIfwSvkA98UIo5T9vq3S8a1TwHk6QYTW77x1TRNaipMmukT8+FYG6DAtZlk0n8jA1R/ZneeaBMFaeei42zJ7Q8omEIg== Received: from BN0PR04CA0184.namprd04.prod.outlook.com (2603:10b6:408:e9::9) by BY5PR12MB5000.namprd12.prod.outlook.com (2603:10b6:a03:1d7::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22; Thu, 3 Nov 2022 16:18:15 +0000 Received: from BN8NAM11FT067.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e9:cafe::30) by BN0PR04CA0184.outlook.office365.com (2603:10b6:408:e9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22 via Frontend Transport; Thu, 3 Nov 2022 16:18:14 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by BN8NAM11FT067.mail.protection.outlook.com (10.13.177.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:18:14 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:18:01 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:18:01 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:17:54 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 12/17] vfio/migration: Implement VFIO migration protocol v2 Date: Thu, 3 Nov 2022 18:16:15 +0200 Message-ID: <20221103161620.13120-13-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT067:EE_|BY5PR12MB5000:EE_ X-MS-Office365-Filtering-Correlation-Id: dbae212a-a208-4f5c-5ef1-08dabdb702b4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kV0M1wK1AlS6ORw0qlyYrph/OGaX68grmmMC8MQdZ99m7pzlLPMcdS47gtnH5TT0rgiCF2aFfodtefQZxt37/SMHcEELUAJhTqOHl5m1zYSlNAev7mYMPYQ9om1ZPXNBm1deer7oqfwgQGQrKW8xZ6fX/y53WZg+IRnSwaRS/ni6EPekP+JDQNwtPT1+5eXYerlqkx/4SsczDxp9Rq74GKVuZQj1Rr6AKdK+OMoEzgZ30oNBjicvcxA+GJOOvsnKH9e5ufnK60zLl17WNpN5ds1prMxTavh9MrIl0UJCqiTUnPLfIxKkaXg63FZQPKZRSDu5c56rknr+P/IhiZEqx/6wT1DXKnovlljtPgDNn13etmuTPdvYOUJjUDwIPH9thRbbhxCGzx/UXiCrGqf+2yRveIe3Mg2qe076lv/GG1uT6wtiBKyIWWrr22bo1UZPjIBNuQO3U51BYrxbr78qan8lSgnwmr0B7V+/OEP+9JlWUtZp7IrPlalA9XUOWf0jdlMlBQWcgvnZWMhLsK0hb2pjOoEavjxMD0zcokWK6BJap8S2iQeVuj4xVUfez79MuGSIfrbfRAUXBzXkLTbIzxoxZkIT8dGaOzpdIhFruDvM3MPzxCBRkHTpuCOiJGJGa4sG2mpeNJ9wgIiL8++6vJuS2d42N+3d3Jcl4Lsr0BR8NtaG97zw/MT9kXPDSnG3QHdVTzty7n0x9AACP4cSXqN+Pw9vAfPfgplfT4A65bq4QmPp1uJfr7lhC19CiNKlsvV0+WU/B+vly533qYktX0exzMoqi1Ats18vdbBa5hOCegIERbUT2hvFQ0wyLafxBcWwYRtMed0D2lwgh7mu1IZkOKueEAk9XlReqLSNmQc= X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(376002)(346002)(136003)(39860400002)(396003)(451199015)(36840700001)(46966006)(40470700004)(336012)(478600001)(47076005)(426003)(30864003)(4326008)(70206006)(966005)(70586007)(8676002)(6666004)(2616005)(40460700003)(186003)(7416002)(1076003)(5660300002)(316002)(54906003)(36860700001)(6916009)(2906002)(82310400005)(41300700001)(7636003)(356005)(83380400001)(82740400003)(36756003)(7696005)(8936002)(86362001)(40480700001)(26005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:18:14.6511 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dbae212a-a208-4f5c-5ef1-08dabdb702b4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT067.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB5000 Received-SPF: softfail client-ip=40.107.244.56; envelope-from=avihaih@nvidia.com; helo=NAM12-MW2-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add implementation of VFIO migration protocol v2. The two protocols, v1 and v2, will co-exist and in next patch v1 protocol will be removed. There are several main differences between v1 and v2 protocols: - VFIO device state is now represented as a finite state machine instead of a bitmap. - Migration interface with kernel is now done using VFIO_DEVICE_FEATURE ioctl and normal read() and write() instead of the migration region. - VFIO migration protocol v2 currently doesn't support the pre-copy phase of migration. Detailed information about VFIO migration protocol v2 and difference compared to v1 can be found here [1]. [1] https://lore.kernel.org/all/20220224142024.147653-10-yishaih@nvidia.com/ Signed-off-by: Avihai Horon --- hw/vfio/common.c | 19 +- hw/vfio/migration.c | 386 ++++++++++++++++++++++++++++++---- hw/vfio/trace-events | 4 + include/hw/vfio/vfio-common.h | 5 + 4 files changed, 375 insertions(+), 39 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 617e6cd901..0bdbd1586b 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -355,10 +355,18 @@ static bool vfio_devices_all_dirty_tracking(VFIOContainer *container) return false; } - if ((vbasedev->pre_copy_dirty_page_tracking == ON_OFF_AUTO_OFF) && + if (!migration->v2 && + (vbasedev->pre_copy_dirty_page_tracking == ON_OFF_AUTO_OFF) && (migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING)) { return false; } + + if (migration->v2 && + (vbasedev->pre_copy_dirty_page_tracking == ON_OFF_AUTO_OFF) && + (migration->device_state == VFIO_DEVICE_STATE_RUNNING || + migration->device_state == VFIO_DEVICE_STATE_RUNNING_P2P)) { + return false; + } } } return true; @@ -385,7 +393,14 @@ static bool vfio_devices_all_running_and_mig_active(VFIOContainer *container) return false; } - if (migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING) { + if (!migration->v2 && + migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING) { + continue; + } + + if (migration->v2 && + (migration->device_state == VFIO_DEVICE_STATE_RUNNING || + migration->device_state == VFIO_DEVICE_STATE_RUNNING_P2P)) { continue; } else { return false; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index e784374453..62afc23a8c 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -44,8 +44,84 @@ #define VFIO_MIG_FLAG_DEV_SETUP_STATE (0xffffffffef100003ULL) #define VFIO_MIG_FLAG_DEV_DATA_STATE (0xffffffffef100004ULL) +#define VFIO_MIG_DATA_BUFFER_SIZE (1024 * 1024) + static int64_t bytes_transferred; +static const char *mig_state_to_str(enum vfio_device_mig_state state) +{ + switch (state) { + case VFIO_DEVICE_STATE_ERROR: + return "ERROR"; + case VFIO_DEVICE_STATE_STOP: + return "STOP"; + case VFIO_DEVICE_STATE_RUNNING: + return "RUNNING"; + case VFIO_DEVICE_STATE_STOP_COPY: + return "STOP_COPY"; + case VFIO_DEVICE_STATE_RESUMING: + return "RESUMING"; + case VFIO_DEVICE_STATE_RUNNING_P2P: + return "RUNNING_P2P"; + default: + return "UNKNOWN STATE"; + } +} + +static int vfio_migration_set_state(VFIODevice *vbasedev, + enum vfio_device_mig_state new_state, + enum vfio_device_mig_state recover_state) +{ + VFIOMigration *migration = vbasedev->migration; + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_mig_state), + sizeof(uint64_t))] = {}; + struct vfio_device_feature *feature = (void *)buf; + struct vfio_device_feature_mig_state *mig_state = (void *)feature->data; + + feature->argsz = sizeof(buf); + feature->flags = + VFIO_DEVICE_FEATURE_SET | VFIO_DEVICE_FEATURE_MIG_DEVICE_STATE; + mig_state->device_state = new_state; + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + /* Try to set the device in some good state */ + error_report( + "%s: Failed setting device state to %s, err: %s. Setting device in recover state %s", + vbasedev->name, mig_state_to_str(new_state), + strerror(errno), mig_state_to_str(recover_state)); + + mig_state->device_state = recover_state; + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + hw_error("%s: Failed setting device in recover state, err: %s", + vbasedev->name, strerror(errno)); + } + + migration->device_state = recover_state; + + return -1; + } + + if (mig_state->data_fd != -1) { + if (migration->data_fd != -1) { + /* + * This can happen if the device is asynchronously reset and + * terminates a data transfer. + */ + error_report("%s: data_fd out of sync", vbasedev->name); + close(mig_state->data_fd); + + return -1; + } + + migration->data_fd = mig_state->data_fd; + } + migration->device_state = new_state; + + trace_vfio_migration_set_state(vbasedev->name, mig_state_to_str(new_state)); + + return 0; +} + static inline int vfio_mig_access(VFIODevice *vbasedev, void *val, int count, off_t off, bool iswrite) { @@ -260,6 +336,20 @@ static int vfio_save_buffer(QEMUFile *f, VFIODevice *vbasedev, uint64_t *size) return ret; } +static int vfio_load_buffer(QEMUFile *f, VFIODevice *vbasedev, + uint64_t data_size) +{ + VFIOMigration *migration = vbasedev->migration; + int ret; + + ret = qemu_file_get_to_fd(f, migration->data_fd, data_size); + if (!ret) { + trace_vfio_load_state_device_data(vbasedev->name, data_size); + } + + return ret; +} + static int vfio_v1_load_buffer(QEMUFile *f, VFIODevice *vbasedev, uint64_t data_size) { @@ -394,6 +484,14 @@ static int vfio_load_device_config_state(QEMUFile *f, void *opaque) return qemu_file_get_error(f); } +static void vfio_migration_cleanup(VFIODevice *vbasedev) +{ + VFIOMigration *migration = vbasedev->migration; + + close(migration->data_fd); + migration->data_fd = -1; +} + static void vfio_migration_v1_cleanup(VFIODevice *vbasedev) { VFIOMigration *migration = vbasedev->migration; @@ -405,6 +503,18 @@ static void vfio_migration_v1_cleanup(VFIODevice *vbasedev) /* ---------------------------------------------------------------------- */ +static int vfio_save_setup(QEMUFile *f, void *opaque) +{ + VFIODevice *vbasedev = opaque; + + trace_vfio_save_setup(vbasedev->name); + + qemu_put_be64(f, VFIO_MIG_FLAG_DEV_SETUP_STATE); + qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); + + return qemu_file_get_error(f); +} + static int vfio_v1_save_setup(QEMUFile *f, void *opaque) { VFIODevice *vbasedev = opaque; @@ -448,6 +558,14 @@ static int vfio_v1_save_setup(QEMUFile *f, void *opaque) return 0; } +static void vfio_save_cleanup(void *opaque) +{ + VFIODevice *vbasedev = opaque; + + vfio_migration_cleanup(vbasedev); + trace_vfio_save_cleanup(vbasedev->name); +} + static void vfio_v1_save_cleanup(void *opaque) { VFIODevice *vbasedev = opaque; @@ -456,6 +574,23 @@ static void vfio_v1_save_cleanup(void *opaque) trace_vfio_save_cleanup(vbasedev->name); } +#define VFIO_MIG_PENDING_SIZE (512 * 1024 * 1024) +static void vfio_save_pending(void *opaque, uint64_t threshold_size, + uint64_t *res_precopy, uint64_t *res_postcopy) +{ + VFIODevice *vbasedev = opaque; + + /* + * VFIO migration protocol v2 currently doesn't have an API to get pending + * device state size. Until such API is introduced, report some big + * arbitrary pending size so the device will be taken into account for + * downtime limit calculations. + */ + *res_postcopy += VFIO_MIG_PENDING_SIZE; + + trace_vfio_save_pending(vbasedev->name, *res_precopy, *res_postcopy); +} + static void vfio_v1_save_pending(void *opaque, uint64_t threshold_size, uint64_t *res_precopy, uint64_t *res_postcopy) { @@ -520,6 +655,67 @@ static int vfio_save_iterate(QEMUFile *f, void *opaque) return 0; } +/* Returns 1 if end-of-stream is reached, 0 if more data and -1 if error */ +static int vfio_save_block(QEMUFile *f, VFIOMigration *migration) +{ + ssize_t data_size; + + data_size = read(migration->data_fd, migration->data_buffer, + migration->data_buffer_size); + if (data_size < 0) { + return -1; + } + if (data_size == 0) { + return 1; + } + + qemu_put_be64(f, VFIO_MIG_FLAG_DEV_DATA_STATE); + qemu_put_be64(f, data_size); + qemu_put_buffer(f, migration->data_buffer, data_size); + bytes_transferred += data_size; + + trace_vfio_save_block(migration->vbasedev->name, data_size); + + return qemu_file_get_error(f); +} + +static int vfio_save_complete_precopy(QEMUFile *f, void *opaque) +{ + VFIODevice *vbasedev = opaque; + enum vfio_device_mig_state recover_state; + int ret; + + /* We reach here with device state STOP only */ + recover_state = VFIO_DEVICE_STATE_STOP; + ret = vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_STOP_COPY, + recover_state); + if (ret) { + return ret; + } + + do { + ret = vfio_save_block(f, vbasedev->migration); + if (ret < 0) { + return ret; + } + } while (!ret); + + qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); + ret = qemu_file_get_error(f); + if (ret) { + return ret; + } + + recover_state = VFIO_DEVICE_STATE_ERROR; + ret = vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_STOP, + recover_state); + if (!ret) { + trace_vfio_save_complete_precopy(vbasedev->name); + } + + return ret; +} + static int vfio_v1_save_complete_precopy(QEMUFile *f, void *opaque) { VFIODevice *vbasedev = opaque; @@ -589,6 +785,14 @@ static void vfio_save_state(QEMUFile *f, void *opaque) } } +static int vfio_load_setup(QEMUFile *f, void *opaque) +{ + VFIODevice *vbasedev = opaque; + + return vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_RESUMING, + vbasedev->migration->device_state); +} + static int vfio_v1_load_setup(QEMUFile *f, void *opaque) { VFIODevice *vbasedev = opaque; @@ -616,6 +820,16 @@ static int vfio_v1_load_setup(QEMUFile *f, void *opaque) return ret; } +static int vfio_load_cleanup(void *opaque) +{ + VFIODevice *vbasedev = opaque; + + vfio_migration_cleanup(vbasedev); + trace_vfio_load_cleanup(vbasedev->name); + + return 0; +} + static int vfio_v1_load_cleanup(void *opaque) { VFIODevice *vbasedev = opaque; @@ -658,7 +872,11 @@ static int vfio_load_state(QEMUFile *f, void *opaque, int version_id) uint64_t data_size = qemu_get_be64(f); if (data_size) { - ret = vfio_v1_load_buffer(f, vbasedev, data_size); + if (vbasedev->migration->v2) { + ret = vfio_load_buffer(f, vbasedev, data_size); + } else { + ret = vfio_v1_load_buffer(f, vbasedev, data_size); + } if (ret < 0) { return ret; } @@ -679,6 +897,17 @@ static int vfio_load_state(QEMUFile *f, void *opaque, int version_id) return ret; } +static const SaveVMHandlers savevm_vfio_handlers = { + .save_setup = vfio_save_setup, + .save_cleanup = vfio_save_cleanup, + .save_live_pending = vfio_save_pending, + .save_live_complete_precopy = vfio_save_complete_precopy, + .save_state = vfio_save_state, + .load_setup = vfio_load_setup, + .load_cleanup = vfio_load_cleanup, + .load_state = vfio_load_state, +}; + static SaveVMHandlers savevm_vfio_v1_handlers = { .save_setup = vfio_v1_save_setup, .save_cleanup = vfio_v1_save_cleanup, @@ -693,6 +922,34 @@ static SaveVMHandlers savevm_vfio_v1_handlers = { /* ---------------------------------------------------------------------- */ +static void vfio_vmstate_change(void *opaque, bool running, RunState state) +{ + VFIODevice *vbasedev = opaque; + enum vfio_device_mig_state new_state; + int ret; + + if (running) { + new_state = VFIO_DEVICE_STATE_RUNNING; + } else { + new_state = VFIO_DEVICE_STATE_STOP; + } + + ret = vfio_migration_set_state(vbasedev, new_state, + VFIO_DEVICE_STATE_ERROR); + if (ret) { + /* + * Migration should be aborted in this case, but vm_state_notify() + * currently does not support reporting failures. + */ + if (migrate_get_current()->to_dst_file) { + qemu_file_set_error(migrate_get_current()->to_dst_file, ret); + } + } + + trace_vfio_vmstate_change(vbasedev->name, running, RunState_str(state), + mig_state_to_str(new_state)); +} + static void vfio_v1_vmstate_change(void *opaque, bool running, RunState state) { VFIODevice *vbasedev = opaque; @@ -766,12 +1023,17 @@ static void vfio_migration_state_notifier(Notifier *notifier, void *data) case MIGRATION_STATUS_CANCELLED: case MIGRATION_STATUS_FAILED: bytes_transferred = 0; - ret = vfio_migration_v1_set_state(vbasedev, - ~(VFIO_DEVICE_STATE_V1_SAVING | - VFIO_DEVICE_STATE_V1_RESUMING), - VFIO_DEVICE_STATE_V1_RUNNING); - if (ret) { - error_report("%s: Failed to set state RUNNING", vbasedev->name); + if (migration->v2) { + vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_RUNNING, + VFIO_DEVICE_STATE_ERROR); + } else { + ret = vfio_migration_v1_set_state(vbasedev, + ~(VFIO_DEVICE_STATE_V1_SAVING | + VFIO_DEVICE_STATE_V1_RESUMING), + VFIO_DEVICE_STATE_V1_RUNNING); + if (ret) { + error_report("%s: Failed to set state RUNNING", vbasedev->name); + } } } } @@ -780,12 +1042,35 @@ static void vfio_migration_exit(VFIODevice *vbasedev) { VFIOMigration *migration = vbasedev->migration; - vfio_region_exit(&migration->region); - vfio_region_finalize(&migration->region); + if (migration->v2) { + g_free(migration->data_buffer); + } else { + vfio_region_exit(&migration->region); + vfio_region_finalize(&migration->region); + } g_free(vbasedev->migration); vbasedev->migration = NULL; } +static int vfio_migration_query_flags(VFIODevice *vbasedev, uint64_t *mig_flags) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_migration), + sizeof(uint64_t))] = {}; + struct vfio_device_feature *feature = (void *)buf; + struct vfio_device_feature_migration *mig = (void *)feature->data; + + feature->argsz = sizeof(buf); + feature->flags = VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_MIGRATION; + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + return -EOPNOTSUPP; + } + + *mig_flags = mig->flags; + + return 0; +} + static int vfio_migration_init(VFIODevice *vbasedev) { int ret; @@ -794,6 +1079,7 @@ static int vfio_migration_init(VFIODevice *vbasedev) char id[256] = ""; g_autofree char *path = NULL, *oid = NULL; struct vfio_region_info *info = NULL; + uint64_t mig_flags; if (!vbasedev->ops->vfio_get_object) { return -EINVAL; @@ -804,34 +1090,51 @@ static int vfio_migration_init(VFIODevice *vbasedev) return -EINVAL; } - ret = vfio_get_dev_region_info(vbasedev, - VFIO_REGION_TYPE_MIGRATION_DEPRECATED, - VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED, - &info); - if (ret) { - return ret; - } + ret = vfio_migration_query_flags(vbasedev, &mig_flags); + if (!ret) { + /* Migration v2 */ + /* Basic migration functionality must be supported */ + if (!(mig_flags & VFIO_MIGRATION_STOP_COPY)) { + return -EOPNOTSUPP; + } + vbasedev->migration = g_new0(VFIOMigration, 1); + vbasedev->migration->device_state = VFIO_DEVICE_STATE_RUNNING; + vbasedev->migration->data_buffer_size = VFIO_MIG_DATA_BUFFER_SIZE; + vbasedev->migration->data_buffer = + g_malloc0(vbasedev->migration->data_buffer_size); + vbasedev->migration->data_fd = -1; + vbasedev->migration->v2 = true; + } else { + /* Migration v1 */ + ret = vfio_get_dev_region_info(vbasedev, + VFIO_REGION_TYPE_MIGRATION_DEPRECATED, + VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED, + &info); + if (ret) { + return ret; + } - vbasedev->migration = g_new0(VFIOMigration, 1); - vbasedev->migration->device_state_v1 = VFIO_DEVICE_STATE_V1_RUNNING; - vbasedev->migration->vm_running = runstate_is_running(); + vbasedev->migration = g_new0(VFIOMigration, 1); + vbasedev->migration->device_state_v1 = VFIO_DEVICE_STATE_V1_RUNNING; + vbasedev->migration->vm_running = runstate_is_running(); - ret = vfio_region_setup(obj, vbasedev, &vbasedev->migration->region, - info->index, "migration"); - if (ret) { - error_report("%s: Failed to setup VFIO migration region %d: %s", - vbasedev->name, info->index, strerror(-ret)); - goto err; - } + ret = vfio_region_setup(obj, vbasedev, &vbasedev->migration->region, + info->index, "migration"); + if (ret) { + error_report("%s: Failed to setup VFIO migration region %d: %s", + vbasedev->name, info->index, strerror(-ret)); + goto err; + } - if (!vbasedev->migration->region.size) { - error_report("%s: Invalid zero-sized VFIO migration region %d", - vbasedev->name, info->index); - ret = -EINVAL; - goto err; - } + if (!vbasedev->migration->region.size) { + error_report("%s: Invalid zero-sized VFIO migration region %d", + vbasedev->name, info->index); + ret = -EINVAL; + goto err; + } - g_free(info); + g_free(info); + } migration = vbasedev->migration; migration->vbasedev = vbasedev; @@ -844,11 +1147,20 @@ static int vfio_migration_init(VFIODevice *vbasedev) } strpadcpy(id, sizeof(id), path, '\0'); - register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, - &savevm_vfio_v1_handlers, vbasedev); + if (migration->v2) { + register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, + &savevm_vfio_handlers, vbasedev); + + migration->vm_state = qdev_add_vm_change_state_handler( + vbasedev->dev, vfio_vmstate_change, vbasedev); + } else { + register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, + &savevm_vfio_v1_handlers, vbasedev); + + migration->vm_state = qdev_add_vm_change_state_handler( + vbasedev->dev, vfio_v1_vmstate_change, vbasedev); + } - migration->vm_state = qdev_add_vm_change_state_handler( - vbasedev->dev, vfio_v1_vmstate_change, vbasedev); migration->migration_state.notify = vfio_migration_state_notifier; add_migration_state_change_notifier(&migration->migration_state); return 0; diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index d88d2b4053..9ef84e24b2 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -149,7 +149,9 @@ vfio_display_edid_write_error(void) "" # migration.c vfio_migration_probe(const char *name) " (%s)" +vfio_migration_set_state(const char *name, const char *state) " (%s) state %s" vfio_migration_v1_set_state(const char *name, uint32_t state) " (%s) state %d" +vfio_vmstate_change(const char *name, int running, const char *reason, const char *dev_state) " (%s) running %d reason %s device state %s" vfio_v1_vmstate_change(const char *name, int running, const char *reason, uint32_t dev_state) " (%s) running %d reason %s device state %d" vfio_migration_state_notifier(const char *name, const char *state) " (%s) state %s" vfio_save_setup(const char *name) " (%s)" @@ -163,6 +165,8 @@ vfio_save_complete_precopy(const char *name) " (%s)" vfio_load_device_config_state(const char *name) " (%s)" vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 vfio_v1_load_state_device_data(const char *name, uint64_t data_offset, uint64_t data_size) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64 +vfio_load_state_device_data(const char *name, uint64_t data_size) " (%s) size 0x%"PRIx64 vfio_load_cleanup(const char *name) " (%s)" vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitmap_size, uint64_t start) "container fd=%d, iova=0x%"PRIx64" size= 0x%"PRIx64" bitmap_size=0x%"PRIx64" start=0x%"PRIx64 vfio_iommu_map_dirty_notify(uint64_t iova_start, uint64_t iova_end) "iommu dirty @ 0x%"PRIx64" - 0x%"PRIx64 +vfio_save_block(const char *name, int data_size) " (%s) data_size %d" diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index bbaf72ba00..2ec3346fea 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -66,6 +66,11 @@ typedef struct VFIOMigration { int vm_running; Notifier migration_state; uint64_t pending_bytes; + enum vfio_device_mig_state device_state; + int data_fd; + void *data_buffer; + size_t data_buffer_size; + bool v2; } VFIOMigration; typedef struct VFIOAddressSpace { From patchwork Thu Nov 3 16:16:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C1708C4332F for ; Thu, 3 Nov 2022 16:23:26 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcxT-0007AM-IQ; Thu, 03 Nov 2022 12:20:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcv8-0001Bn-Mq; Thu, 03 Nov 2022 12:18:32 -0400 Received: from mail-co1nam11on2042.outbound.protection.outlook.com ([40.107.220.42] helo=NAM11-CO1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcv5-0008Cn-AF; Thu, 03 Nov 2022 12:18:22 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=H3NdaCP6CBRv7Afx0sRqtniycjCX5bOFxFcGnNsQ2711N2gHZctiBlNTKPCLBjyztNgEqT8YTmxrZ/I3lSozSY26FORPHdcEaqTSPZLv3OjNIbSFGDyg9CdEXorisDBjy0dsL9c1NyXNegu3leDY9GFKnWvDuH9VsLB1v92LN5WDFRihvK8X9pWTNmwpiw/EH3Dcnu17DL3w3RZeWJIclqQI6DyqF94GzQ8nABf6POwNvgd8NhYV/a/rLBIUkVeAi6D+3nRyKguk+kvHL9sdwkW5gv/cOO1ZQGPZWGg2flQin/Ozm+Y8SpmL3DtretE1N2LJh4b8/SzdzRAnOE3hig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vu9xjUeWujHgXQvI6JDy9po/GtyHmEF7PgptdSVdDLg=; b=DK4Pdyp9J1m1MdiYv3zG6H8g7JHg39zFxV0qSgj0WNDJQ4PXJORSibJ4T7PngfENgED0UrUpwurTxkZlrLdQLYSI3S8pFOKnMp7Y6o8Qgjkldk2o+echIWGQHRN6OM0Y9mPYCTx42QK7wfs6xeVuEgAowKo/n7unyU13vP8AdPsLI6dN4ZrCIwyntpRJuHkTq/n8C/3dTm4Zn+SPe2jj4D5kfpPixkUA9WiIcknq76xqs25PKVDQclDekAZmRuNI7uYVGIunFwMRX51mRVhEZ8gt9tQdexFA+QIvFcOIE2TFBt9EJlaSOSNMwkwX51p2QX2mEz/h5EvF0sAtMg61lg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vu9xjUeWujHgXQvI6JDy9po/GtyHmEF7PgptdSVdDLg=; b=rZzCIchinBnSqFOOZ4Jql3SHjPo9HyRHS9hWcgbjChhsJTvHP3VUPcQMxOY1f77cF5iWxxlERZqsX+hMCULzHvZSUBOHKIpQEiM38SD44S1AIXnmcGlzf61Ze9B0cUmONUxYIr2vPWtcyGdydDRYy7JNLmzDg3N5yAaXsys8RQGLaKfA5rSGhNdJkg2EpJboetm+iftrvwznHjCeDhrUTVXGdZQ8HCyFlI/k2ayHwgNAHoiUvY8kQc8XgubRsvAX/So2ZyaLKFFy53JE4ZFBj8oxaUIiWs3xivr3k5cqv2EfQxTjInQBZAwJFpt4PoflNKUsTdQfuE51AmaXnN0U1g== Received: from DS7PR03CA0085.namprd03.prod.outlook.com (2603:10b6:5:3bb::30) by IA1PR12MB6209.namprd12.prod.outlook.com (2603:10b6:208:3e7::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20; Thu, 3 Nov 2022 16:18:14 +0000 Received: from DM6NAM11FT041.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3bb:cafe::14) by DS7PR03CA0085.outlook.office365.com (2603:10b6:5:3bb::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22 via Frontend Transport; Thu, 3 Nov 2022 16:18:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by DM6NAM11FT041.mail.protection.outlook.com (10.13.172.98) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:18:13 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:18:09 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:18:09 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:18:02 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 13/17] vfio/migration: Remove VFIO migration protocol v1 Date: Thu, 3 Nov 2022 18:16:16 +0200 Message-ID: <20221103161620.13120-14-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT041:EE_|IA1PR12MB6209:EE_ X-MS-Office365-Filtering-Correlation-Id: 559a2086-8632-4ddd-e729-08dabdb701e9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ci6Qt+nsRmbO6Pone8JEZW+66T5AenVmRvRgViKmGUn7XJBxl2mOU8N5yHUZ45DtzcvYW883kOEFfS5xETx1/xkrjvGyExg6UtkxXQmwdSAW7sQ06xJxTCfdL5AYBpyhpx+qeDuknhfT2UMICPO05DNhKSBJwwbs2nj6qtAbNybcq1yCkfoMII9KPgGHw6KoY5qo3XoRZQsP87ZqAber+VJNQHY8Zp1x4KVMtYsQTzj+jSVbtbzOaekqqsBFaYzuu7VhIHX4mS4+zEgkvqEyWdyioimDK1t+i2IFzcNnzHqNNNcwZdRczyeelDhPPnMqxYZzPbih/1gUpuDDTCIpXNzvu0rSIOTksKi8cPj+7vy2xlIehqhtRgKnFgYukoaK2cDwTSz5f6XogXz5YEllMnAw756AVZyyVpBJ8Bbmek7UVN8/R8NCTVzc527QIdZjzsWDq17+5Jk0hZOs6dc+JDtU+A/tVwlJKbXGFidvsANeYorE+rPpB3SP7mXmvu6VRBYSPQEx16DbIQbQDrWIdsAD62H1Q8fgP6LR9EiFHRWOK8d0jCHHBS7zDO21TMPuRRDSK0YWR3929G8FkCMZT47wHLI7VtKHpfLlfkJs4DlROBH6VEnM0ZEhlzdLGvzxC7XkkamRsr+XcyvF3N4M9bOei5bDRdzQW6H0xog/k0XRQdPNSkbdXcGELr9CNC/FizXv6nDDzUIENW2U0Pm7iCGBZsxQhMEuV8mY9xa2DHeA+2rZ5gAmkdmCpF3zWl4YL899l20pNhkiISbf1398Wg== X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(376002)(136003)(39860400002)(346002)(396003)(451199015)(40470700004)(36840700001)(46966006)(41300700001)(70586007)(70206006)(30864003)(5660300002)(2906002)(8676002)(83380400001)(316002)(2616005)(4326008)(40460700003)(8936002)(6916009)(478600001)(186003)(40480700001)(356005)(7636003)(7416002)(36860700001)(82740400003)(426003)(86362001)(7696005)(6666004)(336012)(1076003)(26005)(54906003)(47076005)(36756003)(82310400005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:18:13.3711 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 559a2086-8632-4ddd-e729-08dabdb701e9 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT041.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6209 Received-SPF: softfail client-ip=40.107.220.42; envelope-from=avihaih@nvidia.com; helo=NAM11-CO1-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Now that v2 protocol implementation has been added, remove the deprecated v1 implementation. Signed-off-by: Avihai Horon --- hw/vfio/common.c | 19 +- hw/vfio/migration.c | 697 +--------------------------------- hw/vfio/trace-events | 6 - include/hw/vfio/vfio-common.h | 5 - 4 files changed, 24 insertions(+), 703 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 0bdbd1586b..9ff57d4b27 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -355,14 +355,7 @@ static bool vfio_devices_all_dirty_tracking(VFIOContainer *container) return false; } - if (!migration->v2 && - (vbasedev->pre_copy_dirty_page_tracking == ON_OFF_AUTO_OFF) && - (migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING)) { - return false; - } - - if (migration->v2 && - (vbasedev->pre_copy_dirty_page_tracking == ON_OFF_AUTO_OFF) && + if ((vbasedev->pre_copy_dirty_page_tracking == ON_OFF_AUTO_OFF) && (migration->device_state == VFIO_DEVICE_STATE_RUNNING || migration->device_state == VFIO_DEVICE_STATE_RUNNING_P2P)) { return false; @@ -393,14 +386,8 @@ static bool vfio_devices_all_running_and_mig_active(VFIOContainer *container) return false; } - if (!migration->v2 && - migration->device_state_v1 & VFIO_DEVICE_STATE_V1_RUNNING) { - continue; - } - - if (migration->v2 && - (migration->device_state == VFIO_DEVICE_STATE_RUNNING || - migration->device_state == VFIO_DEVICE_STATE_RUNNING_P2P)) { + if (migration->device_state == VFIO_DEVICE_STATE_RUNNING || + migration->device_state == VFIO_DEVICE_STATE_RUNNING_P2P) { continue; } else { return false; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 62afc23a8c..f8c3228314 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -122,220 +122,6 @@ static int vfio_migration_set_state(VFIODevice *vbasedev, return 0; } -static inline int vfio_mig_access(VFIODevice *vbasedev, void *val, int count, - off_t off, bool iswrite) -{ - int ret; - - ret = iswrite ? pwrite(vbasedev->fd, val, count, off) : - pread(vbasedev->fd, val, count, off); - if (ret < count) { - error_report("vfio_mig_%s %d byte %s: failed at offset 0x%" - HWADDR_PRIx", err: %s", iswrite ? "write" : "read", count, - vbasedev->name, off, strerror(errno)); - return (ret < 0) ? ret : -EINVAL; - } - return 0; -} - -static int vfio_mig_rw(VFIODevice *vbasedev, __u8 *buf, size_t count, - off_t off, bool iswrite) -{ - int ret, done = 0; - __u8 *tbuf = buf; - - while (count) { - int bytes = 0; - - if (count >= 8 && !(off % 8)) { - bytes = 8; - } else if (count >= 4 && !(off % 4)) { - bytes = 4; - } else if (count >= 2 && !(off % 2)) { - bytes = 2; - } else { - bytes = 1; - } - - ret = vfio_mig_access(vbasedev, tbuf, bytes, off, iswrite); - if (ret) { - return ret; - } - - count -= bytes; - done += bytes; - off += bytes; - tbuf += bytes; - } - return done; -} - -#define vfio_mig_read(f, v, c, o) vfio_mig_rw(f, (__u8 *)v, c, o, false) -#define vfio_mig_write(f, v, c, o) vfio_mig_rw(f, (__u8 *)v, c, o, true) - -#define VFIO_MIG_STRUCT_OFFSET(f) \ - offsetof(struct vfio_device_migration_info, f) -/* - * Change the device_state register for device @vbasedev. Bits set in @mask - * are preserved, bits set in @value are set, and bits not set in either @mask - * or @value are cleared in device_state. If the register cannot be accessed, - * the resulting state would be invalid, or the device enters an error state, - * an error is returned. - */ - -static int vfio_migration_v1_set_state(VFIODevice *vbasedev, uint32_t mask, - uint32_t value) -{ - VFIOMigration *migration = vbasedev->migration; - VFIORegion *region = &migration->region; - off_t dev_state_off = region->fd_offset + - VFIO_MIG_STRUCT_OFFSET(device_state); - uint32_t device_state; - int ret; - - ret = vfio_mig_read(vbasedev, &device_state, sizeof(device_state), - dev_state_off); - if (ret < 0) { - return ret; - } - - device_state = (device_state & mask) | value; - - if (!VFIO_DEVICE_STATE_VALID(device_state)) { - return -EINVAL; - } - - ret = vfio_mig_write(vbasedev, &device_state, sizeof(device_state), - dev_state_off); - if (ret < 0) { - int rret; - - rret = vfio_mig_read(vbasedev, &device_state, sizeof(device_state), - dev_state_off); - - if ((rret < 0) || (VFIO_DEVICE_STATE_IS_ERROR(device_state))) { - hw_error("%s: Device in error state 0x%x", vbasedev->name, - device_state); - return rret ? rret : -EIO; - } - return ret; - } - - migration->device_state_v1 = device_state; - trace_vfio_migration_v1_set_state(vbasedev->name, device_state); - return 0; -} - -static void *get_data_section_size(VFIORegion *region, uint64_t data_offset, - uint64_t data_size, uint64_t *size) -{ - void *ptr = NULL; - uint64_t limit = 0; - int i; - - if (!region->mmaps) { - if (size) { - *size = MIN(data_size, region->size - data_offset); - } - return ptr; - } - - for (i = 0; i < region->nr_mmaps; i++) { - VFIOMmap *map = region->mmaps + i; - - if ((data_offset >= map->offset) && - (data_offset < map->offset + map->size)) { - - /* check if data_offset is within sparse mmap areas */ - ptr = map->mmap + data_offset - map->offset; - if (size) { - *size = MIN(data_size, map->offset + map->size - data_offset); - } - break; - } else if ((data_offset < map->offset) && - (!limit || limit > map->offset)) { - /* - * data_offset is not within sparse mmap areas, find size of - * non-mapped area. Check through all list since region->mmaps list - * is not sorted. - */ - limit = map->offset; - } - } - - if (!ptr && size) { - *size = limit ? MIN(data_size, limit - data_offset) : data_size; - } - return ptr; -} - -static int vfio_save_buffer(QEMUFile *f, VFIODevice *vbasedev, uint64_t *size) -{ - VFIOMigration *migration = vbasedev->migration; - VFIORegion *region = &migration->region; - uint64_t data_offset = 0, data_size = 0, sz; - int ret; - - ret = vfio_mig_read(vbasedev, &data_offset, sizeof(data_offset), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_offset)); - if (ret < 0) { - return ret; - } - - ret = vfio_mig_read(vbasedev, &data_size, sizeof(data_size), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_size)); - if (ret < 0) { - return ret; - } - - trace_vfio_save_buffer(vbasedev->name, data_offset, data_size, - migration->pending_bytes); - - qemu_put_be64(f, data_size); - sz = data_size; - - while (sz) { - void *buf; - uint64_t sec_size; - bool buf_allocated = false; - - buf = get_data_section_size(region, data_offset, sz, &sec_size); - - if (!buf) { - buf = g_try_malloc(sec_size); - if (!buf) { - error_report("%s: Error allocating buffer ", __func__); - return -ENOMEM; - } - buf_allocated = true; - - ret = vfio_mig_read(vbasedev, buf, sec_size, - region->fd_offset + data_offset); - if (ret < 0) { - g_free(buf); - return ret; - } - } - - qemu_put_buffer(f, buf, sec_size); - - if (buf_allocated) { - g_free(buf); - } - sz -= sec_size; - data_offset += sec_size; - } - - ret = qemu_file_get_error(f); - - if (!ret && size) { - *size = data_size; - } - - bytes_transferred += data_size; - return ret; -} - static int vfio_load_buffer(QEMUFile *f, VFIODevice *vbasedev, uint64_t data_size) { @@ -350,96 +136,6 @@ static int vfio_load_buffer(QEMUFile *f, VFIODevice *vbasedev, return ret; } -static int vfio_v1_load_buffer(QEMUFile *f, VFIODevice *vbasedev, - uint64_t data_size) -{ - VFIORegion *region = &vbasedev->migration->region; - uint64_t data_offset = 0, size, report_size; - int ret; - - do { - ret = vfio_mig_read(vbasedev, &data_offset, sizeof(data_offset), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_offset)); - if (ret < 0) { - return ret; - } - - if (data_offset + data_size > region->size) { - /* - * If data_size is greater than the data section of migration region - * then iterate the write buffer operation. This case can occur if - * size of migration region at destination is smaller than size of - * migration region at source. - */ - report_size = size = region->size - data_offset; - data_size -= size; - } else { - report_size = size = data_size; - data_size = 0; - } - - trace_vfio_v1_load_state_device_data(vbasedev->name, data_offset, size); - - while (size) { - void *buf; - uint64_t sec_size; - bool buf_alloc = false; - - buf = get_data_section_size(region, data_offset, size, &sec_size); - - if (!buf) { - buf = g_try_malloc(sec_size); - if (!buf) { - error_report("%s: Error allocating buffer ", __func__); - return -ENOMEM; - } - buf_alloc = true; - } - - qemu_get_buffer(f, buf, sec_size); - - if (buf_alloc) { - ret = vfio_mig_write(vbasedev, buf, sec_size, - region->fd_offset + data_offset); - g_free(buf); - - if (ret < 0) { - return ret; - } - } - size -= sec_size; - data_offset += sec_size; - } - - ret = vfio_mig_write(vbasedev, &report_size, sizeof(report_size), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(data_size)); - if (ret < 0) { - return ret; - } - } while (data_size); - - return 0; -} - -static int vfio_update_pending(VFIODevice *vbasedev) -{ - VFIOMigration *migration = vbasedev->migration; - VFIORegion *region = &migration->region; - uint64_t pending_bytes = 0; - int ret; - - ret = vfio_mig_read(vbasedev, &pending_bytes, sizeof(pending_bytes), - region->fd_offset + VFIO_MIG_STRUCT_OFFSET(pending_bytes)); - if (ret < 0) { - migration->pending_bytes = 0; - return ret; - } - - migration->pending_bytes = pending_bytes; - trace_vfio_update_pending(vbasedev->name, pending_bytes); - return 0; -} - static int vfio_save_device_config_state(QEMUFile *f, void *opaque) { VFIODevice *vbasedev = opaque; @@ -492,15 +188,6 @@ static void vfio_migration_cleanup(VFIODevice *vbasedev) migration->data_fd = -1; } -static void vfio_migration_v1_cleanup(VFIODevice *vbasedev) -{ - VFIOMigration *migration = vbasedev->migration; - - if (migration->region.mmaps) { - vfio_region_unmap(&migration->region); - } -} - /* ---------------------------------------------------------------------- */ static int vfio_save_setup(QEMUFile *f, void *opaque) @@ -515,49 +202,6 @@ static int vfio_save_setup(QEMUFile *f, void *opaque) return qemu_file_get_error(f); } -static int vfio_v1_save_setup(QEMUFile *f, void *opaque) -{ - VFIODevice *vbasedev = opaque; - VFIOMigration *migration = vbasedev->migration; - int ret; - - trace_vfio_save_setup(vbasedev->name); - - qemu_put_be64(f, VFIO_MIG_FLAG_DEV_SETUP_STATE); - - if (migration->region.mmaps) { - /* - * Calling vfio_region_mmap() from migration thread. Memory API called - * from this function require locking the iothread when called from - * outside the main loop thread. - */ - qemu_mutex_lock_iothread(); - ret = vfio_region_mmap(&migration->region); - qemu_mutex_unlock_iothread(); - if (ret) { - error_report("%s: Failed to mmap VFIO migration region: %s", - vbasedev->name, strerror(-ret)); - error_report("%s: Falling back to slow path", vbasedev->name); - } - } - - ret = vfio_migration_v1_set_state(vbasedev, VFIO_DEVICE_STATE_MASK, - VFIO_DEVICE_STATE_V1_SAVING); - if (ret) { - error_report("%s: Failed to set state SAVING", vbasedev->name); - return ret; - } - - qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); - - ret = qemu_file_get_error(f); - if (ret) { - return ret; - } - - return 0; -} - static void vfio_save_cleanup(void *opaque) { VFIODevice *vbasedev = opaque; @@ -566,14 +210,6 @@ static void vfio_save_cleanup(void *opaque) trace_vfio_save_cleanup(vbasedev->name); } -static void vfio_v1_save_cleanup(void *opaque) -{ - VFIODevice *vbasedev = opaque; - - vfio_migration_v1_cleanup(vbasedev); - trace_vfio_save_cleanup(vbasedev->name); -} - #define VFIO_MIG_PENDING_SIZE (512 * 1024 * 1024) static void vfio_save_pending(void *opaque, uint64_t threshold_size, uint64_t *res_precopy, uint64_t *res_postcopy) @@ -591,70 +227,6 @@ static void vfio_save_pending(void *opaque, uint64_t threshold_size, trace_vfio_save_pending(vbasedev->name, *res_precopy, *res_postcopy); } -static void vfio_v1_save_pending(void *opaque, uint64_t threshold_size, - uint64_t *res_precopy, uint64_t *res_postcopy) -{ - VFIODevice *vbasedev = opaque; - VFIOMigration *migration = vbasedev->migration; - int ret; - - ret = vfio_update_pending(vbasedev); - if (ret) { - return; - } - - *res_precopy += migration->pending_bytes; - - trace_vfio_save_pending(vbasedev->name, *res_precopy, *res_postcopy); -} - -static int vfio_save_iterate(QEMUFile *f, void *opaque) -{ - VFIODevice *vbasedev = opaque; - VFIOMigration *migration = vbasedev->migration; - uint64_t data_size; - int ret; - - qemu_put_be64(f, VFIO_MIG_FLAG_DEV_DATA_STATE); - - if (migration->pending_bytes == 0) { - ret = vfio_update_pending(vbasedev); - if (ret) { - return ret; - } - - if (migration->pending_bytes == 0) { - qemu_put_be64(f, 0); - qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); - /* indicates data finished, goto complete phase */ - return 1; - } - } - - ret = vfio_save_buffer(f, vbasedev, &data_size); - if (ret) { - error_report("%s: vfio_save_buffer failed %s", vbasedev->name, - strerror(errno)); - return ret; - } - - qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); - - ret = qemu_file_get_error(f); - if (ret) { - return ret; - } - - /* - * Reset pending_bytes as .save_live_pending is not called during savevm or - * snapshot case, in such case vfio_update_pending() at the start of this - * function updates pending_bytes. - */ - migration->pending_bytes = 0; - trace_vfio_save_iterate(vbasedev->name, data_size); - return 0; -} - /* Returns 1 if end-of-stream is reached, 0 if more data and -1 if error */ static int vfio_save_block(QEMUFile *f, VFIOMigration *migration) { @@ -716,62 +288,6 @@ static int vfio_save_complete_precopy(QEMUFile *f, void *opaque) return ret; } -static int vfio_v1_save_complete_precopy(QEMUFile *f, void *opaque) -{ - VFIODevice *vbasedev = opaque; - VFIOMigration *migration = vbasedev->migration; - uint64_t data_size; - int ret; - - ret = vfio_migration_v1_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_RUNNING, - VFIO_DEVICE_STATE_V1_SAVING); - if (ret) { - error_report("%s: Failed to set state STOP and SAVING", - vbasedev->name); - return ret; - } - - ret = vfio_update_pending(vbasedev); - if (ret) { - return ret; - } - - while (migration->pending_bytes > 0) { - qemu_put_be64(f, VFIO_MIG_FLAG_DEV_DATA_STATE); - ret = vfio_save_buffer(f, vbasedev, &data_size); - if (ret < 0) { - error_report("%s: Failed to save buffer", vbasedev->name); - return ret; - } - - if (data_size == 0) { - break; - } - - ret = vfio_update_pending(vbasedev); - if (ret) { - return ret; - } - } - - qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); - - ret = qemu_file_get_error(f); - if (ret) { - return ret; - } - - ret = vfio_migration_v1_set_state(vbasedev, ~VFIO_DEVICE_STATE_V1_SAVING, - 0); - if (ret) { - error_report("%s: Failed to set state STOPPED", vbasedev->name); - return ret; - } - - trace_vfio_save_complete_precopy(vbasedev->name); - return ret; -} - static void vfio_save_state(QEMUFile *f, void *opaque) { VFIODevice *vbasedev = opaque; @@ -793,33 +309,6 @@ static int vfio_load_setup(QEMUFile *f, void *opaque) vbasedev->migration->device_state); } -static int vfio_v1_load_setup(QEMUFile *f, void *opaque) -{ - VFIODevice *vbasedev = opaque; - VFIOMigration *migration = vbasedev->migration; - int ret = 0; - - if (migration->region.mmaps) { - ret = vfio_region_mmap(&migration->region); - if (ret) { - error_report("%s: Failed to mmap VFIO migration region %d: %s", - vbasedev->name, migration->region.nr, - strerror(-ret)); - error_report("%s: Falling back to slow path", vbasedev->name); - } - } - - ret = vfio_migration_v1_set_state(vbasedev, ~VFIO_DEVICE_STATE_MASK, - VFIO_DEVICE_STATE_V1_RESUMING); - if (ret) { - error_report("%s: Failed to set state RESUMING", vbasedev->name); - if (migration->region.mmaps) { - vfio_region_unmap(&migration->region); - } - } - return ret; -} - static int vfio_load_cleanup(void *opaque) { VFIODevice *vbasedev = opaque; @@ -830,15 +319,6 @@ static int vfio_load_cleanup(void *opaque) return 0; } -static int vfio_v1_load_cleanup(void *opaque) -{ - VFIODevice *vbasedev = opaque; - - vfio_migration_v1_cleanup(vbasedev); - trace_vfio_load_cleanup(vbasedev->name); - return 0; -} - static int vfio_load_state(QEMUFile *f, void *opaque, int version_id) { VFIODevice *vbasedev = opaque; @@ -872,11 +352,7 @@ static int vfio_load_state(QEMUFile *f, void *opaque, int version_id) uint64_t data_size = qemu_get_be64(f); if (data_size) { - if (vbasedev->migration->v2) { - ret = vfio_load_buffer(f, vbasedev, data_size); - } else { - ret = vfio_v1_load_buffer(f, vbasedev, data_size); - } + ret = vfio_load_buffer(f, vbasedev, data_size); if (ret < 0) { return ret; } @@ -908,18 +384,6 @@ static const SaveVMHandlers savevm_vfio_handlers = { .load_state = vfio_load_state, }; -static SaveVMHandlers savevm_vfio_v1_handlers = { - .save_setup = vfio_v1_save_setup, - .save_cleanup = vfio_v1_save_cleanup, - .save_live_pending = vfio_v1_save_pending, - .save_live_iterate = vfio_save_iterate, - .save_live_complete_precopy = vfio_v1_save_complete_precopy, - .save_state = vfio_save_state, - .load_setup = vfio_v1_load_setup, - .load_cleanup = vfio_v1_load_cleanup, - .load_state = vfio_load_state, -}; - /* ---------------------------------------------------------------------- */ static void vfio_vmstate_change(void *opaque, bool running, RunState state) @@ -950,70 +414,12 @@ static void vfio_vmstate_change(void *opaque, bool running, RunState state) mig_state_to_str(new_state)); } -static void vfio_v1_vmstate_change(void *opaque, bool running, RunState state) -{ - VFIODevice *vbasedev = opaque; - VFIOMigration *migration = vbasedev->migration; - uint32_t value, mask; - int ret; - - if (vbasedev->migration->vm_running == running) { - return; - } - - if (running) { - /* - * Here device state can have one of _SAVING, _RESUMING or _STOP bit. - * Transition from _SAVING to _RUNNING can happen if there is migration - * failure, in that case clear _SAVING bit. - * Transition from _RESUMING to _RUNNING occurs during resuming - * phase, in that case clear _RESUMING bit. - * In both the above cases, set _RUNNING bit. - */ - mask = ~VFIO_DEVICE_STATE_MASK; - value = VFIO_DEVICE_STATE_V1_RUNNING; - } else { - /* - * Here device state could be either _RUNNING or _SAVING|_RUNNING. Reset - * _RUNNING bit - */ - mask = ~VFIO_DEVICE_STATE_V1_RUNNING; - - /* - * When VM state transition to stop for savevm command, device should - * start saving data. - */ - if (state == RUN_STATE_SAVE_VM) { - value = VFIO_DEVICE_STATE_V1_SAVING; - } else { - value = 0; - } - } - - ret = vfio_migration_v1_set_state(vbasedev, mask, value); - if (ret) { - /* - * Migration should be aborted in this case, but vm_state_notify() - * currently does not support reporting failures. - */ - error_report("%s: Failed to set device state 0x%x", vbasedev->name, - (migration->device_state_v1 & mask) | value); - if (migrate_get_current()->to_dst_file) { - qemu_file_set_error(migrate_get_current()->to_dst_file, ret); - } - } - vbasedev->migration->vm_running = running; - trace_vfio_v1_vmstate_change(vbasedev->name, running, RunState_str(state), - (migration->device_state_v1 & mask) | value); -} - static void vfio_migration_state_notifier(Notifier *notifier, void *data) { MigrationState *s = data; VFIOMigration *migration = container_of(notifier, VFIOMigration, migration_state); VFIODevice *vbasedev = migration->vbasedev; - int ret; trace_vfio_migration_state_notifier(vbasedev->name, MigrationStatus_str(s->state)); @@ -1023,31 +429,14 @@ static void vfio_migration_state_notifier(Notifier *notifier, void *data) case MIGRATION_STATUS_CANCELLED: case MIGRATION_STATUS_FAILED: bytes_transferred = 0; - if (migration->v2) { - vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_RUNNING, - VFIO_DEVICE_STATE_ERROR); - } else { - ret = vfio_migration_v1_set_state(vbasedev, - ~(VFIO_DEVICE_STATE_V1_SAVING | - VFIO_DEVICE_STATE_V1_RESUMING), - VFIO_DEVICE_STATE_V1_RUNNING); - if (ret) { - error_report("%s: Failed to set state RUNNING", vbasedev->name); - } - } + vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_RUNNING, + VFIO_DEVICE_STATE_ERROR); } } static void vfio_migration_exit(VFIODevice *vbasedev) { - VFIOMigration *migration = vbasedev->migration; - - if (migration->v2) { - g_free(migration->data_buffer); - } else { - vfio_region_exit(&migration->region); - vfio_region_finalize(&migration->region); - } + g_free(vbasedev->migration->data_buffer); g_free(vbasedev->migration); vbasedev->migration = NULL; } @@ -1078,7 +467,6 @@ static int vfio_migration_init(VFIODevice *vbasedev) VFIOMigration *migration; char id[256] = ""; g_autofree char *path = NULL, *oid = NULL; - struct vfio_region_info *info = NULL; uint64_t mig_flags; if (!vbasedev->ops->vfio_get_object) { @@ -1091,53 +479,22 @@ static int vfio_migration_init(VFIODevice *vbasedev) } ret = vfio_migration_query_flags(vbasedev, &mig_flags); - if (!ret) { - /* Migration v2 */ - /* Basic migration functionality must be supported */ - if (!(mig_flags & VFIO_MIGRATION_STOP_COPY)) { - return -EOPNOTSUPP; - } - vbasedev->migration = g_new0(VFIOMigration, 1); - vbasedev->migration->device_state = VFIO_DEVICE_STATE_RUNNING; - vbasedev->migration->data_buffer_size = VFIO_MIG_DATA_BUFFER_SIZE; - vbasedev->migration->data_buffer = - g_malloc0(vbasedev->migration->data_buffer_size); - vbasedev->migration->data_fd = -1; - vbasedev->migration->v2 = true; - } else { - /* Migration v1 */ - ret = vfio_get_dev_region_info(vbasedev, - VFIO_REGION_TYPE_MIGRATION_DEPRECATED, - VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED, - &info); - if (ret) { - return ret; - } - - vbasedev->migration = g_new0(VFIOMigration, 1); - vbasedev->migration->device_state_v1 = VFIO_DEVICE_STATE_V1_RUNNING; - vbasedev->migration->vm_running = runstate_is_running(); - - ret = vfio_region_setup(obj, vbasedev, &vbasedev->migration->region, - info->index, "migration"); - if (ret) { - error_report("%s: Failed to setup VFIO migration region %d: %s", - vbasedev->name, info->index, strerror(-ret)); - goto err; - } - - if (!vbasedev->migration->region.size) { - error_report("%s: Invalid zero-sized VFIO migration region %d", - vbasedev->name, info->index); - ret = -EINVAL; - goto err; - } + if (ret) { + return ret; + } - g_free(info); + /* Basic migration functionality must be supported */ + if (!(mig_flags & VFIO_MIGRATION_STOP_COPY)) { + return -EOPNOTSUPP; } + vbasedev->migration = g_new0(VFIOMigration, 1); migration = vbasedev->migration; migration->vbasedev = vbasedev; + migration->device_state = VFIO_DEVICE_STATE_RUNNING; + migration->data_buffer_size = VFIO_MIG_DATA_BUFFER_SIZE; + migration->data_buffer = g_malloc0(migration->data_buffer_size); + migration->data_fd = -1; oid = vmstate_if_get_id(VMSTATE_IF(DEVICE(obj))); if (oid) { @@ -1147,28 +504,16 @@ static int vfio_migration_init(VFIODevice *vbasedev) } strpadcpy(id, sizeof(id), path, '\0'); - if (migration->v2) { - register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, - &savevm_vfio_handlers, vbasedev); - - migration->vm_state = qdev_add_vm_change_state_handler( - vbasedev->dev, vfio_vmstate_change, vbasedev); - } else { - register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, - &savevm_vfio_v1_handlers, vbasedev); - - migration->vm_state = qdev_add_vm_change_state_handler( - vbasedev->dev, vfio_v1_vmstate_change, vbasedev); - } + register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, &savevm_vfio_handlers, + vbasedev); + migration->vm_state = qdev_add_vm_change_state_handler(vbasedev->dev, + vfio_vmstate_change, + vbasedev); migration->migration_state.notify = vfio_migration_state_notifier; add_migration_state_change_notifier(&migration->migration_state); - return 0; -err: - g_free(info); - vfio_migration_exit(vbasedev); - return ret; + return 0; } /* ---------------------------------------------------------------------- */ diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 9ef84e24b2..cbd6590dd9 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -150,21 +150,15 @@ vfio_display_edid_write_error(void) "" # migration.c vfio_migration_probe(const char *name) " (%s)" vfio_migration_set_state(const char *name, const char *state) " (%s) state %s" -vfio_migration_v1_set_state(const char *name, uint32_t state) " (%s) state %d" vfio_vmstate_change(const char *name, int running, const char *reason, const char *dev_state) " (%s) running %d reason %s device state %s" -vfio_v1_vmstate_change(const char *name, int running, const char *reason, uint32_t dev_state) " (%s) running %d reason %s device state %d" vfio_migration_state_notifier(const char *name, const char *state) " (%s) state %s" vfio_save_setup(const char *name) " (%s)" vfio_save_cleanup(const char *name) " (%s)" -vfio_save_buffer(const char *name, uint64_t data_offset, uint64_t data_size, uint64_t pending) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64" pending 0x%"PRIx64 -vfio_update_pending(const char *name, uint64_t pending) " (%s) pending 0x%"PRIx64 vfio_save_device_config_state(const char *name) " (%s)" vfio_save_pending(const char *name, uint64_t precopy, uint64_t postcopy) " (%s) precopy 0x%"PRIx64" postcopy 0x%"PRIx64 -vfio_save_iterate(const char *name, int data_size) " (%s) data_size %d" vfio_save_complete_precopy(const char *name) " (%s)" vfio_load_device_config_state(const char *name) " (%s)" vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 -vfio_v1_load_state_device_data(const char *name, uint64_t data_offset, uint64_t data_size) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64 vfio_load_state_device_data(const char *name, uint64_t data_size) " (%s) size 0x%"PRIx64 vfio_load_cleanup(const char *name) " (%s)" vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitmap_size, uint64_t start) "container fd=%d, iova=0x%"PRIx64" size= 0x%"PRIx64" bitmap_size=0x%"PRIx64" start=0x%"PRIx64 diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 2ec3346fea..76d470178f 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -61,16 +61,11 @@ typedef struct VFIORegion { typedef struct VFIOMigration { struct VFIODevice *vbasedev; VMChangeStateEntry *vm_state; - VFIORegion region; - uint32_t device_state_v1; - int vm_running; Notifier migration_state; - uint64_t pending_bytes; enum vfio_device_mig_state device_state; int data_fd; void *data_buffer; size_t data_buffer_size; - bool v2; } VFIOMigration; typedef struct VFIOAddressSpace { From patchwork Thu Nov 3 16:16:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0287BC4332F for ; Thu, 3 Nov 2022 16:23:31 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcxW-0007Yj-E1; Thu, 03 Nov 2022 12:20:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcve-0001Zo-Fv; Thu, 03 Nov 2022 12:18:56 -0400 Received: from mail-dm6nam04on2088.outbound.protection.outlook.com ([40.107.102.88] helo=NAM04-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcvY-0008HE-Pe; Thu, 03 Nov 2022 12:18:51 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=m53iRSUI3RDI0ZXc927HsjmCxm6S1aj8p6QIRwmdCWNMARqMElQfLmU/JA0SdcFbokVElz9jB27Xoo5ET1z2A76Xv1Ecql+dV3S7L8QX5NtPjy5o3qs37N4JeMF9qFVaixEl1yoCixS6X2IYrixnaKBi3wiEt9a+xub6YBUoU2Hu5L0X35Gl9iQWgmI89RpL3yAsYjcM8rc4GwZltWBhl27ncIUZOk+fY2iOyhaTL/Ut9e898KHIqw3XPr5YOXiEcS2IFiBMJtL6rv22rg7xYiKBrngzwJfJtCupbbwmLtZUean7beiay5Z/C5AbovdOJaVMWipaeTaCLoqkvR5I5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UWB92MjN1yYMLJTc0eJ/jgyz5tTb6+lW9x+GXrTLhAQ=; b=YOwoqS3LskBdCrLThdOO+pAipjXCIaBCsUfFruB/UxqsLqCF4excrqPrCfJmWb74ZGFhkTdWm8ySXCEDW2D6Kq/1OJ8bl61rPIesQnjkKrfx4YY0wjpzRKNPSK3kpdbjmr7VsLSG6JcNBRTv4tsxFAumDO3D9z4RKMq5RAiCxrK9PTFJ1D6cLvnLPoowbk8RZUn49qYRhw5tRBoqKMVaOD6DiSjbqvCPwA/Gdhw5ablJ7oxMt/lH8LfwrrvXNQwyBJh5qHXtAD3r35zxQtn2jTKg3dlA8LVZ5xczl5v/cZx8kAxQ8Jow1GJr8UcfciMSAzwS8bMm1ZTL2+L1jPYzRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UWB92MjN1yYMLJTc0eJ/jgyz5tTb6+lW9x+GXrTLhAQ=; b=GPoSXo8fJwUO8BXq4uErHOPpAr12Hthw8FtLfnaxPZYNxhmzuwNH0UrwZfvejtFuxbEb21I8b73zr62/5V+ycd8wKRM+12gu0AKjFC6oqOQ4PvNyH5iLwlV4uDVpa4mtCVqx2Tv5+Jtz6VOhnkafdXwmWHr0ptfmUtJ2K5835XFPf1T7t2NnU8MNc6bCg2kkMeBdrGrZq9e1zFk5hW1bvlBQv0p+hQ5CQIj6cYmkwTLOnWpAYbQsZ8smx2YIP2wIwQEBswFSLyyjMcYcWvYrxR4Ij/pbEIcfPx20BRz33xOzWm6gUnqUaWvSvTJaKmBFqjnT0thy0PwSULWFlNB49A== Received: from MW4P222CA0024.NAMP222.PROD.OUTLOOK.COM (2603:10b6:303:114::29) by MN2PR12MB4408.namprd12.prod.outlook.com (2603:10b6:208:26c::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.16; Thu, 3 Nov 2022 16:18:30 +0000 Received: from CO1NAM11FT074.eop-nam11.prod.protection.outlook.com (2603:10b6:303:114:cafe::46) by MW4P222CA0024.outlook.office365.com (2603:10b6:303:114::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.18 via Frontend Transport; Thu, 3 Nov 2022 16:18:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CO1NAM11FT074.mail.protection.outlook.com (10.13.174.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:18:29 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:18:16 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:18:16 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:18:09 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 14/17] vfio/migration: Reset device if setting recover state fails Date: Thu, 3 Nov 2022 18:16:17 +0200 Message-ID: <20221103161620.13120-15-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT074:EE_|MN2PR12MB4408:EE_ X-MS-Office365-Filtering-Correlation-Id: 6579aae6-daf4-4d24-9674-08dabdb70bb6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: i8x0TSHZHPkYhtVwTtTwi9IPspwkvPd8WwIaendN7wHck5B4K3qDVcx5VbPeH5Ks9l5vHhhb76NUeI159qXmtIfbgztmf5VAlegSGaCXF/hEQqH4NChnS30o0DqrAlRBmsr3nxnCGMjLEEVj1ZqJTInHrO14e5yFVpW3DNJ2xYZtTDLe8jkEcGSAW8NQQFg3XBuBS4Ktvk7fmRtViqgKENo5Xa9f3+FVX6xeGAocu5CHmEBhflV9PLeBAL6+RhaiWazwXrXEawnu3xjf1zkVJMYY0E7DOrYBbEVKB1pVo326hsUPAuQ+f+/BwNXqOtBcJ5MNiGcUtos7QJWbvNbdChQ0olze/Szea8y+ELOmCWBF33RWwDt9+guElwfO7WpDEXkrRC85e58HSkHx4HNb25PoniH4XpvL2aZD+xsaDM8v5oPzTKLttKr2hpKRXn3YI3wDASupPG59Lnn9VgXwV1LVnXvf8nPvzKcCSiSj4HMam88mG+9syiBowsO8KmNZiWamGqWcQB/Lopjwync1Qtba0a6DPctrNoVe7HoDjjutk4qX2TddOQ1VmBDgqhj9uRAEgy+7cncaqjsI7nhFqMpHg/y3CBPfZF5twdjmjR0b+WQAaVmy0A6idRe5lejaynncopNo6daO7ELlxm3h/8XctgHxoVollnPJfZ+vKJc0Nb71rlOfsUTHF9Xrfc3ZshHuu+Oca0dMsmHhcdq3vGmRW5Eslw9CVEczCICbNTiK26F/dB3YqsWJSY3rSk+1fpwXIiLT5pIrbAdr2e3T3A== X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(136003)(346002)(396003)(376002)(451199015)(46966006)(40470700004)(36840700001)(40480700001)(4326008)(70206006)(70586007)(41300700001)(5660300002)(8936002)(7416002)(8676002)(6666004)(36860700001)(478600001)(2906002)(82310400005)(26005)(54906003)(6916009)(316002)(7696005)(40460700003)(86362001)(356005)(426003)(83380400001)(47076005)(186003)(1076003)(82740400003)(2616005)(36756003)(7636003)(336012); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:18:29.8589 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6579aae6-daf4-4d24-9674-08dabdb70bb6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT074.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4408 Received-SPF: softfail client-ip=40.107.102.88; envelope-from=avihaih@nvidia.com; helo=NAM04-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org If vfio_migration_set_state() fails to set the device in the requested state it tries to put it in a recover state. If setting the device in the recover state fails as well, hw_error is triggered and the VM is aborted. To improve user experience and avoid VM data loss, reset the device with VFIO_RESET_DEVICE instead of aborting the VM. Signed-off-by: Avihai Horon --- hw/vfio/migration.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index f8c3228314..e8068b9147 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -92,8 +92,18 @@ static int vfio_migration_set_state(VFIODevice *vbasedev, mig_state->device_state = recover_state; if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { - hw_error("%s: Failed setting device in recover state, err: %s", - vbasedev->name, strerror(errno)); + error_report( + "%s: Failed setting device in recover state, err: %s. Resetting device", + vbasedev->name, strerror(errno)); + + if (ioctl(vbasedev->fd, VFIO_DEVICE_RESET)) { + hw_error("%s: Failed resetting device, err: %s", vbasedev->name, + strerror(errno)); + } + + migration->device_state = VFIO_DEVICE_STATE_RUNNING; + + return -1; } migration->device_state = recover_state; From patchwork Thu Nov 3 16:16:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030544 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 879D9C4332F for ; Thu, 3 Nov 2022 16:22:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcwi-0005V2-0q; Thu, 03 Nov 2022 12:20:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcva-0001Ys-Ko; Thu, 03 Nov 2022 12:18:56 -0400 Received: from mail-mw2nam10on2076.outbound.protection.outlook.com ([40.107.94.76] helo=NAM10-MW2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcvY-0008IS-Oc; Thu, 03 Nov 2022 12:18:50 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Hi7inmLrzsQkqiWRYlm54N8AssePMpSS16rnuO09wOpmXOAHU662jQJUVnv7FHJUGriiUDljiVOg0Ta2AfpvoV0DXHkMh3lT7TMGYRsknsIg1DxvQ0zygRtjo8D08AFoee2N28hzq3TRAijNBkTOyJxFw2b4K5b2xu+zqjcI9Lzq2a9q7VUqHzFLQ2WuyhCDMzT7b2G6aBpH7USvMSjfEnQbXqbdf9pcDjKy5LNLZuXOvDmkgsT3WrOVWN108uyfsIzmfB8DI3MLC5WshFTVxV76eHiNBWzOFzq8eHg1jmT5fFnOLdlkpz0lBC2DcP3ER5m7pil8FGycssxiiNUBlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=67k9XJXT/vlKcWf+8i7haH9uebtBGQ3rwwwCyvPBOtw=; b=JqYtvHI/HgmnEJpNvoZI2IUec91H4PFaF11n/NA4+pOqgJdxZzFzk9taYTw0xJJo70vVe696PDfELzmIaaXQvFZBH8BV43C7xFG9lFLqkjp+o7daOx2h9HfmGedYSJjrT/KW6FY6D0Gav5/ytIxjaATpGttOXM3PDj0FcOwO0ugD4RqzdnGlMW6pLTRv7Jd9E2+LsVOQD+nLe1mf9vpsXSciQC0Z2Q18+tS7ro4mEAj2NWv75Ra1lBZSLI0HX5fju7ENUIBWoI/E/L0pyJJIk6d4i9DuMs0IMF2gE/m6JbKvBU3FRGHQtqXsu6FLxvtXyE6jlhQHbWFqtCAZQT+teg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=67k9XJXT/vlKcWf+8i7haH9uebtBGQ3rwwwCyvPBOtw=; b=nwdjuvaFg8FWkD/QLo6mFjo1BMcesZYrhgGS5Hx8cQGjqwzJVWvEItLOlBKPZjtkAMYY2CIxaWeZzL2hZDGI/OwsoNB5KKVKTqjMNK6KmMH9wOyb9L9saaY6nhGKdGwSbaQcBzG/gLq8yPDp7la3N+4Kz3d004DpJ5tQTGGesrvOAuV2LN+h4Jvo7ErenoDGCu8ww+vSrvLe9ZzJ7e6HUY/xi8YUqJqMRmkI4+qKtk0ALXyWyoDQbmGjRto+LsEmTSDShf9zEgHK3RPlGmSxkyEecgYpzJ2X2XG7BTglx8gy6YwB3LL1+R1fl2CE1yWMg4AS4AtlGT2YNP8xK2Z+nQ== Received: from MW4PR03CA0137.namprd03.prod.outlook.com (2603:10b6:303:8c::22) by DM6PR12MB4434.namprd12.prod.outlook.com (2603:10b6:5:2ad::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22; Thu, 3 Nov 2022 16:18:36 +0000 Received: from CO1NAM11FT092.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8c:cafe::53) by MW4PR03CA0137.outlook.office365.com (2603:10b6:303:8c::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22 via Frontend Transport; Thu, 3 Nov 2022 16:18:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CO1NAM11FT092.mail.protection.outlook.com (10.13.175.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:18:35 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:18:24 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:18:24 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:18:17 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 15/17] vfio: Alphabetize migration section of VFIO trace-events file Date: Thu, 3 Nov 2022 18:16:18 +0200 Message-ID: <20221103161620.13120-16-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT092:EE_|DM6PR12MB4434:EE_ X-MS-Office365-Filtering-Correlation-Id: f496a91b-f4d5-4791-607e-08dabdb70f5e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qYHIijfokfKXRKyyJYb73FKpDe2E5cAiLSC3IXkPI6n6cYkpVeQBLqFGNpiIuhazV4m9bAmPsOaoLNz5syZsHsPnVHeWlAgCCGaQUHu5n3bmWr36lekrYg3P8g4lpCTzhMcx5X36G57oMP/xBaAaHbVWV5MK4gLE6oQ1Qi2H2OlW+QNN55bO8ljBv0D7MVsCubm/mvmTNMJWVPqWE+L9s7bQCPxMVlsYBCUCmMcJdmPyGFIcVQr251nJmh3PpMUVXKcQv63Rl+EtjrluMSWan1DRayyO8WCR6YqqhteHsPexnmxKUJ94tb/PtXKplgJk4WoE1i5SmJk55NM0gVritd31cvwsOSK3UCRmKCVyMbJ1xVLcVAkHIPQAyq4uluY1SENloFpPJ1HTuJmarbPpxOZo2RnOd2Mj23APkUJLraaMeYd6hFBiwbMgN6+gDvQkU1aauPJbzGRAMgAzM1J7hUuUX/9bC0dLevNi4SqWe6yaHfPjPi3xJCrsjrl950QwvWfz4jgVtfkPKMiYzo0qwj5d8XPllkZRGOwzJ66H0gehU6fj3rbIOxXNvWExnmsVwAQ8xwaxvotQMDaDJF0o7wPko7Yml+zyX9rIiUGBrv6hXem6n7vOYx0iUmtrRf8ROsIR1/+GHEykOs00n8XpG/VRPMdlb26pOJNCwTZJHKslOtDTLrPRSS9yYqpY8TWiBfmuLd29cWLI66j4kEOpuu6uQCGjnV925YWkjLAu9T7YmUZrqn65KNj3gZMb/8FEir6kbv7weWh3YHJSlntcyg== X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(396003)(376002)(136003)(346002)(39860400002)(451199015)(36840700001)(40470700004)(46966006)(2616005)(186003)(82740400003)(26005)(7696005)(36860700001)(426003)(336012)(47076005)(40460700003)(2906002)(7416002)(40480700001)(1076003)(316002)(8936002)(54906003)(70586007)(6916009)(478600001)(82310400005)(4326008)(5660300002)(41300700001)(8676002)(70206006)(356005)(83380400001)(36756003)(7636003)(86362001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:18:35.9917 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f496a91b-f4d5-4791-607e-08dabdb70f5e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT092.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4434 Received-SPF: softfail client-ip=40.107.94.76; envelope-from=avihaih@nvidia.com; helo=NAM10-MW2-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sort the migration section of VFIO trace events file alphabetically and move two misplaced traces to common.c section. Signed-off-by: Avihai Horon --- hw/vfio/trace-events | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index cbd6590dd9..59f48af4ee 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -119,6 +119,8 @@ vfio_region_sparse_mmap_header(const char *name, int index, int nr_areas) "Devic vfio_region_sparse_mmap_entry(int i, unsigned long start, unsigned long end) "sparse entry %d [0x%lx - 0x%lx]" vfio_get_dev_region(const char *name, int index, uint32_t type, uint32_t subtype) "%s index %d, %08x/%0x8" vfio_dma_unmap_overflow_workaround(void) "" +vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitmap_size, uint64_t start) "container fd=%d, iova=0x%"PRIx64" size= 0x%"PRIx64" bitmap_size=0x%"PRIx64" start=0x%"PRIx64 +vfio_iommu_map_dirty_notify(uint64_t iova_start, uint64_t iova_end) "iommu dirty @ 0x%"PRIx64" - 0x%"PRIx64 # platform.c vfio_platform_base_device_init(char *name, int groupid) "%s belongs to group #%d" @@ -148,19 +150,17 @@ vfio_display_edid_update(uint32_t prefx, uint32_t prefy) "%ux%u" vfio_display_edid_write_error(void) "" # migration.c +vfio_load_cleanup(const char *name) " (%s)" +vfio_load_device_config_state(const char *name) " (%s)" +vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 +vfio_load_state_device_data(const char *name, uint64_t data_size) " (%s) size 0x%"PRIx64 vfio_migration_probe(const char *name) " (%s)" vfio_migration_set_state(const char *name, const char *state) " (%s) state %s" -vfio_vmstate_change(const char *name, int running, const char *reason, const char *dev_state) " (%s) running %d reason %s device state %s" vfio_migration_state_notifier(const char *name, const char *state) " (%s) state %s" -vfio_save_setup(const char *name) " (%s)" +vfio_save_block(const char *name, int data_size) " (%s) data_size %d" vfio_save_cleanup(const char *name) " (%s)" +vfio_save_complete_precopy(const char *name) " (%s)" vfio_save_device_config_state(const char *name) " (%s)" vfio_save_pending(const char *name, uint64_t precopy, uint64_t postcopy) " (%s) precopy 0x%"PRIx64" postcopy 0x%"PRIx64 -vfio_save_complete_precopy(const char *name) " (%s)" -vfio_load_device_config_state(const char *name) " (%s)" -vfio_load_state(const char *name, uint64_t data) " (%s) data 0x%"PRIx64 -vfio_load_state_device_data(const char *name, uint64_t data_size) " (%s) size 0x%"PRIx64 -vfio_load_cleanup(const char *name) " (%s)" -vfio_get_dirty_bitmap(int fd, uint64_t iova, uint64_t size, uint64_t bitmap_size, uint64_t start) "container fd=%d, iova=0x%"PRIx64" size= 0x%"PRIx64" bitmap_size=0x%"PRIx64" start=0x%"PRIx64 -vfio_iommu_map_dirty_notify(uint64_t iova_start, uint64_t iova_end) "iommu dirty @ 0x%"PRIx64" - 0x%"PRIx64 -vfio_save_block(const char *name, int data_size) " (%s) data_size %d" +vfio_save_setup(const char *name) " (%s)" +vfio_vmstate_change(const char *name, int running, const char *reason, const char *dev_state) " (%s) running %d reason %s device state %s" From patchwork Thu Nov 3 16:16:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030546 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A462AC43217 for ; Thu, 3 Nov 2022 16:22:56 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcxV-0007Sa-Jm; Thu, 03 Nov 2022 12:20:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcve-0001Zp-Gj; Thu, 03 Nov 2022 12:18:56 -0400 Received: from mail-mw2nam10on2083.outbound.protection.outlook.com ([40.107.94.83] helo=NAM10-MW2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcvY-0008If-QY; Thu, 03 Nov 2022 12:18:52 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZOdZKHiRD0N4KsL5jIYbu03Gp4L6KRvzUYbfxEDgMS2yz4VzdRnCH2Sy3kI3hOe4zZC12XPgNj1gGcpHXeNzzSUEgXXxKdxYHV4zn6Kolx8uaraHjNngq72pFFtnZvJ9dwbLnl3E0CLRmDP+bkVj1olRCzCobn1a0/Z7SaYsbXDa6gErb7iUoJe0OFySWwz8CFniwMi9BdzBZYT0S58g2ZguE4vL0Ua6rU4srXsZgDH4v6Yxg819Iw4QAKX0NSrE3rysT/QFtuouuslFnGJdttwn/s+V5FB9O7SdbJJw45gRhLuJhNhxj7pQ+oStdAvjTTjkywfjOmyryHK77hU0ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GoMNWay27aQJm1B0UbLOVlQDLewk2rcI3IG58Ufnsl0=; b=IGxFBlJ5J2IvKpyFzveraT2OTG99nXpzVjh9i1tjli7vKtCnKJybCu0z6yLU37VWfrSqBmQRhSJfN4wGhdzt6p+FvD40Qjf0uUJHSMcrrENFfXZfc4qBgSI+JBmSjOUBh5nYS3BJQV0IMLkerLuulWwySTCOU7gFxXmWz8l8+XykaN5yEozfu3HHMVntEft0o2sEYWHHUsy+bc/IH7MAWAUCzSGpmHWS3cBrT0zdXxX0U+9IyvXKfQrkrngzc8ryy+GZ/xww2CPuyznX3bRUASLyyf5Bl4cvYaDi0jJzN4SFQBVQfb5O1NwJ/tKyEViAH+zb7H9BiX0/Gxm3IuMlbg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GoMNWay27aQJm1B0UbLOVlQDLewk2rcI3IG58Ufnsl0=; b=MQKaz2sVq6uI5tOsf7+1kz2kLXmVCo5A+3V4+YRI6zLk112iPt25yrqik7FDrDxW2EZGxRGiEHmQKOeQAfFUmxrE2vOAj5TVDH6D3FEcu1dQEuc80iTvd8i1H0iIQKHjfUQWN/lePU/qfvzuX8ozupkwgaCZfk0L7bn0axIrGAZ1kkk2WHugWW7jZsbfZFk2DjjxYwJ9C1X3gHLK9Bq3HiO948LpocN7MzHH0f6dPlIcJzBnCL3xWRgFJpTanZwwkMQY0eNC8GmR7QGY0TOacNtfYV1Dvdy6AgQ4R0uOH9JzyJbsNI9XswXQ6yV0qY1Ea5fK0YWMHNp/FjNeosjSdg== Received: from MW4PR03CA0140.namprd03.prod.outlook.com (2603:10b6:303:8c::25) by CH0PR12MB5172.namprd12.prod.outlook.com (2603:10b6:610:bb::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22; Thu, 3 Nov 2022 16:18:41 +0000 Received: from CO1NAM11FT092.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8c:cafe::e) by MW4PR03CA0140.outlook.office365.com (2603:10b6:303:8c::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.21 via Frontend Transport; Thu, 3 Nov 2022 16:18:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CO1NAM11FT092.mail.protection.outlook.com (10.13.175.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:18:41 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:18:32 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:18:31 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:18:24 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 16/17] docs/devel: Align vfio-migration docs to VFIO migration v2 Date: Thu, 3 Nov 2022 18:16:19 +0200 Message-ID: <20221103161620.13120-17-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT092:EE_|CH0PR12MB5172:EE_ X-MS-Office365-Filtering-Correlation-Id: 6d305f41-0e3e-4461-189b-08dabdb71271 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IEoDKXLgbaVWGzoyLULiwrINreu1KRaCcyw7rqLLweCiIO2rpt6LV27J8EpAvQRpJQtWOIPUHd8DDshUZ7OjWuCaZO/8lRBlu+Bxy8qfaqlqcpY5uUkINsZVuVZm3/xAw4UHhzjp2DTluL8OY6Er0ctXwPtpqnhhTOPgW4/SeSZqwQNFypw0ljFe1ce4+ODHTtqt2tIK1K4QHijHGdBzwNsxAPuJrVzitmqIfzQivO0qRJM9V/OQ8GP6+ylR5+ZDcdcBmYAazwTg43kYCr+QGDux65pJKwCEgQr9Stqc1bQgCEdwW+mJkP7oTaau0YnJjAaDprIOtaGIC0HcqRo/eifdoV32165URzw/67OH9esZ2QobEDd4TA6bZJ6dV1lLzOCTvC+11YuDNaDfzMCsv5WsNuu0ZB+fxkNYPXKem9aKK4Zxz2qVmaedpkL+mVfifq7bTML5zJPbYJpxcV3JWnxkVVfn8AGGUw5gTEvVM5/ne7BGy2wJpntqoDn8lD8nIluhpSgSM8/BEtUt9KMTGc7mfwt5dukCB+Q5E339k789vgXrFTdhhQKiTsVeN4kSyJC/AUUKmi4L7I6n01GkdVUfz7s9YFTqzgy59fMDAc/xVYGwgqkGilW5OZiB4j1Sz82UNgdRG6QTlus2v23oONSOZsUVp1EDTGEhol1oLMe+naNuBLkld8roy/INkZiQwIgn8jno2x4pK/Il00okqa4BiqaDPbCvQWBxeD3GZrfynfpoCs6eB8sS/D9cIhwiWQIiRdnjzSCGLRKuqiXtgQ== X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(376002)(346002)(136003)(396003)(451199015)(36840700001)(40470700004)(46966006)(7636003)(356005)(40480700001)(82310400005)(26005)(40460700003)(36756003)(86362001)(41300700001)(8676002)(70206006)(4326008)(70586007)(186003)(2616005)(336012)(8936002)(7416002)(5660300002)(478600001)(1076003)(7696005)(54906003)(6916009)(316002)(6666004)(36860700001)(82740400003)(2906002)(426003)(47076005)(83380400001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:18:41.1475 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6d305f41-0e3e-4461-189b-08dabdb71271 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT092.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5172 Received-SPF: softfail client-ip=40.107.94.83; envelope-from=avihaih@nvidia.com; helo=NAM10-MW2-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Align the vfio-migration documentation to VFIO migration protocol v2. Signed-off-by: Avihai Horon --- docs/devel/vfio-migration.rst | 68 ++++++++++++++++------------------- 1 file changed, 30 insertions(+), 38 deletions(-) diff --git a/docs/devel/vfio-migration.rst b/docs/devel/vfio-migration.rst index 9ff6163c88..ad991b7eeb 100644 --- a/docs/devel/vfio-migration.rst +++ b/docs/devel/vfio-migration.rst @@ -7,46 +7,39 @@ the guest is running on source host and restoring this saved state on the destination host. This document details how saving and restoring of VFIO devices is done in QEMU. -Migration of VFIO devices consists of two phases: the optional pre-copy phase, -and the stop-and-copy phase. The pre-copy phase is iterative and allows to -accommodate VFIO devices that have a large amount of data that needs to be -transferred. The iterative pre-copy phase of migration allows for the guest to -continue whilst the VFIO device state is transferred to the destination, this -helps to reduce the total downtime of the VM. VFIO devices can choose to skip -the pre-copy phase of migration by returning pending_bytes as zero during the -pre-copy phase. +Migration of VFIO devices currently consists of a single stop-and-copy phase. +During the stop-and-copy phase the guest is stopped and the entire VFIO device +data is transferred to the destination. + +The pre-copy phase of migration is currently not supported for VFIO devices, +so VFIO device data is not transferred during pre-copy phase. A detailed description of the UAPI for VFIO device migration can be found in -the comment for the ``vfio_device_migration_info`` structure in the header -file linux-headers/linux/vfio.h. +the comment for the ``vfio_device_mig_state`` structure in the header file +linux-headers/linux/vfio.h. VFIO implements the device hooks for the iterative approach as follows: -* A ``save_setup`` function that sets up the migration region and sets _SAVING - flag in the VFIO device state. +* A ``save_setup`` function that sets up migration on the source. -* A ``load_setup`` function that sets up the migration region on the - destination and sets _RESUMING flag in the VFIO device state. +* A ``load_setup`` function that sets the VFIO device on the destination in + _RESUMING state. * A ``save_live_pending`` function that reads pending_bytes from the vendor driver, which indicates the amount of data that the vendor driver has yet to save for the VFIO device. -* A ``save_live_iterate`` function that reads the VFIO device's data from the - vendor driver through the migration region during iterative phase. - * A ``save_state`` function to save the device config space if it is present. -* A ``save_live_complete_precopy`` function that resets _RUNNING flag from the - VFIO device state and iteratively copies the remaining data for the VFIO - device until the vendor driver indicates that no data remains (pending bytes - is zero). +* A ``save_live_complete_precopy`` function that sets the VFIO device in + _STOP_COPY state and iteratively copies the data for the VFIO device until + the vendor driver indicates that no data remains. * A ``load_state`` function that loads the config section and the data - sections that are generated by the save functions above + sections that are generated by the save functions above. * ``cleanup`` functions for both save and load that perform any migration - related cleanup, including unmapping the migration region + related cleanup. The VFIO migration code uses a VM state change handler to change the VFIO @@ -71,13 +64,13 @@ tracking can identify dirtied pages, but any page pinned by the vendor driver can also be written by the device. There is currently no device or IOMMU support for dirty page tracking in hardware. -By default, dirty pages are tracked when the device is in pre-copy as well as -stop-and-copy phase. So, a page pinned by the vendor driver will be copied to -the destination in both phases. Copying dirty pages in pre-copy phase helps -QEMU to predict if it can achieve its downtime tolerances. If QEMU during -pre-copy phase keeps finding dirty pages continuously, then it understands -that even in stop-and-copy phase, it is likely to find dirty pages and can -predict the downtime accordingly. +By default, dirty pages are tracked during pre-copy as well as stop-and-copy +phase. So, a page pinned by the vendor driver will be copied to the destination +in both phases. Copying dirty pages in pre-copy phase helps QEMU to predict if +it can achieve its downtime tolerances. If QEMU during pre-copy phase keeps +finding dirty pages continuously, then it understands that even in stop-and-copy +phase, it is likely to find dirty pages and can predict the downtime +accordingly. QEMU also provides a per device opt-out option ``pre-copy-dirty-page-tracking`` which disables querying the dirty bitmap during pre-copy phase. If it is set to @@ -111,23 +104,22 @@ Live migration save path | migrate_init spawns migration_thread Migration thread then calls each device's .save_setup() - (RUNNING, _SETUP, _RUNNING|_SAVING) + (RUNNING, _SETUP, _RUNNING) | - (RUNNING, _ACTIVE, _RUNNING|_SAVING) + (RUNNING, _ACTIVE, _RUNNING) If device is active, get pending_bytes by .save_live_pending() If total pending_bytes >= threshold_size, call .save_live_iterate() - Data of VFIO device for pre-copy phase is copied Iterate till total pending bytes converge and are less than threshold | On migration completion, vCPU stops and calls .save_live_complete_precopy for - each active device. The VFIO device is then transitioned into _SAVING state - (FINISH_MIGRATE, _DEVICE, _SAVING) + each active device. The VFIO device is then transitioned into _STOP_COPY state + (FINISH_MIGRATE, _DEVICE, _STOP_COPY) | For the VFIO device, iterate in .save_live_complete_precopy until pending data is 0 - (FINISH_MIGRATE, _DEVICE, _STOPPED) + (FINISH_MIGRATE, _DEVICE, _STOP) | - (FINISH_MIGRATE, _COMPLETED, _STOPPED) + (FINISH_MIGRATE, _COMPLETED, _STOP) Migraton thread schedules cleanup bottom half and exits Live migration resume path @@ -136,7 +128,7 @@ Live migration resume path :: Incoming migration calls .load_setup for each device - (RESTORE_VM, _ACTIVE, _STOPPED) + (RESTORE_VM, _ACTIVE, _STOP) | For each device, .load_state is called for that device section data (RESTORE_VM, _ACTIVE, _RESUMING) From patchwork Thu Nov 3 16:16:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avihai Horon X-Patchwork-Id: 13030541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EF600C4332F for ; Thu, 3 Nov 2022 16:22:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqcwi-0005e8-VZ; Thu, 03 Nov 2022 12:20:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcvh-0001cK-BU; Thu, 03 Nov 2022 12:18:59 -0400 Received: from mail-sn1anam02on2050.outbound.protection.outlook.com ([40.107.96.50] helo=NAM02-SN1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqcvf-0008K5-1j; Thu, 03 Nov 2022 12:18:57 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YjLd67zYEaEbqrIhjhIeYUkA7y+e0Oaw0fNmX4LdsilQLVwEuN+fls1deGCyVgpD1w8KpZdxZDTysG+Lz03ahF/YOqk45Ew/iJKe7g/hPN+gx89ydalfsd0zeRI+e8FrruygTzlmpvhCYt2dPwqnAyzelBElYytPclgDoDbFume4EIuVcuNgLAxj4Em0M4WmWbuTnAv7uzUNcr4ic3Ukrs9hrLjc01wsihPGneyuXeu2w8yLju9r6srafa2t8VgXtuTUkjfe5WIK9v+RJkawHrm9+uUUBJgYiFmPdnmfa7NqS1aRBs35mF0sfGL8ZJAkzLeyU09mC3VmVGCLEQ9CTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rShIJnG7YAGu3zdDEePp439THE+u/jJkUvwts6qsDg8=; b=VEwZ/LjDytYDvt0A/8wzTcyRsfo3g5VMugbct9l+D82iPdE+krRzKRhnM+1FTHVZtJYoCs3yAI+MNNFyKQ20QGmeCfdEXI1XEUCAvySO5e2/PPj/EMDoTpBFW14gmkYrybFlRk6no4ZY355w4NffpffyAxPwuoc46EGwODgdaaW5ByFoRxmVaOwDDcqoz5jSZmx4nHxw8nAw8WAP5O1RrR+1Nj5OHdS61PLM0AuMgcq3j4ILWRSISldP6/E4LLuIrF/RF35r8/+ks5wHDG2QVeGHZIuDNMnC1YEo/5bojq1SSMJsved9+ZgmE6crAOVRnZX6AcZWzXVba7zZScNJMw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rShIJnG7YAGu3zdDEePp439THE+u/jJkUvwts6qsDg8=; b=MLk490j0FJ+WPbKwvwphUp7BmH3qZDQ4speAqGGCwTZTocjwi2r6MFTfyMc/n7N/hOwsfYXp/MHlSW6q1ntOwu4PcPmHT2s8nbpGbAT7edLxo47oqVPKVyNgZw+GHuQqcgbDW96zf08or4e02eAJCQ7zXmaqhsw7WWQAaS/HLq7labWRtpgBw6RMYS/scE/KyHAXYJyBhj5phc7GIakvv3RzxMRratMzrDGGT6+K0d7NuTB1EAxgXX/lupq9lY69z0mg3iSCWxqmXwmpfZtibHpnWYJw0ZK46JxST9Tuju8oZloD88chrnzc7jeD+LgVGQ95mKGRsbvyyNhJcwj3nw== Received: from MW4PR03CA0136.namprd03.prod.outlook.com (2603:10b6:303:8c::21) by CH0PR12MB5186.namprd12.prod.outlook.com (2603:10b6:610:b9::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.19; Thu, 3 Nov 2022 16:18:47 +0000 Received: from CO1NAM11FT016.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8c:cafe::bd) by MW4PR03CA0136.outlook.office365.com (2603:10b6:303:8c::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.21 via Frontend Transport; Thu, 3 Nov 2022 16:18:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CO1NAM11FT016.mail.protection.outlook.com (10.13.175.141) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Thu, 3 Nov 2022 16:18:47 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Thu, 3 Nov 2022 09:18:39 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Thu, 3 Nov 2022 09:18:39 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.29 via Frontend Transport; Thu, 3 Nov 2022 09:18:32 -0700 From: Avihai Horon To: CC: Alex Williamson , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , David Hildenbrand , "Ilya Leoshkevich" , Thomas Huth , "Juan Quintela" , "Dr. David Alan Gilbert" , "Michael S. Tsirkin" , Cornelia Huck , Paolo Bonzini , Stefan Hajnoczi , Fam Zheng , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow , , , Kunkun Jiang , "Zhang, Chen" , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Shay Drory , Avihai Horon , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 17/17] vfio/migration: Query device data size in vfio_save_pending() Date: Thu, 3 Nov 2022 18:16:20 +0200 Message-ID: <20221103161620.13120-18-avihaih@nvidia.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20221103161620.13120-1-avihaih@nvidia.com> References: <20221103161620.13120-1-avihaih@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT016:EE_|CH0PR12MB5186:EE_ X-MS-Office365-Filtering-Correlation-Id: ab337005-8327-4c9a-a34d-08dabdb7161b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lZ73Up8VJj8SkN7Xt5HBWr2SGXjGwkX6JLhIyK2K9yEtYbSRp4nF0VFR1VXG667r48ekoMEU5rCyDPjh0nLT7JdkGd9FCXBFTB1LdSA+LIqE4LEPPpH3qMi2pEvkj9c0Hh4nJq0y6FU0773hTQd/0bQZXCgnuFEoaInz9QAJdH+KoE6wnLGb+hHWNIHBYAM9hDe0U9ZbyZ4F0SUfUlJ24DMmaTT1UPEtBzDZStnn+2LJtBj/0fu47+o02z8cYTDE7WZFHFuCl9J58H/sReFlSvJLD+SuqNkKsUdhC+Gckvt38iqE+DFn5frVDJ+zRRKD6qWqEk+ojHXSK3Rv2vDR4EK93L/OzqNzxks+qWuCw0VEJ8rQc3ygvlafDSKV22OiForIXSAqJbBPohrxFDn/c8s0ZUmqd7viCbHEkfuilion+gB+hh13TnytECD5EglAhiGj8j99Fb3Z8YUMs9yQTb19AKjFh8tktKq0RlHxaHGOaTQ5hlrVPiXw5iV9OxdJ4IjpoT0yrGdAhaxn/mgxOd7kg8OuHPoIY7qTf2Yp8iZ9dco340kss4Ki9IfNLUSPXTl81ff/Z4Ekg+qkf/fYrxbZjGR4WKHsSswsNObM4HGZBwydc4qjFDFgUgNjzRgQ6+ztIU7IsDNvAmOH93WeRRqUADTE2b25OCL0jVcrAwwfgvgvIFNaUkoExBggnyNIpfWkfolqRSshLBrl3GOWTO3nGeAvf/SEHYIeKt5cJDgHPWn97PgnC5h2PT8F9C0iIFdWPJ7UeisLSI8doaIv1g== X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(396003)(376002)(346002)(136003)(451199015)(40470700004)(36840700001)(46966006)(83380400001)(82740400003)(36860700001)(336012)(2906002)(426003)(47076005)(6916009)(40480700001)(186003)(8676002)(70206006)(4326008)(2616005)(478600001)(70586007)(82310400005)(41300700001)(1076003)(356005)(7416002)(86362001)(7696005)(36756003)(6666004)(26005)(8936002)(7636003)(5660300002)(54906003)(40460700003)(316002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 16:18:47.2983 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ab337005-8327-4c9a-a34d-08dabdb7161b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT016.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5186 Received-SPF: softfail client-ip=40.107.96.50; envelope-from=avihaih@nvidia.com; helo=NAM02-SN1-obe.outbound.protection.outlook.com X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.047, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Use VFIO_DEVICE_FEATURE_MIG_DATA_SIZE uAPI to query the device data size and report it in vfio_save_pending() instead of the hardcoded value that is currently used. Signed-off-by: Avihai Horon --- hw/vfio/migration.c | 27 ++++++++++++++++++++------- linux-headers/linux/vfio.h | 13 +++++++++++++ 2 files changed, 33 insertions(+), 7 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index e8068b9147..8ade901383 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -225,14 +225,27 @@ static void vfio_save_pending(void *opaque, uint64_t threshold_size, uint64_t *res_precopy, uint64_t *res_postcopy) { VFIODevice *vbasedev = opaque; + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_mig_data_size), + sizeof(uint64_t))] = {}; + struct vfio_device_feature *feature = (void *)buf; + struct vfio_device_feature_mig_data_size *mig_data_size = + (void *)feature->data; + + feature->argsz = sizeof(buf); + feature->flags = + VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_MIG_DATA_SIZE; - /* - * VFIO migration protocol v2 currently doesn't have an API to get pending - * device state size. Until such API is introduced, report some big - * arbitrary pending size so the device will be taken into account for - * downtime limit calculations. - */ - *res_postcopy += VFIO_MIG_PENDING_SIZE; + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + if (errno != ENOTTY) { + return; + } + + /* Kernel doesn't support VFIO_DEVICE_FEATURE_MIG_DATA_SIZE */ + *res_postcopy += VFIO_MIG_PENDING_SIZE; + } else { + *res_postcopy += mig_data_size->stop_copy_length; + } trace_vfio_save_pending(vbasedev->name, *res_precopy, *res_postcopy); } diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index ede44b5572..5c4ddf424f 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -986,6 +986,19 @@ enum vfio_device_mig_state { VFIO_DEVICE_STATE_RUNNING_P2P = 5, }; +/* + * Upon VFIO_DEVICE_FEATURE_GET read back the estimated data length that will + * be required to complete stop copy. + * + * Note: Can be called on each device state. + */ + +struct vfio_device_feature_mig_data_size { + __aligned_u64 stop_copy_length; +}; + +#define VFIO_DEVICE_FEATURE_MIG_DATA_SIZE 9 + /* -------- API for Type1 VFIO IOMMU -------- */ /**