From patchwork Fri Jan 6 04:06:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 9500015 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E050460413 for ; Fri, 6 Jan 2017 04:09:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CAFCE280DE for ; Fri, 6 Jan 2017 04:09:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BDCEC28480; Fri, 6 Jan 2017 04:09:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5AD00280DE for ; Fri, 6 Jan 2017 04:09:54 +0000 (UTC) Received: from localhost ([::1]:50013 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cPLqj-0001kq-F1 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 05 Jan 2017 23:09:53 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57600) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cPLnP-0007RN-9A for qemu-devel@nongnu.org; Thu, 05 Jan 2017 23:06:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cPLnO-0001NC-Al for qemu-devel@nongnu.org; Thu, 05 Jan 2017 23:06:27 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51526) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cPLnO-0001N4-27 for qemu-devel@nongnu.org; Thu, 05 Jan 2017 23:06:26 -0500 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 40F1F552CE for ; Fri, 6 Jan 2017 04:06:26 +0000 (UTC) Received: from pxdev.xzpeter.org (vpn1-5-53.pek2.redhat.com [10.72.5.53]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v0646FLA020563; Thu, 5 Jan 2017 23:06:21 -0500 From: Peter Xu To: qemu-devel@nongnu.org Date: Fri, 6 Jan 2017 12:06:12 +0800 Message-Id: <1483675573-12636-2-git-send-email-peterx@redhat.com> In-Reply-To: <1483675573-12636-1-git-send-email-peterx@redhat.com> References: <1483675573-12636-1-git-send-email-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 06 Jan 2017 04:06:26 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 1/2] migration: allow to prioritize save state entries X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , Jason Wang , mst@redhat.com, "Dr. David Alan Gilbert" , peterx@redhat.com, Amit Shah , Paolo Bonzini Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP During migration, save state entries are saved/loaded without a specific order - we just traverse the savevm_state.handlers list and do it one by one. This might not be enough. There are requirements that we need to load specific device's vmstate first before others. For example, VT-d IOMMU contains DMA address remapping information, which is required by all the PCI devices to do address translations. We need to make sure IOMMU's device state is loaded before the rest of the PCI devices, so that DMA address translation can work properly. This patch provide a VMStateDescription.priority value to allow specify the priority of the saved states. The loadvm operation will be done with those devices with higher vmsd priority. Before this patch, we are possibly achieving the ordering requirement by an assumption that the ordering will be the same with the ordering that objects are created. A better way is to mark it out explicitly in the VMStateDescription table, like what this patch does. Current ordering logic is still naive and slow, but after all that's not a critical path so IMO it's a workable solution for now. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- include/migration/vmstate.h | 6 ++++++ migration/savevm.c | 34 ++++++++++++++++++++++++++++++---- 2 files changed, 36 insertions(+), 4 deletions(-) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 1638ee5..1a22887 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -186,6 +186,11 @@ enum VMStateFlags { VMS_MULTIPLY_ELEMENTS = 0x4000, }; +typedef enum { + MIG_PRI_DEFAULT = 0, + MIG_PRI_MAX, +} MigrationPriority; + typedef struct { const char *name; size_t offset; @@ -207,6 +212,7 @@ struct VMStateDescription { int version_id; int minimum_version_id; int minimum_version_id_old; + MigrationPriority priority; LoadStateHandler *load_state_old; int (*pre_load)(void *opaque); int (*post_load)(void *opaque, int version_id); diff --git a/migration/savevm.c b/migration/savevm.c index 0363372..f9c06e9 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -532,6 +532,34 @@ static int calculate_compat_instance_id(const char *idstr) return instance_id; } +static inline MigrationPriority save_state_priority(SaveStateEntry *se) +{ + if (se->vmsd) { + return se->vmsd->priority; + } + return MIG_PRI_DEFAULT; +} + +static void savevm_state_handler_insert(SaveStateEntry *nse) +{ + MigrationPriority priority = save_state_priority(nse); + SaveStateEntry *se; + + assert(priority <= MIG_PRI_MAX); + + QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { + if (save_state_priority(se) < priority) { + break; + } + } + + if (se) { + QTAILQ_INSERT_BEFORE(se, nse, entry); + } else { + QTAILQ_INSERT_TAIL(&savevm_state.handlers, nse, entry); + } +} + /* TODO: Individual devices generally have very little idea about the rest of the system, so instance_id should be removed/replaced. Meanwhile pass -1 as instance_id if you do not already have a clearly @@ -578,8 +606,7 @@ int register_savevm_live(DeviceState *dev, se->instance_id = instance_id; } assert(!se->compat || se->instance_id == 0); - /* add at the end of list */ - QTAILQ_INSERT_TAIL(&savevm_state.handlers, se, entry); + savevm_state_handler_insert(se); return 0; } @@ -662,8 +689,7 @@ int vmstate_register_with_alias_id(DeviceState *dev, int instance_id, se->instance_id = instance_id; } assert(!se->compat || se->instance_id == 0); - /* add at the end of list */ - QTAILQ_INSERT_TAIL(&savevm_state.handlers, se, entry); + savevm_state_handler_insert(se); return 0; }