From patchwork Tue Jan 17 11:55:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13104544 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 683EBC63797 for ; Tue, 17 Jan 2023 11:56:12 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pHkYs-0000i9-Nu; Tue, 17 Jan 2023 06:55:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pHkYq-0000hA-Ru for qemu-devel@nongnu.org; Tue, 17 Jan 2023 06:55:28 -0500 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pHkYn-0008A9-Bm for qemu-devel@nongnu.org; Tue, 17 Jan 2023 06:55:28 -0500 Received: by mail-pf1-x429.google.com with SMTP id c26so18816374pfp.10 for ; Tue, 17 Jan 2023 03:55:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Vf5jNMRum0lU93ysVQ8c+v+UiH2X4F5XN02QztzkGHI=; b=e/cNcEgu2inC5XKVxZCYUS5uD274sfs+s1rVzaLLoIOfAO3e5rZo/gLzHmDYeQESHW qMAfL3xnbaox2kiTmElJCoEzgEReupxmK74ZPomhWoRgMGd2AQpJrd4QzjeoKnDYxYXl L8yFSD1gt1sF0E7VoWjlUtb/skpiox8FCdREiJZznO76+pfobA5nCuscSJU6AjxyT/7n ir6v5tJuePipA740jSYE42LNNdFy07+PlHD7j7sWf7YUmXXpEVzb4dpC1Joh/eBqDrx7 N7p8vLvCTWOp1woMXDLX4MCaSVi6BFDvS8C0ElggKi9oKkeMdWwPjsUtrcGPfh05BawB xRdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Vf5jNMRum0lU93ysVQ8c+v+UiH2X4F5XN02QztzkGHI=; b=MWUx/QSqLk1nfPEtFMiEMO6MqwYg3ZecGoIRp56wBS33X8RszMAiGLZZXqa71dLSyQ 2RXC/arCUgCM0zY7G/pdoKCmTe62XnUfcbDRvU9q9+aiBwNZAqHQ0MQ/9qYieRtoeABP /E3zBDHfQBFMGt02kzA+FAlAmuuPeZdqp6YNwCag16RB9JtQHdC6X06CyxOd7VGX7ClZ t68KU9McjSv+r+EOzMO1ozqJuP9es70yqWRbQYdH8NhcchGn5EGBkESvbgdCwvZxl/kk T8xN6AIIEvjTae0GCQjLQtFMlKoMHtr/eKmAPXT281JCNJJwYMoWI1v5/ZOAZIf0WO7X GpTQ== X-Gm-Message-State: AFqh2koeTH3KH+aloPpcI05HA8nwVgpPnSRKscx7+kknnucoccO7pRiq CMZXDajKZ3VQj22NuHiTxv4HOTADXqXJ5wkU X-Google-Smtp-Source: AMrXdXsuaWgYTwcPj+w760bTnEQz+pO/FK3MIQUjH4PnMAj6dD7Lp53a0eNvBSiEQikxo4j61mUdgQ== X-Received: by 2002:a05:6a00:3020:b0:58d:92ff:8a2a with SMTP id ay32-20020a056a00302000b0058d92ff8a2amr2967816pfb.23.1673956520812; Tue, 17 Jan 2023 03:55:20 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.28]) by smtp.gmail.com with ESMTPSA id c196-20020a621ccd000000b0058a666aea32sm14003708pfc.147.2023.01.17.03.55.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jan 2023 03:55:20 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [RFC v5 1/3] rcu: introduce rcu_read_is_locked() Date: Tue, 17 Jan 2023 19:55:09 +0800 Message-Id: <20230117115511.3215273-2-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230117115511.3215273-1-xuchuangxclwt@bytedance.com> References: <20230117115511.3215273-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pf1-x429.google.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.001, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu Reviewed-by: Juan Quintela --- include/qemu/rcu.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index b063c6fde8..719916d9d3 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h @@ -119,6 +119,13 @@ static inline void rcu_read_unlock(void) } } +static inline bool rcu_read_is_locked(void) +{ + struct rcu_reader_data *p_rcu_reader = get_ptr_rcu_reader(); + + return p_rcu_reader->depth > 0; +} + extern void synchronize_rcu(void); /* From patchwork Tue Jan 17 11:55:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13104543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5C6F7C63797 for ; Tue, 17 Jan 2023 11:55:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pHkYr-0000hY-Os; Tue, 17 Jan 2023 06:55:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pHkYp-0000gt-LZ for qemu-devel@nongnu.org; Tue, 17 Jan 2023 06:55:28 -0500 Received: from mail-pg1-x52f.google.com ([2607:f8b0:4864:20::52f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pHkYn-0008BS-3u for qemu-devel@nongnu.org; Tue, 17 Jan 2023 06:55:27 -0500 Received: by mail-pg1-x52f.google.com with SMTP id 141so21779864pgc.0 for ; Tue, 17 Jan 2023 03:55:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pHEyRvEt3t7J15YLDPwGUSeYDGUOiCOIPr6S00z/+hs=; b=tInR7XtWZ2y0qs5M54WiUopx9e/XkLKXVN/mm0oBUqCzf5q+/XXD0L49O/IwdKYXAv MgBCb9BljWNI547OjZQgMRanMljXOH7XZ96qmO/lD2fzadmNmRFI6elOZj32DUKPrdXs B3w+ykmG66D59SO5ro7J5qRSA6umviLFnxWToAEmeb5qA9QA1oAtmnsgxbvCFAT+XS+S 9RnKSpjKsEeYIHk+SlEzdElPw8yTOj5SmmYw1kav25E9VBfesb4o+Gzp5K98hr26cgeY zpcQu6NjrUgsLK81Ms9cv+ult7Qj78xq/PKlpl2Y2miAU8pew+KyqyWAv6A1Nw9dPZkq ySWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pHEyRvEt3t7J15YLDPwGUSeYDGUOiCOIPr6S00z/+hs=; b=xJtwdjR/qgl9BSjCe3NeqkLbegDssIQKhJQsE8GG6ZTXh0dz2mWLX0otw5/LkhsrCA ndUotFHSJFD5ocTT/p4++rIA1zc2QRPG2+E3DUC4N7xeilAO1GeFjNXLlVb6/PhlYA5m g71KC6EVkeQAE9XV20kPH9rMbkSgOkRMB+T28NUiSjt2Xz1C858qi/HxGOUIQ3l9QQFV E4c5E+xW87QV50FH3DUyvaAGu7tQsz2QeYfLk7DiW42rRi8YQaYDVasmvvpTKgzF+rSB JoRq6BUoTjlAlu69kZ4B1nhYg2hXato4DoPM0dE52pytWybhdcpz5VebKxEHPLEU5J/y 6YxQ== X-Gm-Message-State: AFqh2kpz428tRC7ZV+m5eQdEu9MPABJa/C2d5wLD8dLT7IHbDV4qmAjE hUCM8/xoUznhiiwKQgplsGocRzjIxVnpD8TT X-Google-Smtp-Source: AMrXdXuHkozBhoa9yFsUfEr8rP2DZaJTlD2Gs0B3Ku3ynvqqeVv5XMcSze00WwBazKGD6lz4qkU/7Q== X-Received: by 2002:a05:6a00:1496:b0:586:35dd:91c3 with SMTP id v22-20020a056a00149600b0058635dd91c3mr4254640pfu.29.1673956523458; Tue, 17 Jan 2023 03:55:23 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.28]) by smtp.gmail.com with ESMTPSA id c196-20020a621ccd000000b0058a666aea32sm14003708pfc.147.2023.01.17.03.55.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jan 2023 03:55:23 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [RFC v5 2/3] memory: add depth assert in address_space_to_flatview Date: Tue, 17 Jan 2023 19:55:10 +0800 Message-Id: <20230117115511.3215273-3-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230117115511.3215273-1-xuchuangxclwt@bytedance.com> References: <20230117115511.3215273-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52f; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pg1-x52f.google.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.001, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu Reviewed-by: Juan Quintela --- include/exec/memory.h | 15 +++++++++++++++ softmmu/memory.c | 5 +++++ 2 files changed, 20 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 91f8a2395a..ce13ebb763 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -27,6 +27,7 @@ #include "qemu/notify.h" #include "qom/object.h" #include "qemu/rcu.h" +#include "qemu/main-loop.h" #define RAM_ADDR_INVALID (~(ram_addr_t)0) @@ -1069,8 +1070,22 @@ struct FlatView { MemoryRegion *root; }; +bool memory_region_transaction_in_progress(void); + static inline FlatView *address_space_to_flatview(AddressSpace *as) { + /* + * Before using any flatview, sanity check we're not during a memory + * region transaction or the map can be invalid. Note that this can + * also be called during commit phase of memory transaction, but that + * should also only happen when the depth decreases to 0 first. + * Meanwhile it's safe to access current_map with RCU read lock held + * even if during a memory transaction. It means the user can bear + * with an obsolete map. + */ + assert((!memory_region_transaction_in_progress() && + qemu_mutex_iothread_locked()) || + rcu_read_is_locked()); return qatomic_rcu_read(&as->current_map); } diff --git a/softmmu/memory.c b/softmmu/memory.c index bc0be3f62c..856c37fd0a 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -1116,6 +1116,11 @@ void memory_region_transaction_commit(void) } } +bool memory_region_transaction_in_progress(void) +{ + return memory_region_transaction_depth != 0; +} + static void memory_region_destructor_none(MemoryRegion *mr) { } From patchwork Tue Jan 17 11:55:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13104545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 13BFFC3DA78 for ; Tue, 17 Jan 2023 11:56:23 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pHkYt-0000id-Vc; Tue, 17 Jan 2023 06:55:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pHkYr-0000hH-9b for qemu-devel@nongnu.org; Tue, 17 Jan 2023 06:55:29 -0500 Received: from mail-pg1-x536.google.com ([2607:f8b0:4864:20::536]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pHkYp-0008Dl-Jo for qemu-devel@nongnu.org; Tue, 17 Jan 2023 06:55:28 -0500 Received: by mail-pg1-x536.google.com with SMTP id h192so21753611pgc.7 for ; Tue, 17 Jan 2023 03:55:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zpXsPMWlC5EFkME77Z0xQ4jlbPMSa230wUSbV2lRrIk=; b=b4d/cvLZnKQL2vmHBucsYoTA2IL56KL6I1img+cCeCIz+bEFZl1csrwv3EgNpHwpba DWsB3hQ9GGP0QGU4gXWOuc/1DbwZpuftStS2+3JPHl1Wrf73Nf6KvAl+is6DvXIOe2/G QDvHIFOxpdqj6wILKw/iVpbUveWp5QAwcpxPYKOf75UydinL3TErUM93QgRRSktEY4R+ S7eSzxMKQiRPJCY+5l21TA3+jKzqdVNcefRkaUUqNpFtTqMrg+fQXYoHyueE8dg8oXSC Wz7cX4qvCUYcI/NbnX8UXN9vxb43/mnQjhAJjfGfg7I6oI8lEeR6lsgG9uruiCxMtUYd svyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zpXsPMWlC5EFkME77Z0xQ4jlbPMSa230wUSbV2lRrIk=; b=a9G0MBpJH4VhnDy3RJFKeXyi4gv0wmQRelHIdB6Tt2jHYm4OkDWyeXJJZH2X7Tz8UT A4iDMlbjLnRf1nn7gI2bDR21pYkzF2xubJinVglaH/Y+Pf9dFHgPFJH5hPheuAWWx2Yp gKyu7Ijz7tDjnz5UveLyeqxM3cUfbn3mshqxcjlih7DYNGjGdgk1iboPNNCSw7M1bi4t fmjYqtDfcXbM1xMLWYl+egzXqtT0vKRGOxZLhxV2cT/h+Ei0gnV8CungtonE8sgAhFs0 JkRttR3He6QEyHtWTLojtuNKkGiXQ7dVYUd3pqMkWkY0H+pjXopcMOdPapL/OVvrqLpG Prwg== X-Gm-Message-State: AFqh2koHyQlY92B5IPcxaDb1Sdqz1qYP8MWjguFsNoOf84mEn4M/Gi1C 1vhbmDU9QTd1b/r7+oJYw5jK7DIN78tpGkg7 X-Google-Smtp-Source: AMrXdXsyV1lUitd3w/D0G4eS+0oNQVOEWm5xGWtuu82MZtIIWCwPw58EASMPDcukpE9x5RLI5tSDrw== X-Received: by 2002:a62:27c6:0:b0:58d:8ea0:ce31 with SMTP id n189-20020a6227c6000000b0058d8ea0ce31mr2864928pfn.27.1673956526112; Tue, 17 Jan 2023 03:55:26 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.28]) by smtp.gmail.com with ESMTPSA id c196-20020a621ccd000000b0058a666aea32sm14003708pfc.147.2023.01.17.03.55.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jan 2023 03:55:25 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [RFC v5 3/3] migration: reduce time of loading non-iterable vmstate Date: Tue, 17 Jan 2023 19:55:11 +0800 Message-Id: <20230117115511.3215273-4-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230117115511.3215273-1-xuchuangxclwt@bytedance.com> References: <20230117115511.3215273-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::536; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pg1-x536.google.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.001, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start). Most of the time is spent committing memory region changes repeatedly. This patch packs all the changes to memory region during the period of loading non-iterable vmstate in a single memory transaction. With the increase of devices, this patch will greatly improve the performance. Here are the test1 results: test info: - Host - Intel(R) Xeon(R) Platinum 8260 CPU - NVIDIA Mellanox ConnectX-5 - VM - 32 CPUs 128GB RAM VM - 8 16-queue vhost-net device - 16 4-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 150 ms 740+ ms after about 30 ms 630+ ms In test2, we keep the number of the device the same as test1, reduce the number of queues per device: Here are the test2 results: test info: - Host - Intel(R) Xeon(R) Platinum 8260 CPU - NVIDIA Mellanox ConnectX-5 - VM - 32 CPUs 128GB RAM VM - 8 1-queue vhost-net device - 16 1-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 90 ms about 250 ms after about 25 ms about 160 ms In test3, we keep the number of queues per device the same as test1, reduce the number of devices: Here are the test3 results: test info: - Host - Intel(R) Xeon(R) Platinum 8260 CPU - NVIDIA Mellanox ConnectX-5 - VM - 32 CPUs 128GB RAM VM - 1 16-queue vhost-net device - 1 4-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 20 ms about 70 ms after about 11 ms about 60 ms As we can see from the test results above, both the number of queues and the number of devices have a great impact on the time of loading non-iterable vmstate. The growth of the number of devices and queues will lead to more mr commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu Reviewed-by: Juan Quintela --- migration/savevm.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..8ca6d396f4 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2617,6 +2617,16 @@ int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis) uint8_t section_type; int ret = 0; + /* + * Call memory_region_transaction_begin() before loading vmstate. + * This call is paired with memory_region_transaction_commit() at + * the end of qemu_loadvm_state_main(), in order to pack all the + * changes to memory region during the period of loading + * non-iterable vmstate in a single memory transaction. + * This operation will reduce time of loading non-iterable vmstate + */ + memory_region_transaction_begin(); + retry: while (true) { section_type = qemu_get_byte(f); @@ -2684,6 +2694,14 @@ out: goto retry; } } + + /* + * Call memory_region_transaction_commit() after loading vmstate. + * At this point, qemu actually completes all the previous memory + * region transactions. + */ + memory_region_transaction_commit(); + return ret; }