From patchwork Fri Dec 23 14:23:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13081004 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A5288C3DA79 for ; Fri, 23 Dec 2022 14:24:44 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p8ixf-0001h0-Lb; Fri, 23 Dec 2022 09:23:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p8ixZ-0001g1-3l for qemu-devel@nongnu.org; Fri, 23 Dec 2022 09:23:41 -0500 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p8ixX-0000lU-4M for qemu-devel@nongnu.org; Fri, 23 Dec 2022 09:23:40 -0500 Received: by mail-pl1-x634.google.com with SMTP id w20so5072542ply.12 for ; Fri, 23 Dec 2022 06:23:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UCBFuBCG3b1Fjv3ba+NdH9dU2b+uFqE2EzjnxotwM2c=; b=u3Le6DOYFo9+On/H7sy8w6pPgoui+/F5lP7cP6ruOWd+ioqJipg/wSMw5IXm/wcXf4 AiERlJSlsu1tW48/dXPgwTYWipLRG75c9L7Gm/EHqZTX2Upid8nzUZ2zXGuIi/A1H01o iOvXmwfmO7jOaH527VFDrVn2ODyIIQZemaETOatZTipkZK0HHCyvTnLgEGxun795QW6I M8UcdbH7a3s3SfmobsZ7plF5Dkx4q3lOw83RszEz586wLyj3/94dL8xDmHUkYk+2mO3x 8R1xyo92iTUS9BSSo4uRbeWhQKRGyQM8RuaJ5Legt+GtK8UqBgUOWgQQGLdSSS1tlOE7 I80w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UCBFuBCG3b1Fjv3ba+NdH9dU2b+uFqE2EzjnxotwM2c=; b=jcG+5lQkhQPIAogrj1VZSncDvNUAxIENskCrlC8JFVlbxxJtRzZ9tYDOB1CwrzYLFw 3DimSd2+ingHphE9ZUQZZFKqaBgE1LT30EyVnvpCfyOrTT322S6+6oB8Pfn09Elem+WA xkf9AoO/zJwwT953aTFYNlZaXMYRpyq6fdMdjOq8rXVPplelh4ukqp4aZBbrBzBexakz 9e6w0pn0MiEVcYN7mtT56YcUt86/uf42vVvSS8eit4FJTr5p6sjjKwh1QCEKZ6MQP9Yz 0GodvwIbkSQfZOk/DP+vcX8sQUnKkr9M28moN2UlgUQskgSeRpdbtQopmaK76hmoMhvZ u84w== X-Gm-Message-State: AFqh2ko5qPHX5padDz/sa7kcqpzwFXWY3Rwxo8mqL71v0HIoMqjmrxt/ 96Aa0QaJeyTU7PVuyFD6FnBmM+oGh/hs7RIo X-Google-Smtp-Source: AMrXdXsz2DjjqkT+jqY+EVBkm/IZFq++QunWhzTW2ta9snqQtPd2Jz3TGmELQEfk+TgCzkAzpkey4w== X-Received: by 2002:a17:90b:d98:b0:221:4cd7:3401 with SMTP id bg24-20020a17090b0d9800b002214cd73401mr11499234pjb.10.1671805410885; Fri, 23 Dec 2022 06:23:30 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.12]) by smtp.gmail.com with ESMTPSA id a16-20020a17090a6d9000b00223fa0148b8sm4751734pjk.41.2022.12.23.06.23.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Dec 2022 06:23:30 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [RFC v4 1/3] rcu: introduce rcu_read_locked() Date: Fri, 23 Dec 2022 22:23:05 +0800 Message-Id: <20221223142307.1614945-2-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221223142307.1614945-1-xuchuangxclwt@bytedance.com> References: <20221223142307.1614945-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pl1-x634.google.com X-Spam_score_int: 6 X-Spam_score: 0.6 X-Spam_bar: / X-Spam_report: (0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org add rcu_read_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index b063c6fde8..42cbd0080f 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h @@ -119,6 +119,13 @@ static inline void rcu_read_unlock(void) } } +static inline bool rcu_read_locked(void) +{ + struct rcu_reader_data *p_rcu_reader = get_ptr_rcu_reader(); + + return p_rcu_reader->depth > 0; +} + extern void synchronize_rcu(void); /* From patchwork Fri Dec 23 14:23:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13081003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A08FCC3DA79 for ; Fri, 23 Dec 2022 14:24:29 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p8ixi-0001hz-0H; Fri, 23 Dec 2022 09:23:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p8ixX-0001fr-Lz for qemu-devel@nongnu.org; Fri, 23 Dec 2022 09:23:40 -0500 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p8ixV-0000lu-QX for qemu-devel@nongnu.org; Fri, 23 Dec 2022 09:23:39 -0500 Received: by mail-pl1-x62e.google.com with SMTP id 4so5116509plj.3 for ; Fri, 23 Dec 2022 06:23:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1/oHzdOrXSbYFdgGl9jwSvJhvS7tn2pp+QhccB4A3yg=; b=nh1MZ+EgoI0u63LP3Zx0mQ+o3q6wfjclAMnskSld+g7nK12d3ubAOsdsG1Ma4NE/cW Pm05Wfq3fEXcvWrdbNPi+avBqxtqHC3uNXGgcBzJ/JpaaQimU4dJzGHMW5qWlZxzoQ6h 1A+3z/1xQgKnYY9br8eAUo8HZxlu/GsN4TBgeYkk/XSji3XA9xtYA+ttD8rL6VrB6bD/ TQdENYcnCTpZOzpAiADvCRUdYEfXWNiLjDDAdMo504O97Wvzx0gh7YdTVZQijSe2BY2H s5DKh6httFfjIjbMzt1qSABo3g3iYkhZXv5NvEmXW88dnBsWYClvc/VmqoGdjtw5GAGR LfmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1/oHzdOrXSbYFdgGl9jwSvJhvS7tn2pp+QhccB4A3yg=; b=bmOeEf9FyXnZ8gehLnnEdfJwBehVLcR45jf/eV+aKcn8eiDytKp7uAfeBYzLsWYgx1 E8E/QLaI9taafX3EhqeqvGA7rajCsz7EoRPFCUSqq3TAjOqnOQjQD1QB77CFJ5alDSqY 6xXZj9gQ9FITTXG8AyGfxf5ZIlcFTK2CX13W/ICCEok1c5QP0s6yPsYtaCisAuqfnu6H v/GxWhZnAyhLSnT74JcCMJ8PhnNFbZCqExPacxQnMp5VOJVSHPx+eoZXrzsV8zs6Bnaa N+sIbfdcADAA53Bd+buWYVl6e5U5B0b9Ojj/O0SgWu+9sHM6zOubZbd1eCIj+gINNjrx dBFA== X-Gm-Message-State: AFqh2kr3uBiQ1Wi1YkBnbBe8fFsJoGWFJHDvKxHYarKKIjOAz8VWjPhS Ii7iH1TKboDzuTyB4ghhVDSFuRmecCdZ2N9N X-Google-Smtp-Source: AMrXdXsQRmDzWWbiFqoUzcQZ0B33gc+9jHEUIoZy4iMrePCJuQJi8Jqiy0muqP8Qi0BoLW0NDpiQzg== X-Received: by 2002:a17:90a:7d0f:b0:219:7f29:3152 with SMTP id g15-20020a17090a7d0f00b002197f293152mr10641552pjl.39.1671805414778; Fri, 23 Dec 2022 06:23:34 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.12]) by smtp.gmail.com with ESMTPSA id a16-20020a17090a6d9000b00223fa0148b8sm4751734pjk.41.2022.12.23.06.23.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Dec 2022 06:23:34 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [RFC v4 2/3] memory: add depth assert in address_space_to_flatview Date: Fri, 23 Dec 2022 22:23:06 +0800 Message-Id: <20221223142307.1614945-3-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221223142307.1614945-1-xuchuangxclwt@bytedance.com> References: <20221223142307.1614945-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62e; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pl1-x62e.google.com X-Spam_score_int: 6 X-Spam_score: 0.6 X-Spam_bar: / X-Spam_report: (0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 +++++++++ softmmu/memory.c | 5 +++++ 2 files changed, 14 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 91f8a2395a..66c43b4862 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -1069,8 +1069,17 @@ struct FlatView { MemoryRegion *root; }; +int memory_region_transaction_get_depth(void); + static inline FlatView *address_space_to_flatview(AddressSpace *as) { + /* + * Before using any flatview, sanity check we're not during a memory + * region transaction or the map can be invalid. Note that this can + * also be called during commit phase of memory transaction, but that + * should also only happen when the depth decreases to 0 first. + */ + assert(memory_region_transaction_get_depth() == 0 || rcu_read_locked()); return qatomic_rcu_read(&as->current_map); } diff --git a/softmmu/memory.c b/softmmu/memory.c index bc0be3f62c..01192e2e5b 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -1116,6 +1116,11 @@ void memory_region_transaction_commit(void) } } +int memory_region_transaction_get_depth(void) +{ + return memory_region_transaction_depth; +} + static void memory_region_destructor_none(MemoryRegion *mr) { } From patchwork Fri Dec 23 14:23:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13081002 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A7EA9C4332F for ; Fri, 23 Dec 2022 14:24:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p8ixg-0001hZ-Ay; Fri, 23 Dec 2022 09:23:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p8ixa-0001g9-LH for qemu-devel@nongnu.org; Fri, 23 Dec 2022 09:23:43 -0500 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p8ixY-0000me-Ag for qemu-devel@nongnu.org; Fri, 23 Dec 2022 09:23:42 -0500 Received: by mail-pl1-x62e.google.com with SMTP id d7so5089079pll.9 for ; Fri, 23 Dec 2022 06:23:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TclGryZiOEctLPI49CtSoCWNEcsJRMvUbalqBdcMECA=; b=kQjB5wFCy7xFQpqH/t9MIkQ/O/dGCs8S3I3FhODKKAAzr6pGJhdJyhN/MWlrrvJ7Gw AEohhU/RmhWH9Oa6GVirjchc9dHjv4W5dbX+bvd2d5OmMEYbT48vXYVmFI/HHba8wBjc INT2RFgsESoTIzx5Z5z5yXRcYDzQaKVhNa+AHvSwidkbS29NVSPt+2STl7jMAAKMZhid mwcuAz8bqfewaRYiQZeT8sHsh2eVeG6nQN3LU+7GrGBSgIsQMOL2DfxpGcQEkNUoZzDc URRvRR1xwqBBwDqneFiXNuofmjRK1rM7s+ZSJSZZ/bhCe31Ds935of5jDkTBs++FCZRi /ZSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TclGryZiOEctLPI49CtSoCWNEcsJRMvUbalqBdcMECA=; b=ZAkaEG+2IDXSLadqVP+nsjrsBDFiV2UIaMzd1WWviEgvy20BAIHjifXWBnEJkH2XaV kXruQfxEZesUZprQPCWbkMqE3kM+aRTCp8cxmLZoNMdDFJ97AG/4MkregNm8XOIz2JM8 EoA4rSdNx0QzndtaBVqZHpjL9GcVJQhgwPDzaLoLNZWyGLzYqeutB6ssJ6SHMNnkwuyc tlyY3ZdZfhguQz6vJzNj6bJkpPXuk4odERTnSp323s6BRFMIUx8fxoqruB1qW9+lRlrI PvRz+m7c8Rszb+6aOoW8t0w2pjXuOgj8gsuGJPZwC5b1/LT2QQHTfVu1VU9qfg4aHnia 2Zww== X-Gm-Message-State: AFqh2kpL/s4UPEzbJvtXMxmsiTmk7Gk43xulYm3pyRZGIrl2GSW7lrRs 2DwRggau1Pv9KWRjElcfF5FwwL89VifsGRqd X-Google-Smtp-Source: AMrXdXu5N4aM9/TIieyTI2aSQXmgHQ8p0byMWn/nYkX9OEh4N+U+CPla8M65DoDniyeqPFxfqen63A== X-Received: by 2002:a17:90a:420b:b0:218:ff8e:5d6b with SMTP id o11-20020a17090a420b00b00218ff8e5d6bmr11370142pjg.16.1671805418925; Fri, 23 Dec 2022 06:23:38 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.12]) by smtp.gmail.com with ESMTPSA id a16-20020a17090a6d9000b00223fa0148b8sm4751734pjk.41.2022.12.23.06.23.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Dec 2022 06:23:38 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [RFC v4 3/3] migration: reduce time of loading non-iterable vmstate Date: Fri, 23 Dec 2022 22:23:07 +0800 Message-Id: <20221223142307.1614945-4-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221223142307.1614945-1-xuchuangxclwt@bytedance.com> References: <20221223142307.1614945-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62e; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pl1-x62e.google.com X-Spam_score_int: 6 X-Spam_score: 0.6 X-Spam_bar: / X-Spam_report: (0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start). Most of the time is spent committing memory region changes repeatedly. This patch packs all the changes to memory region during the period of loading non-iterable vmstate in a single memory transaction. With the increase of devices, this patch will greatly improve the performance. Here are the test1 results: test info: - Host - Intel(R) Xeon(R) Platinum 8260 CPU - NVIDIA Mellanox ConnectX-5 - VM - 32 CPUs 128GB RAM VM - 8 16-queue vhost-net device - 16 4-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 150 ms 740+ ms after about 30 ms 630+ ms In test2, we keep the number of the device the same as test1, reduce the number of queues per device: Here are the test2 results: test info: - Host - Intel(R) Xeon(R) Platinum 8260 CPU - NVIDIA Mellanox ConnectX-5 - VM - 32 CPUs 128GB RAM VM - 8 1-queue vhost-net device - 16 1-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 90 ms about 250 ms after about 25 ms about 160 ms In test3, we keep the number of queues per device the same as test1, reduce the number of devices: Here are the test3 results: test info: - Host - Intel(R) Xeon(R) Platinum 8260 CPU - NVIDIA Mellanox ConnectX-5 - VM - 32 CPUs 128GB RAM VM - 1 16-queue vhost-net device - 1 4-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 20 ms about 70 ms after about 11 ms about 60 ms As we can see from the test results above, both the number of queues and the number of devices have a great impact on the time of loading non-iterable vmstate. The growth of the number of devices and queues will lead to more mr commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu Signed-off-by: Chuang Xu --- migration/savevm.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..19785e5a54 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2617,6 +2617,9 @@ int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis) uint8_t section_type; int ret = 0; + /* call memory_region_transaction_begin() before loading vmstate */ + memory_region_transaction_begin(); + retry: while (true) { section_type = qemu_get_byte(f); @@ -2684,6 +2687,10 @@ out: goto retry; } } + + /* call memory_region_transaction_commit() after loading vmstate */ + memory_region_transaction_commit(); + return ret; }