From patchwork Tue Dec 13 13:35:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13072086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D34D6C4332F for ; Tue, 13 Dec 2022 13:36:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p55RQ-0000zy-Qz; Tue, 13 Dec 2022 08:35:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p55RO-0000ya-0L for qemu-devel@nongnu.org; Tue, 13 Dec 2022 08:35:26 -0500 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p55RM-0002yN-C3 for qemu-devel@nongnu.org; Tue, 13 Dec 2022 08:35:25 -0500 Received: by mail-pl1-x62d.google.com with SMTP id w23so15609838ply.12 for ; Tue, 13 Dec 2022 05:35:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0u/ggCD2dpzlHztgUVICOjgPHyTnEFx12/17GnxGqa0=; b=vy/hLU2+rTggSbyvOzwetXdX25lUFmfh3nH6NyG40cc6U70W4FzOqoSCuO0N0mWg0/ rjiy5eqZnH9d3TGMVL3joUc5Uim8CcjxOk4yxCEj8nqJ5iwvNuwISB7VDL/86KjWa995 9WXxMazcCDcMsYCvVjgSSplxHeWBXSZo3T3bleI3vCKfTAXCD4gwMZ+81rjaqww6cvAH qJQq2mfi2TyRjM8VpCgoBAS+oVhsew7ugLWAV0pNOJHOmTUWDxlV4cjiCzDOqz29noES WY8J6SN1qwgmUJf19CRGptUfulPm/gpCBicCjrClcVmw2C/nP0yC+cgbm/McLJbQAPlQ PZWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0u/ggCD2dpzlHztgUVICOjgPHyTnEFx12/17GnxGqa0=; b=dy7YXXOoUsnhbfQERLFyhrNcSFoGo2s2N2C05KcXYwsevv5/CBTkhnV3fQJU5U2Pob mJmlGyjuYlXp9CvCZersWCvNkkKDaE3WrldDOtI8y/nYOL2mIRT0VkekpRHTtmiAWi/+ oU3kLEVUcAM08h9U2U6QJWLWsUahDpeyN3hm7T/uKxXWDuCAls8Kwgms919ks4PaChHO 926xgjShJHd0+mTWVuhniE5dsubtwhDPDQuxOJPPIomIq99CMAzX+sxTdG0ZlAluSpIM 0+Ds5fJVG/Rn0hSw+dO1ZfFPTlB3ZMv/rECYdNJwUQMq0LsbyzYtBIC3z+4HAVeymVuM y0PQ== X-Gm-Message-State: ANoB5pl2mB7OFZvQQCIgdRh6RlE6s5qPn6j8sKVb2yzs9zVq0uSExI7s sGlqIlVzGEOgFeqX24jZVfDQhSA3GXa8WTpt X-Google-Smtp-Source: AA0mqf5WJkLFXWrDyCpAqWhxjX3cFqZ6ZL1Ck2T624d22z+S36cypve4YKXoDO/6KSF1ettDv+0nhg== X-Received: by 2002:a17:902:cf03:b0:186:8568:be7e with SMTP id i3-20020a170902cf0300b001868568be7emr25173734plg.15.1670938522776; Tue, 13 Dec 2022 05:35:22 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.12]) by smtp.gmail.com with ESMTPSA id b18-20020a170903229200b00182a9c27acfsm8440033plh.227.2022.12.13.05.35.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 05:35:22 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, f4bug@amsat.org, mst@redhat.com, zhouyibo@bytedance.com, Chuang Xu Subject: [RFC v3 1/3] memory: add depth assert in address_space_to_flatview Date: Tue, 13 Dec 2022 21:35:08 +0800 Message-Id: <20221213133510.1279488-2-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221213133510.1279488-1-xuchuangxclwt@bytedance.com> References: <20221213133510.1279488-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pl1-x62d.google.com X-Spam_score_int: 6 X-Spam_score: 0.6 X-Spam_bar: / X-Spam_report: (0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu Signed-off-by: Chuang Xu Signed-off-by: Chuang Xu Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 +++++++++ softmmu/memory.c | 1 - 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index 91f8a2395a..b43cd46084 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -1069,8 +1069,17 @@ struct FlatView { MemoryRegion *root; }; +static unsigned memory_region_transaction_depth; + static inline FlatView *address_space_to_flatview(AddressSpace *as) { + /* + * Before using any flatview, sanity check we're not during a memory + * region transaction or the map can be invalid. Note that this can + * also be called during commit phase of memory transaction, but that + * should also only happen when the depth decreases to 0 first. + */ + assert(memory_region_transaction_depth == 0); return qatomic_rcu_read(&as->current_map); } diff --git a/softmmu/memory.c b/softmmu/memory.c index bc0be3f62c..f177c40cd8 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -37,7 +37,6 @@ //#define DEBUG_UNASSIGNED -static unsigned memory_region_transaction_depth; static bool memory_region_update_pending; static bool ioeventfd_update_pending; unsigned int global_dirty_tracking; From patchwork Tue Dec 13 13:35:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13072084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0FAE2C4708E for ; Tue, 13 Dec 2022 13:36:21 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p55Ra-00013m-FO; Tue, 13 Dec 2022 08:35:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p55RT-00012z-4n for qemu-devel@nongnu.org; Tue, 13 Dec 2022 08:35:32 -0500 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p55RR-0002zc-8q for qemu-devel@nongnu.org; Tue, 13 Dec 2022 08:35:30 -0500 Received: by mail-pl1-x629.google.com with SMTP id m4so15640036pls.4 for ; Tue, 13 Dec 2022 05:35:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Bq+9ICfvwoh9xLI+z54DwrGyhNaLYVE9QK8/eKTA4gQ=; b=5gi8Vn0pVR2D8HqkUNtXsG6P1E4G0fiDt8gIoPtfz/nHvuvr4lCnKqYNAy4RmPvxUW CnEVwDXfrYgiOACVBXpxFg332zjSsz3ofV9qUlU50qrc5FkIhg8oDu01IrN5fXc5NiXQ iXls84q5OH0RAEsYUORDkHtS3CzVsKjkr+BWEx6fc3gOBD/KS4usKD0NbejvJnscTtpw ZGVw5vEh5liXg9R/MU+VZaZCj34WtCmRMhJCgz5LJUEgDenOXGPPoutpsi4kVomtEALK c629wi+75WOAC97Lg0usQCQU6Q3PGjptK/X9OOelM8BCvDqdCAPZdHGc2UXmR0c8IUvm xiLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Bq+9ICfvwoh9xLI+z54DwrGyhNaLYVE9QK8/eKTA4gQ=; b=iYnBdYcty3nKz0Tgd3I37o5s5pu9DqHxIVjL2yDieBaGpeNJeU+MsCwXULBrMCSEKQ j10v2shvHsp+YM/DwYhaOq/H4iOzBHmqXTXl76Vh3dAjqtQyD1vaGlLTDD1oBJlrrn3I QDIAzp9TktdtvQzAMhpQnAdazQhUMDyXf0fnO4qVEedn21nbqxyz4J2XpYAB3HQ4lze8 EmFcJ4KHNsVaL/qCpZgMUDPXdg+ORrUstQOchRNfnA4qnAMSFA5pA2NbKbUGaTLWAXPj T4/StLwHfZ0sCA7HbCEOaeJq9ytXmuy/04yspeLI0jsEDMoLlZriYsXFYxNtxrAyziA0 JLcw== X-Gm-Message-State: ANoB5pnj9cRBNIooey+v5q2+MUBou37O5qUzaLFQwfFBQAVNe43dx6ec S9pA4ueHJsquDh0jucxjgJgxtAJ6BYIvp9Sg X-Google-Smtp-Source: AA0mqf6bq3yFF38wiObCHKM+CFm6+rFvv1C19xEPhQcmglJkv7oBx+C02pjF1z89XrMt2rWbSoKRxA== X-Received: by 2002:a17:902:8343:b0:186:95c9:ddc9 with SMTP id z3-20020a170902834300b0018695c9ddc9mr23098857pln.55.1670938527368; Tue, 13 Dec 2022 05:35:27 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.12]) by smtp.gmail.com with ESMTPSA id b18-20020a170903229200b00182a9c27acfsm8440033plh.227.2022.12.13.05.35.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 05:35:26 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, f4bug@amsat.org, mst@redhat.com, zhouyibo@bytedance.com, Chuang Xu Subject: [RFC v3 2/3] virtio: support delay of checks in virtio_load() Date: Tue, 13 Dec 2022 21:35:09 +0800 Message-Id: <20221213133510.1279488-3-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221213133510.1279488-1-xuchuangxclwt@bytedance.com> References: <20221213133510.1279488-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pl1-x629.google.com X-Spam_score_int: 6 X-Spam_score: 0.6 X-Spam_bar: / X-Spam_report: (0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Delay checks in virtio_load() to avoid possible address_space_to_flatview() call during memory region's begin/commit. Signed-off-by: Chuang Xu Signed-off-by: Chuang Xu --- hw/virtio/virtio.c | 37 +++++++++++++++++++++++++++---------- include/hw/virtio/virtio.h | 2 ++ 2 files changed, 29 insertions(+), 10 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index eb6347ab5d..f556e565c6 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -3642,8 +3642,26 @@ int virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id) vdev->start_on_kick = true; } + vdev->delay_check = true; + + if (vdc->post_load) { + ret = vdc->post_load(vdev); + if (ret) { + return ret; + } + } + + return 0; +} + +static void virtio_load_check_delay(VirtIODevice *vdev) +{ RCU_READ_LOCK_GUARD(); - for (i = 0; i < num; i++) { + for (int i = 0; i < VIRTIO_QUEUE_MAX; i++) { + if (vdev->vq[i].vring.num == 0) { + break; + } + if (vdev->vq[i].vring.desc) { uint16_t nheads; @@ -3696,19 +3714,12 @@ int virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id) i, vdev->vq[i].vring.num, vdev->vq[i].last_avail_idx, vdev->vq[i].used_idx); - return -1; + abort(); } } } - if (vdc->post_load) { - ret = vdc->post_load(vdev); - if (ret) { - return ret; - } - } - - return 0; + return; } void virtio_cleanup(VirtIODevice *vdev) @@ -3722,6 +3733,11 @@ static void virtio_vmstate_change(void *opaque, bool running, RunState state) BusState *qbus = qdev_get_parent_bus(DEVICE(vdev)); VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); bool backend_run = running && virtio_device_started(vdev, vdev->status); + + if (vdev->delay_check) { + virtio_load_check_delay(vdev); + vdev->delay_check = false; + } vdev->vm_running = running; if (backend_run) { @@ -3789,6 +3805,7 @@ void virtio_init(VirtIODevice *vdev, uint16_t device_id, size_t config_size) virtio_vmstate_change, vdev); vdev->device_endian = virtio_default_endian(); vdev->use_guest_notifier_mask = true; + vdev->delay_check = false; } /* diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index acfd4df125..269e80d04a 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -135,6 +135,8 @@ struct VirtIODevice AddressSpace *dma_as; QLIST_HEAD(, VirtQueue) *vector_queues; QTAILQ_ENTRY(VirtIODevice) next; + /* @delay_check: delay checks in virtio_load */ + bool delay_check; }; struct VirtioDeviceClass { From patchwork Tue Dec 13 13:35:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13072085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0FAA1C4167B for ; Tue, 13 Dec 2022 13:36:22 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p55Rn-00014h-2K; Tue, 13 Dec 2022 08:36:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p55RY-00013j-Cm for qemu-devel@nongnu.org; Tue, 13 Dec 2022 08:35:36 -0500 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p55RW-0002zc-Jo for qemu-devel@nongnu.org; Tue, 13 Dec 2022 08:35:35 -0500 Received: by mail-pl1-x629.google.com with SMTP id m4so15640442pls.4 for ; Tue, 13 Dec 2022 05:35:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zfmFKXvPbYDVO7QdoAaOz/IKaZzu933l8Zdg5qycnZA=; b=bO1sytPBMwImixy5yKcyKuDN2Jp/WkqPGf+ClLAH7XblbkijqLKOoHesA81+dEBPdv L3P8WEu1aMw7E87Sx98HK6fPc0M2LYQWufQIIGR52t/sCKWj/2v0gsc/QaVLu8DOwLlR RDfz5a0B00OHu7jvNk3TsedfwX9Q0rpUH/c5TiNZKXYblf/qWhxvQBW/XbTeFvmGU1V2 njqPhu92Xl9h3mGTx3xItUh9wbueM20G1bO9zN8ADvOkxBGJiEFH/+S+TiFUlR+RBaam SueVf+mhEPmajgHR0MFlU9D9IfL8348CS+YQbPhYMWqeWBKwNc1mCuFXJ+3AjBQEQ7sr DJOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zfmFKXvPbYDVO7QdoAaOz/IKaZzu933l8Zdg5qycnZA=; b=qHGK/IHOFKUq9VsdQQ784MrczO1bpClpHEed+s+OaxskIVUMwb9DXjM8t9bYEEncIw kmNRdERTmBdVhJA/S7o1NAQC38ZRgBwzCDeHIRIH7D+BE2diDO1GG5QCWVo9jHMFi0sJ r1GtzqsGdLfkrOwRNeI7tNLacYG6U3PLaxne1SFbyZqKjlexzJIAaFaSVCrUA8VbgE77 6Rhlwq9b0PmfFf7GNkcgA37b09JFITXeTA2CQga748rfmjENNIzTLEXP5nBKwMP+erzE psxzNMD/fblj9SJGyzLYDSft+SqgSxTKH5bJoKELrvCtTGU2Kz+f8aguaA58Cqs7oNY6 ElFw== X-Gm-Message-State: ANoB5pmzkQBTtPccaB/ibgTMRwjWExflIuyAgWYKWdHizUWLJbZVoaUZ Xa+IQB3LzJSG5oMev3Q5hNiqhRC1pCC3lxng X-Google-Smtp-Source: AA0mqf7nUBHrEE8Wd69u+eFPeY4JS9SV8kJcLXbuEcobchf1tpgFWf/YPTVuylU9oac2uPs6ZNRA8Q== X-Received: by 2002:a17:902:f114:b0:18f:aca1:b0c9 with SMTP id e20-20020a170902f11400b0018faca1b0c9mr4419515plb.53.1670938533704; Tue, 13 Dec 2022 05:35:33 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.12]) by smtp.gmail.com with ESMTPSA id b18-20020a170903229200b00182a9c27acfsm8440033plh.227.2022.12.13.05.35.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 05:35:33 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, f4bug@amsat.org, mst@redhat.com, zhouyibo@bytedance.com, Chuang Xu Subject: [RFC v3 3/3] migration: reduce time of loading non-iterable vmstate Date: Tue, 13 Dec 2022 21:35:10 +0800 Message-Id: <20221213133510.1279488-4-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221213133510.1279488-1-xuchuangxclwt@bytedance.com> References: <20221213133510.1279488-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pl1-x629.google.com X-Spam_score_int: 6 X-Spam_score: 0.6 X-Spam_bar: / X-Spam_report: (0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start). Most of the time is spent committing memory region changes repeatedly. This patch packs all the changes to memory region during the period of loading non-iterable vmstate in a single memory transaction. With the increase of devices, this patch will greatly improve the performance. Here are the test results: test vm info: - 32 CPUs 128GB RAM - 8 16-queue vhost-net device - 16 4-queue vhost-user-blk device. time of loading non-iterable vmstate before about 210 ms after about 40 ms Signed-off-by: Chuang Xu --- migration/savevm.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..19785e5a54 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2617,6 +2617,9 @@ int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis) uint8_t section_type; int ret = 0; + /* call memory_region_transaction_begin() before loading vmstate */ + memory_region_transaction_begin(); + retry: while (true) { section_type = qemu_get_byte(f); @@ -2684,6 +2687,10 @@ out: goto retry; } } + + /* call memory_region_transaction_commit() after loading vmstate */ + memory_region_transaction_commit(); + return ret; }