From patchwork Tue Oct 31 09:01:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 87942C4332F for ; Tue, 31 Oct 2023 09:08:10 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkdH-0007xs-3v; Tue, 31 Oct 2023 05:01:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdF-0007xV-SC for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:01:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdE-0006o7-F1 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:01:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742911; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VM08G+LHFsIqxiXEmkhxqOFMNoBjDlLFnd3SYaD5rAM=; b=C2Cj/Oqm3HRLoGPnNI7t9QySTr2kumOKW01m/D+H7agvWTNXUEgTk2AYlh6LQnDFxJ+alt Wu80VTCByEfq1vDc9wU/t2Gl8Pb0KQkqXLdiB1C9DXYgIhWG3sxTXZP3i6Hl2nHmH4Nx0Q /s9s5Mrfhj7KtLdXLfHByHCx8dek06o= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-374--Xde7hmzNciTnm9qmbpKhg-1; Tue, 31 Oct 2023 05:01:50 -0400 X-MC-Unique: -Xde7hmzNciTnm9qmbpKhg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 870C31C060C9; Tue, 31 Oct 2023 09:01:49 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 28D1C1C060AE; Tue, 31 Oct 2023 09:01:47 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 01/38] migration/doc: Add contents Date: Tue, 31 Oct 2023 10:01:05 +0100 Message-ID: <20231031090142.13122-2-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231018112827.1325-2-quintela@redhat.com> --- docs/devel/migration.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst index c3e1400c0c..4d6a98ae58 100644 --- a/docs/devel/migration.rst +++ b/docs/devel/migration.rst @@ -28,6 +28,8 @@ the guest to be stopped. Typically the time that the guest is unresponsive during live migration is the low hundred of milliseconds (notice that this depends on a lot of things). +.. contents:: + Transports ========== From patchwork Tue Oct 31 09:01:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 62EA2C4332F for ; Tue, 31 Oct 2023 09:07:44 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkde-00087Z-6v; Tue, 31 Oct 2023 05:02:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdW-00082X-Hw for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdT-0006pT-67 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742926; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wCFHcvyVWdab+kL4xChxY/YWGrNKy4P/u8/VAcJabY0=; b=S+7LmoX78xuL005rs6yLzCLhFg7sxs5GJEjLVKPeR6kOefb9KCjhhsFYFpU+yJP4i0By4T HPgsj2Zjylk+Mia6ix/15J7XicsJTYExvhQj2sMBufl1ak8FVvVAVkV9cf8ItHmKtz6kHD o3AZbv2nvqxxA2gdlq8TU5Yb6Cqt1fY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-202-lYLwbCPpPpyuKXx1sscNvA-1; Tue, 31 Oct 2023 05:01:53 -0400 X-MC-Unique: lYLwbCPpPpyuKXx1sscNvA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9B59A185A784; Tue, 31 Oct 2023 09:01:52 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id CB2E51C060AE; Tue, 31 Oct 2023 09:01:49 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , Vladimir Sementsov-Ogievskiy Subject: [PULL 02/38] migration/doc: Add documentation for backwards compatiblity Date: Tue, 31 Oct 2023 10:01:06 +0100 Message-ID: <20231031090142.13122-3-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org State what are the requeriments to get migration working between qemu versions. And once there explain how one is supposed to implement a new feature/default value and not break migration. Reviewed-by: Vladimir Sementsov-Ogievskiy Acked-by: Peter Xu Signed-off-by: Juan Quintela Message-ID: <20231018112827.1325-3-quintela@redhat.com> --- docs/devel/migration.rst | 219 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 219 insertions(+) diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst index 4d6a98ae58..6fe275b1ec 100644 --- a/docs/devel/migration.rst +++ b/docs/devel/migration.rst @@ -919,3 +919,222 @@ versioned machine types to cut down on the combinations that will need support. This is also useful when newer versions of firmware outgrow the padding. + +Backwards compatibility +======================= + +How backwards compatibility works +--------------------------------- + +When we do migration, we have two QEMU processes: the source and the +target. There are two cases, they are the same version or they are +different versions. The easy case is when they are the same version. +The difficult one is when they are different versions. + +There are two things that are different, but they have very similar +names and sometimes get confused: + +- QEMU version +- machine type version + +Let's start with a practical example, we start with: + +- qemu-system-x86_64 (v5.2), from now on qemu-5.2. +- qemu-system-x86_64 (v5.1), from now on qemu-5.1. + +Related to this are the "latest" machine types defined on each of +them: + +- pc-q35-5.2 (newer one in qemu-5.2) from now on pc-5.2 +- pc-q35-5.1 (newer one in qemu-5.1) from now on pc-5.1 + +First of all, migration is only supposed to work if you use the same +machine type in both source and destination. The QEMU hardware +configuration needs to be the same also on source and destination. +Most aspects of the backend configuration can be changed at will, +except for a few cases where the backend features influence frontend +device feature exposure. But that is not relevant for this section. + +I am going to list the number of combinations that we can have. Let's +start with the trivial ones, QEMU is the same on source and +destination: + +1 - qemu-5.2 -M pc-5.2 -> migrates to -> qemu-5.2 -M pc-5.2 + + This is the latest QEMU with the latest machine type. + This have to work, and if it doesn't work it is a bug. + +2 - qemu-5.1 -M pc-5.1 -> migrates to -> qemu-5.1 -M pc-5.1 + + Exactly the same case than the previous one, but for 5.1. + Nothing to see here either. + +This are the easiest ones, we will not talk more about them in this +section. + +Now we start with the more interesting cases. Consider the case where +we have the same QEMU version in both sides (qemu-5.2) but we are using +the latest machine type for that version (pc-5.2) but one of an older +QEMU version, in this case pc-5.1. + +3 - qemu-5.2 -M pc-5.1 -> migrates to -> qemu-5.2 -M pc-5.1 + + It needs to use the definition of pc-5.1 and the devices as they + were configured on 5.1, but this should be easy in the sense that + both sides are the same QEMU and both sides have exactly the same + idea of what the pc-5.1 machine is. + +4 - qemu-5.1 -M pc-5.2 -> migrates to -> qemu-5.1 -M pc-5.2 + + This combination is not possible as the qemu-5.1 doen't understand + pc-5.2 machine type. So nothing to worry here. + +Now it comes the interesting ones, when both QEMU processes are +different. Notice also that the machine type needs to be pc-5.1, +because we have the limitation than qemu-5.1 doesn't know pc-5.2. So +the possible cases are: + +5 - qemu-5.2 -M pc-5.1 -> migrates to -> qemu-5.1 -M pc-5.1 + + This migration is known as newer to older. We need to make sure + when we are developing 5.2 we need to take care about not to break + migration to qemu-5.1. Notice that we can't make updates to + qemu-5.1 to understand whatever qemu-5.2 decides to change, so it is + in qemu-5.2 side to make the relevant changes. + +6 - qemu-5.1 -M pc-5.1 -> migrates to -> qemu-5.2 -M pc-5.1 + + This migration is known as older to newer. We need to make sure + than we are able to receive migrations from qemu-5.1. The problem is + similar to the previous one. + +If qemu-5.1 and qemu-5.2 were the same, there will not be any +compatibility problems. But the reason that we create qemu-5.2 is to +get new features, devices, defaults, etc. + +If we get a device that has a new feature, or change a default value, +we have a problem when we try to migrate between different QEMU +versions. + +So we need a way to tell qemu-5.2 that when we are using machine type +pc-5.1, it needs to **not** use the feature, to be able to migrate to +real qemu-5.1. + +And the equivalent part when migrating from qemu-5.1 to qemu-5.2. +qemu-5.2 has to expect that it is not going to get data for the new +feature, because qemu-5.1 doesn't know about it. + +How do we tell QEMU about these device feature changes? In +hw/core/machine.c:hw_compat_X_Y arrays. + +If we change a default value, we need to put back the old value on +that array. And the device, during initialization needs to look at +that array to see what value it needs to get for that feature. And +what are we going to put in that array, the value of a property. + +To create a property for a device, we need to use one of the +DEFINE_PROP_*() macros. See include/hw/qdev-properties.h to find the +macros that exist. With it, we set the default value for that +property, and that is what it is going to get in the latest released +version. But if we want a different value for a previous version, we +can change that in the hw_compat_X_Y arrays. + +hw_compat_X_Y is an array of registers that have the format: + +- name_device +- name_property +- value + +Let's see a practical example. + +In qemu-5.2 virtio-blk-device got multi queue support. This is a +change that is not backward compatible. In qemu-5.1 it has one +queue. In qemu-5.2 it has the same number of queues as the number of +cpus in the system. + +When we are doing migration, if we migrate from a device that has 4 +queues to a device that have only one queue, we don't know where to +put the extra information for the other 3 queues, and we fail +migration. + +Similar problem when we migrate from qemu-5.1 that has only one queue +to qemu-5.2, we only sent information for one queue, but destination +has 4, and we have 3 queues that are not properly initialized and +anything can happen. + +So, how can we address this problem. Easy, just convince qemu-5.2 +that when it is running pc-5.1, it needs to set the number of queues +for virtio-blk-devices to 1. + +That way we fix the cases 5 and 6. + +5 - qemu-5.2 -M pc-5.1 -> migrates to -> qemu-5.1 -M pc-5.1 + + qemu-5.2 -M pc-5.1 sets number of queues to be 1. + qemu-5.1 -M pc-5.1 expects number of queues to be 1. + + correct. migration works. + +6 - qemu-5.1 -M pc-5.1 -> migrates to -> qemu-5.2 -M pc-5.1 + + qemu-5.1 -M pc-5.1 sets number of queues to be 1. + qemu-5.2 -M pc-5.1 expects number of queues to be 1. + + correct. migration works. + +And now the other interesting case, case 3. In this case we have: + +3 - qemu-5.2 -M pc-5.1 -> migrates to -> qemu-5.2 -M pc-5.1 + + Here we have the same QEMU in both sides. So it doesn't matter a + lot if we have set the number of queues to 1 or not, because + they are the same. + + WRONG! + + Think what happens if we do one of this double migrations: + + A -> migrates -> B -> migrates -> C + + where: + + A: qemu-5.1 -M pc-5.1 + B: qemu-5.2 -M pc-5.1 + C: qemu-5.2 -M pc-5.1 + + migration A -> B is case 6, so number of queues needs to be 1. + + migration B -> C is case 3, so we don't care. But actually we + care because we haven't started the guest in qemu-5.2, it came + migrated from qemu-5.1. So to be in the safe place, we need to + always use number of queues 1 when we are using pc-5.1. + +Now, how was this done in reality? The following commit shows how it +was done:: + + commit 9445e1e15e66c19e42bea942ba810db28052cd05 + Author: Stefan Hajnoczi + Date: Tue Aug 18 15:33:47 2020 +0100 + + virtio-blk-pci: default num_queues to -smp N + +The relevant parts for migration are:: + + @@ -1281,7 +1284,8 @@ static Property virtio_blk_properties[] = { + #endif + DEFINE_PROP_BIT("request-merging", VirtIOBlock, conf.request_merging, 0, + true), + - DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, 1), + + DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, + + VIRTIO_BLK_AUTO_NUM_QUEUES), + DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size, 256), + +It changes the default value of num_queues. But it fishes it for old +machine types to have the right value:: + + @@ -31,6 +31,7 @@ + GlobalProperty hw_compat_5_1[] = { + ... + + { "virtio-blk-device", "num-queues", "1"}, + ... + }; From patchwork Tue Oct 31 09:01:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DF7D7C4332F for ; Tue, 31 Oct 2023 09:08:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkdU-0007zd-GN; Tue, 31 Oct 2023 05:02:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdO-0007yx-Ht for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdM-0006ob-Ih for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742918; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0wLme3tPLHrsytkaTz20AbsMQdVkg50xoNnDIj0EyAo=; b=cVlNfIOKzCmwUPalvGNh/Gxlz0MJumo8aEwoe+1aAfnbVg7lYLSPB70Ui8uIkLGO0cgh3B MnNNioIszvt+iFdMdMQYZi7pgNyQLODIHaNIENWyg25Co9RRz9FLic+Sj7KRCtiwNLigie HdE0GLvlsDYXj5CUCz3k18OGQ/fJpes= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-390-NjjZGfGoMBmxH64qk-RTcA-1; Tue, 31 Oct 2023 05:01:55 -0400 X-MC-Unique: NjjZGfGoMBmxH64qk-RTcA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4E5AE3C100A7; Tue, 31 Oct 2023 09:01:55 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id E3B2A1C060AE; Tue, 31 Oct 2023 09:01:52 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 03/38] migration/doc: How to migrate when hosts have different features Date: Tue, 31 Oct 2023 10:01:07 +0100 Message-ID: <20231031090142.13122-4-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sometimes devices have different features depending of things outside of qemu. For instance the kernel. Document how to handle that cases. Acked-by: Peter Xu Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231018112827.1325-4-quintela@redhat.com> --- docs/devel/migration.rst | 97 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst index 6fe275b1ec..974505e4a7 100644 --- a/docs/devel/migration.rst +++ b/docs/devel/migration.rst @@ -1138,3 +1138,100 @@ machine types to have the right value:: + { "virtio-blk-device", "num-queues", "1"}, ... }; + +A device with diferent features on both sides +--------------------------------------------- + +Let's assume that we are using the same QEMU binary on both sides, +just to make the things easier. But we have a device that has +different features on both sides of the migration. That can be +because the devices are different, because the kernel driver of both +devices have different features, whatever. + +How can we get this to work with migration. The way to do that is +"theoretically" easy. You have to get the features that the device +has in the source of the migration. The features that the device has +on the target of the migration, you get the intersection of the +features of both sides, and that is the way that you should launch +QEMU. + +Notice that this is not completely related to QEMU. The most +important thing here is that this should be handled by the managing +application that launches QEMU. If QEMU is configured correctly, the +migration will succeed. + +That said, actually doing it is complicated. Almost all devices are +bad at being able to be launched with only some features enabled. +With one big exception: cpus. + +You can read the documentation for QEMU x86 cpu models here: + +https://qemu-project.gitlab.io/qemu/system/qemu-cpu-models.html + +See when they talk about migration they recommend that one chooses the +newest cpu model that is supported for all cpus. + +Let's say that we have: + +Host A: + +Device X has the feature Y + +Host B: + +Device X has not the feature Y + +If we try to migrate without any care from host A to host B, it will +fail because when migration tries to load the feature Y on +destination, it will find that the hardware is not there. + +Doing this would be the equivalent of doing with cpus: + +Host A: + +$ qemu-system-x86_64 -cpu host + +Host B: + +$ qemu-system-x86_64 -cpu host + +When both hosts have different cpu features this is guaranteed to +fail. Especially if Host B has less features than host A. If host A +has less features than host B, sometimes it works. Important word of +last sentence is "sometimes". + +So, forgetting about cpu models and continuing with the -cpu host +example, let's see that the differences of the cpus is that Host A and +B have the following features: + +Features: 'pcid' 'stibp' 'taa-no' +Host A: X X +Host B: X + +And we want to migrate between them, the way configure both QEMU cpu +will be: + +Host A: + +$ qemu-system-x86_64 -cpu host,pcid=off,stibp=off + +Host B: + +$ qemu-system-x86_64 -cpu host,taa-no=off + +And you would be able to migrate between them. It is responsability +of the management application or of the user to make sure that the +configuration is correct. QEMU doesn't know how to look at this kind +of features in general. + +Notice that we don't recomend to use -cpu host for migration. It is +used in this example because it makes the example simpler. + +Other devices have worse control about individual features. If they +want to be able to migrate between hosts that show different features, +the device needs a way to configure which ones it is going to use. + +In this section we have considered that we are using the same QEMU +binary in both sides of the migration. If we use different QEMU +versions process, then we need to have into account all other +differences and the examples become even more complicated. From patchwork Tue Oct 31 09:01:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441284 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 85B63C4332F for ; Tue, 31 Oct 2023 09:03:54 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkdb-00085I-8S; Tue, 31 Oct 2023 05:02:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdQ-0007zH-Fy for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdO-0006oo-4u for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742920; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ag5wALHI/WZ7Tu3k00h4RSWbnUl7APd5QuUK7txDLXI=; b=grTjsOMvuEkjtP1c0S5n+WOWrTC8EghFv2693bW3r7J5VwZPMJ+CNe4pyyOODSPm7dOy11 LXd/W3pfKGX8x2edlrxUxl8xmtcnktKGAww216ZbLgdLsZjOHhGmn4aEkG5ZvxzjapyqDP QD3bACB7cxzy2yjiLlRAiPtKvvXhI+M= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-178-KG-xpdgNMkmrHZiGpDCAqg-1; Tue, 31 Oct 2023 05:01:58 -0400 X-MC-Unique: KG-xpdgNMkmrHZiGpDCAqg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 02BF529AB3E3; Tue, 31 Oct 2023 09:01:58 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 96FD81C060AE; Tue, 31 Oct 2023 09:01:55 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 04/38] migration/doc: We broke backwards compatibility Date: Tue, 31 Oct 2023 10:01:08 +0100 Message-ID: <20231031090142.13122-5-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org When we detect that we have broken backwards compatibility in a released version, we can't do anything for that version. But once we fix that bug on the next released version, we can "mitigate" that problem when migrating to new versions to give a way out of that machine until it does a hard reboot. Acked-by: Peter Xu Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231018112827.1325-5-quintela@redhat.com> --- docs/devel/migration.rst | 202 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 202 insertions(+) diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst index 974505e4a7..be913630c3 100644 --- a/docs/devel/migration.rst +++ b/docs/devel/migration.rst @@ -1235,3 +1235,205 @@ In this section we have considered that we are using the same QEMU binary in both sides of the migration. If we use different QEMU versions process, then we need to have into account all other differences and the examples become even more complicated. + +How to mitigate when we have a backward compatibility error +----------------------------------------------------------- + +We broke migration for old machine types continuously during +development. But as soon as we find that there is a problem, we fix +it. The problem is what happens when we detect after we have done a +release that something has gone wrong. + +Let see how it worked with one example. + +After the release of qemu-8.0 we found a problem when doing migration +of the machine type pc-7.2. + +- $ qemu-7.2 -M pc-7.2 -> qemu-7.2 -M pc-7.2 + + This migration works + +- $ qemu-8.0 -M pc-7.2 -> qemu-8.0 -M pc-7.2 + + This migration works + +- $ qemu-8.0 -M pc-7.2 -> qemu-7.2 -M pc-7.2 + + This migration fails + +- $ qemu-7.2 -M pc-7.2 -> qemu-8.0 -M pc-7.2 + + This migration fails + +So clearly something fails when migration between qemu-7.2 and +qemu-8.0 with machine type pc-7.2. The error messages, and git bisect +pointed to this commit. + +In qemu-8.0 we got this commit:: + + commit 010746ae1db7f52700cb2e2c46eb94f299cfa0d2 + Author: Jonathan Cameron + Date: Thu Mar 2 13:37:02 2023 +0000 + + hw/pci/aer: Implement PCI_ERR_UNCOR_MASK register + + +The relevant bits of the commit for our example are this ones:: + + --- a/hw/pci/pcie_aer.c + +++ b/hw/pci/pcie_aer.c + @@ -112,6 +112,10 @@ int pcie_aer_init(PCIDevice *dev, + + pci_set_long(dev->w1cmask + offset + PCI_ERR_UNCOR_STATUS, + PCI_ERR_UNC_SUPPORTED); + + pci_set_long(dev->config + offset + PCI_ERR_UNCOR_MASK, + + PCI_ERR_UNC_MASK_DEFAULT); + + pci_set_long(dev->wmask + offset + PCI_ERR_UNCOR_MASK, + + PCI_ERR_UNC_SUPPORTED); + + pci_set_long(dev->config + offset + PCI_ERR_UNCOR_SEVER, + PCI_ERR_UNC_SEVERITY_DEFAULT); + +The patch changes how we configure PCI space for AER. But QEMU fails +when the PCI space configuration is different between source and +destination. + +The following commit shows how this got fixed:: + + commit 5ed3dabe57dd9f4c007404345e5f5bf0e347317f + Author: Leonardo Bras + Date: Tue May 2 21:27:02 2023 -0300 + + hw/pci: Disable PCI_ERR_UNCOR_MASK register for machine type < 8.0 + + [...] + +The relevant parts of the fix in QEMU are as follow: + +First, we create a new property for the device to be able to configure +the old behaviour or the new behaviour:: + + diff --git a/hw/pci/pci.c b/hw/pci/pci.c + index 8a87ccc8b0..5153ad63d6 100644 + --- a/hw/pci/pci.c + +++ b/hw/pci/pci.c + @@ -79,6 +79,8 @@ static Property pci_props[] = { + DEFINE_PROP_STRING("failover_pair_id", PCIDevice, + failover_pair_id), + DEFINE_PROP_UINT32("acpi-index", PCIDevice, acpi_index, 0), + + DEFINE_PROP_BIT("x-pcie-err-unc-mask", PCIDevice, cap_present, + + QEMU_PCIE_ERR_UNC_MASK_BITNR, true), + DEFINE_PROP_END_OF_LIST() + }; + +Notice that we enable the feature for new machine types. + +Now we see how the fix is done. This is going to depend on what kind +of breakage happens, but in this case it is quite simple:: + + diff --git a/hw/pci/pcie_aer.c b/hw/pci/pcie_aer.c + index 103667c368..374d593ead 100644 + --- a/hw/pci/pcie_aer.c + +++ b/hw/pci/pcie_aer.c + @@ -112,10 +112,13 @@ int pcie_aer_init(PCIDevice *dev, uint8_t cap_ver, + uint16_t offset, + + pci_set_long(dev->w1cmask + offset + PCI_ERR_UNCOR_STATUS, + PCI_ERR_UNC_SUPPORTED); + - pci_set_long(dev->config + offset + PCI_ERR_UNCOR_MASK, + - PCI_ERR_UNC_MASK_DEFAULT); + - pci_set_long(dev->wmask + offset + PCI_ERR_UNCOR_MASK, + - PCI_ERR_UNC_SUPPORTED); + + + + if (dev->cap_present & QEMU_PCIE_ERR_UNC_MASK) { + + pci_set_long(dev->config + offset + PCI_ERR_UNCOR_MASK, + + PCI_ERR_UNC_MASK_DEFAULT); + + pci_set_long(dev->wmask + offset + PCI_ERR_UNCOR_MASK, + + PCI_ERR_UNC_SUPPORTED); + + } + + pci_set_long(dev->config + offset + PCI_ERR_UNCOR_SEVER, + PCI_ERR_UNC_SEVERITY_DEFAULT); + +I.e. If the property bit is enabled, we configure it as we did for +qemu-8.0. If the property bit is not set, we configure it as it was in 7.2. + +And now, everything that is missing is disabling the feature for old +machine types:: + + diff --git a/hw/core/machine.c b/hw/core/machine.c + index 47a34841a5..07f763eb2e 100644 + --- a/hw/core/machine.c + +++ b/hw/core/machine.c + @@ -48,6 +48,7 @@ GlobalProperty hw_compat_7_2[] = { + { "e1000e", "migrate-timadj", "off" }, + { "virtio-mem", "x-early-migration", "false" }, + { "migration", "x-preempt-pre-7-2", "true" }, + + { TYPE_PCI_DEVICE, "x-pcie-err-unc-mask", "off" }, + }; + const size_t hw_compat_7_2_len = G_N_ELEMENTS(hw_compat_7_2); + +And now, when qemu-8.0.1 is released with this fix, all combinations +are going to work as supposed. + +- $ qemu-7.2 -M pc-7.2 -> qemu-7.2 -M pc-7.2 (works) +- $ qemu-8.0.1 -M pc-7.2 -> qemu-8.0.1 -M pc-7.2 (works) +- $ qemu-8.0.1 -M pc-7.2 -> qemu-7.2 -M pc-7.2 (works) +- $ qemu-7.2 -M pc-7.2 -> qemu-8.0.1 -M pc-7.2 (works) + +So the normality has been restored and everything is ok, no? + +Not really, now our matrix is much bigger. We started with the easy +cases, migration from the same version to the same version always +works: + +- $ qemu-7.2 -M pc-7.2 -> qemu-7.2 -M pc-7.2 +- $ qemu-8.0 -M pc-7.2 -> qemu-8.0 -M pc-7.2 +- $ qemu-8.0.1 -M pc-7.2 -> qemu-8.0.1 -M pc-7.2 + +Now the interesting ones. When the QEMU processes versions are +different. For the 1st set, their fail and we can do nothing, both +versions are released and we can't change anything. + +- $ qemu-7.2 -M pc-7.2 -> qemu-8.0 -M pc-7.2 +- $ qemu-8.0 -M pc-7.2 -> qemu-7.2 -M pc-7.2 + +This two are the ones that work. The whole point of making the +change in qemu-8.0.1 release was to fix this issue: + +- $ qemu-7.2 -M pc-7.2 -> qemu-8.0.1 -M pc-7.2 +- $ qemu-8.0.1 -M pc-7.2 -> qemu-7.2 -M pc-7.2 + +But now we found that qemu-8.0 neither can migrate to qemu-7.2 not +qemu-8.0.1. + +- $ qemu-8.0 -M pc-7.2 -> qemu-8.0.1 -M pc-7.2 +- $ qemu-8.0.1 -M pc-7.2 -> qemu-8.0 -M pc-7.2 + +So, if we start a pc-7.2 machine in qemu-8.0 we can't migrate it to +anything except to qemu-8.0. + +Can we do better? + +Yeap. If we know that we are going to do this migration: + +- $ qemu-8.0 -M pc-7.2 -> qemu-8.0.1 -M pc-7.2 + +We can launch the appropriate devices with:: + + --device...,x-pci-e-err-unc-mask=on + +And now we can receive a migration from 8.0. And from now on, we can +do that migration to new machine types if we remember to enable that +property for pc-7.2. Notice that we need to remember, it is not +enough to know that the source of the migration is qemu-8.0. Think of +this example: + +$ qemu-8.0 -M pc-7.2 -> qemu-8.0.1 -M pc-7.2 -> qemu-8.2 -M pc-7.2 + +In the second migration, the source is not qemu-8.0, but we still have +that "problem" and have that property enabled. Notice that we need to +continue having this mark/property until we have this machine +rebooted. But it is not a normal reboot (that don't reload QEMU) we +need the machine to poweroff/poweron on a fixed QEMU. And from now +on we can use the proper real machine. From patchwork Tue Oct 31 09:01:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E831DC4332F for ; Tue, 31 Oct 2023 09:03:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkdh-0008AI-4q; Tue, 31 Oct 2023 05:02:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdd-00086v-KL for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkda-0006rA-Mq for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742933; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wSytcPgXUihOAo8jO1TEKPhY/Cs4SiCxtxaCJgHSibA=; b=iamWVlqqoLfCse/IoWTW9CBWC6vf8e2sm+QmbPfiFeZbHZGEXqZdCTMVUIQSosg7lcbweQ Zyte5azMCqPvZ0lxmzuc2x00E8GJKtRwX7osHYOCqIZWE5wy1iB/p3M3iYEcwLhdIPUhza RxQc6D7ZbNOUMTbUIhD7NG24XZ9mwog= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-225-INxsbuvsNhae8lCVBAtL9w-1; Tue, 31 Oct 2023 05:02:01 -0400 X-MC-Unique: INxsbuvsNhae8lCVBAtL9w-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DC59A3822549; Tue, 31 Oct 2023 09:02:00 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4A1B61C060AE; Tue, 31 Oct 2023 09:01:58 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 05/38] migration: Receiving a zero page non zero is an error Date: Tue, 31 Oct 2023 10:01:09 +0100 Message-ID: <20231031090142.13122-6-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We don't allow non zero compressed pages since: commit 3edcd7e6ebae3ef0ac178eed5f4225803159562d Author: Peter Lieven Date: Tue Mar 26 10:58:35 2013 +0100 migration: search for zero instead of dup pages RDMA case is a bit more complicated, but they don't handle it since: commit a1febc4950f2c6232c002f401d7cd409f6fa6a88 Author: Richard Henderson Date: Mon Aug 29 11:46:14 2016 -0700 cutils: Export only buffer_is_zero Reviewed-by: Fabiano Rosas Reviewed-by: Peter Xu Signed-off-by: Juan Quintela Message-ID: <20231019085259.13307-2-quintela@redhat.com> --- migration/ram.c | 15 +++++++++++---- migration/rdma.c | 6 +++++- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 92769902bb..4bfb20c94a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3715,16 +3715,18 @@ int ram_load_postcopy(QEMUFile *f, int channel) switch (flags & ~RAM_SAVE_FLAG_CONTINUE) { case RAM_SAVE_FLAG_ZERO: ch = qemu_get_byte(f); + if (ch != 0) { + error_report("Found a zero page with value %d", ch); + ret = -EINVAL; + break; + } /* * Can skip to set page_buffer when * this is a zero page and (block->page_size == TARGET_PAGE_SIZE). */ - if (ch || !matches_target_page_size) { + if (!matches_target_page_size) { memset(page_buffer, ch, TARGET_PAGE_SIZE); } - if (ch) { - tmp_page->all_zero = false; - } break; case RAM_SAVE_FLAG_PAGE: @@ -4030,6 +4032,11 @@ static int ram_load_precopy(QEMUFile *f) case RAM_SAVE_FLAG_ZERO: ch = qemu_get_byte(f); + if (ch != 0) { + error_report("Found a zero page with value %d", ch); + ret = -EINVAL; + break; + } ram_handle_compressed(host, ch, TARGET_PAGE_SIZE); break; diff --git a/migration/rdma.c b/migration/rdma.c index 2a1852ec7f..2d963fd147 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3592,7 +3592,11 @@ int rdma_registration_handle(QEMUFile *f) host_addr = block->local_host_addr + (comp->offset - block->offset); - + if (comp->value) { + error_report("rdma: Zero page with non-zero (%d) value", + comp->value); + goto err; + } ram_handle_compressed(host_addr, comp->value, comp->length); break; From patchwork Tue Oct 31 09:01:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A9CD1C4167B for ; Tue, 31 Oct 2023 09:03:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkdf-00089S-Aj; Tue, 31 Oct 2023 05:02:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdX-00082n-Qi for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdW-0006q7-5U for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742929; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l9wUW2OQfxfDQsGzuQG2LFoAYA/fNyU0RqVbl7N/h3E=; b=OSmSRpOyUHVYUnX04DwSuIGqAY6YmyNgaP37i7kQMrwPEpf54nuw0y/WgOY8G7DqIzhJnL WocyJqqDfKPG4TfKtIATaAbOA6UAwR0CttfazMYA9vaaAR3PdQ6H8ev7IlWhIYzMdeopMy 4Y7jpMJg8Ex/1uR72DpRwXMROb0YaFc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-241-U_m1cco0OYip-IxPD_qqgw-1; Tue, 31 Oct 2023 05:02:04 -0400 X-MC-Unique: U_m1cco0OYip-IxPD_qqgw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 97816811E7B; Tue, 31 Oct 2023 09:02:03 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3004A1C060AE; Tue, 31 Oct 2023 09:02:01 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 06/38] migration: Rename ram_handle_compressed() to ram_handle_zero() Date: Tue, 31 Oct 2023 10:01:10 +0100 Message-ID: <20231031090142.13122-7-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Now that we know it only handles zero, we can remove the ch parameter. Reviewed-by: Fabiano Rosas Reviewed-by: Peter Xu Signed-off-by: Juan Quintela Message-ID: <20231019085259.13307-3-quintela@redhat.com> --- migration/ram.h | 2 +- migration/ram.c | 10 +++++----- migration/rdma.c | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/migration/ram.h b/migration/ram.h index 145c915ca7..3f724b2f02 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -60,7 +60,7 @@ int ram_discard_range(const char *block_name, uint64_t start, size_t length); int ram_postcopy_incoming_init(MigrationIncomingState *mis); int ram_load_postcopy(QEMUFile *f, int channel); -void ram_handle_compressed(void *host, uint8_t ch, uint64_t size); +void ram_handle_zero(void *host, uint64_t size); void ram_transferred_add(uint64_t bytes); void ram_release_page(const char *rbname, uint64_t offset); diff --git a/migration/ram.c b/migration/ram.c index 4bfb20c94a..e0ad732ee8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3446,7 +3446,7 @@ static inline void *colo_cache_from_block_offset(RAMBlock *block, } /** - * ram_handle_compressed: handle the zero page case + * ram_handle_zero: handle the zero page case * * If a page (or a whole RDMA chunk) has been * determined to be zero, then zap it. @@ -3455,10 +3455,10 @@ static inline void *colo_cache_from_block_offset(RAMBlock *block, * @ch: what the page is filled from. We only support zero * @size: size of the zero page */ -void ram_handle_compressed(void *host, uint8_t ch, uint64_t size) +void ram_handle_zero(void *host, uint64_t size) { - if (ch != 0 || !buffer_is_zero(host, size)) { - memset(host, ch, size); + if (!buffer_is_zero(host, size)) { + memset(host, 0, size); } } @@ -4037,7 +4037,7 @@ static int ram_load_precopy(QEMUFile *f) ret = -EINVAL; break; } - ram_handle_compressed(host, ch, TARGET_PAGE_SIZE); + ram_handle_zero(host, TARGET_PAGE_SIZE); break; case RAM_SAVE_FLAG_PAGE: diff --git a/migration/rdma.c b/migration/rdma.c index 2d963fd147..e3493e3b3e 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3597,7 +3597,7 @@ int rdma_registration_handle(QEMUFile *f) comp->value); goto err; } - ram_handle_compressed(host_addr, comp->value, comp->length); + ram_handle_zero(host_addr, comp->length); break; case RDMA_CONTROL_REGISTER_FINISHED: From patchwork Tue Oct 31 09:01:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 63065C4332F for ; Tue, 31 Oct 2023 09:06:05 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkde-00088E-MP; Tue, 31 Oct 2023 05:02:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdX-00082m-HO for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdW-0006q3-4M for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742928; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Pnb5H2GzAD+4zbTepRAdsHMXrQuph0eNuqqAzzPZHHc=; b=K/YrzXORRm8XuBG8yNA03B8gL13yQxlzKJ3XiqzQWsP95n1ixajuI3IQsqUVWPsRGiXiuF pa7jclR5HphdA1KU9LrNOCLRH2v83Jn0qvsCXYLG5UIKp4+9CLigw4764iKbmMZwGMyylF WY5UamKob65FQp/xazx0MsHKbV/24vs= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-336-lhYZwqNWMZ6wVMhe7vNRiA-1; Tue, 31 Oct 2023 05:02:06 -0400 X-MC-Unique: lhYZwqNWMZ6wVMhe7vNRiA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 481AD1C060C9; Tue, 31 Oct 2023 09:02:06 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id DFF741C060B1; Tue, 31 Oct 2023 09:02:03 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 07/38] migration: Give one error if trying to set MULTIFD and XBZRLE Date: Tue, 31 Oct 2023 10:01:11 +0100 Message-ID: <20231031090142.13122-8-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-2-quintela@redhat.com> --- migration/options.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/migration/options.c b/migration/options.c index 42fb818956..b8c3c3218d 100644 --- a/migration/options.c +++ b/migration/options.c @@ -618,6 +618,13 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) } } + if (new_caps[MIGRATION_CAPABILITY_MULTIFD]) { + if (new_caps[MIGRATION_CAPABILITY_XBZRLE]) { + error_setg(errp, "Multifd is not compatible with xbzrle"); + return false; + } + } + return true; } From patchwork Tue Oct 31 09:01:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 63604C4332F for ; Tue, 31 Oct 2023 09:03:04 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkdh-0008AH-53; Tue, 31 Oct 2023 05:02:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdd-00086t-JG for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkda-0006qx-7r for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742932; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bzn6XgU4nv/l3ZtGp4EWYxpwxg6jKvAv8cH4Fj755pE=; b=cNwVknQIX+wzkREKuuZuIucLyHP2tieFQniFf9HsjWZarUngV58hguS5MpKg2fgCDS6K8O W5zi9j2FufFE+aPszy1qEQPXQSSZdnlC5+nahh4n1oOxfUyFFpouxkVPfeGmGKRTd2u0Q5 +6wrLgGT/FpM4XJn5PKaXEdoTSNeLAc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-688-y7YUOZlVOEiO7jLMnD1Zzg-1; Tue, 31 Oct 2023 05:02:09 -0400 X-MC-Unique: y7YUOZlVOEiO7jLMnD1Zzg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EBBE88A67E3; Tue, 31 Oct 2023 09:02:08 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8C66D1C060AE; Tue, 31 Oct 2023 09:02:06 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 08/38] migration: Give one error if trying to set COMPRESSION and XBZRLE Date: Tue, 31 Oct 2023 10:01:12 +0100 Message-ID: <20231031090142.13122-9-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-3-quintela@redhat.com> --- migration/options.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/migration/options.c b/migration/options.c index b8c3c3218d..37fa1cfe74 100644 --- a/migration/options.c +++ b/migration/options.c @@ -625,6 +625,13 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) } } + if (new_caps[MIGRATION_CAPABILITY_COMPRESS]) { + if (new_caps[MIGRATION_CAPABILITY_XBZRLE]) { + error_setg(errp, "Compression is not compatible with xbzrle"); + return false; + } + } + return true; } From patchwork Tue Oct 31 09:01:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441294 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0EC81C4167D for ; Tue, 31 Oct 2023 09:06:06 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkdr-0000KF-E7; Tue, 31 Oct 2023 05:02:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdp-00007r-3u for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdm-0006sb-G5 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742945; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kNC+yhxdlGzjH/kuVa1H0qDLD4e1tNF8NnwPKMQwLmo=; b=TA69bXKcgHji4T28UMUBX1yWokylCPFVOtjoUjrd7MFIKkhSy2Z0Lfh4B60MWpMiYxef/Y nrzWFITJeRWAL+ToJ0dUqF8i8R9TvefAo0pcLN8+0mhiccpUIz7fH/8IrtAZ45VkytJEaf qdDAXhGB5YBCkM8D/jtCreKVEVLtGQI= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-55-Co5VoTzzMeanaWxsz-VF-Q-1; Tue, 31 Oct 2023 05:02:12 -0400 X-MC-Unique: Co5VoTzzMeanaWxsz-VF-Q-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 998DB29AB3EA; Tue, 31 Oct 2023 09:02:11 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3A0AA1C060AE; Tue, 31 Oct 2023 09:02:09 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 09/38] migration: Remove save_page_use_compression() Date: Tue, 31 Oct 2023 10:01:13 +0100 Message-ID: <20231031090142.13122-10-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org After previous patch, we disable the posiblity that we use compression together with xbzrle. So we can use directly migrate_compress(). Once there, now we don't need the rs parameter, so remove it. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-4-quintela@redhat.com> --- migration/ram.c | 34 +++++++--------------------------- 1 file changed, 7 insertions(+), 27 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index e0ad732ee8..8246663f64 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1291,8 +1291,6 @@ static int ram_save_multifd_page(QEMUFile *file, RAMBlock *block, return 1; } -static bool save_page_use_compression(RAMState *rs); - static int send_queued_data(CompressParam *param) { PageSearchStatus *pss = &ram_state->pss[RAM_CHANNEL_PRECOPY]; @@ -1329,9 +1327,9 @@ static int send_queued_data(CompressParam *param) return len; } -static void ram_flush_compressed_data(RAMState *rs) +static void ram_flush_compressed_data(void) { - if (!save_page_use_compression(rs)) { + if (!migrate_compress()) { return; } @@ -1393,7 +1391,7 @@ static int find_dirty_block(RAMState *rs, PageSearchStatus *pss) * Also If xbzrle is on, stop using the data compression at this * point. In theory, xbzrle can do better than compression. */ - ram_flush_compressed_data(rs); + ram_flush_compressed_data(); /* Hit the end of the list */ pss->block = QLIST_FIRST_RCU(&ram_list.blocks); @@ -2042,24 +2040,6 @@ int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t len) return 0; } -static bool save_page_use_compression(RAMState *rs) -{ - if (!migrate_compress()) { - return false; - } - - /* - * If xbzrle is enabled (e.g., after first round of migration), stop - * using the data compression. In theory, xbzrle can do better than - * compression. - */ - if (rs->xbzrle_started) { - return false; - } - - return true; -} - /* * try to compress the page before posting it out, return true if the page * has been properly handled by compression, otherwise needs other @@ -2068,7 +2048,7 @@ static bool save_page_use_compression(RAMState *rs) static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, ram_addr_t offset) { - if (!save_page_use_compression(rs)) { + if (!migrate_compress()) { return false; } @@ -2083,7 +2063,7 @@ static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, * much CPU resource. */ if (pss->block != pss->last_sent_block) { - ram_flush_compressed_data(rs); + ram_flush_compressed_data(); return false; } @@ -3135,7 +3115,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) * page is sent in one chunk. */ if (migrate_postcopy_ram()) { - ram_flush_compressed_data(rs); + ram_flush_compressed_data(); } /* @@ -3236,7 +3216,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) } qemu_mutex_unlock(&rs->bitmap_mutex); - ram_flush_compressed_data(rs); + ram_flush_compressed_data(); int ret = rdma_registration_stop(f, RAM_CONTROL_FINISH); if (ret < 0) { From patchwork Tue Oct 31 09:01:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 28AFCC4332F for ; Tue, 31 Oct 2023 09:06:18 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkdn-0008TJ-SC; Tue, 31 Oct 2023 05:02:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdl-0008EN-4k for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdi-0006sC-U6 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742942; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FD/xrtx5Ex3UkXu99HPCWJPi/sgl1QuP9uITLor6Oos=; b=Zk19yZEjBaQ1qEhS/wqbf0eGKuSCf55NVtYLX+Yyof9bUV2fXPJfL8u57sndlp/zcSpkzs pCKnc1a9GUYqR3Uyb7qnbUk+7Q/d+AnW1fuOlRZ08mGNinV1H8qtPjWq0miFZaVbotOtRl 34PaqzkF/jFa9k1Lxa9pZiHPPf/5c9I= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-49-58hp-CY1N3a9YPu57ENTng-1; Tue, 31 Oct 2023 05:02:15 -0400 X-MC-Unique: 58hp-CY1N3a9YPu57ENTng-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8141485A58B; Tue, 31 Oct 2023 09:02:14 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id E13B31C060AE; Tue, 31 Oct 2023 09:02:11 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , Lukas Straub Subject: [PULL 10/38] migration: Make compress_data_with_multithreads return bool Date: Tue, 31 Oct 2023 10:01:14 +0100 Message-ID: <20231031090142.13122-11-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Reviewed-by: Lukas Straub Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-5-quintela@redhat.com> --- migration/ram-compress.h | 4 ++-- migration/ram-compress.c | 17 ++++++++++------- migration/ram.c | 3 +-- 3 files changed, 13 insertions(+), 11 deletions(-) diff --git a/migration/ram-compress.h b/migration/ram-compress.h index e55d3b50bd..b228640092 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -60,8 +60,8 @@ void compress_threads_save_cleanup(void); int compress_threads_save_setup(void); void flush_compressed_data(int (send_queued_data(CompressParam *))); -int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, - int (send_queued_data(CompressParam *))); +bool compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, + int (send_queued_data(CompressParam *))); int wait_for_decompress_done(void); void compress_threads_load_cleanup(void); diff --git a/migration/ram-compress.c b/migration/ram-compress.c index d037dfe6cf..ef03d60a6d 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -260,10 +260,13 @@ static inline void set_compress_params(CompressParam *param, RAMBlock *block, param->trigger = true; } -int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, - int (send_queued_data(CompressParam *))) +/* + * Return true when it compress a page + */ +bool compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, + int (send_queued_data(CompressParam *))) { - int thread_count, pages = -1; + int thread_count; bool wait = migrate_compress_wait_thread(); thread_count = migrate_compress_threads(); @@ -281,8 +284,8 @@ retry: qemu_cond_signal(¶m->cond); qemu_mutex_unlock(¶m->mutex); - pages = 1; - break; + qemu_mutex_unlock(&comp_done_lock); + return true; } } @@ -290,13 +293,13 @@ retry: * wait for the free thread if the user specifies 'compress-wait-thread', * otherwise we will post the page out in the main thread as normal page. */ - if (pages < 0 && wait) { + if (wait) { qemu_cond_wait(&comp_done_cond, &comp_done_lock); goto retry; } qemu_mutex_unlock(&comp_done_lock); - return pages; + return false; } /* return the size after decompression, or negative value on error */ diff --git a/migration/ram.c b/migration/ram.c index 8246663f64..63a575ae90 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2067,8 +2067,7 @@ static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, return false; } - if (compress_page_with_multi_thread(pss->block, offset, - send_queued_data) > 0) { + if (compress_page_with_multi_thread(pss->block, offset, send_queued_data)) { return true; } From patchwork Tue Oct 31 09:01:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441312 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CF519C00142 for ; Tue, 31 Oct 2023 09:08:10 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxke2-0000jr-14; Tue, 31 Oct 2023 05:02:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxke0-0000hp-An for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdy-0006uj-OJ for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742957; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v07XEObK+8HRkY6PIJKRF8OCoGMMogRwRKEZtGkEDCc=; b=EiawGvqFrSEakbl2yzZLdL4E0CNuY2+Xqz+nU8yQ6QYtWVxco+XD02akA9pHsjjDorzf5m 6iJeAPD5F4o6wgdeVuPIZadjANAgD5rVyXpcRW0Pl5m8pK2GEsQOKKKslxSma84hJnnTXe Kls7ogpQxpgL+Z01xYfvUIm+o7B2Jm8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-655-Rt5vKY5RP3O57pE23OkKaQ-1; Tue, 31 Oct 2023 05:02:17 -0400 X-MC-Unique: Rt5vKY5RP3O57pE23OkKaQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5FDC78A67E4; Tue, 31 Oct 2023 09:02:17 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id C66741C060AE; Tue, 31 Oct 2023 09:02:14 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , Lukas Straub Subject: [PULL 11/38] migration: Simplify compress_page_with_multithread() Date: Tue, 31 Oct 2023 10:01:15 +0100 Message-ID: <20231031090142.13122-12-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Move the goto to a while true. Reviewed-by: Lukas Straub Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-6-quintela@redhat.com> --- migration/ram-compress.c | 48 ++++++++++++++++++++-------------------- 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/migration/ram-compress.c b/migration/ram-compress.c index ef03d60a6d..a991b15b7a 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -271,35 +271,35 @@ bool compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, thread_count = migrate_compress_threads(); qemu_mutex_lock(&comp_done_lock); -retry: - for (int i = 0; i < thread_count; i++) { - if (comp_param[i].done) { - CompressParam *param = &comp_param[i]; - qemu_mutex_lock(¶m->mutex); - param->done = false; - send_queued_data(param); - assert(qemu_file_buffer_empty(param->file)); - compress_reset_result(param); - set_compress_params(param, block, offset); - qemu_cond_signal(¶m->cond); - qemu_mutex_unlock(¶m->mutex); + while (true) { + for (int i = 0; i < thread_count; i++) { + if (comp_param[i].done) { + CompressParam *param = &comp_param[i]; + qemu_mutex_lock(¶m->mutex); + param->done = false; + send_queued_data(param); + assert(qemu_file_buffer_empty(param->file)); + compress_reset_result(param); + set_compress_params(param, block, offset); + + qemu_cond_signal(¶m->cond); + qemu_mutex_unlock(¶m->mutex); + qemu_mutex_unlock(&comp_done_lock); + return true; + } + } + if (!wait) { qemu_mutex_unlock(&comp_done_lock); - return true; + return false; } - } - - /* - * wait for the free thread if the user specifies 'compress-wait-thread', - * otherwise we will post the page out in the main thread as normal page. - */ - if (wait) { + /* + * wait for a free thread if the user specifies + * 'compress-wait-thread', otherwise we will post the page out + * in the main thread as normal page. + */ qemu_cond_wait(&comp_done_cond, &comp_done_lock); - goto retry; } - qemu_mutex_unlock(&comp_done_lock); - - return false; } /* return the size after decompression, or negative value on error */ From patchwork Tue Oct 31 09:01:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441296 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6C0BDC4167D for ; Tue, 31 Oct 2023 09:06:20 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkdr-0000KG-DT; Tue, 31 Oct 2023 05:02:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdo-00006V-Vq for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdm-0006sd-Fx for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742945; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MAvCgT0ohAvu13qO9O2qXDeVppbBAyGNHtVWe+19H/o=; b=h9FBB7yb+7Vy9uzj5Gd0Tusj9pwpYVhwrJfHaj5oAEU/GlOSaFfZWVqPTMFcWgGhvzG8SB U5T4k2f04zdd++5o2iLyZRV6vQIXC94HIcI1MU3y3IgYOWBVFuhBA7rjotRFhOOV7rqamy LP9+cdOnMX2lRfPK8kS+4f5/IOJk+ZU= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-522-rWmXFxJUOGyfWschV86yfg-1; Tue, 31 Oct 2023 05:02:20 -0400 X-MC-Unique: rWmXFxJUOGyfWschV86yfg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3891B1C060CB; Tue, 31 Oct 2023 09:02:20 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id A758F1C060AE; Tue, 31 Oct 2023 09:02:17 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , Lukas Straub Subject: [PULL 12/38] migration: Move busy++ to migrate_with_multithread Date: Tue, 31 Oct 2023 10:01:16 +0100 Message-ID: <20231031090142.13122-13-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org And now we can simplify save_compress_page(). Reviewed-by: Lukas Straub Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-7-quintela@redhat.com> --- migration/ram-compress.c | 1 + migration/ram.c | 8 ++------ 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/migration/ram-compress.c b/migration/ram-compress.c index a991b15b7a..f56e1f8e69 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -291,6 +291,7 @@ bool compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, } if (!wait) { qemu_mutex_unlock(&comp_done_lock); + compression_counters.busy++; return false; } /* diff --git a/migration/ram.c b/migration/ram.c index 63a575ae90..46209388ec 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2067,12 +2067,8 @@ static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, return false; } - if (compress_page_with_multi_thread(pss->block, offset, send_queued_data)) { - return true; - } - - compression_counters.busy++; - return false; + return compress_page_with_multi_thread(pss->block, offset, + send_queued_data); } /** From patchwork Tue Oct 31 09:01:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3A624C4332F for ; Tue, 31 Oct 2023 09:03:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkeB-0001AM-OR; Tue, 31 Oct 2023 05:02:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxke8-00010E-EL for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxke6-0006xf-K1 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742966; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uW1Z+v64y+umSQ32w8EgxYbosCFqhn1os2yU1vJh/YQ=; b=WbAgwawC4W44ugeWGbd3USwCEn14NUjT8gCkCnxMBIQCgla4rCgou1GomCR+ok/ogY27og uCeR36i0MsIMxNKINDpuDIs+0Wp6CbyZalGa6JeqKVzq+5zMmEa/Er9EESnMnFwiquFMU1 gXxGGT4WTzUuHDZ6uo59bMm2Ku8kysQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-614-c8otk8lIMhC-Sx9GIx9kmw-1; Tue, 31 Oct 2023 05:02:23 -0400 X-MC-Unique: c8otk8lIMhC-Sx9GIx9kmw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0BB11811E7D; Tue, 31 Oct 2023 09:02:23 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8105E1C060AE; Tue, 31 Oct 2023 09:02:20 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , Lukas Straub Subject: [PULL 13/38] migration: Create compress_update_rates() Date: Tue, 31 Oct 2023 10:01:17 +0100 Message-ID: <20231031090142.13122-14-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org So we can move more compression_counters stuff to ram-compress.c. Create compression_counters struct to add the stuff that was on MigrationState. Reviewed-by: Lukas Straub Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-8-quintela@redhat.com> --- migration/ram-compress.h | 1 + migration/ram.h | 1 - migration/ram-compress.c | 42 +++++++++++++++++++++++++++++++++++++++- migration/ram.c | 29 +-------------------------- 4 files changed, 43 insertions(+), 30 deletions(-) diff --git a/migration/ram-compress.h b/migration/ram-compress.h index b228640092..76dacd3ec7 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -71,5 +71,6 @@ void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len); void populate_compress(MigrationInfo *info); uint64_t ram_compressed_pages(void); void update_compress_thread_counts(const CompressParam *param, int bytes_xmit); +void compress_update_rates(uint64_t page_count); #endif diff --git a/migration/ram.h b/migration/ram.h index 3f724b2f02..9f3ad1ee81 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -34,7 +34,6 @@ #include "io/channel.h" extern XBZRLECacheStats xbzrle_counters; -extern CompressionStats compression_counters; /* Should be holding either ram_list.mutex, or the RCU lock. */ #define RAMBLOCK_FOREACH_NOT_IGNORED(block) \ diff --git a/migration/ram-compress.c b/migration/ram-compress.c index f56e1f8e69..af42cab0fe 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -41,7 +41,20 @@ #include "ram.h" #include "migration-stats.h" -CompressionStats compression_counters; +static struct { + int64_t pages; + int64_t busy; + double busy_rate; + int64_t compressed_size; + double compression_rate; + /* compression statistics since the beginning of the period */ + /* amount of count that no free thread to compress data */ + uint64_t compress_thread_busy_prev; + /* amount bytes after compression */ + uint64_t compressed_size_prev; + /* amount of compressed pages */ + uint64_t compress_pages_prev; +} compression_counters; static CompressParam *comp_param; static QemuThread *compress_threads; @@ -518,3 +531,30 @@ void update_compress_thread_counts(const CompressParam *param, int bytes_xmit) compression_counters.pages++; } +void compress_update_rates(uint64_t page_count) +{ + if (!migrate_compress()) { + return; + } + compression_counters.busy_rate = (double)(compression_counters.busy - + compression_counters.compress_thread_busy_prev) / page_count; + compression_counters.compress_thread_busy_prev = + compression_counters.busy; + + double compressed_size = compression_counters.compressed_size - + compression_counters.compressed_size_prev; + if (compressed_size) { + double uncompressed_size = (compression_counters.pages - + compression_counters.compress_pages_prev) * + qemu_target_page_size(); + + /* Compression-Ratio = Uncompressed-size / Compressed-size */ + compression_counters.compression_rate = + uncompressed_size / compressed_size; + + compression_counters.compress_pages_prev = + compression_counters.pages; + compression_counters.compressed_size_prev = + compression_counters.compressed_size; + } +} diff --git a/migration/ram.c b/migration/ram.c index 46209388ec..f7daf2226e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -369,13 +369,6 @@ struct RAMState { bool xbzrle_started; /* Are we on the last stage of migration */ bool last_stage; - /* compression statistics since the beginning of the period */ - /* amount of count that no free thread to compress data */ - uint64_t compress_thread_busy_prev; - /* amount bytes after compression */ - uint64_t compressed_size_prev; - /* amount of compressed pages */ - uint64_t compress_pages_prev; /* total handled target pages at the beginning of period */ uint64_t target_page_count_prev; @@ -945,7 +938,6 @@ uint64_t ram_get_total_transferred_pages(void) static void migration_update_rates(RAMState *rs, int64_t end_time) { uint64_t page_count = rs->target_page_count - rs->target_page_count_prev; - double compressed_size; /* calculate period counters */ stat64_set(&mig_stats.dirty_pages_rate, @@ -973,26 +965,7 @@ static void migration_update_rates(RAMState *rs, int64_t end_time) rs->xbzrle_pages_prev = xbzrle_counters.pages; rs->xbzrle_bytes_prev = xbzrle_counters.bytes; } - - if (migrate_compress()) { - compression_counters.busy_rate = (double)(compression_counters.busy - - rs->compress_thread_busy_prev) / page_count; - rs->compress_thread_busy_prev = compression_counters.busy; - - compressed_size = compression_counters.compressed_size - - rs->compressed_size_prev; - if (compressed_size) { - double uncompressed_size = (compression_counters.pages - - rs->compress_pages_prev) * TARGET_PAGE_SIZE; - - /* Compression-Ratio = Uncompressed-size / Compressed-size */ - compression_counters.compression_rate = - uncompressed_size / compressed_size; - - rs->compress_pages_prev = compression_counters.pages; - rs->compressed_size_prev = compression_counters.compressed_size; - } - } + compress_update_rates(page_count); } /* From patchwork Tue Oct 31 09:01:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441280 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 530EDC4332F for ; Tue, 31 Oct 2023 09:03:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkdw-0000Sv-Ia; Tue, 31 Oct 2023 05:02:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdr-0000LM-M2 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdq-0006tD-5R for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742949; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8N9eMhXSGAX5ioZRsrn869KMrCb4AUfc/pU+hNfVZPk=; b=GTyYDkdhQBIugGAE1Qi5D/x2XQrGkn1k/Wynbz5rwzolOzP/Fz0RD4wlWYeDfVa7HuT8wf Gkhb/ghBQiusgqu5XsJawyx4PTRhoA0jC9irNJQmaTem+OjPeLV7glfIRx3GoKn2cJE7P1 NlcXXzMupqk15LzoervWP3Z36HiwSTI= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-619-avwqIWeLP32wDgxj6sFxNQ-1; Tue, 31 Oct 2023 05:02:26 -0400 X-MC-Unique: avwqIWeLP32wDgxj6sFxNQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AF9A91C060CC; Tue, 31 Oct 2023 09:02:25 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 526C11C060AE; Tue, 31 Oct 2023 09:02:23 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 14/38] migration: Export send_queued_data() Date: Tue, 31 Oct 2023 10:01:18 +0100 Message-ID: <20231031090142.13122-15-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This function is only used for compression. So we rename it as compress_send_queued_data(). We put it on ram-compress.h because we are moving it later to ram-compress.c. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-9-quintela@redhat.com> --- migration/ram-compress.h | 1 + migration/ram.c | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/migration/ram-compress.h b/migration/ram-compress.h index 76dacd3ec7..636281ed97 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -72,5 +72,6 @@ void populate_compress(MigrationInfo *info); uint64_t ram_compressed_pages(void); void update_compress_thread_counts(const CompressParam *param, int bytes_xmit); void compress_update_rates(uint64_t page_count); +int compress_send_queued_data(CompressParam *param); #endif diff --git a/migration/ram.c b/migration/ram.c index f7daf2226e..b6d485358e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1264,7 +1264,7 @@ static int ram_save_multifd_page(QEMUFile *file, RAMBlock *block, return 1; } -static int send_queued_data(CompressParam *param) +int compress_send_queued_data(CompressParam *param) { PageSearchStatus *pss = &ram_state->pss[RAM_CHANNEL_PRECOPY]; MigrationState *ms = migrate_get_current(); @@ -1306,7 +1306,7 @@ static void ram_flush_compressed_data(void) return; } - flush_compressed_data(send_queued_data); + flush_compressed_data(compress_send_queued_data); } #define PAGE_ALL_CLEAN 0 @@ -2041,7 +2041,7 @@ static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, } return compress_page_with_multi_thread(pss->block, offset, - send_queued_data); + compress_send_queued_data); } /** From patchwork Tue Oct 31 09:01:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441282 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 64876C4332F for ; Tue, 31 Oct 2023 09:03:39 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkdy-0000az-9k; Tue, 31 Oct 2023 05:02:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdv-0000PE-C3 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdt-0006td-4q for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742952; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UCKjJA4W78PJZMYf4U2CLeARXWY7OhYhyDrkh0cEZKU=; b=LATHGQ44q3YiMFC+DjS6pu/0oYDsHPV+EU+MgsaYrhQwBVL1/SmwK7g5RMpHEMvJpk5M15 0ca9wJEgRD3bHcS5M+nbtVexfxlu1mAuIYgKt+eyJgjMqpecU4Z9XCGpCvAamiKyXMrt1p mFNF1b2N/kYkEHnPrBG/KDDd1+1773I= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-630-8Y7GJ8ECMMWu3A4bayNxJg-1; Tue, 31 Oct 2023 05:02:29 -0400 X-MC-Unique: 8Y7GJ8ECMMWu3A4bayNxJg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5E1A9185A782; Tue, 31 Oct 2023 09:02:28 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 028301C060B0; Tue, 31 Oct 2023 09:02:25 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 15/38] migration: Move ram_flush_compressed_data() to ram-compress.c Date: Tue, 31 Oct 2023 10:01:19 +0100 Message-ID: <20231031090142.13122-16-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org As we export it, rename it compress_flush_data(). Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-10-quintela@redhat.com> --- migration/ram-compress.h | 1 + migration/ram-compress.c | 9 +++++++++ migration/ram.c | 17 ++++------------- 3 files changed, 14 insertions(+), 13 deletions(-) diff --git a/migration/ram-compress.h b/migration/ram-compress.h index 636281ed97..7ba01e2882 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -73,5 +73,6 @@ uint64_t ram_compressed_pages(void); void update_compress_thread_counts(const CompressParam *param, int bytes_xmit); void compress_update_rates(uint64_t page_count); int compress_send_queued_data(CompressParam *param); +void compress_flush_data(void); #endif diff --git a/migration/ram-compress.c b/migration/ram-compress.c index af42cab0fe..1443a1cb45 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -558,3 +558,12 @@ void compress_update_rates(uint64_t page_count) compression_counters.compressed_size; } } + +void compress_flush_data(void) +{ + if (!migrate_compress()) { + return; + } + + flush_compressed_data(compress_send_queued_data); +} diff --git a/migration/ram.c b/migration/ram.c index b6d485358e..849752ef29 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1300,15 +1300,6 @@ int compress_send_queued_data(CompressParam *param) return len; } -static void ram_flush_compressed_data(void) -{ - if (!migrate_compress()) { - return; - } - - flush_compressed_data(compress_send_queued_data); -} - #define PAGE_ALL_CLEAN 0 #define PAGE_TRY_AGAIN 1 #define PAGE_DIRTY_FOUND 2 @@ -1364,7 +1355,7 @@ static int find_dirty_block(RAMState *rs, PageSearchStatus *pss) * Also If xbzrle is on, stop using the data compression at this * point. In theory, xbzrle can do better than compression. */ - ram_flush_compressed_data(); + compress_flush_data(); /* Hit the end of the list */ pss->block = QLIST_FIRST_RCU(&ram_list.blocks); @@ -2036,7 +2027,7 @@ static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, * much CPU resource. */ if (pss->block != pss->last_sent_block) { - ram_flush_compressed_data(); + compress_flush_data(); return false; } @@ -3083,7 +3074,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) * page is sent in one chunk. */ if (migrate_postcopy_ram()) { - ram_flush_compressed_data(); + compress_flush_data(); } /* @@ -3184,7 +3175,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) } qemu_mutex_unlock(&rs->bitmap_mutex); - ram_flush_compressed_data(); + compress_flush_data(); int ret = rdma_registration_stop(f, RAM_CONTROL_FINISH); if (ret < 0) { From patchwork Tue Oct 31 09:01:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3989AC4332F for ; Tue, 31 Oct 2023 09:05:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxke0-0000hn-9O; Tue, 31 Oct 2023 05:02:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdx-0000Vo-8R for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdv-0006tw-6u for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742954; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0x7e3xlK2kWrRG/MI826k6pyKhvtiFg3nKIHazDXcTY=; b=hY4uy2yqOsaXfLSpdw4nnWIigpB7utQqW7NhYaAnEn3l+mA2pMoK+rCsOciuqlZqSqH1sf lAwCyFGQG0/nsZg155yBfBTa4k1JyLTRztAn/4XnZ2ywF15G4TmAkuj+0k0kzpnlSh254K KihCYqHQ0+vMr4U5MM1hjeja+Di0F1A= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-653-zJYS-2f8PhqjVimWiD5W9w-1; Tue, 31 Oct 2023 05:02:31 -0400 X-MC-Unique: zJYS-2f8PhqjVimWiD5W9w-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0C11B29AB3E9; Tue, 31 Oct 2023 09:02:31 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id A4C0E1C060AE; Tue, 31 Oct 2023 09:02:28 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 16/38] migration: Merge flush_compressed_data() and compress_flush_data() Date: Tue, 31 Oct 2023 10:01:20 +0100 Message-ID: <20231031090142.13122-17-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-11-quintela@redhat.com> --- migration/ram-compress.h | 1 - migration/ram-compress.c | 17 ++++++----------- 2 files changed, 6 insertions(+), 12 deletions(-) diff --git a/migration/ram-compress.h b/migration/ram-compress.h index 7ba01e2882..e222887fb7 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -59,7 +59,6 @@ typedef struct CompressParam CompressParam; void compress_threads_save_cleanup(void); int compress_threads_save_setup(void); -void flush_compressed_data(int (send_queued_data(CompressParam *))); bool compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, int (send_queued_data(CompressParam *))); diff --git a/migration/ram-compress.c b/migration/ram-compress.c index 1443a1cb45..036e44085b 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -241,10 +241,14 @@ static inline void compress_reset_result(CompressParam *param) param->offset = 0; } -void flush_compressed_data(int (send_queued_data(CompressParam *))) +void compress_flush_data(void) { int thread_count = migrate_compress_threads(); + if (!migrate_compress()) { + return; + } + qemu_mutex_lock(&comp_done_lock); for (int i = 0; i < thread_count; i++) { while (!comp_param[i].done) { @@ -257,7 +261,7 @@ void flush_compressed_data(int (send_queued_data(CompressParam *))) qemu_mutex_lock(&comp_param[i].mutex); if (!comp_param[i].quit) { CompressParam *param = &comp_param[i]; - send_queued_data(param); + compress_send_queued_data(param); assert(qemu_file_buffer_empty(param->file)); compress_reset_result(param); } @@ -558,12 +562,3 @@ void compress_update_rates(uint64_t page_count) compression_counters.compressed_size; } } - -void compress_flush_data(void) -{ - if (!migrate_compress()) { - return; - } - - flush_compressed_data(compress_send_queued_data); -} From patchwork Tue Oct 31 09:01:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441292 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 977CBC4332F for ; Tue, 31 Oct 2023 09:05:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxke3-0000lp-NU; Tue, 31 Oct 2023 05:02:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxke1-0000jA-I7 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkdz-0006wZ-61 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742958; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=65HtqjEgp5Bqh3Qad91zv3o+mLXRJbK00jYVKDpCWLY=; b=ixQKO9njPu+ZFrVak8sWeI1pvHRGKgFGvfPwPmH8a610k3oKv9iT+OAhk0YOIIexaxHfpJ 7PmL/kS/JrAFTuR6+DcaEOEvJOM1oIjRZK8hXHiFLHz+qCK7PKzmSt8ZmflVNhvRLgzgju uvQII1RVF3NeIM8Q0/RPv2KPPXTAnZU= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-612-xavfIDNkOzevx3MXse27kw-1; Tue, 31 Oct 2023 05:02:34 -0400 X-MC-Unique: xavfIDNkOzevx3MXse27kw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AC8151C060CD; Tue, 31 Oct 2023 09:02:33 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4F9FE1C060AE; Tue, 31 Oct 2023 09:02:31 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 17/38] migration: Rename ram_compressed_pages() to compress_ram_pages() Date: Tue, 31 Oct 2023 10:01:21 +0100 Message-ID: <20231031090142.13122-18-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We are moving to have all functions exported from ram-compress.c to start with compress_. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231019110724.15324-12-quintela@redhat.com> --- migration/ram-compress.h | 2 +- migration/ram-compress.c | 2 +- migration/ram.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/migration/ram-compress.h b/migration/ram-compress.h index e222887fb7..0d89a2f55e 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -68,7 +68,7 @@ int compress_threads_load_setup(QEMUFile *f); void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len); void populate_compress(MigrationInfo *info); -uint64_t ram_compressed_pages(void); +uint64_t compress_ram_pages(void); void update_compress_thread_counts(const CompressParam *param, int bytes_xmit); void compress_update_rates(uint64_t page_count); int compress_send_queued_data(CompressParam *param); diff --git a/migration/ram-compress.c b/migration/ram-compress.c index 036e44085b..fa4388f6a6 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -516,7 +516,7 @@ void populate_compress(MigrationInfo *info) info->compression->compression_rate = compression_counters.compression_rate; } -uint64_t ram_compressed_pages(void) +uint64_t compress_ram_pages(void) { return compression_counters.pages; } diff --git a/migration/ram.c b/migration/ram.c index 849752ef29..6335564035 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -932,7 +932,7 @@ uint64_t ram_get_total_transferred_pages(void) { return stat64_get(&mig_stats.normal_pages) + stat64_get(&mig_stats.zero_pages) + - ram_compressed_pages() + xbzrle_counters.pages; + compress_ram_pages() + xbzrle_counters.pages; } static void migration_update_rates(RAMState *rs, int64_t end_time) From patchwork Tue Oct 31 09:01:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441322 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0DBA7C4332F for ; Tue, 31 Oct 2023 09:09:29 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxke8-0000wc-S8; Tue, 31 Oct 2023 05:02:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxke5-0000mt-0x for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxke2-0006xB-M0 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742962; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zNTqpR4R/RiiCgDdxYzqqoUc01rIqkGdDcl+05m0oFw=; b=XZwziNumJlAiWHj82ozhzP29w6pDSzpru2N6fflT49KVtoVnyZIi+qTn15sJSA0f/HZvB5 ZcOaZ28KfC5OkIjUTRyLXzm9o+kCxQ3T1+QdPyJjCLNn74PTN3FGmQ8Xe1VkV04iXt/+n+ o68WZfWKsIUW0cHapbO27BturYMjeNc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-678-N9_3Ct25M56TAOt46Hkfcw-1; Tue, 31 Oct 2023 05:02:37 -0400 X-MC-Unique: N9_3Ct25M56TAOt46Hkfcw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BC0CA185A78E; Tue, 31 Oct 2023 09:02:36 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3459F1C060AE; Tue, 31 Oct 2023 09:02:33 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , Thomas Huth Subject: [PULL 18/38] migration/ram: Fix compilation with -Wshadow=local Date: Tue, 31 Oct 2023 10:01:22 +0100 Message-ID: <20231031090142.13122-19-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth Rename the variable here to avoid that it shadows a variable from the beginning of the function scope. With this change the code now successfully compiles with -Wshadow=local. Signed-off-by: Thomas Huth Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231024092220.55305-1-thuth@redhat.com> --- migration/ram.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 6335564035..024dedb6b1 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3147,6 +3147,8 @@ static int ram_save_complete(QEMUFile *f, void *opaque) rs->last_stage = !migration_in_colo_state(); WITH_RCU_READ_LOCK_GUARD() { + int rdma_reg_ret; + if (!migration_in_postcopy()) { migration_bitmap_sync_precopy(rs, true); } @@ -3177,9 +3179,9 @@ static int ram_save_complete(QEMUFile *f, void *opaque) compress_flush_data(); - int ret = rdma_registration_stop(f, RAM_CONTROL_FINISH); - if (ret < 0) { - qemu_file_set_error(f, ret); + rdma_reg_ret = rdma_registration_stop(f, RAM_CONTROL_FINISH); + if (rdma_reg_ret < 0) { + qemu_file_set_error(f, rdma_reg_ret); } } From patchwork Tue Oct 31 09:01:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441324 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7369CC4332F for ; Tue, 31 Oct 2023 09:09:35 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkeD-0001FD-SZ; Tue, 31 Oct 2023 05:02:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeA-00016r-V1 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxke8-0006yW-Fb for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742967; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/NRSpUIijsQTQV6/Ab+Fo4l/3wg2rU3H4n9wtIAj/fI=; b=g1+WPvkNLnMSKyX+UMCV7C9WDltifAAlJKhmQEcYiG/qafU6Da2o3j/U4mUFjRkaYG49EF HOEuzLdzFe2rD6hFwXzfeA3jiNY76lWibzgYkFetjpCcV1Y6zCCbYHtXO546Ot4JKo/6PO 7Yly64IC0uFlyJ5Semkb36ca6HFKSd4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-696-H24PkZb7O5qhnhgxTm9hFA-1; Tue, 31 Oct 2023 05:02:40 -0400 X-MC-Unique: H24PkZb7O5qhnhgxTm9hFA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8F90685A5B5; Tue, 31 Oct 2023 09:02:39 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 099CA1C060AE; Tue, 31 Oct 2023 09:02:36 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= Subject: [PULL 19/38] migration: rename vmstate_save_needed->vmstate_section_needed Date: Tue, 31 Oct 2023 10:01:23 +0100 Message-ID: <20231031090142.13122-20-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Marc-André Lureau The function is used on save at this point. The following commits will use it on load. Signed-off-by: Marc-André Lureau Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231024084043.2926316-5-marcandre.lureau@redhat.com> --- include/migration/vmstate.h | 2 +- migration/savevm.c | 2 +- migration/vmstate.c | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 1a31fb7293..1af181877c 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -1202,7 +1202,7 @@ int vmstate_save_state_v(QEMUFile *f, const VMStateDescription *vmsd, void *opaque, JSONWriter *vmdesc, int version_id, Error **errp); -bool vmstate_save_needed(const VMStateDescription *vmsd, void *opaque); +bool vmstate_section_needed(const VMStateDescription *vmsd, void *opaque); #define VMSTATE_INSTANCE_ID_ANY -1 diff --git a/migration/savevm.c b/migration/savevm.c index 8622f229e5..ca5c7cebe0 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -985,7 +985,7 @@ static int vmstate_save(QEMUFile *f, SaveStateEntry *se, JSONWriter *vmdesc) if ((!se->ops || !se->ops->save_state) && !se->vmsd) { return 0; } - if (se->vmsd && !vmstate_save_needed(se->vmsd, se->opaque)) { + if (se->vmsd && !vmstate_section_needed(se->vmsd, se->opaque)) { trace_savevm_section_skip(se->idstr, se->section_id); return 0; } diff --git a/migration/vmstate.c b/migration/vmstate.c index 1cf9e45b85..16e33a5d34 100644 --- a/migration/vmstate.c +++ b/migration/vmstate.c @@ -324,7 +324,7 @@ static void vmsd_desc_field_end(const VMStateDescription *vmsd, } -bool vmstate_save_needed(const VMStateDescription *vmsd, void *opaque) +bool vmstate_section_needed(const VMStateDescription *vmsd, void *opaque) { if (vmsd->needed && !vmsd->needed(opaque)) { /* optional section not needed */ @@ -522,7 +522,7 @@ static int vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd, trace_vmstate_subsection_save_top(vmsd->name); while (sub && *sub) { - if (vmstate_save_needed(*sub, opaque)) { + if (vmstate_section_needed(*sub, opaque)) { const VMStateDescription *vmsdsub = *sub; uint8_t len; From patchwork Tue Oct 31 09:01:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441286 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6D34BC4332F for ; Tue, 31 Oct 2023 09:04:00 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkeB-00017i-75; Tue, 31 Oct 2023 05:02:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxke9-00014a-K5 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxke7-0006yC-QT for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742967; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=12UW4S7DwReY5i7WbgLvv6myQHnsED050vB3pRolHAs=; b=Fz/8XIB61afmL6/2JetRoqGHua6nhjf5t3xsUXyIhgiOYN5iZ7Owht3aOxLyUCgrcr0chw 6LAgyCDJwTKy1iy0o+0nOIdiZxL5Zy6H3/sgTZix3ZhF5+MmqxrilhAhjIJhIBxiqxfX3w o/21OweOlZnspttOsnSB051oo7UmHVI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-443-AkrXjiNGPpuDj7Y-wn-Mwg-1; Tue, 31 Oct 2023 05:02:42 -0400 X-MC-Unique: AkrXjiNGPpuDj7Y-wn-Mwg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6DA1E8A67E3; Tue, 31 Oct 2023 09:02:42 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id D6FDA1C060B0; Tue, 31 Oct 2023 09:02:39 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= Subject: [PULL 20/38] migration: set file error on subsection loading Date: Tue, 31 Oct 2023 10:01:24 +0100 Message-ID: <20231031090142.13122-21-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Marc-André Lureau commit 13cde50889237 ("vmstate: Return error in case of error") sets QemuFile error to stop reading from it and report to the caller (checked by unit tests). We should do the same on subsection loading error. Signed-off-by: Marc-André Lureau Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Message-ID: <20231024084043.2926316-8-marcandre.lureau@redhat.com> --- migration/vmstate.c | 1 + 1 file changed, 1 insertion(+) diff --git a/migration/vmstate.c b/migration/vmstate.c index 16e33a5d34..9c36803c8a 100644 --- a/migration/vmstate.c +++ b/migration/vmstate.c @@ -179,6 +179,7 @@ int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd, assert(field->flags == VMS_END); ret = vmstate_subsection_load(f, vmsd, opaque); if (ret != 0) { + qemu_file_set_error(f, ret); return ret; } if (vmsd->post_load) { From patchwork Tue Oct 31 09:01:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 18F4BC4167B for ; Tue, 31 Oct 2023 09:04:26 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkeK-0001Ns-Qg; Tue, 31 Oct 2023 05:03:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeD-0001El-G4 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeA-0006zc-9e for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742969; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w5jcWCJuarmvQpIJ2dG0VPWVcQ2F1rKSx6Uy0+MjrJ4=; b=U4PtjNgzlE6q2q21boDZgi8TqRC6aEczexFgW/oKG1aqfIdYfRg4Y40QDydKPw0kU9E9/n DszBXbfN4a8SM+D5U/XPl3SU3pO7wRJ4qNVMF9QSHfih6DnFFwFcH+yeFPBJYdbLUINbRS oxVrGDlPmHWT6yorc/UaWK/2GsL986U= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-178-g03ZC8bbNuaMA6ePHGxC0g-1; Tue, 31 Oct 2023 05:02:45 -0400 X-MC-Unique: g03ZC8bbNuaMA6ePHGxC0g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 19E6A3C100A7; Tue, 31 Oct 2023 09:02:45 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id B12551C060AE; Tue, 31 Oct 2023 09:02:42 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 21/38] qemu-iotests: Filter warnings about block migration being deprecated Date: Tue, 31 Oct 2023 10:01:25 +0100 Message-ID: <20231031090142.13122-22-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Create a new filter that removes the two warnings for test 183. Reviewed-by: Hanna Czenczek Signed-off-by: Juan Quintela Message-ID: <20231018115513.2163-2-quintela@redhat.com> --- tests/qemu-iotests/183 | 2 +- tests/qemu-iotests/common.filter | 7 +++++++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/183 b/tests/qemu-iotests/183 index ee62939e72..b85770458e 100755 --- a/tests/qemu-iotests/183 +++ b/tests/qemu-iotests/183 @@ -90,7 +90,7 @@ echo reply="$(_send_qemu_cmd $src \ "{ 'execute': 'migrate', 'arguments': { 'uri': 'unix:${MIG_SOCKET}', 'blk': true } }" \ - 'return\|error')" + 'return\|error' | _filter_migration_block_deprecated)" echo "$reply" if echo "$reply" | grep "compiled without old-style" > /dev/null; then _notrun "migrate -b support not compiled in" diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index fc3c64bcb8..2846c83808 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -359,5 +359,12 @@ _filter_qcow2_compression_type_bit() -e 's/\(incompatible_features.*\), 3\(,.*\)/\1\2/' } +# filter warnings caused for block migration deprecation +_filter_migration_block_deprecated() +{ + gsed -e '/warning: parameter .blk. is deprecated; use blockdev-mirror with NBD instead/d' \ + -e '/warning: block migration is deprecated; use blockdev-mirror with NBD instead/d' +} + # make sure this script returns success true From patchwork Tue Oct 31 09:01:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441310 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4AD0BC4332F for ; Tue, 31 Oct 2023 09:08:05 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkeQ-0001VQ-Lg; Tue, 31 Oct 2023 05:03:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeF-0001IH-F5 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeC-00070M-FV for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:02:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742971; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0I3xavpLLPL9UCs9aLYAbhUG+U/r+saIeKaKicQp828=; b=Xi+rkMJVRC4znj/cOUEPUKT2WhXwujLn40nS9RLSQ7qg+KGeTKqYi4wI5BeVyBEePlHR4M lHYQN7LGCmwqu7IRx1MNLERnYv8I2ez9yL3uEDM+sjo4wLBZDEi0DlHE2QC+VOTYk/7CKu cPVccmvAopVClxH5iXvk+MfZDwQDE8k= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-524-hp0ady_qOyuS0kk3a7hPHg-1; Tue, 31 Oct 2023 05:02:48 -0400 X-MC-Unique: hp0ady_qOyuS0kk3a7hPHg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E00B710B0703; Tue, 31 Oct 2023 09:02:47 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 625711C060AE; Tue, 31 Oct 2023 09:02:45 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , Thomas Huth Subject: [PULL 22/38] migration: migrate 'inc' command option is deprecated. Date: Tue, 31 Oct 2023 10:01:26 +0100 Message-ID: <20231031090142.13122-23-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Use blockdev-mirror with NBD instead. Reviewed-by: Thomas Huth Acked-by: Stefan Hajnoczi Reviewed-by: Markus Armbruster Signed-off-by: Juan Quintela Message-ID: <20231018115513.2163-3-quintela@redhat.com> --- docs/about/deprecated.rst | 8 ++++++++ qapi/migration.json | 8 +++++++- migration/migration-hmp-cmds.c | 5 +++++ migration/migration.c | 5 +++++ 4 files changed, 25 insertions(+), 1 deletion(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 4e0eb2fe02..1c8c8c34c4 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -469,3 +469,11 @@ Migration ``skipped`` field in Migration stats has been deprecated. It hasn't been used for more than 10 years. +``inc`` migrate command option (since 8.2) +'''''''''''''''''''''''''''''''''''''''''' + +Use blockdev-mirror with NBD instead. + +As an intermediate step the ``inc`` functionality can be achieved by +setting the ``block-incremental`` migration parameter to ``true``. +But this parameter is also deprecated. diff --git a/qapi/migration.json b/qapi/migration.json index db3df12d6c..fa7f4f2575 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1524,6 +1524,11 @@ # # @resume: resume one paused migration, default "off". (since 3.0) # +# Features: +# +# @deprecated: Member @inc is deprecated. Use blockdev-mirror with +# NBD instead. +# # Returns: nothing on success # # Since: 0.14 @@ -1545,7 +1550,8 @@ # <- { "return": {} } ## { 'command': 'migrate', - 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', + 'data': {'uri': 'str', '*blk': 'bool', + '*inc': { 'type': 'bool', 'features': [ 'deprecated' ] }, '*detach': 'bool', '*resume': 'bool' } } ## diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index a82597f18e..83176f5bae 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -745,6 +745,11 @@ void hmp_migrate(Monitor *mon, const QDict *qdict) const char *uri = qdict_get_str(qdict, "uri"); Error *err = NULL; + if (inc) { + warn_report("option '-i' is deprecated;" + " use blockdev-mirror with NBD instead"); + } + qmp_migrate(uri, !!blk, blk, !!inc, inc, false, false, true, resume, &err); if (hmp_handle_error(mon, err)) { diff --git a/migration/migration.c b/migration/migration.c index 67547eb6a1..06a706ad90 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1620,6 +1620,11 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, { Error *local_err = NULL; + if (blk_inc) { + warn_report("parameter 'inc' is deprecated;" + " use blockdev-mirror with NBD instead"); + } + if (resume) { if (s->state != MIGRATION_STATUS_POSTCOPY_PAUSED) { error_setg(errp, "Cannot resume if there is no " From patchwork Tue Oct 31 09:01:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B4A47C4332F for ; Tue, 31 Oct 2023 09:09:09 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkeU-0001jJ-7R; Tue, 31 Oct 2023 05:03:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeL-0001VP-T5 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeK-00072I-7f for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742979; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=10JfSxom2lk2mbOjj76bRdexz5YSQtyyggKlwWnHJ6E=; b=LFVN5zdlDokmG1jZ9YmQWY2Wu/ki6EwODPFMzlUGKtC05DK4ijvBiJ3aw0Z6pt8M9Dq9yw LHj+HTDhUnkQD/VQI+NqBEISARGftaR3+pdf+Ax6Zq5NlHbme2YWHJu+n0BkxZkNtt9i2m z/uM12G6JdSPaV53/DCpNEcnm/pPpdA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-410-OgGxdt4oOOmwWC_iKjl4NQ-1; Tue, 31 Oct 2023 05:02:51 -0400 X-MC-Unique: OgGxdt4oOOmwWC_iKjl4NQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ADAC2185A786; Tue, 31 Oct 2023 09:02:50 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 300BE1C060AE; Tue, 31 Oct 2023 09:02:48 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , Thomas Huth Subject: [PULL 23/38] migration: migrate 'blk' command option is deprecated. Date: Tue, 31 Oct 2023 10:01:27 +0100 Message-ID: <20231031090142.13122-24-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Use blocked-mirror with NBD instead. Acked-by: Stefan Hajnoczi Reviewed-by: Thomas Huth Reviewed-by: Markus Armbruster Signed-off-by: Juan Quintela Message-ID: <20231018115513.2163-4-quintela@redhat.com> --- docs/about/deprecated.rst | 9 +++++++++ qapi/migration.json | 7 ++++--- migration/migration-hmp-cmds.c | 5 +++++ migration/migration.c | 5 +++++ 4 files changed, 23 insertions(+), 3 deletions(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 1c8c8c34c4..c1613468e6 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -477,3 +477,12 @@ Use blockdev-mirror with NBD instead. As an intermediate step the ``inc`` functionality can be achieved by setting the ``block-incremental`` migration parameter to ``true``. But this parameter is also deprecated. + +``blk`` migrate command option (since 8.2) +'''''''''''''''''''''''''''''''''''''''''' + +Use blockdev-mirror with NBD instead. + +As an intermediate step the ``blk`` functionality can be achieved by +setting the ``block`` migration capability to ``true``. But this +capability is also deprecated. diff --git a/qapi/migration.json b/qapi/migration.json index fa7f4f2575..3765c2b662 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1526,8 +1526,8 @@ # # Features: # -# @deprecated: Member @inc is deprecated. Use blockdev-mirror with -# NBD instead. +# @deprecated: Members @inc and @blk are deprecated. Use +# blockdev-mirror with NBD instead. # # Returns: nothing on success # @@ -1550,7 +1550,8 @@ # <- { "return": {} } ## { 'command': 'migrate', - 'data': {'uri': 'str', '*blk': 'bool', + 'data': {'uri': 'str', + '*blk': { 'type': 'bool', 'features': [ 'deprecated' ] }, '*inc': { 'type': 'bool', 'features': [ 'deprecated' ] }, '*detach': 'bool', '*resume': 'bool' } } diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 83176f5bae..dfe98da355 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -750,6 +750,11 @@ void hmp_migrate(Monitor *mon, const QDict *qdict) " use blockdev-mirror with NBD instead"); } + if (blk) { + warn_report("option '-b' is deprecated;" + " use blockdev-mirror with NBD instead"); + } + qmp_migrate(uri, !!blk, blk, !!inc, inc, false, false, true, resume, &err); if (hmp_handle_error(mon, err)) { diff --git a/migration/migration.c b/migration/migration.c index 06a706ad90..cb2d7161b5 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1625,6 +1625,11 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc, " use blockdev-mirror with NBD instead"); } + if (blk) { + warn_report("parameter 'blk' is deprecated;" + " use blockdev-mirror with NBD instead"); + } + if (resume) { if (s->state != MIGRATION_STATUS_POSTCOPY_PAUSED) { error_setg(errp, "Cannot resume if there is no " From patchwork Tue Oct 31 09:01:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DE0C5C4332F for ; Tue, 31 Oct 2023 09:10:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkeu-0002rO-5R; Tue, 31 Oct 2023 05:03:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeZ-00020s-QF for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeX-00079G-T5 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742993; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cgY98SFPwcTkFT1EcyY0UJrzdGnE86XwuN/qvH01nOg=; b=WPDJRBiP+Q6LhNOBvAW/cxV3eADXcLOsoJfkr1kI4yebLvLV+RANaPJG35sCZZeRZuEOU/ D+5N/JVA00bl1axKkz9/JrbiIqow0ZmTwVj/Jq9dpVjNABTjcx1AGQDXA7n7kt0LdU/y7n dSTl2HQIn8h4Be1zCA4NmU8mZxQDg4Q= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-219-3bHIKcz_MDeEczM95KsXnA-1; Tue, 31 Oct 2023 05:02:54 -0400 X-MC-Unique: 3bHIKcz_MDeEczM95KsXnA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 122348A5DDB; Tue, 31 Oct 2023 09:02:54 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id F20D11C060AE; Tue, 31 Oct 2023 09:02:50 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 24/38] migration: Deprecate block migration Date: Tue, 31 Oct 2023 10:01:28 +0100 Message-ID: <20231031090142.13122-25-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org It is obsolete. It is better to use driver-mirror with NBD instead. CC: Kevin Wolf CC: Eric Blake CC: Stefan Hajnoczi CC: Hanna Czenczek Acked-by: Stefan Hajnoczi Reviewed-by: Markus Armbruster Signed-off-by: Juan Quintela Message-ID: <20231018115513.2163-5-quintela@redhat.com> --- docs/about/deprecated.rst | 10 ++++++++++ qapi/migration.json | 29 ++++++++++++++++++++++++----- migration/block.c | 3 +++ migration/options.c | 9 ++++++++- 4 files changed, 45 insertions(+), 6 deletions(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index c1613468e6..35c8d7d1d4 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -486,3 +486,13 @@ Use blockdev-mirror with NBD instead. As an intermediate step the ``blk`` functionality can be achieved by setting the ``block`` migration capability to ``true``. But this capability is also deprecated. + +block migration (since 8.2) +''''''''''''''''''''''''''' + +Block migration is too inflexible. It needs to migrate all block +devices or none. + +Please see "QMP invocation for live storage migration with +``blockdev-mirror`` + NBD" in docs/interop/live-block-operations.rst +for a detailed explanation. diff --git a/qapi/migration.json b/qapi/migration.json index 3765c2b662..e3b00a215b 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -269,11 +269,15 @@ # average memory load of the virtual CPU indirectly. Note that # zero means guest doesn't dirty memory. (Since 8.1) # +# Features: +# +# @deprecated: Member @disk is deprecated because block migration is. +# # Since: 0.14 ## { 'struct': 'MigrationInfo', 'data': {'*status': 'MigrationStatus', '*ram': 'MigrationStats', - '*disk': 'MigrationStats', + '*disk': { 'type': 'MigrationStats', 'features': [ 'deprecated' ] }, '*vfio': 'VfioStats', '*xbzrle-cache': 'XBZRLECacheStats', '*total-time': 'int', @@ -525,6 +529,9 @@ # # Features: # +# @deprecated: Member @block is deprecated. Use blockdev-mirror with +# NBD instead. +# # @unstable: Members @x-colo and @x-ignore-shared are experimental. # # Since: 1.2 @@ -534,7 +541,8 @@ 'compress', 'events', 'postcopy-ram', { 'name': 'x-colo', 'features': [ 'unstable' ] }, 'release-ram', - 'block', 'return-path', 'pause-before-switchover', 'multifd', + { 'name': 'block', 'features': [ 'deprecated' ] }, + 'return-path', 'pause-before-switchover', 'multifd', 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, 'validate-uuid', 'background-snapshot', @@ -835,6 +843,9 @@ # # Features: # +# @deprecated: Member @block-incremental is deprecated. Use +# blockdev-mirror with NBD instead. +# # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. # @@ -850,7 +861,7 @@ 'tls-creds', 'tls-hostname', 'tls-authz', 'max-bandwidth', 'avail-switchover-bandwidth', 'downtime-limit', { 'name': 'x-checkpoint-delay', 'features': [ 'unstable' ] }, - 'block-incremental', + { 'name': 'block-incremental', 'features': [ 'deprecated' ] }, 'multifd-channels', 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', @@ -1011,6 +1022,9 @@ # # Features: # +# @deprecated: Member @block-incremental is deprecated. Use +# blockdev-mirror with NBD instead. +# # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. # @@ -1040,7 +1054,8 @@ '*downtime-limit': 'uint64', '*x-checkpoint-delay': { 'type': 'uint32', 'features': [ 'unstable' ] }, - '*block-incremental': 'bool', + '*block-incremental': { 'type': 'bool', + 'features': [ 'deprecated' ] }, '*multifd-channels': 'uint8', '*xbzrle-cache-size': 'size', '*max-postcopy-bandwidth': 'size', @@ -1225,6 +1240,9 @@ # # Features: # +# @deprecated: Member @block-incremental is deprecated. Use +# blockdev-mirror with NBD instead. +# # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. # @@ -1251,7 +1269,8 @@ '*downtime-limit': 'uint64', '*x-checkpoint-delay': { 'type': 'uint32', 'features': [ 'unstable' ] }, - '*block-incremental': 'bool', + '*block-incremental': { 'type': 'bool', + 'features': [ 'deprecated' ] }, '*multifd-channels': 'uint8', '*xbzrle-cache-size': 'size', '*max-postcopy-bandwidth': 'size', diff --git a/migration/block.c b/migration/block.c index b60698d6e2..acffe88f84 100644 --- a/migration/block.c +++ b/migration/block.c @@ -731,6 +731,9 @@ static int block_save_setup(QEMUFile *f, void *opaque) trace_migration_block_save("setup", block_mig_state.submitted, block_mig_state.transferred); + warn_report("block migration is deprecated;" + " use blockdev-mirror with NBD instead"); + ret = init_blk_migration(f); if (ret < 0) { return ret; diff --git a/migration/options.c b/migration/options.c index 37fa1cfe74..ae8ab47e32 100644 --- a/migration/options.c +++ b/migration/options.c @@ -12,6 +12,7 @@ */ #include "qemu/osdep.h" +#include "qemu/error-report.h" #include "exec/target_page.h" #include "qapi/clone-visitor.h" #include "qapi/error.h" @@ -473,10 +474,14 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) if (new_caps[MIGRATION_CAPABILITY_BLOCK]) { error_setg(errp, "QEMU compiled without old-style (blk/-b, inc/-i) " "block migration"); - error_append_hint(errp, "Use drive_mirror+NBD instead.\n"); + error_append_hint(errp, "Use blockdev-mirror with NBD instead.\n"); return false; } #endif + if (new_caps[MIGRATION_CAPABILITY_BLOCK]) { + warn_report("block migration is deprecated;" + " use blockdev-mirror with NBD instead"); + } #ifndef CONFIG_REPLICATION if (new_caps[MIGRATION_CAPABILITY_X_COLO]) { @@ -1400,6 +1405,8 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) } if (params->has_block_incremental) { + warn_report("block migration is deprecated;" + " use blockdev-mirror with NBD instead"); s->parameters.block_incremental = params->block_incremental; } if (params->has_multifd_channels) { From patchwork Tue Oct 31 09:01:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441298 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70E1DC4332F for ; Tue, 31 Oct 2023 09:06:35 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxker-0002S1-1x; Tue, 31 Oct 2023 05:03:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeY-0001xZ-KJ for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeW-00078y-Iq for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742991; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dMjS6eAVEaI9hWwneUQidWJQnE5AjKKwrLBa2z8lOyo=; b=KX72bfJYG0OfXnbOhEhyZ2qRWb/ggryBbXJ++JU0m5qukJI44D/TO0bdX7zMXBERCisxZe rldpfxnseUI7RSU4m8RazsTSp2X9R66nlKXaH20B3oJSj5M5TqieFY/sC+BdR3egDHCHo2 cGOTZcHNnF6+FtQPCaJtfnpEGMdygM8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-523-FcsaPkEOMiS-XBR-SXehMA-1; Tue, 31 Oct 2023 05:02:57 -0400 X-MC-Unique: FcsaPkEOMiS-XBR-SXehMA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B3FB881F443; Tue, 31 Oct 2023 09:02:56 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 54B831C060B0; Tue, 31 Oct 2023 09:02:54 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 25/38] migration: Deprecate old compression method Date: Tue, 31 Oct 2023 10:01:29 +0100 Message-ID: <20231031090142.13122-26-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Acked-by: Stefan Hajnoczi Acked-by: Peter Xu Reviewed-by: Markus Armbruster Signed-off-by: Juan Quintela Message-ID: <20231018115513.2163-6-quintela@redhat.com> --- docs/about/deprecated.rst | 8 +++++ qapi/migration.json | 63 ++++++++++++++++++++++++++------------- migration/options.c | 13 ++++++++ 3 files changed, 64 insertions(+), 20 deletions(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 35c8d7d1d4..ecccd5d3fc 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -496,3 +496,11 @@ devices or none. Please see "QMP invocation for live storage migration with ``blockdev-mirror`` + NBD" in docs/interop/live-block-operations.rst for a detailed explanation. + +old compression method (since 8.2) +'''''''''''''''''''''''''''''''''' + +Compression method fails too much. Too many races. We are going to +remove it if nobody fixes it. For starters, migration-test +compression tests are disabled becase they fail randomly. If you need +compression, use multifd compression methods. diff --git a/qapi/migration.json b/qapi/migration.json index e3b00a215b..e6610af428 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -272,6 +272,10 @@ # Features: # # @deprecated: Member @disk is deprecated because block migration is. +# Member @compression is deprecated because it is unreliable and +# untested. It is recommended to use multifd migration, which +# offers an alternative compression implementation that is +# reliable and tested. # # Since: 0.14 ## @@ -289,7 +293,7 @@ '*blocked-reasons': ['str'], '*postcopy-blocktime': 'uint32', '*postcopy-vcpu-blocktime': ['uint32'], - '*compression': 'CompressionStats', + '*compression': { 'type': 'CompressionStats', 'features': [ 'deprecated' ] }, '*socket-address': ['SocketAddress'], '*dirty-limit-throttle-time-per-round': 'uint64', '*dirty-limit-ring-full-time': 'uint64'} } @@ -530,7 +534,10 @@ # Features: # # @deprecated: Member @block is deprecated. Use blockdev-mirror with -# NBD instead. +# NBD instead. Member @compression is deprecated because it is +# unreliable and untested. It is recommended to use multifd +# migration, which offers an alternative compression +# implementation that is reliable and tested. # # @unstable: Members @x-colo and @x-ignore-shared are experimental. # @@ -538,7 +545,8 @@ ## { 'enum': 'MigrationCapability', 'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks', - 'compress', 'events', 'postcopy-ram', + { 'name': 'compress', 'features': [ 'deprecated' ] }, + 'events', 'postcopy-ram', { 'name': 'x-colo', 'features': [ 'unstable' ] }, 'release-ram', { 'name': 'block', 'features': [ 'deprecated' ] }, @@ -844,7 +852,9 @@ # Features: # # @deprecated: Member @block-incremental is deprecated. Use -# blockdev-mirror with NBD instead. +# blockdev-mirror with NBD instead. Members @compress-level, +# @compress-threads, @decompress-threads and @compress-wait-thread +# are deprecated because @compression is deprecated. # # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. @@ -854,8 +864,11 @@ { 'enum': 'MigrationParameter', 'data': ['announce-initial', 'announce-max', 'announce-rounds', 'announce-step', - 'compress-level', 'compress-threads', 'decompress-threads', - 'compress-wait-thread', 'throttle-trigger-threshold', + { 'name': 'compress-level', 'features': [ 'deprecated' ] }, + { 'name': 'compress-threads', 'features': [ 'deprecated' ] }, + { 'name': 'decompress-threads', 'features': [ 'deprecated' ] }, + { 'name': 'compress-wait-thread', 'features': [ 'deprecated' ] }, + 'throttle-trigger-threshold', 'cpu-throttle-initial', 'cpu-throttle-increment', 'cpu-throttle-tailslow', 'tls-creds', 'tls-hostname', 'tls-authz', 'max-bandwidth', @@ -1023,7 +1036,9 @@ # Features: # # @deprecated: Member @block-incremental is deprecated. Use -# blockdev-mirror with NBD instead. +# blockdev-mirror with NBD instead. Members @compress-level, +# @compress-threads, @decompress-threads and @compress-wait-thread +# are deprecated because @compression is deprecated. # # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. @@ -1038,10 +1053,14 @@ '*announce-max': 'size', '*announce-rounds': 'size', '*announce-step': 'size', - '*compress-level': 'uint8', - '*compress-threads': 'uint8', - '*compress-wait-thread': 'bool', - '*decompress-threads': 'uint8', + '*compress-level': { 'type': 'uint8', + 'features': [ 'deprecated' ] }, + '*compress-threads': { 'type': 'uint8', + 'features': [ 'deprecated' ] }, + '*compress-wait-thread': { 'type': 'bool', + 'features': [ 'deprecated' ] }, + '*decompress-threads': { 'type': 'uint8', + 'features': [ 'deprecated' ] }, '*throttle-trigger-threshold': 'uint8', '*cpu-throttle-initial': 'uint8', '*cpu-throttle-increment': 'uint8', @@ -1078,7 +1097,7 @@ # Example: # # -> { "execute": "migrate-set-parameters" , -# "arguments": { "compress-level": 1 } } +# "arguments": { "multifd-channels": 5 } } # <- { "return": {} } ## { 'command': 'migrate-set-parameters', 'boxed': true, @@ -1241,7 +1260,9 @@ # Features: # # @deprecated: Member @block-incremental is deprecated. Use -# blockdev-mirror with NBD instead. +# blockdev-mirror with NBD instead. Members @compress-level, +# @compress-threads, @decompress-threads and @compress-wait-thread +# are deprecated because @compression is deprecated. # # @unstable: Members @x-checkpoint-delay and @x-vcpu-dirty-limit-period # are experimental. @@ -1253,10 +1274,14 @@ '*announce-max': 'size', '*announce-rounds': 'size', '*announce-step': 'size', - '*compress-level': 'uint8', - '*compress-threads': 'uint8', - '*compress-wait-thread': 'bool', - '*decompress-threads': 'uint8', + '*compress-level': { 'type': 'uint8', + 'features': [ 'deprecated' ] }, + '*compress-threads': { 'type': 'uint8', + 'features': [ 'deprecated' ] }, + '*compress-wait-thread': { 'type': 'bool', + 'features': [ 'deprecated' ] }, + '*decompress-threads': { 'type': 'uint8', + 'features': [ 'deprecated' ] }, '*throttle-trigger-threshold': 'uint8', '*cpu-throttle-initial': 'uint8', '*cpu-throttle-increment': 'uint8', @@ -1296,10 +1321,8 @@ # # -> { "execute": "query-migrate-parameters" } # <- { "return": { -# "decompress-threads": 2, +# "multifd-channels": 2, # "cpu-throttle-increment": 10, -# "compress-threads": 8, -# "compress-level": 1, # "cpu-throttle-initial": 20, # "max-bandwidth": 33554432, # "downtime-limit": 300 diff --git a/migration/options.c b/migration/options.c index ae8ab47e32..9a39826ca5 100644 --- a/migration/options.c +++ b/migration/options.c @@ -483,6 +483,11 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) " use blockdev-mirror with NBD instead"); } + if (new_caps[MIGRATION_CAPABILITY_COMPRESS]) { + warn_report("old compression method is deprecated;" + " use multifd compression methods instead"); + } + #ifndef CONFIG_REPLICATION if (new_caps[MIGRATION_CAPABILITY_X_COLO]) { error_setg(errp, "QEMU compiled without replication module" @@ -1335,18 +1340,26 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) /* TODO use QAPI_CLONE() instead of duplicating it inline */ if (params->has_compress_level) { + warn_report("old compression is deprecated;" + " use multifd compression methods instead"); s->parameters.compress_level = params->compress_level; } if (params->has_compress_threads) { + warn_report("old compression is deprecated;" + " use multifd compression methods instead"); s->parameters.compress_threads = params->compress_threads; } if (params->has_compress_wait_thread) { + warn_report("old compression is deprecated;" + " use multifd compression methods instead"); s->parameters.compress_wait_thread = params->compress_wait_thread; } if (params->has_decompress_threads) { + warn_report("old compression is deprecated;" + " use multifd compression methods instead"); s->parameters.decompress_threads = params->decompress_threads; } From patchwork Tue Oct 31 09:01:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441308 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 61E3DC4332F for ; Tue, 31 Oct 2023 09:07:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkeX-0001vT-QN; Tue, 31 Oct 2023 05:03:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeR-0001fY-HP for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeP-00074z-Re for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742985; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P38lkBR2tXfHat2gPQMUxrdNBlKPaKNGbHcoKTIPWWE=; b=VdMPWZw5mgoxXfq+8OtZuhErVTaFW2fz1Zr7q/VenHThZoDfxLYrLRdWbTIVPcdezdvtIY 6x9bNMj8Vlptnb8rhDfVo7Ol79NbqopdAHbnnsQXe5GFn9prqsg1fQwDLPjLHjjzTdHb3l P5aolA+maKqpeS1ZEo8o7URYmpbMW3U= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-322-PlgYXqqZMuyiIrT1aaYYXQ-1; Tue, 31 Oct 2023 05:03:00 -0400 X-MC-Unique: PlgYXqqZMuyiIrT1aaYYXQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 87532811E8D; Tue, 31 Oct 2023 09:02:59 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 093281C060B0; Tue, 31 Oct 2023 09:02:56 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , Thomas Huth Subject: [PULL 26/38] migration: Stop migration immediately in RDMA error paths Date: Tue, 31 Oct 2023 10:01:30 +0100 Message-ID: <20231031090142.13122-27-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Xu In multiple places, RDMA errors are handled in a strange way, where it only sets qemu_file_set_error() but not stop the migration immediately. It's not obvious what will happen later if there is already an error. Make all such failures stop migration immediately. Cc: Zhijian Li (Fujitsu) Cc: Markus Armbruster Cc: Juan Quintela Cc: Fabiano Rosas Reported-by: Thomas Huth Signed-off-by: Peter Xu Reviewed-by: Juan Quintela Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231024163933.516546-1-peterx@redhat.com> --- migration/ram.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 024dedb6b1..20e6153114 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2973,11 +2973,13 @@ static int ram_save_setup(QEMUFile *f, void *opaque) ret = rdma_registration_start(f, RAM_CONTROL_SETUP); if (ret < 0) { qemu_file_set_error(f, ret); + return ret; } ret = rdma_registration_stop(f, RAM_CONTROL_SETUP); if (ret < 0) { qemu_file_set_error(f, ret); + return ret; } migration_ops = g_malloc0(sizeof(MigrationOps)); @@ -3043,6 +3045,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) ret = rdma_registration_start(f, RAM_CONTROL_ROUND); if (ret < 0) { qemu_file_set_error(f, ret); + goto out; } t0 = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); @@ -3147,8 +3150,6 @@ static int ram_save_complete(QEMUFile *f, void *opaque) rs->last_stage = !migration_in_colo_state(); WITH_RCU_READ_LOCK_GUARD() { - int rdma_reg_ret; - if (!migration_in_postcopy()) { migration_bitmap_sync_precopy(rs, true); } @@ -3156,6 +3157,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) ret = rdma_registration_start(f, RAM_CONTROL_FINISH); if (ret < 0) { qemu_file_set_error(f, ret); + return ret; } /* try transferring iterative blocks of memory */ @@ -3171,24 +3173,21 @@ static int ram_save_complete(QEMUFile *f, void *opaque) break; } if (pages < 0) { - ret = pages; - break; + qemu_mutex_unlock(&rs->bitmap_mutex); + return pages; } } qemu_mutex_unlock(&rs->bitmap_mutex); compress_flush_data(); - rdma_reg_ret = rdma_registration_stop(f, RAM_CONTROL_FINISH); - if (rdma_reg_ret < 0) { - qemu_file_set_error(f, rdma_reg_ret); + ret = rdma_registration_stop(f, RAM_CONTROL_FINISH); + if (ret < 0) { + qemu_file_set_error(f, ret); + return ret; } } - if (ret < 0) { - return ret; - } - ret = multifd_send_sync_main(rs->pss[RAM_CHANNEL_PRECOPY].pss_channel); if (ret < 0) { return ret; From patchwork Tue Oct 31 09:01:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9DE37C4332F for ; Tue, 31 Oct 2023 09:06:22 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkew-0003Jh-QL; Tue, 31 Oct 2023 05:03:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeh-0002C3-5y for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeP-000751-Ts for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742985; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ec/gsbC0ZU31b9WDXxljm1D+NF4ldiXb3H5sGtwKquc=; b=da/aEsT5Sgv2EDZ8yI1/DIIKoRnN16z1yD6Uh6rlzBFHbOV6LIcnJUIdjeiVbHpGPhs/Dr LRiBB5izqJKBOyGA05CuJgC79o5caNHu2BxI8BgOKidz6KZOvdefYQ0nO+UT6Bk9zEAzEA 99TLz3ZhDA/f/mPC/fcH7oKB79mcfZM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-541-_GWVKVe-P4y1VOgA5jYhPg-1; Tue, 31 Oct 2023 05:03:02 -0400 X-MC-Unique: _GWVKVe-P4y1VOgA5jYhPg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 35E57185A79B; Tue, 31 Oct 2023 09:03:02 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id CC4391C060AE; Tue, 31 Oct 2023 09:02:59 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 27/38] qemu-file: Don't increment qemu_file_transferred at qemu_file_fill_buffer Date: Tue, 31 Oct 2023 10:01:31 +0100 Message-ID: <20231031090142.13122-28-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We only call qemu_file_transferred_* on the sending side. Remove the increment at qemu_file_fill_buffer() and add asserts to qemu_file_transferred* functions. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-2-quintela@redhat.com> --- migration/qemu-file.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 3fb25148d1..6814c562e6 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -337,7 +337,6 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEMUFile *f) if (len > 0) { f->buf_size += len; - f->total_transferred += len; } else if (len == 0) { qemu_file_set_error_obj(f, -EIO, local_error); } else { @@ -627,6 +626,8 @@ uint64_t qemu_file_transferred_noflush(QEMUFile *f) uint64_t ret = f->total_transferred; int i; + g_assert(qemu_file_is_writable(f)); + for (i = 0; i < f->iovcnt; i++) { ret += f->iov[i].iov_len; } @@ -636,6 +637,7 @@ uint64_t qemu_file_transferred_noflush(QEMUFile *f) uint64_t qemu_file_transferred(QEMUFile *f) { + g_assert(qemu_file_is_writable(f)); qemu_fflush(f); return f->total_transferred; } From patchwork Tue Oct 31 09:01:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441320 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4ACECC4332F for ; Tue, 31 Oct 2023 09:09:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkf1-0003rG-JL; Tue, 31 Oct 2023 05:03:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkei-0002F3-Vb for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkef-0007AU-9Q for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742998; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8lTpCK8bZSi+vuXOZlQ6YxReYIXK1KoMX+JOW0rufZE=; b=hsZOGfQiDz8/Betr9jHe59mszpkWj8MsVeQZyCr7vl32nTPWeom6lTCALcfpCgYPDNdAuk Ry/BM4HaEDDwoE2u9bUg3YPF2I/WCBzBjBq1pQzwQWQPGHPfjXRMqkzjAQBObcO2VoZjQI esdQvjyVxzTk3Db5PXcvMWmJ/PmR0dw= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-18-ScGKAoC1MzqkvJ6o8oMQGg-1; Tue, 31 Oct 2023 05:03:05 -0400 X-MC-Unique: ScGKAoC1MzqkvJ6o8oMQGg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D65A73822E84; Tue, 31 Oct 2023 09:03:04 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 79AA11C060AE; Tue, 31 Oct 2023 09:03:02 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 28/38] qemu_file: Use a stat64 for qemu_file_transferred Date: Tue, 31 Oct 2023 10:01:32 +0100 Message-ID: <20231031090142.13122-29-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This way we can read it from any thread. I checked that it gives the same value as the current one. We never use two qemu_files at the same time. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-3-quintela@redhat.com> --- migration/migration-stats.h | 4 ++++ migration/qemu-file.c | 5 +++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index 2358caad63..b7795e7914 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -81,6 +81,10 @@ typedef struct { * Number of bytes sent during precopy stage. */ Stat64 precopy_bytes; + /* + * Number of bytes transferred with QEMUFile. + */ + Stat64 qemu_file_transferred; /* * Amount of transferred data at the start of current cycle. */ diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 6814c562e6..384985f534 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -283,6 +283,7 @@ void qemu_fflush(QEMUFile *f) } else { uint64_t size = iov_size(f->iov, f->iovcnt); f->total_transferred += size; + stat64_add(&mig_stats.qemu_file_transferred, size); } qemu_iovec_release_ram(f); @@ -623,7 +624,7 @@ int coroutine_mixed_fn qemu_get_byte(QEMUFile *f) uint64_t qemu_file_transferred_noflush(QEMUFile *f) { - uint64_t ret = f->total_transferred; + uint64_t ret = stat64_get(&mig_stats.qemu_file_transferred); int i; g_assert(qemu_file_is_writable(f)); @@ -639,7 +640,7 @@ uint64_t qemu_file_transferred(QEMUFile *f) { g_assert(qemu_file_is_writable(f)); qemu_fflush(f); - return f->total_transferred; + return stat64_get(&mig_stats.qemu_file_transferred); } void qemu_put_be16(QEMUFile *f, unsigned int v) From patchwork Tue Oct 31 09:01:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441316 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EC136C4167B for ; Tue, 31 Oct 2023 09:08:29 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkei-0002AF-TL; Tue, 31 Oct 2023 05:03:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeX-0001vf-Kl for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeW-00078t-73 for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698742991; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2+kGTMGW3qhYBMLkFA3W6KsHWTLkquKA1hlQj0Q/xVg=; b=ggZaK2ee5m0WTjrmvCZ18BqFhH5jlDemTiRzeV4UW7uHdHJVHf7GAuYk9Lb8aLCBwDzdjO u4uI4lEqbJNZu6JPqndTW9Cdy4liXklqsAeeCpmoG1bDZVFyjNEX2ow9+Yd3ln5iDNpBh7 wm80I2zENmv0FOiaIuDzkMDMoNsAiTk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-614-xC1biQb2OA2pMpHaCELwzg-1; Tue, 31 Oct 2023 05:03:08 -0400 X-MC-Unique: xC1biQb2OA2pMpHaCELwzg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 86922811E7E; Tue, 31 Oct 2023 09:03:07 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 29A1A1C060B0; Tue, 31 Oct 2023 09:03:05 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 29/38] qemu_file: total_transferred is not used anymore Date: Tue, 31 Oct 2023 10:01:33 +0100 Message-ID: <20231031090142.13122-30-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-4-quintela@redhat.com> --- migration/qemu-file.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 384985f534..641ab703cc 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -41,9 +41,6 @@ struct QEMUFile { QIOChannel *ioc; bool is_writable; - /* The sum of bytes transferred on the wire */ - uint64_t total_transferred; - int buf_index; int buf_size; /* 0 when writing */ uint8_t buf[IO_BUF_SIZE]; @@ -282,7 +279,6 @@ void qemu_fflush(QEMUFile *f) qemu_file_set_error_obj(f, -EIO, local_error); } else { uint64_t size = iov_size(f->iov, f->iovcnt); - f->total_transferred += size; stat64_add(&mig_stats.qemu_file_transferred, size); } From patchwork Tue Oct 31 09:01:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 78FCFC4332F for ; Tue, 31 Oct 2023 09:09:25 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkf4-00049k-Rb; Tue, 31 Oct 2023 05:03:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkes-0002gK-AX for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeq-0007Ce-JO for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698743010; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tF+ygvJ0bwO3q4A/qBLLXKpVMn37FRDSfsWOUVUJl+I=; b=J2w5Jw0zcMHARLhChipyS5/vQxfyeNDjP1bw6YZXXFIdLq3TEMcZ/3asTL3HwZ7iMIeexS kTKOaqoDuDmQAbRPSTC2jQC54TFU+uBbg/SzuQhul4vuwKuvxvR/hjN2zlO1p83LKUs4Jx nU1BVnuHAGIDps/FAA9AmhBqFvrs8fA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-628-v7wI9djZNemqu3NJx3KF5w-1; Tue, 31 Oct 2023 05:03:10 -0400 X-MC-Unique: v7wI9djZNemqu3NJx3KF5w-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 727C981F44B; Tue, 31 Oct 2023 09:03:10 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id CBF921C060AE; Tue, 31 Oct 2023 09:03:07 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 30/38] migration: Use the number of transferred bytes directly Date: Tue, 31 Oct 2023 10:01:34 +0100 Message-ID: <20231031090142.13122-31-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We only use migration_transferred_bytes() to calculate the rate_limit, for that we don't need to flush whatever is on the qemu_file buffer. Remember that the buffer is really small (normal case is 32K if we use iov's can be 64 * TARGET_PAGE_SIZE), so this is not relevant to calculations. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-5-quintela@redhat.com> --- migration/migration-stats.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/migration-stats.c b/migration/migration-stats.c index 4cc989d975..1d9197b4c3 100644 --- a/migration/migration-stats.c +++ b/migration/migration-stats.c @@ -63,7 +63,7 @@ uint64_t migration_transferred_bytes(QEMUFile *f) { uint64_t multifd = stat64_get(&mig_stats.multifd_bytes); uint64_t rdma = stat64_get(&mig_stats.rdma_bytes); - uint64_t qemu_file = qemu_file_transferred(f); + uint64_t qemu_file = stat64_get(&mig_stats.qemu_file_transferred); trace_migration_transferred_bytes(qemu_file, multifd, rdma); return qemu_file + multifd + rdma; From patchwork Tue Oct 31 09:01:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441288 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C3911C4332F for ; Tue, 31 Oct 2023 09:04:42 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkf3-00046H-Pa; Tue, 31 Oct 2023 05:03:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeo-0002Ku-8P for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkem-0007CC-Ky for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698743006; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PcqbjGXJuRCp6TIW51sCb5QOIYpZ1WT4y0e1Wgyy7AY=; b=YRJzN5as0i93cIQ4ddEMAWUNDNoYD2AyARfNr+6LLu/ikeb32DtSoMM/AZBtsv/7PvPQ0t rzkHDYaT7lNYlc1MCX1iCvc0M+iHVPZUut82Y5pGS5j7vxz8XZMgF/q+qZvUBOJ0kuwh7U kcga8DcqPtCwWbEWwrY1wq7Gp8CdCFA= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-620-VjQKpgtyPVCk12AWlM7QPw-1; Tue, 31 Oct 2023 05:03:13 -0400 X-MC-Unique: VjQKpgtyPVCk12AWlM7QPw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 29E173C1042A; Tue, 31 Oct 2023 09:03:13 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id B58201C060AE; Tue, 31 Oct 2023 09:03:10 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 31/38] qemu_file: Remove unused qemu_file_transferred() Date: Tue, 31 Oct 2023 10:01:35 +0100 Message-ID: <20231031090142.13122-32-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-6-quintela@redhat.com> --- migration/qemu-file.h | 18 ------------------ migration/qemu-file.c | 7 ------- 2 files changed, 25 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index a29c37b0d0..8b71152754 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -33,24 +33,6 @@ QEMUFile *qemu_file_new_input(QIOChannel *ioc); QEMUFile *qemu_file_new_output(QIOChannel *ioc); int qemu_fclose(QEMUFile *f); -/* - * qemu_file_transferred: - * - * Report the total number of bytes transferred with - * this file. - * - * For writable files, any pending buffers will be - * flushed, so the reported value will be equal to - * the number of bytes transferred on the wire. - * - * For readable files, the reported value will be - * equal to the number of bytes transferred on the - * wire. - * - * Returns: the total bytes transferred - */ -uint64_t qemu_file_transferred(QEMUFile *f); - /* * qemu_file_transferred_noflush: * diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 641ab703cc..efa5f11033 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -632,13 +632,6 @@ uint64_t qemu_file_transferred_noflush(QEMUFile *f) return ret; } -uint64_t qemu_file_transferred(QEMUFile *f) -{ - g_assert(qemu_file_is_writable(f)); - qemu_fflush(f); - return stat64_get(&mig_stats.qemu_file_transferred); -} - void qemu_put_be16(QEMUFile *f, unsigned int v) { qemu_put_byte(f, v >> 8); From patchwork Tue Oct 31 09:01:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 30ED3C4332F for ; Tue, 31 Oct 2023 09:09:32 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkeu-0002uX-9R; Tue, 31 Oct 2023 05:03:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkei-0002F5-6z for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkef-0007Ae-Dn for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698743000; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QjSL1hcG9VuYm/7FTdP+GJGNi8SYhcnfOVzKe90L+6k=; b=QW5ULXMsoxiIcSvxRY/iN8NJYDMuJ0kTkQVRVWR+cbBiTXPPQ1aR1NauntJueTzbTJZKfn BHf6gx5lKA4QsN9Omk49GVNSO/qZNljj2WC0lcX96VToBdWmrFlYXmutrtV7wMx1iwe4OV XnRmL67zE2EQ2943Mp+/DS01s698glA= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-79-FG3gJ5aLNo6mh1oG1xtdyA-1; Tue, 31 Oct 2023 05:03:16 -0400 X-MC-Unique: FG3gJ5aLNo6mh1oG1xtdyA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CE3F93C10422; Tue, 31 Oct 2023 09:03:15 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 718791C060AE; Tue, 31 Oct 2023 09:03:13 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 32/38] qemu-file: Remove _noflush from qemu_file_transferred_noflush() Date: Tue, 31 Oct 2023 10:01:36 +0100 Message-ID: <20231031090142.13122-33-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org qemu_file_transferred() don't exist anymore, so we can reuse the name. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-7-quintela@redhat.com> Signed-off-by: Juan Quintela --- migration/qemu-file.h | 9 ++++----- migration/block.c | 4 ++-- migration/qemu-file.c | 2 +- migration/savevm.c | 6 +++--- migration/vmstate.c | 4 ++-- 5 files changed, 12 insertions(+), 13 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 8b71152754..1b2f6b8d8f 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -34,15 +34,14 @@ QEMUFile *qemu_file_new_output(QIOChannel *ioc); int qemu_fclose(QEMUFile *f); /* - * qemu_file_transferred_noflush: + * qemu_file_transferred: * - * As qemu_file_transferred except for writable files, where no flush - * is performed and the reported amount will include the size of any - * queued buffers, on top of the amount actually transferred. + * No flush is performed and the reported amount will include the size + * of any queued buffers, on top of the amount actually transferred. * * Returns: the total bytes transferred and queued */ -uint64_t qemu_file_transferred_noflush(QEMUFile *f); +uint64_t qemu_file_transferred(QEMUFile *f); /* * put_buffer without copying the buffer. diff --git a/migration/block.c b/migration/block.c index acffe88f84..a15f9bddcb 100644 --- a/migration/block.c +++ b/migration/block.c @@ -755,7 +755,7 @@ static int block_save_setup(QEMUFile *f, void *opaque) static int block_save_iterate(QEMUFile *f, void *opaque) { int ret; - uint64_t last_bytes = qemu_file_transferred_noflush(f); + uint64_t last_bytes = qemu_file_transferred(f); trace_migration_block_save("iterate", block_mig_state.submitted, block_mig_state.transferred); @@ -807,7 +807,7 @@ static int block_save_iterate(QEMUFile *f, void *opaque) } qemu_put_be64(f, BLK_MIG_FLAG_EOS); - uint64_t delta_bytes = qemu_file_transferred_noflush(f) - last_bytes; + uint64_t delta_bytes = qemu_file_transferred(f) - last_bytes; return (delta_bytes > 0); } diff --git a/migration/qemu-file.c b/migration/qemu-file.c index efa5f11033..0158db2a54 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -618,7 +618,7 @@ int coroutine_mixed_fn qemu_get_byte(QEMUFile *f) return result; } -uint64_t qemu_file_transferred_noflush(QEMUFile *f) +uint64_t qemu_file_transferred(QEMUFile *f) { uint64_t ret = stat64_get(&mig_stats.qemu_file_transferred); int i; diff --git a/migration/savevm.c b/migration/savevm.c index ca5c7cebe0..9ec78abd53 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -927,9 +927,9 @@ static int vmstate_load(QEMUFile *f, SaveStateEntry *se) static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se, JSONWriter *vmdesc) { - uint64_t old_offset = qemu_file_transferred_noflush(f); + uint64_t old_offset = qemu_file_transferred(f); se->ops->save_state(f, se->opaque); - uint64_t size = qemu_file_transferred_noflush(f) - old_offset; + uint64_t size = qemu_file_transferred(f) - old_offset; if (vmdesc) { json_writer_int64(vmdesc, "size", size); @@ -3053,7 +3053,7 @@ bool save_snapshot(const char *name, bool overwrite, const char *vmstate, goto the_end; } ret = qemu_savevm_state(f, errp); - vm_state_size = qemu_file_transferred_noflush(f); + vm_state_size = qemu_file_transferred(f); ret2 = qemu_fclose(f); if (ret < 0) { goto the_end; diff --git a/migration/vmstate.c b/migration/vmstate.c index 9c36803c8a..b7723a4187 100644 --- a/migration/vmstate.c +++ b/migration/vmstate.c @@ -387,7 +387,7 @@ int vmstate_save_state_v(QEMUFile *f, const VMStateDescription *vmsd, void *curr_elem = first_elem + size * i; vmsd_desc_field_start(vmsd, vmdesc_loop, field, i, n_elems); - old_offset = qemu_file_transferred_noflush(f); + old_offset = qemu_file_transferred(f); if (field->flags & VMS_ARRAY_OF_POINTER) { assert(curr_elem); curr_elem = *(void **)curr_elem; @@ -417,7 +417,7 @@ int vmstate_save_state_v(QEMUFile *f, const VMStateDescription *vmsd, return ret; } - written_bytes = qemu_file_transferred_noflush(f) - old_offset; + written_bytes = qemu_file_transferred(f) - old_offset; vmsd_desc_field_end(vmsd, vmdesc_loop, field, written_bytes, i); /* Compressed arrays only care about the first element */ From patchwork Tue Oct 31 09:01:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6AFA2C4332F for ; Tue, 31 Oct 2023 09:08:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkey-0003Zu-Pu; Tue, 31 Oct 2023 05:03:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkek-0002Fm-DJ for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeh-0007BA-Or for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698743003; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gVvfzwqt9WXnmvnFt6LXme9hyJVoOdI8qQBUN/vFVMk=; b=ZaDAAJ+FhMPEogyY2rNkU5mPCBSlEv8QIvtH49bMHDLFZB2IXyk9I7c4Frbvx+CerzDOe4 JImsfzaCAj3rB8s1zK10W0AFR7m9q7ypTlumMOTFNXrzLxc5QeiVCQCrn9ZLu1LmjJ4Zti 73wUNfBHPUW+yP2VxiMW7I8KcXwf3tw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-353-OdHeBjRcNn2cMTjZzUIrSg-1; Tue, 31 Oct 2023 05:03:19 -0400 X-MC-Unique: OdHeBjRcNn2cMTjZzUIrSg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A768985A58A; Tue, 31 Oct 2023 09:03:18 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 21E8F1C060AE; Tue, 31 Oct 2023 09:03:15 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 33/38] migration: migration_transferred_bytes() don't need the QEMUFile Date: Tue, 31 Oct 2023 10:01:37 +0100 Message-ID: <20231031090142.13122-34-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-8-quintela@redhat.com> --- migration/migration-stats.h | 4 +--- migration/migration-stats.c | 6 +++--- migration/migration.c | 6 +++--- 3 files changed, 7 insertions(+), 9 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index b7795e7914..e3863bf9bb 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -137,11 +137,9 @@ void migration_rate_set(uint64_t new_rate); /** * migration_transferred_bytes: Return number of bytes transferred * - * @f: QEMUFile used for main migration channel - * * Returns how many bytes have we transferred since the beginning of * the migration. It accounts for bytes sent through any migration * channel, multifd, qemu_file, rdma, .... */ -uint64_t migration_transferred_bytes(QEMUFile *f); +uint64_t migration_transferred_bytes(void); #endif diff --git a/migration/migration-stats.c b/migration/migration-stats.c index 1d9197b4c3..4ae8c0c722 100644 --- a/migration/migration-stats.c +++ b/migration/migration-stats.c @@ -30,7 +30,7 @@ bool migration_rate_exceeded(QEMUFile *f) } uint64_t rate_limit_start = stat64_get(&mig_stats.rate_limit_start); - uint64_t rate_limit_current = migration_transferred_bytes(f); + uint64_t rate_limit_current = migration_transferred_bytes(); uint64_t rate_limit_used = rate_limit_current - rate_limit_start; if (rate_limit_max > 0 && rate_limit_used > rate_limit_max) { @@ -56,10 +56,10 @@ void migration_rate_set(uint64_t limit) void migration_rate_reset(QEMUFile *f) { - stat64_set(&mig_stats.rate_limit_start, migration_transferred_bytes(f)); + stat64_set(&mig_stats.rate_limit_start, migration_transferred_bytes()); } -uint64_t migration_transferred_bytes(QEMUFile *f) +uint64_t migration_transferred_bytes(void) { uint64_t multifd = stat64_get(&mig_stats.multifd_bytes); uint64_t rdma = stat64_get(&mig_stats.rdma_bytes); diff --git a/migration/migration.c b/migration/migration.c index cb2d7161b5..04d4546150 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2704,7 +2704,7 @@ static MigThrError migration_detect_error(MigrationState *s) static void migration_calculate_complete(MigrationState *s) { - uint64_t bytes = migration_transferred_bytes(s->to_dst_file); + uint64_t bytes = migration_transferred_bytes(); int64_t end_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); int64_t transfer_time; @@ -2730,7 +2730,7 @@ static void update_iteration_initial_status(MigrationState *s) * wrong speed calculation. */ s->iteration_start_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); - s->iteration_initial_bytes = migration_transferred_bytes(s->to_dst_file); + s->iteration_initial_bytes = migration_transferred_bytes(); s->iteration_initial_pages = ram_get_total_transferred_pages(); } @@ -2749,7 +2749,7 @@ static void migration_update_counters(MigrationState *s, } switchover_bw = migrate_avail_switchover_bandwidth(); - current_bytes = migration_transferred_bytes(s->to_dst_file); + current_bytes = migration_transferred_bytes(); transferred = current_bytes - s->iteration_initial_bytes; time_spent = current_time - s->iteration_start_time; bandwidth = (double)transferred / time_spent; From patchwork Tue Oct 31 09:01:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441326 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AFAA5C4332F for ; Tue, 31 Oct 2023 09:10:23 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxket-0002mY-Bp; Tue, 31 Oct 2023 05:03:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxken-0002KD-0r for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkek-0007By-Nx for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698743006; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n8I8b00ywXtgCefxVuG8Ld+7VEOyo7QAMJMHTl3NboQ=; b=DVrx989Lqx8lhgq4Fvn2yk2Hvhax0JEMsGz+fx6d/DFK0+Ok4pijN8BuEUXE8Utj+9clAk dSQUXaTwu5GWGX+b4nU+LT0NP8Y1tRDwf3JYDVk3A/wrkvqNMoPGH52BWAYZ5QVjMKOFvH ri1cA5+M79m8s3cqSD2zTp3qOqkyaLo= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-186-LWWYnzg7Oz6rjp8UryLpOQ-1; Tue, 31 Oct 2023 05:03:22 -0400 X-MC-Unique: LWWYnzg7Oz6rjp8UryLpOQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7BF0D101A54C; Tue, 31 Oct 2023 09:03:21 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id EF1EA1C060AE; Tue, 31 Oct 2023 09:03:18 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 34/38] migration: migration_rate_limit_reset() don't need the QEMUFile Date: Tue, 31 Oct 2023 10:01:38 +0100 Message-ID: <20231031090142.13122-35-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-9-quintela@redhat.com> --- migration/migration-stats.h | 4 +--- migration/migration-stats.c | 2 +- migration/migration.c | 2 +- 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index e3863bf9bb..68f3939188 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -120,10 +120,8 @@ uint64_t migration_rate_get(void); * migration_rate_reset: Reset the rate limit counter. * * This is called when we know we start a new transfer cycle. - * - * @f: QEMUFile used for main migration channel */ -void migration_rate_reset(QEMUFile *f); +void migration_rate_reset(void); /** * migration_rate_set: Set the maximum amount that can be transferred. diff --git a/migration/migration-stats.c b/migration/migration-stats.c index 4ae8c0c722..f690b98a03 100644 --- a/migration/migration-stats.c +++ b/migration/migration-stats.c @@ -54,7 +54,7 @@ void migration_rate_set(uint64_t limit) stat64_set(&mig_stats.rate_limit_max, limit / XFER_LIMIT_RATIO); } -void migration_rate_reset(QEMUFile *f) +void migration_rate_reset(void) { stat64_set(&mig_stats.rate_limit_start, migration_transferred_bytes()); } diff --git a/migration/migration.c b/migration/migration.c index 04d4546150..a25a2f3c54 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2785,7 +2785,7 @@ static void migration_update_counters(MigrationState *s, stat64_get(&mig_stats.dirty_bytes_last_sync) / expected_bw_per_ms; } - migration_rate_reset(s->to_dst_file); + migration_rate_reset(); update_iteration_initial_status(s); From patchwork Tue Oct 31 09:01:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441290 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3E268C4167B for ; Tue, 31 Oct 2023 09:05:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkfF-0005WM-NP; Tue, 31 Oct 2023 05:03:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkfD-0005CS-8E for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkfB-0007Gn-Lw for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698743033; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W79d6K2ZBM5QCtmCBTtqfT3+lDpsLxhv0xvj+rJ2sXU=; b=VE42AIuFDeitopHIHOD4QhkTsXF3X/9Mnyl88eCHPCeLdspXFstICIX4VurJfXE/hXcXIP 3FErm1TTrgkZd39q/PFuWTUpvqSwzQetpgORMQXgGCGYJT7+h4TnD9OR+5LAKWwK6ufelC Ds/BXZhHDDbj1a+v1fIw81hMkql0hgM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-58-SPV1jI5tMWe22vwhpEg3Kg-1; Tue, 31 Oct 2023 05:03:24 -0400 X-MC-Unique: SPV1jI5tMWe22vwhpEg3Kg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2BE12811E7E; Tue, 31 Oct 2023 09:03:24 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id C2D231C060AE; Tue, 31 Oct 2023 09:03:21 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 35/38] qemu-file: Simplify qemu_file_get_error() Date: Tue, 31 Oct 2023 10:01:39 +0100 Message-ID: <20231031090142.13122-36-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org If we pass a NULL error is the same that returning directly the value. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-10-quintela@redhat.com> --- migration/qemu-file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 0158db2a54..7e738743ce 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -204,7 +204,7 @@ void qemu_file_set_error_obj(QEMUFile *f, int ret, Error *err) */ int qemu_file_get_error(QEMUFile *f) { - return qemu_file_get_error_obj(f, NULL); + return f->last_error; } /* From patchwork Tue Oct 31 09:01:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441314 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B0B7CC4167D for ; Tue, 31 Oct 2023 09:08:14 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkf0-0003ej-Bx; Tue, 31 Oct 2023 05:03:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkes-0002lG-Tr for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkeq-0007Ci-Jy for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698743010; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gzYf61QPolPgq39fnfrhBsobCf/WZvu1CQ7mneB5U4s=; b=Ytr6FX+XDLMi3PIqgFee6hjZHCYOmJSyuUWsjkh0k8nst9C2YypLiTmnPkQfx8nyeRQXs6 NB0fKSEvqR4l8ZQgRA2GLHPanAofJhCQtMTZLC2N3523l+SmXnl1edOf1t6MY6F29wXM77 uTjFiE0D4k7gBLALAgfnoa6H2IwJdpA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-311-GuXv93ISMEKZER4TnY4URg-1; Tue, 31 Oct 2023 05:03:27 -0400 X-MC-Unique: GuXv93ISMEKZER4TnY4URg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D5E048F77C4; Tue, 31 Oct 2023 09:03:26 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 714C31C060AE; Tue, 31 Oct 2023 09:03:24 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 36/38] migration: Use migration_transferred_bytes() Date: Tue, 31 Oct 2023 10:01:40 +0100 Message-ID: <20231031090142.13122-37-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org There are only two differnces with the old value: - the amount of QEMUFile that hasn't yet been flushed. It can be discussed what is more exact, the new or the old one. - the amount of transferred bytes that we forgot to account for (the newer is better, i.e. exact). Notice that this two values are used to: a - present to the user b - calculate the rate_limit So a few KB here and there is not going to make a difference. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-11-quintela@redhat.com> --- migration/migration.c | 2 +- migration/ram.c | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index a25a2f3c54..aa7b791833 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -942,7 +942,7 @@ static void populate_ram_info(MigrationInfo *info, MigrationState *s) size_t page_size = qemu_target_page_size(); info->ram = g_malloc0(sizeof(*info->ram)); - info->ram->transferred = stat64_get(&mig_stats.transferred); + info->ram->transferred = migration_transferred_bytes(); info->ram->total = ram_bytes_total(); info->ram->duplicate = stat64_get(&mig_stats.zero_pages); /* legacy value. It is not used anymore */ diff --git a/migration/ram.c b/migration/ram.c index 20e6153114..8dd36e3d2b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -557,7 +557,7 @@ void mig_throttle_counter_reset(void) rs->time_last_bitmap_sync = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); rs->num_dirty_pages_period = 0; - rs->bytes_xfer_prev = stat64_get(&mig_stats.transferred); + rs->bytes_xfer_prev = migration_transferred_bytes(); } /** @@ -1003,7 +1003,7 @@ static void migration_trigger_throttle(RAMState *rs) { uint64_t threshold = migrate_throttle_trigger_threshold(); uint64_t bytes_xfer_period = - stat64_get(&mig_stats.transferred) - rs->bytes_xfer_prev; + migration_transferred_bytes() - rs->bytes_xfer_prev; uint64_t bytes_dirty_period = rs->num_dirty_pages_period * TARGET_PAGE_SIZE; uint64_t bytes_dirty_threshold = bytes_xfer_period * threshold / 100; @@ -1073,7 +1073,7 @@ static void migration_bitmap_sync(RAMState *rs, bool last_stage) /* reset period counters */ rs->time_last_bitmap_sync = end_time; rs->num_dirty_pages_period = 0; - rs->bytes_xfer_prev = stat64_get(&mig_stats.transferred); + rs->bytes_xfer_prev = migration_transferred_bytes(); } if (migrate_events()) { uint64_t generation = stat64_get(&mig_stats.dirty_sync_count); From patchwork Tue Oct 31 09:01:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441318 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B93D0C4167D for ; Tue, 31 Oct 2023 09:09:09 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkf5-0004Ai-HU; Tue, 31 Oct 2023 05:03:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkez-0003ev-EM for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkex-0007FH-HP for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698743018; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VPzZaKGcaSECpnJZBaytwit7A0fCgcaBXi1fArd4hN0=; b=JEoQKzmm27UUb7bkS+vfc1CKlJaNszTqedxG64D330wlOChewo82E6AD4Yok8EtDvC7cBd f4w2qMXIcVLIpPCg8egZqpdx6/79SPEAmMlvaJeUoKYzL8w3IGrMgqYrK6fiZW0I6fq3e5 iMKQ+Obunhr6fTkRDNXVPs/5GG+GG2I= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-179-95h_zTKdPwuZ-T8vXP0zTQ-1; Tue, 31 Oct 2023 05:03:29 -0400 X-MC-Unique: 95h_zTKdPwuZ-T8vXP0zTQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 85CA229ABA13; Tue, 31 Oct 2023 09:03:29 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 29D711C060AE; Tue, 31 Oct 2023 09:03:27 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas Subject: [PULL 37/38] migration: Remove transferred atomic counter Date: Tue, 31 Oct 2023 10:01:41 +0100 Message-ID: <20231031090142.13122-38-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org After last commit, it is a write only variable. Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-12-quintela@redhat.com> --- migration/migration-stats.h | 4 ---- migration/multifd.c | 3 --- migration/ram.c | 1 - 3 files changed, 8 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index 68f3939188..05290ade76 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -97,10 +97,6 @@ typedef struct { * Number of bytes sent through RDMA. */ Stat64 rdma_bytes; - /* - * Total number of bytes transferred. - */ - Stat64 transferred; /* * Number of pages transferred that were full of zeros. */ diff --git a/migration/multifd.c b/migration/multifd.c index e2a45c667a..ec58c58082 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -188,7 +188,6 @@ static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) return -1; } stat64_add(&mig_stats.multifd_bytes, size); - stat64_add(&mig_stats.transferred, size); return 0; } @@ -733,8 +732,6 @@ static void *multifd_send_thread(void *opaque) stat64_add(&mig_stats.multifd_bytes, p->next_packet_size + p->packet_len); - stat64_add(&mig_stats.transferred, - p->next_packet_size + p->packet_len); p->next_packet_size = 0; qemu_mutex_lock(&p->mutex); p->pending_job--; diff --git a/migration/ram.c b/migration/ram.c index 8dd36e3d2b..cec9bd31d9 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -448,7 +448,6 @@ void ram_transferred_add(uint64_t bytes) } else { stat64_add(&mig_stats.downtime_bytes, bytes); } - stat64_add(&mig_stats.transferred, bytes); } struct MigrationOps { From patchwork Tue Oct 31 09:01:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 13441289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 78CF6C4167B for ; Tue, 31 Oct 2023 09:04:45 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxkfB-0004qm-AW; Tue, 31 Oct 2023 05:03:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkf9-0004ax-3H for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxkf7-0007GK-2D for qemu-devel@nongnu.org; Tue, 31 Oct 2023 05:03:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698743027; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ni1kuRiso1jGYq5sm4grqkz4KA3VXrdMN/V5792pZt0=; b=ClxFoIewpyanYzo1hCC9n9XZvJsAvtAUgYDNN0ad+5RsMrGdoIvLed5KXn5ws+pYPKIj7Y dr21mRrIflx/Xc2v3MMS9Oqzmt37dqnbEQfZego+PZJqQzUY3A5Lvb4O9tSlr7VUCuTCNK Mq8MNs14JeMG35PttTmjrG+BNkkBXCk= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-459-AcpQ_R2oPia8nlQ6w-2Mkg-1; Tue, 31 Oct 2023 05:03:32 -0400 X-MC-Unique: AcpQ_R2oPia8nlQ6w-2Mkg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 592DF29ABA16; Tue, 31 Oct 2023 09:03:32 +0000 (UTC) Received: from secure.mitica (unknown [10.39.195.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id CDFBC1C060AE; Tue, 31 Oct 2023 09:03:29 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Cc: Kevin Wolf , libvir-list@redhat.com, Stefan Hajnoczi , Fam Zheng , Hanna Reitz , Li Zhijian , Peter Xu , Leonardo Bras , Juan Quintela , Eric Blake , Markus Armbruster , Hailiang Zhang , qemu-block@nongnu.org, Fabiano Rosas , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 38/38] qemu-file: Make qemu_fflush() return errors Date: Tue, 31 Oct 2023 10:01:42 +0100 Message-ID: <20231031090142.13122-39-quintela@redhat.com> In-Reply-To: <20231031090142.13122-1-quintela@redhat.com> References: <20231031090142.13122-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.483, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This let us simplify code of this shape. qemu_fflush(f); int ret = qemu_file_get_error(f); if (ret) { return ret; } into: int ret = qemu_fflush(f); if (ret) { return ret; } I updated all callers where there is any error check. qemu_fclose() don't need to check for f->last_error because qemu_fflush() returns it at the beggining of the function. Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Fabiano Rosas Signed-off-by: Juan Quintela Message-ID: <20231025091117.6342-13-quintela@redhat.com> Signed-off-by: Juan Quintela --- migration/qemu-file.h | 2 +- migration/colo.c | 11 +++-------- migration/migration.c | 7 +------ migration/qemu-file.c | 23 +++++++---------------- migration/ram.c | 22 +++++++--------------- migration/rdma.c | 4 +--- migration/savevm.c | 3 +-- 7 files changed, 21 insertions(+), 51 deletions(-) diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 1b2f6b8d8f..1774116f79 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -71,7 +71,7 @@ void qemu_file_set_error_obj(QEMUFile *f, int ret, Error *err); void qemu_file_set_error(QEMUFile *f, int ret); int qemu_file_shutdown(QEMUFile *f); QEMUFile *qemu_file_get_return_path(QEMUFile *f); -void qemu_fflush(QEMUFile *f); +int qemu_fflush(QEMUFile *f); void qemu_file_set_blocking(QEMUFile *f, bool block); int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size); diff --git a/migration/colo.c b/migration/colo.c index 72f4f7b37e..4447e34914 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -314,9 +314,7 @@ static void colo_send_message(QEMUFile *f, COLOMessage msg, return; } qemu_put_be32(f, msg); - qemu_fflush(f); - - ret = qemu_file_get_error(f); + ret = qemu_fflush(f); if (ret < 0) { error_setg_errno(errp, -ret, "Can't send COLO message"); } @@ -335,9 +333,7 @@ static void colo_send_message_value(QEMUFile *f, COLOMessage msg, return; } qemu_put_be64(f, value); - qemu_fflush(f); - - ret = qemu_file_get_error(f); + ret = qemu_fflush(f); if (ret < 0) { error_setg_errno(errp, -ret, "Failed to send value for message:%s", COLOMessage_str(msg)); @@ -483,8 +479,7 @@ static int colo_do_checkpoint_transaction(MigrationState *s, } qemu_put_buffer(s->to_dst_file, bioc->data, bioc->usage); - qemu_fflush(s->to_dst_file); - ret = qemu_file_get_error(s->to_dst_file); + ret = qemu_fflush(s->to_dst_file); if (ret < 0) { goto out; } diff --git a/migration/migration.c b/migration/migration.c index aa7b791833..6abcbefd9c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -305,12 +305,7 @@ static int migrate_send_rp_message(MigrationIncomingState *mis, qemu_put_be16(mis->to_src_file, (unsigned int)message_type); qemu_put_be16(mis->to_src_file, len); qemu_put_buffer(mis->to_src_file, data, len); - qemu_fflush(mis->to_src_file); - - /* It's possible that qemu file got error during sending */ - ret = qemu_file_get_error(mis->to_src_file); - - return ret; + return qemu_fflush(mis->to_src_file); } /* Request one page from the source VM at the given start address. diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 7e738743ce..d64500310d 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -262,14 +262,14 @@ static void qemu_iovec_release_ram(QEMUFile *f) * This will flush all pending data. If data was only partially flushed, it * will set an error state. */ -void qemu_fflush(QEMUFile *f) +int qemu_fflush(QEMUFile *f) { if (!qemu_file_is_writable(f)) { - return; + return f->last_error; } - if (qemu_file_get_error(f)) { - return; + if (f->last_error) { + return f->last_error; } if (f->iovcnt > 0) { Error *local_error = NULL; @@ -287,6 +287,7 @@ void qemu_fflush(QEMUFile *f) f->buf_index = 0; f->iovcnt = 0; + return f->last_error; } /* @@ -353,22 +354,12 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEMUFile *f) */ int qemu_fclose(QEMUFile *f) { - int ret, ret2; - qemu_fflush(f); - ret = qemu_file_get_error(f); - - ret2 = qio_channel_close(f->ioc, NULL); + int ret = qemu_fflush(f); + int ret2 = qio_channel_close(f->ioc, NULL); if (ret >= 0) { ret = ret2; } g_clear_pointer(&f->ioc, object_unref); - - /* If any error was spotted before closing, we should report it - * instead of the close() return value. - */ - if (f->last_error) { - ret = f->last_error; - } error_free(f->last_error_obj); g_free(f); trace_qemu_file_fclose(); diff --git a/migration/ram.c b/migration/ram.c index cec9bd31d9..34724e8fe8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -305,17 +305,15 @@ int64_t ramblock_recv_bitmap_send(QEMUFile *file, qemu_put_be64(file, size); qemu_put_buffer(file, (const uint8_t *)le_bitmap, size); + g_free(le_bitmap); /* * Mark as an end, in case the middle part is screwed up due to * some "mysterious" reason. */ qemu_put_be64(file, RAMBLOCK_RECV_BITMAP_ENDING); - qemu_fflush(file); - - g_free(le_bitmap); - - if (qemu_file_get_error(file)) { - return qemu_file_get_error(file); + int ret = qemu_fflush(file); + if (ret) { + return ret; } return size + sizeof(size); @@ -2996,9 +2994,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) } qemu_put_be64(f, RAM_SAVE_FLAG_EOS); - qemu_fflush(f); - - return 0; + return qemu_fflush(f); } /** @@ -3118,10 +3114,8 @@ out: } qemu_put_be64(f, RAM_SAVE_FLAG_EOS); - qemu_fflush(f); ram_transferred_add(8); - - ret = qemu_file_get_error(f); + ret = qemu_fflush(f); } if (ret < 0) { return ret; @@ -3196,9 +3190,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_FLUSH); } qemu_put_be64(f, RAM_SAVE_FLAG_EOS); - qemu_fflush(f); - - return 0; + return qemu_fflush(f); } static void ram_state_pending_estimate(void *opaque, uint64_t *must_precopy, diff --git a/migration/rdma.c b/migration/rdma.c index e3493e3b3e..2938db4f64 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3853,9 +3853,7 @@ int rdma_registration_start(QEMUFile *f, uint64_t flags) trace_rdma_registration_start(flags); qemu_put_be64(f, RAM_SAVE_FLAG_HOOK); - qemu_fflush(f); - - return 0; + return qemu_fflush(f); } /* diff --git a/migration/savevm.c b/migration/savevm.c index 9ec78abd53..c7835e9c73 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1583,8 +1583,7 @@ int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only, } flush: - qemu_fflush(f); - return 0; + return qemu_fflush(f); } /* Give an estimate of the amount left to be transferred,