From patchwork Thu Oct 17 06:42:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong Huang X-Patchwork-Id: 13839501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 664F7D2F7F4 for ; Thu, 17 Oct 2024 06:43:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t1KE8-0001gE-3J; Thu, 17 Oct 2024 02:43:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t1KE5-0001fF-Ey for qemu-devel@nongnu.org; Thu, 17 Oct 2024 02:43:13 -0400 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t1KE3-0005F4-M6 for qemu-devel@nongnu.org; Thu, 17 Oct 2024 02:43:13 -0400 Received: by mail-pj1-x1030.google.com with SMTP id 98e67ed59e1d1-2e2bd347124so428210a91.1 for ; Wed, 16 Oct 2024 23:43:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1729147390; x=1729752190; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=T6S1/+0v2iIFTN9IgGH3Oc8MyncjueTnrzbQJcQEhw8=; b=doI/06lQliCs4Vk34jLYvocAnAipMTopoQ1pltv4w5HKiiUU/LiUev8JyMWuAjAPAx AsEiui3jTRssBRnKD7idWDtOPd4j03/HWXp5EBMPU/heRGzVinY0abYzbVFEIPlORSSW EHkkGk2oJiwjGz2rMrr2OWJPPYONEkndwyqcg0eljVTID52PtvkloRKbvyH/marQNNQ1 12bTrXZUJAVk+9vomykjL5mhOMb4zC9xqXyyU6Avd9vJ0KudBCmIPL5rw1Ks1w8JAGCE 1yE33v89lDlR0nxoRP/xHehACQ0ukEWvPssa9Y8C4XpGOKBz4mxARqVf49fI8Gim1TAC /MSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729147390; x=1729752190; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=T6S1/+0v2iIFTN9IgGH3Oc8MyncjueTnrzbQJcQEhw8=; b=jAgjk0FE561FnGxeyy0KkVGdE+EtXGfkETzYwTaqbOktnaheJGETklZlAEGQgh55xr MoNWyjEl/AENcSKffrT9bXpw/4XdElOVvpk34dtfOmN7ZMbatRMFl6K4TiBAtTnLUVMO 8+2ekZHo9LVcGWwRrPlLJOcCzBPjabUPzcVpVpj8nO5v2ID+wMeDhVgNYFZv9fBffwXh nrStsRfEnaMz9wJrKbFUToKfAfS0Viz/9ZqNwUDc7IIH5ihp7Euv2ZwBtwnW3ScJZPM0 8JIRQLCGAl/EW/8WVI64jG87Jb8pU3sNX86MCQ5JZ7QHvlMRcrDQza7kLsRETdWW1UHQ kvVQ== X-Gm-Message-State: AOJu0YyDJBpZ26o/+ySk4RnuJGzrFZOZIWHkA9IQ+DaGUVGC0nO+8mqg 9wbmRffzm/MDdE0Z83CtYcdevFLQs5FsXY8ivzNJI9jF896Wap2LOzVEsr37GDMdjqAur+SkJ6r pJDhTkg== X-Google-Smtp-Source: AGHT+IHlcjSGqarS+g1bhBvZ+/5z18UzVO8XJTedPcuZrk2rXbJ2yPGb6Ke6AXkQXWyOH6X/CTjG1Q== X-Received: by 2002:a17:90b:4c8c:b0:2e2:c66f:5527 with SMTP id 98e67ed59e1d1-2e3ab8bcf91mr7405582a91.29.1729147389467; Wed, 16 Oct 2024 23:43:09 -0700 (PDT) Received: from Hyman-Dev-Euler.zelin.local ([85.208.104.155]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e3e08eba12sm1044372a91.29.2024.10.16.23.43.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Oct 2024 23:43:09 -0700 (PDT) From: yong.huang@smartx.com To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Richard Henderson , Paolo Bonzini , yong.huang@smartx.com Subject: [PATCH v4 1/6] accel/tcg/icount-common: Remove the reference to the unused header file Date: Thu, 17 Oct 2024 14:42:50 +0800 Message-Id: <5e33b423d0b8506e5cb33fff42b50aa301b7731b.1729146786.git.yong.huang@smartx.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1030; envelope-from=yong.huang@smartx.com; helo=mail-pj1-x1030.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hyman Huang Signed-off-by: Hyman Huang Reviewed-by: Fabiano Rosas --- accel/tcg/icount-common.c | 1 - 1 file changed, 1 deletion(-) diff --git a/accel/tcg/icount-common.c b/accel/tcg/icount-common.c index 8d3d3a7e9d..30bf8500dc 100644 --- a/accel/tcg/icount-common.c +++ b/accel/tcg/icount-common.c @@ -36,7 +36,6 @@ #include "sysemu/runstate.h" #include "hw/core/cpu.h" #include "sysemu/cpu-timers.h" -#include "sysemu/cpu-throttle.h" #include "sysemu/cpu-timers-internal.h" /* From patchwork Thu Oct 17 06:42:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong Huang X-Patchwork-Id: 13839500 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84449D2F7F3 for ; Thu, 17 Oct 2024 06:43:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t1KE8-0001gM-Im; Thu, 17 Oct 2024 02:43:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t1KE7-0001fd-14 for qemu-devel@nongnu.org; Thu, 17 Oct 2024 02:43:15 -0400 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t1KE5-0005Fk-IF for qemu-devel@nongnu.org; Thu, 17 Oct 2024 02:43:14 -0400 Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-7ea0ff74b15so389787a12.3 for ; Wed, 16 Oct 2024 23:43:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1729147392; x=1729752192; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CuqvH2NTqcFSEDwUKN/STKKD5/FUD3STbfz6kJBD0EE=; b=Z/yaVSZsSEzNV+K9BBqATAp7yrmMommD/+wshFHC8v8anl83KLMq0P8avKHiKaz52O Mh+83sJqXDHX3CV1sl35exVNmy9QoOEpAItEgcgWMAUcrcu9sOWRAt70kX5ltuB0QZnJ NgZHQKqnpjY9ddAru+7wIN0tzojniq0Fzpj8P1yp0i+T/wPkNGdmbmoBrfDVvz508z5L pqG+JSSuVcNnZNrYgmkt6nu71rFwFoDDhmCs4dT4DSrDidttf3yDGhneQOrahrcQv67j 0AxH90WY8LmVN7k3pxx7iG760bqqPR7HEtc9OvVq4niZ6hUQm3H6Fs4j2cPu3I90oyW4 jTcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729147392; x=1729752192; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CuqvH2NTqcFSEDwUKN/STKKD5/FUD3STbfz6kJBD0EE=; b=SxYmkW2EHjGsoxVwgaphXlqqPmQuMROh+hlrZMJMdyUtI7oec5fue42AnWztBYth4j xTxUEccRyLg5fLTig8NhQMjqFLCtPDVU3L8HCvddZfrKWzlcvEYE9VfccQhDXTUXaiGl JSzVIap/jw7zZW4tut9sZ5uUygWzmaad6HLGfjHNm9ox9LCyJXhz1Op4UNe3Dz9hKS0L AsiVUbWxvQndUVdIbmQya7EdUjghzCIH/y65Rkr1QJtAckL3IMa27o4ZsczceI2UBPLK cFAnAVXt9N0S6elY5vuFtOf7vjjWsOHKOLjNogBudkZ3l2Y4SxE/jmiQffvLpDqZlopH yWzw== X-Gm-Message-State: AOJu0YxnyUOqtdPMiuVDTV9/KNCIBf1wwisKYY8caTh2lMnV+2dTikAw gq7wqAmsJC7D8323eJxlBNLvPtBYSEfWnkGBbsc4FVWty/6+KhUCIWLDLj2l/dV0Idtve/XOaPD U4Ndirw== X-Google-Smtp-Source: AGHT+IGVJkp+f34DuAMjC6+QzWkaoK+6LCbLo4DlA8737VvkMJsmiR2vLkqLQXqDdKc+kMWC8dW/iQ== X-Received: by 2002:a05:6a21:3a43:b0:1d9:21cb:ecdb with SMTP id adf61e73a8af0-1d921cbed64mr1000367637.41.1729147391434; Wed, 16 Oct 2024 23:43:11 -0700 (PDT) Received: from Hyman-Dev-Euler.zelin.local ([85.208.104.155]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e3e08eba12sm1044372a91.29.2024.10.16.23.43.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Oct 2024 23:43:11 -0700 (PDT) From: yong.huang@smartx.com To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Richard Henderson , Paolo Bonzini , yong.huang@smartx.com Subject: [PATCH v4 2/6] migration: Stop CPU throttling conditionally Date: Thu, 17 Oct 2024 14:42:51 +0800 Message-Id: X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=yong.huang@smartx.com; helo=mail-pg1-x530.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hyman Huang Since CPU throttling only occurs when auto-converge is on, stop it conditionally. Signed-off-by: Hyman Huang Reviewed-by: Fabiano Rosas --- migration/migration.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 021faee2f3..37a200a177 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3289,7 +3289,9 @@ static MigIterateState migration_iteration_run(MigrationState *s) static void migration_iteration_finish(MigrationState *s) { /* If we enabled cpu throttling for auto-converge, turn it off. */ - cpu_throttle_stop(); + if (migrate_auto_converge()) { + cpu_throttle_stop(); + } bql_lock(); switch (s->state) { From patchwork Thu Oct 17 06:42:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong Huang X-Patchwork-Id: 13839504 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DABBCD2F7EF for ; Thu, 17 Oct 2024 06:44:43 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t1KEA-0001gg-8K; Thu, 17 Oct 2024 02:43:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t1KE9-0001gX-JP for qemu-devel@nongnu.org; Thu, 17 Oct 2024 02:43:17 -0400 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t1KE7-0005Gr-Sa for qemu-devel@nongnu.org; Thu, 17 Oct 2024 02:43:17 -0400 Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-2e2bb1efe78so406068a91.1 for ; Wed, 16 Oct 2024 23:43:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1729147394; x=1729752194; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kD9H2eZxfztUOY4sM1Cm1FWeUikKoaPzRcd0R4WpRJc=; b=g0pACeirIbirlYI1PNxKq1pe0GbrGUsWCldv0KTCYMdxphz9cErK6w7TbdusQ7JHQT UgldLzY6RTLkug4O0HqyLAfU3MaWSnx9qptGXae5FsmRJuNtbCrb2FUDTAiAiasYguUe R7ptOhWoikIgCVyX/LNT1AZab0mQl3pUVhOl/2FfezOUq3zbBqCnRwhmApTcqqC94Tbw Nfjkw3TrMfhZudJV3GDFZoBtfS99CcZ8OENneMwRiXGrieZDYF+giMjHew5f38zqOxWM h4oHYBmsjRog/LXfXsFEqi41r19HzEKV8sKiCNeuwB6LyIh2eI5LvYgb9oWLp1gh+cIA 64wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729147394; x=1729752194; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kD9H2eZxfztUOY4sM1Cm1FWeUikKoaPzRcd0R4WpRJc=; b=f8RcAZjvVfDvXks7L/g3p60ZHJPwDuF6MJfLD96GBOK9HapmvyzjorU32j9vr6v+U4 zRCxfH5H+cOvn+O6Ev9IFX26sFEWioDbPWB2+QalrGa1CIyLdA7Bvxh7khMkRaVy4kDs CXyj5HVekXDF77dyJlncX2F1cKKuvP9h85xtcf6dCKz/4k2oy6evDIglRCiDDaLcv5g+ U9NRRRx789z1TUzJn6R9C/8/XD7Yi68QhIKMWdAs8PTYhIpRNISez7NPPfLDnhG3oWgd gClSQd6gTi9757tR9g1Upd7emj7u/9dWDJr1q/mIsu26DQUTMLX7W2m9x1MQRAMdVfql a25g== X-Gm-Message-State: AOJu0YztNCmlCXKLynYtneSWwp4d17BAd14oTtEAhswAifqzcTDVW+U6 Uvubc5sz0NsVVm3VmOPlPUB7H2ghXS+i/S4dlkVMD2QgyQfWfgs00ngKqltHXdxk78dNZihJX7a mnX6CoQ== X-Google-Smtp-Source: AGHT+IHdozgOdudHQcrsqoXC2kB/nT3IoXDzAtZT8zKk5SPi/sKegswRILxhQjgKzXWdUOLIRUvMfg== X-Received: by 2002:a17:90b:3706:b0:2e2:ada8:2986 with SMTP id 98e67ed59e1d1-2e3dc29e5fdmr3433093a91.16.1729147393730; Wed, 16 Oct 2024 23:43:13 -0700 (PDT) Received: from Hyman-Dev-Euler.zelin.local ([85.208.104.155]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e3e08eba12sm1044372a91.29.2024.10.16.23.43.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Oct 2024 23:43:13 -0700 (PDT) From: yong.huang@smartx.com To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Richard Henderson , Paolo Bonzini , yong.huang@smartx.com Subject: [PATCH v4 3/6] migration: Move cpu-throttole.c from system to migration Date: Thu, 17 Oct 2024 14:42:52 +0800 Message-Id: X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=yong.huang@smartx.com; helo=mail-pj1-x102a.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hyman Huang Move cpu-throttle.c from system to migration since it's only used for migration; this makes us avoid exporting the util functions and variables in misc.h but export them in migration.h when implementing the periodic ramblock dirty sync feature in the upcoming commits. Since CPU throttle timers are only used in migration, move their registry to migration_object_init. Signed-off-by: Hyman Huang Reviewed-by: Fabiano Rosas --- {system => migration}/cpu-throttle.c | 2 +- {include/sysemu => migration}/cpu-throttle.h | 0 migration/meson.build | 1 + migration/migration.c | 5 ++++- migration/ram.c | 2 +- migration/trace-events | 3 +++ system/cpu-timers.c | 3 --- system/meson.build | 1 - system/trace-events | 3 --- 9 files changed, 10 insertions(+), 10 deletions(-) rename {system => migration}/cpu-throttle.c (99%) rename {include/sysemu => migration}/cpu-throttle.h (100%) diff --git a/system/cpu-throttle.c b/migration/cpu-throttle.c similarity index 99% rename from system/cpu-throttle.c rename to migration/cpu-throttle.c index 7632dc6143..fa47ee2e21 100644 --- a/system/cpu-throttle.c +++ b/migration/cpu-throttle.c @@ -27,7 +27,7 @@ #include "hw/core/cpu.h" #include "qemu/main-loop.h" #include "sysemu/cpus.h" -#include "sysemu/cpu-throttle.h" +#include "cpu-throttle.h" #include "trace.h" /* vcpu throttling controls */ diff --git a/include/sysemu/cpu-throttle.h b/migration/cpu-throttle.h similarity index 100% rename from include/sysemu/cpu-throttle.h rename to migration/cpu-throttle.h diff --git a/migration/meson.build b/migration/meson.build index 66d3de86f0..d53cf3417a 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -13,6 +13,7 @@ system_ss.add(files( 'block-dirty-bitmap.c', 'channel.c', 'channel-block.c', + 'cpu-throttle.c', 'dirtyrate.c', 'exec.c', 'fd.c', diff --git a/migration/migration.c b/migration/migration.c index 37a200a177..2e10ca77af 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -24,7 +24,7 @@ #include "socket.h" #include "sysemu/runstate.h" #include "sysemu/sysemu.h" -#include "sysemu/cpu-throttle.h" +#include "cpu-throttle.h" #include "rdma.h" #include "ram.h" #include "migration/global_state.h" @@ -263,6 +263,9 @@ void migration_object_init(void) ram_mig_init(); dirty_bitmap_mig_init(); + + /* Initialize cpu throttle timers */ + cpu_throttle_init(); } typedef struct { diff --git a/migration/ram.c b/migration/ram.c index 326ce7eb79..54d352b152 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -52,7 +52,7 @@ #include "exec/target_page.h" #include "qemu/rcu_queue.h" #include "migration/colo.h" -#include "sysemu/cpu-throttle.h" +#include "cpu-throttle.h" #include "savevm.h" #include "qemu/iov.h" #include "multifd.h" diff --git a/migration/trace-events b/migration/trace-events index c65902f042..9a19599804 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -378,3 +378,6 @@ migration_block_progression(unsigned percent) "Completed %u%%" # page_cache.c migration_pagecache_init(int64_t max_num_items) "Setting cache buckets to %" PRId64 migration_pagecache_insert(void) "Error allocating page" + +# cpu-throttle.c +cpu_throttle_set(int new_throttle_pct) "set guest CPU throttled by %d%%" diff --git a/system/cpu-timers.c b/system/cpu-timers.c index 0b31c9a1b6..856e502e34 100644 --- a/system/cpu-timers.c +++ b/system/cpu-timers.c @@ -35,7 +35,6 @@ #include "sysemu/runstate.h" #include "hw/core/cpu.h" #include "sysemu/cpu-timers.h" -#include "sysemu/cpu-throttle.h" #include "sysemu/cpu-timers-internal.h" /* clock and ticks */ @@ -272,6 +271,4 @@ void cpu_timers_init(void) seqlock_init(&timers_state.vm_clock_seqlock); qemu_spin_init(&timers_state.vm_clock_lock); vmstate_register(NULL, 0, &vmstate_timers, &timers_state); - - cpu_throttle_init(); } diff --git a/system/meson.build b/system/meson.build index a296270cb0..4952f4b2c7 100644 --- a/system/meson.build +++ b/system/meson.build @@ -10,7 +10,6 @@ system_ss.add(files( 'balloon.c', 'bootdevice.c', 'cpus.c', - 'cpu-throttle.c', 'cpu-timers.c', 'datadir.c', 'dirtylimit.c', diff --git a/system/trace-events b/system/trace-events index 074d001e90..2ed1d59b1f 100644 --- a/system/trace-events +++ b/system/trace-events @@ -44,6 +44,3 @@ dirtylimit_state_finalize(void) dirtylimit_throttle_pct(int cpu_index, uint64_t pct, int64_t time_us) "CPU[%d] throttle percent: %" PRIu64 ", throttle adjust time %"PRIi64 " us" dirtylimit_set_vcpu(int cpu_index, uint64_t quota) "CPU[%d] set dirty page rate limit %"PRIu64 dirtylimit_vcpu_execute(int cpu_index, int64_t sleep_time_us) "CPU[%d] sleep %"PRIi64 " us" - -# cpu-throttle.c -cpu_throttle_set(int new_throttle_pct) "set guest CPU throttled by %d%%" From patchwork Thu Oct 17 06:42:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong Huang X-Patchwork-Id: 13839503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 72053D2F7EF for ; Thu, 17 Oct 2024 06:44:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t1KED-0001hN-8P; Thu, 17 Oct 2024 02:43:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t1KEB-0001h2-Id for qemu-devel@nongnu.org; Thu, 17 Oct 2024 02:43:19 -0400 Received: from mail-pj1-x102e.google.com ([2607:f8b0:4864:20::102e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t1KEA-0005HX-0i for qemu-devel@nongnu.org; Thu, 17 Oct 2024 02:43:19 -0400 Received: by mail-pj1-x102e.google.com with SMTP id 98e67ed59e1d1-2e2bd0e2c4fso470675a91.3 for ; Wed, 16 Oct 2024 23:43:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1729147396; x=1729752196; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v+ajNe5sP2rJbGsXRQh/6mYHUXwdnX8x9rsRLQEatk8=; b=oaG9FReK0D0OTywrTSFMbmQlsuO3RbCzqgy1JpNscEvSKGx3OoFBKYK3CRbygaTV8W xsgMPi1cz2qcQWsFVbNmBYK/0SItlyUltH6ocAnnmsuRMqJCS9HKHZrk+IXiIE6sDSzO pMnoGKG3XjM/CuAnm0TwrZIGTBPfKASKOfunbNGx/pS92XlAEC2Y0ZIcaSnlLsjrwe30 d2YU8lfllw+efN+JwFMZr2VIjsXBrjRoO1bU3XFcuvTVMNUg+VtK29zOHZI1tvScqZdX mcs2fb/FhrTtNWqFTYM7Kjp+lXSxIPm4SnCFhsgCC2HryTXJx0QG6PkwjpUgze7Le0d/ DU6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729147396; x=1729752196; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v+ajNe5sP2rJbGsXRQh/6mYHUXwdnX8x9rsRLQEatk8=; b=Ct/5MxbZL1TATjLJOTkmodIoU6c+/87Bbt8PeGxLqTDBI2IbjfPjcyVBqk9ylGIvyy eeJBion45tA7Q7uymKxxCY8+wXOa3EG707Z5AneDMX7ewxdMdmTKOEVKKPbmisoalflj ijGkiQB7srsSib+PUtbdx3qNf8g45vDA7Rf7pvTCm6uwtexDiWkAN/ZfZhLFMFCftyNT YuI7B05eRB/jhSwpeN+fDs4wZAsC9/HzC7VmxetziRRZljTDl1NgtDs5KntaQDgXlFGw V+MW3lC0/lKTEm30NTlUIBx+wm/6ZilhmrJYnfaleIZjfTvnS0efUy3Mt7/La1U8ppEY EeLQ== X-Gm-Message-State: AOJu0YxoQKgBhW96fDhwvu6BPtZupxesZJfQrd+50uQbTCfkEZIa7BAg UN2Qkyg75mFx31yVuoYQBuhOhZSwN8PXRvdyJ/Vzx6p8wKJJEhlW2Ks1d1SZiWm6zEhtMmC/91Z PdcJ5Rw== X-Google-Smtp-Source: AGHT+IHM/NxZwKeGkGKk+QRfXw/mZD5dY+1mZCe5iJEGNEO6JUt9a3tY6wEushZpFJIxzytxqnOoVg== X-Received: by 2002:a17:90a:c708:b0:2e2:b64e:f501 with SMTP id 98e67ed59e1d1-2e2f0d7f1fbmr23998858a91.30.1729147395727; Wed, 16 Oct 2024 23:43:15 -0700 (PDT) Received: from Hyman-Dev-Euler.zelin.local ([85.208.104.155]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e3e08eba12sm1044372a91.29.2024.10.16.23.43.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Oct 2024 23:43:15 -0700 (PDT) From: yong.huang@smartx.com To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Richard Henderson , Paolo Bonzini , yong.huang@smartx.com Subject: [PATCH v4 4/6] migration: Remove "rs" parameter in migration_bitmap_sync_precopy Date: Thu, 17 Oct 2024 14:42:53 +0800 Message-Id: <283c335d61463bf477160da91b24da45cdaf3e43.1729146786.git.yong.huang@smartx.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102e; envelope-from=yong.huang@smartx.com; helo=mail-pj1-x102e.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hyman Huang The global static variable ram_state in fact is referred to by the "rs" parameter in migration_bitmap_sync_precopy. For ease of calling by the callees, use the global variable directly in migration_bitmap_sync_precopy and remove "rs" parameter. The migration_bitmap_sync_precopy will be exported in the next commit. Signed-off-by: Hyman Huang Reviewed-by: Peter Xu --- migration/ram.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 54d352b152..9b5b350405 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1088,9 +1088,10 @@ static void migration_bitmap_sync(RAMState *rs, bool last_stage) } } -static void migration_bitmap_sync_precopy(RAMState *rs, bool last_stage) +static void migration_bitmap_sync_precopy(bool last_stage) { Error *local_err = NULL; + assert(ram_state); /* * The current notifier usage is just an optimization to migration, so we @@ -1101,7 +1102,7 @@ static void migration_bitmap_sync_precopy(RAMState *rs, bool last_stage) local_err = NULL; } - migration_bitmap_sync(rs, last_stage); + migration_bitmap_sync(ram_state, last_stage); if (precopy_notify(PRECOPY_NOTIFY_AFTER_BITMAP_SYNC, &local_err)) { error_report_err(local_err); @@ -2782,7 +2783,7 @@ static bool ram_init_bitmaps(RAMState *rs, Error **errp) if (!ret) { goto out_unlock; } - migration_bitmap_sync_precopy(rs, false); + migration_bitmap_sync_precopy(false); } } out_unlock: @@ -3248,7 +3249,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) WITH_RCU_READ_LOCK_GUARD() { if (!migration_in_postcopy()) { - migration_bitmap_sync_precopy(rs, true); + migration_bitmap_sync_precopy(true); } ret = rdma_registration_start(f, RAM_CONTROL_FINISH); @@ -3330,7 +3331,7 @@ static void ram_state_pending_exact(void *opaque, uint64_t *must_precopy, if (!migration_in_postcopy()) { bql_lock(); WITH_RCU_READ_LOCK_GUARD() { - migration_bitmap_sync_precopy(rs, false); + migration_bitmap_sync_precopy(false); } bql_unlock(); } From patchwork Thu Oct 17 06:42:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong Huang X-Patchwork-Id: 13839502 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8B58DD2F7F2 for ; Thu, 17 Oct 2024 06:43:46 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t1KEF-0001iG-QC; Thu, 17 Oct 2024 02:43:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t1KED-0001hh-ST for qemu-devel@nongnu.org; Thu, 17 Oct 2024 02:43:21 -0400 Received: from mail-pg1-x536.google.com ([2607:f8b0:4864:20::536]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t1KEC-0005IQ-2d for qemu-devel@nongnu.org; Thu, 17 Oct 2024 02:43:21 -0400 Received: by mail-pg1-x536.google.com with SMTP id 41be03b00d2f7-7ea7e250c54so417096a12.0 for ; Wed, 16 Oct 2024 23:43:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1729147398; x=1729752198; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gkuUteniMACK6TVPruWS74ndE9dexfSbMnrEVOVSYNY=; b=26GoC+EOMnV3KCSGqLEOLfSJEJ5jm0FjVmVX0i2ONh1nQ+l2f7mP1Gv3f3BmgVwzYM +nYx0IiR9s5t3RTLlU62zC9vrmo1VUtGmyL6PiDKznnRqYufu1hQm/QbckjfXGTCzEfB t1CtC3d3FlYRJ5Lmg8ydIrddu/nWx/HnTxwh6Lw1tIonaV/827BO31wc2C188GB/T9j/ g68KwBsKd7/P6f7VY4ehDdw9fOsOTJcv8P0pd/Qh/5wv6j9Sk/3IVk/IRUGq1C0fAVGk Grqx34lJnuRGNpDLiT8hq5jfgE9GnQjsn5Z7ZMT2LhVtX/WZh33ppkEQsptGNEhm8iMT xEYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729147398; x=1729752198; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gkuUteniMACK6TVPruWS74ndE9dexfSbMnrEVOVSYNY=; b=XE3VPMT5JgS/4uq8U/DhJVwFZjJ7WoNVVwWdnB8FvApakxs4qKAJ4xuLwqwrw7meIM EiAmeKTqSg6aBLu3oKrD3jhAATIErwe2tAZH+ECLelQLyvU+VFRdFA6i1EXNmLya+TbN nJ30uoTQ5eyEyopSnMCOBrFnfIsFuPu6mMx9WipiLnc0izv5de3XLIMW7b8QQkpHJh0Q GfwmGU9pjUGWUfPWJUJa6py1RwX+ioCBw5SQIBcet45pIXtDdS9dF/EO7fW2ksjmKMjh 9up9GHaUj3N0mtgbnxQ/SKn52zVTMPUx3oyku6x1+r/gW+RF8YTaBqmPtDtxN6xp6osM xNcg== X-Gm-Message-State: AOJu0YyT2xdEe6sxoeir4fbU5jE7jCDg97NIK3VLgkOXyM9uaIa4KDHM r5HMA/oAwuH0Y/2NkU7iv3Z3JrxRx8r0Jao0HQBqHGHF1DfZE/uF/e01mW10YVdTAhoEjSf8Gnt oiFUQMQ== X-Google-Smtp-Source: AGHT+IGpjBhb0YHl1JDz/HyUUYop39j+CjmYDawSdpfuLNuqiZBUCjMSwamDkA0u51mn6rCTnCNtmw== X-Received: by 2002:a05:6a20:438d:b0:1d6:2378:cffd with SMTP id adf61e73a8af0-1d8c93f5dbemr27552546637.0.1729147397880; Wed, 16 Oct 2024 23:43:17 -0700 (PDT) Received: from Hyman-Dev-Euler.zelin.local ([85.208.104.155]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e3e08eba12sm1044372a91.29.2024.10.16.23.43.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Oct 2024 23:43:17 -0700 (PDT) From: yong.huang@smartx.com To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Richard Henderson , Paolo Bonzini , yong.huang@smartx.com Subject: [PATCH v4 5/6] migration: Support periodic RAMBlock dirty bitmap sync Date: Thu, 17 Oct 2024 14:42:54 +0800 Message-Id: X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::536; envelope-from=yong.huang@smartx.com; helo=mail-pg1-x536.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hyman Huang When VM is configured with huge memory, the current throttle logic doesn't look like to scale, because migration_trigger_throttle() is only called for each iteration, so it won't be invoked for a long time if one iteration can take a long time. The periodic dirty sync aims to fix the above issue by synchronizing the ramblock from remote dirty bitmap and, when necessary, triggering the CPU throttle multiple times during a long iteration. This is a trade-off between synchronization overhead and CPU throttle impact. Signed-off-by: Hyman Huang Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu Acked-by: Hyman Huang --- migration/cpu-throttle.c | 65 +++++++++++++++++++++++++++++++++++++++- migration/cpu-throttle.h | 14 +++++++++ migration/migration.c | 14 +++++++-- migration/migration.h | 1 + migration/ram.c | 2 +- migration/trace-events | 1 + 6 files changed, 93 insertions(+), 4 deletions(-) diff --git a/migration/cpu-throttle.c b/migration/cpu-throttle.c index fa47ee2e21..342681cdd4 100644 --- a/migration/cpu-throttle.c +++ b/migration/cpu-throttle.c @@ -28,16 +28,22 @@ #include "qemu/main-loop.h" #include "sysemu/cpus.h" #include "cpu-throttle.h" +#include "migration.h" +#include "migration-stats.h" #include "trace.h" /* vcpu throttling controls */ -static QEMUTimer *throttle_timer; +static QEMUTimer *throttle_timer, *throttle_dirty_sync_timer; static unsigned int throttle_percentage; +static bool throttle_dirty_sync_timer_active; #define CPU_THROTTLE_PCT_MIN 1 #define CPU_THROTTLE_PCT_MAX 99 #define CPU_THROTTLE_TIMESLICE_NS 10000000 +/* Making sure RAMBlock dirty bitmap is synchronized every five seconds */ +#define CPU_THROTTLE_DIRTY_SYNC_TIMESLICE_MS 5000 + static void cpu_throttle_thread(CPUState *cpu, run_on_cpu_data opaque) { double pct; @@ -112,6 +118,7 @@ void cpu_throttle_set(int new_throttle_pct) void cpu_throttle_stop(void) { qatomic_set(&throttle_percentage, 0); + cpu_throttle_dirty_sync_timer(false); } bool cpu_throttle_active(void) @@ -124,8 +131,64 @@ int cpu_throttle_get_percentage(void) return qatomic_read(&throttle_percentage); } +void cpu_throttle_dirty_sync_timer_tick(void *opaque) +{ + static uint64_t prev_sync_cnt; + uint64_t sync_cnt = stat64_get(&mig_stats.dirty_sync_count); + + /* + * The first iteration copies all memory anyhow and has no + * effect on guest performance, therefore omit it to avoid + * paying extra for the sync penalty. + */ + if (sync_cnt <= 1) { + goto end; + } + + if (sync_cnt == prev_sync_cnt) { + trace_cpu_throttle_dirty_sync(); + WITH_RCU_READ_LOCK_GUARD() { + migration_bitmap_sync_precopy(false); + } + } + +end: + prev_sync_cnt = stat64_get(&mig_stats.dirty_sync_count); + + timer_mod(throttle_dirty_sync_timer, + qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL_RT) + + CPU_THROTTLE_DIRTY_SYNC_TIMESLICE_MS); +} + +static bool cpu_throttle_dirty_sync_active(void) +{ + return qatomic_read(&throttle_dirty_sync_timer_active); +} + +void cpu_throttle_dirty_sync_timer(bool enable) +{ + assert(throttle_dirty_sync_timer); + + if (enable) { + if (!cpu_throttle_dirty_sync_active()) { + timer_mod(throttle_dirty_sync_timer, + qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL_RT) + + CPU_THROTTLE_DIRTY_SYNC_TIMESLICE_MS); + qatomic_set(&throttle_dirty_sync_timer_active, 1); + } + } else { + if (cpu_throttle_dirty_sync_active()) { + timer_del(throttle_dirty_sync_timer); + qatomic_set(&throttle_dirty_sync_timer_active, 0); + } + } +} + void cpu_throttle_init(void) { throttle_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL_RT, cpu_throttle_timer_tick, NULL); + throttle_dirty_sync_timer = + timer_new_ms(QEMU_CLOCK_VIRTUAL_RT, + cpu_throttle_dirty_sync_timer_tick, NULL); } diff --git a/migration/cpu-throttle.h b/migration/cpu-throttle.h index d65bdef6d0..420702b8d3 100644 --- a/migration/cpu-throttle.h +++ b/migration/cpu-throttle.h @@ -65,4 +65,18 @@ bool cpu_throttle_active(void); */ int cpu_throttle_get_percentage(void); +/** + * cpu_throttle_dirty_sync_timer_tick: + * + * Dirty sync timer hook. + */ +void cpu_throttle_dirty_sync_timer_tick(void *opaque); + +/** + * cpu_throttle_dirty_sync_timer: + * + * Start or stop the dirty sync timer. + */ +void cpu_throttle_dirty_sync_timer(bool enable); + #endif /* SYSEMU_CPU_THROTTLE_H */ diff --git a/migration/migration.c b/migration/migration.c index 2e10ca77af..f673e30069 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3291,12 +3291,17 @@ static MigIterateState migration_iteration_run(MigrationState *s) static void migration_iteration_finish(MigrationState *s) { - /* If we enabled cpu throttling for auto-converge, turn it off. */ + bql_lock(); + + /* + * If we enabled cpu throttling for auto-converge, turn it off. + * Stopping CPU throttle should be serialized by BQL to avoid + * racing for the throttle_dirty_sync_timer. + */ if (migrate_auto_converge()) { cpu_throttle_stop(); } - bql_lock(); switch (s->state) { case MIGRATION_STATUS_COMPLETED: runstate_set(RUN_STATE_POSTMIGRATE); @@ -3513,6 +3518,11 @@ static void *migration_thread(void *opaque) qemu_savevm_send_colo_enable(s->to_dst_file); } + if (migrate_auto_converge()) { + /* Start RAMBlock dirty bitmap sync timer */ + cpu_throttle_dirty_sync_timer(true); + } + bql_lock(); ret = qemu_savevm_state_setup(s->to_dst_file, &local_err); bql_unlock(); diff --git a/migration/migration.h b/migration/migration.h index 38aa1402d5..fbd0d19092 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -537,4 +537,5 @@ int migration_rp_wait(MigrationState *s); */ void migration_rp_kick(MigrationState *s); +void migration_bitmap_sync_precopy(bool last_stage); #endif diff --git a/migration/ram.c b/migration/ram.c index 9b5b350405..d284f63854 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1088,7 +1088,7 @@ static void migration_bitmap_sync(RAMState *rs, bool last_stage) } } -static void migration_bitmap_sync_precopy(bool last_stage) +void migration_bitmap_sync_precopy(bool last_stage) { Error *local_err = NULL; assert(ram_state); diff --git a/migration/trace-events b/migration/trace-events index 9a19599804..0638183056 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -381,3 +381,4 @@ migration_pagecache_insert(void) "Error allocating page" # cpu-throttle.c cpu_throttle_set(int new_throttle_pct) "set guest CPU throttled by %d%%" +cpu_throttle_dirty_sync(void) "" From patchwork Thu Oct 17 06:42:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong Huang X-Patchwork-Id: 13839498 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CDA8DD2F7EF for ; Thu, 17 Oct 2024 06:43:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t1KEG-0001iI-Fa; Thu, 17 Oct 2024 02:43:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t1KEF-0001ht-Cs for qemu-devel@nongnu.org; Thu, 17 Oct 2024 02:43:23 -0400 Received: from mail-pj1-x1032.google.com ([2607:f8b0:4864:20::1032]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t1KED-0005Jr-Tr for qemu-devel@nongnu.org; Thu, 17 Oct 2024 02:43:23 -0400 Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-2e2cc469c62so432760a91.2 for ; Wed, 16 Oct 2024 23:43:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1729147400; x=1729752200; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yMnx/3A6xdIIvLj6aaYgJOG0jaU0Jmn5Vj6VTmTpdGs=; b=aI50HVv4AdtwNXttA48piL7jfY5t3jVZL+kmw9UY3534Zj8pRgZQwM7w42ME22blQ5 jN6l92AIySgcWW5o5jK9HK/UpxqP7KLpHKwbczOEyQLB9tMtqrZNUQTz0yiA5u0/x+1P vqmC/8ub8C9lwxXDr9a6iHQL21dA3/8gdoWzVPtEvKCwDcfdqNKyZZmAb7NC18/WbMj4 G7u5zt78cPvH5a+OVluww7R9vt7zwzLZY7JojqCD63auL5LNNJhY9JEK0L0PouLxXDML 7ryJJpTjFNEmnWxT9e/aksZuopQQuO8y+3fH13/kj8MutLOaddfJ/++6KUO2iW9/c3j3 gkKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729147400; x=1729752200; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yMnx/3A6xdIIvLj6aaYgJOG0jaU0Jmn5Vj6VTmTpdGs=; b=Po1zzMxOVmBAft6d9Xo7HH4JrxJIpJdJXGzWmVKzlrpZ+HkXlYxVYP9hNt+lCV3G5+ pZ4q+Zkh8RBFN9CthZceqT8IBbpbqWygrgnqVZgvmJBLRuvm47Wtkkw8aPz1p3QMMXG7 1he68Rq0N+zdzaUx+soT8zxsETM66CIWSL3aXrg89I/swsksN8y0ekZgJyvDo1I569Yg GJak3XL9gTAcrdJfLoVE08/BaVzcdHayC5uCCNp5V6CfIPCxIwqAC6ThtaVpbmpEtiD6 wq09U5zZAhs33Mk8jpm87Mi3g1fE0shcf7pOnZzIke0nrjwo8hG7Z9DRAClYvz3hCWt0 dlWA== X-Gm-Message-State: AOJu0YwSkx8ifvDhAjHA837SOa4TnYMdOtEZoCzuYFE4NeADTww4x4y/ 2jrY2naP93b2AAYov2JyLo3cfwNyvAWDEBZU0qDgRd8fYxI4WPwYmLtwmv5ozncMnfRWQAYt/Ju bC+wQ1g== X-Google-Smtp-Source: AGHT+IGWcofpFCaTqLPfkIlm95VBexhfdRfXYNh+PzP11VZ1wqGjquG/JE0np6sJ2+Ku4YY6POBEIg== X-Received: by 2002:a17:90a:8a92:b0:2e2:e660:96d5 with SMTP id 98e67ed59e1d1-2e3152f4b3bmr22148378a91.24.1729147399890; Wed, 16 Oct 2024 23:43:19 -0700 (PDT) Received: from Hyman-Dev-Euler.zelin.local ([85.208.104.155]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e3e08eba12sm1044372a91.29.2024.10.16.23.43.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Oct 2024 23:43:19 -0700 (PDT) From: yong.huang@smartx.com To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Richard Henderson , Paolo Bonzini , yong.huang@smartx.com Subject: [PATCH v4 6/6] tests/migration: Add case for periodic ramblock dirty sync Date: Thu, 17 Oct 2024 14:42:55 +0800 Message-Id: X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1032; envelope-from=yong.huang@smartx.com; helo=mail-pj1-x1032.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hyman Huang Signed-off-by: Hyman Huang Reviewed-by: Peter Xu Reviewed-by: Fabiano Rosas --- tests/qtest/migration-test.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 95e45b5029..e6a2803e71 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2791,6 +2791,8 @@ static void test_migrate_auto_converge(void) * so we need to decrease a bandwidth. */ const int64_t init_pct = 5, inc_pct = 25, max_pct = 95; + uint64_t prev_dirty_sync_cnt, dirty_sync_cnt; + int max_try_count, hit = 0; if (test_migrate_start(&from, &to, uri, &args)) { return; @@ -2827,6 +2829,36 @@ static void test_migrate_auto_converge(void) } while (true); /* The first percentage of throttling should be at least init_pct */ g_assert_cmpint(percentage, >=, init_pct); + + /* + * End the loop when the dirty sync count greater than 1. + */ + while ((dirty_sync_cnt = get_migration_pass(from)) < 2) { + usleep(1000 * 1000); + } + + prev_dirty_sync_cnt = dirty_sync_cnt; + + /* + * The RAMBlock dirty sync count must changes in 5 seconds, here we set + * the timeout to 10 seconds to ensure it changes. + * + * Note that migrate_ensure_non_converge set the max-bandwidth to 3MB/s, + * while the qtest mem is >= 100MB, one iteration takes at least 33s (100/3) + * to complete; this ensures that the RAMBlock dirty sync occurs. + */ + max_try_count = 10; + while (--max_try_count) { + dirty_sync_cnt = get_migration_pass(from); + if (dirty_sync_cnt != prev_dirty_sync_cnt) { + hit = 1; + break; + } + prev_dirty_sync_cnt = dirty_sync_cnt; + sleep(1); + } + g_assert_cmpint(hit, ==, 1); + /* Now, when we tested that throttling works, let it converge */ migrate_ensure_converge(from);