From patchwork Wed Sep 4 12:43:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790769 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C39DCD3431 for ; Wed, 4 Sep 2024 12:45:39 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpN7-0003BF-09; Wed, 04 Sep 2024 08:44:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpN6-00038M-7L for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:28 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpN4-0005Ow-OZ for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:27 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 064CF1F7BE; Wed, 4 Sep 2024 12:44:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453863; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bhJR1jldO74JnbozaOH6vB5Eq5RsqFjmHTbdjHPW+tw=; b=oBZa+FG6Ww+Z1L7ZyJP/BiVARAHtePMXnro3YIl8S31EH5zww4yNHlbmPsQ6PV537xf5bQ PVs7aeR/Z+ctRPiqzZZSsrzCzsjF1bhw+/QVQz11w0A4mSZmH3fmHnSXgA2OPhEhuqyuFK +V10bIFVnPyf2LL8icWn8/u8H55uouU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453863; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bhJR1jldO74JnbozaOH6vB5Eq5RsqFjmHTbdjHPW+tw=; b=hSuVvFc7nYEzNUUk8AGaGmvmWChM4pUENM98dErfYaSz3QpDrBXiAnhmj9z3537EMVEK3b 6sZNfrvNXiNev4CA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453863; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bhJR1jldO74JnbozaOH6vB5Eq5RsqFjmHTbdjHPW+tw=; b=oBZa+FG6Ww+Z1L7ZyJP/BiVARAHtePMXnro3YIl8S31EH5zww4yNHlbmPsQ6PV537xf5bQ PVs7aeR/Z+ctRPiqzZZSsrzCzsjF1bhw+/QVQz11w0A4mSZmH3fmHnSXgA2OPhEhuqyuFK +V10bIFVnPyf2LL8icWn8/u8H55uouU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453863; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bhJR1jldO74JnbozaOH6vB5Eq5RsqFjmHTbdjHPW+tw=; b=hSuVvFc7nYEzNUUk8AGaGmvmWChM4pUENM98dErfYaSz3QpDrBXiAnhmj9z3537EMVEK3b 6sZNfrvNXiNev4CA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 9B231139D2; Wed, 4 Sep 2024 12:44:21 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QBIEGCVW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:21 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Steve Sistare Subject: [PULL 01/34] migration: delete unused parameter mis Date: Wed, 4 Sep 2024 09:43:44 -0300 Message-Id: <20240904124417.14565-2-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-1.93 / 50.00]; BAYES_HAM(-2.15)[95.89%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.19)[-0.927]; MIME_GOOD(-0.10)[text/plain]; FUZZY_BLOCKED(0.00)[rspamd.com]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:mid] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Steve Sistare Signed-off-by: Steve Sistare Reviewed-by: Fabiano Rosas Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/savevm.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/migration/savevm.c b/migration/savevm.c index 6bb404b9c8..d500eae979 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2578,8 +2578,7 @@ static bool check_section_footer(QEMUFile *f, SaveStateEntry *se) } static int -qemu_loadvm_section_start_full(QEMUFile *f, MigrationIncomingState *mis, - uint8_t type) +qemu_loadvm_section_start_full(QEMUFile *f, uint8_t type) { bool trace_downtime = (type == QEMU_VM_SECTION_FULL); uint32_t instance_id, version_id, section_id; @@ -2657,8 +2656,7 @@ qemu_loadvm_section_start_full(QEMUFile *f, MigrationIncomingState *mis, } static int -qemu_loadvm_section_part_end(QEMUFile *f, MigrationIncomingState *mis, - uint8_t type) +qemu_loadvm_section_part_end(QEMUFile *f, uint8_t type) { bool trace_downtime = (type == QEMU_VM_SECTION_END); int64_t start_ts, end_ts; @@ -2893,14 +2891,14 @@ retry: switch (section_type) { case QEMU_VM_SECTION_START: case QEMU_VM_SECTION_FULL: - ret = qemu_loadvm_section_start_full(f, mis, section_type); + ret = qemu_loadvm_section_start_full(f, section_type); if (ret < 0) { goto out; } break; case QEMU_VM_SECTION_PART: case QEMU_VM_SECTION_END: - ret = qemu_loadvm_section_part_end(f, mis, section_type); + ret = qemu_loadvm_section_part_end(f, section_type); if (ret < 0) { goto out; } From patchwork Wed Sep 4 12:43:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DC92CCD37B4 for ; Wed, 4 Sep 2024 12:48:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpN8-0003He-JQ; Wed, 04 Sep 2024 08:44:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpN6-0003Ad-Qe for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:28 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpN4-0005PR-Og for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:28 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 99F062191F; Wed, 4 Sep 2024 12:44:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453864; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YcVSdsvCvRjLleMdDu1UVJDKr274FIyU0pgazixF2Xw=; b=LWDRJmRleuhWhUOYk+10RoCzeEUPj3TTUH2Vx0FMqay8FDzw4VbT61Z56DXwdyAupzmu9W MXieGxrcFJgtFQaDG71107gjDzb5TOK7789y6vCnbEosVO80ZG5wrHFzJSWEVbslxmlb+T gVfm6grEpcMu5lOOxf+sYUMh3aWzlPo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453864; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YcVSdsvCvRjLleMdDu1UVJDKr274FIyU0pgazixF2Xw=; b=occpXvqyGYZATnkNrquhI6C9ZHcNTEIy5Ns1kCTeRJYrQLfS98hOXEzwwu4xVlVTVXDLd1 MEJMdqW1DnaYt8Bg== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=LWDRJmRl; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=occpXvqy DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453864; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YcVSdsvCvRjLleMdDu1UVJDKr274FIyU0pgazixF2Xw=; b=LWDRJmRleuhWhUOYk+10RoCzeEUPj3TTUH2Vx0FMqay8FDzw4VbT61Z56DXwdyAupzmu9W MXieGxrcFJgtFQaDG71107gjDzb5TOK7789y6vCnbEosVO80ZG5wrHFzJSWEVbslxmlb+T gVfm6grEpcMu5lOOxf+sYUMh3aWzlPo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453864; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YcVSdsvCvRjLleMdDu1UVJDKr274FIyU0pgazixF2Xw=; b=occpXvqyGYZATnkNrquhI6C9ZHcNTEIy5Ns1kCTeRJYrQLfS98hOXEzwwu4xVlVTVXDLd1 MEJMdqW1DnaYt8Bg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 70A63139D2; Wed, 4 Sep 2024 12:44:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QAkDDidW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:23 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 02/34] tests/qtest/migration: Remove vmstate-static-checker test Date: Wed, 4 Sep 2024 09:43:45 -0300 Message-Id: <20240904124417.14565-3-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 99F062191F X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:mid,suse.de:email]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org I fumbled one of my last pull requests when fixing in-tree an issue with commit 87d67fadb9 ("monitor: Stop removing non-duplicated fds"). Basically mixed-up my `git add -p` and `git checkout -p` and committed a piece of test infra that has not been reviewed yet. This has not caused any bad symptoms because the test is not enabled by default anywhere: make check doesn't use two qemu binaries and the CI doesn't have PYTHON set for the compat tests. Besides, the test works fine anyway, it would not break anything. Remove this because it was never intended to be merged. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- tests/qtest/libqtest.c | 17 +++----- tests/qtest/libqtest.h | 2 - tests/qtest/migration-test.c | 82 ------------------------------------ 3 files changed, 6 insertions(+), 95 deletions(-) diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c index 1326e34291..9d07de1fbd 100644 --- a/tests/qtest/libqtest.c +++ b/tests/qtest/libqtest.c @@ -514,7 +514,12 @@ static QTestState *qtest_init_internal(const char *qemu_bin, kill(s->qemu_pid, SIGSTOP); } #endif - return s; + + /* ask endianness of the target */ + + s->big_endian = qtest_query_target_endianness(s); + + return s; } QTestState *qtest_init_without_qmp_handshake(const char *extra_args) @@ -522,21 +527,11 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args) return qtest_init_internal(qtest_qemu_binary(NULL), extra_args); } -QTestState *qtest_init_with_env_no_handshake(const char *var, - const char *extra_args) -{ - return qtest_init_internal(qtest_qemu_binary(var), extra_args); -} - QTestState *qtest_init_with_env(const char *var, const char *extra_args) { QTestState *s = qtest_init_internal(qtest_qemu_binary(var), extra_args); QDict *greeting; - /* ask endianness of the target */ - - s->big_endian = qtest_query_target_endianness(s); - /* Read the QMP greeting and then do the handshake */ greeting = qtest_qmp_receive(s); qobject_unref(greeting); diff --git a/tests/qtest/libqtest.h b/tests/qtest/libqtest.h index c261b7e0b3..beb96b18eb 100644 --- a/tests/qtest/libqtest.h +++ b/tests/qtest/libqtest.h @@ -68,8 +68,6 @@ QTestState *qtest_init(const char *extra_args); */ QTestState *qtest_init_with_env(const char *var, const char *extra_args); -QTestState *qtest_init_with_env_no_handshake(const char *var, - const char *extra_args); /** * qtest_init_without_qmp_handshake: * @extra_args: other arguments to pass to QEMU. CAUTION: these diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 6c06100d91..334b63cbaa 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -64,7 +64,6 @@ static QTestMigrationState dst_state; #define DIRTYLIMIT_TOLERANCE_RANGE 25 /* MB/s */ #define ANALYZE_SCRIPT "scripts/analyze-migration.py" -#define VMSTATE_CHECKER_SCRIPT "scripts/vmstate-static-checker.py" #define QEMU_VM_FILE_MAGIC 0x5145564d #define FILE_TEST_FILENAME "migfile" @@ -1692,85 +1691,6 @@ static void test_analyze_script(void) test_migrate_end(from, to, false); cleanup("migfile"); } - -static void test_vmstate_checker_script(void) -{ - g_autofree gchar *cmd_src = NULL; - g_autofree gchar *cmd_dst = NULL; - g_autofree gchar *vmstate_src = NULL; - g_autofree gchar *vmstate_dst = NULL; - const char *machine_alias, *machine_opts = ""; - g_autofree char *machine = NULL; - const char *arch = qtest_get_arch(); - int pid, wstatus; - const char *python = g_getenv("PYTHON"); - - if (!getenv(QEMU_ENV_SRC) && !getenv(QEMU_ENV_DST)) { - g_test_skip("Test needs two different QEMU versions"); - return; - } - - if (!python) { - g_test_skip("PYTHON variable not set"); - return; - } - - if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) { - if (g_str_equal(arch, "i386")) { - machine_alias = "pc"; - } else { - machine_alias = "q35"; - } - } else if (g_str_equal(arch, "s390x")) { - machine_alias = "s390-ccw-virtio"; - } else if (strcmp(arch, "ppc64") == 0) { - machine_alias = "pseries"; - } else if (strcmp(arch, "aarch64") == 0) { - machine_alias = "virt"; - } else { - g_assert_not_reached(); - } - - if (!qtest_has_machine(machine_alias)) { - g_autofree char *msg = g_strdup_printf("machine %s not supported", machine_alias); - g_test_skip(msg); - return; - } - - machine = resolve_machine_version(machine_alias, QEMU_ENV_SRC, - QEMU_ENV_DST); - - vmstate_src = g_strdup_printf("%s/vmstate-src", tmpfs); - vmstate_dst = g_strdup_printf("%s/vmstate-dst", tmpfs); - - cmd_dst = g_strdup_printf("-machine %s,%s -dump-vmstate %s", - machine, machine_opts, vmstate_dst); - cmd_src = g_strdup_printf("-machine %s,%s -dump-vmstate %s", - machine, machine_opts, vmstate_src); - - qtest_init_with_env_no_handshake(QEMU_ENV_SRC, cmd_src); - qtest_init_with_env_no_handshake(QEMU_ENV_DST, cmd_dst); - - pid = fork(); - if (!pid) { - close(1); - open("/dev/null", O_WRONLY); - execl(python, python, VMSTATE_CHECKER_SCRIPT, - "-s", vmstate_src, - "-d", vmstate_dst, - NULL); - g_assert_not_reached(); - } - - g_assert(waitpid(pid, &wstatus, 0) == pid); - if (!WIFEXITED(wstatus) || WEXITSTATUS(wstatus) != 0) { - g_test_message("Failed to run vmstate-static-checker.py"); - g_test_fail(); - } - - cleanup("vmstate-src"); - cleanup("vmstate-dst"); -} #endif static void test_precopy_common(MigrateCommon *args) @@ -3823,8 +3743,6 @@ int main(int argc, char **argv) migration_test_add("/migration/bad_dest", test_baddest); #ifndef _WIN32 migration_test_add("/migration/analyze-script", test_analyze_script); - migration_test_add("/migration/vmstate-checker-script", - test_vmstate_checker_script); #endif if (is_x86) { From patchwork Wed Sep 4 12:43:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790783 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EE03DCD3431 for ; Wed, 4 Sep 2024 12:47:39 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNA-0003OI-7y; Wed, 04 Sep 2024 08:44:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpN8-0003Ha-Hg for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:30 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpN6-0005QW-Rn for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:30 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6DFE6219FF; Wed, 4 Sep 2024 12:44:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453867; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h8BaQiVelmckmlwaMiq3JJV9tWP+l6Wiavo1PrlqEfw=; b=TXMwhzjc6XjAnKmQWevCL9x7gsVm2nJkZ1mPlNnuL/Bb9LxhPkfpZuMU65r0gTUyZ2Y0/J CASTTx1emWxVelEFKYTM578Nknr3Gf5Ila9et4HWYATmZI3zIaXAUDKRrUJGEemJ7iYkPr 1OMACx+ai6KHXNgvZxZSX31aMiAICwI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453867; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h8BaQiVelmckmlwaMiq3JJV9tWP+l6Wiavo1PrlqEfw=; b=w9HCwqbdpgXzirw/4eef7ZeqXWDvkeyTr+YhJ16IfQje4C1sORCwwlnK7PXAluEii7A2yu 1I74HcJlADDIrhDQ== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=sHloRYph; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="slt3SS/x" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453866; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h8BaQiVelmckmlwaMiq3JJV9tWP+l6Wiavo1PrlqEfw=; b=sHloRYphbcPcSKqiucih/yLSUL36w/BhSkj2ll+Drz7CYdY+4s17bctErs09S4Y1g0sGnr h/payUctl4j0P6CQF99VFJVa8DK8q0qrvXPFpOMBRkvBkyjWV3UAV7IxvVHSamU8xGeAa1 V9N8SrRPC9me3WSqvbFybnxgIazJH7k= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453866; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h8BaQiVelmckmlwaMiq3JJV9tWP+l6Wiavo1PrlqEfw=; b=slt3SS/xGT0eN2SRZjmTlV1Id1gj9yhgikuZce9b9GxLfQl9OB50J6XXm+UzW0MkGzbRlw yikVVqlXrEycq4BQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 10AF0139D2; Wed, 4 Sep 2024 12:44:24 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id qL0SMihW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:24 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Peter Maydell Subject: [PULL 03/34] tests/qtest/migration-test: Fix bootfile cleanup handling Date: Wed, 4 Sep 2024 09:43:46 -0300 Message-Id: <20240904124417.14565-4-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6DFE6219FF X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:mid,suse.de:email,linaro.org:email]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Maydell If you invoke the migration-test binary in such a way that it doesn't run any tests, then we never call bootfile_create(), and at the end of main() bootfile_delete() will try to unlink(NULL), which is not valid. This can happen if for instance you tell the test binary to run a subset of tests that turns out to be empty, like this: (cd build/asan && QTEST_QEMU_BINARY=./qemu-system-x86_64 ./tests/qtest/migration-test --tap -k -p bang) # random seed: R02S6501b289ff8ced4231ba452c3a87bc6f # Skipping test: userfaultfd not available 1..0 ../../tests/qtest/migration-test.c:182:12: runtime error: null pointer passed as argument 1, which is declared to never be null /usr/include/unistd.h:858:48: note: nonnull attribute specified here Handle this by making bootfile_delete() not needing to do anything because bootfile_create() was never called. Signed-off-by: Peter Maydell Reviewed-by: Fabiano Rosas [fixed conflict with aee07f2563] Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 334b63cbaa..37ef99c980 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -145,6 +145,9 @@ static char *bootpath; static void bootfile_delete(void) { + if (!bootpath) { + return; + } unlink(bootpath); g_free(bootpath); bootpath = NULL; @@ -156,10 +159,7 @@ static void bootfile_create(char *dir, bool suspend_me) unsigned char *content; size_t len; - if (bootpath) { - bootfile_delete(); - } - + bootfile_delete(); bootpath = g_strdup_printf("%s/bootsect", dir); if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) { /* the assembled x86 boot sector should be exactly one sector large */ From patchwork Wed Sep 4 12:43:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4EAD3CD37B4 for ; Wed, 4 Sep 2024 12:46:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNB-0003S7-86; Wed, 04 Sep 2024 08:44:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpN9-0003LM-Dk for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:31 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpN7-0005Qs-Lj for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:31 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 48495219D2; Wed, 4 Sep 2024 12:44:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453868; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mXVgqh4miadmpincum38z66yhal5qVKzuhQP668uObs=; b=Xx76rvF3WSt/BYG5eikgXrTUYHjmqAz4F5WeZW96Ty07F/GmnCjR53fKmsGcujqRJe8Xfq wiqZOGvlA5e8evz80hrEBOBKy4XWiAdos3fZlfTv5ttaFXqAyXlNmwU3/cTaSArjzy6SLu RUD2XUg0oNkkHSYrUA0UHt/cmZ0ZeOg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453868; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mXVgqh4miadmpincum38z66yhal5qVKzuhQP668uObs=; b=f8Mu4F4AhYTQOncVOYZzyz6OapkzgsXs4b3ZK11rnKAVKHdZlWzkQQ8FakSgIi+zZvE216 91c/D1P+WnNA7+Cw== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453868; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mXVgqh4miadmpincum38z66yhal5qVKzuhQP668uObs=; b=Xx76rvF3WSt/BYG5eikgXrTUYHjmqAz4F5WeZW96Ty07F/GmnCjR53fKmsGcujqRJe8Xfq wiqZOGvlA5e8evz80hrEBOBKy4XWiAdos3fZlfTv5ttaFXqAyXlNmwU3/cTaSArjzy6SLu RUD2XUg0oNkkHSYrUA0UHt/cmZ0ZeOg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453868; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mXVgqh4miadmpincum38z66yhal5qVKzuhQP668uObs=; b=f8Mu4F4AhYTQOncVOYZzyz6OapkzgsXs4b3ZK11rnKAVKHdZlWzkQQ8FakSgIi+zZvE216 91c/D1P+WnNA7+Cw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DA47213A5F; Wed, 4 Sep 2024 12:44:26 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id wJzBJypW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:26 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Peter Maydell Subject: [PULL 04/34] tests/qtest/migration-test: Don't leak resp in multifd_mapped_ram_fdset_end() Date: Wed, 4 Sep 2024 09:43:47 -0300 Message-Id: <20240904124417.14565-5-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.79 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.19)[-0.927]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[linaro.org:email,suse.de:mid,suse.de:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Maydell In multifd_mapped_ram_fdset_end() we call qtest_qmp() but forgot to unref the response QDict we get back, which means it is leaked: Indirect leak of 4120 byte(s) in 1 object(s) allocated from: #0 0x55c0c095d318 in __interceptor_calloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/tests/qtest/migration-test+0x22f318) (BuildI d: 07f667506452d6c467dbc06fd95191966d3e91b4) #1 0x7f186f939c50 in g_malloc0 debian/build/deb/../../../glib/gmem.c:161:13 #2 0x55c0c0ae9b01 in qdict_new qobject/qdict.c:30:13 #3 0x55c0c0afc16c in parse_object qobject/json-parser.c:317:12 #4 0x55c0c0afb90f in parse_value qobject/json-parser.c:545:16 #5 0x55c0c0afb579 in json_parser_parse qobject/json-parser.c:579:14 #6 0x55c0c0afa21d in json_message_process_token qobject/json-streamer.c:92:12 #7 0x55c0c0bca2e5 in json_lexer_feed_char qobject/json-lexer.c:313:13 #8 0x55c0c0bc97ce in json_lexer_feed qobject/json-lexer.c:350:9 #9 0x55c0c0afabbc in json_message_parser_feed qobject/json-streamer.c:121:5 #10 0x55c0c09cbd52 in qmp_fd_receive tests/qtest/libqmp.c:86:9 #11 0x55c0c09be69b in qtest_qmp_receive_dict tests/qtest/libqtest.c:760:12 #12 0x55c0c09bca77 in qtest_qmp_receive tests/qtest/libqtest.c:741:27 #13 0x55c0c09bee9d in qtest_vqmp tests/qtest/libqtest.c:812:12 #14 0x55c0c09bd257 in qtest_qmp tests/qtest/libqtest.c:835:16 #15 0x55c0c0a87747 in multifd_mapped_ram_fdset_end tests/qtest/migration-test.c:2393:12 #16 0x55c0c0a85eb3 in test_file_common tests/qtest/migration-test.c:1978:9 #17 0x55c0c0a746a3 in test_multifd_file_mapped_ram_fdset tests/qtest/migration-test.c:2437:5 #18 0x55c0c0a93237 in migration_test_wrapper tests/qtest/migration-helpers.c:458:5 #19 0x7f186f958aed in test_case_run debian/build/deb/../../../glib/gtestutils.c:2930:15 #20 0x7f186f958aed in g_test_run_suite_internal debian/build/deb/../../../glib/gtestutils.c:3018:16 #21 0x7f186f95880a in g_test_run_suite_internal debian/build/deb/../../../glib/gtestutils.c:3035:18 #22 0x7f186f95880a in g_test_run_suite_internal debian/build/deb/../../../glib/gtestutils.c:3035:18 #23 0x7f186f95880a in g_test_run_suite_internal debian/build/deb/../../../glib/gtestutils.c:3035:18 #24 0x7f186f95880a in g_test_run_suite_internal debian/build/deb/../../../glib/gtestutils.c:3035:18 #25 0x7f186f95880a in g_test_run_suite_internal debian/build/deb/../../../glib/gtestutils.c:3035:18 #26 0x7f186f958faa in g_test_run_suite debian/build/deb/../../../glib/gtestutils.c:3109:18 #27 0x7f186f959055 in g_test_run debian/build/deb/../../../glib/gtestutils.c:2231:7 #28 0x7f186f959055 in g_test_run debian/build/deb/../../../glib/gtestutils.c:2218:1 #29 0x55c0c0a6e427 in main tests/qtest/migration-test.c:4033:11 Unref the object after we've confirmed that it is what we expect. Signed-off-by: Peter Maydell Reviewed-by: Fabiano Rosas Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 37ef99c980..b775ffed81 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2315,6 +2315,7 @@ static void multifd_mapped_ram_fdset_end(QTestState *from, QTestState *to, g_assert(qdict_haskey(resp, "return")); fdsets = qdict_get_qlist(resp, "return"); g_assert(fdsets && qlist_empty(fdsets)); + qobject_unref(resp); } static void *multifd_mapped_ram_fdset_dio(QTestState *from, QTestState *to) From patchwork Wed Sep 4 12:43:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C036ECD3431 for ; Wed, 4 Sep 2024 12:47:02 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNC-0003WP-AO; Wed, 04 Sep 2024 08:44:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNB-0003SQ-8Z for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:33 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpN9-0005Rm-JI for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:32 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 1EE961F38E; Wed, 4 Sep 2024 12:44:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453870; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GqaP/FCR+HWdzVs0QVMLDNAsRWxwmc3xCegKzUxtsN8=; b=Ava6dQEZiRDFXQSAJ+BEw20Uudp8LvvzCFGd0ZkIZVy+GwJJijHcrLhqHpnFKTGyXrG+Sm pGpDxygFhrNmylXu1VFMPkrXZuzMX9TMupCGDO/EwmFnfv5Zj/AphAc5AZDvIAHRSTC3pz +mR8XPfmPUzPnngJ7Es2KrPrARn2hH4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453870; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GqaP/FCR+HWdzVs0QVMLDNAsRWxwmc3xCegKzUxtsN8=; b=KYenGYCetC5k6R9d8w6qRLVX4Uv/FkN2E+Hj+jHiem+/3t6pyCk0t1IH0eF/J291+IKv3E wMY/Wgf18K4tj7BA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453870; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GqaP/FCR+HWdzVs0QVMLDNAsRWxwmc3xCegKzUxtsN8=; b=Ava6dQEZiRDFXQSAJ+BEw20Uudp8LvvzCFGd0ZkIZVy+GwJJijHcrLhqHpnFKTGyXrG+Sm pGpDxygFhrNmylXu1VFMPkrXZuzMX9TMupCGDO/EwmFnfv5Zj/AphAc5AZDvIAHRSTC3pz +mR8XPfmPUzPnngJ7Es2KrPrARn2hH4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453870; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GqaP/FCR+HWdzVs0QVMLDNAsRWxwmc3xCegKzUxtsN8=; b=KYenGYCetC5k6R9d8w6qRLVX4Uv/FkN2E+Hj+jHiem+/3t6pyCk0t1IH0eF/J291+IKv3E wMY/Wgf18K4tj7BA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B38F5139D2; Wed, 4 Sep 2024 12:44:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WBwxHixW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:28 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Peter Maydell Subject: [PULL 05/34] tests/qtest/migration-test: Fix leaks in calc_dirtyrate_ready() Date: Wed, 4 Sep 2024 09:43:48 -0300 Message-Id: <20240904124417.14565-6-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.78 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.18)[-0.925]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:mid,linaro.org:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Maydell In calc_dirtyrate_ready() we g_strdup() a string but then never free it: Direct leak of 19 byte(s) in 2 object(s) allocated from: #0 0x55ead613413e in malloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/tests/qtest/migration-test+0x22f13e) (BuildId: e7cd5c37b2987a1af682b43ee5240b98bb316737) #1 0x7f7a13d39738 in g_malloc debian/build/deb/../../../glib/gmem.c:128:13 #2 0x7f7a13d4e583 in g_strdup debian/build/deb/../../../glib/gstrfuncs.c:361:17 #3 0x55ead6266f48 in calc_dirtyrate_ready tests/qtest/migration-test.c:3409:14 #4 0x55ead62669fe in wait_for_calc_dirtyrate_complete tests/qtest/migration-test.c:3422:13 #5 0x55ead6253df7 in test_vcpu_dirty_limit tests/qtest/migration-test.c:3562:9 #6 0x55ead626a407 in migration_test_wrapper tests/qtest/migration-helpers.c:456:5 We also fail to unref the QMP rsp_return, so we leak that also. Rather than duplicating the string, use the in-place value from the qdict, and then unref the qdict. Signed-off-by: Peter Maydell Reviewed-by: Fabiano Rosas Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index b775ffed81..97f99c1316 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -3318,15 +3318,18 @@ static QDict *query_vcpu_dirty_limit(QTestState *who) static bool calc_dirtyrate_ready(QTestState *who) { QDict *rsp_return; - gchar *status; + const char *status; + bool ready; rsp_return = query_dirty_rate(who); g_assert(rsp_return); - status = g_strdup(qdict_get_str(rsp_return, "status")); + status = qdict_get_str(rsp_return, "status"); g_assert(status); + ready = g_strcmp0(status, "measuring"); + qobject_unref(rsp_return); - return g_strcmp0(status, "measuring"); + return ready; } static void wait_for_calc_dirtyrate_complete(QTestState *who, From patchwork Wed Sep 4 12:43:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CCBEDCD3431 for ; Wed, 4 Sep 2024 12:48:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNE-0003gg-RK; Wed, 04 Sep 2024 08:44:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpND-0003Z9-0E for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:35 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNB-0005SO-8l for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:34 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id E86862191F; Wed, 4 Sep 2024 12:44:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453872; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qi/XzRa+1wsS1LuzhOV4ldJ/cldmkU5okDmZg2VwaI4=; b=yus6klPTXXznuJ2MXhCbRajMTwHH68uH14+Wr25BZ+/dYKm8I9X4RVWac44waVnDNaXDXe Yf6W8dVfHJgXxBUU7wnQI8my4A6cgIzyK0CC4UB4g96NUkx5InzWCp5fDkJsRz/STReaHO mfvUKEhFPP/KoqenlZRpaaHu5IksDFg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453872; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qi/XzRa+1wsS1LuzhOV4ldJ/cldmkU5okDmZg2VwaI4=; b=cfysV+y+ZEWJIM+TVOBQIlT/OdX6MEt6wRdMgPF9XpN7C3EUmI5PwL5uv9FWUDCMUM/CDB GJdsKV4k8JrV5iCQ== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453871; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qi/XzRa+1wsS1LuzhOV4ldJ/cldmkU5okDmZg2VwaI4=; b=NgVnCv3icurnoyRCy0fsyGgSIcPXu/jKge6Rq0l4NIe7Jlv+7V70QEVYwFhzaSdvuJxz+o /4X+wUFnsLSDiiuETs1nx3IXISNoSfUy/brEBaJv2xGQN6Ff2g88noeXDLOfUnF6IS3oLW VYEU9juZry7/Ua+qQVVKZvweQVBdKqU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453871; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qi/XzRa+1wsS1LuzhOV4ldJ/cldmkU5okDmZg2VwaI4=; b=renisp8KyRitIJkL+WjV5ITllep5UcD6Fw5z2MFWOQcChjcuvJyHk1+Dh4xbbiRWbyxJm4 GTPzoUEKGq5yTKAw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 88E41139D2; Wed, 4 Sep 2024 12:44:30 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cKwBFC5W2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:30 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Peter Maydell Subject: [PULL 06/34] tests/qtest/migration-helpers: Fix migrate_get_socket_address() leak Date: Wed, 4 Sep 2024 09:43:49 -0300 Message-Id: <20240904124417.14565-7-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.78 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.18)[-0.925]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[linaro.org:email,suse.de:mid,suse.de:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Maydell In migrate_get_socket_address() we leak the SocketAddressList: (cd build/asan && \ ASAN_OPTIONS="fast_unwind_on_malloc=0:strip_path_prefix=/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/../../" QTEST_QEMU_BINARY=./qemu-system-x86_64 \ ./tests/qtest/migration-test --tap -k -p /x86_64/migration/multifd/tcp/tls/psk/match ) [...] Direct leak of 16 byte(s) in 1 object(s) allocated from: #0 0x563d7f22f318 in __interceptor_calloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/tests/qtest/migration-test+0x22f318) (BuildId: 2ad6282fb5d076c863ab87f41a345d46dc965ded) #1 0x7f9de3b39c50 in g_malloc0 debian/build/deb/../../../glib/gmem.c:161:13 #2 0x563d7f3a119c in qobject_input_start_list qapi/qobject-input-visitor.c:336:17 #3 0x563d7f390fbf in visit_start_list qapi/qapi-visit-core.c:80:10 #4 0x563d7f3882ef in visit_type_SocketAddressList /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/qapi/qapi-visit-sockets.c:519:10 #5 0x563d7f3658c9 in migrate_get_socket_address tests/qtest/migration-helpers.c:97:5 #6 0x563d7f362e24 in migrate_get_connect_uri tests/qtest/migration-helpers.c:111:13 #7 0x563d7f362bb2 in migrate_qmp tests/qtest/migration-helpers.c:222:23 #8 0x563d7f3533cd in test_precopy_common tests/qtest/migration-test.c:1817:5 #9 0x563d7f34dc1c in test_multifd_tcp_tls_psk_match tests/qtest/migration-test.c:3185:5 #10 0x563d7f365337 in migration_test_wrapper tests/qtest/migration-helpers.c:458:5 The code fishes out the SocketAddress from the list to return it, and the callers are freeing that, but nothing frees the list. Since this function is called in only two places, the simple fix is to make it return the SocketAddressList rather than just a SocketAddress, and then the callers can easily access the SocketAddress, and free the whole SocketAddressList when they're done. Signed-off-by: Peter Maydell Reviewed-by: Fabiano Rosas Signed-off-by: Fabiano Rosas --- tests/qtest/migration-helpers.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helpers.c index 84f49db85e..7cbb9831e7 100644 --- a/tests/qtest/migration-helpers.c +++ b/tests/qtest/migration-helpers.c @@ -82,11 +82,10 @@ static QDict *SocketAddress_to_qdict(SocketAddress *addr) return dict; } -static SocketAddress *migrate_get_socket_address(QTestState *who) +static SocketAddressList *migrate_get_socket_address(QTestState *who) { QDict *rsp; SocketAddressList *addrs; - SocketAddress *addr; Visitor *iv = NULL; QObject *object; @@ -95,36 +94,35 @@ static SocketAddress *migrate_get_socket_address(QTestState *who) iv = qobject_input_visitor_new(object); visit_type_SocketAddressList(iv, NULL, &addrs, &error_abort); - addr = addrs->value; visit_free(iv); qobject_unref(rsp); - return addr; + return addrs; } static char * migrate_get_connect_uri(QTestState *who) { - SocketAddress *addrs; + SocketAddressList *addrs; char *connect_uri; addrs = migrate_get_socket_address(who); - connect_uri = SocketAddress_to_str(addrs); + connect_uri = SocketAddress_to_str(addrs->value); - qapi_free_SocketAddress(addrs); + qapi_free_SocketAddressList(addrs); return connect_uri; } static QDict * migrate_get_connect_qdict(QTestState *who) { - SocketAddress *addrs; + SocketAddressList *addrs; QDict *connect_qdict; addrs = migrate_get_socket_address(who); - connect_qdict = SocketAddress_to_qdict(addrs); + connect_qdict = SocketAddress_to_qdict(addrs->value); - qapi_free_SocketAddress(addrs); + qapi_free_SocketAddressList(addrs); return connect_qdict; } From patchwork Wed Sep 4 12:43:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 93D92CD37B4 for ; Wed, 4 Sep 2024 12:45:46 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNH-0003oR-7Y; Wed, 04 Sep 2024 08:44:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNE-0003fw-JT for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:36 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNC-0005TT-T5 for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:36 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id C113A1F38E; Wed, 4 Sep 2024 12:44:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453873; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6DXoshtaUxP0FpcVJs8eo7ClCgFXi1fAoaWy/y+V1sI=; b=sUtGOqc3uQNkB0rbj/5tr2oRXKfw3wI5nl+meDYtS22uU/WNdFM3D9BcYPrMezrbyvsqU1 9AhGFmxMHcHP0LdX6L3Jq4rm93hOJ7Wuv8x4Sbwhl8O/3fLenpUHS31fuEQGa4Xo9WP798 M7NOmGrsai8er7Y9JbGKiBOcGg2+ASo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453873; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6DXoshtaUxP0FpcVJs8eo7ClCgFXi1fAoaWy/y+V1sI=; b=xe6uBCc2EnB2Hbj1+BU6vXGjQQ/Te2MQBAS53TPXCPcnqPd1sDcHXoXDi07jPzHncsqchW r2VAsSbDKGyEQFAQ== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453873; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6DXoshtaUxP0FpcVJs8eo7ClCgFXi1fAoaWy/y+V1sI=; b=sUtGOqc3uQNkB0rbj/5tr2oRXKfw3wI5nl+meDYtS22uU/WNdFM3D9BcYPrMezrbyvsqU1 9AhGFmxMHcHP0LdX6L3Jq4rm93hOJ7Wuv8x4Sbwhl8O/3fLenpUHS31fuEQGa4Xo9WP798 M7NOmGrsai8er7Y9JbGKiBOcGg2+ASo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453873; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6DXoshtaUxP0FpcVJs8eo7ClCgFXi1fAoaWy/y+V1sI=; b=xe6uBCc2EnB2Hbj1+BU6vXGjQQ/Te2MQBAS53TPXCPcnqPd1sDcHXoXDi07jPzHncsqchW r2VAsSbDKGyEQFAQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5FE74139D2; Wed, 4 Sep 2024 12:44:32 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cFHkCTBW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:32 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Peter Maydell Subject: [PULL 07/34] tests/qtest/migration-test: Free QCRyptoTLSTestCertReq objects Date: Wed, 4 Sep 2024 09:43:50 -0300 Message-Id: <20240904124417.14565-8-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.79 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.19)[-0.930]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[linaro.org:email,suse.de:mid,suse.de:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Maydell In the migration test we create several TLS certificates with the TLS_* macros from crypto-tls-x509-helpers.h. These macros create both a QCryptoTLSCertReq object which must be deinitialized and also an on-disk certificate file. The migration test currently removes the on-disk file in test_migrate_tls_x509_finish() but never deinitializes the QCryptoTLSCertReq, which means that memory allocated as part of it is leaked: Indirect leak of 2 byte(s) in 1 object(s) allocated from: #0 0x5558ba33712e in malloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/tests/qtest/migration-test+0x22f12e) (BuildId: 4c8618f663e538538cad19d35233124cea161491) #1 0x7f64afc131f4 (/lib/x86_64-linux-gnu/libtasn1.so.6+0x81f4) (BuildId: 2fde6ecb43c586fe4077118f771077aa1298e7ea) #2 0x7f64afc18d58 in asn1_write_value (/lib/x86_64-linux-gnu/libtasn1.so.6+0xdd58) (BuildId: 2fde6ecb43c586fe4077118f771077aa1298e7ea) #3 0x7f64af8fc678 in gnutls_x509_crt_set_version (/lib/x86_64-linux-gnu/libgnutls.so.30+0xe7678) (BuildId: 97b8f99f392f1fd37b969a7164bcea884e23649b) #4 0x5558ba470035 in test_tls_generate_cert tests/unit/crypto-tls-x509-helpers.c:234:5 #5 0x5558ba464e4a in test_migrate_tls_x509_start_common tests/qtest/migration-test.c:1058:5 #6 0x5558ba462c8a in test_migrate_tls_x509_start_default_host tests/qtest/migration-test.c:1123:12 #7 0x5558ba45ab40 in test_precopy_common tests/qtest/migration-test.c:1786:21 #8 0x5558ba450015 in test_precopy_unix_tls_x509_default_host tests/qtest/migration-test.c:2077:5 #9 0x5558ba46d3c7 in migration_test_wrapper tests/qtest/migration-helpers.c:456:5 (and similar reports). The only function currently provided to deinit a QCryptoTLSCertReq is test_tls_discard_cert(), which also removes the on-disk certificate file. For the migration tests we need to retain the on-disk files until we've finished running the test, so the simplest fix is to provide a new function test_tls_deinit_cert() which does only the cleanup of the QCryptoTLSCertReq, and call it in the right places. Signed-off-by: Peter Maydell Reviewed-by: Fabiano Rosas Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 3 +++ tests/unit/crypto-tls-x509-helpers.c | 12 ++++++++++-- tests/unit/crypto-tls-x509-helpers.h | 6 ++++++ 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 97f99c1316..f0f0335c6b 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1061,12 +1061,15 @@ test_migrate_tls_x509_start_common(QTestState *from, QCRYPTO_TLS_TEST_CLIENT_HOSTILE_NAME : QCRYPTO_TLS_TEST_CLIENT_NAME, data->clientcert); + test_tls_deinit_cert(&servercertreq); } TLS_CERT_REQ_SIMPLE_SERVER(clientcertreq, cacertreq, data->servercert, args->certhostname, args->certipaddr); + test_tls_deinit_cert(&clientcertreq); + test_tls_deinit_cert(&cacertreq); qtest_qmp_assert_success(from, "{ 'execute': 'object-add'," diff --git a/tests/unit/crypto-tls-x509-helpers.c b/tests/unit/crypto-tls-x509-helpers.c index 3e74ec5b5d..b316155d6a 100644 --- a/tests/unit/crypto-tls-x509-helpers.c +++ b/tests/unit/crypto-tls-x509-helpers.c @@ -502,8 +502,7 @@ void test_tls_write_cert_chain(const char *filename, g_free(buffer); } - -void test_tls_discard_cert(QCryptoTLSTestCertReq *req) +void test_tls_deinit_cert(QCryptoTLSTestCertReq *req) { if (!req->crt) { return; @@ -511,6 +510,15 @@ void test_tls_discard_cert(QCryptoTLSTestCertReq *req) gnutls_x509_crt_deinit(req->crt); req->crt = NULL; +} + +void test_tls_discard_cert(QCryptoTLSTestCertReq *req) +{ + if (!req->crt) { + return; + } + + test_tls_deinit_cert(req); if (getenv("QEMU_TEST_DEBUG_CERTS") == NULL) { unlink(req->filename); diff --git a/tests/unit/crypto-tls-x509-helpers.h b/tests/unit/crypto-tls-x509-helpers.h index 562c160653..2a0f7c04fd 100644 --- a/tests/unit/crypto-tls-x509-helpers.h +++ b/tests/unit/crypto-tls-x509-helpers.h @@ -73,6 +73,12 @@ void test_tls_generate_cert(QCryptoTLSTestCertReq *req, void test_tls_write_cert_chain(const char *filename, gnutls_x509_crt_t *certs, size_t ncerts); +/* + * Deinitialize the QCryptoTLSTestCertReq, but don't delete the certificate + * file on disk. (The caller is then responsible for doing that themselves. + */ +void test_tls_deinit_cert(QCryptoTLSTestCertReq *req); +/* Deinit the QCryptoTLSTestCertReq, and delete the certificate file */ void test_tls_discard_cert(QCryptoTLSTestCertReq *req); void test_tls_init(const char *keyfile); From patchwork Wed Sep 4 12:43:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790782 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8070FCD484A for ; Wed, 4 Sep 2024 12:47:03 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNK-0003vR-0u; Wed, 04 Sep 2024 08:44:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNG-0003nc-Im for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:39 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNE-0005UF-Ri for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:38 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 975242191F; Wed, 4 Sep 2024 12:44:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453875; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BhCq7dKgVbsaeLH4ffCexHZniHpgK/O1LsShezIDRYo=; b=DiPaBbtUTqNfnjGuqq1myGEMZalXX1XMwhX5yTI+tb2EZ+S6sud4MAPwXCwO7cMo2c9EWA maZWfLgP1m6cwhfpIRFKfHZrlv109yZ2W3QPgMYTifY7ajbDMXjGjFTvsY7Ixoq+rkzuVU 7LFdrKcsxiXVuSH1cM+sXGeNnkE7iS8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453875; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BhCq7dKgVbsaeLH4ffCexHZniHpgK/O1LsShezIDRYo=; b=equoGoffzwq8gsW3/U3JVT6gbQ6alTCoq4qCUKAFT0kPDLe4w4DHa+q27O9G1ISLJ1NT0b CEqZEiLY2/Gl0tDQ== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453875; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BhCq7dKgVbsaeLH4ffCexHZniHpgK/O1LsShezIDRYo=; b=DiPaBbtUTqNfnjGuqq1myGEMZalXX1XMwhX5yTI+tb2EZ+S6sud4MAPwXCwO7cMo2c9EWA maZWfLgP1m6cwhfpIRFKfHZrlv109yZ2W3QPgMYTifY7ajbDMXjGjFTvsY7Ixoq+rkzuVU 7LFdrKcsxiXVuSH1cM+sXGeNnkE7iS8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453875; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BhCq7dKgVbsaeLH4ffCexHZniHpgK/O1LsShezIDRYo=; b=equoGoffzwq8gsW3/U3JVT6gbQ6alTCoq4qCUKAFT0kPDLe4w4DHa+q27O9G1ISLJ1NT0b CEqZEiLY2/Gl0tDQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 38BB0139D2; Wed, 4 Sep 2024 12:44:34 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 6E4yADJW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:34 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Peter Maydell Subject: [PULL 08/34] tests/unit/crypto-tls-x509-helpers: deinit privkey in test_tls_cleanup Date: Wed, 4 Sep 2024 09:43:51 -0300 Message-Id: <20240904124417.14565-9-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.79 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.19)[-0.927]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:email,linaro.org:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Maydell We create a gnutls_x509_privkey_t in test_tls_init(), but forget to deinit it in test_tls_cleanup(), resulting in leaks reported in hte migration test such as: Indirect leak of 8 byte(s) in 1 object(s) allocated from: #0 0x55fa6d11c12e in malloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/tests/qtest/migration-test+0x22f12e) (BuildId: 852a267993587f557f50e5715f352f43720077ba) #1 0x7f073982685d in __gmp_default_allocate (/lib/x86_64-linux-gnu/libgmp.so.10+0xa85d) (BuildId: f110719303ddbea25a5e89ff730fec520eed67b0) #2 0x7f0739836193 in __gmpz_realloc (/lib/x86_64-linux-gnu/libgmp.so.10+0x1a193) (BuildId: f110719303ddbea25a5e89ff730fec520eed67b0) #3 0x7f0739836594 in __gmpz_import (/lib/x86_64-linux-gnu/libgmp.so.10+0x1a594) (BuildId: f110719303ddbea25a5e89ff730fec520eed67b0) #4 0x7f07398a91ed in nettle_mpz_set_str_256_u (/lib/x86_64-linux-gnu/libhogweed.so.6+0xb1ed) (BuildId: 3cc4a3474de72db89e9dcc93bfb95fe377f48c37) #5 0x7f073a146a5a (/lib/x86_64-linux-gnu/libgnutls.so.30+0x131a5a) (BuildId: 97b8f99f392f1fd37b969a7164bcea884e23649b) #6 0x7f073a07192c (/lib/x86_64-linux-gnu/libgnutls.so.30+0x5c92c) (BuildId: 97b8f99f392f1fd37b969a7164bcea884e23649b) #7 0x7f073a078333 (/lib/x86_64-linux-gnu/libgnutls.so.30+0x63333) (BuildId: 97b8f99f392f1fd37b969a7164bcea884e23649b) #8 0x7f073a0e8353 (/lib/x86_64-linux-gnu/libgnutls.so.30+0xd3353) (BuildId: 97b8f99f392f1fd37b969a7164bcea884e23649b) #9 0x7f073a0ef0ac in gnutls_x509_privkey_import (/lib/x86_64-linux-gnu/libgnutls.so.30+0xda0ac) (BuildId: 97b8f99f392f1fd37b969a7164bcea884e23649b) #10 0x55fa6d2547e3 in test_tls_load_key tests/unit/crypto-tls-x509-helpers.c:99:11 #11 0x55fa6d25460c in test_tls_init tests/unit/crypto-tls-x509-helpers.c:128:15 #12 0x55fa6d2495c4 in test_migrate_tls_x509_start_common tests/qtest/migration-test.c:1044:5 #13 0x55fa6d24c23a in test_migrate_tls_x509_start_reject_anon_client tests/qtest/migration-test.c:1216:12 #14 0x55fa6d23fb40 in test_precopy_common tests/qtest/migration-test.c:1789:21 #15 0x55fa6d236b7c in test_precopy_tcp_tls_x509_reject_anon_client tests/qtest/migration-test.c:2614:5 (Oddly, there is no reported leak in the x509 unit tests, even though those also use test_tls_init() and test_tls_cleanup().) Deinit the privkey in test_tls_cleanup(). Signed-off-by: Peter Maydell Reviewed-by: Fabiano Rosas Signed-off-by: Fabiano Rosas --- tests/unit/crypto-tls-x509-helpers.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/unit/crypto-tls-x509-helpers.c b/tests/unit/crypto-tls-x509-helpers.c index b316155d6a..2daecc416c 100644 --- a/tests/unit/crypto-tls-x509-helpers.c +++ b/tests/unit/crypto-tls-x509-helpers.c @@ -135,6 +135,7 @@ void test_tls_init(const char *keyfile) void test_tls_cleanup(const char *keyfile) { asn1_delete_structure(&pkix_asn1); + gnutls_x509_privkey_deinit(privkey); unlink(keyfile); } From patchwork Wed Sep 4 12:43:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E4878CD3431 for ; Wed, 4 Sep 2024 12:46:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNL-00044b-MG; Wed, 04 Sep 2024 08:44:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNI-0003v2-FM for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:40 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNG-0005Ut-Mr for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:40 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6CD212119F; Wed, 4 Sep 2024 12:44:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453877; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z2Ri8HctTIXsXSaBfIqxljhxeFAgkjIWbGYXT0sfccc=; b=dfs6ckQVBxMw6kI4JyxWrA3OXTZTFWHeaMPPhzyp0hdc6jyFQHADox4cuwiErz9pLuU2M/ irnbjy+MlsfB3ScQdQ9cK1Wn9jibDonERQZZHpIbAcX3XVPf4feaXYA0AtoXZ0Y8t9QEY/ Hrlsl4w8tcsAQO6FnYF6weo/FdItPJE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453877; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z2Ri8HctTIXsXSaBfIqxljhxeFAgkjIWbGYXT0sfccc=; b=N9F48+vfk7EG5HpLcJxrHl7o1+/0mYmy/kv2EZrFdjZ+CtAiw6tv5RSLCGEdPrl0s9Tmy/ +sOL0YgaUxIQz5DA== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453877; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z2Ri8HctTIXsXSaBfIqxljhxeFAgkjIWbGYXT0sfccc=; b=dfs6ckQVBxMw6kI4JyxWrA3OXTZTFWHeaMPPhzyp0hdc6jyFQHADox4cuwiErz9pLuU2M/ irnbjy+MlsfB3ScQdQ9cK1Wn9jibDonERQZZHpIbAcX3XVPf4feaXYA0AtoXZ0Y8t9QEY/ Hrlsl4w8tcsAQO6FnYF6weo/FdItPJE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453877; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z2Ri8HctTIXsXSaBfIqxljhxeFAgkjIWbGYXT0sfccc=; b=N9F48+vfk7EG5HpLcJxrHl7o1+/0mYmy/kv2EZrFdjZ+CtAiw6tv5RSLCGEdPrl0s9Tmy/ +sOL0YgaUxIQz5DA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0D43F139D2; Wed, 4 Sep 2024 12:44:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id kEFXMTNW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:35 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Peter Maydell Subject: [PULL 09/34] tests/qtest/migration-helpers: Don't dup argument to qdict_put_str() Date: Wed, 4 Sep 2024 09:43:52 -0300 Message-Id: <20240904124417.14565-10-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.79 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.19)[-0.927]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[linaro.org:email,suse.de:mid,suse.de:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Maydell In migrate_set_ports() we call qdict_put_str() with a value string which we g_strdup(). However qdict_put_str() takes a copy of the value string, it doesn't take ownership of it, so the g_strdup() only results in a leak: Direct leak of 6 byte(s) in 1 object(s) allocated from: #0 0x56298023713e in malloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/tests/qtest/migration-test+0x22f13e) (BuildId: b2b9174a5a54707a7f76bca51cdc95d2aa08bac1) #1 0x7fba0ad39738 in g_malloc debian/build/deb/../../../glib/gmem.c:128:13 #2 0x7fba0ad4e583 in g_strdup debian/build/deb/../../../glib/gstrfuncs.c:361:17 #3 0x56298036b16e in migrate_set_ports tests/qtest/migration-helpers.c:145:49 #4 0x56298036ad1c in migrate_qmp tests/qtest/migration-helpers.c:228:9 #5 0x56298035b3dd in test_precopy_common tests/qtest/migration-test.c:1820:5 #6 0x5629803549dc in test_multifd_tcp_channels_none tests/qtest/migration-test.c:3077:5 #7 0x56298036d427 in migration_test_wrapper tests/qtest/migration-helpers.c:456:5 Drop the unnecessary g_strdup() call. Signed-off-by: Peter Maydell Reviewed-by: Fabiano Rosas Signed-off-by: Fabiano Rosas --- tests/qtest/migration-helpers.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helpers.c index 7cbb9831e7..a43d180c80 100644 --- a/tests/qtest/migration-helpers.c +++ b/tests/qtest/migration-helpers.c @@ -142,7 +142,7 @@ static void migrate_set_ports(QTestState *to, QList *channel_list) qdict_haskey(addr, "port") && (strcmp(qdict_get_str(addrdict, "port"), "0") == 0)) { addr_port = qdict_get_str(addr, "port"); - qdict_put_str(addrdict, "port", g_strdup(addr_port)); + qdict_put_str(addrdict, "port", addr_port); } } From patchwork Wed Sep 4 12:43:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EC71DCD4849 for ; Wed, 4 Sep 2024 12:46:31 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNP-0004JL-Bs; Wed, 04 Sep 2024 08:44:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNL-00045r-8Z for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:43 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNJ-0005Vc-Ld for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:43 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 456692191F; Wed, 4 Sep 2024 12:44:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453879; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EBCOf/JUetC7ZUytRZGZMdkk3yEjJLBkIUqiPgUQdVY=; b=ITlBQOMmQzY+C+7L5D4BTNfGCQ08+MEmTt+Qr5dSSrlSa3wk3AW42TF6MquuxxAEEiff8K JQNxDX4kJI41NZ3fPpgGuExEfPfdK4Yetskq43ZDGIoaIKpu/sv6AJNIu7EQiuL7Da0bqx jMRonMWmaQzNR3NengHTAlVF0V3+2Uo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453879; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EBCOf/JUetC7ZUytRZGZMdkk3yEjJLBkIUqiPgUQdVY=; b=b+V6By0CsCcwmQ9BzGUYlzYJGZm2T4k3xr9aS1C6v4eeS7UTN02T/pCTDx4vGObPqkl1K4 Ky2QT+ZMyJtrNtDA== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453879; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EBCOf/JUetC7ZUytRZGZMdkk3yEjJLBkIUqiPgUQdVY=; b=ITlBQOMmQzY+C+7L5D4BTNfGCQ08+MEmTt+Qr5dSSrlSa3wk3AW42TF6MquuxxAEEiff8K JQNxDX4kJI41NZ3fPpgGuExEfPfdK4Yetskq43ZDGIoaIKpu/sv6AJNIu7EQiuL7Da0bqx jMRonMWmaQzNR3NengHTAlVF0V3+2Uo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453879; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EBCOf/JUetC7ZUytRZGZMdkk3yEjJLBkIUqiPgUQdVY=; b=b+V6By0CsCcwmQ9BzGUYlzYJGZm2T4k3xr9aS1C6v4eeS7UTN02T/pCTDx4vGObPqkl1K4 Ky2QT+ZMyJtrNtDA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D6ACC139D2; Wed, 4 Sep 2024 12:44:37 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id gIjaJjVW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:37 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Peter Maydell Subject: [PULL 10/34] tests/qtest/migration-test: Don't strdup in get_dirty_rate() Date: Wed, 4 Sep 2024 09:43:53 -0300 Message-Id: <20240904124417.14565-11-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.79 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.19)[-0.925]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[linaro.org:email,suse.de:email,suse.de:mid]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Maydell We g_strdup() the "status" string we get out of the qdict in get_dirty_rate(), but we never free it. Since we only use this string while the dictionary is still valid, we don't need to strdup at all; drop the unnecessary call to avoid this leak: Direct leak of 18 byte(s) in 2 object(s) allocated from: #0 0x564b3e01913e in malloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/tests/qtest/migration-test+0x22f13e) (BuildId: d6403a811332fcc846f93c45e23abfd06d1e67c4) #1 0x7f2f278ff738 in g_malloc debian/build/deb/../../../glib/gmem.c:128:13 #2 0x7f2f27914583 in g_strdup debian/build/deb/../../../glib/gstrfuncs.c:361:17 #3 0x564b3e14bb5b in get_dirty_rate tests/qtest/migration-test.c:3447:14 #4 0x564b3e138e00 in test_vcpu_dirty_limit tests/qtest/migration-test.c:3565:16 #5 0x564b3e14f417 in migration_test_wrapper tests/qtest/migration-helpers.c:456:5 Signed-off-by: Peter Maydell Reviewed-by: Fabiano Rosas Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index f0f0335c6b..3818595040 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -3355,7 +3355,7 @@ static void wait_for_calc_dirtyrate_complete(QTestState *who, static int64_t get_dirty_rate(QTestState *who) { QDict *rsp_return; - gchar *status; + const char *status; QList *rates; const QListEntry *entry; QDict *rate; @@ -3364,7 +3364,7 @@ static int64_t get_dirty_rate(QTestState *who) rsp_return = query_dirty_rate(who); g_assert(rsp_return); - status = g_strdup(qdict_get_str(rsp_return, "status")); + status = qdict_get_str(rsp_return, "status"); g_assert(status); g_assert_cmpstr(status, ==, "measured"); From patchwork Wed Sep 4 12:43:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 406F7CD4849 for ; Wed, 4 Sep 2024 12:48:58 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNV-0004fX-0n; Wed, 04 Sep 2024 08:44:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNM-00049J-6a for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:45 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNK-0005Vz-DX for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:43 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 1AA591F38E; Wed, 4 Sep 2024 12:44:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453881; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mmzN9h1hq3ADXXvfx51VoENPQLC4RFTA8VHLqVTGY5U=; b=bLxxq6yWHSidfzuPEfV1p1c++wQyrMH06znY4GPcDcLESsGQHU1onYHYJSCmTrAVX/6aMm xJKCO71dr2d9p5PWqvxM1jebISzL36VI1z9DfWjW7SIesVNE9xZ1AmjvCbPNrexrUXAxdu sXrJFVpUpvhFLnFFT/4TjcJmwuWiTHM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453881; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mmzN9h1hq3ADXXvfx51VoENPQLC4RFTA8VHLqVTGY5U=; b=NAjyow80rZ7LgpBibPxpOCs62lq1Yf4gQPIfBNXtQS96IFG39CzQIzswSxED1JXCEvw8hB DoNUNo3EXzFxrqBw== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=bLxxq6yW; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=NAjyow80 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453881; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mmzN9h1hq3ADXXvfx51VoENPQLC4RFTA8VHLqVTGY5U=; b=bLxxq6yWHSidfzuPEfV1p1c++wQyrMH06znY4GPcDcLESsGQHU1onYHYJSCmTrAVX/6aMm xJKCO71dr2d9p5PWqvxM1jebISzL36VI1z9DfWjW7SIesVNE9xZ1AmjvCbPNrexrUXAxdu sXrJFVpUpvhFLnFFT/4TjcJmwuWiTHM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453881; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mmzN9h1hq3ADXXvfx51VoENPQLC4RFTA8VHLqVTGY5U=; b=NAjyow80rZ7LgpBibPxpOCs62lq1Yf4gQPIfBNXtQS96IFG39CzQIzswSxED1JXCEvw8hB DoNUNo3EXzFxrqBw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id AF0E6139D2; Wed, 4 Sep 2024 12:44:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id aDE6HTdW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:39 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Peter Maydell Subject: [PULL 11/34] tests/qtest/migration-test: Don't leak QTestState in test_multifd_tcp_cancel() Date: Wed, 4 Sep 2024 09:43:54 -0300 Message-Id: <20240904124417.14565-12-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1AA591F38E X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[linaro.org:email,suse.de:email,suse.de:dkim,suse.de:mid]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Maydell In test_multifd_tcp_cancel() we create three QEMU processes: 'from', 'to' and 'to2'. We clean up (via qtest_quit()) 'from' and 'to2' when we call test_migrate_end(), but never clean up 'to', which results in this leak: Direct leak of 336 byte(s) in 1 object(s) allocated from: #0 0x55e984fcd328 in __interceptor_calloc (/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/asan/tests/qtest/migration-test+0x22f328) (BuildId: 710d409b68bb04427009e9ca6e1b63ff8af785d3) #1 0x7f0878b39c50 in g_malloc0 debian/build/deb/../../../glib/gmem.c:161:13 #2 0x55e98503a172 in qtest_spawn_qemu tests/qtest/libqtest.c:397:21 #3 0x55e98502bc4a in qtest_init_internal tests/qtest/libqtest.c:471:9 #4 0x55e98502c5b7 in qtest_init_with_env tests/qtest/libqtest.c:533:21 #5 0x55e9850eef0f in test_migrate_start tests/qtest/migration-test.c:857:11 #6 0x55e9850eb01d in test_multifd_tcp_cancel tests/qtest/migration-test.c:3297:9 #7 0x55e985103407 in migration_test_wrapper tests/qtest/migration-helpers.c:456:5 Call qtest_quit() on 'to' to clean it up once it has exited. Signed-off-by: Peter Maydell Reviewed-by: Fabiano Rosas Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 3818595040..6aca6760ef 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -3242,6 +3242,7 @@ static void test_multifd_tcp_cancel(void) /* Make sure QEMU process "to" exited */ qtest_set_expected_status(to, EXIT_FAILURE); qtest_wait_qemu(to); + qtest_quit(to); args = (MigrateStart){ .only_target = true, From patchwork Wed Sep 4 12:43:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790784 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 92F3ACD3431 for ; Wed, 4 Sep 2024 12:47:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNY-0004vp-Ne; Wed, 04 Sep 2024 08:44:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNP-0004MC-LT for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:47 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNM-0005Wb-2j for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:46 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id AB1901F7BC; Wed, 4 Sep 2024 12:44:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453882; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZoUl0udIRRV4d9HW5hBW3oAtA8isHXavDrhrtHZ82rM=; b=ZThUmcKCBPTndMfQbXoDp6Jl15Cu8KEHH7Gr2DkYXSYQmzOlxhvI85rlxq6flUZMUItx7l UqUjCoVyoydgggkM5uyuhL4Q9I6tRgf0EVDfz7z1Gjm5He/2aw6zMgi8LDl5kHB0IW/IeR wzyAI0M7cGB1dVyHIeHlDMsRJB1I5EI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453882; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZoUl0udIRRV4d9HW5hBW3oAtA8isHXavDrhrtHZ82rM=; b=ifw/eeOy87ueB7OZiMLxEYHF2faevfQJv+BNaJ+H3ql0fTmPcD01I9S+9rI7qH98nUYlJw rG6EUyiPTzgoFaBA== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=ZThUmcKC; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="ifw/eeOy" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453882; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZoUl0udIRRV4d9HW5hBW3oAtA8isHXavDrhrtHZ82rM=; b=ZThUmcKCBPTndMfQbXoDp6Jl15Cu8KEHH7Gr2DkYXSYQmzOlxhvI85rlxq6flUZMUItx7l UqUjCoVyoydgggkM5uyuhL4Q9I6tRgf0EVDfz7z1Gjm5He/2aw6zMgi8LDl5kHB0IW/IeR wzyAI0M7cGB1dVyHIeHlDMsRJB1I5EI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453882; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZoUl0udIRRV4d9HW5hBW3oAtA8isHXavDrhrtHZ82rM=; b=ifw/eeOy87ueB7OZiMLxEYHF2faevfQJv+BNaJ+H3ql0fTmPcD01I9S+9rI7qH98nUYlJw rG6EUyiPTzgoFaBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 84184139D2; Wed, 4 Sep 2024 12:44:41 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id yOjDEjlW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:41 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 12/34] migration/multifd: Reduce access to p->pages Date: Wed, 4 Sep 2024 09:43:55 -0300 Message-Id: <20240904124417.14565-13-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: AB1901F7BC X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:mid,suse.de:email]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org I'm about to replace the p->pages pointer with an opaque pointer, so do a cleanup now to reduce direct accesses to p->page, which makes the next diffs cleaner. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-qpl.c | 8 +++++--- migration/multifd-uadk.c | 9 +++++---- migration/multifd-zlib.c | 2 +- migration/multifd-zstd.c | 2 +- migration/multifd.c | 13 +++++++------ 5 files changed, 19 insertions(+), 15 deletions(-) diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c index 9265098ee7..f8c84c52cf 100644 --- a/migration/multifd-qpl.c +++ b/migration/multifd-qpl.c @@ -404,13 +404,14 @@ retry: static void multifd_qpl_compress_pages_slow_path(MultiFDSendParams *p) { QplData *qpl = p->compress_data; + MultiFDPages_t *pages = p->pages; uint32_t size = p->page_size; qpl_job *job = qpl->sw_job; uint8_t *zbuf = qpl->zbuf; uint8_t *buf; - for (int i = 0; i < p->pages->normal_num; i++) { - buf = p->pages->block->host + p->pages->offset[i]; + for (int i = 0; i < pages->normal_num; i++) { + buf = pages->block->host + pages->offset[i]; multifd_qpl_prepare_comp_job(job, buf, zbuf, size); if (qpl_execute_job(job) == QPL_STS_OK) { multifd_qpl_fill_packet(i, p, zbuf, job->total_out); @@ -498,6 +499,7 @@ static void multifd_qpl_compress_pages(MultiFDSendParams *p) static int multifd_qpl_send_prepare(MultiFDSendParams *p, Error **errp) { QplData *qpl = p->compress_data; + MultiFDPages_t *pages = p->pages; uint32_t len = 0; if (!multifd_send_prepare_common(p)) { @@ -505,7 +507,7 @@ static int multifd_qpl_send_prepare(MultiFDSendParams *p, Error **errp) } /* The first IOV is used to store the compressed page lengths */ - len = p->pages->normal_num * sizeof(uint32_t); + len = pages->normal_num * sizeof(uint32_t); multifd_qpl_fill_iov(p, (uint8_t *) qpl->zlen, len); if (qpl->hw_avail) { multifd_qpl_compress_pages(p); diff --git a/migration/multifd-uadk.c b/migration/multifd-uadk.c index d12353fb21..b8ba3cd9c1 100644 --- a/migration/multifd-uadk.c +++ b/migration/multifd-uadk.c @@ -174,19 +174,20 @@ static int multifd_uadk_send_prepare(MultiFDSendParams *p, Error **errp) uint32_t hdr_size; uint8_t *buf = uadk_data->buf; int ret = 0; + MultiFDPages_t *pages = p->pages; if (!multifd_send_prepare_common(p)) { goto out; } - hdr_size = p->pages->normal_num * sizeof(uint32_t); + hdr_size = pages->normal_num * sizeof(uint32_t); /* prepare the header that stores the lengths of all compressed data */ prepare_next_iov(p, uadk_data->buf_hdr, hdr_size); - for (int i = 0; i < p->pages->normal_num; i++) { + for (int i = 0; i < pages->normal_num; i++) { struct wd_comp_req creq = { .op_type = WD_DIR_COMPRESS, - .src = p->pages->block->host + p->pages->offset[i], + .src = pages->block->host + pages->offset[i], .src_len = p->page_size, .dst = buf, /* Set dst_len to double the src in case compressed out >= page_size */ @@ -214,7 +215,7 @@ static int multifd_uadk_send_prepare(MultiFDSendParams *p, Error **errp) */ if (!uadk_data->handle || creq.dst_len >= p->page_size) { uadk_data->buf_hdr[i] = cpu_to_be32(p->page_size); - prepare_next_iov(p, p->pages->block->host + p->pages->offset[i], + prepare_next_iov(p, pages->block->host + pages->offset[i], p->page_size); buf += p->page_size; } diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 2ced69487e..65f8aba5c8 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -147,7 +147,7 @@ static int zlib_send_prepare(MultiFDSendParams *p, Error **errp) * with compression. zlib does not guarantee that this is safe, * therefore copy the page before calling deflate(). */ - memcpy(z->buf, p->pages->block->host + pages->offset[i], p->page_size); + memcpy(z->buf, pages->block->host + pages->offset[i], p->page_size); zs->avail_in = p->page_size; zs->next_in = z->buf; diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index ca17b7e310..cb6075a9a5 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -138,7 +138,7 @@ static int zstd_send_prepare(MultiFDSendParams *p, Error **errp) if (i == pages->normal_num - 1) { flush = ZSTD_e_flush; } - z->in.src = p->pages->block->host + pages->offset[i]; + z->in.src = pages->block->host + pages->offset[i]; z->in.size = p->page_size; z->in.pos = 0; diff --git a/migration/multifd.c b/migration/multifd.c index a6db05502a..0bd9c2253e 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -114,11 +114,11 @@ static void multifd_set_file_bitmap(MultiFDSendParams *p) assert(pages->block); - for (int i = 0; i < p->pages->normal_num; i++) { + for (int i = 0; i < pages->normal_num; i++) { ramblock_set_file_bmap_atomic(pages->block, pages->offset[i], true); } - for (int i = p->pages->normal_num; i < p->pages->num; i++) { + for (int i = pages->normal_num; i < pages->num; i++) { ramblock_set_file_bmap_atomic(pages->block, pages->offset[i], false); } } @@ -417,7 +417,7 @@ void multifd_send_fill_packet(MultiFDSendParams *p) int i; packet->flags = cpu_to_be32(p->flags); - packet->pages_alloc = cpu_to_be32(p->pages->allocated); + packet->pages_alloc = cpu_to_be32(pages->allocated); packet->normal_pages = cpu_to_be32(pages->normal_num); packet->zero_pages = cpu_to_be32(zero_num); packet->next_packet_size = cpu_to_be32(p->next_packet_size); @@ -953,7 +953,7 @@ static void *multifd_send_thread(void *opaque) if (migrate_mapped_ram()) { ret = file_write_ramblock_iov(p->c, p->iov, p->iovs_num, - p->pages->block, &local_err); + pages->block, &local_err); } else { ret = qio_channel_writev_full_all(p->c, p->iov, p->iovs_num, NULL, 0, p->write_flags, @@ -969,7 +969,7 @@ static void *multifd_send_thread(void *opaque) stat64_add(&mig_stats.normal_pages, pages->normal_num); stat64_add(&mig_stats.zero_pages, pages->num - pages->normal_num); - multifd_pages_reset(p->pages); + multifd_pages_reset(pages); p->next_packet_size = 0; /* @@ -1690,9 +1690,10 @@ void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) bool multifd_send_prepare_common(MultiFDSendParams *p) { + MultiFDPages_t *pages = p->pages; multifd_send_zero_page_detect(p); - if (!p->pages->normal_num) { + if (!pages->normal_num) { p->next_packet_size = 0; return false; } From patchwork Wed Sep 4 12:43:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790804 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C9270CD37B4 for ; Wed, 4 Sep 2024 12:50:09 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNZ-0004x5-49; Wed, 04 Sep 2024 08:44:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNQ-0004Pe-J5 for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:49 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNN-0005Wt-Em for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:48 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 4E8821F7BD; Wed, 4 Sep 2024 12:44:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453884; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m20m0r4QWCLMsSs8oz0T1IcCV7Rz+rDYdRTAHBTA5V0=; b=DKqaC2KG7ZMIoXf5XHbBvjNqm0rRSrZq3cRtdlNNPMwI4CLM8EcOQqe5HJwyChV8LUGgnG fuhKfn472/tm8EGfYz0HZF+3iiZk0GQSWd8UA3T4kNTr738OZ5u2XULLdCusr3yu53CwMI O1f+IJL4lLmfMbnxVUTy46erLuU83Fs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453884; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m20m0r4QWCLMsSs8oz0T1IcCV7Rz+rDYdRTAHBTA5V0=; b=OFm0CobitZzSliU65W1qwvUa+68m0z99dArwK/MamKGOHsW6+QNbZQicin7243wkFVo5KZ fH1v8Dxm5HvekgDg== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453884; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m20m0r4QWCLMsSs8oz0T1IcCV7Rz+rDYdRTAHBTA5V0=; b=DKqaC2KG7ZMIoXf5XHbBvjNqm0rRSrZq3cRtdlNNPMwI4CLM8EcOQqe5HJwyChV8LUGgnG fuhKfn472/tm8EGfYz0HZF+3iiZk0GQSWd8UA3T4kNTr738OZ5u2XULLdCusr3yu53CwMI O1f+IJL4lLmfMbnxVUTy46erLuU83Fs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453884; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m20m0r4QWCLMsSs8oz0T1IcCV7Rz+rDYdRTAHBTA5V0=; b=OFm0CobitZzSliU65W1qwvUa+68m0z99dArwK/MamKGOHsW6+QNbZQicin7243wkFVo5KZ fH1v8Dxm5HvekgDg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 233ED139D2; Wed, 4 Sep 2024 12:44:42 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id iKKtNjpW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:42 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 13/34] migration/multifd: Inline page_size and page_count Date: Wed, 4 Sep 2024 09:43:56 -0300 Message-Id: <20240904124417.14565-14-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.79 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.19)[-0.940]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The MultiFD*Params structures are for per-channel data. Constant values should not be there because that needlessly wastes cycles and storage. The page_size and page_count fall into this category so move them inline in multifd.h. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-qpl.c | 10 +++++++--- migration/multifd-uadk.c | 36 ++++++++++++++++++++--------------- migration/multifd-zero-page.c | 4 ++-- migration/multifd-zlib.c | 14 ++++++++------ migration/multifd-zstd.c | 11 ++++++----- migration/multifd.c | 33 ++++++++++++++++---------------- migration/multifd.h | 18 ++++++++++-------- 7 files changed, 71 insertions(+), 55 deletions(-) diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c index f8c84c52cf..db60c05795 100644 --- a/migration/multifd-qpl.c +++ b/migration/multifd-qpl.c @@ -233,8 +233,10 @@ static void multifd_qpl_deinit(QplData *qpl) static int multifd_qpl_send_setup(MultiFDSendParams *p, Error **errp) { QplData *qpl; + uint32_t page_size = multifd_ram_page_size(); + uint32_t page_count = multifd_ram_page_count(); - qpl = multifd_qpl_init(p->page_count, p->page_size, errp); + qpl = multifd_qpl_init(page_count, page_size, errp); if (!qpl) { return -1; } @@ -245,7 +247,7 @@ static int multifd_qpl_send_setup(MultiFDSendParams *p, Error **errp) * additional two IOVs are used to store packet header and compressed data * length */ - p->iov = g_new0(struct iovec, p->page_count + 2); + p->iov = g_new0(struct iovec, page_count + 2); return 0; } @@ -534,8 +536,10 @@ out: static int multifd_qpl_recv_setup(MultiFDRecvParams *p, Error **errp) { QplData *qpl; + uint32_t page_size = multifd_ram_page_size(); + uint32_t page_count = multifd_ram_page_count(); - qpl = multifd_qpl_init(p->page_count, p->page_size, errp); + qpl = multifd_qpl_init(page_count, page_size, errp); if (!qpl) { return -1; } diff --git a/migration/multifd-uadk.c b/migration/multifd-uadk.c index b8ba3cd9c1..1ed1c6afe6 100644 --- a/migration/multifd-uadk.c +++ b/migration/multifd-uadk.c @@ -114,8 +114,10 @@ static void multifd_uadk_uninit_sess(struct wd_data *wd) static int multifd_uadk_send_setup(MultiFDSendParams *p, Error **errp) { struct wd_data *wd; + uint32_t page_size = multifd_ram_page_size(); + uint32_t page_count = multifd_ram_page_count(); - wd = multifd_uadk_init_sess(p->page_count, p->page_size, true, errp); + wd = multifd_uadk_init_sess(page_count, page_size, true, errp); if (!wd) { return -1; } @@ -128,7 +130,7 @@ static int multifd_uadk_send_setup(MultiFDSendParams *p, Error **errp) * length */ - p->iov = g_new0(struct iovec, p->page_count + 2); + p->iov = g_new0(struct iovec, page_count + 2); return 0; } @@ -172,6 +174,7 @@ static int multifd_uadk_send_prepare(MultiFDSendParams *p, Error **errp) { struct wd_data *uadk_data = p->compress_data; uint32_t hdr_size; + uint32_t page_size = multifd_ram_page_size(); uint8_t *buf = uadk_data->buf; int ret = 0; MultiFDPages_t *pages = p->pages; @@ -188,7 +191,7 @@ static int multifd_uadk_send_prepare(MultiFDSendParams *p, Error **errp) struct wd_comp_req creq = { .op_type = WD_DIR_COMPRESS, .src = pages->block->host + pages->offset[i], - .src_len = p->page_size, + .src_len = page_size, .dst = buf, /* Set dst_len to double the src in case compressed out >= page_size */ .dst_len = p->page_size * 2, @@ -201,7 +204,7 @@ static int multifd_uadk_send_prepare(MultiFDSendParams *p, Error **errp) p->id, ret, creq.status); return -1; } - if (creq.dst_len < p->page_size) { + if (creq.dst_len < page_size) { uadk_data->buf_hdr[i] = cpu_to_be32(creq.dst_len); prepare_next_iov(p, buf, creq.dst_len); buf += creq.dst_len; @@ -213,11 +216,11 @@ static int multifd_uadk_send_prepare(MultiFDSendParams *p, Error **errp) * than page_size as well because at the receive end we can skip the * decompression. But it is tricky to find the right number here. */ - if (!uadk_data->handle || creq.dst_len >= p->page_size) { - uadk_data->buf_hdr[i] = cpu_to_be32(p->page_size); + if (!uadk_data->handle || creq.dst_len >= page_size) { + uadk_data->buf_hdr[i] = cpu_to_be32(page_size); prepare_next_iov(p, pages->block->host + pages->offset[i], - p->page_size); - buf += p->page_size; + page_size); + buf += page_size; } } out: @@ -239,8 +242,10 @@ out: static int multifd_uadk_recv_setup(MultiFDRecvParams *p, Error **errp) { struct wd_data *wd; + uint32_t page_size = multifd_ram_page_size(); + uint32_t page_count = multifd_ram_page_count(); - wd = multifd_uadk_init_sess(p->page_count, p->page_size, false, errp); + wd = multifd_uadk_init_sess(page_count, page_size, false, errp); if (!wd) { return -1; } @@ -281,6 +286,7 @@ static int multifd_uadk_recv(MultiFDRecvParams *p, Error **errp) uint32_t flags = p->flags & MULTIFD_FLAG_COMPRESSION_MASK; uint32_t hdr_len = p->normal_num * sizeof(uint32_t); uint32_t data_len = 0; + uint32_t page_size = multifd_ram_page_size(); uint8_t *buf = uadk_data->buf; int ret = 0; @@ -307,7 +313,7 @@ static int multifd_uadk_recv(MultiFDRecvParams *p, Error **errp) for (int i = 0; i < p->normal_num; i++) { uadk_data->buf_hdr[i] = be32_to_cpu(uadk_data->buf_hdr[i]); data_len += uadk_data->buf_hdr[i]; - assert(uadk_data->buf_hdr[i] <= p->page_size); + assert(uadk_data->buf_hdr[i] <= page_size); } /* read compressed data */ @@ -323,12 +329,12 @@ static int multifd_uadk_recv(MultiFDRecvParams *p, Error **errp) .src = buf, .src_len = uadk_data->buf_hdr[i], .dst = p->host + p->normal[i], - .dst_len = p->page_size, + .dst_len = page_size, }; - if (uadk_data->buf_hdr[i] == p->page_size) { - memcpy(p->host + p->normal[i], buf, p->page_size); - buf += p->page_size; + if (uadk_data->buf_hdr[i] == page_size) { + memcpy(p->host + p->normal[i], buf, page_size); + buf += page_size; continue; } @@ -344,7 +350,7 @@ static int multifd_uadk_recv(MultiFDRecvParams *p, Error **errp) p->id, ret, creq.status); return -1; } - if (creq.dst_len != p->page_size) { + if (creq.dst_len != page_size) { error_setg(errp, "multifd %u: decompressed length error", p->id); return -1; } diff --git a/migration/multifd-zero-page.c b/migration/multifd-zero-page.c index e1b8370f88..cc624e36b3 100644 --- a/migration/multifd-zero-page.c +++ b/migration/multifd-zero-page.c @@ -63,7 +63,7 @@ void multifd_send_zero_page_detect(MultiFDSendParams *p) while (i <= j) { uint64_t offset = pages->offset[i]; - if (!buffer_is_zero(rb->host + offset, p->page_size)) { + if (!buffer_is_zero(rb->host + offset, multifd_ram_page_size())) { i++; continue; } @@ -81,7 +81,7 @@ void multifd_recv_zero_page_process(MultiFDRecvParams *p) for (int i = 0; i < p->zero_num; i++) { void *page = p->host + p->zero[i]; if (ramblock_recv_bitmap_test_byte_offset(p->block, p->zero[i])) { - memset(page, 0, p->page_size); + memset(page, 0, multifd_ram_page_size()); } else { ramblock_recv_bitmap_set_offset(p->block, p->zero[i]); } diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 65f8aba5c8..e47d7f70dc 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -127,6 +127,7 @@ static int zlib_send_prepare(MultiFDSendParams *p, Error **errp) struct zlib_data *z = p->compress_data; z_stream *zs = &z->zs; uint32_t out_size = 0; + uint32_t page_size = multifd_ram_page_size(); int ret; uint32_t i; @@ -147,8 +148,8 @@ static int zlib_send_prepare(MultiFDSendParams *p, Error **errp) * with compression. zlib does not guarantee that this is safe, * therefore copy the page before calling deflate(). */ - memcpy(z->buf, pages->block->host + pages->offset[i], p->page_size); - zs->avail_in = p->page_size; + memcpy(z->buf, pages->block->host + pages->offset[i], page_size); + zs->avail_in = page_size; zs->next_in = z->buf; zs->avail_out = available; @@ -260,7 +261,8 @@ static int zlib_recv(MultiFDRecvParams *p, Error **errp) uint32_t in_size = p->next_packet_size; /* we measure the change of total_out */ uint32_t out_size = zs->total_out; - uint32_t expected_size = p->normal_num * p->page_size; + uint32_t page_size = multifd_ram_page_size(); + uint32_t expected_size = p->normal_num * page_size; uint32_t flags = p->flags & MULTIFD_FLAG_COMPRESSION_MASK; int ret; int i; @@ -296,7 +298,7 @@ static int zlib_recv(MultiFDRecvParams *p, Error **errp) flush = Z_SYNC_FLUSH; } - zs->avail_out = p->page_size; + zs->avail_out = page_size; zs->next_out = p->host + p->normal[i]; /* @@ -310,8 +312,8 @@ static int zlib_recv(MultiFDRecvParams *p, Error **errp) do { ret = inflate(zs, flush); } while (ret == Z_OK && zs->avail_in - && (zs->total_out - start) < p->page_size); - if (ret == Z_OK && (zs->total_out - start) < p->page_size) { + && (zs->total_out - start) < page_size); + if (ret == Z_OK && (zs->total_out - start) < page_size) { error_setg(errp, "multifd %u: inflate generated too few output", p->id); return -1; diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index cb6075a9a5..1812fd1b48 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -139,7 +139,7 @@ static int zstd_send_prepare(MultiFDSendParams *p, Error **errp) flush = ZSTD_e_flush; } z->in.src = pages->block->host + pages->offset[i]; - z->in.size = p->page_size; + z->in.size = multifd_ram_page_size(); z->in.pos = 0; /* @@ -254,7 +254,8 @@ static int zstd_recv(MultiFDRecvParams *p, Error **errp) { uint32_t in_size = p->next_packet_size; uint32_t out_size = 0; - uint32_t expected_size = p->normal_num * p->page_size; + uint32_t page_size = multifd_ram_page_size(); + uint32_t expected_size = p->normal_num * page_size; uint32_t flags = p->flags & MULTIFD_FLAG_COMPRESSION_MASK; struct zstd_data *z = p->compress_data; int ret; @@ -286,7 +287,7 @@ static int zstd_recv(MultiFDRecvParams *p, Error **errp) for (i = 0; i < p->normal_num; i++) { ramblock_recv_bitmap_set_offset(p->block, p->normal[i]); z->out.dst = p->host + p->normal[i]; - z->out.size = p->page_size; + z->out.size = page_size; z->out.pos = 0; /* @@ -300,8 +301,8 @@ static int zstd_recv(MultiFDRecvParams *p, Error **errp) do { ret = ZSTD_decompressStream(z->zds, &z->out, &z->in); } while (ret > 0 && (z->in.size - z->in.pos > 0) - && (z->out.pos < p->page_size)); - if (ret > 0 && (z->out.pos < p->page_size)) { + && (z->out.pos < page_size)); + if (ret > 0 && (z->out.pos < page_size)) { error_setg(errp, "multifd %u: decompressStream buffer too small", p->id); return -1; diff --git a/migration/multifd.c b/migration/multifd.c index 0bd9c2253e..3dfed8a005 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -133,15 +133,17 @@ static void multifd_set_file_bitmap(MultiFDSendParams *p) */ static int nocomp_send_setup(MultiFDSendParams *p, Error **errp) { + uint32_t page_count = multifd_ram_page_count(); + if (migrate_zero_copy_send()) { p->write_flags |= QIO_CHANNEL_WRITE_FLAG_ZERO_COPY; } if (multifd_use_packets()) { /* We need one extra place for the packet header */ - p->iov = g_new0(struct iovec, p->page_count + 1); + p->iov = g_new0(struct iovec, page_count + 1); } else { - p->iov = g_new0(struct iovec, p->page_count); + p->iov = g_new0(struct iovec, page_count); } return 0; @@ -165,14 +167,15 @@ static void nocomp_send_cleanup(MultiFDSendParams *p, Error **errp) static void multifd_send_prepare_iovs(MultiFDSendParams *p) { MultiFDPages_t *pages = p->pages; + uint32_t page_size = multifd_ram_page_size(); for (int i = 0; i < pages->normal_num; i++) { p->iov[p->iovs_num].iov_base = pages->block->host + pages->offset[i]; - p->iov[p->iovs_num].iov_len = p->page_size; + p->iov[p->iovs_num].iov_len = page_size; p->iovs_num++; } - p->next_packet_size = pages->normal_num * p->page_size; + p->next_packet_size = pages->normal_num * page_size; } /** @@ -237,7 +240,7 @@ static int nocomp_send_prepare(MultiFDSendParams *p, Error **errp) */ static int nocomp_recv_setup(MultiFDRecvParams *p, Error **errp) { - p->iov = g_new0(struct iovec, p->page_count); + p->iov = g_new0(struct iovec, multifd_ram_page_count()); return 0; } @@ -288,7 +291,7 @@ static int nocomp_recv(MultiFDRecvParams *p, Error **errp) for (int i = 0; i < p->normal_num; i++) { p->iov[i].iov_base = p->host + p->normal[i]; - p->iov[i].iov_len = p->page_size; + p->iov[i].iov_len = multifd_ram_page_size(); ramblock_recv_bitmap_set_offset(p->block, p->normal[i]); } return qio_channel_readv_all(p->c, p->iov, p->normal_num, errp); @@ -447,6 +450,8 @@ void multifd_send_fill_packet(MultiFDSendParams *p) static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) { MultiFDPacket_t *packet = p->packet; + uint32_t page_count = multifd_ram_page_count(); + uint32_t page_size = multifd_ram_page_size(); int i; packet->magic = be32_to_cpu(packet->magic); @@ -472,10 +477,10 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) * If we received a packet that is 100 times bigger than expected * just stop migration. It is a magic number. */ - if (packet->pages_alloc > p->page_count) { + if (packet->pages_alloc > page_count) { error_setg(errp, "multifd: received packet " "with size %u and expected a size of %u", - packet->pages_alloc, p->page_count) ; + packet->pages_alloc, page_count) ; return -1; } @@ -521,7 +526,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) for (i = 0; i < p->normal_num; i++) { uint64_t offset = be64_to_cpu(packet->offset[i]); - if (offset > (p->block->used_length - p->page_size)) { + if (offset > (p->block->used_length - page_size)) { error_setg(errp, "multifd: offset too long %" PRIu64 " (max " RAM_ADDR_FMT ")", offset, p->block->used_length); @@ -533,7 +538,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) for (i = 0; i < p->zero_num; i++) { uint64_t offset = be64_to_cpu(packet->offset[p->normal_num + i]); - if (offset > (p->block->used_length - p->page_size)) { + if (offset > (p->block->used_length - page_size)) { error_setg(errp, "multifd: offset too long %" PRIu64 " (max " RAM_ADDR_FMT ")", offset, p->block->used_length); @@ -1157,7 +1162,7 @@ bool multifd_send_setup(void) { MigrationState *s = migrate_get_current(); int thread_count, ret = 0; - uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); + uint32_t page_count = multifd_ram_page_count(); bool use_packets = multifd_use_packets(); uint8_t i; @@ -1191,8 +1196,6 @@ bool multifd_send_setup(void) p->packet->version = cpu_to_be32(MULTIFD_VERSION); } p->name = g_strdup_printf("mig/src/send_%d", i); - p->page_size = qemu_target_page_size(); - p->page_count = page_count; p->write_flags = 0; if (!multifd_new_send_channel_create(p, &local_err)) { @@ -1569,7 +1572,7 @@ static void *multifd_recv_thread(void *opaque) int multifd_recv_setup(Error **errp) { int thread_count; - uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); + uint32_t page_count = multifd_ram_page_count(); bool use_packets = multifd_use_packets(); uint8_t i; @@ -1613,8 +1616,6 @@ int multifd_recv_setup(Error **errp) p->name = g_strdup_printf("mig/dst/recv_%d", i); p->normal = g_new0(ram_addr_t, page_count); p->zero = g_new0(ram_addr_t, page_count); - p->page_count = page_count; - p->page_size = qemu_target_page_size(); } for (i = 0; i < thread_count; i++) { diff --git a/migration/multifd.h b/migration/multifd.h index 0ecd6f47d7..a2bba23af9 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -13,6 +13,7 @@ #ifndef QEMU_MIGRATION_MULTIFD_H #define QEMU_MIGRATION_MULTIFD_H +#include "exec/target_page.h" #include "ram.h" typedef struct MultiFDRecvData MultiFDRecvData; @@ -106,10 +107,6 @@ typedef struct { QIOChannel *c; /* packet allocated len */ uint32_t packet_len; - /* guest page size */ - uint32_t page_size; - /* number of pages in a full packet */ - uint32_t page_count; /* multifd flags for sending ram */ int write_flags; @@ -173,10 +170,6 @@ typedef struct { QIOChannel *c; /* packet allocated len */ uint32_t packet_len; - /* guest page size */ - uint32_t page_size; - /* number of pages in a full packet */ - uint32_t page_count; /* syncs main thread and channels */ QemuSemaphore sem_sync; @@ -254,4 +247,13 @@ static inline void multifd_send_prepare_header(MultiFDSendParams *p) void multifd_channel_connect(MultiFDSendParams *p, QIOChannel *ioc); +static inline uint32_t multifd_ram_page_size(void) +{ + return qemu_target_page_size(); +} + +static inline uint32_t multifd_ram_page_count(void) +{ + return MULTIFD_PACKET_SIZE / qemu_target_page_size(); +} #endif From patchwork Wed Sep 4 12:43:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790800 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 73293CD37B4 for ; Wed, 4 Sep 2024 12:49:12 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNW-0004l5-3e; Wed, 04 Sep 2024 08:44:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNR-0004Rv-59 for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:49 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNP-0005Xc-EB for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:48 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DFCE21F7BE; Wed, 4 Sep 2024 12:44:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453885; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y0HSli6qqhvsrwlkZtUULc7TpZRojqiByjP50yocx8g=; b=nimcgAKY6tPK3qc2f2LmV7IH1OHISwreKoqbXE0qn7rc2zJnRdMsc9ZlgtmBLqZkiqZ0qf fIACoTYoGxH7EnS4Zc++SYugvMqfyMLgFoX2CQqcJzY3GnntKL1HFSqK5HwKprIGAcxAQ6 MYFCmsecXgtlzLGH6v4vtJwoIKis80U= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453885; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y0HSli6qqhvsrwlkZtUULc7TpZRojqiByjP50yocx8g=; b=innLr/fG+mvKW1GiZwTzcGlL8MCItbZrybFXYaRQq3R50WIfpNFY0JGLGVvzvyM+Vhs2Jw bTLRvn2+c812l2Bg== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453885; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y0HSli6qqhvsrwlkZtUULc7TpZRojqiByjP50yocx8g=; b=nimcgAKY6tPK3qc2f2LmV7IH1OHISwreKoqbXE0qn7rc2zJnRdMsc9ZlgtmBLqZkiqZ0qf fIACoTYoGxH7EnS4Zc++SYugvMqfyMLgFoX2CQqcJzY3GnntKL1HFSqK5HwKprIGAcxAQ6 MYFCmsecXgtlzLGH6v4vtJwoIKis80U= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453885; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y0HSli6qqhvsrwlkZtUULc7TpZRojqiByjP50yocx8g=; b=innLr/fG+mvKW1GiZwTzcGlL8MCItbZrybFXYaRQq3R50WIfpNFY0JGLGVvzvyM+Vhs2Jw bTLRvn2+c812l2Bg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B855B139D2; Wed, 4 Sep 2024 12:44:44 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id uMyIHzxW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:44 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 14/34] migration/multifd: Remove pages->allocated Date: Wed, 4 Sep 2024 09:43:57 -0300 Message-Id: <20240904124417.14565-15-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.79 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.19)[-0.926]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:mid]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This value never changes and is always the same as page_count. We don't need a copy of it per-channel plus one in the extra slot. Remove it. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.c | 6 ++---- migration/multifd.h | 2 -- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 3dfed8a005..30e5c687d3 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -396,7 +396,6 @@ static MultiFDPages_t *multifd_pages_init(uint32_t n) { MultiFDPages_t *pages = g_new0(MultiFDPages_t, 1); - pages->allocated = n; pages->offset = g_new0(ram_addr_t, n); return pages; @@ -405,7 +404,6 @@ static MultiFDPages_t *multifd_pages_init(uint32_t n) static void multifd_pages_clear(MultiFDPages_t *pages) { multifd_pages_reset(pages); - pages->allocated = 0; g_free(pages->offset); pages->offset = NULL; g_free(pages); @@ -420,7 +418,7 @@ void multifd_send_fill_packet(MultiFDSendParams *p) int i; packet->flags = cpu_to_be32(p->flags); - packet->pages_alloc = cpu_to_be32(pages->allocated); + packet->pages_alloc = cpu_to_be32(multifd_ram_page_count()); packet->normal_pages = cpu_to_be32(pages->normal_num); packet->zero_pages = cpu_to_be32(zero_num); packet->next_packet_size = cpu_to_be32(p->next_packet_size); @@ -651,7 +649,7 @@ static inline bool multifd_queue_empty(MultiFDPages_t *pages) static inline bool multifd_queue_full(MultiFDPages_t *pages) { - return pages->num == pages->allocated; + return pages->num == multifd_ram_page_count(); } static inline void multifd_enqueue(MultiFDPages_t *pages, ram_addr_t offset) diff --git a/migration/multifd.h b/migration/multifd.h index a2bba23af9..660a9882c2 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -76,8 +76,6 @@ typedef struct { uint32_t num; /* number of normal pages */ uint32_t normal_num; - /* number of allocated pages */ - uint32_t allocated; /* offset of each page */ ram_addr_t *offset; RAMBlock *block; From patchwork Wed Sep 4 12:43:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 42BA6CD37B4 for ; Wed, 4 Sep 2024 12:49:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNW-0004lF-5D; Wed, 04 Sep 2024 08:44:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNS-0004X1-GF for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:52 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNQ-0005YH-UX for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:50 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 7F37A2119F; Wed, 4 Sep 2024 12:44:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453887; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NlMmLQqIoc99biOand2+mbxP6OE6deuFa9cOXS8yzLU=; b=BChTr/bfqDsjQ2KvK5/paAfb7xX++sh2ZPLapQTgu7b4mUmxdR7jq3KOFpnZp5roU1xMnW ghnK8yl4n2naPOZmv3cAguTQIpsYHa9mKb+13/pyfrZCm1JMiqL8o0QmVwTOOQBmYXCw+J O0BtoldNmHHEDX6wsiBS78HwuErdB5Y= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453887; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NlMmLQqIoc99biOand2+mbxP6OE6deuFa9cOXS8yzLU=; b=ncbl+zSke2QAy8QqVoQDbX9crbSHqIOsHaE1Gop8Mstul6bXdxJ/RD2Kfqaunub8qNk9dp mJh4yKo73IeVNaCw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="BChTr/bf"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=ncbl+zSk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453887; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NlMmLQqIoc99biOand2+mbxP6OE6deuFa9cOXS8yzLU=; b=BChTr/bfqDsjQ2KvK5/paAfb7xX++sh2ZPLapQTgu7b4mUmxdR7jq3KOFpnZp5roU1xMnW ghnK8yl4n2naPOZmv3cAguTQIpsYHa9mKb+13/pyfrZCm1JMiqL8o0QmVwTOOQBmYXCw+J O0BtoldNmHHEDX6wsiBS78HwuErdB5Y= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453887; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NlMmLQqIoc99biOand2+mbxP6OE6deuFa9cOXS8yzLU=; b=ncbl+zSke2QAy8QqVoQDbX9crbSHqIOsHaE1Gop8Mstul6bXdxJ/RD2Kfqaunub8qNk9dp mJh4yKo73IeVNaCw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 56DCE139D2; Wed, 4 Sep 2024 12:44:46 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id sGezBz5W2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:46 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 15/34] migration/multifd: Pass in MultiFDPages_t to file_write_ramblock_iov Date: Wed, 4 Sep 2024 09:43:58 -0300 Message-Id: <20240904124417.14565-16-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 7F37A2119F X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:mid,suse.de:email]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We want to stop dereferencing 'pages' so it can be replaced by an opaque pointer in the next patches. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/file.c | 3 ++- migration/file.h | 2 +- migration/multifd.c | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/migration/file.c b/migration/file.c index 6451a21c86..7f11e26f5c 100644 --- a/migration/file.c +++ b/migration/file.c @@ -196,12 +196,13 @@ void file_start_incoming_migration(FileMigrationArgs *file_args, Error **errp) } int file_write_ramblock_iov(QIOChannel *ioc, const struct iovec *iov, - int niov, RAMBlock *block, Error **errp) + int niov, MultiFDPages_t *pages, Error **errp) { ssize_t ret = 0; int i, slice_idx, slice_num; uintptr_t base, next, offset; size_t len; + RAMBlock *block = pages->block; slice_idx = 0; slice_num = 1; diff --git a/migration/file.h b/migration/file.h index 9f71e87f74..1a1115f7f1 100644 --- a/migration/file.h +++ b/migration/file.h @@ -21,6 +21,6 @@ int file_parse_offset(char *filespec, uint64_t *offsetp, Error **errp); void file_cleanup_outgoing_migration(void); bool file_send_channel_create(gpointer opaque, Error **errp); int file_write_ramblock_iov(QIOChannel *ioc, const struct iovec *iov, - int niov, RAMBlock *block, Error **errp); + int niov, MultiFDPages_t *pages, Error **errp); int multifd_file_recv_data(MultiFDRecvParams *p, Error **errp); #endif diff --git a/migration/multifd.c b/migration/multifd.c index 30e5c687d3..640e4450ff 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -956,7 +956,7 @@ static void *multifd_send_thread(void *opaque) if (migrate_mapped_ram()) { ret = file_write_ramblock_iov(p->c, p->iov, p->iovs_num, - pages->block, &local_err); + pages, &local_err); } else { ret = qio_channel_writev_full_all(p->c, p->iov, p->iovs_num, NULL, 0, p->write_flags, From patchwork Wed Sep 4 12:43:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790794 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DC31ECD4849 for ; Wed, 4 Sep 2024 12:48:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNX-0004pN-4B; Wed, 04 Sep 2024 08:44:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNU-0004dX-6d for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:52 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNS-0005Yc-8h for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:51 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 233582191F; Wed, 4 Sep 2024 12:44:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453889; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zBNH5hoMByRQw4O6EKyHHEbwBKscWoD9zGn8eAJCu0A=; b=xKsRHyWPqYkHQFlluVX60mHVi01J6qMSou5213r/XPycDpWFbmIWtfJoe300nwe+JBzdkm M9W6LJg9yDRPpZhZ3Gzk/dZX0yYopNjCVTTD0PcyMyMxW/rUfgfc3HYyislPyjL2AQLy8e 7rnnYB/IU3iBDr3WafQWM9CuRAtB1Hg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453889; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zBNH5hoMByRQw4O6EKyHHEbwBKscWoD9zGn8eAJCu0A=; b=L42n4J1/nTg5XfsYcWHKCngOSfVDyh/u77Ap3aKwNelLezqAlYmqqA+I3YlkgcsW/QPOZW hqnGwVdUC1Q1MiBA== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=xKsRHyWP; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="L42n4J1/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453889; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zBNH5hoMByRQw4O6EKyHHEbwBKscWoD9zGn8eAJCu0A=; b=xKsRHyWPqYkHQFlluVX60mHVi01J6qMSou5213r/XPycDpWFbmIWtfJoe300nwe+JBzdkm M9W6LJg9yDRPpZhZ3Gzk/dZX0yYopNjCVTTD0PcyMyMxW/rUfgfc3HYyislPyjL2AQLy8e 7rnnYB/IU3iBDr3WafQWM9CuRAtB1Hg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453889; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zBNH5hoMByRQw4O6EKyHHEbwBKscWoD9zGn8eAJCu0A=; b=L42n4J1/nTg5XfsYcWHKCngOSfVDyh/u77Ap3aKwNelLezqAlYmqqA+I3YlkgcsW/QPOZW hqnGwVdUC1Q1MiBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id EA1ED139D2; Wed, 4 Sep 2024 12:44:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SCerKz9W2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:47 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 16/34] migration/multifd: Introduce MultiFDSendData Date: Wed, 4 Sep 2024 09:43:59 -0300 Message-Id: <20240904124417.14565-17-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 233582191F X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:dkim,suse.de:mid]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add a new data structure to replace p->pages in the multifd channel. This new structure will hide the multifd payload type behind an union, so we don't need to add a new field to the channel each time we want to handle a different data type. This also allow us to keep multifd_send_pages() as is, without needing to complicate the pointer switching. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/migration/multifd.h b/migration/multifd.h index 660a9882c2..7bb4a2cbc4 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -17,6 +17,7 @@ #include "ram.h" typedef struct MultiFDRecvData MultiFDRecvData; +typedef struct MultiFDSendData MultiFDSendData; bool multifd_send_setup(void); void multifd_send_shutdown(void); @@ -88,6 +89,31 @@ struct MultiFDRecvData { off_t file_offset; }; +typedef enum { + MULTIFD_PAYLOAD_NONE, + MULTIFD_PAYLOAD_RAM, +} MultiFDPayloadType; + +typedef union MultiFDPayload { + MultiFDPages_t ram; +} MultiFDPayload; + +struct MultiFDSendData { + MultiFDPayloadType type; + MultiFDPayload u; +}; + +static inline bool multifd_payload_empty(MultiFDSendData *data) +{ + return data->type == MULTIFD_PAYLOAD_NONE; +} + +static inline void multifd_set_payload_type(MultiFDSendData *data, + MultiFDPayloadType type) +{ + data->type = type; +} + typedef struct { /* Fields are only written at creating/deletion time */ /* No lock required for them, they are read only */ From patchwork Wed Sep 4 12:44:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2CB4CCD37B4 for ; Wed, 4 Sep 2024 12:50:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNZ-0004x4-5H; Wed, 04 Sep 2024 08:44:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNV-0004jW-L4 for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:53 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNT-0005ZF-VI for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:53 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B78FB2119F; Wed, 4 Sep 2024 12:44:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453890; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=39t9Cv16AimeidENCBivcEBi5CMLaC52094MjW94ySo=; b=wZwvuHvn+mdRfFGA9LNvDlNc5lpBmOPxaTK5vk/hf3tTzDxIu2RNgc6r/Ov/H3x5xw+52+ O1JczO7XLlGiESas2/oaVgbaK8mlAIeh+IK6c33f6Zz0Dvvbpk4+HhVUM/JmFf0Mc+VoVj yn5FMRXGqgqZjbsqaImhtmNXOq97WYE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453890; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=39t9Cv16AimeidENCBivcEBi5CMLaC52094MjW94ySo=; b=FYJDQH/DY4ju4P9lB7cax0fm9YnKHgslljrSjZmJI2sznZgR8pF4D9FfkQCapGyVJ62ylv tNNB4KDrNkkn+bDA== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=wZwvuHvn; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="FYJDQH/D" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453890; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=39t9Cv16AimeidENCBivcEBi5CMLaC52094MjW94ySo=; b=wZwvuHvn+mdRfFGA9LNvDlNc5lpBmOPxaTK5vk/hf3tTzDxIu2RNgc6r/Ov/H3x5xw+52+ O1JczO7XLlGiESas2/oaVgbaK8mlAIeh+IK6c33f6Zz0Dvvbpk4+HhVUM/JmFf0Mc+VoVj yn5FMRXGqgqZjbsqaImhtmNXOq97WYE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453890; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=39t9Cv16AimeidENCBivcEBi5CMLaC52094MjW94ySo=; b=FYJDQH/DY4ju4P9lB7cax0fm9YnKHgslljrSjZmJI2sznZgR8pF4D9FfkQCapGyVJ62ylv tNNB4KDrNkkn+bDA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8F140139D2; Wed, 4 Sep 2024 12:44:49 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id CDFDFUFW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:49 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 17/34] migration/multifd: Make MultiFDPages_t:offset a flexible array member Date: Wed, 4 Sep 2024 09:44:00 -0300 Message-Id: <20240904124417.14565-18-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: B78FB2119F X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:dkim,suse.de:mid]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We're about to use MultiFDPages_t from inside the MultiFDSendData payload union, which means we cannot have pointers to allocated data inside the pages structure, otherwise we'd lose the reference to that memory once another payload type touches the union. Move the offset array into the end of the structure and turn it into a flexible array member, so it is allocated along with the rest of MultiFDSendData in the next patches. Note that other pointers, such as the ramblock pointer are still fine as long as the storage for them is not owned by the migration code and can be correctly released at some point. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.c | 19 ++++++++++++------- migration/multifd.h | 4 ++-- 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 640e4450ff..717e71f539 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -98,6 +98,17 @@ struct { MultiFDMethods *ops; } *multifd_recv_state; +static size_t multifd_ram_payload_size(void) +{ + uint32_t n = multifd_ram_page_count(); + + /* + * We keep an array of page offsets at the end of MultiFDPages_t, + * add space for it in the allocation. + */ + return sizeof(MultiFDPages_t) + n * sizeof(ram_addr_t); +} + static bool multifd_use_packets(void) { return !migrate_mapped_ram(); @@ -394,18 +405,12 @@ static int multifd_recv_initial_packet(QIOChannel *c, Error **errp) static MultiFDPages_t *multifd_pages_init(uint32_t n) { - MultiFDPages_t *pages = g_new0(MultiFDPages_t, 1); - - pages->offset = g_new0(ram_addr_t, n); - - return pages; + return g_malloc0(multifd_ram_payload_size()); } static void multifd_pages_clear(MultiFDPages_t *pages) { multifd_pages_reset(pages); - g_free(pages->offset); - pages->offset = NULL; g_free(pages); } diff --git a/migration/multifd.h b/migration/multifd.h index 7bb4a2cbc4..a7fdd97f70 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -77,9 +77,9 @@ typedef struct { uint32_t num; /* number of normal pages */ uint32_t normal_num; + RAMBlock *block; /* offset of each page */ - ram_addr_t *offset; - RAMBlock *block; + ram_addr_t offset[]; } MultiFDPages_t; struct MultiFDRecvData { From patchwork Wed Sep 4 12:44:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790776 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DC20DCD37B4 for ; Wed, 4 Sep 2024 12:46:31 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNa-00054L-Tb; Wed, 04 Sep 2024 08:44:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNY-0004sy-1r for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:56 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNV-0005Za-NA for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:55 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 535361F38E; Wed, 4 Sep 2024 12:44:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453892; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NGSG7UuMyDvGvnyCspfLz8qZ9BkR6lzkG/QyTlxaVZs=; b=Lkl7f6EokhPiuqEBmG9DCURGz375X57vdYgQg0uFrOR1epxWuO8JfvOBTUXAuE2QlyXDkf CZnIA4QwcXcWMTEQ0UZP3qbF7DcJKx0QrmZFu0KIZK067EO5PZ2VPngVcQJkijxEKv8jrh +rMsrqFAUvx6w0qUKm958yutj7MbK40= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453892; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NGSG7UuMyDvGvnyCspfLz8qZ9BkR6lzkG/QyTlxaVZs=; b=0XRRxiCRZe1vHzc+m2lh42WlXEoaR3OsMLVO629de3w6Fk4BVjENSySTCnVKfzhRzy9zg5 xFA5X7h3VMOjw2Bw== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=Lkl7f6Eo; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=0XRRxiCR DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453892; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NGSG7UuMyDvGvnyCspfLz8qZ9BkR6lzkG/QyTlxaVZs=; b=Lkl7f6EokhPiuqEBmG9DCURGz375X57vdYgQg0uFrOR1epxWuO8JfvOBTUXAuE2QlyXDkf CZnIA4QwcXcWMTEQ0UZP3qbF7DcJKx0QrmZFu0KIZK067EO5PZ2VPngVcQJkijxEKv8jrh +rMsrqFAUvx6w0qUKm958yutj7MbK40= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453892; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NGSG7UuMyDvGvnyCspfLz8qZ9BkR6lzkG/QyTlxaVZs=; b=0XRRxiCRZe1vHzc+m2lh42WlXEoaR3OsMLVO629de3w6Fk4BVjENSySTCnVKfzhRzy9zg5 xFA5X7h3VMOjw2Bw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 2CE39139D2; Wed, 4 Sep 2024 12:44:50 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0HIkOUJW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:50 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 18/34] migration/multifd: Replace p->pages with an union pointer Date: Wed, 4 Sep 2024 09:44:01 -0300 Message-Id: <20240904124417.14565-19-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 535361F38E X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCVD_TLS_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[3]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:mid,suse.de:email] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We want multifd to be able to handle more types of data than just ram pages. To start decoupling multifd from pages, replace p->pages (MultiFDPages_t) with the new type MultiFDSendData that hides the client payload inside an union. The general idea here is to isolate functions that *need* to handle MultiFDPages_t and move them in the future to multifd-ram.c, while multifd.c will stay with only the core functions that handle MultiFDSendData/MultiFDRecvData. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-qpl.c | 6 +-- migration/multifd-uadk.c | 2 +- migration/multifd-zero-page.c | 2 +- migration/multifd-zlib.c | 2 +- migration/multifd-zstd.c | 2 +- migration/multifd.c | 83 +++++++++++++++++++++-------------- migration/multifd.h | 7 +-- 7 files changed, 57 insertions(+), 47 deletions(-) diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c index db60c05795..21153f1987 100644 --- a/migration/multifd-qpl.c +++ b/migration/multifd-qpl.c @@ -406,7 +406,7 @@ retry: static void multifd_qpl_compress_pages_slow_path(MultiFDSendParams *p) { QplData *qpl = p->compress_data; - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; uint32_t size = p->page_size; qpl_job *job = qpl->sw_job; uint8_t *zbuf = qpl->zbuf; @@ -437,7 +437,7 @@ static void multifd_qpl_compress_pages_slow_path(MultiFDSendParams *p) static void multifd_qpl_compress_pages(MultiFDSendParams *p) { QplData *qpl = p->compress_data; - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; uint32_t size = p->page_size; QplHwJob *hw_job; uint8_t *buf; @@ -501,7 +501,7 @@ static void multifd_qpl_compress_pages(MultiFDSendParams *p) static int multifd_qpl_send_prepare(MultiFDSendParams *p, Error **errp) { QplData *qpl = p->compress_data; - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; uint32_t len = 0; if (!multifd_send_prepare_common(p)) { diff --git a/migration/multifd-uadk.c b/migration/multifd-uadk.c index 1ed1c6afe6..9d99807af5 100644 --- a/migration/multifd-uadk.c +++ b/migration/multifd-uadk.c @@ -177,7 +177,7 @@ static int multifd_uadk_send_prepare(MultiFDSendParams *p, Error **errp) uint32_t page_size = multifd_ram_page_size(); uint8_t *buf = uadk_data->buf; int ret = 0; - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; if (!multifd_send_prepare_common(p)) { goto out; diff --git a/migration/multifd-zero-page.c b/migration/multifd-zero-page.c index cc624e36b3..6506a4aa89 100644 --- a/migration/multifd-zero-page.c +++ b/migration/multifd-zero-page.c @@ -46,7 +46,7 @@ static void swap_page_offset(ram_addr_t *pages_offset, int a, int b) */ void multifd_send_zero_page_detect(MultiFDSendParams *p) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; RAMBlock *rb = pages->block; int i = 0; int j = pages->num - 1; diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index e47d7f70dc..66517c1067 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -123,7 +123,7 @@ static void zlib_send_cleanup(MultiFDSendParams *p, Error **errp) */ static int zlib_send_prepare(MultiFDSendParams *p, Error **errp) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; struct zlib_data *z = p->compress_data; z_stream *zs = &z->zs; uint32_t out_size = 0; diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index 1812fd1b48..04ac711cf4 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -119,7 +119,7 @@ static void zstd_send_cleanup(MultiFDSendParams *p, Error **errp) */ static int zstd_send_prepare(MultiFDSendParams *p, Error **errp) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; struct zstd_data *z = p->compress_data; int ret; uint32_t i; diff --git a/migration/multifd.c b/migration/multifd.c index 717e71f539..c310d28532 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -49,8 +49,7 @@ typedef struct { struct { MultiFDSendParams *params; - /* array of pages to sent */ - MultiFDPages_t *pages; + MultiFDSendData *data; /* * Global number of generated multifd packets. * @@ -109,6 +108,28 @@ static size_t multifd_ram_payload_size(void) return sizeof(MultiFDPages_t) + n * sizeof(ram_addr_t); } +static MultiFDSendData *multifd_send_data_alloc(void) +{ + size_t max_payload_size, size_minus_payload; + + /* + * MultiFDPages_t has a flexible array at the end, account for it + * when allocating MultiFDSendData. Use max() in case other types + * added to the union in the future are larger than + * (MultiFDPages_t + flex array). + */ + max_payload_size = MAX(multifd_ram_payload_size(), sizeof(MultiFDPayload)); + + /* + * Account for any holes the compiler might insert. We can't pack + * the structure because that misaligns the members and triggers + * Waddress-of-packed-member. + */ + size_minus_payload = sizeof(MultiFDSendData) - sizeof(MultiFDPayload); + + return g_malloc0(size_minus_payload + max_payload_size); +} + static bool multifd_use_packets(void) { return !migrate_mapped_ram(); @@ -121,7 +142,7 @@ void multifd_send_channel_created(void) static void multifd_set_file_bitmap(MultiFDSendParams *p) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; assert(pages->block); @@ -177,7 +198,7 @@ static void nocomp_send_cleanup(MultiFDSendParams *p, Error **errp) static void multifd_send_prepare_iovs(MultiFDSendParams *p) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; uint32_t page_size = multifd_ram_page_size(); for (int i = 0; i < pages->normal_num; i++) { @@ -403,21 +424,10 @@ static int multifd_recv_initial_packet(QIOChannel *c, Error **errp) return msg.id; } -static MultiFDPages_t *multifd_pages_init(uint32_t n) -{ - return g_malloc0(multifd_ram_payload_size()); -} - -static void multifd_pages_clear(MultiFDPages_t *pages) -{ - multifd_pages_reset(pages); - g_free(pages); -} - void multifd_send_fill_packet(MultiFDSendParams *p) { MultiFDPacket_t *packet = p->packet; - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; uint64_t packet_num; uint32_t zero_num = pages->num - pages->normal_num; int i; @@ -599,7 +609,7 @@ static bool multifd_send_pages(void) int i; static int next_channel; MultiFDSendParams *p = NULL; /* make happy gcc */ - MultiFDPages_t *pages = multifd_send_state->pages; + MultiFDSendData *tmp; if (multifd_send_should_exit()) { return false; @@ -634,11 +644,14 @@ static bool multifd_send_pages(void) * qatomic_store_release() in multifd_send_thread(). */ smp_mb_acquire(); - assert(!p->pages->num); - multifd_send_state->pages = p->pages; - p->pages = pages; + + assert(!p->data->u.ram.num); + + tmp = multifd_send_state->data; + multifd_send_state->data = p->data; + p->data = tmp; /* - * Making sure p->pages is setup before marking pending_job=true. Pairs + * Making sure p->data is setup before marking pending_job=true. Pairs * with the qatomic_load_acquire() in multifd_send_thread(). */ qatomic_store_release(&p->pending_job, true); @@ -668,7 +681,7 @@ bool multifd_queue_page(RAMBlock *block, ram_addr_t offset) MultiFDPages_t *pages; retry: - pages = multifd_send_state->pages; + pages = &multifd_send_state->data->u.ram; /* If the queue is empty, we can already enqueue now */ if (multifd_queue_empty(pages)) { @@ -798,8 +811,8 @@ static bool multifd_send_cleanup_channel(MultiFDSendParams *p, Error **errp) qemu_sem_destroy(&p->sem_sync); g_free(p->name); p->name = NULL; - multifd_pages_clear(p->pages); - p->pages = NULL; + g_free(p->data); + p->data = NULL; p->packet_len = 0; g_free(p->packet); p->packet = NULL; @@ -816,8 +829,8 @@ static void multifd_send_cleanup_state(void) qemu_sem_destroy(&multifd_send_state->channels_ready); g_free(multifd_send_state->params); multifd_send_state->params = NULL; - multifd_pages_clear(multifd_send_state->pages); - multifd_send_state->pages = NULL; + g_free(multifd_send_state->data); + multifd_send_state->data = NULL; g_free(multifd_send_state); multifd_send_state = NULL; } @@ -866,11 +879,13 @@ int multifd_send_sync_main(void) { int i; bool flush_zero_copy; + MultiFDPages_t *pages; if (!migrate_multifd()) { return 0; } - if (multifd_send_state->pages->num) { + pages = &multifd_send_state->data->u.ram; + if (pages->num) { if (!multifd_send_pages()) { error_report("%s: multifd_send_pages fail", __func__); return -1; @@ -945,11 +960,11 @@ static void *multifd_send_thread(void *opaque) } /* - * Read pending_job flag before p->pages. Pairs with the + * Read pending_job flag before p->data. Pairs with the * qatomic_store_release() in multifd_send_pages(). */ if (qatomic_load_acquire(&p->pending_job)) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; p->iovs_num = 0; assert(pages->num); @@ -961,7 +976,7 @@ static void *multifd_send_thread(void *opaque) if (migrate_mapped_ram()) { ret = file_write_ramblock_iov(p->c, p->iov, p->iovs_num, - pages, &local_err); + &p->data->u.ram, &local_err); } else { ret = qio_channel_writev_full_all(p->c, p->iov, p->iovs_num, NULL, 0, p->write_flags, @@ -981,7 +996,7 @@ static void *multifd_send_thread(void *opaque) p->next_packet_size = 0; /* - * Making sure p->pages is published before saying "we're + * Making sure p->data is published before saying "we're * free". Pairs with the smp_mb_acquire() in * multifd_send_pages(). */ @@ -1176,7 +1191,7 @@ bool multifd_send_setup(void) thread_count = migrate_multifd_channels(); multifd_send_state = g_malloc0(sizeof(*multifd_send_state)); multifd_send_state->params = g_new0(MultiFDSendParams, thread_count); - multifd_send_state->pages = multifd_pages_init(page_count); + multifd_send_state->data = multifd_send_data_alloc(); qemu_sem_init(&multifd_send_state->channels_created, 0); qemu_sem_init(&multifd_send_state->channels_ready, 0); qatomic_set(&multifd_send_state->exiting, 0); @@ -1189,7 +1204,7 @@ bool multifd_send_setup(void) qemu_sem_init(&p->sem, 0); qemu_sem_init(&p->sem_sync, 0); p->id = i; - p->pages = multifd_pages_init(page_count); + p->data = multifd_send_data_alloc(); if (use_packets) { p->packet_len = sizeof(MultiFDPacket_t) @@ -1694,7 +1709,7 @@ void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) bool multifd_send_prepare_common(MultiFDSendParams *p) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; multifd_send_zero_page_detect(p); if (!pages->normal_num) { diff --git a/migration/multifd.h b/migration/multifd.h index a7fdd97f70..c2ba4cad13 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -152,12 +152,7 @@ typedef struct { */ bool pending_job; bool pending_sync; - /* array of pages to sent. - * The owner of 'pages' depends of 'pending_job' value: - * pending_job == 0 -> migration_thread can use it. - * pending_job != 0 -> multifd_channel can use it. - */ - MultiFDPages_t *pages; + MultiFDSendData *data; /* thread local variables. No locking required */ From patchwork Wed Sep 4 12:44:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790788 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B0C07CD37B4 for ; Wed, 4 Sep 2024 12:48:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNb-00056m-DU; Wed, 04 Sep 2024 08:44:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNZ-0004x9-1W for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:57 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNX-0005bR-FJ for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:56 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id E43EA2119F; Wed, 4 Sep 2024 12:44:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453894; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OMCk6K5d6G6+9Y8txB3BHcZSoOMR6I+KaV+SlQizkAs=; b=yVBK7SEAyQhbK1Z8/ClRZz4G8mo8LnyWXi5IFDOvE9p0MDPmuL22yObR+8kxhEjV/LJye2 rPHhxXUNXyimSeeUiDDf2TnSe7vCeYIENkNsF/h0OTs3kZk1rl0yojAWaYRbBzBr4dNd/H kjsuMQcvHTu1lqGKbjO8NPKqGaLNAPA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453894; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OMCk6K5d6G6+9Y8txB3BHcZSoOMR6I+KaV+SlQizkAs=; b=pLN6MR0N9aJ63UJ2d95sTyreG8ShPkunIwNdkiF/CJyKoswBI9WSjaeKWYchbV0QaEclvq kVA1vgfhP5kgtPCA== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=Iux7mLF0; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=l7fJkt+8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453893; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OMCk6K5d6G6+9Y8txB3BHcZSoOMR6I+KaV+SlQizkAs=; b=Iux7mLF09aU3B7F6eIvnyCjmNAjhiUqAD7+z+VB5XON3dKFdAbvQLOSLFeZUqZheVHDMag 2TdL1bV2dSVHn8dMw8HhXeJvUFvdTaglf6c4QujOvbvdQSznvTKAuySD/h+6SlpG+myuyQ fRLTWoP7b+AwGRYcXQy1nRXyliglVQg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453893; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OMCk6K5d6G6+9Y8txB3BHcZSoOMR6I+KaV+SlQizkAs=; b=l7fJkt+8OfnVSN4O/vp0pRL2aJeUK8TJBskpUmBxqpE42ONgV3U81utRGEEzNG+IM1Ke9J nV+MreF70XC/AyBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id BE4C0139D2; Wed, 4 Sep 2024 12:44:52 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id IObxIERW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:52 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 19/34] migration/multifd: Move pages accounting into multifd_send_zero_page_detect() Date: Wed, 4 Sep 2024 09:44:02 -0300 Message-Id: <20240904124417.14565-20-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: E43EA2119F X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:mid,suse.de:email]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org All references to pages are being removed from the multifd worker threads in order to allow multifd to deal with different payload types. multifd_send_zero_page_detect() is called by all multifd migration paths that deal with pages and is the last spot where zero pages and normal page amounts are adjusted. Move the pages accounting into that function. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-zero-page.c | 7 ++++++- migration/multifd.c | 2 -- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/migration/multifd-zero-page.c b/migration/multifd-zero-page.c index 6506a4aa89..f1e988a959 100644 --- a/migration/multifd-zero-page.c +++ b/migration/multifd-zero-page.c @@ -14,6 +14,7 @@ #include "qemu/cutils.h" #include "exec/ramblock.h" #include "migration.h" +#include "migration-stats.h" #include "multifd.h" #include "options.h" #include "ram.h" @@ -53,7 +54,7 @@ void multifd_send_zero_page_detect(MultiFDSendParams *p) if (!multifd_zero_page_enabled()) { pages->normal_num = pages->num; - return; + goto out; } /* @@ -74,6 +75,10 @@ void multifd_send_zero_page_detect(MultiFDSendParams *p) } pages->normal_num = i; + +out: + stat64_add(&mig_stats.normal_pages, pages->normal_num); + stat64_add(&mig_stats.zero_pages, pages->num - pages->normal_num); } void multifd_recv_zero_page_process(MultiFDRecvParams *p) diff --git a/migration/multifd.c b/migration/multifd.c index c310d28532..410b7e12cc 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -989,8 +989,6 @@ static void *multifd_send_thread(void *opaque) stat64_add(&mig_stats.multifd_bytes, p->next_packet_size + p->packet_len); - stat64_add(&mig_stats.normal_pages, pages->normal_num); - stat64_add(&mig_stats.zero_pages, pages->num - pages->normal_num); multifd_pages_reset(pages); p->next_packet_size = 0; From patchwork Wed Sep 4 12:44:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CFF58CD37B4 for ; Wed, 4 Sep 2024 12:47:01 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNc-0005Ar-I5; Wed, 04 Sep 2024 08:45:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNa-00053U-LE for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:58 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNY-0005bx-Qn for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:44:58 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 859D91F7BC; Wed, 4 Sep 2024 12:44:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453895; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=41Iha+9TwuH9aA17NRpJF0mKGVR2kS9cNIKotE3QSTI=; b=UeDvBcKUT2mrQ4jnp6MiZ2IW40AHBfoK0wiFW9u0MDnEdDS9HwKBZyvWQt7lv19wTqVI7O iszFKzFvhstTc3MVqbLxKpH0ng6yO7mmoYwApIx6Wx2d7NDrGykrc3teGqs8FuZsFaT1pa Q2lmJ9PjIj4ZE76h0L/I6gRWQGPYaUE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453895; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=41Iha+9TwuH9aA17NRpJF0mKGVR2kS9cNIKotE3QSTI=; b=Ah134rsl1/L5QvShycM3S1Irze3QLKpaTRnlspM4LNenyBXK7qJD6zIOPxJOZ2QyJoE/ub cADVgS8TxnKvXmDA== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=UeDvBcKU; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=Ah134rsl DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453895; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=41Iha+9TwuH9aA17NRpJF0mKGVR2kS9cNIKotE3QSTI=; b=UeDvBcKUT2mrQ4jnp6MiZ2IW40AHBfoK0wiFW9u0MDnEdDS9HwKBZyvWQt7lv19wTqVI7O iszFKzFvhstTc3MVqbLxKpH0ng6yO7mmoYwApIx6Wx2d7NDrGykrc3teGqs8FuZsFaT1pa Q2lmJ9PjIj4ZE76h0L/I6gRWQGPYaUE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453895; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=41Iha+9TwuH9aA17NRpJF0mKGVR2kS9cNIKotE3QSTI=; b=Ah134rsl1/L5QvShycM3S1Irze3QLKpaTRnlspM4LNenyBXK7qJD6zIOPxJOZ2QyJoE/ub cADVgS8TxnKvXmDA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5B9DE139D2; Wed, 4 Sep 2024 12:44:54 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 6H/fCEZW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:54 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 20/34] migration/multifd: Remove total pages tracing Date: Wed, 4 Sep 2024 09:44:03 -0300 Message-Id: <20240904124417.14565-21-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 859D91F7BC X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:mid,suse.de:email]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The total_normal_pages and total_zero_pages elements are used only for the end tracepoints of the multifd threads. These are not super useful since they record per-channel numbers and are just the sum of all the pages that are transmitted per-packet, for which we already have tracepoints. Remove the totals from the tracing. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.c | 12 ++---------- migration/multifd.h | 8 -------- migration/trace-events | 4 ++-- 3 files changed, 4 insertions(+), 20 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 410b7e12cc..df8dfcc98f 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -453,8 +453,6 @@ void multifd_send_fill_packet(MultiFDSendParams *p) } p->packets_sent++; - p->total_normal_pages += pages->normal_num; - p->total_zero_pages += zero_num; trace_multifd_send(p->id, packet_num, pages->normal_num, zero_num, p->flags, p->next_packet_size); @@ -516,8 +514,6 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) p->next_packet_size = be32_to_cpu(packet->next_packet_size); p->packet_num = be64_to_cpu(packet->packet_num); p->packets_recved++; - p->total_normal_pages += p->normal_num; - p->total_zero_pages += p->zero_num; trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->zero_num, p->flags, p->next_packet_size); @@ -1036,8 +1032,7 @@ out: rcu_unregister_thread(); migration_threads_remove(thread); - trace_multifd_send_thread_end(p->id, p->packets_sent, p->total_normal_pages, - p->total_zero_pages); + trace_multifd_send_thread_end(p->id, p->packets_sent); return NULL; } @@ -1561,7 +1556,6 @@ static void *multifd_recv_thread(void *opaque) qemu_sem_wait(&p->sem_sync); } } else { - p->total_normal_pages += p->data->size / qemu_target_page_size(); p->data->size = 0; /* * Order data->size update before clearing @@ -1578,9 +1572,7 @@ static void *multifd_recv_thread(void *opaque) } rcu_unregister_thread(); - trace_multifd_recv_thread_end(p->id, p->packets_recved, - p->total_normal_pages, - p->total_zero_pages); + trace_multifd_recv_thread_end(p->id, p->packets_recved); return NULL; } diff --git a/migration/multifd.h b/migration/multifd.h index c2ba4cad13..9175104aea 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -162,10 +162,6 @@ typedef struct { uint32_t next_packet_size; /* packets sent through this channel */ uint64_t packets_sent; - /* non zero pages sent through this channel */ - uint64_t total_normal_pages; - /* zero pages sent through this channel */ - uint64_t total_zero_pages; /* buffers to send */ struct iovec *iov; /* number of iovs used */ @@ -218,10 +214,6 @@ typedef struct { RAMBlock *block; /* ramblock host address */ uint8_t *host; - /* non zero pages recv through this channel */ - uint64_t total_normal_pages; - /* zero pages recv through this channel */ - uint64_t total_zero_pages; /* buffers to recv */ struct iovec *iov; /* Pages that are not zero */ diff --git a/migration/trace-events b/migration/trace-events index 0b7c3324fb..0887cef912 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -134,7 +134,7 @@ multifd_recv_sync_main(long packet_num) "packet num %ld" multifd_recv_sync_main_signal(uint8_t id) "channel %u" multifd_recv_sync_main_wait(uint8_t id) "iter %u" multifd_recv_terminate_threads(bool error) "error %d" -multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t normal_pages, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PRIu64 " zero pages %" PRIu64 +multifd_recv_thread_end(uint8_t id, uint64_t packets) "channel %u packets %" PRIu64 multifd_recv_thread_start(uint8_t id) "%u" multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal_pages, uint32_t zero_pages, uint32_t flags, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " normal pages %u zero pages %u flags 0x%x next packet size %u" multifd_send_error(uint8_t id) "channel %u" @@ -142,7 +142,7 @@ multifd_send_sync_main(long packet_num) "packet num %ld" multifd_send_sync_main_signal(uint8_t id) "channel %u" multifd_send_sync_main_wait(uint8_t id) "channel %u" multifd_send_terminate_threads(void) "" -multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_pages, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PRIu64 " zero pages %" PRIu64 +multifd_send_thread_end(uint8_t id, uint64_t packets) "channel %u packets %" PRIu64 multifd_send_thread_start(uint8_t id) "%u" multifd_tls_outgoing_handshake_start(void *ioc, void *tioc, const char *hostname) "ioc=%p tioc=%p hostname=%s" multifd_tls_outgoing_handshake_error(void *ioc, const char *err) "ioc=%p err=%s" From patchwork Wed Sep 4 12:44:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790792 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A31E0CD37B4 for ; Wed, 4 Sep 2024 12:48:35 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNf-0005Lr-8H; Wed, 04 Sep 2024 08:45:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNc-0005Bs-O2 for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:00 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNa-0005cy-Jn for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:00 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 5B30D1F38E; Wed, 4 Sep 2024 12:44:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453897; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8ukWOmEgHLqBfOxNv4qwgTd9uIIxaNm442ZeTMbCSNY=; b=PNHBYonHBdGBm3CosTu+1EDWx2qsDTpSgsawcQ6KBstNYgntCD9Ug4xB57lUfrpO85z7s4 XXMN50cc3k8MAcqCcD32subDLuEq2rj7l2BxxJpFC8JEQtzjCv3i0sLeojqmy2eGTDrKOF 9Buqj110o4FioENNtPjqo8yy9ueJm1Y= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453897; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8ukWOmEgHLqBfOxNv4qwgTd9uIIxaNm442ZeTMbCSNY=; b=utz1qPKs/Cqt2GWMq6zoN/XL66g9F+rovRNC1Ms+Kkwk5zj+2hjm2pVEFnGxnw9cQZhliN Ge9vY0ISZDdVm6Bw== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453897; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8ukWOmEgHLqBfOxNv4qwgTd9uIIxaNm442ZeTMbCSNY=; b=PNHBYonHBdGBm3CosTu+1EDWx2qsDTpSgsawcQ6KBstNYgntCD9Ug4xB57lUfrpO85z7s4 XXMN50cc3k8MAcqCcD32subDLuEq2rj7l2BxxJpFC8JEQtzjCv3i0sLeojqmy2eGTDrKOF 9Buqj110o4FioENNtPjqo8yy9ueJm1Y= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453897; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8ukWOmEgHLqBfOxNv4qwgTd9uIIxaNm442ZeTMbCSNY=; b=utz1qPKs/Cqt2GWMq6zoN/XL66g9F+rovRNC1Ms+Kkwk5zj+2hjm2pVEFnGxnw9cQZhliN Ge9vY0ISZDdVm6Bw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id EF9A5139D2; Wed, 4 Sep 2024 12:44:55 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SC/3LEdW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:55 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , =?utf-8?q?Philippe_Mathie?= =?utf-8?q?u-Daud=C3=A9?= Subject: [PULL 21/34] migration/multifd: Isolate ram pages packet data Date: Wed, 4 Sep 2024 09:44:04 -0300 Message-Id: <20240904124417.14565-22-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-3.30 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.20)[-0.990]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_ALL(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[linaro.org:email,suse.de:email,suse.de:mid] Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org While we cannot yet disentangle the multifd packet from page data, we can make the code a bit cleaner by setting the page-related fields in a separate function. Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.c | 99 +++++++++++++++++++++++++----------------- migration/trace-events | 5 ++- 2 files changed, 63 insertions(+), 41 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index df8dfcc98f..d64fcdf4ac 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -424,65 +424,61 @@ static int multifd_recv_initial_packet(QIOChannel *c, Error **errp) return msg.id; } -void multifd_send_fill_packet(MultiFDSendParams *p) +static void multifd_ram_fill_packet(MultiFDSendParams *p) { MultiFDPacket_t *packet = p->packet; MultiFDPages_t *pages = &p->data->u.ram; - uint64_t packet_num; uint32_t zero_num = pages->num - pages->normal_num; - int i; - packet->flags = cpu_to_be32(p->flags); packet->pages_alloc = cpu_to_be32(multifd_ram_page_count()); packet->normal_pages = cpu_to_be32(pages->normal_num); packet->zero_pages = cpu_to_be32(zero_num); - packet->next_packet_size = cpu_to_be32(p->next_packet_size); - - packet_num = qatomic_fetch_inc(&multifd_send_state->packet_num); - packet->packet_num = cpu_to_be64(packet_num); if (pages->block) { strncpy(packet->ramblock, pages->block->idstr, 256); } - for (i = 0; i < pages->num; i++) { + for (int i = 0; i < pages->num; i++) { /* there are architectures where ram_addr_t is 32 bit */ uint64_t temp = pages->offset[i]; packet->offset[i] = cpu_to_be64(temp); } + trace_multifd_send_ram_fill(p->id, pages->normal_num, zero_num); +} + +void multifd_send_fill_packet(MultiFDSendParams *p) +{ + MultiFDPacket_t *packet = p->packet; + uint64_t packet_num; + + memset(packet, 0, p->packet_len); + + packet->magic = cpu_to_be32(MULTIFD_MAGIC); + packet->version = cpu_to_be32(MULTIFD_VERSION); + + packet->flags = cpu_to_be32(p->flags); + packet->next_packet_size = cpu_to_be32(p->next_packet_size); + + packet_num = qatomic_fetch_inc(&multifd_send_state->packet_num); + packet->packet_num = cpu_to_be64(packet_num); + p->packets_sent++; - trace_multifd_send(p->id, packet_num, pages->normal_num, zero_num, - p->flags, p->next_packet_size); + multifd_ram_fill_packet(p); + + trace_multifd_send_fill(p->id, packet_num, + p->flags, p->next_packet_size); } -static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) +static int multifd_ram_unfill_packet(MultiFDRecvParams *p, Error **errp) { MultiFDPacket_t *packet = p->packet; uint32_t page_count = multifd_ram_page_count(); uint32_t page_size = multifd_ram_page_size(); int i; - packet->magic = be32_to_cpu(packet->magic); - if (packet->magic != MULTIFD_MAGIC) { - error_setg(errp, "multifd: received packet " - "magic %x and expected magic %x", - packet->magic, MULTIFD_MAGIC); - return -1; - } - - packet->version = be32_to_cpu(packet->version); - if (packet->version != MULTIFD_VERSION) { - error_setg(errp, "multifd: received packet " - "version %u and expected version %u", - packet->version, MULTIFD_VERSION); - return -1; - } - - p->flags = be32_to_cpu(packet->flags); - packet->pages_alloc = be32_to_cpu(packet->pages_alloc); /* * If we received a packet that is 100 times bigger than expected @@ -511,13 +507,6 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) return -1; } - p->next_packet_size = be32_to_cpu(packet->next_packet_size); - p->packet_num = be64_to_cpu(packet->packet_num); - p->packets_recved++; - - trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->zero_num, - p->flags, p->next_packet_size); - if (p->normal_num == 0 && p->zero_num == 0) { return 0; } @@ -559,6 +548,40 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) return 0; } +static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) +{ + MultiFDPacket_t *packet = p->packet; + int ret = 0; + + packet->magic = be32_to_cpu(packet->magic); + if (packet->magic != MULTIFD_MAGIC) { + error_setg(errp, "multifd: received packet " + "magic %x and expected magic %x", + packet->magic, MULTIFD_MAGIC); + return -1; + } + + packet->version = be32_to_cpu(packet->version); + if (packet->version != MULTIFD_VERSION) { + error_setg(errp, "multifd: received packet " + "version %u and expected version %u", + packet->version, MULTIFD_VERSION); + return -1; + } + + p->flags = be32_to_cpu(packet->flags); + p->next_packet_size = be32_to_cpu(packet->next_packet_size); + p->packet_num = be64_to_cpu(packet->packet_num); + p->packets_recved++; + + ret = multifd_ram_unfill_packet(p, errp); + + trace_multifd_recv_unfill(p->id, p->packet_num, p->flags, + p->next_packet_size); + + return ret; +} + static bool multifd_send_should_exit(void) { return qatomic_read(&multifd_send_state->exiting); @@ -1203,8 +1226,6 @@ bool multifd_send_setup(void) p->packet_len = sizeof(MultiFDPacket_t) + sizeof(uint64_t) * page_count; p->packet = g_malloc0(p->packet_len); - p->packet->magic = cpu_to_be32(MULTIFD_MAGIC); - p->packet->version = cpu_to_be32(MULTIFD_VERSION); } p->name = g_strdup_printf("mig/src/send_%d", i); p->write_flags = 0; diff --git a/migration/trace-events b/migration/trace-events index 0887cef912..c65902f042 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -128,7 +128,7 @@ postcopy_preempt_reset_channel(void) "" # multifd.c multifd_new_send_channel_async(uint8_t id) "channel %u" multifd_new_send_channel_async_error(uint8_t id, void *err) "channel=%u err=%p" -multifd_recv(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t zero, uint32_t flags, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " normal pages %u zero pages %u flags 0x%x next packet size %u" +multifd_recv_unfill(uint8_t id, uint64_t packet_num, uint32_t flags, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " flags 0x%x next packet size %u" multifd_recv_new_channel(uint8_t id) "channel %u" multifd_recv_sync_main(long packet_num) "packet num %ld" multifd_recv_sync_main_signal(uint8_t id) "channel %u" @@ -136,7 +136,8 @@ multifd_recv_sync_main_wait(uint8_t id) "iter %u" multifd_recv_terminate_threads(bool error) "error %d" multifd_recv_thread_end(uint8_t id, uint64_t packets) "channel %u packets %" PRIu64 multifd_recv_thread_start(uint8_t id) "%u" -multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal_pages, uint32_t zero_pages, uint32_t flags, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " normal pages %u zero pages %u flags 0x%x next packet size %u" +multifd_send_fill(uint8_t id, uint64_t packet_num, uint32_t flags, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " flags 0x%x next packet size %u" +multifd_send_ram_fill(uint8_t id, uint32_t normal, uint32_t zero) "channel %u normal pages %u zero pages %u" multifd_send_error(uint8_t id) "channel %u" multifd_send_sync_main(long packet_num) "packet num %ld" multifd_send_sync_main_signal(uint8_t id) "channel %u" From patchwork Wed Sep 4 12:44:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790770 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DAF1FCD37B4 for ; Wed, 4 Sep 2024 12:45:40 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNf-0005OF-Ur; Wed, 04 Sep 2024 08:45:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNe-0005Gq-2f for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:02 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNc-0005di-8f for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:01 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id ECEB22119F; Wed, 4 Sep 2024 12:44:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453899; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1ZzH6tbwU1tnRJepeYyskBXA12nUW4vAOqU+UVM33TM=; b=n3NXldXhNy96FKLhscpqiE2RdS7GaHb2jcVwvjFs4MbXpcXOzIsvj7EyX9gNk9+QMGs0cB iXOO6wNo1aW9GpgsQD5vj6YZw6JZo8iV6j14omM0TK6W1CmakjZ7EAx5oJGlav/Jnshb3P v15/0M+Ay04dOyPQBOq2dzL9K2skQRc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453899; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1ZzH6tbwU1tnRJepeYyskBXA12nUW4vAOqU+UVM33TM=; b=wO1HleKLn6DR7DdmAx7WrKw74XOY/D7+FCpAhP4NbTEjrt4jcILBhk/+iNnyg9A6zktA8X j/PBH0rii4cKlcAw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=hq401+jX; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="5m+yPv/l" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453898; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1ZzH6tbwU1tnRJepeYyskBXA12nUW4vAOqU+UVM33TM=; b=hq401+jXL5innj9TSdHJqde3tmUF+nPyXcS/Tt3mn23dTdRkbajJLSN8y3QB/l+AMEGkmp W9C/K3mqahaH83lZoLO5VjcPK0UjHqfr8DgoVLpalDX5Ry9bkhqXy4wgRPDBvwVwnbBc0Z AsK1J7vUqg/upUJE5urur7oDn1KUX9w= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453898; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1ZzH6tbwU1tnRJepeYyskBXA12nUW4vAOqU+UVM33TM=; b=5m+yPv/lWsi+egcq3Em7jcgcgYbpRnbmxv96eRAtEJNj+2QaoLtYcyOSZvb/vFQmfREgrA w6av9j2S1RdqbLDQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id C5E7B139D2; Wed, 4 Sep 2024 12:44:57 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id iJu+IklW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:57 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 22/34] migration/multifd: Don't send ram data during SYNC Date: Wed, 4 Sep 2024 09:44:05 -0300 Message-Id: <20240904124417.14565-23-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: ECEB22119F X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCVD_TLS_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[3]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:mid,suse.de:email] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Skip saving and loading any ram data in the packet in the case of a SYNC. This fixes a shortcoming of the current code which requires a reset of the MultiFDPages_t fields right after the previous pending_job finishes, otherwise the very next job might be a SYNC and multifd_send_fill_packet() will put the stale values in the packet. By not calling multifd_ram_fill_packet(), we can stop resetting MultiFDPages_t in the multifd core and leave that to the client code. Actually moving the reset function is not yet done because pages->num==0 is used by the client code to determine whether the MultiFDPages_t needs to be flushed. The subsequent patches will replace that with a generic flag that is not dependent on MultiFDPages_t. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index d64fcdf4ac..3a164c124d 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -452,6 +452,7 @@ void multifd_send_fill_packet(MultiFDSendParams *p) { MultiFDPacket_t *packet = p->packet; uint64_t packet_num; + bool sync_packet = p->flags & MULTIFD_FLAG_SYNC; memset(packet, 0, p->packet_len); @@ -466,7 +467,9 @@ void multifd_send_fill_packet(MultiFDSendParams *p) p->packets_sent++; - multifd_ram_fill_packet(p); + if (!sync_packet) { + multifd_ram_fill_packet(p); + } trace_multifd_send_fill(p->id, packet_num, p->flags, p->next_packet_size); @@ -574,7 +577,9 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) p->packet_num = be64_to_cpu(packet->packet_num); p->packets_recved++; - ret = multifd_ram_unfill_packet(p, errp); + if (!(p->flags & MULTIFD_FLAG_SYNC)) { + ret = multifd_ram_unfill_packet(p, errp); + } trace_multifd_recv_unfill(p->id, p->packet_num, p->flags, p->next_packet_size); @@ -1536,7 +1541,9 @@ static void *multifd_recv_thread(void *opaque) flags = p->flags; /* recv methods don't know how to handle the SYNC flag */ p->flags &= ~MULTIFD_FLAG_SYNC; - has_data = p->normal_num || p->zero_num; + if (!(flags & MULTIFD_FLAG_SYNC)) { + has_data = p->normal_num || p->zero_num; + } qemu_mutex_unlock(&p->mutex); } else { /* From patchwork Wed Sep 4 12:44:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790786 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 76BBECD37B4 for ; Wed, 4 Sep 2024 12:48:14 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNg-0005R0-MT; Wed, 04 Sep 2024 08:45:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNg-0005Oc-1N for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:04 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNd-0005eR-TF for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:03 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 88096219AE; Wed, 4 Sep 2024 12:45:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453900; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5ku2Yclsnnj7YP7PuEDeeWjtItuKjNrIIAnBEqjVWMI=; b=HR4v3PxkH0s8+eS21lczadfjUDOhktr3ESG+wkxyRj5uZdBI4h96K7OtBEjooDbR3RYB7f 2qXxDcFAvVkzSFsqfNJ4vL1MZPa66b28Vrxe+cQ8lVT4ahWX3IUkqMesocBfUjCMEakdy0 4fv8kj20UrdV08WydIQ6Y6bQKMc3WRQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453900; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5ku2Yclsnnj7YP7PuEDeeWjtItuKjNrIIAnBEqjVWMI=; b=WJpMTK51+co9ql/V+3o0AE8ChjTQnOVA2LspLb8+1nvNmf0w/uvvgFLGVB2OjzZz2FWiY4 S9If3oOwt53FVMCg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453900; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5ku2Yclsnnj7YP7PuEDeeWjtItuKjNrIIAnBEqjVWMI=; b=HR4v3PxkH0s8+eS21lczadfjUDOhktr3ESG+wkxyRj5uZdBI4h96K7OtBEjooDbR3RYB7f 2qXxDcFAvVkzSFsqfNJ4vL1MZPa66b28Vrxe+cQ8lVT4ahWX3IUkqMesocBfUjCMEakdy0 4fv8kj20UrdV08WydIQ6Y6bQKMc3WRQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453900; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5ku2Yclsnnj7YP7PuEDeeWjtItuKjNrIIAnBEqjVWMI=; b=WJpMTK51+co9ql/V+3o0AE8ChjTQnOVA2LspLb8+1nvNmf0w/uvvgFLGVB2OjzZz2FWiY4 S9If3oOwt53FVMCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 62FFD139D2; Wed, 4 Sep 2024 12:44:59 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0PCsCktW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:44:59 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 23/34] migration/multifd: Replace multifd_send_state->pages with client data Date: Wed, 4 Sep 2024 09:44:06 -0300 Message-Id: <20240904124417.14565-24-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.79 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.19)[-0.935]; MIME_GOOD(-0.10)[text/plain]; FUZZY_BLOCKED(0.00)[rspamd.com]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:email] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Multifd currently has a simple scheduling mechanism that distributes work to the various channels by keeping storage space within each channel and an extra space that is given to the client. Each time the client fills the space with data and calls into multifd, that space is given to the next idle channel and a free storage space is taken from the channel and given to client for the next iteration. This means we always need (#multifd_channels + 1) memory slots to operate multifd. This is fine, except that the presence of this one extra memory slot doesn't allow different types of payloads to be processed at the same time in different channels, i.e. the data type of multifd_send_state->pages needs to be the same as p->pages. For each new data type different from MultiFDPage_t that is to be handled, this logic would need to be duplicated by adding new fields to multifd_send_state, to the channels and to multifd_send_pages(). Fix this situation by moving the extra slot into the client and using only the generic type MultiFDSendData in the multifd core. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.c | 79 ++++++++++++++++++++++++++------------------- migration/multifd.h | 3 ++ migration/ram.c | 2 ++ 3 files changed, 50 insertions(+), 34 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 3a164c124d..cb7a121eb0 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -49,7 +49,6 @@ typedef struct { struct { MultiFDSendParams *params; - MultiFDSendData *data; /* * Global number of generated multifd packets. * @@ -97,6 +96,8 @@ struct { MultiFDMethods *ops; } *multifd_recv_state; +static MultiFDSendData *multifd_ram_send; + static size_t multifd_ram_payload_size(void) { uint32_t n = multifd_ram_page_count(); @@ -130,6 +131,17 @@ static MultiFDSendData *multifd_send_data_alloc(void) return g_malloc0(size_minus_payload + max_payload_size); } +void multifd_ram_save_setup(void) +{ + multifd_ram_send = multifd_send_data_alloc(); +} + +void multifd_ram_save_cleanup(void) +{ + g_free(multifd_ram_send); + multifd_ram_send = NULL; +} + static bool multifd_use_packets(void) { return !migrate_mapped_ram(); @@ -610,25 +622,20 @@ static void multifd_send_kick_main(MultiFDSendParams *p) } /* - * How we use multifd_send_state->pages and channel->pages? + * multifd_send() works by exchanging the MultiFDSendData object + * provided by the caller with an unused MultiFDSendData object from + * the next channel that is found to be idle. * - * We create a pages for each channel, and a main one. Each time that - * we need to send a batch of pages we interchange the ones between - * multifd_send_state and the channel that is sending it. There are - * two reasons for that: - * - to not have to do so many mallocs during migration - * - to make easier to know what to free at the end of migration + * The channel owns the data until it finishes transmitting and the + * caller owns the empty object until it fills it with data and calls + * this function again. No locking necessary. * - * This way we always know who is the owner of each "pages" struct, - * and we don't need any locking. It belongs to the migration thread - * or to the channel thread. Switching is safe because the migration - * thread is using the channel mutex when changing it, and the channel - * have to had finish with its own, otherwise pending_job can't be - * false. + * Switching is safe because both the migration thread and the channel + * thread have barriers in place to serialize access. * * Returns true if succeed, false otherwise. */ -static bool multifd_send_pages(void) +static bool multifd_send(MultiFDSendData **send_data) { int i; static int next_channel; @@ -669,11 +676,16 @@ static bool multifd_send_pages(void) */ smp_mb_acquire(); - assert(!p->data->u.ram.num); + assert(multifd_payload_empty(p->data)); - tmp = multifd_send_state->data; - multifd_send_state->data = p->data; + /* + * Swap the pointers. The channel gets the client data for + * transferring and the client gets back an unused data slot. + */ + tmp = *send_data; + *send_data = p->data; p->data = tmp; + /* * Making sure p->data is setup before marking pending_job=true. Pairs * with the qatomic_load_acquire() in multifd_send_thread(). @@ -705,7 +717,12 @@ bool multifd_queue_page(RAMBlock *block, ram_addr_t offset) MultiFDPages_t *pages; retry: - pages = &multifd_send_state->data->u.ram; + pages = &multifd_ram_send->u.ram; + + if (multifd_payload_empty(multifd_ram_send)) { + multifd_pages_reset(pages); + multifd_set_payload_type(multifd_ram_send, MULTIFD_PAYLOAD_RAM); + } /* If the queue is empty, we can already enqueue now */ if (multifd_queue_empty(pages)) { @@ -723,7 +740,7 @@ retry: * After flush, always retry. */ if (pages->block != block || multifd_queue_full(pages)) { - if (!multifd_send_pages()) { + if (!multifd_send(&multifd_ram_send)) { return false; } goto retry; @@ -853,8 +870,6 @@ static void multifd_send_cleanup_state(void) qemu_sem_destroy(&multifd_send_state->channels_ready); g_free(multifd_send_state->params); multifd_send_state->params = NULL; - g_free(multifd_send_state->data); - multifd_send_state->data = NULL; g_free(multifd_send_state); multifd_send_state = NULL; } @@ -903,15 +918,14 @@ int multifd_send_sync_main(void) { int i; bool flush_zero_copy; - MultiFDPages_t *pages; if (!migrate_multifd()) { return 0; } - pages = &multifd_send_state->data->u.ram; - if (pages->num) { - if (!multifd_send_pages()) { - error_report("%s: multifd_send_pages fail", __func__); + + if (!multifd_payload_empty(multifd_ram_send)) { + if (!multifd_send(&multifd_ram_send)) { + error_report("%s: multifd_send fail", __func__); return -1; } } @@ -985,13 +999,11 @@ static void *multifd_send_thread(void *opaque) /* * Read pending_job flag before p->data. Pairs with the - * qatomic_store_release() in multifd_send_pages(). + * qatomic_store_release() in multifd_send(). */ if (qatomic_load_acquire(&p->pending_job)) { - MultiFDPages_t *pages = &p->data->u.ram; - p->iovs_num = 0; - assert(pages->num); + assert(!multifd_payload_empty(p->data)); ret = multifd_send_state->ops->send_prepare(p, &local_err); if (ret != 0) { @@ -1014,13 +1026,13 @@ static void *multifd_send_thread(void *opaque) stat64_add(&mig_stats.multifd_bytes, p->next_packet_size + p->packet_len); - multifd_pages_reset(pages); p->next_packet_size = 0; + multifd_set_payload_type(p->data, MULTIFD_PAYLOAD_NONE); /* * Making sure p->data is published before saying "we're * free". Pairs with the smp_mb_acquire() in - * multifd_send_pages(). + * multifd_send(). */ qatomic_store_release(&p->pending_job, false); } else { @@ -1212,7 +1224,6 @@ bool multifd_send_setup(void) thread_count = migrate_multifd_channels(); multifd_send_state = g_malloc0(sizeof(*multifd_send_state)); multifd_send_state->params = g_new0(MultiFDSendParams, thread_count); - multifd_send_state->data = multifd_send_data_alloc(); qemu_sem_init(&multifd_send_state->channels_created, 0); qemu_sem_init(&multifd_send_state->channels_ready, 0); qatomic_set(&multifd_send_state->exiting, 0); diff --git a/migration/multifd.h b/migration/multifd.h index 9175104aea..5fa384d9af 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -267,4 +267,7 @@ static inline uint32_t multifd_ram_page_count(void) { return MULTIFD_PACKET_SIZE / qemu_target_page_size(); } + +void multifd_ram_save_setup(void); +void multifd_ram_save_cleanup(void); #endif diff --git a/migration/ram.c b/migration/ram.c index edec1a2d07..1815b2557b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2387,6 +2387,7 @@ static void ram_save_cleanup(void *opaque) ram_bitmaps_destroy(); xbzrle_cleanup(); + multifd_ram_save_cleanup(); ram_state_cleanup(rsp); g_free(migration_ops); migration_ops = NULL; @@ -3058,6 +3059,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque, Error **errp) migration_ops = g_malloc0(sizeof(MigrationOps)); if (migrate_multifd()) { + multifd_ram_save_setup(); migration_ops->ram_save_target_page = ram_save_target_page_multifd; } else { migration_ops->ram_save_target_page = ram_save_target_page_legacy; From patchwork Wed Sep 4 12:44:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790796 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E28EACD484A for ; Wed, 4 Sep 2024 12:48:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNi-0005YX-Qj; Wed, 04 Sep 2024 08:45:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNh-0005TK-CA for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:05 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNf-0005fC-GR for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:05 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 249902119F; Wed, 4 Sep 2024 12:45:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453902; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Jr+HOGkMYcWF2YiEqUGvJYvmhhFs2+ydbQignoNGMA=; b=XMff969g6afOWmqBsjWeDZFty1xZKxBNkjwTFYtJ0nS6JrpMLfTIijVsEFa5x1F9kfRijy G5U3H19pDryumjlaIDh3TshascAva/A8CetXS/Ky6s7UD9SUftjSJVERSM9pyhknj7swJo vhFzXUX8DfPVIJYf1dh0CwE1v1k7n30= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453902; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Jr+HOGkMYcWF2YiEqUGvJYvmhhFs2+ydbQignoNGMA=; b=rbauHAKQC3L24MXik1jtczOUqdQ5rWJIWRg5RNjCTbJjNTx/+TOQL0oye9k5OV6YHWzqUX whctu73vKP48oMAg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453902; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Jr+HOGkMYcWF2YiEqUGvJYvmhhFs2+ydbQignoNGMA=; b=XMff969g6afOWmqBsjWeDZFty1xZKxBNkjwTFYtJ0nS6JrpMLfTIijVsEFa5x1F9kfRijy G5U3H19pDryumjlaIDh3TshascAva/A8CetXS/Ky6s7UD9SUftjSJVERSM9pyhknj7swJo vhFzXUX8DfPVIJYf1dh0CwE1v1k7n30= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453902; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Jr+HOGkMYcWF2YiEqUGvJYvmhhFs2+ydbQignoNGMA=; b=rbauHAKQC3L24MXik1jtczOUqdQ5rWJIWRg5RNjCTbJjNTx/+TOQL0oye9k5OV6YHWzqUX whctu73vKP48oMAg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id F2ED9139D2; Wed, 4 Sep 2024 12:45:00 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SLnaLUxW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:45:00 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 24/34] migration/multifd: Allow multifd sync without flush Date: Wed, 4 Sep 2024 09:44:07 -0300 Message-Id: <20240904124417.14565-25-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.79 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.19)[-0.933]; MIME_GOOD(-0.10)[text/plain]; FUZZY_BLOCKED(0.00)[rspamd.com]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:mid] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Separate the multifd sync from flushing the client data to the channels. These two operations are closely related but not strictly necessary to be executed together. The multifd sync is intrinsic to how multifd works. The multiple channels operate independently and may finish IO out of order in relation to each other. This applies also between the source and destination QEMU. Flushing the data that is left in the client-owned data structures (e.g. MultiFDPages_t) prior to sync is usually the right thing to do, but that is particular to how the ram migration is implemented with several passes over dirty data. Make these two routines separate, allowing future code to call the sync by itself if needed. This also allows the usage of multifd_ram_send to be isolated to ram code. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.c | 13 +++++++++---- migration/multifd.h | 1 + migration/ram.c | 8 ++++---- 3 files changed, 14 insertions(+), 8 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index cb7a121eb0..ce08257706 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -914,11 +914,8 @@ static int multifd_zero_copy_flush(QIOChannel *c) return ret; } -int multifd_send_sync_main(void) +int multifd_ram_flush_and_sync(void) { - int i; - bool flush_zero_copy; - if (!migrate_multifd()) { return 0; } @@ -930,6 +927,14 @@ int multifd_send_sync_main(void) } } + return multifd_send_sync_main(); +} + +int multifd_send_sync_main(void) +{ + int i; + bool flush_zero_copy; + flush_zero_copy = migrate_zero_copy_send(); for (i = 0; i < migrate_multifd_channels(); i++) { diff --git a/migration/multifd.h b/migration/multifd.h index 5fa384d9af..00c872dfda 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -270,4 +270,5 @@ static inline uint32_t multifd_ram_page_count(void) void multifd_ram_save_setup(void); void multifd_ram_save_cleanup(void); +int multifd_ram_flush_and_sync(void); #endif diff --git a/migration/ram.c b/migration/ram.c index 1815b2557b..67ca3d5d51 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1326,7 +1326,7 @@ static int find_dirty_block(RAMState *rs, PageSearchStatus *pss) (!migrate_multifd_flush_after_each_section() || migrate_mapped_ram())) { QEMUFile *f = rs->pss[RAM_CHANNEL_PRECOPY].pss_channel; - int ret = multifd_send_sync_main(); + int ret = multifd_ram_flush_and_sync(); if (ret < 0) { return ret; } @@ -3066,7 +3066,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque, Error **errp) } bql_unlock(); - ret = multifd_send_sync_main(); + ret = multifd_ram_flush_and_sync(); bql_lock(); if (ret < 0) { error_setg(errp, "%s: multifd synchronization failed", __func__); @@ -3213,7 +3213,7 @@ out: && migration_is_setup_or_active()) { if (migrate_multifd() && migrate_multifd_flush_after_each_section() && !migrate_mapped_ram()) { - ret = multifd_send_sync_main(); + ret = multifd_ram_flush_and_sync(); if (ret < 0) { return ret; } @@ -3285,7 +3285,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) } } - ret = multifd_send_sync_main(); + ret = multifd_ram_flush_and_sync(); if (ret < 0) { return ret; } From patchwork Wed Sep 4 12:44:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 80913CD3431 for ; Wed, 4 Sep 2024 12:49:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNl-0005gM-7H; Wed, 04 Sep 2024 08:45:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNk-0005dK-3m for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:08 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNh-0005nm-36 for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:07 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B4F74219FF; Wed, 4 Sep 2024 12:45:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453903; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3N/cbFy0MXlrNeVqrxMB3+ZmXakOnSm1XJUUwoNWCgA=; b=uY88279U6+4+YTWuFsm+1ZZ8XszLQXQz8Sk1lpa6mzksVYfLCzXbdcQ0IwKPpSiJlwULen WLCh2KGCJhkI1gD1T+dnEBgGHULbiwgb7C+AjRpEkziY+KnVOgFw0Vb/3KuGlwCjfrIvH5 p4AdVdIm6Go6qQmVxpdgKjvD26HsUmY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453903; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3N/cbFy0MXlrNeVqrxMB3+ZmXakOnSm1XJUUwoNWCgA=; b=mX16B1to8im6PZ0+uGC9aszx5HnI+AlboA/8+M8f+Gd3ChLJXK+jwfUAjfD0KgOqDAU1IB MXRSdlrJALFyb+Cw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=uY88279U; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=mX16B1to DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453903; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3N/cbFy0MXlrNeVqrxMB3+ZmXakOnSm1XJUUwoNWCgA=; b=uY88279U6+4+YTWuFsm+1ZZ8XszLQXQz8Sk1lpa6mzksVYfLCzXbdcQ0IwKPpSiJlwULen WLCh2KGCJhkI1gD1T+dnEBgGHULbiwgb7C+AjRpEkziY+KnVOgFw0Vb/3KuGlwCjfrIvH5 p4AdVdIm6Go6qQmVxpdgKjvD26HsUmY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453903; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3N/cbFy0MXlrNeVqrxMB3+ZmXakOnSm1XJUUwoNWCgA=; b=mX16B1to8im6PZ0+uGC9aszx5HnI+AlboA/8+M8f+Gd3ChLJXK+jwfUAjfD0KgOqDAU1IB MXRSdlrJALFyb+Cw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8ED8C139D2; Wed, 4 Sep 2024 12:45:02 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id EJ9oFU5W2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:45:02 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 25/34] migration/multifd: Standardize on multifd ops names Date: Wed, 4 Sep 2024 09:44:08 -0300 Message-Id: <20240904124417.14565-26-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: B4F74219FF X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:mid,suse.de:email]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add the multifd_ prefix to all functions and remove the useless docstrings. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-qpl.c | 57 ---------------------------- migration/multifd-uadk.c | 55 --------------------------- migration/multifd-zlib.c | 81 ++++++---------------------------------- migration/multifd-zstd.c | 81 ++++++---------------------------------- migration/multifd.c | 78 ++++++-------------------------------- 5 files changed, 36 insertions(+), 316 deletions(-) diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c index 21153f1987..75041a4c4d 100644 --- a/migration/multifd-qpl.c +++ b/migration/multifd-qpl.c @@ -220,16 +220,6 @@ static void multifd_qpl_deinit(QplData *qpl) } } -/** - * multifd_qpl_send_setup: set up send side - * - * Set up the channel with QPL compression. - * - * Returns 0 on success or -1 on error - * - * @p: Params for the channel being used - * @errp: pointer to an error - */ static int multifd_qpl_send_setup(MultiFDSendParams *p, Error **errp) { QplData *qpl; @@ -251,14 +241,6 @@ static int multifd_qpl_send_setup(MultiFDSendParams *p, Error **errp) return 0; } -/** - * multifd_qpl_send_cleanup: clean up send side - * - * Close the channel and free memory. - * - * @p: Params for the channel being used - * @errp: pointer to an error - */ static void multifd_qpl_send_cleanup(MultiFDSendParams *p, Error **errp) { multifd_qpl_deinit(p->compress_data); @@ -487,17 +469,6 @@ static void multifd_qpl_compress_pages(MultiFDSendParams *p) } } -/** - * multifd_qpl_send_prepare: prepare data to be able to send - * - * Create a compressed buffer with all the pages that we are going to - * send. - * - * Returns 0 on success or -1 on error - * - * @p: Params for the channel being used - * @errp: pointer to an error - */ static int multifd_qpl_send_prepare(MultiFDSendParams *p, Error **errp) { QplData *qpl = p->compress_data; @@ -523,16 +494,6 @@ out: return 0; } -/** - * multifd_qpl_recv_setup: set up receive side - * - * Create the compressed channel and buffer. - * - * Returns 0 on success or -1 on error - * - * @p: Params for the channel being used - * @errp: pointer to an error - */ static int multifd_qpl_recv_setup(MultiFDRecvParams *p, Error **errp) { QplData *qpl; @@ -547,13 +508,6 @@ static int multifd_qpl_recv_setup(MultiFDRecvParams *p, Error **errp) return 0; } -/** - * multifd_qpl_recv_cleanup: set up receive side - * - * Close the channel and free memory. - * - * @p: Params for the channel being used - */ static void multifd_qpl_recv_cleanup(MultiFDRecvParams *p) { multifd_qpl_deinit(p->compress_data); @@ -694,17 +648,6 @@ static int multifd_qpl_decompress_pages(MultiFDRecvParams *p, Error **errp) } return 0; } -/** - * multifd_qpl_recv: read the data from the channel into actual pages - * - * Read the compressed buffer, and uncompress it into the actual - * pages. - * - * Returns 0 on success or -1 on error - * - * @p: Params for the channel being used - * @errp: pointer to an error - */ static int multifd_qpl_recv(MultiFDRecvParams *p, Error **errp) { QplData *qpl = p->compress_data; diff --git a/migration/multifd-uadk.c b/migration/multifd-uadk.c index 9d99807af5..db2549f59b 100644 --- a/migration/multifd-uadk.c +++ b/migration/multifd-uadk.c @@ -103,14 +103,6 @@ static void multifd_uadk_uninit_sess(struct wd_data *wd) g_free(wd); } -/** - * multifd_uadk_send_setup: setup send side - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ static int multifd_uadk_send_setup(MultiFDSendParams *p, Error **errp) { struct wd_data *wd; @@ -134,14 +126,6 @@ static int multifd_uadk_send_setup(MultiFDSendParams *p, Error **errp) return 0; } -/** - * multifd_uadk_send_cleanup: cleanup send side - * - * Close the channel and return memory. - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ static void multifd_uadk_send_cleanup(MultiFDSendParams *p, Error **errp) { struct wd_data *wd = p->compress_data; @@ -159,17 +143,6 @@ static inline void prepare_next_iov(MultiFDSendParams *p, void *base, p->iovs_num++; } -/** - * multifd_uadk_send_prepare: prepare data to be able to send - * - * Create a compressed buffer with all the pages that we are going to - * send. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ static int multifd_uadk_send_prepare(MultiFDSendParams *p, Error **errp) { struct wd_data *uadk_data = p->compress_data; @@ -229,16 +202,6 @@ out: return 0; } -/** - * multifd_uadk_recv_setup: setup receive side - * - * Create the compressed channel and buffer. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ static int multifd_uadk_recv_setup(MultiFDRecvParams *p, Error **errp) { struct wd_data *wd; @@ -253,13 +216,6 @@ static int multifd_uadk_recv_setup(MultiFDRecvParams *p, Error **errp) return 0; } -/** - * multifd_uadk_recv_cleanup: cleanup receive side - * - * Close the channel and return memory. - * - * @p: Params for the channel that we are using - */ static void multifd_uadk_recv_cleanup(MultiFDRecvParams *p) { struct wd_data *wd = p->compress_data; @@ -268,17 +224,6 @@ static void multifd_uadk_recv_cleanup(MultiFDRecvParams *p) p->compress_data = NULL; } -/** - * multifd_uadk_recv: read the data from the channel into actual pages - * - * Read the compressed buffer, and uncompress it into the actual - * pages. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ static int multifd_uadk_recv(MultiFDRecvParams *p, Error **errp) { struct wd_data *uadk_data = p->compress_data; diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 66517c1067..6787538762 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -34,17 +34,7 @@ struct zlib_data { /* Multifd zlib compression */ -/** - * zlib_send_setup: setup send side - * - * Setup each channel with zlib compression. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static int zlib_send_setup(MultiFDSendParams *p, Error **errp) +static int multifd_zlib_send_setup(MultiFDSendParams *p, Error **errp) { struct zlib_data *z = g_new0(struct zlib_data, 1); z_stream *zs = &z->zs; @@ -86,15 +76,7 @@ err_free_z: return -1; } -/** - * zlib_send_cleanup: cleanup send side - * - * Close the channel and return memory. - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static void zlib_send_cleanup(MultiFDSendParams *p, Error **errp) +static void multifd_zlib_send_cleanup(MultiFDSendParams *p, Error **errp) { struct zlib_data *z = p->compress_data; @@ -110,18 +92,7 @@ static void zlib_send_cleanup(MultiFDSendParams *p, Error **errp) p->iov = NULL; } -/** - * zlib_send_prepare: prepare date to be able to send - * - * Create a compressed buffer with all the pages that we are going to - * send. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static int zlib_send_prepare(MultiFDSendParams *p, Error **errp) +static int multifd_zlib_send_prepare(MultiFDSendParams *p, Error **errp) { MultiFDPages_t *pages = &p->data->u.ram; struct zlib_data *z = p->compress_data; @@ -189,17 +160,7 @@ out: return 0; } -/** - * zlib_recv_setup: setup receive side - * - * Create the compressed channel and buffer. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static int zlib_recv_setup(MultiFDRecvParams *p, Error **errp) +static int multifd_zlib_recv_setup(MultiFDRecvParams *p, Error **errp) { struct zlib_data *z = g_new0(struct zlib_data, 1); z_stream *zs = &z->zs; @@ -225,14 +186,7 @@ static int zlib_recv_setup(MultiFDRecvParams *p, Error **errp) return 0; } -/** - * zlib_recv_cleanup: setup receive side - * - * For no compression this function does nothing. - * - * @p: Params for the channel that we are using - */ -static void zlib_recv_cleanup(MultiFDRecvParams *p) +static void multifd_zlib_recv_cleanup(MultiFDRecvParams *p) { struct zlib_data *z = p->compress_data; @@ -243,18 +197,7 @@ static void zlib_recv_cleanup(MultiFDRecvParams *p) p->compress_data = NULL; } -/** - * zlib_recv: read the data from the channel into actual pages - * - * Read the compressed buffer, and uncompress it into the actual - * pages. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static int zlib_recv(MultiFDRecvParams *p, Error **errp) +static int multifd_zlib_recv(MultiFDRecvParams *p, Error **errp) { struct zlib_data *z = p->compress_data; z_stream *zs = &z->zs; @@ -335,12 +278,12 @@ static int zlib_recv(MultiFDRecvParams *p, Error **errp) } static MultiFDMethods multifd_zlib_ops = { - .send_setup = zlib_send_setup, - .send_cleanup = zlib_send_cleanup, - .send_prepare = zlib_send_prepare, - .recv_setup = zlib_recv_setup, - .recv_cleanup = zlib_recv_cleanup, - .recv = zlib_recv + .send_setup = multifd_zlib_send_setup, + .send_cleanup = multifd_zlib_send_cleanup, + .send_prepare = multifd_zlib_send_prepare, + .recv_setup = multifd_zlib_recv_setup, + .recv_cleanup = multifd_zlib_recv_cleanup, + .recv = multifd_zlib_recv }; static void multifd_zlib_register(void) diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index 04ac711cf4..1576b1e2ad 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -37,17 +37,7 @@ struct zstd_data { /* Multifd zstd compression */ -/** - * zstd_send_setup: setup send side - * - * Setup each channel with zstd compression. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static int zstd_send_setup(MultiFDSendParams *p, Error **errp) +static int multifd_zstd_send_setup(MultiFDSendParams *p, Error **errp) { struct zstd_data *z = g_new0(struct zstd_data, 1); int res; @@ -83,15 +73,7 @@ static int zstd_send_setup(MultiFDSendParams *p, Error **errp) return 0; } -/** - * zstd_send_cleanup: cleanup send side - * - * Close the channel and return memory. - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static void zstd_send_cleanup(MultiFDSendParams *p, Error **errp) +static void multifd_zstd_send_cleanup(MultiFDSendParams *p, Error **errp) { struct zstd_data *z = p->compress_data; @@ -106,18 +88,7 @@ static void zstd_send_cleanup(MultiFDSendParams *p, Error **errp) p->iov = NULL; } -/** - * zstd_send_prepare: prepare date to be able to send - * - * Create a compressed buffer with all the pages that we are going to - * send. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static int zstd_send_prepare(MultiFDSendParams *p, Error **errp) +static int multifd_zstd_send_prepare(MultiFDSendParams *p, Error **errp) { MultiFDPages_t *pages = &p->data->u.ram; struct zstd_data *z = p->compress_data; @@ -176,17 +147,7 @@ out: return 0; } -/** - * zstd_recv_setup: setup receive side - * - * Create the compressed channel and buffer. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static int zstd_recv_setup(MultiFDRecvParams *p, Error **errp) +static int multifd_zstd_recv_setup(MultiFDRecvParams *p, Error **errp) { struct zstd_data *z = g_new0(struct zstd_data, 1); int ret; @@ -220,14 +181,7 @@ static int zstd_recv_setup(MultiFDRecvParams *p, Error **errp) return 0; } -/** - * zstd_recv_cleanup: setup receive side - * - * For no compression this function does nothing. - * - * @p: Params for the channel that we are using - */ -static void zstd_recv_cleanup(MultiFDRecvParams *p) +static void multifd_zstd_recv_cleanup(MultiFDRecvParams *p) { struct zstd_data *z = p->compress_data; @@ -239,18 +193,7 @@ static void zstd_recv_cleanup(MultiFDRecvParams *p) p->compress_data = NULL; } -/** - * zstd_recv: read the data from the channel into actual pages - * - * Read the compressed buffer, and uncompress it into the actual - * pages. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static int zstd_recv(MultiFDRecvParams *p, Error **errp) +static int multifd_zstd_recv(MultiFDRecvParams *p, Error **errp) { uint32_t in_size = p->next_packet_size; uint32_t out_size = 0; @@ -323,12 +266,12 @@ static int zstd_recv(MultiFDRecvParams *p, Error **errp) } static MultiFDMethods multifd_zstd_ops = { - .send_setup = zstd_send_setup, - .send_cleanup = zstd_send_cleanup, - .send_prepare = zstd_send_prepare, - .recv_setup = zstd_recv_setup, - .recv_cleanup = zstd_recv_cleanup, - .recv = zstd_recv + .send_setup = multifd_zstd_send_setup, + .send_cleanup = multifd_zstd_send_cleanup, + .send_prepare = multifd_zstd_send_prepare, + .recv_setup = multifd_zstd_recv_setup, + .recv_cleanup = multifd_zstd_recv_cleanup, + .recv = multifd_zstd_recv }; static void multifd_zstd_register(void) diff --git a/migration/multifd.c b/migration/multifd.c index ce08257706..9f40bb2f16 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -167,15 +167,7 @@ static void multifd_set_file_bitmap(MultiFDSendParams *p) } } -/* Multifd without compression */ - -/** - * nocomp_send_setup: setup send side - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static int nocomp_send_setup(MultiFDSendParams *p, Error **errp) +static int multifd_nocomp_send_setup(MultiFDSendParams *p, Error **errp) { uint32_t page_count = multifd_ram_page_count(); @@ -193,15 +185,7 @@ static int nocomp_send_setup(MultiFDSendParams *p, Error **errp) return 0; } -/** - * nocomp_send_cleanup: cleanup send side - * - * For no compression this function does nothing. - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static void nocomp_send_cleanup(MultiFDSendParams *p, Error **errp) +static void multifd_nocomp_send_cleanup(MultiFDSendParams *p, Error **errp) { g_free(p->iov); p->iov = NULL; @@ -222,18 +206,7 @@ static void multifd_send_prepare_iovs(MultiFDSendParams *p) p->next_packet_size = pages->normal_num * page_size; } -/** - * nocomp_send_prepare: prepare date to be able to send - * - * For no compression we just have to calculate the size of the - * packet. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static int nocomp_send_prepare(MultiFDSendParams *p, Error **errp) +static int multifd_nocomp_send_prepare(MultiFDSendParams *p, Error **errp) { bool use_zero_copy_send = migrate_zero_copy_send(); int ret; @@ -272,46 +245,19 @@ static int nocomp_send_prepare(MultiFDSendParams *p, Error **errp) return 0; } -/** - * nocomp_recv_setup: setup receive side - * - * For no compression this function does nothing. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static int nocomp_recv_setup(MultiFDRecvParams *p, Error **errp) +static int multifd_nocomp_recv_setup(MultiFDRecvParams *p, Error **errp) { p->iov = g_new0(struct iovec, multifd_ram_page_count()); return 0; } -/** - * nocomp_recv_cleanup: setup receive side - * - * For no compression this function does nothing. - * - * @p: Params for the channel that we are using - */ -static void nocomp_recv_cleanup(MultiFDRecvParams *p) +static void multifd_nocomp_recv_cleanup(MultiFDRecvParams *p) { g_free(p->iov); p->iov = NULL; } -/** - * nocomp_recv: read the data from the channel - * - * For no compression we just need to read things into the correct place. - * - * Returns 0 for success or -1 for error - * - * @p: Params for the channel that we are using - * @errp: pointer to an error - */ -static int nocomp_recv(MultiFDRecvParams *p, Error **errp) +static int multifd_nocomp_recv(MultiFDRecvParams *p, Error **errp) { uint32_t flags; @@ -342,12 +288,12 @@ static int nocomp_recv(MultiFDRecvParams *p, Error **errp) } static MultiFDMethods multifd_nocomp_ops = { - .send_setup = nocomp_send_setup, - .send_cleanup = nocomp_send_cleanup, - .send_prepare = nocomp_send_prepare, - .recv_setup = nocomp_recv_setup, - .recv_cleanup = nocomp_recv_cleanup, - .recv = nocomp_recv + .send_setup = multifd_nocomp_send_setup, + .send_cleanup = multifd_nocomp_send_cleanup, + .send_prepare = multifd_nocomp_send_prepare, + .recv_setup = multifd_nocomp_recv_setup, + .recv_cleanup = multifd_nocomp_recv_cleanup, + .recv = multifd_nocomp_recv }; static MultiFDMethods *multifd_ops[MULTIFD_COMPRESSION__MAX] = { From patchwork Wed Sep 4 12:44:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BD3C5CD4849 for ; Wed, 4 Sep 2024 12:46:39 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpNm-0005o4-Gw; Wed, 04 Sep 2024 08:45:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNl-0005gf-24 for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:09 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNj-0005rN-8Y for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:08 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 89FA42191F; Wed, 4 Sep 2024 12:45:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453905; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7HyhEyYY8UmrP9bOy+Hm1xugaK/bChxs6lv5yF3HDyY=; b=h29JHPy+6707ESlOZXNvs+xQ5SnLW4sL7U3jgY6n1XAO/+Si+f2NWKeZejUSrBlbP8SxGW 9WgHNXn/CsvyoEFJekmytRLDrOOFPnyi3XGfsbGDkZZL429O5o8D+ZjgZQd0C2Jqdu2/za 2uXf8d14GpOqB1GpFcuKi0XzsJauCP8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453905; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7HyhEyYY8UmrP9bOy+Hm1xugaK/bChxs6lv5yF3HDyY=; b=dWj8of+qQ2ld3UbBLs57gy2fnCm9P2Mnpp4LKGopdz8steUyeVAHZ0P0u1iLdmt5b/WSRG c4B/uu184R+RzBAQ== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=h29JHPy+; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=dWj8of+q DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453905; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7HyhEyYY8UmrP9bOy+Hm1xugaK/bChxs6lv5yF3HDyY=; b=h29JHPy+6707ESlOZXNvs+xQ5SnLW4sL7U3jgY6n1XAO/+Si+f2NWKeZejUSrBlbP8SxGW 9WgHNXn/CsvyoEFJekmytRLDrOOFPnyi3XGfsbGDkZZL429O5o8D+ZjgZQd0C2Jqdu2/za 2uXf8d14GpOqB1GpFcuKi0XzsJauCP8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453905; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7HyhEyYY8UmrP9bOy+Hm1xugaK/bChxs6lv5yF3HDyY=; b=dWj8of+qQ2ld3UbBLs57gy2fnCm9P2Mnpp4LKGopdz8steUyeVAHZ0P0u1iLdmt5b/WSRG c4B/uu184R+RzBAQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 2C320139D2; Wed, 4 Sep 2024 12:45:03 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QLLKOE9W2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:45:03 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Prasad Pandit Subject: [PULL 26/34] migration/multifd: Register nocomp ops dynamically Date: Wed, 4 Sep 2024 09:44:09 -0300 Message-Id: <20240904124417.14565-27-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 89FA42191F X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:dkim,suse.de:mid]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Prior to moving the ram code into multifd-nocomp.c, change the code to register the nocomp ops dynamically so we don't need to have the ops structure defined in multifd.c. While here, move the ops struct initialization to the end of the file to make the next diff cleaner. Reviewed-by: Prasad Pandit Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.c | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 9f40bb2f16..e100836cbe 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -287,22 +287,12 @@ static int multifd_nocomp_recv(MultiFDRecvParams *p, Error **errp) return qio_channel_readv_all(p->c, p->iov, p->normal_num, errp); } -static MultiFDMethods multifd_nocomp_ops = { - .send_setup = multifd_nocomp_send_setup, - .send_cleanup = multifd_nocomp_send_cleanup, - .send_prepare = multifd_nocomp_send_prepare, - .recv_setup = multifd_nocomp_recv_setup, - .recv_cleanup = multifd_nocomp_recv_cleanup, - .recv = multifd_nocomp_recv -}; - -static MultiFDMethods *multifd_ops[MULTIFD_COMPRESSION__MAX] = { - [MULTIFD_COMPRESSION_NONE] = &multifd_nocomp_ops, -}; +static MultiFDMethods *multifd_ops[MULTIFD_COMPRESSION__MAX] = {}; void multifd_register_ops(int method, MultiFDMethods *ops) { - assert(0 < method && method < MULTIFD_COMPRESSION__MAX); + assert(0 <= method && method < MULTIFD_COMPRESSION__MAX); + assert(!multifd_ops[method]); multifd_ops[method] = ops; } @@ -1701,3 +1691,19 @@ bool multifd_send_prepare_common(MultiFDSendParams *p) return true; } + +static MultiFDMethods multifd_nocomp_ops = { + .send_setup = multifd_nocomp_send_setup, + .send_cleanup = multifd_nocomp_send_cleanup, + .send_prepare = multifd_nocomp_send_prepare, + .recv_setup = multifd_nocomp_recv_setup, + .recv_cleanup = multifd_nocomp_recv_cleanup, + .recv = multifd_nocomp_recv +}; + +static void multifd_nocomp_register(void) +{ + multifd_register_ops(MULTIFD_COMPRESSION_NONE, &multifd_nocomp_ops); +} + +migration_init(multifd_nocomp_register); From patchwork Wed Sep 4 12:44:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4FC37CD37B4 for ; Wed, 4 Sep 2024 12:48:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpOi-0000mO-8X; Wed, 04 Sep 2024 08:46:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNr-00068N-At for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:16 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNk-0005rw-QO for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:12 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 611A51F38E; Wed, 4 Sep 2024 12:45:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453907; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vu3lO01kqO9ruq0I9ow3cIczfDroX1+TjeIk0q8bgLs=; b=s/BFVjc4Ztn+4wyDY44Lt9gwQidS3q2Y/du8U6KzBuq29AsjbeO0mpOY4wVx/LxR5ImdOh +MPa0HoRHbDcJsBydl80aiqLNsk6EA1tPAY/utXrUV1gRApcRJNUIJxwKwIgv2hRhRwkQ9 YQtjUTnOz2OkAdU9By4bR/rRy3/oMpM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453907; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vu3lO01kqO9ruq0I9ow3cIczfDroX1+TjeIk0q8bgLs=; b=WFpE3sflv7bMadmmG6YeTBLmLbVw00aomt3Cz+I/hPZ5/6+J+6Z1s1HA6nRzMnznBxXfh5 YzUeQZ4C6G29HxAQ== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453907; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vu3lO01kqO9ruq0I9ow3cIczfDroX1+TjeIk0q8bgLs=; b=s/BFVjc4Ztn+4wyDY44Lt9gwQidS3q2Y/du8U6KzBuq29AsjbeO0mpOY4wVx/LxR5ImdOh +MPa0HoRHbDcJsBydl80aiqLNsk6EA1tPAY/utXrUV1gRApcRJNUIJxwKwIgv2hRhRwkQ9 YQtjUTnOz2OkAdU9By4bR/rRy3/oMpM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453907; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vu3lO01kqO9ruq0I9ow3cIczfDroX1+TjeIk0q8bgLs=; b=WFpE3sflv7bMadmmG6YeTBLmLbVw00aomt3Cz+I/hPZ5/6+J+6Z1s1HA6nRzMnznBxXfh5 YzUeQZ4C6G29HxAQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 02356139D2; Wed, 4 Sep 2024 12:45:05 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id uPyjLlFW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:45:05 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 27/34] migration/multifd: Move nocomp code into multifd-nocomp.c Date: Wed, 4 Sep 2024 09:44:10 -0300 Message-Id: <20240904124417.14565-28-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.79 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.19)[-0.936]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In preparation for adding new payload types to multifd, move most of the no-compression code into multifd-nocomp.c. Let's try to keep a semblance of layering by not mixing general multifd control flow with the details of transmitting pages of ram. There are still some pieces leftover, namely the p->normal, p->zero, etc variables that we use for zero page tracking and the packet allocation which is heavily dependent on the ram code. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/meson.build | 1 + migration/multifd-nocomp.c | 394 +++++++++++++++++++++++++++++++++++++ migration/multifd.c | 377 +---------------------------------- migration/multifd.h | 5 + 4 files changed, 402 insertions(+), 375 deletions(-) create mode 100644 migration/multifd-nocomp.c diff --git a/migration/meson.build b/migration/meson.build index 5ce2acb41e..77f3abf08e 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -21,6 +21,7 @@ system_ss.add(files( 'migration-hmp-cmds.c', 'migration.c', 'multifd.c', + 'multifd-nocomp.c', 'multifd-zlib.c', 'multifd-zero-page.c', 'options.c', diff --git a/migration/multifd-nocomp.c b/migration/multifd-nocomp.c new file mode 100644 index 0000000000..53ea9f9c83 --- /dev/null +++ b/migration/multifd-nocomp.c @@ -0,0 +1,394 @@ +/* + * Multifd RAM migration without compression + * + * Copyright (c) 2019-2020 Red Hat Inc + * + * Authors: + * Juan Quintela + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "exec/ramblock.h" +#include "exec/target_page.h" +#include "file.h" +#include "multifd.h" +#include "options.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "trace.h" + +static MultiFDSendData *multifd_ram_send; + +size_t multifd_ram_payload_size(void) +{ + uint32_t n = multifd_ram_page_count(); + + /* + * We keep an array of page offsets at the end of MultiFDPages_t, + * add space for it in the allocation. + */ + return sizeof(MultiFDPages_t) + n * sizeof(ram_addr_t); +} + +void multifd_ram_save_setup(void) +{ + multifd_ram_send = multifd_send_data_alloc(); +} + +void multifd_ram_save_cleanup(void) +{ + g_free(multifd_ram_send); + multifd_ram_send = NULL; +} + +static void multifd_set_file_bitmap(MultiFDSendParams *p) +{ + MultiFDPages_t *pages = &p->data->u.ram; + + assert(pages->block); + + for (int i = 0; i < pages->normal_num; i++) { + ramblock_set_file_bmap_atomic(pages->block, pages->offset[i], true); + } + + for (int i = pages->normal_num; i < pages->num; i++) { + ramblock_set_file_bmap_atomic(pages->block, pages->offset[i], false); + } +} + +static int multifd_nocomp_send_setup(MultiFDSendParams *p, Error **errp) +{ + uint32_t page_count = multifd_ram_page_count(); + + if (migrate_zero_copy_send()) { + p->write_flags |= QIO_CHANNEL_WRITE_FLAG_ZERO_COPY; + } + + if (!migrate_mapped_ram()) { + /* We need one extra place for the packet header */ + p->iov = g_new0(struct iovec, page_count + 1); + } else { + p->iov = g_new0(struct iovec, page_count); + } + + return 0; +} + +static void multifd_nocomp_send_cleanup(MultiFDSendParams *p, Error **errp) +{ + g_free(p->iov); + p->iov = NULL; + return; +} + +static void multifd_send_prepare_iovs(MultiFDSendParams *p) +{ + MultiFDPages_t *pages = &p->data->u.ram; + uint32_t page_size = multifd_ram_page_size(); + + for (int i = 0; i < pages->normal_num; i++) { + p->iov[p->iovs_num].iov_base = pages->block->host + pages->offset[i]; + p->iov[p->iovs_num].iov_len = page_size; + p->iovs_num++; + } + + p->next_packet_size = pages->normal_num * page_size; +} + +static int multifd_nocomp_send_prepare(MultiFDSendParams *p, Error **errp) +{ + bool use_zero_copy_send = migrate_zero_copy_send(); + int ret; + + multifd_send_zero_page_detect(p); + + if (migrate_mapped_ram()) { + multifd_send_prepare_iovs(p); + multifd_set_file_bitmap(p); + + return 0; + } + + if (!use_zero_copy_send) { + /* + * Only !zerocopy needs the header in IOV; zerocopy will + * send it separately. + */ + multifd_send_prepare_header(p); + } + + multifd_send_prepare_iovs(p); + p->flags |= MULTIFD_FLAG_NOCOMP; + + multifd_send_fill_packet(p); + + if (use_zero_copy_send) { + /* Send header first, without zerocopy */ + ret = qio_channel_write_all(p->c, (void *)p->packet, + p->packet_len, errp); + if (ret != 0) { + return -1; + } + } + + return 0; +} + +static int multifd_nocomp_recv_setup(MultiFDRecvParams *p, Error **errp) +{ + p->iov = g_new0(struct iovec, multifd_ram_page_count()); + return 0; +} + +static void multifd_nocomp_recv_cleanup(MultiFDRecvParams *p) +{ + g_free(p->iov); + p->iov = NULL; +} + +static int multifd_nocomp_recv(MultiFDRecvParams *p, Error **errp) +{ + uint32_t flags; + + if (migrate_mapped_ram()) { + return multifd_file_recv_data(p, errp); + } + + flags = p->flags & MULTIFD_FLAG_COMPRESSION_MASK; + + if (flags != MULTIFD_FLAG_NOCOMP) { + error_setg(errp, "multifd %u: flags received %x flags expected %x", + p->id, flags, MULTIFD_FLAG_NOCOMP); + return -1; + } + + multifd_recv_zero_page_process(p); + + if (!p->normal_num) { + return 0; + } + + for (int i = 0; i < p->normal_num; i++) { + p->iov[i].iov_base = p->host + p->normal[i]; + p->iov[i].iov_len = multifd_ram_page_size(); + ramblock_recv_bitmap_set_offset(p->block, p->normal[i]); + } + return qio_channel_readv_all(p->c, p->iov, p->normal_num, errp); +} + +static void multifd_pages_reset(MultiFDPages_t *pages) +{ + /* + * We don't need to touch offset[] array, because it will be + * overwritten later when reused. + */ + pages->num = 0; + pages->normal_num = 0; + pages->block = NULL; +} + +void multifd_ram_fill_packet(MultiFDSendParams *p) +{ + MultiFDPacket_t *packet = p->packet; + MultiFDPages_t *pages = &p->data->u.ram; + uint32_t zero_num = pages->num - pages->normal_num; + + packet->pages_alloc = cpu_to_be32(multifd_ram_page_count()); + packet->normal_pages = cpu_to_be32(pages->normal_num); + packet->zero_pages = cpu_to_be32(zero_num); + + if (pages->block) { + strncpy(packet->ramblock, pages->block->idstr, 256); + } + + for (int i = 0; i < pages->num; i++) { + /* there are architectures where ram_addr_t is 32 bit */ + uint64_t temp = pages->offset[i]; + + packet->offset[i] = cpu_to_be64(temp); + } + + trace_multifd_send_ram_fill(p->id, pages->normal_num, + zero_num); +} + +int multifd_ram_unfill_packet(MultiFDRecvParams *p, Error **errp) +{ + MultiFDPacket_t *packet = p->packet; + uint32_t page_count = multifd_ram_page_count(); + uint32_t page_size = multifd_ram_page_size(); + int i; + + packet->pages_alloc = be32_to_cpu(packet->pages_alloc); + /* + * If we received a packet that is 100 times bigger than expected + * just stop migration. It is a magic number. + */ + if (packet->pages_alloc > page_count) { + error_setg(errp, "multifd: received packet " + "with size %u and expected a size of %u", + packet->pages_alloc, page_count) ; + return -1; + } + + p->normal_num = be32_to_cpu(packet->normal_pages); + if (p->normal_num > packet->pages_alloc) { + error_setg(errp, "multifd: received packet " + "with %u normal pages and expected maximum pages are %u", + p->normal_num, packet->pages_alloc) ; + return -1; + } + + p->zero_num = be32_to_cpu(packet->zero_pages); + if (p->zero_num > packet->pages_alloc - p->normal_num) { + error_setg(errp, "multifd: received packet " + "with %u zero pages and expected maximum zero pages are %u", + p->zero_num, packet->pages_alloc - p->normal_num) ; + return -1; + } + + if (p->normal_num == 0 && p->zero_num == 0) { + return 0; + } + + /* make sure that ramblock is 0 terminated */ + packet->ramblock[255] = 0; + p->block = qemu_ram_block_by_name(packet->ramblock); + if (!p->block) { + error_setg(errp, "multifd: unknown ram block %s", + packet->ramblock); + return -1; + } + + p->host = p->block->host; + for (i = 0; i < p->normal_num; i++) { + uint64_t offset = be64_to_cpu(packet->offset[i]); + + if (offset > (p->block->used_length - page_size)) { + error_setg(errp, "multifd: offset too long %" PRIu64 + " (max " RAM_ADDR_FMT ")", + offset, p->block->used_length); + return -1; + } + p->normal[i] = offset; + } + + for (i = 0; i < p->zero_num; i++) { + uint64_t offset = be64_to_cpu(packet->offset[p->normal_num + i]); + + if (offset > (p->block->used_length - page_size)) { + error_setg(errp, "multifd: offset too long %" PRIu64 + " (max " RAM_ADDR_FMT ")", + offset, p->block->used_length); + return -1; + } + p->zero[i] = offset; + } + + return 0; +} + +static inline bool multifd_queue_empty(MultiFDPages_t *pages) +{ + return pages->num == 0; +} + +static inline bool multifd_queue_full(MultiFDPages_t *pages) +{ + return pages->num == multifd_ram_page_count(); +} + +static inline void multifd_enqueue(MultiFDPages_t *pages, ram_addr_t offset) +{ + pages->offset[pages->num++] = offset; +} + +/* Returns true if enqueue successful, false otherwise */ +bool multifd_queue_page(RAMBlock *block, ram_addr_t offset) +{ + MultiFDPages_t *pages; + +retry: + pages = &multifd_ram_send->u.ram; + + if (multifd_payload_empty(multifd_ram_send)) { + multifd_pages_reset(pages); + multifd_set_payload_type(multifd_ram_send, MULTIFD_PAYLOAD_RAM); + } + + /* If the queue is empty, we can already enqueue now */ + if (multifd_queue_empty(pages)) { + pages->block = block; + multifd_enqueue(pages, offset); + return true; + } + + /* + * Not empty, meanwhile we need a flush. It can because of either: + * + * (1) The page is not on the same ramblock of previous ones, or, + * (2) The queue is full. + * + * After flush, always retry. + */ + if (pages->block != block || multifd_queue_full(pages)) { + if (!multifd_send(&multifd_ram_send)) { + return false; + } + goto retry; + } + + /* Not empty, and we still have space, do it! */ + multifd_enqueue(pages, offset); + return true; +} + +int multifd_ram_flush_and_sync(void) +{ + if (!migrate_multifd()) { + return 0; + } + + if (!multifd_payload_empty(multifd_ram_send)) { + if (!multifd_send(&multifd_ram_send)) { + error_report("%s: multifd_send fail", __func__); + return -1; + } + } + + return multifd_send_sync_main(); +} + +bool multifd_send_prepare_common(MultiFDSendParams *p) +{ + MultiFDPages_t *pages = &p->data->u.ram; + multifd_send_zero_page_detect(p); + + if (!pages->normal_num) { + p->next_packet_size = 0; + return false; + } + + multifd_send_prepare_header(p); + + return true; +} + +static MultiFDMethods multifd_nocomp_ops = { + .send_setup = multifd_nocomp_send_setup, + .send_cleanup = multifd_nocomp_send_cleanup, + .send_prepare = multifd_nocomp_send_prepare, + .recv_setup = multifd_nocomp_recv_setup, + .recv_cleanup = multifd_nocomp_recv_cleanup, + .recv = multifd_nocomp_recv +}; + +static void multifd_nocomp_register(void) +{ + multifd_register_ops(MULTIFD_COMPRESSION_NONE, &multifd_nocomp_ops); +} + +migration_init(multifd_nocomp_register); diff --git a/migration/multifd.c b/migration/multifd.c index e100836cbe..0c07a2040b 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -96,20 +96,7 @@ struct { MultiFDMethods *ops; } *multifd_recv_state; -static MultiFDSendData *multifd_ram_send; - -static size_t multifd_ram_payload_size(void) -{ - uint32_t n = multifd_ram_page_count(); - - /* - * We keep an array of page offsets at the end of MultiFDPages_t, - * add space for it in the allocation. - */ - return sizeof(MultiFDPages_t) + n * sizeof(ram_addr_t); -} - -static MultiFDSendData *multifd_send_data_alloc(void) +MultiFDSendData *multifd_send_data_alloc(void) { size_t max_payload_size, size_minus_payload; @@ -131,17 +118,6 @@ static MultiFDSendData *multifd_send_data_alloc(void) return g_malloc0(size_minus_payload + max_payload_size); } -void multifd_ram_save_setup(void) -{ - multifd_ram_send = multifd_send_data_alloc(); -} - -void multifd_ram_save_cleanup(void) -{ - g_free(multifd_ram_send); - multifd_ram_send = NULL; -} - static bool multifd_use_packets(void) { return !migrate_mapped_ram(); @@ -152,141 +128,6 @@ void multifd_send_channel_created(void) qemu_sem_post(&multifd_send_state->channels_created); } -static void multifd_set_file_bitmap(MultiFDSendParams *p) -{ - MultiFDPages_t *pages = &p->data->u.ram; - - assert(pages->block); - - for (int i = 0; i < pages->normal_num; i++) { - ramblock_set_file_bmap_atomic(pages->block, pages->offset[i], true); - } - - for (int i = pages->normal_num; i < pages->num; i++) { - ramblock_set_file_bmap_atomic(pages->block, pages->offset[i], false); - } -} - -static int multifd_nocomp_send_setup(MultiFDSendParams *p, Error **errp) -{ - uint32_t page_count = multifd_ram_page_count(); - - if (migrate_zero_copy_send()) { - p->write_flags |= QIO_CHANNEL_WRITE_FLAG_ZERO_COPY; - } - - if (multifd_use_packets()) { - /* We need one extra place for the packet header */ - p->iov = g_new0(struct iovec, page_count + 1); - } else { - p->iov = g_new0(struct iovec, page_count); - } - - return 0; -} - -static void multifd_nocomp_send_cleanup(MultiFDSendParams *p, Error **errp) -{ - g_free(p->iov); - p->iov = NULL; - return; -} - -static void multifd_send_prepare_iovs(MultiFDSendParams *p) -{ - MultiFDPages_t *pages = &p->data->u.ram; - uint32_t page_size = multifd_ram_page_size(); - - for (int i = 0; i < pages->normal_num; i++) { - p->iov[p->iovs_num].iov_base = pages->block->host + pages->offset[i]; - p->iov[p->iovs_num].iov_len = page_size; - p->iovs_num++; - } - - p->next_packet_size = pages->normal_num * page_size; -} - -static int multifd_nocomp_send_prepare(MultiFDSendParams *p, Error **errp) -{ - bool use_zero_copy_send = migrate_zero_copy_send(); - int ret; - - multifd_send_zero_page_detect(p); - - if (!multifd_use_packets()) { - multifd_send_prepare_iovs(p); - multifd_set_file_bitmap(p); - - return 0; - } - - if (!use_zero_copy_send) { - /* - * Only !zerocopy needs the header in IOV; zerocopy will - * send it separately. - */ - multifd_send_prepare_header(p); - } - - multifd_send_prepare_iovs(p); - p->flags |= MULTIFD_FLAG_NOCOMP; - - multifd_send_fill_packet(p); - - if (use_zero_copy_send) { - /* Send header first, without zerocopy */ - ret = qio_channel_write_all(p->c, (void *)p->packet, - p->packet_len, errp); - if (ret != 0) { - return -1; - } - } - - return 0; -} - -static int multifd_nocomp_recv_setup(MultiFDRecvParams *p, Error **errp) -{ - p->iov = g_new0(struct iovec, multifd_ram_page_count()); - return 0; -} - -static void multifd_nocomp_recv_cleanup(MultiFDRecvParams *p) -{ - g_free(p->iov); - p->iov = NULL; -} - -static int multifd_nocomp_recv(MultiFDRecvParams *p, Error **errp) -{ - uint32_t flags; - - if (!multifd_use_packets()) { - return multifd_file_recv_data(p, errp); - } - - flags = p->flags & MULTIFD_FLAG_COMPRESSION_MASK; - - if (flags != MULTIFD_FLAG_NOCOMP) { - error_setg(errp, "multifd %u: flags received %x flags expected %x", - p->id, flags, MULTIFD_FLAG_NOCOMP); - return -1; - } - - multifd_recv_zero_page_process(p); - - if (!p->normal_num) { - return 0; - } - - for (int i = 0; i < p->normal_num; i++) { - p->iov[i].iov_base = p->host + p->normal[i]; - p->iov[i].iov_len = multifd_ram_page_size(); - ramblock_recv_bitmap_set_offset(p->block, p->normal[i]); - } - return qio_channel_readv_all(p->c, p->iov, p->normal_num, errp); -} - static MultiFDMethods *multifd_ops[MULTIFD_COMPRESSION__MAX] = {}; void multifd_register_ops(int method, MultiFDMethods *ops) @@ -296,18 +137,6 @@ void multifd_register_ops(int method, MultiFDMethods *ops) multifd_ops[method] = ops; } -/* Reset a MultiFDPages_t* object for the next use */ -static void multifd_pages_reset(MultiFDPages_t *pages) -{ - /* - * We don't need to touch offset[] array, because it will be - * overwritten later when reused. - */ - pages->num = 0; - pages->normal_num = 0; - pages->block = NULL; -} - static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) { MultiFDInit_t msg = {}; @@ -372,30 +201,6 @@ static int multifd_recv_initial_packet(QIOChannel *c, Error **errp) return msg.id; } -static void multifd_ram_fill_packet(MultiFDSendParams *p) -{ - MultiFDPacket_t *packet = p->packet; - MultiFDPages_t *pages = &p->data->u.ram; - uint32_t zero_num = pages->num - pages->normal_num; - - packet->pages_alloc = cpu_to_be32(multifd_ram_page_count()); - packet->normal_pages = cpu_to_be32(pages->normal_num); - packet->zero_pages = cpu_to_be32(zero_num); - - if (pages->block) { - strncpy(packet->ramblock, pages->block->idstr, 256); - } - - for (int i = 0; i < pages->num; i++) { - /* there are architectures where ram_addr_t is 32 bit */ - uint64_t temp = pages->offset[i]; - - packet->offset[i] = cpu_to_be64(temp); - } - - trace_multifd_send_ram_fill(p->id, pages->normal_num, zero_num); -} - void multifd_send_fill_packet(MultiFDSendParams *p) { MultiFDPacket_t *packet = p->packet; @@ -423,82 +228,6 @@ void multifd_send_fill_packet(MultiFDSendParams *p) p->flags, p->next_packet_size); } -static int multifd_ram_unfill_packet(MultiFDRecvParams *p, Error **errp) -{ - MultiFDPacket_t *packet = p->packet; - uint32_t page_count = multifd_ram_page_count(); - uint32_t page_size = multifd_ram_page_size(); - int i; - - packet->pages_alloc = be32_to_cpu(packet->pages_alloc); - /* - * If we received a packet that is 100 times bigger than expected - * just stop migration. It is a magic number. - */ - if (packet->pages_alloc > page_count) { - error_setg(errp, "multifd: received packet " - "with size %u and expected a size of %u", - packet->pages_alloc, page_count) ; - return -1; - } - - p->normal_num = be32_to_cpu(packet->normal_pages); - if (p->normal_num > packet->pages_alloc) { - error_setg(errp, "multifd: received packet " - "with %u normal pages and expected maximum pages are %u", - p->normal_num, packet->pages_alloc) ; - return -1; - } - - p->zero_num = be32_to_cpu(packet->zero_pages); - if (p->zero_num > packet->pages_alloc - p->normal_num) { - error_setg(errp, "multifd: received packet " - "with %u zero pages and expected maximum zero pages are %u", - p->zero_num, packet->pages_alloc - p->normal_num) ; - return -1; - } - - if (p->normal_num == 0 && p->zero_num == 0) { - return 0; - } - - /* make sure that ramblock is 0 terminated */ - packet->ramblock[255] = 0; - p->block = qemu_ram_block_by_name(packet->ramblock); - if (!p->block) { - error_setg(errp, "multifd: unknown ram block %s", - packet->ramblock); - return -1; - } - - p->host = p->block->host; - for (i = 0; i < p->normal_num; i++) { - uint64_t offset = be64_to_cpu(packet->offset[i]); - - if (offset > (p->block->used_length - page_size)) { - error_setg(errp, "multifd: offset too long %" PRIu64 - " (max " RAM_ADDR_FMT ")", - offset, p->block->used_length); - return -1; - } - p->normal[i] = offset; - } - - for (i = 0; i < p->zero_num; i++) { - uint64_t offset = be64_to_cpu(packet->offset[p->normal_num + i]); - - if (offset > (p->block->used_length - page_size)) { - error_setg(errp, "multifd: offset too long %" PRIu64 - " (max " RAM_ADDR_FMT ")", - offset, p->block->used_length); - return -1; - } - p->zero[i] = offset; - } - - return 0; -} - static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) { MultiFDPacket_t *packet = p->packet; @@ -571,7 +300,7 @@ static void multifd_send_kick_main(MultiFDSendParams *p) * * Returns true if succeed, false otherwise. */ -static bool multifd_send(MultiFDSendData **send_data) +bool multifd_send(MultiFDSendData **send_data) { int i; static int next_channel; @@ -632,61 +361,6 @@ static bool multifd_send(MultiFDSendData **send_data) return true; } -static inline bool multifd_queue_empty(MultiFDPages_t *pages) -{ - return pages->num == 0; -} - -static inline bool multifd_queue_full(MultiFDPages_t *pages) -{ - return pages->num == multifd_ram_page_count(); -} - -static inline void multifd_enqueue(MultiFDPages_t *pages, ram_addr_t offset) -{ - pages->offset[pages->num++] = offset; -} - -/* Returns true if enqueue successful, false otherwise */ -bool multifd_queue_page(RAMBlock *block, ram_addr_t offset) -{ - MultiFDPages_t *pages; - -retry: - pages = &multifd_ram_send->u.ram; - - if (multifd_payload_empty(multifd_ram_send)) { - multifd_pages_reset(pages); - multifd_set_payload_type(multifd_ram_send, MULTIFD_PAYLOAD_RAM); - } - - /* If the queue is empty, we can already enqueue now */ - if (multifd_queue_empty(pages)) { - pages->block = block; - multifd_enqueue(pages, offset); - return true; - } - - /* - * Not empty, meanwhile we need a flush. It can because of either: - * - * (1) The page is not on the same ramblock of previous ones, or, - * (2) The queue is full. - * - * After flush, always retry. - */ - if (pages->block != block || multifd_queue_full(pages)) { - if (!multifd_send(&multifd_ram_send)) { - return false; - } - goto retry; - } - - /* Not empty, and we still have space, do it! */ - multifd_enqueue(pages, offset); - return true; -} - /* Multifd send side hit an error; remember it and prepare to quit */ static void multifd_send_set_error(Error *err) { @@ -850,22 +524,6 @@ static int multifd_zero_copy_flush(QIOChannel *c) return ret; } -int multifd_ram_flush_and_sync(void) -{ - if (!migrate_multifd()) { - return 0; - } - - if (!multifd_payload_empty(multifd_ram_send)) { - if (!multifd_send(&multifd_ram_send)) { - error_report("%s: multifd_send fail", __func__); - return -1; - } - } - - return multifd_send_sync_main(); -} - int multifd_send_sync_main(void) { int i; @@ -1676,34 +1334,3 @@ void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) QEMU_THREAD_JOINABLE); qatomic_inc(&multifd_recv_state->count); } - -bool multifd_send_prepare_common(MultiFDSendParams *p) -{ - MultiFDPages_t *pages = &p->data->u.ram; - multifd_send_zero_page_detect(p); - - if (!pages->normal_num) { - p->next_packet_size = 0; - return false; - } - - multifd_send_prepare_header(p); - - return true; -} - -static MultiFDMethods multifd_nocomp_ops = { - .send_setup = multifd_nocomp_send_setup, - .send_cleanup = multifd_nocomp_send_cleanup, - .send_prepare = multifd_nocomp_send_prepare, - .recv_setup = multifd_nocomp_recv_setup, - .recv_cleanup = multifd_nocomp_recv_cleanup, - .recv = multifd_nocomp_recv -}; - -static void multifd_nocomp_register(void) -{ - multifd_register_ops(MULTIFD_COMPRESSION_NONE, &multifd_nocomp_ops); -} - -migration_init(multifd_nocomp_register); diff --git a/migration/multifd.h b/migration/multifd.h index 00c872dfda..a3e35196d1 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -257,6 +257,8 @@ static inline void multifd_send_prepare_header(MultiFDSendParams *p) } void multifd_channel_connect(MultiFDSendParams *p, QIOChannel *ioc); +bool multifd_send(MultiFDSendData **send_data); +MultiFDSendData *multifd_send_data_alloc(void); static inline uint32_t multifd_ram_page_size(void) { @@ -271,4 +273,7 @@ static inline uint32_t multifd_ram_page_count(void) void multifd_ram_save_setup(void); void multifd_ram_save_cleanup(void); int multifd_ram_flush_and_sync(void); +size_t multifd_ram_payload_size(void); +void multifd_ram_fill_packet(MultiFDSendParams *p); +int multifd_ram_unfill_packet(MultiFDRecvParams *p, Error **errp); #endif From patchwork Wed Sep 4 12:44:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790790 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 10D34CD37B4 for ; Wed, 4 Sep 2024 12:48:30 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpO0-0006s8-QT; Wed, 04 Sep 2024 08:45:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNr-00068M-BC for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:16 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNm-0005sl-Fq for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:12 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 40E8D1F7C5; Wed, 4 Sep 2024 12:45:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453909; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1fVnmqMUCoeEJ0o7aTnw/7to6eULdZeK0MhAeGNEGjY=; b=kSSWvKkQmubgT96I5fYS/c0d73GGuchBSdpRQ83iHlj8gwNehJBii7mAfSlYSK8V+3vRXF QRCqwHz++7RjMlp9D2l3tpUCEUadOMvV2EVh9xwXb4we00BaqihtZn4y8lzl6a4a11gT3z BwxD/6Hn6yUUGKF9e4JGwk1eOIo8EX0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453909; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1fVnmqMUCoeEJ0o7aTnw/7to6eULdZeK0MhAeGNEGjY=; b=2h0xyeOgzo7XikCgvQ4yjo5hatoya1pNKQ0fXl0Ehso7XY2Sr/sv6+xBSht/2fD6qfcicU wMGAwGGCrD4LRODw== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453909; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1fVnmqMUCoeEJ0o7aTnw/7to6eULdZeK0MhAeGNEGjY=; b=kSSWvKkQmubgT96I5fYS/c0d73GGuchBSdpRQ83iHlj8gwNehJBii7mAfSlYSK8V+3vRXF QRCqwHz++7RjMlp9D2l3tpUCEUadOMvV2EVh9xwXb4we00BaqihtZn4y8lzl6a4a11gT3z BwxD/6Hn6yUUGKF9e4JGwk1eOIo8EX0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453909; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1fVnmqMUCoeEJ0o7aTnw/7to6eULdZeK0MhAeGNEGjY=; b=2h0xyeOgzo7XikCgvQ4yjo5hatoya1pNKQ0fXl0Ehso7XY2Sr/sv6+xBSht/2fD6qfcicU wMGAwGGCrD4LRODw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id CB761139D2; Wed, 4 Sep 2024 12:45:07 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 4Ko3JFNW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:45:07 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , =?utf-8?q?Philippe_Mathie?= =?utf-8?q?u-Daud=C3=A9?= Subject: [PULL 28/34] migration/multifd: Make MultiFDMethods const Date: Wed, 4 Sep 2024 09:44:11 -0300 Message-Id: <20240904124417.14565-29-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-3.30 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.20)[-0.988]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_ALL(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[linaro.org:email,suse.de:mid,suse.de:email] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The methods are defined at module_init time and don't ever change. Make them const. Suggested-by: Philippe Mathieu-Daudé Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-nocomp.c | 2 +- migration/multifd-qpl.c | 2 +- migration/multifd-uadk.c | 2 +- migration/multifd-zlib.c | 2 +- migration/multifd-zstd.c | 2 +- migration/multifd.c | 8 ++++---- migration/multifd.h | 2 +- 7 files changed, 10 insertions(+), 10 deletions(-) diff --git a/migration/multifd-nocomp.c b/migration/multifd-nocomp.c index 53ea9f9c83..f294d1b0b2 100644 --- a/migration/multifd-nocomp.c +++ b/migration/multifd-nocomp.c @@ -377,7 +377,7 @@ bool multifd_send_prepare_common(MultiFDSendParams *p) return true; } -static MultiFDMethods multifd_nocomp_ops = { +static const MultiFDMethods multifd_nocomp_ops = { .send_setup = multifd_nocomp_send_setup, .send_cleanup = multifd_nocomp_send_cleanup, .send_prepare = multifd_nocomp_send_prepare, diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c index 75041a4c4d..b0f1e2ba46 100644 --- a/migration/multifd-qpl.c +++ b/migration/multifd-qpl.c @@ -694,7 +694,7 @@ static int multifd_qpl_recv(MultiFDRecvParams *p, Error **errp) return multifd_qpl_decompress_pages_slow_path(p, errp); } -static MultiFDMethods multifd_qpl_ops = { +static const MultiFDMethods multifd_qpl_ops = { .send_setup = multifd_qpl_send_setup, .send_cleanup = multifd_qpl_send_cleanup, .send_prepare = multifd_qpl_send_prepare, diff --git a/migration/multifd-uadk.c b/migration/multifd-uadk.c index db2549f59b..89f6a72f0e 100644 --- a/migration/multifd-uadk.c +++ b/migration/multifd-uadk.c @@ -305,7 +305,7 @@ static int multifd_uadk_recv(MultiFDRecvParams *p, Error **errp) return 0; } -static MultiFDMethods multifd_uadk_ops = { +static const MultiFDMethods multifd_uadk_ops = { .send_setup = multifd_uadk_send_setup, .send_cleanup = multifd_uadk_send_cleanup, .send_prepare = multifd_uadk_send_prepare, diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 6787538762..8cf8a26bb4 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -277,7 +277,7 @@ static int multifd_zlib_recv(MultiFDRecvParams *p, Error **errp) return 0; } -static MultiFDMethods multifd_zlib_ops = { +static const MultiFDMethods multifd_zlib_ops = { .send_setup = multifd_zlib_send_setup, .send_cleanup = multifd_zlib_send_cleanup, .send_prepare = multifd_zlib_send_prepare, diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index 1576b1e2ad..53da33e048 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -265,7 +265,7 @@ static int multifd_zstd_recv(MultiFDRecvParams *p, Error **errp) return 0; } -static MultiFDMethods multifd_zstd_ops = { +static const MultiFDMethods multifd_zstd_ops = { .send_setup = multifd_zstd_send_setup, .send_cleanup = multifd_zstd_send_cleanup, .send_prepare = multifd_zstd_send_prepare, diff --git a/migration/multifd.c b/migration/multifd.c index 0c07a2040b..b89715fdc2 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -76,7 +76,7 @@ struct { */ int exiting; /* multifd ops */ - MultiFDMethods *ops; + const MultiFDMethods *ops; } *multifd_send_state; struct { @@ -93,7 +93,7 @@ struct { uint64_t packet_num; int exiting; /* multifd ops */ - MultiFDMethods *ops; + const MultiFDMethods *ops; } *multifd_recv_state; MultiFDSendData *multifd_send_data_alloc(void) @@ -128,9 +128,9 @@ void multifd_send_channel_created(void) qemu_sem_post(&multifd_send_state->channels_created); } -static MultiFDMethods *multifd_ops[MULTIFD_COMPRESSION__MAX] = {}; +static const MultiFDMethods *multifd_ops[MULTIFD_COMPRESSION__MAX] = {}; -void multifd_register_ops(int method, MultiFDMethods *ops) +void multifd_register_ops(int method, const MultiFDMethods *ops) { assert(0 <= method && method < MULTIFD_COMPRESSION__MAX); assert(!multifd_ops[method]); diff --git a/migration/multifd.h b/migration/multifd.h index a3e35196d1..13e7a88c01 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -243,7 +243,7 @@ typedef struct { int (*recv)(MultiFDRecvParams *p, Error **errp); } MultiFDMethods; -void multifd_register_ops(int method, MultiFDMethods *ops); +void multifd_register_ops(int method, const MultiFDMethods *ops); void multifd_send_fill_packet(MultiFDSendParams *p); bool multifd_send_prepare_common(MultiFDSendParams *p); void multifd_send_zero_page_detect(MultiFDSendParams *p); From patchwork Wed Sep 4 12:44:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790772 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4801DCD37B4 for ; Wed, 4 Sep 2024 12:45:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpO9-0007Dr-HF; Wed, 04 Sep 2024 08:45:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNt-0006NK-9V for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:20 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNo-0005tK-Ka for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:16 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 080351F7C1; Wed, 4 Sep 2024 12:45:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453911; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YZFXooq4MKnjtghAnsVB52i6ZSWkI8umAQzT08aOBq4=; b=Y+FT6fnU3DLJKS2RrtSFbdChRPt1c+yTXzZb9Q79U3i0Zp0R0W+VoOeLn8KLA4sS9wK3Ab ZIwDGKorBhL6S5iGLk1r+rm0f/6ecEI2j2r/O8hLLBE4AB3p9mgHjmoympHqrTwpVui4uP t+/GAMdfI8gZnXHSkTYa3aUA0nRJh1Y= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453911; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YZFXooq4MKnjtghAnsVB52i6ZSWkI8umAQzT08aOBq4=; b=GvjZJbRP6l/8BS9NwzIxCnvRRdxyB9KvtDecJkvrZoKOCj2GRqSabZrsC5M8j3+uJzYuWf jgOoHtjxXbNk8LAA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453911; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YZFXooq4MKnjtghAnsVB52i6ZSWkI8umAQzT08aOBq4=; b=Y+FT6fnU3DLJKS2RrtSFbdChRPt1c+yTXzZb9Q79U3i0Zp0R0W+VoOeLn8KLA4sS9wK3Ab ZIwDGKorBhL6S5iGLk1r+rm0f/6ecEI2j2r/O8hLLBE4AB3p9mgHjmoympHqrTwpVui4uP t+/GAMdfI8gZnXHSkTYa3aUA0nRJh1Y= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453911; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YZFXooq4MKnjtghAnsVB52i6ZSWkI8umAQzT08aOBq4=; b=GvjZJbRP6l/8BS9NwzIxCnvRRdxyB9KvtDecJkvrZoKOCj2GRqSabZrsC5M8j3+uJzYuWf jgOoHtjxXbNk8LAA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A0B0A139D2; Wed, 4 Sep 2024 12:45:09 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id kGOOGVVW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:45:09 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , =?utf-8?q?Philippe_Mathie?= =?utf-8?q?u-Daud=C3=A9?= Subject: [PULL 29/34] migration/multifd: Stop changing the packet on recv side Date: Wed, 4 Sep 2024 09:44:12 -0300 Message-Id: <20240904124417.14565-30-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-3.30 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.20)[-0.990]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_ALL(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:mid,linaro.org:email] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org As observed by Philippe, the multifd_ram_unfill_packet() function currently leaves the MultiFDPacket structure with mixed endianness. This is harmless, but ultimately not very clean. Stop touching the received packet and do the necessary work using stack variables instead. While here tweak the error strings and fix the space before semicolons. Also remove the "100 times bigger" comment because it's just one possible explanation for a size mismatch and it doesn't even match the code. CC: Philippe Mathieu-Daudé Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-nocomp.c | 29 ++++++++++++----------------- migration/multifd.c | 20 +++++++++----------- 2 files changed, 21 insertions(+), 28 deletions(-) diff --git a/migration/multifd-nocomp.c b/migration/multifd-nocomp.c index f294d1b0b2..07c63f4a72 100644 --- a/migration/multifd-nocomp.c +++ b/migration/multifd-nocomp.c @@ -220,33 +220,28 @@ int multifd_ram_unfill_packet(MultiFDRecvParams *p, Error **errp) MultiFDPacket_t *packet = p->packet; uint32_t page_count = multifd_ram_page_count(); uint32_t page_size = multifd_ram_page_size(); + uint32_t pages_per_packet = be32_to_cpu(packet->pages_alloc); int i; - packet->pages_alloc = be32_to_cpu(packet->pages_alloc); - /* - * If we received a packet that is 100 times bigger than expected - * just stop migration. It is a magic number. - */ - if (packet->pages_alloc > page_count) { - error_setg(errp, "multifd: received packet " - "with size %u and expected a size of %u", - packet->pages_alloc, page_count) ; + if (pages_per_packet > page_count) { + error_setg(errp, "multifd: received packet with %u pages, expected %u", + pages_per_packet, page_count); return -1; } p->normal_num = be32_to_cpu(packet->normal_pages); - if (p->normal_num > packet->pages_alloc) { - error_setg(errp, "multifd: received packet " - "with %u normal pages and expected maximum pages are %u", - p->normal_num, packet->pages_alloc) ; + if (p->normal_num > pages_per_packet) { + error_setg(errp, "multifd: received packet with %u non-zero pages, " + "which exceeds maximum expected pages %u", + p->normal_num, pages_per_packet); return -1; } p->zero_num = be32_to_cpu(packet->zero_pages); - if (p->zero_num > packet->pages_alloc - p->normal_num) { - error_setg(errp, "multifd: received packet " - "with %u zero pages and expected maximum zero pages are %u", - p->zero_num, packet->pages_alloc - p->normal_num) ; + if (p->zero_num > pages_per_packet - p->normal_num) { + error_setg(errp, + "multifd: received packet with %u zero pages, expected maximum %u", + p->zero_num, pages_per_packet - p->normal_num); return -1; } diff --git a/migration/multifd.c b/migration/multifd.c index b89715fdc2..2a8cd9174c 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -230,22 +230,20 @@ void multifd_send_fill_packet(MultiFDSendParams *p) static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) { - MultiFDPacket_t *packet = p->packet; + const MultiFDPacket_t *packet = p->packet; + uint32_t magic = be32_to_cpu(packet->magic); + uint32_t version = be32_to_cpu(packet->version); int ret = 0; - packet->magic = be32_to_cpu(packet->magic); - if (packet->magic != MULTIFD_MAGIC) { - error_setg(errp, "multifd: received packet " - "magic %x and expected magic %x", - packet->magic, MULTIFD_MAGIC); + if (magic != MULTIFD_MAGIC) { + error_setg(errp, "multifd: received packet magic %x, expected %x", + magic, MULTIFD_MAGIC); return -1; } - packet->version = be32_to_cpu(packet->version); - if (packet->version != MULTIFD_VERSION) { - error_setg(errp, "multifd: received packet " - "version %u and expected version %u", - packet->version, MULTIFD_VERSION); + if (version != MULTIFD_VERSION) { + error_setg(errp, "multifd: received packet version %u, expected %u", + version, MULTIFD_VERSION); return -1; } From patchwork Wed Sep 4 12:44:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790774 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CB2C9CD3431 for ; Wed, 4 Sep 2024 12:46:23 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpOO-0007Ue-Lv; Wed, 04 Sep 2024 08:45:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNw-0006iM-Lh for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:22 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNr-0005tW-3s for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:18 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 990E11F7BF; Wed, 4 Sep 2024 12:45:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453912; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jeA7NbEryeXb4kRUTHNDcX3dp21ybv7ZvsSmNAMr7AU=; b=lumbuvv2NVX2fSqV8SIVomWAyl5jxjZ/sqIB6RuZ09rfFHKPTxMfd66i93HdGMVsFtEtMi 8x+0LwlMegv8QCKXNt/V+fl7dNna+Ba9CUQBHDy3UzAZPl9HV8PGy7dXFXceM83vglFoTB 82BNF99KAWT/eozpbS9QUmSZGm1KGis= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453912; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jeA7NbEryeXb4kRUTHNDcX3dp21ybv7ZvsSmNAMr7AU=; b=l8GwgQnAekLcg9rcItG0C2J0r/BKK3Gqtyd7Z5pDxFZ45r6+qG+3nPY2JbXfngb+iCv9fI 9vl9ViTkaILTSQAw== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453912; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jeA7NbEryeXb4kRUTHNDcX3dp21ybv7ZvsSmNAMr7AU=; b=lumbuvv2NVX2fSqV8SIVomWAyl5jxjZ/sqIB6RuZ09rfFHKPTxMfd66i93HdGMVsFtEtMi 8x+0LwlMegv8QCKXNt/V+fl7dNna+Ba9CUQBHDy3UzAZPl9HV8PGy7dXFXceM83vglFoTB 82BNF99KAWT/eozpbS9QUmSZGm1KGis= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453912; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jeA7NbEryeXb4kRUTHNDcX3dp21ybv7ZvsSmNAMr7AU=; b=l8GwgQnAekLcg9rcItG0C2J0r/BKK3Gqtyd7Z5pDxFZ45r6+qG+3nPY2JbXfngb+iCv9fI 9vl9ViTkaILTSQAw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 734E9139D2; Wed, 4 Sep 2024 12:45:11 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id mCmUDldW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:45:11 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 30/34] migration/multifd: Fix p->iov leak in multifd-uadk.c Date: Wed, 4 Sep 2024 09:44:13 -0300 Message-Id: <20240904124417.14565-31-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.78 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.18)[-0.917]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:mid]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The send_cleanup() hook should free the p->iov that was allocated at send_setup(). This was missed because the UADK code is conditional on the presence of the accelerator, so it's not tested by default. Fixes: 819dd20636 ("migration/multifd: Add UADK initialization") Reported-by: Peter Xu Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-uadk.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/multifd-uadk.c b/migration/multifd-uadk.c index 89f6a72f0e..6e6a290ae9 100644 --- a/migration/multifd-uadk.c +++ b/migration/multifd-uadk.c @@ -132,6 +132,8 @@ static void multifd_uadk_send_cleanup(MultiFDSendParams *p, Error **errp) multifd_uadk_uninit_sess(wd); p->compress_data = NULL; + g_free(p->iov); + p->iov = NULL; } static inline void prepare_next_iov(MultiFDSendParams *p, void *base, From patchwork Wed Sep 4 12:44:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790798 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 31C60CD3431 for ; Wed, 4 Sep 2024 12:48:58 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpP7-0002sN-Ef; Wed, 04 Sep 2024 08:46:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNw-0006iD-Kf for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:22 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNt-0005ti-0a for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:18 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 392281F7BE; Wed, 4 Sep 2024 12:45:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453914; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Fz4HZz0nXhBt4vizW/ev4ZCnz++jAefiPW1WfFxMH3w=; b=PLuyFLYVzWKryDkR03BukmqvkDBSlra4Uhkmek0dAWqq0zFBKkr/vgYXU1lPRSUik71Fc2 /1Bfaamhm6heMCW5ADiTzRVCNLNZ0gbAwVTeiL2hR3jJ2ayIbREqHRWlTFQ9MmtMsZFNcN t9BmMlBCC6C7N/chFq1NsuK5+QYO/mk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453914; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Fz4HZz0nXhBt4vizW/ev4ZCnz++jAefiPW1WfFxMH3w=; b=fE8XqKBL6NdVyc5yf9WwCHHecGNbzbNGBZB+t6ctpaQ7fiiM6+d1A57OugqW4q337bi3O7 NOiSRxNkQ9YdbGCQ== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=PLuyFLYV; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=fE8XqKBL DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453914; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Fz4HZz0nXhBt4vizW/ev4ZCnz++jAefiPW1WfFxMH3w=; b=PLuyFLYVzWKryDkR03BukmqvkDBSlra4Uhkmek0dAWqq0zFBKkr/vgYXU1lPRSUik71Fc2 /1Bfaamhm6heMCW5ADiTzRVCNLNZ0gbAwVTeiL2hR3jJ2ayIbREqHRWlTFQ9MmtMsZFNcN t9BmMlBCC6C7N/chFq1NsuK5+QYO/mk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453914; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Fz4HZz0nXhBt4vizW/ev4ZCnz++jAefiPW1WfFxMH3w=; b=fE8XqKBL6NdVyc5yf9WwCHHecGNbzbNGBZB+t6ctpaQ7fiiM6+d1A57OugqW4q337bi3O7 NOiSRxNkQ9YdbGCQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0FF3D139D2; Wed, 4 Sep 2024 12:45:12 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id yL4KMlhW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:45:12 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 31/34] migration/multifd: Add a couple of asserts for p->iov Date: Wed, 4 Sep 2024 09:44:14 -0300 Message-Id: <20240904124417.14565-32-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 392281F7BE X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_TLS_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCVD_VIA_SMTP_AUTH(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; RCPT_COUNT_THREE(0.00)[3]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:dkim,suse.de:mid] X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Check that p->iov is indeed always allocated and freed by the MultiFDMethods hooks. Suggested-by: Peter Xu Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/multifd.c b/migration/multifd.c index 2a8cd9174c..9b200f4ad9 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -466,6 +466,7 @@ static bool multifd_send_cleanup_channel(MultiFDSendParams *p, Error **errp) g_free(p->packet); p->packet = NULL; multifd_send_state->ops->send_cleanup(p, errp); + assert(!p->iov); return *errp == NULL; } @@ -871,6 +872,7 @@ bool multifd_send_setup(void) migrate_set_error(s, local_err); goto err; } + assert(p->iov); } return true; From patchwork Wed Sep 4 12:44:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E0243CD4849 for ; Wed, 4 Sep 2024 12:48:31 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpP8-00036Q-Sc; Wed, 04 Sep 2024 08:46:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNw-0006i6-6d for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:22 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNt-0005u0-98 for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:19 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id C8A4C1F38E; Wed, 4 Sep 2024 12:45:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453915; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MNzb/EaSJ6kInfzVq5bNLBikfqN947TiDCwvTjFUn1o=; b=zxUkvX/ZGQzOvw3FzHPsZErWa6IS5xnIagWG1zZa6MW1VW60CVQi3lbS5kvC1a2cyECkS9 yS32UK2u7XWjAfF179adPT0JwPYYG7Yty199q4jI+YTPbP1dM5wFsG3LuPZ1ezaEIoZZ+D n9l7zTRcaWG6kwDPsVpBEyNZQ87UuCY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453915; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MNzb/EaSJ6kInfzVq5bNLBikfqN947TiDCwvTjFUn1o=; b=5UdEBBjIR1gMAYoPFRsF/+2LhbzmILSoBkAIbPBRqs0uudfGd7EIw4VA4cD33Rq2Tn7voj wDQQLhcyUTPdTjBQ== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453915; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MNzb/EaSJ6kInfzVq5bNLBikfqN947TiDCwvTjFUn1o=; b=zxUkvX/ZGQzOvw3FzHPsZErWa6IS5xnIagWG1zZa6MW1VW60CVQi3lbS5kvC1a2cyECkS9 yS32UK2u7XWjAfF179adPT0JwPYYG7Yty199q4jI+YTPbP1dM5wFsG3LuPZ1ezaEIoZZ+D n9l7zTRcaWG6kwDPsVpBEyNZQ87UuCY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453915; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MNzb/EaSJ6kInfzVq5bNLBikfqN947TiDCwvTjFUn1o=; b=5UdEBBjIR1gMAYoPFRsF/+2LhbzmILSoBkAIbPBRqs0uudfGd7EIw4VA4cD33Rq2Tn7voj wDQQLhcyUTPdTjBQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A27EA139D2; Wed, 4 Sep 2024 12:45:14 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QG8nGlpW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:45:14 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson Subject: [PULL 32/34] migration/multifd: Add documentation for multifd methods Date: Wed, 4 Sep 2024 09:44:15 -0300 Message-Id: <20240904124417.14565-33-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.79 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.19)[-0.934]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:mid,suse.de:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add documentation clarifying the usage of the multifd methods. The general idea is that the client code calls into multifd to trigger send/recv of data and multifd then calls these hooks back from the worker threads at opportune moments so the client can process a portion of the data. Suggested-by: Peter Xu Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.h | 76 +++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 70 insertions(+), 6 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index 13e7a88c01..3bb96e9558 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -229,17 +229,81 @@ typedef struct { } MultiFDRecvParams; typedef struct { - /* Setup for sending side */ + /* + * The send_setup, send_cleanup, send_prepare are only called on + * the QEMU instance at the migration source. + */ + + /* + * Setup for sending side. Called once per channel during channel + * setup phase. + * + * Must allocate p->iov. If packets are in use (default), one + * extra iovec must be allocated for the packet header. Any memory + * allocated in this hook must be released at send_cleanup. + * + * p->write_flags may be used for passing flags to the QIOChannel. + * + * p->compression_data may be used by compression methods to store + * compression data. + */ int (*send_setup)(MultiFDSendParams *p, Error **errp); - /* Cleanup for sending side */ + + /* + * Cleanup for sending side. Called once per channel during + * channel cleanup phase. + */ void (*send_cleanup)(MultiFDSendParams *p, Error **errp); - /* Prepare the send packet */ + + /* + * Prepare the send packet. Called as a result of multifd_send() + * on the client side, with p pointing to the MultiFDSendParams of + * a channel that is currently idle. + * + * Must populate p->iov with the data to be sent, increment + * p->iovs_num to match the amount of iovecs used and set + * p->next_packet_size with the amount of data currently present + * in p->iov. + * + * Must indicate whether this is a compression packet by setting + * p->flags. + * + * As a last step, if packets are in use (default), must prepare + * the packet by calling multifd_send_fill_packet(). + */ int (*send_prepare)(MultiFDSendParams *p, Error **errp); - /* Setup for receiving side */ + + /* + * The recv_setup, recv_cleanup, recv are only called on the QEMU + * instance at the migration destination. + */ + + /* + * Setup for receiving side. Called once per channel during + * channel setup phase. May be empty. + * + * May allocate data structures for the receiving of data. May use + * p->iov. Compression methods may use p->compress_data. + */ int (*recv_setup)(MultiFDRecvParams *p, Error **errp); - /* Cleanup for receiving side */ + + /* + * Cleanup for receiving side. Called once per channel during + * channel cleanup phase. May be empty. + */ void (*recv_cleanup)(MultiFDRecvParams *p); - /* Read all data */ + + /* + * Data receive method. Called as a result of multifd_recv() on + * the client side, with p pointing to the MultiFDRecvParams of a + * channel that is currently idle. Only called if there is data + * available to receive. + * + * Must validate p->flags according to what was set at + * send_prepare. + * + * Must read the data from the QIOChannel p->c. + */ int (*recv)(MultiFDRecvParams *p, Error **errp); } MultiFDMethods; From patchwork Wed Sep 4 12:44:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3DEDACD37B4 for ; Wed, 4 Sep 2024 12:48:19 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpP8-00030B-2i; Wed, 04 Sep 2024 08:46:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNy-0006n8-9t; Wed, 04 Sep 2024 08:45:22 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNw-0005uU-93; Wed, 04 Sep 2024 08:45:22 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 1996E219AE; Wed, 4 Sep 2024 12:45:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453918; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FsP7vs9wLXy+/Qf4dErPzeH4vBCbn1FKP69bjd7sNkE=; b=0zr1qR8k9vAiHLBRnhCLjZFrHjgQB7KK0qVJo2EhicPJyO2axUqGf9ADDvbyc19E3LW0mZ je4L55RyGyXoVyAbcS8B2ToSfFNOeCPQHeIQpmCm63FFkOCA180GtvOX/2cAShZZuDzlpm LBMZLVH29kCxdWW2gK2ZLRhIcOCEntM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453918; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FsP7vs9wLXy+/Qf4dErPzeH4vBCbn1FKP69bjd7sNkE=; b=/NzjHL2Rc4zco/UGgV/b+Vk1kct8dx4yeDOHUZns2P2wGnzPu9vRIkwurR7LTJkOwV0mTH 87QWtb5k/ttc73BA== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453918; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FsP7vs9wLXy+/Qf4dErPzeH4vBCbn1FKP69bjd7sNkE=; b=0zr1qR8k9vAiHLBRnhCLjZFrHjgQB7KK0qVJo2EhicPJyO2axUqGf9ADDvbyc19E3LW0mZ je4L55RyGyXoVyAbcS8B2ToSfFNOeCPQHeIQpmCm63FFkOCA180GtvOX/2cAShZZuDzlpm LBMZLVH29kCxdWW2gK2ZLRhIcOCEntM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453918; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FsP7vs9wLXy+/Qf4dErPzeH4vBCbn1FKP69bjd7sNkE=; b=/NzjHL2Rc4zco/UGgV/b+Vk1kct8dx4yeDOHUZns2P2wGnzPu9vRIkwurR7LTJkOwV0mTH 87QWtb5k/ttc73BA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 3F629139D2; Wed, 4 Sep 2024 12:45:16 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id AAD0AVxW2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:45:16 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Arman Nabiev , qemu-stable@nongnu.org, Peter Maydell Subject: [PULL 33/34] target/ppc: Fix migration of CPUs with TLB_EMB TLB type Date: Wed, 4 Sep 2024 09:44:16 -0300 Message-Id: <20240904124417.14565-34-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [0.20 / 50.00]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; BAYES_HAM(-0.00)[41.71%]; RCVD_VIA_SMTP_AUTH(0.00)[]; TAGGED_RCPT(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[linaro.org:email,suse.de:email,suse.de:mid]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_CC(0.00)[redhat.com,linaro.org,gmail.com,nongnu.org]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FREEMAIL_ENVRCPT(0.00)[gmail.com] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Arman Nabiev In vmstate_tlbemb a cut-and-paste error meant we gave this vmstate subsection the same "cpu/tlb6xx" name as the vmstate_tlb6xx subsection. This breaks migration load for any CPU using the TLB_EMB CPU type, because when we see the "tlb6xx" name in the incoming data we try to interpret it as a vmstate_tlb6xx subsection, which it isn't the right format for: $ qemu-system-ppc -drive if=none,format=qcow2,file=/home/petmay01/test-images/virt/dummy.qcow2 -monitor stdio -M bamboo QEMU 9.0.92 monitor - type 'help' for more information (qemu) savevm foo (qemu) loadvm foo Missing section footer for cpu Error: Error -22 while loading VM state Correct the incorrect vmstate section name. Since migration for these CPU types was completely broken before, we don't need to care that this is a migration compatibility break. This affects the PPC 405, 440, 460 and e200 CPU families. Cc: qemu-stable@nongnu.org Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2522 Reviewed-by: Peter Maydell Signed-off-by: Arman Nabiev Signed-off-by: Fabiano Rosas --- target/ppc/machine.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/ppc/machine.c b/target/ppc/machine.c index 731dd8df35..d433fd45fc 100644 --- a/target/ppc/machine.c +++ b/target/ppc/machine.c @@ -621,7 +621,7 @@ static bool tlbemb_needed(void *opaque) } static const VMStateDescription vmstate_tlbemb = { - .name = "cpu/tlb6xx", + .name = "cpu/tlbemb", .version_id = 1, .minimum_version_id = 1, .needed = tlbemb_needed, From patchwork Wed Sep 4 12:44:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13790789 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 56CFECD3431 for ; Wed, 4 Sep 2024 12:48:25 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slpP4-0002Eb-3s; Wed, 04 Sep 2024 08:46:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slpNz-0006tw-Jh for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:24 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1slpNx-0005uy-6x for qemu-devel@nongnu.org; Wed, 04 Sep 2024 08:45:23 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id DFBA92119F; Wed, 4 Sep 2024 12:45:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453919; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MMW5DoYt/OXE6ijsDBvuukEAH9fCu8UyO1E91FfUEXs=; b=V57zSILIGHQMfnWHNXCqeDFnkK971K9cHwButqiwJ6/B68XYp8s8DDyySTGPds+iSlhNnS Yn48p+H7e98yRjX9MrpvubTF3aTLTNr5CtOPsYTkgy8xFl3ggXoTsqdQ84DmnisnmT63SE MkXsFTcP/AgY5eKpuzYpkv5j5lZwF38= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453919; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MMW5DoYt/OXE6ijsDBvuukEAH9fCu8UyO1E91FfUEXs=; b=URnrHXzw1L3p6T8uFXve++XZehNHbmlYssJPsK18+7h3LX5codfyQTcN64ukZL7z2f2q+y BKPY/r3HJWzYiNCw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=V57zSILI; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=URnrHXzw DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1725453919; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MMW5DoYt/OXE6ijsDBvuukEAH9fCu8UyO1E91FfUEXs=; b=V57zSILIGHQMfnWHNXCqeDFnkK971K9cHwButqiwJ6/B68XYp8s8DDyySTGPds+iSlhNnS Yn48p+H7e98yRjX9MrpvubTF3aTLTNr5CtOPsYTkgy8xFl3ggXoTsqdQ84DmnisnmT63SE MkXsFTcP/AgY5eKpuzYpkv5j5lZwF38= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1725453919; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MMW5DoYt/OXE6ijsDBvuukEAH9fCu8UyO1E91FfUEXs=; b=URnrHXzw1L3p6T8uFXve++XZehNHbmlYssJPsK18+7h3LX5codfyQTcN64ukZL7z2f2q+y BKPY/r3HJWzYiNCw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 83D2E139D2; Wed, 4 Sep 2024 12:45:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 6NKqEl5W2GZuVQAAD6G6ig (envelope-from ); Wed, 04 Sep 2024 12:45:18 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Richard Henderson , Thomas Huth Subject: [PULL 34/34] tests/qtest/migration: Add a check for the availability of the "pc" machine Date: Wed, 4 Sep 2024 09:44:17 -0300 Message-Id: <20240904124417.14565-35-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240904124417.14565-1-farosas@suse.de> References: <20240904124417.14565-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: DFBA92119F X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_TLS_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCVD_VIA_SMTP_AUTH(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; RCPT_COUNT_THREE(0.00)[4]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:mid,suse.de:email] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Thomas Huth The test_vcpu_dirty_limit is the only test that does not check for the availability of the machine before starting the test, so it fails when QEMU has been configured with --without-default-devices. Add a check for the "pc" machine type to fix it. Signed-off-by: Thomas Huth Reviewed-by: Fabiano Rosas Signed-off-by: Fabiano Rosas --- tests/qtest/migration-test.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 6aca6760ef..9d08101643 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -3952,8 +3952,10 @@ int main(int argc, char **argv) if (g_str_equal(arch, "x86_64") && has_kvm && kvm_dirty_ring_supported()) { migration_test_add("/migration/dirty_ring", test_precopy_unix_dirty_ring); - migration_test_add("/migration/vcpu_dirty_limit", - test_vcpu_dirty_limit); + if (qtest_has_machine("pc")) { + migration_test_add("/migration/vcpu_dirty_limit", + test_vcpu_dirty_limit); + } } ret = g_test_run();