From patchwork Thu May 23 19:05:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C2756C25B7E for ; Thu, 23 May 2024 19:07:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADlH-0001wj-Lv; Thu, 23 May 2024 15:05:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlG-0001wK-QP for qemu-devel@nongnu.org; Thu, 23 May 2024 15:05:58 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlF-0004FX-3y for qemu-devel@nongnu.org; Thu, 23 May 2024 15:05:58 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D3ADC20410; Thu, 23 May 2024 19:05:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491155; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vv1wgOdsadIPSi81PyTrC26xDTPAu/UHUd/1ECzKYOI=; b=2BtgqYXHwIRylVWWMeTQL1QWfmc1Rw2h/Is6PjLtWhdn0urwkE/FanQietQySEuo05jlS7 RqQ/W+sHboQ7QXT8m936fam+xEvN24Nmu/buDzcm8Ifxn4f/CRJh6EHTaxe7ZDnmbzuVTR Bwef9LVaKstW6+IpS8qJ38D/8uVE6Ac= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491155; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vv1wgOdsadIPSi81PyTrC26xDTPAu/UHUd/1ECzKYOI=; b=IJElDfq7adCD8OKxz6lPa5r7tTUDV/igAdZTi2yfuszk4ufTtgL9Vp79T3ki2FHwbDYzF4 FAWjIKFNH1zjv8AA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491154; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vv1wgOdsadIPSi81PyTrC26xDTPAu/UHUd/1ECzKYOI=; b=XHcCOSB2uhwBubnR4AhTiSYsk/MrQsn4bdxohJ7iE+CwOt9szf8bnIrN7hlZEaazlEeljH lAFXU1tQfX6GPhOuIj+gzBOVmxYnUT0jDLhLTLw+TfizOQMMGjJQcO0FHlF8RVr2n+zPsm 1/8UgjDIvxTBR/DOq0RNh445SiwDnV4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491154; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vv1wgOdsadIPSi81PyTrC26xDTPAu/UHUd/1ECzKYOI=; b=zvLOFcV69KYKLktytdAFhZxW1FlPZC3pKBlFf4khhvqFpkXpRDsnccJynOVRpZf2HsI/fF cscaQRyUVB9+d4DA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0846013A6B; Thu, 23 May 2024 19:05:52 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 2JY4MJCTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:05:52 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig Subject: [PATCH v2 01/18] migration: Fix file migration with fdset Date: Thu, 23 May 2024 16:05:31 -0300 Message-Id: <20240523190548.23977-2-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-3.30 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_ALL(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCPT_COUNT_FIVE(0.00)[6]; FROM_HAS_DN(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org When the "file:" migration support was added we missed the special case in the qemu_open_old implementation that allows for a particular file name format to be used to refer to a set of file descriptors that have been previously provided to QEMU via the add-fd QMP command. When using this fdset feature, we should not truncate the migration file because being given an fd means that the management layer is in control of the file and will likely already have some data written to it. This is further indicated by the presence of the 'offset' argument, which indicates the start of the region where QEMU is allowed to write. Fix the issue by replacing the O_TRUNC flag on open by an ftruncate call, which will take the offset into consideration. Fixes: 385f510df5 ("migration: file URI offset") Suggested-by: Daniel P. Berrangé Signed-off-by: Fabiano Rosas Reviewed-by: Prasad Pandit Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrangé --- migration/file.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/migration/file.c b/migration/file.c index ab18ba505a..ba5b5c44ff 100644 --- a/migration/file.c +++ b/migration/file.c @@ -84,12 +84,19 @@ void file_start_outgoing_migration(MigrationState *s, trace_migration_file_outgoing(filename); - fioc = qio_channel_file_new_path(filename, O_CREAT | O_WRONLY | O_TRUNC, - 0600, errp); + fioc = qio_channel_file_new_path(filename, O_CREAT | O_WRONLY, 0600, errp); if (!fioc) { return; } + if (ftruncate(fioc->fd, offset)) { + error_setg_errno(errp, errno, + "failed to truncate migration file to offset %" PRIx64, + offset); + object_unref(OBJECT(fioc)); + return; + } + outgoing_args.fname = g_strdup(filename); ioc = QIO_CHANNEL(fioc); From patchwork Thu May 23 19:05:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9709DC25B75 for ; Thu, 23 May 2024 19:07:15 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADlL-00020h-D4; Thu, 23 May 2024 15:06:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlJ-0001xv-93 for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:01 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlH-0004Fl-CH for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:01 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id CF08B22B73; Thu, 23 May 2024 19:05:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491158; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fmpEEye7ycGf1aJ3oDklUvl3X7zM1pah4teZ4b0oRJc=; b=MZ+soJFuM7c3j5XrZba9z7IU5eUADY08N3XtPQaES7tfkoGXTlPKuowtBMMbnfouXXCgxf FavwHlKwfBrnfYs/eStN7mlzTc4VW3CjVBRbKRHBLhIzAA/bBSEEQKRcCM/CTFKPt01XB1 0WIVxhKvR7Gdu6lwk1z99b2G5RaS2Fw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491158; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fmpEEye7ycGf1aJ3oDklUvl3X7zM1pah4teZ4b0oRJc=; b=gcsHvmaDU9HySVd456A3VwIzHyDZWroloQTy/JutGFIAapSTR+bWztm9LLLatNKjh4Cx7Q Xtnwa5NZA4ll+pCA== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=OlEPem0r; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=RPVpaMpQ DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491157; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fmpEEye7ycGf1aJ3oDklUvl3X7zM1pah4teZ4b0oRJc=; b=OlEPem0r2UOVAodlgajALjtwDrfgVXPm/mdVWyARZtJUmQ1e8xgCEPBFrW6GeR1n5IHjo4 SpKXa6nmKlG8tpJAZcg+Tr8ymnqu+I2XD4/jJdNVPUE3FiFxBGjKs6fNmLUaCrfNakjxtp GKEgEW6jJh1WBFQ178609F3BYQFAgHI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491157; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fmpEEye7ycGf1aJ3oDklUvl3X7zM1pah4teZ4b0oRJc=; b=RPVpaMpQpuom+LTLSsEgID1JKFS56fPAqqqluEE5Kx42AkfYN8EHFuhPYkwh+VFKx1fkib 5R9mPQDFYysKCyDA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 500F513A6B; Thu, 23 May 2024 19:05:55 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SDc5BpOTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:05:55 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH v2 02/18] tests/qtest/migration: Fix file migration offset check Date: Thu, 23 May 2024 16:05:32 -0300 Message-Id: <20240523190548.23977-3-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Action: no action X-Rspamd-Queue-Id: CF08B22B73 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:dkim,suse.de:email]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_SEVEN(0.00)[9]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; TO_DN_SOME(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org When doing file migration, QEMU accepts an offset that should be skipped when writing the migration stream to the file. The purpose of the offset is to allow the management layer to put its own metadata at the start of the file. We have tests for this in migration-test, but only testing that the migration stream starts at the correct offset and not that it actually leaves the data intact. Unsurprisingly, there's been a bug in that area that the tests didn't catch. Fix the tests to write some data to the offset region and check that it's actually there after the migration. While here, switch to using g_get_file_contents() which is more portable than mmap(). Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrangé --- tests/qtest/migration-test.c | 79 ++++++++++++++++++++++-------------- 1 file changed, 48 insertions(+), 31 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index b7e3406471..ec905543cf 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -67,6 +67,7 @@ static QTestMigrationState dst_state; #define QEMU_VM_FILE_MAGIC 0x5145564d #define FILE_TEST_FILENAME "migfile" #define FILE_TEST_OFFSET 0x1000 +#define FILE_TEST_MARKER 'X' #define QEMU_ENV_SRC "QTEST_QEMU_BINARY_SRC" #define QEMU_ENV_DST "QTEST_QEMU_BINARY_DST" @@ -1716,10 +1717,43 @@ finish: test_migrate_end(from, to, args->result == MIG_TEST_SUCCEED); } +static void file_dirty_offset_region(void) +{ + g_autofree char *path = g_strdup_printf("%s/%s", tmpfs, FILE_TEST_FILENAME); + size_t size = FILE_TEST_OFFSET; + g_autofree char *data = g_new0(char, size); + + memset(data, FILE_TEST_MARKER, size); + g_assert(g_file_set_contents(path, data, size, NULL)); +} + +static void file_check_offset_region(void) +{ + g_autofree char *path = g_strdup_printf("%s/%s", tmpfs, FILE_TEST_FILENAME); + size_t size = FILE_TEST_OFFSET; + g_autofree char *expected = g_new0(char, size); + g_autofree char *actual = NULL; + uint64_t *stream_start; + + /* + * Ensure the skipped offset region's data has not been touched + * and the migration stream starts at the right place. + */ + + memset(expected, FILE_TEST_MARKER, size); + + g_assert(g_file_get_contents(path, &actual, NULL, NULL)); + g_assert(!memcmp(actual, expected, size)); + + stream_start = (uint64_t *)(actual + size); + g_assert_cmpint(cpu_to_be64(*stream_start) >> 32, ==, QEMU_VM_FILE_MAGIC); +} + static void test_file_common(MigrateCommon *args, bool stop_src) { QTestState *from, *to; void *data_hook = NULL; + bool check_offset = false; if (test_migrate_start(&from, &to, args->listen_uri, &args->start)) { return; @@ -1732,6 +1766,16 @@ static void test_file_common(MigrateCommon *args, bool stop_src) */ g_assert_false(args->live); + if (g_strrstr(args->connect_uri, "offset=")) { + check_offset = true; + /* + * This comes before the start_hook because it's equivalent to + * a management application creating the file and writing to + * it so hooks should expect the file to be already present. + */ + file_dirty_offset_region(); + } + if (args->start_hook) { data_hook = args->start_hook(from, to); } @@ -1766,6 +1810,10 @@ static void test_file_common(MigrateCommon *args, bool stop_src) wait_for_serial("dest_serial"); + if (check_offset) { + file_check_offset_region(); + } + finish: if (args->finish_hook) { args->finish_hook(from, to, data_hook); @@ -1965,36 +2013,6 @@ static void test_precopy_file(void) test_file_common(&args, true); } -static void file_offset_finish_hook(QTestState *from, QTestState *to, - void *opaque) -{ -#if defined(__linux__) - g_autofree char *path = g_strdup_printf("%s/%s", tmpfs, FILE_TEST_FILENAME); - size_t size = FILE_TEST_OFFSET + sizeof(QEMU_VM_FILE_MAGIC); - uintptr_t *addr, *p; - int fd; - - fd = open(path, O_RDONLY); - g_assert(fd != -1); - addr = mmap(NULL, size, PROT_READ, MAP_SHARED, fd, 0); - g_assert(addr != MAP_FAILED); - - /* - * Ensure the skipped offset contains zeros and the migration - * stream starts at the right place. - */ - p = addr; - while (p < addr + FILE_TEST_OFFSET / sizeof(uintptr_t)) { - g_assert(*p == 0); - p++; - } - g_assert_cmpint(cpu_to_be64(*p) >> 32, ==, QEMU_VM_FILE_MAGIC); - - munmap(addr, size); - close(fd); -#endif -} - static void test_precopy_file_offset(void) { g_autofree char *uri = g_strdup_printf("file:%s/%s,offset=%d", tmpfs, @@ -2003,7 +2021,6 @@ static void test_precopy_file_offset(void) MigrateCommon args = { .connect_uri = uri, .listen_uri = "defer", - .finish_hook = file_offset_finish_hook, }; test_file_common(&args, false); From patchwork Thu May 23 19:05:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 81393C25B75 for ; Thu, 23 May 2024 19:09:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADlN-00022K-JT; Thu, 23 May 2024 15:06:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlM-00021Y-6w for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:04 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlK-0004GA-Ei for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:03 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D0AD52040D; Thu, 23 May 2024 19:06:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491160; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ER5Peeluxy6JHmSRH7zQ8d3YDDvk/6q9I4V6pgtQEYE=; b=1VMJu6e1iveSpV6QYuluTlc9Q/Nyty9M9X33/s/qVrr/IlM6rdvqJJFCQFnfBaKcRPPXc1 getx6Y7fU4kK1KGK4GP1BmTkjrUx/ln4wiPAkvolg88xR6YM8MLtXlVmIzseYYD11VVJU0 VoENooGvbryvQ8GlcwdhO3IaDirySUQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491160; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ER5Peeluxy6JHmSRH7zQ8d3YDDvk/6q9I4V6pgtQEYE=; b=tCORUTJvo1nPfEALMVo84uUvYWoRtlBg/s8U8MVC8okaTSz0nqw+k/cz4rYQrWIODDA5a6 /PRGcjCK+iZsaAAg== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491160; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ER5Peeluxy6JHmSRH7zQ8d3YDDvk/6q9I4V6pgtQEYE=; b=1VMJu6e1iveSpV6QYuluTlc9Q/Nyty9M9X33/s/qVrr/IlM6rdvqJJFCQFnfBaKcRPPXc1 getx6Y7fU4kK1KGK4GP1BmTkjrUx/ln4wiPAkvolg88xR6YM8MLtXlVmIzseYYD11VVJU0 VoENooGvbryvQ8GlcwdhO3IaDirySUQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491160; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ER5Peeluxy6JHmSRH7zQ8d3YDDvk/6q9I4V6pgtQEYE=; b=tCORUTJvo1nPfEALMVo84uUvYWoRtlBg/s8U8MVC8okaTSz0nqw+k/cz4rYQrWIODDA5a6 /PRGcjCK+iZsaAAg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4BEA713A6B; Thu, 23 May 2024 19:05:58 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0Fn+BJaTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:05:58 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH v2 03/18] tests/qtest/migration: Add a precopy file test with fdset Date: Thu, 23 May 2024 16:05:33 -0300 Message-Id: <20240523190548.23977-4-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_SEVEN(0.00)[9]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add a test for file migration using fdset. The passing of fds is more complex than using a file path. This is also the scenario where it's most important we ensure that the initial migration stream offset is respected because the fdset interface is the one used by the management layer when providing a non empty migration file. Note that fd passing is not available on Windows, so anything that uses add-fd needs to exclude that platform. Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- tests/qtest/migration-test.c | 44 ++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index ec905543cf..5bb53f4839 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2013,6 +2013,46 @@ static void test_precopy_file(void) test_file_common(&args, true); } +#ifndef _WIN32 +static void fdset_add_fds(QTestState *qts, const char *file, int flags, + int num_fds) +{ + for (int i = 0; i < num_fds; i++) { + int fd; + + fd = open(file, flags, 0660); + assert(fd != -1); + + qtest_qmp_fds_assert_success(qts, &fd, 1, "{'execute': 'add-fd', " + "'arguments': {'fdset-id': 1}}"); + close(fd); + } +} + +static void *file_offset_fdset_start_hook(QTestState *from, QTestState *to) +{ + g_autofree char *file = g_strdup_printf("%s/%s", tmpfs, FILE_TEST_FILENAME); + + fdset_add_fds(from, file, O_WRONLY, 1); + fdset_add_fds(to, file, O_RDONLY, 1); + + return NULL; +} + +static void test_precopy_file_offset_fdset(void) +{ + g_autofree char *uri = g_strdup_printf("file:/dev/fdset/1,offset=%d", + FILE_TEST_OFFSET); + MigrateCommon args = { + .connect_uri = uri, + .listen_uri = "defer", + .start_hook = file_offset_fdset_start_hook, + }; + + test_file_common(&args, false); +} +#endif + static void test_precopy_file_offset(void) { g_autofree char *uri = g_strdup_printf("file:%s/%s,offset=%d", tmpfs, @@ -3521,6 +3561,10 @@ int main(int argc, char **argv) test_precopy_file); migration_test_add("/migration/precopy/file/offset", test_precopy_file_offset); +#ifndef _WIN32 + migration_test_add("/migration/precopy/file/offset/fdset", + test_precopy_file_offset_fdset); +#endif migration_test_add("/migration/precopy/file/offset/bad", test_precopy_file_offset_bad); From patchwork Thu May 23 19:05:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D41FEC25B7A for ; Thu, 23 May 2024 19:07:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADlR-00025F-8R; Thu, 23 May 2024 15:06:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlO-00023p-V7 for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:06 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlN-0004GT-8J for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:06 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D14BE22B73; Thu, 23 May 2024 19:06:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491164; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JHN+EgKe2ZmfVVqlpa/J85xjZ9uaHa7RSXl4aXKQzw0=; b=SG0YyFYY7dFSGX9mjIcSrycgRi/QGXIT4kP8VWTx4bekMH3nCFxELK3ijxkMolN7LmdVGW 83otzAadOodyWSkolFk4orv4SuPlBWkDzmjdt21YTCc2jaxKtsbjSlLcojl/EBJw9D4oWu BiGSZzyvDlDTiTCbpFVcE+j8jJ4kStQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491164; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JHN+EgKe2ZmfVVqlpa/J85xjZ9uaHa7RSXl4aXKQzw0=; b=/U36flAp6HWTV1BkmFgFX7Hyoh/vZdp0amFZYFjKoKOu5/wE96Sk95yJvKzeKm2KsJDuic NHoIneetTes0RqBw== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491163; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JHN+EgKe2ZmfVVqlpa/J85xjZ9uaHa7RSXl4aXKQzw0=; b=RQ6HTyDpatvmR7GI6EZREr6OCjNKb8tp0Flj6E7EN7kilVcRlKjNbFE3uzD8xDzn73MJ2O kImLrsoijjmM7Jtz81NfIUHKuhuS6lnmg+n+5Wxs+ZwVns/M4zQAQ+pHmq8bvc3amBQH6F JaWzl2Wyv5dvaLlPJgiTAY60vSAig7A= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491163; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JHN+EgKe2ZmfVVqlpa/J85xjZ9uaHa7RSXl4aXKQzw0=; b=Rex4bWw+3bwFzlgJjReAztETL9VaMoDotDiQxjy4Fz98egWT5SHtU6HGx5frDhgcPw0INs EhFHM5X3D7/3uFCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 519C813A6B; Thu, 23 May 2024 19:06:01 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id oMuNBpmTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:01 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig , "Dr . David Alan Gilbert" , =?utf-8?q?Philippe_Mathieu-Da?= =?utf-8?q?ud=C3=A9?= , Paolo Bonzini Subject: [PATCH v2 04/18] monitor: Drop monitor_fdset_dup_fd_add() Date: Thu, 23 May 2024 16:05:34 -0300 Message-Id: <20240523190548.23977-5-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-3.30 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_SEVEN(0.00)[9]; RCVD_TLS_ALL(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Xu This function is not needed, one remove function should already work. Clean it up. Here the code doesn't really care about whether we need to keep that dupfd around if close() failed: when that happens something got very wrong, keeping the dup_fd around the fdsets may not help that situation so far. Cc: Dr. David Alan Gilbert Cc: Markus Armbruster Cc: Philippe Mathieu-Daudé Cc: Paolo Bonzini Cc: Daniel P. Berrangé Signed-off-by: Peter Xu [add missing return statement, removal during traversal is not safe] Signed-off-by: Fabiano Rosas --- include/monitor/monitor.h | 1 - monitor/fds.c | 28 ++++++---------------------- stubs/fdset.c | 5 ----- util/osdep.c | 15 +-------------- 4 files changed, 7 insertions(+), 42 deletions(-) diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h index 965f5d5450..fd9b3f538c 100644 --- a/include/monitor/monitor.h +++ b/include/monitor/monitor.h @@ -53,7 +53,6 @@ AddfdInfo *monitor_fdset_add_fd(int fd, bool has_fdset_id, int64_t fdset_id, const char *opaque, Error **errp); int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags); void monitor_fdset_dup_fd_remove(int dup_fd); -int64_t monitor_fdset_dup_fd_find(int dup_fd); void monitor_register_hmp(const char *name, bool info, void (*cmd)(Monitor *mon, const QDict *qdict)); diff --git a/monitor/fds.c b/monitor/fds.c index d86c2c674c..fb9f58c056 100644 --- a/monitor/fds.c +++ b/monitor/fds.c @@ -458,7 +458,7 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags) #endif } -static int64_t monitor_fdset_dup_fd_find_remove(int dup_fd, bool remove) +void monitor_fdset_dup_fd_remove(int dup_fd) { MonFdset *mon_fdset; MonFdsetFd *mon_fdset_fd_dup; @@ -467,31 +467,15 @@ static int64_t monitor_fdset_dup_fd_find_remove(int dup_fd, bool remove) QLIST_FOREACH(mon_fdset, &mon_fdsets, next) { QLIST_FOREACH(mon_fdset_fd_dup, &mon_fdset->dup_fds, next) { if (mon_fdset_fd_dup->fd == dup_fd) { - if (remove) { - QLIST_REMOVE(mon_fdset_fd_dup, next); - g_free(mon_fdset_fd_dup); - if (QLIST_EMPTY(&mon_fdset->dup_fds)) { - monitor_fdset_cleanup(mon_fdset); - } - return -1; - } else { - return mon_fdset->id; + QLIST_REMOVE(mon_fdset_fd_dup, next); + g_free(mon_fdset_fd_dup); + if (QLIST_EMPTY(&mon_fdset->dup_fds)) { + monitor_fdset_cleanup(mon_fdset); } + return; } } } - - return -1; -} - -int64_t monitor_fdset_dup_fd_find(int dup_fd) -{ - return monitor_fdset_dup_fd_find_remove(dup_fd, false); -} - -void monitor_fdset_dup_fd_remove(int dup_fd) -{ - monitor_fdset_dup_fd_find_remove(dup_fd, true); } int monitor_fd_param(Monitor *mon, const char *fdname, Error **errp) diff --git a/stubs/fdset.c b/stubs/fdset.c index d7c39a28ac..389e368a29 100644 --- a/stubs/fdset.c +++ b/stubs/fdset.c @@ -9,11 +9,6 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags) return -1; } -int64_t monitor_fdset_dup_fd_find(int dup_fd) -{ - return -1; -} - void monitor_fdset_dup_fd_remove(int dupfd) { } diff --git a/util/osdep.c b/util/osdep.c index e996c4744a..2d9749d060 100644 --- a/util/osdep.c +++ b/util/osdep.c @@ -393,21 +393,8 @@ int qemu_open_old(const char *name, int flags, ...) int qemu_close(int fd) { - int64_t fdset_id; - /* Close fd that was dup'd from an fdset */ - fdset_id = monitor_fdset_dup_fd_find(fd); - if (fdset_id != -1) { - int ret; - - ret = close(fd); - if (ret == 0) { - monitor_fdset_dup_fd_remove(fd); - } - - return ret; - } - + monitor_fdset_dup_fd_remove(fd); return close(fd); } From patchwork Thu May 23 19:05:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 67C19C25B7C for ; Thu, 23 May 2024 19:08:43 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADlT-000280-HB; Thu, 23 May 2024 15:06:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlR-00025A-3T for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:09 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlP-0004Ge-Fp for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:08 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 232F622B78; Thu, 23 May 2024 19:06:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491166; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ntI6Ev69Q4/AWtXPMvGfO4VO9JtK9Spzna77OmuCw9c=; b=Go53YdA1SA6+a5WbRJO+pUKq/veH8Xx+iV1z5aWdhfkKeMA/XyErsdweaBp2qR/3Jr1fpm OVaIwm07bnuiaYErIat9zE0Cb/TB/J0sFQqLnXxLoqUK7Kd+R4th/LGaTTkYrP8X0ZuKV8 AsF96vFzmAFpbmXfSW4RnQcPAw29+lk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491166; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ntI6Ev69Q4/AWtXPMvGfO4VO9JtK9Spzna77OmuCw9c=; b=V7CytLSLPBfezC7lkOZC2Gf1Qmv99sdXLILUsRSmntGqJQF9pFQ/GnQroZUQhKRWh4YH8j 1Rr2LrDA/JOyPMAA== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=Go53YdA1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=V7CytLSL DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491166; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ntI6Ev69Q4/AWtXPMvGfO4VO9JtK9Spzna77OmuCw9c=; b=Go53YdA1SA6+a5WbRJO+pUKq/veH8Xx+iV1z5aWdhfkKeMA/XyErsdweaBp2qR/3Jr1fpm OVaIwm07bnuiaYErIat9zE0Cb/TB/J0sFQqLnXxLoqUK7Kd+R4th/LGaTTkYrP8X0ZuKV8 AsF96vFzmAFpbmXfSW4RnQcPAw29+lk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491166; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ntI6Ev69Q4/AWtXPMvGfO4VO9JtK9Spzna77OmuCw9c=; b=V7CytLSLPBfezC7lkOZC2Gf1Qmv99sdXLILUsRSmntGqJQF9pFQ/GnQroZUQhKRWh4YH8j 1Rr2LrDA/JOyPMAA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4BB8F13A6B; Thu, 23 May 2024 19:06:04 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0JvzBJyTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:04 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig Subject: [PATCH v2 05/18] monitor: Introduce monitor_fdset_*free Date: Thu, 23 May 2024 16:05:35 -0300 Message-Id: <20240523190548.23977-6-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spamd-Result: default: False [-0.03 / 50.00]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; BAYES_HAM(-0.02)[54.66%]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_FIVE(0.00)[6]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: 232F622B78 X-Spamd-Bar: / Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Introduce two new functions to remove and free no longer used fds and fdsets. We need those to decouple the remove/free routines from monitor_fdset_cleanup() which will go away in the next patches. The new functions: - monitor_fdset_free() will be used when a monitor connection closes and when an fd is removed to cleanup any fdset that is now empty. - monitor_fdset_fd_free() will be used to remove one or more fds that have been explicitly targeted by qmp_remove_fd(). Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- monitor/fds.c | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/monitor/fds.c b/monitor/fds.c index fb9f58c056..101e21720d 100644 --- a/monitor/fds.c +++ b/monitor/fds.c @@ -167,6 +167,22 @@ int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp) return -1; } +static void monitor_fdset_free(MonFdset *mon_fdset) +{ + if (QLIST_EMPTY(&mon_fdset->fds) && QLIST_EMPTY(&mon_fdset->dup_fds)) { + QLIST_REMOVE(mon_fdset, next); + g_free(mon_fdset); + } +} + +static void monitor_fdset_fd_free(MonFdsetFd *mon_fdset_fd) +{ + close(mon_fdset_fd->fd); + g_free(mon_fdset_fd->opaque); + QLIST_REMOVE(mon_fdset_fd, next); + g_free(mon_fdset_fd); +} + static void monitor_fdset_cleanup(MonFdset *mon_fdset) { MonFdsetFd *mon_fdset_fd; @@ -176,17 +192,11 @@ static void monitor_fdset_cleanup(MonFdset *mon_fdset) if ((mon_fdset_fd->removed || (QLIST_EMPTY(&mon_fdset->dup_fds) && mon_refcount == 0)) && runstate_is_running()) { - close(mon_fdset_fd->fd); - g_free(mon_fdset_fd->opaque); - QLIST_REMOVE(mon_fdset_fd, next); - g_free(mon_fdset_fd); + monitor_fdset_fd_free(mon_fdset_fd); } } - if (QLIST_EMPTY(&mon_fdset->fds) && QLIST_EMPTY(&mon_fdset->dup_fds)) { - QLIST_REMOVE(mon_fdset, next); - g_free(mon_fdset); - } + monitor_fdset_free(mon_fdset); } void monitor_fdsets_cleanup(void) From patchwork Thu May 23 19:05:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6F57FC25B75 for ; Thu, 23 May 2024 19:08:42 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADlW-00029O-2c; Thu, 23 May 2024 15:06:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlT-00028G-TF for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:11 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlR-0004H0-VK for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:11 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id A716022B79; Thu, 23 May 2024 19:06:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491168; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3zM6yJwrUnBGwhUnSZK8j874ter6lal7RTb8Pv37dNs=; b=Wg3LQHahoag2mLxT2rDJ/FJolYa/JcopKAgRUY1L4CJbaYSpC7OYTwluC8qaLPLjzPNUf1 5B9yPLkDt62tmqnruns2vOnX6UwrfefG0131OAUaxC99R+Qdg5i95egGCFaO/mid7VejdI a6Io0b5D+ZhOyDlmZuXr70M+xeJf+Ew= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491168; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3zM6yJwrUnBGwhUnSZK8j874ter6lal7RTb8Pv37dNs=; b=8xy3hrevmWTi0iR6zU+5N62Jhc/h77jL7qy7B+fOulx6hpUeKAlQ2D0YU6upafhc0sxEqN /5v7/vFKoOR5xZBQ== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491168; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3zM6yJwrUnBGwhUnSZK8j874ter6lal7RTb8Pv37dNs=; b=Wg3LQHahoag2mLxT2rDJ/FJolYa/JcopKAgRUY1L4CJbaYSpC7OYTwluC8qaLPLjzPNUf1 5B9yPLkDt62tmqnruns2vOnX6UwrfefG0131OAUaxC99R+Qdg5i95egGCFaO/mid7VejdI a6Io0b5D+ZhOyDlmZuXr70M+xeJf+Ew= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491168; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3zM6yJwrUnBGwhUnSZK8j874ter6lal7RTb8Pv37dNs=; b=8xy3hrevmWTi0iR6zU+5N62Jhc/h77jL7qy7B+fOulx6hpUeKAlQ2D0YU6upafhc0sxEqN /5v7/vFKoOR5xZBQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 984FA13A6B; Thu, 23 May 2024 19:06:06 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id yLbXF56TT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:06 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig , "Dr. David Alan Gilbert" Subject: [PATCH v2 06/18] monitor: Stop removing non-duplicated fds Date: Thu, 23 May 2024 16:05:36 -0300 Message-Id: <20240523190548.23977-7-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_SEVEN(0.00)[7]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We've been up until now cleaning up any file descriptors that have been passed into QEMU and never duplicated[1,2]. A file descriptor without duplicates indicates that no part of QEMU has made use of it. This approach is starting to show some cracks now that we're starting to consume fds from the migration code: - Doing cleanup every time the last monitor connection closes works to reap unused fds, but also has the side effect of forcing the management layer to pass the file descriptors again in case of a disconnect/re-connect, if that happened to be the only monitor connection. Another side effect is that removing an fd with qmp_remove_fd() is effectively delayed until the last monitor connection closes. The reliance on mon_refcount is also problematic because it's racy. - Checking runstate_is_running() skips the cleanup unless the VM is running and avoids premature cleanup of the fds, but also has the side effect of blocking the legitimate removal of an fd via qmp_remove_fd() if the VM happens to be in another state. This affects qmp_remove_fd() and qmp_query_fdsets() in particular because requesting a removal at a bad time (guest stopped) might cause an fd to never be removed, or to be removed at a much later point in time, causing the query command to continue showing the supposedly removed fd/fdset. Note that file descriptors that *have* been duplicated are owned by the code that uses them and will be removed after qemu_close() is called. Therefore we've decided that the best course of action to avoid the undesired side-effects is to stop managing non-duplicated file descriptors. 1- efb87c1697 ("monitor: Clean up fd sets on monitor disconnect") 2- ebe52b592d ("monitor: Prevent removing fd from set during init") Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- monitor/fds.c | 15 ++++++++------- monitor/hmp.c | 2 -- monitor/monitor-internal.h | 1 - monitor/qmp.c | 2 -- 4 files changed, 8 insertions(+), 12 deletions(-) diff --git a/monitor/fds.c b/monitor/fds.c index 101e21720d..f7b98814fa 100644 --- a/monitor/fds.c +++ b/monitor/fds.c @@ -169,6 +169,11 @@ int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp) static void monitor_fdset_free(MonFdset *mon_fdset) { + /* + * Only remove an empty fdset. The fds are owned by the user and + * should have been removed with qmp_remove_fd(). The dup_fds are + * owned by QEMU and should have been removed with qemu_close(). + */ if (QLIST_EMPTY(&mon_fdset->fds) && QLIST_EMPTY(&mon_fdset->dup_fds)) { QLIST_REMOVE(mon_fdset, next); g_free(mon_fdset); @@ -189,9 +194,7 @@ static void monitor_fdset_cleanup(MonFdset *mon_fdset) MonFdsetFd *mon_fdset_fd_next; QLIST_FOREACH_SAFE(mon_fdset_fd, &mon_fdset->fds, next, mon_fdset_fd_next) { - if ((mon_fdset_fd->removed || - (QLIST_EMPTY(&mon_fdset->dup_fds) && mon_refcount == 0)) && - runstate_is_running()) { + if (mon_fdset_fd->removed) { monitor_fdset_fd_free(mon_fdset_fd); } } @@ -206,7 +209,7 @@ void monitor_fdsets_cleanup(void) QEMU_LOCK_GUARD(&mon_fdsets_lock); QLIST_FOREACH_SAFE(mon_fdset, &mon_fdsets, next, mon_fdset_next) { - monitor_fdset_cleanup(mon_fdset); + monitor_fdset_free(mon_fdset); } } @@ -479,9 +482,7 @@ void monitor_fdset_dup_fd_remove(int dup_fd) if (mon_fdset_fd_dup->fd == dup_fd) { QLIST_REMOVE(mon_fdset_fd_dup, next); g_free(mon_fdset_fd_dup); - if (QLIST_EMPTY(&mon_fdset->dup_fds)) { - monitor_fdset_cleanup(mon_fdset); - } + monitor_fdset_free(mon_fdset); return; } } diff --git a/monitor/hmp.c b/monitor/hmp.c index 69c1b7e98a..460e8832f6 100644 --- a/monitor/hmp.c +++ b/monitor/hmp.c @@ -1437,11 +1437,9 @@ static void monitor_event(void *opaque, QEMUChrEvent event) monitor_resume(mon); } qemu_mutex_unlock(&mon->mon_lock); - mon_refcount++; break; case CHR_EVENT_CLOSED: - mon_refcount--; monitor_fdsets_cleanup(); break; diff --git a/monitor/monitor-internal.h b/monitor/monitor-internal.h index 252de85681..cb628f681d 100644 --- a/monitor/monitor-internal.h +++ b/monitor/monitor-internal.h @@ -168,7 +168,6 @@ extern bool qmp_dispatcher_co_shutdown; extern QmpCommandList qmp_commands, qmp_cap_negotiation_commands; extern QemuMutex monitor_lock; extern MonitorList mon_list; -extern int mon_refcount; extern HMPCommand hmp_cmds[]; diff --git a/monitor/qmp.c b/monitor/qmp.c index a239945e8d..5e538f34c0 100644 --- a/monitor/qmp.c +++ b/monitor/qmp.c @@ -466,7 +466,6 @@ static void monitor_qmp_event(void *opaque, QEMUChrEvent event) data = qmp_greeting(mon); qmp_send_response(mon, data); qobject_unref(data); - mon_refcount++; break; case CHR_EVENT_CLOSED: /* @@ -479,7 +478,6 @@ static void monitor_qmp_event(void *opaque, QEMUChrEvent event) json_message_parser_destroy(&mon->parser); json_message_parser_init(&mon->parser, handle_qmp_command, mon, NULL); - mon_refcount--; monitor_fdsets_cleanup(); break; case CHR_EVENT_BREAK: From patchwork Thu May 23 19:05:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672206 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 53669C25B75 for ; Thu, 23 May 2024 19:07:47 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADlW-00029g-Th; Thu, 23 May 2024 15:06:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlV-00029J-Ve for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:13 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlU-0004IX-7d for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:13 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id EE2602040D; Thu, 23 May 2024 19:06:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491171; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mwtD5RitfHwo1/3GuiieS4h4aHhJQYNEx9e0JR0CQDo=; b=GTvi8PblbEzD9Koy+pDsEKQBOoL4QbrMZWgW3d987AuLeOfqz41vLzJ/WZz/ef19dnhpou wFC2Ika7QDQRysI1hBSqCu89jAw7RmRCqlTvuBppuHnzkD4DmhOig8rrQYpfw7hCVZp1BT 0KRFb+ctMBpGEOBONwuZgqKQYsBv0Cc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491171; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mwtD5RitfHwo1/3GuiieS4h4aHhJQYNEx9e0JR0CQDo=; b=X98rZU8Iu1VdSXgwabhyKnvgxTYTpWGpSb7u6I6cMiESMjBjcY59AkJM4iyiQzFGraLRz9 4Ijl8VZAbBqSZACg== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="L/h+uycb"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=79yYwZpL DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491170; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mwtD5RitfHwo1/3GuiieS4h4aHhJQYNEx9e0JR0CQDo=; b=L/h+uycbaiWoGYyS5NdNKySegEYlweLXd/Dyf4u5rQkxI7MujRRu5JmZP4g/vbNycKa0C9 mq6g1DKai8QOGgQjBNxWL0eW+TrDcc3TIvWyJ80l+/EuUn3QnfdWlrmuezUh3sXr7Wps/3 dUcMpOem/JfUkxG0xHg9WtT5jwIeuJ4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491170; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mwtD5RitfHwo1/3GuiieS4h4aHhJQYNEx9e0JR0CQDo=; b=79yYwZpL3y6BifCho9tJhe33ZwTxag7vmkAy9k93nu8k0O8xvEtw4rwyXnKttu2glO4qzn j+c5YOvUyLtHN8Dw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 2369D13A6B; Thu, 23 May 2024 19:06:08 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id GL/ONqCTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:08 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig Subject: [PATCH v2 07/18] monitor: Simplify fdset and fd removal Date: Thu, 23 May 2024 16:05:37 -0300 Message-Id: <20240523190548.23977-8-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.04 / 50.00]; BAYES_HAM(-2.03)[95.23%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_FIVE(0.00)[6]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Action: no action X-Rspamd-Queue-Id: EE2602040D X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Remove fds right away instead of setting the ->removed flag. We don't need the extra complexity of having a cleanup function reap the removed entries at a later time. Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- monitor/fds.c | 27 ++++++--------------------- 1 file changed, 6 insertions(+), 21 deletions(-) diff --git a/monitor/fds.c b/monitor/fds.c index f7b98814fa..fa723e1883 100644 --- a/monitor/fds.c +++ b/monitor/fds.c @@ -43,7 +43,6 @@ struct mon_fd_t { typedef struct MonFdsetFd MonFdsetFd; struct MonFdsetFd { int fd; - bool removed; char *opaque; QLIST_ENTRY(MonFdsetFd) next; }; @@ -188,20 +187,6 @@ static void monitor_fdset_fd_free(MonFdsetFd *mon_fdset_fd) g_free(mon_fdset_fd); } -static void monitor_fdset_cleanup(MonFdset *mon_fdset) -{ - MonFdsetFd *mon_fdset_fd; - MonFdsetFd *mon_fdset_fd_next; - - QLIST_FOREACH_SAFE(mon_fdset_fd, &mon_fdset->fds, next, mon_fdset_fd_next) { - if (mon_fdset_fd->removed) { - monitor_fdset_fd_free(mon_fdset_fd); - } - } - - monitor_fdset_free(mon_fdset); -} - void monitor_fdsets_cleanup(void) { MonFdset *mon_fdset; @@ -276,7 +261,7 @@ void qmp_get_win32_socket(const char *infos, const char *fdname, Error **errp) void qmp_remove_fd(int64_t fdset_id, bool has_fd, int64_t fd, Error **errp) { MonFdset *mon_fdset; - MonFdsetFd *mon_fdset_fd; + MonFdsetFd *mon_fdset_fd, *mon_fdset_fd_next; char fd_str[60]; QEMU_LOCK_GUARD(&mon_fdsets_lock); @@ -284,21 +269,22 @@ void qmp_remove_fd(int64_t fdset_id, bool has_fd, int64_t fd, Error **errp) if (mon_fdset->id != fdset_id) { continue; } - QLIST_FOREACH(mon_fdset_fd, &mon_fdset->fds, next) { + QLIST_FOREACH_SAFE(mon_fdset_fd, &mon_fdset->fds, next, + mon_fdset_fd_next) { if (has_fd) { if (mon_fdset_fd->fd != fd) { continue; } - mon_fdset_fd->removed = true; + monitor_fdset_fd_free(mon_fdset_fd); break; } else { - mon_fdset_fd->removed = true; + monitor_fdset_fd_free(mon_fdset_fd); } } if (has_fd && !mon_fdset_fd) { goto error; } - monitor_fdset_cleanup(mon_fdset); + monitor_fdset_free(mon_fdset); return; } @@ -408,7 +394,6 @@ AddfdInfo *monitor_fdset_add_fd(int fd, bool has_fdset_id, int64_t fdset_id, mon_fdset_fd = g_malloc0(sizeof(*mon_fdset_fd)); mon_fdset_fd->fd = fd; - mon_fdset_fd->removed = false; mon_fdset_fd->opaque = g_strdup(opaque); QLIST_INSERT_HEAD(&mon_fdset->fds, mon_fdset_fd, next); From patchwork Thu May 23 19:05:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672198 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 92CF4C25B7A for ; Thu, 23 May 2024 19:07:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADlZ-0002AM-LY; Thu, 23 May 2024 15:06:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlY-0002A2-Eq for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:16 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlW-0004Ij-Pk for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:16 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 7F2752040F; Thu, 23 May 2024 19:06:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491173; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dc/TOwAU9vhSOb+rAc6rFNHJjK+Pe45jsesCu2QG4kc=; b=n2zs8You/LhDkcxSuPzr7hMfwzA2Q6Co55WZurXfeaD4smC5zyu7Gq8O64p4UrHY2meUk2 q4FcZAmPvneGc4XJ859ELw81IYWE58Drvd243BdTmrbUEmnjjKz7diMm8YdbYxrQmCET+P jn8Jskr49nhuHLD5w1Xxij5DumlqoNM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491173; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dc/TOwAU9vhSOb+rAc6rFNHJjK+Pe45jsesCu2QG4kc=; b=VS8DPy6sNW3rNYk83TjNlBSS8xzP95XxNG/GXKPFM9dgaX6y9/C2RDanUkBmbVb+SbtfF2 29xtR/OwZ/ym2LDA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491173; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dc/TOwAU9vhSOb+rAc6rFNHJjK+Pe45jsesCu2QG4kc=; b=n2zs8You/LhDkcxSuPzr7hMfwzA2Q6Co55WZurXfeaD4smC5zyu7Gq8O64p4UrHY2meUk2 q4FcZAmPvneGc4XJ859ELw81IYWE58Drvd243BdTmrbUEmnjjKz7diMm8YdbYxrQmCET+P jn8Jskr49nhuHLD5w1Xxij5DumlqoNM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491173; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dc/TOwAU9vhSOb+rAc6rFNHJjK+Pe45jsesCu2QG4kc=; b=VS8DPy6sNW3rNYk83TjNlBSS8xzP95XxNG/GXKPFM9dgaX6y9/C2RDanUkBmbVb+SbtfF2 29xtR/OwZ/ym2LDA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 7064A13A6B; Thu, 23 May 2024 19:06:11 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id KDjnDaOTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:11 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig , Paolo Bonzini Subject: [PATCH v2 08/18] monitor: Report errors from monitor_fdset_dup_fd_add Date: Thu, 23 May 2024 16:05:38 -0300 Message-Id: <20240523190548.23977-9-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_SEVEN(0.00)[7]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org I'm keeping the EACCES because callers expect to be able to look at errno. Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- include/monitor/monitor.h | 2 +- monitor/fds.c | 10 +++++++++- stubs/fdset.c | 2 +- util/osdep.c | 10 +--------- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h index fd9b3f538c..c3740ec616 100644 --- a/include/monitor/monitor.h +++ b/include/monitor/monitor.h @@ -51,7 +51,7 @@ int monitor_read_password(MonitorHMP *mon, ReadLineFunc *readline_func, AddfdInfo *monitor_fdset_add_fd(int fd, bool has_fdset_id, int64_t fdset_id, const char *opaque, Error **errp); -int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags); +int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags, Error **errp); void monitor_fdset_dup_fd_remove(int dup_fd); void monitor_register_hmp(const char *name, bool info, diff --git a/monitor/fds.c b/monitor/fds.c index fa723e1883..d93d2e695b 100644 --- a/monitor/fds.c +++ b/monitor/fds.c @@ -404,9 +404,10 @@ AddfdInfo *monitor_fdset_add_fd(int fd, bool has_fdset_id, int64_t fdset_id, return fdinfo; } -int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags) +int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags, Error **errp) { #ifdef _WIN32 + error_setg(errp, "Platform does not support fd passing (fdset)"); return -ENOENT; #else MonFdset *mon_fdset; @@ -426,6 +427,8 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags) QLIST_FOREACH(mon_fdset_fd, &mon_fdset->fds, next) { mon_fd_flags = fcntl(mon_fdset_fd->fd, F_GETFL); if (mon_fd_flags == -1) { + error_setg(errp, "Failed to read file status flags for fd=%d", + mon_fdset_fd->fd); return -1; } @@ -437,11 +440,15 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags) if (fd == -1) { errno = EACCES; + error_setg(errp, + "Failed to find file descriptor with matching flags=0x%x", + flags); return -1; } dup_fd = qemu_dup_flags(fd, flags); if (dup_fd == -1) { + error_setg(errp, "Failed to dup() given file descriptor fd=%d", fd); return -1; } @@ -451,6 +458,7 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags) return dup_fd; } + error_setg(errp, "Failed to find fdset /dev/fdset/%" PRId64, fdset_id); errno = ENOENT; return -1; #endif diff --git a/stubs/fdset.c b/stubs/fdset.c index 389e368a29..2950fd91fd 100644 --- a/stubs/fdset.c +++ b/stubs/fdset.c @@ -3,7 +3,7 @@ #include "monitor/monitor.h" #include "../monitor/monitor-internal.h" -int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags) +int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags, Error **errp) { errno = ENOSYS; return -1; diff --git a/util/osdep.c b/util/osdep.c index 2d9749d060..0cb8f98bf6 100644 --- a/util/osdep.c +++ b/util/osdep.c @@ -305,7 +305,6 @@ qemu_open_internal(const char *name, int flags, mode_t mode, Error **errp) /* Attempt dup of fd from fd set */ if (strstart(name, "/dev/fdset/", &fdset_id_str)) { int64_t fdset_id; - int dupfd; fdset_id = qemu_parse_fdset(fdset_id_str); if (fdset_id == -1) { @@ -314,14 +313,7 @@ qemu_open_internal(const char *name, int flags, mode_t mode, Error **errp) return -1; } - dupfd = monitor_fdset_dup_fd_add(fdset_id, flags); - if (dupfd == -1) { - error_setg_errno(errp, errno, "Could not dup FD for %s flags %x", - name, flags); - return -1; - } - - return dupfd; + return monitor_fdset_dup_fd_add(fdset_id, flags, errp); } #endif From patchwork Thu May 23 19:05:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9A217C41513 for ; Thu, 23 May 2024 19:07:15 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADlc-0002B5-AC; Thu, 23 May 2024 15:06:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADla-0002AW-Sh for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:18 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlZ-0004Iv-8j for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:18 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C7F0F22B73; Thu, 23 May 2024 19:06:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491175; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wtQ1u0erj1/fAKsKQRglQTtk58OkABCFdk6wbusgBdY=; b=LvKxKJDcM1CXVrIT0jemfZAdJLBS4Dmj3OxGMVTwY00gOxH3/lTJW2RZdXDBm/mRgvKldo X3zP/c2oPhbuwVivK2gdIW7KQa9HTAxqf5QKOW8ZrvPUfNFnUCf5N5cqSbMRZ4bfJtFh+Y s7sMpi0bofuNqt0TzjU17ugdTrWYP8c= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491175; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wtQ1u0erj1/fAKsKQRglQTtk58OkABCFdk6wbusgBdY=; b=OpciNtucqAEkiIkRZpT1ZqtgSXOOi/DW810VHyytVoHu9kSui3a6gxzQS1A+ROJTWxfHoU 6AGJvTg/L5bDY7Aw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=LvKxKJDc; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=OpciNtuc DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491175; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wtQ1u0erj1/fAKsKQRglQTtk58OkABCFdk6wbusgBdY=; b=LvKxKJDcM1CXVrIT0jemfZAdJLBS4Dmj3OxGMVTwY00gOxH3/lTJW2RZdXDBm/mRgvKldo X3zP/c2oPhbuwVivK2gdIW7KQa9HTAxqf5QKOW8ZrvPUfNFnUCf5N5cqSbMRZ4bfJtFh+Y s7sMpi0bofuNqt0TzjU17ugdTrWYP8c= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491175; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wtQ1u0erj1/fAKsKQRglQTtk58OkABCFdk6wbusgBdY=; b=OpciNtucqAEkiIkRZpT1ZqtgSXOOi/DW810VHyytVoHu9kSui3a6gxzQS1A+ROJTWxfHoU 6AGJvTg/L5bDY7Aw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id EF64313A6B; Thu, 23 May 2024 19:06:13 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id mEoTLaWTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:13 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig Subject: [PATCH v2 09/18] io: Stop using qemu_open_old in channel-file Date: Thu, 23 May 2024 16:05:39 -0300 Message-Id: <20240523190548.23977-10-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Action: no action X-Rspamd-Queue-Id: C7F0F22B73 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Result: default: False [-3.51 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:dkim,suse.de:email]; DKIM_TRACE(0.00)[suse.de:+] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We want to make use of the Error object to report fdset errors from qemu_open_internal() and passing the error pointer to qemu_open_old() would require changing all callers. Move the file channel to the new API instead. Reviewed-by: Daniel P. Berrangé Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- io/channel-file.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/io/channel-file.c b/io/channel-file.c index 6436cfb6ae..2ea8d08360 100644 --- a/io/channel-file.c +++ b/io/channel-file.c @@ -68,11 +68,13 @@ qio_channel_file_new_path(const char *path, ioc = QIO_CHANNEL_FILE(object_new(TYPE_QIO_CHANNEL_FILE)); - ioc->fd = qemu_open_old(path, flags, mode); + if (flags & O_CREAT) { + ioc->fd = qemu_create(path, flags & ~O_CREAT, mode, errp); + } else { + ioc->fd = qemu_open(path, flags, errp); + } if (ioc->fd < 0) { object_unref(OBJECT(ioc)); - error_setg_errno(errp, errno, - "Unable to open %s", path); return NULL; } From patchwork Thu May 23 19:05:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672202 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A0B21C25B7A for ; Thu, 23 May 2024 19:07:33 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADlv-0002G9-Hj; Thu, 23 May 2024 15:06:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlp-0002Ek-44 for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:34 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlb-0004JP-RY for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:32 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5C1E722B79; Thu, 23 May 2024 19:06:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491178; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Pp89Cnn3FmnMv8zeYpnAlQzcELRXERoRwrQNWr+s7A=; b=zkoIySyhgtL8/x9NZcIqlYwb2U5XcBmG/J0gSP9NPuofAgYN1mVV4PB4KD15rgMnUCfSKp fYD0hXap9Zqvgic2u0MlIPW7V55/wSEW2Gaza8E31nWqZGYJ0r9JwNq/QZNwWtnWEDjwk0 il0dc6yUmUBs31eGYCxnpc6HcvsCAr4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491178; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Pp89Cnn3FmnMv8zeYpnAlQzcELRXERoRwrQNWr+s7A=; b=9qZleLagIKj7W3mZTX3KeeZsYi7pBVe9tyJJwFEItcXiYl/Ez82VjQLYaKl24Ff0ksvTaO A1DkXW/S3uhTSUDg== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=zkoIySyh; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=9qZleLag DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491178; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Pp89Cnn3FmnMv8zeYpnAlQzcELRXERoRwrQNWr+s7A=; b=zkoIySyhgtL8/x9NZcIqlYwb2U5XcBmG/J0gSP9NPuofAgYN1mVV4PB4KD15rgMnUCfSKp fYD0hXap9Zqvgic2u0MlIPW7V55/wSEW2Gaza8E31nWqZGYJ0r9JwNq/QZNwWtnWEDjwk0 il0dc6yUmUBs31eGYCxnpc6HcvsCAr4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491178; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Pp89Cnn3FmnMv8zeYpnAlQzcELRXERoRwrQNWr+s7A=; b=9qZleLagIKj7W3mZTX3KeeZsYi7pBVe9tyJJwFEItcXiYl/Ez82VjQLYaKl24Ff0ksvTaO A1DkXW/S3uhTSUDg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4C57A13A6B; Thu, 23 May 2024 19:06:16 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id iP8OBaiTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:16 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig , Eric Blake Subject: [PATCH v2 10/18] migration: Add direct-io parameter Date: Thu, 23 May 2024 16:05:40 -0300 Message-Id: <20240523190548.23977-11-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Action: no action X-Rspamd-Queue-Id: 5C1E722B79 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:dkim,suse.de:email]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_SEVEN(0.00)[7]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; TO_DN_SOME(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add the direct-io migration parameter that tells the migration code to use O_DIRECT when opening the migration stream file whenever possible. This is currently only used with the mapped-ram migration that has a clear window guaranteed to perform aligned writes. Acked-by: Markus Armbruster Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- include/qemu/osdep.h | 2 ++ migration/migration-hmp-cmds.c | 11 +++++++++++ migration/options.c | 28 ++++++++++++++++++++++++++++ migration/options.h | 1 + qapi/migration.json | 21 ++++++++++++++++++--- util/osdep.c | 9 +++++++++ 6 files changed, 69 insertions(+), 3 deletions(-) diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h index f61edcfdc2..191916f38e 100644 --- a/include/qemu/osdep.h +++ b/include/qemu/osdep.h @@ -612,6 +612,8 @@ int qemu_lock_fd_test(int fd, int64_t start, int64_t len, bool exclusive); bool qemu_has_ofd_lock(void); #endif +bool qemu_has_direct_io(void); + #if defined(__HAIKU__) && defined(__i386__) #define FMT_pid "%ld" #elif defined(WIN64) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 9f0e8029e0..7d608d26e1 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -351,6 +351,13 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict) monitor_printf(mon, "%s: %s\n", MigrationParameter_str(MIGRATION_PARAMETER_MODE), qapi_enum_lookup(&MigMode_lookup, params->mode)); + + if (params->has_direct_io) { + monitor_printf(mon, "%s: %s\n", + MigrationParameter_str( + MIGRATION_PARAMETER_DIRECT_IO), + params->direct_io ? "on" : "off"); + } } qapi_free_MigrationParameters(params); @@ -624,6 +631,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_mode = true; visit_type_MigMode(v, param, &p->mode, &err); break; + case MIGRATION_PARAMETER_DIRECT_IO: + p->has_direct_io = true; + visit_type_bool(v, param, &p->direct_io, &err); + break; default: assert(0); } diff --git a/migration/options.c b/migration/options.c index 5ab5b6d85d..6f614761b8 100644 --- a/migration/options.c +++ b/migration/options.c @@ -702,6 +702,18 @@ bool migrate_cpu_throttle_tailslow(void) return s->parameters.cpu_throttle_tailslow; } +bool migrate_direct_io(void) +{ + MigrationState *s = migrate_get_current(); + + /* For now O_DIRECT is only supported with mapped-ram */ + if (!s->capabilities[MIGRATION_CAPABILITY_MAPPED_RAM]) { + return false; + } + + return s->parameters.direct_io; +} + uint64_t migrate_downtime_limit(void) { MigrationState *s = migrate_get_current(); @@ -905,6 +917,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->mode = s->parameters.mode; params->has_zero_page_detection = true; params->zero_page_detection = s->parameters.zero_page_detection; + params->has_direct_io = true; + params->direct_io = s->parameters.direct_io; return params; } @@ -937,6 +951,7 @@ void migrate_params_init(MigrationParameters *params) params->has_vcpu_dirty_limit = true; params->has_mode = true; params->has_zero_page_detection = true; + params->has_direct_io = true; } /* @@ -1110,6 +1125,11 @@ bool migrate_params_check(MigrationParameters *params, Error **errp) return false; } + if (params->has_direct_io && params->direct_io && !qemu_has_direct_io()) { + error_setg(errp, "No build-time support for direct-io"); + return false; + } + return true; } @@ -1216,6 +1236,10 @@ static void migrate_params_test_apply(MigrateSetParameters *params, if (params->has_zero_page_detection) { dest->zero_page_detection = params->zero_page_detection; } + + if (params->has_direct_io) { + dest->direct_io = params->direct_io; + } } static void migrate_params_apply(MigrateSetParameters *params, Error **errp) @@ -1341,6 +1365,10 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) if (params->has_zero_page_detection) { s->parameters.zero_page_detection = params->zero_page_detection; } + + if (params->has_direct_io) { + s->parameters.direct_io = params->direct_io; + } } void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) diff --git a/migration/options.h b/migration/options.h index 4b21cc2669..a2397026db 100644 --- a/migration/options.h +++ b/migration/options.h @@ -69,6 +69,7 @@ uint32_t migrate_checkpoint_delay(void); uint8_t migrate_cpu_throttle_increment(void); uint8_t migrate_cpu_throttle_initial(void); bool migrate_cpu_throttle_tailslow(void); +bool migrate_direct_io(void); uint64_t migrate_downtime_limit(void); uint8_t migrate_max_cpu_throttle(void); uint64_t migrate_max_bandwidth(void); diff --git a/qapi/migration.json b/qapi/migration.json index a351fd3714..7efbaf1e25 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -812,6 +812,10 @@ # See description in @ZeroPageDetection. Default is 'multifd'. # (since 9.0) # +# @direct-io: Open migration files with O_DIRECT when possible. This +# only has effect if the @mapped-ram capability is enabled. (since +# 9.1) +# # Features: # # @unstable: Members @x-checkpoint-delay and @@ -836,7 +840,8 @@ { 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable'] }, 'vcpu-dirty-limit', 'mode', - 'zero-page-detection'] } + 'zero-page-detection', + 'direct-io'] } ## # @MigrateSetParameters: @@ -982,6 +987,10 @@ # See description in @ZeroPageDetection. Default is 'multifd'. # (since 9.0) # +# @direct-io: Open migration files with O_DIRECT when possible. This +# only has effect if the @mapped-ram capability is enabled. (since +# 9.1) +# # Features: # # @unstable: Members @x-checkpoint-delay and @@ -1021,7 +1030,8 @@ 'features': [ 'unstable' ] }, '*vcpu-dirty-limit': 'uint64', '*mode': 'MigMode', - '*zero-page-detection': 'ZeroPageDetection'} } + '*zero-page-detection': 'ZeroPageDetection', + '*direct-io': 'bool' } } ## # @migrate-set-parameters: @@ -1181,6 +1191,10 @@ # See description in @ZeroPageDetection. Default is 'multifd'. # (since 9.0) # +# @direct-io: Open migration files with O_DIRECT when possible. This +# only has effect if the @mapped-ram capability is enabled. (since +# 9.1) +# # Features: # # @unstable: Members @x-checkpoint-delay and @@ -1217,7 +1231,8 @@ 'features': [ 'unstable' ] }, '*vcpu-dirty-limit': 'uint64', '*mode': 'MigMode', - '*zero-page-detection': 'ZeroPageDetection'} } + '*zero-page-detection': 'ZeroPageDetection', + '*direct-io': 'bool' } } ## # @query-migrate-parameters: diff --git a/util/osdep.c b/util/osdep.c index 0cb8f98bf6..4900251d2e 100644 --- a/util/osdep.c +++ b/util/osdep.c @@ -277,6 +277,15 @@ int qemu_lock_fd_test(int fd, int64_t start, int64_t len, bool exclusive) } #endif +bool qemu_has_direct_io(void) +{ +#ifdef O_DIRECT + return true; +#else + return false; +#endif +} + static int qemu_open_cloexec(const char *name, int flags, mode_t mode) { int ret; From patchwork Thu May 23 19:05:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672204 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3999BC41513 for ; Thu, 23 May 2024 19:07:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADmF-0002WZ-Nk; Thu, 23 May 2024 15:07:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlq-0002Fi-Sj for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:35 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADle-0004Jc-3U for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:34 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id A779E22B7B; Thu, 23 May 2024 19:06:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491180; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8VFeZYA/tlbO8xdmHq8foYs81r9KIVu31YTivWBguyo=; b=McewFK8zqbiOSLaFbb/jo9OJatCvG/ll9gXipuNvt/yCHNqDWtIy7s0fUJOERguHusBtFQ BoyNJN+YPJpxEUXfX1/e295eMnYv2W4xce2kMMaQ90ulQfsir4Bj+xZr935ZHVG/CawsrX hQZL5qijfC3174EMeHI+eRgvCSfta4I= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491180; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8VFeZYA/tlbO8xdmHq8foYs81r9KIVu31YTivWBguyo=; b=UwQBa7n20vpNieeDKwdwqliP4PcbQbyqW47EqDBsASpgHc9srvvY2zCNugRkH5bVU0tA0v LnXOqjSxFwVM3fDw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=McewFK8z; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=UwQBa7n2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491180; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8VFeZYA/tlbO8xdmHq8foYs81r9KIVu31YTivWBguyo=; b=McewFK8zqbiOSLaFbb/jo9OJatCvG/ll9gXipuNvt/yCHNqDWtIy7s0fUJOERguHusBtFQ BoyNJN+YPJpxEUXfX1/e295eMnYv2W4xce2kMMaQ90ulQfsir4Bj+xZr935ZHVG/CawsrX hQZL5qijfC3174EMeHI+eRgvCSfta4I= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491180; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8VFeZYA/tlbO8xdmHq8foYs81r9KIVu31YTivWBguyo=; b=UwQBa7n20vpNieeDKwdwqliP4PcbQbyqW47EqDBsASpgHc9srvvY2zCNugRkH5bVU0tA0v LnXOqjSxFwVM3fDw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D057C13A6B; Thu, 23 May 2024 19:06:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cE57JaqTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:18 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig Subject: [PATCH v2 11/18] migration/multifd: Add direct-io support Date: Thu, 23 May 2024 16:05:41 -0300 Message-Id: <20240523190548.23977-12-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Action: no action X-Rspamd-Queue-Id: A779E22B7B X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_FIVE(0.00)[6]; DKIM_TRACE(0.00)[suse.de:+] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org When multifd is used along with mapped-ram, we can take benefit of a filesystem that supports the O_DIRECT flag and perform direct I/O in the multifd threads. This brings a significant performance improvement because direct-io writes bypass the page cache which would otherwise be thrashed by the multifd data which is unlikely to be needed again in a short period of time. To be able to use a multifd channel opened with O_DIRECT, we must ensure that a certain aligment is used. Filesystems usually require a block-size alignment for direct I/O. The way to achieve this is by enabling the mapped-ram feature, which already aligns its I/O properly (see MAPPED_RAM_FILE_OFFSET_ALIGNMENT at ram.c). By setting O_DIRECT on the multifd channels, all writes to the same file descriptor need to be aligned as well, even the ones that come from outside multifd, such as the QEMUFile I/O from the main migration code. This makes it impossible to use the same file descriptor for the QEMUFile and for the multifd channels. The various flags and metadata written by the main migration code will always be unaligned by virtue of their small size. To workaround this issue, we'll require a second file descriptor to be used exclusively for direct I/O. The second file descriptor can be obtained by QEMU by re-opening the migration file (already possible), or by being provided by the user or management application (support to be added in future patches). Signed-off-by: Fabiano Rosas --- migration/file.c | 31 ++++++++++++++++++++++++++----- migration/file.h | 1 - migration/migration.c | 23 +++++++++++++++++++++++ 3 files changed, 49 insertions(+), 6 deletions(-) diff --git a/migration/file.c b/migration/file.c index ba5b5c44ff..ac4d206492 100644 --- a/migration/file.c +++ b/migration/file.c @@ -50,12 +50,31 @@ void file_cleanup_outgoing_migration(void) outgoing_args.fname = NULL; } +static void file_enable_direct_io(int *flags) +{ +#ifdef O_DIRECT + if (migrate_direct_io()) { + *flags |= O_DIRECT; + } +#else + /* it should have been rejected when setting the parameter */ + g_assert_not_reached(); +#endif +} + bool file_send_channel_create(gpointer opaque, Error **errp) { QIOChannelFile *ioc; int flags = O_WRONLY; bool ret = true; + /* + * Attempt to enable O_DIRECT for the secondary channels. These + * are used for sending ram pages and writes should be guaranteed + * to be aligned to at least page size. + */ + file_enable_direct_io(&flags); + ioc = qio_channel_file_new_path(outgoing_args.fname, flags, 0, errp); if (!ioc) { ret = false; @@ -116,21 +135,23 @@ static gboolean file_accept_incoming_migration(QIOChannel *ioc, return G_SOURCE_REMOVE; } -void file_create_incoming_channels(QIOChannel *ioc, Error **errp) +static void file_create_incoming_channels(QIOChannel *ioc, char *filename, + Error **errp) { - int i, fd, channels = 1; + int i, channels = 1; g_autofree QIOChannel **iocs = NULL; + int flags = O_RDONLY; if (migrate_multifd()) { channels += migrate_multifd_channels(); + file_enable_direct_io(&flags); } iocs = g_new0(QIOChannel *, channels); - fd = QIO_CHANNEL_FILE(ioc)->fd; iocs[0] = ioc; for (i = 1; i < channels; i++) { - QIOChannelFile *fioc = qio_channel_file_new_dupfd(fd, errp); + QIOChannelFile *fioc = qio_channel_file_new_path(filename, flags, 0, errp); if (!fioc) { while (i) { @@ -170,7 +191,7 @@ void file_start_incoming_migration(FileMigrationArgs *file_args, Error **errp) return; } - file_create_incoming_channels(QIO_CHANNEL(fioc), errp); + file_create_incoming_channels(QIO_CHANNEL(fioc), filename, errp); } int file_write_ramblock_iov(QIOChannel *ioc, const struct iovec *iov, diff --git a/migration/file.h b/migration/file.h index 7699c04677..9f71e87f74 100644 --- a/migration/file.h +++ b/migration/file.h @@ -20,7 +20,6 @@ void file_start_outgoing_migration(MigrationState *s, int file_parse_offset(char *filespec, uint64_t *offsetp, Error **errp); void file_cleanup_outgoing_migration(void); bool file_send_channel_create(gpointer opaque, Error **errp); -void file_create_incoming_channels(QIOChannel *ioc, Error **errp); int file_write_ramblock_iov(QIOChannel *ioc, const struct iovec *iov, int niov, RAMBlock *block, Error **errp); int multifd_file_recv_data(MultiFDRecvParams *p, Error **errp); diff --git a/migration/migration.c b/migration/migration.c index e1b269624c..e03c80b3aa 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -155,6 +155,16 @@ static bool migration_needs_seekable_channel(void) return migrate_mapped_ram(); } +static bool migration_needs_extra_fds(void) +{ + /* + * When doing direct-io, multifd requires two different, + * non-duplicated file descriptors so we can use one of them for + * unaligned IO. + */ + return migrate_multifd() && migrate_direct_io(); +} + static bool transport_supports_seeking(MigrationAddress *addr) { if (addr->transport == MIGRATION_ADDRESS_TYPE_FILE) { @@ -164,6 +174,12 @@ static bool transport_supports_seeking(MigrationAddress *addr) return false; } +static bool transport_supports_extra_fds(MigrationAddress *addr) +{ + /* file: works because QEMU can open it multiple times */ + return addr->transport == MIGRATION_ADDRESS_TYPE_FILE; +} + static bool migration_channels_and_transport_compatible(MigrationAddress *addr, Error **errp) @@ -180,6 +196,13 @@ migration_channels_and_transport_compatible(MigrationAddress *addr, return false; } + if (migration_needs_extra_fds() && + !transport_supports_extra_fds(addr)) { + error_setg(errp, + "Migration requires a transport that allows for extra fds (e.g. file)"); + return false; + } + return true; } From patchwork Thu May 23 19:05:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672208 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 178C5C25B75 for ; Thu, 23 May 2024 19:08:17 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADmM-0002oM-5P; Thu, 23 May 2024 15:07:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADls-0002G6-5w for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:36 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlg-0004Jy-Ti for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:35 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id A977622B7D; Thu, 23 May 2024 19:06:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491183; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qPONvFeKbrPhZNJo/pMWwmXaBvgYfqthr1wvcEaHRu8=; b=ghh3I/AHxO4uJnVve+BowpoO5G3mMSmQPMDGB8usf+0jLLXGkTp95O+ZgV8+uiGsNH7Xk0 IwquriwCP57HTTkkCLV29jFL4TJ5MYutXGFmhRMFIuMsezbktmwW03jQG78QfhxaqwhXWN jMhlfp3DmlGZ0aqjUJaUemjE5aq2VZs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491183; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qPONvFeKbrPhZNJo/pMWwmXaBvgYfqthr1wvcEaHRu8=; b=FqGCzEgRe7qD4BhxBb1BgfYsjIn2PqveN5i9fVDRwlxgBqT8ZhB2MmS8t7dr7I4wo5YqcF 7n2yBumezdz+4oBQ== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="ghh3I/AH"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=FqGCzEgR DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491183; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qPONvFeKbrPhZNJo/pMWwmXaBvgYfqthr1wvcEaHRu8=; b=ghh3I/AHxO4uJnVve+BowpoO5G3mMSmQPMDGB8usf+0jLLXGkTp95O+ZgV8+uiGsNH7Xk0 IwquriwCP57HTTkkCLV29jFL4TJ5MYutXGFmhRMFIuMsezbktmwW03jQG78QfhxaqwhXWN jMhlfp3DmlGZ0aqjUJaUemjE5aq2VZs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491183; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qPONvFeKbrPhZNJo/pMWwmXaBvgYfqthr1wvcEaHRu8=; b=FqGCzEgRe7qD4BhxBb1BgfYsjIn2PqveN5i9fVDRwlxgBqT8ZhB2MmS8t7dr7I4wo5YqcF 7n2yBumezdz+4oBQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 2898613A6B; Thu, 23 May 2024 19:06:20 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id kLQFOKyTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:20 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH v2 12/18] tests/qtest/migration: Add tests for file migration with direct-io Date: Thu, 23 May 2024 16:05:42 -0300 Message-Id: <20240523190548.23977-13-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCVD_TLS_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCPT_COUNT_SEVEN(0.00)[9]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim, suse.de:email, imap1.dmz-prg2.suse.org:helo, imap1.dmz-prg2.suse.org:rdns] X-Rspamd-Action: no action X-Rspamd-Queue-Id: A977622B7D X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The tests are only allowed to run in systems that know about the O_DIRECT flag and in filesystems which support it. Note: this also brings back migrate_set_parameter_bool() which went away when we removed the compression tests. I copied it verbatim. Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- - updated the aligment comment - added an inline version of probe function so were able to call it without any ifdef - simplified the _start function that sets the capability - used qemu_try_memalign instead of aligned_alloc --- tests/qtest/migration-helpers.c | 44 +++++++++++++++++++++++ tests/qtest/migration-helpers.h | 8 +++++ tests/qtest/migration-test.c | 62 +++++++++++++++++++++++++++++++++ 3 files changed, 114 insertions(+) diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helpers.c index ce6d6615b5..0ac49ceb54 100644 --- a/tests/qtest/migration-helpers.c +++ b/tests/qtest/migration-helpers.c @@ -18,6 +18,7 @@ #include "qapi/error.h" #include "qapi/qmp/qlist.h" #include "qemu/cutils.h" +#include "qemu/memalign.h" #include "migration-helpers.h" @@ -473,3 +474,46 @@ void migration_test_add(const char *path, void (*fn)(void)) qtest_add_data_func_full(path, test, migration_test_wrapper, migration_test_destroy); } + +#ifdef O_DIRECT +/* + * Probe for O_DIRECT support on the filesystem. Since this is used + * for tests, be conservative, if anything fails, assume it's + * unsupported. + */ +bool probe_o_direct_support(const char *tmpfs) +{ + g_autofree char *filename = g_strdup_printf("%s/probe-o-direct", tmpfs); + int fd, flags = O_CREAT | O_RDWR | O_TRUNC | O_DIRECT; + void *buf; + ssize_t ret, len; + uint64_t offset; + + fd = open(filename, flags, 0660); + if (fd < 0) { + unlink(filename); + return false; + } + + /* + * Using 1MB alignment as conservative choice to satisfy any + * plausible architecture default page size, and/or filesystem + * alignment restrictions. + */ + len = 0x100000; + offset = 0x100000; + + buf = qemu_try_memalign(len, len); + g_assert(buf); + + ret = pwrite(fd, buf, len, offset); + unlink(filename); + g_free(buf); + + if (ret < 0) { + return false; + } + + return true; +} +#endif diff --git a/tests/qtest/migration-helpers.h b/tests/qtest/migration-helpers.h index 1339835698..50095fca4a 100644 --- a/tests/qtest/migration-helpers.h +++ b/tests/qtest/migration-helpers.h @@ -54,5 +54,13 @@ char *find_common_machine_version(const char *mtype, const char *var1, const char *var2); char *resolve_machine_version(const char *alias, const char *var1, const char *var2); +#ifdef O_DIRECT +bool probe_o_direct_support(const char *tmpfs); +#else +static inline bool probe_o_direct_support(const char *tmpfs) +{ + return false; +} +#endif void migration_test_add(const char *path, void (*fn)(void)); #endif /* MIGRATION_HELPERS_H */ diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 5bb53f4839..37017ff2d5 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -428,6 +428,38 @@ static void migrate_set_parameter_str(QTestState *who, const char *parameter, migrate_check_parameter_str(who, parameter, value); } +static long long migrate_get_parameter_bool(QTestState *who, + const char *parameter) +{ + QDict *rsp; + int result; + + rsp = qtest_qmp_assert_success_ref( + who, "{ 'execute': 'query-migrate-parameters' }"); + result = qdict_get_bool(rsp, parameter); + qobject_unref(rsp); + return !!result; +} + +static void migrate_check_parameter_bool(QTestState *who, const char *parameter, + int value) +{ + int result; + + result = migrate_get_parameter_bool(who, parameter); + g_assert_cmpint(result, ==, value); +} + +static void migrate_set_parameter_bool(QTestState *who, const char *parameter, + int value) +{ + qtest_qmp_assert_success(who, + "{ 'execute': 'migrate-set-parameters'," + "'arguments': { %s: %i } }", + parameter, value); + migrate_check_parameter_bool(who, parameter, value); +} + static void migrate_ensure_non_converge(QTestState *who) { /* Can't converge with 1ms downtime + 3 mbs bandwidth limit */ @@ -2178,6 +2210,33 @@ static void test_multifd_file_mapped_ram(void) test_file_common(&args, true); } +static void *multifd_mapped_ram_dio_start(QTestState *from, QTestState *to) +{ + migrate_multifd_mapped_ram_start(from, to); + + migrate_set_parameter_bool(from, "direct-io", true); + migrate_set_parameter_bool(to, "direct-io", true); + + return NULL; +} + +static void test_multifd_file_mapped_ram_dio(void) +{ + g_autofree char *uri = g_strdup_printf("file:%s/%s", tmpfs, + FILE_TEST_FILENAME); + MigrateCommon args = { + .connect_uri = uri, + .listen_uri = "defer", + .start_hook = multifd_mapped_ram_dio_start, + }; + + if (!probe_o_direct_support(tmpfs)) { + g_test_skip("Filesystem does not support O_DIRECT"); + return; + } + + test_file_common(&args, true); +} static void test_precopy_tcp_plain(void) { @@ -3586,6 +3645,9 @@ int main(int argc, char **argv) migration_test_add("/migration/multifd/file/mapped-ram/live", test_multifd_file_mapped_ram_live); + migration_test_add("/migration/multifd/file/mapped-ram/dio", + test_multifd_file_mapped_ram_dio); + #ifdef CONFIG_GNUTLS migration_test_add("/migration/precopy/unix/tls/psk", test_precopy_unix_tls_psk); From patchwork Thu May 23 19:05:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1DFE3C25B75 for ; Thu, 23 May 2024 19:08:09 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADlz-0002IM-JQ; Thu, 23 May 2024 15:06:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADls-0002G2-1W for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:36 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlj-0004K5-5U for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:35 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id F1FCC22B7E; Thu, 23 May 2024 19:06:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491186; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rLoeI57E2p6A0O3qEBvqik55xUqwKlM7a7ONvohzAaA=; b=lebN6kUyDGyx5tHaTZIUPPO575NSitOLczZ9Ca7eWoiZGzyteoogwQP9OlsXsZOSaVTbGg JipJsoqie7/XNZZ47ObhW0KVu6aJsMpnyub7aCzjkx6NDu3ybpd9Vylw9cKAdETStPQM6T V4gfRNLJwrvkq1dmreSDgaUltyGs+FI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491186; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rLoeI57E2p6A0O3qEBvqik55xUqwKlM7a7ONvohzAaA=; b=wpatSGTDtjMRQf3S8NnBQ9EYSrXsJN9nCfEg8O1J90kvqRlRDtrIMK9eqWH8hesMBQ4mn6 8v/FmUAvgC1vEVDw== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491186; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rLoeI57E2p6A0O3qEBvqik55xUqwKlM7a7ONvohzAaA=; b=lebN6kUyDGyx5tHaTZIUPPO575NSitOLczZ9Ca7eWoiZGzyteoogwQP9OlsXsZOSaVTbGg JipJsoqie7/XNZZ47ObhW0KVu6aJsMpnyub7aCzjkx6NDu3ybpd9Vylw9cKAdETStPQM6T V4gfRNLJwrvkq1dmreSDgaUltyGs+FI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491186; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rLoeI57E2p6A0O3qEBvqik55xUqwKlM7a7ONvohzAaA=; b=wpatSGTDtjMRQf3S8NnBQ9EYSrXsJN9nCfEg8O1J90kvqRlRDtrIMK9eqWH8hesMBQ4mn6 8v/FmUAvgC1vEVDw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 271D413A6B; Thu, 23 May 2024 19:06:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 2CSeN6+TT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:23 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig Subject: [PATCH v2 13/18] monitor: fdset: Match against O_DIRECT Date: Thu, 23 May 2024 16:05:43 -0300 Message-Id: <20240523190548.23977-14-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We're about to enable the use of O_DIRECT in the migration code and due to the alignment restrictions imposed by filesystems we need to make sure the flag is only used when doing aligned IO. The migration will do parallel IO to different regions of a file, so we need to use more than one file descriptor. Those cannot be obtained by duplicating (dup()) since duplicated file descriptors share the file status flags, including O_DIRECT. If one migration channel does unaligned IO while another sets O_DIRECT to do aligned IO, the filesystem would fail the unaligned operation. The add-fd QMP command along with the fdset code are specifically designed to allow the user to pass a set of file descriptors with different access flags into QEMU to be later fetched by code that needs to alternate between those flags when doing IO. Extend the fdset matching to behave the same with the O_DIRECT flag. Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- monitor/fds.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/monitor/fds.c b/monitor/fds.c index d93d2e695b..a6580e215e 100644 --- a/monitor/fds.c +++ b/monitor/fds.c @@ -419,6 +419,11 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags, Error **errp) int fd = -1; int dup_fd; int mon_fd_flags; + int mask = O_ACCMODE; + +#ifdef O_DIRECT + mask |= O_DIRECT; +#endif if (mon_fdset->id != fdset_id) { continue; @@ -432,7 +437,7 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags, Error **errp) return -1; } - if ((flags & O_ACCMODE) == (mon_fd_flags & O_ACCMODE)) { + if ((flags & mask) == (mon_fd_flags & mask)) { fd = mon_fdset_fd->fd; break; } From patchwork Thu May 23 19:05:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672200 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70886C25B75 for ; Thu, 23 May 2024 19:07:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADm8-0002Ja-4T; Thu, 23 May 2024 15:06:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlt-0002GW-Lj for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:38 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlm-0004KC-0M for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:37 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 467D322B73; Thu, 23 May 2024 19:06:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491188; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+xua7VagbC1TpTuVfzZaNgQ0/sKLFKwgSqqvOnvA/00=; b=Vjsq2D853b5vLwPopVR2pmhPN5UdwSj4VJx1S/zJ0WFUL8qyIrIxKNLCpcyrd+3LCCelqb 0/RjI8oSrz4tKjTA/Ir+1gZeaScbNF3G5iBu6XzczNejOzmpP7de9V1ES47w3TcSU0dQwh 6ahQsXAw8rwRNujyoVq9NeZmrW90INo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491188; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+xua7VagbC1TpTuVfzZaNgQ0/sKLFKwgSqqvOnvA/00=; b=ffSq3vjJISf5DHZb8mmVfrI0VVORYIN/9qIOpMQV4qOCkJaZxG7yP4Vl6sFws0zzrHvwP5 +47NShxoGTNrDHBg== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=Vjsq2D85; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=ffSq3vjJ DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491188; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+xua7VagbC1TpTuVfzZaNgQ0/sKLFKwgSqqvOnvA/00=; b=Vjsq2D853b5vLwPopVR2pmhPN5UdwSj4VJx1S/zJ0WFUL8qyIrIxKNLCpcyrd+3LCCelqb 0/RjI8oSrz4tKjTA/Ir+1gZeaScbNF3G5iBu6XzczNejOzmpP7de9V1ES47w3TcSU0dQwh 6ahQsXAw8rwRNujyoVq9NeZmrW90INo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491188; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+xua7VagbC1TpTuVfzZaNgQ0/sKLFKwgSqqvOnvA/00=; b=ffSq3vjJISf5DHZb8mmVfrI0VVORYIN/9qIOpMQV4qOCkJaZxG7yP4Vl6sFws0zzrHvwP5 +47NShxoGTNrDHBg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6F72313A6B; Thu, 23 May 2024 19:06:26 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id aPnMDbKTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:26 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig Subject: [PATCH v2 14/18] migration: Add documentation for fdset with multifd + file Date: Thu, 23 May 2024 16:05:44 -0300 Message-Id: <20240523190548.23977-15-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:dkim,suse.de:email]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_FIVE(0.00)[6]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Action: no action X-Rspamd-Queue-Id: 467D322B73 X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org With the last few changes to the fdset infrastructure, we now allow multifd to use an fdset when migrating to a file. This is useful for the scenario where the management layer wants to have control over the migration file. By receiving the file descriptors directly, QEMU can delegate some high level operating system operations to the management layer (such as mandatory access control). The management layer might also want to add its own headers before the migration stream. Document the "file:/dev/fdset/#" syntax for the multifd migration with mapped-ram. The requirements for the fdset mechanism are: - the fdset must contain two fds that are not duplicates between themselves; - if direct-io is to be used, exactly one of the fds must have the O_DIRECT flag set; - the file must be opened with WRONLY on the migration source side; - the file must be opened with RDONLY on the migration destination side. Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- docs/devel/migration/main.rst | 24 +++++++++++++++++++----- docs/devel/migration/mapped-ram.rst | 6 +++++- 2 files changed, 24 insertions(+), 6 deletions(-) diff --git a/docs/devel/migration/main.rst b/docs/devel/migration/main.rst index 495cdcb112..784c899dca 100644 --- a/docs/devel/migration/main.rst +++ b/docs/devel/migration/main.rst @@ -47,11 +47,25 @@ over any transport. QEMU interference. Note that QEMU does not flush cached file data/metadata at the end of migration. -In addition, support is included for migration using RDMA, which -transports the page data using ``RDMA``, where the hardware takes care of -transporting the pages, and the load on the CPU is much lower. While the -internals of RDMA migration are a bit different, this isn't really visible -outside the RAM migration code. + The file migration also supports using a file that has already been + opened. A set of file descriptors is passed to QEMU via an "fdset" + (see add-fd QMP command documentation). This method allows a + management application to have control over the migration file + opening operation. There are, however, strict requirements to this + interface if the multifd capability is enabled: + + - the fdset must contain two file descriptors that are not + duplicates between themselves; + - if the direct-io capability is to be used, exactly one of the + file descriptors must have the O_DIRECT flag set; + - the file must be opened with WRONLY on the migration source side + and RDONLY on the migration destination side. + +- rdma migration: support is included for migration using RDMA, which + transports the page data using ``RDMA``, where the hardware takes + care of transporting the pages, and the load on the CPU is much + lower. While the internals of RDMA migration are a bit different, + this isn't really visible outside the RAM migration code. All these migration protocols use the same infrastructure to save/restore state devices. This infrastructure is shared with the diff --git a/docs/devel/migration/mapped-ram.rst b/docs/devel/migration/mapped-ram.rst index fa4cefd9fc..e6505511f0 100644 --- a/docs/devel/migration/mapped-ram.rst +++ b/docs/devel/migration/mapped-ram.rst @@ -16,7 +16,7 @@ location in the file, rather than constantly being added to a sequential stream. Having the pages at fixed offsets also allows the usage of O_DIRECT for save/restore of the migration stream as the pages are ensured to be written respecting O_DIRECT alignment -restrictions (direct-io support not yet implemented). +restrictions. Usage ----- @@ -35,6 +35,10 @@ Use a ``file:`` URL for migration: Mapped-ram migration is best done non-live, i.e. by stopping the VM on the source side before migrating. +For best performance enable the ``direct-io`` capability as well: + + ``migrate_set_capability direct-io on`` + Use-cases --------- From patchwork Thu May 23 19:05:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 39969C25B7A for ; Thu, 23 May 2024 19:07:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADmM-0002v8-UU; Thu, 23 May 2024 15:07:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlt-0002GX-Ng for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:38 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlp-0004KJ-Kq for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:37 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 3D6E32040D; Thu, 23 May 2024 19:06:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491191; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sniV4ADiqxDqLReNOG6JjHF4fkYiKMuydvXvoREL6Bs=; b=zau8v39nBp01WAQq6alJ19sPqOuwXSnY7WnMEFruDqsiO9z+Vp9pXyjeekwsWq8xTg38I6 bgmR1MXMjdExOYies9lcqIIhC6qTUerdxIS2FTlVPOSMtuR4OmK/SVA+XK12SNNc4p7RrJ CmttIeoVZbcfPc0TPs31i/PZ2jr1n1A= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491191; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sniV4ADiqxDqLReNOG6JjHF4fkYiKMuydvXvoREL6Bs=; b=l33cmH4A5PczwiVR3Bdo9C2vUFTimb2dVa9Ix1m2nWP3IhRk8BTIyt8p9K8T1sIeP4J3Sz 4fcI2715YLvvvKCw== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491191; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sniV4ADiqxDqLReNOG6JjHF4fkYiKMuydvXvoREL6Bs=; b=zau8v39nBp01WAQq6alJ19sPqOuwXSnY7WnMEFruDqsiO9z+Vp9pXyjeekwsWq8xTg38I6 bgmR1MXMjdExOYies9lcqIIhC6qTUerdxIS2FTlVPOSMtuR4OmK/SVA+XK12SNNc4p7RrJ CmttIeoVZbcfPc0TPs31i/PZ2jr1n1A= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491191; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sniV4ADiqxDqLReNOG6JjHF4fkYiKMuydvXvoREL6Bs=; b=l33cmH4A5PczwiVR3Bdo9C2vUFTimb2dVa9Ix1m2nWP3IhRk8BTIyt8p9K8T1sIeP4J3Sz 4fcI2715YLvvvKCw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B658E13A6B; Thu, 23 May 2024 19:06:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id oJ8oH7STT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:28 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH v2 15/18] tests/qtest/migration: Add a test for mapped-ram with passing of fds Date: Thu, 23 May 2024 16:05:45 -0300 Message-Id: <20240523190548.23977-16-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_SEVEN(0.00)[9]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,imap1.dmz-prg2.suse.org:helo]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add a multifd test for mapped-ram with passing of fds into QEMU. This is how libvirt will consume the feature. There are a couple of details to the fdset mechanism: - multifd needs two distinct file descriptors (not duplicated with dup()) so it can enable O_DIRECT only on the channels that do aligned IO. The dup() system call creates file descriptors that share status flags, of which O_DIRECT is one. - the open() access mode flags used for the fds passed into QEMU need to match the flags QEMU uses to open the file. Currently O_WRONLY for src and O_RDONLY for dst. Note that fdset code goes under _WIN32 because fd passing is not supported on Windows. Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- tests/qtest/migration-test.c | 101 +++++++++++++++++++++++++++++++++-- 1 file changed, 98 insertions(+), 3 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 37017ff2d5..5ced3b90c9 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2047,11 +2047,18 @@ static void test_precopy_file(void) #ifndef _WIN32 static void fdset_add_fds(QTestState *qts, const char *file, int flags, - int num_fds) + int num_fds, bool direct_io) { for (int i = 0; i < num_fds; i++) { int fd; +#ifdef O_DIRECT + /* only secondary channels can use direct-io */ + if (direct_io && i != 0) { + flags |= O_DIRECT; + } +#endif + fd = open(file, flags, 0660); assert(fd != -1); @@ -2065,8 +2072,8 @@ static void *file_offset_fdset_start_hook(QTestState *from, QTestState *to) { g_autofree char *file = g_strdup_printf("%s/%s", tmpfs, FILE_TEST_FILENAME); - fdset_add_fds(from, file, O_WRONLY, 1); - fdset_add_fds(to, file, O_RDONLY, 1); + fdset_add_fds(from, file, O_WRONLY, 1, false); + fdset_add_fds(to, file, O_RDONLY, 1, false); return NULL; } @@ -2238,6 +2245,87 @@ static void test_multifd_file_mapped_ram_dio(void) test_file_common(&args, true); } +#ifndef _WIN32 +static void multifd_mapped_ram_fdset_end(QTestState *from, QTestState *to, + void *opaque) +{ + QDict *resp; + QList *fdsets; + + /* + * Remove the fdsets after migration, otherwise a second migration + * would fail due fdset reuse. + */ + qtest_qmp_assert_success(from, "{'execute': 'remove-fd', " + "'arguments': { 'fdset-id': 1}}"); + + resp = qtest_qmp(from, "{'execute': 'query-fdsets', " + "'arguments': {}}"); + g_assert(qdict_haskey(resp, "return")); + fdsets = qdict_get_qlist(resp, "return"); + g_assert(fdsets && qlist_empty(fdsets)); +} + +static void *multifd_mapped_ram_fdset_dio(QTestState *from, QTestState *to) +{ + g_autofree char *file = g_strdup_printf("%s/%s", tmpfs, FILE_TEST_FILENAME); + + fdset_add_fds(from, file, O_WRONLY, 2, true); + fdset_add_fds(to, file, O_RDONLY, 2, true); + + migrate_multifd_mapped_ram_start(from, to); + migrate_set_parameter_bool(from, "direct-io", true); + migrate_set_parameter_bool(to, "direct-io", true); + + return NULL; +} + +static void *multifd_mapped_ram_fdset(QTestState *from, QTestState *to) +{ + g_autofree char *file = g_strdup_printf("%s/%s", tmpfs, FILE_TEST_FILENAME); + + fdset_add_fds(from, file, O_WRONLY, 2, false); + fdset_add_fds(to, file, O_RDONLY, 2, false); + + migrate_multifd_mapped_ram_start(from, to); + + return NULL; +} + +static void test_multifd_file_mapped_ram_fdset(void) +{ + g_autofree char *uri = g_strdup_printf("file:/dev/fdset/1,offset=%d", + FILE_TEST_OFFSET); + MigrateCommon args = { + .connect_uri = uri, + .listen_uri = "defer", + .start_hook = multifd_mapped_ram_fdset, + .finish_hook = multifd_mapped_ram_fdset_end, + }; + + test_file_common(&args, true); +} + +static void test_multifd_file_mapped_ram_fdset_dio(void) +{ + g_autofree char *uri = g_strdup_printf("file:/dev/fdset/1,offset=%d", + FILE_TEST_OFFSET); + MigrateCommon args = { + .connect_uri = uri, + .listen_uri = "defer", + .start_hook = multifd_mapped_ram_fdset_dio, + .finish_hook = multifd_mapped_ram_fdset_end, + }; + + if (!probe_o_direct_support(tmpfs)) { + g_test_skip("Filesystem does not support O_DIRECT"); + return; + } + + test_file_common(&args, true); +} +#endif /* !_WIN32 */ + static void test_precopy_tcp_plain(void) { MigrateCommon args = { @@ -3648,6 +3736,13 @@ int main(int argc, char **argv) migration_test_add("/migration/multifd/file/mapped-ram/dio", test_multifd_file_mapped_ram_dio); +#ifndef _WIN32 + qtest_add_func("/migration/multifd/file/mapped-ram/fdset", + test_multifd_file_mapped_ram_fdset); + qtest_add_func("/migration/multifd/file/mapped-ram/fdset/dio", + test_multifd_file_mapped_ram_fdset_dio); +#endif + #ifdef CONFIG_GNUTLS migration_test_add("/migration/precopy/unix/tls/psk", test_precopy_unix_tls_psk); From patchwork Thu May 23 19:05:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7C54AC41513 for ; Thu, 23 May 2024 19:07:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADmQ-0003Ff-AA; Thu, 23 May 2024 15:07:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlu-0002GY-5h for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:38 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlq-0004KZ-Ua for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:37 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 882DE20410; Thu, 23 May 2024 19:06:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491193; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vl2VTZZ7htmokSj3DhF+Hd+04nY+uwgKE6y9zGEpmro=; b=SKqit/j3lG+8BaKprymQ99mprirziwP2Kal5VikfXG5957aT+1BEx8efYzaAt3dirmLNaU FT2m8zTgBtdWdhxw4FaC3JwFBkzx2GSC8xp5FTUFPEDNXepRONAm6orY7Aj7gCXXNMapdr zBN9nRFOOd89Smen8+K4AlIj5wKeE9M= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491193; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vl2VTZZ7htmokSj3DhF+Hd+04nY+uwgKE6y9zGEpmro=; b=9NOZV9IvsJdEGJbF0v2c4WgQVUg3Z7r3WX6UCgCV3XpfYg1zJ2Qr3H+aR6N9Hs+HifM6jp gmT3QsiD2Z42U4DQ== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="SKqit/j3"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=9NOZV9Iv DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491193; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vl2VTZZ7htmokSj3DhF+Hd+04nY+uwgKE6y9zGEpmro=; b=SKqit/j3lG+8BaKprymQ99mprirziwP2Kal5VikfXG5957aT+1BEx8efYzaAt3dirmLNaU FT2m8zTgBtdWdhxw4FaC3JwFBkzx2GSC8xp5FTUFPEDNXepRONAm6orY7Aj7gCXXNMapdr zBN9nRFOOd89Smen8+K4AlIj5wKeE9M= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491193; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vl2VTZZ7htmokSj3DhF+Hd+04nY+uwgKE6y9zGEpmro=; b=9NOZV9IvsJdEGJbF0v2c4WgQVUg3Z7r3WX6UCgCV3XpfYg1zJ2Qr3H+aR6N9Hs+HifM6jp gmT3QsiD2Z42U4DQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B004F13A6B; Thu, 23 May 2024 19:06:31 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id MOKiHbeTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:31 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig Subject: [PATCH v2 16/18] io/channel-file: Add direct-io support Date: Thu, 23 May 2024 16:05:46 -0300 Message-Id: <20240523190548.23977-17-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCPT_COUNT_FIVE(0.00)[6]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Action: no action X-Rspamd-Queue-Id: 882DE20410 X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add support for setting/clearing the O_DIRECT flag on a file descriptor. This will be used for enabling O_DIRECT in the main migration channel when multifd is not in use. Signed-off-by: Fabiano Rosas --- include/io/channel-file.h | 8 ++++++++ io/channel-file.c | 25 +++++++++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/include/io/channel-file.h b/include/io/channel-file.h index d373a4e44d..ecb4450e8e 100644 --- a/include/io/channel-file.h +++ b/include/io/channel-file.h @@ -107,4 +107,12 @@ qio_channel_file_new_path(const char *path, mode_t mode, Error **errp); +/** + * qio_channel_file_set_direct_io: + * @ioc: the QIOChannel object + * @enabled: the desired state of the O_DIRECT flag + * @errp: pointer to initialized error object + */ +void qio_channel_file_set_direct_io(QIOChannel *ioc, bool enabled, + Error **errp); #endif /* QIO_CHANNEL_FILE_H */ diff --git a/io/channel-file.c b/io/channel-file.c index 2ea8d08360..a89cd3a6d5 100644 --- a/io/channel-file.c +++ b/io/channel-file.c @@ -231,6 +231,31 @@ static int qio_channel_file_set_blocking(QIOChannel *ioc, #endif } +void qio_channel_file_set_direct_io(QIOChannel *ioc, bool enabled, Error **errp) +{ +#ifdef O_DIRECT + QIOChannelFile *fioc = QIO_CHANNEL_FILE(ioc); + int flags = fcntl(fioc->fd, F_GETFL); + + if (flags == -1) { + error_setg_errno(errp, errno, "Unable to read file descriptor flags"); + return; + } + + if (enabled) { + flags |= O_DIRECT; + } else { + flags &= ~O_DIRECT; + } + + if (fcntl(fioc->fd, F_SETFL, flags) == -1) { + error_setg_errno(errp, errno, "Unable to set file descriptor flags"); + return; + } +#else + error_setg(errp, "System does not support O_DIRECT"); +#endif +} static off_t qio_channel_file_seek(QIOChannel *ioc, off_t offset, From patchwork Thu May 23 19:05:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672212 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E541AC25B75 for ; Thu, 23 May 2024 19:08:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADmK-0002hn-Hs; Thu, 23 May 2024 15:07:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlv-0002Gb-GI for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:39 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlt-0004LD-2Q for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:38 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D368220412; Thu, 23 May 2024 19:06:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491195; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OjnxH7khDXmrflJOrKVMiCdjvXgeY6BwXBZDPxXoGaw=; b=g/9ZYB2QtEraUnqA+pfGWx0n5HhZMWF0ebivw5RKRWS3ZdrBRCqTP55e/xnuZExUo9Xmdk iZrS16H44a2SU5DMy3oMl9gmyh0mO3LQPhNkyZjFbe0f2veqRxwWJs5xpfiQhbGxFvc+Pz 2IqKTWHn8AK1lpY8/ZOIGZKhQq3aPo8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491195; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OjnxH7khDXmrflJOrKVMiCdjvXgeY6BwXBZDPxXoGaw=; b=gaYFYbhX1BLt/DTndEKFkFGnJ0+YZQKdt0OKC9y0OcczkiKEmapjhWxiti6/nU58KCP1Lm gSgBRzD/c+R1xRBA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491195; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OjnxH7khDXmrflJOrKVMiCdjvXgeY6BwXBZDPxXoGaw=; b=g/9ZYB2QtEraUnqA+pfGWx0n5HhZMWF0ebivw5RKRWS3ZdrBRCqTP55e/xnuZExUo9Xmdk iZrS16H44a2SU5DMy3oMl9gmyh0mO3LQPhNkyZjFbe0f2veqRxwWJs5xpfiQhbGxFvc+Pz 2IqKTWHn8AK1lpY8/ZOIGZKhQq3aPo8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491195; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OjnxH7khDXmrflJOrKVMiCdjvXgeY6BwXBZDPxXoGaw=; b=gaYFYbhX1BLt/DTndEKFkFGnJ0+YZQKdt0OKC9y0OcczkiKEmapjhWxiti6/nU58KCP1Lm gSgBRzD/c+R1xRBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0A65813A6B; Thu, 23 May 2024 19:06:33 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id APrUMLmTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:33 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig Subject: [PATCH v2 17/18] migration: Add direct-io helpers Date: Thu, 23 May 2024 16:05:47 -0300 Message-Id: <20240523190548.23977-18-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,imap1.dmz-prg2.suse.org:helo]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We support using O_DIRECT with multifd by isolating the main migration channel which does unaligned IO from the multifd channels that do aligned IO. When multifd is not enabled, we can still use O_DIRECT, if we judiciously enable/disable the flag to avoid the unaligned IO. Add helpers to enable/disable direct-io around the aligned parts. Signed-off-by: Fabiano Rosas --- migration/migration.c | 31 +++++++++++++++++++++++++++++++ migration/migration.h | 2 ++ migration/qemu-file.c | 29 +++++++++++++++++++++++++++++ migration/qemu-file.h | 2 +- 4 files changed, 63 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index e03c80b3aa..be128f95da 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -420,6 +420,37 @@ static void migrate_generate_event(int new_state) } } +bool migration_direct_io_start(QEMUFile *file, Error **errp) +{ + if (!migrate_direct_io() || migrate_multifd()) { + return true; + } + + /* flush any potentially unaligned IO before setting O_DIRECT */ + qemu_fflush(file); + + if (!qemu_file_set_direct_io(file, true)) { + error_setg(errp, "Failed to enable direct-io"); + return false; + } + + return true; +} + +bool migration_direct_io_finish(QEMUFile *file, Error **errp) +{ + if (!migrate_direct_io() || migrate_multifd()) { + return true; + } + + if (!qemu_file_set_direct_io(file, false)) { + error_setg(errp, "Failed to disable direct-io"); + return false; + } + + return true; +} + /* * Send a message on the return channel back to the source * of the migration. diff --git a/migration/migration.h b/migration/migration.h index 6af01362d4..4d808563b5 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -536,4 +536,6 @@ int migration_rp_wait(MigrationState *s); */ void migration_rp_kick(MigrationState *s); +bool migration_direct_io_start(QEMUFile *file, Error **errp); +bool migration_direct_io_finish(QEMUFile *file, Error **errp); #endif diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 9ccbbb0099..4796be918f 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -52,6 +52,11 @@ struct QEMUFile { int last_error; Error *last_error_obj; + /* + * Whether O_DIRECT is in effect. Used to catch code attempting + * unaligned IO. + */ + bool dio; }; /* @@ -281,6 +286,9 @@ int qemu_fflush(QEMUFile *f) } if (f->iovcnt > 0) { Error *local_error = NULL; + + assert(!f->dio); + if (qio_channel_writev_all(f->ioc, f->iov, f->iovcnt, &local_error) < 0) { @@ -429,6 +437,8 @@ void qemu_put_buffer_async(QEMUFile *f, const uint8_t *buf, size_t size, return; } + assert(!f->dio); + add_to_iovec(f, buf, size, may_free); } @@ -440,6 +450,8 @@ void qemu_put_buffer(QEMUFile *f, const uint8_t *buf, size_t size) return; } + assert(!f->dio); + while (size > 0) { l = IO_BUF_SIZE - f->buf_index; if (l > size) { @@ -558,6 +570,8 @@ off_t qemu_get_offset(QEMUFile *f) void qemu_put_byte(QEMUFile *f, int v) { + assert(!f->dio); + if (f->last_error) { return; } @@ -865,3 +879,18 @@ int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size) return 0; } + +bool qemu_file_set_direct_io(QEMUFile *f, bool enabled) +{ + Error *local_err = NULL; + + qio_channel_file_set_direct_io(f->ioc, enabled, &local_err); + if (local_err) { + qemu_file_set_error_obj(f, -EINVAL, local_err); + return false; + } + + f->dio = enabled; + + return true; +} diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 11c2120edd..118853b21c 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -79,5 +79,5 @@ size_t qemu_get_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, off_t pos); QIOChannel *qemu_file_get_ioc(QEMUFile *file); - +bool qemu_file_set_direct_io(QEMUFile *f, bool enabled); #endif From patchwork Thu May 23 19:05:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13672209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 13D83C25B75 for ; Thu, 23 May 2024 19:08:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sADmO-00035o-7E; Thu, 23 May 2024 15:07:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sADlz-0002IT-NU for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:45 -0400 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sADlw-0004Lj-Hq for qemu-devel@nongnu.org; Thu, 23 May 2024 15:06:42 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DC0782040D; Thu, 23 May 2024 19:06:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491198; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qcAnGN7Dn8Hx+2ymPsQ94bgP0wRUUi69NxEcC3s4Wwk=; b=ToCIBhb/Cm11PcvUFoUcWxAJgSCWw2/BiuDynvZK4sXWXXR/ccQzfkNakbKDS4SLHjm5Pz p+3cLr4rstwy3QNYXiTqDildILB1yN46QalpygzP6lzWF8b//aCYZobSTpPc5ZJdGLe9Ht il0+6UjCkgW+XHJYwxk6k5p98P9fRr8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491198; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qcAnGN7Dn8Hx+2ymPsQ94bgP0wRUUi69NxEcC3s4Wwk=; b=TobcceVeDKKeSygCTyPqV0TCtDkVvcQblVLzuf5tNY95HUpkpfUlHPi9Gpjg1+6g93Tmcd 6kmEz0YxhAs0QsDA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1716491198; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qcAnGN7Dn8Hx+2ymPsQ94bgP0wRUUi69NxEcC3s4Wwk=; b=ToCIBhb/Cm11PcvUFoUcWxAJgSCWw2/BiuDynvZK4sXWXXR/ccQzfkNakbKDS4SLHjm5Pz p+3cLr4rstwy3QNYXiTqDildILB1yN46QalpygzP6lzWF8b//aCYZobSTpPc5ZJdGLe9Ht il0+6UjCkgW+XHJYwxk6k5p98P9fRr8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1716491198; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qcAnGN7Dn8Hx+2ymPsQ94bgP0wRUUi69NxEcC3s4Wwk=; b=TobcceVeDKKeSygCTyPqV0TCtDkVvcQblVLzuf5tNY95HUpkpfUlHPi9Gpjg1+6g93Tmcd 6kmEz0YxhAs0QsDA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5214C13A6B; Thu, 23 May 2024 19:06:36 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id uHObBryTT2bUEQAAD6G6ig (envelope-from ); Thu, 23 May 2024 19:06:36 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: berrange@redhat.com, armbru@redhat.com, Peter Xu , Claudio Fontana , Jim Fehlig , Thomas Huth , Laurent Vivier , Paolo Bonzini Subject: [PATCH v2 18/18] migration/ram: Add direct-io support to precopy file migration Date: Thu, 23 May 2024 16:05:48 -0300 Message-Id: <20240523190548.23977-19-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240523190548.23977-1-farosas@suse.de> References: <20240523190548.23977-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_SEVEN(0.00)[9]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,imap1.dmz-prg2.suse.org:helo]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We've recently added support for direct-io with multifd, which brings performance benefits, but creates a non-uniform user interface by coupling direct-io with the multifd capability. This means that users cannot keep the direct-io flag enabled while disabling multifd. Libvirt in particular already has support for direct-io and parallel migration separately from each other, so it would be a regression to now require both options together. It's relatively simple for QEMU to add support for direct-io migration without multifd, so let's do this in order to keep both options decoupled. We cannot simply enable the O_DIRECT flag, however, because not all IO performed by the migration thread satisfies the alignment requirements of O_DIRECT. There are many small read & writes that add headers and synchronization flags to the stream, which at the moment are required to always be present. Fortunately, due to fixed-ram migration there is a discernible moment where only RAM pages are written to the migration file. Enable direct-io during that moment. Signed-off-by: Fabiano Rosas --- migration/ram.c | 40 ++++++++++++++++++++++++++++-------- tests/qtest/migration-test.c | 30 +++++++++++++++++++++++++++ 2 files changed, 61 insertions(+), 9 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index ceea586b06..5183d1f97c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3131,6 +3131,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) int i; int64_t t0; int done = 0; + Error **errp = NULL; /* * We'll take this lock a little bit long, but it's okay for two reasons. @@ -3154,6 +3155,10 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) goto out; } + if (!migration_direct_io_start(f, errp)) { + return -errno; + } + t0 = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); i = 0; while ((ret = migration_rate_exceeded(f)) == 0 || @@ -3194,6 +3199,9 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) } i++; } + if (!migration_direct_io_finish(f, errp)) { + return -errno; + } } } @@ -3242,7 +3250,8 @@ static int ram_save_complete(QEMUFile *f, void *opaque) { RAMState **temp = opaque; RAMState *rs = *temp; - int ret = 0; + int ret = 0, pages; + Error **errp = NULL; rs->last_stage = !migration_in_colo_state(); @@ -3257,25 +3266,30 @@ static int ram_save_complete(QEMUFile *f, void *opaque) return ret; } + if (!migration_direct_io_start(f, errp)) { + return -errno; + } + /* try transferring iterative blocks of memory */ /* flush all remaining blocks regardless of rate limiting */ qemu_mutex_lock(&rs->bitmap_mutex); while (true) { - int pages; - pages = ram_find_and_save_block(rs); - /* no more blocks to sent */ - if (pages == 0) { + if (pages <= 0) { break; } - if (pages < 0) { - qemu_mutex_unlock(&rs->bitmap_mutex); - return pages; - } } qemu_mutex_unlock(&rs->bitmap_mutex); + if (!migration_direct_io_finish(f, errp)) { + return -errno; + } + + if (pages < 0) { + return pages; + } + ret = rdma_registration_stop(f, RAM_CONTROL_FINISH); if (ret < 0) { qemu_file_set_error(f, ret); @@ -3920,6 +3934,10 @@ static bool read_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block, void *host; size_t read, unread, size; + if (!migration_direct_io_start(f, errp)) { + return false; + } + for (set_bit_idx = find_first_bit(bitmap, num_pages); set_bit_idx < num_pages; set_bit_idx = find_next_bit(bitmap, num_pages, clear_bit_idx + 1)) { @@ -3955,6 +3973,10 @@ static bool read_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block, } } + if (!migration_direct_io_finish(f, errp)) { + return false; + } + return true; err: diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 5ced3b90c9..8c6a122c20 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2245,6 +2245,34 @@ static void test_multifd_file_mapped_ram_dio(void) test_file_common(&args, true); } +static void *mapped_ram_dio_start(QTestState *from, QTestState *to) +{ + migrate_mapped_ram_start(from, to); + + migrate_set_parameter_bool(from, "direct-io", true); + migrate_set_parameter_bool(to, "direct-io", true); + + return NULL; +} + +static void test_precopy_file_mapped_ram_dio(void) +{ + g_autofree char *uri = g_strdup_printf("file:%s/%s", tmpfs, + FILE_TEST_FILENAME); + MigrateCommon args = { + .connect_uri = uri, + .listen_uri = "defer", + .start_hook = mapped_ram_dio_start, + }; + + if (!probe_o_direct_support(tmpfs)) { + g_test_skip("Filesystem does not support O_DIRECT"); + return; + } + + test_file_common(&args, true); +} + #ifndef _WIN32 static void multifd_mapped_ram_fdset_end(QTestState *from, QTestState *to, void *opaque) @@ -3735,6 +3763,8 @@ int main(int argc, char **argv) migration_test_add("/migration/multifd/file/mapped-ram/dio", test_multifd_file_mapped_ram_dio); + migration_test_add("/migration/precopy/file/mapped-ram/dio", + test_precopy_file_mapped_ram_dio); #ifndef _WIN32 qtest_add_func("/migration/multifd/file/mapped-ram/fdset",