From patchwork Mon Jul 22 17:59:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13738990 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C5179C3DA5D for ; Mon, 22 Jul 2024 18:00:15 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sVxJr-0002se-Ck; Mon, 22 Jul 2024 13:59:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sVxJl-0002oE-Tn for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:25 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sVxJh-000359-Tl for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:25 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 47DBA21B11; Mon, 22 Jul 2024 17:59:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671160; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xtr1ul5ovyW20vpLM4Cci52AvWBkmI6LBuM6z6QgBA4=; b=TrXYKjBrgLDsOzKkvnlP7maEVZf6gDoURxPXnrRKFJTirbmJg9cSzF+bAVbXngXb2lHpWc spWmZ+A3LEHbNtVwHPbH83oSd8mOiZVa5WPIM/WOE5GFsCdfxME1Ie39cHHWej/CjWSQZ/ o7NgbPX4hNv/Iq3qWAdrb+Cgwxsxggw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671160; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xtr1ul5ovyW20vpLM4Cci52AvWBkmI6LBuM6z6QgBA4=; b=5O9W9asAcscvQ/CIq3ZZs6z8ncCI2ULIMJBC4LfjAbvP2jB1gZi3ZPir2iNUmfMb/clfNV GtSCZtmdHL1IerDg== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=TrXYKjBr; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=5O9W9asA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671160; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xtr1ul5ovyW20vpLM4Cci52AvWBkmI6LBuM6z6QgBA4=; b=TrXYKjBrgLDsOzKkvnlP7maEVZf6gDoURxPXnrRKFJTirbmJg9cSzF+bAVbXngXb2lHpWc spWmZ+A3LEHbNtVwHPbH83oSd8mOiZVa5WPIM/WOE5GFsCdfxME1Ie39cHHWej/CjWSQZ/ o7NgbPX4hNv/Iq3qWAdrb+Cgwxsxggw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671160; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xtr1ul5ovyW20vpLM4Cci52AvWBkmI6LBuM6z6QgBA4=; b=5O9W9asAcscvQ/CIq3ZZs6z8ncCI2ULIMJBC4LfjAbvP2jB1gZi3ZPir2iNUmfMb/clfNV GtSCZtmdHL1IerDg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1D8D513996; Mon, 22 Jul 2024 17:59:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QMVHNfadnmYnCQAAD6G6ig (envelope-from ); Mon, 22 Jul 2024 17:59:18 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , "Maciej S . Szmigiero" Subject: [RFC PATCH v2 1/9] migration/multifd: Reduce access to p->pages Date: Mon, 22 Jul 2024 14:59:06 -0300 Message-Id: <20240722175914.24022-2-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240722175914.24022-1-farosas@suse.de> References: <20240722175914.24022-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Rspamd-Queue-Id: 47DBA21B11 X-Spamd-Result: default: False [-2.81 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; MIME_TRACE(0.00)[0:+]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_TLS_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:dkim,suse.de:email]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org I'm about to replace the p->pages pointer with an opaque pointer, so do a cleanup now to reduce direct accesses to p->page, which makes the next diffs cleaner. Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- migration/multifd-qpl.c | 8 +++++--- migration/multifd-uadk.c | 9 +++++---- migration/multifd-zlib.c | 2 +- migration/multifd-zstd.c | 2 +- migration/multifd.c | 13 +++++++------ 5 files changed, 19 insertions(+), 15 deletions(-) diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c index 9265098ee7..f8c84c52cf 100644 --- a/migration/multifd-qpl.c +++ b/migration/multifd-qpl.c @@ -404,13 +404,14 @@ retry: static void multifd_qpl_compress_pages_slow_path(MultiFDSendParams *p) { QplData *qpl = p->compress_data; + MultiFDPages_t *pages = p->pages; uint32_t size = p->page_size; qpl_job *job = qpl->sw_job; uint8_t *zbuf = qpl->zbuf; uint8_t *buf; - for (int i = 0; i < p->pages->normal_num; i++) { - buf = p->pages->block->host + p->pages->offset[i]; + for (int i = 0; i < pages->normal_num; i++) { + buf = pages->block->host + pages->offset[i]; multifd_qpl_prepare_comp_job(job, buf, zbuf, size); if (qpl_execute_job(job) == QPL_STS_OK) { multifd_qpl_fill_packet(i, p, zbuf, job->total_out); @@ -498,6 +499,7 @@ static void multifd_qpl_compress_pages(MultiFDSendParams *p) static int multifd_qpl_send_prepare(MultiFDSendParams *p, Error **errp) { QplData *qpl = p->compress_data; + MultiFDPages_t *pages = p->pages; uint32_t len = 0; if (!multifd_send_prepare_common(p)) { @@ -505,7 +507,7 @@ static int multifd_qpl_send_prepare(MultiFDSendParams *p, Error **errp) } /* The first IOV is used to store the compressed page lengths */ - len = p->pages->normal_num * sizeof(uint32_t); + len = pages->normal_num * sizeof(uint32_t); multifd_qpl_fill_iov(p, (uint8_t *) qpl->zlen, len); if (qpl->hw_avail) { multifd_qpl_compress_pages(p); diff --git a/migration/multifd-uadk.c b/migration/multifd-uadk.c index d12353fb21..b8ba3cd9c1 100644 --- a/migration/multifd-uadk.c +++ b/migration/multifd-uadk.c @@ -174,19 +174,20 @@ static int multifd_uadk_send_prepare(MultiFDSendParams *p, Error **errp) uint32_t hdr_size; uint8_t *buf = uadk_data->buf; int ret = 0; + MultiFDPages_t *pages = p->pages; if (!multifd_send_prepare_common(p)) { goto out; } - hdr_size = p->pages->normal_num * sizeof(uint32_t); + hdr_size = pages->normal_num * sizeof(uint32_t); /* prepare the header that stores the lengths of all compressed data */ prepare_next_iov(p, uadk_data->buf_hdr, hdr_size); - for (int i = 0; i < p->pages->normal_num; i++) { + for (int i = 0; i < pages->normal_num; i++) { struct wd_comp_req creq = { .op_type = WD_DIR_COMPRESS, - .src = p->pages->block->host + p->pages->offset[i], + .src = pages->block->host + pages->offset[i], .src_len = p->page_size, .dst = buf, /* Set dst_len to double the src in case compressed out >= page_size */ @@ -214,7 +215,7 @@ static int multifd_uadk_send_prepare(MultiFDSendParams *p, Error **errp) */ if (!uadk_data->handle || creq.dst_len >= p->page_size) { uadk_data->buf_hdr[i] = cpu_to_be32(p->page_size); - prepare_next_iov(p, p->pages->block->host + p->pages->offset[i], + prepare_next_iov(p, pages->block->host + pages->offset[i], p->page_size); buf += p->page_size; } diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 2ced69487e..65f8aba5c8 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -147,7 +147,7 @@ static int zlib_send_prepare(MultiFDSendParams *p, Error **errp) * with compression. zlib does not guarantee that this is safe, * therefore copy the page before calling deflate(). */ - memcpy(z->buf, p->pages->block->host + pages->offset[i], p->page_size); + memcpy(z->buf, pages->block->host + pages->offset[i], p->page_size); zs->avail_in = p->page_size; zs->next_in = z->buf; diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index ca17b7e310..cb6075a9a5 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -138,7 +138,7 @@ static int zstd_send_prepare(MultiFDSendParams *p, Error **errp) if (i == pages->normal_num - 1) { flush = ZSTD_e_flush; } - z->in.src = p->pages->block->host + pages->offset[i]; + z->in.src = pages->block->host + pages->offset[i]; z->in.size = p->page_size; z->in.pos = 0; diff --git a/migration/multifd.c b/migration/multifd.c index 0b4cbaddfe..c95fce2222 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -114,11 +114,11 @@ static void multifd_set_file_bitmap(MultiFDSendParams *p) assert(pages->block); - for (int i = 0; i < p->pages->normal_num; i++) { + for (int i = 0; i < pages->normal_num; i++) { ramblock_set_file_bmap_atomic(pages->block, pages->offset[i], true); } - for (int i = p->pages->normal_num; i < p->pages->num; i++) { + for (int i = pages->normal_num; i < pages->num; i++) { ramblock_set_file_bmap_atomic(pages->block, pages->offset[i], false); } } @@ -417,7 +417,7 @@ void multifd_send_fill_packet(MultiFDSendParams *p) int i; packet->flags = cpu_to_be32(p->flags); - packet->pages_alloc = cpu_to_be32(p->pages->allocated); + packet->pages_alloc = cpu_to_be32(pages->allocated); packet->normal_pages = cpu_to_be32(pages->normal_num); packet->zero_pages = cpu_to_be32(zero_num); packet->next_packet_size = cpu_to_be32(p->next_packet_size); @@ -953,7 +953,7 @@ static void *multifd_send_thread(void *opaque) if (migrate_mapped_ram()) { ret = file_write_ramblock_iov(p->c, p->iov, p->iovs_num, - p->pages->block, &local_err); + pages->block, &local_err); } else { ret = qio_channel_writev_full_all(p->c, p->iov, p->iovs_num, NULL, 0, p->write_flags, @@ -969,7 +969,7 @@ static void *multifd_send_thread(void *opaque) stat64_add(&mig_stats.normal_pages, pages->normal_num); stat64_add(&mig_stats.zero_pages, pages->num - pages->normal_num); - multifd_pages_reset(p->pages); + multifd_pages_reset(pages); p->next_packet_size = 0; /* @@ -1684,9 +1684,10 @@ void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) bool multifd_send_prepare_common(MultiFDSendParams *p) { + MultiFDPages_t *pages = p->pages; multifd_send_zero_page_detect(p); - if (!p->pages->normal_num) { + if (!pages->normal_num) { p->next_packet_size = 0; return false; } From patchwork Mon Jul 22 17:59:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13738996 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8A2EFC3DA5D for ; Mon, 22 Jul 2024 18:01:31 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sVxJu-000352-0v; Mon, 22 Jul 2024 13:59:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sVxJn-0002pW-PV for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:29 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sVxJj-00035N-O3 for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:26 -0400 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id E2E1821B6A; Mon, 22 Jul 2024 17:59:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671161; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+0k8HamURW5P5sG+J1YUFAc7dteadeAjS9LJG4WDNI4=; b=MCzUW6b3jzKcCokc+7cnZWSAt1bmsTgYLnB2VCZ4f2n5q2L91P222GzKGSnrjVcMr3WZnZ +vqff45WNVnRMvnc+I4XG6yq58HON5RhlaFDFb/kETol2KpcWG8hmMEISYkjt9ISIYODpL iijV2LpDmCAMGXTFhuzFocRERevOtDM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671161; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+0k8HamURW5P5sG+J1YUFAc7dteadeAjS9LJG4WDNI4=; b=EQTZbJDzDycwLvFhg+h9FkswfRQ+fZgZ0X0omTQKO0tcOWgRnJ8pAh4JVddVfY0p6Fp/1Z PZjf4j5rt+qqQrAA== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671161; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+0k8HamURW5P5sG+J1YUFAc7dteadeAjS9LJG4WDNI4=; b=MCzUW6b3jzKcCokc+7cnZWSAt1bmsTgYLnB2VCZ4f2n5q2L91P222GzKGSnrjVcMr3WZnZ +vqff45WNVnRMvnc+I4XG6yq58HON5RhlaFDFb/kETol2KpcWG8hmMEISYkjt9ISIYODpL iijV2LpDmCAMGXTFhuzFocRERevOtDM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671161; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+0k8HamURW5P5sG+J1YUFAc7dteadeAjS9LJG4WDNI4=; b=EQTZbJDzDycwLvFhg+h9FkswfRQ+fZgZ0X0omTQKO0tcOWgRnJ8pAh4JVddVfY0p6Fp/1Z PZjf4j5rt+qqQrAA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id BC4A113996; Mon, 22 Jul 2024 17:59:20 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cCB3IPidnmYnCQAAD6G6ig (envelope-from ); Mon, 22 Jul 2024 17:59:20 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , "Maciej S . Szmigiero" Subject: [RFC PATCH v2 2/9] migration/multifd: Pass in MultiFDPages_t to file_write_ramblock_iov Date: Mon, 22 Jul 2024 14:59:07 -0300 Message-Id: <20240722175914.24022-3-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240722175914.24022-1-farosas@suse.de> References: <20240722175914.24022-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [0.40 / 50.00]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FROM_EQ_ENVFROM(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,imap1.dmz-prg2.suse.org:helo]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We want to stop dereferencing 'pages' so it can be replaced by an opaque pointer in the next patches. Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- migration/file.c | 3 ++- migration/file.h | 2 +- migration/multifd.c | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/migration/file.c b/migration/file.c index db870f2cf0..7a8c9e3957 100644 --- a/migration/file.c +++ b/migration/file.c @@ -198,12 +198,13 @@ void file_start_incoming_migration(FileMigrationArgs *file_args, Error **errp) } int file_write_ramblock_iov(QIOChannel *ioc, const struct iovec *iov, - int niov, RAMBlock *block, Error **errp) + int niov, MultiFDPages_t *pages, Error **errp) { ssize_t ret = 0; int i, slice_idx, slice_num; uintptr_t base, next, offset; size_t len; + RAMBlock *block = pages->block; slice_idx = 0; slice_num = 1; diff --git a/migration/file.h b/migration/file.h index 9f71e87f74..1a1115f7f1 100644 --- a/migration/file.h +++ b/migration/file.h @@ -21,6 +21,6 @@ int file_parse_offset(char *filespec, uint64_t *offsetp, Error **errp); void file_cleanup_outgoing_migration(void); bool file_send_channel_create(gpointer opaque, Error **errp); int file_write_ramblock_iov(QIOChannel *ioc, const struct iovec *iov, - int niov, RAMBlock *block, Error **errp); + int niov, MultiFDPages_t *pages, Error **errp); int multifd_file_recv_data(MultiFDRecvParams *p, Error **errp); #endif diff --git a/migration/multifd.c b/migration/multifd.c index c95fce2222..20a767157e 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -953,7 +953,7 @@ static void *multifd_send_thread(void *opaque) if (migrate_mapped_ram()) { ret = file_write_ramblock_iov(p->c, p->iov, p->iovs_num, - pages->block, &local_err); + pages, &local_err); } else { ret = qio_channel_writev_full_all(p->c, p->iov, p->iovs_num, NULL, 0, p->write_flags, From patchwork Mon Jul 22 17:59:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13738989 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 10F80C3DA7E for ; Mon, 22 Jul 2024 18:00:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sVxJs-0002yr-Ba; Mon, 22 Jul 2024 13:59:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sVxJn-0002pV-PS for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:28 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sVxJl-000374-FR for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:27 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9695121B8C; Mon, 22 Jul 2024 17:59:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671163; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iG9KIiyOBzYdvie4pn+w1wJTeVwhMXBO9AnRUlQiMu4=; b=k7dla34je0L3e8Cz9F/i2e8/HMuddlnm8s+0NNDvd/6oweUq1BVdVjnSmJGCd8Ck3xpto3 AxeFk+Zqww9Lj+842Dnrcak6Z0vi/sNILGrgyL3Jwh5fkxZmRR2NrBF59SNST9YdKm7vAQ BjYh0IpeVXSKi+aDaRcJKnicuTxsTVk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671163; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iG9KIiyOBzYdvie4pn+w1wJTeVwhMXBO9AnRUlQiMu4=; b=+zGijYWSQ8t6nSJI3FHLI2CZr6Eo36UG3dQNMQhIvOKbv4/QZi7Sr47dKWgf9IlfVdWj6H jKlchDiNvYk1mRAw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=k7dla34j; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=+zGijYWS DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671163; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iG9KIiyOBzYdvie4pn+w1wJTeVwhMXBO9AnRUlQiMu4=; b=k7dla34je0L3e8Cz9F/i2e8/HMuddlnm8s+0NNDvd/6oweUq1BVdVjnSmJGCd8Ck3xpto3 AxeFk+Zqww9Lj+842Dnrcak6Z0vi/sNILGrgyL3Jwh5fkxZmRR2NrBF59SNST9YdKm7vAQ BjYh0IpeVXSKi+aDaRcJKnicuTxsTVk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671163; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iG9KIiyOBzYdvie4pn+w1wJTeVwhMXBO9AnRUlQiMu4=; b=+zGijYWSQ8t6nSJI3FHLI2CZr6Eo36UG3dQNMQhIvOKbv4/QZi7Sr47dKWgf9IlfVdWj6H jKlchDiNvYk1mRAw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 606CB13996; Mon, 22 Jul 2024 17:59:22 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id kLsJCvqdnmYnCQAAD6G6ig (envelope-from ); Mon, 22 Jul 2024 17:59:22 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , "Maciej S . Szmigiero" Subject: [RFC PATCH v2 3/9] migration/multifd: Introduce MultiFDSendData Date: Mon, 22 Jul 2024 14:59:08 -0300 Message-Id: <20240722175914.24022-4-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240722175914.24022-1-farosas@suse.de> References: <20240722175914.24022-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [0.19 / 50.00]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FUZZY_BLOCKED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCPT_COUNT_THREE(0.00)[3]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: 9695121B8C X-Rspamd-Action: no action X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Bar: / Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add a new data structure to replace p->pages in the multifd channel. This new structure will hide the multifd payload type behind an union, so we don't need to add a new field to the channel each time we want to handle a different data type. This also allow us to keep multifd_send_pages() as is, without needing to complicate the pointer switching. Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- migration/multifd.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/migration/multifd.h b/migration/multifd.h index 0ecd6f47d7..c7b1ebe099 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -16,6 +16,7 @@ #include "ram.h" typedef struct MultiFDRecvData MultiFDRecvData; +typedef struct MultiFDSendData MultiFDSendData; bool multifd_send_setup(void); void multifd_send_shutdown(void); @@ -89,6 +90,29 @@ struct MultiFDRecvData { off_t file_offset; }; +typedef enum { + MULTIFD_PAYLOAD_NONE, + MULTIFD_PAYLOAD_RAM, +} MultiFDPayloadType; + +struct MultiFDSendData { + MultiFDPayloadType type; + union { + MultiFDPages_t ram; + } u; +}; + +static inline bool multifd_payload_empty(MultiFDSendData *data) +{ + return data->type == MULTIFD_PAYLOAD_NONE; +} + +static inline void multifd_set_payload_type(MultiFDSendData *data, + MultiFDPayloadType type) +{ + data->type = type; +} + typedef struct { /* Fields are only written at creating/deletion time */ /* No lock required for them, they are read only */ From patchwork Mon Jul 22 17:59:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13738997 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C07C2C3DA63 for ; Mon, 22 Jul 2024 18:01:33 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sVxJt-00032x-Ck; Mon, 22 Jul 2024 13:59:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sVxJp-0002qW-Nm for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:29 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sVxJm-00037E-Rz for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:28 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 34AF021AE5; Mon, 22 Jul 2024 17:59:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671165; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oK5XkDoeInZOoB1Qlg91lWOBq6MpNV/Nl58Zm0hrvNo=; b=gFI4YJy22zfJ9JfTnICLDUm3H6AjqGYDTD9lviH1JU9BV7/vLuL4QJaLDhXrb/j744UerT FnGoy0mq0H+V9cQHvpYxLkuFG7DtLN082ydqplEWZtX/mK63/Ff6WJPwfB5EwM0cWzMLnE kR4OAY5BurRvP8uWZIBfgv6JC7+WQuM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671165; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oK5XkDoeInZOoB1Qlg91lWOBq6MpNV/Nl58Zm0hrvNo=; b=2+7yX51mqTGoJ6Tv43ZUB8ZM6Y8o3ZRCI2sx0WOFUtncB2dht1tQLJo0KDsXDGDkIkKVsk mjYgOS1pyYc2+wCw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=gFI4YJy2; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=2+7yX51m DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671165; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oK5XkDoeInZOoB1Qlg91lWOBq6MpNV/Nl58Zm0hrvNo=; b=gFI4YJy22zfJ9JfTnICLDUm3H6AjqGYDTD9lviH1JU9BV7/vLuL4QJaLDhXrb/j744UerT FnGoy0mq0H+V9cQHvpYxLkuFG7DtLN082ydqplEWZtX/mK63/Ff6WJPwfB5EwM0cWzMLnE kR4OAY5BurRvP8uWZIBfgv6JC7+WQuM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671165; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oK5XkDoeInZOoB1Qlg91lWOBq6MpNV/Nl58Zm0hrvNo=; b=2+7yX51mqTGoJ6Tv43ZUB8ZM6Y8o3ZRCI2sx0WOFUtncB2dht1tQLJo0KDsXDGDkIkKVsk mjYgOS1pyYc2+wCw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0C02313996; Mon, 22 Jul 2024 17:59:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id kAkNMfudnmYnCQAAD6G6ig (envelope-from ); Mon, 22 Jul 2024 17:59:23 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , "Maciej S . Szmigiero" Subject: [RFC PATCH v2 4/9] migration/multifd: Make MultiFDPages_t:offset a flexible array member Date: Mon, 22 Jul 2024 14:59:09 -0300 Message-Id: <20240722175914.24022-5-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240722175914.24022-1-farosas@suse.de> References: <20240722175914.24022-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [0.19 / 50.00]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:dkim,suse.de:email]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FUZZY_BLOCKED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCPT_COUNT_THREE(0.00)[3]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: 34AF021AE5 X-Rspamd-Action: no action X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Bar: / Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We're about to use MultiFDPages_t from inside the MultiFDSendData payload union, which means we cannot have pointers to allocated data inside the pages structure, otherwise we'd lose the reference to that memory once another payload type touches the union. Move the offset array into the end of the structure and turn it into a flexible array member, so it is allocated along with the rest of MultiFDSendData in the next patches. Note that the ramblock pointer is still fine because the storage for it is not owned by the migration code. Signed-off-by: Fabiano Rosas --- migration/multifd.c | 21 ++++++--------------- migration/multifd.h | 4 ++-- 2 files changed, 8 insertions(+), 17 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 20a767157e..440319b361 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -389,22 +389,10 @@ static int multifd_recv_initial_packet(QIOChannel *c, Error **errp) return msg.id; } -static MultiFDPages_t *multifd_pages_init(uint32_t n) -{ - MultiFDPages_t *pages = g_new0(MultiFDPages_t, 1); - - pages->allocated = n; - pages->offset = g_new0(ram_addr_t, n); - - return pages; -} - static void multifd_pages_clear(MultiFDPages_t *pages) { multifd_pages_reset(pages); pages->allocated = 0; - g_free(pages->offset); - pages->offset = NULL; g_free(pages); } @@ -1169,7 +1157,9 @@ bool multifd_send_setup(void) thread_count = migrate_multifd_channels(); multifd_send_state = g_malloc0(sizeof(*multifd_send_state)); multifd_send_state->params = g_new0(MultiFDSendParams, thread_count); - multifd_send_state->pages = multifd_pages_init(page_count); + multifd_send_state->pages = g_malloc0(sizeof(MultiFDPages_t) + + page_count * sizeof(ram_addr_t)); + multifd_send_state->pages->allocated = page_count; qemu_sem_init(&multifd_send_state->channels_created, 0); qemu_sem_init(&multifd_send_state->channels_ready, 0); qatomic_set(&multifd_send_state->exiting, 0); @@ -1181,8 +1171,9 @@ bool multifd_send_setup(void) qemu_sem_init(&p->sem, 0); qemu_sem_init(&p->sem_sync, 0); p->id = i; - p->pages = multifd_pages_init(page_count); - + p->pages = g_malloc0(sizeof(MultiFDPages_t) + + page_count * sizeof(ram_addr_t)); + p->pages->allocated = page_count; if (use_packets) { p->packet_len = sizeof(MultiFDPacket_t) + sizeof(uint64_t) * page_count; diff --git a/migration/multifd.h b/migration/multifd.h index c7b1ebe099..12d4247e23 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -78,9 +78,9 @@ typedef struct { uint32_t normal_num; /* number of allocated pages */ uint32_t allocated; + RAMBlock *block; /* offset of each page */ - ram_addr_t *offset; - RAMBlock *block; + ram_addr_t offset[]; } MultiFDPages_t; struct MultiFDRecvData { From patchwork Mon Jul 22 17:59:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13738994 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5E344C3DA59 for ; Mon, 22 Jul 2024 18:01:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sVxJv-0003D6-Sb; Mon, 22 Jul 2024 13:59:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sVxJu-00037I-DK for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:34 -0400 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sVxJo-00037P-GW for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:33 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D839E1FB42; Mon, 22 Jul 2024 17:59:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671166; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HN9/IQFX1axqLwfA8q+DCIUChYwXgxrYqTMU0Ie3HHM=; b=DLR5AXwJUnMUpGcHBEE9AHSnX7cUFyr0hH6rDO50nM9cjChCI3SGdXpSuji9YDpJS5/3VD 9ZuoQIDm9YYUOaRFvfxXKyWqZgFTZdWMxc5j4qodOE+vxh7XW1N0cZ7SEtmfu1b6Dmg4xJ gL8s7c0tiv7wTIWhVw5MJo6rrvgmwh4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671166; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HN9/IQFX1axqLwfA8q+DCIUChYwXgxrYqTMU0Ie3HHM=; b=N+7XNe+GMGQnlf4q5c+KVK4vqEzUfCwEiJgkceMoUSJpOMSfYEoAci4LWtMjJwQgMqkCSn ujcGz30I1V7ni0Dg== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=DLR5AXwJ; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=N+7XNe+G DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671166; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HN9/IQFX1axqLwfA8q+DCIUChYwXgxrYqTMU0Ie3HHM=; b=DLR5AXwJUnMUpGcHBEE9AHSnX7cUFyr0hH6rDO50nM9cjChCI3SGdXpSuji9YDpJS5/3VD 9ZuoQIDm9YYUOaRFvfxXKyWqZgFTZdWMxc5j4qodOE+vxh7XW1N0cZ7SEtmfu1b6Dmg4xJ gL8s7c0tiv7wTIWhVw5MJo6rrvgmwh4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671166; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HN9/IQFX1axqLwfA8q+DCIUChYwXgxrYqTMU0Ie3HHM=; b=N+7XNe+GMGQnlf4q5c+KVK4vqEzUfCwEiJgkceMoUSJpOMSfYEoAci4LWtMjJwQgMqkCSn ujcGz30I1V7ni0Dg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B0E1213996; Mon, 22 Jul 2024 17:59:25 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id IH7BHf2dnmYnCQAAD6G6ig (envelope-from ); Mon, 22 Jul 2024 17:59:25 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , "Maciej S . Szmigiero" Subject: [RFC PATCH v2 5/9] migration/multifd: Replace p->pages with an union pointer Date: Mon, 22 Jul 2024 14:59:10 -0300 Message-Id: <20240722175914.24022-6-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240722175914.24022-1-farosas@suse.de> References: <20240722175914.24022-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [0.19 / 50.00]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:dkim,suse.de:email]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FUZZY_BLOCKED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCPT_COUNT_THREE(0.00)[3]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: D839E1FB42 X-Rspamd-Action: no action X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Bar: / Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We want multifd to be able to handle more types of data than just ram pages. To start decoupling multifd from pages, replace p->pages (MultiFDPages_t) with the new type MultiFDSendData that hides the client payload inside an union. The general idea here is to isolate functions that *need* to handle MultiFDPages_t and move them in the future to multifd-ram.c, while multifd.c will stay with only the core functions that handle MultiFDSendData/MultiFDRecvData. Signed-off-by: Fabiano Rosas --- migration/multifd-qpl.c | 6 +-- migration/multifd-uadk.c | 2 +- migration/multifd-zero-page.c | 2 +- migration/multifd-zlib.c | 2 +- migration/multifd-zstd.c | 2 +- migration/multifd.c | 84 +++++++++++++++++++++-------------- migration/multifd.h | 7 +-- 7 files changed, 58 insertions(+), 47 deletions(-) diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c index f8c84c52cf..17ca16824d 100644 --- a/migration/multifd-qpl.c +++ b/migration/multifd-qpl.c @@ -404,7 +404,7 @@ retry: static void multifd_qpl_compress_pages_slow_path(MultiFDSendParams *p) { QplData *qpl = p->compress_data; - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; uint32_t size = p->page_size; qpl_job *job = qpl->sw_job; uint8_t *zbuf = qpl->zbuf; @@ -435,7 +435,7 @@ static void multifd_qpl_compress_pages_slow_path(MultiFDSendParams *p) static void multifd_qpl_compress_pages(MultiFDSendParams *p) { QplData *qpl = p->compress_data; - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; uint32_t size = p->page_size; QplHwJob *hw_job; uint8_t *buf; @@ -499,7 +499,7 @@ static void multifd_qpl_compress_pages(MultiFDSendParams *p) static int multifd_qpl_send_prepare(MultiFDSendParams *p, Error **errp) { QplData *qpl = p->compress_data; - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; uint32_t len = 0; if (!multifd_send_prepare_common(p)) { diff --git a/migration/multifd-uadk.c b/migration/multifd-uadk.c index b8ba3cd9c1..a561988d2f 100644 --- a/migration/multifd-uadk.c +++ b/migration/multifd-uadk.c @@ -174,7 +174,7 @@ static int multifd_uadk_send_prepare(MultiFDSendParams *p, Error **errp) uint32_t hdr_size; uint8_t *buf = uadk_data->buf; int ret = 0; - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; if (!multifd_send_prepare_common(p)) { goto out; diff --git a/migration/multifd-zero-page.c b/migration/multifd-zero-page.c index e1b8370f88..efc0424f74 100644 --- a/migration/multifd-zero-page.c +++ b/migration/multifd-zero-page.c @@ -46,7 +46,7 @@ static void swap_page_offset(ram_addr_t *pages_offset, int a, int b) */ void multifd_send_zero_page_detect(MultiFDSendParams *p) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; RAMBlock *rb = pages->block; int i = 0; int j = pages->num - 1; diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 65f8aba5c8..cec552e9b4 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -123,7 +123,7 @@ static void zlib_send_cleanup(MultiFDSendParams *p, Error **errp) */ static int zlib_send_prepare(MultiFDSendParams *p, Error **errp) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; struct zlib_data *z = p->compress_data; z_stream *zs = &z->zs; uint32_t out_size = 0; diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index cb6075a9a5..ff72c6d90a 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -119,7 +119,7 @@ static void zstd_send_cleanup(MultiFDSendParams *p, Error **errp) */ static int zstd_send_prepare(MultiFDSendParams *p, Error **errp) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; struct zstd_data *z = p->compress_data; int ret; uint32_t i; diff --git a/migration/multifd.c b/migration/multifd.c index 440319b361..f64b053e44 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -49,8 +49,7 @@ typedef struct { struct { MultiFDSendParams *params; - /* array of pages to sent */ - MultiFDPages_t *pages; + MultiFDSendData *data; /* * Global number of generated multifd packets. * @@ -98,6 +97,27 @@ struct { MultiFDMethods *ops; } *multifd_recv_state; +static size_t multifd_ram_payload_size(void) +{ + uint32_t n = MULTIFD_PACKET_SIZE / qemu_target_page_size(); + + return sizeof(MultiFDPages_t) + n * sizeof(ram_addr_t); +} + +static MultiFDSendData *multifd_send_data_alloc(void) +{ + size_t max_payload_size; + + /* + * MultiFDPages_t carries extra data as a flexible array + * member. When adding new payload types, change this to use the + * larger between multifd_ram_payload_size() and the new type(s). + */ + max_payload_size = multifd_ram_payload_size(); + + return g_malloc0(sizeof(MultiFDPayloadType) + max_payload_size); +} + static bool multifd_use_packets(void) { return !migrate_mapped_ram(); @@ -110,7 +130,7 @@ void multifd_send_channel_created(void) static void multifd_set_file_bitmap(MultiFDSendParams *p) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; assert(pages->block); @@ -164,7 +184,7 @@ static void nocomp_send_cleanup(MultiFDSendParams *p, Error **errp) static void multifd_send_prepare_iovs(MultiFDSendParams *p) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; for (int i = 0; i < pages->normal_num; i++) { p->iov[p->iovs_num].iov_base = pages->block->host + pages->offset[i]; @@ -389,17 +409,10 @@ static int multifd_recv_initial_packet(QIOChannel *c, Error **errp) return msg.id; } -static void multifd_pages_clear(MultiFDPages_t *pages) -{ - multifd_pages_reset(pages); - pages->allocated = 0; - g_free(pages); -} - void multifd_send_fill_packet(MultiFDSendParams *p) { MultiFDPacket_t *packet = p->packet; - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; uint64_t packet_num; uint32_t zero_num = pages->num - pages->normal_num; int i; @@ -579,7 +592,7 @@ static bool multifd_send_pages(void) int i; static int next_channel; MultiFDSendParams *p = NULL; /* make happy gcc */ - MultiFDPages_t *pages = multifd_send_state->pages; + MultiFDSendData *tmp; if (multifd_send_should_exit()) { return false; @@ -614,11 +627,14 @@ static bool multifd_send_pages(void) * qatomic_store_release() in multifd_send_thread(). */ smp_mb_acquire(); - assert(!p->pages->num); - multifd_send_state->pages = p->pages; - p->pages = pages; + + assert(!p->data->u.ram.num); + + tmp = multifd_send_state->data; + multifd_send_state->data = p->data; + p->data = tmp; /* - * Making sure p->pages is setup before marking pending_job=true. Pairs + * Making sure p->data is setup before marking pending_job=true. Pairs * with the qatomic_load_acquire() in multifd_send_thread(). */ qatomic_store_release(&p->pending_job, true); @@ -648,7 +664,7 @@ bool multifd_queue_page(RAMBlock *block, ram_addr_t offset) MultiFDPages_t *pages; retry: - pages = multifd_send_state->pages; + pages = &multifd_send_state->data->u.ram; /* If the queue is empty, we can already enqueue now */ if (multifd_queue_empty(pages)) { @@ -778,8 +794,8 @@ static bool multifd_send_cleanup_channel(MultiFDSendParams *p, Error **errp) qemu_sem_destroy(&p->sem_sync); g_free(p->name); p->name = NULL; - multifd_pages_clear(p->pages); - p->pages = NULL; + g_free(p->data); + p->data = NULL; p->packet_len = 0; g_free(p->packet); p->packet = NULL; @@ -796,8 +812,8 @@ static void multifd_send_cleanup_state(void) qemu_sem_destroy(&multifd_send_state->channels_ready); g_free(multifd_send_state->params); multifd_send_state->params = NULL; - multifd_pages_clear(multifd_send_state->pages); - multifd_send_state->pages = NULL; + g_free(multifd_send_state->data); + multifd_send_state->data = NULL; g_free(multifd_send_state); multifd_send_state = NULL; } @@ -846,11 +862,13 @@ int multifd_send_sync_main(void) { int i; bool flush_zero_copy; + MultiFDPages_t *pages; if (!migrate_multifd()) { return 0; } - if (multifd_send_state->pages->num) { + pages = &multifd_send_state->data->u.ram; + if (pages->num) { if (!multifd_send_pages()) { error_report("%s: multifd_send_pages fail", __func__); return -1; @@ -925,11 +943,11 @@ static void *multifd_send_thread(void *opaque) } /* - * Read pending_job flag before p->pages. Pairs with the + * Read pending_job flag before p->data. Pairs with the * qatomic_store_release() in multifd_send_pages(). */ if (qatomic_load_acquire(&p->pending_job)) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; p->iovs_num = 0; assert(pages->num); @@ -941,7 +959,7 @@ static void *multifd_send_thread(void *opaque) if (migrate_mapped_ram()) { ret = file_write_ramblock_iov(p->c, p->iov, p->iovs_num, - pages, &local_err); + &p->data->u.ram, &local_err); } else { ret = qio_channel_writev_full_all(p->c, p->iov, p->iovs_num, NULL, 0, p->write_flags, @@ -961,7 +979,7 @@ static void *multifd_send_thread(void *opaque) p->next_packet_size = 0; /* - * Making sure p->pages is published before saying "we're + * Making sure p->data is published before saying "we're * free". Pairs with the smp_mb_acquire() in * multifd_send_pages(). */ @@ -1157,9 +1175,8 @@ bool multifd_send_setup(void) thread_count = migrate_multifd_channels(); multifd_send_state = g_malloc0(sizeof(*multifd_send_state)); multifd_send_state->params = g_new0(MultiFDSendParams, thread_count); - multifd_send_state->pages = g_malloc0(sizeof(MultiFDPages_t) + - page_count * sizeof(ram_addr_t)); - multifd_send_state->pages->allocated = page_count; + multifd_send_state->data = multifd_send_data_alloc(); + multifd_send_state->data->u.ram.allocated = page_count; qemu_sem_init(&multifd_send_state->channels_created, 0); qemu_sem_init(&multifd_send_state->channels_ready, 0); qatomic_set(&multifd_send_state->exiting, 0); @@ -1171,9 +1188,8 @@ bool multifd_send_setup(void) qemu_sem_init(&p->sem, 0); qemu_sem_init(&p->sem_sync, 0); p->id = i; - p->pages = g_malloc0(sizeof(MultiFDPages_t) + - page_count * sizeof(ram_addr_t)); - p->pages->allocated = page_count; + p->data = multifd_send_data_alloc(); + p->data->u.ram.allocated = page_count; if (use_packets) { p->packet_len = sizeof(MultiFDPacket_t) + sizeof(uint64_t) * page_count; @@ -1675,7 +1691,7 @@ void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) bool multifd_send_prepare_common(MultiFDSendParams *p) { - MultiFDPages_t *pages = p->pages; + MultiFDPages_t *pages = &p->data->u.ram; multifd_send_zero_page_detect(p); if (!pages->normal_num) { diff --git a/migration/multifd.h b/migration/multifd.h index 12d4247e23..c9c01579a0 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -155,12 +155,7 @@ typedef struct { */ bool pending_job; bool pending_sync; - /* array of pages to sent. - * The owner of 'pages' depends of 'pending_job' value: - * pending_job == 0 -> migration_thread can use it. - * pending_job != 0 -> multifd_channel can use it. - */ - MultiFDPages_t *pages; + MultiFDSendData *data; /* thread local variables. No locking required */ From patchwork Mon Jul 22 17:59:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13738998 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 528D6C3DA5D for ; Mon, 22 Jul 2024 18:01:47 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sVxJu-00038U-RD; Mon, 22 Jul 2024 13:59:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sVxJt-00034F-K6 for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:33 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sVxJp-00037c-Rw for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:33 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8080C21B11; Mon, 22 Jul 2024 17:59:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671168; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5F8uqsyx4QWoTlzopVcW8Z9o1ZrQmpekAfO9pBHU+Sw=; b=sEcYDgA0WzEnIyO7IlIog2vzJmT83jXNiz7JDKCELVNOzzwWk3yMxBLi+PqdfQqViFMzKK 3BGzt7JoibwBkhqecR3yi+yRGimVGSSJMJXXDaP3yjHmtxVZwpah+O8jv4oMnRwSe0Ejpq vOWEp+/0xobjwzlD1CiFqaNvvl2qmfk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671168; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5F8uqsyx4QWoTlzopVcW8Z9o1ZrQmpekAfO9pBHU+Sw=; b=Fbr7djNGbJ9P+RQYAlyX5mIAApT98tOJoXBgRcQJ8mOSbWj6pTMyQLxRzYi6rYToLKJIux DnFW7xO05C0VGzCQ== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=sEcYDgA0; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=Fbr7djNG DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671168; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5F8uqsyx4QWoTlzopVcW8Z9o1ZrQmpekAfO9pBHU+Sw=; b=sEcYDgA0WzEnIyO7IlIog2vzJmT83jXNiz7JDKCELVNOzzwWk3yMxBLi+PqdfQqViFMzKK 3BGzt7JoibwBkhqecR3yi+yRGimVGSSJMJXXDaP3yjHmtxVZwpah+O8jv4oMnRwSe0Ejpq vOWEp+/0xobjwzlD1CiFqaNvvl2qmfk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671168; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5F8uqsyx4QWoTlzopVcW8Z9o1ZrQmpekAfO9pBHU+Sw=; b=Fbr7djNGbJ9P+RQYAlyX5mIAApT98tOJoXBgRcQJ8mOSbWj6pTMyQLxRzYi6rYToLKJIux DnFW7xO05C0VGzCQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 58E4D13996; Mon, 22 Jul 2024 17:59:27 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id gEwhCP+dnmYnCQAAD6G6ig (envelope-from ); Mon, 22 Jul 2024 17:59:27 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , "Maciej S . Szmigiero" Subject: [RFC PATCH v2 6/9] migration/multifd: Move pages accounting into multifd_send_zero_page_detect() Date: Mon, 22 Jul 2024 14:59:11 -0300 Message-Id: <20240722175914.24022-7-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240722175914.24022-1-farosas@suse.de> References: <20240722175914.24022-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [0.19 / 50.00]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:dkim,suse.de:email]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FUZZY_BLOCKED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCPT_COUNT_THREE(0.00)[3]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: 8080C21B11 X-Rspamd-Action: no action X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Bar: / Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org All references to pages are being removed from the multifd worker threads in order to allow multifd to deal with different payload types. multifd_send_zero_page_detect() is called by all multifd migration paths that deal with pages and is the last spot where zero pages and normal page amounts are adjusted. Move the pages accounting into that function. Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu --- migration/multifd-zero-page.c | 7 ++++++- migration/multifd.c | 2 -- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/migration/multifd-zero-page.c b/migration/multifd-zero-page.c index efc0424f74..899e8864c6 100644 --- a/migration/multifd-zero-page.c +++ b/migration/multifd-zero-page.c @@ -14,6 +14,7 @@ #include "qemu/cutils.h" #include "exec/ramblock.h" #include "migration.h" +#include "migration-stats.h" #include "multifd.h" #include "options.h" #include "ram.h" @@ -53,7 +54,7 @@ void multifd_send_zero_page_detect(MultiFDSendParams *p) if (!multifd_zero_page_enabled()) { pages->normal_num = pages->num; - return; + goto out; } /* @@ -74,6 +75,10 @@ void multifd_send_zero_page_detect(MultiFDSendParams *p) } pages->normal_num = i; + +out: + stat64_add(&mig_stats.normal_pages, pages->normal_num); + stat64_add(&mig_stats.zero_pages, pages->num - pages->normal_num); } void multifd_recv_zero_page_process(MultiFDRecvParams *p) diff --git a/migration/multifd.c b/migration/multifd.c index f64b053e44..fcdb12e04f 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -972,8 +972,6 @@ static void *multifd_send_thread(void *opaque) stat64_add(&mig_stats.multifd_bytes, p->next_packet_size + p->packet_len); - stat64_add(&mig_stats.normal_pages, pages->normal_num); - stat64_add(&mig_stats.zero_pages, pages->num - pages->normal_num); multifd_pages_reset(pages); p->next_packet_size = 0; From patchwork Mon Jul 22 17:59:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13738995 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F1A38C3DA63 for ; Mon, 22 Jul 2024 18:01:18 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sVxJy-0003Lf-04; Mon, 22 Jul 2024 13:59:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sVxJw-0003FT-Ck for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:36 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sVxJr-00037n-Nr for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:36 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 27B1021B6A; Mon, 22 Jul 2024 17:59:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671170; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cs55Q4tDVXnDLGxgGhNE0DNXcMeFtVN9CDpXBLNZuJg=; b=ZuhrJdS/OHUyd22bNFB2xETP3GlmVZDFcClhtTLAdzcaDycEBCaMYehn3JheJdjO3oe21P bAxhiFVUhY4xxeJfQOAhU3mMQuQibtRrw8I/D/6opTKfEHUG72vrTPPzyXmlHtsJaJvYnI AH+6+Jk7VL1a93/jqAqFi8NPsyEW4Bk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671170; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cs55Q4tDVXnDLGxgGhNE0DNXcMeFtVN9CDpXBLNZuJg=; b=UpKyL6vveLTgAXPcTb7k44bCmX+qBKih+7GTi8hZwBUfAXF83c1gLLHpEmoRs6JWlupiGp ntm+0ty9TCHTvXCg== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="ZuhrJdS/"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=UpKyL6vv DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671170; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cs55Q4tDVXnDLGxgGhNE0DNXcMeFtVN9CDpXBLNZuJg=; b=ZuhrJdS/OHUyd22bNFB2xETP3GlmVZDFcClhtTLAdzcaDycEBCaMYehn3JheJdjO3oe21P bAxhiFVUhY4xxeJfQOAhU3mMQuQibtRrw8I/D/6opTKfEHUG72vrTPPzyXmlHtsJaJvYnI AH+6+Jk7VL1a93/jqAqFi8NPsyEW4Bk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671170; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cs55Q4tDVXnDLGxgGhNE0DNXcMeFtVN9CDpXBLNZuJg=; b=UpKyL6vveLTgAXPcTb7k44bCmX+qBKih+7GTi8hZwBUfAXF83c1gLLHpEmoRs6JWlupiGp ntm+0ty9TCHTvXCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id F2FA513996; Mon, 22 Jul 2024 17:59:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id eKzGLQCenmYnCQAAD6G6ig (envelope-from ); Mon, 22 Jul 2024 17:59:28 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , "Maciej S . Szmigiero" Subject: [RFC PATCH v2 7/9] migration/multifd: Isolate ram pages packet data Date: Mon, 22 Jul 2024 14:59:12 -0300 Message-Id: <20240722175914.24022-8-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240722175914.24022-1-farosas@suse.de> References: <20240722175914.24022-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Rspamd-Queue-Id: 27B1021B6A X-Spamd-Result: default: False [-2.81 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_TWO(0.00)[2]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DKIM_TRACE(0.00)[suse.de:+]; RCPT_COUNT_THREE(0.00)[3]; RCVD_TLS_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo, imap1.dmz-prg2.suse.org:rdns, suse.de:dkim, suse.de:email] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org While we cannot yet disentangle the multifd packet from page data, we can make the code a bit cleaner by setting the page-related fields in a separate function. Signed-off-by: Fabiano Rosas --- migration/multifd.c | 97 ++++++++++++++++++++++++++++----------------- 1 file changed, 61 insertions(+), 36 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index fcdb12e04f..d25b8658b2 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -409,65 +409,62 @@ static int multifd_recv_initial_packet(QIOChannel *c, Error **errp) return msg.id; } -void multifd_send_fill_packet(MultiFDSendParams *p) +static void multifd_ram_fill_packet(MultiFDSendParams *p) { MultiFDPacket_t *packet = p->packet; MultiFDPages_t *pages = &p->data->u.ram; - uint64_t packet_num; uint32_t zero_num = pages->num - pages->normal_num; - int i; - packet->flags = cpu_to_be32(p->flags); packet->pages_alloc = cpu_to_be32(pages->allocated); packet->normal_pages = cpu_to_be32(pages->normal_num); packet->zero_pages = cpu_to_be32(zero_num); - packet->next_packet_size = cpu_to_be32(p->next_packet_size); - - packet_num = qatomic_fetch_inc(&multifd_send_state->packet_num); - packet->packet_num = cpu_to_be64(packet_num); if (pages->block) { strncpy(packet->ramblock, pages->block->idstr, 256); } - for (i = 0; i < pages->num; i++) { + for (int i = 0; i < pages->num; i++) { /* there are architectures where ram_addr_t is 32 bit */ uint64_t temp = pages->offset[i]; packet->offset[i] = cpu_to_be64(temp); } - p->packets_sent++; p->total_normal_pages += pages->normal_num; p->total_zero_pages += zero_num; +} - trace_multifd_send(p->id, packet_num, pages->normal_num, zero_num, +void multifd_send_fill_packet(MultiFDSendParams *p) +{ + MultiFDPacket_t *packet = p->packet; + uint64_t packet_num; + + memset(packet, 0, p->packet_len); + + packet->magic = cpu_to_be32(MULTIFD_MAGIC); + packet->version = cpu_to_be32(MULTIFD_VERSION); + + packet->flags = cpu_to_be32(p->flags); + packet->next_packet_size = cpu_to_be32(p->next_packet_size); + + packet_num = qatomic_fetch_inc(&multifd_send_state->packet_num); + packet->packet_num = cpu_to_be64(packet_num); + + p->packets_sent++; + + multifd_ram_fill_packet(p); + + trace_multifd_send(p->id, packet_num, + be32_to_cpu(packet->normal_pages), + be32_to_cpu(packet->zero_pages), p->flags, p->next_packet_size); } -static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) +static int multifd_ram_unfill_packet(MultiFDRecvParams *p, Error **errp) { MultiFDPacket_t *packet = p->packet; int i; - packet->magic = be32_to_cpu(packet->magic); - if (packet->magic != MULTIFD_MAGIC) { - error_setg(errp, "multifd: received packet " - "magic %x and expected magic %x", - packet->magic, MULTIFD_MAGIC); - return -1; - } - - packet->version = be32_to_cpu(packet->version); - if (packet->version != MULTIFD_VERSION) { - error_setg(errp, "multifd: received packet " - "version %u and expected version %u", - packet->version, MULTIFD_VERSION); - return -1; - } - - p->flags = be32_to_cpu(packet->flags); - packet->pages_alloc = be32_to_cpu(packet->pages_alloc); /* * If we received a packet that is 100 times bigger than expected @@ -496,15 +493,9 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) return -1; } - p->next_packet_size = be32_to_cpu(packet->next_packet_size); - p->packet_num = be64_to_cpu(packet->packet_num); - p->packets_recved++; p->total_normal_pages += p->normal_num; p->total_zero_pages += p->zero_num; - trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->zero_num, - p->flags, p->next_packet_size); - if (p->normal_num == 0 && p->zero_num == 0) { return 0; } @@ -546,6 +537,40 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) return 0; } +static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) +{ + MultiFDPacket_t *packet = p->packet; + int ret = 0; + + packet->magic = be32_to_cpu(packet->magic); + if (packet->magic != MULTIFD_MAGIC) { + error_setg(errp, "multifd: received packet " + "magic %x and expected magic %x", + packet->magic, MULTIFD_MAGIC); + return -1; + } + + packet->version = be32_to_cpu(packet->version); + if (packet->version != MULTIFD_VERSION) { + error_setg(errp, "multifd: received packet " + "version %u and expected version %u", + packet->version, MULTIFD_VERSION); + return -1; + } + + p->flags = be32_to_cpu(packet->flags); + p->next_packet_size = be32_to_cpu(packet->next_packet_size); + p->packet_num = be64_to_cpu(packet->packet_num); + p->packets_recved++; + + ret = multifd_ram_unfill_packet(p, errp); + + trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->zero_num, + p->flags, p->next_packet_size); + + return ret; +} + static bool multifd_send_should_exit(void) { return qatomic_read(&multifd_send_state->exiting); From patchwork Mon Jul 22 17:59:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13738991 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1B0EDC3DA7F for ; Mon, 22 Jul 2024 18:00:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sVxJy-0003NQ-E0; Mon, 22 Jul 2024 13:59:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sVxJw-0003Fo-Ex for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:36 -0400 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sVxJt-00037t-MQ for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:36 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C0F7A21AE5; Mon, 22 Jul 2024 17:59:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671171; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vyOCNk8Ot7Z6gA0zkZ6O5tbVbGEXVWoBumOmL4EGmj4=; b=lAmPiCkQcntPn5++te68F1vgqieuJp+Oz7DxQRN7Q6CF403IKD7jG/6wY00fM0OPv141jk g3MbLDvzzIfvj3Cn/b6a6hdsYI4NnrKxl4qePPsIVFo8jZyfdOetVG+JV72UgwKV6LMRqj AmE7Zvbt5LxZ/QwF7sBG/5osy7YgSOo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671171; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vyOCNk8Ot7Z6gA0zkZ6O5tbVbGEXVWoBumOmL4EGmj4=; b=LKUEvKhb7NnjvBV5hHd5PEB5PZMzbmuLddyPMOfCRNf6p8iy2wMe6/+kFz7p8vN8F4Dk/9 ucPkoKPDQsX011DA== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=lAmPiCkQ; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=LKUEvKhb DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671171; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vyOCNk8Ot7Z6gA0zkZ6O5tbVbGEXVWoBumOmL4EGmj4=; b=lAmPiCkQcntPn5++te68F1vgqieuJp+Oz7DxQRN7Q6CF403IKD7jG/6wY00fM0OPv141jk g3MbLDvzzIfvj3Cn/b6a6hdsYI4NnrKxl4qePPsIVFo8jZyfdOetVG+JV72UgwKV6LMRqj AmE7Zvbt5LxZ/QwF7sBG/5osy7YgSOo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671171; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vyOCNk8Ot7Z6gA0zkZ6O5tbVbGEXVWoBumOmL4EGmj4=; b=LKUEvKhb7NnjvBV5hHd5PEB5PZMzbmuLddyPMOfCRNf6p8iy2wMe6/+kFz7p8vN8F4Dk/9 ucPkoKPDQsX011DA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 98CA713996; Mon, 22 Jul 2024 17:59:30 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0NS2FwKenmYnCQAAD6G6ig (envelope-from ); Mon, 22 Jul 2024 17:59:30 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , "Maciej S . Szmigiero" Subject: [RFC PATCH v2 8/9] migration/multifd: Don't send ram data during SYNC Date: Mon, 22 Jul 2024 14:59:13 -0300 Message-Id: <20240722175914.24022-9-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240722175914.24022-1-farosas@suse.de> References: <20240722175914.24022-1-farosas@suse.de> MIME-Version: 1.0 X-Spamd-Result: default: False [0.19 / 50.00]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FUZZY_BLOCKED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCPT_COUNT_THREE(0.00)[3]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Queue-Id: C0F7A21AE5 X-Rspamd-Action: no action X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Bar: / Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Skip saving and loading any ram data in the packet in the case of a SYNC. This fixes a shortcoming of the current code which requires a reset of the MultiFDPages_t fields right after the previous pending_job finishes, otherwise the very next job might be a SYNC and multifd_send_fill_packet() will put the stale values in the packet. By not calling multifd_ram_fill_packet(), we can stop resetting MultiFDPages_t in the multifd core and leave that to the client code. Actually moving the reset function is not yet done because pages->num==0 is used by the client code to determine whether the MultiFDPages_t needs to be flushed. The subsequent patches will replace that with a generic flag that is not dependent on MultiFDPages_t. Signed-off-by: Fabiano Rosas --- migration/multifd.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index d25b8658b2..4394ca6ade 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -438,6 +438,7 @@ void multifd_send_fill_packet(MultiFDSendParams *p) { MultiFDPacket_t *packet = p->packet; uint64_t packet_num; + bool sync_packet = p->flags & MULTIFD_FLAG_SYNC; memset(packet, 0, p->packet_len); @@ -452,7 +453,9 @@ void multifd_send_fill_packet(MultiFDSendParams *p) p->packets_sent++; - multifd_ram_fill_packet(p); + if (!sync_packet) { + multifd_ram_fill_packet(p); + } trace_multifd_send(p->id, packet_num, be32_to_cpu(packet->normal_pages), @@ -563,7 +566,12 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) p->packet_num = be64_to_cpu(packet->packet_num); p->packets_recved++; - ret = multifd_ram_unfill_packet(p, errp); + if (p->flags & MULTIFD_FLAG_SYNC) { + p->normal_num = 0; + p->zero_num = 0; + } else { + ret = multifd_ram_unfill_packet(p, errp); + } trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->zero_num, p->flags, p->next_packet_size); From patchwork Mon Jul 22 17:59:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 13738992 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 53BC4C3DA63 for ; Mon, 22 Jul 2024 18:00:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sVxK0-0003U2-4T; Mon, 22 Jul 2024 13:59:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sVxJy-0003PY-Rn for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:38 -0400 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sVxJv-000386-UQ for qemu-devel@nongnu.org; Mon, 22 Jul 2024 13:59:38 -0400 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6B4CD21B8C; Mon, 22 Jul 2024 17:59:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671173; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n6BPhaF+EZHTqmz/osYdJ7aHl1MRXlZSXHJwyPxwg10=; b=sTn+bC6CoapCX/rlFoeIGh3EWc3nT7NNaCm5dmEKjbhz6mjucjDluqHiOHw2MNPY/lioLz QVKWLfOl59XQcSRfFa8Dr17PMMH8UJiA8AOReUfILNqmePul2Ccd6s+jP6N8Cw8uUMgpKR AV3Y9HAKcd4PBgVH9s3Hy11Lkks/oQo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671173; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n6BPhaF+EZHTqmz/osYdJ7aHl1MRXlZSXHJwyPxwg10=; b=9fFa3i3ODvk3TYCo2Q/EJgvVGftmXH1YV6FtquxMoZ5/qsG+SWsfRplk9EQpiNZPypz3ng 8TFR4Srk4RC4IABQ== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=sTn+bC6C; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=9fFa3i3O DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1721671173; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n6BPhaF+EZHTqmz/osYdJ7aHl1MRXlZSXHJwyPxwg10=; b=sTn+bC6CoapCX/rlFoeIGh3EWc3nT7NNaCm5dmEKjbhz6mjucjDluqHiOHw2MNPY/lioLz QVKWLfOl59XQcSRfFa8Dr17PMMH8UJiA8AOReUfILNqmePul2Ccd6s+jP6N8Cw8uUMgpKR AV3Y9HAKcd4PBgVH9s3Hy11Lkks/oQo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1721671173; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n6BPhaF+EZHTqmz/osYdJ7aHl1MRXlZSXHJwyPxwg10=; b=9fFa3i3ODvk3TYCo2Q/EJgvVGftmXH1YV6FtquxMoZ5/qsG+SWsfRplk9EQpiNZPypz3ng 8TFR4Srk4RC4IABQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4148113996; Mon, 22 Jul 2024 17:59:32 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cF9lAgSenmYnCQAAD6G6ig (envelope-from ); Mon, 22 Jul 2024 17:59:32 +0000 From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , "Maciej S . Szmigiero" Subject: [RFC PATCH v2 9/9] migration/multifd: Replace multifd_send_state->pages with client data Date: Mon, 22 Jul 2024 14:59:14 -0300 Message-Id: <20240722175914.24022-10-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240722175914.24022-1-farosas@suse.de> References: <20240722175914.24022-1-farosas@suse.de> MIME-Version: 1.0 X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Rspamd-Queue-Id: 6B4CD21B8C X-Spamd-Result: default: False [-2.81 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; MIME_TRACE(0.00)[0:+]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_TLS_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:dkim,suse.de:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Multifd currently has a simple scheduling mechanism that distributes work to the various channels by keeping storage space within each channel and an extra space that is given to the client. Each time the client fills the space with data and calls into multifd, that space is given to the next idle channel and a free storage space is taken from the channel and given to client for the next iteration. This means we always need (#multifd_channels + 1) memory slots to operate multifd. This is fine, except that the presence of this one extra memory slot doesn't allow different types of payloads to be processed at the same time in different channels, i.e. the data type of multifd_send_state->pages needs to be the same as p->pages. For each new data type different from MultiFDPage_t that is to be handled, this logic would need to be duplicated by adding new fields to multifd_send_state, to the channels and to multifd_send_pages(). Fix this situation by moving the extra slot into the client and using only the generic type MultiFDSendData in the multifd core. Signed-off-by: Fabiano Rosas --- migration/multifd.c | 58 ++++++++++++++++++++++++++++----------------- migration/multifd.h | 2 ++ migration/ram.c | 1 + 3 files changed, 39 insertions(+), 22 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 4394ca6ade..0a85951d58 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -49,7 +49,6 @@ typedef struct { struct { MultiFDSendParams *params; - MultiFDSendData *data; /* * Global number of generated multifd packets. * @@ -97,6 +96,8 @@ struct { MultiFDMethods *ops; } *multifd_recv_state; +/* TODO: move these to multifd-ram.c */ +static MultiFDSendData *multifd_ram_send; static size_t multifd_ram_payload_size(void) { uint32_t n = MULTIFD_PACKET_SIZE / qemu_target_page_size(); @@ -118,6 +119,14 @@ static MultiFDSendData *multifd_send_data_alloc(void) return g_malloc0(sizeof(MultiFDPayloadType) + max_payload_size); } +void multifd_ram_save_setup(void) +{ + uint32_t n = MULTIFD_PACKET_SIZE / qemu_target_page_size(); + + multifd_ram_send = multifd_send_data_alloc(); + multifd_ram_send->u.ram.allocated = n; +} + static bool multifd_use_packets(void) { return !migrate_mapped_ram(); @@ -620,7 +629,7 @@ static void multifd_send_kick_main(MultiFDSendParams *p) * * Returns true if succeed, false otherwise. */ -static bool multifd_send_pages(void) +static bool multifd_send(MultiFDSendData **send_data) { int i; static int next_channel; @@ -661,11 +670,16 @@ static bool multifd_send_pages(void) */ smp_mb_acquire(); - assert(!p->data->u.ram.num); + assert(multifd_payload_empty(p->data)); - tmp = multifd_send_state->data; - multifd_send_state->data = p->data; + /* + * Swap the pointers. The channel gets the client data for + * transferring and the client gets back an unused data slot. + */ + tmp = *send_data; + *send_data = p->data; p->data = tmp; + /* * Making sure p->data is setup before marking pending_job=true. Pairs * with the qatomic_load_acquire() in multifd_send_thread(). @@ -697,7 +711,12 @@ bool multifd_queue_page(RAMBlock *block, ram_addr_t offset) MultiFDPages_t *pages; retry: - pages = &multifd_send_state->data->u.ram; + pages = &multifd_ram_send->u.ram; + + if (multifd_payload_empty(multifd_ram_send)) { + multifd_pages_reset(pages); + multifd_set_payload_type(multifd_ram_send, MULTIFD_PAYLOAD_RAM); + } /* If the queue is empty, we can already enqueue now */ if (multifd_queue_empty(pages)) { @@ -715,7 +734,7 @@ retry: * After flush, always retry. */ if (pages->block != block || multifd_queue_full(pages)) { - if (!multifd_send_pages()) { + if (!multifd_send(&multifd_ram_send)) { return false; } goto retry; @@ -833,6 +852,8 @@ static bool multifd_send_cleanup_channel(MultiFDSendParams *p, Error **errp) g_free(p->packet); p->packet = NULL; multifd_send_state->ops->send_cleanup(p, errp); + g_free(p->data); + p->data = NULL; return *errp == NULL; } @@ -845,8 +866,6 @@ static void multifd_send_cleanup_state(void) qemu_sem_destroy(&multifd_send_state->channels_ready); g_free(multifd_send_state->params); multifd_send_state->params = NULL; - g_free(multifd_send_state->data); - multifd_send_state->data = NULL; g_free(multifd_send_state); multifd_send_state = NULL; } @@ -895,15 +914,14 @@ int multifd_send_sync_main(void) { int i; bool flush_zero_copy; - MultiFDPages_t *pages; if (!migrate_multifd()) { return 0; } - pages = &multifd_send_state->data->u.ram; - if (pages->num) { - if (!multifd_send_pages()) { - error_report("%s: multifd_send_pages fail", __func__); + + if (!multifd_payload_empty(multifd_ram_send)) { + if (!multifd_send(&multifd_ram_send)) { + error_report("%s: multifd_send fail", __func__); return -1; } } @@ -977,13 +995,11 @@ static void *multifd_send_thread(void *opaque) /* * Read pending_job flag before p->data. Pairs with the - * qatomic_store_release() in multifd_send_pages(). + * qatomic_store_release() in multifd_send(). */ if (qatomic_load_acquire(&p->pending_job)) { - MultiFDPages_t *pages = &p->data->u.ram; - p->iovs_num = 0; - assert(pages->num); + assert(!multifd_payload_empty(p->data)); ret = multifd_send_state->ops->send_prepare(p, &local_err); if (ret != 0) { @@ -1006,13 +1022,13 @@ static void *multifd_send_thread(void *opaque) stat64_add(&mig_stats.multifd_bytes, p->next_packet_size + p->packet_len); - multifd_pages_reset(pages); p->next_packet_size = 0; + multifd_set_payload_type(p->data, MULTIFD_PAYLOAD_NONE); /* * Making sure p->data is published before saying "we're * free". Pairs with the smp_mb_acquire() in - * multifd_send_pages(). + * multifd_send(). */ qatomic_store_release(&p->pending_job, false); } else { @@ -1206,8 +1222,6 @@ bool multifd_send_setup(void) thread_count = migrate_multifd_channels(); multifd_send_state = g_malloc0(sizeof(*multifd_send_state)); multifd_send_state->params = g_new0(MultiFDSendParams, thread_count); - multifd_send_state->data = multifd_send_data_alloc(); - multifd_send_state->data->u.ram.allocated = page_count; qemu_sem_init(&multifd_send_state->channels_created, 0); qemu_sem_init(&multifd_send_state->channels_ready, 0); qatomic_set(&multifd_send_state->exiting, 0); diff --git a/migration/multifd.h b/migration/multifd.h index c9c01579a0..04c000f435 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -113,6 +113,8 @@ static inline void multifd_set_payload_type(MultiFDSendData *data, data->type = type; } +void multifd_ram_save_setup(void); + typedef struct { /* Fields are only written at creating/deletion time */ /* No lock required for them, they are read only */ diff --git a/migration/ram.c b/migration/ram.c index edec1a2d07..2b90396b3c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3058,6 +3058,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque, Error **errp) migration_ops = g_malloc0(sizeof(MigrationOps)); if (migrate_multifd()) { + multifd_ram_save_setup(); migration_ops->ram_save_target_page = ram_save_target_page_multifd; } else { migration_ops->ram_save_target_page = ram_save_target_page_legacy;