From patchwork Mon May 8 19:10:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 13234835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4D4E1C77B7F for ; Mon, 8 May 2023 19:11:44 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pw6GD-0007Dm-4a; Mon, 08 May 2023 15:11:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pw6GA-0007DP-Sx for qemu-devel@nongnu.org; Mon, 08 May 2023 15:10:58 -0400 Received: from mout.web.de ([212.227.15.4]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pw6G9-0003wa-9e for qemu-devel@nongnu.org; Mon, 08 May 2023 15:10:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=s29768273; t=1683573053; i=lukasstraub2@web.de; bh=cMt5vCpBN0KI8tnXhcVX0yiidCI3Z21rKwnl39vBX5Y=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=GkQgUkAvffQ7bxKLlkVxZF3QBT/RGaL3L9SU8kNSWEwNw1BOHZB+JTqcb34XJyksd 5iWiGE6/f/yZdEYhvdcYmb4W3cmgl1xBjUViIiCIn8dYs0PVvl8AO7IsjigfVgoIeD 44+fTST/ch7qBRUbfQlxAZolJR05ftEK6Kr+d54vytRdgG0OrPPkumi5jrvlr94hUZ tf6xig5G/wqPaR8VCCpW7FNlHQCnKKCMNq+nUQLPJKPu6Cpr8geLEYxsP0v3XGhQcm VkO+0Ogng2G36MPgnS3cVBMEGcPJbeFGZOTPfibPlOeHM8/EYoihmn6XF3f2b42bPR obxzFbKH/uEDQ== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from gecko.fritz.box ([82.207.254.98]) by smtp.web.de (mrweb006 [213.165.67.108]) with ESMTPSA (Nemesis) id 1MmQYX-1qdzeG1VPV-00i9XW; Mon, 08 May 2023 21:10:53 +0200 Date: Mon, 8 May 2023 21:10:52 +0200 From: Lukas Straub To: qemu-devel Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v2 1/6] ram: Add public helper to set colo bitmap Message-ID: <22d83cb428f37929563155531bfb69fd8953cc61.1683572883.git.lukasstraub2@web.de> In-Reply-To: References: MIME-Version: 1.0 X-Provags-ID: V03:K1:f8gZ6rAjohk1tekSySYhZYJEOk1esAs1lE+fHTyjAwHTB23PUZ6 beDmN45RSfzoOnjfbZ0bzmChkvaVcUs/0w0O+ciz2+ZnPzlY3LisFCyxZDNVF7w8/d0yJqG ptXt8pXfhFLpDw1YTWlh8YhuUYHobu68nZQP01yRbjmCjJfW7CdtiuvonbFlDF0XJOSi9QW WrtuTkTZ8nr3aYb7NXX7g== UI-OutboundReport: notjunk:1;M01:P0:tjVevSqgFc4=;/20tj7JIfVqY7VRglN5qVovrMJJ Xl7xz9OF9cQ3wkSkTl84oOwOx1m3hFEvV0L7OU/VyZt9OxLQZCSUFgGI+8FIyS73gBhkFZbgK J/rhZ8yV+4xqcJ2zX8Y7/Jtp9yQGUd54HoInQmPnlVKSUvanJQI6DAjjMkHVidJdCZS5q9ic7 h7/HdzTW7Tu1RIS20sZw1OMnCrWn16WcR/Goutf8G0R6Xc1OM4j5Tv5b3ZcMDFhAMrTcymmDV FLs2j9tYDsu2+JG2FfylBZ5NIP6MDsLnod9Z3gZA7xQEUvDJvtfMyG6A6zYJzhil2WcjzwU+B X8Kw8f4x8emZZ0JSxAPOWxg1sXGSOdh9iesojJFpvA5lGIFlA5Za3bCvRzvyXlimp6TPpIsND QgNZ9mTDxYYZ4k8ZwNlBIHZDI80lzBe0MgWWfREfJ8OYBn7TklI1IW8bCmRPu9NjgT4/Lum3z Mwtk13YVx4NwZh+CQG5EH1eHx/GN4JRdQOEd0gYWZ2CdK+WfBFAneKdNoOiJaTdaHijWuiaij B0oKe/Ml240xOpx05RBI0v4ecukgsWflwqPt6FaPCM4Gi7VoL10RoV3IDce+FdOaELfbLwpmh Oc09yGlTYdrgfSUcnamqv9QPjGCTyI+azLw5MYy1Jv/NwpiY+a22P2LvVl+vxgSli5YBb/Nze n6rCCAN3THxQdBZaORcc8vPCqeJGxondyxaC0YGAqp0BwPzoN58rz7DRTybIcxSJjbfl6CkSD qTsyeDaoxVSlGezWd7wtw72UE/bIkkOl8zLOZUxEpnOKGWidG+JruKSjW1YPo+HJyzr1ryciM JvTx93stT9JWqjylbiLMzjPUh9qG2ecp60uwBK6HS3/EstMxEuUS3o1WsaIGpbMBHKqVm90nr w/7fmiQL0IrVJu5FHEJ6S5z/1pquze3BkitIIvzmgXUTdiLJKDUWViwyE83aYC5xQrTFQQuFE QS3y/Pkf7we9I84jeHvAx4jpG48= Received-SPF: pass client-ip=212.227.15.4; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The overhead of the mutex in non-multifd mode is negligible, because in that case its just the single thread taking the mutex. This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela --- migration/ram.c | 17 ++++++++++++++--- migration/ram.h | 1 + 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 5e7bf20ca5..2d3fd2112a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3633,6 +3633,18 @@ static ram_addr_t host_page_offset_from_ram_block_offset(RAMBlock *block, return ((uintptr_t)block->host + offset) & (block->page_size - 1); } +void colo_record_bitmap(RAMBlock *block, ram_addr_t *normal, uint normal_num) +{ + qemu_mutex_lock(&ram_state->bitmap_mutex); + for (int i = 0; i < normal_num; i++) { + ram_addr_t offset = normal[i]; + ram_state->migration_dirty_pages += !test_and_set_bit( + offset >> TARGET_PAGE_BITS, + block->bmap); + } + qemu_mutex_unlock(&ram_state->bitmap_mutex); +} + static inline void *colo_cache_from_block_offset(RAMBlock *block, ram_addr_t offset, bool record_bitmap) { @@ -3650,9 +3662,8 @@ static inline void *colo_cache_from_block_offset(RAMBlock *block, * It help us to decide which pages in ram cache should be flushed * into VM's RAM later. */ - if (record_bitmap && - !test_and_set_bit(offset >> TARGET_PAGE_BITS, block->bmap)) { - ram_state->migration_dirty_pages++; + if (record_bitmap) { + colo_record_bitmap(block, &offset, 1); } return block->colo_cache + offset; } diff --git a/migration/ram.h b/migration/ram.h index 6fffbeb5f1..887d1fbae6 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -82,6 +82,7 @@ int colo_init_ram_cache(void); void colo_flush_ram_cache(void); void colo_release_ram_cache(void); void colo_incoming_start_dirty_log(void); +void colo_record_bitmap(RAMBlock *block, ram_addr_t *normal, uint normal_num); /* Background snapshot */ bool ram_write_tracking_available(void); From patchwork Mon May 8 19:10:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 13234836 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 864C6C77B75 for ; Mon, 8 May 2023 19:11:45 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pw6GH-0007ER-MB; Mon, 08 May 2023 15:11:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pw6GF-0007E7-06 for qemu-devel@nongnu.org; Mon, 08 May 2023 15:11:03 -0400 Received: from mout.web.de ([212.227.15.4]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pw6GD-00041A-EH for qemu-devel@nongnu.org; Mon, 08 May 2023 15:11:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=s29768273; t=1683573057; i=lukasstraub2@web.de; bh=0QH53IRgOPhT49cv5vsdm5gVQIKWEzSU5q9yPGwkZWk=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=ltRJdQoTuEKMo+tbTwvlbVE1gpc1B73hee6jZImk1QF0rr1Fvqp+R7rpNn/+s4ZEh 7fIfmHt2VpKDTje+kADEUZZDuZCmpKQvEUE/oEKJRI2ADkbkTa29VbzPWjmYl35VEx a/thCBW3dZQAlUpKfux3e0Xnnp3h4Dwq9P7zO69IwxiPsP9+Ebr+uig6NdrYopPbYS aQjGyKomnHE4GV7qA99ckpkvFAvaHIAESyYqMzDf15wSSIokUnnD8aZn26HHA8yUxx 0vX/+vg4Wm7zEHyL4HaT5GvEwxeS66FVMZPFsz+1RxVGIBlHF13kHrI4jojBuZ2/0L QXX7wtK1/4HAA== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from gecko.fritz.box ([82.207.254.98]) by smtp.web.de (mrweb005 [213.165.67.108]) with ESMTPSA (Nemesis) id 1MUCz1-1pnXff3d3b-00RIQj; Mon, 08 May 2023 21:10:56 +0200 Date: Mon, 8 May 2023 21:10:55 +0200 From: Lukas Straub To: qemu-devel Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v2 2/6] ram: Let colo_flush_ram_cache take the bitmap_mutex Message-ID: <35cb23ba854151d38a31e3a5c8a1020e4283cb4a.1683572883.git.lukasstraub2@web.de> In-Reply-To: References: MIME-Version: 1.0 X-Provags-ID: V03:K1:Jdr5G3n2E6kpeS4hvyaa7eCAvWjLD1iXSr+vQ3RGLXYjjc3EBN3 DWXtQtj17JMQjEKdAcQqsT8gFq3WbNY8lhLFicRRCOiVP0hCQ1PyWgJKL/y2jXcnvQsJ34r b06JuauNTh2+zYmcrQ59mmjNvDbB/6vNVtbvkye7H90mKhmUznpEhleXDrVC0/iH6cjS9n5 7nn5HT0kgvPZ2XPPiuMZA== UI-OutboundReport: notjunk:1;M01:P0:Ul7hvAOpr4Q=;ivephjLNZcFV353pSwe2W7VMcYW vKNiUGHxT8OoXIdAjYchjAnpaJIhom9mulM1DF4ASJugDiDgjqtzY1ZO2wIOKa5iGCSugolQw aULxRvhrmPXv5cu2+6Gp2mBzzc8GOM0LWKpmffVBSohE6maHlyGjK/3CPEqHSvIpr7pPC7QoN PlRHgwxzCget1VHGX/r/aZ0XxqrsFZIHozOqZXkWcQJMINBQsji+0CZ6XCQOlxzu20oWILNlE fNxZwaRm3bFg+B+VNzFMvaxWtzbXyDVxVYJjY/ho0jVbnpCLNybnZjS8HunWpMa459P3PNraX sS9nF8HZHOt/dcjM0F3x8fCghvcK8goORN1rY1WycipUCfmQOZZX7Q+eCBdVtZ9nKfEc5twjf b/eXTspJ7/DYpvTrvMZYxFKjLBl7tmDnAJlH0CoAIYRQfG0Eihx3U+Ss01eTMEEH0Da+mrNm/ EbOxbVzpZtI9ejDwB/V/0VpStPty8YSgEx+FA5k52fagq99/YhBU+v1N4OXVVWP3Szx8GMilo 2YtNE69jB8PvmKRUr+MU7tGadaci7Gm2u7jJJcnm7F0y26imY3Ggo6l09hEptezp0T5INdMjN ThhghTNLxv9vDfxv5usHxfzswO6ksgfAv80Mu5cnpXGSOIUSqRUgO1210J3CZQw3CIwjxGtUq 0kt+Fe+WMAlLcoOCV3F5vB8oGUbd4Ppvv5RmRYKF43Cpx/pjmtujEOBzAbAve80SxziWIXrZy +b/hvX0VjzBKu2AYS1fTI0a4SI5plmAM7tR0Y5i+otr3CJivJvhx08hj/33wjCyKroLwlX29T 1noTVNB+b5OohDd2V+ehp2qVLFQFf7NrFFirPhTW+Xjaxy+fKYvrG64I0de4UJD9zMKx2Dlbl Oh9+9jwQuwy4Ner4/jNnJTr7+xTk+y+xnqfAIu9WnvTYG9JsWp95vhEj5m/BwsH20yN/VuiDf Eve+C+ArLSUphu+EA1mRMe9Wb/4= Received-SPF: pass client-ip=212.227.15.4; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This is not required, colo_flush_ram_cache does not run concurrently with the multifd threads since the cache is only flushed after everything has been received. But it makes me more comfortable. This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela --- migration/ram.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 2d3fd2112a..f9e7aeda12 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -4230,6 +4230,7 @@ void colo_flush_ram_cache(void) unsigned long offset = 0; memory_global_dirty_log_sync(); + qemu_mutex_lock(&ram_state->bitmap_mutex); WITH_RCU_READ_LOCK_GUARD() { RAMBLOCK_FOREACH_NOT_IGNORED(block) { ramblock_sync_dirty_bitmap(ram_state, block); @@ -4264,6 +4265,7 @@ void colo_flush_ram_cache(void) } } } + qemu_mutex_unlock(&ram_state->bitmap_mutex); trace_colo_flush_ram_cache_end(); } From patchwork Mon May 8 19:11:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 13234839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 98C9FC7EE2A for ; Mon, 8 May 2023 19:12:19 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pw6GO-0007GD-VO; Mon, 08 May 2023 15:11:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pw6GM-0007Fc-TR for qemu-devel@nongnu.org; Mon, 08 May 2023 15:11:10 -0400 Received: from mout.web.de ([212.227.15.4]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pw6GI-00043e-2z for qemu-devel@nongnu.org; Mon, 08 May 2023 15:11:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=s29768273; t=1683573061; i=lukasstraub2@web.de; bh=RMPaB96clX712MJVaf16l7dBlsMK+Puiiaf/vcCDJIw=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=K287YoeBj1St36rKkXvZ7A0eKGLaFOtlyk1vVLaSAjZPUZdfrWljGef2ybaJy6S4J 39CzJxny799tPoiGMZjqwnvuyw47PCncJRwumHCOpLwT2qloUhN2cqYIi2/BAusFFN p5nk83MfEF3dGZQopKEAHO6+4CxqSoFcRKQnvhj4Wy9vszxQYUr7DHH9maBLmr7NC5 mERlu77r4ZwklbJb5QfJ9EvMOa1AeCizjaHadTujlQuDUevvTTyMu4NEx4c4Lujm07 0KTHbqEEpAMSuBXitcW7JDtKoUzGNO8IjFLJsD1/ALUdNMN/b1lQPTsrg7JFrOjRDr g7cERqDEVDn+g== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from gecko.fritz.box ([82.207.254.98]) by smtp.web.de (mrweb005 [213.165.67.108]) with ESMTPSA (Nemesis) id 1MV2KQ-1povFT1JPK-00S7IC; Mon, 08 May 2023 21:11:01 +0200 Date: Mon, 8 May 2023 21:11:00 +0200 From: Lukas Straub To: qemu-devel Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v2 3/6] multifd: Introduce multifd-internal.h Message-ID: <613b971dbdd71a7719667ae3fd607b8542afefe2.1683572883.git.lukasstraub2@web.de> In-Reply-To: References: MIME-Version: 1.0 X-Provags-ID: V03:K1:eWpObQv2KgCUZM1/uuYwN/9RChh6RUHRoQmw9NDk5ps753hZySC 05NoM1g0JbfJCk98yWjquxRuxVB6I9bHDmXbcKQ1eZ92JSddE0uz8/KVr7L8onx9xJT87LV 7to8nnX3Its6+J/jprcfgWOcrlAz+4BFtRDif1mGa1oZpRVtToKxoh39jMRDwopCeMmc9GH 1lfHz4j0QFHID0Gn2j9cg== UI-OutboundReport: notjunk:1;M01:P0:GaatC1caZas=;roL9yCczVF7we35eoBYUugV46cM FU4LxnOwT14HkTvQCRjrPuzwBwTuZ3jlic6N6miMSFdlJrhuxZpRDNpgfpMwl1oZkFywGLG+S /Qcam6WtlD0Co54xre46yWzP6pjVAj6n++yKyZWf6ViVa7rP5iJL3N//hKDmj3iNfRNSqeydN Zzw1EgnTl0AuQPX4QoxDU9KhGuxe9MIGNgv4IxqjP9S1/8Trt0kogXQHdf/AqjsXafX8cgf9P hhWtmZinercDqkhnZn4mCaMNs8hvXmXwHn66/aXhzcVMntJJb8YyIURH1lME6HRP9+o7VdxSI Q8K8krXskxkmd/hjkKHXkmee2NZ0YUsFYIoXYOWGLa5BxcNq2SRby23FpH/npNV71OD4lv5ag HNZyP9wHVu2nkDvJdpD8rYzlD0K8omgjKNx8/xOnH8x7Qfed2pETc3pYOGCsKTPT2C/IbCCpB ax2FW0IH4e6f6m7/zOBcZgC+oCzqJ4A5ljNXkcsrQXsjnG+4IldMceezeSBMl14v4F32clMxn zzUBsXPVlfiwuTesZBObajLVb137ItbVuvATZfWtv/660BMu727yBUyUA8BVxnxS2SCjMnLCh xPslMcbIA5zcoHVWwCWxX+m/7R4RVY1SCnqWLnT/XQH2N9dbbkvQees57tKxwq+0AhnZ3hojB lUmnoDBuh4hb5EbGts/9d7UxVnW5Ms/0AALzR0XxBFiVedJVBk/VotNJ39Ez766eUxhybq8B/ kRe32MFb9Cn9JyIIiLLNIQNW7KNyrMHj2PD0BjeqCuf5utoO84VeaYVrjY6LwX/r6/t66Gd/6 qsl5SBGKxQ3ehwIPmcBUoyPwTc7LCDjhuXPy2e79wJ6nnPEF2TONAYYoajP3YSeupoE2g//YZ QJyXrHrCoI8vz1w0+64a622G/N7ksJTl+P/KLTeBKyc5C3P6F6bEQJcbxlPI7tdVFieJez0mT v12Ww/gP4QCSmlwGq3Op0fPsbV8= Received-SPF: pass client-ip=212.227.15.4; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Introduce multifd-internal.h so code that would normally go into multifd.c can go into an extra file. This way, multifd.c hopefully won't grow to 4000 lines like ram.c This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub --- migration/multifd-internal.h | 34 ++++++++++++++++++++++++++++++++++ migration/multifd.c | 15 ++++----------- 2 files changed, 38 insertions(+), 11 deletions(-) create mode 100644 migration/multifd-internal.h diff --git a/migration/multifd-internal.h b/migration/multifd-internal.h new file mode 100644 index 0000000000..6eeaa028e7 --- /dev/null +++ b/migration/multifd-internal.h @@ -0,0 +1,34 @@ +/* + * Internal Multifd header + * + * Copyright (c) 2019-2020 Red Hat Inc + * + * Authors: + * Juan Quintela + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifdef QEMU_MIGRATION_MULTIFD_INTERNAL_H +#error Only include this header directly +#endif +#define QEMU_MIGRATION_MULTIFD_INTERNAL_H + +#ifndef MULTIFD_INTERNAL +#error This header is internal to multifd +#endif + +struct MultiFDRecvState { + MultiFDRecvParams *params; + /* number of created threads */ + int count; + /* syncs main thread and channels */ + QemuSemaphore sem_sync; + /* global number of generated multifd packets */ + uint64_t packet_num; + /* multifd ops */ + MultiFDMethods *ops; +}; + +extern struct MultiFDRecvState *multifd_recv_state; diff --git a/migration/multifd.c b/migration/multifd.c index 4e71c19292..f6bad69b6c 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -31,6 +31,9 @@ #include "io/channel-socket.h" #include "yank_functions.h" +#define MULTIFD_INTERNAL +#include "multifd-internal.h" + /* Multiple fd's */ #define MULTIFD_MAGIC 0x11223344U @@ -967,17 +970,7 @@ int multifd_save_setup(Error **errp) return 0; } -struct { - MultiFDRecvParams *params; - /* number of created threads */ - int count; - /* syncs main thread and channels */ - QemuSemaphore sem_sync; - /* global number of generated multifd packets */ - uint64_t packet_num; - /* multifd ops */ - MultiFDMethods *ops; -} *multifd_recv_state; +struct MultiFDRecvState *multifd_recv_state; static void multifd_recv_terminate_threads(Error *err) { From patchwork Mon May 8 19:11:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 13234838 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5BC01C77B7F for ; Mon, 8 May 2023 19:12:18 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pw6GP-0007GV-VZ; Mon, 08 May 2023 15:11:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pw6GO-0007G2-JA for qemu-devel@nongnu.org; Mon, 08 May 2023 15:11:12 -0400 Received: from mout.web.de ([212.227.15.3]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pw6GK-000448-Sn for qemu-devel@nongnu.org; Mon, 08 May 2023 15:11:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=s29768273; t=1683573065; i=lukasstraub2@web.de; bh=RLJxB0vqXLDgxgpf3+7W0iM4sU9TCrsn1WfbSjL6Mi0=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=QWZUdqWKtV7i4AMWQJdYHGBjrXgEWiECHS1f8OSxaZY0WFKDgMx7zdH/WBnPUSblq bU53pFV+WjJDuVeFbGuCgqBmIy+2/gL9vzdrSxpLuO+0VyF1M5/lTzyBQr/9HnwjSe U6U2dutZPEgifVZ5qWWZFPwHMm564+qKp9ybsZiy+5g8ee0pPhks8kuU3iQnBgR28B /gqb8gTq88ScYqmGkBU8JbJZTJPLORaWiv2U+O5XRdTpKzIIdkoDONeRiHplXiC905 MjExn2ck/3kIlrmNcXFX7L9ykAdNhnH3qamcTq0JWW0Nv5XCu7NKOvbG828bP5DoOp pfjWuHFfjCJuQ== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from gecko.fritz.box ([82.207.254.98]) by smtp.web.de (mrweb005 [213.165.67.108]) with ESMTPSA (Nemesis) id 1MQPdl-1pZm1Z3WG9-00MOik; Mon, 08 May 2023 21:11:04 +0200 Date: Mon, 8 May 2023 21:11:03 +0200 From: Lukas Straub To: qemu-devel Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v2 4/6] multifd: Introduce a overridable revc_pages method Message-ID: <276b19d89a054aadce0eb69269e114e164a1da9d.1683572883.git.lukasstraub2@web.de> In-Reply-To: References: MIME-Version: 1.0 X-Provags-ID: V03:K1:YaOgXu2wN4wgyqsCGz9e4iTpGRW8VX0oW7tmdfsbX67UA2ho9cC 7rPqRKWedt+Pp4oss4OjsKhVltNfHo2ko9NZZGJ6vORTE9Mxek80hxQIlw0+4g9cEUQYOjc hDxTFxFtcCHXfvV9rIKSNGBwLSKNar9vU+Whajh2lTr9aQqZmaEiq7UP9AG33ethOEHYU4H /uWHkVmv3i5p5Hxm98+tg== UI-OutboundReport: notjunk:1;M01:P0:Zt8udie3o9k=;JGCM98op9+j6EnrL3tpRwl42VLN r1yHWoqtJgrsfesLiZ5kIfINMp6NR/55x1AkR8JdTSDJuniVi8Z10dPvu/RbFh/8ap2iXuc7Y B0TYqGJMhaBzKQqOsswOVZmV2+RiFUBQxgY8KATiRb6r430P4nZFt8D0j6JJPNKlq/vWEP/XB /YkIAyZtZu5p1ZV/tI0pjeRtRlRNJpYRKhUU+oF/mYtyqkuf/VV9tk+qyEq4PKZZ9YTIZN77D AIbNxVByykq0bRUfymUWHgd9ZLopS+qUGoX7Q/qS/6Eg3SHILa/EjqDajiKBLrTuzpYKjMUKF xRtU3SCbVUMz00rP36i0/KvCcD2jcdz68/MjedPpWM73edHrPZcDl+v/cNBObpaO7rHB/3/Oh KHXvGcSn3XFfhoZ4zzyWHqpXVsvV3E6sZ3VVraMBc8zkVIMCgzp2lzgZ4yV45tHO/85FDryXF 3TuKmWHNOqnTlDBbLPJ8c4zijJEQhV31MFlP1HQsd5Sk8z1QdonhnB415EbjxpKv+UIcnRy7G us6lzoGf5dpCn5bkmKlZWM0/MHHfbmN+a/VLwx4SsWub6iUIKesjBU4+QW4k1vZUqdlceaEjI TE1upyxr1RDXsGhsJdszebgsM2gg0JeNFhkGgCXGeDRFf15S+t2IgfE1G3I7ZJ+QpT/x1ZMUY WVxsx23NthAHcewBAMXon4xCLHojuqaDb3lu0eUgpuFLczTC2Q6cW/iGpFojYDj6BG+2UiBu7 9YQWVxZVCj8ICNxyVcxLIMH77Jxic6AnpwGpQ0PpUUjm8O4ox0kJ5ZYIuCnmEXmG/76mPryNO yJB4xQexlogacIKJxf3QWvkCYxr39PbUiSnAS52fdb78KOB/ZHBMbFw2qS8+hop/OhvCH0fBO NXiHwpNE4TIMrDutE14LnPeUqPl6VB7zPK28TmbJGOT6wWcj86Xkh7IWonvJU6Wj/hF4QiZTu a23SKWQp+RM3ZxP1mOeRUgBN/6U= Received-SPF: pass client-ip=212.227.15.3; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This allows to override the behaviour around recv_pages. Think of it like a "multifd_colo" child class of multifd. This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub --- migration/meson.build | 1 + migration/multifd-colo.c | 39 +++++++++++++++++++++++++++++ migration/multifd-internal.h | 5 ++++ migration/multifd.c | 48 ++++++++++++++++++++++++++++-------- 4 files changed, 83 insertions(+), 10 deletions(-) create mode 100644 migration/multifd-colo.c diff --git a/migration/meson.build b/migration/meson.build index da1897fadf..22ab6c6d73 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -23,6 +23,7 @@ softmmu_ss.add(files( 'migration.c', 'multifd.c', 'multifd-zlib.c', + 'multifd-colo.c', 'options.c', 'postcopy-ram.c', 'savevm.c', diff --git a/migration/multifd-colo.c b/migration/multifd-colo.c new file mode 100644 index 0000000000..c035d15e87 --- /dev/null +++ b/migration/multifd-colo.c @@ -0,0 +1,39 @@ +/* + * multifd colo implementation + * + * Copyright (c) Lukas Straub + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "exec/target_page.h" +#include "exec/ramblock.h" +#include "qemu/error-report.h" +#include "qapi/error.h" +#include "ram.h" +#include "multifd.h" +#include "io/channel-socket.h" + +#define MULTIFD_INTERNAL +#include "multifd-internal.h" + +static int multifd_colo_recv_pages(MultiFDRecvParams *p, Error **errp) +{ + return multifd_recv_state->ops->recv_pages(p, errp); +} + +int multifd_colo_load_setup(Error **errp) +{ + int ret; + + ret = _multifd_load_setup(errp); + if (ret) { + return ret; + } + + multifd_recv_state->recv_pages = multifd_colo_recv_pages; + + return 0; +} diff --git a/migration/multifd-internal.h b/migration/multifd-internal.h index 6eeaa028e7..82357f1d88 100644 --- a/migration/multifd-internal.h +++ b/migration/multifd-internal.h @@ -29,6 +29,11 @@ struct MultiFDRecvState { uint64_t packet_num; /* multifd ops */ MultiFDMethods *ops; + /* overridable recv method */ + int (*recv_pages)(MultiFDRecvParams *p, Error **errp); }; extern struct MultiFDRecvState *multifd_recv_state; + +int _multifd_load_setup(Error **errp); +int multifd_colo_load_setup(Error **errp); diff --git a/migration/multifd.c b/migration/multifd.c index f6bad69b6c..fb5e8859de 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -1126,7 +1126,7 @@ static void *multifd_recv_thread(void *opaque) qemu_mutex_unlock(&p->mutex); if (p->normal_num) { - ret = multifd_recv_state->ops->recv_pages(p, &local_err); + ret = multifd_recv_state->recv_pages(p, &local_err); if (ret != 0) { break; } @@ -1152,20 +1152,12 @@ static void *multifd_recv_thread(void *opaque) return NULL; } -int multifd_load_setup(Error **errp) +int _multifd_load_setup(Error **errp) { int thread_count; uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); uint8_t i; - /* - * Return successfully if multiFD recv state is already initialised - * or multiFD is not enabled. - */ - if (multifd_recv_state || !migrate_multifd()) { - return 0; - } - thread_count = migrate_multifd_channels(); multifd_recv_state = g_malloc0(sizeof(*multifd_recv_state)); multifd_recv_state->params = g_new0(MultiFDRecvParams, thread_count); @@ -1204,6 +1196,42 @@ int multifd_load_setup(Error **errp) return 0; } +static int multifd_normal_recv_pages(MultiFDRecvParams *p, Error **errp) +{ + return multifd_recv_state->ops->recv_pages(p, errp); +} + +static int multifd_normal_load_setup(Error **errp) +{ + int ret; + + ret = _multifd_load_setup(errp); + if (ret) { + return ret; + } + + multifd_recv_state->recv_pages = multifd_normal_recv_pages; + + return 0; +} + +int multifd_load_setup(Error **errp) +{ + /* + * Return successfully if multiFD recv state is already initialised + * or multiFD is not enabled. + */ + if (multifd_recv_state || !migrate_multifd()) { + return 0; + } + + if (migrate_colo()) { + return multifd_colo_load_setup(errp); + } else { + return multifd_normal_load_setup(errp); + } +} + bool multifd_recv_all_channels_created(void) { int thread_count = migrate_multifd_channels(); From patchwork Mon May 8 19:11:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 13234837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6BC0BC77B75 for ; Mon, 8 May 2023 19:12:02 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pw6GR-0007H3-II; Mon, 08 May 2023 15:11:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pw6GQ-0007GZ-JP for qemu-devel@nongnu.org; Mon, 08 May 2023 15:11:14 -0400 Received: from mout.web.de ([212.227.15.4]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pw6GO-00044l-QI for qemu-devel@nongnu.org; Mon, 08 May 2023 15:11:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=s29768273; t=1683573068; i=lukasstraub2@web.de; bh=VWfgn0vy+usTipXAMejAs4Eb46fij7bBgTFx5TjexRE=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=tEwfCMx3IUVsT0B9KaIq3NhKJxjCPNdZeD6nZGkh8IBON5B2Bc60mcX/8JT0it7dx ZJ1y4UENRV3+qImGvArCfDXvYcZDmihOyWx5+93wnpJTHWHtkJUcchhIvoTmdsTL+d 3EvSr/T/jBQWsRs5r4yhbBn6k7R0RFE0pF15h8KlHGVx4vCk2JKpj5l5UhWCjkdt0k 7mUPBBtMjNfylRuTjy69ySuFRMCCV5qTnGEKma5emFfACaGgF8QoPSiIQ/211iUVPH I2pGw6CXj5anYhq9x2ItY6CB+mnDMb5Z7tGUoleAvympktS8XEMjlFdCDHtuJvJe8o jGpQooMc9BEuw== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from gecko.fritz.box ([82.207.254.98]) by smtp.web.de (mrweb005 [213.165.67.108]) with ESMTPSA (Nemesis) id 1Meler-1qWGQT1Og1-00akKA; Mon, 08 May 2023 21:11:08 +0200 Date: Mon, 8 May 2023 21:11:07 +0200 From: Lukas Straub To: qemu-devel Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v2 5/6] multifd: Add the ramblock to MultiFDRecvParams Message-ID: <88135197411df1a71d7832962b39abf60faf0021.1683572883.git.lukasstraub2@web.de> In-Reply-To: References: MIME-Version: 1.0 X-Provags-ID: V03:K1:Kk4+CG4/VqX2tK8marXR1eUJOk5ILk4a9QF77yNcA1RYqFNsdc8 PDt+PQ7Ms/B9rdfoWdoqRzD6G6WfYhqrn0N7fQCqrnk+GQo4HzZjbQsPVTWzw1EQs6/eb+p 3aSHgrRpb8Z2pp6QXMk6+5C+CxJoJ1L6sNPHOcN3Ycgyk+6tAvrmIQ4tLsujmtLcss38RlU sqqZCsSFh8cciC38h557Q== UI-OutboundReport: notjunk:1;M01:P0:ms2Gc0sDEoA=;lgzm1hrbRIA0pCFVeUedHGG0Sjt QFTtAvsj2GVUSAXBnD7fVbNin2XTPEEgR74CITUDb2D8FhvuKPGFn+RcjfseBiS1AuHwkDTlt 2EP6dzzYhkrRVlOJMMcP3u3Jp/xnWQDtsIYas/Z+l+vYrdRTfcdXQE31+iREpf9HCxXO0x9T9 hHXfh3UUlOv9y07JLUXFWa31QCFUia0Wo/VqAXHx0dgvUJk35AOj05fWXdp8wsWW5uf/Pi0DD kmKk3UQJ1TRO84RoJ/Tj5IVnoxHQbX8bF1RjIeM6BH9NFaB0KZ3NcwzuStpvb5B4Q8BwAgEY9 red2+VRKcIDKfKPBVvfM/b3NUxzOOJ0ezFmzVbuzkjGcMYh7iwo7lEj0hL5uBZEuoMDGgAjKb 06bCaX842QzerW3GyUhRjSAHMzQmdhy6vqzRGbsA0IG46xcmMbiD2dJMEIkjtBZoGwzlFhF8t aGLUyZnb043fj/g0qrPVsvrkzs8YtKV6AdTqylc/gVKPEYsrNFozYsLXKb51E+1kbDhLntUHO WSOmPKdK39Ydiw+i1UqYcOksGfO701KRab/RwXHI0XHSWQnOJgACvRDvzavk8PP0IuUwCj7wU 6XAAwudS7vc1g6cAl7960gaSTSdkBSTBbJ3d/GvkRKiURa1lRjU3TedgTSq/g9Pc3mVbMXQYG pJ6lLaD7n0kSLk5gYZSufqYJkJHkl6ggl0xSlSj0XF1MIiLOfj1zS3Bl1yn5yq45l1J2ql5+k 2pFP7sF6lmjs1aodgKypcMrDqSRukQhMuEdMW4chDOIXewZmKrswiVtwPTUz6Bnvikxoo9Fxy Byf9S9Y+DZub7c16qsvf2owTETwVqN5eMRtJRQadRDCKaGkMW4xv28UGG6MCNYIq0wowUDuCZ 2DLsp6VEG+zXdFfZKKWqkPbTlwksG0hpqeLx1cjBpgQ26MX2JG4CUxvuKQ2qoaYBU2cUfSAgZ Gr/wgLovn8sJzCmFZnb/gFA772w= Received-SPF: pass client-ip=212.227.15.4; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela --- migration/multifd.c | 11 +++++------ migration/multifd.h | 2 ++ 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index fb5e8859de..fddbf86596 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -284,7 +284,6 @@ static void multifd_send_fill_packet(MultiFDSendParams *p) static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) { MultiFDPacket_t *packet = p->packet; - RAMBlock *block; int i; packet->magic = be32_to_cpu(packet->magic); @@ -334,21 +333,21 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) /* make sure that ramblock is 0 terminated */ packet->ramblock[255] = 0; - block = qemu_ram_block_by_name(packet->ramblock); - if (!block) { + p->block = qemu_ram_block_by_name(packet->ramblock); + if (!p->block) { error_setg(errp, "multifd: unknown ram block %s", packet->ramblock); return -1; } - p->host = block->host; + p->host = p->block->host; for (i = 0; i < p->normal_num; i++) { uint64_t offset = be64_to_cpu(packet->offset[i]); - if (offset > (block->used_length - p->page_size)) { + if (offset > (p->block->used_length - p->page_size)) { error_setg(errp, "multifd: offset too long %" PRIu64 " (max " RAM_ADDR_FMT ")", - offset, block->used_length); + offset, p->block->used_length); return -1; } p->normal[i] = offset; diff --git a/migration/multifd.h b/migration/multifd.h index 7cfc265148..a835643b48 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -175,6 +175,8 @@ typedef struct { uint32_t next_packet_size; /* packets sent through this channel */ uint64_t num_packets; + /* ramblock */ + RAMBlock *block; /* ramblock host address */ uint8_t *host; /* non zero pages recv through this channel */ From patchwork Mon May 8 19:11:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 13234841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BFB55C77B75 for ; Mon, 8 May 2023 19:13:13 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pw6GZ-0007Ia-CN; Mon, 08 May 2023 15:11:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pw6GX-0007I3-BI for qemu-devel@nongnu.org; Mon, 08 May 2023 15:11:21 -0400 Received: from mout.web.de ([212.227.15.14]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pw6GS-00045s-Ju for qemu-devel@nongnu.org; Mon, 08 May 2023 15:11:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=s29768273; t=1683573072; i=lukasstraub2@web.de; bh=A+4cqBKUkLELvquPN55Hz1t4Aq7uehn7miNGhcLw8AM=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=JaagfLx+dhi3P4EFrTzTCzZ/3GP02J1ywngevFzIK/aPZyyQ+p1Zx371KGZyOVKcq SbCfPUNwXX+poz0VgMW9Y3EqD1qlrPoEFLrZvhQvaFyQkTgIsPzLQXo2AS6HOFUbyC NZ4xhBQil7lGPXFkyotVsp5PDSM+Jk4DMAiLA5Pnl9Z+vArzOyfC4bJCL7Z9r+EwF6 /79ou6SPPOp+QeRJXAY/vz434K2sS5k2crGAdxPCNP9+bGQW6yvc18ceuheAdemvvF 1JNHgLmBcEYFQcDkDhcOT1gQP7RHi9YlTd9mrPmxOJn9eGocK1JlhuLl+XHPFQUG3g xJdeRrkyEK0og== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from gecko.fritz.box ([82.207.254.98]) by smtp.web.de (mrweb006 [213.165.67.108]) with ESMTPSA (Nemesis) id 1MlsKF-1qeXts3Toh-00ihe4; Mon, 08 May 2023 21:11:11 +0200 Date: Mon, 8 May 2023 21:11:10 +0200 From: Lukas Straub To: qemu-devel Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v2 6/6] multifd: Add colo support Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Provags-ID: V03:K1:Gr0zXIk2hmORKG6JrlenkUOIlyHVLlag+e350nwznqhB3Klxxz9 9Xv7v0ONf+pDH5xd8HoMUbsDAbR2kDmZwTjSIvBHofXxJMGauEhyIDUvQz0qV7KH4Zl4pvp dl574WeOtxmnBcHWuhbA9/tbYfkPudWB7h5/AlJNnVpVAcyf69MohWsx0NEbtnD9a8jRVBk U6qpJLtWauX087FJyyp+A== UI-OutboundReport: notjunk:1;M01:P0:MB0m002iubo=;qWENqjIyDLqaZKYrRoqqtWau0NQ kB/pVa4dpo7Iluk9z4mbuanCmekQMn28tbDixmIncVHYGH7ZAYwMXWm3kF8MlL6hFfqHPtXFR wCO7QAzxTI1z/tZ8mmmTUvwXHEZzABynZO2Jj91G2Zueb4gA829etxwr6B54HuZ24k0J8kEuh 5P9tOwMV9EaqZLOr8VLQ7fpHG2JNlOkyJIKPaTjUogVDVsFCgnKOGZtUdGda0F3WziEuLH0aF nJJ+m00Xpepor9Nx4qhTthawbLpLyyMYxhYVA58Jq6iaAnHRPi+NtObNC3JZjY2G7d6LPribp y3VzU9dsZnXph0ggmRQEW3YR3KCcKgixg7IKE5qknEyzgElosAKa8t0r1j6xqaMCxYnEEGE8/ bT184yW0b+8qfWAELT3/F+87iBefaY0TSlbcWfO2Iz2A9KYQe4aFJrRqQQEOE2j1yb/VP6lpB nLN955wO/L3Q8oNvUB8SkmtfWEW5NATbG0dmCiSJhaJ/E+V6GVRYfe8SV3JeFqLKWfFry8uiX dANs3P1jbIBSsEsvd/0E7aozxoWCP3mdgs/bBPWAYUDwybErXlS0H461UhzJs0xOBzFoEjqMK DxbNZpwQ/iRKrAMF6KWKuTabRJLFVxdPN5X87tXEobltQvHY2Xgxt4BxvDU64p17P5athBSKm GJANpI/uYtHQ3+3XAmwPA3xRtL7sLgtcKDFIAH62s3ueKNeJBzXBTQpbqyiKRHHVYvH5yD21K f5OGDwPGAP8LF2KbwT1BI/iPdyXG8G536Eu7LlRHiD+S0maOA9Dtsa3ZTLTZWmEaDB5L6WJ2U 96tngDwqlK7RFrNxaQVSq8TPFrwMU8NDSCBY3eJAoYB4KW6AR//Ow3ZWxlrjJPnklk2bA8Qvn DJshN6j9FKNgG19ks+TR7enq1WOr4Pm5EvTwgWi/4cTBZiICENJftMugmOaxrZrLmborgQ//C EEWLiJRJv5n+KxPe7TTjK3u9BKY= Received-SPF: pass client-ip=212.227.15.14; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Like in the normal ram_load() path, put the received pages into the colo cache and mark the pages in the bitmap so that they will be flushed to the guest later. Signed-off-by: Lukas Straub --- migration/multifd-colo.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/migration/multifd-colo.c b/migration/multifd-colo.c index c035d15e87..305a1b7000 100644 --- a/migration/multifd-colo.c +++ b/migration/multifd-colo.c @@ -15,13 +15,41 @@ #include "ram.h" #include "multifd.h" #include "io/channel-socket.h" +#include "migration/colo.h" #define MULTIFD_INTERNAL #include "multifd-internal.h" static int multifd_colo_recv_pages(MultiFDRecvParams *p, Error **errp) { - return multifd_recv_state->ops->recv_pages(p, errp); + int ret = 0; + + /* + * While we're still in precopy mode, we copy received pages to both guest + * and cache. No need to set dirty bits, since guest and cache memory are + * in sync. + */ + if (migration_incoming_in_colo_state()) { + colo_record_bitmap(p->block, p->normal, p->normal_num); + } + + p->host = p->block->colo_cache; + ret = multifd_recv_state->ops->recv_pages(p, errp); + if (ret != 0) { + p->host = p->block->host; + return ret; + } + + if (!migration_incoming_in_colo_state()) { + for (int i = 0; i < p->normal_num; i++) { + void *guest = p->block->host + p->normal[i]; + void *cache = p->host + p->normal[i]; + memcpy(guest, cache, p->page_size); + } + } + + p->host = p->block->host; + return ret; } int multifd_colo_load_setup(Error **errp)