From patchwork Fri Nov 1 13:47:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859424 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8DE0AE6B27E for ; Fri, 1 Nov 2024 13:49:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s0Y-0003yz-9K; Fri, 01 Nov 2024 09:48:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0U-0003yN-9c for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:07 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0S-0002C5-65 for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:05 -0400 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1Dg9nW026799; Fri, 1 Nov 2024 13:47:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=bcZigjOuSf9Js/+5i9Gmss9ynPMJJKwYfOsefq2wnpM=; b= Y0htrr88z4MB6rSiISyVFWP8W6sivQVB2QHJjjRiiMmrI7OfMe8wuliUeY9KuNR7 IQ8oWUbeP8p7rg5jYR/2TrjjKzkL1FI8WrlvVoq/oNaNxId8dAdIVUXrsUqA1bnR qCsX+zly1QsxJkqn0lMiqSXtVlWWFdAVl4GzDkO1YBh0R12gtKYZAAtkgoOaTRYo zHP+SaWqbiywSgE9upLr4MWpmcjTEv3xKZ5CXBnBIwPW2NBuJUF5qebknlVoztO/ e5BHgqNwSwyV7APf3xqvGHdS4xYRr1vCfzeBM+hX4obR6UUR6CJvozq/FBARfngs 8r4+nZgydjnTkmIgpzJ7tQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42gryscbay-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:47:58 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1DBssZ010281; Fri, 1 Nov 2024 13:47:57 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91pt57-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:47:57 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuO2006031; Fri, 1 Nov 2024 13:47:57 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-2; Fri, 01 Nov 2024 13:47:57 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 01/16] machine: anon-alloc option Date: Fri, 1 Nov 2024 06:47:40 -0700 Message-Id: <1730468875-249970-2-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010099 X-Proofpoint-ORIG-GUID: UZY9sQXPLefqlfHams_hHcx5MphhF1cn X-Proofpoint-GUID: UZY9sQXPLefqlfHams_hHcx5MphhF1cn Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Allocate anonymous memory using mmap MAP_ANON or memfd_create depending on the value of the anon-alloc machine property. This option applies to memory allocated as a side effect of creating various devices. It does not apply to memory-backend-objects, whether explicitly specified on the command line, or implicitly created by the -m command line option. The memfd option is intended to support new migration modes, in which the memory region can be transferred in place to a new QEMU process, by sending the memfd file descriptor to the process. Memory contents are preserved, and if the mode also transfers device descriptors, then pages that are locked in memory for DMA remain locked. This behavior is a pre-requisite for supporting vfio, vdpa, and iommufd devices with the new modes. Signed-off-by: Steve Sistare Signed-off-by: David Hildenbrand Signed-off-by: David Hildenbrand --- hw/core/machine.c | 19 +++++++++++++++++++ include/hw/boards.h | 1 + qapi/machine.json | 14 ++++++++++++++ qemu-options.hx | 11 +++++++++++ system/physmem.c | 35 +++++++++++++++++++++++++++++++++++ system/trace-events | 3 +++ 6 files changed, 83 insertions(+) diff --git a/hw/core/machine.c b/hw/core/machine.c index adaba17..a89a32b 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -460,6 +460,20 @@ static void machine_set_mem_merge(Object *obj, bool value, Error **errp) ms->mem_merge = value; } +static int machine_get_anon_alloc(Object *obj, Error **errp) +{ + MachineState *ms = MACHINE(obj); + + return ms->anon_alloc; +} + +static void machine_set_anon_alloc(Object *obj, int value, Error **errp) +{ + MachineState *ms = MACHINE(obj); + + ms->anon_alloc = value; +} + static bool machine_get_usb(Object *obj, Error **errp) { MachineState *ms = MACHINE(obj); @@ -1078,6 +1092,11 @@ static void machine_class_init(ObjectClass *oc, void *data) object_class_property_set_description(oc, "mem-merge", "Enable/disable memory merge support"); + object_class_property_add_enum(oc, "anon-alloc", "AnonAllocOption", + &AnonAllocOption_lookup, + machine_get_anon_alloc, + machine_set_anon_alloc); + object_class_property_add_bool(oc, "usb", machine_get_usb, machine_set_usb); object_class_property_set_description(oc, "usb", diff --git a/include/hw/boards.h b/include/hw/boards.h index 5966069..5a87647 100644 --- a/include/hw/boards.h +++ b/include/hw/boards.h @@ -393,6 +393,7 @@ struct MachineState { bool enable_graphics; ConfidentialGuestSupport *cgs; HostMemoryBackend *memdev; + AnonAllocOption anon_alloc; /* * convenience alias to ram_memdev_id backend memory region * or to numa container memory region diff --git a/qapi/machine.json b/qapi/machine.json index 3cc055b..f634c40 100644 --- a/qapi/machine.json +++ b/qapi/machine.json @@ -1898,3 +1898,17 @@ { 'command': 'x-query-interrupt-controllers', 'returns': 'HumanReadableText', 'features': [ 'unstable' ]} + +## +# @AnonAllocOption: +# +# An enumeration of the options for allocating anonymous guest memory. +# +# @mmap: allocate using mmap MAP_ANON +# +# @memfd: allocate using memfd_create +# +# Since: 9.2 +## +{ 'enum': 'AnonAllocOption', + 'data': [ 'mmap', 'memfd' ] } diff --git a/qemu-options.hx b/qemu-options.hx index dacc979..fdd6bf2 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -38,6 +38,7 @@ DEF("machine", HAS_ARG, QEMU_OPTION_machine, \ " nvdimm=on|off controls NVDIMM support (default=off)\n" " memory-encryption=@var{} memory encryption object to use (default=none)\n" " hmat=on|off controls ACPI HMAT support (default=off)\n" + " anon-alloc=mmap|memfd allocate anonymous guest RAM using mmap MAP_ANON or memfd_create (default: mmap)\n" " memory-backend='backend-id' specifies explicitly provided backend for main RAM (default=none)\n" " cxl-fmw.0.targets.0=firsttarget,cxl-fmw.0.targets.1=secondtarget,cxl-fmw.0.size=size[,cxl-fmw.0.interleave-granularity=granularity]\n", QEMU_ARCH_ALL) @@ -101,6 +102,16 @@ SRST Enables or disables ACPI Heterogeneous Memory Attribute Table (HMAT) support. The default is off. + ``anon-alloc=mmap|memfd`` + Allocate anonymous guest RAM using mmap MAP_ANON (the default) + or memfd_create. This option applies to memory allocated as a + side effect of creating various devices. It does not apply to + memory-backend-objects, whether explicitly specified on the + command line, or implicitly created by the -m command line + option. + + Some migration modes require anon-alloc=memfd. + ``memory-backend='id'`` An alternative to legacy ``-mem-path`` and ``mem-prealloc`` options. Allows to use a memory backend as main RAM. diff --git a/system/physmem.c b/system/physmem.c index dc1db3a..174f7e0 100644 --- a/system/physmem.c +++ b/system/physmem.c @@ -47,6 +47,7 @@ #include "qemu/qemu-print.h" #include "qemu/log.h" #include "qemu/memalign.h" +#include "qemu/memfd.h" #include "exec/memory.h" #include "exec/ioport.h" #include "sysemu/dma.h" @@ -69,6 +70,8 @@ #include "qemu/pmem.h" +#include "qapi/qapi-types-migration.h" +#include "migration/options.h" #include "migration/vmstate.h" #include "qemu/range.h" @@ -1849,6 +1852,35 @@ static void ram_block_add(RAMBlock *new_block, Error **errp) qemu_mutex_unlock_ramlist(); return; } + + } else if (current_machine->anon_alloc == ANON_ALLOC_OPTION_MEMFD && + !object_dynamic_cast(new_block->mr->parent_obj.parent, + TYPE_MEMORY_BACKEND)) { + size_t max_length = new_block->max_length; + MemoryRegion *mr = new_block->mr; + const char *name = memory_region_name(mr); + + new_block->mr->align = QEMU_VMALLOC_ALIGN; + new_block->flags |= RAM_SHARED; + + if (new_block->fd == -1) { + new_block->fd = qemu_memfd_create(name, max_length + mr->align, + 0, 0, 0, errp); + } + + if (new_block->fd >= 0) { + int mfd = new_block->fd; + qemu_set_cloexec(mfd); + new_block->host = file_ram_alloc(new_block, max_length, mfd, + false, 0, errp); + } + if (!new_block->host) { + qemu_mutex_unlock_ramlist(); + return; + } + memory_try_enable_merging(new_block->host, new_block->max_length); + free_on_error = true; + } else { new_block->host = qemu_anon_ram_alloc(new_block->max_length, &new_block->mr->align, @@ -1932,6 +1964,9 @@ static void ram_block_add(RAMBlock *new_block, Error **errp) ram_block_notify_add(new_block->host, new_block->used_length, new_block->max_length); } + trace_ram_block_add(memory_region_name(new_block->mr), new_block->flags, + new_block->fd, new_block->used_length, + new_block->max_length); return; out_free: diff --git a/system/trace-events b/system/trace-events index 074d001..267daca 100644 --- a/system/trace-events +++ b/system/trace-events @@ -47,3 +47,6 @@ dirtylimit_vcpu_execute(int cpu_index, int64_t sleep_time_us) "CPU[%d] sleep %"P # cpu-throttle.c cpu_throttle_set(int new_throttle_pct) "set guest CPU throttled by %d%%" + +#physmem.c +ram_block_add(const char *name, uint32_t flags, int fd, size_t used_length, size_t max_length) "%s, flags %u, fd %d, len %zu, maxlen %zu" From patchwork Fri Nov 1 13:47:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859422 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 241A8E6B278 for ; Fri, 1 Nov 2024 13:49:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s0a-0003zW-8J; Fri, 01 Nov 2024 09:48:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0X-0003z4-6z for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:09 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0U-0002C9-Hp for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:08 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1Dfiar023759; Fri, 1 Nov 2024 13:47:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=c9uS44G7ynQoLuJTNrIMsiM9YHe2zbMmlbNR2XgY2tM=; b= mJSANESEtPrWqq6KV/t66VRdPKtD2tzVufl/QmDHrFBAqeOycgLUyaj2feM/b9yQ jtca5wdabWBT2qWk77FB1Fu8DGcttP1Jm58ZH+g7w41o5Y4b1fpmOgzj9lIp/xzK 5mKmAWs14vWcIPkuNQl6g237BKqhAGUVUnhHBWlpzwTJ81o4nILFKqox+q6cWTcj vFWzaUK39OZQkxTJsOzhfFhUClEdqVpDgfuBFFBMc5yaZNGFp4rH1+Bi2m6nIOLf /rHhiBoSebGEzFlv68rBY5jZV60Efzu/qhxZyJ8Du+Bhfc4quWw+Sx3Ek7jKYBLc m/A6Px7sMZFo/ReO+TDOug== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grc94a24-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:47:59 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1CloLb010012; Fri, 1 Nov 2024 13:47:58 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91pt60-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:47:58 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuO4006031; Fri, 1 Nov 2024 13:47:58 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-3; Fri, 01 Nov 2024 13:47:58 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 02/16] migration: cpr-state Date: Fri, 1 Nov 2024 06:47:41 -0700 Message-Id: <1730468875-249970-3-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010099 X-Proofpoint-GUID: 6XpCEVmmDWzxV38Pzd2uhZQa1NahEpGy X-Proofpoint-ORIG-GUID: 6XpCEVmmDWzxV38Pzd2uhZQa1NahEpGy Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org CPR must save state that is needed after QEMU is restarted, when devices are realized. Thus the extra state cannot be saved in the migration stream, as objects must already exist before that stream can be loaded. Instead, define auxilliary state structures and vmstate descriptions, not associated with any registered object, and serialize the aux state to a cpr-specific stream in cpr_state_save. Deserialize in cpr_state_load after QEMU restarts, before devices are realized. Provide accessors for clients to register file descriptors for saving. The mechanism for passing the fd's to the new process will be specific to each migration mode, and added in subsequent patches. Signed-off-by: Steve Sistare Reviewed-by: Fabiano Rosas Reviewed-by: Peter Xu --- include/migration/cpr.h | 23 ++++++ migration/cpr.c | 192 ++++++++++++++++++++++++++++++++++++++++++++++++ migration/meson.build | 1 + migration/migration.c | 1 + migration/trace-events | 5 ++ 5 files changed, 222 insertions(+) create mode 100644 include/migration/cpr.h create mode 100644 migration/cpr.c diff --git a/include/migration/cpr.h b/include/migration/cpr.h new file mode 100644 index 0000000..6e4781c --- /dev/null +++ b/include/migration/cpr.h @@ -0,0 +1,23 @@ +/* + * Copyright (c) 2021, 2024 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef MIGRATION_CPR_H +#define MIGRATION_CPR_H + +#define QEMU_CPR_FILE_MAGIC 0x51435052 +#define QEMU_CPR_FILE_VERSION 0x00000001 + +void cpr_save_fd(const char *name, int id, int fd); +void cpr_delete_fd(const char *name, int id); +int cpr_find_fd(const char *name, int id); + +int cpr_state_save(Error **errp); +int cpr_state_load(Error **errp); +void cpr_state_close(void); +struct QIOChannel *cpr_state_ioc(void); + +#endif diff --git a/migration/cpr.c b/migration/cpr.c new file mode 100644 index 0000000..be1dc92 --- /dev/null +++ b/migration/cpr.c @@ -0,0 +1,192 @@ +/* + * Copyright (c) 2021-2024 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "migration/cpr.h" +#include "migration/misc.h" +#include "migration/options.h" +#include "migration/qemu-file.h" +#include "migration/savevm.h" +#include "migration/vmstate.h" +#include "sysemu/runstate.h" +#include "trace.h" + +/*************************************************************************/ +/* cpr state container for all information to be saved. */ + +typedef QLIST_HEAD(CprFdList, CprFd) CprFdList; + +typedef struct CprState { + CprFdList fds; +} CprState; + +static CprState cpr_state; + +/****************************************************************************/ + +typedef struct CprFd { + char *name; + unsigned int namelen; + int id; + int fd; + QLIST_ENTRY(CprFd) next; +} CprFd; + +static const VMStateDescription vmstate_cpr_fd = { + .name = "cpr fd", + .version_id = 1, + .minimum_version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_UINT32(namelen, CprFd), + VMSTATE_VBUFFER_ALLOC_UINT32(name, CprFd, 0, NULL, namelen), + VMSTATE_INT32(id, CprFd), + VMSTATE_INT32(fd, CprFd), + VMSTATE_END_OF_LIST() + } +}; + +void cpr_save_fd(const char *name, int id, int fd) +{ + CprFd *elem = g_new0(CprFd, 1); + + trace_cpr_save_fd(name, id, fd); + elem->name = g_strdup(name); + elem->namelen = strlen(name) + 1; + elem->id = id; + elem->fd = fd; + QLIST_INSERT_HEAD(&cpr_state.fds, elem, next); +} + +static CprFd *find_fd(CprFdList *head, const char *name, int id) +{ + CprFd *elem; + + QLIST_FOREACH(elem, head, next) { + if (!strcmp(elem->name, name) && elem->id == id) { + return elem; + } + } + return NULL; +} + +void cpr_delete_fd(const char *name, int id) +{ + CprFd *elem = find_fd(&cpr_state.fds, name, id); + + if (elem) { + QLIST_REMOVE(elem, next); + g_free(elem->name); + g_free(elem); + } + + trace_cpr_delete_fd(name, id); +} + +int cpr_find_fd(const char *name, int id) +{ + CprFd *elem = find_fd(&cpr_state.fds, name, id); + int fd = elem ? elem->fd : -1; + + trace_cpr_find_fd(name, id, fd); + return fd; +} +/*************************************************************************/ +#define CPR_STATE "CprState" + +static const VMStateDescription vmstate_cpr_state = { + .name = CPR_STATE, + .version_id = 1, + .minimum_version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_QLIST_V(fds, CprState, 1, vmstate_cpr_fd, CprFd, next), + VMSTATE_END_OF_LIST() + } +}; +/*************************************************************************/ + +static QEMUFile *cpr_state_file; + +QIOChannel *cpr_state_ioc(void) +{ + return qemu_file_get_ioc(cpr_state_file); +} + +int cpr_state_save(Error **errp) +{ + int ret; + QEMUFile *f; + + /* set f based on mode in a later patch in this series */ + return 0; + + qemu_put_be32(f, QEMU_CPR_FILE_MAGIC); + qemu_put_be32(f, QEMU_CPR_FILE_VERSION); + + ret = vmstate_save_state(f, &vmstate_cpr_state, &cpr_state, 0); + if (ret) { + error_setg(errp, "vmstate_save_state error %d", ret); + qemu_fclose(f); + return ret; + } + + /* + * Close the socket only partially so we can later detect when the other + * end closes by getting a HUP event. + */ + qemu_fflush(f); + qio_channel_shutdown(qemu_file_get_ioc(f), QIO_CHANNEL_SHUTDOWN_WRITE, + NULL); + cpr_state_file = f; + return 0; +} + +int cpr_state_load(Error **errp) +{ + int ret; + uint32_t v; + QEMUFile *f; + + /* set f based on other parameters in a later patch in this series */ + return 0; + + v = qemu_get_be32(f); + if (v != QEMU_CPR_FILE_MAGIC) { + error_setg(errp, "Not a migration stream (bad magic %x)", v); + qemu_fclose(f); + return -EINVAL; + } + v = qemu_get_be32(f); + if (v != QEMU_CPR_FILE_VERSION) { + error_setg(errp, "Unsupported migration stream version %d", v); + qemu_fclose(f); + return -ENOTSUP; + } + + ret = vmstate_load_state(f, &vmstate_cpr_state, &cpr_state, 1); + if (ret) { + error_setg(errp, "vmstate_load_state error %d", ret); + qemu_fclose(f); + return ret; + } + + /* + * Let the caller decide when to close the socket (and generate a HUP event + * for the sending side). + */ + cpr_state_file = f; + + return ret; +} + +void cpr_state_close(void) +{ + if (cpr_state_file) { + qemu_fclose(cpr_state_file); + cpr_state_file = NULL; + } +} diff --git a/migration/meson.build b/migration/meson.build index 66d3de8..e5f4211 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -13,6 +13,7 @@ system_ss.add(files( 'block-dirty-bitmap.c', 'channel.c', 'channel-block.c', + 'cpr.c', 'dirtyrate.c', 'exec.c', 'fd.c', diff --git a/migration/migration.c b/migration/migration.c index 021faee..6dc7c09 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -27,6 +27,7 @@ #include "sysemu/cpu-throttle.h" #include "rdma.h" #include "ram.h" +#include "migration/cpr.h" #include "migration/global_state.h" #include "migration/misc.h" #include "migration.h" diff --git a/migration/trace-events b/migration/trace-events index c65902f..5356fb5 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -341,6 +341,11 @@ colo_receive_message(const char *msg) "Receive '%s' message" # colo-failover.c colo_failover_set_state(const char *new_state) "new state %s" +# cpr.c +cpr_save_fd(const char *name, int id, int fd) "%s, id %d, fd %d" +cpr_delete_fd(const char *name, int id) "%s, id %d" +cpr_find_fd(const char *name, int id, int fd) "%s, id %d returns %d" + # block-dirty-bitmap.c send_bitmap_header_enter(void) "" send_bitmap_bits(uint32_t flags, uint64_t start_sector, uint32_t nr_sectors, uint64_t data_size) "flags: 0x%x, start_sector: %" PRIu64 ", nr_sectors: %" PRIu32 ", data_size: %" PRIu64 From patchwork Fri Nov 1 13:47:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2BCEE6B27C for ; Fri, 1 Nov 2024 13:50:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s0x-00047i-Cy; Fri, 01 Nov 2024 09:48:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0v-00041J-9Q for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:33 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0t-0002GS-KF for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:33 -0400 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1Dfe4L029184; Fri, 1 Nov 2024 13:48:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=mFMfP6A/2uG91S8VBhFG+/Sw+haZOLY6FbviW3+2dlA=; b= Xkjq477PkrTMqoIGhfTu1iNJuSs1WK9DmXPg3p6brdPJRLtNGpuu5unztgloDxUS ayrAyLYyS7h6ChebNVmYqnsFuJGRqbkxRjqNANvrFkqTFRU8xhBL3Hd6zIc6jxv7 4zA8prOJO0GE1uPxdXjEPJKblPCXk/Jc4LAbqnp07FrOhJrIuImeFYzfkKUf9u3C EteREzTQPEx9De5GYCULsKkhF1twtM4m3P+doXDTPt/kT+akjvFan18LwH2Wt0Z7 r7SS7Kpxsf/8qyXYG4pApYforqoFLB9YfextbNDGG2mPJL5lbCNbfJKprjRQ8U2M 6yMJMFPYccgVsTmg3W5iUA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grdxv9w8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:25 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1Cc7Pq009727; Fri, 1 Nov 2024 13:48:24 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91pt6s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:24 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuO6006031; Fri, 1 Nov 2024 13:47:58 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-4; Fri, 01 Nov 2024 13:47:58 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 03/16] physmem: preserve ram blocks for cpr Date: Fri, 1 Nov 2024 06:47:42 -0700 Message-Id: <1730468875-249970-4-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-GUID: aLkqzbqR9b80TzF8Loqc2HyedvLR15Hh X-Proofpoint-ORIG-GUID: aLkqzbqR9b80TzF8Loqc2HyedvLR15Hh Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Save the memfd for anonymous ramblocks in CPR state, along with a name that uniquely identifies it. The block's idstr is not yet set, so it cannot be used for this purpose. Find the saved memfd in new QEMU when creating a block. QEMU hard-codes the length of some internally-created blocks, so to guard against that length changing, use lseek to get the actual length of an incoming memfd. Signed-off-by: Steve Sistare --- system/physmem.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/system/physmem.c b/system/physmem.c index 174f7e0..cd468eb 100644 --- a/system/physmem.c +++ b/system/physmem.c @@ -72,6 +72,7 @@ #include "qapi/qapi-types-migration.h" #include "migration/options.h" +#include "migration/cpr.h" #include "migration/vmstate.h" #include "qemu/range.h" @@ -1663,6 +1664,19 @@ void qemu_ram_unset_idstr(RAMBlock *block) } } +static char *cpr_name(RAMBlock *block) +{ + MemoryRegion *mr = block->mr; + const char *mr_name = memory_region_name(mr); + g_autofree char *id = mr->dev ? qdev_get_dev_path(mr->dev) : NULL; + + if (id) { + return g_strdup_printf("%s/%s", id, mr_name); + } else { + return g_strdup(mr_name); + } +} + size_t qemu_ram_pagesize(RAMBlock *rb) { return rb->page_size; @@ -1858,14 +1872,23 @@ static void ram_block_add(RAMBlock *new_block, Error **errp) TYPE_MEMORY_BACKEND)) { size_t max_length = new_block->max_length; MemoryRegion *mr = new_block->mr; - const char *name = memory_region_name(mr); + g_autofree char *name = cpr_name(new_block); new_block->mr->align = QEMU_VMALLOC_ALIGN; new_block->flags |= RAM_SHARED; + new_block->fd = cpr_find_fd(name, 0); if (new_block->fd == -1) { new_block->fd = qemu_memfd_create(name, max_length + mr->align, 0, 0, 0, errp); + cpr_save_fd(name, 0, new_block->fd); + } else if (lseek(new_block->fd, 0, SEEK_END) < max_length && + ftruncate(new_block->fd, max_length)) { + error_setg_errno(errp, errno, + "cannot grow ram block %s fd %d to %ld bytes", + name, new_block->fd, max_length); + qemu_mutex_unlock_ramlist(); + return; } if (new_block->fd >= 0) { @@ -1875,6 +1898,7 @@ static void ram_block_add(RAMBlock *new_block, Error **errp) false, 0, errp); } if (!new_block->host) { + cpr_delete_fd(name, 0); qemu_mutex_unlock_ramlist(); return; } @@ -2182,6 +2206,8 @@ static void reclaim_ramblock(RAMBlock *block) void qemu_ram_free(RAMBlock *block) { + g_autofree char *name = NULL; + if (!block) { return; } @@ -2192,6 +2218,8 @@ void qemu_ram_free(RAMBlock *block) } qemu_mutex_lock_ramlist(); + name = cpr_name(block); + cpr_delete_fd(name, 0); QLIST_REMOVE_RCU(block, next); ram_list.mru_block = NULL; /* Write list before version */ From patchwork Fri Nov 1 13:47:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BD5ADE6B278 for ; Fri, 1 Nov 2024 13:49:40 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s0y-0004Ba-Kz; Fri, 01 Nov 2024 09:48:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0v-00041T-Nf for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:33 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0u-0002Gg-6z for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:33 -0400 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1Dfdxr009296; Fri, 1 Nov 2024 13:48:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=0VPeUxWKuni5VVu9Z1JrKdeDCKAw5m+u4mI+NxHmpNE=; b= PJj3ofmIR7YeedMjeMSrZ1ZzkrNZm5kNsV0L+VX7hpl6NHtcx58d6nzWAAHTfEoi 8X08gf0tQPLO2oorExEVugm1flHARIILQPTrrVeJXCX2p53sQnPcFYMyoSrSE8w8 UbEhdXS5Q6j/jJfHzHAGGdo7ShyNNC52IZExSMxL32YXFQbxi87Ae8rCA/iSjbE3 KfKgvjPj6nztocGM40h3KOKAeDzwCGEKs+EQ902i+waYGbowoI3oYFR5V//KOrZy m2/ALW4Zgjn1nd15rpTc4x/hC8ilPwfJWKPvZj9MAC/MFSsUB2ASKdVdMPCNFVJq 4vwPRHxQVulTP22eWlwI1A== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grdpca0s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:30 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1BbZG0010235; Fri, 1 Nov 2024 13:48:28 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptjk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:28 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuO8006031; Fri, 1 Nov 2024 13:48:28 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-5; Fri, 01 Nov 2024 13:48:27 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 04/16] hostmem-memfd: preserve for cpr Date: Fri, 1 Nov 2024 06:47:43 -0700 Message-Id: <1730468875-249970-5-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-ORIG-GUID: riiM93bJYzu3gtnM-I-zuABC1TXc_niM X-Proofpoint-GUID: riiM93bJYzu3gtnM-I-zuABC1TXc_niM Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Preserve memory-backend-memfd memory objects during cpr-transfer. Signed-off-by: Steve Sistare Acked-by: Peter Xu --- backends/hostmem-memfd.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/backends/hostmem-memfd.c b/backends/hostmem-memfd.c index 6a3c89a..2740222 100644 --- a/backends/hostmem-memfd.c +++ b/backends/hostmem-memfd.c @@ -17,6 +17,7 @@ #include "qemu/module.h" #include "qapi/error.h" #include "qom/object.h" +#include "migration/cpr.h" #define TYPE_MEMORY_BACKEND_MEMFD "memory-backend-memfd" @@ -35,15 +36,19 @@ static bool memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) { HostMemoryBackendMemfd *m = MEMORY_BACKEND_MEMFD(backend); - g_autofree char *name = NULL; + g_autofree char *name = host_memory_backend_get_name(backend); + int fd = cpr_find_fd(name, 0); uint32_t ram_flags; - int fd; if (!backend->size) { error_setg(errp, "can't create backend with size 0"); return false; } + if (fd >= 0) { + goto have_fd; + } + fd = qemu_memfd_create(TYPE_MEMORY_BACKEND_MEMFD, backend->size, m->hugetlb, m->hugetlbsize, m->seal ? F_SEAL_GROW | F_SEAL_SHRINK | F_SEAL_SEAL : 0, @@ -51,9 +56,10 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) if (fd == -1) { return false; } + cpr_save_fd(name, 0, fd); +have_fd: backend->aligned = true; - name = host_memory_backend_get_name(backend); ram_flags = backend->share ? RAM_SHARED : 0; ram_flags |= backend->reserve ? 0 : RAM_NORESERVE; ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0; From patchwork Fri Nov 1 13:47:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859417 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B15BE6B278 for ; Fri, 1 Nov 2024 13:48:46 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s10-0004EG-3j; Fri, 01 Nov 2024 09:48:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0x-000492-Kp for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:35 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0v-0002Gz-TN for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:35 -0400 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1DfegV010310; Fri, 1 Nov 2024 13:48:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=jPMk7MCI/w6pg3ewTl45pooOG+03vx9ydl0E4fEeRBA=; b= UJ5dUk0LcE0yY1X/4fCUx9Cc4wYrVY5Ruaa3BZlL6k38RZ3h8VyOddcSaau5QYTu Bmfy2ppll8pVE/EZvWUQGn5ZAWKFbmNtS0lB4Zs2IWgYy/kuOmSf+eASsnaXB/0M zh9hZofNuBwxcpp/KRe8DHsqowTADXE7/bcU0++C4YIsfQx/I3WVG/W1uxxAPrzJ 877kW7+vuvl6o10RraGLoZJcJeNuy7NUS6gzhvQq+rUj9uHslV+k3MERFfpxvHlH s6K/nzZ0xK/W7sexMrHzZJ1YsVkiMlfNTprHSPtGwaG3cYmO3WDVRfz+nRKvA10r oazbXvcb/DkJyY2PX0Mdjg== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grc247gv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:31 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1AnKHP010118; Fri, 1 Nov 2024 13:48:30 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptjv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:30 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuOA006031; Fri, 1 Nov 2024 13:48:29 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-6; Fri, 01 Nov 2024 13:48:29 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 05/16] migration: SCM_RIGHTS for QEMUFile Date: Fri, 1 Nov 2024 06:47:44 -0700 Message-Id: <1730468875-249970-6-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-GUID: 3hL5kCsFG5tfyk3bo-lcgskGwUjyhsRe X-Proofpoint-ORIG-GUID: 3hL5kCsFG5tfyk3bo-lcgskGwUjyhsRe Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Define functions to put/get file descriptors to/from a QEMUFile, for qio channels that support SCM_RIGHTS. Maintain ordering such that put(A), put(fd), put(B) followed by get(A), get(fd), get(B) always succeeds. Other get orderings may succeed but are not guaranteed. Signed-off-by: Steve Sistare Reviewed-by: Peter Xu --- migration/qemu-file.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++--- migration/qemu-file.h | 2 ++ migration/trace-events | 2 ++ 3 files changed, 83 insertions(+), 4 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index b6d2f58..7f951ab 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -37,6 +37,11 @@ #define IO_BUF_SIZE 32768 #define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64) +typedef struct FdEntry { + QTAILQ_ENTRY(FdEntry) entry; + int fd; +} FdEntry; + struct QEMUFile { QIOChannel *ioc; bool is_writable; @@ -51,6 +56,9 @@ struct QEMUFile { int last_error; Error *last_error_obj; + + bool fd_pass; + QTAILQ_HEAD(, FdEntry) fds; }; /* @@ -109,6 +117,8 @@ static QEMUFile *qemu_file_new_impl(QIOChannel *ioc, bool is_writable) object_ref(ioc); f->ioc = ioc; f->is_writable = is_writable; + f->fd_pass = qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_FD_PASS); + QTAILQ_INIT(&f->fds); return f; } @@ -310,6 +320,10 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEMUFile *f) int len; int pending; Error *local_error = NULL; + g_autofree int *fds = NULL; + size_t nfd = 0; + int **pfds = f->fd_pass ? &fds : NULL; + size_t *pnfd = f->fd_pass ? &nfd : NULL; assert(!qemu_file_is_writable(f)); @@ -325,10 +339,9 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEMUFile *f) } do { - len = qio_channel_read(f->ioc, - (char *)f->buf + pending, - IO_BUF_SIZE - pending, - &local_error); + struct iovec iov = { f->buf + pending, IO_BUF_SIZE - pending }; + len = qio_channel_readv_full(f->ioc, &iov, 1, pfds, pnfd, 0, + &local_error); if (len == QIO_CHANNEL_ERR_BLOCK) { if (qemu_in_coroutine()) { qio_channel_yield(f->ioc, G_IO_IN); @@ -348,9 +361,65 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEMUFile *f) qemu_file_set_error_obj(f, len, local_error); } + for (int i = 0; i < nfd; i++) { + FdEntry *fde = g_new0(FdEntry, 1); + fde->fd = fds[i]; + QTAILQ_INSERT_TAIL(&f->fds, fde, entry); + } + return len; } +int qemu_file_put_fd(QEMUFile *f, int fd) +{ + int ret = 0; + QIOChannel *ioc = qemu_file_get_ioc(f); + Error *err = NULL; + struct iovec iov = { (void *)" ", 1 }; + + /* + * Send a dummy byte so qemu_fill_buffer on the receiving side does not + * fail with a len=0 error. Flush first to maintain ordering wrt other + * data. + */ + + qemu_fflush(f); + if (qio_channel_writev_full(ioc, &iov, 1, &fd, 1, 0, &err) < 1) { + error_report_err(error_copy(err)); + qemu_file_set_error_obj(f, -EIO, err); + ret = -1; + } + trace_qemu_file_put_fd(f->ioc->name, fd, ret); + return ret; +} + +int qemu_file_get_fd(QEMUFile *f) +{ + int fd = -1; + FdEntry *fde; + + if (!f->fd_pass) { + Error *err = NULL; + error_setg(&err, "%s does not support fd passing", f->ioc->name); + error_report_err(error_copy(err)); + qemu_file_set_error_obj(f, -EIO, err); + goto out; + } + + /* Force the dummy byte and its fd passenger to appear. */ + qemu_peek_byte(f, 0); + + fde = QTAILQ_FIRST(&f->fds); + if (fde) { + qemu_get_byte(f); /* Drop the dummy byte */ + fd = fde->fd; + QTAILQ_REMOVE(&f->fds, fde, entry); + } +out: + trace_qemu_file_get_fd(f->ioc->name, fd); + return fd; +} + /** Closes the file * * Returns negative error value if any error happened on previous operations or @@ -361,11 +430,17 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEMUFile *f) */ int qemu_fclose(QEMUFile *f) { + FdEntry *fde, *next; int ret = qemu_fflush(f); int ret2 = qio_channel_close(f->ioc, NULL); if (ret >= 0) { ret = ret2; } + QTAILQ_FOREACH_SAFE(fde, &f->fds, entry, next) { + warn_report("qemu_fclose: received fd %d was never claimed", fde->fd); + close(fde->fd); + g_free(fde); + } g_clear_pointer(&f->ioc, object_unref); error_free(f->last_error_obj); g_free(f); diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 11c2120..3e47a20 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -79,5 +79,7 @@ size_t qemu_get_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, off_t pos); QIOChannel *qemu_file_get_ioc(QEMUFile *file); +int qemu_file_put_fd(QEMUFile *f, int fd); +int qemu_file_get_fd(QEMUFile *f); #endif diff --git a/migration/trace-events b/migration/trace-events index 5356fb5..345506b 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -88,6 +88,8 @@ put_qlist_end(const char *field_name, const char *vmsd_name) "%s(%s)" # qemu-file.c qemu_file_fclose(void) "" +qemu_file_put_fd(const char *name, int fd, int ret) "ioc %s, fd %d -> status %d" +qemu_file_get_fd(const char *name, int fd) "ioc %s -> fd %d" # ram.c get_queued_page(const char *block_name, uint64_t tmp_offset, unsigned long page_abs) "%s/0x%" PRIx64 " page_abs=0x%lx" From patchwork Fri Nov 1 13:47:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859418 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4F00CE6B27C for ; Fri, 1 Nov 2024 13:49:11 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s13-0004MC-D7; Fri, 01 Nov 2024 09:48:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s10-0004GU-V3 for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:39 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0z-0002I2-4H for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:38 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1Dffpk023740; Fri, 1 Nov 2024 13:48:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=e74YKrpv2MegEIZU64hNJKOHGS3V5LCX3aEL+so5b6c=; b= myIl4H+H8OVARXIFoy4cS6RGNh5k6gUkPwZXf2m0DqtGdivDgi/c/ID8JOhJtwgf ociAb+HKZD4ia1hAbQJYGkWRpJwbJsrPwNtbfChUiYwjak12N6Udp2kG5BcW6O+1 KJUYw7Un0FgnP8YeKQGglYfjIwF89CuvIZHd4FCk4ATqifLy1BFHC/JsG8nxr4W3 r16haOruWr4/Z6mapZppLMP2EFjkCwwvfnPdRPQOoUGlHeMN7YwSo1gSeyjK+W1p bsdqiLeAj+PtGKR+XyVBCZCyn/DHUg2c/+/ZHfcjBht8CHUdLiiXrl9XJJKzpwYH hkerqHKp9LsZcGaxdwHYVw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grc94a3b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:31 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1CVS7d010127; Fri, 1 Nov 2024 13:48:31 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptk1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:31 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuOC006031; Fri, 1 Nov 2024 13:48:30 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-7; Fri, 01 Nov 2024 13:48:30 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 06/16] migration: VMSTATE_FD Date: Fri, 1 Nov 2024 06:47:45 -0700 Message-Id: <1730468875-249970-7-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-GUID: Wi57TUCZZ-x8LJU9fhNyChjuJ7UlGvC7 X-Proofpoint-ORIG-GUID: Wi57TUCZZ-x8LJU9fhNyChjuJ7UlGvC7 Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Define VMSTATE_FD for declaring a file descriptor field in a VMStateDescription. Signed-off-by: Steve Sistare Reviewed-by: Peter Xu --- include/migration/vmstate.h | 9 +++++++++ migration/vmstate-types.c | 23 +++++++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index f313f2f..a1dfab4 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -230,6 +230,7 @@ extern const VMStateInfo vmstate_info_uint8; extern const VMStateInfo vmstate_info_uint16; extern const VMStateInfo vmstate_info_uint32; extern const VMStateInfo vmstate_info_uint64; +extern const VMStateInfo vmstate_info_fd; /** Put this in the stream when migrating a null pointer.*/ #define VMS_NULLPTR_MARKER (0x30U) /* '0' */ @@ -902,6 +903,9 @@ extern const VMStateInfo vmstate_info_qlist; #define VMSTATE_UINT64_V(_f, _s, _v) \ VMSTATE_SINGLE(_f, _s, _v, vmstate_info_uint64, uint64_t) +#define VMSTATE_FD_V(_f, _s, _v) \ + VMSTATE_SINGLE(_f, _s, _v, vmstate_info_fd, int32_t) + #ifdef CONFIG_LINUX #define VMSTATE_U8_V(_f, _s, _v) \ @@ -936,6 +940,9 @@ extern const VMStateInfo vmstate_info_qlist; #define VMSTATE_UINT64(_f, _s) \ VMSTATE_UINT64_V(_f, _s, 0) +#define VMSTATE_FD(_f, _s) \ + VMSTATE_FD_V(_f, _s, 0) + #ifdef CONFIG_LINUX #define VMSTATE_U8(_f, _s) \ @@ -1009,6 +1016,8 @@ extern const VMStateInfo vmstate_info_qlist; #define VMSTATE_UINT64_TEST(_f, _s, _t) \ VMSTATE_SINGLE_TEST(_f, _s, _t, 0, vmstate_info_uint64, uint64_t) +#define VMSTATE_FD_TEST(_f, _s, _t) \ + VMSTATE_SINGLE_TEST(_f, _s, _t, 0, vmstate_info_fd, int32_t) #define VMSTATE_TIMER_PTR_TEST(_f, _s, _test) \ VMSTATE_POINTER_TEST(_f, _s, _test, vmstate_info_timer, QEMUTimer *) diff --git a/migration/vmstate-types.c b/migration/vmstate-types.c index e83bfcc..f31deb3 100644 --- a/migration/vmstate-types.c +++ b/migration/vmstate-types.c @@ -314,6 +314,29 @@ const VMStateInfo vmstate_info_uint64 = { .put = put_uint64, }; +/* File descriptor communicated via SCM_RIGHTS */ + +static int get_fd(QEMUFile *f, void *pv, size_t size, + const VMStateField *field) +{ + int32_t *v = pv; + *v = qemu_file_get_fd(f); + return 0; +} + +static int put_fd(QEMUFile *f, void *pv, size_t size, + const VMStateField *field, JSONWriter *vmdesc) +{ + int32_t *v = pv; + return qemu_file_put_fd(f, *v); +} + +const VMStateInfo vmstate_info_fd = { + .name = "fd", + .get = get_fd, + .put = put_fd, +}; + static int get_nullptr(QEMUFile *f, void *pv, size_t size, const VMStateField *field) From patchwork Fri Nov 1 13:47:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859420 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84FAAE6B27C for ; Fri, 1 Nov 2024 13:49:20 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s12-0004IQ-BH; Fri, 01 Nov 2024 09:48:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0z-0004CX-6N for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:37 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0w-0002HO-MM for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:36 -0400 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1DfegQ009327; Fri, 1 Nov 2024 13:48:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=Ng077bxtBe+BwuWE8Ytq2oKycG1Y8NibxMVLxV82HMA=; b= nroRRMuLUVQFciWMJGqEfv/uG63pFxfYmNVM4yHADa7btCDN8ize/PsKm3y1mJqA dPPrf3TkR+93xJ8e8IriptsOP9Qu/D0jtxwkNfYjWpZFMcolKJQ2M0Pm3m7Qyfkn 2HCTc9Ourqm7wbJDzZgrjLMZQUutxBjFZIr0L/faGbolI5vXJfOh/34OYqdPWIRe 82wyXM6UEt0EUR3osr3SeszUc7HaRol8HaOY4HayRMPyYHp+TLu6+oabV6PlLaBj Xti44UDKE3qxX9ssfmxc0BZBv91IQArIrG3mQZlThUECzIMjKwBLEZK1DjTSRdD4 lZrwUKxFhiueiGcD6ezepg== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grdpca0w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:32 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1CloLm010012; Fri, 1 Nov 2024 13:48:31 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptk8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:31 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuOE006031; Fri, 1 Nov 2024 13:48:30 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-8; Fri, 01 Nov 2024 13:48:30 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 07/16] migration: cpr-transfer save and load Date: Fri, 1 Nov 2024 06:47:46 -0700 Message-Id: <1730468875-249970-8-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-ORIG-GUID: Yz7naI_RtePmTy93SEhtIRIbb9gebNYx X-Proofpoint-GUID: Yz7naI_RtePmTy93SEhtIRIbb9gebNYx Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add functions to create a QEMUFile based on a unix URI, for saving or loading, for use by cpr-transfer mode to preserve CPR state. Signed-off-by: Steve Sistare Reviewed-by: Peter Xu --- include/migration/cpr.h | 3 ++ migration/cpr-transfer.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++++ migration/meson.build | 1 + 3 files changed, 85 insertions(+) create mode 100644 migration/cpr-transfer.c diff --git a/include/migration/cpr.h b/include/migration/cpr.h index 6e4781c..ae318da 100644 --- a/include/migration/cpr.h +++ b/include/migration/cpr.h @@ -20,4 +20,7 @@ int cpr_state_load(Error **errp); void cpr_state_close(void); struct QIOChannel *cpr_state_ioc(void); +QEMUFile *cpr_transfer_output(const char *uri, Error **errp); +QEMUFile *cpr_transfer_input(const char *uri, Error **errp); + #endif diff --git a/migration/cpr-transfer.c b/migration/cpr-transfer.c new file mode 100644 index 0000000..fb9ecd8 --- /dev/null +++ b/migration/cpr-transfer.c @@ -0,0 +1,81 @@ +/* + * Copyright (c) 2022, 2024 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "io/channel-file.h" +#include "io/channel-socket.h" +#include "io/net-listener.h" +#include "migration/cpr.h" +#include "migration/migration.h" +#include "migration/savevm.h" +#include "migration/qemu-file.h" +#include "migration/vmstate.h" + +QEMUFile *cpr_transfer_output(const char *uri, Error **errp) +{ + g_autoptr(MigrationChannel) channel = NULL; + QIOChannel *ioc; + + if (!migrate_uri_parse(uri, &channel, errp)) { + return NULL; + } + + if (channel->addr->transport == MIGRATION_ADDRESS_TYPE_SOCKET && + channel->addr->u.socket.type == SOCKET_ADDRESS_TYPE_UNIX) { + + QIOChannelSocket *sioc = qio_channel_socket_new(); + SocketAddress *saddr = &channel->addr->u.socket; + + if (qio_channel_socket_connect_sync(sioc, saddr, errp)) { + object_unref(OBJECT(sioc)); + return NULL; + } + ioc = QIO_CHANNEL(sioc); + + } else { + error_setg(errp, "bad cpr-uri %s; must be unix:", uri); + return NULL; + } + + qio_channel_set_name(ioc, "cpr-out"); + return qemu_file_new_output(ioc); +} + +QEMUFile *cpr_transfer_input(const char *uri, Error **errp) +{ + g_autoptr(MigrationChannel) channel = NULL; + QIOChannel *ioc; + + if (!migrate_uri_parse(uri, &channel, errp)) { + return NULL; + } + + if (channel->addr->transport == MIGRATION_ADDRESS_TYPE_SOCKET && + channel->addr->u.socket.type == SOCKET_ADDRESS_TYPE_UNIX) { + + QIOChannelSocket *sioc; + SocketAddress *saddr = &channel->addr->u.socket; + QIONetListener *listener = qio_net_listener_new(); + + qio_net_listener_set_name(listener, "cpr-socket-listener"); + if (qio_net_listener_open_sync(listener, saddr, 1, errp) < 0) { + object_unref(OBJECT(listener)); + return NULL; + } + + sioc = qio_net_listener_wait_client(listener); + ioc = QIO_CHANNEL(sioc); + + } else { + error_setg(errp, "bad cpr-uri %s; must be unix:", uri); + return NULL; + } + + qio_channel_set_name(ioc, "cpr-in"); + return qemu_file_new_input(ioc); +} diff --git a/migration/meson.build b/migration/meson.build index e5f4211..684ba98 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -14,6 +14,7 @@ system_ss.add(files( 'channel.c', 'channel-block.c', 'cpr.c', + 'cpr-transfer.c', 'dirtyrate.c', 'exec.c', 'fd.c', From patchwork Fri Nov 1 13:47:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859423 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 89424E6B27C for ; Fri, 1 Nov 2024 13:49:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s12-0004IU-AV; Fri, 01 Nov 2024 09:48:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0z-0004Dk-Nt for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:37 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0x-0002Hc-R1 for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:37 -0400 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1DffEY026685; Fri, 1 Nov 2024 13:48:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=KdYcQra8WuKmwlcs0CeO5clpBnDxsDFsYeuREwqG5RQ=; b= cD3+EXJFG/zWZm3kFtIgnOT6fZWh/vM7fC5sPSuHoMA0kBFXX/xzUVRDAN7tT9u7 nUJmJKAUgRCgzmkQJVQtjxWKfuZOCn1DPFnnKULAygA/KvcVd0LQ4tHwi0FgOtDT 78K5b1NKhr3On6C8MS1w11vm1wV3mTFxOf1IH9PAAE3vZybxcTWogZvPTLOP3l6M /BnHh7m+/DlPG1yFEG+pgXgAguyV59MoTaguBwcUXPzvIbbkTgR4oreJtxi0JTti 1BDUoVRc29+p4Dmat0paodf7OgRRdc82i/mIiI0+JSWxVVEKSyZjqmtufI8uNT3V c4ustvLYeMGm/z21KdThgQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grgmme6m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:33 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1DWwk7010337; Fri, 1 Nov 2024 13:48:32 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptkm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:32 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuOG006031; Fri, 1 Nov 2024 13:48:31 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-9; Fri, 01 Nov 2024 13:48:31 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 08/16] migration: cpr-uri parameter Date: Fri, 1 Nov 2024 06:47:47 -0700 Message-Id: <1730468875-249970-9-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-GUID: aLNMdBH_TF8pXUs2_ErjPb9FVEOa78Hn X-Proofpoint-ORIG-GUID: aLNMdBH_TF8pXUs2_ErjPb9FVEOa78Hn Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Define the cpr-uri migration parameter to specify the URI to which CPR vmstate is saved for cpr-transfer mode. Signed-off-by: Steve Sistare Acked-by: Peter Xu --- migration/migration-hmp-cmds.c | 10 ++++++++++ migration/options.c | 28 ++++++++++++++++++++++++++++ migration/options.h | 1 + qapi/migration.json | 18 +++++++++++++++--- 4 files changed, 54 insertions(+), 3 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 20d1a6e..79d8c66 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -358,6 +358,11 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict) MIGRATION_PARAMETER_DIRECT_IO), params->direct_io ? "on" : "off"); } + + assert(params->cpr_uri); + monitor_printf(mon, "%s: '%s'\n", + MigrationParameter_str(MIGRATION_PARAMETER_CPR_URI), + params->cpr_uri); } qapi_free_MigrationParameters(params); @@ -639,6 +644,11 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_direct_io = true; visit_type_bool(v, param, &p->direct_io, &err); break; + case MIGRATION_PARAMETER_CPR_URI: + p->cpr_uri = g_new0(StrOrNull, 1); + p->cpr_uri->type = QTYPE_QSTRING; + visit_type_str(v, param, &p->cpr_uri->u.s, &err); + break; default: g_assert_not_reached(); } diff --git a/migration/options.c b/migration/options.c index ad8d698..82de1d8 100644 --- a/migration/options.c +++ b/migration/options.c @@ -172,6 +172,8 @@ Property migration_properties[] = { DEFINE_PROP_ZERO_PAGE_DETECTION("zero-page-detection", MigrationState, parameters.zero_page_detection, ZERO_PAGE_DETECTION_MULTIFD), + DEFINE_PROP_STRING("cpr-uri", MigrationState, + parameters.cpr_uri), /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -837,6 +839,13 @@ ZeroPageDetection migrate_zero_page_detection(void) return s->parameters.zero_page_detection; } +const char *migrate_cpr_uri(void) +{ + MigrationState *s = migrate_get_current(); + + return s->parameters.cpr_uri; +} + /* parameters helpers */ AnnounceParameters *migrate_announce_params(void) @@ -922,6 +931,7 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->zero_page_detection = s->parameters.zero_page_detection; params->has_direct_io = true; params->direct_io = s->parameters.direct_io; + params->cpr_uri = g_strdup(s->parameters.cpr_uri); return params; } @@ -956,6 +966,7 @@ void migrate_params_init(MigrationParameters *params) params->has_mode = true; params->has_zero_page_detection = true; params->has_direct_io = true; + params->cpr_uri = g_strdup(""); } /* @@ -1255,6 +1266,11 @@ static void migrate_params_test_apply(MigrateSetParameters *params, if (params->has_direct_io) { dest->direct_io = params->direct_io; } + + if (params->cpr_uri) { + assert(params->cpr_uri->type == QTYPE_QSTRING); + dest->cpr_uri = params->cpr_uri->u.s; + } } static void migrate_params_apply(MigrateSetParameters *params, Error **errp) @@ -1387,6 +1403,12 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) if (params->has_direct_io) { s->parameters.direct_io = params->direct_io; } + + if (params->cpr_uri) { + g_free(s->parameters.cpr_uri); + assert(params->cpr_uri->type == QTYPE_QSTRING); + s->parameters.cpr_uri = g_strdup(params->cpr_uri->u.s); + } } void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) @@ -1413,6 +1435,12 @@ void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) params->tls_authz->u.s = strdup(""); } + if (params->cpr_uri && params->cpr_uri->type == QTYPE_QNULL) { + qobject_unref(params->cpr_uri->u.n); + params->cpr_uri->type = QTYPE_QSTRING; + params->cpr_uri->u.s = strdup(""); + } + migrate_params_test_apply(params, &tmp); if (!migrate_params_check(&tmp, errp)) { diff --git a/migration/options.h b/migration/options.h index 79084ee..2b4c082 100644 --- a/migration/options.h +++ b/migration/options.h @@ -84,6 +84,7 @@ const char *migrate_tls_creds(void); const char *migrate_tls_hostname(void); uint64_t migrate_xbzrle_cache_size(void); ZeroPageDetection migrate_zero_page_detection(void); +const char *migrate_cpr_uri(void); /* parameters helpers */ diff --git a/qapi/migration.json b/qapi/migration.json index 3af6aa1..5bf3e49 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -844,6 +844,9 @@ # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) # +# @cpr-uri: URI for an additional migration channel needed by +# @cpr-transfer mode. (Since 9.2) +# # Features: # # @unstable: Members @x-checkpoint-delay and @@ -870,7 +873,8 @@ 'vcpu-dirty-limit', 'mode', 'zero-page-detection', - 'direct-io'] } + 'direct-io', + 'cpr-uri'] } ## # @MigrateSetParameters: @@ -1025,6 +1029,9 @@ # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) # +# @cpr-uri: URI for an additional migration channel needed by +# @cpr-transfer mode. (Since 9.2) +# # Features: # # @unstable: Members @x-checkpoint-delay and @@ -1066,7 +1073,8 @@ '*vcpu-dirty-limit': 'uint64', '*mode': 'MigMode', '*zero-page-detection': 'ZeroPageDetection', - '*direct-io': 'bool' } } + '*direct-io': 'bool', + '*cpr-uri': 'StrOrNull' } } ## # @migrate-set-parameters: @@ -1235,6 +1243,9 @@ # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) # +# @cpr-uri: URI for an additional migration channel needed by +# @cpr-transfer mode. (Since 9.2) +# # Features: # # @unstable: Members @x-checkpoint-delay and @@ -1273,7 +1284,8 @@ '*vcpu-dirty-limit': 'uint64', '*mode': 'MigMode', '*zero-page-detection': 'ZeroPageDetection', - '*direct-io': 'bool' } } + '*direct-io': 'bool', + '*cpr-uri': 'str' } } ## # @query-migrate-parameters: From patchwork Fri Nov 1 13:47:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859430 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 93A7EE6B278 for ; Fri, 1 Nov 2024 13:50:32 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s15-0004TN-NC; Fri, 01 Nov 2024 09:48:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s13-0004Mb-Gc for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:41 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s11-0002J4-KG for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:41 -0400 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1Dfgww029215; Fri, 1 Nov 2024 13:48:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=VkW1dq6TTWsU3T6/rVxk+iWHYtUQ3SfxCOyN+XDGssQ=; b= TGlmHrtIuFduxpSLATU+OHV+HdxoM/RlEwP5DZPv0uIl7s8KT7Nbtw3wovTZj1aJ Et4zecmALwh6HuzTpL6O5oLRKUFJA42T7nd8S6daUKU32NpUr739s1a/wuZwFZbx BUCZqO0uui/qjYLLKKmhUhO58tiCjRjKnXDPWogYP1NJ5Mv2SR20Lky/4ekUe14l ynekpTELoz7+5NMW1F9Tn7NOjzFM8FgbfL8QFiaIgpgMTM8k3ZjREWgiWjPKDFNM d2CsPVXbOW2AR9T8/4flc6zKESxy3d7VvBqeKJOotZzqg3FRTpSoPVKHSuo4BRxH 4CyqBGamhSAEDIOsQum3Fw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grdxv9x0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:33 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1CXiVD009960; Fri, 1 Nov 2024 13:48:33 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptkx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:33 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuOI006031; Fri, 1 Nov 2024 13:48:32 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-10; Fri, 01 Nov 2024 13:48:32 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 09/16] migration: cpr-uri option Date: Fri, 1 Nov 2024 06:47:48 -0700 Message-Id: <1730468875-249970-10-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-GUID: -c8mQU8Le5lkvJM4CYNvECXLl-8o7HUD X-Proofpoint-ORIG-GUID: -c8mQU8Le5lkvJM4CYNvECXLl-8o7HUD Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Define the cpr-uri QEMU command-line option to specify the URI from which CPR vmstate is loaded for cpr-transfer mode. Signed-off-by: Steve Sistare Acked-by: Peter Xu --- include/migration/cpr.h | 3 +++ migration/cpr.c | 12 ++++++++++++ qemu-options.hx | 8 ++++++++ system/vl.c | 4 ++++ 4 files changed, 27 insertions(+) diff --git a/include/migration/cpr.h b/include/migration/cpr.h index ae318da..c9c291f 100644 --- a/include/migration/cpr.h +++ b/include/migration/cpr.h @@ -15,6 +15,9 @@ void cpr_save_fd(const char *name, int id, int fd); void cpr_delete_fd(const char *name, int id); int cpr_find_fd(const char *name, int id); +void cpr_set_cpr_uri(const char *uri); +const char *cpr_get_cpr_uri(void); + int cpr_state_save(Error **errp); int cpr_state_load(Error **errp); void cpr_state_close(void); diff --git a/migration/cpr.c b/migration/cpr.c index be1dc92..b72d1f4 100644 --- a/migration/cpr.c +++ b/migration/cpr.c @@ -116,6 +116,18 @@ QIOChannel *cpr_state_ioc(void) return qemu_file_get_ioc(cpr_state_file); } +static char *cpr_uri; + +void cpr_set_cpr_uri(const char *uri) +{ + cpr_uri = g_strdup(uri); +} + +const char *cpr_get_cpr_uri(void) +{ + return cpr_uri; +} + int cpr_state_save(Error **errp) { int ret; diff --git a/qemu-options.hx b/qemu-options.hx index fdd6bf2..89bbc9f 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -4922,6 +4922,14 @@ SRST ERST +DEF("cpr-uri", HAS_ARG, QEMU_OPTION_cpr_uri, \ + "-cpr-uri unix:socketpath\n", + QEMU_ARCH_ALL) +SRST +``-cpr-uri unix:socketpath`` + URI for incoming CPR state, for the cpr-transfer migration mode. +ERST + DEF("incoming", HAS_ARG, QEMU_OPTION_incoming, \ "-incoming tcp:[host]:port[,to=maxport][,ipv4=on|off][,ipv6=on|off]\n" \ "-incoming rdma:host:port[,ipv4=on|off][,ipv6=on|off]\n" \ diff --git a/system/vl.c b/system/vl.c index df26264..5d08fade 100644 --- a/system/vl.c +++ b/system/vl.c @@ -77,6 +77,7 @@ #include "hw/block/block.h" #include "hw/i386/x86.h" #include "hw/i386/pc.h" +#include "migration/cpr.h" #include "migration/misc.h" #include "migration/snapshot.h" #include "sysemu/tpm.h" @@ -3479,6 +3480,9 @@ void qemu_init(int argc, char **argv) exit(1); } break; + case QEMU_OPTION_cpr_uri: + cpr_set_cpr_uri(optarg); + break; case QEMU_OPTION_incoming: if (!incoming) { runstate_set(RUN_STATE_INMIGRATE); From patchwork Fri Nov 1 13:47:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E5354E6B27C for ; Fri, 1 Nov 2024 13:49:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s13-0004Lo-9J; Fri, 01 Nov 2024 09:48:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s10-0004GH-Jy for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:38 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s0z-0002I9-4F for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:38 -0400 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1DfdpW009299; Fri, 1 Nov 2024 13:48:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=uwIYRe4SV6Wm2OcD+4vHFa67wx9C2XCjgtT/TGXsxpY=; b= jJEZ5H8Q6Ab4iN7bnjP+MRccSLMK3u70L5j0+tkHyWAzk9RSYpnoLmL2BZGBG2Dg PeIoqKFSK7Ac3PIejfOdXz5KmjpultSj0a4Of1R5aQ1GrSI9RcL94U0p1Sv9FG6w 3Uiq+OWvGsBbl9OTZ77d0mP3SOC/cPFnovORDdyEqP5mGnWa3SYCogzX/IM++9L4 FCHAfP3uxlGXXvgTR1UxIICjUFYeJKy1plO0/bsEaZyGlekAhfEc5MfgmzNw1U8K St/7IvheEgzJiCurqLpu8ZsZG4Jg1PnmxURvwkj75yItHwPHsOS3hLpOryXD2EYB nVdhaQxpAfR1GVbwczuVnQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grdpca10-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:34 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1DgLfM010133; Fri, 1 Nov 2024 13:48:33 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptm8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:33 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuOK006031; Fri, 1 Nov 2024 13:48:33 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-11; Fri, 01 Nov 2024 13:48:33 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 10/16] migration: split qmp_migrate Date: Fri, 1 Nov 2024 06:47:49 -0700 Message-Id: <1730468875-249970-11-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-ORIG-GUID: WK_VIPraSCxOtKuQrUzKnA4_j-5CXKPl X-Proofpoint-GUID: WK_VIPraSCxOtKuQrUzKnA4_j-5CXKPl Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Split qmp_migrate into start and finish functions. Finish will be called asynchronously in a subsequent patch, but for now, call it immediately. No functional change. Signed-off-by: Steve Sistare --- migration/migration.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index 6dc7c09..86b3f39 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1521,6 +1521,7 @@ static void migrate_fd_error(MigrationState *s, const Error *error) static void migrate_fd_cancel(MigrationState *s) { int old_state ; + bool setup = (s->state == MIGRATION_STATUS_SETUP); trace_migrate_fd_cancel(); @@ -1565,6 +1566,15 @@ static void migrate_fd_cancel(MigrationState *s) s->block_inactive = false; } } + + /* + * If qmp_migrate_finish has not been called, then there is no path that + * will complete the cancellation. Do it now. + */ + if (setup && !s->to_dst_file) { + migrate_set_state(&s->state, s->state, MIGRATION_STATUS_CANCELLED); + vm_resume(s->vm_old_state); + } } void migration_add_notifier_mode(NotifierWithReturn *notify, @@ -2072,6 +2082,9 @@ static bool migrate_prepare(MigrationState *s, bool resume, Error **errp) return true; } +static void qmp_migrate_finish(MigrationAddress *addr, bool resume_requested, + Error **errp); + void qmp_migrate(const char *uri, bool has_channels, MigrationChannelList *channels, bool has_detach, bool detach, bool has_resume, bool resume, Error **errp) @@ -2118,6 +2131,20 @@ void qmp_migrate(const char *uri, bool has_channels, return; } + qmp_migrate_finish(addr, resume_requested, errp); + + if (local_err) { + migrate_fd_error(s, local_err); + error_propagate(errp, local_err); + } +} + +static void qmp_migrate_finish(MigrationAddress *addr, bool resume_requested, + Error **errp) +{ + MigrationState *s = migrate_get_current(); + Error *local_err = NULL; + if (!resume_requested) { if (!yank_register_instance(MIGRATION_YANK_INSTANCE, errp)) { return; From patchwork Fri Nov 1 13:47:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859432 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 46E4AE6B27D for ; Fri, 1 Nov 2024 13:50:41 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s17-0004ac-5x; Fri, 01 Nov 2024 09:48:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s15-0004Pt-5U for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:43 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s12-0002JT-JD for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:42 -0400 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1Dg9HE026803; Fri, 1 Nov 2024 13:48:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=cisfb0ijx5L+5BrhQN+axrJq0nER1Scead8pkn2CIdc=; b= ZzIMCOQWR5HxwpWKbsGksnnKQJc7HxNJRPJ0z5J4tsn2NWNDtLRlh/RX94xZy32c cN4mtmS+gwtczkO336z2pFC10izpSw+x/FF+DgB4gp+bsIR6d9BK+kwaVaLTm2iH GoZuwuR4pghVvkITTi7YrA2d4agELeAcbUA0fPu4a+nc7Qsy0tFUew7gyHoOFzKE svkqb3nk0lPhJPooS+Vv/XtB7WhyQiJFvMTmMSDda30oLyrVWBg9D7yKYuC1MNc7 ax2R6fyXoqg6MW8EDmqtdwLfSHkEpW/fiM0/2PewrMx4gapRke6Ct8mJxrW6qfWW 5qMzJgjkxgShEWBde+gf2g== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42gryscbcd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:35 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1BiUW2010024; Fri, 1 Nov 2024 13:48:34 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptmc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:34 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuOM006031; Fri, 1 Nov 2024 13:48:34 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-12; Fri, 01 Nov 2024 13:48:33 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 11/16] migration: cpr-transfer mode Date: Fri, 1 Nov 2024 06:47:50 -0700 Message-Id: <1730468875-249970-12-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-ORIG-GUID: Cdb4wxuvwxbpAdGRhmqESyD3kR1aTt2j X-Proofpoint-GUID: Cdb4wxuvwxbpAdGRhmqESyD3kR1aTt2j Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add the cpr-transfer migration mode. Usage: qemu-system-$arch -machine anon-alloc=memfd ... start new QEMU with "-incoming -cpr-uri " Issue commands to old QEMU: migrate_set_parameter mode cpr-transfer migrate_set_parameter cpr-uri migrate -d The migrate command stops the VM, saves CPR state to uri-2, saves normal migration state to uri-1, and old QEMU enters the postmigrate state. The user starts new QEMU on the same host as old QEMU, with the same arguments as old QEMU, plus the -incoming option. Guest RAM is preserved in place, albeit with new virtual addresses in new QEMU. This mode requires a second migration channel, specified by the cpr-uri migration property on the outgoing side, and by the cpr-uri QEMU command-line option on the incoming side. The channel must be a type, such as unix socket, that supports SCM_RIGHTS. Memory-backend objects must have the share=on attribute, but memory-backend-epc is not supported. The VM must be started with the '-machine anon-alloc=memfd' option, which allows anonymous memory to be transferred in place to the new process. The memfds are kept open by sending the descriptors to new QEMU via the cpr-uri, which must support SCM_RIGHTS, and they are mmap'd in new QEMU. Signed-off-by: Steve Sistare --- migration/cpr.c | 29 ++++++++++++++--- migration/migration.c | 81 +++++++++++++++++++++++++++++++++++++++++++++-- migration/migration.h | 2 ++ migration/options.c | 12 +++++-- migration/ram.c | 2 ++ migration/trace-events | 2 ++ migration/vmstate-types.c | 1 + qapi/migration.json | 35 +++++++++++++++++++- stubs/vmstate.c | 7 ++++ system/vl.c | 6 ++++ 10 files changed, 167 insertions(+), 10 deletions(-) diff --git a/migration/cpr.c b/migration/cpr.c index b72d1f4..3f3ef43 100644 --- a/migration/cpr.c +++ b/migration/cpr.c @@ -45,7 +45,7 @@ static const VMStateDescription vmstate_cpr_fd = { VMSTATE_UINT32(namelen, CprFd), VMSTATE_VBUFFER_ALLOC_UINT32(name, CprFd, 0, NULL, namelen), VMSTATE_INT32(id, CprFd), - VMSTATE_INT32(fd, CprFd), + VMSTATE_FD(fd, CprFd), VMSTATE_END_OF_LIST() } }; @@ -132,9 +132,18 @@ int cpr_state_save(Error **errp) { int ret; QEMUFile *f; + MigMode mode = migrate_mode(); - /* set f based on mode in a later patch in this series */ - return 0; + if (mode == MIG_MODE_CPR_TRANSFER) { + f = cpr_transfer_output(migrate_cpr_uri(), errp); + } else { + return 0; + } + if (!f) { + return -1; + } + + trace_cpr_state_save(MigMode_str(mode), migrate_cpr_uri()); qemu_put_be32(f, QEMU_CPR_FILE_MAGIC); qemu_put_be32(f, QEMU_CPR_FILE_VERSION); @@ -162,9 +171,19 @@ int cpr_state_load(Error **errp) int ret; uint32_t v; QEMUFile *f; + MigMode mode = 0; - /* set f based on other parameters in a later patch in this series */ - return 0; + if (cpr_uri) { + mode = MIG_MODE_CPR_TRANSFER; + f = cpr_transfer_input(cpr_uri, errp); + } else { + return 0; + } + if (!f) { + return -1; + } + + trace_cpr_state_load(MigMode_str(mode), cpr_uri); v = qemu_get_be32(f); if (v != QEMU_CPR_FILE_MAGIC) { diff --git a/migration/migration.c b/migration/migration.c index 86b3f39..5a53d01 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -76,6 +76,7 @@ static NotifierWithReturnList migration_state_notifiers[] = { NOTIFIER_ELEM_INIT(migration_state_notifiers, MIG_MODE_NORMAL), NOTIFIER_ELEM_INIT(migration_state_notifiers, MIG_MODE_CPR_REBOOT), + NOTIFIER_ELEM_INIT(migration_state_notifiers, MIG_MODE_CPR_TRANSFER), }; /* Messages sent on the return path from destination to source */ @@ -109,6 +110,7 @@ static int migration_maybe_pause(MigrationState *s, static void migrate_fd_cancel(MigrationState *s); static bool close_return_path_on_source(MigrationState *s); static void migration_completion_end(MigrationState *s); +static void migrate_hup_delete(MigrationState *s); static void migration_downtime_start(MigrationState *s) { @@ -204,6 +206,12 @@ migration_channels_and_transport_compatible(MigrationAddress *addr, return false; } + if (migrate_mode() == MIG_MODE_CPR_TRANSFER && + addr->transport == MIGRATION_ADDRESS_TYPE_FILE) { + error_setg(errp, "Migration requires streamable transport (eg unix)"); + return false; + } + return true; } @@ -316,6 +324,7 @@ void migration_cancel(const Error *error) qmp_cancel_vcpu_dirty_limit(false, -1, NULL); } migrate_fd_cancel(current_migration); + migrate_hup_delete(current_migration); } void migration_shutdown(void) @@ -416,6 +425,7 @@ void migration_incoming_state_destroy(void) mis->postcopy_qemufile_dst = NULL; } + cpr_set_cpr_uri(NULL); yank_unregister_instance(MIGRATION_YANK_INSTANCE); } @@ -717,6 +727,9 @@ static void qemu_start_incoming_migration(const char *uri, bool has_channels, } else { error_setg(errp, "unknown migration protocol: %s", uri); } + + /* Close cpr socket to tell source that we are listening */ + cpr_state_close(); } static void process_incoming_migration_bh(void *opaque) @@ -1413,6 +1426,8 @@ static void migrate_fd_cleanup(MigrationState *s) s->vmdesc = NULL; qemu_savevm_state_cleanup(); + cpr_state_close(); + migrate_hup_delete(s); close_return_path_on_source(s); @@ -1573,6 +1588,8 @@ static void migrate_fd_cancel(MigrationState *s) */ if (setup && !s->to_dst_file) { migrate_set_state(&s->state, s->state, MIGRATION_STATUS_CANCELLED); + cpr_state_close(); + migrate_hup_delete(s); vm_resume(s->vm_old_state); } } @@ -1707,7 +1724,9 @@ bool migration_thread_is_self(void) bool migrate_mode_is_cpr(MigrationState *s) { - return s->parameters.mode == MIG_MODE_CPR_REBOOT; + MigMode mode = s->parameters.mode; + return mode == MIG_MODE_CPR_REBOOT || + mode == MIG_MODE_CPR_TRANSFER; } int migrate_init(MigrationState *s, Error **errp) @@ -2042,6 +2061,12 @@ static bool migrate_prepare(MigrationState *s, bool resume, Error **errp) return false; } + if (migrate_mode() == MIG_MODE_CPR_TRANSFER && + (!s->parameters.cpr_uri || !s->parameters.cpr_uri[0])) { + error_setg(errp, "cpr-transfer mode requires setting cpr-uri"); + return false; + } + if (migration_is_blocked(errp)) { return false; } @@ -2085,6 +2110,37 @@ static bool migrate_prepare(MigrationState *s, bool resume, Error **errp) static void qmp_migrate_finish(MigrationAddress *addr, bool resume_requested, Error **errp); +static void migrate_hup_add(MigrationState *s, QIOChannel *ioc, GSourceFunc cb, + void *opaque) +{ + s->hup_source = qio_channel_create_watch(ioc, G_IO_HUP); + g_source_set_callback(s->hup_source, cb, opaque, NULL); + g_source_attach(s->hup_source, NULL); +} + +static void migrate_hup_delete(MigrationState *s) +{ + if (s->hup_source) { + g_source_destroy(s->hup_source); + g_source_unref(s->hup_source); + s->hup_source = NULL; + } +} + +static gboolean qmp_migrate_finish_cb(QIOChannel *channel, + GIOCondition cond, + void *opaque) +{ + MigrationAddress *addr = opaque; + + qmp_migrate_finish(addr, false, NULL); + + cpr_state_close(); + migrate_hup_delete(migrate_get_current()); + qapi_free_MigrationAddress(addr); + return G_SOURCE_REMOVE; +} + void qmp_migrate(const char *uri, bool has_channels, MigrationChannelList *channels, bool has_detach, bool detach, bool has_resume, bool resume, Error **errp) @@ -2131,8 +2187,29 @@ void qmp_migrate(const char *uri, bool has_channels, return; } - qmp_migrate_finish(addr, resume_requested, errp); + if (cpr_state_save(&local_err)) { + goto out; + } + /* + * For cpr-transfer, the target may not be listening yet on the migration + * channel, because first it must finish cpr_load_state. The target tells + * us it is listening by closing the cpr-state socket. Wait for that HUP + * event before connecting in qmp_migrate_finish. + * + * The HUP could occur because the target fails while reading CPR state, + * in which case the target will not listen for the incoming migration + * connection, so qmp_migrate_finish will fail to connect, and then recover. + */ + if (s->parameters.mode == MIG_MODE_CPR_TRANSFER) { + migrate_hup_add(s, cpr_state_ioc(), (GSourceFunc)qmp_migrate_finish_cb, + QAPI_CLONE(MigrationAddress, addr)); + + } else { + qmp_migrate_finish(addr, resume_requested, errp); + } + +out: if (local_err) { migrate_fd_error(s, local_err); error_propagate(errp, local_err); diff --git a/migration/migration.h b/migration/migration.h index 38aa140..74c167b 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -457,6 +457,8 @@ struct MigrationState { bool switchover_acked; /* Is this a rdma migration */ bool rdma_migration; + + GSource *hup_source; }; void migrate_set_state(MigrationStatus *state, MigrationStatus old_state, diff --git a/migration/options.c b/migration/options.c index 82de1d8..3733bc9 100644 --- a/migration/options.c +++ b/migration/options.c @@ -22,6 +22,7 @@ #include "qapi/qmp/qnull.h" #include "sysemu/runstate.h" #include "migration/colo.h" +#include "migration/cpr.h" #include "migration/misc.h" #include "migration.h" #include "migration-stats.h" @@ -747,9 +748,16 @@ uint64_t migrate_max_postcopy_bandwidth(void) MigMode migrate_mode(void) { - MigrationState *s = migrate_get_current(); - MigMode mode = s->parameters.mode; + MigMode mode; + /* + * cpr_uri is only set during the early cpr-transfer loading stage, + * after which it is cleared. + */ + if (cpr_get_cpr_uri()) { + return MIG_MODE_CPR_TRANSFER; + } + mode = migrate_get_current()->parameters.mode; assert(mode >= 0 && mode < MIG_MODE__MAX); return mode; } diff --git a/migration/ram.c b/migration/ram.c index 326ce7e..bafe41b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -216,7 +216,9 @@ static bool postcopy_preempt_active(void) bool migrate_ram_is_ignored(RAMBlock *block) { + MigMode mode = migrate_mode(); return !qemu_ram_is_migratable(block) || + mode == MIG_MODE_CPR_TRANSFER || (migrate_ignore_shared() && qemu_ram_is_shared(block) && qemu_ram_is_named_file(block)); } diff --git a/migration/trace-events b/migration/trace-events index 345506b..455dec5 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -347,6 +347,8 @@ colo_failover_set_state(const char *new_state) "new state %s" cpr_save_fd(const char *name, int id, int fd) "%s, id %d, fd %d" cpr_delete_fd(const char *name, int id) "%s, id %d" cpr_find_fd(const char *name, int id, int fd) "%s, id %d returns %d" +cpr_state_save(const char *mode, const char *uri) "%s to %s" +cpr_state_load(const char *mode, const char *uri) "%s from %s" # block-dirty-bitmap.c send_bitmap_header_enter(void) "" diff --git a/migration/vmstate-types.c b/migration/vmstate-types.c index f31deb3..2210f0c 100644 --- a/migration/vmstate-types.c +++ b/migration/vmstate-types.c @@ -15,6 +15,7 @@ #include "qemu-file.h" #include "migration.h" #include "migration/vmstate.h" +#include "migration/client-options.h" #include "qemu/error-report.h" #include "qemu/queue.h" #include "trace.h" diff --git a/qapi/migration.json b/qapi/migration.json index 5bf3e49..3328d1b 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -614,9 +614,42 @@ # or COLO. # # (since 8.2) +# +# @cpr-transfer: This mode allows the user to transfer a guest to a +# new QEMU instance on the same host with minimal guest pause +# time, by preserving guest RAM in place, albeit with new virtual +# addresses in new QEMU. +# +# The user starts new QEMU on the same host as old QEMU, with the +# the same arguments as old QEMU, plus the -incoming option. The +# user issues the migrate command to old QEMU, which stops the VM, +# saves state to the migration channels, and enters the +# postmigrate state. Execution resumes in new QEMU. +# +# This mode requires a second migration channel, specified by the +# cpr-uri migration property on the outgoing side, and by the +# cpr-uri QEMU command-line option on the incoming side. The +# channel must be a type, such as unix socket, that supports +# SCM_RIGHTS. +# +# Memory-backend objects must have the share=on attribute, but +# memory-backend-epc and memory-backend-ram are not supported. +# The VM must be started with the '-machine anon-alloc=memfd' +# option. +# +# The incoming migration channel cannot be a file type, and for +# the tcp type, the port cannot be 0 (meaning dynamically choose +# a port). +# +# When using -incoming defer, you must issue the migrate command +# to old QEMU before issuing any monitor commands to new QEMU. +# However, new QEMU does not open and read the migration stream +# until you issue the migrate incoming command. +# +# (since 9.2) ## { 'enum': 'MigMode', - 'data': [ 'normal', 'cpr-reboot' ] } + 'data': [ 'normal', 'cpr-reboot', 'cpr-transfer' ] } ## # @ZeroPageDetection: diff --git a/stubs/vmstate.c b/stubs/vmstate.c index 8513d92..c190762 100644 --- a/stubs/vmstate.c +++ b/stubs/vmstate.c @@ -1,5 +1,7 @@ #include "qemu/osdep.h" #include "migration/vmstate.h" +#include "qapi/qapi-types-migration.h" +#include "migration/client-options.h" int vmstate_register_with_alias_id(VMStateIf *obj, uint32_t instance_id, @@ -21,3 +23,8 @@ bool vmstate_check_only_migratable(const VMStateDescription *vmsd) { return true; } + +MigMode migrate_mode(void) +{ + return MIG_MODE_NORMAL; +} diff --git a/system/vl.c b/system/vl.c index 5d08fade..9bd0e33 100644 --- a/system/vl.c +++ b/system/vl.c @@ -3714,6 +3714,12 @@ void qemu_init(int argc, char **argv) qemu_create_machine(machine_opts_dict); + /* + * Load incoming CPR state before any devices are created, because it + * contains file descriptors that are needed in device initialization code. + */ + cpr_state_load(&error_fatal); + suspend_mux_open(); qemu_disable_default_devices(); From patchwork Fri Nov 1 13:47:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9AE3CE6B27C for ; Fri, 1 Nov 2024 13:50:45 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s15-0004QW-BX; Fri, 01 Nov 2024 09:48:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s12-0004Ix-BV for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:40 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s10-0002It-Qe for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:40 -0400 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1DfeYU026645; Fri, 1 Nov 2024 13:48:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=dpCPRDwBsiH7p1vvYqDWQ9A8+6soeOcX/CfHOL0dUus=; b= DHkYvTHZboVxaMXiPN/ZClnxf4H3MIAA3mkEiT5VA6SHCwC1uP51RoJ8w+7Kbdys 6mA9DlTygjmk8DDRQ/BZA3/JBCjkVAgWBI6zwpLdcj49AwrEouux/jSxRbzhxJMt A179c9o9CNRT7b7N0anPD+Ov2rgwigfdkf9OCSu5cdPxRQqK/Q7ieH77QBbfvC9f 5/fKZppO/DUDfZPPRN9cv7+cEj2nEg+gy+sqq3EHNBC3peTAQvjSOWi8glAOQVnp Z6aKZUK5cCZDoSmW+5aRBIa1AX479A/wIpxPTvRJOVKPM4sGJzaD60RrAeB3ldd0 jS3OahMg+M4w3im+IYY+Zg== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grgmme6r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:36 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1CVS7e010127; Fri, 1 Nov 2024 13:48:35 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptmk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:35 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuOO006031; Fri, 1 Nov 2024 13:48:34 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-13; Fri, 01 Nov 2024 13:48:34 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 12/16] tests/migration-test: memory_backend Date: Fri, 1 Nov 2024 06:47:51 -0700 Message-Id: <1730468875-249970-13-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-GUID: sWfDJ-nKruDgsPt3Ulyc5mW5u2yOUBJF X-Proofpoint-ORIG-GUID: sWfDJ-nKruDgsPt3Ulyc5mW5u2yOUBJF Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Allow each migration test to define its own memory backend, replacing the standard "-m " specification. Signed-off-by: Steve Sistare Reviewed-by: Fabiano Rosas --- tests/qtest/migration-test.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 95e45b5..a008316 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -609,6 +609,11 @@ typedef struct { const char *opts_target; /* suspend the src before migrating to dest. */ bool suspend_me; + /* + * Format string for the main memory backend, containing one %s where the + * size is plugged in. If omitted, "-m %s" is used. + */ + const char *memory_backend; } MigrateStart; /* @@ -727,6 +732,7 @@ static int test_migrate_start(QTestState **from, QTestState **to, const char *memory_size; const char *machine_alias, *machine_opts = ""; g_autofree char *machine = NULL; + g_autofree char *memory_backend = NULL; if (args->use_shmem) { if (!g_file_test("/dev/shm", G_FILE_TEST_IS_DIR)) { @@ -802,6 +808,12 @@ static int test_migrate_start(QTestState **from, QTestState **to, memory_size, shmem_path); } + if (args->memory_backend) { + memory_backend = g_strdup_printf(args->memory_backend, memory_size); + } else { + memory_backend = g_strdup_printf("-m %s ", memory_size); + } + if (args->use_dirty_ring) { kvm_opts = ",dirty-ring-size=4096"; } @@ -820,12 +832,12 @@ static int test_migrate_start(QTestState **from, QTestState **to, cmd_source = g_strdup_printf("-accel kvm%s -accel tcg " "-machine %s,%s " "-name source,debug-threads=on " - "-m %s " + "%s " "-serial file:%s/src_serial " "%s %s %s %s %s", kvm_opts ? kvm_opts : "", machine, machine_opts, - memory_size, tmpfs, + memory_backend, tmpfs, arch_opts ? arch_opts : "", arch_source ? arch_source : "", shmem_opts ? shmem_opts : "", @@ -841,13 +853,13 @@ static int test_migrate_start(QTestState **from, QTestState **to, cmd_target = g_strdup_printf("-accel kvm%s -accel tcg " "-machine %s,%s " "-name target,debug-threads=on " - "-m %s " + "%s " "-serial file:%s/dest_serial " "-incoming %s " "%s %s %s %s %s", kvm_opts ? kvm_opts : "", machine, machine_opts, - memory_size, tmpfs, uri, + memory_backend, tmpfs, uri, arch_opts ? arch_opts : "", arch_target ? arch_target : "", shmem_opts ? shmem_opts : "", From patchwork Fri Nov 1 13:47:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859428 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3B69CE6B27D for ; Fri, 1 Nov 2024 13:50:02 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s18-0004jN-8P; Fri, 01 Nov 2024 09:48:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s15-0004Up-Vk for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:44 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s14-0002KI-4B for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:43 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1Dffpl023740; Fri, 1 Nov 2024 13:48:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=IU/Mn09d2+EbLtQ8eeojMxP360v0lOYoa9uwDhEpKzo=; b= gxDyccqtlqyxC4PkpRt4Sc7PFTonit3xFpBj7WR2S5pp5DWuUVzaCGzrKgbTsDVu p0Ug7PXi5wvFULTr1k6EF4ji+FG46yx7wzJhACohEHVZAublACF3Dn4lbdOmRmbT KREPAJ7+MHW2TIaE7Sb9Ri/fcmFWv8ett/EdLkzlAcK2pfbxYQRoyMmCivI4NFo3 9LGEyhFqTK8tZobjFmD77SHTVJwlSQVGHufGaP048TE7XgKIm/lZIrJ/X+e/Ywy3 fiJ6fhn3WIT8S7J1jW88/j+YKotNq09pwuBjygIVzKO1IfXXsr9bH0QsH9oyhPYT 6d6wynIjQOQ3DmD4lFgl+w== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grc94a3m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:36 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1CJBnP010117; Fri, 1 Nov 2024 13:48:36 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptmu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:35 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuOQ006031; Fri, 1 Nov 2024 13:48:35 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-14; Fri, 01 Nov 2024 13:48:35 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 13/16] tests/qtest: defer connection Date: Fri, 1 Nov 2024 06:47:52 -0700 Message-Id: <1730468875-249970-14-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-GUID: vSx5fPTEJJZgkTCV3NrWmgsFaBO3aNrB X-Proofpoint-ORIG-GUID: vSx5fPTEJJZgkTCV3NrWmgsFaBO3aNrB Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add an option to defer making the connecting to the monitor and qtest sockets when calling qtest_init_with_env. The client makes the connection later by calling qtest_connect_deferred and qtest_qmp_handshake. Signed-off-by: Steve Sistare --- tests/qtest/libqtest.c | 69 +++++++++++++++++++++++++++++--------------- tests/qtest/libqtest.h | 19 +++++++++++- tests/qtest/migration-test.c | 4 +-- 3 files changed, 65 insertions(+), 27 deletions(-) diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c index 9d07de1..95408fb 100644 --- a/tests/qtest/libqtest.c +++ b/tests/qtest/libqtest.c @@ -75,6 +75,8 @@ struct QTestState { int fd; int qmp_fd; + int sock; + int qmpsock; pid_t qemu_pid; /* our child QEMU process */ int wstatus; #ifdef _WIN32 @@ -443,7 +445,8 @@ static QTestState *G_GNUC_PRINTF(2, 3) qtest_spawn_qemu(const char *qemu_bin, } static QTestState *qtest_init_internal(const char *qemu_bin, - const char *extra_args) + const char *extra_args, + bool defer_connect) { QTestState *s; int sock, qmpsock, i; @@ -485,22 +488,17 @@ static QTestState *qtest_init_internal(const char *qemu_bin, qtest_client_set_rx_handler(s, qtest_client_socket_recv_line); qtest_client_set_tx_handler(s, qtest_client_socket_send); - s->fd = socket_accept(sock); - if (s->fd >= 0) { - s->qmp_fd = socket_accept(qmpsock); - } - unlink(socket_path); - unlink(qmp_socket_path); - g_free(socket_path); - g_free(qmp_socket_path); - - g_assert(s->fd >= 0 && s->qmp_fd >= 0); - s->rx = g_string_new(""); for (i = 0; i < MAX_IRQ; i++) { s->irq_level[i] = false; } + s->sock = sock; + s->qmpsock = qmpsock; + if (!defer_connect) { + qtest_connect_deferred(s); + } + /* * Stopping QEMU for debugging is not supported on Windows. * @@ -515,34 +513,57 @@ static QTestState *qtest_init_internal(const char *qemu_bin, } #endif + return s; +} + +void qtest_connect_deferred(QTestState *s) +{ + g_autofree gchar *socket_path = NULL; + g_autofree gchar *qmp_socket_path = NULL; + + socket_path = g_strdup_printf("%s/qtest-%d.sock", + g_get_tmp_dir(), getpid()); + qmp_socket_path = g_strdup_printf("%s/qtest-%d.qmp", + g_get_tmp_dir(), getpid()); + + s->fd = socket_accept(s->sock); + if (s->fd >= 0) { + s->qmp_fd = socket_accept(s->qmpsock); + } + unlink(socket_path); + unlink(qmp_socket_path); + g_assert(s->fd >= 0 && s->qmp_fd >= 0); /* ask endianness of the target */ - s->big_endian = qtest_query_target_endianness(s); - - return s; } QTestState *qtest_init_without_qmp_handshake(const char *extra_args) { - return qtest_init_internal(qtest_qemu_binary(NULL), extra_args); + return qtest_init_internal(qtest_qemu_binary(NULL), extra_args, false); } -QTestState *qtest_init_with_env(const char *var, const char *extra_args) +void qtest_qmp_handshake(QTestState *s) { - QTestState *s = qtest_init_internal(qtest_qemu_binary(var), extra_args); - QDict *greeting; - /* Read the QMP greeting and then do the handshake */ - greeting = qtest_qmp_receive(s); + QDict *greeting = qtest_qmp_receive(s); qobject_unref(greeting); qobject_unref(qtest_qmp(s, "{ 'execute': 'qmp_capabilities' }")); +} +QTestState *qtest_init_with_env(const char *var, const char *extra_args, + bool defer_connect) +{ + QTestState *s = qtest_init_internal(qtest_qemu_binary(var), extra_args, + defer_connect); + if (!defer_connect) { + qtest_qmp_handshake(s); + } return s; } QTestState *qtest_init(const char *extra_args) { - return qtest_init_with_env(NULL, extra_args); + return qtest_init_with_env(NULL, extra_args, false); } QTestState *qtest_vinitf(const char *fmt, va_list ap) @@ -1523,7 +1544,7 @@ static struct MachInfo *qtest_get_machines(const char *var) silence_spawn_log = !g_test_verbose(); - qts = qtest_init_with_env(qemu_var, "-machine none"); + qts = qtest_init_with_env(qemu_var, "-machine none", false); response = qtest_qmp(qts, "{ 'execute': 'query-machines' }"); g_assert(response); list = qdict_get_qlist(response, "return"); @@ -1578,7 +1599,7 @@ static struct CpuModel *qtest_get_cpu_models(void) silence_spawn_log = !g_test_verbose(); - qts = qtest_init_with_env(NULL, "-machine none"); + qts = qtest_init_with_env(NULL, "-machine none", false); response = qtest_qmp(qts, "{ 'execute': 'query-cpu-definitions' }"); g_assert(response); list = qdict_get_qlist(response, "return"); diff --git a/tests/qtest/libqtest.h b/tests/qtest/libqtest.h index beb96b1..db76f2c 100644 --- a/tests/qtest/libqtest.h +++ b/tests/qtest/libqtest.h @@ -60,13 +60,15 @@ QTestState *qtest_init(const char *extra_args); * @var: Environment variable from where to take the QEMU binary * @extra_args: Other arguments to pass to QEMU. CAUTION: these * arguments are subject to word splitting and shell evaluation. + * @defer_connect: do not connect to qemu monitor and qtest socket. * * Like qtest_init(), but use a different environment variable for the * QEMU binary. * * Returns: #QTestState instance. */ -QTestState *qtest_init_with_env(const char *var, const char *extra_args); +QTestState *qtest_init_with_env(const char *var, const char *extra_args, + bool defer_connect); /** * qtest_init_without_qmp_handshake: @@ -78,6 +80,21 @@ QTestState *qtest_init_with_env(const char *var, const char *extra_args); QTestState *qtest_init_without_qmp_handshake(const char *extra_args); /** + * qtest_connect_deferred: + * @s: #QTestState instance to connect + * Connect to qemu monitor and qtest socket, after deferring them in + * qtest_init_with_env. Does not handshake with the monitor. + */ +void qtest_connect_deferred(QTestState *s); + +/** + * qtest_qmp_handshake: + * @s: #QTestState instance to operate on. + * Perform handshake after connecting to qemu monitor. + */ +void qtest_qmp_handshake(QTestState *s); + +/** * qtest_init_with_serial: * @extra_args: other arguments to pass to QEMU. CAUTION: these * arguments are subject to word splitting and shell evaluation. diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index a008316..d359b10 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -844,7 +844,7 @@ static int test_migrate_start(QTestState **from, QTestState **to, args->opts_source ? args->opts_source : "", ignore_stderr); if (!args->only_target) { - *from = qtest_init_with_env(QEMU_ENV_SRC, cmd_source); + *from = qtest_init_with_env(QEMU_ENV_SRC, cmd_source, false); qtest_qmp_set_event_callback(*from, migrate_watch_for_events, &src_state); @@ -865,7 +865,7 @@ static int test_migrate_start(QTestState **from, QTestState **to, shmem_opts ? shmem_opts : "", args->opts_target ? args->opts_target : "", ignore_stderr); - *to = qtest_init_with_env(QEMU_ENV_DST, cmd_target); + *to = qtest_init_with_env(QEMU_ENV_DST, cmd_target, false); qtest_qmp_set_event_callback(*to, migrate_watch_for_events, &dst_state); From patchwork Fri Nov 1 13:47:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859427 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69DC4E6B27C for ; Fri, 1 Nov 2024 13:50:01 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s17-0004ZU-3A; Fri, 01 Nov 2024 09:48:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s13-0004Mx-N5 for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:41 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s11-0002JF-WD for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:41 -0400 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1Dfepx002452; Fri, 1 Nov 2024 13:48:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=r15lkJcNKH6RmQ6mqc+evCZ2vhqf3qbkcpilJbyVa+c=; b= aUVETD6fTYTRQFgSNEBIZNfgDQhGd1OGBX3+ROF8Ury9gi+87hYk5I3zfOBsdBwf yvCpk7wl0cgg+EgiGjI8cG1+F0larzJrWaoWByE/Zf/0oecOg2qlPlrfPxDpIqI1 uo6dltAAf42s7PVc4nM/SCmATE9rIb5nDXCwYX0YKK12lpOT0qUxojsBuFzpUpDc Kzq8FXh9najLHZnX525162EvKSEFj6WUlxyvoskwlzm+bZqmVB+g5SlNTje580Yn TDIuz9zYULySvbb0nkk3RfQRcfQPKvlDPytSUK8qQbr5QyIoONYvQtcYl/OLFI+i Dmrcd+OtdY/SLafpW1+nTw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grdqm6n6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:37 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1ClLur010140; Fri, 1 Nov 2024 13:48:36 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptn3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:36 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuOS006031; Fri, 1 Nov 2024 13:48:36 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-15; Fri, 01 Nov 2024 13:48:35 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 14/16] tests/migration-test: defer connection Date: Fri, 1 Nov 2024 06:47:53 -0700 Message-Id: <1730468875-249970-15-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-GUID: WIfudnOXxKJ3CY9EInXwU622yb_LgOWr X-Proofpoint-ORIG-GUID: WIfudnOXxKJ3CY9EInXwU622yb_LgOWr Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add an option to defer connection to the target monitor, needed by the cpr-transfer test. Signed-off-by: Steve Sistare Reviewed-by: Fabiano Rosas --- tests/qtest/migration-test.c | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index d359b10..dfeb8d2 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -614,6 +614,9 @@ typedef struct { * size is plugged in. If omitted, "-m %s" is used. */ const char *memory_backend; + + /* Do not connect to target monitor and qtest sockets in qtest_init */ + bool defer_target_connect; } MigrateStart; /* @@ -733,6 +736,7 @@ static int test_migrate_start(QTestState **from, QTestState **to, const char *machine_alias, *machine_opts = ""; g_autofree char *machine = NULL; g_autofree char *memory_backend = NULL; + const char *events; if (args->use_shmem) { if (!g_file_test("/dev/shm", G_FILE_TEST_IS_DIR)) { @@ -850,22 +854,31 @@ static int test_migrate_start(QTestState **from, QTestState **to, &src_state); } + /* + * If the monitor connection is deferred, enable events on the command line + * so none are missed. This is for testing only, do not set migration + * options like this in general. + */ + events = args->defer_target_connect ? "-global migration.x-events=on" : ""; + cmd_target = g_strdup_printf("-accel kvm%s -accel tcg " "-machine %s,%s " "-name target,debug-threads=on " "%s " "-serial file:%s/dest_serial " "-incoming %s " - "%s %s %s %s %s", + "%s %s %s %s %s %s", kvm_opts ? kvm_opts : "", machine, machine_opts, memory_backend, tmpfs, uri, + events, arch_opts ? arch_opts : "", arch_target ? arch_target : "", shmem_opts ? shmem_opts : "", args->opts_target ? args->opts_target : "", ignore_stderr); - *to = qtest_init_with_env(QEMU_ENV_DST, cmd_target, false); + *to = qtest_init_with_env(QEMU_ENV_DST, cmd_target, + args->defer_target_connect); qtest_qmp_set_event_callback(*to, migrate_watch_for_events, &dst_state); @@ -883,7 +896,9 @@ static int test_migrate_start(QTestState **from, QTestState **to, * to mimic as closer as that. */ migrate_set_capability(*from, "events", true); - migrate_set_capability(*to, "events", true); + if (!args->defer_target_connect) { + migrate_set_capability(*to, "events", true); + } return 0; } @@ -1751,6 +1766,11 @@ static void test_precopy_common(MigrateCommon *args) migrate_qmp(from, to, args->connect_uri, args->connect_channels, "{}"); + if (args->start.defer_target_connect) { + qtest_connect_deferred(to); + qtest_qmp_handshake(to); + } + if (args->result != MIG_TEST_SUCCEED) { bool allow_active = args->result == MIG_TEST_FAIL; wait_for_migration_fail(from, allow_active); From patchwork Fri Nov 1 13:47:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859426 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 71C73E6B278 for ; Fri, 1 Nov 2024 13:50:00 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s15-0004TK-Lw; Fri, 01 Nov 2024 09:48:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s15-0004PE-01 for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:43 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s12-0002Ji-S4 for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:42 -0400 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1Dfe93002483; Fri, 1 Nov 2024 13:48:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=VOkxPPuTOnYY8RIIFWn2R7GdnHJAr8Vt81EQDGD0vA4=; b= G6CXDMhpOgNt9nXvaWaFUdlPNxmGL5iMMEpeIwO4FYNDR+Mq5h2rBW5JnUzI8Rdb FlWq4Ozn984vBYVWLmCrpS2HTCKZ5vNZM6KvgBLL4RanktPoQ+MGOqmplD+Qh9ji LyoZ9RhRoG4Ss7kpcS2ISah+CM9WvnDSFotM/jxgDoZBsQ1vUOCvEs5lkwgQsAr7 XrIrffWIvBmrtBIeD8+0Lw8i6mFTONKVOWkDxeXL5ltcQHzUfJbMdMs7a9rqWLqI e+bWtaZnVX3sKshj0yA20hXPJ4/kGFw3OpkosYv6tpi3dH1w+mom1w3CKZJQscnX tc0089e/DpQk/gbGFIQE3Q== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grdqm6n7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:38 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1C3RTS010191; Fri, 1 Nov 2024 13:48:37 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptn9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:37 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuOU006031; Fri, 1 Nov 2024 13:48:36 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-16; Fri, 01 Nov 2024 13:48:36 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 15/16] migration-test: cpr-transfer Date: Fri, 1 Nov 2024 06:47:54 -0700 Message-Id: <1730468875-249970-16-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-GUID: E49JGIyCjDAsu-LYjq3voq3xxlaeCMRD X-Proofpoint-ORIG-GUID: E49JGIyCjDAsu-LYjq3voq3xxlaeCMRD Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add a migration test for cpr-transfer mode. Defer the connection to the target monitor, else the test hangs because in cpr-transfer mode QEMU does not listen for monitor connections until we send the migrate command to source QEMU. To test -incoming defer, send a migrate incoming command to the target, after sending the migrate command to the source, as required by cpr-transfer mode. Signed-off-by: Steve Sistare --- tests/qtest/migration-test.c | 59 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index dfeb8d2..e364e9b 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1769,6 +1769,9 @@ static void test_precopy_common(MigrateCommon *args) if (args->start.defer_target_connect) { qtest_connect_deferred(to); qtest_qmp_handshake(to); + if (!strcmp(args->listen_uri, "defer")) { + migrate_incoming_qmp(to, args->connect_uri, "{}"); + } } if (args->result != MIG_TEST_SUCCEED) { @@ -2413,6 +2416,58 @@ static void test_multifd_file_mapped_ram_fdset_dio(void) } #endif /* !_WIN32 */ +static char *get_cpr_uri(void) +{ + return g_strdup_printf("unix:%s/cpr.sock", tmpfs); +} + +static void *test_mode_transfer_start(QTestState *from, QTestState *to) +{ + g_autofree char *cpr_uri = get_cpr_uri(); + + migrate_set_parameter_str(from, "mode", "cpr-transfer"); + migrate_set_parameter_str(from, "cpr-uri", cpr_uri); + return NULL; +} + +/* + * cpr-transfer mode cannot use the target monitor prior to starting the + * migration, and cannot connect synchronously to the monitor, so defer + * the target connection. + */ +static void test_mode_transfer_common(bool incoming_defer) +{ + g_autofree char *cpr_uri = get_cpr_uri(); + g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs); + + const char *opts = "-machine anon-alloc=memfd -nodefaults"; + g_autofree char *opts_target = g_strdup_printf("-cpr-uri %s %s", + cpr_uri, opts); + + MigrateCommon args = { + .start.opts_source = opts, + .start.opts_target = opts_target, + .start.defer_target_connect = true, + .start.memory_backend = "-object memory-backend-memfd,id=pc.ram,size=%s" + " -machine memory-backend=pc.ram", + .listen_uri = incoming_defer ? "defer" : uri, + .connect_uri = uri, + .start_hook = test_mode_transfer_start, + }; + + test_precopy_common(&args); +} + +static void test_mode_transfer(void) +{ + test_mode_transfer_common(NULL); +} + +static void test_mode_transfer_defer(void) +{ + test_mode_transfer_common(true); +} + static void test_precopy_tcp_plain(void) { MigrateCommon args = { @@ -3866,6 +3921,10 @@ int main(int argc, char **argv) migration_test_add("/migration/mode/reboot", test_mode_reboot); } + migration_test_add("/migration/mode/transfer", test_mode_transfer); + migration_test_add("/migration/mode/transfer/defer", + test_mode_transfer_defer); + migration_test_add("/migration/precopy/file/mapped-ram", test_precopy_file_mapped_ram); migration_test_add("/migration/precopy/file/mapped-ram/live", From patchwork Fri Nov 1 13:47:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13859429 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9FF7AE6B278 for ; Fri, 1 Nov 2024 13:50:05 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t6s18-0004hH-8Y; Fri, 01 Nov 2024 09:48:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s15-0004QL-8B for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:43 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t6s13-0002Ju-50 for qemu-devel@nongnu.org; Fri, 01 Nov 2024 09:48:42 -0400 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1DfekZ010313; Fri, 1 Nov 2024 13:48:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=IGuir1SixE9HwG3JNgK0+QzvhULJSvM7M0POG8w4yRM=; b= dZ/q2pQQEY7/vwb8RIio5kOe4XqBdsf5jpXSZesofvoJGKmiX1rf7DCTxA1CTNtf WpLqGZ68Hdhh5mZTVDB4ucDYkktAFT2TmYrWXScerkuteuuZ325Qs9+rEKFi5tLi 5IWsuhDotROMtJI6bHY2GpYew/SivRVa/RF6UTSvYKHffk2RoRx8ZINXxU1XGzHD vqSdIgCPEyFByLd6f7/oZF5Ii5Y8haMv8LSyEqaZ01lJlkaSawBPWlH7mVfuxpgR /5CCDIjiU0/TQRBQ8236hfGfanPKTDcyHjQgMvQrOVafLloO0vBpE9pfOhsRnAXo eqWT9c3/3ujwicd571l/fA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42grc247h2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:38 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A1BFOf2010296; Fri, 1 Nov 2024 13:48:37 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42hn91ptnf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 01 Nov 2024 13:48:37 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A1DhuOW006031; Fri, 1 Nov 2024 13:48:37 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 42hn91pt43-17; Fri, 01 Nov 2024 13:48:37 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V3 16/16] migration: cpr-transfer documentation Date: Fri, 1 Nov 2024 06:47:55 -0700 Message-Id: <1730468875-249970-17-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> References: <1730468875-249970-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-01_08,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411010100 X-Proofpoint-GUID: lMenP9T4wO4MwAESgNqMNRFPs0157g4X X-Proofpoint-ORIG-GUID: lMenP9T4wO4MwAESgNqMNRFPs0157g4X Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Steve Sistare --- docs/devel/migration/CPR.rst | 144 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 142 insertions(+), 2 deletions(-) diff --git a/docs/devel/migration/CPR.rst b/docs/devel/migration/CPR.rst index 63c3647..732d5a6 100644 --- a/docs/devel/migration/CPR.rst +++ b/docs/devel/migration/CPR.rst @@ -5,7 +5,7 @@ CPR is the umbrella name for a set of migration modes in which the VM is migrated to a new QEMU instance on the same host. It is intended for use when the goal is to update host software components that run the VM, such as QEMU or even the host kernel. At this time, -cpr-reboot is the only available mode. +the cpr-reboot and cpr-transfer modes are available. Because QEMU is restarted on the same host, with access to the same local devices, CPR is allowed in certain cases where normal migration @@ -53,7 +53,7 @@ RAM is copied to the migration URI. Outgoing: * Set the migration mode parameter to ``cpr-reboot``. * Set the ``x-ignore-shared`` capability if desired. - * Issue the ``migrate`` command. It is recommended the the URI be a + * Issue the ``migrate`` command. It is recommended the URI be a ``file`` type, but one can use other types such as ``exec``, provided the command captures all the data from the outgoing side, and provides all the data to the incoming side. @@ -145,3 +145,143 @@ Caveats cpr-reboot mode may not be used with postcopy, background-snapshot, or COLO. + +cpr-transfer mode +----------------- + +This mode allows the user to transfer a guest to a new QEMU instance +on the same host with minimal guest pause time, by preserving guest +RAM in place, albeit with new virtual addresses in new QEMU. + +The user starts new QEMU on the same host as old QEMU, with the +same arguments as old QEMU, plus the ``-incoming option``. The user +issues the migrate command to old QEMU, which stops the VM, saves +state to the migration channels, and enters the postmigrate state. +Execution resumes in new QEMU. + +This mode requires a second migration channel, specified by the +``cpr-uri`` migration property on the outgoing side, and by the +``cpr-uri`` QEMU command-line option on the incoming side. The +channel must be a type, such as unix socket, that supports SCM_RIGHTS. + +Usage +^^^^^ + +Memory backend objects must have the ``share=on`` attribute. + +The VM must be started with the ``-machine anon-alloc=memfd`` +option. This causes implicit RAM blocks (those not described by +a memory-backend object) to be allocated by mmap'ing a memfd. +Examples include VGA and ROM. + +Outgoing: + * Set the migration mode parameter to ``cpr-transfer``. + * Set the ``cpr-uri`` parameter. It must be a ``unix`` type. + * Issue the ``migrate`` command. + +Incoming: + * Start new QEMU with the ``-incoming`` and ``-cpr-uri`` options. + * If the VM was running when the outgoing ``migrate`` command was + issued, then QEMU automatically resumes VM execution. + +Caveats +^^^^^^^ + +cpr-transfer mode may not be used with postcopy, background-snapshot, +or COLO. + +memory-backend-epc and memory-backend-ram are not supported. + +The incoming migration channel cannot be a file type. + +If the incoming migration channel is a tcp type, then the port cannot +be 0 (meaning dynamically choose a port). + +When using ``-incoming defer``, you must issue the migrate command to +old QEMU before issuing any monitor commands to new QEMU, because new +QEMU blocks waiting to read from cpr-uri before starting its monitor, +and old QEMU does not write to cpr-uri until the migrate command is +issued. However, new QEMU does not open and read the migration stream +until you issue the migrate incoming command. + +Example 1: incoming channel +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In these examples, we simply restart the same version of QEMU, but +in a real scenario one would start new QEMU on the incoming side. +Note that new QEMU does not print the monitor prompt until old QEMU +has issued the migrate command. + +:: + + Outgoing: Incoming: + + # qemu-kvm -monitor stdio + -object memory-backend-file,id=ram0,size=4G, + mem-path=/dev/shm/ram0,share=on -m 4G + -machine anon-alloc=memfd' + ... + # qemu-kvm -incoming tcp:0:44444 + -cpr-uri unix:cpr.sock + ... + QEMU 9.2.50 monitor + (qemu) info status + VM status: running + (qemu) migrate_set_parameter mode cpr-transfer + (qemu) migrate_set_parameter cpr-uri unix:cpr.sock + (qemu) migrate -d tcp:0:44444 + + QEMU 9.2.50 monitor + (qemu) info status + VM status: running + + (qemu) info status + VM status: paused (postmigrate) + + +Example 2: incoming defer +^^^^^^^^^^^^^^^^^^^^^^^^^ + +This example uses ``-incoming defer`` to hot plug a device before +accepting the migration stream. Again note you must issue the +migrate command to old QEMU before you can issue any monitor +commands to new QEMU. + + +:: + + Outgoing: Incoming: + + # qemu-kvm -monitor stdio + -object memory-backend-file,id=ram0,size=4G, + mem-path=/dev/shm/ram0,share=on -m 4G + -machine anon-alloc=memfd' + ... + # qemu-kvm -incoming defer + -cpr-uri unix:cpr.sock + ... + QEMU 9.2.50 monitor + (qemu) device_add pcie-root-port + (qemu) migrate_set_parameter mode cpr-transfer + (qemu) migrate_set_parameter cpr-uri unix:cpr.sock + (qemu) migrate -d tcp:0:44444 + + QEMU 9.2.50 monitor + (qemu) info status + VM status: paused (inmigrate) + (qemu) device_add pcie-root-port + (qemu) migrate_incoming tcp:0:44444 + (qemu) info status + VM status: running + + (qemu) info status + VM status: paused (postmigrate) + +Futures +^^^^^^^ + +cpr-transfer mode is based on a capability to transfer open file +descriptors from old to new QEMU. In the future, descriptors for +vfio, iommufd, vhost, and char devices could be transferred, +preserving those devices and their kernel state without interruption, +even if they do not explicitly support live migration.