From patchwork Mon Dec 2 13:19:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 58355D7831F for ; Mon, 2 Dec 2024 13:34:58 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Zh-0004L1-JR; Mon, 02 Dec 2024 08:34:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Li-0007yB-UQ for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:37 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lc-0005g6-DF for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:23 -0500 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2DAgQp023933; Mon, 2 Dec 2024 13:20:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=IiV/SS9De5zI5JVcAHpFXS2iXMkHLB4glwh8SI5UR9o=; b= Pft8Cc/J3kvPM5AlkRGQBxc7HAXBBhsm/lf+JfrprmclX58bU7eak/ZoDXO5jfLD Yy5sIkxriEReK79gU9LA7Y2AYxb1olYIfAK5WE0AM0JaOmgl6QwKuwKfnvmWlDSj Ie+PgOerKgvoXT1hdv+d90UcqVAqkHUCizzW7rTNBjOMLRs6h9lC5AZaXo58cavg 8ZoZsChLVJbOWNBOWowa8XSsjqww3KYxGxGgChernzLZBz2Rsl9+fgOrkQIQrZYK VNW4Wa8jhd6xlkQFGcxfjxEO7tKxzmoJCs4BXGk7ng3NO8U+06jWXM21Sfyw7fgT Aj3lGErt9D/Fn/UA/w21Tg== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437trbk4ab-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:14 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2C0hvX032882; Mon, 2 Dec 2024 13:20:13 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtjv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:13 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEZ032806; Mon, 2 Dec 2024 13:20:13 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-2; Mon, 02 Dec 2024 13:20:13 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 01/19] backends/hostmem-shm: factor out allocation of "anonymous shared memory with an fd" Date: Mon, 2 Dec 2024 05:19:53 -0800 Message-Id: <1733145611-62315-2-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-GUID: 6wrt4RFUnpY-Ih81k-utkKCxJChNKThk X-Proofpoint-ORIG-GUID: 6wrt4RFUnpY-Ih81k-utkKCxJChNKThk Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Let's factor it out so we can reuse it. Signed-off-by: David Hildenbrand --- backends/hostmem-shm.c | 45 ++++----------------------------------- include/qemu/osdep.h | 2 ++ meson.build | 8 +++++-- util/oslib-posix.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++ util/oslib-win32.c | 11 ++++++++++ 5 files changed, 81 insertions(+), 43 deletions(-) diff --git a/backends/hostmem-shm.c b/backends/hostmem-shm.c index 374edc3..837b9f1 100644 --- a/backends/hostmem-shm.c +++ b/backends/hostmem-shm.c @@ -25,11 +25,9 @@ struct HostMemoryBackendShm { static bool shm_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) { - g_autoptr(GString) shm_name = g_string_new(NULL); g_autofree char *backend_name = NULL; uint32_t ram_flags; - int fd, oflag; - mode_t mode; + int fd; if (!backend->size) { error_setg(errp, "can't create shm backend with size 0"); @@ -41,48 +39,13 @@ shm_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) return false; } - /* - * Let's use `mode = 0` because we don't want other processes to open our - * memory unless we share the file descriptor with them. - */ - mode = 0; - oflag = O_RDWR | O_CREAT | O_EXCL; - backend_name = host_memory_backend_get_name(backend); - - /* - * Some operating systems allow creating anonymous POSIX shared memory - * objects (e.g. FreeBSD provides the SHM_ANON constant), but this is not - * defined by POSIX, so let's create a unique name. - * - * From Linux's shm_open(3) man-page: - * For portable use, a shared memory object should be identified - * by a name of the form /somename;" - */ - g_string_printf(shm_name, "/qemu-" FMT_pid "-shm-%s", getpid(), - backend_name); - - fd = shm_open(shm_name->str, oflag, mode); + fd = qemu_shm_alloc(backend->size, errp); if (fd < 0) { - error_setg_errno(errp, errno, - "failed to create POSIX shared memory"); - return false; - } - - /* - * We have the file descriptor, so we no longer need to expose the - * POSIX shared memory object. However it will remain allocated as long as - * there are file descriptors pointing to it. - */ - shm_unlink(shm_name->str); - - if (ftruncate(fd, backend->size) == -1) { - error_setg_errno(errp, errno, - "failed to resize POSIX shared memory to %" PRIu64, - backend->size); - close(fd); return false; } + /* Let's do the same as memory-backend-ram,share=on would do. */ + backend_name = host_memory_backend_get_name(backend); ram_flags = RAM_SHARED; ram_flags |= backend->reserve ? 0 : RAM_NORESERVE; diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h index fdff07f..e2a0e15 100644 --- a/include/qemu/osdep.h +++ b/include/qemu/osdep.h @@ -509,6 +509,8 @@ int qemu_daemon(int nochdir, int noclose); void *qemu_anon_ram_alloc(size_t size, uint64_t *align, bool shared, bool noreserve); void qemu_anon_ram_free(void *ptr, size_t size); +int qemu_shm_alloc(size_t size, Error **errp); +bool qemu_shm_available(void); #ifdef _WIN32 #define HAVE_CHARDEV_SERIAL 1 diff --git a/meson.build b/meson.build index b09bfb1..4f58783 100644 --- a/meson.build +++ b/meson.build @@ -3706,9 +3706,13 @@ libqemuutil = static_library('qemuutil', build_by_default: false, sources: util_ss.sources() + stub_ss.sources() + genh, dependencies: [util_ss.dependencies(), libm, threads, glib, socket, malloc]) +qemuutil_deps = [event_loop_base] +if host_os != 'windows' + qemuutil_deps += [rt] +endif qemuutil = declare_dependency(link_with: libqemuutil, sources: genh + version_res, - dependencies: [event_loop_base]) + dependencies: qemuutil_deps) if have_system or have_user decodetree = generator(find_program('scripts/decodetree.py'), @@ -4361,7 +4365,7 @@ if have_tools subdir('contrib/elf2dmp') executable('qemu-edid', files('qemu-edid.c', 'hw/display/edid-generate.c'), - dependencies: qemuutil, + dependencies: [qemuutil, rt], install: true) if have_vhost_user diff --git a/util/oslib-posix.c b/util/oslib-posix.c index 11b35e4..f8c3724 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -931,3 +931,61 @@ void qemu_close_all_open_fd(const int *skip, unsigned int nskip) qemu_close_all_open_fd_fallback(skip, nskip, open_max); } } + +bool qemu_shm_available(void) +{ + return true; +} + +int qemu_shm_alloc(size_t size, Error **errp) +{ + g_autoptr(GString) shm_name = g_string_new(NULL); + int fd, oflag, cur_sequence; + static int sequence; + mode_t mode; + + cur_sequence = qatomic_fetch_inc(&sequence); + + /* + * Let's use `mode = 0` because we don't want other processes to open our + * memory unless we share the file descriptor with them. + */ + mode = 0; + oflag = O_RDWR | O_CREAT | O_EXCL; + + /* + * Some operating systems allow creating anonymous POSIX shared memory + * objects (e.g. FreeBSD provides the SHM_ANON constant), but this is not + * defined by POSIX, so let's create a unique name. + * + * From Linux's shm_open(3) man-page: + * For portable use, a shared memory object should be identified + * by a name of the form /somename;" + */ + g_string_printf(shm_name, "/qemu-" FMT_pid "-shm-%d", getpid(), + cur_sequence); + + fd = shm_open(shm_name->str, oflag, mode); + if (fd < 0) { + error_setg_errno(errp, errno, + "failed to create POSIX shared memory"); + return -1; + } + + /* + * We have the file descriptor, so we no longer need to expose the + * POSIX shared memory object. However it will remain allocated as long as + * there are file descriptors pointing to it. + */ + shm_unlink(shm_name->str); + + if (ftruncate(fd, size) == -1) { + error_setg_errno(errp, errno, + "failed to resize POSIX shared memory to %" PRIu64, + size); + close(fd); + return -1; + } + + return fd; +} diff --git a/util/oslib-win32.c b/util/oslib-win32.c index b623830..aef5779 100644 --- a/util/oslib-win32.c +++ b/util/oslib-win32.c @@ -877,3 +877,14 @@ void qemu_win32_map_free(void *ptr, HANDLE h, Error **errp) } CloseHandle(h); } + +bool qemu_shm_available(void) +{ + return false; +} + +int qemu_shm_alloc(size_t size, Error **errp) +{ + error_setg("Shared memory is not supported."); + return -1; +} From patchwork Mon Dec 2 13:19:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890682 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 59FFED78322 for ; Mon, 2 Dec 2024 13:43:19 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Zl-0004iO-NM; Mon, 02 Dec 2024 08:34:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Li-0007yC-Uf for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:37 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lc-0005gI-DT for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:23 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26XML9013787; Mon, 2 Dec 2024 13:20:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=KAIQZpXnGy6pVdwuvfo7Shh7bliaiotYtNJFyvqtlhM=; b= ouFiaGvmIbBMCYjUKCScmUKpvMuyzuCn6s2ICf8q95ZC6QQk/cyNPO15xrHPgMbg IB7tgg+kYGyXqrHyIDYtm4w7IuC7AnJkvXwD/OclnzrvarIfOd7SrCt7jaYdJ/p0 HDwXS9lLFeYmDKDYsMNCADcrmHLqgwx0ODhRj5523qZ6Vqpk/XOqO506JdZnl6Pp 9VT+3f75Ij34LVoiGX9mvoMP3JoOq8o0YcaBSynM8HK7uTjdvoSxC/R2ZO5w8xOg DddRiOM7Z//fBwGO0YMheXNqCaVLMah7Qsqv0NL3FHFIClqnOuDeSjCTk+eZibUp QbwS84yxN89kSEL6sT4Tiw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437s4c333t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:15 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2CEJ7T032829; Mon, 2 Dec 2024 13:20:14 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtke-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:14 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEb032806; Mon, 2 Dec 2024 13:20:13 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-3; Mon, 02 Dec 2024 13:20:13 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 02/19] physmem: fd-based shared memory Date: Mon, 2 Dec 2024 05:19:54 -0800 Message-Id: <1733145611-62315-3-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: _grpuIs03P-XbjUKAO7KEPnM8ztbpaEw X-Proofpoint-GUID: _grpuIs03P-XbjUKAO7KEPnM8ztbpaEw Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Create MAP_SHARED RAMBlocks by mmap'ing a file descriptor rather than using MAP_ANON, so the memory can be accessed in another process by passing and mmap'ing the fd. This will allow CPR to support memory-backend-ram and memory-backend-shm objects, provided the user creates them with share=on. Use memfd_create if available because it has no constraints. If not, use POSIX shm_open. However, this may fail if the shm mount size is too small, even if the system has free memory, so for backwards compatibility fall back to qemu_anon_ram_alloc/MAP_ANON on shm_open failure. For backwards compatibility on Windows, always use MAP_ANON. share=on has no purpose there, but the syntax is accepted, and must continue to work. Exclude Xen. Xen ignores RAM_SHARED and does its own allocation. Signed-off-by: Steve Sistare --- system/physmem.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++++---- system/trace-events | 1 + 2 files changed, 81 insertions(+), 5 deletions(-) diff --git a/system/physmem.c b/system/physmem.c index dc1db3a..b0c4b22 100644 --- a/system/physmem.c +++ b/system/physmem.c @@ -47,6 +47,7 @@ #include "qemu/qemu-print.h" #include "qemu/log.h" #include "qemu/memalign.h" +#include "qemu/memfd.h" #include "exec/memory.h" #include "exec/ioport.h" #include "sysemu/dma.h" @@ -2057,6 +2058,70 @@ RAMBlock *qemu_ram_alloc_from_file(ram_addr_t size, MemoryRegion *mr, } #endif +static bool qemu_memfd_available(void) +{ + static int has_memfd = -1; + + if (has_memfd < 0) { + has_memfd = qemu_memfd_check(0); + } + return has_memfd; +} + +/* + * We want anonymous shared memory, similar to MAP_SHARED|MAP_ANON, but + * some users want the fd. Allocate shm explicitly to get an fd. + */ +static bool qemu_ram_alloc_shared(RAMBlock *new_block, Error **errp) +{ + size_t max_length = new_block->max_length; + MemoryRegion *mr = new_block->mr; + const char *name = memory_region_name(mr); + int fd; + + if (qemu_memfd_available()) { + fd = qemu_memfd_create(name, max_length + mr->align, 0, 0, 0, errp); + if (fd < 0) { + return false; + } + } else if (!qemu_shm_available()) { + /* + * Backwards compatibility for Windows. The user may specify a + * memory backend with shared=on, and Windows ignores shared. + * Fall back to qemu_anon_ram_alloc. + */ + return true; + } else { + Error *local_err = NULL; + + fd = qemu_shm_alloc(max_length, &local_err); + if (fd < 0) { + /* + * Backwards compatibility in case the shm mount size is too small. + * Previous QEMU versions called qemu_anon_ram_alloc for anonymous + * shared memory, which could succeed. + */ + error_prepend(&local_err, + "Retrying using MAP_ANON|MAP_SHARED because: "); + warn_report_err(local_err); + return true; + } + } + + new_block->mr->align = QEMU_VMALLOC_ALIGN; + new_block->host = file_ram_alloc(new_block, max_length, fd, false, 0, errp); + + if (new_block->host) { + qemu_set_cloexec(fd); + new_block->fd = fd; + trace_qemu_ram_alloc_shared(name, max_length, fd, new_block->host); + return true; + } + + close(fd); + return false; +} + static RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size, void (*resized)(const char*, @@ -2089,13 +2154,23 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size, new_block->page_size = qemu_real_host_page_size(); new_block->host = host; new_block->flags = ram_flags; + + if (!host && !xen_enabled()) { + if ((new_block->flags & RAM_SHARED) && + !qemu_ram_alloc_shared(new_block, &local_err)) { + goto err; + } + } + ram_block_add(new_block, &local_err); - if (local_err) { - g_free(new_block); - error_propagate(errp, local_err); - return NULL; + if (!local_err) { + return new_block; } - return new_block; + +err: + g_free(new_block); + error_propagate(errp, local_err); + return NULL; } RAMBlock *qemu_ram_alloc_from_ptr(ram_addr_t size, void *host, diff --git a/system/trace-events b/system/trace-events index 5bbc3fb..831a60c 100644 --- a/system/trace-events +++ b/system/trace-events @@ -33,6 +33,7 @@ address_space_map(void *as, uint64_t addr, uint64_t len, bool is_write, uint32_t find_ram_offset(uint64_t size, uint64_t offset) "size: 0x%" PRIx64 " @ 0x%" PRIx64 find_ram_offset_loop(uint64_t size, uint64_t candidate, uint64_t offset, uint64_t next, uint64_t mingap) "trying size: 0x%" PRIx64 " @ 0x%" PRIx64 ", offset: 0x%" PRIx64" next: 0x%" PRIx64 " mingap: 0x%" PRIx64 ram_block_discard_range(const char *rbname, void *hva, size_t length, bool need_madvise, bool need_fallocate, int ret) "%s@%p + 0x%zx: madvise: %d fallocate: %d ret: %d" +qemu_ram_alloc_shared(const char *name, size_t max_length, int fd, void *host) "%s size %zu fd %d host %p" # cpus.c vm_stop_flush_all(int ret) "ret %d" From patchwork Mon Dec 2 13:19:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890678 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 371F0D78321 for ; Mon, 2 Dec 2024 13:41:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Yg-0001VC-B6; Mon, 02 Dec 2024 08:33:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lj-0007yK-2C for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:37 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Ld-0005gZ-TD for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:25 -0500 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2DFUQ8024793; Mon, 2 Dec 2024 13:20:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=uFwQmKSMvGGZhZgzNKndV+LvyQx4KW3Hsn+qjliCObU=; b= Hx6GsHssbCvxUf1zEs5v2EzLIx1JS/tsMYZ3R0xB3/FJXs/CxHJfEkX7HpnCPP5l PKnjaQtW79Bj9YSBAwPAKilP7hRMiaM6OLaVwaxmbpldTx7WCybpCnsxIK1NI6yS MJRYLBrjPmGktujHDZp3d6HCsgxc8Y2XuTOeUcJWmd8ndT1vnyd4+qcEf0OjNfo9 anpza6CjI0F+2fVdKc3LKnYcRqwZYZHH/8f7yj74evs6MPaDDbkLVuJerDESpH6u a8usLW0cezzwue8H1DwgzISOJvdoQTyvPuOVzuyBE4vEeuGkjl6WhxWy/id21bDM QQZV2Eyqe53bEd5CbYjifA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437sg23462-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:16 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2Bk0pV032875; Mon, 2 Dec 2024 13:20:15 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtky-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:15 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEd032806; Mon, 2 Dec 2024 13:20:14 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-4; Mon, 02 Dec 2024 13:20:14 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 03/19] memory: add RAM_PRIVATE Date: Mon, 2 Dec 2024 05:19:55 -0800 Message-Id: <1733145611-62315-4-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: EcAlZ-TZKFRiYuZVWHfvnzPXQR2rwsLS X-Proofpoint-GUID: EcAlZ-TZKFRiYuZVWHfvnzPXQR2rwsLS Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_FILL_THIS_FORM_SHORT=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Define the RAM_PRIVATE flag. In RAMBlock creation functions, if MAP_SHARED is 0 in the flags parameter, in a subsequent patch the implementation may still create a shared mapping if other conditions require it. Callers who specifically want a private mapping, eg for objects specified by the user, must pass RAM_PRIVATE. After RAMBlock creation, MAP_SHARED in the block's flags indicates whether the block is shared or private, and MAP_PRIVATE is omitted. Signed-off-by: Steve Sistare --- backends/hostmem-epc.c | 2 +- backends/hostmem-file.c | 2 +- backends/hostmem-memfd.c | 2 +- backends/hostmem-ram.c | 2 +- include/exec/memory.h | 10 ++++++++++ system/physmem.c | 15 ++++++++++++--- 6 files changed, 26 insertions(+), 7 deletions(-) diff --git a/backends/hostmem-epc.c b/backends/hostmem-epc.c index 6c024d6..3148ffa 100644 --- a/backends/hostmem-epc.c +++ b/backends/hostmem-epc.c @@ -36,7 +36,7 @@ sgx_epc_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) backend->aligned = true; name = object_get_canonical_path(OBJECT(backend)); - ram_flags = (backend->share ? RAM_SHARED : 0) | RAM_PROTECTED; + ram_flags = (backend->share ? RAM_SHARED : RAM_PRIVATE) | RAM_PROTECTED; return memory_region_init_ram_from_fd(&backend->mr, OBJECT(backend), name, backend->size, ram_flags, fd, 0, errp); } diff --git a/backends/hostmem-file.c b/backends/hostmem-file.c index 7e5072e..8cc10cb 100644 --- a/backends/hostmem-file.c +++ b/backends/hostmem-file.c @@ -82,7 +82,7 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) backend->aligned = true; name = host_memory_backend_get_name(backend); - ram_flags = backend->share ? RAM_SHARED : 0; + ram_flags = backend->share ? RAM_SHARED : RAM_PRIVATE; ram_flags |= fb->readonly ? RAM_READONLY_FD : 0; ram_flags |= fb->rom == ON_OFF_AUTO_ON ? RAM_READONLY : 0; ram_flags |= backend->reserve ? 0 : RAM_NORESERVE; diff --git a/backends/hostmem-memfd.c b/backends/hostmem-memfd.c index 9f890a8..1bcae4b 100644 --- a/backends/hostmem-memfd.c +++ b/backends/hostmem-memfd.c @@ -52,7 +52,7 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) backend->aligned = true; name = host_memory_backend_get_name(backend); - ram_flags = backend->share ? RAM_SHARED : 0; + ram_flags = backend->share ? RAM_SHARED : RAM_PRIVATE; ram_flags |= backend->reserve ? 0 : RAM_NORESERVE; ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0; return memory_region_init_ram_from_fd(&backend->mr, OBJECT(backend), name, diff --git a/backends/hostmem-ram.c b/backends/hostmem-ram.c index f7d81af..b380563 100644 --- a/backends/hostmem-ram.c +++ b/backends/hostmem-ram.c @@ -28,7 +28,7 @@ ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) } name = host_memory_backend_get_name(backend); - ram_flags = backend->share ? RAM_SHARED : 0; + ram_flags = backend->share ? RAM_SHARED : RAM_PRIVATE; ram_flags |= backend->reserve ? 0 : RAM_NORESERVE; ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0; return memory_region_init_ram_flags_nomigrate(&backend->mr, OBJECT(backend), diff --git a/include/exec/memory.h b/include/exec/memory.h index 9458e28..0ac21cc 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -246,6 +246,16 @@ typedef struct IOMMUTLBEvent { /* RAM can be private that has kvm guest memfd backend */ #define RAM_GUEST_MEMFD (1 << 12) +/* + * In RAMBlock creation functions, if MAP_SHARED is 0 in the flags parameter, + * the implementation may still create a shared mapping if other conditions + * require it. Callers who specifically want a private mapping, eg objects + * specified by the user, must pass RAM_PRIVATE. + * After RAMBlock creation, MAP_SHARED in the block's flags indicates whether + * the block is shared or private, and MAP_PRIVATE is omitted. + */ +#define RAM_PRIVATE (1 << 13) + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, hwaddr start, hwaddr end, diff --git a/system/physmem.c b/system/physmem.c index b0c4b22..36f0811 100644 --- a/system/physmem.c +++ b/system/physmem.c @@ -1949,7 +1949,11 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, { RAMBlock *new_block; Error *local_err = NULL; - int64_t file_size, file_align; + int64_t file_size, file_align, share_flags; + + share_flags = ram_flags & (RAM_PRIVATE | RAM_SHARED); + assert(share_flags != (RAM_SHARED | RAM_PRIVATE)); + ram_flags &= ~RAM_PRIVATE; /* Just support these ram flags by now. */ assert((ram_flags & ~(RAM_SHARED | RAM_PMEM | RAM_NORESERVE | @@ -2132,7 +2136,11 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size, { RAMBlock *new_block; Error *local_err = NULL; - int align; + int align, share_flags; + + share_flags = ram_flags & (RAM_PRIVATE | RAM_SHARED); + assert(share_flags != (RAM_SHARED | RAM_PRIVATE)); + ram_flags &= ~RAM_PRIVATE; assert((ram_flags & ~(RAM_SHARED | RAM_RESIZEABLE | RAM_PREALLOC | RAM_NORESERVE | RAM_GUEST_MEMFD)) == 0); @@ -2183,7 +2191,8 @@ RAMBlock *qemu_ram_alloc_from_ptr(ram_addr_t size, void *host, RAMBlock *qemu_ram_alloc(ram_addr_t size, uint32_t ram_flags, MemoryRegion *mr, Error **errp) { - assert((ram_flags & ~(RAM_SHARED | RAM_NORESERVE | RAM_GUEST_MEMFD)) == 0); + assert((ram_flags & ~(RAM_SHARED | RAM_NORESERVE | RAM_GUEST_MEMFD | + RAM_PRIVATE)) == 0); return qemu_ram_alloc_internal(size, size, NULL, NULL, ram_flags, mr, errp); } From patchwork Mon Dec 2 13:19:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890662 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 872D6D78321 for ; Mon, 2 Dec 2024 13:38:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Zb-0003yt-OR; Mon, 02 Dec 2024 08:34:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lj-0007yN-4L for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:37 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lc-0005h1-N9 for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:25 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26XFUh013671; Mon, 2 Dec 2024 13:20:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=l/Vc2l6s/DVzvU00oZ5/69o+yzZSdPoGvqx3QUG7o8A=; b= MPO5KDeKHzJtOdQfIUoEJY+e0LKNTtKs+VUYw0GXNc8r5+MmAsy3uulpJCNg+xPx fKXY+LNrZ52dVMpP7PLSHS5ZmIGUgcqaSM3G7AAAVm23xs+y4324JPhPjSuh7z9I tUzgCk1G1kaeJfW1glXpST2GkTzDXxlA786HMslb5gxEh1NuXpB+XFipSVKPVpub qywvjEh0F+zxId2RIzKBgr1Rf9phq0BBD/Xfgy1CWt+R0cpefSgiyLeWA6YlcO+t UbZ7QWSsSZRI+3pHHNNCYQJmjUTXshCMMY0Tj43AzvWOGW7daaJwHqlOl7rFuFgX 2xpWcnk7UYUREgqKvViTtA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437s4c333w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:16 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2BcwcH032644; Mon, 2 Dec 2024 13:20:16 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtmm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:15 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEf032806; Mon, 2 Dec 2024 13:20:15 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-5; Mon, 02 Dec 2024 13:20:15 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 04/19] machine: aux-ram-share option Date: Mon, 2 Dec 2024 05:19:56 -0800 Message-Id: <1733145611-62315-5-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: _V1EcolF7YMQF8qT4u5Y8zj1wa8s8qDi X-Proofpoint-GUID: _V1EcolF7YMQF8qT4u5Y8zj1wa8s8qDi Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Allocate auxilliary guest RAM as an anonymous file that is shareable with an external process. This option applies to memory allocated as a side effect of creating various devices. It does not apply to memory-backend-objects, whether explicitly specified on the command line, or implicitly created by the -m command line option. This option is intended to support new migration modes, in which the memory region can be transferred in place to a new QEMU process, by sending the memfd file descriptor to the process. Memory contents are preserved, and if the mode also transfers device descriptors, then pages that are locked in memory for DMA remain locked. This behavior is a pre-requisite for supporting vfio, vdpa, and iommufd devices with the new modes. Signed-off-by: Steve Sistare --- hw/core/machine.c | 18 ++++++++++++++++++ include/hw/boards.h | 1 + qemu-options.hx | 15 +++++++++++++++ system/physmem.c | 3 +++ 4 files changed, 37 insertions(+) diff --git a/hw/core/machine.c b/hw/core/machine.c index a35c4a8..b299b40 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -437,6 +437,20 @@ static void machine_set_mem_merge(Object *obj, bool value, Error **errp) ms->mem_merge = value; } +static bool machine_get_aux_ram_share(Object *obj, Error **errp) +{ + MachineState *ms = MACHINE(obj); + + return ms->aux_ram_share; +} + +static void machine_set_aux_ram_share(Object *obj, bool value, Error **errp) +{ + MachineState *ms = MACHINE(obj); + + ms->aux_ram_share = value; +} + static bool machine_get_usb(Object *obj, Error **errp) { MachineState *ms = MACHINE(obj); @@ -1129,6 +1143,10 @@ static void machine_class_init(ObjectClass *oc, void *data) object_class_property_set_description(oc, "mem-merge", "Enable/disable memory merge support"); + object_class_property_add_bool(oc, "aux-ram-share", + machine_get_aux_ram_share, + machine_set_aux_ram_share); + object_class_property_add_bool(oc, "usb", machine_get_usb, machine_set_usb); object_class_property_set_description(oc, "usb", diff --git a/include/hw/boards.h b/include/hw/boards.h index 36fbb9b..922ecd4 100644 --- a/include/hw/boards.h +++ b/include/hw/boards.h @@ -410,6 +410,7 @@ struct MachineState { bool enable_graphics; ConfidentialGuestSupport *cgs; HostMemoryBackend *memdev; + bool aux_ram_share; /* * convenience alias to ram_memdev_id backend memory region * or to numa container memory region diff --git a/qemu-options.hx b/qemu-options.hx index dacc979..02b9118 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -38,6 +38,9 @@ DEF("machine", HAS_ARG, QEMU_OPTION_machine, \ " nvdimm=on|off controls NVDIMM support (default=off)\n" " memory-encryption=@var{} memory encryption object to use (default=none)\n" " hmat=on|off controls ACPI HMAT support (default=off)\n" +#ifdef CONFIG_POSIX + " aux-ram-share=on|off allocate auxiliary guest RAM as shared (default: off)\n" +#endif " memory-backend='backend-id' specifies explicitly provided backend for main RAM (default=none)\n" " cxl-fmw.0.targets.0=firsttarget,cxl-fmw.0.targets.1=secondtarget,cxl-fmw.0.size=size[,cxl-fmw.0.interleave-granularity=granularity]\n", QEMU_ARCH_ALL) @@ -101,6 +104,18 @@ SRST Enables or disables ACPI Heterogeneous Memory Attribute Table (HMAT) support. The default is off. +#ifdef CONFIG_POSIX + ``aux-ram-share=on|off`` + Allocate auxiliary guest RAM as an anonymous file that is + shareable with an external process. This option applies to + memory allocated as a side effect of creating various devices. + It does not apply to memory-backend-objects, whether explicitly + specified on the command line, or implicitly created by the -m + command line option. + + Some migration modes require aux-ram-share=on. +#endif + ``memory-backend='id'`` An alternative to legacy ``-mem-path`` and ``mem-prealloc`` options. Allows to use a memory backend as main RAM. diff --git a/system/physmem.c b/system/physmem.c index 36f0811..0bcb2cc 100644 --- a/system/physmem.c +++ b/system/physmem.c @@ -2164,6 +2164,9 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size, new_block->flags = ram_flags; if (!host && !xen_enabled()) { + if (!share_flags && current_machine->aux_ram_share) { + new_block->flags |= RAM_SHARED; + } if ((new_block->flags & RAM_SHARED) && !qemu_ram_alloc_shared(new_block, &local_err)) { goto err; From patchwork Mon Dec 2 13:19:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0E5C8D78320 for ; Mon, 2 Dec 2024 13:38:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Zh-0004Ky-Hz; Mon, 02 Dec 2024 08:34:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lj-0007yJ-0m for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:37 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lc-0005h8-DT for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:23 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26XMLA013787; Mon, 2 Dec 2024 13:20:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=t2gVpnzKhmx8a/07hIqePpIJtTtvQZcreYKCDaL7UtQ=; b= A/bha6Gc4X7jIzXSxgMALJ2pKCT5xw5k7GLGWPbDM2qqdCvBJl4IaXiQ/RnD2uNP ijKa56mtrcoJX8xez2OgyQYa6KVt1HMsU/b/8V3tkvhVmHtm2gyReTZI4qGWBK8N n8hO5r5CgiD8ROZ1UBIPiua4NPmtvnbNtMpU15u+odzB18n8C5Qs3u4NptGRnCTz AlhmoD6mYufFTKdxa3Z6O7oIFyn6edTguOWh1NVrbVsZOMLyw2uagv8pKOb/EIEV MxX4hai6Ki8pJCDTMbzOBwGnxMTlz+ua62eJplV7USQfglyG3d7uKJvdi4UpHNMI /qyxhO8hlQjBY3sGZNOPsg== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437s4c333x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:17 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2Bk0pW032875; Mon, 2 Dec 2024 13:20:17 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtn7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:16 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEh032806; Mon, 2 Dec 2024 13:20:16 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-6; Mon, 02 Dec 2024 13:20:16 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 05/19] migration: cpr-state Date: Mon, 2 Dec 2024 05:19:57 -0800 Message-Id: <1733145611-62315-6-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: W-jMrteg6m8EddmOWYXeGxOjDj03dVxb X-Proofpoint-GUID: W-jMrteg6m8EddmOWYXeGxOjDj03dVxb Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org CPR must save state that is needed after QEMU is restarted, when devices are realized. Thus the extra state cannot be saved in the migration channel, as objects must already exist before that channel can be loaded. Instead, define auxilliary state structures and vmstate descriptions, not associated with any registered object, and serialize the aux state to a cpr-specific channel in cpr_state_save. Deserialize in cpr_state_load after QEMU restarts, before devices are realized. Provide accessors for clients to register file descriptors for saving. The mechanism for passing the fd's to the new process will be specific to each migration mode, and added in subsequent patches. Signed-off-by: Steve Sistare Reviewed-by: Fabiano Rosas Reviewed-by: Peter Xu --- include/migration/cpr.h | 25 ++++++ migration/cpr.c | 198 ++++++++++++++++++++++++++++++++++++++++++++++++ migration/meson.build | 1 + migration/migration.c | 1 + migration/trace-events | 7 ++ 5 files changed, 232 insertions(+) create mode 100644 include/migration/cpr.h create mode 100644 migration/cpr.c diff --git a/include/migration/cpr.h b/include/migration/cpr.h new file mode 100644 index 0000000..201d66d --- /dev/null +++ b/include/migration/cpr.h @@ -0,0 +1,25 @@ +/* + * Copyright (c) 2021, 2024 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef MIGRATION_CPR_H +#define MIGRATION_CPR_H + +#include "qapi/qapi-types-migration.h" + +#define QEMU_CPR_FILE_MAGIC 0x51435052 +#define QEMU_CPR_FILE_VERSION 0x00000001 + +void cpr_save_fd(const char *name, int id, int fd); +void cpr_delete_fd(const char *name, int id); +int cpr_find_fd(const char *name, int id); + +int cpr_state_save(MigrationChannel *channel, Error **errp); +int cpr_state_load(Error **errp); +void cpr_state_close(void); +struct QIOChannel *cpr_state_ioc(void); + +#endif diff --git a/migration/cpr.c b/migration/cpr.c new file mode 100644 index 0000000..1e2878c --- /dev/null +++ b/migration/cpr.c @@ -0,0 +1,198 @@ +/* + * Copyright (c) 2021-2024 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "migration/cpr.h" +#include "migration/misc.h" +#include "migration/options.h" +#include "migration/qemu-file.h" +#include "migration/savevm.h" +#include "migration/vmstate.h" +#include "sysemu/runstate.h" +#include "trace.h" + +/*************************************************************************/ +/* cpr state container for all information to be saved. */ + +typedef QLIST_HEAD(CprFdList, CprFd) CprFdList; + +typedef struct CprState { + CprFdList fds; +} CprState; + +static CprState cpr_state; + +/****************************************************************************/ + +typedef struct CprFd { + char *name; + unsigned int namelen; + int id; + int fd; + QLIST_ENTRY(CprFd) next; +} CprFd; + +static const VMStateDescription vmstate_cpr_fd = { + .name = "cpr fd", + .version_id = 1, + .minimum_version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_UINT32(namelen, CprFd), + VMSTATE_VBUFFER_ALLOC_UINT32(name, CprFd, 0, NULL, namelen), + VMSTATE_INT32(id, CprFd), + VMSTATE_INT32(fd, CprFd), + VMSTATE_END_OF_LIST() + } +}; + +void cpr_save_fd(const char *name, int id, int fd) +{ + CprFd *elem = g_new0(CprFd, 1); + + trace_cpr_save_fd(name, id, fd); + elem->name = g_strdup(name); + elem->namelen = strlen(name) + 1; + elem->id = id; + elem->fd = fd; + QLIST_INSERT_HEAD(&cpr_state.fds, elem, next); +} + +static CprFd *find_fd(CprFdList *head, const char *name, int id) +{ + CprFd *elem; + + QLIST_FOREACH(elem, head, next) { + if (!strcmp(elem->name, name) && elem->id == id) { + return elem; + } + } + return NULL; +} + +void cpr_delete_fd(const char *name, int id) +{ + CprFd *elem = find_fd(&cpr_state.fds, name, id); + + if (elem) { + QLIST_REMOVE(elem, next); + g_free(elem->name); + g_free(elem); + } + + trace_cpr_delete_fd(name, id); +} + +int cpr_find_fd(const char *name, int id) +{ + CprFd *elem = find_fd(&cpr_state.fds, name, id); + int fd = elem ? elem->fd : -1; + + trace_cpr_find_fd(name, id, fd); + return fd; +} +/*************************************************************************/ +#define CPR_STATE "CprState" + +static const VMStateDescription vmstate_cpr_state = { + .name = CPR_STATE, + .version_id = 1, + .minimum_version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_QLIST_V(fds, CprState, 1, vmstate_cpr_fd, CprFd, next), + VMSTATE_END_OF_LIST() + } +}; +/*************************************************************************/ + +static QEMUFile *cpr_state_file; + +QIOChannel *cpr_state_ioc(void) +{ + return qemu_file_get_ioc(cpr_state_file); +} + +int cpr_state_save(MigrationChannel *channel, Error **errp) +{ + int ret; + QEMUFile *f; + MigMode mode = migrate_mode(); + + trace_cpr_state_save(MigMode_str(mode)); + + /* set f based on mode in a later patch in this series */ + return 0; + + qemu_put_be32(f, QEMU_CPR_FILE_MAGIC); + qemu_put_be32(f, QEMU_CPR_FILE_VERSION); + + ret = vmstate_save_state(f, &vmstate_cpr_state, &cpr_state, 0); + if (ret) { + error_setg(errp, "vmstate_save_state error %d", ret); + qemu_fclose(f); + return ret; + } + + /* + * Close the socket only partially so we can later detect when the other + * end closes by getting a HUP event. + */ + qemu_fflush(f); + qio_channel_shutdown(qemu_file_get_ioc(f), QIO_CHANNEL_SHUTDOWN_WRITE, + NULL); + cpr_state_file = f; + return 0; +} + +int cpr_state_load(Error **errp) +{ + int ret; + uint32_t v; + QEMUFile *f; + MigMode mode = 0; + + /* set f and mode based on other parameters later in this patch series */ + return 0; + + trace_cpr_state_load(MigMode_str(mode)); + + v = qemu_get_be32(f); + if (v != QEMU_CPR_FILE_MAGIC) { + error_setg(errp, "Not a migration stream (bad magic %x)", v); + qemu_fclose(f); + return -EINVAL; + } + v = qemu_get_be32(f); + if (v != QEMU_CPR_FILE_VERSION) { + error_setg(errp, "Unsupported migration stream version %d", v); + qemu_fclose(f); + return -ENOTSUP; + } + + ret = vmstate_load_state(f, &vmstate_cpr_state, &cpr_state, 1); + if (ret) { + error_setg(errp, "vmstate_load_state error %d", ret); + qemu_fclose(f); + return ret; + } + + /* + * Let the caller decide when to close the socket (and generate a HUP event + * for the sending side). + */ + cpr_state_file = f; + + return ret; +} + +void cpr_state_close(void) +{ + if (cpr_state_file) { + qemu_fclose(cpr_state_file); + cpr_state_file = NULL; + } +} diff --git a/migration/meson.build b/migration/meson.build index d53cf34..039f0f9 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -13,6 +13,7 @@ system_ss.add(files( 'block-dirty-bitmap.c', 'channel.c', 'channel-block.c', + 'cpr.c', 'cpu-throttle.c', 'dirtyrate.c', 'exec.c', diff --git a/migration/migration.c b/migration/migration.c index 8c5bd0a..83dabc7 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -27,6 +27,7 @@ #include "sysemu/cpu-throttle.h" #include "rdma.h" #include "ram.h" +#include "migration/cpr.h" #include "migration/global_state.h" #include "migration/misc.h" #include "migration.h" diff --git a/migration/trace-events b/migration/trace-events index bb0e0cc..89c0244 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -342,6 +342,13 @@ colo_receive_message(const char *msg) "Receive '%s' message" # colo-failover.c colo_failover_set_state(const char *new_state) "new state %s" +# cpr.c +cpr_save_fd(const char *name, int id, int fd) "%s, id %d, fd %d" +cpr_delete_fd(const char *name, int id) "%s, id %d" +cpr_find_fd(const char *name, int id, int fd) "%s, id %d returns %d" +cpr_state_save(const char *mode) "%s mode" +cpr_state_load(const char *mode) "%s mode" + # block-dirty-bitmap.c send_bitmap_header_enter(void) "" send_bitmap_bits(uint32_t flags, uint64_t start_sector, uint32_t nr_sectors, uint64_t data_size) "flags: 0x%x, start_sector: %" PRIu64 ", nr_sectors: %" PRIu32 ", data_size: %" PRIu64 From patchwork Mon Dec 2 13:19:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890656 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1CA10D7831F for ; Mon, 2 Dec 2024 13:37:25 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6bR-0007YR-56; Mon, 02 Dec 2024 08:36:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lj-0007yH-1q for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:37 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lc-0005he-ND for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:23 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26XMLB013787; Mon, 2 Dec 2024 13:20:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=76RsVoAGMj38dWsNKVn+ivLB7EnCOcYrTG3VXeLBVB4=; b= DLU0A8jG4MEPweEjVAEWpSRkrgx7aSt888pb4+cuIEml+itdCNpRAcwqJ+qBM0qB iAe4Zf3hiyTMO+CoNuHU58IEQ+IQEsp1DFO/SWl7WQDkSJpS3xPhA2HhkZsI4+rl WywqYti7yPikUxOHTcdxMVpkG3joju4suNX4ess8C3OGggcAolyqMBrPjmMAUzUB QamsMWH0aeczeHcrZOnNx2fVTWYuoS0aCvuE0hCT95d4dZihTnchws5UC8xNvCAR V5sh7ofvjb6H8xSOia2m4oDthE3L2VTobhdEB6IXuX97byZRJMVK9kI5d56SWbz1 AvQ5d6phpc8ODNezBf9afQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437s4c3342-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:18 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2DIXe3032684; Mon, 2 Dec 2024 13:20:17 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtnq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:17 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEj032806; Mon, 2 Dec 2024 13:20:17 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-7; Mon, 02 Dec 2024 13:20:17 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 06/19] physmem: preserve ram blocks for cpr Date: Mon, 2 Dec 2024 05:19:58 -0800 Message-Id: <1733145611-62315-7-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: HF56IY68lu2-gGBrqXRGeOzuzecSzKEX X-Proofpoint-GUID: HF56IY68lu2-gGBrqXRGeOzuzecSzKEX Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Save the memfd for ramblocks in CPR state, along with a name that uniquely identifies it. The block's idstr is not yet set, so it cannot be used for this purpose. Find the saved memfd in new QEMU when creating a block. If the block size is larger in new QEMU, extend the block using fallocate, and the extra space will be useable after a guest reset. Signed-off-by: Steve Sistare --- system/physmem.c | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/system/physmem.c b/system/physmem.c index 0bcb2cc..aa095a3 100644 --- a/system/physmem.c +++ b/system/physmem.c @@ -70,6 +70,7 @@ #include "qemu/pmem.h" +#include "migration/cpr.h" #include "migration/vmstate.h" #include "qemu/range.h" @@ -1661,6 +1662,19 @@ void qemu_ram_unset_idstr(RAMBlock *block) } } +static char *cpr_name(RAMBlock *block) +{ + MemoryRegion *mr = block->mr; + const char *mr_name = memory_region_name(mr); + g_autofree char *id = mr->dev ? qdev_get_dev_path(mr->dev) : NULL; + + if (id) { + return g_strdup_printf("%s/%s", id, mr_name); + } else { + return g_strdup(mr_name); + } +} + size_t qemu_ram_pagesize(RAMBlock *rb) { return rb->page_size; @@ -2080,8 +2094,18 @@ static bool qemu_ram_alloc_shared(RAMBlock *new_block, Error **errp) { size_t max_length = new_block->max_length; MemoryRegion *mr = new_block->mr; - const char *name = memory_region_name(mr); - int fd; + g_autofree char *name = cpr_name(new_block); + int fd = cpr_find_fd(name, 0); + + if (fd >= 0) { + if (lseek(fd, 0, SEEK_END) < max_length && ftruncate(fd, max_length)) { + error_setg_errno(errp, errno, + "cannot grow ram block %s fd %d to %ld bytes", + name, fd, max_length); + goto err; + } + goto have_fd; + } if (qemu_memfd_available()) { fd = qemu_memfd_create(name, max_length + mr->align, 0, 0, 0, errp); @@ -2111,7 +2135,9 @@ static bool qemu_ram_alloc_shared(RAMBlock *new_block, Error **errp) return true; } } + cpr_save_fd(name, 0, fd); +have_fd: new_block->mr->align = QEMU_VMALLOC_ALIGN; new_block->host = file_ram_alloc(new_block, max_length, fd, false, 0, errp); @@ -2122,6 +2148,8 @@ static bool qemu_ram_alloc_shared(RAMBlock *new_block, Error **errp) return true; } +err: + cpr_delete_fd(name, 0); close(fd); return false; } @@ -2234,6 +2262,8 @@ static void reclaim_ramblock(RAMBlock *block) void qemu_ram_free(RAMBlock *block) { + g_autofree char *name = NULL; + if (!block) { return; } @@ -2244,6 +2274,8 @@ void qemu_ram_free(RAMBlock *block) } qemu_mutex_lock_ramlist(); + name = cpr_name(block); + cpr_delete_fd(name, 0); QLIST_REMOVE_RCU(block, next); ram_list.mru_block = NULL; /* Write list before version */ From patchwork Mon Dec 2 13:19:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890654 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BA0ABD7831F for ; Mon, 2 Dec 2024 13:37:17 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Zq-00056D-Jd; Mon, 02 Dec 2024 08:35:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lj-0007yO-55 for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:37 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lf-0005k0-6Y for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:26 -0500 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26WwQG005193; Mon, 2 Dec 2024 13:20:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=TXd9jXCmZXqRMNmbI/fqeBAMqJD3CKnnnlbau9xifCM=; b= fFHffPgTcYhrPE7duNa+qmImNPwsFeedQowGUoSYv5FLICxKe614ysdJxRGphdp/ 2r3jtKyTdtO14130YmzsRmn16CgnmSaEYNDM6Uwv6Q06/jQHTnMQrYUMT7tmbTW9 po1n9hgnaXEpV2MSFkw8QhMkLlWPK0VQ/sgv/+h+5BH8F7vHA0WcYwsI4zUpJpX0 bcGpJe3DaGFeaf+HUHEwvI6kR4HSOrm+mHlXEyF9zAjgdFHO1XNQlW35uqgyxK7c mfYG0Ivxy4hNDkE2HscfwGF54vUyPu9DtqktyrEt+cY6clEW8qOJhB3wH52fl9+T 2vUYqJ5f6AC3SJYI7gIlDg== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437tk8txgr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:19 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2CmKYO032740; Mon, 2 Dec 2024 13:20:18 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtpd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:18 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEl032806; Mon, 2 Dec 2024 13:20:18 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-8; Mon, 02 Dec 2024 13:20:17 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 07/19] hostmem-memfd: preserve for cpr Date: Mon, 2 Dec 2024 05:19:59 -0800 Message-Id: <1733145611-62315-8-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: 62KhHrDx2vfPcxHKQHwhPP2732f-48nA X-Proofpoint-GUID: 62KhHrDx2vfPcxHKQHwhPP2732f-48nA Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Preserve memory-backend-memfd memory objects during cpr-transfer. Signed-off-by: Steve Sistare Acked-by: Peter Xu --- backends/hostmem-memfd.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/backends/hostmem-memfd.c b/backends/hostmem-memfd.c index 1bcae4b..497d63b 100644 --- a/backends/hostmem-memfd.c +++ b/backends/hostmem-memfd.c @@ -17,6 +17,7 @@ #include "qemu/module.h" #include "qapi/error.h" #include "qom/object.h" +#include "migration/cpr.h" OBJECT_DECLARE_SIMPLE_TYPE(HostMemoryBackendMemfd, MEMORY_BACKEND_MEMFD) @@ -33,15 +34,19 @@ static bool memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) { HostMemoryBackendMemfd *m = MEMORY_BACKEND_MEMFD(backend); - g_autofree char *name = NULL; + g_autofree char *name = host_memory_backend_get_name(backend); + int fd = cpr_find_fd(name, 0); uint32_t ram_flags; - int fd; if (!backend->size) { error_setg(errp, "can't create backend with size 0"); return false; } + if (fd >= 0) { + goto have_fd; + } + fd = qemu_memfd_create(TYPE_MEMORY_BACKEND_MEMFD, backend->size, m->hugetlb, m->hugetlbsize, m->seal ? F_SEAL_GROW | F_SEAL_SHRINK | F_SEAL_SEAL : 0, @@ -49,9 +54,10 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) if (fd == -1) { return false; } + cpr_save_fd(name, 0, fd); +have_fd: backend->aligned = true; - name = host_memory_backend_get_name(backend); ram_flags = backend->share ? RAM_SHARED : RAM_PRIVATE; ram_flags |= backend->reserve ? 0 : RAM_NORESERVE; ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0; From patchwork Mon Dec 2 13:20:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890684 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8E41AD78321 for ; Mon, 2 Dec 2024 13:43:40 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Zl-0004in-V1; Mon, 02 Dec 2024 08:34:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lj-0007yG-1A for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:37 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Le-0005k2-Pu for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:24 -0500 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26WuBV023891; Mon, 2 Dec 2024 13:20:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=tWyVrtCSuYibWdU8IB2GVRbDR33+hFLngtrMzltO5Jk=; b= TuNEwAw/mXlska/700ZmsPyfRpZqQd6aSjLVUAcsqA8pcfa4IHTiU6I+qTQLjj3K mofRYAYnFO1f7eCRU5Kq7BjW/TVGff7XYt1XH4hz/1WGjk/UO7G9ohTbNjzf+QHQ 1BAuKKoDWaYKhjRoCv89aOt6h7bp03GchAOshztdc/kpepnKB/l3cyvDKv+V/7L1 K6SNDL+HfYLCtPIYuzGtkNa0Brx1JbFeB+lA++NHaHG00JvyTmAa+obCWmYKtCSz FgLeg7lB7OEztGTynWmemgbAsQpLF0ioQoADll8g2pN3Zx4BZ6I0JD9ZmzOKvO05 SMlTA0hDx2G6CTfrcvNevA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437trbk4b4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:20 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2CLxW1032761; Mon, 2 Dec 2024 13:20:19 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtq1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:19 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEn032806; Mon, 2 Dec 2024 13:20:19 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-9; Mon, 02 Dec 2024 13:20:18 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 08/19] hostmem-shm: preserve for cpr Date: Mon, 2 Dec 2024 05:20:00 -0800 Message-Id: <1733145611-62315-9-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-GUID: y_dRIZym8uOZGfNwbfUDFxudLmUZIZJR X-Proofpoint-ORIG-GUID: y_dRIZym8uOZGfNwbfUDFxudLmUZIZJR Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Preserve memory-backend-shm memory objects during cpr-transfer. Signed-off-by: Steve Sistare --- backends/hostmem-shm.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/backends/hostmem-shm.c b/backends/hostmem-shm.c index 837b9f1..0946f2a 100644 --- a/backends/hostmem-shm.c +++ b/backends/hostmem-shm.c @@ -13,6 +13,7 @@ #include "qemu/osdep.h" #include "sysemu/hostmem.h" #include "qapi/error.h" +#include "migration/cpr.h" #define TYPE_MEMORY_BACKEND_SHM "memory-backend-shm" @@ -25,9 +26,9 @@ struct HostMemoryBackendShm { static bool shm_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) { - g_autofree char *backend_name = NULL; + g_autofree char *backend_name = host_memory_backend_get_name(backend); uint32_t ram_flags; - int fd; + int fd = cpr_find_fd(backend_name, 0); if (!backend->size) { error_setg(errp, "can't create shm backend with size 0"); @@ -39,13 +40,18 @@ shm_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) return false; } + if (fd >= 0) { + goto have_fd; + } + fd = qemu_shm_alloc(backend->size, errp); if (fd < 0) { return false; } + cpr_save_fd(backend_name, 0, fd); +have_fd: /* Let's do the same as memory-backend-ram,share=on would do. */ - backend_name = host_memory_backend_get_name(backend); ram_flags = RAM_SHARED; ram_flags |= backend->reserve ? 0 : RAM_NORESERVE; From patchwork Mon Dec 2 13:20:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4A61DD78323 for ; Mon, 2 Dec 2024 13:41:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Zr-00059J-2W; Mon, 02 Dec 2024 08:35:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Ln-00081k-S2 for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:41 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lg-0005mX-Nb for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:28 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2DIY83012846; Mon, 2 Dec 2024 13:20:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=rq8NerJdvkXQsuBr1MGXnTxqN79ZMc4lonX0yPws4+o=; b= oRFzjM3OylhpQ84GYimDxD3twmioTXOVTrGaljPfOKyJGCs8KVajJ4Gzadhnd3AU Q0gH570PFGod3IJAghpa6hmx2KRdu2XgXVPErBasm7MZJrUAVl1SXYneqV4yopQ+ lnrLpAFeXZ9bBVRoF7vzWoGjkVSgc7XShl6zjPFBq2b77llv94I4+TVWZgoywVF8 B2LSgcIE7h4ypyn+K+nIbgGUMES348mQRJkqA2/VftVJR8k1ZyXeDUoaBEvZG+tH 8NoUcY4CblbMshZ2PlWfhwYP8xk+TtHr7AsSXsj4dXvtkpGRIrwvQfgD9bzYhZhQ Isao3uzbI5xFed+w8Z0w6w== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437s4c3347-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:21 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2CmKYQ032740; Mon, 2 Dec 2024 13:20:20 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtqm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:20 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEp032806; Mon, 2 Dec 2024 13:20:19 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-10; Mon, 02 Dec 2024 13:20:19 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 09/19] migration: incoming channel Date: Mon, 2 Dec 2024 05:20:01 -0800 Message-Id: <1733145611-62315-10-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: A3pdK6DAAW_0IoLvDL9HdR45Wd6R0WUt X-Proofpoint-GUID: A3pdK6DAAW_0IoLvDL9HdR45Wd6R0WUt Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Extend the -incoming option to allow an @MigrationChannel to be specified. This allows channels other than 'main' to be described on the command line, which will be needed for CPR. Signed-off-by: Steve Sistare --- include/migration/misc.h | 2 ++ migration/migration.c | 18 ++++++++++++++---- migration/migration.h | 2 -- qemu-options.hx | 17 +++++++++++++++++ system/vl.c | 35 +++++++++++++++++++++++++++++++++-- 5 files changed, 66 insertions(+), 8 deletions(-) diff --git a/include/migration/misc.h b/include/migration/misc.h index 804eb23..259d4aa 100644 --- a/include/migration/misc.h +++ b/include/migration/misc.h @@ -106,4 +106,6 @@ bool migration_incoming_postcopy_advised(void); /* True if background snapshot is active */ bool migration_in_bg_snapshot(void); +bool migrate_uri_parse(const char *uri, MigrationChannel **channel, + Error **errp); #endif diff --git a/migration/migration.c b/migration/migration.c index 83dabc7..a5cf148 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2046,6 +2046,7 @@ void qmp_migrate(const char *uri, bool has_channels, MigrationState *s = migrate_get_current(); g_autoptr(MigrationChannel) channel = NULL; MigrationAddress *addr = NULL; + MigrationChannel *channelv[MIGRATION_CHANNEL_TYPE__MAX] = { NULL }; /* * Having preliminary checks for uri and channel @@ -2056,12 +2057,21 @@ void qmp_migrate(const char *uri, bool has_channels, } if (channels) { - /* To verify that Migrate channel list has only item */ - if (channels->next) { - error_setg(errp, "Channel list has more than one entries"); + for ( ; channels; channels = channels->next) { + MigrationChannelType type = channels->value->channel_type; + + if (channelv[type]) { + error_setg(errp, "Channel list has more than one %s entry", + MigrationChannelType_str(type)); + return; + } + channelv[type] = channels->value; + } + addr = channelv[MIGRATION_CHANNEL_TYPE_MAIN]->addr; + if (!addr) { + error_setg(errp, "Channel list has no main entry"); return; } - addr = channels->value->addr; } if (uri) { diff --git a/migration/migration.h b/migration/migration.h index 0956e92..5cd0f29 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -522,8 +522,6 @@ bool check_dirty_bitmap_mig_alias_map(const BitmapMigrationNodeAliasList *bbm, Error **errp); void migrate_add_address(SocketAddress *address); -bool migrate_uri_parse(const char *uri, MigrationChannel **channel, - Error **errp); int foreach_not_ignored_block(RAMBlockIterFunc func, void *opaque); #define qemu_ram_foreach_block \ diff --git a/qemu-options.hx b/qemu-options.hx index 02b9118..fab50ce 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -4937,10 +4937,17 @@ DEF("incoming", HAS_ARG, QEMU_OPTION_incoming, \ "-incoming exec:cmdline\n" \ " accept incoming migration on given file descriptor\n" \ " or from given external command\n" \ + "-incoming @MigrationChannel\n" \ + " accept incoming migration on the channel\n" \ "-incoming defer\n" \ " wait for the URI to be specified via migrate_incoming\n", QEMU_ARCH_ALL) SRST +The -incoming option specifies the migration channel for an incoming +migration. It may be used multiple times to specify multiple +migration channel types. The channel type is specified in @MigrationChannel, +and is 'main' for all other forms of -incoming. + ``-incoming tcp:[host]:port[,to=maxport][,ipv4=on|off][,ipv6=on|off]`` \ ``-incoming rdma:host:port[,ipv4=on|off][,ipv6=on|off]`` @@ -4960,6 +4967,16 @@ SRST Accept incoming migration as an output from specified external command. +``-incoming @MigrationChannel`` + Accept incoming migration on the channel. See the QAPI documentation + for the syntax of the @MigrationChannel data element. For example: + :: + + -incoming '{"channel-type": "main", + "addr": { "transport": "socket", + "type": "unix", + "path": "my.sock" }}' + ``-incoming defer`` Wait for the URI to be specified via migrate\_incoming. The monitor can be used to change settings (such as migration parameters) prior diff --git a/system/vl.c b/system/vl.c index 4151a79..2c24c60 100644 --- a/system/vl.c +++ b/system/vl.c @@ -123,6 +123,7 @@ #include "qapi/qapi-visit-block-core.h" #include "qapi/qapi-visit-compat.h" #include "qapi/qapi-visit-machine.h" +#include "qapi/qapi-visit-migration.h" #include "qapi/qapi-visit-ui.h" #include "qapi/qapi-commands-block-core.h" #include "qapi/qapi-commands-migration.h" @@ -159,6 +160,7 @@ typedef struct DeviceOption { static const char *cpu_option; static const char *mem_path; static const char *incoming; +static MigrationChannelList *incoming_channels; static const char *loadvm; static const char *accelerators; static bool have_custom_ram_size; @@ -1821,6 +1823,35 @@ static void object_option_add_visitor(Visitor *v) QTAILQ_INSERT_TAIL(&object_opts, opt, next); } +static void incoming_option_parse(const char *str) +{ + MigrationChannel *channel; + + if (str[0] == '{') { + QObject *obj = qobject_from_json(str, &error_fatal); + Visitor *v = qobject_input_visitor_new(obj); + + qobject_unref(obj); + visit_type_MigrationChannel(v, "channel", &channel, &error_fatal); + visit_free(v); + } else if (!strcmp(str, "defer")) { + channel = NULL; + } else { + migrate_uri_parse(str, &channel, &error_fatal); + } + + /* New incoming spec replaces the previous */ + + if (incoming_channels) { + qapi_free_MigrationChannelList(incoming_channels); + } + if (channel) { + incoming_channels = g_new0(MigrationChannelList, 1); + incoming_channels->value = channel; + } + incoming = str; +} + static void object_option_parse(const char *str) { QemuOpts *opts; @@ -2730,7 +2761,7 @@ void qmp_x_exit_preconfig(Error **errp) if (incoming) { Error *local_err = NULL; if (strcmp(incoming, "defer") != 0) { - qmp_migrate_incoming(incoming, false, NULL, true, true, + qmp_migrate_incoming(NULL, true, incoming_channels, true, true, &local_err); if (local_err) { error_reportf_err(local_err, "-incoming %s: ", incoming); @@ -3477,7 +3508,7 @@ void qemu_init(int argc, char **argv) if (!incoming) { runstate_set(RUN_STATE_INMIGRATE); } - incoming = optarg; + incoming_option_parse(optarg); break; case QEMU_OPTION_only_migratable: only_migratable = 1; From patchwork Mon Dec 2 13:20:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 59E9FD78321 for ; Mon, 2 Dec 2024 13:40:50 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6bg-00080f-5P; Mon, 02 Dec 2024 08:36:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Ln-00081j-Re for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:41 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Li-0005n9-JI for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:28 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26WwEN012851; Mon, 2 Dec 2024 13:20:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=n8GAaLWehzVdaQSW0sNL/nudX90REaKNsDMkGSNfea0=; b= Jv7Dgvg95RLlIl4HAF60VxuJkDXlx0WXTNfXzEFicnWI4KUkyXN9ORgS9T5SarS2 yQqSozzAJPvPKB7rRx9dBhVRyqoGzvsbkBMJ+fiMAwQIxDqY1wLhX+bDXsA581r+ OioTeihSAL51YEAMjqBhJQ6TPhVEaR1/N45qBJ2xPmET9T1FZXrXAryT6K1nKOWB PQTCfHq49p7VZtY4CzJ7lznXSWSe1WN7+PgHTXHG8mfOCcsqfmoNn13yq7TyKj7G +OvJnM05qCQSIphMu8uykSTxd1ZPaSN7YIwxqZEFMy4bgYbynsS93JmFV7wr43Oc arG4xDDN65hoyfinErSTHw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437s4c3348-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:22 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2DIXe5032684; Mon, 2 Dec 2024 13:20:21 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtra-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:21 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEr032806; Mon, 2 Dec 2024 13:20:20 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-11; Mon, 02 Dec 2024 13:20:20 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 10/19] migration: cpr channel Date: Mon, 2 Dec 2024 05:20:02 -0800 Message-Id: <1733145611-62315-11-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: XlFtZckGl6MIUh_6UZDggTBP5DKiiIY3 X-Proofpoint-GUID: XlFtZckGl6MIUh_6UZDggTBP5DKiiIY3 Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add the 'cpr' channel type, and stash the incoming cpr channel for use in a subsequent patch. Signed-off-by: Steve Sistare --- include/migration/cpr.h | 3 +++ migration/cpr.c | 15 +++++++++++++++ qapi/migration.json | 3 ++- system/vl.c | 6 ++++++ 4 files changed, 26 insertions(+), 1 deletion(-) diff --git a/include/migration/cpr.h b/include/migration/cpr.h index 201d66d..e833fae 100644 --- a/include/migration/cpr.h +++ b/include/migration/cpr.h @@ -17,6 +17,9 @@ void cpr_save_fd(const char *name, int id, int fd); void cpr_delete_fd(const char *name, int id); int cpr_find_fd(const char *name, int id); +void cpr_set_cpr_channel(MigrationChannel *channel); +MigrationChannel *cpr_get_cpr_channel(void); + int cpr_state_save(MigrationChannel *channel, Error **errp); int cpr_state_load(Error **errp); void cpr_state_close(void); diff --git a/migration/cpr.c b/migration/cpr.c index 1e2878c..f4a795f 100644 --- a/migration/cpr.c +++ b/migration/cpr.c @@ -116,6 +116,21 @@ QIOChannel *cpr_state_ioc(void) return qemu_file_get_ioc(cpr_state_file); } +static MigrationChannel *cpr_channel; + +void cpr_set_cpr_channel(MigrationChannel *channel) +{ + if (cpr_channel) { + qapi_free_MigrationChannel(cpr_channel); + } + cpr_channel = channel; +} + +MigrationChannel *cpr_get_cpr_channel(void) +{ + return cpr_channel; +} + int cpr_state_save(MigrationChannel *channel, Error **errp) { int ret; diff --git a/qapi/migration.json b/qapi/migration.json index a605dc2..a26960b 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1578,11 +1578,12 @@ # The migration channel-type request options. # # @main: Main outbound migration channel. +# @cpr: cpr state channel. # # Since: 8.1 ## { 'enum': 'MigrationChannelType', - 'data': [ 'main' ] } + 'data': [ 'main', 'cpr' ] } ## # @MigrationChannel: diff --git a/system/vl.c b/system/vl.c index 2c24c60..40e049e 100644 --- a/system/vl.c +++ b/system/vl.c @@ -77,6 +77,7 @@ #include "hw/block/block.h" #include "hw/i386/x86.h" #include "hw/i386/pc.h" +#include "migration/cpr.h" #include "migration/misc.h" #include "migration/snapshot.h" #include "sysemu/tpm.h" @@ -1834,6 +1835,11 @@ static void incoming_option_parse(const char *str) qobject_unref(obj); visit_type_MigrationChannel(v, "channel", &channel, &error_fatal); visit_free(v); + + if (channel->channel_type == MIGRATION_CHANNEL_TYPE_CPR) { + cpr_set_cpr_channel(channel); + return; + } } else if (!strcmp(str, "defer")) { channel = NULL; } else { From patchwork Mon Dec 2 13:20:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 123BED78322 for ; Mon, 2 Dec 2024 13:44:00 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Zv-0005Ps-6h; Mon, 02 Dec 2024 08:35:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lo-00081l-J9 for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:41 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Li-0005nX-Ni for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:29 -0500 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26X3oZ015184; Mon, 2 Dec 2024 13:20:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=rPSayIr1as1CoNo4I53Fie8Y5xVuvIsQZGh4MXjNa5c=; b= eIWwuVdqyf2Q5Z1NqcHm7LSjT6NKwBZltyBCkNJMBmrX8aCSZ83/ReeNwUh5TRQO RfHSOxqkzHqOty3w0N0VpupLcalGX7qcTrvOiZChydf8s8QXdk5xdQQ9Lv7HdLjb ESOnmHOvTC+mbkq3KROTw5OmMED5uTfegFAtel54QhOT2rVLTEGhFlJ/iY0xtyPY 7Oij1jwtCbtuPSB/PhnAGWYZmhvzA6SeMcA5ZleN9ciMrvToiE26pB/bQq9VbYKj 1GOmjrhyRLT13cZ0/eMblgpoJ9jyuJhjiveyCOppZkf3duzH1Tc3kcGQvoXX+flp Ajt+2hKdLWJxnGUkd+2xQQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437smab02u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:22 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2CQxAM032065; Mon, 2 Dec 2024 13:20:21 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtrs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:21 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEt032806; Mon, 2 Dec 2024 13:20:21 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-12; Mon, 02 Dec 2024 13:20:21 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 11/19] migration: SCM_RIGHTS for QEMUFile Date: Mon, 2 Dec 2024 05:20:03 -0800 Message-Id: <1733145611-62315-12-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: PIeb2vryjAh79XhXqFEUGuZ3qUdcKQNJ X-Proofpoint-GUID: PIeb2vryjAh79XhXqFEUGuZ3qUdcKQNJ Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Define functions to put/get file descriptors to/from a QEMUFile, for qio channels that support SCM_RIGHTS. Maintain ordering such that put(A), put(fd), put(B) followed by get(A), get(fd), get(B) always succeeds. Other get orderings may succeed but are not guaranteed. Signed-off-by: Steve Sistare Reviewed-by: Peter Xu --- migration/qemu-file.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++--- migration/qemu-file.h | 2 ++ migration/trace-events | 2 ++ 3 files changed, 83 insertions(+), 4 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index b6d2f58..728c7dd 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -37,6 +37,11 @@ #define IO_BUF_SIZE 32768 #define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64) +typedef struct FdEntry { + QTAILQ_ENTRY(FdEntry) entry; + int fd; +} FdEntry; + struct QEMUFile { QIOChannel *ioc; bool is_writable; @@ -51,6 +56,9 @@ struct QEMUFile { int last_error; Error *last_error_obj; + + bool can_pass_fd; + QTAILQ_HEAD(, FdEntry) fds; }; /* @@ -109,6 +117,8 @@ static QEMUFile *qemu_file_new_impl(QIOChannel *ioc, bool is_writable) object_ref(ioc); f->ioc = ioc; f->is_writable = is_writable; + f->can_pass_fd = qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_FD_PASS); + QTAILQ_INIT(&f->fds); return f; } @@ -310,6 +320,10 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEMUFile *f) int len; int pending; Error *local_error = NULL; + g_autofree int *fds = NULL; + size_t nfd = 0; + int **pfds = f->can_pass_fd ? &fds : NULL; + size_t *pnfd = f->can_pass_fd ? &nfd : NULL; assert(!qemu_file_is_writable(f)); @@ -325,10 +339,9 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEMUFile *f) } do { - len = qio_channel_read(f->ioc, - (char *)f->buf + pending, - IO_BUF_SIZE - pending, - &local_error); + struct iovec iov = { f->buf + pending, IO_BUF_SIZE - pending }; + len = qio_channel_readv_full(f->ioc, &iov, 1, pfds, pnfd, 0, + &local_error); if (len == QIO_CHANNEL_ERR_BLOCK) { if (qemu_in_coroutine()) { qio_channel_yield(f->ioc, G_IO_IN); @@ -348,9 +361,65 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEMUFile *f) qemu_file_set_error_obj(f, len, local_error); } + for (int i = 0; i < nfd; i++) { + FdEntry *fde = g_new0(FdEntry, 1); + fde->fd = fds[i]; + QTAILQ_INSERT_TAIL(&f->fds, fde, entry); + } + return len; } +int qemu_file_put_fd(QEMUFile *f, int fd) +{ + int ret = 0; + QIOChannel *ioc = qemu_file_get_ioc(f); + Error *err = NULL; + struct iovec iov = { (void *)" ", 1 }; + + /* + * Send a dummy byte so qemu_fill_buffer on the receiving side does not + * fail with a len=0 error. Flush first to maintain ordering wrt other + * data. + */ + + qemu_fflush(f); + if (qio_channel_writev_full(ioc, &iov, 1, &fd, 1, 0, &err) < 1) { + error_report_err(error_copy(err)); + qemu_file_set_error_obj(f, -EIO, err); + ret = -1; + } + trace_qemu_file_put_fd(f->ioc->name, fd, ret); + return ret; +} + +int qemu_file_get_fd(QEMUFile *f) +{ + int fd = -1; + FdEntry *fde; + + if (!f->can_pass_fd) { + Error *err = NULL; + error_setg(&err, "%s does not support fd passing", f->ioc->name); + error_report_err(error_copy(err)); + qemu_file_set_error_obj(f, -EIO, err); + goto out; + } + + /* Force the dummy byte and its fd passenger to appear. */ + qemu_peek_byte(f, 0); + + fde = QTAILQ_FIRST(&f->fds); + if (fde) { + qemu_get_byte(f); /* Drop the dummy byte */ + fd = fde->fd; + QTAILQ_REMOVE(&f->fds, fde, entry); + } +out: + trace_qemu_file_get_fd(f->ioc->name, fd); + return fd; +} + /** Closes the file * * Returns negative error value if any error happened on previous operations or @@ -361,11 +430,17 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEMUFile *f) */ int qemu_fclose(QEMUFile *f) { + FdEntry *fde, *next; int ret = qemu_fflush(f); int ret2 = qio_channel_close(f->ioc, NULL); if (ret >= 0) { ret = ret2; } + QTAILQ_FOREACH_SAFE(fde, &f->fds, entry, next) { + warn_report("qemu_fclose: received fd %d was never claimed", fde->fd); + close(fde->fd); + g_free(fde); + } g_clear_pointer(&f->ioc, object_unref); error_free(f->last_error_obj); g_free(f); diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 11c2120..3e47a20 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -79,5 +79,7 @@ size_t qemu_get_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, off_t pos); QIOChannel *qemu_file_get_ioc(QEMUFile *file); +int qemu_file_put_fd(QEMUFile *f, int fd); +int qemu_file_get_fd(QEMUFile *f); #endif diff --git a/migration/trace-events b/migration/trace-events index 89c0244..9388f81 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -88,6 +88,8 @@ put_qlist_end(const char *field_name, const char *vmsd_name) "%s(%s)" # qemu-file.c qemu_file_fclose(void) "" +qemu_file_put_fd(const char *name, int fd, int ret) "ioc %s, fd %d -> status %d" +qemu_file_get_fd(const char *name, int fd) "ioc %s -> fd %d" # ram.c get_queued_page(const char *block_name, uint64_t tmp_offset, unsigned long page_abs) "%s/0x%" PRIx64 " page_abs=0x%lx" From patchwork Mon Dec 2 13:20:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890688 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AA43ED78322 for ; Mon, 2 Dec 2024 13:45:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Zv-0005Oq-4U; Mon, 02 Dec 2024 08:35:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lo-00081m-O8 for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:41 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lj-0005nw-5X for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:30 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26X0YZ008619; Mon, 2 Dec 2024 13:20:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=7OQmLlx/PoPLowHM1tzFKaWQQg/KV0GS0o4j8Vd8xL0=; b= JlwHAyXfXuj7MYbpLxfW5WGSB/kEy7XVH4whlSSNSr1ypcFjbJyjQcgtc4ptP9Yk 4LUT/TcUYgRGwK4vmevurtpWCSRMRiYFzp56IwS7r0jwEJ5CLBxtE/ygi2tI1YjD rmqBu8vtHSphCDysf3xc/oqcRiTzDoeAE2DK1igVwRLMb+wgRzTBtipV6OMhYd7s 40+hzyFEJheOCqpJaCdouSdA9Om0PpxfBxlZKAwYRNSr+5v+rVHHDRVR93BlwkCk Kid7TI8T8MIvGGx9EsUyRr6tpLfwItNr6XirvsNa2Qt6j6S907vHCdMAIsXySDg7 d+uDPOT4kofXcJOVWrEORA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437s9ytymk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:23 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2CFJhN032114; Mon, 2 Dec 2024 13:20:22 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jts5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:22 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEv032806; Mon, 2 Dec 2024 13:20:22 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-13; Mon, 02 Dec 2024 13:20:21 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 12/19] migration: VMSTATE_FD Date: Mon, 2 Dec 2024 05:20:04 -0800 Message-Id: <1733145611-62315-13-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-GUID: M2o9xaSHQbRZAE62WGbzUFVWho6UTEfi X-Proofpoint-ORIG-GUID: M2o9xaSHQbRZAE62WGbzUFVWho6UTEfi Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Define VMSTATE_FD for declaring a file descriptor field in a VMStateDescription. Signed-off-by: Steve Sistare Reviewed-by: Peter Xu --- include/migration/vmstate.h | 9 +++++++++ migration/vmstate-types.c | 23 +++++++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index f313f2f..a1dfab4 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -230,6 +230,7 @@ extern const VMStateInfo vmstate_info_uint8; extern const VMStateInfo vmstate_info_uint16; extern const VMStateInfo vmstate_info_uint32; extern const VMStateInfo vmstate_info_uint64; +extern const VMStateInfo vmstate_info_fd; /** Put this in the stream when migrating a null pointer.*/ #define VMS_NULLPTR_MARKER (0x30U) /* '0' */ @@ -902,6 +903,9 @@ extern const VMStateInfo vmstate_info_qlist; #define VMSTATE_UINT64_V(_f, _s, _v) \ VMSTATE_SINGLE(_f, _s, _v, vmstate_info_uint64, uint64_t) +#define VMSTATE_FD_V(_f, _s, _v) \ + VMSTATE_SINGLE(_f, _s, _v, vmstate_info_fd, int32_t) + #ifdef CONFIG_LINUX #define VMSTATE_U8_V(_f, _s, _v) \ @@ -936,6 +940,9 @@ extern const VMStateInfo vmstate_info_qlist; #define VMSTATE_UINT64(_f, _s) \ VMSTATE_UINT64_V(_f, _s, 0) +#define VMSTATE_FD(_f, _s) \ + VMSTATE_FD_V(_f, _s, 0) + #ifdef CONFIG_LINUX #define VMSTATE_U8(_f, _s) \ @@ -1009,6 +1016,8 @@ extern const VMStateInfo vmstate_info_qlist; #define VMSTATE_UINT64_TEST(_f, _s, _t) \ VMSTATE_SINGLE_TEST(_f, _s, _t, 0, vmstate_info_uint64, uint64_t) +#define VMSTATE_FD_TEST(_f, _s, _t) \ + VMSTATE_SINGLE_TEST(_f, _s, _t, 0, vmstate_info_fd, int32_t) #define VMSTATE_TIMER_PTR_TEST(_f, _s, _test) \ VMSTATE_POINTER_TEST(_f, _s, _test, vmstate_info_timer, QEMUTimer *) diff --git a/migration/vmstate-types.c b/migration/vmstate-types.c index e83bfcc..f31deb3 100644 --- a/migration/vmstate-types.c +++ b/migration/vmstate-types.c @@ -314,6 +314,29 @@ const VMStateInfo vmstate_info_uint64 = { .put = put_uint64, }; +/* File descriptor communicated via SCM_RIGHTS */ + +static int get_fd(QEMUFile *f, void *pv, size_t size, + const VMStateField *field) +{ + int32_t *v = pv; + *v = qemu_file_get_fd(f); + return 0; +} + +static int put_fd(QEMUFile *f, void *pv, size_t size, + const VMStateField *field, JSONWriter *vmdesc) +{ + int32_t *v = pv; + return qemu_file_put_fd(f, *v); +} + +const VMStateInfo vmstate_info_fd = { + .name = "fd", + .get = get_fd, + .put = put_fd, +}; + static int get_nullptr(QEMUFile *f, void *pv, size_t size, const VMStateField *field) From patchwork Mon Dec 2 13:20:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B21B2D78321 for ; Mon, 2 Dec 2024 13:43:05 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Wg-0006k0-If; Mon, 02 Dec 2024 08:31:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lw-00082P-Pz for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:47 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lo-0005oM-3d for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:39 -0500 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26XhdX006129; Mon, 2 Dec 2024 13:20:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=ifO6Q95YO4NBRU+rtzLgiEN95kUc7QMXkoyWvLHpnt8=; b= E5F7FV2g41D2u62FJwjBHKn67eAC7Pt5uG75fa+Mli/m5GdgrD+LaVBJVDIZgR43 5o/T8/Ilx/EGSc27/xtNgIZzhP/m7uNyZQR98p10mbrUw27p4Mfne2BXzSM+3LDl ILMwWGYU66gWx/70GaTzyxgtwNI5ZhvfCj7j7O/u5e8l07eDr+bIVW+73oAfxGeZ qjr/helPIw2Uv61IOo8NLywGFqDardHViCUSjrG8VQD4KIi9HfAetyEqZZzQ3c3j a+hks0x3PWBDpq5RByqTgTqNpcBIj/Ivfbi9WHCCytLReI03piRisde18BDdfH/e qdG3tCDF8P7KnN9Jnk4lAw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437tk8txgy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:23 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2C6OXX032775; Mon, 2 Dec 2024 13:20:23 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtst-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:23 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCEx032806; Mon, 2 Dec 2024 13:20:22 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-14; Mon, 02 Dec 2024 13:20:22 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 13/19] migration: cpr-transfer save and load Date: Mon, 2 Dec 2024 05:20:05 -0800 Message-Id: <1733145611-62315-14-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: UZDToqFlPDqH6gwxVHpBtJU9idD43S5w X-Proofpoint-GUID: UZDToqFlPDqH6gwxVHpBtJU9idD43S5w Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add functions to create a QEMUFile based on a unix URI, for saving or loading, for use by cpr-transfer mode to preserve CPR state. Signed-off-by: Steve Sistare Reviewed-by: Peter Xu --- include/migration/cpr.h | 3 ++ migration/cpr-transfer.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++ migration/meson.build | 1 + migration/trace-events | 2 ++ 4 files changed, 82 insertions(+) create mode 100644 migration/cpr-transfer.c diff --git a/include/migration/cpr.h b/include/migration/cpr.h index e833fae..51ac7f4 100644 --- a/include/migration/cpr.h +++ b/include/migration/cpr.h @@ -25,4 +25,7 @@ int cpr_state_load(Error **errp); void cpr_state_close(void); struct QIOChannel *cpr_state_ioc(void); +QEMUFile *cpr_transfer_output(MigrationChannel *channel, Error **errp); +QEMUFile *cpr_transfer_input(MigrationChannel *channel, Error **errp); + #endif diff --git a/migration/cpr-transfer.c b/migration/cpr-transfer.c new file mode 100644 index 0000000..0fbdf66 --- /dev/null +++ b/migration/cpr-transfer.c @@ -0,0 +1,76 @@ +/* + * Copyright (c) 2022, 2024 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "io/channel-file.h" +#include "io/channel-socket.h" +#include "io/net-listener.h" +#include "migration/cpr.h" +#include "migration/migration.h" +#include "migration/savevm.h" +#include "migration/qemu-file.h" +#include "migration/vmstate.h" +#include "trace.h" + +QEMUFile *cpr_transfer_output(MigrationChannel *channel, Error **errp) +{ + QIOChannel *ioc; + MigrationAddress *addr = channel->addr; + + if (addr->transport == MIGRATION_ADDRESS_TYPE_SOCKET && + addr->u.socket.type == SOCKET_ADDRESS_TYPE_UNIX) { + + QIOChannelSocket *sioc = qio_channel_socket_new(); + SocketAddress *saddr = &addr->u.socket; + + if (qio_channel_socket_connect_sync(sioc, saddr, errp)) { + object_unref(OBJECT(sioc)); + return NULL; + } + ioc = QIO_CHANNEL(sioc); + + } else { + error_setg(errp, "bad cpr channel address; must be unix"); + return NULL; + } + + trace_cpr_transfer_output(addr->u.socket.u.q_unix.path); + qio_channel_set_name(ioc, "cpr-out"); + return qemu_file_new_output(ioc); +} + +QEMUFile *cpr_transfer_input(MigrationChannel *channel, Error **errp) +{ + QIOChannel *ioc; + MigrationAddress *addr = channel->addr; + + if (addr->transport == MIGRATION_ADDRESS_TYPE_SOCKET && + addr->u.socket.type == SOCKET_ADDRESS_TYPE_UNIX) { + + QIOChannelSocket *sioc; + SocketAddress *saddr = &addr->u.socket; + QIONetListener *listener = qio_net_listener_new(); + + qio_net_listener_set_name(listener, "cpr-socket-listener"); + if (qio_net_listener_open_sync(listener, saddr, 1, errp) < 0) { + object_unref(OBJECT(listener)); + return NULL; + } + + sioc = qio_net_listener_wait_client(listener); + ioc = QIO_CHANNEL(sioc); + + } else { + error_setg(errp, "bad cpr channel socket type; must be unix"); + return NULL; + } + + trace_cpr_transfer_input(addr->u.socket.u.q_unix.path); + qio_channel_set_name(ioc, "cpr-in"); + return qemu_file_new_input(ioc); +} diff --git a/migration/meson.build b/migration/meson.build index 039f0f9..d89435b 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -14,6 +14,7 @@ system_ss.add(files( 'channel.c', 'channel-block.c', 'cpr.c', + 'cpr-transfer.c', 'cpu-throttle.c', 'dirtyrate.c', 'exec.c', diff --git a/migration/trace-events b/migration/trace-events index 9388f81..1dd394d 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -350,6 +350,8 @@ cpr_delete_fd(const char *name, int id) "%s, id %d" cpr_find_fd(const char *name, int id, int fd) "%s, id %d returns %d" cpr_state_save(const char *mode) "%s mode" cpr_state_load(const char *mode) "%s mode" +cpr_transfer_input(const char *path) "%s" +cpr_transfer_output(const char *path) "%s" # block-dirty-bitmap.c send_bitmap_header_enter(void) "" From patchwork Mon Dec 2 13:20:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890674 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EDB93D78321 for ; Mon, 2 Dec 2024 13:41:30 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Zz-0005bs-Qj; Mon, 02 Dec 2024 08:35:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lw-00082N-Mj for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:47 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Ll-0005oI-1K for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:32 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2DIY85012846; Mon, 2 Dec 2024 13:20:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=kIJQkFkP2pXfWartjKaoYqVVOADcqox68MZZCrXT9F8=; b= Y8J33yU5ZY870LFpOqAyQpl70jUSPbEcxvLe7FNkOdgXkuS1OsEvjBn8vI+/KPGV PkyNPkxlFbICPhovyHZGhdFqY53eDEJ/FbRb/KsOv6/TyCienLOyJr9v8hb/Mxuz pPoFNJ4m9mDtgluONlEUW6eEeWpcv9Ve/TVzi+ArD+rTcPNnp5042uwr8VKLT80N /YcWigTLbt1M8PQq+piB0Tgz5eEZ47qUEHbe+wO8SifZAo2HzvmJeuZO1iBpW4j1 yo8kMzoUCa1Mjtw7YMhcuMzbMQVL+aLvZ4+o8zUz+guXEvWSarNRiIVKdtpHYQ22 0o16pqUOFbCR98R7ALo8tg== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437s4c334d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:25 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2CjRJA032128; Mon, 2 Dec 2024 13:20:24 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtt8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:24 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCF1032806; Mon, 2 Dec 2024 13:20:23 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-15; Mon, 02 Dec 2024 13:20:23 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 14/19] migration: cpr-transfer mode Date: Mon, 2 Dec 2024 05:20:06 -0800 Message-Id: <1733145611-62315-15-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: cmWpJV6Jw1wxJDjedXcLB531kSvVA0-V X-Proofpoint-GUID: cmWpJV6Jw1wxJDjedXcLB531kSvVA0-V Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add the cpr-transfer migration mode. Usage: qemu-system-$arch -machine aux-ram-share=on ... start new QEMU with "-incoming -incoming " Issue commands to old QEMU: migrate_set_parameter mode cpr-transfer {"execute": "migrate", ... {"channel-type": "main"...}, {"channel-type": "cpr"...} ... } The migrate command stops the VM, saves CPR state to cpr-channel, saves normal migration state to main-uri, and old QEMU enters the postmigrate state. The user starts new QEMU on the same host as old QEMU, with the same arguments as old QEMU, plus two -incoming options. Guest RAM is preserved in place, albeit with new virtual addresses in new QEMU. This mode requires a second migration channel of type "cpr", in the channel arguments on the outgoing side, and in a second -incoming command-line parameter on the incoming side. Memory-backend objects must have the share=on attribute, but memory-backend-epc is not supported. The VM must be started with the '-machine aux-ram-share=on' option, which allows anonymous memory to be transferred in place to the new process. The memfds are kept open by sending the descriptors to new QEMU via the CPR channel, which must support SCM_RIGHTS, and they are mmap'd in new QEMU. The implementation splits qmp_migrate into start and finish functions. Start sends CPR state to new QEMU, which responds by closing the CPR channel. Old QEMU detects the HUP then calls finish, which connects the main migration channel. Signed-off-by: Steve Sistare --- migration/cpr.c | 23 +++++++--- migration/migration.c | 107 +++++++++++++++++++++++++++++++++++++++++++++- migration/migration.h | 2 + migration/options.c | 12 +++++- migration/ram.c | 2 + migration/vmstate-types.c | 1 + qapi/migration.json | 37 +++++++++++++++- stubs/vmstate.c | 7 +++ system/vl.c | 6 +++ 9 files changed, 188 insertions(+), 9 deletions(-) diff --git a/migration/cpr.c b/migration/cpr.c index f4a795f..560546d 100644 --- a/migration/cpr.c +++ b/migration/cpr.c @@ -45,7 +45,7 @@ static const VMStateDescription vmstate_cpr_fd = { VMSTATE_UINT32(namelen, CprFd), VMSTATE_VBUFFER_ALLOC_UINT32(name, CprFd, 0, NULL, namelen), VMSTATE_INT32(id, CprFd), - VMSTATE_INT32(fd, CprFd), + VMSTATE_FD(fd, CprFd), VMSTATE_END_OF_LIST() } }; @@ -139,8 +139,14 @@ int cpr_state_save(MigrationChannel *channel, Error **errp) trace_cpr_state_save(MigMode_str(mode)); - /* set f based on mode in a later patch in this series */ - return 0; + if (mode == MIG_MODE_CPR_TRANSFER) { + f = cpr_transfer_output(channel, errp); + } else { + return 0; + } + if (!f) { + return -1; + } qemu_put_be32(f, QEMU_CPR_FILE_MAGIC); qemu_put_be32(f, QEMU_CPR_FILE_VERSION); @@ -170,8 +176,15 @@ int cpr_state_load(Error **errp) QEMUFile *f; MigMode mode = 0; - /* set f and mode based on other parameters later in this patch series */ - return 0; + if (cpr_channel) { + mode = MIG_MODE_CPR_TRANSFER; + f = cpr_transfer_input(cpr_channel, errp); + } else { + return 0; + } + if (!f) { + return -1; + } trace_cpr_state_load(MigMode_str(mode)); diff --git a/migration/migration.c b/migration/migration.c index a5cf148..81a1ced 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -76,6 +76,7 @@ static NotifierWithReturnList migration_state_notifiers[] = { NOTIFIER_ELEM_INIT(migration_state_notifiers, MIG_MODE_NORMAL), NOTIFIER_ELEM_INIT(migration_state_notifiers, MIG_MODE_CPR_REBOOT), + NOTIFIER_ELEM_INIT(migration_state_notifiers, MIG_MODE_CPR_TRANSFER), }; /* Messages sent on the return path from destination to source */ @@ -109,6 +110,7 @@ static int migration_maybe_pause(MigrationState *s, static void migrate_fd_cancel(MigrationState *s); static bool close_return_path_on_source(MigrationState *s); static void migration_completion_end(MigrationState *s); +static void migrate_hup_delete(MigrationState *s); static void migration_downtime_start(MigrationState *s) { @@ -204,6 +206,12 @@ migration_channels_and_transport_compatible(MigrationAddress *addr, return false; } + if (migrate_mode() == MIG_MODE_CPR_TRANSFER && + addr->transport == MIGRATION_ADDRESS_TYPE_FILE) { + error_setg(errp, "Migration requires streamable transport (eg unix)"); + return false; + } + return true; } @@ -319,6 +327,7 @@ void migration_cancel(const Error *error) qmp_cancel_vcpu_dirty_limit(false, -1, NULL); } migrate_fd_cancel(current_migration); + migrate_hup_delete(current_migration); } void migration_shutdown(void) @@ -419,6 +428,7 @@ void migration_incoming_state_destroy(void) mis->postcopy_qemufile_dst = NULL; } + cpr_set_cpr_channel(NULL); yank_unregister_instance(MIGRATION_YANK_INSTANCE); } @@ -720,6 +730,9 @@ static void qemu_start_incoming_migration(const char *uri, bool has_channels, } else { error_setg(errp, "unknown migration protocol: %s", uri); } + + /* Close cpr socket to tell source that we are listening */ + cpr_state_close(); } static void process_incoming_migration_bh(void *opaque) @@ -1395,6 +1408,8 @@ static void migrate_fd_cleanup(MigrationState *s) s->vmdesc = NULL; qemu_savevm_state_cleanup(); + cpr_state_close(); + migrate_hup_delete(s); close_return_path_on_source(s); @@ -1506,6 +1521,7 @@ static void migrate_fd_error(MigrationState *s, const Error *error) static void migrate_fd_cancel(MigrationState *s) { int old_state ; + bool setup = (s->state == MIGRATION_STATUS_SETUP); trace_migrate_fd_cancel(); @@ -1550,6 +1566,17 @@ static void migrate_fd_cancel(MigrationState *s) s->block_inactive = false; } } + + /* + * If qmp_migrate_finish has not been called, then there is no path that + * will complete the cancellation. Do it now. + */ + if (setup && !s->to_dst_file) { + migrate_set_state(&s->state, s->state, MIGRATION_STATUS_CANCELLED); + cpr_state_close(); + migrate_hup_delete(s); + vm_resume(s->vm_old_state); + } } void migration_add_notifier_mode(NotifierWithReturn *notify, @@ -1662,7 +1689,9 @@ bool migration_thread_is_self(void) bool migrate_mode_is_cpr(MigrationState *s) { - return s->parameters.mode == MIG_MODE_CPR_REBOOT; + MigMode mode = s->parameters.mode; + return mode == MIG_MODE_CPR_REBOOT || + mode == MIG_MODE_CPR_TRANSFER; } int migrate_init(MigrationState *s, Error **errp) @@ -2037,6 +2066,40 @@ static bool migrate_prepare(MigrationState *s, bool resume, Error **errp) return true; } +static void qmp_migrate_finish(MigrationAddress *addr, bool resume_requested, + Error **errp); + +static void migrate_hup_add(MigrationState *s, QIOChannel *ioc, GSourceFunc cb, + void *opaque) +{ + s->hup_source = qio_channel_create_watch(ioc, G_IO_HUP); + g_source_set_callback(s->hup_source, cb, opaque, NULL); + g_source_attach(s->hup_source, NULL); +} + +static void migrate_hup_delete(MigrationState *s) +{ + if (s->hup_source) { + g_source_destroy(s->hup_source); + g_source_unref(s->hup_source); + s->hup_source = NULL; + } +} + +static gboolean qmp_migrate_finish_cb(QIOChannel *channel, + GIOCondition cond, + void *opaque) +{ + MigrationAddress *addr = opaque; + + qmp_migrate_finish(addr, false, NULL); + + cpr_state_close(); + migrate_hup_delete(migrate_get_current()); + qapi_free_MigrationAddress(addr); + return G_SOURCE_REMOVE; +} + void qmp_migrate(const char *uri, bool has_channels, MigrationChannelList *channels, bool has_detach, bool detach, bool has_resume, bool resume, Error **errp) @@ -2047,6 +2110,7 @@ void qmp_migrate(const char *uri, bool has_channels, g_autoptr(MigrationChannel) channel = NULL; MigrationAddress *addr = NULL; MigrationChannel *channelv[MIGRATION_CHANNEL_TYPE__MAX] = { NULL }; + MigrationChannel *cpr_channel = NULL; /* * Having preliminary checks for uri and channel @@ -2067,6 +2131,7 @@ void qmp_migrate(const char *uri, bool has_channels, } channelv[type] = channels->value; } + cpr_channel = channelv[MIGRATION_CHANNEL_TYPE_CPR]; addr = channelv[MIGRATION_CHANNEL_TYPE_MAIN]->addr; if (!addr) { error_setg(errp, "Channel list has no main entry"); @@ -2087,12 +2152,52 @@ void qmp_migrate(const char *uri, bool has_channels, return; } + if (s->parameters.mode == MIG_MODE_CPR_TRANSFER && !cpr_channel) { + error_setg(errp, "missing 'cpr' migration channel"); + return; + } + resume_requested = has_resume && resume; if (!migrate_prepare(s, resume_requested, errp)) { /* Error detected, put into errp */ return; } + if (cpr_state_save(cpr_channel, &local_err)) { + goto out; + } + + /* + * For cpr-transfer, the target may not be listening yet on the migration + * channel, because first it must finish cpr_load_state. The target tells + * us it is listening by closing the cpr-state socket. Wait for that HUP + * event before connecting in qmp_migrate_finish. + * + * The HUP could occur because the target fails while reading CPR state, + * in which case the target will not listen for the incoming migration + * connection, so qmp_migrate_finish will fail to connect, and then recover. + */ + if (s->parameters.mode == MIG_MODE_CPR_TRANSFER) { + migrate_hup_add(s, cpr_state_ioc(), (GSourceFunc)qmp_migrate_finish_cb, + QAPI_CLONE(MigrationAddress, addr)); + + } else { + qmp_migrate_finish(addr, resume_requested, errp); + } + +out: + if (local_err) { + migrate_fd_error(s, local_err); + error_propagate(errp, local_err); + } +} + +static void qmp_migrate_finish(MigrationAddress *addr, bool resume_requested, + Error **errp) +{ + MigrationState *s = migrate_get_current(); + Error *local_err = NULL; + if (!resume_requested) { if (!yank_register_instance(MIGRATION_YANK_INSTANCE, errp)) { return; diff --git a/migration/migration.h b/migration/migration.h index 5cd0f29..bdae228 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -471,6 +471,8 @@ struct MigrationState { bool switchover_acked; /* Is this a rdma migration */ bool rdma_migration; + + GSource *hup_source; }; void migrate_set_state(MigrationStatus *state, MigrationStatus old_state, diff --git a/migration/options.c b/migration/options.c index ad8d698..bb24bc9 100644 --- a/migration/options.c +++ b/migration/options.c @@ -22,6 +22,7 @@ #include "qapi/qmp/qnull.h" #include "sysemu/runstate.h" #include "migration/colo.h" +#include "migration/cpr.h" #include "migration/misc.h" #include "migration.h" #include "migration-stats.h" @@ -745,9 +746,16 @@ uint64_t migrate_max_postcopy_bandwidth(void) MigMode migrate_mode(void) { - MigrationState *s = migrate_get_current(); - MigMode mode = s->parameters.mode; + MigMode mode; + /* + * cpr_channel is only set during the early cpr-transfer loading stage, + * after which it is cleared. + */ + if (cpr_get_cpr_channel()) { + return MIG_MODE_CPR_TRANSFER; + } + mode = migrate_get_current()->parameters.mode; assert(mode >= 0 && mode < MIG_MODE__MAX); return mode; } diff --git a/migration/ram.c b/migration/ram.c index 05ff9eb..fa6a909 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -216,7 +216,9 @@ static bool postcopy_preempt_active(void) bool migrate_ram_is_ignored(RAMBlock *block) { + MigMode mode = migrate_mode(); return !qemu_ram_is_migratable(block) || + mode == MIG_MODE_CPR_TRANSFER || (migrate_ignore_shared() && qemu_ram_is_shared(block) && qemu_ram_is_named_file(block)); } diff --git a/migration/vmstate-types.c b/migration/vmstate-types.c index f31deb3..2210f0c 100644 --- a/migration/vmstate-types.c +++ b/migration/vmstate-types.c @@ -15,6 +15,7 @@ #include "qemu-file.h" #include "migration.h" #include "migration/vmstate.h" +#include "migration/client-options.h" #include "qemu/error-report.h" #include "qemu/queue.h" #include "trace.h" diff --git a/qapi/migration.json b/qapi/migration.json index a26960b..1bc963f 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -614,9 +614,44 @@ # or COLO. # # (since 8.2) +# +# @cpr-transfer: This mode allows the user to transfer a guest to a +# new QEMU instance on the same host with minimal guest pause +# time, by preserving guest RAM in place, albeit with new virtual +# addresses in new QEMU. +# +# The user starts new QEMU on the same host as old QEMU, with the +# the same arguments as old QEMU, plus the -incoming option. The +# user issues the migrate command to old QEMU, which stops the VM, +# saves state to the migration channels, and enters the +# postmigrate state. Execution resumes in new QEMU. +# +# This mode requires a second migration channel type "cpr" in the +# channel arguments on the outgoing side. The channel must be a +# type, such as unix socket, that supports SCM_RIGHTS. However, +# the cpr channel cannot be added to the list of channels for a +# migrate-incoming command, because it must be read before new +# QEMU opens a monitor. Instead, the user passes the channel as a +# second -incoming command-line argument to new QEMU using JSON +# syntax. +# +# Memory-backend objects must have the share=on attribute, but +# memory-backend-epc is not supported. The VM must be started +# with the '-machine aux-ram-share=on' option. +# +# The incoming migration channel cannot be a file type, and for +# the tcp type, the port cannot be 0 (meaning dynamically choose +# a port). +# +# When using -incoming defer, you must issue the migrate command +# to old QEMU before issuing any monitor commands to new QEMU. +# However, new QEMU does not open and read the migration stream +# until you issue the migrate incoming command. +# +# (since 10.0) ## { 'enum': 'MigMode', - 'data': [ 'normal', 'cpr-reboot' ] } + 'data': [ 'normal', 'cpr-reboot', 'cpr-transfer' ] } ## # @ZeroPageDetection: diff --git a/stubs/vmstate.c b/stubs/vmstate.c index 8513d92..c190762 100644 --- a/stubs/vmstate.c +++ b/stubs/vmstate.c @@ -1,5 +1,7 @@ #include "qemu/osdep.h" #include "migration/vmstate.h" +#include "qapi/qapi-types-migration.h" +#include "migration/client-options.h" int vmstate_register_with_alias_id(VMStateIf *obj, uint32_t instance_id, @@ -21,3 +23,8 @@ bool vmstate_check_only_migratable(const VMStateDescription *vmsd) { return true; } + +MigMode migrate_mode(void) +{ + return MIG_MODE_NORMAL; +} diff --git a/system/vl.c b/system/vl.c index 40e049e..d965be8 100644 --- a/system/vl.c +++ b/system/vl.c @@ -3741,6 +3741,12 @@ void qemu_init(int argc, char **argv) qemu_create_machine(machine_opts_dict); + /* + * Load incoming CPR state before any devices are created, because it + * contains file descriptors that are needed in device initialization code. + */ + cpr_state_load(&error_fatal); + suspend_mux_open(); qemu_disable_default_devices(); From patchwork Mon Dec 2 13:20:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9D8D0D78321 for ; Mon, 2 Dec 2024 13:37:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Zz-0005cA-So; Mon, 02 Dec 2024 08:35:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lw-00082O-OX for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:47 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Ln-0005qF-IN for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:36 -0500 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26WwwF016645; Mon, 2 Dec 2024 13:20:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=ozP1Yoc6awP/psqrTDYAdBGKGPDf1Mz9oBrsbPNytDw=; b= fzXFce+07enPsPQSRKuwKmpbVPXORGergurXJHBzwqzE9pPE6mt729vxhQPRzPoj bGiyowvYgJD9QqmO1apQxZEXZPrDLkG+nbN6ojp8nTdoC9xlVWj6Bx80D5mi2OsK yxqIfwK/dFcLT35uxf/ShEi0Lgn7KrBb04auoISLhuxuR7JFEATLpgKXCZQm2+da CrGzqjB1Av4KQ9D/swEuRBra9Ed32GxGekISyUjzgbz2arQMwa204PwOKFNdskm2 HdVUSXn2xPjFP6OEH4Lb+MRaPmC3B1UYVcMwJfu1l9gvQL6TJHcubcdl9Pxti7q2 uuOHV/KzH/fuzuvpk34A4A== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437u8t2xtw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:25 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2C2cLu032113; Mon, 2 Dec 2024 13:20:24 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jttr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:24 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCF3032806; Mon, 2 Dec 2024 13:20:24 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-16; Mon, 02 Dec 2024 13:20:24 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 15/19] tests/migration-test: memory_backend Date: Mon, 2 Dec 2024 05:20:07 -0800 Message-Id: <1733145611-62315-16-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: YhYZAIzkvm0W5JLO-YZkL-nLDP6f7yEo X-Proofpoint-GUID: YhYZAIzkvm0W5JLO-YZkL-nLDP6f7yEo Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Allow each migration test to define its own memory backend, replacing the standard "-m " specification. Signed-off-by: Steve Sistare Reviewed-by: Fabiano Rosas --- tests/qtest/migration-test.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 74d3000..64e1c50 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -609,6 +609,11 @@ typedef struct { const char *opts_target; /* suspend the src before migrating to dest. */ bool suspend_me; + /* + * Format string for the main memory backend, containing one %s where the + * size is plugged in. If omitted, "-m %s" is used. + */ + const char *memory_backend; } MigrateStart; /* @@ -727,6 +732,7 @@ static int test_migrate_start(QTestState **from, QTestState **to, const char *memory_size; const char *machine_alias, *machine_opts = ""; g_autofree char *machine = NULL; + g_autofree char *memory_backend = NULL; if (args->use_shmem) { if (!g_file_test("/dev/shm", G_FILE_TEST_IS_DIR)) { @@ -802,6 +808,12 @@ static int test_migrate_start(QTestState **from, QTestState **to, memory_size, shmem_path); } + if (args->memory_backend) { + memory_backend = g_strdup_printf(args->memory_backend, memory_size); + } else { + memory_backend = g_strdup_printf("-m %s ", memory_size); + } + if (args->use_dirty_ring) { kvm_opts = ",dirty-ring-size=4096"; } @@ -820,12 +832,12 @@ static int test_migrate_start(QTestState **from, QTestState **to, cmd_source = g_strdup_printf("-accel kvm%s -accel tcg " "-machine %s,%s " "-name source,debug-threads=on " - "-m %s " + "%s " "-serial file:%s/src_serial " "%s %s %s %s %s", kvm_opts ? kvm_opts : "", machine, machine_opts, - memory_size, tmpfs, + memory_backend, tmpfs, arch_opts ? arch_opts : "", arch_source ? arch_source : "", shmem_opts ? shmem_opts : "", @@ -841,13 +853,13 @@ static int test_migrate_start(QTestState **from, QTestState **to, cmd_target = g_strdup_printf("-accel kvm%s -accel tcg " "-machine %s,%s " "-name target,debug-threads=on " - "-m %s " + "%s " "-serial file:%s/dest_serial " "-incoming %s " "%s %s %s %s %s", kvm_opts ? kvm_opts : "", machine, machine_opts, - memory_size, tmpfs, uri, + memory_backend, tmpfs, uri, arch_opts ? arch_opts : "", arch_target ? arch_target : "", shmem_opts ? shmem_opts : "", From patchwork Mon Dec 2 13:20:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 56191D78320 for ; Mon, 2 Dec 2024 13:36:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6a3-0005o3-FD; Mon, 02 Dec 2024 08:35:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6M4-00086G-FD for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:50 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lw-0005qA-MQ for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:42 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26XFUj013671; Mon, 2 Dec 2024 13:20:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=dZPhlS1YCwPtdJZdfwGT937kznXrOYhmpvBlJatq8KE=; b= Y21MOLlapQfGTqYB8Ua5K3l25FJN5c9vgAFwwU9OKliSX86xfsH3EqdV44k/ju/9 IsI3T5gvYR5H+/oi9rzHITD24ke7zgXpY1ItEkQ6/FWt9Y4anh61JGfNz9zth2kS s5aW4uZrxTYPQ/xhc8SPaK63SNxQMgc8tmvc9ugyVv9xFGlrA0O4AWFMOfVDsMvU Xl8HnxZ06NNsdtnkQjgqAdVgWg3Z92M8fxZd/WDUIOzu9k1QxRv7JIzvdEjI9fZt fGZVrtqo/4GgXw67PYeGG7aZdFsLuFX62zuKM3SRKe+saoi9t96OWpJlyDJWAlub P/r4lTJxujmMLix7ha5zJQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437s4c334h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:26 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2CLJX9032060; Mon, 2 Dec 2024 13:20:25 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtua-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:25 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCF5032806; Mon, 2 Dec 2024 13:20:25 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-17; Mon, 02 Dec 2024 13:20:24 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 16/19] tests/qtest: defer connection Date: Mon, 2 Dec 2024 05:20:08 -0800 Message-Id: <1733145611-62315-17-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: -19NQH7eSHqUd1Rz7qJ7uX4l6-6qlLk1 X-Proofpoint-GUID: -19NQH7eSHqUd1Rz7qJ7uX4l6-6qlLk1 Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add an option to defer making the connecting to the monitor and qtest sockets when calling qtest_init_with_env. The client makes the connection later by calling qtest_connect_deferred and qtest_qmp_handshake. A test cannot specify port=0 for a deferred connection, because qmp_migrate cannot query for the assigned port, because the monitor is not connected yet. However, even if the test does not specify port=0, qmp_migrate -> migrate_set_ports unconditionally queries connection parameters. Modify migrate_set_ports to only query when port=0. Signed-off-by: Steve Sistare --- tests/qtest/libqtest.c | 80 +++++++++++++++++++++++++---------------- tests/qtest/libqtest.h | 19 +++++++++- tests/qtest/migration-helpers.c | 19 +++++----- tests/qtest/migration-test.c | 4 +-- 4 files changed, 80 insertions(+), 42 deletions(-) diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c index 817fd7a..31c4032 100644 --- a/tests/qtest/libqtest.c +++ b/tests/qtest/libqtest.c @@ -75,6 +75,8 @@ struct QTestState { int fd; int qmp_fd; + int sock; + int qmpsock; pid_t qemu_pid; /* our child QEMU process */ int wstatus; #ifdef _WIN32 @@ -442,18 +444,19 @@ static QTestState *G_GNUC_PRINTF(2, 3) qtest_spawn_qemu(const char *qemu_bin, return s; } +static char *qtest_socket_path(const char *suffix) +{ + return g_strdup_printf("%s/qtest-%d.%s", g_get_tmp_dir(), getpid(), suffix); +} + static QTestState *qtest_init_internal(const char *qemu_bin, - const char *extra_args) + const char *extra_args, + bool defer_connect) { QTestState *s; int sock, qmpsock, i; - gchar *socket_path; - gchar *qmp_socket_path; - - socket_path = g_strdup_printf("%s/qtest-%d.sock", - g_get_tmp_dir(), getpid()); - qmp_socket_path = g_strdup_printf("%s/qtest-%d.qmp", - g_get_tmp_dir(), getpid()); + g_autofree gchar *socket_path = qtest_socket_path("sock"); + g_autofree gchar *qmp_socket_path = qtest_socket_path("qmp"); /* * It's possible that if an earlier test run crashed it might @@ -485,22 +488,17 @@ static QTestState *qtest_init_internal(const char *qemu_bin, qtest_client_set_rx_handler(s, qtest_client_socket_recv_line); qtest_client_set_tx_handler(s, qtest_client_socket_send); - s->fd = socket_accept(sock); - if (s->fd >= 0) { - s->qmp_fd = socket_accept(qmpsock); - } - unlink(socket_path); - unlink(qmp_socket_path); - g_free(socket_path); - g_free(qmp_socket_path); - - g_assert(s->fd >= 0 && s->qmp_fd >= 0); - s->rx = g_string_new(""); for (i = 0; i < MAX_IRQ; i++) { s->irq_level[i] = false; } + s->sock = sock; + s->qmpsock = qmpsock; + if (!defer_connect) { + qtest_connect_deferred(s); + } + /* * Stopping QEMU for debugging is not supported on Windows. * @@ -515,34 +513,54 @@ static QTestState *qtest_init_internal(const char *qemu_bin, } #endif + return s; +} + +void qtest_connect_deferred(QTestState *s) +{ + g_autofree gchar *socket_path = qtest_socket_path("sock"); + g_autofree gchar *qmp_socket_path = qtest_socket_path("qmp"); + + g_assert(s->sock >= 0 && s->qmpsock >= 0); + s->fd = socket_accept(s->sock); + if (s->fd >= 0) { + s->qmp_fd = socket_accept(s->qmpsock); + } + unlink(socket_path); + unlink(qmp_socket_path); + g_assert(s->fd >= 0 && s->qmp_fd >= 0); + s->sock = s->qmpsock = -1; /* ask endianness of the target */ - s->big_endian = qtest_query_target_endianness(s); - - return s; } QTestState *qtest_init_without_qmp_handshake(const char *extra_args) { - return qtest_init_internal(qtest_qemu_binary(NULL), extra_args); + return qtest_init_internal(qtest_qemu_binary(NULL), extra_args, false); } -QTestState *qtest_init_with_env(const char *var, const char *extra_args) +void qtest_qmp_handshake(QTestState *s) { - QTestState *s = qtest_init_internal(qtest_qemu_binary(var), extra_args); - QDict *greeting; - /* Read the QMP greeting and then do the handshake */ - greeting = qtest_qmp_receive(s); + QDict *greeting = qtest_qmp_receive(s); qobject_unref(greeting); qobject_unref(qtest_qmp(s, "{ 'execute': 'qmp_capabilities' }")); +} +QTestState *qtest_init_with_env(const char *var, const char *extra_args, + bool defer_connect) +{ + QTestState *s = qtest_init_internal(qtest_qemu_binary(var), extra_args, + defer_connect); + if (!defer_connect) { + qtest_qmp_handshake(s); + } return s; } QTestState *qtest_init(const char *extra_args) { - return qtest_init_with_env(NULL, extra_args); + return qtest_init_with_env(NULL, extra_args, false); } QTestState *qtest_vinitf(const char *fmt, va_list ap) @@ -1523,7 +1541,7 @@ static struct MachInfo *qtest_get_machines(const char *var) silence_spawn_log = !g_test_verbose(); - qts = qtest_init_with_env(qemu_var, "-machine none"); + qts = qtest_init_with_env(qemu_var, "-machine none", false); response = qtest_qmp(qts, "{ 'execute': 'query-machines' }"); g_assert(response); list = qdict_get_qlist(response, "return"); @@ -1578,7 +1596,7 @@ static struct CpuModel *qtest_get_cpu_models(void) silence_spawn_log = !g_test_verbose(); - qts = qtest_init_with_env(NULL, "-machine none"); + qts = qtest_init_with_env(NULL, "-machine none", false); response = qtest_qmp(qts, "{ 'execute': 'query-cpu-definitions' }"); g_assert(response); list = qdict_get_qlist(response, "return"); diff --git a/tests/qtest/libqtest.h b/tests/qtest/libqtest.h index beb96b1..db76f2c 100644 --- a/tests/qtest/libqtest.h +++ b/tests/qtest/libqtest.h @@ -60,13 +60,15 @@ QTestState *qtest_init(const char *extra_args); * @var: Environment variable from where to take the QEMU binary * @extra_args: Other arguments to pass to QEMU. CAUTION: these * arguments are subject to word splitting and shell evaluation. + * @defer_connect: do not connect to qemu monitor and qtest socket. * * Like qtest_init(), but use a different environment variable for the * QEMU binary. * * Returns: #QTestState instance. */ -QTestState *qtest_init_with_env(const char *var, const char *extra_args); +QTestState *qtest_init_with_env(const char *var, const char *extra_args, + bool defer_connect); /** * qtest_init_without_qmp_handshake: @@ -78,6 +80,21 @@ QTestState *qtest_init_with_env(const char *var, const char *extra_args); QTestState *qtest_init_without_qmp_handshake(const char *extra_args); /** + * qtest_connect_deferred: + * @s: #QTestState instance to connect + * Connect to qemu monitor and qtest socket, after deferring them in + * qtest_init_with_env. Does not handshake with the monitor. + */ +void qtest_connect_deferred(QTestState *s); + +/** + * qtest_qmp_handshake: + * @s: #QTestState instance to operate on. + * Perform handshake after connecting to qemu monitor. + */ +void qtest_qmp_handshake(QTestState *s); + +/** * qtest_init_with_serial: * @extra_args: other arguments to pass to QEMU. CAUTION: these * arguments are subject to word splitting and shell evaluation. diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helpers.c index 3f8ba7f..9f39401 100644 --- a/tests/qtest/migration-helpers.c +++ b/tests/qtest/migration-helpers.c @@ -127,25 +127,28 @@ migrate_get_connect_qdict(QTestState *who) static void migrate_set_ports(QTestState *to, QList *channel_list) { - QDict *addr; + g_autoptr(QDict) addr = NULL; QListEntry *entry; const char *addr_port = NULL; - addr = migrate_get_connect_qdict(to); - QLIST_FOREACH_ENTRY(channel_list, entry) { QDict *channel = qobject_to(QDict, qlist_entry_obj(entry)); QDict *addrdict = qdict_get_qdict(channel, "addr"); - if (qdict_haskey(addrdict, "port") && - qdict_haskey(addr, "port") && - (strcmp(qdict_get_str(addrdict, "port"), "0") == 0)) { + if (!qdict_haskey(addrdict, "port") || + strcmp(qdict_get_str(addrdict, "port"), "0")) { + continue; + } + + if (!addr) { + addr = migrate_get_connect_qdict(to); + } + + if (qdict_haskey(addr, "port")) { addr_port = qdict_get_str(addr, "port"); qdict_put_str(addrdict, "port", addr_port); } } - - qobject_unref(addr); } bool migrate_watch_for_events(QTestState *who, const char *name, diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 64e1c50..b7001b0 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -844,7 +844,7 @@ static int test_migrate_start(QTestState **from, QTestState **to, args->opts_source ? args->opts_source : "", ignore_stderr); if (!args->only_target) { - *from = qtest_init_with_env(QEMU_ENV_SRC, cmd_source); + *from = qtest_init_with_env(QEMU_ENV_SRC, cmd_source, false); qtest_qmp_set_event_callback(*from, migrate_watch_for_events, &src_state); @@ -865,7 +865,7 @@ static int test_migrate_start(QTestState **from, QTestState **to, shmem_opts ? shmem_opts : "", args->opts_target ? args->opts_target : "", ignore_stderr); - *to = qtest_init_with_env(QEMU_ENV_DST, cmd_target); + *to = qtest_init_with_env(QEMU_ENV_DST, cmd_target, false); qtest_qmp_set_event_callback(*to, migrate_watch_for_events, &dst_state); From patchwork Mon Dec 2 13:20:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B12DAD78323 for ; Mon, 2 Dec 2024 13:39:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6a3-0005nT-Cd; Mon, 02 Dec 2024 08:35:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6M5-000871-BJ for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:52 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6M0-0005qK-Pz for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:46 -0500 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B26WvkD024799; Mon, 2 Dec 2024 13:20:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=djKZ8Tk2Z3XOin/4L7hl0xtlsXfXjyoXfa58BNaPARc=; b= LqR3qLvsM4C6VCvSuRP3XuMHFju5L9nW32P1DuJGgJX09NT4lGMKQpb1eFpOcRPa KGKkCGut+tTJnh8yWh3r1qtoC251ztFLbc6VdThAOXihx7MDouKD+bnbnZI/kPwX bTok3vD9MO6SuW3G31UDDcYIsuxSoHo50Hu/3+2pZoOdprNiikYXuYj6RFwkZRxh 2tbdIKTS7eBUf3UNKVFLyiB80V08ZMYZIWzeXnCSVTuwA97vvR1zbc1IwmfuUxOU PZhnPpB2eeImsZ1coy9iK9K+B6ue3BODogLFfyfS1wNqh1LcnsToPHxrw9zAzqYm sJ1RqnyknItIb4UDY2sz8w== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437sg2347d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:27 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2CVsYs032908; Mon, 2 Dec 2024 13:20:26 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtuq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:26 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCF7032806; Mon, 2 Dec 2024 13:20:25 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-18; Mon, 02 Dec 2024 13:20:25 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 17/19] tests/migration-test: defer connection Date: Mon, 2 Dec 2024 05:20:09 -0800 Message-Id: <1733145611-62315-18-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: ODQ1lFjqfusBKUxsvONEamQfp11NKfs4 X-Proofpoint-GUID: ODQ1lFjqfusBKUxsvONEamQfp11NKfs4 Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add an option to defer connection to the target monitor, needed by the cpr-transfer test. Signed-off-by: Steve Sistare Reviewed-by: Fabiano Rosas --- tests/qtest/migration-test.c | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index b7001b0..8bc665d 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -614,6 +614,9 @@ typedef struct { * size is plugged in. If omitted, "-m %s" is used. */ const char *memory_backend; + + /* Do not connect to target monitor and qtest sockets in qtest_init */ + bool defer_target_connect; } MigrateStart; /* @@ -733,6 +736,7 @@ static int test_migrate_start(QTestState **from, QTestState **to, const char *machine_alias, *machine_opts = ""; g_autofree char *machine = NULL; g_autofree char *memory_backend = NULL; + const char *events; if (args->use_shmem) { if (!g_file_test("/dev/shm", G_FILE_TEST_IS_DIR)) { @@ -850,22 +854,31 @@ static int test_migrate_start(QTestState **from, QTestState **to, &src_state); } + /* + * If the monitor connection is deferred, enable events on the command line + * so none are missed. This is for testing only, do not set migration + * options like this in general. + */ + events = args->defer_target_connect ? "-global migration.x-events=on" : ""; + cmd_target = g_strdup_printf("-accel kvm%s -accel tcg " "-machine %s,%s " "-name target,debug-threads=on " "%s " "-serial file:%s/dest_serial " "-incoming %s " - "%s %s %s %s %s", + "%s %s %s %s %s %s", kvm_opts ? kvm_opts : "", machine, machine_opts, memory_backend, tmpfs, uri, + events, arch_opts ? arch_opts : "", arch_target ? arch_target : "", shmem_opts ? shmem_opts : "", args->opts_target ? args->opts_target : "", ignore_stderr); - *to = qtest_init_with_env(QEMU_ENV_DST, cmd_target, false); + *to = qtest_init_with_env(QEMU_ENV_DST, cmd_target, + args->defer_target_connect); qtest_qmp_set_event_callback(*to, migrate_watch_for_events, &dst_state); @@ -883,7 +896,9 @@ static int test_migrate_start(QTestState **from, QTestState **to, * to mimic as closer as that. */ migrate_set_capability(*from, "events", true); - migrate_set_capability(*to, "events", true); + if (!args->defer_target_connect) { + migrate_set_capability(*to, "events", true); + } return 0; } @@ -1753,6 +1768,11 @@ static void test_precopy_common(MigrateCommon *args) migrate_qmp(from, to, args->connect_uri, args->connect_channels, "{}"); + if (args->start.defer_target_connect) { + qtest_connect_deferred(to); + qtest_qmp_handshake(to); + } + if (args->result != MIG_TEST_SUCCEED) { bool allow_active = args->result == MIG_TEST_FAIL; wait_for_migration_fail(from, allow_active); From patchwork Mon Dec 2 13:20:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 64E82D7831F for ; Mon, 2 Dec 2024 13:38:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6Xe-0007iz-06; Mon, 02 Dec 2024 08:32:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6MA-0008CM-VD for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:56 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6M3-0005qY-VC for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:49 -0500 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2DJCO7023886; Mon, 2 Dec 2024 13:20:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=TWNSOPxczOSPwzHxzQ0Ytn0JQ6tNKAw9ubDxtoSqYm8=; b= XEsicbX1eO48bnUrJrE1LZehYmQQESXCamQDAGSIrDy9Be7tskN2nAQOoxnoh+cX 9nRTT25fwCJ4BaMCeO4dtI8BIHrgJRD1atSSxYFyI2RrVbINRs7CJ5yM2/B+QWfm UgOj5I2saHJqy55aL/gbq4HJ3pwKgkScEGVSp0KguI/FFP2/A5M4cMS8waRflFJn 8UQcCIo4G9rfIt0PC1+iXkARVUzs6/VIENa7dhwAG1TnQCJ8H4ZHm2zOveUzXEvC A9ZmYE1677zlHTkgBJGpVUv+1++/pQzWv1bfQrBXm5iMvIh9Xk18JPhtui1SzNaT +JFkKq8yNBY9uAOgYs/79w== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437trbk4c3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:28 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2D44PA032669; Mon, 2 Dec 2024 13:20:27 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtv5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:27 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCF9032806; Mon, 2 Dec 2024 13:20:26 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-19; Mon, 02 Dec 2024 13:20:26 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 18/19] migration-test: cpr-transfer Date: Mon, 2 Dec 2024 05:20:10 -0800 Message-Id: <1733145611-62315-19-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-GUID: h5VqSifPlv8pnPAx06l9C18Cklmn0JpG X-Proofpoint-ORIG-GUID: h5VqSifPlv8pnPAx06l9C18Cklmn0JpG Received-SPF: pass client-ip=205.220.177.32; envelope-from=steven.sistare@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add a migration test for cpr-transfer mode. Defer the connection to the target monitor, else the test hangs because in cpr-transfer mode QEMU does not listen for monitor connections until we send the migrate command to source QEMU. To test -incoming defer, send a migrate incoming command to the target, after sending the migrate command to the source, as required by cpr-transfer mode. Signed-off-by: Steve Sistare --- tests/qtest/migration-test.c | 72 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 8bc665d..4eb641c 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1729,6 +1729,7 @@ static void test_precopy_common(MigrateCommon *args) { QTestState *from, *to; void *data_hook = NULL; + const char *connect_uri; if (test_migrate_start(&from, &to, args->listen_uri, &args->start)) { return; @@ -1766,11 +1767,16 @@ static void test_precopy_common(MigrateCommon *args) goto finish; } - migrate_qmp(from, to, args->connect_uri, args->connect_channels, "{}"); + /* If has channels, then connect_uri is only used for listen defer */ + connect_uri = args->connect_channels ? NULL : args->connect_uri; + migrate_qmp(from, to, connect_uri, args->connect_channels, "{}"); if (args->start.defer_target_connect) { qtest_connect_deferred(to); qtest_qmp_handshake(to); + if (!strcmp(args->listen_uri, "defer")) { + migrate_incoming_qmp(to, args->connect_uri, "{}"); + } } if (args->result != MIG_TEST_SUCCEED) { @@ -2415,6 +2421,66 @@ static void test_multifd_file_mapped_ram_fdset_dio(void) } #endif /* !_WIN32 */ +static void *test_mode_transfer_start(QTestState *from, QTestState *to) +{ + migrate_set_parameter_str(from, "mode", "cpr-transfer"); + return NULL; +} + +/* + * cpr-transfer mode cannot use the target monitor prior to starting the + * migration, and cannot connect synchronously to the monitor, so defer + * the target connection. + */ +static void test_mode_transfer_common(bool incoming_defer) +{ + g_autofree char *cpr_path = g_strdup_printf("%s/cpr.sock", tmpfs); + g_autofree char *mig_path = g_strdup_printf("%s/migsocket", tmpfs); + g_autofree char *uri = g_strdup_printf("unix:%s", mig_path); + + const char *opts = "-machine aux-ram-share=on -nodefaults"; + g_autofree char *opts_target = g_strdup_printf( + "-incoming \\{\\\'channel-type\\\':\\\'cpr\\\'," + "\\\'addr\\\':\\{\\\'transport\\\':\\\'socket\\\'," + "\\\'type\\\':\\\'unix\\\',\\\'path\\\':\\\'%s\\\'\\}\\} %s", + cpr_path, opts); + + g_autofree char *channels = g_strdup_printf( + "[ { 'channel-type': 'main'," + " 'addr': { 'transport': 'socket'," + " 'type': 'unix'," + " 'path': '%s' } }," + " { 'channel-type': 'cpr'," + " 'addr': { 'transport': 'socket'," + " 'type': 'unix'," + " 'path': '%s' } } ]", + mig_path, cpr_path); + + MigrateCommon args = { + .start.opts_source = opts, + .start.opts_target = opts_target, + .start.defer_target_connect = true, + .start.memory_backend = "-object memory-backend-memfd,id=pc.ram,size=%s" + " -machine memory-backend=pc.ram", + .listen_uri = incoming_defer ? "defer" : uri, + .connect_uri = incoming_defer ? uri : NULL, + .connect_channels = channels, + .start_hook = test_mode_transfer_start, + }; + + test_precopy_common(&args); +} + +static void test_mode_transfer(void) +{ + test_mode_transfer_common(NULL); +} + +static void test_mode_transfer_defer(void) +{ + test_mode_transfer_common(true); +} + static void test_precopy_tcp_plain(void) { MigrateCommon args = { @@ -3905,6 +3971,10 @@ int main(int argc, char **argv) migration_test_add("/migration/mode/reboot", test_mode_reboot); } + migration_test_add("/migration/mode/transfer", test_mode_transfer); + migration_test_add("/migration/mode/transfer/defer", + test_mode_transfer_defer); + migration_test_add("/migration/precopy/file/mapped-ram", test_precopy_file_mapped_ram); migration_test_add("/migration/precopy/file/mapped-ram/live", From patchwork Mon Dec 2 13:20:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 13890651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3D18BD78321 for ; Mon, 2 Dec 2024 13:35:21 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI6XG-0007JI-2p; Mon, 02 Dec 2024 08:32:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6M4-00086H-8T for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:50 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI6Lw-0005qq-L8 for qemu-devel@nongnu.org; Mon, 02 Dec 2024 08:20:42 -0500 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2DFJ6t015023; Mon, 2 Dec 2024 13:20:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :date:from:in-reply-to:message-id:references:subject:to; s= corp-2023-11-20; bh=yzUhmGyI6Anp6d6IumCZHkZ+5KaNq+VjA3ZSwUCd4Ng=; b= f1xhTTu5OJE/KnQH1S2JZbUbKQtg86EjJjqsei1FMzfLTbkgwFjnbSPw46cdTsGG d+JuQckUX8hP4G8EGO24riW/6n1FexawSVOKzuGsrh4DjY0vWn6o1u0KBV73FWhr eWff3zX25qkymG8hl2QtVYr9dbcPvt+sccf1zLZbSagz+qawO4d9/xzp+7AsvYUs cqJMeRfhVFhdadnoqo7Cz1oA0XVPfKJ0GNxyFN8nKDFBma63zGEs7kXAoHRJupge /pRAzZleIRpuJjCnNf8oq4px/Xzlesd+NW8M2HFiefimHrxWP47MWhkNBrJxMwZL MP8JkeSpIH3SmhVirI0+kw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437smab035-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:28 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2CLJXA032060; Mon, 2 Dec 2024 13:20:27 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 437s56jtvj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 02 Dec 2024 13:20:27 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4B2DKCFB032806; Mon, 2 Dec 2024 13:20:27 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 437s56jthv-20; Mon, 02 Dec 2024 13:20:27 +0000 From: Steve Sistare To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , David Hildenbrand , Marcel Apfelbaum , Eduardo Habkost , Philippe Mathieu-Daude , Paolo Bonzini , "Daniel P. Berrange" , Markus Armbruster , Steve Sistare Subject: [PATCH V4 19/19] migration: cpr-transfer documentation Date: Mon, 2 Dec 2024 05:20:11 -0800 Message-Id: <1733145611-62315-20-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> References: <1733145611-62315-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_09,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020116 X-Proofpoint-ORIG-GUID: rXQ__ndRcHuDRg6fFJyNdMC1lmIuwDf1 X-Proofpoint-GUID: rXQ__ndRcHuDRg6fFJyNdMC1lmIuwDf1 Received-SPF: pass client-ip=205.220.165.32; envelope-from=steven.sistare@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.444, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Steve Sistare --- docs/devel/migration/CPR.rst | 176 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 174 insertions(+), 2 deletions(-) diff --git a/docs/devel/migration/CPR.rst b/docs/devel/migration/CPR.rst index 63c3647..a8a57c0 100644 --- a/docs/devel/migration/CPR.rst +++ b/docs/devel/migration/CPR.rst @@ -5,7 +5,7 @@ CPR is the umbrella name for a set of migration modes in which the VM is migrated to a new QEMU instance on the same host. It is intended for use when the goal is to update host software components that run the VM, such as QEMU or even the host kernel. At this time, -cpr-reboot is the only available mode. +the cpr-reboot and cpr-transfer modes are available. Because QEMU is restarted on the same host, with access to the same local devices, CPR is allowed in certain cases where normal migration @@ -53,7 +53,7 @@ RAM is copied to the migration URI. Outgoing: * Set the migration mode parameter to ``cpr-reboot``. * Set the ``x-ignore-shared`` capability if desired. - * Issue the ``migrate`` command. It is recommended the the URI be a + * Issue the ``migrate`` command. It is recommended the URI be a ``file`` type, but one can use other types such as ``exec``, provided the command captures all the data from the outgoing side, and provides all the data to the incoming side. @@ -145,3 +145,175 @@ Caveats cpr-reboot mode may not be used with postcopy, background-snapshot, or COLO. + +cpr-transfer mode +----------------- + +This mode allows the user to transfer a guest to a new QEMU instance +on the same host with minimal guest pause time, by preserving guest +RAM in place, albeit with new virtual addresses in new QEMU. + +The user starts new QEMU on the same host as old QEMU, with the +same arguments as old QEMU, plus the ``-incoming option``. The user +issues the migrate command to old QEMU, which stops the VM, saves +state to the migration channels, and enters the postmigrate state. +Execution resumes in new QEMU. + +This mode requires a second migration channel type "cpr" in the +channel arguments on the outgoing side. The channel must be a type, +such as unix socket, that supports SCM_RIGHTS. However, the cpr +channel cannot be added to the list of channels for a migrate-incoming +command, because it must be read before new QEMU opens a monitor. +Instead, the user passes the channel as a second -incoming +command-line argument to new QEMU using JSON syntax. + +Usage +^^^^^ + +Memory backend objects must have the ``share=on`` attribute. + +The VM must be started with the ``-machine aux-ram-share=on`` +option. This causes implicit RAM blocks (those not described by +a memory-backend object) to be allocated by mmap'ing a memfd. +Examples include VGA and ROM. + +Outgoing: + * Set the migration mode parameter to ``cpr-transfer``. + * Issue the ``migrate`` command, containing a main channel and + a cpr channel. + +Incoming: + * Start new QEMU with two ``-incoming`` options. + * If the VM was running when the outgoing ``migrate`` command was + issued, then QEMU automatically resumes VM execution. + +Caveats +^^^^^^^ + +cpr-transfer mode may not be used with postcopy, background-snapshot, +or COLO. + +memory-backend-epc is not supported. + +The main incoming migration channel cannot be a file type. + +If the main incoming migration channel is a tcp type, then the port +cannot be 0 (meaning dynamically choose a port). + +When using ``-incoming defer``, you must issue the migrate command to +old QEMU before issuing any monitor commands to new QEMU, because new +QEMU blocks waiting to read from the cpr channel before starting its +monitor, and old QEMU does not write to the channel until the migrate +command is issued. However, new QEMU does not open and read the +main migration channel until you issue the migrate incoming command. + +Example 1: incoming channel +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In these examples, we simply restart the same version of QEMU, but +in a real scenario one would start new QEMU on the incoming side. +Note that new QEMU does not print the monitor prompt until old QEMU +has issued the migrate command. The outgoing side uses QMP because +HMP cannot specify a CPR channel. Some QMP responses are omitted for +brevity. + +:: + + Outgoing: Incoming: + + # qemu-kvm -qmp stdio + -object memory-backend-file,id=ram0,size=4G, + mem-path=/dev/shm/ram0,share=on -m 4G + -machine aux-ram-share=on + ... + # qemu-kvm -monitor stdio + -incoming tcp:0:44444 + -incoming '{"channel-type": "cpr", + "addr": { "transport": "socket", + "type": "unix", "path": "cpr.sock"}}' + ... + {"execute":"qmp_capabilities"} + + {"execute": "query-status"} + {"return": {"status": "running", + "running": true}} + + {"execute":"migrate-set-parameters", + "arguments":{"mode":"cpr-transfer"}} + + {"execute": "migrate", "arguments": { "channels": [ + {"channel-type": "main", + "addr": { "transport": "socket", "type": "inet", + "host": "0", "port": "44444" }}, + {"channel-type": "cpr", + "addr": { "transport": "socket", "type": "unix", + "path": "cpr.sock" }}]}} + + QEMU 10.0.50 monitor + (qemu) info status + VM status: running + + {"execute": "query-status"} + {"return": {"status": "postmigrate", + "running": false}} + +Example 2: incoming defer +^^^^^^^^^^^^^^^^^^^^^^^^^ + +This example uses ``-incoming defer`` to hot plug a device before +accepting the main migration channel. Again note you must issue the +migrate command to old QEMU before you can issue any monitor +commands to new QEMU. + + +:: + + Outgoing: Incoming: + + # qemu-kvm -monitor stdio + -object memory-backend-file,id=ram0,size=4G, + mem-path=/dev/shm/ram0,share=on -m 4G + -machine aux-ram-share=on + ... + # qemu-kvm -monitor stdio + -incoming defer + -incoming '{"channel-type": "cpr", + "addr": { "transport": "socket", + "type": "unix", "path": "cpr.sock"}}' + ... + {"execute":"qmp_capabilities"} + + {"execute": "device_add", + "arguments": {"driver": "pcie-root-port"}} + + {"execute":"migrate-set-parameters", + "arguments":{"mode":"cpr-transfer"}} + + {"execute": "migrate", "arguments": { "channels": [ + {"channel-type": "main", + "addr": { "transport": "socket", "type": "inet", + "host": "0", "port": "44444" }}, + {"channel-type": "cpr", + "addr": { "transport": "socket", "type": "unix", + "path": "cpr.sock" }}]}} + + QEMU 10.0.50 monitor + (qemu) info status + VM status: paused (inmigrate) + (qemu) device_add pcie-root-port + (qemu) migrate_incoming tcp:0:44444 + (qemu) info status + VM status: running + + {"execute": "query-status"} + {"return": {"status": "postmigrate", + "running": false}} + +Futures +^^^^^^^ + +cpr-transfer mode is based on a capability to transfer open file +descriptors from old to new QEMU. In the future, descriptors for +vfio, iommufd, vhost, and char devices could be transferred, +preserving those devices and their kernel state without interruption, +even if they do not explicitly support live migration.