From patchwork Fri May 19 09:46:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Het Gala X-Patchwork-Id: 13248124 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D753EC77B75 for ; Fri, 19 May 2023 09:48:25 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzwh1-0000A1-51; Fri, 19 May 2023 05:46:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzwgx-00009I-KH for qemu-devel@nongnu.org; Fri, 19 May 2023 05:46:32 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzwgu-0008RG-Sj for qemu-devel@nongnu.org; Fri, 19 May 2023 05:46:31 -0400 Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34J5ECPG005134; Fri, 19 May 2023 02:46:25 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h=from : to : cc : subject : date : message-id : content-type : content-transfer-encoding : mime-version; s=proofpoint20171006; bh=AML5V1W2qYtBY5E//HZroVr6nI1KHXS1OqP9uCSahSY=; b=M7elDAdb1WsaAvZdmFKS+ix45/aw03MmVzCUrGweGoLcP70S+J/w5AQ7Rr//R5AkmX7H jJdVlvgKu1QKVa0Gu1YBxsDR/LMy/oC8LZKr/xmyAypRgWrSmCCOZlD/Wu5fJEgaC02q VZ3mIeSt0sFdFmvGIrZRtVa/aZtrP9rTfs0AkY3y9WG4H48Ikxdv1w8I4XpvfWw/TfFs Rt56Vcp6++5/CkwbH1Vp2ZOFcHXI8GxD6APw/KaezmhGQEVOz48HV3BAkWJxiohCfZRG cadUqYKBkFHSTFXRWWWwRnkP+9oaOIy+Bur33hO3jZyR8vT3lUJVVwGIFrKzCkOeRlev kg== Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam02lp2049.outbound.protection.outlook.com [104.47.56.49]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 3qmw0vda29-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 May 2023 02:46:25 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SOscLAeHpku3mQrTizaPBA1ChdGxx7U7+KVCJ/U79vc2WCMG8xQX1M4w96DadznBxC5x8p7NMj4DCD3TODFhH6t+ITJmmCexOQC23QdnuAs1AOKDitbKMMZNp9g0IlkDq6bQCmlzAzClEgjvD+ir/itZb8oQpzRuvsiZKAMtZQDh8mj+r0XQrunK1WumMFCfm8mSI+xTw6LpJ7HB7QQI0f/Yp60qpL2uMiIYeLhj0Sx+asfDMoZRwz6qDmmpL+ThgTQ52+zXKgR7jPU5cYPZTN+WA6DD4lmf/qHF3Yx6r9LGBJeC/hp7Ujs5xhjYyX5XLH3G0m5dg7kSp0FMaOZm/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AML5V1W2qYtBY5E//HZroVr6nI1KHXS1OqP9uCSahSY=; b=jUywp+RAi+U63jcp0K5alfo5o/nMGQqahWoHbfnB5h4WL109QBhqbBpFcrpvxTy6bBjI7CJzJupb5KC/6h+0NLPU6QTxARo5FhtJYhNksqa9lfVQMgBvsUBMwFW7TSJA7ACH9gcqJPAfFwFCdM7uWxQRgugmzSNn1G5aDGzajfN3sSauF94bWdeJXwIvFvilbgtGoWGgLGF10O8ccA6Xo8XDGd5LxwYZ/e+J2m0nqT+N/Aet0QmvjKPToExg0/8RaTihTB6aXYM7NFGmq98+KxKtw+5KfKWHvWZNavEZUTxRGSr2kFXQGJavIif4vrvo94GcjFWGFCVIQyTc0aMu/w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AML5V1W2qYtBY5E//HZroVr6nI1KHXS1OqP9uCSahSY=; b=VTBshwIFxD6mCWjIpC3Jwki65k1HdbeymYvLAKlj6zzL+/LB67oTU3xR1oEbQO6FTNWwb4kx/fGY2hCYQExqLLBop2Eh79bXeYb1kxmPTAvlEGe21kmXpvp2vxR6IIBzuH1om/qeX/dLmcgS4WwFeRx9fv/P3OkSd04YXFfOoRaiUNui3+h9KCeFOX7CJCf8d1yVS/MFkfCJF0EcOTL7bU9spgH99S8xgnov61n7ejNZjAupTy/Y54MB3lnXri43sjBdNBPkGQLSmmzqTufrEeKGMo8H6ZmGTaSag+NuxUqLfPWWkVLbnlfUr1+gXn73mAyMY+H5PH1CTOkvcV0xbQ== Received: from BYAPR02MB4343.namprd02.prod.outlook.com (2603:10b6:a03:57::18) by DM8PR02MB7912.namprd02.prod.outlook.com (2603:10b6:8:11::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6411.21; Fri, 19 May 2023 09:46:23 +0000 Received: from BYAPR02MB4343.namprd02.prod.outlook.com ([fe80::4500:569:a3d6:71c9]) by BYAPR02MB4343.namprd02.prod.outlook.com ([fe80::4500:569:a3d6:71c9%7]) with mapi id 15.20.6411.019; Fri, 19 May 2023 09:46:23 +0000 From: Het Gala To: qemu-devel@nongnu.org Cc: prerna.saxena@nutanix.com, quintela@redhat.com, dgilbert@redhat.com, pbonzini@redhat.com, berrange@redhat.com, armbru@redhat.com, eblake@redhat.com, manish.mishra@nutanix.com, aravind.retnakaran@nutanix.com, Het Gala Subject: [PATCH v5 0/9] migration: Modify 'migrate' and 'migrate-incoming' QAPI commands for migration Date: Fri, 19 May 2023 09:46:08 +0000 Message-Id: <20230519094617.7078-1-het.gala@nutanix.com> X-Mailer: git-send-email 2.22.3 X-ClientProxiedBy: BY5PR13CA0031.namprd13.prod.outlook.com (2603:10b6:a03:180::44) To BYAPR02MB4343.namprd02.prod.outlook.com (2603:10b6:a03:57::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR02MB4343:EE_|DM8PR02MB7912:EE_ X-MS-Office365-Filtering-Correlation-Id: 2e192e4e-fc07-4946-75f4-08db584de7a9 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: d1oY3XwB/FfYHjRRP/Sjb1H7Nn2HmLHsCqw9/jeGHlRo8B9tTqwix+occ9vd221j56YRNyNc8WeehY6fLhaIONqgtaN32t8hQjusvVaZjTDvrraWdBdB/EReKCDjLMmXf76mturKphf2DTuyfkF4u5p4Oc+gf7elNiNn87fG/0M6yU/56IQePS9j20pWtDmZ9s8RHGapxSe0dxq4ZjYwXDT2aQ5EHcwofE8DTXQOwFcw2XbsxmYmwkoev9WAC/wMW0TIPdDNDWRAFoh5slDrJoCNizLJJ0PWeUHC1xpZhTRUWjFw1i9H52W7x5TYavZ0DLJi042RNILOAKoXBSRXHCXy6iiOuLDN6sbw8IKVwHiYd1gDRYx49W5OFqRSb9qS68mPB+BZ3CnNyzNqBdvQP5dAwSjwwBjuQcNMweisXEqSAxzKh8DwypHTL+f9KJv2Q+Q3RVxgSGnkupxwlOrkP0YJbBjeqje4mvg8ehffMFTWS318iWxVRfBt4tSJSTfd9lbklm8UHb0/oZFuPGeD9oc3W8poi95Ws6UKMkkhNWLES6bZk7SCwEaixDSI+cAmheWz3jiMV1XaqZA9t6rkUgUWbvM6WI4MvQhvLm0KoQeDK5h5fdUt4M+WDI5dFQ+kBWYIsEUBm8qo04g6tHol9A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR02MB4343.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(39860400002)(136003)(376002)(396003)(366004)(346002)(451199021)(36756003)(44832011)(8676002)(8936002)(5660300002)(316002)(41300700001)(66476007)(38350700002)(4326008)(38100700002)(6916009)(86362001)(66556008)(66946007)(966005)(83380400001)(107886003)(6512007)(6506007)(1076003)(26005)(2906002)(186003)(52116002)(6486002)(478600001)(6666004)(2616005); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?5hbO7Y1x7FZ1AbxH1q4gi+/BveuA?= =?utf-8?q?x8pbYXSSJFoz49tsA/lnBVl8LJ6Fq3RU2km68aY1V4styenZhPNga77Umq4j+k6XS?= =?utf-8?q?ISgfQK0FVXMTmgkYvXSCxfZOS36gf7A+Sqeg5wZ3a8w+FGlO0O/oK49hGpAz3mZOb?= =?utf-8?q?XkEshCysVD59V1vG1ux/Kli3KnArzaMxrgTBqwmuu+AYuWhwF67fAa2ukKuHRMIIj?= =?utf-8?q?076xqL3JGULGTaIyJhx3XHKZl7uoWPHKa5tQRSxTKuPY5RM2cgiSbzofXzHwxl9Vs?= =?utf-8?q?P+ny+fvZ1Ty6J36EeZBxbgNAMusOFprRODSTWxAfg7MiAMuAGCFZvQO8QB2yGuq2U?= =?utf-8?q?75+Xl5bzHAd/Ucu0PNbQ2H9F5cbqrD2e9JVnY8xoxRfxTfioMbpimFjGA5yzFGEkR?= =?utf-8?q?Ut2CRmvzLS+KZVxxXEYXeZ52KQDEfZDRteqWw0JIJQ4F8Jcl8uLJvBNAQ6wrzLUnd?= =?utf-8?q?8bOdXXixCPgOMyM+zoj3/NiiwIG/ASZOSghiDpHjUetnm34/PSQs+TrxJahLaFMEH?= =?utf-8?q?0KVT20SZHpS7ujzzwLGqDCJqYRRkjI1V3O1QP7HlFRbN83MwUmM6nVJRkPEZkUyrZ?= =?utf-8?q?U3EEZ31Sd2ho5W4oBj4GzA+WCQYykC/Nk03ShhTWrRowsghkbop4U4Piz2TuIS9B7?= =?utf-8?q?J2pDeQkn6gOVTn9XY41MNOY/9/s7vYCs5AoQs0JTBPuVTWZGJOLknTG1rGZI4gati?= =?utf-8?q?xn6WZzhBIE77ZzGuCtlMHLOmhI8orA29v5J9cE1IrBiuMdN78mQ52cIowITNiyfgT?= =?utf-8?q?LRjSvDD7knmxBNWST65aCWsq7bCuq5LYhlD03ULnxEechr/QO0b2Ys5ULVjvFHkN9?= =?utf-8?q?3sd/meQ3ShqMPH15HogxTeTXsYFrIXZNeeSizZTZiiY/LfdwvEr2buAw8nIET1cZk?= =?utf-8?q?P4UviYM9lJ3AfoRfPnhKbsKQlpNqwwmdCHMAUtnAGtNL44BPWvppcA/e8pLPWEPx/?= =?utf-8?q?S1TBqS+Krc3sb2prAfKo2NzQkQfd5FC4bGtCAWoZPCE3DZBdbSTkx8VOBiOV+fUNj?= =?utf-8?q?NptB9+ZWv3Rp4HmqN5oaS2uIUnFkJ8sC2GjPO8iDFK0oTL0/+7WZBgl/FctMpFLpY?= =?utf-8?q?NDNnE9bwJMmQBZwKdg42Fk6bzo8X+ZwDIR3+oZQgWiedAxrBxgvblTpKUih6z+mSY?= =?utf-8?q?WniNBwFqj8H7yjes5unGkY/FVeh9subLeBZqoA8BS2A24eNdplfWeX91uUb0SsLQs?= =?utf-8?q?Vd6wCF6oL/YC+hm7TBNmU8LH3VPSio1hdZVf4NfHIEpfxHVveVAaoLRuVp7g5CUNM?= =?utf-8?q?OBmJwgCK/lyv8cBPmmy+IV734z+0d+LasvZJwigbymwdjnnagyqiA12sb833yslyi?= =?utf-8?q?U9aNTxtkt4NNOS7OImlyOuA/vKkBJOQL9MV/0TC0e4X58TBKhxdci1PPWvQKz2OpP?= =?utf-8?q?kPcSLv+lJGpbn8gmSxEwDGC7oKwRxPeNGNeivwYcA6fNOCwp3iYmkl4BP/PFT5126?= =?utf-8?q?TilKxhbvagkYM+IW5mvaQUx75Y33hoEqRTn1Ob8XY098tAvUeb1JmQS+9PmANwgNL?= =?utf-8?q?3p8KTx6sjDxv?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2e192e4e-fc07-4946-75f4-08db584de7a9 X-MS-Exchange-CrossTenant-AuthSource: BYAPR02MB4343.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 May 2023 09:46:22.7688 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nOerhdSroB6hBLVPJVMih0e7tb9cFFQqJW8MGruJMa1WA5DM5LQ6eHKMjeZFBvDq9f45XP6sicYIHoIK4ZQxOw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR02MB7912 X-Proofpoint-ORIG-GUID: 1qoBaV_wBTuE3ps8j-e_jQJ4g7goBluG X-Proofpoint-GUID: 1qoBaV_wBTuE3ps8j-e_jQJ4g7goBluG X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-19_05,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass client-ip=148.163.155.12; envelope-from=het.gala@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This is v5 patchset of modified 'migrate' and 'migrate-incoming' QAPI design for upstream review. Would like to thank all the maintainers that actively participated in the v4 patchset discussion and gave insightful suggestions to improve the patches. Link to previous upstream community patchset links: v1: https://lists.gnu.org/archive/html/qemu-devel/2022-12/msg04339.html v2: https://lists.gnu.org/archive/html/qemu-devel/2023-02/msg02106.html v3: https://lists.gnu.org/archive/html/qemu-devel/2023-02/msg02473.html v4: https://lists.gnu.org/archive/html/qemu-devel/2023-05/msg03064.html v4 -> v5 changelog: ------------------- - Improved majorly on cleanly freeing objects across the patches by using g_auto(), g_autoptr(), gnew0() macros wherever necessary and preventing unecessary use goto statements. - Simplified error statement propogation in migration code flow. - qapi: changed version form 8.0 -> 8.1 - For HMP commands, decided to keep using URI syntax forever, so depricated implementation of migrate_channel_from_qdict() for converting URI -> MigrateChannel struct, as "char *uri" is already getting converted into modified struct using another API in the workflow. - Additional tcp test case for multifd with modified QAPI syntax in qtests. Abstract: --------- Current QAPI 'migrate' command design (for initiating a migration stream) contains information regarding different migrate transport mechanism (tcp / unix / exec), dest-host IP address, and binding port number in form of a string. Thus the design does seem to have some design issues. Some of the issues, stated below are: 1. Use of string URIs is a data encoding scheme within a data encoding scheme. QEMU code should directly be able to work with the results from QAPI, without resorting to do a second level of parsing (eg. socket_parse()). 2. For features / parameters related to migration, the migration tunables needs to be defined and updated upfront. For example, 'migrate-set-capability' and 'migrate-set-parameter' is required to enable multifd capability and multifd-number of channels respectively. Instead, 'Multifd-channels' can directly be represented as a single additional parameter to 'migrate' QAPI. 'migrate-set-capability' and 'migrate-set-parameter' commands could be used for runtime tunables that need setting after migration has already started. The current patchset focuses on solving the first problem of multi-level encoding of URIs. The patch defines 'migrate' command as a QAPI discriminated union for the various transport backends (like socket, exec and rdma), and on basis of transport backends, different migration parameters are defined. (uri) string --> (channel) Channel-type Transport-type Migration parameters based on transport type ------------------------------------------------------------------------------ Het Gala (9): migration: introduced 'MigrateAddress' in QAPI for migration wire protocol. migration: convert uri parameter into 'MigrateAddress' struct migration: convert socket backend to accept MigrateAddress struct migration: convert rdma backend to accept MigrateAddress struct migration: convert exec backend to accept MigrateAddress struct. migration: modified migration QAPIs to accept 'channels' argument for migration migration: modify migration_channels_and_uri_compatible() to incorporate newer migration QAPI syntax migration: Introduced MigrateChannelList struct to migration code flow. migration: adding test case for modified QAPI syntax migration/exec.c | 62 ++++++++--- migration/exec.h | 8 +- migration/migration-hmp-cmds.c | 16 ++- migration/migration.c | 183 ++++++++++++++++++++++++++------- migration/rdma.c | 34 +++--- migration/rdma.h | 6 +- migration/socket.c | 39 ++----- migration/socket.h | 7 +- qapi/migration.json | 145 +++++++++++++++++++++++++- softmmu/vl.c | 2 +- tests/qtest/migration-test.c | 47 +++++++++ 11 files changed, 426 insertions(+), 123 deletions(-)