From patchwork Wed May 10 00:18:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Rafa=C3=ABl_Kooi?= X-Patchwork-Id: 13236446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CCFDEC77B7C for ; Wed, 10 May 2023 05:14:24 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.532612.828882 (Exim 4.92) (envelope-from ) id 1pwc9N-0001Mk-WD; Wed, 10 May 2023 05:14:06 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 532612.828882; Wed, 10 May 2023 05:14:05 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pwc9N-0001Lj-Oo; Wed, 10 May 2023 05:14:05 +0000 Received: by outflank-mailman (input) for mailman id 532612; Wed, 10 May 2023 00:18:39 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pwXXT-0004li-Sy for xen-devel@lists.xenproject.org; Wed, 10 May 2023 00:18:39 +0000 Received: from EUR02-AM0-obe.outbound.protection.outlook.com (mail-am0eur02olkn20822.outbound.protection.outlook.com [2a01:111:f400:fe13::822]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 36252708-eec8-11ed-b229-6b7b168915f2; Wed, 10 May 2023 02:18:38 +0200 (CEST) Received: from DU0P192MB1700.EURP192.PROD.OUTLOOK.COM (2603:10a6:10:3bf::6) by AM9P192MB0871.EURP192.PROD.OUTLOOK.COM (2603:10a6:20b:1fb::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.33; Wed, 10 May 2023 00:18:38 +0000 Received: from DU0P192MB1700.EURP192.PROD.OUTLOOK.COM ([fe80::5056:b334:c71f:b047]) by DU0P192MB1700.EURP192.PROD.OUTLOOK.COM ([fe80::5056:b334:c71f:b047%6]) with mapi id 15.20.6363.033; Wed, 10 May 2023 00:18:38 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 36252708-eec8-11ed-b229-6b7b168915f2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JpxuxsysUsDSy+raGfwZoQG51JT6QRIdG++XXpcSoUesQw9F8jwVr0CgyxdY5R1VWVm5xIdvfHI3sWj6Z0zJJQVYrvodrggawSqPEizDJuK7Jnrj8qjgZhvnW4eb8YyOOcvawrzlq6Hzk/idD2yl1nWTqFB4j3yphKgZ+G3GHeahTanKOZ9jhTI5OZPRvtzbUuBuUFyz4tx2uI6vdxgj+JtpecFWHwvtaQ9dGM7L55c88nhG0qC6TVGbyCDBr1ZetSQnlkrB25+06IgTPZP7cUxjU59sMb0EWsMq5qpe5Akaaj/zFEQDctr5asjw4mPRl1HJG60hHHzmkr+IBo9+LQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Co0A8XpUzSdvSffHfW1JcfS0NgGlehxe6pGZTt39HIE=; b=T5BOE9bB0TEiD5VWiQpmV7JGoup47DkMRhy1Y6WnVG6R2Nz1dioJx3xrlqr1RpOh3FRtFmDx0gBOiksojwbH5dXSMEqhHvL8Kh0gJ7fNogjiTh/s9lIWBkABKx4fkgKywqHX3q0V8Boavg0lJXNNgQAVHYwvEuBIux+5J9L71X51i9v0TbF2pBXAUZIR5ClNSc9LK/zUtly9yxCA7QHxbVeyAxg0OK0qusSGVfSQtsZujKowVj5SFYkEMs701RmSmruWKJDz+6+Rmmp23tTVDq2OuwMfIzrllGOmbp4eZV7Sn648HQa/x43ojBic33Omlf6dOCsLZpEL0EKEKRpYgw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Co0A8XpUzSdvSffHfW1JcfS0NgGlehxe6pGZTt39HIE=; b=RIat4IdLGcLGFwu7+3bnNxBZdsF8F5KYs7DP+LXUPfgq9THaa7pZ3Os8sFfsjAwB5fdvJH8a97htQNrNib0c/qCJjmclRDcHntrpmJEHkBlK7ucGLq2HUTK6M35AbXNY/AOYFub+JqqhpiCH22WjkJxEB2ArhChsMEZK7HZ89DCHKfykWiRH7syXrbPvpQs8lN3Th/iDFcumvSTmhVFoZr4zjg0zHcOv1H8loSrUPA51+ZRWKsJMSo2ZS6Km0ofGotSrUD/OfWCesFINzSX1+3omg2HMZRW2oOhwnZ2Be6bFTaf8SqGtF6gcBCz3l8hMe5Z6ar0poKpGVNYgUppsmA== From: =?utf-8?q?Rafa=C3=ABl_Kooi?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Rafa=C3=ABl_Kooi?= , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [XEN PATCH 2/2] x86/Dom0: Use streaming decompression for ZSTD compressed kernels Date: Wed, 10 May 2023 02:18:22 +0200 Message-ID: X-Mailer: git-send-email 2.40.0 In-Reply-To: References: X-TMN: [HcumDLXUQx2Ka6y8dbImpEzA8tER5uiK1S3VfeyYV5wT2Hi4Nw5fZ9XqlsqI5KUK] X-ClientProxiedBy: LO2P265CA0052.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:60::16) To DU0P192MB1700.EURP192.PROD.OUTLOOK.COM (2603:10a6:10:3bf::6) X-Microsoft-Original-Message-ID: MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU0P192MB1700:EE_|AM9P192MB0871:EE_ X-MS-Office365-Filtering-Correlation-Id: 91a01a52-a7fb-4686-ab79-08db50ec19c7 X-MS-Exchange-SLBlob-MailProps: ZILSnhm0P3k2XayEnZ/FXgv66CsfMCS1d7vfnlsnkxPz9PUXJadlSovgYnwo3lJ6PxRKlGP/SxcNROzPIHNCRtP3M+1kuBI/U+oo7Dpgd+Cwj1g1PGktYMMwagUf+Fjl1181pDhQnaVi/teLqHASkpl1GFzJW0BraS7JuIUeP2gOtW5TRGOYoiZbWtidRWc/NYabT2xh+8CWAfawcyO751KLlO+DDiFD8fo3sPVzPiFeHju87Lu126nUFDfVAnRWdxvr/TZaiZqEVKUEQEohD0UYOHjqDDJ7pePb0jY3qFbHK6TxMrCQYycSpVsASUuuoPPsQMG52S/tDRs2Jk5kKcNPCVKtBZ1iez8uVDvf30oRt+KPzvxcD3hDlWqPXIDd1VVL3kAE1Nh/zllFkiujAt8NOPLwJJPO03XhzLoJlGzBaDUBIYFSN4hFpvshdp4zpkfny5Ksv52QXI6KJKv9X38f5uskkhQ60EVXVzSTglTsFdjcarpU573Thb8ezfJ97eWc8ER6TAFxzAc1FBPI81N2QfEByfP4FfKnfk3ee6VjwkXptZDjGd+hfl9Qi2qUPG1Oym80RH//ULTm1WL4/LTpz9cyIBB2KPi6Hnf+ca9C0bxJwy182OxXb1KnM0Ri15/D81jvL2biMVcBWPkpwRoVSDni8iuB1SXRhJ37Spoe4Rcb6Yo9qPkZFt80WagRwOqCuin9ejZHacIp3uQ0ZkoGM6YPzlVvJK03GLNndk3+54cGbPV2+hIH6ZUHa1O2Q0apHGOImIIjRJYBZ4RjjNpXuy8+YQLV X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: W5pC3f03pX93RXyqnro6AjJ8bptsX7vbdbLudov9rnBmZnW4pJtG5Z3hUX7qylEk6zEYu5mOu6Hq/ZLU7WrkoTFMb2H50RAfAIDWRLf8U+pW1w/Ez7DRmObRgD+2q7VPNyvxvjBanHX+FU42Aex0LbpdfH7i5nPl2b21UTu2J84F8uAs+8JBzeiC6wDXSazpagrBVnvSPsnzr0V5rNPQI8YCn2+rWqgo/kf/90jzOACVKY0ryFi3x24n66K+s2FFANPvbyk3N7JQzTO5UPwxKbNfegx0ApnURgrnf8bwwtfy/2MQnowLBYaqmFM2+TDy/NbxV7k6DgKkEioLakTeeqCxr+1tPQuaR9O16bLFMyOzb1EaNwIzuw7tny4dl5I1/Mgz7R1BJ3wXKQ+OUZ3taqwxQKTQqA6Q2xzyd2PXWQ8pMODyqkJeNhtjneDOXA4CilWl3z6ZExjcqDIvdX/N0WYGTVMvbxzk79mMDlZHZp9D++4irUe7XjqPWBNeJ7xyw8HhNXWLmx+i2RaqbL2/BQuz5zJLCgw6pCphhbZjqcIdgoVtB29AoJwkMEfPiSwRJBgDknUw5jCKHWh3FwHP1R+i0zuNETbepDlMdEjapyec33qkfjqPxdhR1PJIzVWv X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?e2azFZxZRwrJWr7B/+Jqi64AYsMA?= =?utf-8?q?iFQz/qFxCG+52b4nV/ojdKJ5hjGgMyraxq1N8J3yLoriIQvTLFSvQot6zpmP2w9TL?= =?utf-8?q?d6gNnDIBrEK837+X3EHvfOJpStUhCGlv+2Fso2bpQb6ThMcOrmQgrxDCQ34wP6otU?= =?utf-8?q?f8wxAGE1ii0Iy9o9wrCHfHeBN5CMAIJj2RpvtuuHRLZlbi8/VJTgpM+Xv751UMd6/?= =?utf-8?q?gNuOHZ9vp6J9/PySvN4RKQhcD3ax+U6k+p2lSDzbuhpUXXR7hQVXll+nrstq7qnn5?= =?utf-8?q?BINCHTemuFPOtOgMoVCYaad10VICYVOw4bO7bdvj9bAaTeaqIPObv5KUTornwcyDN?= =?utf-8?q?N/cKHf1IDrLDfUhHA6ybIU4s/ul5AeYaMwCtJEfQAFsfKWXOycLS/RsLW9CFK0PAP?= =?utf-8?q?naoKGZh2mMOkdzx481eLiAIBJ3oXL7BDkhisqq7to+P+9trkSu/1nNC9AhJGqtoMx?= =?utf-8?q?Im3kJGw8KHkqiEieno9LTG/3Z6FUTgf0C9lQshacGJgGEpvbPN9gKdY6tHXtrFmW7?= =?utf-8?q?WdlJOgH98PzUCWu4IEKXZ8rOTmkgbHBSIyvqgIn+cE2mgzZTWgp2vHVHgxbG3PKWU?= =?utf-8?q?qIoZv9xuk//aj1tCwRjQP2IDoUc0NpSnqG2BLyMYQxAUSgj4Y8xIltHipTGHwkcqD?= =?utf-8?q?+ongtL9uku8RQgAF3KpjfPnpcI/+8HCuVwdrKjVTCFZ6VgStmj7RGeO5CBy8sQ/zO?= =?utf-8?q?JsNIoXFv+YJcG1c9vAiwGxGgEQ4e+cvvzN3LrGODPfBJQH6/U8nId/lNKfDnfKvoR?= =?utf-8?q?hgMxSTlB15dPDOttRE9Drdq5nSsUyY41zDgZ0cDqJYBIDOEfKJ6EziFiMH+FMbkq0?= =?utf-8?q?ZglscVovDXqZ7ayvxrACWP4nZgi2kF4mrR90beVp5XgNG6V3Cx54XA1vLlS6s1DM4?= =?utf-8?q?gdI7E7PJyCPn02SQh2EXwEQUwcRtU/mcvgxpD3CKg3qn8CRWkYm8SwuHVgp8O9Egr?= =?utf-8?q?43CvnIa+I4dpwacywD2J9TmLXFxdDjthlspKAKY3iEGrmYLHrSBkRpDQ0BXjDN+Kw?= =?utf-8?q?6rmIo6bHyIjRdLww5CjErgnCve4InfaT2Ng2ia4o3YstDiwzwMbwNKBt04Fw8IRQE?= =?utf-8?q?oFtPoAFhcC6Unn+pQ140XPN6EtmSe5mKAT/2UKKxOJN0OiL6zyMNwgAgiht8k3mJj?= =?utf-8?q?jmezjFzUR2TY8Ioec6RmGNxoaheqj1iDd4Wl6Q+7kAu5X9TXpFwL8kUNi1MOmOcCx?= =?utf-8?q?Pyu64IICbOiGS1Su2qRXN7n3AXIgcsdmdvkeRTsMwq8Xrx4eLFd8qMRBfmMI=3D?= X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-fb43a.templateTenant X-MS-Exchange-CrossTenant-Network-Message-Id: 91a01a52-a7fb-4686-ab79-08db50ec19c7 X-MS-Exchange-CrossTenant-AuthSource: DU0P192MB1700.EURP192.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2023 00:18:38.1978 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9P192MB0871 On Arch Linux kernel decompression will fail when Xen has been unified with the kernel and initramfs as a single binary. This change works for both streaming and non-streaming ZSTD content. Signed-off-by: Rafaël Kooi Reviewed-by: tag - you can only add such tags once they were offered to --- xen/common/decompress.c | 37 +++++++++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 6 deletions(-) diff --git a/xen/common/decompress.c b/xen/common/decompress.c index 989336983f..cde754ffb1 100644 --- a/xen/common/decompress.c +++ b/xen/common/decompress.c @@ -3,11 +3,26 @@ #include #include +typedef struct _ZSTD_state +{ + void *write_buf; + unsigned int write_pos; +} ZSTD_state; + static void __init cf_check error(const char *msg) { printk("%s\n", msg); } +static int __init cf_check ZSTD_flush(void *buf, unsigned int pos, + void *userptr) +{ + ZSTD_state *state = (ZSTD_state*)userptr; + memcpy(state->write_buf + state->write_pos, buf, pos); + state->write_pos += pos; + return pos; +} + int __init decompress(void *inbuf, unsigned int len, void *outbuf) { #if 0 /* Not needed here yet. */ @@ -17,22 +32,32 @@ int __init decompress(void *inbuf, unsigned int len, void *outbuf) #endif if ( len >= 3 && !memcmp(inbuf, "\x42\x5a\x68", 3) ) - return bunzip2(inbuf, len, NULL, NULL, outbuf, NULL, error); + return bunzip2(inbuf, len, NULL, NULL, outbuf, NULL, error, NULL); if ( len >= 6 && !memcmp(inbuf, "\3757zXZ", 6) ) - return unxz(inbuf, len, NULL, NULL, outbuf, NULL, error); + return unxz(inbuf, len, NULL, NULL, outbuf, NULL, error, NULL); if ( len >= 2 && !memcmp(inbuf, "\135\000", 2) ) - return unlzma(inbuf, len, NULL, NULL, outbuf, NULL, error); + return unlzma(inbuf, len, NULL, NULL, outbuf, NULL, error, NULL); if ( len >= 5 && !memcmp(inbuf, "\x89LZO", 5) ) - return unlzo(inbuf, len, NULL, NULL, outbuf, NULL, error); + return unlzo(inbuf, len, NULL, NULL, outbuf, NULL, error, NULL); if ( len >= 2 && !memcmp(inbuf, "\x02\x21", 2) ) - return unlz4(inbuf, len, NULL, NULL, outbuf, NULL, error); + return unlz4(inbuf, len, NULL, NULL, outbuf, NULL, error, NULL); if ( len >= 4 && !memcmp(inbuf, "\x28\xb5\x2f\xfd", 4) ) - return unzstd(inbuf, len, NULL, NULL, outbuf, NULL, error); + { + // NOTE (Rafaël): On Arch Linux the kernel is compressed in a way + // that requires streaming ZSTD decompression. Otherwise decompression + // will fail when using a unified EFI binary. Somehow decompression + // works when not using a unified EFI binary, I suspect this is the + // kernel self decompressing. Or there is a code path that I am not + // aware of that takes care of the use case properly. + + ZSTD_state state = (ZSTD_state){ outbuf, 0 }; + return unzstd(inbuf, len, NULL, ZSTD_flush, NULL, NULL, error, &state); + } return 1; }