From patchwork Thu Mar 6 22:03:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Andryuk X-Patchwork-Id: 14005400 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 51156C282EC for ; Thu, 6 Mar 2025 22:04:19 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.904077.1312064 (Exim 4.92) (envelope-from ) id 1tqJK7-0003LE-O6; Thu, 06 Mar 2025 22:04:11 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 904077.1312064; Thu, 06 Mar 2025 22:04:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tqJK7-0003KF-KC; Thu, 06 Mar 2025 22:04:11 +0000 Received: by outflank-mailman (input) for mailman id 904077; Thu, 06 Mar 2025 22:04:10 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tqJK5-0000St-SH for xen-devel@lists.xenproject.org; Thu, 06 Mar 2025 22:04:09 +0000 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on20613.outbound.protection.outlook.com [2a01:111:f403:2413::613]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ec6ee5e5-fad6-11ef-9898-31a8f345e629; Thu, 06 Mar 2025 23:04:08 +0100 (CET) Received: from BN9PR03CA0068.namprd03.prod.outlook.com (2603:10b6:408:fc::13) by MW4PR12MB7467.namprd12.prod.outlook.com (2603:10b6:303:212::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8511.20; Thu, 6 Mar 2025 22:04:01 +0000 Received: from BN3PEPF0000B372.namprd21.prod.outlook.com (2603:10b6:408:fc:cafe::79) by BN9PR03CA0068.outlook.office365.com (2603:10b6:408:fc::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8511.19 via Frontend Transport; Thu, 6 Mar 2025 22:04:01 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by BN3PEPF0000B372.mail.protection.outlook.com (10.167.243.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8549.1 via Frontend Transport; Thu, 6 Mar 2025 22:04:01 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 6 Mar 2025 16:03:59 -0600 Received: from fedora.mshome.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Thu, 6 Mar 2025 16:03:59 -0600 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ec6ee5e5-fad6-11ef-9898-31a8f345e629 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=acN1sRDCmHjMGSoyfHaDXvzhOaCbbHidpjbHBl19IJo62S5+/R3sU9mSkpeaZN/vVANoggMCMK0lGIkuKjylxJ0NbUIIyDsQYDgOlp8cUcpEsvFO7dMjagppjLNLTQIfoxrRsjzytLQ3gZfkhOji4e39p1c2dS1J/b2uFANsCjl8E2A9Ys8KJ3TIiLcCMVxvVdzUtX71s4qcknrbtxCgiYhcrMhiPjxzifmzPIW6YUcPNScjjV6wEO9WUN54wl4TfYl7rYITL685SywEcMh2iQqeKYTq0+Ozlwg1YN+lX7LkqxnOa4zn7Ow1BMKLSs5QPbHnwWS2vfXYpfRGChNISA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rIC7xasjbalk/ExMtgL5X2h1ecLmO7GR38I48vi/PIw=; b=lbs9N7JC3Vg8gNlIecqSyNj6f+38gNyqckKK8CC3Qv7i3dB722yEvvK7AsWZMGgkZHPYS0BCq7QFMwMzNOnaX7JNYgx1YUfJafoGQXW4bxIYJuxNJcs3GbTVCoLRNnhz3eetHIdz8Xj7YjJAHXs6f51F/QG5ENmT4PlGq98QoNhcC1bxIlBbwM2tJDC8hYqTxOdpReUUaRZrNYBv0Euh+EYX5pNlY9AsS1wp5ICwSnyBvvcqmCBI2eeg2TFtKXfS9NvWZ6c5yntD0o0G5dtxoByuRCr7AtuAH0FqDJEF0RXMUMVd2qzxtKwC5mQk1LbIIJWyeo2cu+M/NdqF3mifVA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rIC7xasjbalk/ExMtgL5X2h1ecLmO7GR38I48vi/PIw=; b=DsozZWtPWYpJyRqYZZ+jC5HU2Pe2xBXhYAdBMjxpEjY2DkWRUGlSvL/ZAqP5bjEXipvmFyx5e8EDVY5zC0vVZi8JhK/n6KnWp5ffmai0MAZTi2gKpmHmFiEPtsiXwO7GXstj1gmcIR4BgCiOoWR7U2MzlXufCwmUluYYjghFI0w= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Jason Andryuk To: CC: Jason Andryuk , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , "Volodymyr Babchuk" Subject: [PATCH 10/23] xen/arm: dom0less delay xenstore initialization Date: Thu, 6 Mar 2025 17:03:30 -0500 Message-ID: <20250306220343.203047-11-jason.andryuk@amd.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250306220343.203047-1-jason.andryuk@amd.com> References: <20250306220343.203047-1-jason.andryuk@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB03.amd.com: jason.andryuk@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN3PEPF0000B372:EE_|MW4PR12MB7467:EE_ X-MS-Office365-Filtering-Correlation-Id: 9cc91e06-025e-412c-5924-08dd5cfacd4a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|82310400026|376014; X-Microsoft-Antispam-Message-Info: zxWcu6lOjk22Bu8pnjwAHqSuyNkuQHIx3VzNR03jn8benuveqsm+zl+fhrGPLolP8JTnzpcwExoJZj/5SZAQRAmkyJZEmVNlPSAb+4iWK6BL8q7kzC2hJ5obZdTB36FT+MBtu1PFGVQN6YCFPMy8j/UCszJ14hYjrndWiNlMbvTBxyI7SVUaB3TzXsqSResLHJYRiK5x2PuA8Gwm9brTHPtQLZopEOzAYjwGCqO2jsjBP45OTCcFoM61oxh8Pr6id1pyoz8YgyRiKytZMYUvkVLmo44F0zWzJul8Qnu3b8k7Xb03ELvdJpuAczF0l9z3Y05yXw8O+N/QCLAbriG5OAFd9SjXcTRRovXe7fQetfviKEpdccmngifcmAsFsB9GZ4jB31eFd8GeWzWw8IR5V5jl5CKGOqITq03hVEhgd3ApPlzZrhAlH9xq/5rcshOlXAzr4S+LaWTGpXdA+AvB6GOCBfcEpGqDcpAQPozFLjFS9enayz9YnPST4m0pZRS45xO4Ff95n8aB7gOKsnQWPGG+G5nFeiY7jlQheS1DLJYWXk6/woAU3oLgB8pEPE5BfQyMBURahpbVwU012bUhrckmRzC1QPTJaYPD/BvhAhJpImQKJd5wTE8o5sReAGnaWXTHiT/BWf5vZmMvAqwK3GZ5RDAFcMiLBnLRzd6gs5bqlfgXZ1xagJdT/rZB2ngjZ+psSxWRhBBhUfss7QLtrZcpmwAWz8Zm1l+ojo8NThXCsZePw1QGyh59f3nKvw7Fs75MypICYdLfL1Jg0iQXNf0b72rq+bWPcY/OF6L7+qIiazhuhoSV2r8RsMeDIQcGdNdSR+1mRVCRv+2cnCIcVe1F8FpuMJif+ngvgK3pjLJcG4c4iudydmnpHsaFgGQ2UsEkmU6JJaYFiCf+uUpdAWWz05mnEBaVV/XoV4kLB2lJSXBy0teTlWaQrx52ziWoUX8gQc+HuqVwPpARiBCZrqPzpb6VTc1Rw587gApnPLTmC8WJDgQl0Uc821TTsxd5colMw8axD4aH4oRhwB/lHG8kAYSiixEeMSBb4hR7bDu5UY3/zQF27AsnfhP8MWpDigS4DDwcTXauvri0yc5xnVm9oPAEMZoRIwaoibs8D8/npS3hQoJIwSW+HUrjWNmF7hvMxkHwimYol4ThDFCSYbtSSHqLpdcIp8S6yA+mHXy9AE0Q4uTiXG3vAaI2GnOunsk1BzcGCff+S0K0B926O+QnfldoPD811v18t0XQZXiVRuMCuYF+pyilqctV1mohxXK3YetZoJanw97dXj7wcbqPripPVR4oUpRigHoawp9eH/1giVtBVl6y4GpR3ANb+BklaQjCQWJ6BX52lq150KjHcXkTfNWBvJFHA6tRvrcuPFWWgIM+6ZULKPe6aXVUZmoAeb5lgIWOuEZOgDFR2d05uQ9Aait9S2yyWykPzUUPOdckneC1dwp46k6kvi+u3Fw1If9DsXSeAKVMgD6geOn/hHjEJR0HsE1W1Wp/u6s= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(82310400026)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Mar 2025 22:04:01.1790 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9cc91e06-025e-412c-5924-08dd5cfacd4a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN3PEPF0000B372.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7467 To allocate the xenstore event channel and initialize the grant table entry, the xenstore domid is neeed. A dom0 is craeted before the domUs, so it is normally available through hardware_domain. With capabilities and dom0less, the xenstore domain may not be created first. Keep the population of the page and HVM_PARAM_STORE_PFN in the normal domain construction, but delay event channel and seeding to after all domUs are created. HVM_PARAM_STORE_PFN now serves as indication to setup xenstore since the device tree is no longer immediately available. 0 means no xenstore. ~0ULL means legacy so only the event channel needs setup, and any other value means to seed the page. dom0 needs to set xs_domid when it is serving as the xenstore domain. Signed-off-by: Jason Andryuk --- xen/arch/arm/dom0less-build.c | 71 +++++++++++++++-------- xen/arch/arm/domain_build.c | 1 + xen/arch/arm/include/asm/dom0less-build.h | 2 + 3 files changed, 51 insertions(+), 23 deletions(-) diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c index f1d5bbb097..3b8153b05e 100644 --- a/xen/arch/arm/dom0less-build.c +++ b/xen/arch/arm/dom0less-build.c @@ -22,6 +22,12 @@ #include static domid_t __initdata xs_domid = DOMID_INVALID; +static bool __initdata need_xenstore; + +void __init set_xs_domid(domid_t domid) +{ + xs_domid = domid; +} bool __init is_dom0less_mode(void) { @@ -697,7 +703,7 @@ static int __init alloc_xenstore_evtchn(struct domain *d) int rc; alloc.dom = d->domain_id; - alloc.remote_dom = hardware_domain->domain_id; + alloc.remote_dom = xs_domid; rc = evtchn_alloc_unbound(&alloc, 0); if ( rc ) { @@ -755,10 +761,6 @@ static int __init alloc_xenstore_page(struct domain *d) interface->connection = XENSTORE_RECONNECT; unmap_domain_page(interface); - if ( xs_domid != DOMID_INVALID ) - gnttab_seed_entry(d, GNTTAB_RESERVED_XENSTORE, xs_domid, - gfn_x(gfn), GTF_permit_access); - return 0; } @@ -767,16 +769,10 @@ static int __init alloc_xenstore_params(struct kernel_info *kinfo) struct domain *d = kinfo->d; int rc = 0; - if ( kinfo->dom0less_feature & (DOM0LESS_XENSTORE | DOM0LESS_XS_LEGACY) ) - { - ASSERT(hardware_domain); - rc = alloc_xenstore_evtchn(d); - if ( rc < 0 ) - return rc; + if ( (kinfo->dom0less_feature & (DOM0LESS_XENSTORE | DOM0LESS_XS_LEGACY)) + == (DOM0LESS_XENSTORE | DOM0LESS_XS_LEGACY) ) d->arch.hvm.params[HVM_PARAM_STORE_PFN] = ~0ULL; - } - - if ( kinfo->dom0less_feature & DOM0LESS_XENSTORE ) + else if ( kinfo->dom0less_feature & DOM0LESS_XENSTORE ) { rc = alloc_xenstore_page(d); if ( rc < 0 ) @@ -848,6 +844,34 @@ static void __init domain_vcpu_affinity(struct domain *d, } } +static void __init initialize_domU_xenstore(void) +{ + struct domain *d; + + if ( xs_domid == DOMID_INVALID ) + return; + + for_each_domain( d ) + { + unsigned long gfn = d->arch.hvm.params[HVM_PARAM_STORE_PFN]; + int rc; + + if ( gfn == 0 ) + continue; + + if ( is_xenstore_domain(d) ) + continue; + + rc = alloc_xenstore_evtchn(d); + if ( rc < 0 ) + panic("%pd: Failed to allocate xenstore_evtchn\n", d); + + if ( gfn != ~0ULL ) + gnttab_seed_entry(d, GNTTAB_RESERVED_XENSTORE, xs_domid, + gfn, GTF_permit_access); + } +} + static int __init construct_domU(struct domain *d, const struct dt_device_node *node) { @@ -888,17 +912,13 @@ static int __init construct_domU(struct domain *d, rc == -ENODATA || (rc == 0 && !strcmp(dom0less_enhanced, "enabled")) ) { - if ( hardware_domain ) - kinfo.dom0less_feature = DOM0LESS_ENHANCED; - else - panic("At the moment, Xenstore support requires dom0 to be present\n"); + need_xenstore = true; + kinfo.dom0less_feature = DOM0LESS_ENHANCED; } else if ( rc == 0 && !strcmp(dom0less_enhanced, "legacy") ) { - if ( hardware_domain ) - kinfo.dom0less_feature = DOM0LESS_ENHANCED_LEGACY; - else - panic("At the moment, Xenstore support requires dom0 to be present\n"); + need_xenstore = true; + kinfo.dom0less_feature = DOM0LESS_ENHANCED_LEGACY; } else if ( rc == 0 && !strcmp(dom0less_enhanced, "no-xenstore") ) kinfo.dom0less_feature = DOM0LESS_ENHANCED_NO_XS; @@ -1181,8 +1201,13 @@ void __init create_domUs(void) dt_node_name(node), rc); if ( d_cfg.flags & XEN_DOMCTL_CDF_xs_domain ) - xs_domid = d->domain_id; + set_xs_domid(d->domain_id); } + + if ( need_xenstore && xs_domid == DOMID_INVALID ) + panic("xenstore requested, but xenstore domain not present\n"); + + initialize_domU_xenstore(); } /* diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 00bc484642..ba6bbb95c1 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -2373,6 +2373,7 @@ void __init create_dom0(void) unsigned int flags = CDF_privileged | CDF_hardware; int rc; + set_xs_domid(0); /* The vGIC for DOM0 is exactly emulating the hardware GIC */ dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE; /* diff --git a/xen/arch/arm/include/asm/dom0less-build.h b/xen/arch/arm/include/asm/dom0less-build.h index 5864944bda..df879f93c8 100644 --- a/xen/arch/arm/include/asm/dom0less-build.h +++ b/xen/arch/arm/include/asm/dom0less-build.h @@ -9,6 +9,7 @@ void create_domUs(void); bool is_dom0less_mode(void); +void set_xs_domid(domid_t); #else /* !CONFIG_DOM0LESS_BOOT */ @@ -17,6 +18,7 @@ static inline bool is_dom0less_mode(void) { return false; } +static inline void set_xs_domid(domid_t) {} #endif /* CONFIG_DOM0LESS_BOOT */