From patchwork Fri Aug 19 10:02:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12948656 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DB259C32771 for ; Fri, 19 Aug 2022 10:03:34 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.390192.627487 (Exim 4.92) (envelope-from ) id 1oOyqa-0000Ts-HT; Fri, 19 Aug 2022 10:03:24 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 390192.627487; Fri, 19 Aug 2022 10:03:24 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOyqa-0000Tj-EQ; Fri, 19 Aug 2022 10:03:24 +0000 Received: by outflank-mailman (input) for mailman id 390192; Fri, 19 Aug 2022 10:03:23 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOyqZ-0008OP-IC for xen-devel@lists.xenproject.org; Fri, 19 Aug 2022 10:03:23 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 2850d0f7-1fa6-11ed-bd2e-47488cf2e6aa; Fri, 19 Aug 2022 12:03:22 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3C83F1042; Fri, 19 Aug 2022 03:03:23 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 378EB3F70D; Fri, 19 Aug 2022 03:03:20 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2850d0f7-1fa6-11ed-bd2e-47488cf2e6aa From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Julien Grall , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v2 1/7] xen/evtchn: Make sure all buckets below d->valid_evtchns are allocated Date: Fri, 19 Aug 2022 11:02:38 +0100 Message-Id: <710e9e6477270212136d6f2047fd15a033fa7d71.1660902588.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 From: Julien Grall Since commit 01280dc19cf3 "evtchn: simplify port_is_valid()", the event channels code assumes that all the buckets below d->valid_evtchns are always allocated. This assumption hold in most of the situation because a guest is not allowed to chose the port. Instead, it will be the first free from port 0. When using Guest Transparent Migration and LiveUpdate, we will only preserve ports that are currently in use. As a guest can open/close event channels, this means the ports may be sparse. The existing implementation of evtchn_allocate_port() is not able to deal with such situation and will end up to override bucket or/and leave some bucket unallocated. The latter will result to a droplet crash if the event channel belongs to an unallocated bucket. This can be solved by making sure that all the buckets below d->valid_evtchns are allocated. There should be no impact for most of the situation but LM/LU as only one bucket would be allocated. For LM/LU, we may end up to allocate multiple buckets if ports in use are sparse. A potential alternative is to check that the bucket is valid in is_port_valid(). This should still possible to do it without taking per-domain lock but will result a couple more of memory access. Signed-off-by: Julien Grall Signed-off-by: Rahul Singh --- Changes in v2: - new patch in this version to fix the security issue --- --- xen/common/event_channel.c | 56 ++++++++++++++++++++++++-------------- 1 file changed, 35 insertions(+), 21 deletions(-) diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c index c2c6f8c151..dbe0a27311 100644 --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -193,6 +193,15 @@ static struct evtchn *alloc_evtchn_bucket(struct domain *d, unsigned int port) return NULL; } +/* + * Allocate a given port and ensure all the buckets up to that ports + * have been allocated. + * + * The last part is important because the rest of the event channel code + * relies on all the buckets up to d->valid_evtchns to be valid. However, + * event channels may be sparsed when restoring a domain during Guest + * Transparent Migration and Live Update. + */ int evtchn_allocate_port(struct domain *d, evtchn_port_t port) { if ( port > d->max_evtchn_port || port >= max_evtchns(d) ) @@ -207,30 +216,35 @@ int evtchn_allocate_port(struct domain *d, evtchn_port_t port) } else { - struct evtchn *chn; - struct evtchn **grp; - - if ( !group_from_port(d, port) ) + do { - grp = xzalloc_array(struct evtchn *, BUCKETS_PER_GROUP); - if ( !grp ) - return -ENOMEM; - group_from_port(d, port) = grp; - } + struct evtchn *chn; + struct evtchn **grp; + unsigned int alloc_port = read_atomic(&d->valid_evtchns); - chn = alloc_evtchn_bucket(d, port); - if ( !chn ) - return -ENOMEM; - bucket_from_port(d, port) = chn; + if ( !group_from_port(d, alloc_port) ) + { + grp = xzalloc_array(struct evtchn *, BUCKETS_PER_GROUP); + if ( !grp ) + return -ENOMEM; + group_from_port(d, alloc_port) = grp; + } - /* - * d->valid_evtchns is used to check whether the bucket can be - * accessed without the per-domain lock. Therefore, - * d->valid_evtchns should be seen *after* the new bucket has - * been setup. - */ - smp_wmb(); - write_atomic(&d->valid_evtchns, d->valid_evtchns + EVTCHNS_PER_BUCKET); + chn = alloc_evtchn_bucket(d, alloc_port); + if ( !chn ) + return -ENOMEM; + bucket_from_port(d, alloc_port) = chn; + + /* + * d->valid_evtchns is used to check whether the bucket can be + * accessed without the per-domain lock. Therefore, + * d->valid_evtchns should be seen *after* the new bucket has + * been setup. + */ + smp_wmb(); + write_atomic(&d->valid_evtchns, + d->valid_evtchns + EVTCHNS_PER_BUCKET); + } while ( port >= read_atomic(&d->valid_evtchns) ); } write_atomic(&d->active_evtchns, d->active_evtchns + 1); From patchwork Fri Aug 19 10:02:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12948657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DF6FCC32771 for ; Fri, 19 Aug 2022 10:04:09 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.390202.627498 (Exim 4.92) (envelope-from ) id 1oOyr9-00012s-Pb; Fri, 19 Aug 2022 10:03:59 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 390202.627498; Fri, 19 Aug 2022 10:03:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOyr9-00012l-MX; Fri, 19 Aug 2022 10:03:59 +0000 Received: by outflank-mailman (input) for mailman id 390202; Fri, 19 Aug 2022 10:03:58 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOyr8-0000LK-MF for xen-devel@lists.xenproject.org; Fri, 19 Aug 2022 10:03:58 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 3d96aeb3-1fa6-11ed-9250-1f966e50362f; Fri, 19 Aug 2022 12:03:57 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 083A11042; Fri, 19 Aug 2022 03:03:59 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DB71B3F70D; Fri, 19 Aug 2022 03:03:55 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 3d96aeb3-1fa6-11ed-9250-1f966e50362f From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu , Stanislav Kinsburskii , Julien Grall Subject: [PATCH v2 2/7] xen/evtchn: Add an helper to reserve/allocate a port Date: Fri, 19 Aug 2022 11:02:39 +0100 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 In a follow-up patch we will be able to either reserve or allocate a port for various event channel helpers. A new wrapper is introduced to either reserved a given port or allocate an empty one if zero. Take the opportunity to replace the open-coded version in evtchn_bind_virq(). Signed-off-by: Stanislav Kinsburskii Signed-off-by: Julien Grall Signed-off-by: Rahul Singh Acked-by: Jan Beulich --- Changes in v2: - new patch in this version --- --- xen/common/event_channel.c | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c index dbe0a27311..194f5346fb 100644 --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -304,6 +304,18 @@ void evtchn_free(struct domain *d, struct evtchn *chn) xsm_evtchn_close_post(chn); } +static int evtchn_get_port(struct domain *d, evtchn_port_t port) +{ + int rc; + + if ( port != 0 ) + rc = evtchn_allocate_port(d, port); + else + rc = get_free_port(d); + + return rc ?: port; +} + int evtchn_alloc_unbound(evtchn_alloc_unbound_t *alloc) { struct evtchn *chn; @@ -461,19 +473,10 @@ int evtchn_bind_virq(evtchn_bind_virq_t *bind, evtchn_port_t port) if ( read_atomic(&v->virq_to_evtchn[virq]) ) ERROR_EXIT(-EEXIST); - if ( port != 0 ) - { - if ( (rc = evtchn_allocate_port(d, port)) != 0 ) - ERROR_EXIT(rc); - } - else - { - int alloc_port = get_free_port(d); - - if ( alloc_port < 0 ) - ERROR_EXIT(alloc_port); - port = alloc_port; - } + port = rc = evtchn_get_port(d, port); + if ( rc < 0 ) + ERROR_EXIT(rc); + rc = 0; chn = evtchn_from_port(d, port); From patchwork Fri Aug 19 10:02:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12948658 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 43080C32771 for ; Fri, 19 Aug 2022 10:04:37 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.390207.627508 (Exim 4.92) (envelope-from ) id 1oOyrd-0001bh-1u; Fri, 19 Aug 2022 10:04:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 390207.627508; Fri, 19 Aug 2022 10:04:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOyrc-0001ba-VW; Fri, 19 Aug 2022 10:04:28 +0000 Received: by outflank-mailman (input) for mailman id 390207; Fri, 19 Aug 2022 10:04:28 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOyrc-0008OP-6g for xen-devel@lists.xenproject.org; Fri, 19 Aug 2022 10:04:28 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 4f3744ff-1fa6-11ed-bd2e-47488cf2e6aa; Fri, 19 Aug 2022 12:04:27 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 945771042; Fri, 19 Aug 2022 03:04:28 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8F67C3F70D; Fri, 19 Aug 2022 03:04:25 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4f3744ff-1fa6-11ed-bd2e-47488cf2e6aa From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Wei Liu Subject: [PATCH v2 3/7] xen/evtchn: restrict the maximum number of evtchn supported for domUs Date: Fri, 19 Aug 2022 11:02:40 +0100 Message-Id: <1a8c49dcc237187cbb9fccaafe1e6533fe68381c.1660902588.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Static event channel support will be added for dom0less domains. Restrict the maximum number of evtchn supported for domUs to avoid allocating a large amount of memory in Xen. Signed-off-by: Rahul Singh --- Changes in v2: - new patch in the version --- --- xen/arch/arm/domain_build.c | 2 +- xen/include/xen/sched.h | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 3fd1186b53..6d447367be 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -3277,7 +3277,7 @@ void __init create_domUs(void) struct xen_domctl_createdomain d_cfg = { .arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE, .flags = XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap, - .max_evtchn_port = -1, + .max_evtchn_port = MAX_EVTCHNS_PORT, .max_grant_frames = -1, .max_maptrack_frames = -1, .grant_opts = XEN_DOMCTL_GRANT_version(opt_gnttab_max_version), diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index e2b3b6daa3..e5cc4f3e3e 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -76,6 +76,9 @@ extern domid_t hardware_domid; /* Maximum number of event channels for any ABI. */ #define MAX_NR_EVTCHNS MAX(EVTCHN_2L_NR_CHANNELS, EVTCHN_FIFO_NR_CHANNELS) +/* Maximum number of event channels supported for domUs. */ +#define MAX_EVTCHNS_PORT 4096 + #define EVTCHNS_PER_BUCKET (PAGE_SIZE / next_power_of_2(sizeof(struct evtchn))) #define EVTCHNS_PER_GROUP (BUCKETS_PER_GROUP * EVTCHNS_PER_BUCKET) #define NR_EVTCHN_GROUPS DIV_ROUND_UP(MAX_NR_EVTCHNS, EVTCHNS_PER_GROUP) From patchwork Fri Aug 19 10:02:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12948659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 60AFCC32771 for ; Fri, 19 Aug 2022 10:06:18 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.390215.627520 (Exim 4.92) (envelope-from ) id 1oOytE-0002Hz-CZ; Fri, 19 Aug 2022 10:06:08 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 390215.627520; Fri, 19 Aug 2022 10:06:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOytE-0002Hs-9k; Fri, 19 Aug 2022 10:06:08 +0000 Received: by outflank-mailman (input) for mailman id 390215; Fri, 19 Aug 2022 10:06:06 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOytC-0002Hf-LH for xen-devel@lists.xenproject.org; Fri, 19 Aug 2022 10:06:06 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 8982e0c5-1fa6-11ed-9250-1f966e50362f; Fri, 19 Aug 2022 12:06:05 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 649E31042; Fri, 19 Aug 2022 03:06:06 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 79E543F70D; Fri, 19 Aug 2022 03:06:03 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8982e0c5-1fa6-11ed-9250-1f966e50362f From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v2 4/7] xen/evtchn: modify evtchn_bind_interdomain to support static evtchn Date: Fri, 19 Aug 2022 11:02:41 +0100 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Static event channel support will be added for dom0less domains. Modify evtchn_bind_interdomain to support static evtchn. It is necessary to have access to the evtchn_bind_interdomain function to do that, so make evtchn_bind_interdomain global and also make it __must_check. evtchn_bind_interdomain() always allocates the next available local port. Static event channel support for dom0less domains requires allocating a specified port. Modify the evtchn_bind_interdomain to accept the port number as an argument and allocate the specified port if available. If the port number argument is zero, the next available port will be allocated. evtchn_bind_interdomain() finds the local domain from "current->domain" pointer. evtchn_bind_interdomain() will be called from the XEN to create static event channel during domain creation. "current" pointer is not valid at that time, therefore modify the evtchn_bind_interdomain() to pass domain as an argument. Signed-off-by: Rahul Singh Acked-by: Jan Beulich Reviewed-by: Julien Grall --- Changes in v2: - Merged patches related to evtchn_bind_interdomain in one patch --- --- xen/common/event_channel.c | 20 ++++++++++++++------ xen/include/xen/event.h | 5 +++++ 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c index 194f5346fb..eed0e94d07 100644 --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -373,11 +373,16 @@ static void double_evtchn_unlock(struct evtchn *lchn, struct evtchn *rchn) evtchn_write_unlock(rchn); } -static int evtchn_bind_interdomain(evtchn_bind_interdomain_t *bind) +/* + * If lport is zero get the next free port and allocate. If port is non-zero + * allocate the specified lport. + */ +int evtchn_bind_interdomain(evtchn_bind_interdomain_t *bind, struct domain *ld, + evtchn_port_t lport) { struct evtchn *lchn, *rchn; - struct domain *ld = current->domain, *rd; - int lport, rc; + struct domain *rd; + int rc; evtchn_port_t rport = bind->remote_port; domid_t rdom = bind->remote_dom; @@ -397,8 +402,11 @@ static int evtchn_bind_interdomain(evtchn_bind_interdomain_t *bind) write_lock(&ld->event_lock); } - if ( (lport = get_free_port(ld)) < 0 ) - ERROR_EXIT(lport); + lport = rc = evtchn_get_port(ld, lport); + if ( rc < 0 ) + ERROR_EXIT(rc); + rc = 0; + lchn = evtchn_from_port(ld, lport); rchn = _evtchn_from_port(rd, rport); @@ -1231,7 +1239,7 @@ long do_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) struct evtchn_bind_interdomain bind_interdomain; if ( copy_from_guest(&bind_interdomain, arg, 1) != 0 ) return -EFAULT; - rc = evtchn_bind_interdomain(&bind_interdomain); + rc = evtchn_bind_interdomain(&bind_interdomain, current->domain, 0); if ( !rc && __copy_to_guest(arg, &bind_interdomain, 1) ) rc = -EFAULT; /* Cleaning up here would be a mess! */ break; diff --git a/xen/include/xen/event.h b/xen/include/xen/event.h index f3021fe304..f29b1123d9 100644 --- a/xen/include/xen/event.h +++ b/xen/include/xen/event.h @@ -74,6 +74,11 @@ int evtchn_allocate_port(struct domain *d, unsigned int port); /* Allocate a new event channel */ int __must_check evtchn_alloc_unbound(evtchn_alloc_unbound_t *alloc); +/* Bind an event channel port to interdomain */ +int __must_check evtchn_bind_interdomain(evtchn_bind_interdomain_t *bind, + struct domain *ld, + evtchn_port_t port); + /* Unmask a local event-channel port. */ int evtchn_unmask(unsigned int port); From patchwork Fri Aug 19 10:02:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12948660 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5A2CAC32773 for ; Fri, 19 Aug 2022 10:06:20 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.390216.627531 (Exim 4.92) (envelope-from ) id 1oOytI-0002Zn-KI; Fri, 19 Aug 2022 10:06:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 390216.627531; Fri, 19 Aug 2022 10:06:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOytI-0002Zg-H8; Fri, 19 Aug 2022 10:06:12 +0000 Received: by outflank-mailman (input) for mailman id 390216; Fri, 19 Aug 2022 10:06:10 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOytG-0002G7-Qw for xen-devel@lists.xenproject.org; Fri, 19 Aug 2022 10:06:10 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 8c2f9a22-1fa6-11ed-bd2e-47488cf2e6aa; Fri, 19 Aug 2022 12:06:09 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CCC781042; Fri, 19 Aug 2022 03:06:10 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C7B8E3F70D; Fri, 19 Aug 2022 03:06:07 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8c2f9a22-1fa6-11ed-bd2e-47488cf2e6aa From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Wei Liu Subject: [PATCH v2 5/7] xen/evtchn: modify evtchn_alloc_unbound to allocate specified port Date: Fri, 19 Aug 2022 11:02:42 +0100 Message-Id: <02234e7aab1e8f3d18e5398f1c4745203df77364.1660902588.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 evtchn_alloc_unbound() always allocates the next available port. Static event channel support for dom0less domains requires allocating a specified port. Modify the evtchn_alloc_unbound() to accept the port number as an argument and allocate the specified port if available. If the port number argument is zero, the next available port will be allocated. Signed-off-by: Rahul Singh Acked-by: Jan Beulich Reviewed-by: Julien Grall --- Changes in v2: - fix minor comments --- --- xen/arch/arm/domain_build.c | 2 +- xen/common/event_channel.c | 17 ++++++++++++----- xen/include/xen/event.h | 3 ++- 3 files changed, 15 insertions(+), 7 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 6d447367be..11a8c6b8b5 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -3171,7 +3171,7 @@ static int __init alloc_xenstore_evtchn(struct domain *d) alloc.dom = d->domain_id; alloc.remote_dom = hardware_domain->domain_id; - rc = evtchn_alloc_unbound(&alloc); + rc = evtchn_alloc_unbound(&alloc, 0); if ( rc ) { printk("Failed allocating event channel for domain\n"); diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c index eed0e94d07..f2d8c2d61a 100644 --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -316,11 +316,15 @@ static int evtchn_get_port(struct domain *d, evtchn_port_t port) return rc ?: port; } -int evtchn_alloc_unbound(evtchn_alloc_unbound_t *alloc) +/* + * If port is zero get the next free port and allocate. If port is non-zero + * allocate the specified port. + */ +int evtchn_alloc_unbound(evtchn_alloc_unbound_t *alloc, evtchn_port_t port) { struct evtchn *chn; struct domain *d; - int port, rc; + int rc; domid_t dom = alloc->dom; d = rcu_lock_domain_by_any_id(dom); @@ -329,8 +333,11 @@ int evtchn_alloc_unbound(evtchn_alloc_unbound_t *alloc) write_lock(&d->event_lock); - if ( (port = get_free_port(d)) < 0 ) - ERROR_EXIT_DOM(port, d); + port = rc = evtchn_get_port(d, port); + if ( rc < 0 ) + ERROR_EXIT(rc); + rc = 0; + chn = evtchn_from_port(d, port); rc = xsm_evtchn_unbound(XSM_TARGET, d, chn, alloc->remote_dom); @@ -1229,7 +1236,7 @@ long do_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) struct evtchn_alloc_unbound alloc_unbound; if ( copy_from_guest(&alloc_unbound, arg, 1) != 0 ) return -EFAULT; - rc = evtchn_alloc_unbound(&alloc_unbound); + rc = evtchn_alloc_unbound(&alloc_unbound, 0); if ( !rc && __copy_to_guest(arg, &alloc_unbound, 1) ) rc = -EFAULT; /* Cleaning up here would be a mess! */ break; diff --git a/xen/include/xen/event.h b/xen/include/xen/event.h index f29b1123d9..8eae9984a9 100644 --- a/xen/include/xen/event.h +++ b/xen/include/xen/event.h @@ -72,7 +72,8 @@ void evtchn_free(struct domain *d, struct evtchn *chn); int evtchn_allocate_port(struct domain *d, unsigned int port); /* Allocate a new event channel */ -int __must_check evtchn_alloc_unbound(evtchn_alloc_unbound_t *alloc); +int __must_check evtchn_alloc_unbound(evtchn_alloc_unbound_t *alloc, + evtchn_port_t port); /* Bind an event channel port to interdomain */ int __must_check evtchn_bind_interdomain(evtchn_bind_interdomain_t *bind, From patchwork Fri Aug 19 10:02:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12948661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 92B1DC32771 for ; Fri, 19 Aug 2022 10:06:24 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.390217.627542 (Exim 4.92) (envelope-from ) id 1oOytL-0002sD-00; Fri, 19 Aug 2022 10:06:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 390217.627542; Fri, 19 Aug 2022 10:06:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOytK-0002s6-SB; Fri, 19 Aug 2022 10:06:14 +0000 Received: by outflank-mailman (input) for mailman id 390217; Fri, 19 Aug 2022 10:06:13 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOytJ-0002G7-EC for xen-devel@lists.xenproject.org; Fri, 19 Aug 2022 10:06:13 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 8d9f2eed-1fa6-11ed-bd2e-47488cf2e6aa; Fri, 19 Aug 2022 12:06:12 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3CFF71042; Fri, 19 Aug 2022 03:06:13 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 879A43F70D; Fri, 19 Aug 2022 03:06:10 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8d9f2eed-1fa6-11ed-bd2e-47488cf2e6aa From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk Subject: [PATCH v2 6/7] xen: introduce xen-evtchn dom0less property Date: Fri, 19 Aug 2022 11:02:43 +0100 Message-Id: <02993cf398573adf9e9bad62aa8d6e753b2c6ab9.1660902588.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Introduce a new sub-node under /chosen node to establish static event channel communication between domains on dom0less systems. An event channel will be created beforehand to allow the domains to send notifications to each other. Signed-off-by: Rahul Singh --- Changes in v2: - no change --- --- docs/misc/arm/device-tree/booting.txt | 63 +++++++++++- xen/arch/arm/domain_build.c | 136 ++++++++++++++++++++++++++ xen/arch/arm/include/asm/domain.h | 1 + xen/arch/arm/include/asm/setup.h | 1 + xen/arch/arm/setup.c | 2 + 5 files changed, 202 insertions(+), 1 deletion(-) diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt index 98253414b8..ec7dbcaf8f 100644 --- a/docs/misc/arm/device-tree/booting.txt +++ b/docs/misc/arm/device-tree/booting.txt @@ -212,7 +212,7 @@ with the following properties: enable only selected interfaces. Under the "xen,domain" compatible node, one or more sub-nodes are present -for the DomU kernel and ramdisk. +for the DomU kernel, ramdisk and static event channel. The kernel sub-node has the following properties: @@ -254,11 +254,43 @@ The ramdisk sub-node has the following properties: property because it will be created by the UEFI stub on boot. This option is needed only when UEFI boot is used. +The static event channel sub-node has the following properties: + +- compatible + + "xen,evtchn" + +- xen,evtchn + + The property is tuples of two numbers + (local-evtchn link-to-foreign-evtchn) where: + + local-evtchn is an integer value that will be used to allocate local port + for a domain to send and receive event notifications to/from the remote + domain. Maximum supported value is 2^17 for FIFO ABI and 4096 for 2L ABI. + It is recommended to use low event channel ID. + + link-to-foreign-evtchn is a single phandle to a remote evtchn to which + local-evtchn will be connected. Example ======= chosen { + + module@0 { + compatible = "multiboot,kernel", "multiboot,module"; + xen,uefi-binary = "..."; + bootargs = "..."; + + /* one sub-node per local event channel */ + ec1: evtchn@1 { + compatible = "xen,evtchn-v1"; + /* local-evtchn link-to-foreign-evtchn */ + xen,evtchn = <0xa &ec2>; + }; + }; + domU1 { compatible = "xen,domain"; #address-cells = <0x2>; @@ -277,6 +309,23 @@ chosen { compatible = "multiboot,ramdisk", "multiboot,module"; reg = <0x0 0x4b000000 0xffffff>; }; + + /* one sub-node per local event channel */ + ec2: evtchn@2 { + compatible = "xen,evtchn-v1"; + /* local-evtchn link-to-foreign-evtchn */ + xen,evtchn = <0xa &ec1>; + }; + + ec3: evtchn@3 { + compatible = "xen,evtchn-v1"; + xen,evtchn = <0xb &ec5>; + }; + + ec4: evtchn@4 { + compatible = "xen,evtchn-v1"; + xen,evtchn = <0xc &ec6>; + }; }; domU2 { @@ -296,6 +345,18 @@ chosen { compatible = "multiboot,ramdisk", "multiboot,module"; reg = <0x0 0x4d000000 0xffffff>; }; + + /* one sub-node per local event channel */ + ec5: evtchn@5 { + compatible = "xen,evtchn-v1"; + /* local-evtchn link-to-foreign-evtchn */ + xen,evtchn = <0xb &ec3>; + }; + + ec6: evtchn@6 { + compatible = "xen,evtchn-v1"; + xen,evtchn = <0xd &ec4>; + }; }; }; diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 11a8c6b8b5..5101bca979 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -3052,6 +3052,141 @@ void __init evtchn_allocate(struct domain *d) d->arch.hvm.params[HVM_PARAM_CALLBACK_IRQ] = val; } +static const void *__init get_evtchn_dt_property( + const struct dt_device_node *np) +{ + const void *prop = NULL; + uint32_t len; + + prop = dt_get_property(np, "xen,evtchn", &len); + if ( !prop ) + return NULL; + + if ( !len ) + { + printk(XENLOG_ERR "xen,evtchn property cannot be empty.\n"); + return ERR_PTR(-EINVAL); + } + + return prop; +} + +static int __init allocate_domain_evtchn(const struct dt_device_node *node) +{ + const void *prop = NULL; + const __be32 *cell; + uint32_t domU1_port, domU2_port, remote_phandle; + const struct dt_device_node *evtchn_node, *remote_node; + struct evtchn_alloc_unbound alloc_unbound; + struct evtchn_bind_interdomain bind_interdomain; + int rc; + + dt_for_each_child_node(node, evtchn_node) + { + struct domain *d, *d1 = NULL, *d2 = NULL; + + if ( !dt_device_is_compatible(evtchn_node, "xen,evtchn-v1") ) + continue; + + prop = get_evtchn_dt_property(evtchn_node); + /* If the property is not found, return without errors */ + if ( !prop || IS_ERR(prop) ) + return IS_ERR(prop) ? PTR_ERR(prop) : 0; + + cell = (const __be32 *)prop; + domU1_port = dt_next_cell(1, &cell); + remote_phandle = dt_next_cell(1, &cell); + + remote_node = dt_find_node_by_phandle(remote_phandle); + if ( !remote_node ) + { + printk(XENLOG_ERR + "evtchn: could not find remote evtchn phandle\n"); + return -EINVAL; + } + + prop = get_evtchn_dt_property(remote_node); + /* If the property is not found, return without errors */ + if ( !prop || IS_ERR(prop) ) + return IS_ERR(prop) ? PTR_ERR(prop) : 0; + + cell = (const __be32 *)prop; + domU2_port = dt_next_cell(1, &cell); + remote_phandle = dt_next_cell(1, &cell); + + if ( evtchn_node->phandle != remote_phandle ) + { + printk(XENLOG_ERR "xen,evtchn property is not setup correctly.\n"); + return -EINVAL; + } + + for_each_domain ( d ) + { + if ( d->arch.node == node ) + { + d1 = d; + continue; + } + if ( d->arch.node == dt_get_parent(remote_node) ) + d2 = d; + } + + if ( !d1 && dt_device_is_compatible(node, "multiboot,kernel") ) + d1 = hardware_domain; + + if ( !d2 && dt_device_is_compatible(dt_get_parent(remote_node), + "multiboot,kernel") ) + d2 = hardware_domain; + + if ( !d1 || !d2 ) + { + printk(XENLOG_ERR "evtchn: could not find domains\n" ); + return -EINVAL; + } + + alloc_unbound.dom = d1->domain_id; + alloc_unbound.remote_dom = d2->domain_id; + + rc = evtchn_alloc_unbound(&alloc_unbound, domU1_port); + if ( rc < 0 && rc != -EBUSY ) + { + printk(XENLOG_ERR + "evtchn_alloc_unbound() failure (Error %d) \n", rc); + return rc; + } + + bind_interdomain.remote_dom = d1->domain_id; + bind_interdomain.remote_port = domU1_port; + + rc = evtchn_bind_interdomain(&bind_interdomain, d2, domU2_port); + if ( rc < 0 && rc != -EBUSY ) + { + printk(XENLOG_ERR + "evtchn_bind_interdomain() failure (Error %d) \n", rc); + return rc; + } + } + + return 0; +} + +void __init allocate_static_evtchn(void) +{ + struct dt_device_node *node; + const struct dt_device_node *chosen = dt_find_node_by_path("/chosen"); + + BUG_ON(chosen == NULL); + dt_for_each_child_node(chosen, node) + { + if ( dt_device_is_compatible(node, "xen,domain") || + dt_device_is_compatible(node, "multiboot,kernel") ) + { + if ( allocate_domain_evtchn(node) != 0 ) + panic("Could not set up domains evtchn\n"); + } + } +} + static void __init find_gnttab_region(struct domain *d, struct kernel_info *kinfo) { @@ -3358,6 +3493,7 @@ void __init create_domUs(void) panic("Error creating domain %s\n", dt_node_name(node)); d->is_console = true; + d->arch.node = node; if ( construct_domU(d, node) != 0 ) panic("Could not set up domain %s\n", dt_node_name(node)); diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h index cd9ce19b4b..51192b28ee 100644 --- a/xen/arch/arm/include/asm/domain.h +++ b/xen/arch/arm/include/asm/domain.h @@ -105,6 +105,7 @@ struct arch_domain #endif bool directmap; + struct dt_device_node *node; } __cacheline_aligned; struct arch_vcpu diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h index 2bb01ecfa8..bac876e68e 100644 --- a/xen/arch/arm/include/asm/setup.h +++ b/xen/arch/arm/include/asm/setup.h @@ -106,6 +106,7 @@ int acpi_make_efi_nodes(void *fdt, struct membank tbl_add[]); void create_domUs(void); void create_dom0(void); +void allocate_static_evtchn(void); void discard_initial_modules(void); void fw_unreserved_regions(paddr_t s, paddr_t e, diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index 500307edc0..8eead619ae 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -1063,6 +1063,8 @@ void __init start_xen(unsigned long boot_phys_offset, if ( acpi_disabled ) create_domUs(); + allocate_static_evtchn(); + /* * This needs to be called **before** heap_init_late() so modules * will be scrubbed (unless suppressed). From patchwork Fri Aug 19 10:02:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rahul Singh X-Patchwork-Id: 12948662 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9DB00C32773 for ; Fri, 19 Aug 2022 10:06:25 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.390218.627553 (Exim 4.92) (envelope-from ) id 1oOytN-0003Bq-8A; Fri, 19 Aug 2022 10:06:17 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 390218.627553; Fri, 19 Aug 2022 10:06:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOytN-0003Bd-4w; Fri, 19 Aug 2022 10:06:17 +0000 Received: by outflank-mailman (input) for mailman id 390218; Fri, 19 Aug 2022 10:06:15 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oOytL-0002G7-NL for xen-devel@lists.xenproject.org; Fri, 19 Aug 2022 10:06:15 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 8f101441-1fa6-11ed-bd2e-47488cf2e6aa; Fri, 19 Aug 2022 12:06:14 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B42D2175A; Fri, 19 Aug 2022 03:06:15 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 259E53F70D; Fri, 19 Aug 2022 03:06:13 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8f101441-1fa6-11ed-bd2e-47488cf2e6aa From: Rahul Singh To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, rahul.singh@arm.com, Stefano Stabellini , Julien Grall , Volodymyr Babchuk Subject: [PATCH v2 7/7] xen/arm: introduce new xen,enhanced property value Date: Fri, 19 Aug 2022 11:02:44 +0100 Message-Id: <2fb69ff7cf9a36dd1294da4f9f4b968ff7076d42.1660902588.git.rahul.singh@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Introduce a new "xen,enhanced" dom0less property value "evtchn" to enable/disable event-channel interfaces for dom0less guests. The configurable option is for domUs only. For dom0 we always set the corresponding property in the Xen code to true. Signed-off-by: Rahul Singh --- Changes in v2: - no change --- --- xen/arch/arm/domain_build.c | 149 ++++++++++++++++-------------- xen/arch/arm/include/asm/kernel.h | 3 + 2 files changed, 82 insertions(+), 70 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 5101bca979..bd8b8475b7 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -1396,85 +1396,92 @@ static int __init make_hypervisor_node(struct domain *d, if ( res ) return res; - if ( !opt_ext_regions ) - { - printk(XENLOG_INFO "%pd: extended regions support is disabled\n", d); - nr_ext_regions = 0; - } - else if ( is_32bit_domain(d) ) - { - printk(XENLOG_WARNING - "%pd: extended regions not supported for 32-bit guests\n", d); - nr_ext_regions = 0; - } - else + if ( kinfo->dom0less_enhanced ) { - ext_regions = xzalloc(struct meminfo); - if ( !ext_regions ) - return -ENOMEM; - - if ( is_domain_direct_mapped(d) ) + if ( !opt_ext_regions ) { - if ( !is_iommu_enabled(d) ) - res = find_unallocated_memory(kinfo, ext_regions); - else - res = find_memory_holes(kinfo, ext_regions); + printk(XENLOG_INFO + "%pd: extended regions support is disabled\n", d); + nr_ext_regions = 0; } - else + else if ( is_32bit_domain(d) ) { - res = find_domU_holes(kinfo, ext_regions); + printk(XENLOG_WARNING + "%pd: extended regions not supported for 32-bit guests\n", d); + nr_ext_regions = 0; } + else + { + ext_regions = xzalloc(struct meminfo); + if ( !ext_regions ) + return -ENOMEM; - if ( res ) - printk(XENLOG_WARNING "%pd: failed to allocate extended regions\n", - d); - nr_ext_regions = ext_regions->nr_banks; - } + if ( is_domain_direct_mapped(d) ) + { + if ( !is_iommu_enabled(d) ) + res = find_unallocated_memory(kinfo, ext_regions); + else + res = find_memory_holes(kinfo, ext_regions); + } + else + { + res = find_domU_holes(kinfo, ext_regions); + } - reg = xzalloc_array(__be32, (nr_ext_regions + 1) * (addrcells + sizecells)); - if ( !reg ) - { - xfree(ext_regions); - return -ENOMEM; - } + if ( res ) + printk(XENLOG_WARNING + "%pd: failed to allocate extended regions\n", d); + nr_ext_regions = ext_regions->nr_banks; + } - /* reg 0 is grant table space */ - cells = ®[0]; - dt_child_set_range(&cells, addrcells, sizecells, - kinfo->gnttab_start, kinfo->gnttab_size); - /* reg 1...N are extended regions */ - for ( i = 0; i < nr_ext_regions; i++ ) - { - u64 start = ext_regions->bank[i].start; - u64 size = ext_regions->bank[i].size; + reg = xzalloc_array(__be32, (nr_ext_regions + 1) * (addrcells + sizecells)); + if ( !reg ) + { + xfree(ext_regions); + return -ENOMEM; + } - printk("%pd: extended region %d: %#"PRIx64"->%#"PRIx64"\n", - d, i, start, start + size); + /* reg 0 is grant table space */ + cells = ®[0]; + dt_child_set_range(&cells, addrcells, sizecells, + kinfo->gnttab_start, kinfo->gnttab_size); + /* reg 1...N are extended regions */ + for ( i = 0; i < nr_ext_regions; i++ ) + { + u64 start = ext_regions->bank[i].start; + u64 size = ext_regions->bank[i].size; - dt_child_set_range(&cells, addrcells, sizecells, start, size); - } + printk("%pd: extended region %d: %#"PRIx64"->%#"PRIx64"\n", + d, i, start, start + size); - res = fdt_property(fdt, "reg", reg, - dt_cells_to_size(addrcells + sizecells) * - (nr_ext_regions + 1)); - xfree(ext_regions); - xfree(reg); + dt_child_set_range(&cells, addrcells, sizecells, start, size); + } - if ( res ) - return res; + res = fdt_property(fdt, "reg", reg, + dt_cells_to_size(addrcells + sizecells) * + (nr_ext_regions + 1)); + xfree(ext_regions); + xfree(reg); - BUG_ON(d->arch.evtchn_irq == 0); + if ( res ) + return res; + } - /* - * Interrupt event channel upcall: - * - Active-low level-sensitive - * - All CPUs - * TODO: Handle properly the cpumask; - */ - set_interrupt(intr, d->arch.evtchn_irq, 0xf, DT_IRQ_TYPE_LEVEL_LOW); - res = fdt_property_interrupts(kinfo, &intr, 1); - if ( res ) - return res; + if ( kinfo->dom0less_evtchn ) + { + BUG_ON(d->arch.evtchn_irq == 0); + + /* + * Interrupt event channel upcall: + * - Active-low level-sensitive + * - All CPUs + * TODO: Handle properly the cpumask; + */ + set_interrupt(intr, d->arch.evtchn_irq, 0xf, DT_IRQ_TYPE_LEVEL_LOW); + res = fdt_property_interrupts(kinfo, &intr, 1); + if ( res ) + return res; + } res = fdt_end_node(fdt); @@ -2891,7 +2898,7 @@ static int __init prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo) goto err; } - if ( kinfo->dom0less_enhanced ) + if ( kinfo->dom0less_enhanced || kinfo->dom0less_evtchn ) { ret = make_hypervisor_node(d, kinfo, addrcells, sizecells); if ( ret ) @@ -3343,11 +3350,11 @@ static int __init construct_domU(struct domain *d, rc == -ENODATA || (rc == 0 && !strcmp(dom0less_enhanced, "enabled")) ) { - if ( hardware_domain ) - kinfo.dom0less_enhanced = true; - else - panic("Tried to use xen,enhanced without dom0\n"); + kinfo.dom0less_enhanced = true; + kinfo.dom0less_evtchn = true; } + else if ( rc == 0 && !strcmp(dom0less_enhanced, "evtchn") ) + kinfo.dom0less_evtchn = true; if ( vcpu_create(d, 0) == NULL ) return -ENOMEM; @@ -3526,6 +3533,8 @@ static int __init construct_dom0(struct domain *d) kinfo.unassigned_mem = dom0_mem; kinfo.d = d; + kinfo.dom0less_enhanced = true; + kinfo.dom0less_evtchn = true; rc = kernel_probe(&kinfo, NULL); if ( rc < 0 ) diff --git a/xen/arch/arm/include/asm/kernel.h b/xen/arch/arm/include/asm/kernel.h index c4dc039b54..7cff19b997 100644 --- a/xen/arch/arm/include/asm/kernel.h +++ b/xen/arch/arm/include/asm/kernel.h @@ -39,6 +39,9 @@ struct kernel_info { /* Enable PV drivers */ bool dom0less_enhanced; + /* Enable event-channel interface */ + bool dom0less_evtchn; + /* GIC phandle */ uint32_t phandle_gic;