| Message ID | 20250212164724.2575624-1-grygorii_strashko@epam.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [RFC] arm: dom0: allow static memory configuration | expand |
On Wed, 12 Feb 2025, Grygorii Strashko wrote: > The Arm Xen allocates memory to place Dom0 following the logic described in > allocate_memory_11() function which is a bit complicated with major > requirement to place Dom0 within the first 128MB of RAM and below 4G. But > this doesn't guarantee it will be placed at the same physical base address > even between two boots with different configuration (changing the Kernel > image size or Initrd size may cause Dom0 base address to change). > > In case of "thin Dom0" use case, when Dom0 implemented with RTOS like > Zephyr, which doesn't use dynamic device-tree parsing, such behavior > causes a lot of inconvenience as it is required to perform modification and > recompiling of Zephyr image to adjust memory layout. > > It also prevents from using Initrd with Zephyr, for example, as it's > expected to be placed at known, fixed address in memory. > > This RFC patch introduces the possibility to place Dom0 at fixed physical > base address, by checking if "chosen" node contains property > "xen,static-mem" and placs Dom0 exactly at the specified memory chunk. > > The implementation follows the same approach as for the static, direct-mapped > guest domain in case of dom0less boot. > > Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com> I fully support this idea and the addition of static memory support to Dom0. However, I would suggest a different approach regarding the device tree binding. Specifically, I would prefer to avoid introducing additional top-level properties for Dom0 under /chosen. Instead, we should create a domain node for Dom0 under /chosen, like we do for other DomUs. Jason is currently working on adding a capability properties to the Dom0less domain nodes, allowing us to specify whether a domain is the hardware domain, the control domain, or both (effectively making it Dom0). Once this is in place, we can use static-mem for Dom0 in the same way as always. > --- > docs/misc/arm/device-tree/booting.txt | 20 ++++++++++++++++++++ > xen/arch/arm/domain_build.c | 12 +++++++++--- > xen/common/device-tree/bootfdt.c | 14 ++++++++++++++ > 3 files changed, 43 insertions(+), 3 deletions(-) > > diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt > index 9c881baccc19..485dcb82de8c 100644 > --- a/docs/misc/arm/device-tree/booting.txt > +++ b/docs/misc/arm/device-tree/booting.txt > @@ -448,6 +448,26 @@ device-tree: > This will reserve a 512MB region starting at the host physical address > 0x30000000 to be exclusively used by DomU1. > > +Dom0 Static Allocation > +====================== > + > +The Memory can be statically allocated to a Dom0 using the property > +"xen,static-mem" defined under the "\chosen" node. This options allows to use > +RTOS as the dom0 kernel, which support only static memory layout. > + > +Below is an DT example: > + > + / { > + chosen { > + #address-cells = <0x1>; > + #size-cells = <0x1>; > + xen,static-mem = <0x50000000 0x8000000>; > + ... > + }; > + > +This will reserve a 128MB region starting at the host physical address > +0x50000000 to be exclusively used by Dom0. > + > Static Event Channel > ==================== > The event channel communication will be established statically between two > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c > index 7b47abade196..8ee280614813 100644 > --- a/xen/arch/arm/domain_build.c > +++ b/xen/arch/arm/domain_build.c > @@ -31,6 +31,7 @@ > #include <asm/cpufeature.h> > #include <asm/dom0less-build.h> > #include <asm/domain_build.h> > +#include <asm/static-memory.h> > #include <asm/static-shmem.h> > #include <xen/event.h> > > @@ -2272,6 +2273,7 @@ int __init construct_domain(struct domain *d, struct kernel_info *kinfo) > static int __init construct_dom0(struct domain *d) > { > struct kernel_info kinfo = KERNEL_INFO_INIT; > + const struct dt_device_node *chosen = dt_find_node_by_path("/chosen"); > int rc; > > /* Sanity! */ > @@ -2305,10 +2307,14 @@ static int __init construct_dom0(struct domain *d) > d->arch.type = kinfo.type; > #endif > find_gnttab_region(d, &kinfo); > - if ( is_domain_direct_mapped(d) ) > - allocate_memory_11(d, &kinfo); > - else > + if ( is_domain_direct_mapped(d) ) { > + if ( !dt_find_property(chosen, "xen,static-mem", NULL) ) > + allocate_memory_11(d, &kinfo); > + else > + assign_static_memory_11(d, &kinfo, chosen); > + } else { > allocate_memory(d, &kinfo); > + } > > rc = process_shm_chosen(d, &kinfo); > if ( rc < 0 ) > diff --git a/xen/common/device-tree/bootfdt.c b/xen/common/device-tree/bootfdt.c > index 529c91e603ab..563a5436fac0 100644 > --- a/xen/common/device-tree/bootfdt.c > +++ b/xen/common/device-tree/bootfdt.c > @@ -413,6 +413,20 @@ static int __init process_chosen_node(const void *fdt, int node, > using_static_heap = true; > } > > + if ( fdt_get_property(fdt, node, "xen,static-mem", NULL) ) > + { > + int rc; > + > + printk("Checking for static static-mem in /chosen\n"); > + > + rc = device_tree_get_meminfo(fdt, node, "xen,static-mem", > + address_cells, size_cells, > + bootinfo_get_reserved_mem(), > + MEMBANK_STATIC_DOMAIN); > + if ( rc ) > + return rc; > + } > + > printk("Checking for initrd in /chosen\n"); > > prop = fdt_get_property(fdt, node, "linux,initrd-start", &len); > -- > 2.34.1 >
Hi Stefano, On 13.02.25 00:11, Stefano Stabellini wrote: > On Wed, 12 Feb 2025, Grygorii Strashko wrote: >> The Arm Xen allocates memory to place Dom0 following the logic described in >> allocate_memory_11() function which is a bit complicated with major >> requirement to place Dom0 within the first 128MB of RAM and below 4G. But >> this doesn't guarantee it will be placed at the same physical base address >> even between two boots with different configuration (changing the Kernel >> image size or Initrd size may cause Dom0 base address to change). >> >> In case of "thin Dom0" use case, when Dom0 implemented with RTOS like >> Zephyr, which doesn't use dynamic device-tree parsing, such behavior >> causes a lot of inconvenience as it is required to perform modification and >> recompiling of Zephyr image to adjust memory layout. >> >> It also prevents from using Initrd with Zephyr, for example, as it's >> expected to be placed at known, fixed address in memory. >> >> This RFC patch introduces the possibility to place Dom0 at fixed physical >> base address, by checking if "chosen" node contains property >> "xen,static-mem" and places Dom0 exactly at the specified memory chunk. >> >> The implementation follows the same approach as for the static, direct-mapped >> guest domain in case of dom0less boot. >> >> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com> > > I fully support this idea and the addition of static memory support to > Dom0. However, I would suggest a different approach regarding the device > tree binding. Specifically, I would prefer to avoid introducing > additional top-level properties for Dom0 under /chosen. That's was major point declaring it RFC. > > Instead, we should create a domain node for Dom0 under /chosen, like we > do for other DomUs. Jason is currently working on adding a capability > properties to the Dom0less domain nodes, allowing us to specify whether > a domain is the hardware domain, the control domain, or both > (effectively making it Dom0). Once this is in place, we can use > static-mem for Dom0 in the same way as always. Good to here that, I assume it can wait (a bit) then. But please note that our requirement here to allow static memory for both dom0less and non-dom0less boot, so here is the question - will bindings and dom0/hwdom/control domain setup be generic? Honestly, for ARM, the discrepancies between boot modes and Xen DT definitions (and actually toolstack) are very confusing :( And now there is also hyperlaunch on the horizon :( [...] BR, -grygorii
On Thu, 13 Feb 2025, Grygorii Strashko wrote: > Hi Stefano, > > On 13.02.25 00:11, Stefano Stabellini wrote: > > On Wed, 12 Feb 2025, Grygorii Strashko wrote: > > > The Arm Xen allocates memory to place Dom0 following the logic described > > > in > > > allocate_memory_11() function which is a bit complicated with major > > > requirement to place Dom0 within the first 128MB of RAM and below 4G. But > > > this doesn't guarantee it will be placed at the same physical base address > > > even between two boots with different configuration (changing the Kernel > > > image size or Initrd size may cause Dom0 base address to change). > > > > > > In case of "thin Dom0" use case, when Dom0 implemented with RTOS like > > > Zephyr, which doesn't use dynamic device-tree parsing, such behavior > > > causes a lot of inconvenience as it is required to perform modification > > > and > > > recompiling of Zephyr image to adjust memory layout. > > > > > > It also prevents from using Initrd with Zephyr, for example, as it's > > > expected to be placed at known, fixed address in memory. > > > > > > This RFC patch introduces the possibility to place Dom0 at fixed physical > > > base address, by checking if "chosen" node contains property > > > "xen,static-mem" and places Dom0 exactly at the specified memory chunk. > > > > > > The implementation follows the same approach as for the static, > > > direct-mapped > > > guest domain in case of dom0less boot. > > > > > > Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com> > > > > I fully support this idea and the addition of static memory support to > > Dom0. However, I would suggest a different approach regarding the device > > tree binding. Specifically, I would prefer to avoid introducing > > additional top-level properties for Dom0 under /chosen. > > That's was major point declaring it RFC. > > > > > Instead, we should create a domain node for Dom0 under /chosen, like we > > do for other DomUs. Jason is currently working on adding a capability > > properties to the Dom0less domain nodes, allowing us to specify whether > > a domain is the hardware domain, the control domain, or both > > (effectively making it Dom0). Once this is in place, we can use > > static-mem for Dom0 in the same way as always. > > Good to here that, I assume it can wait (a bit) then. > > But please note that our requirement here to allow static memory for both > dom0less and > non-dom0less boot, so here is the question - will bindings and > dom0/hwdom/control > domain setup be generic? Yes, they should be generic. > Honestly, for ARM, the discrepancies between boot modes and Xen DT definitions > (and actually toolstack) are very confusing :( And now there is also > hyperlaunch on the horizon :( The good news is that we managed to align the hyperlaunch DT with the dom0less DT and they are now compatible in the last incarnation of the patch series. The ability to use the dom0less domain nodes (not the legacy dom0 nodes) and still be able to specify a dom0 is something that hyperlaunch had from the start and will be of great benefit to ARM as well.
diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt index 9c881baccc19..485dcb82de8c 100644 --- a/docs/misc/arm/device-tree/booting.txt +++ b/docs/misc/arm/device-tree/booting.txt @@ -448,6 +448,26 @@ device-tree: This will reserve a 512MB region starting at the host physical address 0x30000000 to be exclusively used by DomU1. +Dom0 Static Allocation +====================== + +The Memory can be statically allocated to a Dom0 using the property +"xen,static-mem" defined under the "\chosen" node. This options allows to use +RTOS as the dom0 kernel, which support only static memory layout. + +Below is an DT example: + + / { + chosen { + #address-cells = <0x1>; + #size-cells = <0x1>; + xen,static-mem = <0x50000000 0x8000000>; + ... + }; + +This will reserve a 128MB region starting at the host physical address +0x50000000 to be exclusively used by Dom0. + Static Event Channel ==================== The event channel communication will be established statically between two diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 7b47abade196..8ee280614813 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -31,6 +31,7 @@ #include <asm/cpufeature.h> #include <asm/dom0less-build.h> #include <asm/domain_build.h> +#include <asm/static-memory.h> #include <asm/static-shmem.h> #include <xen/event.h> @@ -2272,6 +2273,7 @@ int __init construct_domain(struct domain *d, struct kernel_info *kinfo) static int __init construct_dom0(struct domain *d) { struct kernel_info kinfo = KERNEL_INFO_INIT; + const struct dt_device_node *chosen = dt_find_node_by_path("/chosen"); int rc; /* Sanity! */ @@ -2305,10 +2307,14 @@ static int __init construct_dom0(struct domain *d) d->arch.type = kinfo.type; #endif find_gnttab_region(d, &kinfo); - if ( is_domain_direct_mapped(d) ) - allocate_memory_11(d, &kinfo); - else + if ( is_domain_direct_mapped(d) ) { + if ( !dt_find_property(chosen, "xen,static-mem", NULL) ) + allocate_memory_11(d, &kinfo); + else + assign_static_memory_11(d, &kinfo, chosen); + } else { allocate_memory(d, &kinfo); + } rc = process_shm_chosen(d, &kinfo); if ( rc < 0 ) diff --git a/xen/common/device-tree/bootfdt.c b/xen/common/device-tree/bootfdt.c index 529c91e603ab..563a5436fac0 100644 --- a/xen/common/device-tree/bootfdt.c +++ b/xen/common/device-tree/bootfdt.c @@ -413,6 +413,20 @@ static int __init process_chosen_node(const void *fdt, int node, using_static_heap = true; } + if ( fdt_get_property(fdt, node, "xen,static-mem", NULL) ) + { + int rc; + + printk("Checking for static static-mem in /chosen\n"); + + rc = device_tree_get_meminfo(fdt, node, "xen,static-mem", + address_cells, size_cells, + bootinfo_get_reserved_mem(), + MEMBANK_STATIC_DOMAIN); + if ( rc ) + return rc; + } + printk("Checking for initrd in /chosen\n"); prop = fdt_get_property(fdt, node, "linux,initrd-start", &len);
The Arm Xen allocates memory to place Dom0 following the logic described in allocate_memory_11() function which is a bit complicated with major requirement to place Dom0 within the first 128MB of RAM and below 4G. But this doesn't guarantee it will be placed at the same physical base address even between two boots with different configuration (changing the Kernel image size or Initrd size may cause Dom0 base address to change). In case of "thin Dom0" use case, when Dom0 implemented with RTOS like Zephyr, which doesn't use dynamic device-tree parsing, such behavior causes a lot of inconvenience as it is required to perform modification and recompiling of Zephyr image to adjust memory layout. It also prevents from using Initrd with Zephyr, for example, as it's expected to be placed at known, fixed address in memory. This RFC patch introduces the possibility to place Dom0 at fixed physical base address, by checking if "chosen" node contains property "xen,static-mem" and placs Dom0 exactly at the specified memory chunk. The implementation follows the same approach as for the static, direct-mapped guest domain in case of dom0less boot. Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com> --- docs/misc/arm/device-tree/booting.txt | 20 ++++++++++++++++++++ xen/arch/arm/domain_build.c | 12 +++++++++--- xen/common/device-tree/bootfdt.c | 14 ++++++++++++++ 3 files changed, 43 insertions(+), 3 deletions(-)