| Message ID | 20210316031814.10311-1-dpsmith@apertussolutions.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Introducing hyperlaunch capability design (formerly: DomB mode of dom0less) | expand |
On 3/15/21 11:18 PM, Daniel P. Smith wrote: > We are submitting for inclusion in the Xen documentation: > > - the hyperlaunch design document, and > - the hyperlaunch device tree design document > > to describe a new method for launching the Xen hypervisor. > > The hyperlaunch feature builds upon prior dom0less work and the DomB > project to bring a flexible and security-minded means to launch a > variety of VM configurations as part of the startup of Xen. > > Signed-off-by: Christopher Clark <christopher.clark@starlab.io> > Signed-off by: Daniel P. Smith <dpsmith@apertussolutions.com> > > > Daniel P. Smith (2): > docs/designs/launch: hyperlaunch design document > docs/designs/launch: hyperlaunch device tree > > .../designs/launch/hyperlaunch-devicetree.rst | 343 +++++++ > docs/designs/launch/hyperlaunch.rst | 900 ++++++++++++++++++ > 2 files changed, 1243 insertions(+) > create mode 100644 docs/designs/launch/hyperlaunch-devicetree.rst > create mode 100644 docs/designs/launch/hyperlaunch.rst > To assist in reading, please find attached rendered copies of the design docs. It should be noted that due to poor rendering by pandoc, we forced the tables to stay as ASCII tables in the patches whereas the attached docs have the tables rendered properly by rst2pdf. V/r Daniel P. Smith
On 16.03.2021 04:56, Daniel P. Smith wrote: > To assist in reading, please find attached rendered copies of the design > docs. It should be noted that due to poor rendering by pandoc, we forced > the tables to stay as ASCII tables in the patches whereas the attached > docs have the tables rendered properly by rst2pdf. In section 3.6 I found "As a result, on x86 the hyperlaunch capability does not rely on nor preclude any specific BIOS boot protocol, i.e legacy BIOS boot or UEFI boot. The only requirement is that the boot loader supports the Multiboot2 (MB2) protocol." I'm afraid the two sentences contradict one another, as UEFI on its own doesn't provide MB2 functionality. It is my understanding that you don't require this anyway - what you need is a way to load modules beyond just Dom0 kernel and an initrd. I'm also struggling to see how you mean to associate the (MB2) modules passed to Xen with the individual functions. I.e. I don't see how it will be ensure that the embedded mb-index is in sync with the order or modules actually supplied. As to modules - iirc there are placement restrictions by at least some boot loaders (below 4Gb). If that's correct, do you have any thoughts towards dealing with the limited space available to hold these modules? I've seem systems with lots of memory, but with as little as just 1Gb below the 4Gb boundary. Jan
On Tue, Mar 30, 2021 at 7:31 AM Jan Beulich <jbeulich@suse.com> wrote: > > On 16.03.2021 04:56, Daniel P. Smith wrote: > > To assist in reading, please find attached rendered copies of the design > > docs. It should be noted that due to poor rendering by pandoc, we forced > > the tables to stay as ASCII tables in the patches whereas the attached > > docs have the tables rendered properly by rst2pdf. > > In section 3.6 I found "As a result, on x86 the hyperlaunch capability does > not rely on nor preclude any specific BIOS boot protocol, i.e legacy BIOS > boot or UEFI boot. The only requirement is that the boot loader supports > the Multiboot2 (MB2) protocol." I'm afraid the two sentences contradict > one another, as UEFI on its own doesn't provide MB2 functionality. It is > my understanding that you don't require this anyway - what you need is a > way to load modules beyond just Dom0 kernel and an initrd. Thanks - we'll amend the doc. Given the already sizeable scope of the project, our current approach for host UEFI is to recommend use of GRUB.efi to load Xen and the initial domains via the multiboot2 method. > I'm also struggling to see how you mean to associate the (MB2) modules > passed to Xen with the individual functions. I.e. I don't see how it will > be ensure that the embedded mb-index is in sync with the order or modules > actually supplied. To make sure I'm following the concern raised here, my understanding is: - the multiboot module list is ordered and stable, so that the order that the bootloader provides the modules in is the same order in which Xen receives them, in the multiboot data structure, so the modules can be referenced by an index into that list - for a given Launch Control Module file (expressed in a Device Tree, as described in our second design document), the provided multiboot modules will need to match, both in type (kernel, ramdisk, microcode, xsm policy) and order - we think that at some point the LCM will end up being dynamically generated by an enlightened bootloader, assembling it from the bootloader config file, which will list the modules and their types > As to modules - iirc there are placement restrictions by at least some > boot loaders (below 4Gb). If that's correct, do you have any thoughts > towards dealing with the limited space available to hold these modules? > I've seem systems with lots of memory, but with as little as just 1Gb > below the 4Gb boundary. At this point, I don't think that we'll break anything that currently works, and that hardware or platform limitations can always constrain what is deployable. Christopher
On 07.04.2021 21:23, Christopher Clark wrote: > On Tue, Mar 30, 2021 at 7:31 AM Jan Beulich <jbeulich@suse.com> wrote: >> >> On 16.03.2021 04:56, Daniel P. Smith wrote: >>> To assist in reading, please find attached rendered copies of the design >>> docs. It should be noted that due to poor rendering by pandoc, we forced >>> the tables to stay as ASCII tables in the patches whereas the attached >>> docs have the tables rendered properly by rst2pdf. >> >> In section 3.6 I found "As a result, on x86 the hyperlaunch capability does >> not rely on nor preclude any specific BIOS boot protocol, i.e legacy BIOS >> boot or UEFI boot. The only requirement is that the boot loader supports >> the Multiboot2 (MB2) protocol." I'm afraid the two sentences contradict >> one another, as UEFI on its own doesn't provide MB2 functionality. It is >> my understanding that you don't require this anyway - what you need is a >> way to load modules beyond just Dom0 kernel and an initrd. > > Thanks - we'll amend the doc. Given the already sizeable scope of the > project, our current approach for host UEFI is to recommend use of > GRUB.efi to load Xen and the initial domains via the multiboot2 method. > >> I'm also struggling to see how you mean to associate the (MB2) modules >> passed to Xen with the individual functions. I.e. I don't see how it will >> be ensure that the embedded mb-index is in sync with the order or modules >> actually supplied. > > To make sure I'm following the concern raised here, my understanding is: > > - the multiboot module list is ordered and stable, so that the order > that the bootloader provides the modules in is the same order in which > Xen receives them, in the multiboot data structure, so the modules can > be referenced by an index into that list In a separate context (parallel ongoing discussion under "multiboot2 and module2 boot issues via GRUB2") Andrew raised the (imo valid) point of this getting the more fragile the more modules there are. > - for a given Launch Control Module file (expressed in a Device Tree, as > described in our second design document), the provided multiboot > modules will need to match, both in type (kernel, ramdisk, microcode, > xsm policy) and order "Need to match" meaning what? You don't clarify how boot loader config and device tree blob are meant to be kept in sync. > - we think that at some point the LCM will end up being dynamically > generated by an enlightened bootloader, assembling it from the > bootloader config file, which will list the modules and their types > >> As to modules - iirc there are placement restrictions by at least some >> boot loaders (below 4Gb). If that's correct, do you have any thoughts >> towards dealing with the limited space available to hold these modules? >> I've seem systems with lots of memory, but with as little as just 1Gb >> below the 4Gb boundary. > > At this point, I don't think that we'll break anything that currently > works, and that hardware or platform limitations can always constrain > what is deployable. I'm not concerned of you breaking anything. I merely raised a possible scalability issue of your current proposal. Jan
On Wed, Apr 7, 2021 at 10:56 PM Jan Beulich <jbeulich@suse.com> wrote: > > On 07.04.2021 21:23, Christopher Clark wrote: > > On Tue, Mar 30, 2021 at 7:31 AM Jan Beulich <jbeulich@suse.com> wrote: > >> > >> On 16.03.2021 04:56, Daniel P. Smith wrote: > >>> To assist in reading, please find attached rendered copies of the design > >>> docs. It should be noted that due to poor rendering by pandoc, we forced > >>> the tables to stay as ASCII tables in the patches whereas the attached > >>> docs have the tables rendered properly by rst2pdf. > >> > >> In section 3.6 I found "As a result, on x86 the hyperlaunch capability does > >> not rely on nor preclude any specific BIOS boot protocol, i.e legacy BIOS > >> boot or UEFI boot. The only requirement is that the boot loader supports > >> the Multiboot2 (MB2) protocol." I'm afraid the two sentences contradict > >> one another, as UEFI on its own doesn't provide MB2 functionality. It is > >> my understanding that you don't require this anyway - what you need is a > >> way to load modules beyond just Dom0 kernel and an initrd. > > > > Thanks - we'll amend the doc. Given the already sizeable scope of the > > project, our current approach for host UEFI is to recommend use of > > GRUB.efi to load Xen and the initial domains via the multiboot2 method. > > > >> I'm also struggling to see how you mean to associate the (MB2) modules > >> passed to Xen with the individual functions. I.e. I don't see how it will > >> be ensure that the embedded mb-index is in sync with the order or modules > >> actually supplied. > > > > To make sure I'm following the concern raised here, my understanding is: > > > > - the multiboot module list is ordered and stable, so that the order > > that the bootloader provides the modules in is the same order in which > > Xen receives them, in the multiboot data structure, so the modules can > > be referenced by an index into that list > > In a separate context (parallel ongoing discussion under "multiboot2 > and module2 boot issues via GRUB2") Andrew raised the (imo valid) > point of this getting the more fragile the more modules there are. It is a separate context though: ie. your (controversial!) proposal to use the Xen command line to direct the hypervisor to concatenate multiboot modules to form a single initrd before assembling a domain. Boot via Multiboot2 relies on the positional correctness (ie. ordering) of the modules supplied. Andrew's first message in the thread describes Xen's longstanding convention for handling the multiboot modules that it receives: Xen is capable of taking various things via modules, such as an XSM/Flask policy, or microcode, so has logic to identify these if present and separate them from "other stuff". However, there is a hardcoded expectation that the first module is the dom0 kernel, and the next unrecognised module, if present, is *the* initrd. We are proposing to make that hardcoded expectation become configurable via the definition provided in the Launch Control Module. The modules supplied will have to be correctly ordered, matching that definition. Xen has behaviour today for when the boot materials are incorrect (eg. when dom0 initrd is supplied before the dom0 kernel) and that is reasonable, and the same can apply to misconfigured systems in future too. > > - for a given Launch Control Module file (expressed in a Device Tree, as > > described in our second design document), the provided multiboot > > modules will need to match, both in type (kernel, ramdisk, microcode, > > xsm policy) and order > > "Need to match" meaning what? You don't clarify how boot loader > config and device tree blob are meant to be kept in sync. A given Launch Control Module (or device tree blob) defines the expectation for boot module ordering. The boot loader config needs to be written with an awareness of that ordering. Tooling to support that synchronization can be provided - as noted here: > > - we think that at some point the LCM will end up being dynamically > > generated by an enlightened bootloader, assembling it from the > > bootloader config file, which will list the modules and their types which should help address usability concerns. > >> As to modules - iirc there are placement restrictions by at least some > >> boot loaders (below 4Gb). If that's correct, do you have any thoughts > >> towards dealing with the limited space available to hold these modules? > >> I've seem systems with lots of memory, but with as little as just 1Gb > >> below the 4Gb boundary. > > > > At this point, I don't think that we'll break anything that currently > > works, and that hardware or platform limitations can always constrain > > what is deployable. > > I'm not concerned of you breaking anything. I merely raised a possible > scalability issue of your current proposal. Fair. Thanks for the review. Christopher
We are submitting for inclusion in the Xen documentation: - the hyperlaunch design document, and - the hyperlaunch device tree design document to describe a new method for launching the Xen hypervisor. The hyperlaunch feature builds upon prior dom0less work and the DomB project to bring a flexible and security-minded means to launch a variety of VM configurations as part of the startup of Xen. Signed-off-by: Christopher Clark <christopher.clark@starlab.io> Signed-off by: Daniel P. Smith <dpsmith@apertussolutions.com> Daniel P. Smith (2): docs/designs/launch: hyperlaunch design document docs/designs/launch: hyperlaunch device tree .../designs/launch/hyperlaunch-devicetree.rst | 343 +++++++ docs/designs/launch/hyperlaunch.rst | 900 ++++++++++++++++++ 2 files changed, 1243 insertions(+) create mode 100644 docs/designs/launch/hyperlaunch-devicetree.rst create mode 100644 docs/designs/launch/hyperlaunch.rst