| Message ID | 20221024035416.34068-7-gshan@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | hw/arm/virt: Improve address assignment for high memory regions | expand |
On Mon, Oct 24 2022, Gavin Shan <gshan@redhat.com> wrote: > After the improvement to high memory region address assignment is > applied, the memory layout can be changed, introducing possible > migration breakage. For example, VIRT_HIGH_PCIE_MMIO memory region > is disabled or enabled when the optimization is applied or not, with > the following configuration. The configuration is only achievable by > modifying the source code until more properties are added to allow > users selectively disable those high memory regions. > > pa_bits = 40; > vms->highmem_redists = false; > vms->highmem_ecam = false; > vms->highmem_mmio = true; > > # qemu-system-aarch64 -accel kvm -cpu host \ > -machine virt-7.2,compact-highmem={on, off} \ > -m 4G,maxmem=511G -monitor stdio > > Region compact-highmem=off compact-highmem=on > ---------------------------------------------------------------- > MEM [1GB 512GB] [1GB 512GB] > HIGH_GIC_REDISTS2 [512GB 512GB+64MB] [disabled] > HIGH_PCIE_ECAM [512GB+256MB 512GB+512MB] [disabled] > HIGH_PCIE_MMIO [disabled] [512GB 1TB] > > In order to keep backwords compatibility, we need to disable the > optimization on machine, which is virt-7.1 or ealier than it. It > means the optimization is enabled by default from virt-7.2. Besides, > 'compact-highmem' property is added so that the optimization can be > explicitly enabled or disabled on all machine types by users. > > Signed-off-by: Gavin Shan <gshan@redhat.com> > Reviewed-by: Cornelia Huck <cohuck@redhat.com> > Tested-by: Zhenyu Zhang <zhenyzha@redhat.com> > --- > docs/system/arm/virt.rst | 4 ++++ > hw/arm/virt.c | 32 ++++++++++++++++++++++++++++++++ > include/hw/arm/virt.h | 1 + > 3 files changed, 37 insertions(+) > (...) > diff --git a/hw/arm/virt.c b/hw/arm/virt.c > index 4896f600b4..11b5685432 100644 > --- a/hw/arm/virt.c > +++ b/hw/arm/virt.c > @@ -174,6 +174,12 @@ static const MemMapEntry base_memmap[] = { > * Note the extended_memmap is sized so that it eventually also includes the > * base_memmap entries (VIRT_HIGH_GIC_REDIST2 index is greater than the last > * index of base_memmap). > + * > + * The memory map for these Highmem IO Regions can be in legacy or compact > + * layout, depending on 'compact-highmem' property. With legacy layout, the > + * PA space for one specific region is always reserved, even the region has s/even/even if/ > + * been disabled or doesn't fit into the PA space. However, the PA space for > + * the region won't be reserved in these circumstances with compact layout. > */ > static MemMapEntry extended_memmap[] = { > /* Additional 64 MB redist region (can contain up to 512 redistributors) */
On 10/25/22 12:30, Cornelia Huck wrote: > On Mon, Oct 24 2022, Gavin Shan <gshan@redhat.com> wrote: > >> After the improvement to high memory region address assignment is >> applied, the memory layout can be changed, introducing possible >> migration breakage. For example, VIRT_HIGH_PCIE_MMIO memory region >> is disabled or enabled when the optimization is applied or not, with >> the following configuration. The configuration is only achievable by >> modifying the source code until more properties are added to allow >> users selectively disable those high memory regions. >> >> pa_bits = 40; >> vms->highmem_redists = false; >> vms->highmem_ecam = false; >> vms->highmem_mmio = true; >> >> # qemu-system-aarch64 -accel kvm -cpu host \ >> -machine virt-7.2,compact-highmem={on, off} \ >> -m 4G,maxmem=511G -monitor stdio >> >> Region compact-highmem=off compact-highmem=on >> ---------------------------------------------------------------- >> MEM [1GB 512GB] [1GB 512GB] >> HIGH_GIC_REDISTS2 [512GB 512GB+64MB] [disabled] >> HIGH_PCIE_ECAM [512GB+256MB 512GB+512MB] [disabled] >> HIGH_PCIE_MMIO [disabled] [512GB 1TB] >> >> In order to keep backwords compatibility, we need to disable the >> optimization on machine, which is virt-7.1 or ealier than it. It >> means the optimization is enabled by default from virt-7.2. Besides, >> 'compact-highmem' property is added so that the optimization can be >> explicitly enabled or disabled on all machine types by users. >> >> Signed-off-by: Gavin Shan <gshan@redhat.com> >> Reviewed-by: Cornelia Huck <cohuck@redhat.com> >> Tested-by: Zhenyu Zhang <zhenyzha@redhat.com> >> --- >> docs/system/arm/virt.rst | 4 ++++ >> hw/arm/virt.c | 32 ++++++++++++++++++++++++++++++++ >> include/hw/arm/virt.h | 1 + >> 3 files changed, 37 insertions(+) >> > (...) > >> diff --git a/hw/arm/virt.c b/hw/arm/virt.c >> index 4896f600b4..11b5685432 100644 >> --- a/hw/arm/virt.c >> +++ b/hw/arm/virt.c >> @@ -174,6 +174,12 @@ static const MemMapEntry base_memmap[] = { >> * Note the extended_memmap is sized so that it eventually also includes the >> * base_memmap entries (VIRT_HIGH_GIC_REDIST2 index is greater than the last >> * index of base_memmap). >> + * >> + * The memory map for these Highmem IO Regions can be in legacy or compact >> + * layout, depending on 'compact-highmem' property. With legacy layout, the >> + * PA space for one specific region is always reserved, even the region has > s/even/even if/ Reviewed-by: Eric Auger <eric.auger@redhat.com> Thanks Eric > >> + * been disabled or doesn't fit into the PA space. However, the PA space for >> + * the region won't be reserved in these circumstances with compact layout. >> */ >> static MemMapEntry extended_memmap[] = { >> /* Additional 64 MB redist region (can contain up to 512 redistributors) */
Hi Connie, On 10/25/22 6:30 PM, Cornelia Huck wrote: > On Mon, Oct 24 2022, Gavin Shan <gshan@redhat.com> wrote: > >> After the improvement to high memory region address assignment is >> applied, the memory layout can be changed, introducing possible >> migration breakage. For example, VIRT_HIGH_PCIE_MMIO memory region >> is disabled or enabled when the optimization is applied or not, with >> the following configuration. The configuration is only achievable by >> modifying the source code until more properties are added to allow >> users selectively disable those high memory regions. >> >> pa_bits = 40; >> vms->highmem_redists = false; >> vms->highmem_ecam = false; >> vms->highmem_mmio = true; >> >> # qemu-system-aarch64 -accel kvm -cpu host \ >> -machine virt-7.2,compact-highmem={on, off} \ >> -m 4G,maxmem=511G -monitor stdio >> >> Region compact-highmem=off compact-highmem=on >> ---------------------------------------------------------------- >> MEM [1GB 512GB] [1GB 512GB] >> HIGH_GIC_REDISTS2 [512GB 512GB+64MB] [disabled] >> HIGH_PCIE_ECAM [512GB+256MB 512GB+512MB] [disabled] >> HIGH_PCIE_MMIO [disabled] [512GB 1TB] >> >> In order to keep backwords compatibility, we need to disable the >> optimization on machine, which is virt-7.1 or ealier than it. It >> means the optimization is enabled by default from virt-7.2. Besides, >> 'compact-highmem' property is added so that the optimization can be >> explicitly enabled or disabled on all machine types by users. >> >> Signed-off-by: Gavin Shan <gshan@redhat.com> >> Reviewed-by: Cornelia Huck <cohuck@redhat.com> >> Tested-by: Zhenyu Zhang <zhenyzha@redhat.com> >> --- >> docs/system/arm/virt.rst | 4 ++++ >> hw/arm/virt.c | 32 ++++++++++++++++++++++++++++++++ >> include/hw/arm/virt.h | 1 + >> 3 files changed, 37 insertions(+) >> > > (...) > >> diff --git a/hw/arm/virt.c b/hw/arm/virt.c >> index 4896f600b4..11b5685432 100644 >> --- a/hw/arm/virt.c >> +++ b/hw/arm/virt.c >> @@ -174,6 +174,12 @@ static const MemMapEntry base_memmap[] = { >> * Note the extended_memmap is sized so that it eventually also includes the >> * base_memmap entries (VIRT_HIGH_GIC_REDIST2 index is greater than the last >> * index of base_memmap). >> + * >> + * The memory map for these Highmem IO Regions can be in legacy or compact >> + * layout, depending on 'compact-highmem' property. With legacy layout, the >> + * PA space for one specific region is always reserved, even the region has > > s/even/even if/ > Thanks, it will be improved as suggested in next respin (v7). >> + * been disabled or doesn't fit into the PA space. However, the PA space for >> + * the region won't be reserved in these circumstances with compact layout. >> */ >> static MemMapEntry extended_memmap[] = { >> /* Additional 64 MB redist region (can contain up to 512 redistributors) */ > Thanks, Gavin
diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst index 20442ea2c1..4454706392 100644 --- a/docs/system/arm/virt.rst +++ b/docs/system/arm/virt.rst @@ -94,6 +94,10 @@ highmem address space above 32 bits. The default is ``on`` for machine types later than ``virt-2.12``. +compact-highmem + Set ``on``/``off`` to enable/disable the compact layout for high memory regions. + The default is ``on`` for machine types later than ``virt-7.2``. + gic-version Specify the version of the Generic Interrupt Controller (GIC) to provide. Valid values are: diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 4896f600b4..11b5685432 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -174,6 +174,12 @@ static const MemMapEntry base_memmap[] = { * Note the extended_memmap is sized so that it eventually also includes the * base_memmap entries (VIRT_HIGH_GIC_REDIST2 index is greater than the last * index of base_memmap). + * + * The memory map for these Highmem IO Regions can be in legacy or compact + * layout, depending on 'compact-highmem' property. With legacy layout, the + * PA space for one specific region is always reserved, even the region has + * been disabled or doesn't fit into the PA space. However, the PA space for + * the region won't be reserved in these circumstances with compact layout. */ static MemMapEntry extended_memmap[] = { /* Additional 64 MB redist region (can contain up to 512 redistributors) */ @@ -2351,6 +2357,20 @@ static void virt_set_highmem(Object *obj, bool value, Error **errp) vms->highmem = value; } +static bool virt_get_compact_highmem(Object *obj, Error **errp) +{ + VirtMachineState *vms = VIRT_MACHINE(obj); + + return vms->highmem_compact; +} + +static void virt_set_compact_highmem(Object *obj, bool value, Error **errp) +{ + VirtMachineState *vms = VIRT_MACHINE(obj); + + vms->highmem_compact = value; +} + static bool virt_get_its(Object *obj, Error **errp) { VirtMachineState *vms = VIRT_MACHINE(obj); @@ -2969,6 +2989,13 @@ static void virt_machine_class_init(ObjectClass *oc, void *data) "Set on/off to enable/disable using " "physical address space above 32 bits"); + object_class_property_add_bool(oc, "compact-highmem", + virt_get_compact_highmem, + virt_set_compact_highmem); + object_class_property_set_description(oc, "compact-highmem", + "Set on/off to enable/disable compact " + "layout for high memory regions"); + object_class_property_add_str(oc, "gic-version", virt_get_gic_version, virt_set_gic_version); object_class_property_set_description(oc, "gic-version", @@ -3053,6 +3080,7 @@ static void virt_instance_init(Object *obj) /* High memory is enabled by default */ vms->highmem = true; + vms->highmem_compact = !vmc->no_highmem_compact; vms->gic_version = VIRT_GIC_VERSION_NOSEL; vms->highmem_ecam = !vmc->no_highmem_ecam; @@ -3122,8 +3150,12 @@ DEFINE_VIRT_MACHINE_AS_LATEST(7, 2) static void virt_machine_7_1_options(MachineClass *mc) { + VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc)); + virt_machine_7_2_options(mc); compat_props_add(mc->compat_props, hw_compat_7_1, hw_compat_7_1_len); + /* Compact layout for high memory regions was introduced with 7.2 */ + vmc->no_highmem_compact = true; } DEFINE_VIRT_MACHINE(7, 1) diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h index 709f623741..c7dd59d7f1 100644 --- a/include/hw/arm/virt.h +++ b/include/hw/arm/virt.h @@ -125,6 +125,7 @@ struct VirtMachineClass { bool no_pmu; bool claim_edge_triggered_timers; bool smbios_old_sys_ver; + bool no_highmem_compact; bool no_highmem_ecam; bool no_ged; /* Machines < 4.2 have no support for ACPI GED device */ bool kvm_no_adjvtime;
After the improvement to high memory region address assignment is applied, the memory layout can be changed, introducing possible migration breakage. For example, VIRT_HIGH_PCIE_MMIO memory region is disabled or enabled when the optimization is applied or not, with the following configuration. The configuration is only achievable by modifying the source code until more properties are added to allow users selectively disable those high memory regions. pa_bits = 40; vms->highmem_redists = false; vms->highmem_ecam = false; vms->highmem_mmio = true; # qemu-system-aarch64 -accel kvm -cpu host \ -machine virt-7.2,compact-highmem={on, off} \ -m 4G,maxmem=511G -monitor stdio Region compact-highmem=off compact-highmem=on ---------------------------------------------------------------- MEM [1GB 512GB] [1GB 512GB] HIGH_GIC_REDISTS2 [512GB 512GB+64MB] [disabled] HIGH_PCIE_ECAM [512GB+256MB 512GB+512MB] [disabled] HIGH_PCIE_MMIO [disabled] [512GB 1TB] In order to keep backwords compatibility, we need to disable the optimization on machine, which is virt-7.1 or ealier than it. It means the optimization is enabled by default from virt-7.2. Besides, 'compact-highmem' property is added so that the optimization can be explicitly enabled or disabled on all machine types by users. Signed-off-by: Gavin Shan <gshan@redhat.com> Reviewed-by: Cornelia Huck <cohuck@redhat.com> Tested-by: Zhenyu Zhang <zhenyzha@redhat.com> --- docs/system/arm/virt.rst | 4 ++++ hw/arm/virt.c | 32 ++++++++++++++++++++++++++++++++ include/hw/arm/virt.h | 1 + 3 files changed, 37 insertions(+)