| Message ID | 1502359687-25370-1-git-send-email-tianyu.lan@intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, Aug 10, 2017 at 06:08:07PM +0800, Lan Tianyu wrote: > Intel Xeon phi chip will support 352 logical threads. For HPC > usage case, it will create a huge VM with vcpus number as same as host > cpus. This patch is to increase max vcpu number to 352. If we pick arbitray limits based on size of physical CPUs that happen to be shipping today, we'll continue the cat+mouse game forever trailing latest CPUs that vendors ship. IMHO we should pick a higher number influenced by technical constraints of the q35 impl instead. eg can we go straight to something like 512 or 1024 ? > Signed-off-by: Lan Tianyu <tianyu.lan@intel.com> > --- > hw/i386/pc_q35.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c > index 169a214..5e93749 100644 > --- a/hw/i386/pc_q35.c > +++ b/hw/i386/pc_q35.c > @@ -299,7 +299,7 @@ static void pc_q35_machine_options(MachineClass *m) > m->default_display = "std"; > m->no_floppy = 1; > m->has_dynamic_sysbus = true; > - m->max_cpus = 288; > + m->max_cpus = 352; > } You'll need to introduce machine type back compat support so that we avoid changing the 2.10 q35 machine type - only the 2.11 machine type should have the new limit. Regards, Daniel
On 2017年08月10日 18:26, Daniel P. Berrange wrote: > On Thu, Aug 10, 2017 at 06:08:07PM +0800, Lan Tianyu wrote: >> Intel Xeon phi chip will support 352 logical threads. For HPC >> usage case, it will create a huge VM with vcpus number as same as host >> cpus. This patch is to increase max vcpu number to 352. > > If we pick arbitray limits based on size of physical CPUs that happen > to be shipping today, we'll continue the cat+mouse game forever trailing > latest CPUs that vendors ship. > > IMHO we should pick a higher number influenced by technical constraints > of the q35 impl instead. eg can we go straight to something like 512 or > 1024 ? Sure. 512 should be enough and some arrays is defined according to max vcpu number. > >> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com> >> --- >> hw/i386/pc_q35.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c >> index 169a214..5e93749 100644 >> --- a/hw/i386/pc_q35.c >> +++ b/hw/i386/pc_q35.c >> @@ -299,7 +299,7 @@ static void pc_q35_machine_options(MachineClass *m) >> m->default_display = "std"; >> m->no_floppy = 1; >> m->has_dynamic_sysbus = true; >> - m->max_cpus = 288; >> + m->max_cpus = 352; >> } > > You'll need to introduce machine type back compat support so that we > avoid changing the 2.10 q35 machine type - only the 2.11 machine > type should have the new limit. How about the following change? -static void pc_q35_2_10_machine_options(MachineClass *m) +static void pc_q35_2_11_machine_options(MachineClass *m) { pc_q35_machine_options(m); m->alias = "q35"; +} + +static void pc_q35_2_10_machine_options(MachineClass *m) +{ + pc_q35_2_11_options(m); + m->alias = "q35"; + m->max_cpus = 288; m->numa_auto_assign_ram = numa_legacy_auto_assign_ram; } > > Regards, > Daniel >
On 10/08/2017 12:08, Lan Tianyu wrote: > Intel Xeon phi chip will support 352 logical threads. For HPC > usage case, it will create a huge VM with vcpus number as same as host > cpus. This patch is to increase max vcpu number to 352. Even without Xeon Phi, a 24-core, 8-socket system (E7-8890 v4 for example) will have 384 physical CPUs. 288 was the biggest we could test at the time, but now that the 256 vCPU barrier has been broken I agree with going straight to 512. Paolo
2017-08-10 19:02+0800, Lan Tianyu: > On 2017年08月10日 18:26, Daniel P. Berrange wrote: >> On Thu, Aug 10, 2017 at 06:08:07PM +0800, Lan Tianyu wrote: >>> Intel Xeon phi chip will support 352 logical threads. For HPC >>> usage case, it will create a huge VM with vcpus number as same as host >>> cpus. This patch is to increase max vcpu number to 352. >> >> If we pick arbitray limits based on size of physical CPUs that happen >> to be shipping today, we'll continue the cat+mouse game forever trailing >> latest CPUs that vendors ship. >> >> IMHO we should pick a higher number influenced by technical constraints >> of the q35 impl instead. eg can we go straight to something like 512 or >> 1024 ? > > Sure. 512 should be enough and some arrays is defined according to max > vcpu number. Hm, which arrays are that? I was thinking it is safe to bump it to INT_MAX as the number is only used when setting global max_cpus. Thanks.
On Thu, Aug 10, 2017 at 02:13:30PM +0200, Paolo Bonzini wrote: > On 10/08/2017 12:08, Lan Tianyu wrote: > > Intel Xeon phi chip will support 352 logical threads. For HPC > > usage case, it will create a huge VM with vcpus number as same as host > > cpus. This patch is to increase max vcpu number to 352. > > Even without Xeon Phi, a 24-core, 8-socket system (E7-8890 v4 for > example) will have 384 physical CPUs. 288 was the biggest we could test > at the time, but now that the 256 vCPU barrier has been broken I agree > with going straight to 512. FWIW, linux allows 8192 CPUs on x86_64 already, due to alledged existance of SGI machines with 6096 CPUs.... commit b53b5eda8194214928c8243d711a75dbf51809fc Author: Josh Boyer <jwboyer@redhat.com> Date: Tue Nov 5 09:38:16 2013 -0500 x86/cpu: Increase max CPU count to 8192 The MAXSMP option is intended to enable silly large numbers of CPUs for testing purposes. The current value of 4096 isn't very silly any longer as there are actual SGI machines that approach 6096 CPUs when taking HT into account. Increase the value to a nice round 8192 to account for this and allow for short term future increases. Regards, Daniel
On Thu, Aug 10, 2017 at 02:41:03PM +0200, Radim Krčmář wrote: > 2017-08-10 19:02+0800, Lan Tianyu: > > On 2017年08月10日 18:26, Daniel P. Berrange wrote: > >> On Thu, Aug 10, 2017 at 06:08:07PM +0800, Lan Tianyu wrote: > >>> Intel Xeon phi chip will support 352 logical threads. For HPC > >>> usage case, it will create a huge VM with vcpus number as same as host > >>> cpus. This patch is to increase max vcpu number to 352. > >> > >> If we pick arbitray limits based on size of physical CPUs that happen > >> to be shipping today, we'll continue the cat+mouse game forever trailing > >> latest CPUs that vendors ship. > >> > >> IMHO we should pick a higher number influenced by technical constraints > >> of the q35 impl instead. eg can we go straight to something like 512 or > >> 1024 ? > > > > Sure. 512 should be enough and some arrays is defined according to max > > vcpu number. > > Hm, which arrays are that? I was thinking it is safe to bump it to > INT_MAX as the number is only used when setting global max_cpus. We had a MAX_CPUMASK_BITS macro, and bitmaps whose sizes were defined at compile time based on it. But commit cdda2018e3b9ce0c18938767dfdb1e05a05b67ca removed it. Probably those arrays all use max_cpus, by now (and the default for max_cpus is smp_cpus, not MachineClass::max_cpus). Anyway, if we set it to INT_MAX, there are some cases where more appropriate error checking/reporting could be required because they won't handle overflow very well: * pcms->apic_id_limit initialization at pc_cpus_init() * ACPI code that assumes possible_cpus->cpus[i].arch_id fits in a 32-bit integer * Other x86_cpu_apic_id_from_index() calls in PC code (especially the initialization of possible_cpus->cpus[i].arch_id). Note that x86_cpu_apic_id_from_index(cpu_index) might not fit in 32 bits even if cpu_index <= UINT32_MAX.
2017-08-10 15:16-0300, Eduardo Habkost: > On Thu, Aug 10, 2017 at 02:41:03PM +0200, Radim Krčmář wrote: > > 2017-08-10 19:02+0800, Lan Tianyu: > > > On 2017年08月10日 18:26, Daniel P. Berrange wrote: > > >> On Thu, Aug 10, 2017 at 06:08:07PM +0800, Lan Tianyu wrote: > > >>> Intel Xeon phi chip will support 352 logical threads. For HPC > > >>> usage case, it will create a huge VM with vcpus number as same as host > > >>> cpus. This patch is to increase max vcpu number to 352. > > >> > > >> If we pick arbitray limits based on size of physical CPUs that happen > > >> to be shipping today, we'll continue the cat+mouse game forever trailing > > >> latest CPUs that vendors ship. > > >> > > >> IMHO we should pick a higher number influenced by technical constraints > > >> of the q35 impl instead. eg can we go straight to something like 512 or > > >> 1024 ? > > > > > > Sure. 512 should be enough and some arrays is defined according to max > > > vcpu number. > > > > Hm, which arrays are that? I was thinking it is safe to bump it to > > INT_MAX as the number is only used when setting global max_cpus. > > We had a MAX_CPUMASK_BITS macro, and bitmaps whose sizes were > defined at compile time based on it. But commit > cdda2018e3b9ce0c18938767dfdb1e05a05b67ca removed it. Probably > those arrays all use max_cpus, by now (and the default for > max_cpus is smp_cpus, not MachineClass::max_cpus). Ah, thanks. > Anyway, if we set it to INT_MAX, there are some cases where more > appropriate error checking/reporting could be required because > they won't handle overflow very well: > * pcms->apic_id_limit initialization at pc_cpus_init() > * ACPI code that assumes possible_cpus->cpus[i].arch_id fits > in a 32-bit integer > * Other x86_cpu_apic_id_from_index() calls in PC code > (especially the initialization of possible_cpus->cpus[i].arch_id). > Note that x86_cpu_apic_id_from_index(cpu_index) might not fit > in 32 bits even if cpu_index <= UINT32_MAX. Good point, looks like it all comes to x86_cpu_apic_id_from_index(). Each level of the topology has at most one underutilized bit, so 2^(32 - 3) would be safe. It is still needlessly large for the foreseeable future, but 512 is going to be surpassed pretty soon, so I think that jumping at least to 8k would be better. (8k the current default maximum for Linux and the resulting overcommit of ~20 is bearable for smoke testing on current hardware.)
On 2017年08月11日 03:22, Radim Krčmář wrote: > 2017-08-10 15:16-0300, Eduardo Habkost: >> On Thu, Aug 10, 2017 at 02:41:03PM +0200, Radim Krčmář wrote: >>> 2017-08-10 19:02+0800, Lan Tianyu: >>>> On 2017年08月10日 18:26, Daniel P. Berrange wrote: >>>>> On Thu, Aug 10, 2017 at 06:08:07PM +0800, Lan Tianyu wrote: >>>>>> Intel Xeon phi chip will support 352 logical threads. For HPC >>>>>> usage case, it will create a huge VM with vcpus number as same as host >>>>>> cpus. This patch is to increase max vcpu number to 352. >>>>> >>>>> If we pick arbitray limits based on size of physical CPUs that happen >>>>> to be shipping today, we'll continue the cat+mouse game forever trailing >>>>> latest CPUs that vendors ship. >>>>> >>>>> IMHO we should pick a higher number influenced by technical constraints >>>>> of the q35 impl instead. eg can we go straight to something like 512 or >>>>> 1024 ? >>>> >>>> Sure. 512 should be enough and some arrays is defined according to max >>>> vcpu number. >>> >>> Hm, which arrays are that? I was thinking it is safe to bump it to >>> INT_MAX as the number is only used when setting global max_cpus. >> >> We had a MAX_CPUMASK_BITS macro, and bitmaps whose sizes were >> defined at compile time based on it. But commit >> cdda2018e3b9ce0c18938767dfdb1e05a05b67ca removed it. Probably >> those arrays all use max_cpus, by now (and the default for >> max_cpus is smp_cpus, not MachineClass::max_cpus). > > Ah, thanks. > >> Anyway, if we set it to INT_MAX, there are some cases where more >> appropriate error checking/reporting could be required because >> they won't handle overflow very well: >> * pcms->apic_id_limit initialization at pc_cpus_init() >> * ACPI code that assumes possible_cpus->cpus[i].arch_id fits >> in a 32-bit integer >> * Other x86_cpu_apic_id_from_index() calls in PC code >> (especially the initialization of possible_cpus->cpus[i].arch_id). >> Note that x86_cpu_apic_id_from_index(cpu_index) might not fit >> in 32 bits even if cpu_index <= UINT32_MAX. > > Good point, looks like it all comes to x86_cpu_apic_id_from_index(). > Each level of the topology has at most one underutilized bit, so > 2^(32 - 3) would be safe. > > It is still needlessly large for the foreseeable future, but 512 is > going to be surpassed pretty soon, so I think that jumping at least to > 8k would be better. > (8k the current default maximum for Linux and the resulting overcommit > of ~20 is bearable for smoke testing on current hardware.) > Hi All: Thanks for your input. I tried Qemu with 8192 as max_vcpu and it works normally. I will update patches.
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c index 169a214..5e93749 100644 --- a/hw/i386/pc_q35.c +++ b/hw/i386/pc_q35.c @@ -299,7 +299,7 @@ static void pc_q35_machine_options(MachineClass *m) m->default_display = "std"; m->no_floppy = 1; m->has_dynamic_sysbus = true; - m->max_cpus = 288; + m->max_cpus = 352; } static void pc_q35_2_10_machine_options(MachineClass *m)
Intel Xeon phi chip will support 352 logical threads. For HPC usage case, it will create a huge VM with vcpus number as same as host cpus. This patch is to increase max vcpu number to 352. Signed-off-by: Lan Tianyu <tianyu.lan@intel.com> --- hw/i386/pc_q35.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)