| Message ID | 1606985867-9791-3-git-send-email-hejinyang@loongson.cn (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | memory node | expand |
于 2020年12月3日 GMT+08:00 下午4:57:47, Jinyang He <hejinyang@loongson.cn> 写到: >This problem may only occur on NUMA platforms. When machine start >with the "mem=" parameter on Loongson64, it cannot boot. When parsing the >"mem=" parameter, first all the RAM was removed, and then the memory was >not added by memblock_add_node(), which caused the newly added memory to >be on MAX_NUMNODES. The key to solve this problem is to fix these memory >nodes through memblock_set_node() before bootmem_init() or earlier. So >it would be better to fix it before check_kernel_sections_mem(). >The check_kernel_sections_mem() will check whether the current RAM can be >used by the kernel. If this fix is added after that, it will do a redundant >memblock_add operation. Adding the fixup_region_node() function can also >provide a reference for future platforms using NUMA when encountering >such problems. Hi Jingyang, Is it possible to do it when parsing cmdline to avoid this kind of fixup? Thanks. - Jiaxun > >Signed-off-by: Jinyang He <hejinyang@loongson.cn> >--- > arch/mips/include/asm/bootinfo.h | 1 + > arch/mips/kernel/setup.c | 6 +++++- > arch/mips/loongson64/numa.c | 11 +++++++++++ > 3 files changed, 17 insertions(+), 1 deletion(-) > >diff --git a/arch/mips/include/asm/bootinfo.h b/arch/mips/include/asm/bootinfo.h >index aa03b12..ddc17b1 100644 >--- a/arch/mips/include/asm/bootinfo.h >+++ b/arch/mips/include/asm/bootinfo.h >@@ -91,6 +91,7 @@ const char *get_system_type(void); > extern unsigned long mips_machtype; > > extern void detect_memory_region(phys_addr_t start, phys_addr_t sz_min, phys_addr_t sz_max); >+extern void fixup_region_node(void); > > extern void prom_init(void); > extern void prom_free_prom_memory(void); >diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c >index b3a711e..fe93882 100644 >--- a/arch/mips/kernel/setup.c >+++ b/arch/mips/kernel/setup.c >@@ -110,6 +110,8 @@ void __init detect_memory_region(phys_addr_t start, phys_addr_t sz_min, phys_add > memblock_add(start, size); > } > >+void __weak fixup_region_node(void) {} >+ > /* > * Manage initrd > */ >@@ -631,8 +633,10 @@ static void __init arch_mem_init(char **cmdline_p) > > parse_early_param(); > >- if (usermem) >+ if (usermem) { > pr_info("User-defined physical RAM map overwrite\n"); >+ fixup_region_node(); >+ } > > check_kernel_sections_mem(); > >diff --git a/arch/mips/loongson64/numa.c b/arch/mips/loongson64/numa.c >index c6f0c48..d8661cc 100644 >--- a/arch/mips/loongson64/numa.c >+++ b/arch/mips/loongson64/numa.c >@@ -220,6 +220,17 @@ void __init mem_init(void) > mem_init_print_info(NULL); > } > >+void __init fixup_region_node(void) >+{ >+ phys_addr_t start, end; >+ u64 i; >+ >+ for_each_mem_range(i, &start, &end) { >+ memblock_set_node(start, end - start, >+ &memblock.memory, pa_to_nid(start)); >+ } >+} >+ > /* All PCI device belongs to logical Node-0 */ > int pcibus_to_node(struct pci_bus *bus) > {
On 12/03/2020 05:27 PM, Jiaxun Yang wrote: > > 于 2020年12月3日 GMT+08:00 下午4:57:47, Jinyang He <hejinyang@loongson.cn> 写到: >> This problem may only occur on NUMA platforms. When machine start >> with the "mem=" parameter on Loongson64, it cannot boot. When parsing the >> "mem=" parameter, first all the RAM was removed, and then the memory was >> not added by memblock_add_node(), which caused the newly added memory to >> be on MAX_NUMNODES. The key to solve this problem is to fix these memory >> nodes through memblock_set_node() before bootmem_init() or earlier. So >> it would be better to fix it before check_kernel_sections_mem(). >> The check_kernel_sections_mem() will check whether the current RAM can be >> used by the kernel. If this fix is added after that, it will do a redundant >> memblock_add operation. Adding the fixup_region_node() function can also >> provide a reference for future platforms using NUMA when encountering >> such problems. > Hi Jingyang, > > Is it possible to do it when parsing cmdline to avoid this kind of fixup? > > Thanks. > > - Jiaxun Hi, Jiaxun, Of course. But "memmap=" could cause same problem , "mem=" could parse many times. So it better to fix it after these parse opertion having done. Thanks, Jinyang. >> Signed-off-by: Jinyang He <hejinyang@loongson.cn> >> --- >> arch/mips/include/asm/bootinfo.h | 1 + >> arch/mips/kernel/setup.c | 6 +++++- >> arch/mips/loongson64/numa.c | 11 +++++++++++ >> 3 files changed, 17 insertions(+), 1 deletion(-) >> >> diff --git a/arch/mips/include/asm/bootinfo.h b/arch/mips/include/asm/bootinfo.h >> index aa03b12..ddc17b1 100644 >> --- a/arch/mips/include/asm/bootinfo.h >> +++ b/arch/mips/include/asm/bootinfo.h >> @@ -91,6 +91,7 @@ const char *get_system_type(void); >> extern unsigned long mips_machtype; >> >> extern void detect_memory_region(phys_addr_t start, phys_addr_t sz_min, phys_addr_t sz_max); >> +extern void fixup_region_node(void); >> >> extern void prom_init(void); >> extern void prom_free_prom_memory(void); >> diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c >> index b3a711e..fe93882 100644 >> --- a/arch/mips/kernel/setup.c >> +++ b/arch/mips/kernel/setup.c >> @@ -110,6 +110,8 @@ void __init detect_memory_region(phys_addr_t start, phys_addr_t sz_min, phys_add >> memblock_add(start, size); >> } >> >> +void __weak fixup_region_node(void) {} >> + >> /* >> * Manage initrd >> */ >> @@ -631,8 +633,10 @@ static void __init arch_mem_init(char **cmdline_p) >> >> parse_early_param(); >> >> - if (usermem) >> + if (usermem) { >> pr_info("User-defined physical RAM map overwrite\n"); >> + fixup_region_node(); >> + } >> >> check_kernel_sections_mem(); >> >> diff --git a/arch/mips/loongson64/numa.c b/arch/mips/loongson64/numa.c >> index c6f0c48..d8661cc 100644 >> --- a/arch/mips/loongson64/numa.c >> +++ b/arch/mips/loongson64/numa.c >> @@ -220,6 +220,17 @@ void __init mem_init(void) >> mem_init_print_info(NULL); >> } >> >> +void __init fixup_region_node(void) >> +{ >> + phys_addr_t start, end; >> + u64 i; >> + >> + for_each_mem_range(i, &start, &end) { >> + memblock_set_node(start, end - start, >> + &memblock.memory, pa_to_nid(start)); >> + } >> +} >> + >> /* All PCI device belongs to logical Node-0 */ >> int pcibus_to_node(struct pci_bus *bus) >> {
diff --git a/arch/mips/include/asm/bootinfo.h b/arch/mips/include/asm/bootinfo.h index aa03b12..ddc17b1 100644 --- a/arch/mips/include/asm/bootinfo.h +++ b/arch/mips/include/asm/bootinfo.h @@ -91,6 +91,7 @@ const char *get_system_type(void); extern unsigned long mips_machtype; extern void detect_memory_region(phys_addr_t start, phys_addr_t sz_min, phys_addr_t sz_max); +extern void fixup_region_node(void); extern void prom_init(void); extern void prom_free_prom_memory(void); diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c index b3a711e..fe93882 100644 --- a/arch/mips/kernel/setup.c +++ b/arch/mips/kernel/setup.c @@ -110,6 +110,8 @@ void __init detect_memory_region(phys_addr_t start, phys_addr_t sz_min, phys_add memblock_add(start, size); } +void __weak fixup_region_node(void) {} + /* * Manage initrd */ @@ -631,8 +633,10 @@ static void __init arch_mem_init(char **cmdline_p) parse_early_param(); - if (usermem) + if (usermem) { pr_info("User-defined physical RAM map overwrite\n"); + fixup_region_node(); + } check_kernel_sections_mem(); diff --git a/arch/mips/loongson64/numa.c b/arch/mips/loongson64/numa.c index c6f0c48..d8661cc 100644 --- a/arch/mips/loongson64/numa.c +++ b/arch/mips/loongson64/numa.c @@ -220,6 +220,17 @@ void __init mem_init(void) mem_init_print_info(NULL); } +void __init fixup_region_node(void) +{ + phys_addr_t start, end; + u64 i; + + for_each_mem_range(i, &start, &end) { + memblock_set_node(start, end - start, + &memblock.memory, pa_to_nid(start)); + } +} + /* All PCI device belongs to logical Node-0 */ int pcibus_to_node(struct pci_bus *bus) {
This problem may only occur on NUMA platforms. When machine start with the "mem=" parameter on Loongson64, it cannot boot. When parsing the "mem=" parameter, first all the RAM was removed, and then the memory was not added by memblock_add_node(), which caused the newly added memory to be on MAX_NUMNODES. The key to solve this problem is to fix these memory nodes through memblock_set_node() before bootmem_init() or earlier. So it would be better to fix it before check_kernel_sections_mem(). The check_kernel_sections_mem() will check whether the current RAM can be used by the kernel. If this fix is added after that, it will do a redundant memblock_add operation. Adding the fixup_region_node() function can also provide a reference for future platforms using NUMA when encountering such problems. Signed-off-by: Jinyang He <hejinyang@loongson.cn> --- arch/mips/include/asm/bootinfo.h | 1 + arch/mips/kernel/setup.c | 6 +++++- arch/mips/loongson64/numa.c | 11 +++++++++++ 3 files changed, 17 insertions(+), 1 deletion(-)