| Message ID | 20220411233549.740157-3-song@kernel.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | vmalloc: bpf: introduce VM_ALLOW_HUGE_VMAP | expand |
On Mon, Apr 11, 2022 at 04:35:47PM -0700, Song Liu wrote: > Introduce module_alloc_huge, which allocates huge page backed memory in > module memory space. The primary user of this memory is bpf_prog_pack > (multiple BPF programs sharing a huge page). I kow I lead you downthis road first, but I wonder if we just want to pass a flag to module_alloc instead. This avoids duplicating all the arch overrides.
On Mon, Apr 11, 2022 at 9:20 PM Christoph Hellwig <hch@infradead.org> wrote: > > On Mon, Apr 11, 2022 at 04:35:47PM -0700, Song Liu wrote: > > Introduce module_alloc_huge, which allocates huge page backed memory in > > module memory space. The primary user of this memory is bpf_prog_pack > > (multiple BPF programs sharing a huge page). > > I kow I lead you downthis road first, but I wonder if we just want to > pass a flag to module_alloc instead. This avoids duplicating all the > arch overrides. I don't think we will see many archs support bpf_prog_pack, so the __weak version might be good enough for a long time. Adding an argument to module_alloc seems like more trouble to me. Thanks, Song
diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index b98ffcf4d250..63f6a16c70dc 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -86,6 +86,27 @@ void *module_alloc(unsigned long size) return p; } +void *module_alloc_huge(unsigned long size) +{ + gfp_t gfp_mask = GFP_KERNEL; + void *p; + + if (PAGE_ALIGN(size) > MODULES_LEN) + return NULL; + + p = __vmalloc_node_range(size, MODULE_ALIGN, + MODULES_VADDR + get_module_load_offset(), + MODULES_END, gfp_mask, PAGE_KERNEL, + VM_DEFER_KMEMLEAK | VM_ALLOW_HUGE_VMAP, + NUMA_NO_NODE, __builtin_return_address(0)); + if (p && (kasan_alloc_module_shadow(p, size, gfp_mask) < 0)) { + vfree(p); + return NULL; + } + + return p; +} + #ifdef CONFIG_X86_32 int apply_relocate(Elf32_Shdr *sechdrs, const char *strtab, diff --git a/include/linux/moduleloader.h b/include/linux/moduleloader.h index 9e09d11ffe5b..d34743a88938 100644 --- a/include/linux/moduleloader.h +++ b/include/linux/moduleloader.h @@ -26,6 +26,11 @@ unsigned int arch_mod_section_prepend(struct module *mod, unsigned int section); sections. Returns NULL on failure. */ void *module_alloc(unsigned long size); +/* Allocator used for allocating memory in module memory space. If size is + * greater than PMD_SIZE, allow using huge pages. Returns NULL on failure. + */ +void *module_alloc_huge(unsigned long size); + /* Free memory returned from module_alloc. */ void module_memfree(void *module_region); diff --git a/kernel/module.c b/kernel/module.c index 6cea788fd965..2af20ac3209c 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -2839,6 +2839,14 @@ void * __weak module_alloc(unsigned long size) NUMA_NO_NODE, __builtin_return_address(0)); } +void * __weak module_alloc_huge(unsigned long size) +{ + return __vmalloc_node_range(size, 1, VMALLOC_START, VMALLOC_END, + GFP_KERNEL, PAGE_KERNEL_EXEC, + VM_FLUSH_RESET_PERMS | VM_ALLOW_HUGE_VMAP, + NUMA_NO_NODE, __builtin_return_address(0)); +} + bool __weak module_init_section(const char *name) { return strstarts(name, ".init");
Introduce module_alloc_huge, which allocates huge page backed memory in module memory space. The primary user of this memory is bpf_prog_pack (multiple BPF programs sharing a huge page). Signed-off-by: Song Liu <song@kernel.org> --- arch/x86/kernel/module.c | 21 +++++++++++++++++++++ include/linux/moduleloader.h | 5 +++++ kernel/module.c | 8 ++++++++ 3 files changed, 34 insertions(+)