| Message ID | 20220106022533.2950016-6-song@kernel.org (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | BPF |
| Headers | show |
| Series | bpf_prog_pack allocator | expand |
On Wed, Jan 05, 2022 at 06:25:31PM -0800, Song Liu wrote: > diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c > index 23fb4d51a5da..02c35725cc62 100644 > --- a/arch/x86/kernel/alternative.c > +++ b/arch/x86/kernel/alternative.c > @@ -1102,6 +1102,34 @@ void *text_poke_kgdb(void *addr, const void *opcode, size_t len) > return __text_poke(addr, opcode, len); > } > > +/** > + * text_poke_jit - Update instructions on a live kernel by jit engine > + * @addr: address to modify > + * @opcode: source of the copy > + * @len: length to copy, could be more than 2x PAGE_SIZE > + * > + * Only module memory taking jit text (e.g. for bpf) should be patched. > + */ Maybe: text_poke_copy() - Copy instructions into (an unused part of) RX memory @args... Not safe against concurrent execution; useful for JITs to dump new code blocks into unused regions of RX memory. Can be used in conjunction with synchronize_rcu_tasks() to wait for existing execution to quiesce after having made sure no existing functions pointers are life. or something along those lines? > +void *text_poke_jit(void *addr, const void *opcode, size_t len) > +{ > + unsigned long start = (unsigned long)addr; > + size_t patched = 0; > + > + if (WARN_ON_ONCE(core_kernel_text(start))) > + return NULL; > + > + while (patched < len) { > + unsigned long ptr = start + patched; > + size_t s; > + > + s = min_t(size_t, PAGE_SIZE * 2 - offset_in_page(ptr), len - patched); > + > + __text_poke((void *)ptr, opcode + patched, s); > + patched += s; > + } > + return addr; > +} > + > static void do_sync_core(void *info) > { > sync_core(); > -- > 2.30.2 >
> On Jan 11, 2022, at 4:13 AM, Peter Zijlstra <peterz@infradead.org> wrote: > > On Wed, Jan 05, 2022 at 06:25:31PM -0800, Song Liu wrote: > >> diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c >> index 23fb4d51a5da..02c35725cc62 100644 >> --- a/arch/x86/kernel/alternative.c >> +++ b/arch/x86/kernel/alternative.c >> @@ -1102,6 +1102,34 @@ void *text_poke_kgdb(void *addr, const void *opcode, size_t len) >> return __text_poke(addr, opcode, len); >> } >> >> +/** >> + * text_poke_jit - Update instructions on a live kernel by jit engine >> + * @addr: address to modify >> + * @opcode: source of the copy >> + * @len: length to copy, could be more than 2x PAGE_SIZE >> + * >> + * Only module memory taking jit text (e.g. for bpf) should be patched. >> + */ > > Maybe: > > text_poke_copy() - Copy instructions into (an unused part of) RX memory > @args... > > Not safe against concurrent execution; useful for JITs to dump > new code blocks into unused regions of RX memory. Can be used in > conjunction with synchronize_rcu_tasks() to wait for existing > execution to quiesce after having made sure no existing > functions pointers are life. > > or something along those lines? This sounds good! Thanks! Song
diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index b7421780e4e9..991058c9b4b1 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -44,6 +44,7 @@ extern void text_poke_early(void *addr, const void *opcode, size_t len); extern void *text_poke(void *addr, const void *opcode, size_t len); extern void text_poke_sync(void); extern void *text_poke_kgdb(void *addr, const void *opcode, size_t len); +extern void *text_poke_jit(void *addr, const void *opcode, size_t len); extern int poke_int3_handler(struct pt_regs *regs); extern void text_poke_bp(void *addr, const void *opcode, size_t len, const void *emulate); diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 23fb4d51a5da..02c35725cc62 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -1102,6 +1102,34 @@ void *text_poke_kgdb(void *addr, const void *opcode, size_t len) return __text_poke(addr, opcode, len); } +/** + * text_poke_jit - Update instructions on a live kernel by jit engine + * @addr: address to modify + * @opcode: source of the copy + * @len: length to copy, could be more than 2x PAGE_SIZE + * + * Only module memory taking jit text (e.g. for bpf) should be patched. + */ +void *text_poke_jit(void *addr, const void *opcode, size_t len) +{ + unsigned long start = (unsigned long)addr; + size_t patched = 0; + + if (WARN_ON_ONCE(core_kernel_text(start))) + return NULL; + + while (patched < len) { + unsigned long ptr = start + patched; + size_t s; + + s = min_t(size_t, PAGE_SIZE * 2 - offset_in_page(ptr), len - patched); + + __text_poke((void *)ptr, opcode + patched, s); + patched += s; + } + return addr; +} + static void do_sync_core(void *info) { sync_core();
This will be used by BPF jit compiler to dump JITed binary to a RX huge page, and thus allow multiple BPF programs sharing the a huge (2MB) page. Signed-off-by: Song Liu <song@kernel.org> --- arch/x86/include/asm/text-patching.h | 1 + arch/x86/kernel/alternative.c | 28 ++++++++++++++++++++++++++++ 2 files changed, 29 insertions(+)