Message ID | 20250211023359.1570-2-laoar.shao@gmail.com (mailing list archive) |
---|---|
State | Changes Requested |
Delegated to: | BPF |
Headers | show |
Series | bpf: Reject attaching fexit to __noreturn functions | expand |
On Tue, 2025-02-11 at 10:33 +0800, Yafang Shao wrote: [...] > diff --git a/tools/include/linux/noreturns.h b/tools/include/linux/noreturns.h > new file mode 100644 > index 000000000000..b2174894f9f7 > --- /dev/null > +++ b/tools/include/linux/noreturns.h > @@ -0,0 +1,52 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > + > +/* > + * This is a (sorted!) list of all known __noreturn functions in the kernel. > + * It's needed for objtool to properly reverse-engineer the control flow graph. > + * > + * Yes, this is unfortunate. A better solution is in the works. > + */ I'm probably out of context for this discussion, sorry if I'm raising points already discussed. The DW_AT_noreturn attribute is defined for DWARF. A simple script like [1] could be used to find all functions with this attribute known to DWARF. Using this script I see several functions present in my kernel but not present in the NORETURN list from this patch: - abort - devtmpfs_work_loop - play_dead - rcu_gp_kthread - rcu_tasks_kthread All these are marked as FUNC symbols when doing 'readelf --symbols vmlinux'. 'pahole' could be modified to look for DW_AT_noreturn attributes and add this information in BTF. E.g. by adding special btf_decl_tag to corresponding FUNC definitions. This won't work if kernel is compiled w/o BTF, of-course. [1] https://gist.github.com/eddyz87/d8513a731dfe7e2be52b346aef1de353 > +NORETURN(__fortify_panic) > +NORETURN(__ia32_sys_exit) > +NORETURN(__ia32_sys_exit_group) > +NORETURN(__kunit_abort) > +NORETURN(__module_put_and_kthread_exit) > +NORETURN(__stack_chk_fail) > +NORETURN(__tdx_hypercall_failed) > +NORETURN(__ubsan_handle_builtin_unreachable) > +NORETURN(__x64_sys_exit) > +NORETURN(__x64_sys_exit_group) > +NORETURN(arch_cpu_idle_dead) > +NORETURN(bch2_trans_in_restart_error) > +NORETURN(bch2_trans_restart_error) > +NORETURN(bch2_trans_unlocked_error) > +NORETURN(cpu_bringup_and_idle) > +NORETURN(cpu_startup_entry) > +NORETURN(do_exit) > +NORETURN(do_group_exit) > +NORETURN(do_task_dead) > +NORETURN(ex_handler_msr_mce) > +NORETURN(hlt_play_dead) > +NORETURN(hv_ghcb_terminate) > +NORETURN(kthread_complete_and_exit) > +NORETURN(kthread_exit) > +NORETURN(kunit_try_catch_throw) > +NORETURN(machine_real_restart) > +NORETURN(make_task_dead) > +NORETURN(mpt_halt_firmware) > +NORETURN(nmi_panic_self_stop) > +NORETURN(panic) > +NORETURN(panic_smp_self_stop) > +NORETURN(rest_init) > +NORETURN(rewind_stack_and_make_dead) > +NORETURN(rust_begin_unwind) > +NORETURN(rust_helper_BUG) > +NORETURN(sev_es_terminate) > +NORETURN(snp_abort) > +NORETURN(start_kernel) > +NORETURN(stop_this_cpu) > +NORETURN(usercopy_abort) > +NORETURN(x86_64_start_kernel) > +NORETURN(x86_64_start_reservations) > +NORETURN(xen_cpu_bringup_again) > +NORETURN(xen_start_kernel) [...]
On Tue, Feb 11, 2025 at 12:12 PM Eduard Zingerman <eddyz87@gmail.com> wrote: > > On Tue, 2025-02-11 at 10:33 +0800, Yafang Shao wrote: > > [...] > > > diff --git a/tools/include/linux/noreturns.h b/tools/include/linux/noreturns.h > > new file mode 100644 > > index 000000000000..b2174894f9f7 > > --- /dev/null > > +++ b/tools/include/linux/noreturns.h > > @@ -0,0 +1,52 @@ > > +/* SPDX-License-Identifier: GPL-2.0 */ > > + > > +/* > > + * This is a (sorted!) list of all known __noreturn functions in the kernel. > > + * It's needed for objtool to properly reverse-engineer the control flow graph. > > + * > > + * Yes, this is unfortunate. A better solution is in the works. > > + */ > > I'm probably out of context for this discussion, sorry if I'm raising > points already discussed. Please refer to the discussion at https://lore.kernel.org/bpf/CAADnVQKXgPTQsjUDB3tjZ46aPWvoEcxBCnDXro8WPtNhkGNFyg@mail.gmail.com/ for more details. > > The DW_AT_noreturn attribute is defined for DWARF. A simple script > like [1] could be used to find all functions with this attribute known > to DWARF. Using this script I see several functions present in my > kernel but not present in the NORETURN list from this patch: > - abort > - devtmpfs_work_loop > - play_dead > - rcu_gp_kthread > - rcu_tasks_kthread > > All these are marked as FUNC symbols when doing 'readelf --symbols vmlinux'. > > 'pahole' could be modified to look for DW_AT_noreturn attributes and > add this information in BTF. E.g. by adding special btf_decl_tag to > corresponding FUNC definitions. This won't work if kernel is compiled > w/o BTF, of-course. > > [1] https://gist.github.com/eddyz87/d8513a731dfe7e2be52b346aef1de353 Thank you for your example—it’s been really helpful. Following Alexei's suggestion, we’ll address this in the next step. We need to apply the fix to the current kernel and backport it, so for now, the simplest approach is to implement the current solution. -- Regards Yafang
On Tue, Feb 11, 2025 at 10:33:57AM +0800, Yafang Shao wrote: > It will used by bpf to reject attaching fexit prog to functions > annotated with __noreturn. > > Signed-off-by: Yafang Shao <laoar.shao@gmail.com> > Cc: Josh Poimboeuf <jpoimboe@kernel.org> > Cc: Peter Zijlstra <peterz@infradead.org> > --- > {tools/objtool => include/linux}/noreturns.h | 0 > tools/include/linux/noreturns.h | 52 ++++++++++++++++++++ Instead of moving the file, please make a copy and add it to tools/objtool/sync-check.sh.
On Mon, Feb 10, 2025 at 08:12:32PM -0800, Eduard Zingerman wrote: > I'm probably out of context for this discussion, sorry if I'm raising > points already discussed. > > The DW_AT_noreturn attribute is defined for DWARF. A simple script > like [1] could be used to find all functions with this attribute known > to DWARF. Using this script I see several functions present in my > kernel but not present in the NORETURN list from this patch: > - abort > - devtmpfs_work_loop > - play_dead > - rcu_gp_kthread > - rcu_tasks_kthread > > All these are marked as FUNC symbols when doing 'readelf --symbols vmlinux'. > > 'pahole' could be modified to look for DW_AT_noreturn attributes and > add this information in BTF. E.g. by adding special btf_decl_tag to > corresponding FUNC definitions. This won't work if kernel is compiled > w/o BTF, of-course. > > [1] https://gist.github.com/eddyz87/d8513a731dfe7e2be52b346aef1de353 I also suggested this, I agree this is a much better way to go. noreturns.h is manually maintained based on objtool warnings and I'm not surprised it has missing entries. Alexei mentioned 30k+ noreturns, but when I eliminate dups and __compiletime_assert_* it's a much smaller list: $ ./noreturn_printer vmlinux |sort |uniq |grep -v compiletime_assert arch_cpu_idle_dead external idle.c arch_cpu_idle_dead external process.c cpu_startup_entry external cpu.h cpu_startup_entry external idle.c do_exit external exit.c do_exit external kernel.h do_group_exit external exit.c do_group_exit external task.h do_task_dead external core.c do_task_dead external task.h doublefault_shim external doublefault_32.c ex_handler_msr_mce external core.c ex_handler_msr_mce external extable.h __fortify_panic external fortify-string.h __fortify_panic external string_helpers.c i386_start_kernel external head32.c __ia32_sys_exit external syscalls_32.h __ia32_sys_exit_group external syscalls_32.h kthread_complete_and_exit external kthread.c kthread_exit external kthread.c kthread_exit external kthread.h machine_real_restart external reboot.c make_task_dead external exit.c __module_put_and_kthread_exit external main.c __module_put_and_kthread_exit external module.h nmi_panic_self_stop external panic.c panic external panic.c panic external panic.h panic_smp_self_stop external panic.c play_dead process.c rcu_gp_kthread tree.c rcu_tasks_kthread tasks.h rest_init main.c rewind_stack_and_make_dead external dumpstack.c start_kernel external main.c start_kernel external start_kernel.h stop_this_cpu external process.c stop_this_cpu external processor.h Also, for objtool we could use something based on your program to autogenerate noreturns.h. Only problem is, objtool doesn't currently have a dependency on CONFIG_DEBUG_INFO. Another option we've considered is compiler annotations (or compiler plugins).
On Tue, Feb 11, 2025 at 11:50 PM Josh Poimboeuf <jpoimboe@kernel.org> wrote: > > On Tue, Feb 11, 2025 at 10:33:57AM +0800, Yafang Shao wrote: > > It will used by bpf to reject attaching fexit prog to functions > > annotated with __noreturn. > > > > Signed-off-by: Yafang Shao <laoar.shao@gmail.com> > > Cc: Josh Poimboeuf <jpoimboe@kernel.org> > > Cc: Peter Zijlstra <peterz@infradead.org> > > --- > > {tools/objtool => include/linux}/noreturns.h | 0 > > tools/include/linux/noreturns.h | 52 ++++++++++++++++++++ > > Instead of moving the file, please make a copy Are you suggesting we keep tools/objtool/noreturns.h as is and simply copy it to include/linux/noreturns.h? In other words, the two files would be tools/objtool/noreturns.h and include/linux/noreturns.h. > and add it to > tools/objtool/sync-check.sh. Sure.
On Wed, Feb 12, 2025 at 12:11 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote: > > On Mon, Feb 10, 2025 at 08:12:32PM -0800, Eduard Zingerman wrote: > > I'm probably out of context for this discussion, sorry if I'm raising > > points already discussed. > > > > The DW_AT_noreturn attribute is defined for DWARF. A simple script > > like [1] could be used to find all functions with this attribute known > > to DWARF. Using this script I see several functions present in my > > kernel but not present in the NORETURN list from this patch: > > - abort > > - devtmpfs_work_loop > > - play_dead > > - rcu_gp_kthread > > - rcu_tasks_kthread > > > > All these are marked as FUNC symbols when doing 'readelf --symbols vmlinux'. > > > > 'pahole' could be modified to look for DW_AT_noreturn attributes and > > add this information in BTF. E.g. by adding special btf_decl_tag to > > corresponding FUNC definitions. This won't work if kernel is compiled > > w/o BTF, of-course. > > > > [1] https://gist.github.com/eddyz87/d8513a731dfe7e2be52b346aef1de353 > > I also suggested this, I agree this is a much better way to go. > noreturns.h is manually maintained based on objtool warnings and > I'm not surprised it has missing entries. > > Alexei mentioned 30k+ noreturns, but when I eliminate dups and > __compiletime_assert_* it's a much smaller list: > > $ ./noreturn_printer vmlinux |sort |uniq |grep -v compiletime_assert > arch_cpu_idle_dead external idle.c > arch_cpu_idle_dead external process.c > cpu_startup_entry external cpu.h > cpu_startup_entry external idle.c > do_exit external exit.c > do_exit external kernel.h > do_group_exit external exit.c > do_group_exit external task.h > do_task_dead external core.c > do_task_dead external task.h > doublefault_shim external doublefault_32.c > ex_handler_msr_mce external core.c > ex_handler_msr_mce external extable.h > __fortify_panic external fortify-string.h > __fortify_panic external string_helpers.c > i386_start_kernel external head32.c > __ia32_sys_exit external syscalls_32.h > __ia32_sys_exit_group external syscalls_32.h > kthread_complete_and_exit external kthread.c > kthread_exit external kthread.c > kthread_exit external kthread.h > machine_real_restart external reboot.c > make_task_dead external exit.c > __module_put_and_kthread_exit external main.c > __module_put_and_kthread_exit external module.h > nmi_panic_self_stop external panic.c > panic external panic.c > panic external panic.h > panic_smp_self_stop external panic.c > play_dead process.c > rcu_gp_kthread tree.c > rcu_tasks_kthread tasks.h > rest_init main.c > rewind_stack_and_make_dead external dumpstack.c > start_kernel external main.c > start_kernel external start_kernel.h > stop_this_cpu external process.c > stop_this_cpu external processor.h > > Also, for objtool we could use something based on your program to > autogenerate noreturns.h. Automatically generating noreturns.h is a great idea, as it would make the file accessible for use by others as well. > Only problem is, objtool doesn't currently > have a dependency on CONFIG_DEBUG_INFO. Is there any reason we can't make it dependent on CONFIG_DEBUG_INFO?" > Another option we've considered > is compiler annotations (or compiler plugins).
On Fri, Feb 14, 2025 at 03:26:40PM +0800, Yafang Shao wrote: > On Tue, Feb 11, 2025 at 11:50 PM Josh Poimboeuf <jpoimboe@kernel.org> wrote: > > > > On Tue, Feb 11, 2025 at 10:33:57AM +0800, Yafang Shao wrote: > > > It will used by bpf to reject attaching fexit prog to functions > > > annotated with __noreturn. > > > > > > Signed-off-by: Yafang Shao <laoar.shao@gmail.com> > > > Cc: Josh Poimboeuf <jpoimboe@kernel.org> > > > Cc: Peter Zijlstra <peterz@infradead.org> > > > --- > > > {tools/objtool => include/linux}/noreturns.h | 0 > > > tools/include/linux/noreturns.h | 52 ++++++++++++++++++++ > > > > Instead of moving the file, please make a copy > > Are you suggesting we keep tools/objtool/noreturns.h as is and simply > copy it to include/linux/noreturns.h? In other words, the two files > would be tools/objtool/noreturns.h and include/linux/noreturns.h. Right.
On Fri, Feb 14, 2025 at 03:29:43PM +0800, Yafang Shao wrote: > On Wed, Feb 12, 2025 at 12:11 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote: > > Only problem is, objtool doesn't currently > > have a dependency on CONFIG_DEBUG_INFO. > > Is there any reason we can't make it dependent on CONFIG_DEBUG_INFO?" Objtool is enabled by default on x86 and is pretty much required at this point. We definitely don't want to force enable CONFIG_DEBUG_INFO as that will slow the build down considerably.
On Tue, Feb 11, 2025 at 08:11:22AM -0800, Josh Poimboeuf wrote: > Also, for objtool we could use something based on your program to > autogenerate noreturns.h. Only problem is, objtool doesn't currently > have a dependency on CONFIG_DEBUG_INFO. Another option we've considered > is compiler annotations (or compiler plugins). But we don't need to re-generate the file on every build, right? We can have every DBUG_INFO build verify the file is still complete.
On Fri, Feb 14, 2025 at 12:14:49AM -0800, Josh Poimboeuf wrote: > On Fri, Feb 14, 2025 at 03:29:43PM +0800, Yafang Shao wrote: > > On Wed, Feb 12, 2025 at 12:11 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote: > > > Only problem is, objtool doesn't currently > > > have a dependency on CONFIG_DEBUG_INFO. > > > > Is there any reason we can't make it dependent on CONFIG_DEBUG_INFO?" > > Objtool is enabled by default on x86 and is pretty much required at this > point. x86_64, but yeah, disabling objtool for x86_64 is going to be *really* hard. i386 is still limping along without -- mostly because nobody cared enough to make it work. > We definitely don't want to force enable CONFIG_DEBUG_INFO as > that will slow the build down considerably. This, very much this. Same reason I hardly every use clang to build a kernel -- just too slow.
On Fri, Feb 14, 2025 at 11:48:00AM +0100, Peter Zijlstra wrote: > On Tue, Feb 11, 2025 at 08:11:22AM -0800, Josh Poimboeuf wrote: > > > Also, for objtool we could use something based on your program to > > autogenerate noreturns.h. Only problem is, objtool doesn't currently > > have a dependency on CONFIG_DEBUG_INFO. Another option we've considered > > is compiler annotations (or compiler plugins). > > But we don't need to re-generate the file on every build, right? We can > have every DBUG_INFO build verify the file is still complete. Hm, yeah we could do that. It's still not ideal, as a human now has to go fix those warnings... But it's probably better than the current situation.
diff --git a/tools/objtool/noreturns.h b/include/linux/noreturns.h similarity index 100% rename from tools/objtool/noreturns.h rename to include/linux/noreturns.h diff --git a/tools/include/linux/noreturns.h b/tools/include/linux/noreturns.h new file mode 100644 index 000000000000..b2174894f9f7 --- /dev/null +++ b/tools/include/linux/noreturns.h @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * This is a (sorted!) list of all known __noreturn functions in the kernel. + * It's needed for objtool to properly reverse-engineer the control flow graph. + * + * Yes, this is unfortunate. A better solution is in the works. + */ +NORETURN(__fortify_panic) +NORETURN(__ia32_sys_exit) +NORETURN(__ia32_sys_exit_group) +NORETURN(__kunit_abort) +NORETURN(__module_put_and_kthread_exit) +NORETURN(__stack_chk_fail) +NORETURN(__tdx_hypercall_failed) +NORETURN(__ubsan_handle_builtin_unreachable) +NORETURN(__x64_sys_exit) +NORETURN(__x64_sys_exit_group) +NORETURN(arch_cpu_idle_dead) +NORETURN(bch2_trans_in_restart_error) +NORETURN(bch2_trans_restart_error) +NORETURN(bch2_trans_unlocked_error) +NORETURN(cpu_bringup_and_idle) +NORETURN(cpu_startup_entry) +NORETURN(do_exit) +NORETURN(do_group_exit) +NORETURN(do_task_dead) +NORETURN(ex_handler_msr_mce) +NORETURN(hlt_play_dead) +NORETURN(hv_ghcb_terminate) +NORETURN(kthread_complete_and_exit) +NORETURN(kthread_exit) +NORETURN(kunit_try_catch_throw) +NORETURN(machine_real_restart) +NORETURN(make_task_dead) +NORETURN(mpt_halt_firmware) +NORETURN(nmi_panic_self_stop) +NORETURN(panic) +NORETURN(panic_smp_self_stop) +NORETURN(rest_init) +NORETURN(rewind_stack_and_make_dead) +NORETURN(rust_begin_unwind) +NORETURN(rust_helper_BUG) +NORETURN(sev_es_terminate) +NORETURN(snp_abort) +NORETURN(start_kernel) +NORETURN(stop_this_cpu) +NORETURN(usercopy_abort) +NORETURN(x86_64_start_kernel) +NORETURN(x86_64_start_reservations) +NORETURN(xen_cpu_bringup_again) +NORETURN(xen_start_kernel) diff --git a/tools/objtool/Documentation/objtool.txt b/tools/objtool/Documentation/objtool.txt index 7c3ee959b63c..726db5b2b1a5 100644 --- a/tools/objtool/Documentation/objtool.txt +++ b/tools/objtool/Documentation/objtool.txt @@ -326,7 +326,8 @@ the objtool maintainers. The call from foo() to bar() doesn't return, but bar() is missing the __noreturn annotation. NOTE: In addition to annotating the function - with __noreturn, please also add it to tools/objtool/noreturns.h. + with __noreturn, please also add it to tools/include/linux/noreturns.h and + include/linux/noreturns.h. 4. file.o: warning: objtool: func(): can't find starting instruction or diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 753dbc4f8198..2940ddc56b1a 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -250,7 +250,7 @@ static bool __dead_end_function(struct objtool_file *file, struct symbol *func, #define NORETURN(func) __stringify(func), static const char * const global_noreturns[] = { -#include "noreturns.h" +#include <linux/noreturns.h> }; #undef NORETURN
It will used by bpf to reject attaching fexit prog to functions annotated with __noreturn. Signed-off-by: Yafang Shao <laoar.shao@gmail.com> Cc: Josh Poimboeuf <jpoimboe@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> --- {tools/objtool => include/linux}/noreturns.h | 0 tools/include/linux/noreturns.h | 52 ++++++++++++++++++++ tools/objtool/Documentation/objtool.txt | 3 +- tools/objtool/check.c | 2 +- 4 files changed, 55 insertions(+), 2 deletions(-) rename {tools/objtool => include/linux}/noreturns.h (100%) create mode 100644 tools/include/linux/noreturns.h