| Message ID | 20220830200826.1432338-1-scott@os.amperecomputing.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm64: Work around missing `bti c` in modules | expand |
On Tue, Aug 30, 2022 at 01:08:26PM -0700, D Scott Phillips wrote: > GCC does not insert a `bti c` instruction at the beginning of a function > when all callers reach the function through a direct branch[1]. In the case > of cross-section calls (like __init to non __init), a thunk may be inserted > which uses an indirect branch. If that happens, the first instruction in > the callee function will result in a Branch Target Exception due to the > missing `bti c`. Oh, nice - I think this is a good approach. I'd been poking at things but not come up with anything yet and didn't want to just disable BTI since it'd disable BTI for the fairly large set of users with small enough kernels. It does weaken the protection but is clearly better than just disabling it. > + /* > + * GCC does not insert a `bti c` instruction at the beginning > + * of a function when all callers reach the function through a > + * direct branch. In the case of cross-section calls (like > + * __init to non __init), a thunk may be inserted which uses > + * an indirect branch. If that happens, the first instruction > + * in the callee function will result in a Branch Target > + * Exception due to the missing `bti c`. > + * > + * If that's the case here, clear PSTATE.BTYPE and resume. > + */ This comment should reference the bug, I'm assuming GCC will fix this at which point we should stop doing this. > + if (IS_ENABLED(CONFIG_CC_IS_GCC)) { I think we should add a new Kconfig symbol for this which is currently just def_bool y for GCC but which we can add a version check for when there is a fix so that kernels built with an unaffected toolchain don't have the workaround code. We could do that incrementally but we're more likely to remember if the placeholder is there already.
diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index c75ca36b4a49..dad27e854d8c 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -388,6 +388,15 @@ static void noinstr el1_undef(struct pt_regs *regs) exit_to_kernel_mode(regs); } +static void noinstr el1_bti(struct pt_regs *regs) +{ + enter_from_kernel_mode(regs); + local_daif_inherit(regs); + do_bti(regs); + local_daif_mask(); + exit_to_kernel_mode(regs); +} + static void noinstr el1_dbg(struct pt_regs *regs, unsigned long esr) { unsigned long far = read_sysreg(far_el1); @@ -427,6 +436,9 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_UNKNOWN: el1_undef(regs); break; + case ESR_ELx_EC_BTI: + el1_bti(regs); + break; case ESR_ELx_EC_BREAKPT_CUR: case ESR_ELx_EC_SOFTSTP_CUR: case ESR_ELx_EC_WATCHPT_CUR: diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index b7fed33981f7..f4f1dfa64137 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -501,8 +501,43 @@ NOKPROBE_SYMBOL(do_undefinstr); void do_bti(struct pt_regs *regs) { - BUG_ON(!user_mode(regs)); - force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0); + struct module *mod; + + if (user_mode(regs)) { + force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0); + return; + } + + /* + * GCC does not insert a `bti c` instruction at the beginning + * of a function when all callers reach the function through a + * direct branch. In the case of cross-section calls (like + * __init to non __init), a thunk may be inserted which uses + * an indirect branch. If that happens, the first instruction + * in the callee function will result in a Branch Target + * Exception due to the missing `bti c`. + * + * If that's the case here, clear PSTATE.BTYPE and resume. + */ + if (IS_ENABLED(CONFIG_CC_IS_GCC)) { + preempt_disable(); + mod = __module_text_address(regs->pc); + preempt_enable(); + + if (mod && try_module_get(mod)) { + bool from_init; + + from_init = within_module_init(regs->regs[30], mod); + module_put(mod); + + if (from_init) { + regs->pstate &= ~PSR_BTYPE_MASK; + return; + } + } + } + + die("Oops - BTI", regs, 0); } NOKPROBE_SYMBOL(do_bti);
GCC does not insert a `bti c` instruction at the beginning of a function when all callers reach the function through a direct branch[1]. In the case of cross-section calls (like __init to non __init), a thunk may be inserted which uses an indirect branch. If that happens, the first instruction in the callee function will result in a Branch Target Exception due to the missing `bti c`. Handle Branch Target Exceptions which happen in the kernel due to module calls from __init to non-__init by clearing PSTATE.BTYPE and resuming. [1]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671 Signed-off-by: D Scott Phillips <scott@os.amperecomputing.com> --- arch/arm64/kernel/entry-common.c | 12 ++++++++++ arch/arm64/kernel/traps.c | 39 ++++++++++++++++++++++++++++++-- 2 files changed, 49 insertions(+), 2 deletions(-)