From patchwork Mon Sep 26 18:37:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Desaulniers X-Patchwork-Id: 12989286 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7BCA7C32771 for ; Mon, 26 Sep 2022 18:45:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=0NCZWtqJNRYcIx0LTbr8vyXCtUTgqP/XyMv4ve+ln6M=; b=TBtIAWbnCZBd3scrWij9lXsa04 ExlCKzPNvbN3Vadj7d4aWz8vAhpHlceuQB1vLhgZi83E3Qhmo1oFILY3PPwbz1cjIF5+nnrkwzLRW sj2h/dwUQtD6SQbGaJy8ATPsj9czLG0MxEUnFFSK/z2j2n8T7EdFCL2JvPp79Q3roJxjES0sG/Ij2 5XlMbgQpPik3TU3bc/ZLltyYV6Yan+mjnCG37dsAL4q5ea3XyWiBAErmz/O0AR5AspypcKnUNe3j6 /Jb01QEOyN2PjwsuhybibPaZHVSsenOzBbWX4Q24gV8nnwIMlKnqsX5hgf9Zbvvn5HxrWc47I3/4j PAwmdmug==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oct5T-006NaS-EO; Mon, 26 Sep 2022 18:44:15 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ocszH-006IPY-RK for linux-arm-kernel@lists.infradead.org; Mon, 26 Sep 2022 18:37:53 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-3452b545544so69803387b3.3 for ; Mon, 26 Sep 2022 11:37:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date; bh=znhGVGmm6TLIt/hCvZyRbrWMCYyq9RR6vI5Xw2wMKC4=; b=d68/aWCLPiIhe6kggZL49/Bh7V4/p14HgIMbRCvVM2NXYFgGVn+Xq9brpRwfNhGn1c bzJwZSeSD28a7Y9C2q6pU7Ns5YVEFTZe/NUDWalKdLmiDD2Ycu38MkayfNaqystb+Yqe 2uitdjgk1pKgDrhsqNj5Zi+us9bovm9f4gWEJiOjaim9mYiKX/XM3sYD5w/R4o2mb4mZ CFQt2YiJ+JQ5f5ciYi+1HTq4fleAv03ZClh98oFN7wKat0tgF+afcbUELEguex2NqQsj ebHSIrxp17+FxDtmN9isOGdXvSr28CY+GxWqR7nRTLSnzGkZucq1jvCcRo3Cov2rfSOI 8Xfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date; bh=znhGVGmm6TLIt/hCvZyRbrWMCYyq9RR6vI5Xw2wMKC4=; b=skbwIDT42eCQnLeui7sV5O3uDGXpknRzThuBuaUlZwiJNwIlKBc6ciZEdOOdFhItpA RxJJUMrfUGokCnzVzfKUfLP4g+jRJlVbag6fYnDSLvOM4/XVW5yBU22cHAEdRu8ezZbM 3KvXuo+8kwR3vl8ly5VDy76XCgC4D+YWjhr6YfUGTywYXuZNU1AM3IsHjgqHphNA065G MBI7YG9vjKoNiU7UhKdtnEy5a5plWSU2hk3v5DVL6LJoapwlMrxZCBbBZYmKDxusVHe4 TtPDK7/tCDVbuyF97qzINLRGh71tW39yodLwcfPfAUULND170w7ORSbEw6Fd/6r/6UrZ PLIA== X-Gm-Message-State: ACrzQf1OUx4UqJKk6dqd7+G4KdLx1ZsPMAS6A4DgRPBwu9g9r8SOlxy1 mQ4ywbK7w2KrMZNOqayu3D6Q7X560ZhdndJ8OHM= X-Google-Smtp-Source: AMsMyM4bDMVt1tdnaBnUxux6UCle3vtahK9wHgCGdwWVS1dv1pQjCOyBISO8bmwfU2AlX2gfDPcGjZtYrAVWpRrRAUc= X-Received: from ndesaulniers-desktop.svl.corp.google.com ([2620:0:100e:712:8fb4:fe0d:d74c:5bcb]) (user=ndesaulniers job=sendgmr) by 2002:a25:f628:0:b0:6bc:bb6:aec5 with SMTP id t40-20020a25f628000000b006bc0bb6aec5mr929868ybd.139.1664217469129; Mon, 26 Sep 2022 11:37:49 -0700 (PDT) Date: Mon, 26 Sep 2022 11:37:25 -0700 In-Reply-To: Mime-Version: 1.0 References: X-Developer-Key: i=ndesaulniers@google.com; a=ed25519; pk=UIrHvErwpgNbhCkRZAYSX0CFd/XFEwqX3D0xqtqjNug= X-Developer-Signature: v=1; a=ed25519-sha256; t=1664217445; l=7895; i=ndesaulniers@google.com; s=20220923; h=from:subject; bh=x0RwTwO74GUfeRgTrJP+F6BnwHVqyXRlkCvRcvW1NlM=; b=aATOabxLSS9znVNtm6L5GIkLiNWqOo9CDSaqmGgfeElYHBzA7DRPuBx916vBrYh+uPU8ccbHNwUy 5wYW/6BNB3vDlr9XOrgMb7SvlKfMG6oNcMPwlge2WJWmJuq1ioxM X-Mailer: git-send-email 2.37.3.998.g577e59143f-goog Message-ID: <20220926183725.1112298-1-ndesaulniers@google.com> Subject: [PATCH] ARM: kprobes: move __kretprobe_trampoline to out of line assembler From: Nick Desaulniers To: Russell King , Masami Hiramatsu Cc: "Naveen N . Rao" , Anil S Keshavamurthy , "David S . Miller" , Arnd Bergmann , Nathan Chancellor , Tom Rix , sparkhuang , Ard Biesheuvel , Steven Rostedt , Linus Walleij , Chen Zhongjin , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, llvm@lists.linux.dev, Naresh Kamboju , regressions@lists.linux.dev, lkft-triage@lists.linaro.org, Nick Desaulniers , Linux Kernel Functional Testing , Logan Chien X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220926_113751_923034_2634185C X-CRM114-Status: GOOD ( 22.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org commit 1069c1dd20a3 ("ARM: 9231/1: Recover kretprobes return address for EABI stack unwinder") tickled a bug in clang's integrated assembler where the .save and .pad directives must have corresponding .fnstart directives. The integrated assembler is unaware that the compiler will be generating the .fnstart directive. arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede .save or .vsave directives :3:2: note: instantiated into assembly here .save {sp, lr, pc} ^ arch/arm/probes/kprobes/core.c:412:29: error: .fnstart must precede .pad directive :6:2: note: instantiated into assembly here .pad #52 ^ __kretprobe_trampoline's definition is already entirely inline asm. Move it to out-of-line asm to avoid breaking the build. Link: https://github.com/llvm/llvm-project/issues/57993 Link: https://github.com/ClangBuiltLinux/linux/issues/1718 Reported-by: Nathan Chancellor Reported-by: Linux Kernel Functional Testing Suggested-by: Logan Chien Signed-off-by: Nick Desaulniers Acked-by: Ard Biesheuvel --- Note: I wasn't quite sure if a Fixes tag against 1069c1dd20a3 was appropriate here? Either way, if 1069c1dd20a3 gets picked up for stable without this, it will break clang builds. arch/arm/probes/kprobes/Makefile | 1 + arch/arm/probes/kprobes/core.c | 54 ++---------------- .../arm/probes/kprobes/kretprobe-trampoline.S | 55 +++++++++++++++++++ include/asm-generic/kprobes.h | 13 +++-- 4 files changed, 69 insertions(+), 54 deletions(-) create mode 100644 arch/arm/probes/kprobes/kretprobe-trampoline.S diff --git a/arch/arm/probes/kprobes/Makefile b/arch/arm/probes/kprobes/Makefile index 6159010dac4a..cdbe9dd99e28 100644 --- a/arch/arm/probes/kprobes/Makefile +++ b/arch/arm/probes/kprobes/Makefile @@ -3,6 +3,7 @@ KASAN_SANITIZE_actions-common.o := n KASAN_SANITIZE_actions-arm.o := n KASAN_SANITIZE_actions-thumb.o := n obj-$(CONFIG_KPROBES) += core.o actions-common.o checkers-common.o +obj-$(CONFIG_KPROBES) += kretprobe-trampoline.o obj-$(CONFIG_ARM_KPROBES_TEST) += test-kprobes.o test-kprobes-objs := test-core.o diff --git a/arch/arm/probes/kprobes/core.c b/arch/arm/probes/kprobes/core.c index 1435b508aa36..17d7e0259e63 100644 --- a/arch/arm/probes/kprobes/core.c +++ b/arch/arm/probes/kprobes/core.c @@ -375,58 +375,10 @@ int __kprobes kprobe_exceptions_notify(struct notifier_block *self, return NOTIFY_DONE; } -/* - * When a retprobed function returns, trampoline_handler() is called, - * calling the kretprobe's handler. We construct a struct pt_regs to - * give a view of registers r0-r11, sp, lr, and pc to the user - * return-handler. This is not a complete pt_regs structure, but that - * should be enough for stacktrace from the return handler with or - * without pt_regs. - */ -void __naked __kprobes __kretprobe_trampoline(void) -{ - __asm__ __volatile__ ( - "ldr lr, =__kretprobe_trampoline \n\t" -#ifdef CONFIG_FRAME_POINTER - /* __kretprobe_trampoline makes a framepointer on pt_regs. */ -#ifdef CONFIG_CC_IS_CLANG - "stmdb sp, {sp, lr, pc} \n\t" - "sub sp, sp, #12 \n\t" - /* In clang case, pt_regs->ip = lr. */ - "stmdb sp!, {r0 - r11, lr} \n\t" - /* fp points regs->r11 (fp) */ - "add fp, sp, #44 \n\t" -#else /* !CONFIG_CC_IS_CLANG */ - /* In gcc case, pt_regs->ip = fp. */ - "stmdb sp, {fp, sp, lr, pc} \n\t" - "sub sp, sp, #16 \n\t" - "stmdb sp!, {r0 - r11} \n\t" - /* fp points regs->r15 (pc) */ - "add fp, sp, #60 \n\t" -#endif /* CONFIG_CC_IS_CLANG */ -#else /* !CONFIG_FRAME_POINTER */ - /* store SP, LR on stack and add EABI unwind hint */ - "stmdb sp, {sp, lr, pc} \n\t" - ".save {sp, lr, pc} \n\t" - "sub sp, sp, #16 \n\t" - "stmdb sp!, {r0 - r11} \n\t" - ".pad #52 \n\t" -#endif /* CONFIG_FRAME_POINTER */ - "mov r0, sp \n\t" - "bl trampoline_handler \n\t" - "mov lr, r0 \n\t" - "ldmia sp!, {r0 - r11} \n\t" - "add sp, sp, #16 \n\t" -#ifdef CONFIG_THUMB2_KERNEL - "bx lr \n\t" -#else - "mov pc, lr \n\t" -#endif - : : : "memory"); -} +/*void __kretprobe_trampoline(void);*/ /* Called from __kretprobe_trampoline */ -static __used __kprobes void *trampoline_handler(struct pt_regs *regs) +__kprobes void *trampoline_handler(struct pt_regs *regs) { return (void *)kretprobe_trampoline_handler(regs, (void *)regs->TRAMP_FP); } @@ -434,6 +386,8 @@ static __used __kprobes void *trampoline_handler(struct pt_regs *regs) void __kprobes arch_prepare_kretprobe(struct kretprobe_instance *ri, struct pt_regs *regs) { + extern void __kretprobe_trampoline(void); + ri->ret_addr = (kprobe_opcode_t *)regs->ARM_lr; ri->fp = (void *)regs->TRAMP_FP; diff --git a/arch/arm/probes/kprobes/kretprobe-trampoline.S b/arch/arm/probes/kprobes/kretprobe-trampoline.S new file mode 100644 index 000000000000..261c99b8c17f --- /dev/null +++ b/arch/arm/probes/kprobes/kretprobe-trampoline.S @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include +#include +#include + +/* + * When a retprobed function returns, trampoline_handler() is called, + * calling the kretprobe's handler. We construct a struct pt_regs to + * give a view of registers r0-r11, sp, lr, and pc to the user + * return-handler. This is not a complete pt_regs structure, but that + * should be enough for stacktrace from the return handler with or + * without pt_regs. + */ +__KPROBE +SYM_FUNC_START(__kretprobe_trampoline) +UNWIND(.fnstart) + ldr lr, =__kretprobe_trampoline +#ifdef CONFIG_FRAME_POINTER + /* __kretprobe_trampoline makes a framepointer on pt_regs. */ +#ifdef CONFIG_CC_IS_CLANG + stmdb sp, {sp, lr, pc} + sub sp, sp, #12 + /* In clang case, pt_regs->ip = lr. */ + stmdb sp!, {r0 - r11, lr} + /* fp points regs->r11 (fp) */ + add fp, sp, #44 +#else /* !CONFIG_CC_IS_CLANG */ + /* In gcc case, pt_regs->ip = fp. */ + stmdb sp, {fp, sp, lr, pc} + sub sp, sp, #16 + stmdb sp!, {r0 - r11} + /* fp points regs->r15 (pc) */ + add fp, sp, #60 +#endif /* CONFIG_CC_IS_CLANG */ +#else /* !CONFIG_FRAME_POINTER */ + /* store SP, LR on stack and add EABI unwind hint */ + stmdb sp, {sp, lr, pc} +UNWIND(.save {sp, lr, pc}) + sub sp, sp, #16 + stmdb sp!, {r0 - r11} +UNWIND(.pad #52) +#endif /* CONFIG_FRAME_POINTER */ + mov r0, sp + bl trampoline_handler + mov lr, r0 + ldmia sp!, {r0 - r11} + add sp, sp, #16 +#ifdef CONFIG_THUMB2_KERNEL + bx lr +#else + mov pc, lr +#endif +UNWIND(.fnend) +SYM_FUNC_END(__kretprobe_trampoline) diff --git a/include/asm-generic/kprobes.h b/include/asm-generic/kprobes.h index 060eab094e5a..1509daa281b8 100644 --- a/include/asm-generic/kprobes.h +++ b/include/asm-generic/kprobes.h @@ -2,7 +2,11 @@ #ifndef _ASM_GENERIC_KPROBES_H #define _ASM_GENERIC_KPROBES_H -#if defined(__KERNEL__) && !defined(__ASSEMBLY__) +#ifdef __KERNEL__ + +#ifdef __ASSEMBLY__ +# define __KPROBE .section ".kprobes.text", "ax" +#else #ifdef CONFIG_KPROBES /* * Blacklist ganerating macro. Specify functions which is not probed @@ -16,11 +20,12 @@ static unsigned long __used \ /* Use this to forbid a kprobes attach on very low level functions */ # define __kprobes __section(".kprobes.text") # define nokprobe_inline __always_inline -#else +#else /* !defined(CONFIG_KPROBES) */ # define NOKPROBE_SYMBOL(fname) # define __kprobes # define nokprobe_inline inline -#endif -#endif /* defined(__KERNEL__) && !defined(__ASSEMBLY__) */ +#endif /* defined(CONFIG_KPROBES) */ +#endif /* defined(__ASSEMBLY__) */ +#endif /* defined(__KERNEL__) */ #endif /* _ASM_GENERIC_KPROBES_H */