From patchwork Thu Sep 22 21:03:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12985799 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F8DDECAAD8 for ; Thu, 22 Sep 2022 21:03:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230011AbiIVVDs (ORCPT ); Thu, 22 Sep 2022 17:03:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229701AbiIVVDr (ORCPT ); Thu, 22 Sep 2022 17:03:47 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 541F9DED55 for ; Thu, 22 Sep 2022 14:03:46 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E3F0362265 for ; Thu, 22 Sep 2022 21:03:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 44336C433D6; Thu, 22 Sep 2022 21:03:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1663880625; bh=oomoVeYl/NS7dXqwjk2EddpsaQRK5EqdKhgqMxNE9lk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=embh+h0eN0LfDzty1QEv34XLlK+kjiw1QU6FP+W8lMdtSpJG7gaye/ntpxMYsp3gb o33h0vlNaGTsjgN7Ztwd13+n/04zt2+DCRyfGvudUHunKARBPQmxiwQmC8Hz/olLVr JBfEIAH5mTSPEtMmR/1c2ttUYJxL0+HpusCcwMGbID1+fHxUYsP+ytjdWFuIJVT+RQ Ot8vST0uah/XBXRN//CFKeYsfS0EISQgLXkDghD6/HTbRoCctJdm4qc0uMZ0H9t6ar /f1403E9J29uhRaduXcyTt8M0/yfmvGNJmf2FFuXCPeuoPDokhrLXXqP9op0gnJUiz xPUsNPgZCGu2w== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Masami Hiramatsu , bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Peter Zijlstra , Martynas Pumputis Subject: [PATCHv4 bpf-next 1/6] kprobes: Add new KPROBE_FLAG_ON_FUNC_ENTRY kprobe flag Date: Thu, 22 Sep 2022 23:03:15 +0200 Message-Id: <20220922210320.1076658-2-jolsa@kernel.org> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220922210320.1076658-1-jolsa@kernel.org> References: <20220922210320.1076658-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding KPROBE_FLAG_ON_FUNC_ENTRY kprobe flag to indicate that attach address is on function entry. This is used in following changes in get_func_ip helper to return correct function address. Acked-by: Masami Hiramatsu (Google) Signed-off-by: Jiri Olsa --- include/linux/kprobes.h | 1 + kernel/kprobes.c | 6 +++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/include/linux/kprobes.h b/include/linux/kprobes.h index 55041d2f884d..a0b92be98984 100644 --- a/include/linux/kprobes.h +++ b/include/linux/kprobes.h @@ -103,6 +103,7 @@ struct kprobe { * this flag is only for optimized_kprobe. */ #define KPROBE_FLAG_FTRACE 8 /* probe is using ftrace */ +#define KPROBE_FLAG_ON_FUNC_ENTRY 16 /* probe is on the function entry */ /* Has this kprobe gone ? */ static inline bool kprobe_gone(struct kprobe *p) diff --git a/kernel/kprobes.c b/kernel/kprobes.c index 08350e35aba2..51adc3c94503 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -1606,9 +1606,10 @@ int register_kprobe(struct kprobe *p) struct kprobe *old_p; struct module *probed_mod; kprobe_opcode_t *addr; + bool on_func_entry; /* Adjust probe address from symbol */ - addr = kprobe_addr(p); + addr = _kprobe_addr(p->addr, p->symbol_name, p->offset, &on_func_entry); if (IS_ERR(addr)) return PTR_ERR(addr); p->addr = addr; @@ -1628,6 +1629,9 @@ int register_kprobe(struct kprobe *p) mutex_lock(&kprobe_mutex); + if (on_func_entry) + p->flags |= KPROBE_FLAG_ON_FUNC_ENTRY; + old_p = get_kprobe(p->addr); if (old_p) { /* Since this may unoptimize 'old_p', locking 'text_mutex'. */ From patchwork Thu Sep 22 21:03:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12985800 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCC5DC54EE9 for ; Thu, 22 Sep 2022 21:04:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230306AbiIVVEY (ORCPT ); Thu, 22 Sep 2022 17:04:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46034 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230289AbiIVVEG (ORCPT ); Thu, 22 Sep 2022 17:04:06 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8C0C10FE2A for ; Thu, 22 Sep 2022 14:04:05 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 55E2063220 for ; Thu, 22 Sep 2022 21:04:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0B4C8C433D7; Thu, 22 Sep 2022 21:03:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1663880644; bh=2WnKXTXVQn0092eVlGqi0BKS7kLkBS0mXywwvISTgBY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JMvAsCUUT0GSzArt8QM0slQpVW+u1m5CN+sTztBDpebYB14RclROTjIcpfUEwXDeb NsHlYFxiY0ZejmenJ3WOzApBoh7grKBVZk3xmWl3ii6Hkzo/NnN6twJVWvhaVrVD7H pb5L30zkX4MlbqpeEfIT+BeqXP1VEf2tkmu+MbLSEaDsseoNyMCTH2lmsdJnoqaASQ AkxWYiRp3GS6Kd+j7biGRlcnViAj4jo/pLcx1epogfe6bUEjLsNP5kudRRdx2QkXHM NwsnX1QrapYImnzcL0ApR3er0VaXBzkSbb6MvO7Vp3suStMSdcyVrTLJR/kfvReUHg 8oQa5K8B6Na3g== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Masami Hiramatsu , bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Peter Zijlstra , Martynas Pumputis Subject: [PATCHv4 bpf-next 2/6] ftrace: Keep the resolved addr in kallsyms_callback Date: Thu, 22 Sep 2022 23:03:16 +0200 Message-Id: <20220922210320.1076658-3-jolsa@kernel.org> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220922210320.1076658-1-jolsa@kernel.org> References: <20220922210320.1076658-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Keeping the resolved 'addr' in kallsyms_callback, instead of taking ftrace_location value, because we depend on symbol address in the cookie related code. With CONFIG_X86_KERNEL_IBT option the ftrace_location value differs from symbol address, which screwes the symbol address cookies matching. There are 2 users of this function: - bpf_kprobe_multi_link_attach for which this fix is for - get_ftrace_locations which is used by register_fprobe_syms this function needs to get symbols resolved to addresses, but does not need 'ftrace location addresses' at this point there's another ftrace location translation in the path done by ftrace_set_filter_ips call: register_fprobe_syms addrs = get_ftrace_locations register_fprobe_ips(addrs) ... ftrace_set_filter_ips ... __ftrace_match_addr ip = ftrace_location(ip); ... Reviewed-by: Masami Hiramatsu (Google) Signed-off-by: Jiri Olsa --- kernel/trace/ftrace.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 439e2ab6905e..447d2e2a8549 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -8265,8 +8265,7 @@ static int kallsyms_callback(void *data, const char *name, if (args->addrs[idx]) return 0; - addr = ftrace_location(addr); - if (!addr) + if (!ftrace_location(addr)) return 0; args->addrs[idx] = addr; From patchwork Thu Sep 22 21:03:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12985801 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A894C54EE9 for ; Thu, 22 Sep 2022 21:04:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230296AbiIVVEb (ORCPT ); Thu, 22 Sep 2022 17:04:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230393AbiIVVEV (ORCPT ); Thu, 22 Sep 2022 17:04:21 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61CA9111DCC for ; Thu, 22 Sep 2022 14:04:19 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E9FBA6318A for ; Thu, 22 Sep 2022 21:04:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4B69DC433D6; Thu, 22 Sep 2022 21:04:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1663880658; bh=8zHQ8a1AvPzozmCud6GCIR/pQsoPwADQwX+wZbl60Cw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qylNgZnqKUK57TO1+jGgmDFaMgyQLfXzcizMfIfUsQrK+D/V/mbC0yiBJ2acz5IJD d6J0h3Bz3BX4X7xrEpQoIXYxwSTr0rU6Qhi30IIbcN03Ut8YHlrFoeSqSy/PvYndlB 2tgSjKgAw71qAekpjFW3fJ09A89akRUU3pJob8zr5s6caSOvkjbPBsN40TWWtXhdrS dTXloYPdLLlqBatIe4eU2+7h3a8EwUUlAp3SqNG2Vwsgg4lnacSWVhEY5ThY0azt8p PJZ6bqAVanErkryNj50S0SGx2IHnE/RSj3VdbRN+Vdf6sd+M0MF59lhuNkMLegJEmq /DfiZ+gYNKTWQ== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , "Masami Hiramatsu (Google)" , Peter Zijlstra , Martynas Pumputis Subject: [PATCHv4 bpf-next 3/6] bpf: Use given function address for trampoline ip arg Date: Thu, 22 Sep 2022 23:03:17 +0200 Message-Id: <20220922210320.1076658-4-jolsa@kernel.org> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220922210320.1076658-1-jolsa@kernel.org> References: <20220922210320.1076658-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Using function address given at the generation time as the trampoline ip argument. This way we get directly the function address that we need, so we don't need to: - read the ip from the stack - subtract X86_PATCH_SIZE - subtract ENDBR_INSN_SIZE if CONFIG_X86_KERNEL_IBT is enabled which is not even implemented yet ;-) Signed-off-by: Jiri Olsa --- arch/x86/net/bpf_jit_comp.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index ae89f4143eb4..d4a6183197e9 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -662,7 +662,7 @@ static void emit_mov_imm64(u8 **pprog, u32 dst_reg, */ emit_mov_imm32(&prog, false, dst_reg, imm32_lo); } else { - /* movabsq %rax, imm64 */ + /* movabsq rax, imm64 */ EMIT2(add_1mod(0x48, dst_reg), add_1reg(0xB8, dst_reg)); EMIT(imm32_lo, 4); EMIT(imm32_hi, 4); @@ -2039,13 +2039,14 @@ static int invoke_bpf_mod_ret(const struct btf_func_model *m, u8 **pprog, int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *image_end, const struct btf_func_model *m, u32 flags, struct bpf_tramp_links *tlinks, - void *orig_call) + void *func_addr) { int ret, i, nr_args = m->nr_args, extra_nregs = 0; int regs_off, ip_off, args_off, stack_size = nr_args * 8, run_ctx_off; struct bpf_tramp_links *fentry = &tlinks[BPF_TRAMP_FENTRY]; struct bpf_tramp_links *fexit = &tlinks[BPF_TRAMP_FEXIT]; struct bpf_tramp_links *fmod_ret = &tlinks[BPF_TRAMP_MODIFY_RETURN]; + void *orig_call = func_addr; u8 **branches = NULL; u8 *prog; bool save_ret; @@ -2126,12 +2127,10 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i if (flags & BPF_TRAMP_F_IP_ARG) { /* Store IP address of the traced function: - * mov rax, QWORD PTR [rbp + 8] - * sub rax, X86_PATCH_SIZE + * movabsq rax, func_addr * mov QWORD PTR [rbp - ip_off], rax */ - emit_ldx(&prog, BPF_DW, BPF_REG_0, BPF_REG_FP, 8); - EMIT4(0x48, 0x83, 0xe8, X86_PATCH_SIZE); + emit_mov_imm64(&prog, BPF_REG_0, (long) func_addr >> 32, (u32) (long) func_addr); emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -ip_off); } From patchwork Thu Sep 22 21:03:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12985802 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53317ECAAD8 for ; Thu, 22 Sep 2022 21:04:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230234AbiIVVEr (ORCPT ); Thu, 22 Sep 2022 17:04:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230267AbiIVVEq (ORCPT ); Thu, 22 Sep 2022 17:04:46 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B33132EF2 for ; Thu, 22 Sep 2022 14:04:42 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 6CD27B838BF for ; Thu, 22 Sep 2022 21:04:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3BDCFC433D6; Thu, 22 Sep 2022 21:04:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1663880680; bh=tyIn95ASZrVudNAaDcgs4Mh4QEiwQUf+uUpcrL9YQTg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Dn26HQoCpJK1cLz0KqfCakrpibj82QXit737P1v0E5lCjV/HSUrX5dzaWCGk/Ko7S QhvnIZZ6I9IZ5yhjgS4bsFjkXsWSQhYnyA2vgWe6KnA9O4AdciXxKfkvfvJDwhTd36 iugUeTssIxfLjdSUb3lLfntUMuloWlgBwZzDcGaewMj/naYr4Ez/+M7gX2QYPs9eW3 BwgLz2HTghtIkEQmK0FwLYrTG5VHl2wY8Zf3QHEijvH0dbyCoIoCSqgRlDyabgsxkt 26qnTruGnrAxfu2YfMOSBfwnsjTwbEAhSZE27+qZMIUYeg0fIdDs1bavf8WsCBcnb7 z4sD5CLNJu8yA== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Peter Zijlstra , Martynas Pumputis , bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , "Masami Hiramatsu (Google)" Subject: [PATCHv4 bpf-next 4/6] bpf: Adjust kprobe_multi entry_ip for CONFIG_X86_KERNEL_IBT Date: Thu, 22 Sep 2022 23:03:18 +0200 Message-Id: <20220922210320.1076658-5-jolsa@kernel.org> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220922210320.1076658-1-jolsa@kernel.org> References: <20220922210320.1076658-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Martynas reported bpf_get_func_ip returning +4 address when CONFIG_X86_KERNEL_IBT option is enabled. When CONFIG_X86_KERNEL_IBT is enabled we'll have endbr instruction at the function entry, which screws return value of bpf_get_func_ip() helper that should return the function address. There's short term workaround for kprobe_multi bpf program made by Alexei [1], but we need this fixup also for bpf_get_attach_cookie, that returns cookie based on the entry_ip value. Moving the fixup in the fprobe handler, so both bpf_get_func_ip and bpf_get_attach_cookie get expected function address when CONFIG_X86_KERNEL_IBT option is enabled. Also renaming kprobe_multi_link_handler entry_ip argument to fentry_ip so it's clearer this is an ftrace __fentry__ ip. [1] commit 7f0059b58f02 ("selftests/bpf: Fix kprobe_multi test.") Cc: Peter Zijlstra Reported-by: Martynas Pumputis Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- kernel/trace/bpf_trace.c | 20 +++++++++++++++++-- .../selftests/bpf/progs/kprobe_multi.c | 4 +--- 2 files changed, 19 insertions(+), 5 deletions(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index b05f0310dbd3..ebd1b348beb3 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1028,6 +1028,22 @@ static const struct bpf_func_proto bpf_get_func_ip_proto_tracing = { .arg1_type = ARG_PTR_TO_CTX, }; +#ifdef CONFIG_X86_KERNEL_IBT +static unsigned long get_entry_ip(unsigned long fentry_ip) +{ + u32 instr; + + /* Being extra safe in here in case entry ip is on the page-edge. */ + if (get_kernel_nofault(instr, (u32 *) fentry_ip - 1)) + return fentry_ip; + if (is_endbr(instr)) + fentry_ip -= ENDBR_INSN_SIZE; + return fentry_ip; +} +#else +#define get_entry_ip(fentry_ip) fentry_ip +#endif + BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs) { struct kprobe *kp = kprobe_running(); @@ -2600,13 +2616,13 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link, } static void -kprobe_multi_link_handler(struct fprobe *fp, unsigned long entry_ip, +kprobe_multi_link_handler(struct fprobe *fp, unsigned long fentry_ip, struct pt_regs *regs) { struct bpf_kprobe_multi_link *link; link = container_of(fp, struct bpf_kprobe_multi_link, fp); - kprobe_multi_link_prog_run(link, entry_ip, regs); + kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs); } static int symbols_cmp_r(const void *a, const void *b, const void *priv) diff --git a/tools/testing/selftests/bpf/progs/kprobe_multi.c b/tools/testing/selftests/bpf/progs/kprobe_multi.c index 08f95a8155d1..98c3399e15c0 100644 --- a/tools/testing/selftests/bpf/progs/kprobe_multi.c +++ b/tools/testing/selftests/bpf/progs/kprobe_multi.c @@ -36,15 +36,13 @@ __u64 kretprobe_test6_result = 0; __u64 kretprobe_test7_result = 0; __u64 kretprobe_test8_result = 0; -extern bool CONFIG_X86_KERNEL_IBT __kconfig __weak; - static void kprobe_multi_check(void *ctx, bool is_return) { if (bpf_get_current_pid_tgid() >> 32 != pid) return; __u64 cookie = test_cookie ? bpf_get_attach_cookie(ctx) : 0; - __u64 addr = bpf_get_func_ip(ctx) - (CONFIG_X86_KERNEL_IBT ? 4 : 0); + __u64 addr = bpf_get_func_ip(ctx); #define SET(__var, __addr, __cookie) ({ \ if (((const void *) addr == __addr) && \ From patchwork Thu Sep 22 21:03:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12985803 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21225C54EE9 for ; Thu, 22 Sep 2022 21:05:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229759AbiIVVFB (ORCPT ); Thu, 22 Sep 2022 17:05:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230353AbiIVVE6 (ORCPT ); Thu, 22 Sep 2022 17:04:58 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 76A9687086 for ; Thu, 22 Sep 2022 14:04:56 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 85E4562D46 for ; Thu, 22 Sep 2022 21:04:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 11D8AC433D7; Thu, 22 Sep 2022 21:04:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1663880694; bh=e8Nm5c5eqOXG/hFEZ2bBwEbQQdcrnXwYJva1UH1Kigk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rvDfFEnw5Lk9G8kDPcTt+IRIcZhMv1ZQJ5JhCUFXohqXbccNaMJwwZUo65/9qnSsB nz0CnJk2JqBYjWzJW5TFUC6cKC3vFz1jyQq32Qv1jjLbaCrIRaLMz7isB0cwZkq7X1 1DqFE/hSBP63MUrq0ySx+0xMa9qZ8D4EmAev5oGlo6VnCWCqRyg+TfMiv4LaONpNMs keYS8usSLcP0XufJhvS18XSqQ2E/cOPyjBYe5W7dKlNMKzq4+Lj0Zdd3X3Sx0zz0Or TKZsGvrt0zyYNnA0RpQHHOUTuLlxtoAvWaOsD197Q1k1HJuwgYpj5B8pR1EZPDtBlf lQiHLFQazUHVQ== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , "Masami Hiramatsu (Google)" , Peter Zijlstra , Martynas Pumputis Subject: [PATCHv4 bpf-next 5/6] bpf: Return value in kprobe get_func_ip only for entry address Date: Thu, 22 Sep 2022 23:03:19 +0200 Message-Id: <20220922210320.1076658-6-jolsa@kernel.org> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220922210320.1076658-1-jolsa@kernel.org> References: <20220922210320.1076658-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Changing return value of kprobe's version of bpf_get_func_ip to return zero if the attach address is not on the function's entry point. For kprobes attached in the middle of the function we can't easily get to the function address especially now with the CONFIG_X86_KERNEL_IBT support. If user cares about current IP for kprobes attached within the function body, they can get it with PT_REGS_IP(ctx). Suggested-by: Andrii Nakryiko Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko Acked-by: Martynas Pumputis --- kernel/trace/bpf_trace.c | 5 ++++- tools/testing/selftests/bpf/progs/get_func_ip_test.c | 4 ++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index ebd1b348beb3..688552df95ca 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1048,7 +1048,10 @@ BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs) { struct kprobe *kp = kprobe_running(); - return kp ? (uintptr_t)kp->addr : 0; + if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY)) + return 0; + + return get_entry_ip((uintptr_t)kp->addr); } static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = { diff --git a/tools/testing/selftests/bpf/progs/get_func_ip_test.c b/tools/testing/selftests/bpf/progs/get_func_ip_test.c index a587aeca5ae0..6db70757bc8b 100644 --- a/tools/testing/selftests/bpf/progs/get_func_ip_test.c +++ b/tools/testing/selftests/bpf/progs/get_func_ip_test.c @@ -69,7 +69,7 @@ int test6(struct pt_regs *ctx) { __u64 addr = bpf_get_func_ip(ctx); - test6_result = (const void *) addr == &bpf_fentry_test6 + 5; + test6_result = (const void *) addr == 0; return 0; } @@ -79,6 +79,6 @@ int test7(struct pt_regs *ctx) { __u64 addr = bpf_get_func_ip(ctx); - test7_result = (const void *) addr == &bpf_fentry_test7 + 5; + test7_result = (const void *) addr == 0; return 0; } From patchwork Thu Sep 22 21:03:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12985804 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA9F9C54EE9 for ; Thu, 22 Sep 2022 21:05:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230295AbiIVVFR (ORCPT ); Thu, 22 Sep 2022 17:05:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230274AbiIVVFP (ORCPT ); Thu, 22 Sep 2022 17:05:15 -0400 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08A1FD5889 for ; Thu, 22 Sep 2022 14:05:12 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 84E8ACE2356 for ; Thu, 22 Sep 2022 21:05:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 32705C433D7; Thu, 22 Sep 2022 21:05:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1663880708; bh=e8bbtFYLVSbybF8R+o1g1uIyD4WWd53GoZ4LOoXZOgg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cnyfrnijQX2OXgqKIkkxSsVIlzlc5cQRXVBIDencHe7PacAC6pHrM0/I3nqmcGMd1 HylLib793OHa+SvkBj3wN/2oQOaJiwF+2ENgtQNSmFsmykpeNM32DfcrP1t2h1bR7w fFAGe93ZcJFUIGJbl7chnoAhGBkYCf8mLf7mVb1pMo9L2BYNVmL3OVU104vxYnUcUe e4Xed5c1L8Cw8TnxHCSoAsYJn/l9pSH1MRWVAsBYhB26Zs7PfvA1Agz2fK4WiNfK7t zLODpf8A/XLZn0aN1BRQjyCo+0maC7MO0MrJTWfpM3TxS2JcrLGQ+AzU0bLfNJn1sm DwfVx5e6OERZQ== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , "Masami Hiramatsu (Google)" , Peter Zijlstra , Martynas Pumputis Subject: [PATCHv4 bpf-next 6/6] selftests/bpf: Fix get_func_ip offset test for CONFIG_X86_KERNEL_IBT Date: Thu, 22 Sep 2022 23:03:20 +0200 Message-Id: <20220922210320.1076658-7-jolsa@kernel.org> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220922210320.1076658-1-jolsa@kernel.org> References: <20220922210320.1076658-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net With CONFIG_X86_KERNEL_IBT enabled the test for kprobe with offset won't work because of the extra endbr instruction. As suggested by Andrii adding CONFIG_X86_KERNEL_IBT detection and using appropriate offset value based on that. Also removing test7 program, because it does the same as test6. Suggested-by: Andrii Nakryiko Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- .../bpf/prog_tests/get_func_ip_test.c | 59 +++++++++++++++---- .../selftests/bpf/progs/get_func_ip_test.c | 23 ++++---- 2 files changed, 60 insertions(+), 22 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c b/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c index 938dbd4d7c2f..fede8ef58b5b 100644 --- a/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c +++ b/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c @@ -2,7 +2,7 @@ #include #include "get_func_ip_test.skel.h" -void test_get_func_ip_test(void) +static void test_function_entry(void) { struct get_func_ip_test *skel = NULL; int err, prog_fd; @@ -12,14 +12,6 @@ void test_get_func_ip_test(void) if (!ASSERT_OK_PTR(skel, "get_func_ip_test__open")) return; - /* test6 is x86_64 specifc because of the instruction - * offset, disabling it for all other archs - */ -#ifndef __x86_64__ - bpf_program__set_autoload(skel->progs.test6, false); - bpf_program__set_autoload(skel->progs.test7, false); -#endif - err = get_func_ip_test__load(skel); if (!ASSERT_OK(err, "get_func_ip_test__load")) goto cleanup; @@ -43,11 +35,56 @@ void test_get_func_ip_test(void) ASSERT_EQ(skel->bss->test3_result, 1, "test3_result"); ASSERT_EQ(skel->bss->test4_result, 1, "test4_result"); ASSERT_EQ(skel->bss->test5_result, 1, "test5_result"); + +cleanup: + get_func_ip_test__destroy(skel); +} + +/* test6 is x86_64 specific because of the instruction + * offset, disabling it for all other archs + */ #ifdef __x86_64__ +static void test_function_body(void) +{ + struct get_func_ip_test *skel = NULL; + LIBBPF_OPTS(bpf_test_run_opts, topts); + LIBBPF_OPTS(bpf_kprobe_opts, kopts); + struct bpf_link *link6 = NULL; + int err, prog_fd; + + skel = get_func_ip_test__open(); + if (!ASSERT_OK_PTR(skel, "get_func_ip_test__open")) + return; + + bpf_program__set_autoload(skel->progs.test6, true); + + err = get_func_ip_test__load(skel); + if (!ASSERT_OK(err, "get_func_ip_test__load")) + goto cleanup; + + kopts.offset = skel->kconfig->CONFIG_X86_KERNEL_IBT ? 9 : 5; + + link6 = bpf_program__attach_kprobe_opts(skel->progs.test6, "bpf_fentry_test6", &kopts); + if (!ASSERT_OK_PTR(link6, "link6")) + goto cleanup; + + prog_fd = bpf_program__fd(skel->progs.test1); + err = bpf_prog_test_run_opts(prog_fd, &topts); + ASSERT_OK(err, "test_run"); + ASSERT_EQ(topts.retval, 0, "test_run"); + ASSERT_EQ(skel->bss->test6_result, 1, "test6_result"); - ASSERT_EQ(skel->bss->test7_result, 1, "test7_result"); -#endif cleanup: + bpf_link__destroy(link6); get_func_ip_test__destroy(skel); } +#else +#define test_function_body() +#endif + +void test_get_func_ip_test(void) +{ + test_function_entry(); + test_function_body(); +} diff --git a/tools/testing/selftests/bpf/progs/get_func_ip_test.c b/tools/testing/selftests/bpf/progs/get_func_ip_test.c index 6db70757bc8b..8559e698b40d 100644 --- a/tools/testing/selftests/bpf/progs/get_func_ip_test.c +++ b/tools/testing/selftests/bpf/progs/get_func_ip_test.c @@ -2,6 +2,7 @@ #include #include #include +#include char _license[] SEC("license") = "GPL"; @@ -13,6 +14,16 @@ extern const void bpf_modify_return_test __ksym; extern const void bpf_fentry_test6 __ksym; extern const void bpf_fentry_test7 __ksym; +extern bool CONFIG_X86_KERNEL_IBT __kconfig __weak; + +/* This function is here to have CONFIG_X86_KERNEL_IBT + * used and added to object BTF. + */ +int unused(void) +{ + return CONFIG_X86_KERNEL_IBT ? 0 : 1; +} + __u64 test1_result = 0; SEC("fentry/bpf_fentry_test1") int BPF_PROG(test1, int a) @@ -64,7 +75,7 @@ int BPF_PROG(test5, int a, int *b, int ret) } __u64 test6_result = 0; -SEC("kprobe/bpf_fentry_test6+0x5") +SEC("?kprobe") int test6(struct pt_regs *ctx) { __u64 addr = bpf_get_func_ip(ctx); @@ -72,13 +83,3 @@ int test6(struct pt_regs *ctx) test6_result = (const void *) addr == 0; return 0; } - -__u64 test7_result = 0; -SEC("kprobe/bpf_fentry_test7+5") -int test7(struct pt_regs *ctx) -{ - __u64 addr = bpf_get_func_ip(ctx); - - test7_result = (const void *) addr == 0; - return 0; -}