From patchwork Wed Jan 26 21:48:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 12725855 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 329D3C433EF for ; Wed, 26 Jan 2022 21:48:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230126AbiAZVsb (ORCPT ); Wed, 26 Jan 2022 16:48:31 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:34136 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232750AbiAZVsb (ORCPT ); Wed, 26 Jan 2022 16:48:31 -0500 Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 20QL2Ble026279 for ; Wed, 26 Jan 2022 13:48:31 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=h6bW2gHsSqURPzdA6QliBuoHm5HqRZFE6c440MJPem0=; b=W9UBNbzrcOuPN2qXPaVWlM4T5qXfk+2BHYxMAwUqyF3ZEtDGMf4/lBv3VxIcf5cB4Ff9 Th6zCXtJYBxYwn62GdqFFrvJLrsaR3Smlm1+5nOGSK2w7/EQ41l9NX0SihI4bmVA9k15 3whK/HX3czLVznD04NqW/0gL7w7eWAIw2Gg= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3dts3ef77s-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 26 Jan 2022 13:48:30 -0800 Received: from twshared2974.18.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::e) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 26 Jan 2022 13:48:28 -0800 Received: by devvm1744.ftw0.facebook.com (Postfix, from userid 460691) id 2699B2C9AA2B; Wed, 26 Jan 2022 13:48:16 -0800 (PST) From: Kui-Feng Lee To: , , , CC: Kui-Feng Lee Subject: [PATCH bpf-next 1/5] bpf: Add a flags value on trampoline frames. Date: Wed, 26 Jan 2022 13:48:05 -0800 Message-ID: <20220126214809.3868787-2-kuifeng@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220126214809.3868787-1-kuifeng@fb.com> References: <20220126214809.3868787-1-kuifeng@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: 7NbRflsBSnwSE-xh9PizGTIPcF_X03wT X-Proofpoint-GUID: 7NbRflsBSnwSE-xh9PizGTIPcF_X03wT X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-01-26_08,2022-01-26_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=fb_outbound_notspam policy=fb_outbound score=0 impostorscore=0 malwarescore=0 clxscore=1015 adultscore=0 mlxlogscore=804 phishscore=0 suspectscore=0 mlxscore=0 lowpriorityscore=0 priorityscore=1501 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2201260126 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The flags indicate what values are below nargs. It appears only if one or more values are there. The order in the flags, from LSB to MSB, indicates the order of values in the trampoline frame. LSB is at the highest location, close to the flags and nargs. Signed-off-by: Kui-Feng Lee --- arch/x86/net/bpf_jit_comp.c | 17 ++++++++++++++++- kernel/bpf/verifier.c | 2 +- kernel/trace/bpf_trace.c | 23 ++++++++++++++++++++++- 3 files changed, 39 insertions(+), 3 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index ce1f86f245c9..1c2d3ef57dad 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -1976,7 +1976,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i void *orig_call) { int ret, i, nr_args = m->nr_args; - int regs_off, ip_off, args_off, stack_size = nr_args * 8; + int regs_off, flags_off, ip_off, args_off, stack_size = nr_args * 8; struct bpf_tramp_progs *fentry = &tprogs[BPF_TRAMP_FENTRY]; struct bpf_tramp_progs *fexit = &tprogs[BPF_TRAMP_FEXIT]; struct bpf_tramp_progs *fmod_ret = &tprogs[BPF_TRAMP_MODIFY_RETURN]; @@ -2019,6 +2019,11 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i stack_size += 8; args_off = stack_size; + if (flags & BPF_TRAMP_F_IP_ARG) + stack_size += 8; /* room for flags */ + + flags_off = stack_size; + if (flags & BPF_TRAMP_F_IP_ARG) stack_size += 8; /* room for IP address argument */ @@ -2044,6 +2049,16 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i emit_mov_imm64(&prog, BPF_REG_0, 0, (u32) nr_args); emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -args_off); + if (flags & BPF_TRAMP_F_IP_ARG) { + /* Store flags + * move rax, flags + * mov QWORD PTR [rbp - flags_off], rax + */ + emit_mov_imm64(&prog, BPF_REG_0, 0, + (u32) (flags & BPF_TRAMP_F_IP_ARG)); + emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -flags_off); + } + if (flags & BPF_TRAMP_F_IP_ARG) { /* Store IP address of the traced function: * mov rax, QWORD PTR [rbp + 8] diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 8c5a46d41f28..ff91f5038010 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -13585,7 +13585,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env) if (prog_type == BPF_PROG_TYPE_TRACING && insn->imm == BPF_FUNC_get_func_ip) { /* Load IP address from ctx - 16 */ - insn_buf[0] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, -16); + insn_buf[0] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, -24); new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, 1); if (!new_prog) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 06a9e220069e..bf2c9d11ad05 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1009,10 +1009,31 @@ const struct bpf_func_proto bpf_snprintf_btf_proto = { .arg5_type = ARG_ANYTHING, }; +static int get_trampo_var_off(void *ctx, u32 flag) +{ + int off = 2; /* All variables are placed before flags */ + u32 flags = (u32)((u64 *)ctx)[-2]; + + if (!(flags & flag)) + return -1; /* The variable is not there */ + if (flag & (flag - 1)) + return -1; /* 2 or more bits are set */ + + for (; flags & flag; flags &= flags - 1) + off++; + + return off; +} + BPF_CALL_1(bpf_get_func_ip_tracing, void *, ctx) { /* This helper call is inlined by verifier. */ - return ((u64 *)ctx)[-2]; + int off = get_trampo_var_off(ctx, BPF_TRAMP_F_IP_ARG); + + if (off < 0) + return 0; + + return ((u64 *)ctx)[-off]; } static const struct bpf_func_proto bpf_get_func_ip_proto_tracing = { From patchwork Wed Jan 26 21:48:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 12725853 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FD24C433F5 for ; Wed, 26 Jan 2022 21:48:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232744AbiAZVsX (ORCPT ); Wed, 26 Jan 2022 16:48:23 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:32004 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230126AbiAZVsX (ORCPT ); Wed, 26 Jan 2022 16:48:23 -0500 Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 20QL24wK005673 for ; Wed, 26 Jan 2022 13:48:23 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=vb1bqpvevvJw77qKF/pvSupfkGkXKoZ8kOt2CxtKzJY=; b=JpUHIGmfhngZXmnQ0jTMrK6Qo/x/IaD49W1ccypxgEiLfbK+AJdMCscAfE4/IYHQ8rZL o/blmzfr4KQx0ePv9trSXLMDWNfviCs2kQpE6YlsKsX1OdkLCtbo2KNpFGqV9jJhnxrM GqlUD9X+gKsTXB8nVdeTekmiLOl02qWYXQ4= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3dtguhu02v-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 26 Jan 2022 13:48:22 -0800 Received: from twshared18912.14.frc2.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 26 Jan 2022 13:48:21 -0800 Received: by devvm1744.ftw0.facebook.com (Postfix, from userid 460691) id 278812C9AA2C; Wed, 26 Jan 2022 13:48:16 -0800 (PST) From: Kui-Feng Lee To: , , , CC: Kui-Feng Lee Subject: [PATCH bpf-next 2/5] bpf: Detect if a program needs its program ID. Date: Wed, 26 Jan 2022 13:48:06 -0800 Message-ID: <20220126214809.3868787-3-kuifeng@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220126214809.3868787-1-kuifeng@fb.com> References: <20220126214809.3868787-1-kuifeng@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: AM2kU1n6Q4jmL4IwTP94go5HDeM9o3fP X-Proofpoint-ORIG-GUID: AM2kU1n6Q4jmL4IwTP94go5HDeM9o3fP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-01-26_08,2022-01-26_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=fb_outbound_notspam policy=fb_outbound score=0 malwarescore=0 suspectscore=0 spamscore=0 phishscore=0 adultscore=0 bulkscore=0 lowpriorityscore=0 priorityscore=1501 clxscore=1015 mlxscore=0 impostorscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2201260126 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Scan BPF bytecode to detect if a program calls any functions requiring a program ID. So far, bpf_get_attach_cookie() is the only function that needs a program ID. Signed-off-by: Kui-Feng Lee --- include/linux/filter.h | 3 ++- kernel/bpf/trampoline.c | 7 ++++--- kernel/bpf/verifier.c | 3 +++ 3 files changed, 9 insertions(+), 4 deletions(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index d23e999dc032..4433c5d1bc19 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -572,7 +572,8 @@ struct bpf_prog { has_callchain_buf:1, /* callchain buffer allocated? */ enforce_expected_attach_type:1, /* Enforce expected_attach_type checking at attach time */ call_get_stack:1, /* Do we call bpf_get_stack() or bpf_get_stackid() */ - call_get_func_ip:1; /* Do we call get_func_ip() */ + call_get_func_ip:1, /* Do we call get_func_ip() */ + need_prog_id:1; /* Do we need program ID? */ enum bpf_prog_type type; /* Type of BPF program */ enum bpf_attach_type expected_attach_type; /* For some prog types */ u32 len; /* Number of filter blocks */ diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c index 4b6974a195c1..c65622e4216c 100644 --- a/kernel/bpf/trampoline.c +++ b/kernel/bpf/trampoline.c @@ -181,7 +181,7 @@ static int register_fentry(struct bpf_trampoline *tr, void *new_addr) } static struct bpf_tramp_progs * -bpf_trampoline_get_progs(const struct bpf_trampoline *tr, int *total, bool *ip_arg) +bpf_trampoline_get_progs(const struct bpf_trampoline *tr, int *total, bool *ip_arg, bool *prog_id) { const struct bpf_prog_aux *aux; struct bpf_tramp_progs *tprogs; @@ -200,6 +200,7 @@ bpf_trampoline_get_progs(const struct bpf_trampoline *tr, int *total, bool *ip_a hlist_for_each_entry(aux, &tr->progs_hlist[kind], tramp_hlist) { *ip_arg |= aux->prog->call_get_func_ip; + *prog_id |= aux->prog->need_prog_id; *progs++ = aux->prog; } } @@ -344,10 +345,10 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr) struct bpf_tramp_image *im; struct bpf_tramp_progs *tprogs; u32 flags = BPF_TRAMP_F_RESTORE_REGS; - bool ip_arg = false; + bool ip_arg = false, prog_id = false; int err, total; - tprogs = bpf_trampoline_get_progs(tr, &total, &ip_arg); + tprogs = bpf_trampoline_get_progs(tr, &total, &ip_arg, &prog_id); if (IS_ERR(tprogs)) return PTR_ERR(tprogs); diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index ff91f5038010..0359242e2a81 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -6812,6 +6812,9 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn env->prog->call_get_func_ip = true; } + if (func_id == BPF_FUNC_get_attach_cookie) + env->prog->need_prog_id = true; + if (changes_data) clear_all_pkt_pointers(env); return 0; From patchwork Wed Jan 26 21:48:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 12725858 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 345D7C433F5 for ; Wed, 26 Jan 2022 21:48:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232762AbiAZVsm (ORCPT ); Wed, 26 Jan 2022 16:48:42 -0500 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:20014 "EHLO mx0b-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232753AbiAZVsl (ORCPT ); Wed, 26 Jan 2022 16:48:41 -0500 Received: from pps.filterd (m0109332.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 20QL28mD021130 for ; Wed, 26 Jan 2022 13:48:41 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=hxivEcICclmHFb3RmBIwDP138VPBXCk9Ldy0H+1ovqA=; b=YpRpNwieo2pzqhyyKRYfGuyluzpAHHJ2gp8v0NcNlG/jHl9Q8zReaZZTBm7jRCCeTQO5 CK9Vi9fdm4Kdlao9uED2TCuDzVhO0nHmIP7SlWojkjUi+QRmfIkIBhDzI8LxHhfDqOaH Tv5liBM0O1VhvTzJCG3vYSxJOrdh4b6ex78= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3dtrpf7f65-11 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 26 Jan 2022 13:48:41 -0800 Received: from twshared3205.02.ash9.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 26 Jan 2022 13:48:34 -0800 Received: by devvm1744.ftw0.facebook.com (Postfix, from userid 460691) id 284622C9AA2D; Wed, 26 Jan 2022 13:48:16 -0800 (PST) From: Kui-Feng Lee To: , , , CC: Kui-Feng Lee Subject: [PATCH bpf-next 3/5] bpf, x86: Store program ID to trampoline frames. Date: Wed, 26 Jan 2022 13:48:07 -0800 Message-ID: <20220126214809.3868787-4-kuifeng@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220126214809.3868787-1-kuifeng@fb.com> References: <20220126214809.3868787-1-kuifeng@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: aeitscmT7yWBz4o1LrLVVUpABNcR2npZ X-Proofpoint-ORIG-GUID: aeitscmT7yWBz4o1LrLVVUpABNcR2npZ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-01-26_08,2022-01-26_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=fb_outbound_notspam policy=fb_outbound score=0 impostorscore=0 clxscore=1015 spamscore=0 priorityscore=1501 phishscore=0 mlxlogscore=958 malwarescore=0 mlxscore=0 lowpriorityscore=0 bulkscore=0 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2201260126 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Store the program ID in the trampoline frame before calling a BPF program if any of BPF programs called by the trampoline need its program ID, i.e., need_prog_id is true. Signed-off-by: Kui-Feng Lee --- arch/x86/net/bpf_jit_comp.c | 44 +++++++++++++++++++++++++++---------- include/linux/bpf.h | 2 ++ kernel/bpf/trampoline.c | 3 +++ 3 files changed, 37 insertions(+), 12 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 1c2d3ef57dad..98ed42215887 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -1754,13 +1754,21 @@ static void restore_regs(const struct btf_func_model *m, u8 **prog, int nr_args, } static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog, - struct bpf_prog *p, int stack_size, bool save_ret) + struct bpf_prog *p, int stack_size, + int pgid_off, bool save_ret) { u8 *prog = *pprog; u8 *jmp_insn; /* arg1: mov rdi, progs[i] */ emit_mov_imm64(&prog, BPF_REG_1, (long) p >> 32, (u32) (long) p); + + /* Store BPF program ID + * mov QWORD PTR [rbp - pgid_off], rdi + */ + if (pgid_off > 0) + emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_1, -pgid_off); + if (emit_call(&prog, p->aux->sleepable ? __bpf_prog_enter_sleepable : __bpf_prog_enter, prog)) @@ -1843,14 +1851,14 @@ static int emit_cond_near_jump(u8 **pprog, void *func, void *ip, u8 jmp_cond) static int invoke_bpf(const struct btf_func_model *m, u8 **pprog, struct bpf_tramp_progs *tp, int stack_size, - bool save_ret) + int pgid_off, bool save_ret) { int i; u8 *prog = *pprog; for (i = 0; i < tp->nr_progs; i++) { if (invoke_bpf_prog(m, &prog, tp->progs[i], stack_size, - save_ret)) + pgid_off, save_ret)) return -EINVAL; } *pprog = prog; @@ -1859,7 +1867,7 @@ static int invoke_bpf(const struct btf_func_model *m, u8 **pprog, static int invoke_bpf_mod_ret(const struct btf_func_model *m, u8 **pprog, struct bpf_tramp_progs *tp, int stack_size, - u8 **branches) + int pgid_off, u8 **branches) { u8 *prog = *pprog; int i; @@ -1870,7 +1878,8 @@ static int invoke_bpf_mod_ret(const struct btf_func_model *m, u8 **pprog, emit_mov_imm32(&prog, false, BPF_REG_0, 0); emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -8); for (i = 0; i < tp->nr_progs; i++) { - if (invoke_bpf_prog(m, &prog, tp->progs[i], stack_size, true)) + if (invoke_bpf_prog(m, &prog, tp->progs[i], stack_size, + pgid_off, true)) return -EINVAL; /* mod_ret prog stored return value into [rbp - 8]. Emit: @@ -1976,7 +1985,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i void *orig_call) { int ret, i, nr_args = m->nr_args; - int regs_off, flags_off, ip_off, args_off, stack_size = nr_args * 8; + int regs_off, flags_off, pgid_off = 0, ip_off, args_off, stack_size = nr_args * 8; struct bpf_tramp_progs *fentry = &tprogs[BPF_TRAMP_FENTRY]; struct bpf_tramp_progs *fexit = &tprogs[BPF_TRAMP_FEXIT]; struct bpf_tramp_progs *fmod_ret = &tprogs[BPF_TRAMP_MODIFY_RETURN]; @@ -2005,7 +2014,12 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i * * RBP - args_off [ args count ] always * + * RBP - flags_off [ flags ] BPF_TRAMP_F_IP_ARG or + * BPF_TRAMP_F_PROG_ID flags. + * * RBP - ip_off [ traced function ] BPF_TRAMP_F_IP_ARG flag + * + * RBP - pgid_off [ program ID ] BPF_TRAMP_F_PROG_ID flags. */ /* room for return value of orig_call or fentry prog */ @@ -2019,7 +2033,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i stack_size += 8; args_off = stack_size; - if (flags & BPF_TRAMP_F_IP_ARG) + if (flags & (BPF_TRAMP_F_IP_ARG | BPF_TRAMP_F_PROG_ID)) stack_size += 8; /* room for flags */ flags_off = stack_size; @@ -2029,6 +2043,12 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i ip_off = stack_size; + /* room for program ID */ + if (flags & BPF_TRAMP_F_PROG_ID) { + stack_size += 8; + pgid_off = stack_size; + } + if (flags & BPF_TRAMP_F_SKIP_FRAME) /* skip patched call instruction and point orig_call to actual * body of the kernel function. @@ -2049,13 +2069,13 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i emit_mov_imm64(&prog, BPF_REG_0, 0, (u32) nr_args); emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -args_off); - if (flags & BPF_TRAMP_F_IP_ARG) { + if (flags & (BPF_TRAMP_F_IP_ARG | BPF_TRAMP_F_PROG_ID)) { /* Store flags * move rax, flags * mov QWORD PTR [rbp - flags_off], rax */ emit_mov_imm64(&prog, BPF_REG_0, 0, - (u32) (flags & BPF_TRAMP_F_IP_ARG)); + (u32) (flags & (BPF_TRAMP_F_IP_ARG | BPF_TRAMP_F_PROG_ID))); emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -flags_off); } @@ -2082,7 +2102,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i } if (fentry->nr_progs) - if (invoke_bpf(m, &prog, fentry, regs_off, + if (invoke_bpf(m, &prog, fentry, regs_off, pgid_off, flags & BPF_TRAMP_F_RET_FENTRY_RET)) return -EINVAL; @@ -2092,7 +2112,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i if (!branches) return -ENOMEM; - if (invoke_bpf_mod_ret(m, &prog, fmod_ret, regs_off, + if (invoke_bpf_mod_ret(m, &prog, fmod_ret, regs_off, pgid_off, branches)) { ret = -EINVAL; goto cleanup; @@ -2130,7 +2150,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i } if (fexit->nr_progs) - if (invoke_bpf(m, &prog, fexit, regs_off, false)) { + if (invoke_bpf(m, &prog, fexit, regs_off, pgid_off, false)) { ret = -EINVAL; goto cleanup; } diff --git a/include/linux/bpf.h b/include/linux/bpf.h index e8ec8d2f2fe3..37353745fee5 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -656,6 +656,8 @@ struct btf_func_model { #define BPF_TRAMP_F_IP_ARG BIT(3) /* Return the return value of fentry prog. Only used by bpf_struct_ops. */ #define BPF_TRAMP_F_RET_FENTRY_RET BIT(4) +/* Store BPF program ID on the trampoline stack. */ +#define BPF_TRAMP_F_PROG_ID BIT(5) /* Each call __bpf_prog_enter + call bpf_func + call __bpf_prog_exit is ~50 * bytes on x86. Pick a number to fit into BPF_IMAGE_SIZE / 2 diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c index c65622e4216c..959a33619b36 100644 --- a/kernel/bpf/trampoline.c +++ b/kernel/bpf/trampoline.c @@ -373,6 +373,9 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr) if (ip_arg) flags |= BPF_TRAMP_F_IP_ARG; + if (prog_id) + flags |= BPF_TRAMP_F_PROG_ID; + err = arch_prepare_bpf_trampoline(im, im->image, im->image + PAGE_SIZE, &tr->func.model, flags, tprogs, tr->func.addr); From patchwork Wed Jan 26 21:48:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 12725857 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F074C4332F for ; Wed, 26 Jan 2022 21:48:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232773AbiAZVsd (ORCPT ); Wed, 26 Jan 2022 16:48:33 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:56982 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232753AbiAZVsd (ORCPT ); Wed, 26 Jan 2022 16:48:33 -0500 Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 20QL2CSo026288 for ; Wed, 26 Jan 2022 13:48:33 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=G+L5UcpNoW1pJd0B+pXYwY+knt0rLNsTVuTun1DhQ6k=; b=oUCZQIG1CPGS+25YP0j5YZOP8aISRU9si0XJtBxMs7aAHP2iYjSE2s+tapwo/stxZK9P 7mLLiGDr8kgmzQrXzEl0dzSRIcTnq5MG3yK6VAlnivhhUewILG+LOEoC/oVGzy7fmH21 FPs5w/VZiGSCjrs3WyZcHMBQtYqCZ+XcCUM= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3dts3ef786-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 26 Jan 2022 13:48:32 -0800 Received: from twshared3115.02.ash8.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:11d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 26 Jan 2022 13:48:31 -0800 Received: by devvm1744.ftw0.facebook.com (Postfix, from userid 460691) id 290E12C9AA2E; Wed, 26 Jan 2022 13:48:16 -0800 (PST) From: Kui-Feng Lee To: , , , CC: Kui-Feng Lee Subject: [PATCH bpf-next 4/5] bpf: Attach a cookie to a BPF program. Date: Wed, 26 Jan 2022 13:48:08 -0800 Message-ID: <20220126214809.3868787-5-kuifeng@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220126214809.3868787-1-kuifeng@fb.com> References: <20220126214809.3868787-1-kuifeng@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: 7N2poUs50HJgFYT4NSaNxRI_GvhXMBdE X-Proofpoint-GUID: 7N2poUs50HJgFYT4NSaNxRI_GvhXMBdE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-01-26_08,2022-01-26_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=fb_outbound_notspam policy=fb_outbound score=0 impostorscore=0 malwarescore=0 clxscore=1015 adultscore=0 mlxlogscore=909 phishscore=0 suspectscore=0 mlxscore=0 lowpriorityscore=0 priorityscore=1501 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2201260126 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Extend bpf() to attach a tracing program with a cookie by adding a bpf_cookie field to bpf_attr for raw tracepoints. Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 12 ++++++++---- tools/include/uapi/linux/bpf.h | 1 + tools/lib/bpf/bpf.c | 14 ++++++++++++++ tools/lib/bpf/bpf.h | 1 + tools/lib/bpf/libbpf.map | 1 + 7 files changed, 27 insertions(+), 4 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 37353745fee5..d5196514e9bd 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1004,6 +1004,7 @@ struct bpf_prog_aux { struct work_struct work; struct rcu_head rcu; }; + u64 cookie; }; struct bpf_array_aux { diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 16a7574292a5..3fa27346ab4b 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1425,6 +1425,7 @@ union bpf_attr { struct { /* anonymous struct used by BPF_RAW_TRACEPOINT_OPEN command */ __u64 name; __u32 prog_fd; + __u64 bpf_cookie; } raw_tracepoint; struct { /* anonymous struct for BPF_BTF_LOAD */ diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 72ce1edde950..79d057918c76 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -2696,7 +2696,8 @@ static const struct bpf_link_ops bpf_tracing_link_lops = { static int bpf_tracing_prog_attach(struct bpf_prog *prog, int tgt_prog_fd, - u32 btf_id) + u32 btf_id, + u64 bpf_cookie) { struct bpf_link_primer link_primer; struct bpf_prog *tgt_prog = NULL; @@ -2832,6 +2833,8 @@ static int bpf_tracing_prog_attach(struct bpf_prog *prog, if (err) goto out_unlock; + prog->aux->cookie = bpf_cookie; + err = bpf_trampoline_link_prog(prog, tr); if (err) { bpf_link_cleanup(&link_primer); @@ -3017,7 +3020,7 @@ static int bpf_perf_link_attach(const union bpf_attr *attr, struct bpf_prog *pro } #endif /* CONFIG_PERF_EVENTS */ -#define BPF_RAW_TRACEPOINT_OPEN_LAST_FIELD raw_tracepoint.prog_fd +#define BPF_RAW_TRACEPOINT_OPEN_LAST_FIELD raw_tracepoint.bpf_cookie static int bpf_raw_tracepoint_open(const union bpf_attr *attr) { @@ -3052,7 +3055,7 @@ static int bpf_raw_tracepoint_open(const union bpf_attr *attr) tp_name = prog->aux->attach_func_name; break; } - err = bpf_tracing_prog_attach(prog, 0, 0); + err = bpf_tracing_prog_attach(prog, 0, 0, attr->raw_tracepoint.bpf_cookie); if (err >= 0) return err; goto out_put_prog; @@ -4244,7 +4247,8 @@ static int tracing_bpf_link_attach(const union bpf_attr *attr, bpfptr_t uattr, else if (prog->type == BPF_PROG_TYPE_EXT) return bpf_tracing_prog_attach(prog, attr->link_create.target_fd, - attr->link_create.target_btf_id); + attr->link_create.target_btf_id, + 0); return -EINVAL; } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 16a7574292a5..3fa27346ab4b 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1425,6 +1425,7 @@ union bpf_attr { struct { /* anonymous struct used by BPF_RAW_TRACEPOINT_OPEN command */ __u64 name; __u32 prog_fd; + __u64 bpf_cookie; } raw_tracepoint; struct { /* anonymous struct for BPF_BTF_LOAD */ diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 418b259166f8..c28b017de515 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -1131,6 +1131,20 @@ int bpf_raw_tracepoint_open(const char *name, int prog_fd) return libbpf_err_errno(fd); } +int bpf_raw_tracepoint_cookie_open(const char *name, int prog_fd, __u64 bpf_cookie) +{ + union bpf_attr attr; + int fd; + + memset(&attr, 0, sizeof(attr)); + attr.raw_tracepoint.name = ptr_to_u64(name); + attr.raw_tracepoint.prog_fd = prog_fd; + attr.raw_tracepoint.bpf_cookie = bpf_cookie; + + fd = sys_bpf_fd(BPF_RAW_TRACEPOINT_OPEN, &attr, sizeof(attr)); + return libbpf_err_errno(fd); +} + int bpf_btf_load(const void *btf_data, size_t btf_size, const struct bpf_btf_load_opts *opts) { const size_t attr_sz = offsetofend(union bpf_attr, btf_log_level); diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h index c2e8327010f9..c3d2c6a4cb15 100644 --- a/tools/lib/bpf/bpf.h +++ b/tools/lib/bpf/bpf.h @@ -475,6 +475,7 @@ LIBBPF_API int bpf_prog_query(int target_fd, enum bpf_attach_type type, __u32 query_flags, __u32 *attach_flags, __u32 *prog_ids, __u32 *prog_cnt); LIBBPF_API int bpf_raw_tracepoint_open(const char *name, int prog_fd); +LIBBPF_API int bpf_raw_tracepoint_cookie_open(const char *name, int prog_fd, __u64 bpf_cookie); LIBBPF_API int bpf_task_fd_query(int pid, int fd, __u32 flags, char *buf, __u32 *buf_len, __u32 *prog_id, __u32 *fd_type, __u64 *probe_offset, __u64 *probe_addr); diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index e10f0822845a..05af5bb8de92 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -432,6 +432,7 @@ LIBBPF_0.7.0 { bpf_xdp_detach; bpf_xdp_query; bpf_xdp_query_id; + bpf_raw_tracepoint_cookie_open; libbpf_probe_bpf_helper; libbpf_probe_bpf_map_type; libbpf_probe_bpf_prog_type; From patchwork Wed Jan 26 21:48:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 12725856 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70A4EC433F5 for ; Wed, 26 Jan 2022 21:48:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232750AbiAZVsb (ORCPT ); Wed, 26 Jan 2022 16:48:31 -0500 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:9080 "EHLO mx0b-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232749AbiAZVsb (ORCPT ); Wed, 26 Jan 2022 16:48:31 -0500 Received: from pps.filterd (m0109331.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 20QL20g2014011 for ; Wed, 26 Jan 2022 13:48:31 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=nIWjqKa2lI4YV777ccTsgkjQeX8Z0g0p0BrOJIqKguc=; b=rIA9ri8YLvUQhRDfBtD1fQ3H8zXYjU53iRgthMiVXKPuVGsxsw1zfuwyplDZUcJ8epCa VShmJqVtAjXE6rTRFxjtmAwKoSs6TDVu27cX08/u3XnXbb70BPOzmNzVphYw3NHxPCH/ lL9RU8S7suYE7KUhpXCr/95CHG7TsKdUki8= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3dtvbex1da-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 26 Jan 2022 13:48:31 -0800 Received: from twshared29821.14.frc2.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:11d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 26 Jan 2022 13:48:27 -0800 Received: by devvm1744.ftw0.facebook.com (Postfix, from userid 460691) id 29EA52C9AA2F; Wed, 26 Jan 2022 13:48:16 -0800 (PST) From: Kui-Feng Lee To: , , , CC: Kui-Feng Lee Subject: [PATCH bpf-next 5/5] bpf: Implement bpf_get_attach_cookie() for tracing programs. Date: Wed, 26 Jan 2022 13:48:09 -0800 Message-ID: <20220126214809.3868787-6-kuifeng@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220126214809.3868787-1-kuifeng@fb.com> References: <20220126214809.3868787-1-kuifeng@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: 32e0w5qQeC3z7EDK8d4dKr4ccXac1R6d X-Proofpoint-ORIG-GUID: 32e0w5qQeC3z7EDK8d4dKr4ccXac1R6d X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-01-26_08,2022-01-26_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=fb_outbound_notspam policy=fb_outbound score=0 bulkscore=0 lowpriorityscore=0 phishscore=0 priorityscore=1501 spamscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 impostorscore=0 clxscore=1015 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2201260126 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Implement bpf_get_attach_cookie() for fentry, fexit, and fmod_ret tracing programs. The cookie is from the bpf_prog from the program ID of the current BPF program. Signed-off-by: Kui-Feng Lee Reported-by: kernel test robot --- kernel/trace/bpf_trace.c | 22 +++++++ .../selftests/bpf/prog_tests/bpf_cookie.c | 57 +++++++++++++++++++ .../selftests/bpf/progs/test_bpf_cookie.c | 24 ++++++++ 3 files changed, 103 insertions(+) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index bf2c9d11ad05..7b1f4c8a10de 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1084,6 +1084,26 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto_pe = { .arg1_type = ARG_PTR_TO_CTX, }; +BPF_CALL_1(bpf_get_attach_cookie_tracing, void *, ctx) +{ + const struct bpf_prog *prog; + int off = get_trampo_var_off(ctx, BPF_TRAMP_F_PROG_ID); + + if (off < 0) + return 0; + + prog = (const struct bpf_prog *)((u64 *)ctx)[-off]; + + return prog->aux->cookie; +} + +static const struct bpf_func_proto bpf_get_attach_cookie_proto_tracing = { + .func = bpf_get_attach_cookie_tracing, + .gpl_only = false, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_CTX, +}; + BPF_CALL_3(bpf_get_branch_snapshot, void *, buf, u32, size, u64, flags) { #ifndef CONFIG_X86 @@ -1706,6 +1726,8 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return bpf_prog_has_trampoline(prog) ? &bpf_get_func_ret_proto : NULL; case BPF_FUNC_get_func_arg_cnt: return bpf_prog_has_trampoline(prog) ? &bpf_get_func_arg_cnt_proto : NULL; + case BPF_FUNC_get_attach_cookie: + return bpf_prog_has_trampoline(prog) ? &bpf_get_attach_cookie_proto_tracing : NULL; default: fn = raw_tp_prog_func_proto(func_id, prog); if (!fn && prog->expected_attach_type == BPF_TRACE_ITER) diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c b/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c index 5eea3c3a40fe..41a73c0a3a8e 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c @@ -231,6 +231,61 @@ static void pe_subtest(struct test_bpf_cookie *skel) bpf_link__destroy(link); } +static void tracing_subtest(struct test_bpf_cookie *skel) +{ + __u64 cookie; + __u32 duration = 0, retval; + int prog_fd; + int fentry_fd = -1, fexit_fd = -1, fmod_ret_fd = -1; + + skel->bss->fentry_res = 0; + skel->bss->fexit_res = 0; + + cookie = 0x100000; + prog_fd = bpf_program__fd(skel->progs.fentry_test1); + if (!ASSERT_GE(prog_fd, 0, "fentry.prog_fd")) + return; + fentry_fd = bpf_raw_tracepoint_cookie_open(NULL, prog_fd, cookie); + if (!ASSERT_GE(fentry_fd, 0, "fentry.open")) + return; + + cookie = 0x200000; + prog_fd = bpf_program__fd(skel->progs.fexit_test1); + if (!ASSERT_GE(prog_fd, 0, "fexit.prog_fd")) + goto cleanup; + fexit_fd = bpf_raw_tracepoint_cookie_open(NULL, prog_fd, cookie); + if (!ASSERT_GE(fexit_fd, 0, "fexit.open")) + goto cleanup; + + cookie = 0x300000; + prog_fd = bpf_program__fd(skel->progs.fmod_ret_test); + if (!ASSERT_GE(prog_fd, 0, "fmod_ret.prog_fd")) + goto cleanup; + fmod_ret_fd = bpf_raw_tracepoint_cookie_open(NULL, prog_fd, cookie); + if (!ASSERT_GE(fmod_ret_fd, 0, "fmod_ret.opoen")) + goto cleanup; + + prog_fd = bpf_program__fd(skel->progs.fentry_test1); + bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + + prog_fd = bpf_program__fd(skel->progs.fmod_ret_test); + bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + + ASSERT_EQ(skel->bss->fentry_res, 0x100000, "fentry_res"); + ASSERT_EQ(skel->bss->fexit_res, 0x200000, "fexit_res"); + ASSERT_EQ(skel->bss->fmod_ret_res, 0x300000, "fmod_ret_res"); + +cleanup: + if (fentry_fd >= 0) + close(fentry_fd); + if (fexit_fd >= 0) + close(fexit_fd); + if (fmod_ret_fd >= 0) + close(fmod_ret_fd); +} + void test_bpf_cookie(void) { struct test_bpf_cookie *skel; @@ -249,6 +304,8 @@ void test_bpf_cookie(void) tp_subtest(skel); if (test__start_subtest("perf_event")) pe_subtest(skel); + if (test__start_subtest("tracing")) + tracing_subtest(skel); test_bpf_cookie__destroy(skel); } diff --git a/tools/testing/selftests/bpf/progs/test_bpf_cookie.c b/tools/testing/selftests/bpf/progs/test_bpf_cookie.c index 2d3a7710e2ce..a9f83f46e7b7 100644 --- a/tools/testing/selftests/bpf/progs/test_bpf_cookie.c +++ b/tools/testing/selftests/bpf/progs/test_bpf_cookie.c @@ -14,6 +14,9 @@ int uprobe_res; int uretprobe_res; int tp_res; int pe_res; +int fentry_res; +int fexit_res; +int fmod_ret_res; static void update(void *ctx, int *res) { @@ -82,4 +85,25 @@ int handle_pe(struct pt_regs *ctx) return 0; } +SEC("fentry/bpf_fentry_test1") +int BPF_PROG(fentry_test1, int a) +{ + update(ctx, &fentry_res); + return 0; +} + +SEC("fexit/bpf_fentry_test1") +int BPF_PROG(fexit_test1, int a, int ret) +{ + update(ctx, &fexit_res); + return 0; +} + +SEC("fmod_ret/bpf_modify_return_test") +int BPF_PROG(fmod_ret_test, int _a, int *_b, int _ret) +{ + update(ctx, &fmod_ret_res); + return 1234; +} + char _license[] SEC("license") = "GPL";