From patchwork Tue Feb 22 17:05:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12755693 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BDCFC433EF for ; Tue, 22 Feb 2022 17:07:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234550AbiBVRHc (ORCPT ); Tue, 22 Feb 2022 12:07:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234545AbiBVRHb (ORCPT ); Tue, 22 Feb 2022 12:07:31 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CEDBDDE84; Tue, 22 Feb 2022 09:07:05 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D95F160F89; Tue, 22 Feb 2022 17:07:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 671A4C36AEB; Tue, 22 Feb 2022 17:06:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1645549624; bh=7gGtwxVWHRIA9qaPB3Q5gClXDbN5Gk7FLicmyv8ugAM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pmDa8PxQiE/gifqFM9N2SQADv5cyLNTBgNBio/yCR19/GLZd/GAb0kO66aro5D5NS 7PC11NJZS8d0DQ+HZvX1Q+Vz41kYtOn3L4IlCTOUTfu43rLPsdtwUW55q/zWAzs1nW ZQ4wpZrQZ2Yi37VUjZ2ViFpH5kPUaH9L0f2WEa/s3uzOLx9b0d3RRmDc6XK6ASOLnk 9HdlZB6oTAwZJOC0TWB50x5v4Y7O+NYcoE+tWo0ZKh0hdmhaS0/vg1ujQYfICpJWZJ yTI1UbKLjoeejkpzNWacxUSCWc8Lnc02aPDJRCJetgJdsUk13bMRh50+gBiCJY+SM8 zCGaym35TYntg== From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt Subject: [PATCH 04/10] bpf: Add support to inline bpf_get_func_ip helper on x86 Date: Tue, 22 Feb 2022 18:05:54 +0100 Message-Id: <20220222170600.611515-5-jolsa@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220222170600.611515-1-jolsa@kernel.org> References: <20220222170600.611515-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding support to inline it on x86, because it's single load instruction. Signed-off-by: Jiri Olsa --- kernel/bpf/verifier.c | 21 ++++++++++++++++++++- kernel/trace/bpf_trace.c | 1 + 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index d7473fee247c..f125c33a37c9 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -13635,7 +13635,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env) continue; } - /* Implement bpf_get_func_ip inline. */ + /* Implement tracing bpf_get_func_ip inline. */ if (prog_type == BPF_PROG_TYPE_TRACING && insn->imm == BPF_FUNC_get_func_ip) { /* Load IP address from ctx - 16 */ @@ -13650,6 +13650,25 @@ static int do_misc_fixups(struct bpf_verifier_env *env) continue; } +#ifdef CONFIG_X86 + /* Implement kprobe_multi bpf_get_func_ip inline. */ + if (prog_type == BPF_PROG_TYPE_KPROBE && + eatype == BPF_TRACE_KPROBE_MULTI && + insn->imm == BPF_FUNC_get_func_ip) { + /* Load IP address from ctx (struct pt_regs) ip */ + insn_buf[0] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, + offsetof(struct pt_regs, ip)); + + new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, 1); + if (!new_prog) + return -ENOMEM; + + env->prog = prog = new_prog; + insn = new_prog->insnsi + i + delta; + continue; + } +#endif + patch_call_imm: fn = env->ops->get_func_proto(insn->imm, env->prog); /* all functions that have prototype and verifier allowed diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 64891b7b0885..c1998b9d5531 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1039,6 +1039,7 @@ static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = { BPF_CALL_1(bpf_get_func_ip_kprobe_multi, struct pt_regs *, regs) { + /* This helper call is inlined by verifier on x86. */ return instruction_pointer(regs); }