From patchwork Tue Jan 4 08:09:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703210 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE1B4C4332F for ; Tue, 4 Jan 2022 08:09:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233788AbiADIJy (ORCPT ); Tue, 4 Jan 2022 03:09:54 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:57215 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233811AbiADIJx (ORCPT ); Tue, 4 Jan 2022 03:09:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283793; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=deFFbtnYM3i/92CnuiUzL3EKyDWx93Ga+Yj6cLk2tzA=; b=XlJcEmcpeM5rR+erGWRS+dgNm1uW6n6+l0kNlRZPvaK5TxPUnKVqEhyuqT9yKMloexlvcq 6xCW4B6xdJzMgjYvvTsncgx1DwAqB7NSW+ZrPOrGVxcfgQypWgtVqWzjhRd9PcljIGLr5E NbbHabSQ2Y/ZmmowaK/D3eaaKbLKkUc= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-121-H0NB9X_vMaODOHpYMtySsA-1; Tue, 04 Jan 2022 03:09:52 -0500 X-MC-Unique: H0NB9X_vMaODOHpYMtySsA-1 Received: by mail-ed1-f72.google.com with SMTP id l14-20020aa7cace000000b003f7f8e1cbbdso24622212edt.20 for ; Tue, 04 Jan 2022 00:09:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=deFFbtnYM3i/92CnuiUzL3EKyDWx93Ga+Yj6cLk2tzA=; b=2Xjg1MpfCsFalJEIP3s+W5IlwVdxb2oXF5RWaHLVg6ppDmoRKeQWierH4k4jgts4cl xfrtVrjJ3l3NF1eT/HR2jJeXvrGXoHKOITcXrXjML9rsKPtT4kyz7W03KB+ob6q1Tzjy hLGJBVuKgsO7nUTccaV7BA4p8mQYDifqBhSebg9fXw6y8hPb03yU++ZGwmK02m5oti7I CX0lpZhSeq4JXCIWXpYGAFEjXeJ4Da/IUHKRs/ZtCwJefgvwDBeOhlTlzmYBjGPWtPBB UeUS/YJJpWq3/Ffd2Dg/kOCVPl7f0RzJiaYvkeMAa1ViUNgNChxmwpyjPUsI7g1GekxP qK8g== X-Gm-Message-State: AOAM5335DzkdISK0N42zKzMJrFz1wx3/dvivBtjjt4XdzojSSI34HpUd S4thld6JCsDBoE6g7zSJSCVBvbct8Fuq1GHBV/5IWT9TJmOpJlsbEgAZVr1rJe+QHFAvERJ8aLd 32JvMMzt2OF6yHNpk X-Received: by 2002:a17:906:8a43:: with SMTP id gx3mr37707284ejc.185.1641283790861; Tue, 04 Jan 2022 00:09:50 -0800 (PST) X-Google-Smtp-Source: ABdhPJxyNUpljah6tzm7i53XGrcK3Z5LHujpoocm6FeZnksRU3h8LpWhIH/IXYbvH/5LaWEcQ2c4dQ== X-Received: by 2002:a17:906:8a43:: with SMTP id gx3mr37707268ejc.185.1641283790690; Tue, 04 Jan 2022 00:09:50 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id go10sm11251843ejc.100.2022.01.04.00.09.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:09:50 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 01/13] ftrace: Add ftrace_set_filter_ips function Date: Tue, 4 Jan 2022 09:09:31 +0100 Message-Id: <20220104080943.113249-2-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding ftrace_set_filter_ips function to be able to set filter on multiple ip addresses at once. With the kprobe multi attach interface we have cases where we need to initialize ftrace_ops object with thousands of functions, so having single function diving into ftrace_hash_move_and_update_ops with ftrace_lock is faster. The functions ips are passed as unsigned long array with count. Signed-off-by: Jiri Olsa --- include/linux/ftrace.h | 3 +++ kernel/trace/ftrace.c | 53 +++++++++++++++++++++++++++++++++++------- 2 files changed, 47 insertions(+), 9 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 9999e29187de..60847cbce0da 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -512,6 +512,8 @@ struct dyn_ftrace { int ftrace_set_filter_ip(struct ftrace_ops *ops, unsigned long ip, int remove, int reset); +int ftrace_set_filter_ips(struct ftrace_ops *ops, unsigned long *ips, + unsigned int cnt, int remove, int reset); int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf, int len, int reset); int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf, @@ -802,6 +804,7 @@ static inline unsigned long ftrace_location(unsigned long ip) #define ftrace_regex_open(ops, flag, inod, file) ({ -ENODEV; }) #define ftrace_set_early_filter(ops, buf, enable) do { } while (0) #define ftrace_set_filter_ip(ops, ip, remove, reset) ({ -ENODEV; }) +#define ftrace_set_filter_ips(ops, ips, cnt, remove, reset) ({ -ENODEV; }) #define ftrace_set_filter(ops, buf, len, reset) ({ -ENODEV; }) #define ftrace_set_notrace(ops, buf, len, reset) ({ -ENODEV; }) #define ftrace_free_filter(ops) do { } while (0) diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index be5f6b32a012..39350aa38649 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -4958,7 +4958,7 @@ ftrace_notrace_write(struct file *file, const char __user *ubuf, } static int -ftrace_match_addr(struct ftrace_hash *hash, unsigned long ip, int remove) +__ftrace_match_addr(struct ftrace_hash *hash, unsigned long ip, int remove) { struct ftrace_func_entry *entry; @@ -4976,9 +4976,25 @@ ftrace_match_addr(struct ftrace_hash *hash, unsigned long ip, int remove) return add_hash_entry(hash, ip); } +static int +ftrace_match_addr(struct ftrace_hash *hash, unsigned long *ips, + unsigned int cnt, int remove) +{ + unsigned int i; + int err; + + for (i = 0; i < cnt; i++) { + err = __ftrace_match_addr(hash, ips[i], remove); + if (err) + return err; + } + return 0; +} + static int ftrace_set_hash(struct ftrace_ops *ops, unsigned char *buf, int len, - unsigned long ip, int remove, int reset, int enable) + unsigned long *ips, unsigned int cnt, + int remove, int reset, int enable) { struct ftrace_hash **orig_hash; struct ftrace_hash *hash; @@ -5008,8 +5024,8 @@ ftrace_set_hash(struct ftrace_ops *ops, unsigned char *buf, int len, ret = -EINVAL; goto out_regex_unlock; } - if (ip) { - ret = ftrace_match_addr(hash, ip, remove); + if (ips) { + ret = ftrace_match_addr(hash, ips, cnt, remove); if (ret < 0) goto out_regex_unlock; } @@ -5026,10 +5042,10 @@ ftrace_set_hash(struct ftrace_ops *ops, unsigned char *buf, int len, } static int -ftrace_set_addr(struct ftrace_ops *ops, unsigned long ip, int remove, - int reset, int enable) +ftrace_set_addr(struct ftrace_ops *ops, unsigned long *ips, unsigned int cnt, + int remove, int reset, int enable) { - return ftrace_set_hash(ops, NULL, 0, ip, remove, reset, enable); + return ftrace_set_hash(ops, NULL, 0, ips, cnt, remove, reset, enable); } #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS @@ -5634,10 +5650,29 @@ int ftrace_set_filter_ip(struct ftrace_ops *ops, unsigned long ip, int remove, int reset) { ftrace_ops_init(ops); - return ftrace_set_addr(ops, ip, remove, reset, 1); + return ftrace_set_addr(ops, &ip, 1, remove, reset, 1); } EXPORT_SYMBOL_GPL(ftrace_set_filter_ip); +/** + * ftrace_set_filter_ips - set a functions to filter on in ftrace by addresses + * @ops - the ops to set the filter with + * @ips - the array of addresses to add to or remove from the filter. + * @cnt - the number of addresses in @ips + * @remove - non zero to remove ips from the filter + * @reset - non zero to reset all filters before applying this filter. + * + * Filters denote which functions should be enabled when tracing is enabled + * If @ips array or any ip specified within is NULL , it fails to update filter. + */ +int ftrace_set_filter_ips(struct ftrace_ops *ops, unsigned long *ips, + unsigned int cnt, int remove, int reset) +{ + ftrace_ops_init(ops); + return ftrace_set_addr(ops, ips, cnt, remove, reset, 1); +} +EXPORT_SYMBOL_GPL(ftrace_set_filter_ips); + /** * ftrace_ops_set_global_filter - setup ops to use global filters * @ops - the ops which will use the global filters @@ -5659,7 +5694,7 @@ static int ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len, int reset, int enable) { - return ftrace_set_hash(ops, buf, len, 0, 0, reset, enable); + return ftrace_set_hash(ops, buf, len, NULL, 0, 0, reset, enable); } /** From patchwork Tue Jan 4 08:09:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703211 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07037C433F5 for ; Tue, 4 Jan 2022 08:10:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233829AbiADIKG (ORCPT ); Tue, 4 Jan 2022 03:10:06 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:36880 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233849AbiADIJ7 (ORCPT ); Tue, 4 Jan 2022 03:09:59 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283799; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wC25vJidXOSulDOMTh98GoQfpZN9ZJGJe/vhERr1/WY=; b=EE05GDFr+C+XzLeJEPOStvJs2gtdMByWDDZd2qJ0RlRoXk2NZvjU2Kq4WlQThiwi3VY2Iy prsFSuyPXupEqxOeZoPenTyfSHG2Hp5xq9TzDMbCsEAA1u9bWFlv1AfQFoUlehxTWIGGNl Q7RWK4ly68OnMOrh5acp8CojLb7QtWA= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-86-SWFN2s9UNhWVPwjA9-AuQQ-1; Tue, 04 Jan 2022 03:09:58 -0500 X-MC-Unique: SWFN2s9UNhWVPwjA9-AuQQ-1 Received: by mail-ed1-f71.google.com with SMTP id y10-20020a056402358a00b003f88b132849so24908512edc.0 for ; Tue, 04 Jan 2022 00:09:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wC25vJidXOSulDOMTh98GoQfpZN9ZJGJe/vhERr1/WY=; b=QKhIzLnCJ1wF4i8WM1TwF97+CqoxF0cni5N6dAr+LZJyMrKp6SpeBcym/9G1kAIkKK 3ekw4eBX/m0DzsOm8SX+HswUQkK7/Gl/6nwcfF2FhkrTBTZjzlBCulKedH/LUKf3Kl91 4bhN0+KUCGOX7VwhiUfkF8K0GgrVfl8fR9fPwAhwNZldwKlmk9yDxHv6OtfQfUa33UFJ 4N+IjTINK79/5pdGQxzw0h7tzEmz8IK954m49N5yDl+kse0tu0O/9TZEVrfbtAX3v1ey 2I8dg2+++HHTlaHvCG7+y3we5N4ALKSm+rlBiqQhKM5kYa8S5aMM1b4s+lU5A7T6/YSn CchA== X-Gm-Message-State: AOAM533kqUEL7IaESPJADOCvI4C/qqf+hiakpaYo0qqJBhJBqtfj7znb AjLTtWjf3Yk3NZvAzGhyvGS8Ln8LsRuTvD4crf3Nsh6dYt49dZigmPAd4nIMVQ+mZwCgV+XaOCo D3DDNjNKnOzHwI47A X-Received: by 2002:a17:907:82a6:: with SMTP id mr38mr37035509ejc.744.1641283796926; Tue, 04 Jan 2022 00:09:56 -0800 (PST) X-Google-Smtp-Source: ABdhPJxsV54Czj67uFIcbN98S4rvhYvKQh3lGyoyRpY5hJVl008UQ+Cuw2N++aiqMvvkhk7MQ3Geww== X-Received: by 2002:a17:907:82a6:: with SMTP id mr38mr37035499ejc.744.1641283796706; Tue, 04 Jan 2022 00:09:56 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id 19sm8284197ejv.207.2022.01.04.00.09.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:09:56 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 02/13] kprobe: Keep traced function address Date: Tue, 4 Jan 2022 09:09:32 +0100 Message-Id: <20220104080943.113249-3-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The bpf_get_func_ip_kprobe helper should return traced function address, but it's doing so only for kprobes that are placed on the function entry. If kprobe is placed within the function, bpf_get_func_ip_kprobe returns that address instead of function entry. Storing the function entry directly in kprobe object, so it could be used in bpf_get_func_ip_kprobe helper. Signed-off-by: Jiri Olsa --- include/linux/kprobes.h | 3 +++ kernel/kprobes.c | 12 ++++++++++++ kernel/trace/bpf_trace.c | 2 +- tools/testing/selftests/bpf/progs/get_func_ip_test.c | 4 ++-- 4 files changed, 18 insertions(+), 3 deletions(-) diff --git a/include/linux/kprobes.h b/include/linux/kprobes.h index 8c8f7a4d93af..a204df4fef96 100644 --- a/include/linux/kprobes.h +++ b/include/linux/kprobes.h @@ -74,6 +74,9 @@ struct kprobe { /* Offset into the symbol */ unsigned int offset; + /* traced function address */ + unsigned long func_addr; + /* Called before addr is executed. */ kprobe_pre_handler_t pre_handler; diff --git a/kernel/kprobes.c b/kernel/kprobes.c index d20ae8232835..c4060a8da050 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -1310,6 +1310,7 @@ static void init_aggr_kprobe(struct kprobe *ap, struct kprobe *p) copy_kprobe(p, ap); flush_insn_slot(ap); ap->addr = p->addr; + ap->func_addr = p->func_addr; ap->flags = p->flags & ~KPROBE_FLAG_OPTIMIZED; ap->pre_handler = aggr_pre_handler; /* We don't care the kprobe which has gone. */ @@ -1588,6 +1589,16 @@ static int check_kprobe_address_safe(struct kprobe *p, return ret; } +static unsigned long resolve_func_addr(kprobe_opcode_t *addr) +{ + char str[KSYM_SYMBOL_LEN]; + unsigned long offset; + + if (kallsyms_lookup((unsigned long) addr, NULL, &offset, NULL, str)) + return (unsigned long) addr - offset; + return 0; +} + int register_kprobe(struct kprobe *p) { int ret; @@ -1600,6 +1611,7 @@ int register_kprobe(struct kprobe *p) if (IS_ERR(addr)) return PTR_ERR(addr); p->addr = addr; + p->func_addr = resolve_func_addr(addr); ret = warn_kprobe_rereg(p); if (ret) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 21aa30644219..25631253084a 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1026,7 +1026,7 @@ BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs) { struct kprobe *kp = kprobe_running(); - return kp ? (uintptr_t)kp->addr : 0; + return kp ? (uintptr_t)kp->func_addr : 0; } static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = { diff --git a/tools/testing/selftests/bpf/progs/get_func_ip_test.c b/tools/testing/selftests/bpf/progs/get_func_ip_test.c index a587aeca5ae0..e988aefa567e 100644 --- a/tools/testing/selftests/bpf/progs/get_func_ip_test.c +++ b/tools/testing/selftests/bpf/progs/get_func_ip_test.c @@ -69,7 +69,7 @@ int test6(struct pt_regs *ctx) { __u64 addr = bpf_get_func_ip(ctx); - test6_result = (const void *) addr == &bpf_fentry_test6 + 5; + test6_result = (const void *) addr == &bpf_fentry_test6; return 0; } @@ -79,6 +79,6 @@ int test7(struct pt_regs *ctx) { __u64 addr = bpf_get_func_ip(ctx); - test7_result = (const void *) addr == &bpf_fentry_test7 + 5; + test7_result = (const void *) addr == &bpf_fentry_test7; return 0; } From patchwork Tue Jan 4 08:09:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703212 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD07EC433FE for ; Tue, 4 Jan 2022 08:10:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233859AbiADIKJ (ORCPT ); Tue, 4 Jan 2022 03:10:09 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43641 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233779AbiADIKG (ORCPT ); Tue, 4 Jan 2022 03:10:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283805; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6BFEB//ISNWLrFisbkclN0B/NzP/+Sn4aN88iN/SnSE=; b=O64QoyJxmrfA5ucp88cN0UMUQeV3iCosmFoTHXds4ydY/G5oBMAqeRe6vSlKiyaG0YurrX 4qZ8NpBUoWrknC5XPWF+fM0cUhM1Y7hAaOLeEOb4PtwcpL69a7uDhynxMldi0yiA6d6ZR9 b/Vxbr9RfYtzMljDdrHyETnftrL538M= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-444-UDUetoNgOxyDt7I2iHP_SQ-1; Tue, 04 Jan 2022 03:10:04 -0500 X-MC-Unique: UDUetoNgOxyDt7I2iHP_SQ-1 Received: by mail-ed1-f72.google.com with SMTP id ch27-20020a0564021bdb00b003f8389236f8so24469151edb.19 for ; Tue, 04 Jan 2022 00:10:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6BFEB//ISNWLrFisbkclN0B/NzP/+Sn4aN88iN/SnSE=; b=lGDQsxW549KgBS3Yyv4/EdtVw9x9LCmBK7amI4+xMVwE9OZECEBwAvTS6RPZRj+3Gp y5d4d0ZqztB1jDFfanBy8eJ8G7TpcwtCq/L1OpWt0Wf3ndKh5DIsTfzJs/+OiOx7pkoI NnmibOsbiGR7rsFCPwyXFZlR4Od1Xsc/50dmjOO8dD2c8OdjSwcHpt8ji7+kPGE9d+Xv p8KRgOQ3yBCRZ/ozayI25t8OfCHY7ALEK1xqBvG5n3ubfj7PavJ5gbaVbq9eojqThYyD RsCSD9V+M5scfAtOSBZAcIt/y56KhNpxYnteUylE2cEyB0mUVObO9x8WeUNHzlgaGviT VM4w== X-Gm-Message-State: AOAM530dzQ0EQ1vykvfzwa4OeunQ7wNx7R3J5GR96Gitg6/1ocu/iMzf a4MlEAue+9LZtFySP3wEace5ns/MPjfsgdYsILZPrK77YDHs8Yyw7boFAT9qMsBY/20hmnqG713 qIqD3mjYNvMFvQfZ6 X-Received: by 2002:aa7:d299:: with SMTP id w25mr48023971edq.269.1641283802978; Tue, 04 Jan 2022 00:10:02 -0800 (PST) X-Google-Smtp-Source: ABdhPJybPAhyMNJAbOkPpI/5fB1tOEsb49UmlCJ1ODo++dlxof7LW1b5pCI0lsf7ki81kO1IxWQOxg== X-Received: by 2002:aa7:d299:: with SMTP id w25mr48023957edq.269.1641283802782; Tue, 04 Jan 2022 00:10:02 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id hu8sm11296854ejc.32.2022.01.04.00.10.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:10:02 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 03/13] kprobe: Add support to register multiple ftrace kprobes Date: Tue, 4 Jan 2022 09:09:33 +0100 Message-Id: <20220104080943.113249-4-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding support to register kprobe on multiple addresses within single kprobe object instance. It uses the CONFIG_KPROBES_ON_FTRACE feature (so it's only available for function entry addresses) and separated ftrace_ops object placed directly in the kprobe object. There's new CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE config option, enabled for archs that support multi kprobes. It registers all the provided addresses in the ftrace_ops object filter and adds separate ftrace_ops callback. To register multi kprobe user provides array of addresses or symbols with their count, like: struct kprobe kp = {}; kp.multi.symbols = (const char **) symbols; kp.multi.cnt = cnt; ... err = register_kprobe(&kp); Signed-off-by: Jiri Olsa --- arch/Kconfig | 3 + arch/x86/Kconfig | 1 + arch/x86/kernel/kprobes/ftrace.c | 48 ++++++-- include/linux/kprobes.h | 25 ++++ kernel/kprobes.c | 204 +++++++++++++++++++++++++------ 5 files changed, 235 insertions(+), 46 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index d3c4ab249e9c..0131636e1ef8 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -191,6 +191,9 @@ config HAVE_OPTPROBES config HAVE_KPROBES_ON_FTRACE bool +config HAVE_KPROBES_MULTI_ON_FTRACE + bool + config ARCH_CORRECT_STACKTRACE_ON_KRETPROBE bool help diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 5c2ccb85f2ef..0c870238016a 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -217,6 +217,7 @@ config X86 select HAVE_KERNEL_ZSTD select HAVE_KPROBES select HAVE_KPROBES_ON_FTRACE + select HAVE_KPROBES_MULTI_ON_FTRACE select HAVE_FUNCTION_ERROR_INJECTION select HAVE_KRETPROBES select HAVE_KVM diff --git a/arch/x86/kernel/kprobes/ftrace.c b/arch/x86/kernel/kprobes/ftrace.c index dd2ec14adb77..ac4d256b89c6 100644 --- a/arch/x86/kernel/kprobes/ftrace.c +++ b/arch/x86/kernel/kprobes/ftrace.c @@ -12,22 +12,14 @@ #include "common.h" -/* Ftrace callback handler for kprobes -- called under preempt disabled */ -void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip, - struct ftrace_ops *ops, struct ftrace_regs *fregs) +static void ftrace_handler(struct kprobe *p, unsigned long ip, + struct ftrace_regs *fregs) { struct pt_regs *regs = ftrace_get_regs(fregs); - struct kprobe *p; struct kprobe_ctlblk *kcb; - int bit; - bit = ftrace_test_recursion_trylock(ip, parent_ip); - if (bit < 0) - return; - - p = get_kprobe((kprobe_opcode_t *)ip); if (unlikely(!p) || kprobe_disabled(p)) - goto out; + return; kcb = get_kprobe_ctlblk(); if (kprobe_running()) { @@ -57,11 +49,43 @@ void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip, */ __this_cpu_write(current_kprobe, NULL); } -out: +} + +/* Ftrace callback handler for kprobes -- called under preempt disabled */ +void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *ops, struct ftrace_regs *fregs) +{ + struct kprobe *p; + int bit; + + bit = ftrace_test_recursion_trylock(ip, parent_ip); + if (bit < 0) + return; + + p = get_kprobe((kprobe_opcode_t *)ip); + ftrace_handler(p, ip, fregs); + ftrace_test_recursion_unlock(bit); } NOKPROBE_SYMBOL(kprobe_ftrace_handler); +void kprobe_ftrace_multi_handler(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *ops, struct ftrace_regs *fregs) +{ + struct kprobe *p; + int bit; + + bit = ftrace_test_recursion_trylock(ip, parent_ip); + if (bit < 0) + return; + + p = container_of(ops, struct kprobe, multi.ops); + ftrace_handler(p, ip, fregs); + + ftrace_test_recursion_unlock(bit); +} +NOKPROBE_SYMBOL(kprobe_ftrace_multi_handler); + int arch_prepare_kprobe_ftrace(struct kprobe *p) { p->ainsn.insn = NULL; diff --git a/include/linux/kprobes.h b/include/linux/kprobes.h index a204df4fef96..03fd86ef69cb 100644 --- a/include/linux/kprobes.h +++ b/include/linux/kprobes.h @@ -68,6 +68,16 @@ struct kprobe { /* location of the probe point */ kprobe_opcode_t *addr; +#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE + /* location of the multi probe points */ + struct { + const char **symbols; + kprobe_opcode_t **addrs; + unsigned int cnt; + struct ftrace_ops ops; + } multi; +#endif + /* Allow user to indicate symbol name of the probe point */ const char *symbol_name; @@ -105,6 +115,7 @@ struct kprobe { * this flag is only for optimized_kprobe. */ #define KPROBE_FLAG_FTRACE 8 /* probe is using ftrace */ +#define KPROBE_FLAG_MULTI 16 /* probe multi addresses */ /* Has this kprobe gone ? */ static inline bool kprobe_gone(struct kprobe *p) @@ -130,6 +141,18 @@ static inline bool kprobe_ftrace(struct kprobe *p) return p->flags & KPROBE_FLAG_FTRACE; } +/* Is this ftrace multi kprobe ? */ +static inline bool kprobe_ftrace_multi(struct kprobe *p) +{ + return kprobe_ftrace(p) && (p->flags & KPROBE_FLAG_MULTI); +} + +/* Is this single kprobe ? */ +static inline bool kprobe_single(struct kprobe *p) +{ + return !(p->flags & KPROBE_FLAG_MULTI); +} + /* * Function-return probe - * Note: @@ -365,6 +388,8 @@ static inline void wait_for_kprobe_optimizer(void) { } #ifdef CONFIG_KPROBES_ON_FTRACE extern void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *ops, struct ftrace_regs *fregs); +extern void kprobe_ftrace_multi_handler(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *ops, struct ftrace_regs *fregs); extern int arch_prepare_kprobe_ftrace(struct kprobe *p); #else static inline int arch_prepare_kprobe_ftrace(struct kprobe *p) diff --git a/kernel/kprobes.c b/kernel/kprobes.c index c4060a8da050..e7729e20d85c 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -44,6 +44,7 @@ #include #include #include +#include #define KPROBE_HASH_BITS 6 #define KPROBE_TABLE_SIZE (1 << KPROBE_HASH_BITS) @@ -1022,6 +1023,35 @@ static struct kprobe *alloc_aggr_kprobe(struct kprobe *p) } #endif /* CONFIG_OPTPROBES */ +static int check_kprobe_address(unsigned long addr) +{ + /* Ensure it is not in reserved area nor out of text */ + return !kernel_text_address(addr) || + within_kprobe_blacklist(addr) || + jump_label_text_reserved((void *) addr, (void *) addr) || + static_call_text_reserved((void *) addr, (void *) addr) || + find_bug(addr); +} + +static int check_ftrace_location(unsigned long addr, struct kprobe *p) +{ + unsigned long ftrace_addr; + + ftrace_addr = ftrace_location(addr); + if (ftrace_addr) { +#ifdef CONFIG_KPROBES_ON_FTRACE + /* Given address is not on the instruction boundary */ + if (addr != ftrace_addr) + return -EILSEQ; + if (p) + p->flags |= KPROBE_FLAG_FTRACE; +#else /* !CONFIG_KPROBES_ON_FTRACE */ + return -EINVAL; +#endif + } + return 0; +} + #ifdef CONFIG_KPROBES_ON_FTRACE static struct ftrace_ops kprobe_ftrace_ops __read_mostly = { .func = kprobe_ftrace_handler, @@ -1043,6 +1073,13 @@ static int __arm_kprobe_ftrace(struct kprobe *p, struct ftrace_ops *ops, lockdep_assert_held(&kprobe_mutex); +#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE + if (kprobe_ftrace_multi(p)) { + ret = register_ftrace_function(&p->multi.ops); + WARN(ret < 0, "Failed to register kprobe-multi-ftrace (error %d)\n", ret); + return ret; + } +#endif ret = ftrace_set_filter_ip(ops, (unsigned long)p->addr, 0, 0); if (WARN_ONCE(ret < 0, "Failed to arm kprobe-ftrace at %pS (error %d)\n", p->addr, ret)) return ret; @@ -1081,6 +1118,13 @@ static int __disarm_kprobe_ftrace(struct kprobe *p, struct ftrace_ops *ops, lockdep_assert_held(&kprobe_mutex); +#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE + if (kprobe_ftrace_multi(p)) { + ret = unregister_ftrace_function(&p->multi.ops); + WARN(ret < 0, "Failed to unregister kprobe-ftrace (error %d)\n", ret); + return ret; + } +#endif if (*cnt == 1) { ret = unregister_ftrace_function(ops); if (WARN(ret < 0, "Failed to unregister kprobe-ftrace (error %d)\n", ret)) @@ -1103,6 +1147,94 @@ static int disarm_kprobe_ftrace(struct kprobe *p) ipmodify ? &kprobe_ipmodify_ops : &kprobe_ftrace_ops, ipmodify ? &kprobe_ipmodify_enabled : &kprobe_ftrace_enabled); } + +#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE +/* + * In addition to standard kprobe address check for multi + * ftrace kprobes we also allow only: + * - ftrace managed function entry address + * - kernel core only address + */ +static unsigned long check_ftrace_addr(unsigned long addr) +{ + int err; + + if (!addr) + return -EINVAL; + err = check_ftrace_location(addr, NULL); + if (err) + return err; + if (check_kprobe_address(addr)) + return -EINVAL; + if (__module_text_address(addr)) + return -EINVAL; + return 0; +} + +static int check_ftrace_multi(struct kprobe *p) +{ + kprobe_opcode_t **addrs = p->multi.addrs; + const char **symbols = p->multi.symbols; + unsigned int i, cnt = p->multi.cnt; + unsigned long addr, *ips; + int err; + + if ((symbols && addrs) || (!symbols && !addrs)) + return -EINVAL; + + /* do we want sysctl for this? */ + if (cnt >= 20000) + return -E2BIG; + + ips = kmalloc(sizeof(*ips) * cnt, GFP_KERNEL); + if (!ips) + return -ENOMEM; + + for (i = 0; i < cnt; i++) { + if (symbols) + addr = (unsigned long) kprobe_lookup_name(symbols[i], 0); + else + addr = (unsigned long) addrs[i]; + ips[i] = addr; + } + + jump_label_lock(); + preempt_disable(); + + for (i = 0; i < cnt; i++) { + err = check_ftrace_addr(ips[i]); + if (err) + break; + } + + preempt_enable(); + jump_label_unlock(); + + if (err) + goto out; + + err = ftrace_set_filter_ips(&p->multi.ops, ips, cnt, 0, 0); + if (err) + goto out; + + p->multi.ops.func = kprobe_ftrace_multi_handler; + p->multi.ops.flags = FTRACE_OPS_FL_SAVE_REGS|FTRACE_OPS_FL_DYNAMIC; + + p->flags |= KPROBE_FLAG_MULTI|KPROBE_FLAG_FTRACE; + if (p->post_handler) + p->multi.ops.flags |= FTRACE_OPS_FL_IPMODIFY; + +out: + kfree(ips); + return err; +} + +static void free_ftrace_multi(struct kprobe *p) +{ + ftrace_free_filter(&p->multi.ops); +} +#endif + #else /* !CONFIG_KPROBES_ON_FTRACE */ static inline int arm_kprobe_ftrace(struct kprobe *p) { @@ -1489,6 +1621,9 @@ static struct kprobe *__get_valid_kprobe(struct kprobe *p) lockdep_assert_held(&kprobe_mutex); + if (kprobe_ftrace_multi(p)) + return p; + ap = get_kprobe(p->addr); if (unlikely(!ap)) return NULL; @@ -1520,41 +1655,18 @@ static inline int warn_kprobe_rereg(struct kprobe *p) return ret; } -static int check_ftrace_location(struct kprobe *p) -{ - unsigned long ftrace_addr; - - ftrace_addr = ftrace_location((unsigned long)p->addr); - if (ftrace_addr) { -#ifdef CONFIG_KPROBES_ON_FTRACE - /* Given address is not on the instruction boundary */ - if ((unsigned long)p->addr != ftrace_addr) - return -EILSEQ; - p->flags |= KPROBE_FLAG_FTRACE; -#else /* !CONFIG_KPROBES_ON_FTRACE */ - return -EINVAL; -#endif - } - return 0; -} - static int check_kprobe_address_safe(struct kprobe *p, struct module **probed_mod) { int ret; - ret = check_ftrace_location(p); + ret = check_ftrace_location((unsigned long) p->addr, p); if (ret) return ret; jump_label_lock(); preempt_disable(); - /* Ensure it is not in reserved area nor out of text */ - if (!kernel_text_address((unsigned long) p->addr) || - within_kprobe_blacklist((unsigned long) p->addr) || - jump_label_text_reserved(p->addr, p->addr) || - static_call_text_reserved(p->addr, p->addr) || - find_bug((unsigned long)p->addr)) { + if (check_kprobe_address((unsigned long) p->addr)) { ret = -EINVAL; goto out; } @@ -1599,13 +1711,16 @@ static unsigned long resolve_func_addr(kprobe_opcode_t *addr) return 0; } -int register_kprobe(struct kprobe *p) +static int check_addr(struct kprobe *p, struct module **probed_mod) { int ret; - struct kprobe *old_p; - struct module *probed_mod; kprobe_opcode_t *addr; +#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE + if (p->multi.cnt) + return check_ftrace_multi(p); +#endif + /* Adjust probe address from symbol */ addr = kprobe_addr(p); if (IS_ERR(addr)) @@ -1616,13 +1731,21 @@ int register_kprobe(struct kprobe *p) ret = warn_kprobe_rereg(p); if (ret) return ret; + return check_kprobe_address_safe(p, probed_mod); +} + +int register_kprobe(struct kprobe *p) +{ + struct module *probed_mod = NULL; + struct kprobe *old_p; + int ret; /* User can pass only KPROBE_FLAG_DISABLED to register_kprobe */ p->flags &= KPROBE_FLAG_DISABLED; p->nmissed = 0; INIT_LIST_HEAD(&p->list); - ret = check_kprobe_address_safe(p, &probed_mod); + ret = check_addr(p, &probed_mod); if (ret) return ret; @@ -1644,14 +1767,21 @@ int register_kprobe(struct kprobe *p) if (ret) goto out; - INIT_HLIST_NODE(&p->hlist); - hlist_add_head_rcu(&p->hlist, - &kprobe_table[hash_ptr(p->addr, KPROBE_HASH_BITS)]); + /* + * Multi ftrace kprobes do not have single address, + * so they are not stored in the kprobe_table hash. + */ + if (kprobe_single(p)) { + INIT_HLIST_NODE(&p->hlist); + hlist_add_head_rcu(&p->hlist, + &kprobe_table[hash_ptr(p->addr, KPROBE_HASH_BITS)]); + } if (!kprobes_all_disarmed && !kprobe_disabled(p)) { ret = arm_kprobe(p); if (ret) { - hlist_del_rcu(&p->hlist); + if (kprobe_single(p)) + hlist_del_rcu(&p->hlist); synchronize_rcu(); goto out; } @@ -1778,7 +1908,13 @@ static int __unregister_kprobe_top(struct kprobe *p) return 0; disarmed: - hlist_del_rcu(&ap->hlist); + if (kprobe_single(ap)) + hlist_del_rcu(&ap->hlist); + +#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE + if (kprobe_ftrace_multi(ap)) + free_ftrace_multi(ap); +#endif return 0; } From patchwork Tue Jan 4 08:09:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703213 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35C9FC433EF for ; Tue, 4 Jan 2022 08:10:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233934AbiADIKU (ORCPT ); Tue, 4 Jan 2022 03:10:20 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:37534 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233910AbiADIKM (ORCPT ); Tue, 4 Jan 2022 03:10:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283811; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PrqUMFFn4pZh/YLFjtZhXl9wcsEGaC7EU4s7bjstbP0=; b=MRmZjZeR7IBYbZopgVZPx8LyoU7HOVUT8TyQIV+g5B/+mTuXXO77gpGYf5w1gH5QcBySUP p2SCixTC2tgEOP2Rw3fskldyrHfQshZ+nFI2dZroK/ccOAHmYP8KxEzo0YbUPdBbESXTzQ 3NbNS/as7uJdWRJgG/b15nvEe5mXVPM= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-315-Ui8hecG8OZuqDZQsXkR8NA-1; Tue, 04 Jan 2022 03:10:10 -0500 X-MC-Unique: Ui8hecG8OZuqDZQsXkR8NA-1 Received: by mail-ed1-f71.google.com with SMTP id x19-20020a05640226d300b003f8b80f5729so23373426edd.13 for ; Tue, 04 Jan 2022 00:10:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PrqUMFFn4pZh/YLFjtZhXl9wcsEGaC7EU4s7bjstbP0=; b=czJBPEmzxQi0TbU5GqjGSM5ft3/lSX2xzW/r1keLB6/uAp0SDnlGsyYEEFR9LFG730 RoJ8r4BQctQdZApN+s/AmkQLg/zvG8minfdxwJ//FuJZQ62WinjmefaSReXG+r/kKWGt MHnhq9KTjO36Ofzi8faSnZiUIlD2u/3D4U9IaL/ixviFsPo6eFkt3RmL/Zy1lMSFQb7q GkGAMNK7d97okwlYBj8EDBFrATD0zDTlGFNar4H5K/xx2/5CNhVhk2jCXVDPn7TAkdXE Afv3Qrll4Wi2DKUzclGbfpUqAfZH2A9MeUXzJQMnZvtvzh61KvTAFtRna3RRVVkgLNJE nIcQ== X-Gm-Message-State: AOAM532Q8Tpr29yRRWxC0esqvcqTqvZtk9hRueMWpT5cyxsMr8PiYFZS HGv+Un0pZLZcQvxT/SHw7abC7r/FxzaG93WIyZ5mKC6z6nclOaL7bWSaM7fa67UkawZFf4LuHxp +B0O6q5X+h12XeU8o X-Received: by 2002:a17:907:1c92:: with SMTP id nb18mr37866566ejc.157.1641283809093; Tue, 04 Jan 2022 00:10:09 -0800 (PST) X-Google-Smtp-Source: ABdhPJySXjS/HZTpUVZPV0EmeRXwPa68kUDgpiR6qGwn7STyl529I3LJOrPLNrk7fQlc+nh3IzGqkg== X-Received: by 2002:a17:907:1c92:: with SMTP id nb18mr37866543ejc.157.1641283808870; Tue, 04 Jan 2022 00:10:08 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id u9sm8293523ejh.193.2022.01.04.00.10.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:10:08 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 04/13] kprobe: Add support to register multiple ftrace kretprobes Date: Tue, 4 Jan 2022 09:09:34 +0100 Message-Id: <20220104080943.113249-5-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding support to register kretprobe on multiple addresses within single kprobe object instance. The interface stays same as for kprobes, the patch just adds blacklist check for each address, which is special for kretprobes. Signed-off-by: Jiri Olsa --- include/linux/kprobes.h | 1 + kernel/kprobes.c | 46 ++++++++++++++++++++++++++++++++--------- 2 files changed, 37 insertions(+), 10 deletions(-) diff --git a/include/linux/kprobes.h b/include/linux/kprobes.h index 03fd86ef69cb..a31da6202b5c 100644 --- a/include/linux/kprobes.h +++ b/include/linux/kprobes.h @@ -75,6 +75,7 @@ struct kprobe { kprobe_opcode_t **addrs; unsigned int cnt; struct ftrace_ops ops; + bool check_kretprobe; } multi; #endif diff --git a/kernel/kprobes.c b/kernel/kprobes.c index e7729e20d85c..04fc411ca30c 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -1052,6 +1052,17 @@ static int check_ftrace_location(unsigned long addr, struct kprobe *p) return 0; } +static bool in_kretprobe_blacklist(void *addr) +{ + int i; + + for (i = 0; kretprobe_blacklist[i].name != NULL; i++) { + if (kretprobe_blacklist[i].addr == addr) + return true; + } + return false; +} + #ifdef CONFIG_KPROBES_ON_FTRACE static struct ftrace_ops kprobe_ftrace_ops __read_mostly = { .func = kprobe_ftrace_handler, @@ -1155,7 +1166,8 @@ static int disarm_kprobe_ftrace(struct kprobe *p) * - ftrace managed function entry address * - kernel core only address */ -static unsigned long check_ftrace_addr(unsigned long addr) +static unsigned long check_ftrace_addr(unsigned long addr, + bool check_kretprobe) { int err; @@ -1168,6 +1180,8 @@ static unsigned long check_ftrace_addr(unsigned long addr) return -EINVAL; if (__module_text_address(addr)) return -EINVAL; + if (check_kretprobe && in_kretprobe_blacklist((void *) addr)) + return -EINVAL; return 0; } @@ -1202,7 +1216,7 @@ static int check_ftrace_multi(struct kprobe *p) preempt_disable(); for (i = 0; i < cnt; i++) { - err = check_ftrace_addr(ips[i]); + err = check_ftrace_addr(ips[i], p->multi.check_kretprobe); if (err) break; } @@ -2188,13 +2202,17 @@ int kprobe_on_func_entry(kprobe_opcode_t *addr, const char *sym, unsigned long o return 0; } -int register_kretprobe(struct kretprobe *rp) +static int check_kretprobe_address(struct kretprobe *rp) { int ret; - struct kretprobe_instance *inst; - int i; void *addr; +#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE + if (rp->kp.multi.cnt) { + rp->kp.multi.check_kretprobe = !!kretprobe_blacklist_size; + return 0; + } +#endif ret = kprobe_on_func_entry(rp->kp.addr, rp->kp.symbol_name, rp->kp.offset); if (ret) return ret; @@ -2207,12 +2225,20 @@ int register_kretprobe(struct kretprobe *rp) addr = kprobe_addr(&rp->kp); if (IS_ERR(addr)) return PTR_ERR(addr); - - for (i = 0; kretprobe_blacklist[i].name != NULL; i++) { - if (kretprobe_blacklist[i].addr == addr) - return -EINVAL; - } + if (in_kretprobe_blacklist(addr)) + return -EINVAL; } + return 0; +} + +int register_kretprobe(struct kretprobe *rp) +{ + struct kretprobe_instance *inst; + int i, ret = 0; + + ret = check_kretprobe_address(rp); + if (ret) + return ret; if (rp->data_size > KRETPROBE_MAX_DATA_SIZE) return -E2BIG; From patchwork Tue Jan 4 08:09:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703214 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFFFEC433FE for ; Tue, 4 Jan 2022 08:10:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234003AbiADIKX (ORCPT ); Tue, 4 Jan 2022 03:10:23 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:28568 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233980AbiADIKS (ORCPT ); Tue, 4 Jan 2022 03:10:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283817; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jYMe7lfz1ar5GbZoHBTiKVhZmhUJ2P1sIXs88W+jgrk=; b=XKMNO4ouO+xU/PSLvHb/9q/0tjdkY3Opwjl9+wSv9pvmLBJTKQoKzBxw5nsel0ulZtadUO ZD9WNpXVPjyIuCrYaW8+y3ipYGRqV7d3r996lFuPDL208mc6ea4EurDZXWR8QmLkN/LOnu HLUyjmRAmrVKkCrhdgcfphNMEoGCcr4= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-655-ZDhFd6bYOMOtRjSU3JWI9w-1; Tue, 04 Jan 2022 03:10:16 -0500 X-MC-Unique: ZDhFd6bYOMOtRjSU3JWI9w-1 Received: by mail-ed1-f72.google.com with SMTP id q15-20020a056402518f00b003f87abf9c37so24636030edd.15 for ; Tue, 04 Jan 2022 00:10:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jYMe7lfz1ar5GbZoHBTiKVhZmhUJ2P1sIXs88W+jgrk=; b=2BBRAIb2ko1twpgYJ8T9D3PWdfXJODAYNmQwggQ3WOuBD2ZCcMUZDVk3tGC4lwFaJ6 R1pfIKO019LZbDQ+pViu5a/T/CobK1baxqSFO3MHoIrVMNZ+fJpV/N4zsDpFep9j2pqd Hvh0mDFnPxsCMq2rPzQ+Aaqz/6seIWmXNVzGjYu6s76/hTPE80aVEau9q2my9+5XeXxq C2I07uwXO27lg0uy3QWRcKYk8SJjcj6mkA/apthZpAPR4l1Cz71JQSZTih/fLcwDKoQ/ zp9BObEVnfPyhp4IgEuTT9YoPJr8Gp3Zlp8ora9sU1Klx7gmTPV0GYG3I+RaWly3UOfI 4Xmg== X-Gm-Message-State: AOAM532w7Bl5Y60Eu7zNT/iXz9EH9pMRKgqgIVwnSsS8Fmg1t7vUy3zs vlsidGGG5N9GaL9HWd+SxwyxlnOp8jWSXx242VpfOKroPW9FUXnOL1rAFZEE8uh3j7Bf8+xVVJZ fzszSgtCqrKZsrM++ X-Received: by 2002:a17:907:1b24:: with SMTP id mp36mr40470827ejc.487.1641283815388; Tue, 04 Jan 2022 00:10:15 -0800 (PST) X-Google-Smtp-Source: ABdhPJys/NBfmK+oSvUScUuxZvvogGO5lENwe6I7iUMzwZZ9EBfuaQOOl6Y5x9wI3K2lWpQAADJuHA== X-Received: by 2002:a17:907:1b24:: with SMTP id mp36mr40470817ejc.487.1641283815175; Tue, 04 Jan 2022 00:10:15 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id qa41sm11271437ejc.0.2022.01.04.00.10.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:10:14 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 05/13] kprobe: Allow to get traced function address for multi ftrace kprobes Date: Tue, 4 Jan 2022 09:09:35 +0100 Message-Id: <20220104080943.113249-6-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The current bpf_get_func_ip_kprobe helper does not work properly, when used in ebpf program triggered by the new multi kprobes. We can't use kprobe's func_addr in bpf_get_func_ip_kprobe helper, because there are multiple functions registered for single kprobe object. Adding new per cpu variable current_ftrace_multi_addr and extra address in kretprobe_instance object to keep current traced function address for each cpu for both kprobe handler and kretprobe trampoline. The address value is set/passed as follows, for kprobe: kprobe_ftrace_multi_handler { old = kprobe_ftrace_multi_addr_set(ip); handler.. kprobe_ftrace_multi_addr_set(old); } For kretprobe: kprobe_ftrace_multi_handler { old = kprobe_ftrace_multi_addr_set(ip); ... pre_handler_kretprobe { ri->ftrace_multi_addr = kprobe_ftrace_multi_addr } ... kprobe_ftrace_multi_addr_set(old); } __kretprobe_trampoline_handler { prev_func_addr = kprobe_ftrace_multi_addr_set(ri->ftrace_multi_addr); handler.. kprobe_ftrace_multi_addr_set(prev_func_addr); } Signed-off-by: Jiri Olsa --- arch/x86/kernel/kprobes/ftrace.c | 3 +++ include/linux/kprobes.h | 26 ++++++++++++++++++++++++++ kernel/kprobes.c | 6 ++++++ kernel/trace/bpf_trace.c | 7 ++++++- 4 files changed, 41 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/kprobes/ftrace.c b/arch/x86/kernel/kprobes/ftrace.c index ac4d256b89c6..8caaa58c3a64 100644 --- a/arch/x86/kernel/kprobes/ftrace.c +++ b/arch/x86/kernel/kprobes/ftrace.c @@ -72,6 +72,7 @@ NOKPROBE_SYMBOL(kprobe_ftrace_handler); void kprobe_ftrace_multi_handler(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { + unsigned long old; struct kprobe *p; int bit; @@ -79,8 +80,10 @@ void kprobe_ftrace_multi_handler(unsigned long ip, unsigned long parent_ip, if (bit < 0) return; + old = kprobe_ftrace_multi_addr_set(ip); p = container_of(ops, struct kprobe, multi.ops); ftrace_handler(p, ip, fregs); + kprobe_ftrace_multi_addr_set(old); ftrace_test_recursion_unlock(bit); } diff --git a/include/linux/kprobes.h b/include/linux/kprobes.h index a31da6202b5c..3f0522b9538b 100644 --- a/include/linux/kprobes.h +++ b/include/linux/kprobes.h @@ -191,6 +191,7 @@ struct kretprobe_instance { struct kretprobe_holder *rph; kprobe_opcode_t *ret_addr; void *fp; + unsigned long ftrace_multi_addr; char data[]; }; @@ -387,16 +388,37 @@ static inline void wait_for_kprobe_optimizer(void) { } #endif /* CONFIG_OPTPROBES */ #ifdef CONFIG_KPROBES_ON_FTRACE +DECLARE_PER_CPU(unsigned long, current_ftrace_multi_addr); extern void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *ops, struct ftrace_regs *fregs); extern void kprobe_ftrace_multi_handler(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *ops, struct ftrace_regs *fregs); extern int arch_prepare_kprobe_ftrace(struct kprobe *p); + +static inline unsigned long kprobe_ftrace_multi_addr(void) +{ + return __this_cpu_read(current_ftrace_multi_addr); +} +static inline unsigned long kprobe_ftrace_multi_addr_set(unsigned long addr) +{ + unsigned long old = __this_cpu_read(current_ftrace_multi_addr); + + __this_cpu_write(current_ftrace_multi_addr, addr); + return old; +} #else static inline int arch_prepare_kprobe_ftrace(struct kprobe *p) { return -EINVAL; } +static inline unsigned long kprobe_ftrace_multi_addr_set(unsigned long addr) +{ + return 0; +} +static inline unsigned long kprobe_ftrace_multi_addr(void) +{ + return 0; +} #endif /* CONFIG_KPROBES_ON_FTRACE */ /* Get the kprobe at this addr (if any) - called with preemption disabled */ @@ -514,6 +536,10 @@ static inline int kprobe_get_kallsym(unsigned int symnum, unsigned long *value, { return -ERANGE; } +static inline unsigned long kprobe_ftrace_multi_addr(void) +{ + return 0; +} #endif /* CONFIG_KPROBES */ static inline int disable_kretprobe(struct kretprobe *rp) diff --git a/kernel/kprobes.c b/kernel/kprobes.c index 04fc411ca30c..6ba249f3a0cb 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -1064,6 +1064,8 @@ static bool in_kretprobe_blacklist(void *addr) } #ifdef CONFIG_KPROBES_ON_FTRACE +DEFINE_PER_CPU(unsigned long, current_ftrace_multi_addr); + static struct ftrace_ops kprobe_ftrace_ops __read_mostly = { .func = kprobe_ftrace_handler, .flags = FTRACE_OPS_FL_SAVE_REGS, @@ -2106,11 +2108,14 @@ unsigned long __kretprobe_trampoline_handler(struct pt_regs *regs, rp = get_kretprobe(ri); if (rp && rp->handler) { struct kprobe *prev = kprobe_running(); + unsigned long prev_func_addr; + prev_func_addr = kprobe_ftrace_multi_addr_set(ri->ftrace_multi_addr); __this_cpu_write(current_kprobe, &rp->kp); ri->ret_addr = correct_ret_addr; rp->handler(ri, regs); __this_cpu_write(current_kprobe, prev); + kprobe_ftrace_multi_addr_set(prev_func_addr); } if (first == node) break; @@ -2161,6 +2166,7 @@ static int pre_handler_kretprobe(struct kprobe *p, struct pt_regs *regs) } arch_prepare_kretprobe(ri, regs); + ri->ftrace_multi_addr = kprobe_ftrace_multi_addr(); __llist_add(&ri->llist, ¤t->kretprobe_instances); diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 25631253084a..39f4d476cfca 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1026,7 +1026,12 @@ BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs) { struct kprobe *kp = kprobe_running(); - return kp ? (uintptr_t)kp->func_addr : 0; + if (!kp) + return 0; + if (kprobe_ftrace_multi(kp)) + return (uintptr_t) kprobe_ftrace_multi_addr(); + else + return (uintptr_t) kp->func_addr; } static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = { From patchwork Tue Jan 4 08:09:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703215 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 517D9C433EF for ; Tue, 4 Jan 2022 08:10:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233978AbiADIKb (ORCPT ); Tue, 4 Jan 2022 03:10:31 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:60490 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234006AbiADIKY (ORCPT ); Tue, 4 Jan 2022 03:10:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283823; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QB0M13Hi340QTyzl9UwQ247tdhoefPKuVolRpwS7Wq0=; b=alg0DjU5vh5N/FC5f0d2sk2o67mdSGR1i/aizlNYhWBORW2Eg5GSvYAG3xwUmLk5NslxVp HkCUGshg+DsIDPcnUmZ+fIXEUUzC+km6BEbjrTe/nIpvkbnwVgFODeEGYkQGApdzgC7SY7 JqZLEBLyxjO7eHmlVEkH/DoPxnYAfAM= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-575-4ioLpslcMASbVeJm6hkhYA-1; Tue, 04 Jan 2022 03:10:22 -0500 X-MC-Unique: 4ioLpslcMASbVeJm6hkhYA-1 Received: by mail-ed1-f72.google.com with SMTP id dz8-20020a0564021d4800b003f897935eb3so24727580edb.12 for ; Tue, 04 Jan 2022 00:10:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QB0M13Hi340QTyzl9UwQ247tdhoefPKuVolRpwS7Wq0=; b=eSUiA3vcyZmJaHzuyT3JiykvywENjt50hfOWoRekRF3BhLpHRDtTKV5e/RlmeEQQVc boagYX+NQv6zgsAS0eB7WBgxsKxQ82hrdu1QR2MJLIu7rSjMDllbLWVAMYYb8Iibb/Tz fkVnBJDgvQ+Nko7nbBE1ZV7xKu21OO6HGSecG0JMn70efweE8RmW9xcwH9mRJUqsCdNh z8REw8KRXsWVCjw2nzuMu47ZR1LWGm8EIwY9rbhHk5c7LTg2t7OGsrgyn9aq054SU2UR yPGFplj/D9kDmxY9l5+OYKDtnGchB3z2JQcjZzKbYxZr5Wt2K6HzmimyOGDpPFaHdaCW 8Log== X-Gm-Message-State: AOAM5328c+m6wzbRzI+YUyDs4p3cGZcrIKCrTVROnjUKyui177dZiiGS adbZiqBf6HAMAy+gtYFnS0RlF/vQ9/bqllOMOEUpgir86A9+t3ssuWUMYQqFHhulZxzBmtnkUfV LJeRdCv1JDSd42VIu X-Received: by 2002:a05:6402:2747:: with SMTP id z7mr45776172edd.321.1641283821366; Tue, 04 Jan 2022 00:10:21 -0800 (PST) X-Google-Smtp-Source: ABdhPJx+KmDVeG+o44jzowNPWmFk6FPwCuGUaVo1KM8fWZVoz9xlx17ppDVM9J0ByCFBH8ZC6vK4bQ== X-Received: by 2002:a05:6402:2747:: with SMTP id z7mr45776154edd.321.1641283821241; Tue, 04 Jan 2022 00:10:21 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id di16sm11303023ejc.82.2022.01.04.00.10.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:10:21 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 06/13] samples/kprobes: Add support for multi kprobe interface Date: Tue, 4 Jan 2022 09:09:36 +0100 Message-Id: <20220104080943.113249-7-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding support to test multi kprobe interface. It's now possible to register multiple functions for the module handler, like: # modprobe kprobe_example symbol='sched_fork,kernel_clone' Signed-off-by: Jiri Olsa --- samples/kprobes/kprobe_example.c | 47 +++++++++++++++++++++++++++++--- 1 file changed, 43 insertions(+), 4 deletions(-) diff --git a/samples/kprobes/kprobe_example.c b/samples/kprobes/kprobe_example.c index f991a66b5b02..4ae31184f098 100644 --- a/samples/kprobes/kprobe_example.c +++ b/samples/kprobes/kprobe_example.c @@ -15,8 +15,10 @@ #include #include #include +#include +#include -#define MAX_SYMBOL_LEN 64 +#define MAX_SYMBOL_LEN 4096 static char symbol[MAX_SYMBOL_LEN] = "kernel_clone"; module_param_string(symbol, symbol, sizeof(symbol), 0644); @@ -97,17 +99,54 @@ static void __kprobes handler_post(struct kprobe *p, struct pt_regs *regs, static int __init kprobe_init(void) { + char **symbols = NULL; int ret; + kp.pre_handler = handler_pre; kp.post_handler = handler_post; +#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE + if (strchr(symbol, ',')) { + char *p, *tmp; + int cnt; + + tmp = kstrdup(symbol, GFP_KERNEL); + if (!tmp) + return -ENOMEM; + + p = strchr(tmp, ','); + while (p) { + *p = ' '; + p = strchr(p + 1, ','); + } + + symbols = argv_split(GFP_KERNEL, tmp, &cnt); + kfree(tmp); + if (!symbols) { + ret = -ENOMEM; + goto out; + } + + kp.multi.symbols = (const char **) symbols; + kp.multi.cnt = cnt; + } +#endif + ret = register_kprobe(&kp); if (ret < 0) { pr_err("register_kprobe failed, returned %d\n", ret); - return ret; + goto out; } - pr_info("Planted kprobe at %p\n", kp.addr); - return 0; + + if (symbols) + pr_info("Planted multi kprobe to %s\n", symbol); + else + pr_info("Planted kprobe at %p\n", kp.addr); + +out: + if (symbols) + argv_free(symbols); + return ret; } static void __exit kprobe_exit(void) From patchwork Tue Jan 4 08:09:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703216 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC353C433FE for ; Tue, 4 Jan 2022 08:10:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234023AbiADIKl (ORCPT ); Tue, 4 Jan 2022 03:10:41 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:46582 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233968AbiADIKa (ORCPT ); Tue, 4 Jan 2022 03:10:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283829; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JbESpHRfLlaK1ly+p+PbLjsNNtlw/IID6NZUFBkK2J0=; b=Qjd5QNyiKntvc0fWzJhcTZHFe3o3o/nL4bOwi3BU1ibe3iBo14yrYeqY9aWqk3x06lj8c1 7XvBkDhPwTqU+Pt4ym0XgA9fObnFyfuMtKjXMr6Jn+vAE2v00Lla+LosNtpPV3v452KgsF jJ/nekahypvVF4GwUHSphLF22SppQVg= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-353-Wz5N3saoPtu203LK8iWxoA-1; Tue, 04 Jan 2022 03:10:28 -0500 X-MC-Unique: Wz5N3saoPtu203LK8iWxoA-1 Received: by mail-ed1-f72.google.com with SMTP id z8-20020a056402274800b003f8580bfb99so24712537edd.11 for ; Tue, 04 Jan 2022 00:10:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JbESpHRfLlaK1ly+p+PbLjsNNtlw/IID6NZUFBkK2J0=; b=ewUud4Ryz+vOwEEF/H9Kd34/QUnp/5xl+qcPEpdZVys3/FcLyEQXVq2A25lcxns8oT IP0Rb7Rh6J05Zo7/IDHKmvrA3l1wDqmIzK73cK0P2dm0VRlHL8qJQJKjFu6URzpEYu9b 02fomCI3kYGNuTyBtlXTJzcu2v5GgIWpI8lVJ+nzIWSiCaQdvdCTgYC6OzeR6UaLa27P n8/ZamXJsKJOF0NjHk4iuEdHGO/BPjP0JEar8U25xKgW0kKfXimoCScnipHxE7Ki8E3/ k2vWfALC7zK9fwVKd/XnpDwscG1rx5qQTFlcLy9LoTpLRybaPJ/b2K8IjKgpNlpCrHbw IWSw== X-Gm-Message-State: AOAM531H736EFYcL0bVL61eYoTvI0fESHD/lCr6L+J2QKIKjMbblwKHn BmYx5ugVX1tG4bHHr9/TZoVxhswVHQTdDYjjpIn296ntF4E/bjAKGhioLrzIRFghpu5AteF2mFo SI5Zy+4etZSLkwpFd X-Received: by 2002:a17:907:2d28:: with SMTP id gs40mr37541397ejc.765.1641283827377; Tue, 04 Jan 2022 00:10:27 -0800 (PST) X-Google-Smtp-Source: ABdhPJyWSI+W+Gb0MYq2Q3RCIFt9uJG0i7vLA8kleMXMypThKFPfIgJ/6256a7Vq6em8dK5Du3tmQw== X-Received: by 2002:a17:907:2d28:: with SMTP id gs40mr37541383ejc.765.1641283827248; Tue, 04 Jan 2022 00:10:27 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id ne31sm11347262ejc.48.2022.01.04.00.10.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:10:27 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 07/13] samples/kprobes: Add support for multi kretprobe interface Date: Tue, 4 Jan 2022 09:09:37 +0100 Message-Id: <20220104080943.113249-8-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding support to test multi kprobe interface. It's now possible to register multiple functions for the module handler, like: # modprobe kretprobe_example.ko func='sched_fork,kernel_clone' Signed-off-by: Jiri Olsa --- samples/kprobes/kretprobe_example.c | 43 +++++++++++++++++++++++++++-- 1 file changed, 41 insertions(+), 2 deletions(-) diff --git a/samples/kprobes/kretprobe_example.c b/samples/kprobes/kretprobe_example.c index 228321ecb161..2181cf0d6e4a 100644 --- a/samples/kprobes/kretprobe_example.c +++ b/samples/kprobes/kretprobe_example.c @@ -25,6 +25,8 @@ #include #include #include +#include +#include static char func_name[NAME_MAX] = "kernel_clone"; module_param_string(func, func_name, NAME_MAX, S_IRUGO); @@ -80,17 +82,54 @@ static struct kretprobe my_kretprobe = { static int __init kretprobe_init(void) { + char **symbols = NULL; int ret; my_kretprobe.kp.symbol_name = func_name; + +#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE + if (strchr(func_name, ',')) { + char *p, *tmp; + int cnt; + + tmp = kstrdup(func_name, GFP_KERNEL); + if (!tmp) + return -ENOMEM; + + p = strchr(tmp, ','); + while (p) { + *p = ' '; + p = strchr(p + 1, ','); + } + + symbols = argv_split(GFP_KERNEL, tmp, &cnt); + kfree(tmp); + if (!symbols) { + ret = -ENOMEM; + goto out; + } + + my_kretprobe.kp.multi.symbols = (const char **) symbols; + my_kretprobe.kp.multi.cnt = cnt; + } +#endif ret = register_kretprobe(&my_kretprobe); if (ret < 0) { pr_err("register_kretprobe failed, returned %d\n", ret); return ret; } - pr_info("Planted return probe at %s: %p\n", + + if (symbols) { + pr_info("Planted multi return kprobe to %s\n", func_name); + } else { + pr_info("Planted return probe at %s: %p\n", my_kretprobe.kp.symbol_name, my_kretprobe.kp.addr); - return 0; + } + +out: + if (symbols) + argv_free(symbols); + return ret; } static void __exit kretprobe_exit(void) From patchwork Tue Jan 4 08:09:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703217 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE82CC4332F for ; Tue, 4 Jan 2022 08:10:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234101AbiADIKv (ORCPT ); Tue, 4 Jan 2022 03:10:51 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:36132 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233996AbiADIKg (ORCPT ); Tue, 4 Jan 2022 03:10:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283836; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d9dNAaTWcvbIp4vHuV0yxIeciw50tXTE8p33fm7sGes=; b=Cv397FB+u/d4skte4TTvo6XFvmTEjAi/zabY+gpFhMS2XU0HZ50G1r/euhkHDnKebOz18H NzkWiKIlxncEEQZEd3ac/hZUGWfK2Mkkt6lpnVuXN7y2OoGAxmObgx6moIeQUatYmBBQQB 2K4Mb3f/FLuYNl5iYjpFHhnBbnR0Kp4= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-326-uNiPLcijMDWsgh8lii5mMw-1; Tue, 04 Jan 2022 03:10:34 -0500 X-MC-Unique: uNiPLcijMDWsgh8lii5mMw-1 Received: by mail-ed1-f70.google.com with SMTP id g2-20020a056402424200b003f8ee03207eso17850948edb.7 for ; Tue, 04 Jan 2022 00:10:34 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d9dNAaTWcvbIp4vHuV0yxIeciw50tXTE8p33fm7sGes=; b=K4kbw2pzZOP3gXQAvXaD4NN2FUwnM5v6Pmb/10asWtEVDDALg6gu/mwzkgpPSVE6rO sIzYpLN8BLD0PEGOfJUrOGu4KBz7S14P1IjIkgsR1ARiyR4eYTAibH/4Ec+xqepjtr6K x75dvbw37FzXC3gXcZpfxYs8UPCQd+4IbhpoX+nqdaXRWZvpv4/mdh9JWOwzeqHR/DgT cP80+s7gS2Wl19YqxCTxtu4EG5KSKRHBh9+WPp0nv+FBVWoGj4P2pSO5whW9+YmGwAQ3 iaoXDQ0zzIPriMoVIFWvURiBXMvqB8ve7gxzir7CR4zI1odBpcicnJZPJPN+jvCvhBP3 v/hA== X-Gm-Message-State: AOAM532dqr/cj/Youk5A8ptjtovYZN1lGw8Kz5/SL7uL0lnOjaGiIwQU EMblTT3o7VszpGMqZ6w65wKu+aywACR9+9WGSSsqYAXtJu0PjEhkGHSN5KsZUnEh0itpP5+05r7 HvsrQgVVyW3x4rsxu X-Received: by 2002:a05:6402:26c8:: with SMTP id x8mr48451573edd.149.1641283833514; Tue, 04 Jan 2022 00:10:33 -0800 (PST) X-Google-Smtp-Source: ABdhPJyQypSKunCsrhQU8Y396D/mlBqh08hdLEKOlVGSjC9sDM2Zp3llwS8LfSli6cij6nxfwCzcOA== X-Received: by 2002:a05:6402:26c8:: with SMTP id x8mr48451552edd.149.1641283833298; Tue, 04 Jan 2022 00:10:33 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id zh8sm11174574ejb.1.2022.01.04.00.10.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:10:33 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 08/13] bpf: Add kprobe link for attaching raw kprobes Date: Tue, 4 Jan 2022 09:09:38 +0100 Message-Id: <20220104080943.113249-9-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding new link type BPF_LINK_TYPE_KPROBE to attach kprobes directly through register_kprobe/kretprobe API. Adding new attach type BPF_TRACE_RAW_KPROBE that enables such link for kprobe program. The new link allows to create multiple kprobes link by using new link_create interface: struct { __aligned_u64 addrs; __u32 cnt; __u64 bpf_cookie; } kprobe; Plus new flag BPF_F_KPROBE_RETURN for link_create.flags to create return probe. Signed-off-by: Jiri Olsa --- include/linux/bpf_types.h | 1 + include/uapi/linux/bpf.h | 12 +++ kernel/bpf/syscall.c | 191 ++++++++++++++++++++++++++++++++- tools/include/uapi/linux/bpf.h | 12 +++ 4 files changed, 211 insertions(+), 5 deletions(-) diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h index 48a91c51c015..a9000feab34e 100644 --- a/include/linux/bpf_types.h +++ b/include/linux/bpf_types.h @@ -140,3 +140,4 @@ BPF_LINK_TYPE(BPF_LINK_TYPE_XDP, xdp) #ifdef CONFIG_PERF_EVENTS BPF_LINK_TYPE(BPF_LINK_TYPE_PERF_EVENT, perf) #endif +BPF_LINK_TYPE(BPF_LINK_TYPE_KPROBE, kprobe) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index b0383d371b9a..5216b333c688 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -995,6 +995,7 @@ enum bpf_attach_type { BPF_SK_REUSEPORT_SELECT, BPF_SK_REUSEPORT_SELECT_OR_MIGRATE, BPF_PERF_EVENT, + BPF_TRACE_RAW_KPROBE, __MAX_BPF_ATTACH_TYPE }; @@ -1009,6 +1010,7 @@ enum bpf_link_type { BPF_LINK_TYPE_NETNS = 5, BPF_LINK_TYPE_XDP = 6, BPF_LINK_TYPE_PERF_EVENT = 7, + BPF_LINK_TYPE_KPROBE = 8, MAX_BPF_LINK_TYPE, }; @@ -1111,6 +1113,11 @@ enum bpf_link_type { */ #define BPF_F_SLEEPABLE (1U << 4) +/* link_create flags used in LINK_CREATE command for BPF_TRACE_RAW_KPROBE + * attach type. + */ +#define BPF_F_KPROBE_RETURN (1U << 0) + /* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * @@ -1465,6 +1472,11 @@ union bpf_attr { */ __u64 bpf_cookie; } perf_event; + struct { + __aligned_u64 addrs; + __u32 cnt; + __u64 bpf_cookie; + } kprobe; }; } link_create; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index fa4505f9b611..7e5be63c820f 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -32,6 +32,7 @@ #include #include #include +#include #define IS_FD_ARRAY(map) ((map)->map_type == BPF_MAP_TYPE_PERF_EVENT_ARRAY || \ (map)->map_type == BPF_MAP_TYPE_CGROUP_ARRAY || \ @@ -3014,8 +3015,178 @@ static int bpf_perf_link_attach(const union bpf_attr *attr, struct bpf_prog *pro fput(perf_file); return err; } +#else +static int bpf_perf_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) +{ + return -ENOTSUPP; +} #endif /* CONFIG_PERF_EVENTS */ +#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE + +struct bpf_kprobe_link { + struct bpf_link link; + struct kretprobe rp; + bool is_return; + kprobe_opcode_t **addrs; + u32 cnt; + u64 bpf_cookie; +}; + +static void bpf_kprobe_link_release(struct bpf_link *link) +{ + struct bpf_kprobe_link *kprobe_link; + + kprobe_link = container_of(link, struct bpf_kprobe_link, link); + + if (kprobe_link->is_return) + unregister_kretprobe(&kprobe_link->rp); + else + unregister_kprobe(&kprobe_link->rp.kp); +} + +static void bpf_kprobe_link_dealloc(struct bpf_link *link) +{ + struct bpf_kprobe_link *kprobe_link; + + kprobe_link = container_of(link, struct bpf_kprobe_link, link); + kfree(kprobe_link->addrs); + kfree(kprobe_link); +} + +static const struct bpf_link_ops bpf_kprobe_link_lops = { + .release = bpf_kprobe_link_release, + .dealloc = bpf_kprobe_link_dealloc, +}; + +static int kprobe_link_prog_run(struct bpf_kprobe_link *kprobe_link, + struct pt_regs *regs) +{ + struct bpf_trace_run_ctx run_ctx; + struct bpf_run_ctx *old_run_ctx; + int err; + + if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1)) { + err = 0; + goto out; + } + + old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx); + run_ctx.bpf_cookie = kprobe_link->bpf_cookie; + + rcu_read_lock(); + migrate_disable(); + err = bpf_prog_run(kprobe_link->link.prog, regs); + migrate_enable(); + rcu_read_unlock(); + + bpf_reset_run_ctx(old_run_ctx); + + out: + __this_cpu_dec(bpf_prog_active); + return err; +} + +static int kprobe_dispatcher(struct kprobe *kp, struct pt_regs *regs) +{ + struct bpf_kprobe_link *kprobe_link; + + kprobe_link = container_of(kp, struct bpf_kprobe_link, rp.kp); + return kprobe_link_prog_run(kprobe_link, regs); +} + +static int +kretprobe_dispatcher(struct kretprobe_instance *ri, struct pt_regs *regs) +{ + struct kretprobe *rp = get_kretprobe(ri); + struct bpf_kprobe_link *kprobe_link; + + kprobe_link = container_of(rp, struct bpf_kprobe_link, rp); + return kprobe_link_prog_run(kprobe_link, regs); +} + +static int bpf_kprobe_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) +{ + struct bpf_link_primer link_primer; + struct bpf_kprobe_link *link; + kprobe_opcode_t **addrs; + u32 flags, cnt, size; + void __user *uaddrs; + u64 **tmp; + int err; + + flags = attr->link_create.flags; + if (flags & ~BPF_F_KPROBE_RETURN) + return -EINVAL; + + uaddrs = u64_to_user_ptr(attr->link_create.kprobe.addrs); + cnt = attr->link_create.kprobe.cnt; + size = cnt * sizeof(*tmp); + + tmp = kzalloc(size, GFP_KERNEL); + if (!tmp) + return -ENOMEM; + + if (copy_from_user(tmp, uaddrs, size)) { + err = -EFAULT; + goto error; + } + + /* TODO add extra copy for 32bit archs */ + if (sizeof(u64) != sizeof(void *)) + return -EINVAL; + + addrs = (kprobe_opcode_t **) tmp; + + link = kzalloc(sizeof(*link), GFP_KERNEL); + if (!link) + return -ENOMEM; + + bpf_link_init(&link->link, BPF_LINK_TYPE_KPROBE, &bpf_kprobe_link_lops, prog); + + err = bpf_link_prime(&link->link, &link_primer); + if (err) { + kfree(link); + goto error; + } + + INIT_HLIST_NODE(&link->rp.kp.hlist); + INIT_LIST_HEAD(&link->rp.kp.list); + link->is_return = flags & BPF_F_KPROBE_RETURN; + link->addrs = addrs; + link->cnt = cnt; + link->bpf_cookie = attr->link_create.kprobe.bpf_cookie; + + link->rp.kp.multi.addrs = addrs; + link->rp.kp.multi.cnt = cnt; + + if (link->is_return) + link->rp.handler = kretprobe_dispatcher; + else + link->rp.kp.pre_handler = kprobe_dispatcher; + + if (link->is_return) + err = register_kretprobe(&link->rp); + else + err = register_kprobe(&link->rp.kp); + + if (err) { + bpf_link_cleanup(&link_primer); + return err; + } + return bpf_link_settle(&link_primer); + +error: + kfree(tmp); + return err; +} +#else /* !CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE */ +static int bpf_kprobe_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) +{ + return -ENOTSUPP; +} +#endif + #define BPF_RAW_TRACEPOINT_OPEN_LAST_FIELD raw_tracepoint.prog_fd static int bpf_raw_tracepoint_open(const union bpf_attr *attr) @@ -4242,7 +4413,7 @@ static int tracing_bpf_link_attach(const union bpf_attr *attr, bpfptr_t uattr, return -EINVAL; } -#define BPF_LINK_CREATE_LAST_FIELD link_create.iter_info_len +#define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe.bpf_cookie static int link_create(union bpf_attr *attr, bpfptr_t uattr) { enum bpf_prog_type ptype; @@ -4266,7 +4437,6 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) ret = tracing_bpf_link_attach(attr, uattr, prog); goto out; case BPF_PROG_TYPE_PERF_EVENT: - case BPF_PROG_TYPE_KPROBE: case BPF_PROG_TYPE_TRACEPOINT: if (attr->link_create.attach_type != BPF_PERF_EVENT) { ret = -EINVAL; @@ -4274,6 +4444,14 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) } ptype = prog->type; break; + case BPF_PROG_TYPE_KPROBE: + if (attr->link_create.attach_type != BPF_PERF_EVENT && + attr->link_create.attach_type != BPF_TRACE_RAW_KPROBE) { + ret = -EINVAL; + goto out; + } + ptype = prog->type; + break; default: ptype = attach_type_to_prog_type(attr->link_create.attach_type); if (ptype == BPF_PROG_TYPE_UNSPEC || ptype != prog->type) { @@ -4305,13 +4483,16 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) ret = bpf_xdp_link_attach(attr, prog); break; #endif -#ifdef CONFIG_PERF_EVENTS case BPF_PROG_TYPE_PERF_EVENT: case BPF_PROG_TYPE_TRACEPOINT: - case BPF_PROG_TYPE_KPROBE: ret = bpf_perf_link_attach(attr, prog); break; -#endif + case BPF_PROG_TYPE_KPROBE: + if (attr->link_create.attach_type == BPF_PERF_EVENT) + ret = bpf_perf_link_attach(attr, prog); + else + ret = bpf_kprobe_link_attach(attr, prog); + break; default: ret = -EINVAL; } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index b0383d371b9a..5216b333c688 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -995,6 +995,7 @@ enum bpf_attach_type { BPF_SK_REUSEPORT_SELECT, BPF_SK_REUSEPORT_SELECT_OR_MIGRATE, BPF_PERF_EVENT, + BPF_TRACE_RAW_KPROBE, __MAX_BPF_ATTACH_TYPE }; @@ -1009,6 +1010,7 @@ enum bpf_link_type { BPF_LINK_TYPE_NETNS = 5, BPF_LINK_TYPE_XDP = 6, BPF_LINK_TYPE_PERF_EVENT = 7, + BPF_LINK_TYPE_KPROBE = 8, MAX_BPF_LINK_TYPE, }; @@ -1111,6 +1113,11 @@ enum bpf_link_type { */ #define BPF_F_SLEEPABLE (1U << 4) +/* link_create flags used in LINK_CREATE command for BPF_TRACE_RAW_KPROBE + * attach type. + */ +#define BPF_F_KPROBE_RETURN (1U << 0) + /* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * @@ -1465,6 +1472,11 @@ union bpf_attr { */ __u64 bpf_cookie; } perf_event; + struct { + __aligned_u64 addrs; + __u32 cnt; + __u64 bpf_cookie; + } kprobe; }; } link_create; From patchwork Tue Jan 4 08:09:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703218 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04002C433EF for ; Tue, 4 Jan 2022 08:10:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234042AbiADIKz (ORCPT ); Tue, 4 Jan 2022 03:10:55 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:33172 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233960AbiADIKm (ORCPT ); Tue, 4 Jan 2022 03:10:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283841; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k9zmsPfmvqNNuMjvxyy1I9bmPA8hKeQFPYbvFjf3I4A=; b=D9cT3aaa/ZVK/+1jLaqFjzdWMsFmDqMpqtG/8sFwVoDUiRTT6HbjYTC+we/GRtKX1Z5GEy G/jKy29Y6lcvk5Sa2lqydlGO+QFs7k/OQ8XeB1hw2uqfJ1zgd/JtG8k99FFuT803JstmQ4 6M7KFPrUb3K5R2gAV9gAQVehSyK9eW8= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-522-WXVBgq84N4OgjgYpxSVWGw-1; Tue, 04 Jan 2022 03:10:40 -0500 X-MC-Unique: WXVBgq84N4OgjgYpxSVWGw-1 Received: by mail-ed1-f69.google.com with SMTP id w6-20020a05640234c600b003f916e1b615so15501775edc.17 for ; Tue, 04 Jan 2022 00:10:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=k9zmsPfmvqNNuMjvxyy1I9bmPA8hKeQFPYbvFjf3I4A=; b=FIYnat96HPZeWdl4PeToCBwnWG82Yp2lq6mIlgqejacicKl0yUUjHPIqPDMFzo713d mitkTqweguVh55HwGt9bAW/7do0WIG8W3a1mArFlnhSWMcGvarJDluO14VUCng0ibyJe PpvVYRLu4W/8r9sUsF/imf+8tkUWUHskVgacwks9c33iMOJIZCDsL3HRrF4J1MAa2GO7 /ZVWePg4bivmSo8uOT+OCQa/yK25KpOsfPjmn9aGTIAH+xEM3p324JF6Er/RB9ns7vSA pKKO0UCkPfKaY/78uP8H1G36W2yqTbpuleNgM+jEmqJyR8bJgLtHDKv4K/DgKbeCnpD8 Xz1w== X-Gm-Message-State: AOAM532MWsKlm0WeXfsKVkKxJJn1MPaQdgnlrTxFa3tTE4K6HsEjQVlT l2+CeLGwsQtyYaptqea+VOYVRVsBgLwuD0jph7ZGe4mFzMNtoifQ/4b95vH6OTRIxi7fUDzixvr BDiLMCnfeKwkGGvhr X-Received: by 2002:a17:906:ced9:: with SMTP id si25mr41488671ejb.77.1641283839575; Tue, 04 Jan 2022 00:10:39 -0800 (PST) X-Google-Smtp-Source: ABdhPJzi6A/ih8FmZYPaSWBAndlTnAjTuC5CIwYwH/5ES92eoOzDAHcM0nZAphH3RYwfLOv0tUGPwA== X-Received: by 2002:a17:906:ced9:: with SMTP id si25mr41488662ejb.77.1641283839376; Tue, 04 Jan 2022 00:10:39 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id hd43sm1415724ejc.62.2022.01.04.00.10.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:10:39 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 09/13] libbpf: Add libbpf__kallsyms_parse function Date: Tue, 4 Jan 2022 09:09:39 +0100 Message-Id: <20220104080943.113249-10-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Move the kallsyms parsing in internal libbpf__kallsyms_parse function, so it can be used from other places. It will be used in following changes. Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 62 ++++++++++++++++++++------------- tools/lib/bpf/libbpf_internal.h | 5 +++ 2 files changed, 43 insertions(+), 24 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 9cb99d1e2385..25512b4dbc8c 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -7146,12 +7146,10 @@ static int bpf_object__sanitize_maps(struct bpf_object *obj) return 0; } -static int bpf_object__read_kallsyms_file(struct bpf_object *obj) +int libbpf__kallsyms_parse(void *arg, kallsyms_cb_t cb) { char sym_type, sym_name[500]; unsigned long long sym_addr; - const struct btf_type *t; - struct extern_desc *ext; int ret, err = 0; FILE *f; @@ -7170,35 +7168,51 @@ static int bpf_object__read_kallsyms_file(struct bpf_object *obj) if (ret != 3) { pr_warn("failed to read kallsyms entry: %d\n", ret); err = -EINVAL; - goto out; + break; } - ext = find_extern_by_name(obj, sym_name); - if (!ext || ext->type != EXT_KSYM) - continue; - - t = btf__type_by_id(obj->btf, ext->btf_id); - if (!btf_is_var(t)) - continue; - - if (ext->is_set && ext->ksym.addr != sym_addr) { - pr_warn("extern (ksym) '%s' resolution is ambiguous: 0x%llx or 0x%llx\n", - sym_name, ext->ksym.addr, sym_addr); - err = -EINVAL; - goto out; - } - if (!ext->is_set) { - ext->is_set = true; - ext->ksym.addr = sym_addr; - pr_debug("extern (ksym) %s=0x%llx\n", sym_name, sym_addr); - } + err = cb(arg, sym_addr, sym_type, sym_name); + if (err) + break; } -out: fclose(f); return err; } +static int kallsyms_cb(void *arg, unsigned long long sym_addr, + char sym_type, const char *sym_name) +{ + struct bpf_object *obj = arg; + const struct btf_type *t; + struct extern_desc *ext; + + ext = find_extern_by_name(obj, sym_name); + if (!ext || ext->type != EXT_KSYM) + return 0; + + t = btf__type_by_id(obj->btf, ext->btf_id); + if (!btf_is_var(t)) + return 0; + + if (ext->is_set && ext->ksym.addr != sym_addr) { + pr_warn("extern (ksym) '%s' resolution is ambiguous: 0x%llx or 0x%llx\n", + sym_name, ext->ksym.addr, sym_addr); + return -EINVAL; + } + if (!ext->is_set) { + ext->is_set = true; + ext->ksym.addr = sym_addr; + pr_debug("extern (ksym) %s=0x%llx\n", sym_name, sym_addr); + } + return 0; +} + +static int bpf_object__read_kallsyms_file(struct bpf_object *obj) +{ + return libbpf__kallsyms_parse(obj, kallsyms_cb); +} + static int find_ksym_btf_id(struct bpf_object *obj, const char *ksym_name, __u16 kind, struct btf **res_btf, struct module_btf **res_mod_btf) diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h index 1565679eb432..dc544d239509 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -446,6 +446,11 @@ __s32 btf__find_by_name_kind_own(const struct btf *btf, const char *type_name, extern enum libbpf_strict_mode libbpf_mode; +typedef int (*kallsyms_cb_t)(void *arg, unsigned long long sym_addr, + char sym_type, const char *sym_name); + +int libbpf__kallsyms_parse(void *arg, kallsyms_cb_t cb); + /* handle direct returned errors */ static inline int libbpf_err(int ret) { From patchwork Tue Jan 4 08:09:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703219 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED062C433F5 for ; Tue, 4 Jan 2022 08:11:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233933AbiADILB (ORCPT ); Tue, 4 Jan 2022 03:11:01 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:25729 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234011AbiADIKs (ORCPT ); Tue, 4 Jan 2022 03:10:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283848; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z04l+1thMGEo6+2wn8HMi5HKrAqA7BNf+MRnLwscczA=; b=f+ccUphmbclKoVmjOTIiD0OpsClWH//sQIMDkK6RufQ8ak+elcUpihHifuWp060GzhKjvX VJZTq79uoCZy1CHCPoXas0wL43dvbbDJepFjsK5joDHSqkIg/fCfgDA/f2dFwIe/LzBdV5 1td4yLDB53lOeREAXMGxOS784WYymZQ= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-567-UcgkRv94OXe9nBngE_P8hQ-1; Tue, 04 Jan 2022 03:10:46 -0500 X-MC-Unique: UcgkRv94OXe9nBngE_P8hQ-1 Received: by mail-ed1-f72.google.com with SMTP id dm10-20020a05640222ca00b003f808b5aa18so24680630edb.4 for ; Tue, 04 Jan 2022 00:10:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=z04l+1thMGEo6+2wn8HMi5HKrAqA7BNf+MRnLwscczA=; b=ExHFMVVznQD92lnLxgDDGZSZ5OIYEwseIY78ssKvewHdAPnkatiygF7oVHt4obC/ll /xa11pQaks84PZO+SXRciYWlhlHmeBU3bshnan6tF5wMD3ob0JNNUpNhlhU1LPescgvQ Bo36zkOeIF4CWqF2y0mz6b76u9unFuUrVJuybk7z+a5GRdBc4hYgAU21AISCyIS8GcoS 2wBt5VtFaMd3ay44SQdDdWiTipzQOJg1Lk/1oNU/ulwA9UolNhML6/nbI7dU1zC1CMqa kv5GMty3xiK0RaQKbQfjb8WsFAKbTsD66mR0+Rgbb9bNHESwsgPgHb0hCupGn5DHu9ot 7ebA== X-Gm-Message-State: AOAM531lia1Vc/7yUsw2iMwFXtuzFHe6RKqbQQhwgsstJDei2MmD8mWt yoY0MsUyDnsOgNMor4uVQOaHjxnYHMrj4hd2A2whbqYz8RtBbRfnsPN9Grf/gZWevLi5ah2Z4b7 eSHZIia/HlamKAMWY X-Received: by 2002:aa7:cada:: with SMTP id l26mr48146089edt.376.1641283845536; Tue, 04 Jan 2022 00:10:45 -0800 (PST) X-Google-Smtp-Source: ABdhPJzFkiq5V/GxTl7wd6cp9nRQDMK16ZHw3L6bqox/Q7gvurc8JFtIApm+yKLHFcDKbqDu4mUuiw== X-Received: by 2002:aa7:cada:: with SMTP id l26mr48146070edt.376.1641283845430; Tue, 04 Jan 2022 00:10:45 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id 21sm11279609ejx.83.2022.01.04.00.10.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:10:45 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 10/13] libbpf: Add bpf_link_create support for multi kprobes Date: Tue, 4 Jan 2022 09:09:40 +0100 Message-Id: <20220104080943.113249-11-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding new kprobe struct in bpf_link_create_opts object to pass multi kprobe data to link_create attr API. Signed-off-by: Jiri Olsa --- tools/lib/bpf/bpf.c | 5 +++++ tools/lib/bpf/bpf.h | 7 ++++++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 9b64eed2b003..40cad575ad62 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -848,6 +848,11 @@ int bpf_link_create(int prog_fd, int target_fd, if (!OPTS_ZEROED(opts, perf_event)) return libbpf_err(-EINVAL); break; + case BPF_TRACE_RAW_KPROBE: + attr.link_create.kprobe.addrs = OPTS_GET(opts, kprobe.addrs, 0); + attr.link_create.kprobe.cnt = OPTS_GET(opts, kprobe.cnt, 0); + attr.link_create.kprobe.bpf_cookie = OPTS_GET(opts, kprobe.bpf_cookie, 0); + break; default: if (!OPTS_ZEROED(opts, flags)) return libbpf_err(-EINVAL); diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h index 00619f64a040..9611023138b1 100644 --- a/tools/lib/bpf/bpf.h +++ b/tools/lib/bpf/bpf.h @@ -298,10 +298,15 @@ struct bpf_link_create_opts { struct { __u64 bpf_cookie; } perf_event; + struct { + __u64 addrs; + __u32 cnt; + __u64 bpf_cookie; + } kprobe; }; size_t :0; }; -#define bpf_link_create_opts__last_field perf_event +#define bpf_link_create_opts__last_field kprobe.bpf_cookie LIBBPF_API int bpf_link_create(int prog_fd, int target_fd, enum bpf_attach_type attach_type, From patchwork Tue Jan 4 08:09:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703220 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20094C433F5 for ; Tue, 4 Jan 2022 08:11:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234118AbiADILF (ORCPT ); Tue, 4 Jan 2022 03:11:05 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:55211 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234130AbiADIKy (ORCPT ); Tue, 4 Jan 2022 03:10:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283854; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YmRKjJYYnr2jeX5kW8bAyoIlm+DRH3mC592LzQxJkXE=; b=f9v0z0lm8yEwpfWln55xXGAR1ufSKPlve61SJF0T7hlTZamzJkV8OZbVjcVhwK7tUAjn9l nGcuMVHpEt+mQdf6CJL5PfBHV8qwCWKf04bd19MI+UL0kNgux3+VbZVBvLX5kInPN+PoBR 9AiucxoXqiAyn9L0Ssw2LwbAsT6i82M= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-93-8q0JdkaTOz6g7GJH-VkjYw-1; Tue, 04 Jan 2022 03:10:53 -0500 X-MC-Unique: 8q0JdkaTOz6g7GJH-VkjYw-1 Received: by mail-ed1-f71.google.com with SMTP id z8-20020a056402274800b003f8580bfb99so24713292edd.11 for ; Tue, 04 Jan 2022 00:10:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YmRKjJYYnr2jeX5kW8bAyoIlm+DRH3mC592LzQxJkXE=; b=e1/YJKHJ7mIU6JZYk8yW6qzQqbBThqmn6yI4+5l3iBxNT8bqdG5XnFrc4ym0q0nZ3o UHsxbkB/qdYlelDfvRZTbPbifAyWvtvic61GFL5e7IW73yMDvJ60nJHMX8V6RK9VXy7q UkNAGODSVFwUGl03XMHMgP2KHmtPsWs87qkv+mx4Y8PSYC2Q2cosYrlRJS2sjwb01IDb 0cqFcE7Wpv6DFUm+EOy7R8g3CDPYPuvBUuj7Eaxoa5mPI6TJLUgSOHolmfnRAB22BBlR kKZe0oz2VK++AyMObLmLWTXe/ESiVrBXygJ5ZsZC+JrbkzBxbvqs0SWypWb1K2r0z97b wG5A== X-Gm-Message-State: AOAM530LCXnuqsjnA09LdphH3+BNqGu1VGIo82AWiVHnfD/+izaKtHwD ZLeXSs/XT0rZ+e674yp6PeYF1ZlIYDw4hQVoMI3q1m6JLqpVGwEdOT4RxCnz+DcqhJcQYukU2HK TVMgm9XdfBj1tdMOZ X-Received: by 2002:a17:906:c147:: with SMTP id dp7mr37784166ejc.173.1641283851757; Tue, 04 Jan 2022 00:10:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJwrJiWwHVZWR9cEMfNpKPhzTTeCzxB6S/iuRTpAQN/EOTKmHH18Ismc6bCp+pYKntGFgT+Z6Q== X-Received: by 2002:a17:906:c147:: with SMTP id dp7mr37784148ejc.173.1641283851527; Tue, 04 Jan 2022 00:10:51 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id c3sm14400378edr.33.2022.01.04.00.10.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:10:51 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 11/13] libbpf: Add bpf_program__attach_kprobe_opts for multi kprobes Date: Tue, 4 Jan 2022 09:09:41 +0100 Message-Id: <20220104080943.113249-12-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net SEC("raw_kprobe/bpf_fentry_test*") to attach program to all bpf_fentry_test* functions. Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 124 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 123 insertions(+), 1 deletion(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 25512b4dbc8c..0061ab02fc5a 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -10000,6 +10000,125 @@ static int perf_event_kprobe_open_legacy(const char *probe_name, bool retprobe, return pfd; } +struct kprobe_resolve_raw_kprobe { + const char *name; + __u64 *addrs; + __u32 alloc; + __u32 cnt; +}; + +static bool glob_matches(const char *glob, const char *s) +{ + int n = strlen(glob); + + if (n == 1 && glob[0] == '*') + return true; + + if (glob[0] == '*' && glob[n - 1] == '*') { + const char *subs; + /* substring match */ + + /* this is hacky, but we don't want to allocate + * for no good reason + */ + ((char *)glob)[n - 1] = '\0'; + subs = strstr(s, glob + 1); + ((char *)glob)[n - 1] = '*'; + + return subs != NULL; + } else if (glob[0] == '*') { + size_t nn = strlen(s); + /* suffix match */ + + /* too short for a given suffix */ + if (nn < n - 1) + return false; + return strcmp(s + nn - (n - 1), glob + 1) == 0; + } else if (glob[n - 1] == '*') { + /* prefix match */ + return strncmp(s, glob, n - 1) == 0; + } else { + /* exact match */ + return strcmp(glob, s) == 0; + } +} + +static int +kprobe_resolve_raw_kprobe_cb(void *arg, unsigned long long sym_addr, + char sym_type, const char *sym_name) +{ + struct kprobe_resolve_raw_kprobe *res = arg; + __u64 *p; + + if (!glob_matches(res->name, sym_name)) + return 0; + + if (res->cnt == res->alloc) { + res->alloc = max((__u32) 16, res->alloc * 3 / 2); + p = libbpf_reallocarray(res->addrs, res->alloc, sizeof(__u32)); + if (!p) + return -ENOMEM; + res->addrs = p; + } + res->addrs[res->cnt++] = sym_addr; + return 0; +} + +static struct bpf_link * +attach_raw_kprobe_opts(const struct bpf_program *prog, + const char *func_name, + const struct bpf_kprobe_opts *kopts) +{ + DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts); + struct kprobe_resolve_raw_kprobe res = { + .name = func_name, + }; + struct bpf_link *link = NULL; + char errmsg[STRERR_BUFSIZE]; + int err, link_fd, prog_fd; + __u64 bpf_cookie; + bool retprobe; + + err = libbpf__kallsyms_parse(&res, kprobe_resolve_raw_kprobe_cb); + if (err) + goto error; + if (!res.cnt) { + err = -ENOENT; + goto error; + } + + retprobe = OPTS_GET(kopts, retprobe, false); + bpf_cookie = OPTS_GET(kopts, bpf_cookie, 0); + + opts.kprobe.addrs = (__u64) res.addrs; + opts.kprobe.cnt = res.cnt; + opts.kprobe.bpf_cookie = bpf_cookie; + opts.flags = retprobe ? BPF_F_KPROBE_RETURN : 0; + + link = calloc(1, sizeof(*link)); + if (!link) + return libbpf_err_ptr(-ENOMEM); + link->detach = &bpf_link__detach_fd; + + prog_fd = bpf_program__fd(prog); + link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_RAW_KPROBE, &opts); + if (link_fd < 0) { + err = -errno; + pr_warn("prog '%s': failed to attach to %s: %s\n", + prog->name, res.name, + libbpf_strerror_r(err, errmsg, sizeof(errmsg))); + goto error; + } + link->fd = link_fd; + free(res.addrs); + return link; + +error: + free(link); + free(res.addrs); + return libbpf_err_ptr(err); +} + struct bpf_link * bpf_program__attach_kprobe_opts(const struct bpf_program *prog, const char *func_name, @@ -10016,6 +10135,9 @@ bpf_program__attach_kprobe_opts(const struct bpf_program *prog, if (!OPTS_VALID(opts, bpf_kprobe_opts)) return libbpf_err_ptr(-EINVAL); + if (strchr(func_name, '*')) + return attach_raw_kprobe_opts(prog, func_name, opts); + retprobe = OPTS_GET(opts, retprobe, false); offset = OPTS_GET(opts, offset, 0); pe_opts.bpf_cookie = OPTS_GET(opts, bpf_cookie, 0); @@ -10096,7 +10218,7 @@ static struct bpf_link *attach_kprobe(const struct bpf_program *prog, long cooki else func_name = prog->sec_name + sizeof("kprobe/") - 1; - n = sscanf(func_name, "%m[a-zA-Z0-9_.]+%li", &func, &offset); + n = sscanf(func_name, "%m[a-zA-Z0-9_.*]+%li", &func, &offset); if (n < 1) { err = -EINVAL; pr_warn("kprobe name is invalid: %s\n", func_name); From patchwork Tue Jan 4 08:09:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703221 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAD08C433EF for ; Tue, 4 Jan 2022 08:11:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229867AbiADILY (ORCPT ); Tue, 4 Jan 2022 03:11:24 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55821 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234006AbiADILB (ORCPT ); Tue, 4 Jan 2022 03:11:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283860; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uDTf/DGOXxIRO6kbaoXtydGSDRn5G2e6XLZL3XY5bwo=; b=fENM+a/lWCRoTH+IzKvPDiJ8PPchrPlGbbQJUtZyCMjULZ/1jn5PwDXO90XWhovlnef8Mq oJJuAbd7HLTWP0g0AgtKUtfzVYOxusDH3j37/sUkl9cpxm6Nxqb+KFPnojkI8pAkBMcjeZ vgQ88swtJEdXS2Wd6kWXSLsGpP/NHWU= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-466-apv1CmUyOyuvVad2ASJlqw-1; Tue, 04 Jan 2022 03:10:59 -0500 X-MC-Unique: apv1CmUyOyuvVad2ASJlqw-1 Received: by mail-ed1-f71.google.com with SMTP id h6-20020a056402280600b003f9967993aeso3843284ede.10 for ; Tue, 04 Jan 2022 00:10:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uDTf/DGOXxIRO6kbaoXtydGSDRn5G2e6XLZL3XY5bwo=; b=aEBzRphOUXiIpliKRljyG/AABxp3WEy8yUiCQ4H8nwGfm3Vt1AoYMgWDhjnOIygtJ7 18Brwk8XJaTzG0h+IH+ypGFSW0CtZMjHFXn0SzUKsX7GxWWkH16bx5KUASQS582kF8tZ Q3tSgjK98CjMTNCu46owQBPd6Uv2xFSdeoaTIR4OelVd5Kz5IbyItgJofzoj/+FEAiWd QtkEWGc8V3eeCozhaEyalI0KZwHjGiPjnjRkBsY0lk1XZvNQ+r1pIPrnL/SXlG0iyo+V SWkKB3y2N/+40u6bvkFzYhmSeBDl1pzxXqRL/ZXm1p//PIpBth418P/GVC/RimtIu0Qw b/zQ== X-Gm-Message-State: AOAM530N1gLfSy2NYEsTg62oQbT3b8QaOBJmPJLJSKqu0q4s9Bk2K+Ge bmQeJq6dNjIjF+3SwN7WgvQ8NMV06HDwXQSet/EbZVFYVHyNE0wqHpP+42YXpKBC9rWfvZtKdAJ KtVaLVlsW3tkGC65l X-Received: by 2002:aa7:d783:: with SMTP id s3mr47284050edq.172.1641283857964; Tue, 04 Jan 2022 00:10:57 -0800 (PST) X-Google-Smtp-Source: ABdhPJxBMvHVw8fcuawLiGOLreMDQWzlHCbdPW8CKjAs+4VUl7EscPuiyQQnMoKFCTlLhYONR9SQ9g== X-Received: by 2002:aa7:d783:: with SMTP id s3mr47284028edq.172.1641283857793; Tue, 04 Jan 2022 00:10:57 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id hb17sm11158384ejc.195.2022.01.04.00.10.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:10:57 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 12/13] selftest/bpf: Add raw kprobe attach test Date: Tue, 4 Jan 2022 09:09:42 +0100 Message-Id: <20220104080943.113249-13-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding raw kprobe attach test that uses new interface to attach multiple kprobes with new kprobe link. The test is attaching to bpf_fentry_test* functions and single trampoline program to use bpf_prog_test_run to trigger bpf_fentry_test* functions. Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/raw_kprobe_test.c | 92 +++++++++++++++++++ .../testing/selftests/bpf/progs/raw_kprobe.c | 58 ++++++++++++ 2 files changed, 150 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/raw_kprobe_test.c create mode 100644 tools/testing/selftests/bpf/progs/raw_kprobe.c diff --git a/tools/testing/selftests/bpf/prog_tests/raw_kprobe_test.c b/tools/testing/selftests/bpf/prog_tests/raw_kprobe_test.c new file mode 100644 index 000000000000..5ade44c57c9e --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/raw_kprobe_test.c @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include "raw_kprobe.skel.h" +#include "trace_helpers.h" + +static void test_skel_api(void) +{ + struct raw_kprobe *skel = NULL; + __u32 duration = 0, retval; + int err, prog_fd; + + skel = raw_kprobe__open_and_load(); + if (!ASSERT_OK_PTR(skel, "raw_kprobe__open_and_load")) + goto cleanup; + + err = raw_kprobe__attach(skel); + if (!ASSERT_OK(err, "raw_kprobe__attach")) + goto cleanup; + + prog_fd = bpf_program__fd(skel->progs.test1); + err = bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + ASSERT_OK(err, "test_run"); + ASSERT_EQ(retval, 0, "test_run"); + + ASSERT_EQ(skel->bss->test2_result, 8, "test2_result"); + ASSERT_EQ(skel->bss->test3_result, 8, "test3_result"); + +cleanup: + raw_kprobe__destroy(skel); +} + +static void test_link_api(void) +{ + DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts); + int err, prog_fd, link1_fd = -1, link2_fd = -1; + struct raw_kprobe *skel = NULL; + __u32 duration = 0, retval; + __u64 addrs[8]; + + skel = raw_kprobe__open_and_load(); + if (!ASSERT_OK_PTR(skel, "fentry_raw_skel_load")) + goto cleanup; + + kallsyms_find("bpf_fentry_test1", &addrs[0]); + kallsyms_find("bpf_fentry_test2", &addrs[1]); + kallsyms_find("bpf_fentry_test3", &addrs[2]); + kallsyms_find("bpf_fentry_test4", &addrs[3]); + kallsyms_find("bpf_fentry_test5", &addrs[4]); + kallsyms_find("bpf_fentry_test6", &addrs[5]); + kallsyms_find("bpf_fentry_test7", &addrs[6]); + kallsyms_find("bpf_fentry_test8", &addrs[7]); + + opts.kprobe.addrs = (__u64) addrs; + opts.kprobe.cnt = 8; + + prog_fd = bpf_program__fd(skel->progs.test2); + link1_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_RAW_KPROBE, &opts); + if (!ASSERT_GE(link1_fd, 0, "link_fd")) + goto cleanup; + + opts.flags = BPF_F_KPROBE_RETURN; + prog_fd = bpf_program__fd(skel->progs.test3); + link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_RAW_KPROBE, &opts); + if (!ASSERT_GE(link2_fd, 0, "link_fd")) + goto cleanup; + + skel->bss->test2_result = 0; + skel->bss->test3_result = 0; + + prog_fd = bpf_program__fd(skel->progs.test1); + err = bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + ASSERT_OK(err, "test_run"); + ASSERT_EQ(retval, 0, "test_run"); + + ASSERT_EQ(skel->bss->test2_result, 8, "test2_result"); + ASSERT_EQ(skel->bss->test3_result, 8, "test3_result"); + +cleanup: + if (link1_fd != -1) + close(link1_fd); + if (link2_fd != -1) + close(link2_fd); + raw_kprobe__destroy(skel); +} + +void test_raw_kprobe_test(void) +{ + test_skel_api(); + test_link_api(); +} diff --git a/tools/testing/selftests/bpf/progs/raw_kprobe.c b/tools/testing/selftests/bpf/progs/raw_kprobe.c new file mode 100644 index 000000000000..baf7086203f9 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/raw_kprobe.c @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +extern const void bpf_fentry_test1 __ksym; +extern const void bpf_fentry_test2 __ksym; +extern const void bpf_fentry_test3 __ksym; +extern const void bpf_fentry_test4 __ksym; +extern const void bpf_fentry_test5 __ksym; +extern const void bpf_fentry_test6 __ksym; +extern const void bpf_fentry_test7 __ksym; +extern const void bpf_fentry_test8 __ksym; + +/* No tests, just to trigger bpf_fentry_test* through tracing test_run */ +SEC("fentry/bpf_modify_return_test") +int BPF_PROG(test1) +{ + return 0; +} + +__u64 test2_result = 0; + +SEC("kprobe/bpf_fentry_test*") +int test2(struct pt_regs *ctx) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test2_result += (const void *) addr == &bpf_fentry_test1 || + (const void *) addr == &bpf_fentry_test2 || + (const void *) addr == &bpf_fentry_test3 || + (const void *) addr == &bpf_fentry_test4 || + (const void *) addr == &bpf_fentry_test5 || + (const void *) addr == &bpf_fentry_test6 || + (const void *) addr == &bpf_fentry_test7 || + (const void *) addr == &bpf_fentry_test8; + return 0; +} + +__u64 test3_result = 0; + +SEC("kretprobe/bpf_fentry_test*") +int test3(struct pt_regs *ctx) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test3_result += (const void *) addr == &bpf_fentry_test1 || + (const void *) addr == &bpf_fentry_test2 || + (const void *) addr == &bpf_fentry_test3 || + (const void *) addr == &bpf_fentry_test4 || + (const void *) addr == &bpf_fentry_test5 || + (const void *) addr == &bpf_fentry_test6 || + (const void *) addr == &bpf_fentry_test7 || + (const void *) addr == &bpf_fentry_test8; + return 0; +} From patchwork Tue Jan 4 08:09:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12703222 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4305FC433EF for ; Tue, 4 Jan 2022 08:11:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234058AbiADIL2 (ORCPT ); Tue, 4 Jan 2022 03:11:28 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:44322 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233949AbiADILI (ORCPT ); Tue, 4 Jan 2022 03:11:08 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641283868; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n1Sn68YOpfQdWqtYtUrgCV+sac3UqaI1SOKuBByB+TA=; b=Wr7BspMT4Wy8BClTlj9lsyG274ejeHucLvlStKKnW0/HhOsT9t8t2weypvcpNq1uRY8aNF 2VD8kQmCZV0bhBMbUcsVhNwsAIWuUh21LJ9Umrn7KW7w+Gu3290Dqx/6IKNm+X/zDXunCc B3Q/ZdEYgcc5C5dxuzPjHo9i+a8MtG0= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-153-Hy9X927pMimG2-HyavG2Vg-1; Tue, 04 Jan 2022 03:11:05 -0500 X-MC-Unique: Hy9X927pMimG2-HyavG2Vg-1 Received: by mail-ed1-f72.google.com with SMTP id q15-20020a056402518f00b003f87abf9c37so24637464edd.15 for ; Tue, 04 Jan 2022 00:11:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=n1Sn68YOpfQdWqtYtUrgCV+sac3UqaI1SOKuBByB+TA=; b=0Xprpp/5khLGVmjA7XGveJlP2hZa+93YRHyp/UcH8krDYWk5h0FOiUlSnkL1cQpAxr q1QsAt4fB234aT/aSlKuhiEXsFMaRPqtRbUDVhnltvDBEW4ZyCxMFd5MQWP4YRxx+2gH PbMtEeAKE89nF6xQJUft1Nun7jC56MfkANjZNx9N4YAxijSxar2MFffsc4TpAtiV0jMh LxHVJf2MgLgvhYE/h6NWn/pEtKMpHPH2LJ6jk9qDNeJs27tFh5LpiD/D+9areRlG0I4K BzwANlV53SGtVlC4UwtNXifXV2MkwiWLJkzGRoiEsqgurkJopOncEOYh1ETFRDfoHBpn 6uCQ== X-Gm-Message-State: AOAM530b6hHM8hDWWaAhB24llEyifcwYeM6h9sU0BXY5Pl9QaprOx/X4 rN4H8HrxwC7Fw/mDZR17M4vgZm9OzbGBH2QkfbbdCL+HX1k5mwvcf9WMTcobpq6Wfzng9+7Cd2X Ttko/lUkm1hS79VkK X-Received: by 2002:a17:907:3fa0:: with SMTP id hr32mr40732550ejc.196.1641283864067; Tue, 04 Jan 2022 00:11:04 -0800 (PST) X-Google-Smtp-Source: ABdhPJxY0vOWIbWrowC6YTp97SbKilX2ZYMN/pSg5z78CfP/qlYGdHEk+p7bdrBYems4kOONH6Q1LQ== X-Received: by 2002:a17:907:3fa0:: with SMTP id hr32mr40732544ejc.196.1641283863934; Tue, 04 Jan 2022 00:11:03 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id v8sm14318341edt.10.2022.01.04.00.11.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jan 2022 00:11:03 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" Subject: [PATCH 13/13] selftest/bpf: Add bpf_cookie test for raw_k[ret]probe Date: Tue, 4 Jan 2022 09:09:43 +0100 Message-Id: <20220104080943.113249-14-jolsa@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220104080943.113249-1-jolsa@kernel.org> References: <20220104080943.113249-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding bpf_cookie test for raw_k[ret]probes. Using the bpf_link_create interface directly and single trampoline program to trigger the bpf_fentry_test1 execution. Signed-off-by: Jiri Olsa --- .../selftests/bpf/prog_tests/bpf_cookie.c | 42 +++++++++++++++++++ .../selftests/bpf/progs/test_bpf_cookie.c | 24 ++++++++++- 2 files changed, 65 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c b/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c index 5eea3c3a40fe..aee01dd3a823 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c @@ -57,6 +57,46 @@ static void kprobe_subtest(struct test_bpf_cookie *skel) bpf_link__destroy(retlink2); } +static void rawkprobe_subtest(struct test_bpf_cookie *skel) +{ + DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts); + int err, prog_fd, link1_fd = -1, link2_fd = -1; + __u32 duration = 0, retval; + __u64 addr; + + kallsyms_find("bpf_fentry_test1", &addr); + + opts.kprobe.addrs = (__u64) &addr; + opts.kprobe.cnt = 1; + opts.kprobe.bpf_cookie = 0x1; + prog_fd = bpf_program__fd(skel->progs.handle_raw_kprobe); + + link1_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_RAW_KPROBE, &opts); + if (!ASSERT_GE(link1_fd, 0, "link1_fd")) + return; + + opts.flags = BPF_F_KPROBE_RETURN; + opts.kprobe.bpf_cookie = 0x2; + prog_fd = bpf_program__fd(skel->progs.handle_raw_kretprobe); + + link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_RAW_KPROBE, &opts); + if (!ASSERT_GE(link2_fd, 0, "link2_fd")) + goto cleanup; + + prog_fd = bpf_program__fd(skel->progs.raw_trigger); + err = bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + ASSERT_OK(err, "test_run"); + ASSERT_EQ(retval, 0, "test_run"); + + ASSERT_EQ(skel->bss->raw_kprobe_res, 0x1, "raw_kprobe_res"); + ASSERT_EQ(skel->bss->raw_kretprobe_res, 0x2, "raw_kretprobe_res"); + +cleanup: + close(link1_fd); + close(link2_fd); +} + static void uprobe_subtest(struct test_bpf_cookie *skel) { DECLARE_LIBBPF_OPTS(bpf_uprobe_opts, opts); @@ -243,6 +283,8 @@ void test_bpf_cookie(void) if (test__start_subtest("kprobe")) kprobe_subtest(skel); + if (test__start_subtest("rawkprobe")) + rawkprobe_subtest(skel); if (test__start_subtest("uprobe")) uprobe_subtest(skel); if (test__start_subtest("tracepoint")) diff --git a/tools/testing/selftests/bpf/progs/test_bpf_cookie.c b/tools/testing/selftests/bpf/progs/test_bpf_cookie.c index 2d3a7710e2ce..409f87464b1f 100644 --- a/tools/testing/selftests/bpf/progs/test_bpf_cookie.c +++ b/tools/testing/selftests/bpf/progs/test_bpf_cookie.c @@ -8,8 +8,9 @@ int my_tid; int kprobe_res; -int kprobe_multi_res; int kretprobe_res; +int raw_kprobe_res; +int raw_kretprobe_res; int uprobe_res; int uretprobe_res; int tp_res; @@ -37,6 +38,27 @@ int handle_kretprobe(struct pt_regs *ctx) return 0; } +SEC("kprobe/bpf_fentry_test1") +int handle_raw_kprobe(struct pt_regs *ctx) +{ + update(ctx, &raw_kprobe_res); + return 0; +} + +SEC("kretprobe/bpf_fentry_test1") +int handle_raw_kretprobe(struct pt_regs *ctx) +{ + update(ctx, &raw_kretprobe_res); + return 0; +} + +/* just to trigger bpf_fentry_test1 through tracing test_run */ +SEC("fentry/bpf_modify_return_test") +int BPF_PROG(raw_trigger) +{ + return 0; +} + SEC("uprobe/trigger_func") int handle_uprobe(struct pt_regs *ctx) {