From patchwork Wed Jun 19 22:48:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13704696 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F8FAC2BA1A for ; Wed, 19 Jun 2024 22:49:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 592706B01B1; Wed, 19 Jun 2024 18:49:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D16C8D0092; Wed, 19 Jun 2024 18:49:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D1988D0091; Wed, 19 Jun 2024 18:49:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D45266B00BD for ; Wed, 19 Jun 2024 18:49:48 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 944EB161603 for ; Wed, 19 Jun 2024 22:49:48 +0000 (UTC) X-FDA: 82249132056.03.F671EF2 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf10.hostedemail.com (Postfix) with ESMTP id 572BDC000C for ; Wed, 19 Jun 2024 22:49:46 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=mtxCY5NO; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=2+cvj+i4; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=mtxCY5NO; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=2+cvj+i4; spf=pass (imf10.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718837381; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GJmQfB+SFKwkGljPkMoSf+PzrKg2yTJDdkhioKlFbR0=; b=O0fYYgZaPPgveOlghgyR9xFJpmsqD3XVgES5e8Mco2Ngu0ZEvSJVRmERXZ63Ajwmy3a/QY G+CyqCRpqpn22Ssc8xrESu2K6q4bVJXAuQK+V1u4rc+zU4j3sPnQ4T/BxduGuW1KFYZHXQ unZbUo107vTAy7nmvt1WY030P5S3mho= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=mtxCY5NO; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=2+cvj+i4; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=mtxCY5NO; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=2+cvj+i4; spf=pass (imf10.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718837381; a=rsa-sha256; cv=none; b=uN4PoUJoVi7n8uGexXXdN9o3UwSfFZHMr/HhlGKWhiFPjRjaWi589UwtlTv3bakZBZxyTr FvUmsEMvmpBqWRhCt8N7ZBA3ZCoSdo+wwAGj9Q4w5hbD83CoLEzSGE7vTxfifAoC43v21y om1/F2UQLuDPlTtei7N3S5eal1XAkK4= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5C51F21A5C; Wed, 19 Jun 2024 22:49:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1718837383; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GJmQfB+SFKwkGljPkMoSf+PzrKg2yTJDdkhioKlFbR0=; b=mtxCY5NOZUMLeNOQo2490WnZzrZz5lK6g9Qm7Ex4Shmq2UVpwBxwZP+nWMnZFY2Qk8nBs1 siLzlqttayrh4IykYENtS8rfrWC8ksDwhasAHMwn4I+ihPTkA8zNMcN6XlS3RF8s2wohXI U+CXOQeOPkjiu8wHU+JqsGy5rejV9nA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1718837383; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GJmQfB+SFKwkGljPkMoSf+PzrKg2yTJDdkhioKlFbR0=; b=2+cvj+i4DhZ6XKrKA7se2fWikyWEQBDFIpBjv+eA7nIqPwGQ03LS2LkiZhUbMyFqEGyfvQ qHnQkg3JJUtkfiDQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1718837383; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GJmQfB+SFKwkGljPkMoSf+PzrKg2yTJDdkhioKlFbR0=; b=mtxCY5NOZUMLeNOQo2490WnZzrZz5lK6g9Qm7Ex4Shmq2UVpwBxwZP+nWMnZFY2Qk8nBs1 siLzlqttayrh4IykYENtS8rfrWC8ksDwhasAHMwn4I+ihPTkA8zNMcN6XlS3RF8s2wohXI U+CXOQeOPkjiu8wHU+JqsGy5rejV9nA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1718837383; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GJmQfB+SFKwkGljPkMoSf+PzrKg2yTJDdkhioKlFbR0=; b=2+cvj+i4DhZ6XKrKA7se2fWikyWEQBDFIpBjv+eA7nIqPwGQ03LS2LkiZhUbMyFqEGyfvQ qHnQkg3JJUtkfiDQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 3D08413AAA; Wed, 19 Jun 2024 22:49:43 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SCSeDodgc2aFIAAAD6G6ig (envelope-from ); Wed, 19 Jun 2024 22:49:43 +0000 From: Vlastimil Babka Date: Thu, 20 Jun 2024 00:48:57 +0200 Subject: [PATCH v2 3/7] bpf: support error injection static keys for perf_event attached progs MIME-Version: 1.0 Message-Id: <20240620-fault-injection-statickeys-v2-3-e23947d3d84b@suse.cz> References: <20240620-fault-injection-statickeys-v2-0-e23947d3d84b@suse.cz> In-Reply-To: <20240620-fault-injection-statickeys-v2-0-e23947d3d84b@suse.cz> To: Akinobu Mita , Christoph Lameter , David Rientjes , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" , Masami Hiramatsu , Steven Rostedt , Mark Rutland Cc: Jiri Olsa , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4256; i=vbabka@suse.cz; h=from:subject:message-id; bh=RHlcQH9C6ly4CbQFrHL5k0pueBGzmfFJHg6uosgSnqE=; b=owEBbQGS/pANAwAIAbvgsHXSRYiaAcsmYgBmc2B4jX+g3Xf4kDO7RN55ImULUpPLAS31+cJIL VWj6k4LZzmJATMEAAEIAB0WIQR7u8hBFZkjSJZITfG74LB10kWImgUCZnNgeAAKCRC74LB10kWI mk8pCACesvfkTPT71qvqKXfyLJtHc/HZ29/KezJP030k0GlFWTkG41XeSOf+Jc9jxoC1871A4SK EWFDuN7tbL8sEOdT7cBJU0MBEXQSDEYNDcoGyZxl98cLsAJ9XHVdkhNOzgJJgPRwcnPT10ejR6X Eb39gdyBoKUdHWD8IoRYiCpG6/Wx8PRQUjK59shva6E514qQRimzOvre6ZR5OQrcynjumq8WrVE n12/Bf0XbXqhrgaM1emVNUHExQn1U7iCYwHGEzlGPNvC8Mi+wQG9Xsq7xvjA5OSOUmmmDusj3eC 3soz/74FQINFipML/55CmhoPIFrOsjPyQbm5FZDYbWm25i7L X-Developer-Key: i=vbabka@suse.cz; a=openpgp; fpr=A940D434992C2E8E99103D50224FA7E7CC82A664 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 572BDC000C X-Stat-Signature: 7gh54nc9cs9zdkfqgbzps737knuhmg71 X-Rspam-User: X-HE-Tag: 1718837386-535326 X-HE-Meta: U2FsdGVkX18tsdHnyEXXnN5epwr/czOc8XHe9l/WzBJcJ/izvw1sJRWH3OiReLi+Lt6blCWEjd/mo5S4CChK/5MRHbMr0sRCcmEkgIAYjyJ5FeSW9qccYSerDmGzyFzVLr8A9XwqkzPKumg8F+nYneUJRP7m3la7rATgssMKgQOyhbOxnj9FNra1Mx+MgcrA4taa/2P8SeIULeF6ErVfM/AqaslOBFYkcdZYBCYsb9r3lUH6s1x5/ocaXuoJ1mpzcrLhIVU3U8j2uKK79HjfrVQugXxP1wLujY3FlXfgCvkJXnA6ETpI3cg7WtAvJ9llfuGtrVK4xXi1x9SLdJf6rPx4Wd5U8Vx3W/U8RZe+ZWbRT5FfQawUSHwRYgnp/jj0Tlyg0E4o8AIjC24USzdENfZgyvliTK33AV4c9sBntT+IJgcJyLYDIAzyfc/WqHbZi4bcxz5QKeJmDiR+BgGcn5YFNi5iZeD9CE4It7kx4C7jdiRjOoGxkm4sjQIcTzzcCPxnB6gom0buMdXxrzh4OXAt7uw7GLBxBH39tQSoPIB3PytwvStKhW/DnU3tpbf22H2t2mj/L7h454BneHtksdCqS3yfkEGnuPAiXsjbn5kEPtvy59tvEA6tIl27WUAPCq2DJ0oQG2Vb0L3YiCydf6USbHuT4hfoxPK4MQmdAkyeiLhZVPcvrmL9cuEwf9fwOD2+4+VycXZa+Qpgn45l07e5AXF9MypJiqQGqupwNKe5G0fKupnqhAUWdNr14hVAnSJetIYad2USc5XBmtbooyv+cbcm8LVA3YC7bBedzEx+CHOJhyRttASQXGoliWH7UGwQGJqnkZm2K0IP3KfaARhuZUtozu7nJZed+SYTmmW0pZegyoigXm9Pw5h7QQZnyE4ccindiXZ9IUAXhRkgwMf50SlfKidXZJ7OVmWOZhFUjYzLW4X6OC8YNl1VQ7lIa6EZTwpuIMMpYw/ZS/w Qshq+LXb GB3f12WnqOEfvDgzBB79WWPMu1cnpX0OsO2JQRUvolD4XLvaWNCMivIvxKNouba1f17dNvGOCUKNyLzHBP8cu54MD4+mFZyjStY94OSrXHSK/KSuPudApw8CP32EGFkdwc++UfWK2LNYaK3LglazjFVi2fQyutfCxVyjEwsIMiHj/LD4NHqzc71se+W8gfUm71i5HW4wWf8W93T2EMBHriiZd6U2LO1BYBiPrJ4u8eQP+8MAihSMxMIxT7+mIhLj8F3lDqUS8gLg8xEYD4OA1xodd9apkcxWxB9S2Xxtd97q5s8KCvZG4suZhxuPNHAaNv94alDV5lZF6Zr31iC8Xri3PW3YnPBHct3e9 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Functions marked for error injection can have an associated static key that guards the callsite(s) to avoid overhead of calling an empty function when no error injection is in progress. Outside of the error injection framework itself, bpf programs can be atteched to perf events and override results of error-injectable functions. To make sure these functions are actually called, attaching such bpf programs should control the static key accordingly. Therefore, add the static key's address to struct trace_kprobe and fill it in trace_kprobe_error_injectable(), using get_injection_key() instead of within_error_injection_list(). Introduce trace_kprobe_error_injection_control() to control the static key and call the control function when attaching or detaching programs with kprobe_override to perf events. Signed-off-by: Vlastimil Babka --- kernel/trace/bpf_trace.c | 6 ++++++ kernel/trace/trace_kprobe.c | 30 ++++++++++++++++++++++++++++-- kernel/trace/trace_probe.h | 5 +++++ 3 files changed, 39 insertions(+), 2 deletions(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index f5154c051d2c..944de1c41209 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -2283,6 +2283,9 @@ int perf_event_attach_bpf_prog(struct perf_event *event, rcu_assign_pointer(event->tp_event->prog_array, new_array); bpf_prog_array_free_sleepable(old_array); + if (prog->kprobe_override) + trace_kprobe_error_injection_control(event->tp_event, true); + unlock: mutex_unlock(&bpf_event_mutex); return ret; @@ -2299,6 +2302,9 @@ void perf_event_detach_bpf_prog(struct perf_event *event) if (!event->prog) goto unlock; + if (event->prog->kprobe_override) + trace_kprobe_error_injection_control(event->tp_event, false); + old_array = bpf_event_rcu_dereference(event->tp_event->prog_array); ret = bpf_prog_array_copy(old_array, event->prog, NULL, 0, &new_array); if (ret == -ENOENT) diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c index 16383247bdbf..1c1ee95bd5de 100644 --- a/kernel/trace/trace_kprobe.c +++ b/kernel/trace/trace_kprobe.c @@ -61,6 +61,7 @@ struct trace_kprobe { unsigned long __percpu *nhit; const char *symbol; /* symbol name */ struct trace_probe tp; + struct static_key *ei_key; }; static bool is_trace_kprobe(struct dyn_event *ev) @@ -235,9 +236,34 @@ bool trace_kprobe_on_func_entry(struct trace_event_call *call) bool trace_kprobe_error_injectable(struct trace_event_call *call) { struct trace_kprobe *tk = trace_kprobe_primary_from_call(call); + struct static_key *ei_key; - return tk ? within_error_injection_list(trace_kprobe_address(tk)) : - false; + if (!tk) + return false; + + ei_key = get_injection_key(trace_kprobe_address(tk)); + if (IS_ERR(ei_key)) + return false; + + tk->ei_key = ei_key; + return true; +} + +void trace_kprobe_error_injection_control(struct trace_event_call *call, + bool enable) +{ + struct trace_kprobe *tk = trace_kprobe_primary_from_call(call); + + if (!tk) + return; + + if (!tk->ei_key) + return; + + if (enable) + static_key_slow_inc(tk->ei_key); + else + static_key_slow_dec(tk->ei_key); } static int register_kprobe_event(struct trace_kprobe *tk); diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h index 5803e6a41570..d9ddcabb9f97 100644 --- a/kernel/trace/trace_probe.h +++ b/kernel/trace/trace_probe.h @@ -212,6 +212,8 @@ DECLARE_BASIC_PRINT_TYPE_FUNC(symbol); #ifdef CONFIG_KPROBE_EVENTS bool trace_kprobe_on_func_entry(struct trace_event_call *call); bool trace_kprobe_error_injectable(struct trace_event_call *call); +void trace_kprobe_error_injection_control(struct trace_event_call *call, + bool enabled); #else static inline bool trace_kprobe_on_func_entry(struct trace_event_call *call) { @@ -222,6 +224,9 @@ static inline bool trace_kprobe_error_injectable(struct trace_event_call *call) { return false; } + +static inline void trace_kprobe_error_injection_control(struct trace_event_call *call, + bool enabled) { } #endif /* CONFIG_KPROBE_EVENTS */ struct probe_arg {