From patchwork Fri Jan 19 00:13:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kyle Huey X-Patchwork-Id: 13523263 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2936510F4 for ; Fri, 19 Jan 2024 00:14:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705623255; cv=none; b=HDRSme98QTyWDiHt1KowrA6Kc8uqwvyE4Ix26LocoVpOn9OPRvijwNYxXHJGxwB0D1TK6WfNwWpAYgN7Qoy+Hy3BYAIaI3Sf4zMKt8TbNvbRO29QwPGc5HomGXvEsx79DuYazQj6DjPbmt0YexDZU36msms6jAaINObTvHIXsq0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705623255; c=relaxed/simple; bh=BvdPhHhuuf/fRX3SqKFdqqVaVqqf5FQjd8NlByquZYU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Rwr/uAmv6YncLqVM0SCkBndWAhYIP93pUbRBM/pfei2k/Vb1PdenwuDK7UhUCjAVzhmsGCTDrZsS4Ihn1AHyQWlTyL5jnjC+2F9O0bRYsc2dlSnbnIASR5wcLlvERailhZMlJxURV+VM+Y/rvRDqC0lOOKrVB+bdcGEY7pL03AA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com; spf=pass smtp.mailfrom=kylehuey.com; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b=g5vLseTw; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b="g5vLseTw" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-1d5f1e0e32eso1395885ad.3 for ; Thu, 18 Jan 2024 16:14:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kylehuey.com; s=google; t=1705623253; x=1706228053; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=goCRlABDp65iNRuy7Fdk99c/XuzP9RSm/x9LjyCmqK8=; b=g5vLseTwxcL1+e/Y4krZOtr6EtaiOwYUQSZuxbGwOUr8uMWz/nICL/yTIYe+NmfkvY x0lJAgXKVhLVmjFFCntcSbWlM7htz8Hf8jgvRizk46PvmwYOExB+gglUH16J+xor7cWK zMRZVkDvkuTiRFwLXPKaqy/+BDIhqynGoWKpZ3BWwurBeWF8yPMB/XyqcKd7IRFd7SfT ySZYS+yFFx8GtAVLI0/eTK3Q4KP1MeOChv/suiKL+1Y3mH1Q02gLPRbQvQKQVMcPYyZk lz+BPo9Z2og9ZGxLWEERNuffyOYNgJ8NabEbVnG8/VatSP7wWAvrdEoFpd/OlUVYjTPo fFTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705623253; x=1706228053; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=goCRlABDp65iNRuy7Fdk99c/XuzP9RSm/x9LjyCmqK8=; b=ONCgvMQ0nz2oqL7a9gSei1/C7jQq66g9wwEGKQkU2vXkvmUN82BEWej6Vicz0GhgGC JYILtyp2uhiaExb3Mw3xBRkJMNt4di5vVc4OFHh8O/nqHjcjB2NEILjibwrjVylrELII jcIhJHSwGvOg2AydPr6cWdKI7BNvdLEoIfsla47WnJIp9arOERL18x0F8Ej2qITcd1KL h1KBmsfqbBK2pJouE4gqAADsk3V2xo+b85/pyJ0Vv9MF+PNQljTOonpkGaXviprZzVD4 P09G9JbI9c8a1AqEIylQhR88EB172Rq5VJHXjbXPrP4r8kRlUmpKtKeV2Zh+Q0PTPBGJ sveg== X-Gm-Message-State: AOJu0YzRtkU8hVt4+Yr6GqjbsRpLG/67kHBjrdqHCOcWnGzDCewNm5ML 3BlJyELnz8jVp3SXQ3QnMRp9AadnivRozgPoQ3FLwrpEUGZRaY6ZbTn7ePLGQQ== X-Google-Smtp-Source: AGHT+IEoWDFJEWLLyfNOOcNAQO69wxXDeoE3NDFE/6nVWcyxeKdTpE9bWKe9TiAUfVBp/hdULye9zQ== X-Received: by 2002:a17:902:dac6:b0:1d4:441e:7377 with SMTP id q6-20020a170902dac600b001d4441e7377mr1612768plx.40.1705623253562; Thu, 18 Jan 2024 16:14:13 -0800 (PST) Received: from zhadum.home.kylehuey.com (c-76-126-33-191.hsd1.ca.comcast.net. [76.126.33.191]) by smtp.gmail.com with ESMTPSA id mj7-20020a1709032b8700b001d1d1ef8be6sm1921238plb.267.2024.01.18.16.14.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Jan 2024 16:14:13 -0800 (PST) From: Kyle Huey X-Google-Original-From: Kyle Huey To: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Song Liu Cc: Robert O'Callahan , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Ian Rogers , Adrian Hunter , linux-perf-users@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH v4 1/4] perf/bpf: Call bpf handler directly, not through overflow machinery Date: Thu, 18 Jan 2024 16:13:48 -0800 Message-Id: <20240119001352.9396-2-khuey@kylehuey.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240119001352.9396-1-khuey@kylehuey.com> References: <20240119001352.9396-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To ultimately allow bpf programs attached to perf events to completely suppress all of the effects of a perf event overflow (rather than just the sample output, as they do today), call bpf_overflow_handler() from __perf_event_overflow() directly rather than modifying struct perf_event's overflow_handler. Return the bpf program's return value from bpf_overflow_handler() so that __perf_event_overflow() knows how to proceed. Remove the now unnecessary orig_overflow_handler from struct perf_event. This patch is solely a refactoring and results in no behavior change. Signed-off-by: Kyle Huey Suggested-by: Namhyung Kim Acked-by: Song Liu --- include/linux/perf_event.h | 6 +----- kernel/events/core.c | 28 +++++++++++++++------------- 2 files changed, 16 insertions(+), 18 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index d2a15c0c6f8a..c7f54fd74d89 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -810,7 +810,6 @@ struct perf_event { perf_overflow_handler_t overflow_handler; void *overflow_handler_context; #ifdef CONFIG_BPF_SYSCALL - perf_overflow_handler_t orig_overflow_handler; struct bpf_prog *prog; u64 bpf_cookie; #endif @@ -1357,10 +1356,7 @@ __is_default_overflow_handler(perf_overflow_handler_t overflow_handler) #ifdef CONFIG_BPF_SYSCALL static inline bool uses_default_overflow_handler(struct perf_event *event) { - if (likely(is_default_overflow_handler(event))) - return true; - - return __is_default_overflow_handler(event->orig_overflow_handler); + return is_default_overflow_handler(event); } #else #define uses_default_overflow_handler(event) \ diff --git a/kernel/events/core.c b/kernel/events/core.c index f0f0f71213a1..24a718e7eb98 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -9548,6 +9548,12 @@ static inline bool sample_is_allowed(struct perf_event *event, struct pt_regs *r return true; } +#ifdef CONFIG_BPF_SYSCALL +static int bpf_overflow_handler(struct perf_event *event, + struct perf_sample_data *data, + struct pt_regs *regs); +#endif + /* * Generic event overflow handling, sampling. */ @@ -9617,7 +9623,10 @@ static int __perf_event_overflow(struct perf_event *event, irq_work_queue(&event->pending_irq); } - READ_ONCE(event->overflow_handler)(event, data, regs); +#ifdef CONFIG_BPF_SYSCALL + if (!(event->prog && !bpf_overflow_handler(event, data, regs))) +#endif + READ_ONCE(event->overflow_handler)(event, data, regs); if (*perf_event_fasync(event) && event->pending_kill) { event->pending_wakeup = 1; @@ -10427,9 +10436,9 @@ static void perf_event_free_filter(struct perf_event *event) } #ifdef CONFIG_BPF_SYSCALL -static void bpf_overflow_handler(struct perf_event *event, - struct perf_sample_data *data, - struct pt_regs *regs) +static int bpf_overflow_handler(struct perf_event *event, + struct perf_sample_data *data, + struct pt_regs *regs) { struct bpf_perf_event_data_kern ctx = { .data = data, @@ -10450,10 +10459,8 @@ static void bpf_overflow_handler(struct perf_event *event, rcu_read_unlock(); out: __this_cpu_dec(bpf_prog_active); - if (!ret) - return; - event->orig_overflow_handler(event, data, regs); + return ret; } static int perf_event_set_bpf_handler(struct perf_event *event, @@ -10489,8 +10496,6 @@ static int perf_event_set_bpf_handler(struct perf_event *event, event->prog = prog; event->bpf_cookie = bpf_cookie; - event->orig_overflow_handler = READ_ONCE(event->overflow_handler); - WRITE_ONCE(event->overflow_handler, bpf_overflow_handler); return 0; } @@ -10501,7 +10506,6 @@ static void perf_event_free_bpf_handler(struct perf_event *event) if (!prog) return; - WRITE_ONCE(event->overflow_handler, event->orig_overflow_handler); event->prog = NULL; bpf_prog_put(prog); } @@ -11975,13 +11979,11 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu, overflow_handler = parent_event->overflow_handler; context = parent_event->overflow_handler_context; #if defined(CONFIG_BPF_SYSCALL) && defined(CONFIG_EVENT_TRACING) - if (overflow_handler == bpf_overflow_handler) { + if (parent_event->prog) { struct bpf_prog *prog = parent_event->prog; bpf_prog_inc(prog); event->prog = prog; - event->orig_overflow_handler = - parent_event->orig_overflow_handler; } #endif } From patchwork Fri Jan 19 00:13:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kyle Huey X-Patchwork-Id: 13523264 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3167A184E for ; Fri, 19 Jan 2024 00:14:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705623258; cv=none; b=nrChSuWZztzdd9Cu8sB+aaLxzjH5N3dSs2WrVb3eRqM7vxjaoUEScozYIQxgDJtX3FK83OAhVV+JNpuwTgzniBJAnWhknx2kQ5lD3iNAafJsOzGrUHCfi/AxlvOesUB4/IRG0inDdLqPRaAAE52zFItL3uNczzsuohldeoaMFmA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705623258; c=relaxed/simple; bh=aE2XSvO8OhL2bKh+kya+x2v8NA+iHW4PCGN/ajZfLTA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XplUW4neBiVWWGlkQxN5enaaJ3LO7O8aYLwjY9pO7urnJbMuQX1O1rIua5l84EmJChYtog42f26M/Z/jNd+ZFRA+4l3FAuq+u/YUJK+TIc8ZBt8r58tynaKnOwu9ZNJEBVQEl9+CBv/w7K057JZM3jAYi3b/wXkGOvVkVmgf6Oo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com; spf=pass smtp.mailfrom=kylehuey.com; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b=k4RCG+1D; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b="k4RCG+1D" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-1d711d7a940so1700685ad.1 for ; Thu, 18 Jan 2024 16:14:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kylehuey.com; s=google; t=1705623256; x=1706228056; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TZcU1eaajijXdHtIWhayAvoytSB02YbTVp/d9IoCpKI=; b=k4RCG+1DXT7r+NHdYh2FBFIrU7lK/6+cDzG2SmFd1Vrb1s+N0L6ZesFliazSbah2nY A27FnLwf6VuPQDuWchzfkWJJgioV2vZaGMdU5TU8zzmtbW22I+JYpGgadFCOro6vWT5n WDMPPQJIJZBAUqLzVUZGbsaARHUvnVVg2Kd71iXIeu3t9APSxWWoBWdhdWf19JZXo46E u+RBT7nFnr93yRh3p4KD4RsxXRXLbDcYzdWMMQxtt57EpqkJgt1aZDVMpiiOkRtgG97q GQxteT02B6at5I1qxMiH9E52zHjv83QM4nHE8COxNDZ7C/gWE+m9cuicDD4/hLb9COD6 4OBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705623256; x=1706228056; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TZcU1eaajijXdHtIWhayAvoytSB02YbTVp/d9IoCpKI=; b=OlbBwh2HUczdx+x1ILwGnfB/FxacL5ifQQ8hwfCZ4Qi6z2XEwqaMBxuKVRV7B9XDxB eHPqDnbR3TYkRGN8+sE2vo4R//QIlbciyLrJEaW5+7tiOZURYI6eOk8I6GDkBEIQoWFt blwu+AUCs1aGwUmAqiJse8Mj8w9U7dCxmHEgdwHOALzO3sbKlNQREkJicpp+dY2Y82CQ DyaK9E7kogVkNt2GGYOE+jXjzAaVB19WjiwLaTPFviZwsw0Hz6fCT9yuW2bI9mSLOQcr gew7oPv8WF/xLd3OLcHBeTjJzoRo/m4M7nTptdn6sC6h3D8v/iah6/wtSttIJxSBy9a0 hK0g== X-Gm-Message-State: AOJu0Yylu96mgKk30/d/KCzcx6F/VRFAsyuG4M9qSUpQZg0EfcFe+IyN LAoYxhQy31sVSEYcwmAOBiNnPTCfNR51lRzlDeGRjpA9SzYFbTkktU1BW4+QKQ== X-Google-Smtp-Source: AGHT+IEmEX9cgoOoLlXa/PdbiT/2i5lxpr6nzytvpqJqWne6DoYaTAYruERi3LVO2S0pdvC6eXPZjA== X-Received: by 2002:a17:902:d486:b0:1d7:1b54:8969 with SMTP id c6-20020a170902d48600b001d71b548969mr518993plg.11.1705623256601; Thu, 18 Jan 2024 16:14:16 -0800 (PST) Received: from zhadum.home.kylehuey.com (c-76-126-33-191.hsd1.ca.comcast.net. [76.126.33.191]) by smtp.gmail.com with ESMTPSA id mj7-20020a1709032b8700b001d1d1ef8be6sm1921238plb.267.2024.01.18.16.14.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Jan 2024 16:14:16 -0800 (PST) From: Kyle Huey X-Google-Original-From: Kyle Huey To: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Song Liu Cc: Robert O'Callahan , Will Deacon , Mark Rutland , Russell King , Catalin Marinas , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Alexander Shishkin , Ian Rogers , Adrian Hunter , linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH v4 2/4] perf/bpf: Remove unneeded uses_default_overflow_handler. Date: Thu, 18 Jan 2024 16:13:49 -0800 Message-Id: <20240119001352.9396-3-khuey@kylehuey.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240119001352.9396-1-khuey@kylehuey.com> References: <20240119001352.9396-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Now that struct perf_event's orig_overflow_handler is gone, there's no need for the functions and macros to support looking past overflow_handler to orig_overflow_handler. This patch is solely a refactoring and results in no behavior change. Signed-off-by: Kyle Huey Acked-by: Will Deacon Acked-by: Song Liu --- arch/arm/kernel/hw_breakpoint.c | 8 ++++---- arch/arm64/kernel/hw_breakpoint.c | 4 ++-- include/linux/perf_event.h | 16 ++-------------- 3 files changed, 8 insertions(+), 20 deletions(-) diff --git a/arch/arm/kernel/hw_breakpoint.c b/arch/arm/kernel/hw_breakpoint.c index dc0fb7a81371..054e9199f30d 100644 --- a/arch/arm/kernel/hw_breakpoint.c +++ b/arch/arm/kernel/hw_breakpoint.c @@ -626,7 +626,7 @@ int hw_breakpoint_arch_parse(struct perf_event *bp, hw->address &= ~alignment_mask; hw->ctrl.len <<= offset; - if (uses_default_overflow_handler(bp)) { + if (is_default_overflow_handler(bp)) { /* * Mismatch breakpoints are required for single-stepping * breakpoints. @@ -798,7 +798,7 @@ static void watchpoint_handler(unsigned long addr, unsigned int fsr, * Otherwise, insert a temporary mismatch breakpoint so that * we can single-step over the watchpoint trigger. */ - if (!uses_default_overflow_handler(wp)) + if (!is_default_overflow_handler(wp)) continue; step: enable_single_step(wp, instruction_pointer(regs)); @@ -811,7 +811,7 @@ static void watchpoint_handler(unsigned long addr, unsigned int fsr, info->trigger = addr; pr_debug("watchpoint fired: address = 0x%x\n", info->trigger); perf_bp_event(wp, regs); - if (uses_default_overflow_handler(wp)) + if (is_default_overflow_handler(wp)) enable_single_step(wp, instruction_pointer(regs)); } @@ -886,7 +886,7 @@ static void breakpoint_handler(unsigned long unknown, struct pt_regs *regs) info->trigger = addr; pr_debug("breakpoint fired: address = 0x%x\n", addr); perf_bp_event(bp, regs); - if (uses_default_overflow_handler(bp)) + if (is_default_overflow_handler(bp)) enable_single_step(bp, addr); goto unlock; } diff --git a/arch/arm64/kernel/hw_breakpoint.c b/arch/arm64/kernel/hw_breakpoint.c index 35225632d70a..db2a1861bb97 100644 --- a/arch/arm64/kernel/hw_breakpoint.c +++ b/arch/arm64/kernel/hw_breakpoint.c @@ -654,7 +654,7 @@ static int breakpoint_handler(unsigned long unused, unsigned long esr, perf_bp_event(bp, regs); /* Do we need to handle the stepping? */ - if (uses_default_overflow_handler(bp)) + if (is_default_overflow_handler(bp)) step = 1; unlock: rcu_read_unlock(); @@ -733,7 +733,7 @@ static u64 get_distance_from_watchpoint(unsigned long addr, u64 val, static int watchpoint_report(struct perf_event *wp, unsigned long addr, struct pt_regs *regs) { - int step = uses_default_overflow_handler(wp); + int step = is_default_overflow_handler(wp); struct arch_hw_breakpoint *info = counter_arch_bp(wp); info->trigger = addr; diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index c7f54fd74d89..c8bd5bb6610c 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1341,8 +1341,9 @@ extern int perf_event_output(struct perf_event *event, struct pt_regs *regs); static inline bool -__is_default_overflow_handler(perf_overflow_handler_t overflow_handler) +is_default_overflow_handler(struct perf_event *event) { + perf_overflow_handler_t overflow_handler = event->overflow_handler; if (likely(overflow_handler == perf_event_output_forward)) return true; if (unlikely(overflow_handler == perf_event_output_backward)) @@ -1350,19 +1351,6 @@ __is_default_overflow_handler(perf_overflow_handler_t overflow_handler) return false; } -#define is_default_overflow_handler(event) \ - __is_default_overflow_handler((event)->overflow_handler) - -#ifdef CONFIG_BPF_SYSCALL -static inline bool uses_default_overflow_handler(struct perf_event *event) -{ - return is_default_overflow_handler(event); -} -#else -#define uses_default_overflow_handler(event) \ - is_default_overflow_handler(event) -#endif - extern void perf_event_header__init_id(struct perf_event_header *header, struct perf_sample_data *data, From patchwork Fri Jan 19 00:13:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kyle Huey X-Patchwork-Id: 13523265 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7052B2F37 for ; Fri, 19 Jan 2024 00:14:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705623260; cv=none; b=fPYlZqqqrD+5KPenlR64+r6+rU74nXb3I3pR1O1o4elxxbEWPfSrNXcBeQUG2G/qazcgSedw/h1KDXSM4gOtmQ5/Vtke9mxEbol43yINyE1flcf6xWV9r6ogf6prt2XYNTUGaePDXPuvx66zh0HdCVhj+fgIzzK41HK2g9gL+Do= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705623260; c=relaxed/simple; bh=3622UZiYTMAps9fbTXA59oDeFjLFPk6fc41GnG+9ElA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZCcvT150BGDNrhYgk5MF2nIJV8MxLJqFfsnr4sFZpTApVJWCJDzVcNLyFSXpfINhePdNZB9qNyIezOM7B0pqeKY6H70kAaeaXyh6H/IZHd0otLGtLjx0voolTwRiU6JXG+T1G+aeRNslKB6sRAAxMl+5MQSkk+gs8cbNpHJK26E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com; spf=pass smtp.mailfrom=kylehuey.com; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b=EAHGi4sB; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b="EAHGi4sB" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-1d5f56912daso1581255ad.2 for ; Thu, 18 Jan 2024 16:14:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kylehuey.com; s=google; t=1705623259; x=1706228059; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/hEmM3vmtzdKl0VwvFdVPj5xDGeYkLg3iqjbEyKv8xM=; b=EAHGi4sBFq4E70+oyXfcJgh8JejvoQfg6y70LujDnGNW/nQOK9LEZJMPMZcZYCYo1F 2ncaGimK3K2P/N05NrVZXPV4ips16A8D5lj31KH8UHVEAplH0q4u4wgoO4zN2uZdDxU0 SokB2JECL0+E2SY+ppU5oUWAJU+YvZ/QnM2xulqWen/U98PUR/EiplgxGSafz0xBUWa2 XU3TKx0QjHgl14oL5yg7K56xQUt2WjL8eQVZWBGKuicvbXvqCGET6lHctm35vbiskLTX nQz+dpMWZx2icGdjq5nrRfeDxyj/hDqnrddGgrjvqxxPFEaqnid6C1+SgQDpIj8JGHdJ 3YmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705623259; x=1706228059; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/hEmM3vmtzdKl0VwvFdVPj5xDGeYkLg3iqjbEyKv8xM=; b=rmQ8x47orGYL6Yze8WEzaT05c4c9cCt/5SS3cXqxhCzKtO3rk40RnOW+g6SR0XkxQE tcbvZvrK2JSLOkOCfeTjU+vx3MevqMVNMQy1uQxzOTys8BpgLNurcE8744Jp2ptX6dgk EMyGESbsMqRpOX95+V7Mt126e6XH7NVOSDwUFtEcEx5WbSy31LWA4s/wpsiMIBxv6+cx PVMkWpekPkTeN1LpeD0TB/d0v/CUhyUbF7CeL1Y7SjrktVryPfdPonXRAtVpzXHxJ+5g 3rdMN5p08PQVvGvum0D04rLubQiiWDHuvI3AlCuTOkfEs6Dv84EJqXP0l5gqhQASeIgQ Vpug== X-Gm-Message-State: AOJu0Yywwi+jv6+tF9EosuYinSrPTuuFyHuNRv9PTVLVJdkmpPjXmFpy gJ2niANzumhd3gHhuf7hAXeWrLTtujTCo21iCVbg9OF+xHdRhIgHp9CzbnLsXcSvi0IfsPOy0mJ fKQ== X-Google-Smtp-Source: AGHT+IF9V/6JdoU8QuZRMVkRzj+UeTRxHQatHPLLvh6dQW/ZvCNMdhB946SuxRg0AqXn3f3Ur/VeDQ== X-Received: by 2002:a17:903:26c4:b0:1d4:f29f:e13b with SMTP id jg4-20020a17090326c400b001d4f29fe13bmr1639156plb.40.1705623258923; Thu, 18 Jan 2024 16:14:18 -0800 (PST) Received: from zhadum.home.kylehuey.com (c-76-126-33-191.hsd1.ca.comcast.net. [76.126.33.191]) by smtp.gmail.com with ESMTPSA id mj7-20020a1709032b8700b001d1d1ef8be6sm1921238plb.267.2024.01.18.16.14.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Jan 2024 16:14:18 -0800 (PST) From: Kyle Huey X-Google-Original-From: Kyle Huey To: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Song Liu Cc: Robert O'Callahan , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Ian Rogers , Adrian Hunter , linux-perf-users@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH v4 3/4] perf/bpf: Allow a bpf program to suppress all sample side effects Date: Thu, 18 Jan 2024 16:13:50 -0800 Message-Id: <20240119001352.9396-4-khuey@kylehuey.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240119001352.9396-1-khuey@kylehuey.com> References: <20240119001352.9396-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Returning zero from a bpf program attached to a perf event already suppresses any data output. Return early from __perf_event_overflow() in this case so it will also suppress event_limit accounting, SIGTRAP generation, and F_ASYNC signalling. Signed-off-by: Kyle Huey Acked-by: Song Liu Acked-by: Namhyung Kim --- kernel/events/core.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 24a718e7eb98..a329bec42c4d 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -9574,6 +9574,11 @@ static int __perf_event_overflow(struct perf_event *event, ret = __perf_event_account_interrupt(event, throttle); +#ifdef CONFIG_BPF_SYSCALL + if (event->prog && !bpf_overflow_handler(event, data, regs)) + return ret; +#endif + /* * XXX event_limit might not quite work as expected on inherited * events @@ -9623,10 +9628,7 @@ static int __perf_event_overflow(struct perf_event *event, irq_work_queue(&event->pending_irq); } -#ifdef CONFIG_BPF_SYSCALL - if (!(event->prog && !bpf_overflow_handler(event, data, regs))) -#endif - READ_ONCE(event->overflow_handler)(event, data, regs); + READ_ONCE(event->overflow_handler)(event, data, regs); if (*perf_event_fasync(event) && event->pending_kill) { event->pending_wakeup = 1; From patchwork Fri Jan 19 00:13:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kyle Huey X-Patchwork-Id: 13523266 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D454318A for ; Fri, 19 Jan 2024 00:14:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705623297; cv=none; b=YCfILcIyPnHuzirguQloBh165zP137gDPe4Hi6B/rI2lMQVDM+wn3ucL4fMgAqP5o/94bSmgWsIrqfLkL4OpEv5CBYti4nxjd1DGNY0pT48Mpxzqm0m8EmDAVx/2Yw5gVjYt0H0RM0xRe7Piq5601Y3g3oO5GxvtpT5/cKAuYvY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705623297; c=relaxed/simple; bh=lgztwFbOxvCSZQBlFKLsXosQinxtlxOmxqJ20zMs7Cw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=U/whJMLRQjL+qPimFlfRaVo5650ZfT+qMF30EcUzQRzZ5mg7bfayGYiIBPKvTTzk0ZsxZNYE26cpTQmGIWW+bPGd276Fb3v3mC6YdcWiN/d00B2YSC4GXmT3mkhS4WvHcsxbfJhOHBDtwfuoMHqJu6XKZ4LawYU2HIB/3BIe4JQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com; spf=pass smtp.mailfrom=kylehuey.com; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b=XupO7X72; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylehuey.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kylehuey.com header.i=@kylehuey.com header.b="XupO7X72" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1d50d0c98c3so10800765ad.1 for ; Thu, 18 Jan 2024 16:14:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kylehuey.com; s=google; t=1705623294; x=1706228094; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7KAoEfpbtZugctsvQWmIErusprU1vjdWQA1H3EUTUAE=; b=XupO7X72OFlt2YjQ4lEe/fBPu5e3h5RALmEQrIq4r/6gO+Lrt8U9eBWcOAbgYe5s8M isEVmgid1Y13IvTCIepzoLnb+rUko+oh37FO8xTAYS71wNgw0oHAFz5VRnin3HQiUKxD FvZLxRL1i80qgd9xZtYuERfyP61gAYAgGrD76wuvL+lcQDk5D6KrJ5n16jK30y/6AABY f5LxrDzhvdUWpM7zy7bGpEJZEha+61YfWnmOmvlAHCTDq5hsFim2IgKjeoSFGQvUQyuR beF5GQYqpk71EJhhbzrNpqVUsu9kq+V3EejWqHMO0nulG6Tad+eg0TR5DKNJOo8ejSc8 B4fA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705623294; x=1706228094; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7KAoEfpbtZugctsvQWmIErusprU1vjdWQA1H3EUTUAE=; b=F4Go2x9Owpu1r1GXWu2UrhyrEGyQQvXUZ4V7i9FKYXX9fGHVKd89WBjoI7s7bUxcWI YRYpZ2sbtkYjEcH1K8epTKMi9gE8C1oiNgkZqlpdsQ/nTetUftHW9w62Z/mJiVBMbYaK 9FXKi8V0MGFOwh9iHUPELqwm5PJ95mVmSfK7jlSy0sTxt4Kmqa/HwRksdM5IdMZfiXHM sgt1xwV33kHQVoiq07LUTnyvCR/KhZHQ2uWnOwllCpDWZXT2pwlXAE5D/FF2Sdx8tVVi YdMrSlB+MQNEZFeg5hYERpL+O21rA+9KXQlm6/1gSx7/H4pf7ajgJmEOEhFdRWth+C40 OJCA== X-Gm-Message-State: AOJu0YyDRASqK3VJSKlnPN3fpk8qrAiQvtI0EebUNNpoPxsRHTILaYZh ed8rSEkjgY7CHWm0vIhQWV33A6dL8Ort9F147EOQT6B46aybqNFabzRL6yeIGA== X-Google-Smtp-Source: AGHT+IH7/HWrPOJEMKFFfVf2axY9b8wQCQlVN82voRPP0r/VV1qz9l5kOc3qny64GVMHXyV/ljKGCw== X-Received: by 2002:a17:902:7245:b0:1d5:7524:6d21 with SMTP id c5-20020a170902724500b001d575246d21mr181163pll.42.1705623294291; Thu, 18 Jan 2024 16:14:54 -0800 (PST) Received: from zhadum.home.kylehuey.com (c-76-126-33-191.hsd1.ca.comcast.net. [76.126.33.191]) by smtp.gmail.com with ESMTPSA id mj7-20020a1709032b8700b001d1d1ef8be6sm1921238plb.267.2024.01.18.16.14.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Jan 2024 16:14:53 -0800 (PST) From: Kyle Huey X-Google-Original-From: Kyle Huey To: Kyle Huey , linux-kernel@vger.kernel.org, Andrii Nakryiko , Jiri Olsa , Namhyung Kim , Marco Elver , Yonghong Song , Song Liu Cc: Robert O'Callahan , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Mykola Lysenko , Shuah Khan , bpf@vger.kernel.org, linux-kselftest@vger.kernel.org Subject: [PATCH v4 4/4] selftest/bpf: Test a perf bpf program that suppresses side effects. Date: Thu, 18 Jan 2024 16:13:51 -0800 Message-Id: <20240119001352.9396-5-khuey@kylehuey.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240119001352.9396-1-khuey@kylehuey.com> References: <20240119001352.9396-1-khuey@kylehuey.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net The test sets a hardware breakpoint and uses a bpf program to suppress the side effects of a perf event sample, including I/O availability signals, SIGTRAPs, and decrementing the event counter limit, if the ip matches the expected value. Then the function with the breakpoint is executed multiple times to test that all effects behave as expected. Signed-off-by: Kyle Huey Acked-by: Song Liu --- .../selftests/bpf/prog_tests/perf_skip.c | 136 ++++++++++++++++++ .../selftests/bpf/progs/test_perf_skip.c | 15 ++ 2 files changed, 151 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/perf_skip.c create mode 100644 tools/testing/selftests/bpf/progs/test_perf_skip.c diff --git a/tools/testing/selftests/bpf/prog_tests/perf_skip.c b/tools/testing/selftests/bpf/prog_tests/perf_skip.c new file mode 100644 index 000000000000..c79dae34ae90 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/perf_skip.c @@ -0,0 +1,136 @@ +// SPDX-License-Identifier: GPL-2.0 +#define _GNU_SOURCE + +#include +#include "test_perf_skip.skel.h" +#include +#include +#include + +#ifndef TRAP_PERF +#define TRAP_PERF 6 +#endif + +int sigio_count, sigtrap_count; + +static void handle_sigio(int sig __always_unused) +{ + ++sigio_count; +} + +static void handle_sigtrap(int signum __always_unused, + siginfo_t *info, + void *ucontext __always_unused) +{ + ASSERT_EQ(info->si_code, TRAP_PERF, "wrong si_code"); + ++sigtrap_count; +} + +static noinline int test_function(void) +{ + asm volatile (""); + return 0; +} + +void serial_test_perf_skip(void) +{ + struct sigaction action = {}; + struct sigaction previous_sigtrap; + sighandler_t previous_sigio; + struct test_perf_skip *skel = NULL; + struct perf_event_attr attr = {}; + int perf_fd = -1; + int err; + struct f_owner_ex owner; + struct bpf_link *prog_link = NULL; + + action.sa_flags = SA_SIGINFO | SA_NODEFER; + action.sa_sigaction = handle_sigtrap; + sigemptyset(&action.sa_mask); + if (!ASSERT_OK(sigaction(SIGTRAP, &action, &previous_sigtrap), "sigaction")) + return; + + previous_sigio = signal(SIGIO, handle_sigio); + if (!ASSERT_NEQ(previous_sigio, SIG_ERR, "signal")) + goto cleanup; + + skel = test_perf_skip__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_load")) + goto cleanup; + + attr.type = PERF_TYPE_BREAKPOINT; + attr.size = sizeof(attr); + attr.bp_type = HW_BREAKPOINT_X; + attr.bp_addr = (uintptr_t)test_function; + attr.bp_len = sizeof(long); + attr.sample_period = 1; + attr.sample_type = PERF_SAMPLE_IP; + attr.pinned = 1; + attr.exclude_kernel = 1; + attr.exclude_hv = 1; + attr.precise_ip = 3; + attr.sigtrap = 1; + attr.remove_on_exec = 1; + + perf_fd = syscall(__NR_perf_event_open, &attr, 0, -1, -1, 0); + if (perf_fd < 0 && (errno == ENOENT || errno == EOPNOTSUPP)) { + printf("SKIP:no PERF_TYPE_BREAKPOINT/HW_BREAKPOINT_X\n"); + test__skip(); + goto cleanup; + } + if (!ASSERT_OK(perf_fd < 0, "perf_event_open")) + goto cleanup; + + /* Configure the perf event to signal on sample. */ + err = fcntl(perf_fd, F_SETFL, O_ASYNC); + if (!ASSERT_OK(err, "fcntl(F_SETFL, O_ASYNC)")) + goto cleanup; + + owner.type = F_OWNER_TID; + owner.pid = syscall(__NR_gettid); + err = fcntl(perf_fd, F_SETOWN_EX, &owner); + if (!ASSERT_OK(err, "fcntl(F_SETOWN_EX)")) + goto cleanup; + + /* Allow at most one sample. A sample rejected by bpf should + * not count against this. + */ + err = ioctl(perf_fd, PERF_EVENT_IOC_REFRESH, 1); + if (!ASSERT_OK(err, "ioctl(PERF_EVENT_IOC_REFRESH)")) + goto cleanup; + + prog_link = bpf_program__attach_perf_event(skel->progs.handler, perf_fd); + if (!ASSERT_OK_PTR(prog_link, "bpf_program__attach_perf_event")) + goto cleanup; + + /* Configure the bpf program to suppress the sample. */ + skel->bss->ip = (uintptr_t)test_function; + test_function(); + + ASSERT_EQ(sigio_count, 0, "sigio_count"); + ASSERT_EQ(sigtrap_count, 0, "sigtrap_count"); + + /* Configure the bpf program to allow the sample. */ + skel->bss->ip = 0; + test_function(); + + ASSERT_EQ(sigio_count, 1, "sigio_count"); + ASSERT_EQ(sigtrap_count, 1, "sigtrap_count"); + + /* Test that the sample above is the only one allowed (by perf, not + * by bpf) + */ + test_function(); + + ASSERT_EQ(sigio_count, 1, "sigio_count"); + ASSERT_EQ(sigtrap_count, 1, "sigtrap_count"); + +cleanup: + bpf_link__destroy(prog_link); + if (perf_fd >= 0) + close(perf_fd); + test_perf_skip__destroy(skel); + + signal(SIGIO, previous_sigio); + sigaction(SIGTRAP, &previous_sigtrap, NULL); +} diff --git a/tools/testing/selftests/bpf/progs/test_perf_skip.c b/tools/testing/selftests/bpf/progs/test_perf_skip.c new file mode 100644 index 000000000000..7eb8b6de7a57 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_perf_skip.c @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include + +uintptr_t ip; + +SEC("perf_event") +int handler(struct bpf_perf_event_data *data) +{ + /* Skip events that have the correct ip. */ + return ip != PT_REGS_IP(&data->regs); +} + +char _license[] SEC("license") = "GPL";