From patchwork Mon Apr 22 08:08:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Howard Chu X-Patchwork-Id: 13637865 Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB4284E1D5; Mon, 22 Apr 2024 08:08:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713773302; cv=none; b=EfFYfBqYrvpli987dpBvRZx69k+WTL/wbn69UXjgDps3lvaq2ZjoSmwS9icrmxpMHEliNdHRBoumrdkfPg9/5KDfPxfcZI1A4zMOzByzIm2HqSVvZ5PZq7OYPIJXGXPm5ZXGhiY6IZ1kJeljLroKZZ4eYr8fCx48zM2L/IF184k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713773302; c=relaxed/simple; bh=PvX3UAI3pKBNO7+QP/77uw0Vs7P6rs6XY2pmKssYcYo=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=kUNry6D1AqZBDEq1fcxcFYGWjVP5JshXRkJcv5CGqzX9W7vnUorJk5WyiE/koWybI/HJWtQtVkZ4EVa/6wHUyR4CbJ+ApTJZswrypDmJIrFbCa3FEhztNeH86oIJomiMLEysYqD7Pwse6rBQUhr89nIUnd1/3xP7eAGalFquMH4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=mVeglYmd; arc=none smtp.client-ip=209.85.215.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mVeglYmd" Received: by mail-pg1-f174.google.com with SMTP id 41be03b00d2f7-5e42b4bbfa4so2339799a12.1; Mon, 22 Apr 2024 01:08:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713773300; x=1714378100; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=R8U35ePrc9QgPQW1W05XfxAhRzQZ8Z2BPPc+gusk8hs=; b=mVeglYmdjc4e2HKuaUHRtmPa0BEyLr4el/23Yqju6hxakgS8zrxB4kU/4Z40tLLqIJ 8O0JaqDhUFsJ8JHAa0fynl2wd05qaVcAQItUZedu1cy7GfxODFhKiAPdftlO5CBCUYCW f7/Z2iLN571kmq6BluADKhTBQyx699lcy79HMRIIyBjn1vmaivbNvQAzxYQ+/LZtHbjx T70ivWBFVH5SaXAwtbnuy2T/2YB90/mFe+CTzRVDxQhZykLypkqpo+AnZ7HLqG89hDHy Vehb/CivwMhmFNAFGdsxUQxP6HdEBOsOA8CyogBneF1d9WIeC/X7fcXOm3eJ1Vy9aoUr 723A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713773300; x=1714378100; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=R8U35ePrc9QgPQW1W05XfxAhRzQZ8Z2BPPc+gusk8hs=; b=QarH1McH2RBGjXeefDbu/U4pAk3y6GznOiTDOy17rPwHrMpPtrneV/MvFH27TS1Ldr 8ojXfQzSbb9uB4D8i/qwVTRKFLoTvoP9Ki8vxc9cTzUZLD2tXeWrWKdoARLw0g6RRCHz PeNPM0cGwSZsknOq07Qum8e9D1IQRA9BmpfGJsueFOl+hRg7w11gj+nnLkVxEN2YIlbS 88+NdW83fTH6WYU05uHHlY4nk1Vs/x9y9RkzUBhZvRr3y+SJJomyGrtjuWy0tqDbQ1qb x6zjsIWZNi0/GCS5Z8qaMeqok25TYApwlAA0rrCRaKslrvycFqMjeHu7A+oResD/MMwZ IcBg== X-Forwarded-Encrypted: i=1; AJvYcCWhjYwcusWAaE0uJZo12Ik/yDOlv3KNfQP/i79cpMw+vNiYyj+2c3S6ITaw7qzjLTPGPPJ5mYW6puTag3Wu37MJkMD6mglqmKSgtl7LABKlbjvqPy5qygnI0oKcgG/rLvSPOEakTagNQyjS3B3IAyE0QHWmJ47lsCfuiyAsxgCW7AfR6g== X-Gm-Message-State: AOJu0YwvcjR9m6NjOFHk3JFoWQtyr3ZjZBq89UPhDhvr5PMpNxB31gZ4 7ttZdOvIEF5tIHO+IO1ds0avb/qQH8CXdaj4pKWSTG9ePeHUOPy0 X-Google-Smtp-Source: AGHT+IG67lFmX36ufIM8YRR3lOsp7+DgbGzlYeIKQFqwShCOFjnLcAyOF/T7LDXTqsiUFwpz/9AzgQ== X-Received: by 2002:a05:6a20:3d8c:b0:1ab:82fe:910b with SMTP id s12-20020a056a203d8c00b001ab82fe910bmr12453549pzi.58.1713773300194; Mon, 22 Apr 2024 01:08:20 -0700 (PDT) Received: from localhost.localdomain ([120.229.49.236]) by smtp.gmail.com with ESMTPSA id hi2-20020a17090b30c200b002a22ddac1a1sm7118809pjb.24.2024.04.22.01.08.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 01:08:19 -0700 (PDT) From: Howard Chu To: peterz@infradead.org Cc: mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, yangjihong1@huawei.com, zegao2021@gmail.com, leo.yan@linux.dev, ravi.bangoria@amd.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, Howard Chu Subject: [PATCH v1 1/4] perf record: Dump off-cpu samples directly Date: Mon, 22 Apr 2024 16:08:27 +0800 Message-ID: <20240422080827.1918034-1-howardchu95@gmail.com> X-Mailer: git-send-email 2.44.0 Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Parse off-cpu events using parse_event(). Change the placement of record__config_off_cpu to after record__open because we need to write mmapped fds into BPF's perf_event_array map, also, write sample_id/sample_type into BPF. In record__pushfn and record__aio_pushfn, handle off-cpu samples using off_cpu_strip. This is because the off-cpu samples that we want to write to perf.data is in off-cpu samples' raw_data section: regular samples: [sample: sample_data] off-cpu samples: [sample: [raw_data: sample_data]] We need to extract the real useful sample data out before writing. Hooks record_done just before evlist__disable to stop BPF program from outputting, otherwise, we lose some samples. After samples are collected, change sample_type of off-cpu event to the OFFCPU_SAMPLE_TYPES for parsing correctly, it was PERF_SAMPLE_RAW and some others, because BPF can only output to a specific type of perf_event, which is why `evsel->core.attr.sample_type &= OFFCPU_SAMPLE_TYPES;` is deleted in util/evsel.c. Signed-off-by: Howard Chu --- tools/perf/builtin-record.c | 98 ++++++++++++++++++++++++++++++++++--- tools/perf/util/evsel.c | 8 --- 2 files changed, 91 insertions(+), 15 deletions(-) diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index 2ff718d3e202..c31b23905f1b 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -389,6 +389,8 @@ struct record_aio { static int record__aio_pushfn(struct mmap *map, void *to, void *buf, size_t size) { struct record_aio *aio = to; + char *bf_stripped = NULL; + size_t stripped; /* * map->core.base data pointed by buf is copied into free map->aio.data[] buffer @@ -404,6 +406,31 @@ static int record__aio_pushfn(struct mmap *map, void *to, void *buf, size_t size * from the beginning of the kernel buffer till the end of the data chunk. */ + if (aio->rec->off_cpu) { + if (size == 0) + return 0; + + map->core.start -= size; + size = map->core.end - map->core.start; + + bf_stripped = malloc(size); + + if (bf_stripped == NULL) { + pr_err("Failed to allocate off-cpu strip buffer\n"); + return -ENOMEM; + } + + stripped = off_cpu_strip(aio->rec->evlist, map, bf_stripped, size); + + if (stripped < 0) { + size = (int)stripped; + goto out; + } + + size = stripped; + buf = bf_stripped; + } + if (record__comp_enabled(aio->rec)) { ssize_t compressed = zstd_compress(aio->rec->session, NULL, aio->data + aio->size, mmap__mmap_len(map) - aio->size, @@ -432,6 +459,9 @@ static int record__aio_pushfn(struct mmap *map, void *to, void *buf, size_t size aio->size += size; +out: + free(bf_stripped); + return size; } @@ -635,6 +665,38 @@ static int process_locked_synthesized_event(struct perf_tool *tool, static int record__pushfn(struct mmap *map, void *to, void *bf, size_t size) { struct record *rec = to; + int err; + char *bf_stripped = NULL; + size_t stripped; + + if (rec->off_cpu) { + /* + * We'll read all the events at once without masking. + * When reading the remainder from a map, the size is 0, because + * start is shifted to the end so no more data is to be read. + */ + if (size == 0) + return 0; + + map->core.start -= size; + /* get the total size */ + size = map->core.end - map->core.start; + + bf_stripped = malloc(size); + + if (bf_stripped == NULL) { + pr_err("Failed to allocate off-cpu strip buffer\n"); + return -ENOMEM; + } + + stripped = off_cpu_strip(rec->evlist, map, bf_stripped, size); + + if (stripped < 0) + return (int)stripped; + + size = stripped; + bf = bf_stripped; + } if (record__comp_enabled(rec)) { ssize_t compressed = zstd_compress(rec->session, map, map->data, @@ -648,7 +710,11 @@ static int record__pushfn(struct mmap *map, void *to, void *bf, size_t size) } thread->samples++; - return record__write(rec, map, bf, size); + err = record__write(rec, map, bf, size); + + free(bf_stripped); + + return err; } static volatile sig_atomic_t signr = -1; @@ -1790,6 +1856,7 @@ record__finish_output(struct record *rec) if (rec->buildid_all) perf_session__dsos_hit_all(rec->session); } + perf_session__write_header(rec->session, rec->evlist, fd, true); return; @@ -2501,6 +2568,14 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) } } + if (rec->off_cpu) { + err = record__config_off_cpu(rec); + if (err) { + pr_err("record__config_off_cpu failed, error %d\n", err); + goto out_free_threads; + } + } + /* * Normally perf_session__new would do this, but it doesn't have the * evlist. @@ -2764,6 +2839,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) * disable events in this case. */ if (done && !disabled && !target__none(&opts->target)) { + perf_hooks__invoke_record_done(); trigger_off(&auxtrace_snapshot_trigger); evlist__disable(rec->evlist); disabled = true; @@ -2827,14 +2903,17 @@ static int __cmd_record(struct record *rec, int argc, const char **argv) } else status = err; - if (rec->off_cpu) - rec->bytes_written += off_cpu_write(rec->session); - record__read_lost_samples(rec); record__synthesize(rec, true); /* this will be recalculated during process_buildids() */ rec->samples = 0; + /* change to the correct sample type for parsing */ + if (rec->off_cpu && off_cpu_change_type(rec->evlist)) { + pr_err("ERROR: Failed to change sample type for off-cpu event\n"); + goto out_delete_session; + } + if (!err) { if (!rec->timestamp_filename) { record__finish_output(rec); @@ -3198,7 +3277,7 @@ static int switch_output_setup(struct record *rec) unsigned long val; /* - * If we're using --switch-output-events, then we imply its + * If we're using --switch-output-events, then we imply its * --switch-output=signal, as we'll send a SIGUSR2 from the side band * thread to its parent. */ @@ -4221,9 +4300,14 @@ int cmd_record(int argc, const char **argv) } if (rec->off_cpu) { - err = record__config_off_cpu(rec); + char off_cpu_event[64]; + + snprintf(off_cpu_event, sizeof(off_cpu_event), + "bpf-output/no-inherit=1,name=%s/", OFFCPU_EVENT); + + err = parse_event(rec->evlist, off_cpu_event); if (err) { - pr_err("record__config_off_cpu failed, error %d\n", err); + pr_err("Failed to open off-cpu event\n"); goto out; } } diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index 3536404e9447..c08ae6a3c8d6 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -1092,11 +1092,6 @@ static void evsel__set_default_freq_period(struct record_opts *opts, } } -static bool evsel__is_offcpu_event(struct evsel *evsel) -{ - return evsel__is_bpf_output(evsel) && evsel__name_is(evsel, OFFCPU_EVENT); -} - /* * The enable_on_exec/disabled value strategy: * @@ -1363,9 +1358,6 @@ void evsel__config(struct evsel *evsel, struct record_opts *opts, if (evsel__is_dummy_event(evsel)) evsel__reset_sample_bit(evsel, BRANCH_STACK); - if (evsel__is_offcpu_event(evsel)) - evsel->core.attr.sample_type &= OFFCPU_SAMPLE_TYPES; - arch__post_evsel_config(evsel, attr); } From patchwork Mon Apr 22 08:08:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Howard Chu X-Patchwork-Id: 13637866 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16E5D4E1D1; Mon, 22 Apr 2024 08:08:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713773318; cv=none; b=La5yskMYjo7gI9a+Qe11LpIXavQoItptq6ZjAPFwvzPFdLXNWmnjQOSBZ0NCKV6jJwjw042QsSo1Dlc43NT5JWpOptG266p2XyQpg+iZ/F84tkAeM5ubDOlVinu3HEsVTVF5WcMe5atoddkUCgaFTvkXZVHAUu5TzbGtEbhHGDc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713773318; c=relaxed/simple; bh=LdGPKWSdxBqCVkcC4Xb/Utabt+QS/J2yF5sdnMasMKk=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=PwT2q/saVsZbooQqoVZT4XYW3GZyssFjVEhDb/q9YaXpI3XM9Qd2BFSlz9tNroYJvY2rkKHQ6Laz2qrpBGfio4pPHZ+wm2BuNQjD0awb1K4MdSrGMeecKV0ALhrCBcqvQZhw60KmKtvbSiU0zwcXfi89rGRdTEIxfjbMHXCAfgE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bpjJPA2q; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bpjJPA2q" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1e4bf0b3e06so39572735ad.1; Mon, 22 Apr 2024 01:08:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713773316; x=1714378116; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=+cw3bWmhYT7Pl8nwAcJrztKLEKkt9wnRRADXIvvrBpA=; b=bpjJPA2quDZZ5Y0+95aPVA1ztTfAI1Xrgc3ATe/2K7HoSve9RCW3iN4bIXclNe93TN hqLGeV1sQidJWar6DreD6Xe0hohg3ktwgw0ab9Z78WU94p0eG1y50npWIYlskQXT0l8J RjYzXUO3O1Y2RybT00IritNQ8dV+3l3b+qtPfTvCWmTGkHBRUEIoOEgZOx/ZSQhAPRKM VWBqbcjgUvR1UsDFXYoRHWq5H1GbkPR/NqNVZBifNpRa1hTbSnGQ14aKYb32VrSPu+M6 gh4NCN00X+AmNera71tELlyWW7ztsWX7X/Wokwe9nDwNiz6A1Voo5p/zpCGZcQK2YeuB A4GQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713773316; x=1714378116; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+cw3bWmhYT7Pl8nwAcJrztKLEKkt9wnRRADXIvvrBpA=; b=HUTwhh9ud1LhnNeRrzU+s81yVCvBqTpi5lC7tPGInWsv46fzQQAKDix1JHCNyEVfZV kHiRo+86bgs1Xl0Tu4104DGgrCYMAAfTKR1qetNlnx9dZZ4Z6EUVBc/hZBSHlxFThp3H fS2DxjG6e3nBoICJ7qXTtS1+oieZnl6x+Cpg3ow2HI0HUNqZIlTFS/iWm+2zSfVEjZq/ wwRAJZZakPsSoykwkHqW+S9eeJLR+gyRDjzfovAEqAmdtXUUR57pgM2F7N7PTg/S3CCy cjTjnyPJP46NwuuLUFMzsHl+ufZGTG4ViyIMSLrE4/jIm+x74gH4iOrZ0kjWbWyq2kko qDGg== X-Forwarded-Encrypted: i=1; AJvYcCXytgo1ncJBTK4rHC0ZpXwukrwNv/klwDeLf4XBXrekIOjTx8tclJUTngMKNPww0hilml2iNI5HXwM+YrVD7jTHdWoeygJpSiZKGH5LTxhzBgx+EgQevh8FqHEkmKQ3CDQnwpHx6FS0FdT5iZq4B5jVvbFqRAB0WEXo8Swusw0bNfscug== X-Gm-Message-State: AOJu0YxbAGKD/X51lJLjBkpSVizbnW4BZ1eJaI+xgNvo3OUkBLAwf8oG SuLLQwWmYp/1ukGr5Kb2d7RZ56ULfNcD/kMy0AmCtXzZmYNnQVrE8RDHnsPZbs+O4C4F X-Google-Smtp-Source: AGHT+IEON+7p/n3KrKlbdDm3YY2fOPlXRl9/p0OQ9vcD6btEbKuYMkr+aRjdLMzlYGcX0CahUCZHtg== X-Received: by 2002:a17:903:98c:b0:1e6:3494:6215 with SMTP id mb12-20020a170903098c00b001e634946215mr14168262plb.6.1713773316279; Mon, 22 Apr 2024 01:08:36 -0700 (PDT) Received: from localhost.localdomain ([120.229.49.236]) by smtp.gmail.com with ESMTPSA id p22-20020a170902b09600b001e7b7a7934bsm7505508plr.107.2024.04.22.01.08.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 01:08:36 -0700 (PDT) From: Howard Chu To: peterz@infradead.org Cc: mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, yangjihong1@huawei.com, zegao2021@gmail.com, leo.yan@linux.dev, ravi.bangoria@amd.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, Howard Chu Subject: [PATCH v1 2/4] perf record: Dump off-cpu samples directly Date: Mon, 22 Apr 2024 16:08:57 +0800 Message-ID: <20240422080857.1918678-1-howardchu95@gmail.com> X-Mailer: git-send-email 2.44.0 Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 bpf_perf_event_output the off-cpu sample on sched_switch. Because most of the time can_record() returns 0, therefore we can't collect stacks, when stack trace is collectable, store it in stack_save for later output. If we don't do that, most of the off-cpu samples won't have a stack trace. Because we don't collect total off-cpu time, and stack traces are collected in task_storage, we don't need to worry about maps getting overflow. There is a threshold OUTPUT_THRESHOLD (ns) to decide the minimum off-CPU time to trigger output, it is now set to zero. I need opinions on this value. Signed-off-by: Howard Chu --- tools/perf/util/bpf_skel/off_cpu.bpf.c | 163 ++++++++++++++++++++----- 1 file changed, 135 insertions(+), 28 deletions(-) diff --git a/tools/perf/util/bpf_skel/off_cpu.bpf.c b/tools/perf/util/bpf_skel/off_cpu.bpf.c index d877a0a9731f..81114de2436d 100644 --- a/tools/perf/util/bpf_skel/off_cpu.bpf.c +++ b/tools/perf/util/bpf_skel/off_cpu.bpf.c @@ -17,9 +17,13 @@ #define MAX_STACKS 32 #define MAX_ENTRIES 102400 +#define MAX_CPUS 4096 +#define MAX_OFFCPU_LEN 128 + +/* minimum offcpu time to trigger output */ +#define OUTPUT_THRESHOLD 0ULL struct tstamp_data { - __u32 stack_id; __u32 state; __u64 timestamp; }; @@ -27,17 +31,17 @@ struct tstamp_data { struct offcpu_key { __u32 pid; __u32 tgid; - __u32 stack_id; __u32 state; __u64 cgroup_id; }; -struct { - __uint(type, BPF_MAP_TYPE_STACK_TRACE); - __uint(key_size, sizeof(__u32)); - __uint(value_size, MAX_STACKS * sizeof(__u64)); - __uint(max_entries, MAX_ENTRIES); -} stacks SEC(".maps"); +struct offcpu_array { + u64 array[MAX_OFFCPU_LEN]; +}; + +struct stack_array { + u64 array[MAX_STACKS]; +}; struct { __uint(type, BPF_MAP_TYPE_TASK_STORAGE); @@ -46,13 +50,6 @@ struct { __type(value, struct tstamp_data); } tstamp SEC(".maps"); -struct { - __uint(type, BPF_MAP_TYPE_HASH); - __uint(key_size, sizeof(struct offcpu_key)); - __uint(value_size, sizeof(__u64)); - __uint(max_entries, MAX_ENTRIES); -} off_cpu SEC(".maps"); - struct { __uint(type, BPF_MAP_TYPE_HASH); __uint(key_size, sizeof(__u32)); @@ -74,6 +71,34 @@ struct { __uint(max_entries, 1); } cgroup_filter SEC(".maps"); +struct { + __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(__u32)); + __uint(max_entries, MAX_CPUS); +} offcpu_output SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(struct offcpu_array)); + __uint(max_entries, 1); +} offcpu_data SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(struct stack_array)); + __uint(max_entries, 1); +} stack_frame SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_TASK_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, struct stack_array); +} stack_save SEC(".maps"); + /* new kernel task_struct definition */ struct task_struct___new { long __state; @@ -96,6 +121,8 @@ const volatile bool uses_cgroup_v1 = false; int perf_subsys_id = -1; +u64 sample_id, sample_type; + /* * Old kernel used to call it task_struct->state and now it's '__state'. * Use BPF CO-RE "ignored suffix rule" to deal with it like below: @@ -182,50 +209,130 @@ static inline int can_record(struct task_struct *t, int state) return 1; } +static inline bool check_bounds(int index) +{ + if (index >= 0 && index < MAX_OFFCPU_LEN) + return true; + + return false; +} + +static inline int copy_stack(struct stack_array *from, + struct offcpu_array *to, int n) +{ + int max_stacks = MAX_STACKS, len = 0; + + if (!from) + return len; + + for (int i = 0; i < max_stacks && from->array[i]; ++i) { + if (check_bounds(n + 2 + i)) { + to->array[n + 2 + i] = from->array[i]; + ++len; + } + } + return len; +} + static int off_cpu_stat(u64 *ctx, struct task_struct *prev, struct task_struct *next, int state) { __u64 ts; - __u32 stack_id; struct tstamp_data *pelem; - + struct stack_array *frame, *stack_save_p; ts = bpf_ktime_get_ns(); + int zero = 0, len = 0, size; if (!can_record(prev, state)) goto next; - stack_id = bpf_get_stackid(ctx, &stacks, - BPF_F_FAST_STACK_CMP | BPF_F_USER_STACK); + frame = bpf_map_lookup_elem(&stack_frame, &zero); + if (frame) + len = bpf_get_stack(ctx, frame->array, MAX_STACKS * sizeof(u64), + BPF_F_USER_STACK) / sizeof(u64); + + /* save stacks if collectable */ + if (len > 0) { + stack_save_p = bpf_task_storage_get(&stack_save, prev, NULL, + BPF_LOCAL_STORAGE_GET_F_CREATE); + if (stack_save_p) + for (int i = 0; i < len && i < MAX_STACKS; ++i) + stack_save_p->array[i] = frame->array[i]; + } pelem = bpf_task_storage_get(&tstamp, prev, NULL, BPF_LOCAL_STORAGE_GET_F_CREATE); + if (!pelem) goto next; pelem->timestamp = ts; pelem->state = state; - pelem->stack_id = stack_id; next: pelem = bpf_task_storage_get(&tstamp, next, NULL, 0); + stack_save_p = bpf_task_storage_get(&stack_save, next, NULL, 0); + if (pelem && pelem->timestamp) { struct offcpu_key key = { .pid = next->pid, .tgid = next->tgid, - .stack_id = pelem->stack_id, .state = pelem->state, .cgroup_id = needs_cgroup ? get_cgroup_id(next) : 0, }; - __u64 delta = ts - pelem->timestamp; - __u64 *total; - total = bpf_map_lookup_elem(&off_cpu, &key); - if (total) - *total += delta; - else - bpf_map_update_elem(&off_cpu, &key, &delta, BPF_ANY); + __u64 delta = ts - pelem->timestamp; + struct offcpu_array *data = bpf_map_lookup_elem(&offcpu_data, &zero); + + if (data && delta >= OUTPUT_THRESHOLD) { + int n = 0; + int ip_pos = -1; + + if (sample_type & PERF_SAMPLE_IDENTIFIER && check_bounds(n)) + data->array[n++] = sample_id; + if (sample_type & PERF_SAMPLE_IP && check_bounds(n)) { + ip_pos = n; + data->array[n++] = 0; /* will be updated */ + } + if (sample_type & PERF_SAMPLE_TID && check_bounds(n)) + data->array[n++] = (u64)key.pid << 32 | key.tgid; + if (sample_type & PERF_SAMPLE_TIME && check_bounds(n)) + data->array[n++] = pelem->timestamp; + if (sample_type & PERF_SAMPLE_ID && check_bounds(n)) + data->array[n++] = sample_id; + if (sample_type & PERF_SAMPLE_CPU && check_bounds(n)) + data->array[n++] = 0; + if (sample_type & PERF_SAMPLE_PERIOD && check_bounds(n)) + data->array[n++] = delta; + if (sample_type & PERF_SAMPLE_CALLCHAIN && check_bounds(n + 2)) { + len = 0; + + /* data->array[n] is callchain->nr (updated later) */ + data->array[n + 1] = PERF_CONTEXT_USER; + data->array[n + 2] = 0; + + len = copy_stack(stack_save_p, data, n); + + /* update length of callchain */ + data->array[n] = len + 1; + + /* update sample ip with the first callchain entry */ + if (ip_pos >= 0) + data->array[ip_pos] = data->array[n + 2]; + + /* calculate sample callchain data->array length */ + n += len + 2; + } + if (sample_type & PERF_SAMPLE_CGROUP && check_bounds(n)) + data->array[n++] = key.cgroup_id; + + size = n * sizeof(u64); + if (size >= 0 && size <= MAX_OFFCPU_LEN * sizeof(u64)) + bpf_perf_event_output(ctx, &offcpu_output, BPF_F_CURRENT_CPU, + data, size); + } /* prevent to reuse the timestamp later */ pelem->timestamp = 0; } From patchwork Mon Apr 22 08:09:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Howard Chu X-Patchwork-Id: 13637867 Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C90AF4E1D1; Mon, 22 Apr 2024 08:08:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713773336; cv=none; b=gqBqO/gWjpQlrF6NTUBbKKTR+N9vFZjicDNRD9T8Nf+0nVuOp9Bzj7TDf0IqShCprUAUTrn1HF/m6Yy18WRRhwVYYFbm8dHPMsrggweDxfyKVQTy0c8kXeF595fk909BDsyIuxPkELk1aZwyX1IYSzveORl6nHB0wLYy557ibaA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713773336; c=relaxed/simple; bh=kUI808mePhRZHwg9ymVLzvgutA/xQJuf3RrktNQ3Tho=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=piFuO8aNqdV/PlRiC3e5CiZnfQNIcB8N6nugioMT7VmGe55/iM475M/quAp1Bx76bOjBA6BEEGkT2QqgJ6mOpuSzSmLxbvhPJ1R8JYWcGAx9qe3b9KRVRhXsG8nQPXyIQa/Aj+kYW653rSa/S1j/b2wVhlSwEwUqKJCPhoRnKmQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=I9vJWVfe; arc=none smtp.client-ip=209.85.216.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="I9vJWVfe" Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-2a58209b159so2544501a91.3; Mon, 22 Apr 2024 01:08:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713773334; x=1714378134; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=1YOWqoUrVrzd78R6ZnWxTyIXa2oV+7yQhabwhKzd1GY=; b=I9vJWVfeTiqwdQyCHC8jGmqDxnNUwDIA7Wcizm6vqbQuTR5wb2/wtrwCc2JCF1WAcK Sx/BIHCNwK+56gmPTG1Ol/dHEaXZ2NfcjnykmjdWoVNwMZ3qazGAh+Z0DAl3RBRpla5p 8Uo5YukfvcgZIh1XiZYLsAiK7KaTMkvMTNwgfcFlEAU+LggKnprG8JW+3tQPcJ5KZJ33 NRIijSQJ6/cNEYErm7k7yWFVtzlwG8o4mSiSuZxr2hiCg984kOmpun8AnUUFO5y30XMO PYncKrAR//oPfjDVYWQuIbSV7CK8HbBcxokzXk9ziMMGtlJr/GTTo/DA/MQc4rWhktkh 57Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713773334; x=1714378134; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1YOWqoUrVrzd78R6ZnWxTyIXa2oV+7yQhabwhKzd1GY=; b=LXEa6aOB7AtxsJBsmMqFjUOpeS+cqVbdsZ/AGTXsBo71C9J2gM++hYFf93z73ef1A2 bfyohXH66K0l3bVNFeDbY/ek6IylzOwxWublbKk6O2IiuzIVivQOGPJtOjxirdBxZbP2 z+qZohlU2a8cSEj0/11OLHq+UzbLL6eCC8N1Bzu7pO/TI9HsWEymIXnBNdby9KamSHUu IFWJtQuvTs5/b2lLgdi73cVzwMQSVU9hKHmKYw23Rd/BsC0iGm9VyUME0yEN9BVo3VIZ CjTJoIS3HQIlibTE86NyOBnP8X6FpXPhvZcf7FP1RQtOZP6UJJMzxlFjDfgbytBoTHDm 4wDA== X-Forwarded-Encrypted: i=1; AJvYcCWY2Ob4RIHWTeaw4ERR7WRUL+I/BoVhELi5Z58yQA177C81c1lh9wpjtH9nB5Gd5N1E5YfQJj2sBcnydmlWuK89hDLBW52ABXRtHU9m8ARyxlnFxJA1ytcbyeFjacTU3/fzcOCFpBadZJybOtdi8Cl4QzQtLMo/blXGStXMRfddWdS4UQ== X-Gm-Message-State: AOJu0Yx2LPlEyixMmK82ftEDgj11IvlGtPttcT1OKVJBGEdiDT0l+PNr Kn3abvMZd+Oeu3RgyRpFupGoxPSLZ7PJsD/Y8+4FVC5CYCQR94mT X-Google-Smtp-Source: AGHT+IFCOjRIFdGtEDgDePhaaz9c1QAeRRqmE0j/uLnYZT22Hw8WLLxAEOToxc3ZMMghs4oHeXg6kQ== X-Received: by 2002:a17:90a:d704:b0:2a2:b308:1371 with SMTP id y4-20020a17090ad70400b002a2b3081371mr7354259pju.43.1713773334054; Mon, 22 Apr 2024 01:08:54 -0700 (PDT) Received: from localhost.localdomain ([120.229.49.236]) by smtp.gmail.com with ESMTPSA id pw14-20020a17090b278e00b002ad0ba85c7asm3490372pjb.3.2024.04.22.01.08.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 01:08:53 -0700 (PDT) From: Howard Chu To: peterz@infradead.org Cc: mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, yangjihong1@huawei.com, zegao2021@gmail.com, leo.yan@linux.dev, ravi.bangoria@amd.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, Howard Chu Subject: [PATCH v1 3/4] perf record: Dump off-cpu samples directly Date: Mon, 22 Apr 2024 16:09:14 +0800 Message-ID: <20240422080914.1919018-1-howardchu95@gmail.com> X-Mailer: git-send-email 2.44.0 Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Because we need to extract real sample data from raw_data, an off_cpu_strip function is needed. In off_cpu_strip, we read events one by one, stripping actual samples from raw data. After stripping is done, a stripped buffer will be written to the perf.data(or compressed). The size is end - start without masking, because masking will be handled by perf_mmap__read_event(). Also, there's no need to call perf_mmap__consume() as it will be called in perf_mmap__push(). We read all the data at once, so start will be moved to end, when perf_mmap__push() is called in pushfn the second time, size will be zero, just returns directly. Hook record_done instead of record_end to off_cpu_finish. Although moving record_end hook is also fine because currently, only off-cpu is using these hooks, technically record doesn't end when we need to turn off BPF output, so it may confuse if moved. The reason why there's an additional off_cpu_change_type is that it cannot be put into off_cpu_finish because there are still samples to be read(which requires sample_type to stay PERF_SAMPLE_RAW), and it cannot be hooked to record_end because it has to be put before record__finish_output. Signed-off-by: Howard Chu --- tools/perf/util/bpf_off_cpu.c | 245 +++++++++++++----------------- tools/perf/util/off_cpu.h | 14 +- tools/perf/util/perf-hooks-list.h | 1 + 3 files changed, 121 insertions(+), 139 deletions(-) diff --git a/tools/perf/util/bpf_off_cpu.c b/tools/perf/util/bpf_off_cpu.c index 6af36142dc5a..df1f02938099 100644 --- a/tools/perf/util/bpf_off_cpu.c +++ b/tools/perf/util/bpf_off_cpu.c @@ -12,6 +12,9 @@ #include "util/thread_map.h" #include "util/cgroup.h" #include "util/strlist.h" +#include "util/mmap.h" +#include "util/sample.h" +#include #include #include "bpf_skel/off_cpu.skel.h" @@ -23,51 +26,6 @@ static struct off_cpu_bpf *skel; -struct off_cpu_key { - u32 pid; - u32 tgid; - u32 stack_id; - u32 state; - u64 cgroup_id; -}; - -union off_cpu_data { - struct perf_event_header hdr; - u64 array[1024 / sizeof(u64)]; -}; - -static int off_cpu_config(struct evlist *evlist) -{ - struct evsel *evsel; - struct perf_event_attr attr = { - .type = PERF_TYPE_SOFTWARE, - .config = PERF_COUNT_SW_BPF_OUTPUT, - .size = sizeof(attr), /* to capture ABI version */ - }; - char *evname = strdup(OFFCPU_EVENT); - - if (evname == NULL) - return -ENOMEM; - - evsel = evsel__new(&attr); - if (!evsel) { - free(evname); - return -ENOMEM; - } - - evsel->core.attr.freq = 1; - evsel->core.attr.sample_period = 1; - /* off-cpu analysis depends on stack trace */ - evsel->core.attr.sample_type = PERF_SAMPLE_CALLCHAIN; - - evlist__add(evlist, evsel); - - free(evsel->name); - evsel->name = evname; - - return 0; -} - static void off_cpu_start(void *arg) { struct evlist *evlist = arg; @@ -125,18 +83,29 @@ static void check_sched_switch_args(void) btf__free(btf); } +int off_cpu_change_type(struct evlist *evlist) +{ + struct evsel *evsel; + + evsel = evlist__find_evsel_by_str(evlist, OFFCPU_EVENT); + if (evsel == NULL) + return -1; + + evsel->core.attr.sample_type = OFFCPU_SAMPLE_TYPES; + + return 0; +} + int off_cpu_prepare(struct evlist *evlist, struct target *target, struct record_opts *opts) { int err, fd, i; int ncpus = 1, ntasks = 1, ncgrps = 1; + u64 sid = 0; struct strlist *pid_slist = NULL; struct str_node *pos; - - if (off_cpu_config(evlist) < 0) { - pr_err("Failed to config off-cpu BPF event\n"); - return -1; - } + struct evsel *evsel; + struct perf_cpu pcpu; skel = off_cpu_bpf__open(); if (!skel) { @@ -250,7 +219,6 @@ int off_cpu_prepare(struct evlist *evlist, struct target *target, } if (evlist__first(evlist)->cgrp) { - struct evsel *evsel; u8 val = 1; skel->bss->has_cgroup = 1; @@ -272,6 +240,25 @@ int off_cpu_prepare(struct evlist *evlist, struct target *target, } } + evsel = evlist__find_evsel_by_str(evlist, OFFCPU_EVENT); + if (evsel == NULL) { + pr_err("%s evsel not found\n", OFFCPU_EVENT); + goto out; + } + + if (evsel->core.id) + sid = evsel->core.id[0]; + + skel->bss->sample_id = sid; + skel->bss->sample_type = OFFCPU_SAMPLE_TYPES; + + perf_cpu_map__for_each_cpu(pcpu, i, evsel->core.cpus) { + bpf_map__update_elem(skel->maps.offcpu_output, + &pcpu.cpu, sizeof(int), + xyarray__entry(evsel->core.fd, pcpu.cpu, 0), + sizeof(__u32), BPF_ANY); + } + err = off_cpu_bpf__attach(skel); if (err) { pr_err("Failed to attach off-cpu BPF skeleton\n"); @@ -279,7 +266,7 @@ int off_cpu_prepare(struct evlist *evlist, struct target *target, } if (perf_hooks__set_hook("record_start", off_cpu_start, evlist) || - perf_hooks__set_hook("record_end", off_cpu_finish, evlist)) { + perf_hooks__set_hook("record_done", off_cpu_finish, evlist)) { pr_err("Failed to attach off-cpu skeleton\n"); goto out; } @@ -291,105 +278,91 @@ int off_cpu_prepare(struct evlist *evlist, struct target *target, return -1; } -int off_cpu_write(struct perf_session *session) +ssize_t off_cpu_strip(struct evlist *evlist, struct mmap *mp, char *dst, size_t size) { - int bytes = 0, size; - int fd, stack; - u64 sample_type, val, sid = 0; + /* + * In this function, we read events one by one, + * stripping actual samples from raw data. + * The size is end - start without masking, + * because masking will be handled by + * perf_mmap__read_event() + */ + + union perf_event *event, tmp; + u64 sample_type = OFFCPU_SAMPLE_TYPES; + size_t written = 0, event_sz, write_sz, raw_sz_aligned, offset = 0; + void *src; + int err = 0, n = 0; + struct perf_sample sample; struct evsel *evsel; - struct perf_data_file *file = &session->data->file; - struct off_cpu_key prev, key; - union off_cpu_data data = { - .hdr = { - .type = PERF_RECORD_SAMPLE, - .misc = PERF_RECORD_MISC_USER, - }, - }; - u64 tstamp = OFF_CPU_TIMESTAMP; - - skel->bss->enabled = 0; - evsel = evlist__find_evsel_by_str(session->evlist, OFFCPU_EVENT); + evsel = evlist__find_evsel_by_str(evlist, OFFCPU_EVENT); if (evsel == NULL) { pr_err("%s evsel not found\n", OFFCPU_EVENT); - return 0; - } - - sample_type = evsel->core.attr.sample_type; - - if (sample_type & ~OFFCPU_SAMPLE_TYPES) { - pr_err("not supported sample type: %llx\n", - (unsigned long long)sample_type); return -1; } - if (sample_type & (PERF_SAMPLE_ID | PERF_SAMPLE_IDENTIFIER)) { - if (evsel->core.id) - sid = evsel->core.id[0]; - } + /* for writing sample time*/ + if (sample_type & PERF_SAMPLE_IDENTIFIER) + ++n; + if (sample_type & PERF_SAMPLE_IP) + ++n; + if (sample_type & PERF_SAMPLE_TID) + ++n; + + /* no need for perf_mmap__consume(), it will be handled by perf_mmap__push() */ + while ((event = perf_mmap__read_event(&mp->core)) != NULL) { + event_sz = event->header.size; + write_sz = event_sz; + src = event; + + if (event->header.type == PERF_RECORD_SAMPLE) { + err = evlist__parse_sample(evlist, event, &sample); + if (err) { + pr_err("Failed to parse off-cpu sample\n"); + return -1; + } - fd = bpf_map__fd(skel->maps.off_cpu); - stack = bpf_map__fd(skel->maps.stacks); - memset(&prev, 0, sizeof(prev)); + if (sample.raw_data && evsel->core.id) { + bool flag = false; - while (!bpf_map_get_next_key(fd, &prev, &key)) { - int n = 1; /* start from perf_event_header */ - int ip_pos = -1; + for (u32 i = 0; i < evsel->core.ids; i++) { + if (sample.id == evsel->core.id[i]) { + flag = true; + break; + } + } + if (flag) { + memcpy(&tmp, event, event_sz); - bpf_map_lookup_elem(fd, &key, &val); + /* raw data has extra bits for alignment, discard them */ + raw_sz_aligned = sample.raw_size - sizeof(u32); + memcpy(tmp.sample.array, sample.raw_data, raw_sz_aligned); - if (sample_type & PERF_SAMPLE_IDENTIFIER) - data.array[n++] = sid; - if (sample_type & PERF_SAMPLE_IP) { - ip_pos = n; - data.array[n++] = 0; /* will be updated */ - } - if (sample_type & PERF_SAMPLE_TID) - data.array[n++] = (u64)key.pid << 32 | key.tgid; - if (sample_type & PERF_SAMPLE_TIME) - data.array[n++] = tstamp; - if (sample_type & PERF_SAMPLE_ID) - data.array[n++] = sid; - if (sample_type & PERF_SAMPLE_CPU) - data.array[n++] = 0; - if (sample_type & PERF_SAMPLE_PERIOD) - data.array[n++] = val; - if (sample_type & PERF_SAMPLE_CALLCHAIN) { - int len = 0; - - /* data.array[n] is callchain->nr (updated later) */ - data.array[n + 1] = PERF_CONTEXT_USER; - data.array[n + 2] = 0; - - bpf_map_lookup_elem(stack, &key.stack_id, &data.array[n + 2]); - while (data.array[n + 2 + len]) - len++; - - /* update length of callchain */ - data.array[n] = len + 1; - - /* update sample ip with the first callchain entry */ - if (ip_pos >= 0) - data.array[ip_pos] = data.array[n + 2]; - - /* calculate sample callchain data array length */ - n += len + 2; - } - if (sample_type & PERF_SAMPLE_CGROUP) - data.array[n++] = key.cgroup_id; + write_sz = sizeof(struct perf_event_header) + + raw_sz_aligned; + + /* without this we'll have out of order events */ + if (sample_type & PERF_SAMPLE_TIME) + tmp.sample.array[n] = sample.time; - size = n * sizeof(u64); - data.hdr.size = size; - bytes += size; + tmp.header.size = write_sz; + tmp.header.type = PERF_RECORD_SAMPLE; + tmp.header.misc = PERF_RECORD_MISC_USER; - if (perf_data_file__write(file, &data, size) < 0) { - pr_err("failed to write perf data, error: %m\n"); - return bytes; + src = &tmp; + } + } } + if (offset + event_sz > size || written + write_sz > size) + break; - prev = key; - /* increase dummy timestamp to sort later samples */ - tstamp++; + memcpy(dst, src, write_sz); + + dst += write_sz; + written += write_sz; + offset += event_sz; } - return bytes; + + return written; } diff --git a/tools/perf/util/off_cpu.h b/tools/perf/util/off_cpu.h index 2dd67c60f211..03d2f29cbb47 100644 --- a/tools/perf/util/off_cpu.h +++ b/tools/perf/util/off_cpu.h @@ -20,7 +20,9 @@ struct record_opts; #ifdef HAVE_BPF_SKEL int off_cpu_prepare(struct evlist *evlist, struct target *target, struct record_opts *opts); -int off_cpu_write(struct perf_session *session); +ssize_t off_cpu_strip(struct evlist *evlist, struct mmap *mp, + char *dst, size_t size); +int off_cpu_change_type(struct evlist *evlist); #else static inline int off_cpu_prepare(struct evlist *evlist __maybe_unused, struct target *target __maybe_unused, @@ -28,8 +30,14 @@ static inline int off_cpu_prepare(struct evlist *evlist __maybe_unused, { return -1; } - -static inline int off_cpu_write(struct perf_session *session __maybe_unused) +static inline ssize_t off_cpu_strip(struct evlist *evlist __maybe_unused, + struct mmap *mp __maybe_unused, + char *dst __maybe_unused, + size_t size __maybe_unused) +{ + return -1; +} +static inline int off_cpu_change_type(struct evlist *evlist __maybe_unused) { return -1; } diff --git a/tools/perf/util/perf-hooks-list.h b/tools/perf/util/perf-hooks-list.h index 2867c07ee84e..1ce4d44ace35 100644 --- a/tools/perf/util/perf-hooks-list.h +++ b/tools/perf/util/perf-hooks-list.h @@ -1,3 +1,4 @@ PERF_HOOK(record_start) PERF_HOOK(record_end) +PERF_HOOK(record_done) PERF_HOOK(test) From patchwork Mon Apr 22 08:09:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Howard Chu X-Patchwork-Id: 13637868 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2151D482EB; Mon, 22 Apr 2024 08:09:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713773349; cv=none; b=sj6UDL/iUYMXotaRmxKksI0wkMG3q0drCrwwaVwv+yRZRXMWno0x2YSybecbW2ffZHKMWmW9YoWQQoHgi84/OyPfcCHuNlfVCCS2ZKViwRd6Z19dj6cwjEiso6yT8360ELrkDypidG/S9pb0cRSOt64h5db3yNpn95FRSTYIFdI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713773349; c=relaxed/simple; bh=T2+6PFWukw8gJlL0s+pqeN9FU08ArUTh/EMbsg7hIcs=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=Q91VwVKZRK4JruHeUnzgrAM/J3SFKddWoEBlN8G9HxG+19iTyrYBa8mkreuYr7VmohD4HJesvcDt1ehMIDfczBrT+jvqC6ZpuC51UalH9XPFmZLqSzKr6rJP3AHRWIea2FQBXIo7DtWPlzPZnpHcwWgeoGVaPPhoh7hk3ROkWWE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=O6mWTDte; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="O6mWTDte" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-1e2bbc2048eso33811955ad.3; Mon, 22 Apr 2024 01:09:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713773347; x=1714378147; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=yMjn7Vnjo3uWqMMcIWhkRSYL+xmtvTZqOgcQc3dm7tg=; b=O6mWTDtevBKJ8hCNQ9x6uI9EFWUxdkgGcQz2VS9dktg45+xmsyavpGjs3/gwcxJA8R UhgthoshKlXpg/D7m4ORBz991u1GtSR3njl8bAXPRv+nPFuQ2RACr8HGu6jYxnRQTioZ 6aAmfAkjCAAZStNcwl71gygAZuI/UeTdkOWWx6hOYsYB74Jnkg3P9Xu5kemjngajxqd0 G7zJqg33NcS5/1uCoui8gumFdWt0NaBgYGrfBlzTC5urcIBSLpMzWreueXYPOeKpahlO A3pNU+r9iFIrrrt//vi40+hNLF33VUc+P4CKBykW+tLn3wU4tVHau0s8K2Iy2Pdv+YsM 6qOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713773347; x=1714378147; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yMjn7Vnjo3uWqMMcIWhkRSYL+xmtvTZqOgcQc3dm7tg=; b=oZpF8eGvLiri9QlpUyol5dxO84Qd5zURFmJzk7iixrhhgvEmCeiICIqpa0bH1y36ve JiMKLy48GOFqzh80EPu1AtMwY4siVV/1TuXOL/FAo/XxkBANi+6FWijh0k/gsBxYvs+N rbuFd7JTbUeWZCyoGTcUpU0cEhq0q4BMZ0xii276MGBwBZqP6z1VUpB2FxowaytVXdiy CJNafMTpA0I8M5RUp17aKIo0WaIpfVnLzOCcUERVMt2399ozzZzpPYtNgyzAPDHz3w8e VwWW1qbpanEyP1KrHu5ahGUgngGeHaEEA9Kt7kjY7L4QJOTRbHn249bvRZBZ+ddCYDuu IaWw== X-Forwarded-Encrypted: i=1; AJvYcCWGrAlg6+OSdhYzOQ+Ix2qiFgcbigzPt3JMLsvUY5hx+AQFmdG8BgqSUY+/FAuNO15U9LhM07RLAH6h2ObECPOGNMlfBeTO0Plv1ItadZCNbF9TARHL/AiQtSoGR1EhSlqRJk7drAw9082neBvMF4OPRkY02zm6o03YUe+IG66pQfAlmA== X-Gm-Message-State: AOJu0YzZntH53ICaScQxtELHaDLCjx2Yef6jW6SujL1IMfgSm/Te4EDQ tjmc1TdilQZjGFCfUUpWR77cE2ws5M9QNTkubgEdq7h5E2H0Lt8A X-Google-Smtp-Source: AGHT+IHKF8y46DVne2upxGaOtJCwPUsJ1t9aMP4ctxHXMofPraBr+EXw0uwvqWl8AQDvRSjrmgoInQ== X-Received: by 2002:a17:902:784f:b0:1e0:b562:b229 with SMTP id e15-20020a170902784f00b001e0b562b229mr7703492pln.47.1713773347435; Mon, 22 Apr 2024 01:09:07 -0700 (PDT) Received: from localhost.localdomain ([120.229.49.236]) by smtp.gmail.com with ESMTPSA id r3-20020a170902be0300b001e27462b988sm7512268pls.61.2024.04.22.01.09.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 01:09:07 -0700 (PDT) From: Howard Chu To: peterz@infradead.org Cc: mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, yangjihong1@huawei.com, zegao2021@gmail.com, leo.yan@linux.dev, ravi.bangoria@amd.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, Howard Chu Subject: [PATCH v1 4/4] perf record: Dump off-cpu samples directly Date: Mon, 22 Apr 2024 16:09:29 +0800 Message-ID: <20240422080929.1919319-1-howardchu95@gmail.com> X-Mailer: git-send-email 2.44.0 Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Since `--off-cpu` uses the same ring buffer as hardware samples, and `perf record --off-cpu -e dummy sleep 1` does not enable evlist, off-cpu samples cannot be read.`test_offcpu_basic` fails and is no longer necessary. Signed-off-by: Howard Chu --- tools/perf/tests/shell/record_offcpu.sh | 29 ------------------------- 1 file changed, 29 deletions(-) diff --git a/tools/perf/tests/shell/record_offcpu.sh b/tools/perf/tests/shell/record_offcpu.sh index 67c925f3a15a..c446c0cdee4f 100755 --- a/tools/perf/tests/shell/record_offcpu.sh +++ b/tools/perf/tests/shell/record_offcpu.sh @@ -36,30 +36,6 @@ test_offcpu_priv() { fi } -test_offcpu_basic() { - echo "Basic off-cpu test" - - if ! perf record --off-cpu -e dummy -o ${perfdata} sleep 1 2> /dev/null - then - echo "Basic off-cpu test [Failed record]" - err=1 - return - fi - if ! perf evlist -i ${perfdata} | grep -q "offcpu-time" - then - echo "Basic off-cpu test [Failed no event]" - err=1 - return - fi - if ! perf report -i ${perfdata} -q --percent-limit=90 | grep -E -q sleep - then - echo "Basic off-cpu test [Failed missing output]" - err=1 - return - fi - echo "Basic off-cpu test [Success]" -} - test_offcpu_child() { echo "Child task off-cpu test" @@ -88,13 +64,8 @@ test_offcpu_child() { echo "Child task off-cpu test [Success]" } - test_offcpu_priv -if [ $err = 0 ]; then - test_offcpu_basic -fi - if [ $err = 0 ]; then test_offcpu_child fi