From patchwork Wed Jan 29 00:14:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chun-Tse Shao X-Patchwork-Id: 13953376 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 82ECF155321 for ; Wed, 29 Jan 2025 00:25:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738110316; cv=none; b=B1ySfZ2zliJIBfvvGkMg9EpcjuVGuq2pbkwV021NlsuyWlZAIPcKiUAhtOEFqEQ7E0Dvqtys5CHUae/nPnXiihHznntjAVlV+v7Lc3s+h/cxj4cPYEG6rMgCYg2YHe2lMrJ1d+xGZ0Wsy6W+RxRvGcDlOYtbjXZfh8Y0uZUV7ZE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738110316; c=relaxed/simple; bh=27Hu/QjNl2hArTosMTM15Ibs0q2pm4UaVBt0g8CsV+E=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=mMTWi5xbkilsP4O8tJpM9yDbGVLfGx5zjsI6F1HDPBULDAa7gsN9qSb9XA8zeL2tJnoz/UjZhQC0ZD8t7qJ6PwFoQsdtwxbZw9e2QA86ChL1re0N2aUEz4yM3GyR5q6txJ7N+L8jUL+fkf7iDpw9524wWxl4Y19PaTVOVhHDTuA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ctshao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EQI24tFL; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ctshao.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EQI24tFL" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ee5668e09bso12754415a91.3 for ; Tue, 28 Jan 2025 16:25:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738110314; x=1738715114; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=PXeAr/W67pFJL0V452kTAWbs3WuQao1XWrD/nQMSmJk=; b=EQI24tFLn10xGvZ4HLHJC5iFHY4lTTa/aJSq3iHFTsK6Y2Va3jKT8g6l0uUyCWrzPE Vb22ShpdvAKlMH6Scf0uHctRtFx5y9nXM5MWY32JzrTTCFyDpPnqJQgdcXN8NYhemXlQ n4x0WCnkvp6xzOGAcIkmMtcahmBTr1H8TP4cPTmVBBK7XnblmrDklRVjv3MjoQ78emFA k56UYx+dpBt+DmBc5nZp4ysPhQ+E+Rt0jO39nRrXMxUdit/gQPYPB3k0sRwUfQXsOpJj UTSKrRCt1jaC5ld3XH3qMutgXP6lZimCaHKAxo3r7vpPzVD3Q88XB8kGsk5zG1e2OjQL 1sBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738110314; x=1738715114; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=PXeAr/W67pFJL0V452kTAWbs3WuQao1XWrD/nQMSmJk=; b=m4TMgGUjIDYnyPbpwOwLx+jKj7/7/CBJ/6lR5XDH7hXjULux10jqWSjghLgZewISKQ Tr7L/pOeorsWpmKvuOaeeUzaOQJUjz10ZflnUzrsw6cV2jyVL1d56A68hAFUr6w1Zx71 cEBTz7l2scdl4Ytl1A+26WqrbgEKHV7Mc9cDCkHjxJlb8I3YDi/4gP3I/y2He2jJp/7q apIfel4OX0QWZb/cEofjnMkUzvESx3dFMLuLFdVqiY0q6jilM9Pb2EtVrVqjzKY/tvBN z+ZS/Kp+yuMx5GVlWnTtjT33FeUpUH8FwyKqRKcLzpc06m/bFNR2AYKk4MfBZtVyqi/T i88A== X-Forwarded-Encrypted: i=1; AJvYcCWq4I4/OFIBe2JDNyvUopYw0UHOxvqVZ3N1PeDDhbeveJxNK6aN80cD6TmAJHrcBiJeArc=@vger.kernel.org X-Gm-Message-State: AOJu0YxL0clkm33e+k+ukp2QYvpL8YqUYFEuMu9XKnZUpMTbCFg94HXS qa4/L/jQwE4g9QvXBEuPcdZcLezwLxKrU25XjDMdSBOQIlbYNrdZn1+0adYbD6erh5m9N/NavY7 /lw== X-Google-Smtp-Source: AGHT+IEXM93qHwaO7edPXGrECqoSkr/Auhvr+ArqW++XActsaTaaIYnOSH8wKgfjnEoeuuF4WQsk+oHQunQ= X-Received: from pfbbu11.prod.google.com ([2002:a05:6a00:410b:b0:725:3321:1f0c]) (user=ctshao job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:148c:b0:71e:21:d2d8 with SMTP id d2e1a72fcca58-72fd0bf3be7mr1616751b3a.7.1738110313779; Tue, 28 Jan 2025 16:25:13 -0800 (PST) Date: Tue, 28 Jan 2025 16:14:57 -0800 In-Reply-To: <20250129001905.619859-1-ctshao@google.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250129001905.619859-1-ctshao@google.com> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog Message-ID: <20250129001905.619859-2-ctshao@google.com> Subject: [PATCH v3 1/5] perf lock: Add bpf maps for owner stack tracing From: Chun-Tse Shao To: linux-kernel@vger.kernel.org Cc: Chun-Tse Shao , peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, nathan@kernel.org, ndesaulniers@google.com, morbo@google.com, justinstitt@google.com, linux-perf-users@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Add a struct and few bpf maps in order to tracing owner stack. `struct owner_tracing_data`: Contains owner's pid, stack id, timestamp for when the owner acquires lock, and the count of lock waiters. `stack_buf`: Percpu buffer for retrieving owner stacktrace. `owner_stacks`: For tracing owner stacktrace to customized owner stack id. `owner_data`: For tracing lock_address to `struct owner_tracing_data` in bpf program. `owner_stat`: For reporting owner stacktrace in usermode. Signed-off-by: Chun-Tse Shao --- tools/perf/util/bpf_lock_contention.c | 16 +++++-- .../perf/util/bpf_skel/lock_contention.bpf.c | 42 ++++++++++++++++--- tools/perf/util/bpf_skel/lock_data.h | 7 ++++ 3 files changed, 57 insertions(+), 8 deletions(-) diff --git a/tools/perf/util/bpf_lock_contention.c b/tools/perf/util/bpf_lock_contention.c index fc8666222399..795e2374facc 100644 --- a/tools/perf/util/bpf_lock_contention.c +++ b/tools/perf/util/bpf_lock_contention.c @@ -131,9 +131,19 @@ int lock_contention_prepare(struct lock_contention *con) else bpf_map__set_max_entries(skel->maps.task_data, 1); - if (con->save_callstack) - bpf_map__set_max_entries(skel->maps.stacks, con->map_nr_entries); - else + if (con->save_callstack) { + bpf_map__set_max_entries(skel->maps.stacks, + con->map_nr_entries); + if (con->owner) { + bpf_map__set_value_size(skel->maps.stack_buf, con->max_stack * sizeof(u64)); + bpf_map__set_key_size(skel->maps.owner_stacks, + con->max_stack * sizeof(u64)); + bpf_map__set_max_entries(skel->maps.owner_stacks, con->map_nr_entries); + bpf_map__set_max_entries(skel->maps.owner_data, con->map_nr_entries); + bpf_map__set_max_entries(skel->maps.owner_stat, con->map_nr_entries); + skel->rodata->max_stack = con->max_stack; + } + } else bpf_map__set_max_entries(skel->maps.stacks, 1); if (target__has_cpu(target)) { diff --git a/tools/perf/util/bpf_skel/lock_contention.bpf.c b/tools/perf/util/bpf_skel/lock_contention.bpf.c index 6533ea9b044c..b4961dd86222 100644 --- a/tools/perf/util/bpf_skel/lock_contention.bpf.c +++ b/tools/perf/util/bpf_skel/lock_contention.bpf.c @@ -19,13 +19,37 @@ #define LCB_F_PERCPU (1U << 4) #define LCB_F_MUTEX (1U << 5) -/* callstack storage */ + /* buffer for owner stacktrace */ struct { - __uint(type, BPF_MAP_TYPE_STACK_TRACE); + __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); __uint(key_size, sizeof(__u32)); __uint(value_size, sizeof(__u64)); - __uint(max_entries, MAX_ENTRIES); -} stacks SEC(".maps"); + __uint(max_entries, 1); +} stack_buf SEC(".maps"); + +/* a map for tracing owner stacktrace to owner stack id */ +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(key_size, sizeof(__u64)); // owner stacktrace + __uint(value_size, sizeof(__u64)); // owner stack id + __uint(max_entries, 1); +} owner_stacks SEC(".maps"); + +/* a map for tracing lock address to owner data */ +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(key_size, sizeof(__u64)); // lock address + __uint(value_size, sizeof(struct owner_tracing_data)); + __uint(max_entries, 1); +} owner_data SEC(".maps"); + +/* a map for contention_key (stores owner stack id) to contention data */ +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(key_size, sizeof(struct contention_key)); + __uint(value_size, sizeof(struct contention_data)); + __uint(max_entries, 1); +} owner_stat SEC(".maps"); /* maintain timestamp at the beginning of contention */ struct { @@ -43,6 +67,14 @@ struct { __uint(max_entries, 1); } tstamp_cpu SEC(".maps"); +/* callstack storage */ +struct { + __uint(type, BPF_MAP_TYPE_STACK_TRACE); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(__u64)); + __uint(max_entries, MAX_ENTRIES); +} stacks SEC(".maps"); + /* actual lock contention statistics */ struct { __uint(type, BPF_MAP_TYPE_HASH); @@ -143,6 +175,7 @@ const volatile int needs_callstack; const volatile int stack_skip; const volatile int lock_owner; const volatile int use_cgroup_v2; +const volatile int max_stack; /* determine the key of lock stat */ const volatile int aggr_mode; @@ -466,7 +499,6 @@ int contention_end(u64 *ctx) return 0; need_delete = true; } - duration = bpf_ktime_get_ns() - pelem->timestamp; if ((__s64)duration < 0) { __sync_fetch_and_add(&time_fail, 1); diff --git a/tools/perf/util/bpf_skel/lock_data.h b/tools/perf/util/bpf_skel/lock_data.h index c15f734d7fc4..15f5743bd409 100644 --- a/tools/perf/util/bpf_skel/lock_data.h +++ b/tools/perf/util/bpf_skel/lock_data.h @@ -3,6 +3,13 @@ #ifndef UTIL_BPF_SKEL_LOCK_DATA_H #define UTIL_BPF_SKEL_LOCK_DATA_H +struct owner_tracing_data { + u32 pid; // Who has the lock. + u32 count; // How many waiters for this lock. + u64 timestamp; // The time while the owner acquires lock and contention is going on. + s32 stack_id; // Identifier for `owner_stat`, which stores as value in `owner_stacks` +}; + struct tstamp_data { u64 timestamp; u64 lock; From patchwork Wed Jan 29 00:14:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chun-Tse Shao X-Patchwork-Id: 13953377 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C77B6154BE4 for ; Wed, 29 Jan 2025 00:25:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738110352; cv=none; b=JwxU9bYZ3150v4sI210rrmrg2N05ZJis7PD0AIF9pkLbcb0UxcJssN/Dlj0ndpK9yEDh/4IfUEG3vTW2rlCvS4RZqn8Jy4+R9tNdiNLTl+hMOrgAubGGY7CEZqhN0t3BAe1ZgPLS6vHoHWsi3zhUc7CVJra8tqeDx7Bpx9cRk3U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738110352; c=relaxed/simple; bh=SVTMW1S7JFSDg+bA7DmzZRGGAfUXEG2NGYwvrvXuhZY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Ja2Gwlzp0dAvMUdR1dRXJ5+xN7tnq7QwCM18w+BkypygdlT7GhIu6rEbZrT1IvtBEPzcYRijv+0Ky3YQlgpxGaMPjNQhtQaYD42UU5fm+NjvuIxTMEZXwbDy729IjOEy58PKgi6LdrVBEDddsc1pXNM+dRDIQ59lolf/VidA0ws= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ctshao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lOFJcyYr; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ctshao.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lOFJcyYr" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ee8ced572eso12800205a91.0 for ; Tue, 28 Jan 2025 16:25:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738110350; x=1738715150; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/FqegXtUqY4HfJigVh80j6ehxCeSFMBdL2Y3cibZPD8=; b=lOFJcyYrUmaEGOA3esXyalGKCDfOduTZtWjF8PZjvzUSR4fEMyfoXXsK0c1HQ+Dj34 LfDZZbNfpdRoMJ72TKQXbphQ67wZyvO2d9Wwhu7QVUdQpp9Pz+0egZR2Equ5g4q0T/sT 3wLRMhjAbAbH10j9laeE1k6dfp7MRR8aclHF+wx0w07krtr/VzyJLdjtAspYZMEVkUEW QwRQSklmGmzWmIa3oEFeId4t5gAwiDPW0czVvWUAEakBenxTCFTTayVB6w80ytHRJwLX jYkgSU7YHnBzpa/wINMbBnX5yWuqnomyAgKBaHkjYv+A6Gm4YDgcvquVe8qSkTR7G7aS cTFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738110350; x=1738715150; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/FqegXtUqY4HfJigVh80j6ehxCeSFMBdL2Y3cibZPD8=; b=KWlQU8wuEVHJ05vd78i7xUp5tLXu3T4YGDXb71ZezsLE5I1pF54GW81v0miVSAhDYZ YTZqc2FDNVhyWSK6hh0FUJBWeDjG/hLr4IZocwq2D0PcVOY3bvCiSstED4ZAFqRn5kTd jp7fEJSy5cYrLQv1Nzj9UwakgfdVVQvuJ8dXi7D0+s7pat6viC6M3ITM6ZyYOn0x74X3 xtl7IQWd0zpvskoVIBW/PTZDdX5Lusw9M81WQ+K/DsmHBtOQunlm0Rb7fUDkX908jLOa DTnNcJWhq+EeB7Gpe0yg5WBhljCxKcgj8ZcXwuudZDNU9YE3JZI+RJUv54bgcnkr5A6M MGJQ== X-Forwarded-Encrypted: i=1; AJvYcCVF5x2YyzKHwVQ/GPF/0etrI/yaVswX/9md+gRrR8NlDvaToieTmJdhcCldUSUwqNrE5/8=@vger.kernel.org X-Gm-Message-State: AOJu0YwuR2wC5Vj04zMKfCEv0E1d5RDAy44mVo9lgSWheGsDAmBtIRu4 1/uiaNcQtz9kgbyAyHYQfUUkC96fthfj2wOhEXlsL7DNPaOyt7hyyZUjNMhkX7IpfkhwnNuak4S FbQ== X-Google-Smtp-Source: AGHT+IHTLBUhEsTI+ojUVsezlYeIKuENRrT3nfo1YAKzsVdV0xdMPKwAv1CFbEaTsZ8jmOKgYykzXqLPQG0= X-Received: from pfbay19.prod.google.com ([2002:a05:6a00:3013:b0:725:eeaa:65e2]) (user=ctshao job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:3d0f:b0:725:96f2:9e63 with SMTP id d2e1a72fcca58-72fd0c7cb59mr1768559b3a.24.1738110350325; Tue, 28 Jan 2025 16:25:50 -0800 (PST) Date: Tue, 28 Jan 2025 16:14:58 -0800 In-Reply-To: <20250129001905.619859-1-ctshao@google.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250129001905.619859-1-ctshao@google.com> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog Message-ID: <20250129001905.619859-3-ctshao@google.com> Subject: [PATCH v3 2/5] perf lock: Retrieve owner callstack in bpf program From: Chun-Tse Shao To: linux-kernel@vger.kernel.org Cc: Chun-Tse Shao , peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, nathan@kernel.org, ndesaulniers@google.com, morbo@google.com, justinstitt@google.com, linux-perf-users@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Tracing owner callstack in `contention_begin()` and `contention_end()`, and storing in `owner_stat` bpf map. Signed-off-by: Chun-Tse Shao --- .../perf/util/bpf_skel/lock_contention.bpf.c | 237 +++++++++++++++++- 1 file changed, 235 insertions(+), 2 deletions(-) diff --git a/tools/perf/util/bpf_skel/lock_contention.bpf.c b/tools/perf/util/bpf_skel/lock_contention.bpf.c index b4961dd86222..1ad2a0793c37 100644 --- a/tools/perf/util/bpf_skel/lock_contention.bpf.c +++ b/tools/perf/util/bpf_skel/lock_contention.bpf.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) // Copyright (c) 2022 Google +#include "linux/bpf.h" #include "vmlinux.h" #include #include @@ -7,6 +8,7 @@ #include #include "lock_data.h" +#include /* for collect_lock_syms(). 4096 was rejected by the verifier */ #define MAX_CPUS 1024 @@ -31,7 +33,7 @@ struct { struct { __uint(type, BPF_MAP_TYPE_HASH); __uint(key_size, sizeof(__u64)); // owner stacktrace - __uint(value_size, sizeof(__u64)); // owner stack id + __uint(value_size, sizeof(__s32)); // owner stack id __uint(max_entries, 1); } owner_stacks SEC(".maps"); @@ -197,6 +199,9 @@ int data_fail; int task_map_full; int data_map_full; +struct task_struct *bpf_task_from_pid(s32 pid) __ksym; +void bpf_task_release(struct task_struct *p) __ksym; + static inline __u64 get_current_cgroup_id(void) { struct task_struct *task; @@ -420,6 +425,27 @@ static inline struct tstamp_data *get_tstamp_elem(__u32 flags) return pelem; } +static inline s32 get_owner_stack_id(u64 *stacktrace) +{ + s32 *id; + static s32 id_gen = 1; + + id = bpf_map_lookup_elem(&owner_stacks, stacktrace); + if (id) + return *id; + + // FIXME: currently `a = __sync_fetch_and_add(...)` cause "Invalid usage of the XADD return + // value" error in BPF program: https://github.com/llvm/llvm-project/issues/91888 + bpf_map_update_elem(&owner_stacks, stacktrace, &id_gen, BPF_NOEXIST); + __sync_fetch_and_add(&id_gen, 1); + + id = bpf_map_lookup_elem(&owner_stacks, stacktrace); + if (id) + return *id; + + return -1; +} + SEC("tp_btf/contention_begin") int contention_begin(u64 *ctx) { @@ -437,6 +463,91 @@ int contention_begin(u64 *ctx) pelem->flags = (__u32)ctx[1]; if (needs_callstack) { + u32 i = 0; + u32 id = 0; + int owner_pid; + u64 *buf; + struct task_struct *task; + struct owner_tracing_data *otdata; + + if (!lock_owner) + goto skip_owner_begin; + + task = get_lock_owner(pelem->lock, pelem->flags); + if (!task) + goto skip_owner_begin; + + owner_pid = BPF_CORE_READ(task, pid); + + buf = bpf_map_lookup_elem(&stack_buf, &i); + if (!buf) + goto skip_owner_begin; + for (i = 0; i < max_stack; i++) + buf[i] = 0x0; + + task = bpf_task_from_pid(owner_pid); + if (task) { + bpf_get_task_stack(task, buf, max_stack * sizeof(unsigned long), 0); + bpf_task_release(task); + } + + otdata = bpf_map_lookup_elem(&owner_data, &pelem->lock); + id = get_owner_stack_id(buf); + + // Contention just happens, or corner case `lock` is owned by process not + // `owner_pid`. For the corner case we treat it as unexpected internal error and + // just ignore the precvious tracing record. + if (!otdata || otdata->pid != owner_pid) { + struct owner_tracing_data first = { + .pid = owner_pid, + .timestamp = pelem->timestamp, + .count = 1, + .stack_id = id, + }; + bpf_map_update_elem(&owner_data, &pelem->lock, &first, BPF_ANY); + } + // Contention is ongoing and new waiter joins. + else { + __sync_fetch_and_add(&otdata->count, 1); + + // The owner is the same, but stacktrace might be changed. In this case we + // store/update `owner_stat` based on current owner stack id. + if (id != otdata->stack_id) { + u64 duration = otdata->timestamp - pelem->timestamp; + struct contention_key ckey = { + .stack_id = id, + .pid = 0, + .lock_addr_or_cgroup = 0, + }; + struct contention_data *cdata = + bpf_map_lookup_elem(&owner_stat, &ckey); + + if (!cdata) { + struct contention_data first = { + .total_time = duration, + .max_time = duration, + .min_time = duration, + .count = 1, + .flags = pelem->flags, + }; + bpf_map_update_elem(&owner_stat, &ckey, &first, + BPF_NOEXIST); + } else { + __sync_fetch_and_add(&cdata->total_time, duration); + __sync_fetch_and_add(&cdata->count, 1); + + /* FIXME: need atomic operations */ + if (cdata->max_time < duration) + cdata->max_time = duration; + if (cdata->min_time > duration) + cdata->min_time = duration; + } + + otdata->timestamp = pelem->timestamp; + otdata->stack_id = id; + } + } +skip_owner_begin: pelem->stack_id = bpf_get_stackid(ctx, &stacks, BPF_F_FAST_STACK_CMP | stack_skip); if (pelem->stack_id < 0) @@ -473,6 +584,7 @@ int contention_end(u64 *ctx) struct tstamp_data *pelem; struct contention_key key = {}; struct contention_data *data; + __u64 timestamp; __u64 duration; bool need_delete = false; @@ -499,12 +611,133 @@ int contention_end(u64 *ctx) return 0; need_delete = true; } - duration = bpf_ktime_get_ns() - pelem->timestamp; + timestamp = bpf_ktime_get_ns(); + duration = timestamp - pelem->timestamp; if ((__s64)duration < 0) { __sync_fetch_and_add(&time_fail, 1); goto out; } + if (needs_callstack && lock_owner) { + u64 owner_time; + struct contention_key ckey = {}; + struct contention_data *cdata; + struct owner_tracing_data *otdata; + + otdata = bpf_map_lookup_elem(&owner_data, &pelem->lock); + if (!otdata) + goto skip_owner_end; + + // Update `owner_stat`. + owner_time = timestamp - otdata->timestamp; + ckey.stack_id = otdata->stack_id; + cdata = bpf_map_lookup_elem(&owner_stat, &ckey); + + if (!cdata) { + struct contention_data first = { + .total_time = owner_time, + .max_time = owner_time, + .min_time = owner_time, + .count = 1, + .flags = pelem->flags, + }; + bpf_map_update_elem(&owner_stat, &ckey, &first, BPF_NOEXIST); + } else { + __sync_fetch_and_add(&cdata->total_time, owner_time); + __sync_fetch_and_add(&cdata->count, 1); + + /* FIXME: need atomic operations */ + if (cdata->max_time < owner_time) + cdata->max_time = owner_time; + if (cdata->min_time > owner_time) + cdata->min_time = owner_time; + } + + // No contention is occurring, delete `lock` entry in `owner_data`. + if (otdata->count <= 1) + bpf_map_delete_elem(&owner_data, &pelem->lock); + // Contention is still ongoing, with a new owner (current task). `owner_data` + // should be updated accordingly. + else { + u32 i = 0; + u64 *buf; + + // FIXME: __sync_fetch_and_sub(&otdata->count, 1) causes compile error. + otdata->count--; + + buf = bpf_map_lookup_elem(&stack_buf, &i); + if (!buf) + goto skip_owner_end; + for (i = 0; i < (u32)max_stack; i++) + buf[i] = 0x0; + + // ctx[1] has the return code of the lock function. + // If ctx[1] is not 0, the current task terminates lock waiting without + // acquiring it. Owner is not changed, but we still need to update the owner + // stack. + if (!ctx[1]) { + s32 id = 0; + struct task_struct *task = bpf_task_from_pid(otdata->pid); + + if (task) { + bpf_get_task_stack(task, buf, + max_stack * sizeof(unsigned long), 0); + bpf_task_release(task); + } + + id = get_owner_stack_id(buf); + + // If owner stack is changed, update `owner_data` and `owner_stat` + // accordingly. + if (id != otdata->stack_id) { + u64 duration = otdata->timestamp - pelem->timestamp; + struct contention_key ckey = { + .stack_id = id, + .pid = 0, + .lock_addr_or_cgroup = 0, + }; + struct contention_data *cdata = + bpf_map_lookup_elem(&owner_stat, &ckey); + + if (!cdata) { + struct contention_data first = { + .total_time = duration, + .max_time = duration, + .min_time = duration, + .count = 1, + .flags = pelem->flags, + }; + bpf_map_update_elem(&owner_stat, &ckey, &first, + BPF_NOEXIST); + } else { + __sync_fetch_and_add(&cdata->total_time, duration); + __sync_fetch_and_add(&cdata->count, 1); + + /* FIXME: need atomic operations */ + if (cdata->max_time < duration) + cdata->max_time = duration; + if (cdata->min_time > duration) + cdata->min_time = duration; + } + + otdata->timestamp = pelem->timestamp; + otdata->stack_id = id; + } + } + // If ctx[1] is 0, then update tracinng data with the current task, which is + // the new owner. + else { + otdata->pid = pid; + otdata->timestamp = timestamp; + + bpf_get_task_stack(bpf_get_current_task_btf(), buf, + max_stack * sizeof(unsigned long), 0); + otdata->stack_id = get_owner_stack_id(buf); + } + } + } +skip_owner_end: + switch (aggr_mode) { case LOCK_AGGR_CALLER: key.stack_id = pelem->stack_id; From patchwork Wed Jan 29 00:14:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chun-Tse Shao X-Patchwork-Id: 13953378 Received: from mail-oa1-f73.google.com (mail-oa1-f73.google.com [209.85.160.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0194155C8C for ; Wed, 29 Jan 2025 00:26:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738110391; cv=none; b=aSnqvgR9ZarQaD7f0m0Kc/1IAQ6sgm62VzfgsYjhWHi3uJLXtg2XZz9YPhUOYvnkfeHkrGTW9pGONp7f1MKf4d8764GBm5btbUQEgjNLPUYF3TVYYMnEWeTd0WPr7zSmyfGr+jyw0Y3VFKEkSyp2laz8QgjLHAg5fmwh7VN4NtI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738110391; c=relaxed/simple; bh=ttHPeGDMTcjcBIoqQRbKRxvn64T3Ln0kMiIrAztObZo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=FKK50aMcMAqKoWsUHv00gkxcoQCTC+IifiNFJkQJKRDzyKUiqkFATfSThVbi6r6stvSH8O7BL3A+R9PuCnvWa0KKNGXXcMAM/WM84sRFA5UKSZK5McH4P77PcOW42hArlMFV+6AU8o6Q30lCeH6i4A3CUFG8IESAfbYDH77mei8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ctshao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VBRmaTqG; arc=none smtp.client-ip=209.85.160.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ctshao.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VBRmaTqG" Received: by mail-oa1-f73.google.com with SMTP id 586e51a60fabf-29f9dcd1235so3723018fac.3 for ; Tue, 28 Jan 2025 16:26:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738110388; x=1738715188; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=pWNil8Fjn4Jar+XAMcIdAlTx0TuFDQai1n5OzDTNvzE=; b=VBRmaTqGQ4qYMo1iGsTK3FH/zCoaaQW7MoxpKwxsslOlGk/OuF3wICqLvZ0v4Ber0s a7WXuYZp4pd0FPcvKGTTSivuAsu8JO4Sbu2Tp7Sj8RJORqAjs8eK0NAZx7AVY9c0uCHl XafKEnV+fJcLZV3DqaHtBQEkCT86lJtfzVr53IWro21mfn5F/lhw0tjDPYzvNFn1Y6Q2 k4VzBx+maPMqGmYispp6vkQzZZH+HIq71mRXmX0V7DaW9HPQ6Achz0sKkXeiklgdJpT3 8HuaSpLkARSqc++c13BojKhXoSqZEarZ3n4/YbmfC5CBVBI0rcVEKlEi3a5NXGYoylvA isuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738110388; x=1738715188; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pWNil8Fjn4Jar+XAMcIdAlTx0TuFDQai1n5OzDTNvzE=; b=kcEjo4GOV7K3wlBditwdnvPfW/8LgRReotlUXYrQ+Bw10K38/1Gr/w+l54JjQbDrHP zYTaYWSKxgWso0nu5pOqVhty10uf2RPWlTHQRF2ChajHbGWfjwGDUZ2OLijUFp4bGAec if0Dr+zZByZxlg3kGhEqIGUgvayPq3bOYQBppFA1Dp4y8Erfvg3HTxbQpXIdNbwtLbT1 NGS5FRizknKjv8rgPCym7nZGmDWEnMR2EKXaEfWd8baSTiaKm/J8HhmVyMuqj019XeW2 0A4P22DhlFd5fu8e7JQDO01OYwz+PUEcJuE3JFLYQvJMNWJG8qSLFPtALjKuIYqOOhRl UG/Q== X-Forwarded-Encrypted: i=1; AJvYcCUa/JItswBmv0qa5fozJ24qm2pR/rjLk//g6ONAAu/cmzGZV1XQc/46ZRvGngDOxkvmtEc=@vger.kernel.org X-Gm-Message-State: AOJu0Yyakh2LwVHwIRtqWLJ7yqEIXsWvRhnuNScSx9oPyoElypllDlwe yCgab6U9gHX/N8/GI/pjzvXDjHStbnLPXjmd/I0+t8XdxwlneJgfV2SSK9TYBQR8i1FqOnyWuy9 wVA== X-Google-Smtp-Source: AGHT+IHsDI3JvNuSMQ9/eovun24uNULLxtkUZVYW6kTWXvv63qVQ/psGtmGrSgUv9BBLi2ZcriE/74v7jeg= X-Received: from oabsa15.prod.google.com ([2002:a05:6871:210f:b0:297:1959:a097]) (user=ctshao job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6870:71d6:b0:29e:49b4:92b2 with SMTP id 586e51a60fabf-2b32f0434e2mr647836fac.12.1738110387974; Tue, 28 Jan 2025 16:26:27 -0800 (PST) Date: Tue, 28 Jan 2025 16:14:59 -0800 In-Reply-To: <20250129001905.619859-1-ctshao@google.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250129001905.619859-1-ctshao@google.com> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog Message-ID: <20250129001905.619859-4-ctshao@google.com> Subject: [PATCH v3 3/5] perf lock: Make rb_tree helper functions generic From: Chun-Tse Shao To: linux-kernel@vger.kernel.org Cc: Chun-Tse Shao , peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, nathan@kernel.org, ndesaulniers@google.com, morbo@google.com, justinstitt@google.com, linux-perf-users@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev The rb_tree helper functions can be reused for parsing `owner_lock_stat` into rb tree for sorting. Signed-off-by: Chun-Tse Shao --- tools/perf/builtin-lock.c | 34 +++++++++++++++++++++++----------- 1 file changed, 23 insertions(+), 11 deletions(-) diff --git a/tools/perf/builtin-lock.c b/tools/perf/builtin-lock.c index 5d405cd8e696..9bebc186286f 100644 --- a/tools/perf/builtin-lock.c +++ b/tools/perf/builtin-lock.c @@ -418,16 +418,13 @@ static void combine_lock_stats(struct lock_stat *st) rb_insert_color(&st->rb, &sorted); } -static void insert_to_result(struct lock_stat *st, - int (*bigger)(struct lock_stat *, struct lock_stat *)) +static void insert_to(struct rb_root *rr, struct lock_stat *st, + int (*bigger)(struct lock_stat *, struct lock_stat *)) { - struct rb_node **rb = &result.rb_node; + struct rb_node **rb = &rr->rb_node; struct rb_node *parent = NULL; struct lock_stat *p; - if (combine_locks && st->combined) - return; - while (*rb) { p = container_of(*rb, struct lock_stat, rb); parent = *rb; @@ -439,13 +436,21 @@ static void insert_to_result(struct lock_stat *st, } rb_link_node(&st->rb, parent, rb); - rb_insert_color(&st->rb, &result); + rb_insert_color(&st->rb, rr); } -/* returns left most element of result, and erase it */ -static struct lock_stat *pop_from_result(void) +static inline void insert_to_result(struct lock_stat *st, + int (*bigger)(struct lock_stat *, + struct lock_stat *)) +{ + if (combine_locks && st->combined) + return; + insert_to(&result, st, bigger); +} + +static inline struct lock_stat *pop_from(struct rb_root *rr) { - struct rb_node *node = result.rb_node; + struct rb_node *node = rr->rb_node; if (!node) return NULL; @@ -453,8 +458,15 @@ static struct lock_stat *pop_from_result(void) while (node->rb_left) node = node->rb_left; - rb_erase(node, &result); + rb_erase(node, rr); return container_of(node, struct lock_stat, rb); + +} + +/* returns left most element of result, and erase it */ +static struct lock_stat *pop_from_result(void) +{ + return pop_from(&result); } struct trace_lock_handler { From patchwork Wed Jan 29 00:15:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chun-Tse Shao X-Patchwork-Id: 13953379 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 83AE0155C8C for ; Wed, 29 Jan 2025 00:27:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738110427; cv=none; b=PEmkqEi9cRifFRio3sInTgdRGpyamFz+XpjZoI0oS+tPwW+OjhQMaKv43NXZrOQaQWoRw5A7ES5x1RS0fDUrWVG0Hpv/58O8m3gI71D9fdBX5XxR+i+Cvx+NV7JjXkm+0sH4xwGNDC9exwLiPp0wNr/KEZEjr1NMMzklykvUDGI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738110427; c=relaxed/simple; bh=8Sgp5yu8OBVqSwtXE0quymzsxKTHzx8BbCOes46TEKM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hCTh2sc7O96qvWi7g3oXsl1v721mx3t3P1vKiYGMfl0gkoWZGZnae8SCGwpqgODVdm776rPqpQprCAkWJoYHrJdUDsB/rAXC8NLeNXb7BZbX7IoYseahWUumKHR0nQVczz5XrJG0Eyvsbes3AdPKOX3/+JisC7/5I4HN82nKzJc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ctshao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lUKCHnN/; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ctshao.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lUKCHnN/" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-218cf85639eso186538645ad.3 for ; Tue, 28 Jan 2025 16:27:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738110425; x=1738715225; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=sBEvePub/SR9/IDN5Bs+wN21Z641MzpSYW1O9rZmtN0=; b=lUKCHnN/EKzsGdHBnOen1OcljRUlEcBpZBTRlzMHmGg8qeMQSZldnVyyqYcZ/j0/K1 nWDB1mtJU6abQLgmB36NgX0XeMAewS+xsbtc6BEcoPjZlBIvgtp+jMTPRLijA2ndkcLz R9cJMfeNRjPTVNOnWPYY+c3LKKNYDjZb0V2Iy1M8cOqDzrOnyE+FQPO540NY33OCH06w Bw4KuHiSRsMuSCJkpfMy/Laj0tVOQ+d69uiqQ7XbluPUsM4UGgKNOP7Vzub+RAsErZsh B4V+1ectM2mfy4aCZhA9GB+yOjLhKCc83PjkERmNbDj55JRimUVwAujJinluEawGtPRS TrJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738110425; x=1738715225; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sBEvePub/SR9/IDN5Bs+wN21Z641MzpSYW1O9rZmtN0=; b=J0+7IA1KwJCBvkQMgLJbr1g4VQ+nMu15WYcje7IIMNSKL0PzhaeVZzaeataqhg19/2 AAColtNW6hfwMcpztjBmg2GK70Ghz/7IEcKQOmHuX1f/V5ceZCkFdHMbNjHf9M1mnBLf q+wQI+klhvh4etZAd2Z/xT1e55B9OgBOR/T371NBNzsuBe1JXm+cau4FQ2UbTYIaiaoS eSsa2fipiEf9q1S7lkfsVQoRyHyL9z7gXtXPCj02u/hFnz0zScCfExAKpnSkZFyLX4Hl 6qhrvMcu33YXjMGYlxtOtfJSJy4WPJPpdvahRmx9rzK3P1kpiTPs8vI9ccKCAuNFTrwr n/PQ== X-Forwarded-Encrypted: i=1; AJvYcCX2QlAkJqS6w/vYkGh95IaCl/cgco8uQeHQvX+B8MHZVtUsBDsn5uU4STN57cpB5LLVVeo=@vger.kernel.org X-Gm-Message-State: AOJu0YyfTNHu50zTtgQ3MXlqSbdWlZYdXcmnaCyUbT0ISSgD05M0iiJZ RzIkdPoI9Dx/TWS29QxWqXZlfmyjfJ+/TmwUBEpvcfc8jN38nXzALvj+I2HBC59PCLdabhaDdBm L+Q== X-Google-Smtp-Source: AGHT+IGLXoWuDc+BL3s0utjGdQvHw6YCJ9vbchFUR1aiImf8YniBCyB4qvpzZenwbn+L4cac0UqIBCcC0M0= X-Received: from pjc3.prod.google.com ([2002:a17:90b:2f43:b0:2ef:7352:9e97]) (user=ctshao job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:2446:b0:215:603e:2141 with SMTP id d9443c01a7336-21dd7c625cemr15041855ad.19.1738110424789; Tue, 28 Jan 2025 16:27:04 -0800 (PST) Date: Tue, 28 Jan 2025 16:15:00 -0800 In-Reply-To: <20250129001905.619859-1-ctshao@google.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250129001905.619859-1-ctshao@google.com> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog Message-ID: <20250129001905.619859-5-ctshao@google.com> Subject: [PATCH v3 4/5] perf lock: Report owner stack in usermode From: Chun-Tse Shao To: linux-kernel@vger.kernel.org Cc: Chun-Tse Shao , peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, nathan@kernel.org, ndesaulniers@google.com, morbo@google.com, justinstitt@google.com, linux-perf-users@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev Parse `owner_lock_stat` into a rb tree, and report owner lock stats with stack trace in order. Example output: $ sudo ~/linux/tools/perf/perf lock con -abvo -Y mutex-spin -E3 perf bench sched pipe ... contended total wait max wait avg wait type caller 171 1.55 ms 20.26 us 9.06 us mutex pipe_read+0x57 0xffffffffac6318e7 pipe_read+0x57 0xffffffffac623862 vfs_read+0x332 0xffffffffac62434b ksys_read+0xbb 0xfffffffface604b2 do_syscall_64+0x82 0xffffffffad00012f entry_SYSCALL_64_after_hwframe+0x76 36 193.71 us 15.27 us 5.38 us mutex pipe_write+0x50 0xffffffffac631ee0 pipe_write+0x50 0xffffffffac6241db vfs_write+0x3bb 0xffffffffac6244ab ksys_write+0xbb 0xfffffffface604b2 do_syscall_64+0x82 0xffffffffad00012f entry_SYSCALL_64_after_hwframe+0x76 4 51.22 us 16.47 us 12.80 us mutex do_epoll_wait+0x24d 0xffffffffac691f0d do_epoll_wait+0x24d 0xffffffffac69249b do_epoll_pwait.part.0+0xb 0xffffffffac693ba5 __x64_sys_epoll_pwait+0x95 0xfffffffface604b2 do_syscall_64+0x82 0xffffffffad00012f entry_SYSCALL_64_after_hwframe+0x76 === owner stack trace === 3 31.24 us 15.27 us 10.41 us mutex pipe_read+0x348 0xffffffffac631bd8 pipe_read+0x348 0xffffffffac623862 vfs_read+0x332 0xffffffffac62434b ksys_read+0xbb 0xfffffffface604b2 do_syscall_64+0x82 0xffffffffad00012f entry_SYSCALL_64_after_hwframe+0x76 ... Signed-off-by: Chun-Tse Shao --- tools/perf/builtin-lock.c | 20 ++++++++-- tools/perf/util/bpf_lock_contention.c | 54 +++++++++++++++++++++++++++ tools/perf/util/lock-contention.h | 7 ++++ 3 files changed, 78 insertions(+), 3 deletions(-) diff --git a/tools/perf/builtin-lock.c b/tools/perf/builtin-lock.c index 9bebc186286f..d9b0d7472aea 100644 --- a/tools/perf/builtin-lock.c +++ b/tools/perf/builtin-lock.c @@ -42,6 +42,7 @@ #include #include #include +#include static struct perf_session *session; static struct target target; @@ -1817,6 +1818,22 @@ static void print_contention_result(struct lock_contention *con) break; } + if (con->owner && con->save_callstack) { + struct rb_root root = RB_ROOT; + + if (symbol_conf.field_sep) + fprintf(lock_output, "# owner stack trace:\n"); + else + fprintf(lock_output, "\n=== owner stack trace ===\n\n"); + while ((st = pop_owner_stack_trace(con))) + insert_to(&root, st, compare); + + while ((st = pop_from(&root))) { + print_lock_stat(con, st); + zfree(st); + } + } + if (print_nr_entries) { /* update the total/bad stats */ while ((st = pop_from_result())) { @@ -1962,9 +1979,6 @@ static int check_lock_contention_options(const struct option *options, } } - if (show_lock_owner) - show_thread_stats = true; - return 0; } diff --git a/tools/perf/util/bpf_lock_contention.c b/tools/perf/util/bpf_lock_contention.c index 795e2374facc..cf3267e46589 100644 --- a/tools/perf/util/bpf_lock_contention.c +++ b/tools/perf/util/bpf_lock_contention.c @@ -549,6 +549,60 @@ static const char *lock_contention_get_name(struct lock_contention *con, return name_buf; } +struct lock_stat *pop_owner_stack_trace(struct lock_contention *con) +{ + int stacks_fd, stat_fd; + u64 *stack_trace; + s32 stack_id; + struct contention_key ckey = {}; + struct contention_data cdata = {}; + size_t stack_size = con->max_stack * sizeof(*stack_trace); + struct lock_stat *st; + char name[KSYM_NAME_LEN]; + + stacks_fd = bpf_map__fd(skel->maps.owner_stacks); + stat_fd = bpf_map__fd(skel->maps.owner_stat); + if (!stacks_fd || !stat_fd) + return NULL; + + stack_trace = zalloc(stack_size); + if (stack_trace == NULL) + return NULL; + + if (bpf_map_get_next_key(stacks_fd, NULL, stack_trace)) + return NULL; + + bpf_map_lookup_elem(stacks_fd, stack_trace, &stack_id); + ckey.stack_id = stack_id; + bpf_map_lookup_elem(stat_fd, &ckey, &cdata); + + st = zalloc(sizeof(struct lock_stat)); + if (!st) + return NULL; + + strcpy(name, + stack_trace[0] ? lock_contention_get_name(con, NULL, stack_trace, 0) : "unknown"); + + st->name = strdup(name); + if (!st->name) + return NULL; + + st->flags = cdata.flags; + st->nr_contended = cdata.count; + st->wait_time_total = cdata.total_time; + st->wait_time_max = cdata.max_time; + st->wait_time_min = cdata.min_time; + st->callstack = stack_trace; + + if (cdata.count) + st->avg_wait_time = cdata.total_time / cdata.count; + + bpf_map_delete_elem(stacks_fd, stack_trace); + bpf_map_delete_elem(stat_fd, &ckey); + + return st; +} + int lock_contention_read(struct lock_contention *con) { int fd, stack, err = 0; diff --git a/tools/perf/util/lock-contention.h b/tools/perf/util/lock-contention.h index a09f7fe877df..97fd33c57f17 100644 --- a/tools/perf/util/lock-contention.h +++ b/tools/perf/util/lock-contention.h @@ -168,6 +168,8 @@ int lock_contention_stop(void); int lock_contention_read(struct lock_contention *con); int lock_contention_finish(struct lock_contention *con); +struct lock_stat *pop_owner_stack_trace(struct lock_contention *con); + #else /* !HAVE_BPF_SKEL */ static inline int lock_contention_prepare(struct lock_contention *con __maybe_unused) @@ -187,6 +189,11 @@ static inline int lock_contention_read(struct lock_contention *con __maybe_unuse return 0; } +struct lock_stat *pop_owner_stack_trace(struct lock_contention *con) +{ + return NULL; +} + #endif /* HAVE_BPF_SKEL */ #endif /* PERF_LOCK_CONTENTION_H */ From patchwork Wed Jan 29 00:15:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chun-Tse Shao X-Patchwork-Id: 13953380 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE3F815534E for ; Wed, 29 Jan 2025 00:27:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738110462; cv=none; b=K+a4mh7iQIcObzJds2VrB1mepmT/UprO/OwSPAknGfF6zSlLaa+Ux897KtiPI1XV7u3v0O+h5MaUmHu/3evmjxPMF7s3ksdMoerbbCcfnGO6PNWxSUi0zU1xO9umz0dxfnKoj0/Oy4EOtGJn1RdW85WueJ4IKRFkba8FAiEwm0w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738110462; c=relaxed/simple; bh=JXR8T33WgTBIEDVxRbVK0LNxlccdzx4b2VLjaRPQh0w=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gRqzm4NMfhrgy8MtWtQqcFzypuYhq6NU9bkZdGb47fU1F+y9JRdg1qgxfi87WIYQNL/knI0xALy+rqhdL1ws1tUbcSOO+VDPlXL9XNRGx5eDJqqV4wy3SpyYYyujLTgPcTkp2E1IzuIceI+pRkRyuPZhXrP6lgPoN2xkOSo4doQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ctshao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=nvRFGquY; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ctshao.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="nvRFGquY" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2f5538a2356so11575084a91.2 for ; Tue, 28 Jan 2025 16:27:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738110460; x=1738715260; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/kZ7iqAFWEW/1CVmn2pzdj8BRoAIsJA95X92RXs36J8=; b=nvRFGquYQypX/GMKG5jfKGxtDChsO1vDSr9GUI/hHLe7/D0oI9eS1PosoMoJPE1BEP dX6OVnkNdthZoOTx38OqagAzRcrgQ6546uoSjyKRzD3nzI9AE8UpVTKNG/4yn8HNlnqo n0iLhZGVNlaieHPmDkvL3P246sFVHGauFsb9B4W9zz8gJXwKJBMGFM9UFe4QtNZb8eqr Od3wq84AxXsrIBn1xfoO/eI0IyT6RdnvMIYlNW5r/K26ISmq2oDLSLVtqjd8fo5aRYcW Jr9h1MzBAVv0pQ1ilGxkoxyyUUZ2+zJIcoXa/hX1TJHbyZC0lMQvf9KKEdPKOjwvW/QT Iatg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738110460; x=1738715260; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/kZ7iqAFWEW/1CVmn2pzdj8BRoAIsJA95X92RXs36J8=; b=Gd3YnwPaBQzxrFVIpmg5RZ3JvGeLc+Hbox0lpSGWuVY0acmfqyZNwaCW4iTqIakMHi OoH0WJi2GRpwr7tCDf5AK3hRl0phtzyF37IQFjANPh1AJRvpDGbv/xYYTRoRnCounOnm Lb64OWP/PhgE4hx8sSk0mAK1CSxMJWp/rRg+gwakTX3TRG2uhslimkwD2/+q2FJ6i22T 3//a7Gm0UAXNHghHCixnF6GpC4aEdHxZT+pFU07yoiCtgOzLow7bspu4b5EeVEVcEpO+ jhWzbz2/WiBTEPLgEImXH1farZ+UDKuqTe1QXlT7iKaKWmG2XOGBIO0/8vpG/ISwXioM 1i3Q== X-Forwarded-Encrypted: i=1; AJvYcCWtqD7OaTt8SGTgoPI9H3NE+y+SRYFfbjJ/lS/pkr2csXgJby+7y9yM7FUjV+nUNUdtSoM=@vger.kernel.org X-Gm-Message-State: AOJu0YyV4vTCoc9EaRQEcdaVcPUf1nqzGYI0nceUGzssgBJDYPpbQRQh 6HBRqB1zE2PaWqvuuVaHQhspC+uvzsy77dqWfypYCf8vDxKMhpbte+LFdgQlmFqqQzX9hLW+/ol FWQ== X-Google-Smtp-Source: AGHT+IHuivb002Myaw3lnKxSivwCEVIBuEWsY7r6mdBhKNiyiCBu9zbawayIbipZdTXtPqU4Sr6JtaI2KY0= X-Received: from pjyr14.prod.google.com ([2002:a17:90a:e18e:b0:2e5:5ffc:1c36]) (user=ctshao job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5190:b0:2ee:d63f:d8f with SMTP id 98e67ed59e1d1-2f83abb91d4mr1576644a91.13.1738110460020; Tue, 28 Jan 2025 16:27:40 -0800 (PST) Date: Tue, 28 Jan 2025 16:15:01 -0800 In-Reply-To: <20250129001905.619859-1-ctshao@google.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250129001905.619859-1-ctshao@google.com> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog Message-ID: <20250129001905.619859-6-ctshao@google.com> Subject: [PATCH v3 5/5] perf lock: Update documentation for -o option in contention mode From: Chun-Tse Shao To: linux-kernel@vger.kernel.org Cc: Chun-Tse Shao , peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, nathan@kernel.org, ndesaulniers@google.com, morbo@google.com, justinstitt@google.com, linux-perf-users@vger.kernel.org, bpf@vger.kernel.org, llvm@lists.linux.dev This patch also decouple -o with -t, and shows warning to notify the new behavior for -ov. Signed-off-by: Chun-Tse Shao --- tools/perf/builtin-lock.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/tools/perf/builtin-lock.c b/tools/perf/builtin-lock.c index d9b0d7472aea..b925be06b0d8 100644 --- a/tools/perf/builtin-lock.c +++ b/tools/perf/builtin-lock.c @@ -1818,7 +1818,7 @@ static void print_contention_result(struct lock_contention *con) break; } - if (con->owner && con->save_callstack) { + if (con->owner && con->save_callstack && verbose > 0) { struct rb_root root = RB_ROOT; if (symbol_conf.field_sep) @@ -1979,6 +1979,11 @@ static int check_lock_contention_options(const struct option *options, } } + if (show_lock_owner && !show_thread_stats) { + pr_warning("Now -o try to show owner's callstack instead of pid and comm.\n"); + pr_warning("Please use -t option too to keep the old behavior.\n"); + } + return 0; } @@ -2570,7 +2575,8 @@ int cmd_lock(int argc, const char **argv) "Filter specific address/symbol of locks", parse_lock_addr), OPT_CALLBACK('S', "callstack-filter", NULL, "NAMES", "Filter specific function in the callstack", parse_call_stack), - OPT_BOOLEAN('o', "lock-owner", &show_lock_owner, "show lock owners instead of waiters"), + OPT_BOOLEAN('o', "lock-owner", &show_lock_owner, "show lock owners instead of waiters.\n" + "\t\t\tThis option can be combined with -t, which shows owner's per thread lock stats, or -v, which shows owner's stacktrace"), OPT_STRING_NOEMPTY('x', "field-separator", &symbol_conf.field_sep, "separator", "print result in CSV format with custom separator"), OPT_BOOLEAN(0, "lock-cgroup", &show_lock_cgroups, "show lock stats by cgroup"),