From patchwork Mon Mar 13 20:48:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13173257 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5147C6FD1C for ; Mon, 13 Mar 2023 20:48:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229918AbjCMUsc (ORCPT ); Mon, 13 Mar 2023 16:48:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53178 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229519AbjCMUsb (ORCPT ); Mon, 13 Mar 2023 16:48:31 -0400 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44DCCCA1D; Mon, 13 Mar 2023 13:48:30 -0700 (PDT) Received: by mail-pj1-x102e.google.com with SMTP id h11-20020a17090a2ecb00b00237c740335cso12998108pjs.3; Mon, 13 Mar 2023 13:48:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678740510; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=EKebWal3Uq3B/B29ONbk7l0nGGLiL0OrNfNz2Gv7aas=; b=Jerz2VowYpm604pCMi7BCgakKHVM/UQJ0tpx4zE/aJ7m2XKSvvZaeRYsOPHvU7pPau 2UOTebugH2WetW+MZ2lIQOIr6x0SdEpNKXsdUipe1WOheIn4YqJhP7d87vBPOCwHlcYT NKNz8nmEtaQKTNh5XKYS/PEBuiuQVJaPvb6F8rqf+7ofkNyJxX5xbkaIGyAyEwUmMqSj iW4eNanNOOWaL00/ZPlq/DFfPs35n3ECcLyigxW/yUPjRUTOit8c3ahMwa1NnpbgLJRI fnSj/e7LJB8qYQ62w1bv47RVK1rhnx8n1f2iS5t+QRkukMmyKLiQhjnO9Cwm2sEG/kac Hu/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678740510; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=EKebWal3Uq3B/B29ONbk7l0nGGLiL0OrNfNz2Gv7aas=; b=HYRirpcrXyPhyFodjzCVLxpOPlmyO8cPtfMQXpOUKbSfAhjXAPSR/++rvrpziWv2x7 wse/3w/+jjwkpUMGMGiQV/noKjUBSfVupI3dWk5cDEbia4uGSYc5LK0e8ifVTCRDpMom j10sY7viaVTrpa4w7Jkoi/AtGMScYfFM+KSABcuuXxKRNAX4FbYyr4qnj5t2XobCTxHD fpayhZYLgXdWt7dZe4hqoS6jU7d/SCNti4VKRHtQtraAxttxBoHLZeKjntAITI8UvXRW LI1OuFveXmCpuEAjkui4Z5VbVK6HeWE7SiPOTmuVsSGkzKmxG6c6iOwN8tTXHinrdIO0 xj0Q== X-Gm-Message-State: AO0yUKV4uUV6Uxl7lClAG/MQYdqsy8RFLtU6GnQ/XjzsJd83lxfR7s5h i2WMPn87kr2vCTwVv/F7j8I= X-Google-Smtp-Source: AK7set/NFHPJznZSDKK/RuZNVANqMGU8JhERNa1yuQ+9wRJB0ObpoorvWVvXjeiwbNUEnov1Km85zA== X-Received: by 2002:a17:90b:383:b0:23b:513d:8ff6 with SMTP id ga3-20020a17090b038300b0023b513d8ff6mr6315041pjb.40.1678740509544; Mon, 13 Mar 2023 13:48:29 -0700 (PDT) Received: from moohyul.svl.corp.google.com ([2620:15c:2d4:203:26de:6cd7:2c4f:96d5]) by smtp.gmail.com with ESMTPSA id s12-20020a17090aba0c00b0023af8a3cf6esm265026pjr.48.2023.03.13.13.48.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Mar 2023 13:48:29 -0700 (PDT) Sender: Namhyung Kim From: Namhyung Kim To: Arnaldo Carvalho de Melo , Jiri Olsa Cc: Peter Zijlstra , Ingo Molnar , Song Liu , Hao Luo , Juri Lelli , Will Deacon , Waiman Long , Boqun Feng , Stephane Eranian , LKML , linux-perf-users@vger.kernel.org, bpf@vger.kernel.org, Suren Baghdasaryan Subject: [PATCH 1/4] perf lock contention: Track and show mmap_lock with address Date: Mon, 13 Mar 2023 13:48:22 -0700 Message-Id: <20230313204825.2665483-2-namhyung@kernel.org> X-Mailer: git-send-email 2.40.0.rc1.284.g88254d51c5-goog In-Reply-To: <20230313204825.2665483-1-namhyung@kernel.org> References: <20230313204825.2665483-1-namhyung@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Sometimes there are severe contentions on the mmap_lock and we want see it in the -l/--lock-addr output. However it cannot symbolize the mmap_lock because it's allocated dynamically without symbols. Stephane and Hao gave me an idea separately to display mmap_lock by following the current->mm pointer. I added a flag to mark mmap_lock after comparing the lock address so that it can show them differently. With this change it can show mmap_lock like below: $ sudo ./perf lock con -abl -- sleep 10 contended total wait max wait avg wait address symbol ... 16344 312.30 ms 2.22 ms 19.11 us ffff8cc702595640 17686 310.08 ms 1.49 ms 17.53 us ffff8cc7025952c0 3 84.14 ms 45.79 ms 28.05 ms ffff8cc78114c478 mmap_lock 3557 76.80 ms 68.75 us 21.59 us ffff8cc77ca3af58 1 68.27 ms 68.27 ms 68.27 ms ffff8cda745dfd70 9 54.53 ms 7.96 ms 6.06 ms ffff8cc7642a48b8 mmap_lock 14629 44.01 ms 60.00 us 3.01 us ffff8cc7625f9ca0 3481 42.63 ms 140.71 us 12.24 us ffffffff937906ac vmap_area_lock 16194 38.73 ms 42.15 us 2.39 us ffff8cd397cbc560 11 38.44 ms 10.39 ms 3.49 ms ffff8ccd6d12fbb8 mmap_lock 1 5.43 ms 5.43 ms 5.43 ms ffff8cd70018f0d8 1674 5.38 ms 422.93 us 3.21 us ffffffff92e06080 tasklist_lock 581 4.51 ms 130.68 us 7.75 us ffff8cc9b1259058 5 3.52 ms 1.27 ms 703.23 us ffff8cc754510070 112 3.47 ms 56.47 us 31.02 us ffff8ccee38b3120 381 3.31 ms 73.44 us 8.69 us ffffffff93790690 purge_vmap_area_lock 255 3.19 ms 36.35 us 12.49 us ffff8d053ce30c80 Note that mmap_lock was renamed some time ago and it needs to support old kernels with a different name 'mmap_sem'. Suggested-by: Stephane Eranian Suggested-by: Hao Luo Cc: Suren Baghdasaryan Signed-off-by: Namhyung Kim --- tools/perf/builtin-lock.c | 2 +- .../perf/util/bpf_skel/lock_contention.bpf.c | 41 +++++++++++++++++++ tools/perf/util/bpf_skel/lock_data.h | 6 +++ 3 files changed, 48 insertions(+), 1 deletion(-) diff --git a/tools/perf/builtin-lock.c b/tools/perf/builtin-lock.c index 054997edd98b..c62f4d9363a6 100644 --- a/tools/perf/builtin-lock.c +++ b/tools/perf/builtin-lock.c @@ -1663,7 +1663,7 @@ static void print_contention_result(struct lock_contention *con) break; case LOCK_AGGR_ADDR: pr_info(" %016llx %s\n", (unsigned long long)st->addr, - st->name ? : ""); + (st->flags & LCD_F_MMAP_LOCK) ? "mmap_lock" : st->name); break; default: break; diff --git a/tools/perf/util/bpf_skel/lock_contention.bpf.c b/tools/perf/util/bpf_skel/lock_contention.bpf.c index e6007eaeda1a..f092a78ae2b5 100644 --- a/tools/perf/util/bpf_skel/lock_contention.bpf.c +++ b/tools/perf/util/bpf_skel/lock_contention.bpf.c @@ -92,6 +92,14 @@ struct rw_semaphore___new { atomic_long_t owner; } __attribute__((preserve_access_index)); +struct mm_struct___old { + struct rw_semaphore mmap_sem; +} __attribute__((preserve_access_index)); + +struct mm_struct___new { + struct rw_semaphore mmap_lock; +} __attribute__((preserve_access_index)); + /* control flags */ int enabled; int has_cpu; @@ -204,6 +212,36 @@ static inline struct task_struct *get_lock_owner(__u64 lock, __u32 flags) return task; } +static inline __u32 check_lock_type(__u64 lock, __u32 flags) +{ + struct task_struct *curr; + struct mm_struct___old *mm_old; + struct mm_struct___new *mm_new; + + switch (flags) { + case LCB_F_READ: /* rwsem */ + case LCB_F_WRITE: + curr = bpf_get_current_task_btf(); + if (curr->mm == NULL) + break; + mm_new = (void *)curr->mm; + if (bpf_core_field_exists(mm_new->mmap_lock)) { + if (&mm_new->mmap_lock == (void *)lock) + return LCD_F_MMAP_LOCK; + break; + } + mm_old = (void *)curr->mm; + if (bpf_core_field_exists(mm_old->mmap_sem)) { + if (&mm_old->mmap_sem == (void *)lock) + return LCD_F_MMAP_LOCK; + } + break; + default: + break; + } + return 0; +} + SEC("tp_btf/contention_begin") int contention_begin(u64 *ctx) { @@ -314,6 +352,9 @@ int contention_end(u64 *ctx) .flags = pelem->flags, }; + if (aggr_mode == LOCK_AGGR_ADDR) + first.flags |= check_lock_type(pelem->lock, pelem->flags); + bpf_map_update_elem(&lock_stat, &key, &first, BPF_NOEXIST); bpf_map_delete_elem(&tstamp, &pid); return 0; diff --git a/tools/perf/util/bpf_skel/lock_data.h b/tools/perf/util/bpf_skel/lock_data.h index 3d35fd4407ac..789f20833798 100644 --- a/tools/perf/util/bpf_skel/lock_data.h +++ b/tools/perf/util/bpf_skel/lock_data.h @@ -15,6 +15,12 @@ struct contention_task_data { char comm[TASK_COMM_LEN]; }; +/* + * Upper bits of the flags in the contention_data are used to identify + * some well-known locks which do not have symbols (non-global locks). + */ +#define LCD_F_MMAP_LOCK (1U << 31) + struct contention_data { u64 total_time; u64 min_time; From patchwork Mon Mar 13 20:48:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13173260 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF7F0C7618B for ; Mon, 13 Mar 2023 20:49:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230072AbjCMUsq (ORCPT ); Mon, 13 Mar 2023 16:48:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53458 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229992AbjCMUsf (ORCPT ); Mon, 13 Mar 2023 16:48:35 -0400 Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7965B6187; Mon, 13 Mar 2023 13:48:31 -0700 (PDT) Received: by mail-pj1-x1033.google.com with SMTP id y2so13279204pjg.3; Mon, 13 Mar 2023 13:48:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678740511; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=JMkL+RuV2gz/hzkO+iEW9HUlD0V+BM8BplklCfPzrkE=; b=TrDJUwoqnKQ/G6hrEF9tBK6uK6Tt8itetLQoIX1DE61g0mlVnfnAfJv/wzZ3vVxClO u4Y/mYRnrYHipfL5j3C4tCWfMwR5slRbu0Cu5Mzl4uMbRICLMSho1vor6UQeXVo4RgEM 7GTkSFyG0FHYaZ7kO4zzBDROeJwi1g2+bt4/Yx8/xG4kCZbUxeG/t2MaZMuyCeyREnoW m8VFroBuCZVS3HYr/2HxBd5qka5SZnxcpyeMEs3Ee5JAweEb7oW1wbCi4OSb07sXrX+U qz3tJQYV91qFkOZIV81kHgNuAP0F1XcggZcDk8PsfBK3LmFTtjaYwaVpsVYL2RqXqcnq zL1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678740511; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=JMkL+RuV2gz/hzkO+iEW9HUlD0V+BM8BplklCfPzrkE=; b=bXr6GXM22MdomqzTuvVdpMY3weDG8xvXrAVEFqPmeZ/DoyWnCLb89/gzhfF3DHEyQ/ ABVGTFRDFNgLS2A4z1tsXdJjMY0Cu/uP0GH6EHSwZEGtdEdYl3oy/wBPRhtTKX/Tsqt3 Qo2W2d3mB/+e8f9VXcH1r8UYqXKThubxogOfudPPFlxZFoXmIQ1+SuT6qpJoSty/LIDC Hl4Jx1zw1lVxZv4aaGwWLotjVxVh5OEkR5/INHyhKXTAOgE9sHGzsJ1zGuFl7p1FelBv mftdGE7QBmhBOpCxp+2eqOeIEYWsuCiE+5QZOEHyTpj6rAZA/4EdNHrctQeDbGW6f9kd q85w== X-Gm-Message-State: AO0yUKVWdpVL5Qf8WT1a1J9JTj2tDPhowx4kgP4AI9nrg0qD4guxYkoX LRULImRrbgtokG4EFbx+3Ps= X-Google-Smtp-Source: AK7set/ksRuDZOws2N7QlY5ec1AmU0FdbxEc5J5WSEFPJ+yfMpOuGnjCn8xJG9MnXMNhCSB2lB1nPA== X-Received: by 2002:a17:90b:1b08:b0:23b:308e:b0ab with SMTP id nu8-20020a17090b1b0800b0023b308eb0abmr9028191pjb.34.1678740510896; Mon, 13 Mar 2023 13:48:30 -0700 (PDT) Received: from moohyul.svl.corp.google.com ([2620:15c:2d4:203:26de:6cd7:2c4f:96d5]) by smtp.gmail.com with ESMTPSA id s12-20020a17090aba0c00b0023af8a3cf6esm265026pjr.48.2023.03.13.13.48.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Mar 2023 13:48:30 -0700 (PDT) Sender: Namhyung Kim From: Namhyung Kim To: Arnaldo Carvalho de Melo , Jiri Olsa Cc: Peter Zijlstra , Ingo Molnar , Song Liu , Hao Luo , Juri Lelli , Will Deacon , Waiman Long , Boqun Feng , Stephane Eranian , LKML , linux-perf-users@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 2/4] perf lock contention: Track and show siglock with address Date: Mon, 13 Mar 2023 13:48:23 -0700 Message-Id: <20230313204825.2665483-3-namhyung@kernel.org> X-Mailer: git-send-email 2.40.0.rc1.284.g88254d51c5-goog In-Reply-To: <20230313204825.2665483-1-namhyung@kernel.org> References: <20230313204825.2665483-1-namhyung@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Likewise, we can display siglock by following the pointer like current->sighand->siglock. $ sudo ./perf lock con -abl -- sleep 1 contended total wait max wait avg wait address symbol 16 2.18 ms 305.35 us 136.34 us ffffffff92e06080 tasklist_lock 28 521.78 us 31.16 us 18.63 us ffff8cc703783ec4 7 119.03 us 23.55 us 17.00 us ffff8ccb92479440 15 88.29 us 10.06 us 5.89 us ffff8cd560b5f380 siglock 7 37.67 us 9.16 us 5.38 us ffff8d053daf0c80 5 8.81 us 4.92 us 1.76 us ffff8d053d6b0c80 Signed-off-by: Namhyung Kim --- tools/perf/builtin-lock.c | 3 +-- tools/perf/util/bpf_lock_contention.c | 8 ++++++-- tools/perf/util/bpf_skel/lock_contention.bpf.c | 5 +++++ tools/perf/util/bpf_skel/lock_data.h | 3 ++- 4 files changed, 14 insertions(+), 5 deletions(-) diff --git a/tools/perf/builtin-lock.c b/tools/perf/builtin-lock.c index c62f4d9363a6..c710a5d46638 100644 --- a/tools/perf/builtin-lock.c +++ b/tools/perf/builtin-lock.c @@ -1662,8 +1662,7 @@ static void print_contention_result(struct lock_contention *con) pid, pid == -1 ? "Unknown" : thread__comm_str(t)); break; case LOCK_AGGR_ADDR: - pr_info(" %016llx %s\n", (unsigned long long)st->addr, - (st->flags & LCD_F_MMAP_LOCK) ? "mmap_lock" : st->name); + pr_info(" %016llx %s\n", (unsigned long long)st->addr, st->name); break; default: break; diff --git a/tools/perf/util/bpf_lock_contention.c b/tools/perf/util/bpf_lock_contention.c index fadcacb9d501..51631af3b4d6 100644 --- a/tools/perf/util/bpf_lock_contention.c +++ b/tools/perf/util/bpf_lock_contention.c @@ -169,7 +169,7 @@ int lock_contention_stop(void) static const char *lock_contention_get_name(struct lock_contention *con, struct contention_key *key, - u64 *stack_trace) + u64 *stack_trace, u32 flags) { int idx = 0; u64 addr; @@ -198,6 +198,10 @@ static const char *lock_contention_get_name(struct lock_contention *con, } if (con->aggr_mode == LOCK_AGGR_ADDR) { + if (flags & LCD_F_MMAP_LOCK) + return "mmap_lock"; + if (flags & LCD_F_SIGHAND_LOCK) + return "siglock"; sym = machine__find_kernel_symbol(machine, key->lock_addr, &kmap); if (sym) name = sym->name; @@ -301,7 +305,7 @@ int lock_contention_read(struct lock_contention *con) goto next; } - name = lock_contention_get_name(con, &key, stack_trace); + name = lock_contention_get_name(con, &key, stack_trace, data.flags); st = lock_stat_findnew(ls_key, name, data.flags); if (st == NULL) break; diff --git a/tools/perf/util/bpf_skel/lock_contention.bpf.c b/tools/perf/util/bpf_skel/lock_contention.bpf.c index f092a78ae2b5..4ba34caf84eb 100644 --- a/tools/perf/util/bpf_skel/lock_contention.bpf.c +++ b/tools/perf/util/bpf_skel/lock_contention.bpf.c @@ -236,6 +236,11 @@ static inline __u32 check_lock_type(__u64 lock, __u32 flags) return LCD_F_MMAP_LOCK; } break; + case LCB_F_SPIN: /* spinlock */ + curr = bpf_get_current_task_btf(); + if (&curr->sighand->siglock == (void *)lock) + return LCD_F_SIGHAND_LOCK; + break; default: break; } diff --git a/tools/perf/util/bpf_skel/lock_data.h b/tools/perf/util/bpf_skel/lock_data.h index 789f20833798..5ed1a0955015 100644 --- a/tools/perf/util/bpf_skel/lock_data.h +++ b/tools/perf/util/bpf_skel/lock_data.h @@ -19,7 +19,8 @@ struct contention_task_data { * Upper bits of the flags in the contention_data are used to identify * some well-known locks which do not have symbols (non-global locks). */ -#define LCD_F_MMAP_LOCK (1U << 31) +#define LCD_F_MMAP_LOCK (1U << 31) +#define LCD_F_SIGHAND_LOCK (1U << 30) struct contention_data { u64 total_time; From patchwork Mon Mar 13 20:48:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13173258 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4977FC6FD1C for ; Mon, 13 Mar 2023 20:48:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229551AbjCMUsf (ORCPT ); Mon, 13 Mar 2023 16:48:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229593AbjCMUse (ORCPT ); Mon, 13 Mar 2023 16:48:34 -0400 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABDB5CA1D; Mon, 13 Mar 2023 13:48:32 -0700 (PDT) Received: by mail-pj1-x102f.google.com with SMTP id nn12so13274042pjb.5; Mon, 13 Mar 2023 13:48:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678740512; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=OGQXTm4YA7W1aKAqyz9fcRI2/M5CKVX/Au3o+59MBZU=; b=EYNTUanzwMq5jtd+howQwwD8q32ETnE986COzYzDye4kpCtH3kmyngoU5sOScQUnHc C9oGUoY2YESCNEGMSCyJ0fuywNpJVn+hx+i19msZuio543r2ALqKmMVn9Oa/jLO7fFx9 OBQOJstde6bEM3igY6EjVSjxhpipjPWOCSmP1ebivcApdSXOR364OdrFoUdDuGauS417 A9JuABChw34rl+5sg1hYz1HbpsJ6PX/A6TGWPhB0tjrWrPN3zGHdEvXNCF/Ya3Yd5G4F QaB4GoWRS7GuWVqiGfIVyqvj64fJ9WQNC8qgvM56SOQ/Jnt3srEPZW6iIMvRG8CZjT7/ hCgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678740512; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=OGQXTm4YA7W1aKAqyz9fcRI2/M5CKVX/Au3o+59MBZU=; b=18LJep9h87mN0z/hGQr/F81feNWtWl4vb1wXaTvQ9RSJW5nhxz+01oY2Ix3mZaVK5w vxHeq8dAwn4iJfhk/qCwHfRN95U7smRLPDrGB2dghtOyC3V9p4lpeVthgTVIdv08Bsgz E/9Zn6982o9HNuJRJwYee4YbD1NPrbC62V7Ib9syf3nmUrQrhBaqNoMr48GCz5EQv0gZ 8tRNpjigSyH58We7UmwB4yiJzrRFbUEszRiuavozuvHo9+cWoGt4aAOkO4rP1U6BgHO5 nyfnDtOmZ3DtRnEuvyVIwpA/J4LVKTyV9UebxU1bJtXjPrf36O2f5ISZBucAzEOBwybw VtHQ== X-Gm-Message-State: AO0yUKWcsXQJYQo3te+nF3PJHoWnCVBUqVU+vrzZJ9hGAR/kwYbXn6/Z eC7ljpQvvUf/Kd1NlhrPNdI= X-Google-Smtp-Source: AK7set8Jp2lFGsXpM2X15mHoW8ZLK/qkAhkJg8TvjWpRqQiXKWg3sEi5CBatLGSrJw0eyTCcJ52mTA== X-Received: by 2002:a17:90a:1a0b:b0:234:190d:e636 with SMTP id 11-20020a17090a1a0b00b00234190de636mr37597100pjk.8.1678740512140; Mon, 13 Mar 2023 13:48:32 -0700 (PDT) Received: from moohyul.svl.corp.google.com ([2620:15c:2d4:203:26de:6cd7:2c4f:96d5]) by smtp.gmail.com with ESMTPSA id s12-20020a17090aba0c00b0023af8a3cf6esm265026pjr.48.2023.03.13.13.48.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Mar 2023 13:48:31 -0700 (PDT) Sender: Namhyung Kim From: Namhyung Kim To: Arnaldo Carvalho de Melo , Jiri Olsa Cc: Peter Zijlstra , Ingo Molnar , Song Liu , Hao Luo , Juri Lelli , Will Deacon , Waiman Long , Boqun Feng , Stephane Eranian , LKML , linux-perf-users@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 3/4] perf lock contention: Show per-cpu rq_lock with address Date: Mon, 13 Mar 2023 13:48:24 -0700 Message-Id: <20230313204825.2665483-4-namhyung@kernel.org> X-Mailer: git-send-email 2.40.0.rc1.284.g88254d51c5-goog In-Reply-To: <20230313204825.2665483-1-namhyung@kernel.org> References: <20230313204825.2665483-1-namhyung@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Using the BPF_PROG_RUN mechanism, we can run a raw_tp BPF program to collect some semi-global locks like per-cpu locks. Let's add runqueue locks using bpf_per_cpu_ptr() helper. $ sudo ./perf lock con -abl -- sleep 1 contended total wait max wait avg wait address symbol 248 3.25 ms 32.23 us 13.10 us ffff8cc75cfd2940 siglock 60 217.91 us 9.69 us 3.63 us ffff8cc700061c00 8 70.23 us 13.86 us 8.78 us ffff8cc703629484 4 56.32 us 35.81 us 14.08 us ffff8cc78b66f778 mmap_lock 4 16.70 us 5.18 us 4.18 us ffff8cc7036a0684 3 4.99 us 2.65 us 1.66 us ffff8d053da30c80 rq_lock 2 3.44 us 2.28 us 1.72 us ffff8d053dcf0c80 rq_lock 9 2.51 us 371 ns 278 ns ffff8ccb92479440 2 2.11 us 1.24 us 1.06 us ffff8d053db30c80 rq_lock 2 2.06 us 1.69 us 1.03 us ffff8d053d970c80 rq_lock Signed-off-by: Namhyung Kim --- tools/perf/util/bpf_lock_contention.c | 27 ++++++++++++++-- .../perf/util/bpf_skel/lock_contention.bpf.c | 31 +++++++++++++++++++ tools/perf/util/bpf_skel/lock_data.h | 5 +++ 3 files changed, 61 insertions(+), 2 deletions(-) diff --git a/tools/perf/util/bpf_lock_contention.c b/tools/perf/util/bpf_lock_contention.c index 51631af3b4d6..235fc7150545 100644 --- a/tools/perf/util/bpf_lock_contention.c +++ b/tools/perf/util/bpf_lock_contention.c @@ -151,6 +151,8 @@ int lock_contention_prepare(struct lock_contention *con) skel->bss->needs_callstack = con->save_callstack; skel->bss->lock_owner = con->owner; + bpf_program__set_autoload(skel->progs.collect_lock_syms, false); + lock_contention_bpf__attach(skel); return 0; } @@ -198,14 +200,26 @@ static const char *lock_contention_get_name(struct lock_contention *con, } if (con->aggr_mode == LOCK_AGGR_ADDR) { + int lock_fd = bpf_map__fd(skel->maps.lock_syms); + + /* per-process locks set upper bits of the flags */ if (flags & LCD_F_MMAP_LOCK) return "mmap_lock"; if (flags & LCD_F_SIGHAND_LOCK) return "siglock"; + + /* global locks with symbols */ sym = machine__find_kernel_symbol(machine, key->lock_addr, &kmap); if (sym) - name = sym->name; - return name; + return sym->name; + + /* try semi-global locks collected separately */ + if (!bpf_map_lookup_elem(lock_fd, &key->lock_addr, &flags)) { + if (flags == LOCK_CLASS_RQLOCK) + return "rq_lock"; + } + + return ""; } /* LOCK_AGGR_CALLER: skip lock internal functions */ @@ -258,6 +272,15 @@ int lock_contention_read(struct lock_contention *con) thread__set_comm(idle, "swapper", /*timestamp=*/0); } + if (con->aggr_mode == LOCK_AGGR_ADDR) { + DECLARE_LIBBPF_OPTS(bpf_test_run_opts, opts, + .flags = BPF_F_TEST_RUN_ON_CPU, + ); + int prog_fd = bpf_program__fd(skel->progs.collect_lock_syms); + + bpf_prog_test_run_opts(prog_fd, &opts); + } + /* make sure it loads the kernel map */ map__load(maps__first(machine->kmaps)); diff --git a/tools/perf/util/bpf_skel/lock_contention.bpf.c b/tools/perf/util/bpf_skel/lock_contention.bpf.c index 4ba34caf84eb..2d50c4395733 100644 --- a/tools/perf/util/bpf_skel/lock_contention.bpf.c +++ b/tools/perf/util/bpf_skel/lock_contention.bpf.c @@ -10,6 +10,9 @@ /* default buffer size */ #define MAX_ENTRIES 10240 +/* for collect_lock_syms(). 4096 was rejected by the verifier */ +#define MAX_CPUS 1024 + /* lock contention flags from include/trace/events/lock.h */ #define LCB_F_SPIN (1U << 0) #define LCB_F_READ (1U << 1) @@ -56,6 +59,13 @@ struct { __uint(max_entries, MAX_ENTRIES); } task_data SEC(".maps"); +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(key_size, sizeof(__u64)); + __uint(value_size, sizeof(__u32)); + __uint(max_entries, 16384); +} lock_syms SEC(".maps"); + struct { __uint(type, BPF_MAP_TYPE_HASH); __uint(key_size, sizeof(__u32)); @@ -378,4 +388,25 @@ int contention_end(u64 *ctx) return 0; } +extern struct rq runqueues __ksym; + +SEC("raw_tp/bpf_test_finish") +int BPF_PROG(collect_lock_syms) +{ + __u64 lock_addr; + __u32 lock_flag; + + for (int i = 0; i < MAX_CPUS; i++) { + struct rq *rq = bpf_per_cpu_ptr(&runqueues, i); + + if (rq == NULL) + break; + + lock_addr = (__u64)&rq->__lock; + lock_flag = LOCK_CLASS_RQLOCK; + bpf_map_update_elem(&lock_syms, &lock_addr, &lock_flag, BPF_ANY); + } + return 0; +} + char LICENSE[] SEC("license") = "Dual BSD/GPL"; diff --git a/tools/perf/util/bpf_skel/lock_data.h b/tools/perf/util/bpf_skel/lock_data.h index 5ed1a0955015..e59366f2dba3 100644 --- a/tools/perf/util/bpf_skel/lock_data.h +++ b/tools/perf/util/bpf_skel/lock_data.h @@ -36,4 +36,9 @@ enum lock_aggr_mode { LOCK_AGGR_CALLER, }; +enum lock_class_sym { + LOCK_CLASS_NONE, + LOCK_CLASS_RQLOCK, +}; + #endif /* UTIL_BPF_SKEL_LOCK_DATA_H */ From patchwork Mon Mar 13 20:48:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13173259 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B375C74A4B for ; Mon, 13 Mar 2023 20:48:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230043AbjCMUsm (ORCPT ); Mon, 13 Mar 2023 16:48:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229814AbjCMUsf (ORCPT ); Mon, 13 Mar 2023 16:48:35 -0400 Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 070B216AC6; Mon, 13 Mar 2023 13:48:34 -0700 (PDT) Received: by mail-pl1-x62c.google.com with SMTP id i5so14389698pla.2; Mon, 13 Mar 2023 13:48:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678740513; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=RciDXnCG/KtX4Q4XcZ5CVs3Bi1XerIjeyGzrjFnWXYw=; b=Jvb5hDuwEF3uh8FaYe3r6f5k6p3w1TgfnX8JQkY1WSKdGPWVX1x258Y7IxdX/tPnVJ ej/XpP52Ra9JpL3PF2IlBQFexFLavSvD3Q7LBvmk9Mlr13uS6hZzvuXqXBbAR8MRZZ45 BSlt6+Ph4p76rSxn3PzeKbzy7YZZ8VlKvq4omqujzgOGTZ1r8D8uMMPfm8/iaGlJdjr6 pKIR/UcOLTi9+Z2HyZLHqDYLSaKoADacAhD4yyWcysd4O0IIIkNvCi6SHPFTVk1ejpNL kfU3JibXU58r5JkSD3CoQp5TGV+hKai7TdEDQBUiQ2C3yDTSfifUXz5MYisYYdNBGYNA L5jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678740513; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=RciDXnCG/KtX4Q4XcZ5CVs3Bi1XerIjeyGzrjFnWXYw=; b=MCVIVLL/2ruv2mwdE+LB5BVamV0bSp9+4zl6GMVgp3enBkxlFPaV6mQT6yXShvhfyu +KBB+cOj5yLz2+fyx34f7UZGcV/ntT56nZ5CnjjjrtxfPvbuRJ1ZDJvACjm9gsNg3jte qAzRBA+2p1DrkLEczi17Ibo+j8dumg3dHccXSbUO9e4naPAk3hgrPOhmiiKF8fdz/nPF SJvcpQubdrRp7SfTe+I/tHIX+Bik+451jXO7CznSFkhgNMg6Q4hgMwzitq5iE8kHLZiG rRf3lVkNhJXSqlLw+0K6Sx7Azis0fkbZiert6FWnzbkNzFEClWuFx8hWRFRATBBFbvRi xHug== X-Gm-Message-State: AO0yUKWu/BCy4InfXbglGiamMkb2inuQtLp5/FSW7xbjOEI7vD1pg3SA tSlQNn0uSG64EiU4VFvBtE25CGgOEJk= X-Google-Smtp-Source: AK7set/IfmOLXf/3U5fu+rEDzar/pRMlA/1BLru0aWlZ07K66QKIFddPFsDRbgbAznYNaadOmdxVXQ== X-Received: by 2002:a17:90b:3810:b0:237:62f7:3106 with SMTP id mq16-20020a17090b381000b0023762f73106mr37551968pjb.17.1678740513428; Mon, 13 Mar 2023 13:48:33 -0700 (PDT) Received: from moohyul.svl.corp.google.com ([2620:15c:2d4:203:26de:6cd7:2c4f:96d5]) by smtp.gmail.com with ESMTPSA id s12-20020a17090aba0c00b0023af8a3cf6esm265026pjr.48.2023.03.13.13.48.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Mar 2023 13:48:33 -0700 (PDT) Sender: Namhyung Kim From: Namhyung Kim To: Arnaldo Carvalho de Melo , Jiri Olsa Cc: Peter Zijlstra , Ingo Molnar , Song Liu , Hao Luo , Juri Lelli , Will Deacon , Waiman Long , Boqun Feng , Stephane Eranian , LKML , linux-perf-users@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 4/4] perf lock contention: Show lock type with address Date: Mon, 13 Mar 2023 13:48:25 -0700 Message-Id: <20230313204825.2665483-5-namhyung@kernel.org> X-Mailer: git-send-email 2.40.0.rc1.284.g88254d51c5-goog In-Reply-To: <20230313204825.2665483-1-namhyung@kernel.org> References: <20230313204825.2665483-1-namhyung@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Show lock type names after the symbol of locks if any. This can be useful especially when it doesn't show the lock symbols. The indentation before the lock type parenthesis is to recognize lock symbols more easily. $ sudo ./perf lock con -abl -- sleep 1 contended total wait max wait avg wait address symbol 44 6.13 ms 284.49 us 139.28 us ffffffff92e06080 tasklist_lock (rwlock) 159 983.38 us 12.38 us 6.18 us ffff8cc717c90000 siglock (spinlock) 10 679.90 us 153.35 us 67.99 us ffff8cdc2872aaf8 mmap_lock (rwsem) 9 558.11 us 180.67 us 62.01 us ffff8cd647914038 mmap_lock (rwsem) 78 228.56 us 7.82 us 2.93 us ffff8cc700061c00 (spinlock) 5 41.60 us 16.93 us 8.32 us ffffd853acb41468 (spinlock) 10 37.24 us 5.87 us 3.72 us ffff8cd560b5c200 siglock (spinlock) 4 11.17 us 3.97 us 2.79 us ffff8d053ddf0c80 rq_lock (spinlock) 1 7.86 us 7.86 us 7.86 us ffff8cd64791404c (spinlock) 1 4.13 us 4.13 us 4.13 us ffff8d053d930c80 rq_lock (spinlock) 7 3.98 us 1.67 us 568 ns ffff8ccb92479440 (mutex) 2 2.62 us 2.33 us 1.31 us ffff8cc702e6ede0 (rwlock) Signed-off-by: Namhyung Kim --- tools/perf/builtin-lock.c | 45 ++++++++++++++++++---------- tools/perf/util/bpf_skel/lock_data.h | 2 ++ 2 files changed, 32 insertions(+), 15 deletions(-) diff --git a/tools/perf/builtin-lock.c b/tools/perf/builtin-lock.c index c710a5d46638..a845c0ce5dc8 100644 --- a/tools/perf/builtin-lock.c +++ b/tools/perf/builtin-lock.c @@ -1548,27 +1548,41 @@ static void sort_result(void) static const struct { unsigned int flags; + const char *str; const char *name; } lock_type_table[] = { - { 0, "semaphore" }, - { LCB_F_SPIN, "spinlock" }, - { LCB_F_SPIN | LCB_F_READ, "rwlock:R" }, - { LCB_F_SPIN | LCB_F_WRITE, "rwlock:W"}, - { LCB_F_READ, "rwsem:R" }, - { LCB_F_WRITE, "rwsem:W" }, - { LCB_F_RT, "rtmutex" }, - { LCB_F_RT | LCB_F_READ, "rwlock-rt:R" }, - { LCB_F_RT | LCB_F_WRITE, "rwlock-rt:W"}, - { LCB_F_PERCPU | LCB_F_READ, "pcpu-sem:R" }, - { LCB_F_PERCPU | LCB_F_WRITE, "pcpu-sem:W" }, - { LCB_F_MUTEX, "mutex" }, - { LCB_F_MUTEX | LCB_F_SPIN, "mutex" }, + { 0, "semaphore", "semaphore" }, + { LCB_F_SPIN, "spinlock", "spinlock" }, + { LCB_F_SPIN | LCB_F_READ, "rwlock:R", "rwlock" }, + { LCB_F_SPIN | LCB_F_WRITE, "rwlock:W", "rwlock" }, + { LCB_F_READ, "rwsem:R", "rwsem" }, + { LCB_F_WRITE, "rwsem:W", "rwsem" }, + { LCB_F_RT, "rt=mutex", "rt-mutex" }, + { LCB_F_RT | LCB_F_READ, "rwlock-rt:R", "rwlock-rt" }, + { LCB_F_RT | LCB_F_WRITE, "rwlock-rt:W", "rwlock-rt" }, + { LCB_F_PERCPU | LCB_F_READ, "pcpu-sem:R", "percpu-rwsem" }, + { LCB_F_PERCPU | LCB_F_WRITE, "pcpu-sem:W", "percpu-rwsem" }, + { LCB_F_MUTEX, "mutex", "mutex" }, + { LCB_F_MUTEX | LCB_F_SPIN, "mutex", "mutex" }, /* alias for get_type_flag() */ - { LCB_F_MUTEX | LCB_F_SPIN, "mutex-spin" }, + { LCB_F_MUTEX | LCB_F_SPIN, "mutex-spin", "mutex" }, }; static const char *get_type_str(unsigned int flags) { + flags &= LCB_F_MAX_FLAGS - 1; + + for (unsigned int i = 0; i < ARRAY_SIZE(lock_type_table); i++) { + if (lock_type_table[i].flags == flags) + return lock_type_table[i].str; + } + return "unknown"; +} + +static const char *get_type_name(unsigned int flags) +{ + flags &= LCB_F_MAX_FLAGS - 1; + for (unsigned int i = 0; i < ARRAY_SIZE(lock_type_table); i++) { if (lock_type_table[i].flags == flags) return lock_type_table[i].name; @@ -1662,7 +1676,8 @@ static void print_contention_result(struct lock_contention *con) pid, pid == -1 ? "Unknown" : thread__comm_str(t)); break; case LOCK_AGGR_ADDR: - pr_info(" %016llx %s\n", (unsigned long long)st->addr, st->name); + pr_info(" %016llx %s (%s)\n", (unsigned long long)st->addr, + st->name, get_type_name(st->flags)); break; default: break; diff --git a/tools/perf/util/bpf_skel/lock_data.h b/tools/perf/util/bpf_skel/lock_data.h index e59366f2dba3..1ba61cb4d480 100644 --- a/tools/perf/util/bpf_skel/lock_data.h +++ b/tools/perf/util/bpf_skel/lock_data.h @@ -22,6 +22,8 @@ struct contention_task_data { #define LCD_F_MMAP_LOCK (1U << 31) #define LCD_F_SIGHAND_LOCK (1U << 30) +#define LCB_F_MAX_FLAGS (1U << 7) + struct contention_data { u64 total_time; u64 min_time;