| Message ID | cover.1558081981.git.han_mao@c-sky.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | riscv: Add perf callchain support | expand |
PING? On Fri, May 17, 2019 at 04:43:01PM +0800, Mao Han wrote: > This patch set add perf callchain(FP/DWARF) support for RISC-V. > It comes from the csky version callchain support with some > slight modifications. The patchset base on Linux 5.1. > > CC: Palmer Dabbelt <palmer@sifive.com> > CC: linux-riscv <linux-riscv@lists.infradead.org> > CC: Christoph Hellwig <hch@lst.de> > CC: Guo Ren <guoren@kernel.org> > > Changes since v2: > - fix inconsistent comment > - force to build kernel with -fno-omit-frame-pointer if perf > event is enabled > > Changes since v1: > - simplify implementation and code convention > > Mao Han (3): > riscv: Add perf callchain support > riscv: Add support for perf registers sampling > riscv: Add support for libdw > > arch/riscv/Kconfig | 2 + > arch/riscv/Makefile | 3 + > arch/riscv/include/uapi/asm/perf_regs.h | 42 ++++++++++ > arch/riscv/kernel/Makefile | 4 +- > arch/riscv/kernel/perf_callchain.c | 113 ++++++++++++++++++++++++++ > arch/riscv/kernel/perf_regs.c | 44 ++++++++++ > tools/arch/riscv/include/uapi/asm/perf_regs.h | 42 ++++++++++ > tools/perf/Makefile.config | 6 +- > tools/perf/arch/riscv/Build | 1 + > tools/perf/arch/riscv/Makefile | 3 + > tools/perf/arch/riscv/include/perf_regs.h | 96 ++++++++++++++++++++++ > tools/perf/arch/riscv/util/Build | 2 + > tools/perf/arch/riscv/util/dwarf-regs.c | 72 ++++++++++++++++ > tools/perf/arch/riscv/util/unwind-libdw.c | 57 +++++++++++++ > 14 files changed, 485 insertions(+), 2 deletions(-) > create mode 100644 arch/riscv/include/uapi/asm/perf_regs.h > create mode 100644 arch/riscv/kernel/perf_callchain.c > create mode 100644 arch/riscv/kernel/perf_regs.c > create mode 100644 tools/arch/riscv/include/uapi/asm/perf_regs.h > create mode 100644 tools/perf/arch/riscv/Build > create mode 100644 tools/perf/arch/riscv/Makefile > create mode 100644 tools/perf/arch/riscv/include/perf_regs.h > create mode 100644 tools/perf/arch/riscv/util/Build > create mode 100644 tools/perf/arch/riscv/util/dwarf-regs.c > create mode 100644 tools/perf/arch/riscv/util/unwind-libdw.c > > -- > 2.7.4 > > > _______________________________________________ > linux-riscv mailing list > linux-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-riscv
Hello Mao Han, On Fri, 17 May 2019, Mao Han wrote: > This patch set add perf callchain(FP/DWARF) support for RISC-V. > It comes from the csky version callchain support with some > slight modifications. The patchset base on Linux 5.1. > > CC: Palmer Dabbelt <palmer@sifive.com> > CC: linux-riscv <linux-riscv@lists.infradead.org> > CC: Christoph Hellwig <hch@lst.de> > CC: Guo Ren <guoren@kernel.org> I tried these patches on v5.3-rc4, both on the HiFive Unleashed board with a Debian-based rootfs and QEMU rv64 with a Fedora-based rootfs. For QEMU, I used defconfig, and for the HiFive Unleashed, I added a few more Kconfig directives; and on both, I enabled CONFIG_PERF_EVENTS. I built the perf tools from the kernel tree. Upon running "/root/bin/perf record -e cpu-clock --call-graph fp /bin/ls", I see the backtraces below. The first is on the HiFive Unleashed, the second is on QEMU. Could you take a look and tell me if you see similar issues? And if not, could you please walk me through your process for testing these patches on rv64, so I can reproduce it here? thanks, - Paul # /root/bin/perf record -e cpu-clock --call-graph fp /bin/ls [ 213.258888] Unable to handle kernel paging request at virtual address ffffffd2d979000c [ 213.266025] Oops [#1] [ 213.268278] Modules linked in: [ 213.271323] CPU: 2 PID: 165 Comm: ls Not tainted 5.3.0-rc4-00006-g2615b94055fc #164 [ 213.278961] sepc: ffffffe0005bde76 ra : ffffffe0005bde84 sp : ffffffe1f609d870 [ 213.286166] gp : ffffffe000dbe6c0 tp : ffffffe1f60fa600 t0 : 0000000000046000 [ 213.293372] t1 : 0000000000000002 t2 : 0000002aaaac4890 s0 : ffffffe1f609d8b0 [ 213.300579] s1 : ffffffe1f609d8d8 a0 : 0000000000000004 a1 : 0000000000000000 [ 213.307785] a2 : 10ef8552d9790004 a3 : 0000000000000018 a4 : 0000000000000018 [ 213.314991] a5 : 0000000000000003 a6 : 0000000000000001 a7 : ffffffe1f7b685e0 [ 213.322198] s2 : 10ef8552d9790014 s3 : 0000003fffffffff s4 : 0000000000001ff8 [ 213.329404] s5 : b7e9595db7f9e15f s6 : ffffffe000dbe128 s7 : ffffffe1f8d6f820 [ 213.336610] s8 : 00000031a394a3e8 s9 : ffffffe000656150 s10: 0000000200000120 [ 213.343816] s11: 0000002aaaabc938 t3 : ffffffe1f7b685c0 t4 : 0000000000000001 [ 213.351021] t5 : 0000000000000008 t6 : 0000000000000000 [ 213.356319] sstatus: 0000000200000100 sbadaddr: ffffffd2d979000c scause: 000000000000000d [ 213.364482] ---[ end trace f325867b8e0786aa ]--- [ 213.369083] Kernel panic - not syncing: Fatal exception in interrupt [ 213.375421] SMP: stopping secondary CPUs [ 213.379336] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- # /root/bin/perf record -e cpu-clock --call-graph fp /bin/ls [ 3465.349655] Unable to handle kernel paging request at virtual address d9bfd43b47718688 [ 3465.352356] Oops [#1] [ 3465.353375] Modules linked in: [ 3465.354496] CPU: 0 PID: 9501 Comm: ls Not tainted 5.3.0-rc4-00004-g3bfea1b43363 #165 [ 3465.355904] sepc: ffffffe000039a0a ra : ffffffe000039a18 sp : ffffffe078c5b5e0 [ 3465.361041] gp : ffffffe000837760 tp : ffffffe078d88940 t0 : 0000000000046000 [ 3465.366104] t1 : 0000000000000002 t2 : 000000004387194a s0 : ffffffe078c5b620 [ 3465.366839] s1 : ffffffe078c5b648 a0 : 0000000000000004 a1 : 0000000000000000 [ 3465.369101] a2 : d9bfd43b47718680 a3 : 0000000000000690 a4 : 0000000000000018 [ 3465.370949] a5 : 0000000000000003 a6 : 0000000000000001 a7 : ffffffe0743b16d0 [ 3465.371650] s2 : d9bfd43b47718690 s3 : 0000003fffffffff s4 : 0000000000001ff8 [ 3465.372658] s5 : e85970cf9e9fbbc8 s6 : ffffffe0008371c8 s7 : ffffffe07a1527a0 [ 3465.373570] s8 : 00000326c7b3d418 s9 : ffffffe0000d3326 s10: 0000000000000120 [ 3465.377102] s11: ffffffe078cb4880 t3 : ffffffe0743b16c0 t4 : 0000000000000001 [ 3465.378125] t5 : 0000000000000008 t6 : ffffffe078c5bd60 [ 3465.380603] sstatus: 0000000000000100 sbadaddr: d9bfd43b47718688 scause: 000000000000000d [ 3465.382872] ---[ end trace f0ff9969d564eb82 ]--- [ 3465.383998] Kernel panic - not syncing: Fatal exception in interrupt [ 3465.385812] SMP: stopping secondary CPUs [ 3465.394408] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
Hi Paul, On Fri, Aug 16, 2019 at 10:14:01AM -0700, Paul Walmsley wrote: > Hello Mao Han, > > On Fri, 17 May 2019, Mao Han wrote: > > > This patch set add perf callchain(FP/DWARF) support for RISC-V. > > It comes from the csky version callchain support with some > > slight modifications. The patchset base on Linux 5.1. > > > > CC: Palmer Dabbelt <palmer@sifive.com> > > CC: linux-riscv <linux-riscv@lists.infradead.org> > > CC: Christoph Hellwig <hch@lst.de> > > CC: Guo Ren <guoren@kernel.org> > > I tried these patches on v5.3-rc4, both on the HiFive Unleashed board > with a Debian-based rootfs and QEMU rv64 with a Fedora-based rootfs. For > QEMU, I used defconfig, and for the HiFive Unleashed, I added a few more > Kconfig directives; and on both, I enabled CONFIG_PERF_EVENTS. I built > the perf tools from the kernel tree. > > Upon running "/root/bin/perf record -e cpu-clock --call-graph fp > /bin/ls", I see the backtraces below. The first is on the HiFive > Unleashed, the second is on QEMU. > > Could you take a look and tell me if you see similar issues? And if not, > could you please walk me through your process for testing these patches on > rv64, so I can reproduce it here? > I'v tried the command line above and got similar issues with probability. unwind_frame_kernel can not stop unwind when fp is a quite large value(like 0x70aac93ff0eff584) which can pass the simple stack check. if (kstack_end((void *)frame->fp)) return -EPERM; if (frame->fp & 0x3 || frame->fp < TASK_SIZE) return -EPERM; handle_exception from arch/riscv/kernel/entry.S will use s0(fp) as temp register. The context for this frame is unpredictable. We may add more strict check in unwind_frame_kernel or keep s0 always 0 in handle_exception to fix this issue. Breakpoint 1, unwind_frame_kernel (frame=0xffffffe0000057a0 <rcu_init+118>) at arch/riscv/kernel/perf_callchain.c:14 14 { 1: /x *frame = {fp = 0xffffffe000005ee0, ra = 0xffffffe000124c90} (gdb) Continuing. Breakpoint 1, unwind_frame_kernel (frame=0xffffffe0000057a0 <rcu_init+118>) at arch/riscv/kernel/perf_callchain.c:14 14 { 1: /x *frame = {fp = 0xffffffe000124c74, ra = 0xffffffe000036cba} (gdb) Continuing. Breakpoint 1, unwind_frame_kernel (frame=0xffffffe0000057a0 <rcu_init+118>) at arch/riscv/kernel/perf_callchain.c:14 14 { 1: /x *frame = {fp = 0x70aac57ff0eff584, ra = 0x8082614d64ea740a} (gdb) Some time perf record can output perf.data correct. something like: # perf record -e cpu-clock --call-graph fp ls perf.data [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.012 MB perf.data (102 samples) ] # # perf report # To display the perf.data header info, please use --header/--header-only options. # # # Total Lost Samples: 0 # # Samples: 102 of event 'cpu-clock' # Event count (approx.): 25500000 # # Children Self Command Shared Object Symbol # ........ ........ ....... ................. ................................. # 30.39% 0.00% ls [kernel.kallsyms] [k] ret_from_exception | ---ret_from_exception | |--16.67%--do_page_fault | handle_mm_fault | __handle_mm_fault | | | |--9.80%--filemap_map_pages | | | | | |--3.92%--alloc_set_pte | | | page_add_file_rmap | | | | | --3.92%--filemap_map_pages | | | --0.98%--__do_fault | |--8.82%--schedule | __sched_text_start | finish_task_switch | --4.90%--__kprobes_text_start irq_exit irq_exit 28.43% 0.00% ls [kernel.kallsyms] [k] ret_from_syscall | ---ret_from_syscall | |--15.69%--__se_sys_execve | __do_execve_file | search_binary_handler.part.7 I previous tested these patch with a program(callchain_test.c) mostly running in userspace, so I didn't got this issues. Thanks, Mao Han
On Mon, Aug 19, 2019 at 04:18:01PM +0800, Mao Han wrote: > Hi Paul, > On Fri, Aug 16, 2019 at 10:14:01AM -0700, Paul Walmsley wrote: > > Hello Mao Han, > > > > On Fri, 17 May 2019, Mao Han wrote: > > > > > This patch set add perf callchain(FP/DWARF) support for RISC-V. > > > It comes from the csky version callchain support with some > > > slight modifications. The patchset base on Linux 5.1. > > > > > > CC: Palmer Dabbelt <palmer@sifive.com> > > > CC: linux-riscv <linux-riscv@lists.infradead.org> > > > CC: Christoph Hellwig <hch@lst.de> > > > CC: Guo Ren <guoren@kernel.org> > > > > I tried these patches on v5.3-rc4, both on the HiFive Unleashed board > > with a Debian-based rootfs and QEMU rv64 with a Fedora-based rootfs. For > > QEMU, I used defconfig, and for the HiFive Unleashed, I added a few more > > Kconfig directives; and on both, I enabled CONFIG_PERF_EVENTS. I built > > the perf tools from the kernel tree. > > > > Upon running "/root/bin/perf record -e cpu-clock --call-graph fp > > /bin/ls", I see the backtraces below. The first is on the HiFive > > Unleashed, the second is on QEMU. > > > > Could you take a look and tell me if you see similar issues? And if not, > > could you please walk me through your process for testing these patches on > > rv64, so I can reproduce it here? > > > > I'v tried the command line above and got similar issues with probability. > unwind_frame_kernel can not stop unwind when fp is a quite large > value(like 0x70aac93ff0eff584) which can pass the simple stack check. > if (kstack_end((void *)frame->fp)) > return -EPERM; > if (frame->fp & 0x3 || frame->fp < TASK_SIZE) > return -EPERM; > handle_exception from arch/riscv/kernel/entry.S will use s0(fp) as temp > register. The context for this frame is unpredictable. We may add more > strict check in unwind_frame_kernel or keep s0 always 0 in handle_exception > to fix this issue. > perf record -e cpu-clock --call-graph fp /bin/ls seems can work stably with this change applied. diff --git a/arch/riscv/kernel/perf_callchain.c b/arch/riscv/kernel/perf_callchain.c index 8b57903..dd27c67 100644 --- a/arch/riscv/kernel/perf_callchain.c +++ b/arch/riscv/kernel/perf_callchain.c @@ -16,6 +16,8 @@ static int unwind_frame_kernel(struct stackframe *frame) return -EPERM; if (frame->fp & 0x3 || frame->fp < TASK_SIZE) return -EPERM; + if (frame->fp < CONFIG_PAGE_OFFSET) + return -EPERM; *frame = *((struct stackframe *)frame->fp - 1); if (__kernel_text_address(frame->ra)) { PS: I got some compile error while compiling glibc 2.30 with linux v5.3-rc4 header. vfork.S include linux/sched.h(./include/uapi/linux/sched.h) which has a struct clone_args inside, added by 7f192e3cd316ba58c88dfa26796cf77789dd9872.
On Mon, 19 Aug 2019, Mao Han wrote: > PS: I got some compile error while compiling glibc 2.30 with linux > v5.3-rc4 header. vfork.S include linux/sched.h(./include/uapi/linux/sched.h) > which has a struct clone_args inside, added by > 7f192e3cd316ba58c88dfa26796cf77789dd9872. Noticed that also. Probably the sched.h uapi kernel header file needs an "#ifndef __ASSEMBLY__" around the struct clone_args... - Paul
This patch set add perf callchain(FP/DWARF) support for RISC-V. It comes from the csky version callchain support with some slight modifications. The patchset base on Linux 5.1. CC: Palmer Dabbelt <palmer@sifive.com> CC: linux-riscv <linux-riscv@lists.infradead.org> CC: Christoph Hellwig <hch@lst.de> CC: Guo Ren <guoren@kernel.org> Changes since v2: - fix inconsistent comment - force to build kernel with -fno-omit-frame-pointer if perf event is enabled Changes since v1: - simplify implementation and code convention Mao Han (3): riscv: Add perf callchain support riscv: Add support for perf registers sampling riscv: Add support for libdw arch/riscv/Kconfig | 2 + arch/riscv/Makefile | 3 + arch/riscv/include/uapi/asm/perf_regs.h | 42 ++++++++++ arch/riscv/kernel/Makefile | 4 +- arch/riscv/kernel/perf_callchain.c | 113 ++++++++++++++++++++++++++ arch/riscv/kernel/perf_regs.c | 44 ++++++++++ tools/arch/riscv/include/uapi/asm/perf_regs.h | 42 ++++++++++ tools/perf/Makefile.config | 6 +- tools/perf/arch/riscv/Build | 1 + tools/perf/arch/riscv/Makefile | 3 + tools/perf/arch/riscv/include/perf_regs.h | 96 ++++++++++++++++++++++ tools/perf/arch/riscv/util/Build | 2 + tools/perf/arch/riscv/util/dwarf-regs.c | 72 ++++++++++++++++ tools/perf/arch/riscv/util/unwind-libdw.c | 57 +++++++++++++ 14 files changed, 485 insertions(+), 2 deletions(-) create mode 100644 arch/riscv/include/uapi/asm/perf_regs.h create mode 100644 arch/riscv/kernel/perf_callchain.c create mode 100644 arch/riscv/kernel/perf_regs.c create mode 100644 tools/arch/riscv/include/uapi/asm/perf_regs.h create mode 100644 tools/perf/arch/riscv/Build create mode 100644 tools/perf/arch/riscv/Makefile create mode 100644 tools/perf/arch/riscv/include/perf_regs.h create mode 100644 tools/perf/arch/riscv/util/Build create mode 100644 tools/perf/arch/riscv/util/dwarf-regs.c create mode 100644 tools/perf/arch/riscv/util/unwind-libdw.c