From patchwork Fri Mar 31 12:47:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: pengdonglin X-Patchwork-Id: 13195978 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8FA2EC77B6E for ; Fri, 31 Mar 2023 12:49:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=WcbvommseEahoJuBnS4c1fUo17YjlXctJvysv0MLLg0=; b=aQkTGUDM7zmCjK baz5UY9nECZ0CRXCwVMD0MUo7BbV2gzvgHkRqQ9PVhOzM0nFA+p4EepyFCRYf3ilO55KspTvXI76Z iqsdJXQ70ABGye1JGf6bU22xrrF9epP/ZGSma01aQ7afl+tig9DJgkdoVc8kiHDy1WpQGtFhYCi1Y 35so0KPHmXI5Q5Jz0djDCeB4P86YhnHctB6SkI5MoUDwSZdFLmzfEH7JZHmedsgwwE5cybgNdzuxW +cB7OeFAsI3o2CCHDHTvuzP9J7Whyq2CO22+xHziJiXQbw8Kt0yX7B+P0jUV4b2ccgnCDAPOIIujS A3nx5O/N3zIuKHmaNzVw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1piECB-007PS8-17; Fri, 31 Mar 2023 12:49:31 +0000 Received: from mail-m118111.qiye.163.com ([115.236.118.111]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1piEB9-007Ojv-2c; Fri, 31 Mar 2023 12:48:33 +0000 Received: from ubuntu.localdomain (unknown [117.133.56.22]) by mail-m118111.qiye.163.com (Hmail) with ESMTPA id B18E058035E; Fri, 31 Mar 2023 20:48:00 +0800 (CST) From: Donglin Peng To: mhiramat@kernel.org, rostedt@goodmis.org, linux@armlinux.org.uk, mark.rutland@arm.com, will@kernel.org, catalin.marinas@arm.com, rmk+kernel@armlinux.org.uk, palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, tglx@linutronix.de, dave.hansen@linux.intel.com, x86@kernel.org, bp@alien8.de, hpa@zytor.com, chenhuacai@kernel.org, zhangqing@loongson.cn, kernel@xen0n.name, mingo@redhat.com, peterz@infradead.org, xiehuan09@gmail.com, dinghui@sangfor.com.cn, huangcun@sangfor.com.cn, dolinux.peng@gmail.com Cc: linux-trace-kernel@vger.kernel.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Donglin Peng Subject: [PATCH v10 0/8] function_graph: Support recording and printing the return value of function Date: Fri, 31 Mar 2023 05:47:36 -0700 Message-Id: X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-HM-Spam-Status: e1kfGhgUHx5ZQUpXWQgPGg8OCBgUHx5ZQUlOS1dZFg8aDwILHllBWSg2Ly tZV1koWUFITzdXWS1ZQUlXWQ8JGhUIEh9ZQVlCHU9PVkJCGBpCSx0aQ09OHVUTARMWGhIXJBQOD1 lXWRgSC1lBWUpKTFVKSEhVTk1VSUlZV1kWGg8SFR0UWUFZT0tIVUpISkJIT1VKS0tVS1kG X-HM-Sender-Digest: e1kMHhlZQR0aFwgeV1kSHx4VD1lBWUc6MhQ6OTo6NT0OPU0BKzgQITIL Hz0aCwlVSlVKTUNLSU1NQ0NJT0xDVTMWGhIXVQseFRwfFBUcFxIVOwgaFRwdFAlVGBQWVRgVRVlX WRILWUFZSkpMVUpISFVOTVVJSVlXWQgBWUFCTkhONwY+ X-HM-Tid: 0a8737b5eb4f2eb7kusnb18e058035e X-HM-MType: 1 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230331_054828_151256_A167B7EA X-CRM114-Status: GOOD ( 20.24 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org When using the function_graph tracer to analyze system call failures, it can be time-consuming to analyze the trace logs and locate the kernel function that first returns an error. This change aims to simplify the process by recording the function return value to the 'retval' member of 'ftrace_graph_ent' and printing it when outputing the trace log. Note that even if a function's return type is void, a return value will still be printed, so it should be ignored. If you care about this, the BTF file can be used to obtain the details of function return type. We can implement a tool to process the trace log and display the return value based on its actual type. Here is an example: ... 1) | cgroup_attach_task() { 1) | cgroup_migrate_add_src() { 1) 1.403 us | cset_cgroup_from_root(); /* = 0xffff93fc86f58010 */ 1) 2.154 us | } /* cgroup_migrate_add_src = 0xffffb286c1297d00 */ 1) ! 386.538 us | cgroup_migrate_prepare_dst(); /* = 0x0 */ 1) | cgroup_migrate() { 1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */ 1) | cgroup_migrate_execute() { 1) | cpu_cgroup_can_attach() { 1) | cgroup_taskset_first() { 1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */ 1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */ 1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */ 1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */ 1) 4.369 us | } /* cgroup_migrate_execute = -22 */ 1) 7.143 us | } /* cgroup_migrate = -22 */ 1) | cgroup_migrate_finish() { 1) 0.411 us | put_css_set_locked(); /* = 0x8 */ 1) + 62.397 us | put_css_set_locked(); /* = 0x80000001 */ 1) + 64.742 us | } /* cgroup_migrate_finish = 0x80000000 */ 1) ! 465.605 us | } /* cgroup_attach_task = -22 */ ... After processing the above trace logs using BTF information: ... 1) | cgroup_attach_task() { 1) | cgroup_migrate_add_src() { 1) 1.403 us | cset_cgroup_from_root(); /* = 0xffff93fc86f58010 */ 1) 2.154 us | } /* cgroup_migrate_add_src */ 1) ! 386.538 us | cgroup_migrate_prepare_dst(); /* = 0 */ 1) | cgroup_migrate() { 1) 0.651 us | cgroup_migrate_add_task(); 1) | cgroup_migrate_execute() { 1) | cpu_cgroup_can_attach() { 1) | cgroup_taskset_first() { 1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */ 1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */ 1) 0.380 us | sched_rt_can_attach(); /* = 0 */ 1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */ 1) 4.369 us | } /* cgroup_migrate_execute = -22 */ 1) 7.143 us | } /* cgroup_migrate = -22 */ 1) | cgroup_migrate_finish() { 1) 0.411 us | put_css_set_locked(); 1) + 62.397 us | put_css_set_locked(); 1) + 64.742 us | } /* cgroup_migrate_finish */ 1) ! 465.605 us | } /* cgroup_attach_task = -22 */ ... --- v10: - Fix code style issues for LoongArch - Fix selftest issues - Use CONFIG_FUNCTION_GRAPH_TRACER to control fgraph_ret_regs definition v9: - Fix align issues in ARM asm code - Fix align issues in LoongArch asm code - Update commit messages - Update comments for ftrace_return_to_handler v8: - Fix issues in ARM64 asm code - Fix issues in selftest - Add some comments on CONFIG_HAVE_FUNCTION_GRAPH_RETVAL - Make CONFIG_FUNCTION_GRAPH_RETVAL switable - Modify the control range of CONFIG_HAVE_FUNCTION_GRAPH_RETVAL v7: - Rename trace option 'graph_retval_hex' to 'funcgraph-retval-hex' - Introduce a new structure fgraph_ret_regs for each architecture to hold return registers - Separate each architecture modification info individual patches - Add a test case for funcgraph-retval - Update documentation description - Support LoongArch v6: - Remove the conversion code for short and char types, because these two types are rarely used to store an error code. - Modify the limitations for funcgraph-retval - Optimize the English expression v5: - Pass both the return values to ftrace_return_to_handler - Modify the parameter sequence of ftrace_return_to_handler to decrease the modification of assembly code, thanks to Russell King - Wrap __ftrace_return_to_handler with ftrace_return_to_handler for compatible - Describe the limitations of funcgraph-retval v4: - Modify commit message - Introduce new option graph_retval_hex to control display format - Introduce macro CONFIG_FUNCTION_GRAPH_RETVAL and CONFIG_HAVE_FUNCTION_GRAPH_RETVAL - Add related arch maintainers to review v3: - Modify the commit message: add trace logs processed with the btf tool v2: - Modify the commit message: use BTF to get the return type of function Donglin Peng (8): function_graph: Support recording and printing the return value of function tracing: Add documentation for funcgraph-retval and funcgraph-retval-hex ARM: ftrace: Enable HAVE_FUNCTION_GRAPH_RETVAL arm64: ftrace: Enable HAVE_FUNCTION_GRAPH_RETVAL riscv: ftrace: Enable HAVE_FUNCTION_GRAPH_RETVAL x86/ftrace: Enable HAVE_FUNCTION_GRAPH_RETVAL LoongArch: ftrace: Enable HAVE_FUNCTION_GRAPH_RETVAL selftests/ftrace: Add funcgraph-retval test case Documentation/trace/ftrace.rst | 74 +++++++++++++++ arch/arm/Kconfig | 1 + arch/arm/include/asm/ftrace.h | 22 +++++ arch/arm/kernel/asm-offsets.c | 8 +- arch/arm/kernel/entry-ftrace.S | 10 +- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/ftrace.h | 22 +++++ arch/arm64/kernel/asm-offsets.c | 13 +++ arch/arm64/kernel/entry-ftrace.S | 27 +++--- arch/loongarch/Kconfig | 1 + arch/loongarch/include/asm/ftrace.h | 22 +++++ arch/loongarch/kernel/asm-offsets.c | 15 ++- arch/loongarch/kernel/mcount.S | 14 +-- arch/loongarch/kernel/mcount_dyn.S | 15 +-- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/ftrace.h | 21 +++++ arch/riscv/kernel/mcount.S | 7 +- arch/x86/Kconfig | 1 + arch/x86/include/asm/ftrace.h | 20 ++++ arch/x86/kernel/ftrace_32.S | 8 +- arch/x86/kernel/ftrace_64.S | 7 +- include/linux/ftrace.h | 3 + kernel/trace/Kconfig | 15 +++ kernel/trace/fgraph.c | 23 ++++- kernel/trace/trace.h | 2 + kernel/trace/trace_entries.h | 26 ++++++ kernel/trace/trace_functions_graph.c | 93 +++++++++++++++++-- .../ftrace/test.d/ftrace/fgraph-retval.tc | 43 +++++++++ 28 files changed, 460 insertions(+), 55 deletions(-) create mode 100644 tools/testing/selftests/ftrace/test.d/ftrace/fgraph-retval.tc