From patchwork Fri Apr 24 02:44:08 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AKASHI Takahiro X-Patchwork-Id: 6266291 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id D34509F1C4 for ; Fri, 24 Apr 2015 02:48:09 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id C091420253 for ; Fri, 24 Apr 2015 02:48:08 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AB5A62022A for ; Fri, 24 Apr 2015 02:48:07 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1YlTcx-0003o2-7q; Fri, 24 Apr 2015 02:46:03 +0000 Received: from mail-pa0-f46.google.com ([209.85.220.46]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1YlTcV-0003Nf-Au for linux-arm-kernel@lists.infradead.org; Fri, 24 Apr 2015 02:45:36 +0000 Received: by pacwv17 with SMTP id wv17so13089947pac.0 for ; Thu, 23 Apr 2015 19:45:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ab6/Cw76rlRLqgUUaPOPcXGT1ssej+DL6nK8gOsXIxA=; b=D+8G2OeXC46qIcboN8UR+VqeJpU41mXa2prKcoXS7GPrXEsk9IJKe9d1bnTEJfOmJy T0xzJQ6SsUU24EpNxG9Z2ZOcDUXHNbuVMOGdK9vPttitC0s+6yudEAL2+u0eguSDZTu2 4uqHOGUvFVafX7gEN02oH10XCrAaKsE1am1w5FpJTpSjWpFUOMQrY02ivoe4iXoHta9P eroUhHPV3JOHHo6UUN6qQ4qsIanKQu8khRpigWM0GhXQR0SktZ1crCfumIRh63FCCCmN 4NwyrsCI8gHrFuHSMNFNxYCBEopW/xDHXVb7xGITTZmyT+cdVCqkmtwmO/RmvWCFVyeJ VEIQ== X-Gm-Message-State: ALoCoQme13WNME8u3M5BVEpW+Q32ZlOkCThRhmslnQMCwMcV93NLEVH/g55m7RvF65U+yEn6FTfD X-Received: by 10.68.89.100 with SMTP id bn4mr10813820pbb.48.1429843513761; Thu, 23 Apr 2015 19:45:13 -0700 (PDT) Received: from localhost.localdomain (61-205-1-167m5.grp1.mineo.jp. [61.205.1.167]) by mx.google.com with ESMTPSA id do16sm9488624pac.15.2015.04.23.19.45.06 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 23 Apr 2015 19:45:12 -0700 (PDT) From: AKASHI Takahiro To: rostedt@goodmis.org, mingo@kernel.org, jpoimboe@redhat.com, sjenning@redhat.com, jkosina@suse.cz, vojtech@suse.cz, catalin.marinas@arm.com, will.deacon@arm.com Subject: [RFC 3/4] arm64: ftrace: add DYNAMIC_TRACE_WITH_REGS version Date: Fri, 24 Apr 2015 11:44:08 +0900 Message-Id: <1429843449-7388-4-git-send-email-takahiro.akashi@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1429843449-7388-1-git-send-email-takahiro.akashi@linaro.org> References: <1429843449-7388-1-git-send-email-takahiro.akashi@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150423_194535_472587_B698C8DA X-CRM114-Status: GOOD ( 13.86 ) X-Spam-Score: -0.7 (/) Cc: linaro-kernel@lists.linaro.org, linux-kernel@vger.kernel.org, AKASHI Takahiro , broonie@kernel.org, masami.hiramatsu.pt@hitachi.com, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP CONFIG_DYNAMIC_TRACE_WITH_REGS is a prerequisite for ftrace-based kprobes as well as livepatch. This patch adds ftrace_regs_caller(), which will pass pt_regs info to ftrace handlers, to enable this config. Signed-off-by: AKASHI Takahiro --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/ftrace.h | 4 ++ arch/arm64/kernel/entry-ftrace.S | 124 ++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/ftrace.c | 24 +++++++- 4 files changed, 152 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 1b8e973..c3678ed 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -54,6 +54,7 @@ config ARM64 select HAVE_DMA_ATTRS select HAVE_DMA_CONTIGUOUS select HAVE_DYNAMIC_FTRACE + select HAVE_DYNAMIC_FTRACE_WITH_REGS select HAVE_EFFICIENT_UNALIGNED_ACCESS select HAVE_FTRACE_MCOUNT_RECORD select HAVE_FUNCTION_TRACER diff --git a/arch/arm64/include/asm/ftrace.h b/arch/arm64/include/asm/ftrace.h index c5534fa..5a14269 100644 --- a/arch/arm64/include/asm/ftrace.h +++ b/arch/arm64/include/asm/ftrace.h @@ -16,6 +16,10 @@ #define MCOUNT_ADDR ((unsigned long)_mcount) #define MCOUNT_INSN_SIZE AARCH64_INSN_SIZE +#ifdef CONFIG_DYNAMIC_FTRACE +#define ARCH_SUPPORTS_FTRACE_OPS 1 +#endif + #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S index 08cafc5..fed28eb 100644 --- a/arch/arm64/kernel/entry-ftrace.S +++ b/arch/arm64/kernel/entry-ftrace.S @@ -10,6 +10,7 @@ */ #include +#include #include #include @@ -86,6 +87,95 @@ add \reg, \reg, #8 .endm +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS +/* + * stack layout after mcount_save_regs in _mcount(): + * + * current sp/fp => 0:+---------+ + * in _mcount() |pt_regs | + * | | + * | +-----+ + * | | x29 | -> instrumented function's fp + * | +-----+ + * | | x30 | -> _mcount()'s lr + * | +-----+ (= instrumented function's pc) + * +S_FRAME_SIZE | | + * old sp => :+-----+--- + * when instrumented | | + * function calls | ... | + * _mcount() | | + * | | + * instrumented => +xx:+-----+ + * function's fp | x29 | -> parent's fp + * +-----+ + * | x30 | -> instrumented function's lr (= parent's pc) + * +-----+ + * | ... | + */ + .macro mcount_save_regs + sub sp, sp, #S_FRAME_SIZE + + stp x0, x1, [sp, #8 * 0] + stp x2, x3, [sp, #8 * 2] + stp x4, x5, [sp, #8 * 4] + stp x6, x7, [sp, #8 * 6] + stp x8, x9, [sp, #8 * 8] + stp x10, x11, [sp, #8 * 10] + stp x12, x13, [sp, #8 * 12] + stp x14, x15, [sp, #8 * 14] + stp x16, x17, [sp, #8 * 16] + stp x18, x19, [sp, #8 * 18] + stp x20, x21, [sp, #8 * 20] + stp x22, x23, [sp, #8 * 22] + stp x24, x25, [sp, #8 * 24] + stp x26, x27, [sp, #8 * 26] + stp x28, x29, [sp, #8 * 28] + + ldr x0, [x29, #8] + mcount_adjust_addr x0, x0 + str x0, [sp, #S_LR] + + add x0, sp, #S_FRAME_SIZE + str x0, [sp, #S_SP] + + mcount_adjust_addr x0, x30 + str x0, [sp, #S_PC] + + ldr x0, [sp, #8 * 0] + + mov x29, sp + .endm + + /* for instrumented function */ + .macro mcount_get_lr1 reg + ldr \reg, [x29, #8 * 29] + ldr \reg, [\reg, #8] + mcount_adjust_addr \reg, \reg + .endm + + .macro mcount_restore_rest_regs + ldp x0, x1, [sp, #8 * 0] + ldp x2, x3, [sp, #8 * 2] + ldp x4, x5, [sp, #8 * 4] + ldp x6, x7, [sp, #8 * 6] + ldp x8, x9, [sp, #8 * 8] + ldp x10, x11, [sp, #8 * 10] + ldp x12, x13, [sp, #8 * 12] + ldp x14, x15, [sp, #8 * 14] + ldp x16, x17, [sp, #8 * 16] + ldp x18, x19, [sp, #8 * 18] + ldp x20, x21, [sp, #8 * 20] + ldp x22, x23, [sp, #8 * 22] + ldp x24, x25, [sp, #8 * 24] + ldp x26, x27, [sp, #8 * 26] + ldp x28, x29, [sp, #8 * 28] + ldr x30, [sp, #S_LR] + add sp, sp, #S_FRAME_SIZE + + mcount_enter + .endm +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */ + #ifndef CONFIG_DYNAMIC_FTRACE /* * void _mcount(unsigned long return_address) @@ -156,6 +246,10 @@ ENTRY(ftrace_caller) mcount_get_pc0 x0 // function's pc mcount_get_lr x1 // function's lr + adrp x2, function_trace_op + add x2, x2, #:lo12:function_trace_op + ldr x2, [x2] // current ftrace_ops + mov x3, xzr // pt_regs (NULL) .global ftrace_call ftrace_call: // tracer(pc, lr); @@ -171,6 +265,36 @@ ftrace_graph_call: // ftrace_graph_caller(); mcount_exit ENDPROC(ftrace_caller) + +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS +/* + * void ftrace_regs_caller() + */ +ENTRY(ftrace_regs_caller) + mcount_save_regs + + mcount_get_pc0 x0 // function's pc + mcount_get_lr1 x1 // function's lr + adrp x2, function_trace_op + add x2, x2, #:lo12:function_trace_op + ldr x2, [x2] // current ftrace_ops + mov x3, sp // pt_regs + + .global ftrace_regs_call +ftrace_regs_call: // tracer(pc, lr, ops, regs); + nop // This will be replaced with "bl xxx" + // where xxx can be any kind of tracer. + + mcount_restore_rest_regs + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + b ftrace_graph_call +#endif + + mcount_exit +ENDPROC(ftrace_regs_caller) +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */ + #endif /* CONFIG_DYNAMIC_FTRACE */ ENTRY(ftrace_stub) diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c index c851be7..5440673 100644 --- a/arch/arm64/kernel/ftrace.c +++ b/arch/arm64/kernel/ftrace.c @@ -56,12 +56,21 @@ int ftrace_update_ftrace_func(ftrace_func_t func) { unsigned long pc; u32 new; + int ret; pc = (unsigned long)&ftrace_call; new = aarch64_insn_gen_branch_imm(pc, (unsigned long)func, AARCH64_INSN_BRANCH_LINK); + ret = ftrace_modify_code(pc, 0, new, false); - return ftrace_modify_code(pc, 0, new, false); + if (!ret) { + pc = (unsigned long)&ftrace_regs_call; + new = aarch64_insn_gen_branch_imm(pc, (unsigned long)func, + AARCH64_INSN_BRANCH_LINK); + ret = ftrace_modify_code(pc, 0, new, false); + } + + return ret; } /* @@ -97,6 +106,19 @@ int __init ftrace_dyn_arch_init(void) { return 0; } + +int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, + unsigned long addr) +{ + unsigned long pc = rec->ip; + u32 old, new; + + old = aarch64_insn_gen_branch_imm(pc, old_addr, + AARCH64_INSN_BRANCH_LINK); + new = aarch64_insn_gen_branch_imm(pc, addr, AARCH64_INSN_BRANCH_LINK); + + return ftrace_modify_code(pc, old, new, true); +} #endif /* CONFIG_DYNAMIC_FTRACE */ #ifdef CONFIG_FUNCTION_GRAPH_TRACER