From patchwork Mon Apr 10 14:13:43 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 9672747 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D4E6960244 for ; Mon, 10 Apr 2017 14:15:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC47D2846D for ; Mon, 10 Apr 2017 14:15:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C12C628470; Mon, 10 Apr 2017 14:15:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 181482846D for ; Mon, 10 Apr 2017 14:15:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=CoPua2DsFy4flRfG69cX1iIFB9xbiWahJMJzTaObQB8=; b=kkBNzbo9jCYKz9k9tYnOFE6Xk2 KArWvcPXi5ytptC6AJCr9HR189nm/yK/Zbi8OPt8XKz6l0/Sa7RAAAiji6F+6AeExPJlVxvcM/tAt jRa3M5STOiyeMKaYxxfqsyJHdQ0wUC5NRPDlUXmFKAK0zf1OoguHtJ1XM9OzBa8IJXgx/g3S4qmf6 4bOMtqiAgQK0vxnraIPHu/6JbZIcdfy/Lm5G9386+udpDmBloKRkQmjpDfe2rzRPXUucK072hXD2D Izbh7Iob2OLjphSWZadh07Yvm4v/LRN8g2c3avN8N8HTUXx2OfZCXRUodnKeAM8gKrTggXGsqt3lz wrIoSRZA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1cxa6A-0007e1-VS; Mon, 10 Apr 2017 14:15:18 +0000 Received: from mail-wm0-x230.google.com ([2a00:1450:400c:c09::230]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1cxa5G-0005mz-Q3 for linux-arm-kernel@lists.infradead.org; Mon, 10 Apr 2017 14:14:25 +0000 Received: by mail-wm0-x230.google.com with SMTP id w64so40191751wma.0 for ; Mon, 10 Apr 2017 07:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pL9ByrnVOcQQoSpQxskT7gp4L5EIwT3ZzEY1GPnbrmI=; b=YwU1di3q1JrHN+64/RYtNlC0vdUymgaNTICkBAv1rjdeFgBH2FTNDdQTerTgLTU5p7 /EOX7FV11FBUTPVSWw0PHW3hTWSXZFE5FoTorbRTuKedJMyayqnqC49f/ywsWfrjMvYf rlnukVR6KActNj0fQREC9Qu3qtV3NhomnsyPQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pL9ByrnVOcQQoSpQxskT7gp4L5EIwT3ZzEY1GPnbrmI=; b=OYDeZo4TNI1NO2WBgZISUdhBPrdEgLgWcKljnsGh5x4dEhbcmaXWX7xLq5rcqT6s+m 3OeitHVOgfk8iB27l2taLn1z+KGlSQs8KsaJOyFzAL/90lOfq4AyaQOlarir9iUlH9s/ dvx9Zw6XWGL/3aT6u7fS2kiyzYx/FgaWIePsXXgzV/G1d14/lKsop+cO116iwcUI9FeD aeT3ZCjGQthZlpO97jVWMCkNZ4uHR6HmihxciQ6bUgxAV2KnWRi1CyzjY2cwX2kvd6VD C5ETtYE2g5gHGJHLF/lKBvTpTHJgMI2ti0Szz9zsznoSeITO60T93jzFKCmZ4P7TKNuY oI6A== X-Gm-Message-State: AN3rC/7syMr4cYiziY3OpyAw/S1VvShrAaYOqy65YhfmHMQy++PwoJaj 4WNe2LBG9VP2kcPPG2iStQ== X-Received: by 10.28.98.135 with SMTP id w129mr10795552wmb.68.1491833639952; Mon, 10 Apr 2017 07:13:59 -0700 (PDT) Received: from localhost.localdomain ([196.85.182.219]) by smtp.gmail.com with ESMTPSA id m186sm10435465wmd.21.2017.04.10.07.13.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Apr 2017 07:13:59 -0700 (PDT) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, rostedt@goodmis.org, mingo@redhat.com Subject: [PATCH 2/2] arm64: ftrace: add support for far branches to dynamic ftrace Date: Mon, 10 Apr 2017 15:13:43 +0100 Message-Id: <20170410141343.1045-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20170410141343.1045-1-ard.biesheuvel@linaro.org> References: <20170410141343.1045-1-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170410_071423_051404_2D490792 X-CRM114-Status: GOOD ( 25.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, Ard Biesheuvel , huawei.libin@huawei.com, guohanjun@huawei.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Currently, dynamic ftrace support in the arm64 kernel assumes that all core kernel code is within range of ordinary branch instructions in module code, which is usually the case, but is no longer guaranteed now that we have support for module PLTs and address space randomization. Since all patching of branch instructions involves function calls to ftrace_caller(), we can emit the modules with a trampoline that has unlimited range, and patch the branch instruction to call the trampoline if ftrace_caller() itself is out of range. Signed-off-by: Ard Biesheuvel --- arch/arm64/Kconfig | 2 +- arch/arm64/Makefile | 3 ++ arch/arm64/include/asm/module.h | 3 ++ arch/arm64/kernel/ftrace.c | 37 ++++++++++++++++++-- arch/arm64/kernel/module-plts.c | 10 ++++++ arch/arm64/lib/Makefile | 2 ++ arch/arm64/lib/ftrace-mod.S | 25 +++++++++++++ 7 files changed, 78 insertions(+), 4 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index e7f043efff41..31af7ea72072 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -982,7 +982,7 @@ config RANDOMIZE_BASE config RANDOMIZE_MODULE_REGION_FULL bool "Randomize the module region independently from the core kernel" - depends on RANDOMIZE_BASE && !DYNAMIC_FTRACE + depends on RANDOMIZE_BASE default y help Randomizes the location of the module region without considering the diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index b9a4a934ca05..01498eab5ada 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -68,6 +68,9 @@ endif ifeq ($(CONFIG_ARM64_MODULE_PLTS),y) KBUILD_LDFLAGS_MODULE += -T $(srctree)/arch/arm64/kernel/module.lds +ifeq ($(CONFIG_DYNAMIC_FTRACE),y) +KBUILD_LDFLAGS_MODULE += $(objtree)/arch/arm64/lib/ftrace-mod.o +endif endif # Default value diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/module.h index e12af6754634..2ff2b7782957 100644 --- a/arch/arm64/include/asm/module.h +++ b/arch/arm64/include/asm/module.h @@ -25,6 +25,9 @@ struct mod_arch_specific { struct elf64_shdr *plt; int plt_num_entries; int plt_max_entries; + + /* for CONFIG_DYNAMIC_FTRACE */ + struct elf64_shdr *ftrace_trampoline; }; #endif diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c index a8db6857cad6..8f5a7dc36c1b 100644 --- a/arch/arm64/kernel/ftrace.c +++ b/arch/arm64/kernel/ftrace.c @@ -9,11 +9,14 @@ * published by the Free Software Foundation. */ +#include #include +#include #include #include #include +#include #include #include @@ -63,6 +66,35 @@ int ftrace_update_ftrace_func(ftrace_func_t func) return ftrace_modify_code(pc, 0, new, false); } +#ifdef CONFIG_ARM64_MODULE_PLTS +EXPORT_SYMBOL_GPL(ftrace_caller); +#endif + +static u32 __ftrace_gen_branch(unsigned long pc, unsigned long addr) +{ + long offset = (long)pc - (long)addr; + struct module *mod; + + if (IS_ENABLED(CONFIG_ARM64_MODULE_PLTS) && + addr == (unsigned long)&ftrace_caller && + unlikely(offset < -SZ_128M || offset >= SZ_128M)) { + + /* + * On kernels that support module PLTs, the offset between the + * call and its target may legally exceed the range of an + * ordinary branch instruction. In this case, we need to branch + * via a trampoline in the module. + */ + mod = __module_address(pc); + if (WARN_ON(!mod)) + return AARCH64_BREAK_FAULT; + + addr = (unsigned long)mod->arch.ftrace_trampoline->sh_addr; + } + + return aarch64_insn_gen_branch_imm(pc, addr, AARCH64_INSN_BRANCH_LINK); +} + /* * Turn on the call to ftrace_caller() in instrumented function */ @@ -72,7 +104,7 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) u32 old, new; old = aarch64_insn_gen_nop(); - new = aarch64_insn_gen_branch_imm(pc, addr, AARCH64_INSN_BRANCH_LINK); + new = __ftrace_gen_branch(pc, addr); return ftrace_modify_code(pc, old, new, true); } @@ -87,8 +119,7 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, u32 old = 0, new; if (!IS_ENABLED(CONFIG_ARM64_MODULE_PLTS)) - old = aarch64_insn_gen_branch_imm(pc, addr, - AARCH64_INSN_BRANCH_LINK); + old = __ftrace_gen_branch(pc, addr); new = aarch64_insn_gen_nop(); return ftrace_modify_code(pc, old, new, diff --git a/arch/arm64/kernel/module-plts.c b/arch/arm64/kernel/module-plts.c index 1ce90d8450ae..859c7170e69a 100644 --- a/arch/arm64/kernel/module-plts.c +++ b/arch/arm64/kernel/module-plts.c @@ -162,6 +162,10 @@ int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs, mod->arch.plt = sechdrs + i; else if (sechdrs[i].sh_type == SHT_SYMTAB) syms = (Elf64_Sym *)sechdrs[i].sh_addr; + else if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE) && + strcmp(".text.ftrace_trampoline", + secstrings + sechdrs[i].sh_name) == 0) + mod->arch.ftrace_trampoline = sechdrs + i; } if (!mod->arch.plt) { @@ -173,6 +177,12 @@ int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs, return -ENOEXEC; } + if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE) && !mod->arch.ftrace_trampoline) { + pr_err("%s: module ftrace trampoline section missing\n", + mod->name); + return -ENOEXEC; + } + for (i = 0; i < ehdr->e_shnum; i++) { Elf64_Rela *rels = (void *)ehdr + sechdrs[i].sh_offset; int numrels = sechdrs[i].sh_size / sizeof(Elf64_Rela); diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile index c86b7909ef31..b01dcfa9c002 100644 --- a/arch/arm64/lib/Makefile +++ b/arch/arm64/lib/Makefile @@ -4,6 +4,8 @@ lib-y := bitops.o clear_user.o delay.o copy_from_user.o \ memcmp.o strcmp.o strncmp.o strlen.o strnlen.o \ strchr.o strrchr.o +lib-$(CONFIG_DYNAMIC_FTRACE) += ftrace-mod.o + # Tell the compiler to treat all general purpose registers (with the # exception of the IP registers, which are already handled by the caller # in case of a PLT) as callee-saved, which allows for efficient runtime diff --git a/arch/arm64/lib/ftrace-mod.S b/arch/arm64/lib/ftrace-mod.S new file mode 100644 index 000000000000..ce15b9948851 --- /dev/null +++ b/arch/arm64/lib/ftrace-mod.S @@ -0,0 +1,25 @@ +/* + * Copyright (C) 2017 Linaro Ltd + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ + +#include +#include + + .section ".text.ftrace_trampoline", "ax" +ENTRY(__ftrace_trampoline) + stp x29, x30, [sp, #-16]! + mov x29, sp + + movz x30, #:abs_g3:ftrace_caller + movk x30, #:abs_g2_nc:ftrace_caller + movk x30, #:abs_g1_nc:ftrace_caller + movk x30, #:abs_g0_nc:ftrace_caller + blr x30 + + ldp x29, x30, [sp], #16 + ret +ENDPROC(__ftrace_trampoline)