From patchwork Fri Jun 3 03:26:23 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Long X-Patchwork-Id: 9151611 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id ECA246074E for ; Fri, 3 Jun 2016 03:29:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DDDD027BFA for ; Fri, 3 Jun 2016 03:29:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D27332832B; Fri, 3 Jun 2016 03:29:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6C03A27BFA for ; Fri, 3 Jun 2016 03:29:45 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1b8fmT-0002Is-PN; Fri, 03 Jun 2016 03:28:17 +0000 Received: from mail-qt0-x22c.google.com ([2607:f8b0:400d:c0d::22c]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1b8flJ-0001Gw-Aa for linux-arm-kernel@lists.infradead.org; Fri, 03 Jun 2016 03:27:08 +0000 Received: by mail-qt0-x22c.google.com with SMTP id q45so8497382qtq.1 for ; Thu, 02 Jun 2016 20:26:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=T6czu7YSV15aoMBd5K83jgbZJw9sWeK0ribeZsF07bk=; b=fZXUjEg2+f2WolH/5BT2qp+je0KjB+MfE+Hz86Umzh351TiJiIhSJk79JOGWBFHGTW GlBxSGVm6pzn5eymeP4LCOfo28055ybsBGu40SSMjVN8OIg7BycHn9Z0u3YuxUVYIqJX WLzzM39ZCayiQN80KQMsrA+I8nZcXcUSX35aA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=T6czu7YSV15aoMBd5K83jgbZJw9sWeK0ribeZsF07bk=; b=UyR7NHALhffdmLxsOQbMK++uvtq8sRbWq62dwuC+m9/Zs7S71AswDgZP3k0Efq7qB5 pFGB7gxMqjUQbq8JgU4/MRoZq6kA4LLB4HZYDwVB8v/LTnHa65nuh7EYkJk7AmiJN0/o HGgPSemQaTyay4lLlApzfzuTXMmHeV0QHP5Xtv3Fm769QTpQZYke/Sj1efWlGsUrbgSg HBwmWfOrZYLCNG7KUDNVtFW70PdItvg/Q4BhLcPIclztAlzMUGoJxOXI2S5Zn56l4WpM ucMmdgzfl2wWE3/HxZO7WI5e13lgzT63qnLJqBtreGlm8bCzoDw1PXrwH7IRk+ysxVxM cGHA== X-Gm-Message-State: ALyK8tLOlpdnF6XPU58TL8wt2KvNavgOF2Ku/KEZJFEm8WQM1M/nn2Q8/11hAQjdBlrpJZPD X-Received: by 10.200.51.89 with SMTP id u25mr1359247qta.58.1464924404256; Thu, 02 Jun 2016 20:26:44 -0700 (PDT) Received: from localhost.localdomain (pool-72-71-243-181.cncdnh.fast00.myfairpoint.net. [72.71.243.181]) by smtp.googlemail.com with ESMTPSA id 30sm359841qgx.19.2016.06.02.20.26.42 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 02 Jun 2016 20:26:43 -0700 (PDT) From: David Long To: Catalin Marinas , Huang Shijie , James Morse , Marc Zyngier , Pratyush Anand , Sandeepa Prabhu , Will Deacon , William Cohen , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Steve Capper , Masami Hiramatsu , Li Bin Subject: [PATCH v13 09/10] arm64: Add kernel return probes support (kretprobes) Date: Thu, 2 Jun 2016 23:26:23 -0400 Message-Id: <1464924384-15269-10-git-send-email-dave.long@linaro.org> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1464924384-15269-1-git-send-email-dave.long@linaro.org> References: <1464924384-15269-1-git-send-email-dave.long@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160602_202705_707666_8448FEBC X-CRM114-Status: GOOD ( 20.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jisheng Zhang , Mark Rutland , Daniel Thompson , Vladimir Murzin , Petr Mladek , Ard Biesheuvel , Jens Wiklander , Robin Murphy , Mark Brown , Suzuki K Poulose , Dave P Martin , Andrey Ryabinin , yalin wang , Yang Shi , Zi Shen Lim , John Blackwood , Andrew Morton , =?UTF-8?q?Alex=20Benn=C3=A9e?= , Adam Buchbinder , Christoffer Dall MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Sandeepa Prabhu The pre-handler of this special 'trampoline' kprobe executes the return probe handler functions and restores original return address in ELR_EL1. This way the saved pt_regs still hold the original register context to be carried back to the probed kernel function. Signed-off-by: Sandeepa Prabhu Signed-off-by: David A. Long --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/kprobes.c | 75 ++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 5496b75..47989eb 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -89,6 +89,7 @@ config ARM64 select HAVE_RCU_TABLE_FREE select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES + select HAVE_KRETPROBES if HAVE_KPROBES select IOMMU_DMA if IOMMU_SUPPORT select IRQ_DOMAIN select IRQ_FORCED_THREADING diff --git a/arch/arm64/kernel/kprobes.c b/arch/arm64/kernel/kprobes.c index b35f76f..c116904 100644 --- a/arch/arm64/kernel/kprobes.c +++ b/arch/arm64/kernel/kprobes.c @@ -577,7 +577,80 @@ bool arch_within_kprobe_blacklist(unsigned long addr) void __kprobes __used *trampoline_probe_handler(struct pt_regs *regs) { - return NULL; + struct kretprobe_instance *ri = NULL; + struct hlist_head *head, empty_rp; + struct hlist_node *tmp; + unsigned long flags, orig_ret_addr = 0; + unsigned long trampoline_address = + (unsigned long)&kretprobe_trampoline; + + INIT_HLIST_HEAD(&empty_rp); + kretprobe_hash_lock(current, &head, &flags); + + /* + * It is possible to have multiple instances associated with a given + * task either because multiple functions in the call path have + * a return probe installed on them, and/or more than one return + * probe was registered for a target function. + * + * We can handle this because: + * - instances are always inserted at the head of the list + * - when multiple return probes are registered for the same + * function, the first instance's ret_addr will point to the + * real return address, and all the rest will point to + * kretprobe_trampoline + */ + hlist_for_each_entry_safe(ri, tmp, head, hlist) { + if (ri->task != current) + /* another task is sharing our hash bucket */ + continue; + + if (ri->rp && ri->rp->handler) { + __this_cpu_write(current_kprobe, &ri->rp->kp); + get_kprobe_ctlblk()->kprobe_status = KPROBE_HIT_ACTIVE; + ri->rp->handler(ri, regs); + __this_cpu_write(current_kprobe, NULL); + } + + orig_ret_addr = (unsigned long)ri->ret_addr; + recycle_rp_inst(ri, &empty_rp); + + if (orig_ret_addr != trampoline_address) + /* + * This is the real return address. Any other + * instances associated with this task are for + * other calls deeper on the call stack + */ + break; + } + + kretprobe_assert(ri, orig_ret_addr, trampoline_address); + /* restore the original return address */ + instruction_pointer(regs) = orig_ret_addr; + reset_current_kprobe(); + kretprobe_hash_unlock(current, &flags); + + hlist_for_each_entry_safe(ri, tmp, &empty_rp, hlist) { + hlist_del(&ri->hlist); + kfree(ri); + } + + /* return 1 so that post handlers not called */ + return (void *) orig_ret_addr; +} + +void __kprobes arch_prepare_kretprobe(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + ri->ret_addr = (kprobe_opcode_t *)regs->regs[30]; + + /* replace return addr (x30) with trampoline */ + regs->regs[30] = (long)&kretprobe_trampoline; +} + +int __kprobes arch_trampoline_kprobe(struct kprobe *p) +{ + return 0; } int __init arch_init_kprobes(void)