From patchwork Tue Nov 29 14:18:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13058632 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9044DC46467 for ; Tue, 29 Nov 2022 14:21:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=13P3KwtsvSgVgfOOzbWJ5D01Y5Swu4buCBFOVX+xmns=; b=UfmtosC0zlJYLP kGnKuuoWC59RDjgUih2cMpd/vLorAIh3IVVVYu+hMFz5R+3xvOj1G/uRpUi4s0Q8SdgG4G+w/6RWA SZ4YqyCmVhrshLdz2rR/eFH51QUDZrWO4oluPLNwkrJCZLPsJ/ezBYP8TN2VsuLX5cSJf7k3klc0y 56bKWZ5eVo50thuXeMl676qQhDe3LyaR3b2txVxGT9sNKeQH1sOEdPPoP2HU08GhEn5GS+/0xeYi2 KhCJ2CRk4G2wjzfvuKI2kGW3MN/Z8xKnSQtuVll3buq7N1NCVCdrCN29czYcJ1jHsV1cHWFuIECTT 65Nj7gxti4KD4o0ZI+dA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p01Sl-009AwR-4w; Tue, 29 Nov 2022 14:19:55 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p01RE-009AKA-3s for linux-arm-kernel@lists.infradead.org; Tue, 29 Nov 2022 14:18:21 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 88FB461770; Tue, 29 Nov 2022 14:18:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9BFB9C433D7; Tue, 29 Nov 2022 14:18:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669731499; bh=w89/zLZJU+ZtRAue8a3Z08EMLc059xV0xjCE22WnAZQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jGY/dMOJjUvBdb26BN6Jgj+ylzx6x5E8AwGIaqClyD1c9eKYtjbTAQ/wMCOy/1MS/ 1RnPByVTwgbDHir3TS7ruZL3VYbXsDb4DZ/+JbLnd0i5ukN7h2TCXdS0VPh3gAYfvk ZoFSK/jizeaOAjhCPGA5YYmMt41JNzcuDg5CDVT7LVWaF1NulVf3KYxbQ6deAAQaFp b3mWOumfqLmq5V4EM5mcjdmGUlxKGSAKe350Uz999AQN7a/KTemiVe2eaA8iovW6J6 5p1hTk5J3ntXNxQBbZA2WxzTyluQkaUD5hJjzSA+XUsHvte5uFsOcsDXRPo1rSkUlB 2laxFYgZF8hDA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown Subject: [PATCH 4/4] arm64: ftrace: Add return address protection Date: Tue, 29 Nov 2022 15:18:03 +0100 Message-Id: <20221129141803.1746898-5-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221129141803.1746898-1-ardb@kernel.org> References: <20221129141803.1746898-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2504; i=ardb@kernel.org; h=from:subject; bh=w89/zLZJU+ZtRAue8a3Z08EMLc059xV0xjCE22WnAZQ=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBjhhSavwedSaUtPAhomofTfrdoSfeEHaZx62NNCkym DlcpSV2JAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCY4YUmgAKCRDDTyI5ktmPJBh2C/ 956M22JlltakhF56XuURCC/39T89B3bV7tYKA0O2cpOq/PzECWdfp/LrXcsLwit0vU9i+xhgkXQg78 EcBfIyU0/nI2NQdTxmm/T421rGnP9wLwVh301eUPkcr5GE78ArsdCEtbRqN3JuZ1fwf9oXF0kRtAG+ Yk09D4YjCACKTK1Ntwxbh2ME8C0h6IKiL/Pdn3acyuy4A9z6vreeonj+JaQAJhUiqELh1KGawZf/OX E7QOdz6hBhbcyJdEz4uqlYms34QPfMsYiG2N/hlKRm6fpWt0v6u5zDZlLFSkhSEioVLLyucbdohUiX bfaD7m/sS1DIfNghJlzNB56OWBv8Dtkow3dqjvvj2y1LAxKaGeBuSWYMMUJVDQW5KJVbuyX1KOQk5B 7tGYl/yz61LLfmkw7rM6NJFHwujJAk9lHTTLgr00N4TmaUSFlyO7vgZUs0hw0e9fDkgEPzavBqt26j WmyzXVvdDoNaEHWnVzwhtFuTvKJk7vT+MbIGoMyDsCn8U= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221129_061820_284114_2CFF2315 X-CRM114-Status: GOOD ( 15.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use the newly added asm macros to protect and restore the return address in the ftrace call wrappers, based on whichever method is active (PAC and/or shadow call stack). If the graph tracer is in use, this covers both the return address *to* the ftrace call site as well as the return address *at* the call site, and the latter will either be restored in return_to_handler(), or before returning to the call site. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/entry-ftrace.S | 28 +++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S index 795344ab4ec45889..c744e4dd8c90a352 100644 --- a/arch/arm64/kernel/entry-ftrace.S +++ b/arch/arm64/kernel/entry-ftrace.S @@ -35,6 +35,11 @@ * is missing from the LR and existing chain of frame records. */ .macro ftrace_regs_entry, allregs=0 +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + protect_return_address x9 +#endif + protect_return_address x30 + /* Make room for pt_regs, plus a callee frame */ sub sp, sp, #(PT_REGS_SIZE + 16) @@ -89,7 +94,9 @@ SYM_CODE_START(ftrace_caller) b ftrace_common SYM_CODE_END(ftrace_caller) -SYM_CODE_START(ftrace_common) +SYM_CODE_START_LOCAL(ftrace_common) + alternative_insn nop, "xpaci x30", ARM64_HAS_ADDRESS_AUTH, IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL) + sub x0, x30, #AARCH64_INSN_SIZE // ip (callsite's BL insn) mov x1, x9 // parent_ip (callsite's LR) ldr_l x2, function_trace_op // op @@ -115,9 +122,27 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) ldr x30, [sp, #S_LR] ldr x9, [sp, #S_PC] +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + /* grab the original return address from the stack */ + ldr x10, [sp, #PT_REGS_SIZE + 8] +#endif + /* Restore the callsite's SP */ add sp, sp, #PT_REGS_SIZE + 16 + restore_return_address x9 +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + /* compare the original return address with the actual one */ + cmp x10, x30 + b.ne 0f + + /* + * If they are the same, unprotect it now. If it was modified, it will + * be dealt with in return_to_handler() below. + */ + restore_return_address x30 +0: +#endif ret x9 SYM_CODE_END(ftrace_common) @@ -329,6 +354,7 @@ SYM_CODE_START(return_to_handler) ldp x6, x7, [sp, #48] add sp, sp, #64 + restore_return_address x30 ret SYM_CODE_END(return_to_handler) #endif /* CONFIG_FUNCTION_GRAPH_TRACER */