From patchwork Tue Nov 29 14:18:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13058628 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F3289C4321E for ; Tue, 29 Nov 2022 14:19:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=yfPqQSp8hkL8sKC99oZMiIbbMSW6zUs0RJEk0+KlpLw=; b=B0KadDUhvPXEVt w6rEG7Ub9EdLJJHyTV8Y5fW0Nj3FcHLm8rDGOwDV+JMPaw9zlHSjWusjdK6z+zri5gR59xohkamx8 AYkXu5dZ0551jjulh4j64Oq60wcclcZDK6V3EiwH0tpvPT+Q/R8i1pZqUBa02jwUr1tByAT7UoIzd ceUWtfuv1pfVmZ1bYWm0DuTyiwszZamLZLuKvw+XWSgyRpQz1PAKZMliAYhkqpeUFmSJDMiin3oMd rm+JWnjWNRkmOExIiJi821JdzEPwHv4kPIPjSoqs6fK8Pbm7K3BbTGxsyKBXBq3dBdPHEC43VeiOD dZllBONFWLYkjVSNnX5A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p01RD-009AHQ-6x; Tue, 29 Nov 2022 14:18:19 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p01R7-009AEk-Ed for linux-arm-kernel@lists.infradead.org; Tue, 29 Nov 2022 14:18:14 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 052D96175C; Tue, 29 Nov 2022 14:18:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 151AAC433D7; Tue, 29 Nov 2022 14:18:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669731492; bh=yoIOK1L/FcjLo8o+n+d0GVBySgN8By0KnxnZeV4z3Y4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ntau9BLlBVIt35fzuJniGd7d3o6azjLZ8fnIyqXXvZIU11UVu2Sn6/ZAsVDjGEucQ e0oAFWCeqRgL7IDX7YFTPRXiGMfEKxZ6MXd+H3MnQSqG2TbV4ep2F+yMU2wkyMdvls j5/60I65PMFcBcRLSzJ2EFo8DRwi6oNR8SmWQOnf9qAv7cXaXBpvKs/TQtfm1FRyJ8 YB1T9xANgToLYveIH/X9Hgv7YqyFCNDtYyKFn/OtgDEdmF9tOiltcwvUu96eeDwO8z r90/G6ityDhNBOz7OBMQkVcbgEkcUQqfQCy8k4++fOGBuGPvw8MQg6k0S1xU8B7IeW rWU89usKlGVZA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown Subject: [PATCH 1/4] arm64: assembler: Force error on misuse of .Lframe_local_offset Date: Tue, 29 Nov 2022 15:18:00 +0100 Message-Id: <20221129141803.1746898-2-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221129141803.1746898-1-ardb@kernel.org> References: <20221129141803.1746898-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1032; i=ardb@kernel.org; h=from:subject; bh=yoIOK1L/FcjLo8o+n+d0GVBySgN8By0KnxnZeV4z3Y4=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBjhhSVqdI0xiGpsx73If8ITz6dgm1Px74nNCABgCOK e17LKXqJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCY4YUlQAKCRDDTyI5ktmPJHZuC/ 48wLsfGx1i2DQnNv1Lk0LwZN3YZFGYUDq4c8H9CD88N0aRCZzZ2PBMOX4gjCoyGx7unrc+hziJ3ukZ DVcWoMiKoZRLa2guCa1lWjHJ/kSbfkd2rr5WcikV99UVn5V3Y0aJwnIIUqnglQxbY39elRVEW4Y7mf g3sWbMKAUTfBWRvE9lEl4W+LR27roerpTjJwKPzxvR4PLVpSOgk2oni+bialebnvJKAjH4hVLCysBg Pkw/bZYO+jgIuVkJ0RRrPJP7YrVOjeU1IDiAYODm+4Meut6+2Scygxyfgb0zAB5sWqpLK0a5blMcss r9aaHX503q5M1A8U3YM+GYehe3TnTZdY/0CsLhtICs3P5vNDD3RuszAABLFixds5Up4AxnPhlLN7KT Wukdq3AnwoE7lNtgE+snz2QM8Neyu91ugLiYs4mLynUpOZDfL2HbhZiRLmX+61wDDP2ZOQDhSGZm/z 63oqKl5ofGfR/SSDS1qyCTzDVC7mKjCXvkI+EGnPDor1c= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221129_061813_582178_B0FA4A8B X-CRM114-Status: GOOD ( 11.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The frame_push macro sets a local symbol .Lframe_local_offset to the offset where the local variable area resides in the stack frame. However, while we take care not to nest frame_push and frame_pop sequences, .Lframe_local_offset retains its most recent value, allowing it to be referenced erroneously from outside a frame_push/frame_pop pair. So set it to an obviously wrong value that is guaranteed to trigger a link error in frame_pop. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/assembler.h | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 30eee6473cf0c0ea..3d1714a7eb6411ba 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -752,6 +752,7 @@ alternative_endif .endif ldp x29, x30, [sp], #.Lframe_local_offset + .Lframe_extra .set .Lframe_regcount, -1 + .set .Lframe_local_offset, frame_local_offset_error .endif .endm From patchwork Tue Nov 29 14:18:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13058630 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 97859C4321E for ; Tue, 29 Nov 2022 14:20:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JeXXrNIhl7ry5+ZT3CKUbBHpdLTpUIhrXveDZuZ84Ts=; b=aRMZnvo18nurtX tTcArgecPWBUOUeG6oelZ2ktoRiVXCqD7bvyUu21tyJhuJ9uuS6xQ0jAwjlh3VK/SU8SyQs6klBeW JzE+mk1OsuyvHE6QoE68yCRQrfIfk9XJYSw00R1+lsernJy3Pc/cmedJmuxbBAcAOnLq0uCt5dac6 B9GOGcl03q79HwEJsH/eFlHmNllgxzJsyjYkxTvlm2mGfSYKS+x7rbow9Boylb+MQ6VHLWcgim/c/ 0gqRDjBRFr7biwKcBheoRvMNmWnF0nPWvZFFPbtIdd9rM69tmJu/Y3YIE/MZM+aC4NEOA4nG36QqP vzCMbd5PGxEoySpWagUg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p01S6-009AeR-BD; Tue, 29 Nov 2022 14:19:14 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p01RC-009AHS-0L for linux-arm-kernel@lists.infradead.org; Tue, 29 Nov 2022 14:18:19 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 4FF0EB81699; Tue, 29 Nov 2022 14:18:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 439C3C433D7; Tue, 29 Nov 2022 14:18:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669731495; bh=x72zQXffLuuNmZdX8TIc7uq8kqTybuuDkPbyv5eookQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sJhPhFVPnCmByJeGeuohM03FEPbv39+8eC0tg84NlAZaeUes+MvZKDOCfnVEeQt78 HO1diBrLhnG/cfN2wh3aQDE2kzNyWk93L7O4jM8w990676EWlfWgu0pIZIgrEtEWJu nghE14OLqjrCRhpl+PjcwIwpXmuc2HbeFeZSvMLDaKx0IFSDl1lfNXflD99r3wef2+ rcUXWckSHgXMURFpPcyzqMEtxGudsih7zuiVkJF010z7qeQ6KzlWYL9W8/hB6ttcMI lpc/xd74nI99dCbDqNkOtzuIYzTmsNb5F7J+tpbLTaBC/Hmy37Xz/zLvbKL2YiRSmm eTRUGr41GiDtA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown Subject: [PATCH 2/4] arm64: assembler: Add macros for return address protection Date: Tue, 29 Nov 2022 15:18:01 +0100 Message-Id: <20221129141803.1746898-3-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221129141803.1746898-1-ardb@kernel.org> References: <20221129141803.1746898-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4030; i=ardb@kernel.org; h=from:subject; bh=x72zQXffLuuNmZdX8TIc7uq8kqTybuuDkPbyv5eookQ=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBjhhSXma7NJVNOopbTlw5mJriiM0A2nnzX9EadvFiU pvnMBMSJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCY4YUlwAKCRDDTyI5ktmPJGKRC/ 9ETD3hxxKER96v8rcZlZJZttk16rhdzRiKYTSeJxnhAiWid0eLhq1fDLDCmv54f0fP6Twxnf03ZDim MVmiatQA1f4rV+V2sY7fCv4LMXjwZsoz3HG6sWMCcoNi9Wsd/kkC+GbGeGQaZ+9y08geRwYDIfswAF +w4J5BAtjv8+YIUaL2kC4g95D2IyH3kbjung9Vz4kumISOzjAEV3fPKjM8crV4gAxgJmKVcKzqYMoa AKPSBSUTklY/2peSJfaW/z4Qh0gfSnC7hco6vUu9xBsk1yOagVMrzQW+GslVj1Vtf1OmJMmLK+v277 S2V1Ks/RFcC88Y9UZtyCt1WIwZLauApHb7XofDpMEtiZzYlcPTfs7xSYE5aDtKuIxcw8FLdxIVkjKX wRFDLoZSKpSrMcZMrYZFrTK/nfPTV56xpj2Q/9HB1hcx/dtr7AKieW7bzDzDW34xmWdwPU6BkVsv/+ eDztgRSWLTS1IRIQ/zuv0Ko482EhHV1wiYehN5gH1thsk= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221129_061818_414643_803B168F X-CRM114-Status: GOOD ( 18.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When in-kernel pointer authentication is configured, emit PACIASP and AUTIASP instructions as well as shadow call stack pushes and pops, depending on the configuration. Note that dynamic shadow call stack makes this slightly tricky, as it depends on in-kernel BTI as well. The resulting code will never contain both PAC and shadow call stack operations, even if shadow call stack support is not configured as dynamic. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/assembler.h | 81 ++++++++++++++++++++ 1 file changed, 81 insertions(+) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 3d1714a7eb6411ba..99d74c29ab3cbe05 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -692,6 +692,85 @@ alternative_endif #endif .endm + /* + * protect_return_address - protect the return address value in + * register @reg, either by signing it using PAC and/or by storing it + * on the shadow call stack. + * + * The sequence below emits a shadow call stack push if the feature is + * enabled, and if in-kernel PAC is enabled as well, the instruction + * will be patched into a PACIA instruction involving the same register + * address (and SP as the modifier) if PAC is detected at runtime. + * + * If in-kernel BTI and dynamic shadow call stacks are also configured, + * it becomes a bit more tricky, because then, shadow call stacks will + * only be enabled on non-BTI hardware, regardless of the PAUTH state. + * In that case, we emit one of the following sequences. + * + * PAC+BTI enabled No PAC or BTI BTI without PAC PAC without BTI + * + * B 0f NOP B 0f NOP + * NOP SCS push SCS push NOP + * 0: PACIA NOP NOP PACIA + * + * Note that, due to the code patching occuring at function entry and + * exit, these macros must not be used in code that may execute before + * the boot CPU feature based code patching has completed. + */ + .macro protect_return_address, reg=x30 +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +#if defined(CONFIG_DYNAMIC_SCS) && defined(CONFIG_ARM64_BTI_KERNEL) +alternative_if ARM64_BTI + b .L0_\@ +alternative_else_nop_endif +#endif +alternative_if_not ARM64_HAS_ADDRESS_AUTH +#endif +#ifdef CONFIG_SHADOW_CALL_STACK + str \reg, [x18], #8 +#endif +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +#if !defined(CONFIG_SHADOW_CALL_STACK) || \ + (defined(CONFIG_DYNAMIC_SCS) && defined(CONFIG_ARM64_BTI_KERNEL)) +.L0_\@: nop +#endif +alternative_else +#if defined(CONFIG_DYNAMIC_SCS) && defined(CONFIG_ARM64_BTI_KERNEL) + nop +#endif + .arch_extension pauth + pacia \reg, sp +alternative_endif +#endif + .endm + + /* + * restore_return_address - restore the return address value in + * register @reg, either by authenticating it using PAC and/or + * reloading it from the shadow call stack. + */ + .macro restore_return_address, reg=x30 +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +alternative_if ARM64_HAS_ADDRESS_AUTH + .arch_extension pauth + autia \reg, sp +alternative_else_nop_endif +#if defined(CONFIG_DYNAMIC_SCS) && defined(CONFIG_ARM64_BTI_KERNEL) +alternative_if ARM64_BTI + b .L0_\@ +alternative_else_nop_endif +#endif +alternative_if_not ARM64_HAS_ADDRESS_AUTH +#endif +#ifdef CONFIG_SHADOW_CALL_STACK + ldr \reg, [x18, #-8]! +#endif +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +alternative_else_nop_endif +.L0_\@: +#endif + .endm + /* * frame_push - Push @regcount callee saved registers to the stack, * starting at x19, as well as x29/x30, and set x29 to @@ -699,6 +778,7 @@ alternative_endif * for locals. */ .macro frame_push, regcount:req, extra + protect_return_address __frame st, \regcount, \extra .endm @@ -710,6 +790,7 @@ alternative_endif */ .macro frame_pop __frame ld + restore_return_address .endm .macro __frame_regs, reg1, reg2, op, num From patchwork Tue Nov 29 14:18:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13058631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 81C59C4321E for ; Tue, 29 Nov 2022 14:20:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=c2r/alFfr7hREqIDER1qtSEVXljkZGqLQsTXMiHtePM=; b=0U+yXg/eOSSREJ Q9x+xkPwDl/XphPg5zO36MCsZZoR5atHfxdUwvqBoKk2mvdDsNv0QvilkT8FrDkB8oQrxthW4lRCM NUIpBaRpywzygbqaHZ8T3onL4AjIpHuqGt3+W8i133aFnWVIeyWRLzTCsVNj8XrHmLzlV1Ewokt3i MKmh1ua0HShH5+ZcTjD+9j48qR5wySKdaAhBw68dc34cNvzkJ14aC6dLgFLtSVs12IKcTiaXMjFc1 a2sleU8R6zJGNfGMZUbvfiZYVCSxQ1wWfZXdxn4uoKG/0WPSPKDkiVAbvWdf5UfNIL6dfyVuJ3aZT HO/bYs70kRUbgkp5CFmg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p01SL-009Akm-Cv; Tue, 29 Nov 2022 14:19:29 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p01RD-009AJK-FU for linux-arm-kernel@lists.infradead.org; Tue, 29 Nov 2022 14:18:20 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CBB3D6176F; Tue, 29 Nov 2022 14:18:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 701EAC433D6; Tue, 29 Nov 2022 14:18:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669731497; bh=j5CIKPVnk6x5nzzeaKmG6KC9k0X4ZL1NuiInXmGW3u0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qAdNeDFYEPgXqwlEdlgABNVYIHUZrzjh3SyeWlZs9nMnQ8o+fPaorgZGGn1kyhkjC IXHfPSWMpXBC9wZn3Y1dOiErckyfyIiOWFHWJwv2D1vBDEoMJn3cIEZlSmvM3EK9Yj w5jYjlDS8jfkwZku8BtULBVNiPzeJodVRYNf8zyX+OYJrL/8b3L9HkZaZanHdR3AHy euiJxLRnm/cRcoGFEdyAdhSyA2kmfkfKAB8MIUCXgoBbp9SpQII6bK5lRZEkMuc7z6 Y8EvGt6Fxcr67AN7We6Wk93572y4WNwUVIX7Xgs5lm4YOIGrcDJTjor5e8NnRM2b3b QVca2PfiCCmrw== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown Subject: [PATCH 3/4] arm64: efi: Add return address protection to runtime wrapper Date: Tue, 29 Nov 2022 15:18:02 +0100 Message-Id: <20221129141803.1746898-4-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221129141803.1746898-1-ardb@kernel.org> References: <20221129141803.1746898-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1497; i=ardb@kernel.org; h=from:subject; bh=j5CIKPVnk6x5nzzeaKmG6KC9k0X4ZL1NuiInXmGW3u0=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBjhhSYeoDUBqUnh/xuLIrbR8oSQZ9UJxsZiRYzvgHM bkjeMCKJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCY4YUmAAKCRDDTyI5ktmPJIapDA DHLRUNv1/4nT0Z7RPGXM+uiHKP99O8MoUpuRqtgrlrc5odsC9+9gJBFA7DFHfD+gX0vKF6BFgem0+R 1B59IeKkWLWeSYi4kIaPQET9dZpWCFyIoNbwHLzcrBtcCbzhJwiefkfk7rENiBern2ANcfSXoNVKIh BGOBYDo/CXHBhjLGJeeX5xP3HtWvBcMeBBfKgHc+RqDIo6mV8DFJOrTYtbDXsJntYYuAl36AdltKUU 2pEk7lICRIcZ+BCkR424jyt+Wi2o6eu0kvjFfBzFJXXNBIXaQ4skTtyN4XtiX2Ff89Ye1DPmXnXXrq 3rD2iwQ/myTQ03u/kutST14s/zEipjZ/iX5ozq7NIezig7Zbg1iS4AY8aGsVMhJz2ne+hdmUHctUn5 Zcy8It/JGvSSwGniiqQRfhE957wK/0Fe+sWOrp8CuHFIqwgG5Woa0R0XFjd3AB8ANStK488wcarsLN aBoL0aY4VVINDK6KqzQGe0rMEF0q6QQokG0C/8ytLzZ54= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221129_061819_622402_53318DD6 X-CRM114-Status: GOOD ( 12.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add return address protection to the EFI runtime wrapper so that this code is less likely to be taken advantage for ROP/JOP style attacks. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/efi-rt-wrapper.S | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kernel/efi-rt-wrapper.S b/arch/arm64/kernel/efi-rt-wrapper.S index afd3e81e1b627b87..874da02f3a1664c3 100644 --- a/arch/arm64/kernel/efi-rt-wrapper.S +++ b/arch/arm64/kernel/efi-rt-wrapper.S @@ -6,6 +6,7 @@ #include SYM_FUNC_START(__efi_rt_asm_wrapper) + protect_return_address stp x29, x30, [sp, #-112]! mov x29, sp @@ -46,9 +47,7 @@ SYM_FUNC_START(__efi_rt_asm_wrapper) ldp x1, x2, [sp, #16] cmp x2, x18 ldp x29, x30, [sp], #112 - b.ne 0f - ret -0: + /* * With CONFIG_SHADOW_CALL_STACK, the kernel uses x18 to store a * shadow stack pointer, which we need to restore before returning to @@ -59,7 +58,10 @@ SYM_FUNC_START(__efi_rt_asm_wrapper) #ifdef CONFIG_SHADOW_CALL_STACK ldr_this_cpu x18, __efi_rt_asm_recover_sp + 8, x9 #endif - + b.ne 0f + restore_return_address + ret +0: b efi_handle_corrupted_x18 // tail call SYM_FUNC_END(__efi_rt_asm_wrapper) @@ -74,5 +76,7 @@ SYM_CODE_START(__efi_rt_asm_recover) ldp x27, x28, [sp, #96] ldp x29, x30, [sp], #112 + restore_return_address + b efi_handle_runtime_exception SYM_CODE_END(__efi_rt_asm_recover) From patchwork Tue Nov 29 14:18:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13058632 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9044DC46467 for ; Tue, 29 Nov 2022 14:21:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=13P3KwtsvSgVgfOOzbWJ5D01Y5Swu4buCBFOVX+xmns=; b=UfmtosC0zlJYLP kGnKuuoWC59RDjgUih2cMpd/vLorAIh3IVVVYu+hMFz5R+3xvOj1G/uRpUi4s0Q8SdgG4G+w/6RWA SZ4YqyCmVhrshLdz2rR/eFH51QUDZrWO4oluPLNwkrJCZLPsJ/ezBYP8TN2VsuLX5cSJf7k3klc0y 56bKWZ5eVo50thuXeMl676qQhDe3LyaR3b2txVxGT9sNKeQH1sOEdPPoP2HU08GhEn5GS+/0xeYi2 KhCJ2CRk4G2wjzfvuKI2kGW3MN/Z8xKnSQtuVll3buq7N1NCVCdrCN29czYcJ1jHsV1cHWFuIECTT 65Nj7gxti4KD4o0ZI+dA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p01Sl-009AwR-4w; Tue, 29 Nov 2022 14:19:55 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p01RE-009AKA-3s for linux-arm-kernel@lists.infradead.org; Tue, 29 Nov 2022 14:18:21 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 88FB461770; Tue, 29 Nov 2022 14:18:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9BFB9C433D7; Tue, 29 Nov 2022 14:18:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669731499; bh=w89/zLZJU+ZtRAue8a3Z08EMLc059xV0xjCE22WnAZQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jGY/dMOJjUvBdb26BN6Jgj+ylzx6x5E8AwGIaqClyD1c9eKYtjbTAQ/wMCOy/1MS/ 1RnPByVTwgbDHir3TS7ruZL3VYbXsDb4DZ/+JbLnd0i5ukN7h2TCXdS0VPh3gAYfvk ZoFSK/jizeaOAjhCPGA5YYmMt41JNzcuDg5CDVT7LVWaF1NulVf3KYxbQ6deAAQaFp b3mWOumfqLmq5V4EM5mcjdmGUlxKGSAKe350Uz999AQN7a/KTemiVe2eaA8iovW6J6 5p1hTk5J3ntXNxQBbZA2WxzTyluQkaUD5hJjzSA+XUsHvte5uFsOcsDXRPo1rSkUlB 2laxFYgZF8hDA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown Subject: [PATCH 4/4] arm64: ftrace: Add return address protection Date: Tue, 29 Nov 2022 15:18:03 +0100 Message-Id: <20221129141803.1746898-5-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221129141803.1746898-1-ardb@kernel.org> References: <20221129141803.1746898-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2504; i=ardb@kernel.org; h=from:subject; bh=w89/zLZJU+ZtRAue8a3Z08EMLc059xV0xjCE22WnAZQ=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBjhhSavwedSaUtPAhomofTfrdoSfeEHaZx62NNCkym DlcpSV2JAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCY4YUmgAKCRDDTyI5ktmPJBh2C/ 956M22JlltakhF56XuURCC/39T89B3bV7tYKA0O2cpOq/PzECWdfp/LrXcsLwit0vU9i+xhgkXQg78 EcBfIyU0/nI2NQdTxmm/T421rGnP9wLwVh301eUPkcr5GE78ArsdCEtbRqN3JuZ1fwf9oXF0kRtAG+ Yk09D4YjCACKTK1Ntwxbh2ME8C0h6IKiL/Pdn3acyuy4A9z6vreeonj+JaQAJhUiqELh1KGawZf/OX E7QOdz6hBhbcyJdEz4uqlYms34QPfMsYiG2N/hlKRm6fpWt0v6u5zDZlLFSkhSEioVLLyucbdohUiX bfaD7m/sS1DIfNghJlzNB56OWBv8Dtkow3dqjvvj2y1LAxKaGeBuSWYMMUJVDQW5KJVbuyX1KOQk5B 7tGYl/yz61LLfmkw7rM6NJFHwujJAk9lHTTLgr00N4TmaUSFlyO7vgZUs0hw0e9fDkgEPzavBqt26j WmyzXVvdDoNaEHWnVzwhtFuTvKJk7vT+MbIGoMyDsCn8U= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221129_061820_284114_2CFF2315 X-CRM114-Status: GOOD ( 15.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use the newly added asm macros to protect and restore the return address in the ftrace call wrappers, based on whichever method is active (PAC and/or shadow call stack). If the graph tracer is in use, this covers both the return address *to* the ftrace call site as well as the return address *at* the call site, and the latter will either be restored in return_to_handler(), or before returning to the call site. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/entry-ftrace.S | 28 +++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S index 795344ab4ec45889..c744e4dd8c90a352 100644 --- a/arch/arm64/kernel/entry-ftrace.S +++ b/arch/arm64/kernel/entry-ftrace.S @@ -35,6 +35,11 @@ * is missing from the LR and existing chain of frame records. */ .macro ftrace_regs_entry, allregs=0 +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + protect_return_address x9 +#endif + protect_return_address x30 + /* Make room for pt_regs, plus a callee frame */ sub sp, sp, #(PT_REGS_SIZE + 16) @@ -89,7 +94,9 @@ SYM_CODE_START(ftrace_caller) b ftrace_common SYM_CODE_END(ftrace_caller) -SYM_CODE_START(ftrace_common) +SYM_CODE_START_LOCAL(ftrace_common) + alternative_insn nop, "xpaci x30", ARM64_HAS_ADDRESS_AUTH, IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL) + sub x0, x30, #AARCH64_INSN_SIZE // ip (callsite's BL insn) mov x1, x9 // parent_ip (callsite's LR) ldr_l x2, function_trace_op // op @@ -115,9 +122,27 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) ldr x30, [sp, #S_LR] ldr x9, [sp, #S_PC] +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + /* grab the original return address from the stack */ + ldr x10, [sp, #PT_REGS_SIZE + 8] +#endif + /* Restore the callsite's SP */ add sp, sp, #PT_REGS_SIZE + 16 + restore_return_address x9 +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + /* compare the original return address with the actual one */ + cmp x10, x30 + b.ne 0f + + /* + * If they are the same, unprotect it now. If it was modified, it will + * be dealt with in return_to_handler() below. + */ + restore_return_address x30 +0: +#endif ret x9 SYM_CODE_END(ftrace_common) @@ -329,6 +354,7 @@ SYM_CODE_START(return_to_handler) ldp x6, x7, [sp, #48] add sp, sp, #64 + restore_return_address x30 ret SYM_CODE_END(return_to_handler) #endif /* CONFIG_FUNCTION_GRAPH_TRACER */