From patchwork Tue Aug 30 23:31:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12960142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 47EFBECAAD4 for ; Tue, 30 Aug 2022 23:36:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=1PXfcD+TyKWDAQqaMQRl0aF6Z7T/fVuQgDtYKoCoLZo=; b=cdq0IdSyiNN+8MuLuYUaolgu7M DPf0T1lPIFiONgbPkJgM/GaCD5r/L3lNN5rXkUozXFhJOIuGMr+QOpYj5BYW4S8kjS0mZpKqEdONw ZEnD+JoAflpcv2F0kb7LAPehFOX9VDMtvtAmAyaw84u5+8bnV/xq2Za7OyrB1buZFBOxBCbbyz23X g1dE4nJZB3RNa/LyjFU9WuVlG3ynl24bnB9ITgOJ2/z2VX0Y30cJ4qTwNHxYEZC2OXo8hMxJwLcfl 4Cr7bLsp1p7egziWn5KGJEI9X+mzqKSc8fg6gQ8vfjrGXIYEyZoFk7Gn9fzLSNhgIJujCGtQZXIQV 9322lb/w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oTAkm-002UPT-LZ; Tue, 30 Aug 2022 23:34:45 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oTAie-002T1U-HQ for linux-arm-kernel@lists.infradead.org; Tue, 30 Aug 2022 23:32:34 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id f12-20020a25b6cc000000b0069a9e36de26so876564ybm.16 for ; Tue, 30 Aug 2022 16:32:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:cc:to:from:subject:references :mime-version:message-id:in-reply-to:date:from:to:cc:subject:date; bh=muitaXVvYj1YXYeJfyrYZvippOQN3ekQHC+LkfXxbLc=; b=qZpclqQ6MsBQZUyMbRHl5nEFHLKVNuncCaq+7Pxg2KT6XNS9xA08pBlGHG3JYut1v1 TRvpdM8IY5fuCJoqsdbv+sWXvtCe0pTC8mj6QL8JS54ZIaAga8BWb6ikSMZwyF8wo1Mp 1IzskPITy3h8+PbYnC19RDCtSEjS9+naLaqxPL38hvyOcW+Wjjnkp5Vu6l4DRpZa2MVw /qLkeHfR5fxAiGbQ3ASQz7tXDU/Ojm5437GZxvJMzZGw2uZgFS6bO7kXNuCOzB8SUj8u ve7L7kOPTIzBNV8TWMg8uLMQMD4aJ+E2OFqj7Rzu0cUj+dVwgYL7lld5b1V4BmU78331 S9yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:from:subject:references :mime-version:message-id:in-reply-to:date:x-gm-message-state:from:to :cc:subject:date; bh=muitaXVvYj1YXYeJfyrYZvippOQN3ekQHC+LkfXxbLc=; b=7FWjyDmivPFtZKaD9R9E8QmzYUWjN+mYHes5A4rWBlVzN/Y874Ix2bK8JOysaIDI9s b/cpkrd0UeTsANPoXbVZe+Xm+OS45O2MuClpjz9JCg8QiVaIslVpCPsuTqxvTaX9HDSb pQRuOWmMXmBsGZqyEQ06gpSlheRAokUQMaTrOpC2gA1KqJAqEA0OAaZUW8pKd2mYj1d4 hPj+7ypo8rdumZbarlq0P4X9aHHBhtFphCotibYwNQ0EGuu5mzfSRMqQjBd4eH/MRlm7 qBA74l2inWgpS+X9vYjbPPIhI66Dxu6cdl1tnyKtwrvrYDCe5FViompLbNweRZsFOtjO h+Uw== X-Gm-Message-State: ACgBeo0POC4LkMnbRe14D/CJ1lbzPM7qk+jLJaS7ILyPlgEpgVRkKzAd MCnbU+DS/JzFDTvOY7csNz5s0KiH6duxGZ8t4wA= X-Google-Smtp-Source: AA6agR5mmw1lyTMjEbXO+bj0XMfoKQXjrA7x+3+LfFFUJRrRakcxIdBjBJC8pQKjP8yP5Cacc+4B59TMcKFo3CTBMQ8= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:54d9:7143:6a7d:91d]) (user=samitolvanen job=sendgmr) by 2002:a25:2c8:0:b0:695:feaa:871f with SMTP id 191-20020a2502c8000000b00695feaa871fmr14222519ybc.471.1661902350818; Tue, 30 Aug 2022 16:32:30 -0700 (PDT) Date: Tue, 30 Aug 2022 16:31:29 -0700 In-Reply-To: <20220830233129.30610-1-samitolvanen@google.com> Message-Id: <20220830233129.30610-22-samitolvanen@google.com> Mime-Version: 1.0 References: <20220830233129.30610-1-samitolvanen@google.com> X-Developer-Key: i=samitolvanen@google.com; a=openpgp; fpr=35CCFB63B283D6D3AEB783944CB5F6848BBC56EE X-Developer-Signature: v=1; a=openpgp-sha256; l=10374; i=samitolvanen@google.com; h=from:subject; bh=xx8wjcYE8Du0racyo2HoO5M6w+PYgPKqDWnfLGl3DjU=; b=owEB7QES/pANAwAKAUy19oSLvFbuAcsmYgBjDp3PPFzSBxPnL1McANFd1sNHrlLrdWSxt2cbevI6 OTWO0UqJAbMEAAEKAB0WIQQ1zPtjsoPW0663g5RMtfaEi7xW7gUCYw6dzwAKCRBMtfaEi7xW7uukC/ 9uM+7HvXyQ67VflXgAsC/YncRdBoq4yhl8rYfT2i05OWqPjoWHJG8Ya/UI5iY6oAeiaG49tfhTVdrJ pUyR3viQo7OIVPeVuzfeHP0eguun9EReYSQcKQGYAV4XfwQ4hFEKT254SvMUfU/M/3EWaS7HN4mNz4 QJK4XkW4SPhA7RBnDLN3EQJPwljXDQ7+t5wNksrguQOfmbffYUJPcP/HC8oNr1lIRstWm3BrwbIJaD MAsUDvpax1bIHiYl6DHoiFijKWm88sHuPFQFFnxGE0U4g/pm+dzy+wirAFnCNNK/RGdmP3fdBqzLSI Ul3ndBsIwZACrYvIZ37KXR82etUCnuTGSvqwWPAnY4rizeCV8SEBvqmVV2EWIAClYLSj6ICZQJRyeX rzdvbADWzQdLSvLP7rGWgi5MQSYwwYbc15tpgtuhFe0HwpsWbo8jxfrJUVzM4T5CXHBy3OqY/UZZrp 0/EHk1+pMYKAO0vzlUhH3g28B4J9GI5syiuW13VGIgv/E= X-Mailer: git-send-email 2.37.2.672.g94769d06f0-goog Subject: [PATCH v4 21/21] x86: Add support for CONFIG_CFI_CLANG From: Sami Tolvanen To: linux-kernel@vger.kernel.org Cc: Kees Cook , Josh Poimboeuf , Peter Zijlstra , x86@kernel.org, Catalin Marinas , Will Deacon , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Joao Moreira , Sedat Dilek , Steven Rostedt , linux-hardening@vger.kernel.org, linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220830_163232_670272_E5A6790D X-CRM114-Status: GOOD ( 28.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With CONFIG_CFI_CLANG, the compiler injects a type preamble immediately before each function and a check to validate the target function type before indirect calls: ; type preamble __cfi_function: mov , %eax function: ... ; indirect call check mov -,%r10d add -0x4(%r11),%r10d je .Ltmp1 ud2 .Ltmp1: call __x86_indirect_thunk_r11 Define the __CFI_TYPE helper macro for manually annotating indirectly called assembly function with the identical premable, add error handling code for the ud2 traps emitted for indirect call checks, and allow CONFIG_CFI_CLANG to be selected on x86_64. This produces the following oops on CFI failure (generated using lkdtm): [ 21.441706] CFI failure at lkdtm_indirect_call+0x16/0x20 [lkdtm] (target: lkdtm_increment_int+0x0/0x10 [lkdtm]; expected type: 0x7e0c52a) [ 21.444579] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI [ 21.445296] CPU: 0 PID: 132 Comm: sh Not tainted 5.19.0-rc8-00020-g9f27360e674c #1 [ 21.445296] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 [ 21.445296] RIP: 0010:lkdtm_indirect_call+0x16/0x20 [lkdtm] [ 21.445296] Code: 52 1c c0 48 c7 c1 c5 50 1c c0 e9 25 48 2a cc 0f 1f 44 00 00 49 89 fb 48 c7 c7 50 b4 1c c0 41 ba 5b ad f3 81 45 03 53 f8 [ 21.445296] RSP: 0018:ffffa9f9c02ffdc0 EFLAGS: 00000292 [ 21.445296] RAX: 0000000000000027 RBX: ffffffffc01cb300 RCX: 385cbbd2e070a700 [ 21.445296] RDX: 0000000000000000 RSI: c0000000ffffdfff RDI: ffffffffc01cb450 [ 21.445296] RBP: 0000000000000006 R08: 0000000000000000 R09: ffffffff8d081610 [ 21.445296] R10: 00000000bcc90825 R11: ffffffffc01c2fc0 R12: 0000000000000000 [ 21.445296] R13: ffffa31b827a6000 R14: 0000000000000000 R15: 0000000000000002 [ 21.445296] FS: 00007f08b42216a0(0000) GS:ffffa31b9f400000(0000) knlGS:0000000000000000 [ 21.445296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 21.445296] CR2: 0000000000c76678 CR3: 0000000001940000 CR4: 00000000000006f0 [ 21.445296] Call Trace: [ 21.445296] [ 21.445296] lkdtm_CFI_FORWARD_PROTO+0x30/0x50 [lkdtm] [ 21.445296] direct_entry+0x12d/0x140 [lkdtm] [ 21.445296] full_proxy_write+0x5d/0xb0 [ 21.445296] vfs_write+0x144/0x460 [ 21.445296] ? __x64_sys_wait4+0x5a/0xc0 [ 21.445296] ksys_write+0x69/0xd0 [ 21.445296] do_syscall_64+0x51/0xa0 [ 21.445296] entry_SYSCALL_64_after_hwframe+0x63/0xcd [ 21.445296] RIP: 0033:0x7f08b41a6fe1 [ 21.445296] Code: be 07 00 00 00 41 89 c0 e8 7e ff ff ff 44 89 c7 89 04 24 e8 91 c6 02 00 8b 04 24 48 83 c4 68 c3 48 63 ff b8 01 00 00 03 [ 21.445296] RSP: 002b:00007ffcdf65c2e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [ 21.445296] RAX: ffffffffffffffda RBX: 00007f08b4221690 RCX: 00007f08b41a6fe1 [ 21.445296] RDX: 0000000000000012 RSI: 0000000000c738f0 RDI: 0000000000000001 [ 21.445296] RBP: 0000000000000001 R08: fefefefefefefeff R09: fefefefeffc5ff4e [ 21.445296] R10: 00007f08b42222b0 R11: 0000000000000246 R12: 0000000000c738f0 [ 21.445296] R13: 0000000000000012 R14: 00007ffcdf65c401 R15: 0000000000c70450 [ 21.445296] [ 21.445296] Modules linked in: lkdtm [ 21.445296] Dumping ftrace buffer: [ 21.445296] (ftrace buffer empty) [ 21.471442] ---[ end trace 0000000000000000 ]--- [ 21.471811] RIP: 0010:lkdtm_indirect_call+0x16/0x20 [lkdtm] [ 21.472467] Code: 52 1c c0 48 c7 c1 c5 50 1c c0 e9 25 48 2a cc 0f 1f 44 00 00 49 89 fb 48 c7 c7 50 b4 1c c0 41 ba 5b ad f3 81 45 03 53 f8 [ 21.474400] RSP: 0018:ffffa9f9c02ffdc0 EFLAGS: 00000292 [ 21.474735] RAX: 0000000000000027 RBX: ffffffffc01cb300 RCX: 385cbbd2e070a700 [ 21.475664] RDX: 0000000000000000 RSI: c0000000ffffdfff RDI: ffffffffc01cb450 [ 21.476471] RBP: 0000000000000006 R08: 0000000000000000 R09: ffffffff8d081610 [ 21.477127] R10: 00000000bcc90825 R11: ffffffffc01c2fc0 R12: 0000000000000000 [ 21.477959] R13: ffffa31b827a6000 R14: 0000000000000000 R15: 0000000000000002 [ 21.478657] FS: 00007f08b42216a0(0000) GS:ffffa31b9f400000(0000) knlGS:0000000000000000 [ 21.479577] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 21.480307] CR2: 0000000000c76678 CR3: 0000000001940000 CR4: 00000000000006f0 [ 21.481460] Kernel panic - not syncing: Fatal exception Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- arch/x86/Kconfig | 2 + arch/x86/include/asm/cfi.h | 22 +++++++++ arch/x86/include/asm/linkage.h | 9 ++++ arch/x86/kernel/Makefile | 2 + arch/x86/kernel/cfi.c | 85 ++++++++++++++++++++++++++++++++++ arch/x86/kernel/traps.c | 4 +- 6 files changed, 123 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/asm/cfi.h create mode 100644 arch/x86/kernel/cfi.c diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f9920f1341c8..1fe6a83dac05 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -107,6 +107,8 @@ config X86 select ARCH_SUPPORTS_PAGE_TABLE_CHECK if X86_64 select ARCH_SUPPORTS_NUMA_BALANCING if X86_64 select ARCH_SUPPORTS_KMAP_LOCAL_FORCE_MAP if NR_CPUS <= 4096 + select ARCH_SUPPORTS_CFI_CLANG if X86_64 + select ARCH_USES_CFI_TRAPS if X86_64 && CFI_CLANG select ARCH_SUPPORTS_LTO_CLANG select ARCH_SUPPORTS_LTO_CLANG_THIN select ARCH_USE_BUILTIN_BSWAP diff --git a/arch/x86/include/asm/cfi.h b/arch/x86/include/asm/cfi.h new file mode 100644 index 000000000000..58dacd90daef --- /dev/null +++ b/arch/x86/include/asm/cfi.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_CFI_H +#define _ASM_X86_CFI_H + +/* + * Clang Control Flow Integrity (CFI) support. + * + * Copyright (C) 2022 Google LLC + */ + +#include + +#ifdef CONFIG_CFI_CLANG +enum bug_trap_type handle_cfi_failure(struct pt_regs *regs); +#else +static inline enum bug_trap_type handle_cfi_failure(struct pt_regs *regs) +{ + return BUG_TRAP_TYPE_NONE; +} +#endif /* CONFIG_CFI_CLANG */ + +#endif /* _ASM_X86_CFI_H */ diff --git a/arch/x86/include/asm/linkage.h b/arch/x86/include/asm/linkage.h index 73ca20049835..ebfa05973312 100644 --- a/arch/x86/include/asm/linkage.h +++ b/arch/x86/include/asm/linkage.h @@ -29,6 +29,15 @@ #endif #endif /* CONFIG_RETPOLINE */ +#ifdef CONFIG_CFI_CLANG +#define __CFI_TYPE(name) \ + SYM_START(__cfi_##name, SYM_L_LOCAL, SYM_A_NONE) \ + .fill 11, 1, 0x90 ASM_NL \ + .byte 0xb8 ASM_NL \ + .long __kcfi_typeid_##name ASM_NL \ + SYM_FUNC_END(__cfi_##name) +#endif + #else /* __ASSEMBLY__ */ #if defined(CONFIG_RETHUNK) && !defined(__DISABLE_EXPORTS) && !defined(BUILD_VDSO) diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index a20a5ebfacd7..1286a73ebdbc 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -139,6 +139,8 @@ obj-$(CONFIG_UNWINDER_GUESS) += unwind_guess.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += sev.o +obj-$(CONFIG_CFI_CLANG) += cfi.o + ### # 64 bit specific files ifeq ($(CONFIG_X86_64),y) diff --git a/arch/x86/kernel/cfi.c b/arch/x86/kernel/cfi.c new file mode 100644 index 000000000000..e642c1af2678 --- /dev/null +++ b/arch/x86/kernel/cfi.c @@ -0,0 +1,85 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Clang Control Flow Integrity (CFI) support. + * + * Copyright (C) 2022 Google LLC + */ +#include +#include +#include + +/* + * Returns the target address and the expected type when regs->ip points + * to a compiler-generated CFI trap. + */ +static bool decode_cfi_insn(struct pt_regs *regs, unsigned long *target, + u32 *type) +{ + char buffer[MAX_INSN_SIZE]; + struct insn insn; + int offset = 0; + + *target = *type = 0; + + /* + * The compiler generates the following instruction sequence + * for indirect call checks: + * + *   movl -, %r10d ; 6 bytes + * addl -4(%reg), %r10d ; 4 bytes + * je .Ltmp1 ; 2 bytes + * ud2 ; <- regs->ip + * .Ltmp1: + * + * We can decode the expected type and the target address from the + * movl/addl instructions. + */ + if (copy_from_kernel_nofault(buffer, (void *)regs->ip - 12, MAX_INSN_SIZE)) + return false; + if (insn_decode_kernel(&insn, &buffer[offset])) + return false; + if (insn.opcode.value != 0xBA) + return false; + + *type = -(u32)insn.immediate.value; + + if (copy_from_kernel_nofault(buffer, (void *)regs->ip - 6, MAX_INSN_SIZE)) + return false; + if (insn_decode_kernel(&insn, &buffer[offset])) + return false; + if (insn.opcode.value != 0x3) + return false; + + /* Read the target address from the register. */ + offset = insn_get_modrm_rm_off(&insn, regs); + if (offset < 0) + return false; + + *target = *(unsigned long *)((void *)regs + offset); + + return true; +} + +/* + * Checks if a ud2 trap is because of a CFI failure, and handles the trap + * if needed. Returns a bug_trap_type value similarly to report_bug. + */ +enum bug_trap_type handle_cfi_failure(struct pt_regs *regs) +{ + unsigned long target; + u32 type; + + if (!is_cfi_trap(regs->ip)) + return BUG_TRAP_TYPE_NONE; + + if (!decode_cfi_insn(regs, &target, &type)) + return report_cfi_failure_noaddr(regs, regs->ip); + + return report_cfi_failure(regs, regs->ip, &target, type); +} + +/* + * Ensure that __kcfi_typeid_ symbols are emitted for functions that may + * not be indirectly called with all configurations. + */ +__ADDRESSABLE(memcpy) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index d62b2cb85cea..178015a820f0 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -63,6 +63,7 @@ #include #include #include +#include #ifdef CONFIG_X86_64 #include @@ -313,7 +314,8 @@ static noinstr bool handle_bug(struct pt_regs *regs) */ if (regs->flags & X86_EFLAGS_IF) raw_local_irq_enable(); - if (report_bug(regs->ip, regs) == BUG_TRAP_TYPE_WARN) { + if (report_bug(regs->ip, regs) == BUG_TRAP_TYPE_WARN || + handle_cfi_failure(regs) == BUG_TRAP_TYPE_WARN) { regs->ip += LEN_UD2; handled = true; }