From patchwork Thu Jan 25 18:48:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Torvalds X-Patchwork-Id: 10184641 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id BAFCC60388 for ; Thu, 25 Jan 2018 18:48:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AA3A5288D1 for ; Thu, 25 Jan 2018 18:48:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9E37828901; Thu, 25 Jan 2018 18:48:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 6B36A288D1 for ; Thu, 25 Jan 2018 18:48:19 +0000 (UTC) Received: (qmail 30596 invoked by uid 550); 25 Jan 2018 18:48:17 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 30553 invoked from network); 25 Jan 2018 18:48:16 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=ja7814FZmhBAIUt+Hi1vGDV+RkBkCdJgg/NFcWKirUY=; b=ibX8+xAppus0TCLjv81CNzQgt63okD6EXDoe3benkKpb2uGJySWST9O1AioeNCtrP4 rcDmtdvZZwlRFpxnilyVMK3fGSR8mhLU9inEPfyAhQ8zVXjB/v/lVcjrGUARcFdjaDCE hDFPPhOacoAGFhZl0i4qiAhrmIWWbWa889FQsCVV1n/wcsxAEq6lFInRDiW1ejjCdOe9 N1Exl+E5K5InzsVguEDG9GB5l25uNl25TknKz+z2kRaBgvpsbn35WkVKQiRjNzghoIn1 YdCFuXVNUbpw255oVss9PKFGx3JezNgPYjiYn3RIsq+WmtY/LvjO5jvEvr73CCUz0oyS 7wDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=ja7814FZmhBAIUt+Hi1vGDV+RkBkCdJgg/NFcWKirUY=; b=rHzNIYs/+NzyPFQoC0AM4iJxc4Ps7q05rz3U5lH6sRh25CMLqgZgSx0qmPTBKxI+g9 84JrRLpDjOcIoprcXXD4KzNH4BvAkKjMYuiUK4mgfB+p9FCjOfyVH68f14xQlDFAIitV rN6z8pIMSTjb0PYRNOLrjqcAco4eL6rmY3vqXaHib0iGCVDr6L+PN5vKsHBNLygdjnyj icAkPJRR4vvUZereAzukBfQuOJh7jwUPAQNcFwQHJYxOuJfUlPB6olcgEF8MTpMxW4zg 2pYsFOL2LZpHaNTxx8G70t286p+amA4diqC9Sk1HwjmiBjonwU6IYvRX0JVZdflR746b d6tA== X-Gm-Message-State: AKwxytf+XybwKoIpAbEKlZNc0Wthcy8JrEky71sU3ZG96oFm4XcFmsOL miP8VldutiM7lUx2O2eB77tYLm9g8kd098Hh5+A= X-Google-Smtp-Source: AH8x224urDKThR6QV+zqRnHCCVYEKOqoONT7/j0GXBm70hrTAjJV13zniaFYZNdKo0q4HXBqMXtPWBbQqPoK4fxaACQ= X-Received: by 10.36.47.5 with SMTP id j5mr13642902itj.123.1516906082531; Thu, 25 Jan 2018 10:48:02 -0800 (PST) MIME-Version: 1.0 Sender: linus971@gmail.com In-Reply-To: References: <503224b776b9513885453756e44bab235221124e.1516644136.git.luto@kernel.org> From: Linus Torvalds Date: Thu, 25 Jan 2018 10:48:01 -0800 X-Google-Sender-Auth: InxVL09lJj-RDSuz3gAyGH9ZsUo Message-ID: To: Andy Lutomirski Cc: "the arch/x86 maintainers" , LKML , Greg Kroah-Hartman , Alan Cox , Jann Horn , Samuel Neves , Dan Williams , Kernel Hardening , Borislav Petkov Subject: [kernel-hardening] Re: [PATCH] x86/retpoline/entry: Disable the entire SYSCALL64 fast path with retpolines on X-Virus-Scanned: ClamAV using ClamSMTP On Mon, Jan 22, 2018 at 10:55 AM, Linus Torvalds wrote: > > Honestly, I'd rather get rid of the fast-path entirely. Compared to > all the PTI mess, it's not even noticeable. So I looked at how that would be. Patch attached. Not really "tested", but I'm running the kernel with this patch now, and 'strace' etc works, and honestly, it seems very obvious. Also, code generation for 'do_syscall_64()' does not look horrible. In fact, it doesn't look all that much worse than the fast-path ever did. So the biggest impact of this is the extra register saves (SAVE_EXTRA_REGS) from setting up the full ptregs. And honestly, I hate how that stupid macro still uses "movq reg,off(%rsp)" instead of "pushq %reg". Considering the diffstat: 2 files changed, 2 insertions(+), 121 deletions(-) and how those 100+ lines are nasty assembly code, I do think we should just do it. Comments? Linus arch/x86/entry/entry_64.S | 116 -------------------------------------------- arch/x86/entry/syscall_64.c | 7 +-- 2 files changed, 2 insertions(+), 121 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index ff6f8022612c..65f4d29be69b 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -241,80 +241,6 @@ GLOBAL(entry_SYSCALL_64_after_hwframe) TRACE_IRQS_OFF - /* - * If we need to do entry work or if we guess we'll need to do - * exit work, go straight to the slow path. - */ - movq PER_CPU_VAR(current_task), %r11 - testl $_TIF_WORK_SYSCALL_ENTRY|_TIF_ALLWORK_MASK, TASK_TI_flags(%r11) - jnz entry_SYSCALL64_slow_path - -entry_SYSCALL_64_fastpath: - /* - * Easy case: enable interrupts and issue the syscall. If the syscall - * needs pt_regs, we'll call a stub that disables interrupts again - * and jumps to the slow path. - */ - TRACE_IRQS_ON - ENABLE_INTERRUPTS(CLBR_NONE) -#if __SYSCALL_MASK == ~0 - cmpq $__NR_syscall_max, %rax -#else - andl $__SYSCALL_MASK, %eax - cmpl $__NR_syscall_max, %eax -#endif - ja 1f /* return -ENOSYS (already in pt_regs->ax) */ - movq %r10, %rcx - - /* - * This call instruction is handled specially in stub_ptregs_64. - * It might end up jumping to the slow path. If it jumps, RAX - * and all argument registers are clobbered. - */ -#ifdef CONFIG_RETPOLINE - movq sys_call_table(, %rax, 8), %rax - call __x86_indirect_thunk_rax -#else - call *sys_call_table(, %rax, 8) -#endif -.Lentry_SYSCALL_64_after_fastpath_call: - - movq %rax, RAX(%rsp) -1: - - /* - * If we get here, then we know that pt_regs is clean for SYSRET64. - * If we see that no exit work is required (which we are required - * to check with IRQs off), then we can go straight to SYSRET64. - */ - DISABLE_INTERRUPTS(CLBR_ANY) - TRACE_IRQS_OFF - movq PER_CPU_VAR(current_task), %r11 - testl $_TIF_ALLWORK_MASK, TASK_TI_flags(%r11) - jnz 1f - - LOCKDEP_SYS_EXIT - TRACE_IRQS_ON /* user mode is traced as IRQs on */ - movq RIP(%rsp), %rcx - movq EFLAGS(%rsp), %r11 - addq $6*8, %rsp /* skip extra regs -- they were preserved */ - UNWIND_HINT_EMPTY - jmp .Lpop_c_regs_except_rcx_r11_and_sysret - -1: - /* - * The fast path looked good when we started, but something changed - * along the way and we need to switch to the slow path. Calling - * raise(3) will trigger this, for example. IRQs are off. - */ - TRACE_IRQS_ON - ENABLE_INTERRUPTS(CLBR_ANY) - SAVE_EXTRA_REGS - movq %rsp, %rdi - call syscall_return_slowpath /* returns with IRQs disabled */ - jmp return_from_SYSCALL_64 - -entry_SYSCALL64_slow_path: /* IRQs are off. */ SAVE_EXTRA_REGS movq %rsp, %rdi @@ -393,7 +319,6 @@ syscall_return_via_sysret: /* rcx and r11 are already restored (see code above) */ UNWIND_HINT_EMPTY POP_EXTRA_REGS -.Lpop_c_regs_except_rcx_r11_and_sysret: popq %rsi /* skip r11 */ popq %r10 popq %r9 @@ -424,47 +349,6 @@ syscall_return_via_sysret: USERGS_SYSRET64 END(entry_SYSCALL_64) -ENTRY(stub_ptregs_64) - /* - * Syscalls marked as needing ptregs land here. - * If we are on the fast path, we need to save the extra regs, - * which we achieve by trying again on the slow path. If we are on - * the slow path, the extra regs are already saved. - * - * RAX stores a pointer to the C function implementing the syscall. - * IRQs are on. - */ - cmpq $.Lentry_SYSCALL_64_after_fastpath_call, (%rsp) - jne 1f - - /* - * Called from fast path -- disable IRQs again, pop return address - * and jump to slow path - */ - DISABLE_INTERRUPTS(CLBR_ANY) - TRACE_IRQS_OFF - popq %rax - UNWIND_HINT_REGS extra=0 - jmp entry_SYSCALL64_slow_path - -1: - JMP_NOSPEC %rax /* Called from C */ -END(stub_ptregs_64) - -.macro ptregs_stub func -ENTRY(ptregs_\func) - UNWIND_HINT_FUNC - leaq \func(%rip), %rax - jmp stub_ptregs_64 -END(ptregs_\func) -.endm - -/* Instantiate ptregs_stub for each ptregs-using syscall */ -#define __SYSCALL_64_QUAL_(sym) -#define __SYSCALL_64_QUAL_ptregs(sym) ptregs_stub sym -#define __SYSCALL_64(nr, sym, qual) __SYSCALL_64_QUAL_##qual(sym) -#include - /* * %rdi: prev task * %rsi: next task diff --git a/arch/x86/entry/syscall_64.c b/arch/x86/entry/syscall_64.c index 9c09775e589d..c176d2fab1da 100644 --- a/arch/x86/entry/syscall_64.c +++ b/arch/x86/entry/syscall_64.c @@ -7,14 +7,11 @@ #include #include -#define __SYSCALL_64_QUAL_(sym) sym -#define __SYSCALL_64_QUAL_ptregs(sym) ptregs_##sym - -#define __SYSCALL_64(nr, sym, qual) extern asmlinkage long __SYSCALL_64_QUAL_##qual(sym)(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long); +#define __SYSCALL_64(nr, sym, qual) extern asmlinkage long sym(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long); #include #undef __SYSCALL_64 -#define __SYSCALL_64(nr, sym, qual) [nr] = __SYSCALL_64_QUAL_##qual(sym), +#define __SYSCALL_64(nr, sym, qual) [nr] = sym, extern long sys_ni_syscall(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long);