From patchwork Thu Feb 22 19:14:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Borislav Petkov X-Patchwork-Id: 10236193 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4096F602DC for ; Thu, 22 Feb 2018 19:14:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3836528649 for ; Thu, 22 Feb 2018 19:14:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2B5B728C46; Thu, 22 Feb 2018 19:14:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 0D7B828649 for ; Thu, 22 Feb 2018 19:14:53 +0000 (UTC) Received: (qmail 28133 invoked by uid 550); 22 Feb 2018 19:14:51 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 28111 invoked from network); 22 Feb 2018 19:14:50 -0000 X-Virus-Scanned: Nedap ESD1 at mail.skyhub.de Date: Thu, 22 Feb 2018 20:14:23 +0100 From: Borislav Petkov To: Alexander Popov Cc: kernel-hardening@lists.openwall.com, Kees Cook , PaX Team , Brad Spengler , Ingo Molnar , Andy Lutomirski , Tycho Andersen , Laura Abbott , Mark Rutland , Ard Biesheuvel , Thomas Gleixner , "H . Peter Anvin" , Peter Zijlstra , "Dmitry V . Levin" , x86@kernel.org Subject: Re: [PATCH RFC v8 1/6] x86/entry: Add STACKLEAK erasing the kernel stack at the end of syscalls Message-ID: <20180222191423.GA27395@pd.tnic> References: <1518804657-24905-1-git-send-email-alex.popov@linux.com> <1518804657-24905-2-git-send-email-alex.popov@linux.com> <20180221132443.GA9989@pd.tnic> <1e30ffd0-b35c-fcc2-ec8f-4495aefa7f6f@linux.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <1e30ffd0-b35c-fcc2-ec8f-4495aefa7f6f@linux.com> User-Agent: Mutt/1.9.3 (2018-01-21) X-Virus-Scanned: ClamAV using ClamSMTP On Thu, Feb 22, 2018 at 12:49:44AM +0300, Alexander Popov wrote: > Actually gcc creates a strange erase_kstack(): > > ffffffff81a00010 : > ffffffff81a00010: 5f pop %rdi > ffffffff81a00011: eb 31 jmp ffffffff81a00044 > ffffffff81a00013: 0f 1f 00 nopl (%rax) > ffffffff81a00016: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1) > ffffffff81a0001d: 00 00 00 Something's very strange here. That code looks like: ENTRY(entry_SYSCALL_64_stage2) UNWIND_HINT_EMPTY popq %rdi jmp entry_SYSCALL_64_after_hwframe END(entry_SYSCALL_64_stage2) ENTRY(entry_SYSCALL_64) ... so frankly I wonder what gcc is even doing here?!? And why isn't that erase_kstack symbol completely empty. It gives it that address ffffffff81a00010 and size and this looks really wrong. /me experiments a bit more, notices the ENDPROC()... Ok, I think I know what it is: END(erase_kstack) gives .globl erase_kstack ; .p2align 4, 0x90 ; erase_kstack: # 167 "arch/x86/entry/entry_64.S" .size erase_kstack, .-erase_kstack # 182 "arch/x86/entry/entry_64.S" and that generates a STT_NOTYPE symbol: 67318: ffffffff81a00000 0 NOTYPE GLOBAL DEFAULT 1 erase_kstack In that case, the symbol gets ignored by objdump as it dumps this at that address: ffffffff81a00000 : ffffffff81a00000: 5f pop %rdi ffffffff81a00001: eb 31 jmp ffffffff81a00034 ffffffff81a00003: 0f 1f 00 nopl (%rax) ffffffff81a00006: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1) ffffffff81a0000d: 00 00 00 which is basically the strange erase_kstack() variant you pasted above. while ENDPROC(erase_kstack) gives .globl erase_kstack ; .p2align 4, 0x90 ; erase_kstack: # 167 "arch/x86/entry/entry_64.S" .type erase_kstack, @function ; .size erase_kstack, .-erase_kstack # 182 "arch/x86/entry/entry_64.S" and that generates a STT_FUNC: 67318: ffffffff81a00000 0 FUNC GLOBAL DEFAULT 1 erase_kstack Now, there's this comment over ENDPROC: /* If symbol 'name' is treated as a subroutine (gets called, and returns) * then please use ENDPROC to mark 'name' as STT_FUNC for the benefit of * static analysis tools such as stack depth analyzer. */ #ifndef ENDPROC and I don't think we want to analyze the stack depth of erase_kstack itself. However, even if we did END(erase_kstack), the calls are still in the code: ffffffff81a00111: e8 ea fe ff ff callq ffffffff81a00000 so macro it is. But please call the macro something else, not the same name as the function. > The commit message says: "Full STACKLEAK feature also contains the gcc plugin > which comes in a separate commit". > > And the next commit in this series introduces that plugin. Let me quote its > commit message as well: ... > Tracking the lowest border of the kernel stack with the lowest_stack > variable makes STACKLEAK so efficient (please see the performance > statistics in the cover letter). Aha, there it is. I guess I better continue looking through the patchset. :) > The mm.txt already has this line: > ffffffffffe00000 - ffffffffffffffff (=2 MB) unused hole > > Excuse me, I didn't get what to document. You say /* Poison value points to the unused hole in the virtual memory map */ but we do change that memory map from time to time and there are multiple unused holes. So do something like this so that there are no clashes when someone decides to use that unused hole: --- --- Thx. diff --git a/Documentation/x86/x86_64/mm.txt b/Documentation/x86/x86_64/mm.txt index ea91cb61a602..5d8f4168247d 100644 --- a/Documentation/x86/x86_64/mm.txt +++ b/Documentation/x86/x86_64/mm.txt @@ -24,6 +24,7 @@ ffffffffa0000000 - [fixmap start] (~1526 MB) module mapping space (variable) [fixmap start] - ffffffffff5fffff kernel-internal fixmap range ffffffffff600000 - ffffffffff600fff (=4 kB) legacy vsyscall ABI ffffffffffe00000 - ffffffffffffffff (=2 MB) unused hole +Stackleak poison value in this last hole: 0xffffffffffff4111 Virtual memory map with 5 level page tables: @@ -50,6 +51,7 @@ ffffffffa0000000 - fffffffffeffffff (1520 MB) module mapping space [fixmap start] - ffffffffff5fffff kernel-internal fixmap range ffffffffff600000 - ffffffffff600fff (=4 kB) legacy vsyscall ABI ffffffffffe00000 - ffffffffffffffff (=2 MB) unused hole +Stackleak poison value in this last hole: 0xffffffffffff4111 Architecture defines a 64-bit virtual address. Implementations can support less. Currently supported are 48- and 57-bit virtual addresses. Bits 63