From patchwork Wed Oct 24 12:51:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A . Shutemov" X-Patchwork-Id: 10654623 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8291713BF for ; Wed, 24 Oct 2018 12:51:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 707D32A77F for ; Wed, 24 Oct 2018 12:51:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6E1512A762; Wed, 24 Oct 2018 12:51:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 739612A7EB for ; Wed, 24 Oct 2018 12:51:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 154536B028A; Wed, 24 Oct 2018 08:51:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 104EC6B028B; Wed, 24 Oct 2018 08:51:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E98736B028D; Wed, 24 Oct 2018 08:51:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id 823646B028A for ; Wed, 24 Oct 2018 08:51:33 -0400 (EDT) Received: by mail-pl1-f199.google.com with SMTP id g7-v6so2546878plb.5 for ; Wed, 24 Oct 2018 05:51:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=AYUuyP9PdaBVHaAxknChsL0Vk2q3XWwxuZ+Oqe+taS8=; b=DyseVkUfNRhjEXHmwtIOVHtkJgzpJNKkyt/lC9KuEizCX8LUeT1ZMoUPZwYwObhlbG 5dXjfFM7hLY8C7diYh5m689OCoPgS+NGpxCnNzFPacz2grkCnjJsG3fuFeS43SfYONxw DG6i6w8wwE6tOau9oHGhFQJWzPNCkUZOZcD7TZEeFAKWdVSJ83Gn9Ohk0zQWeBX+9OHY xDIpAs7AJkooSS/OSec1VG97lpG7Q/YD+5eXDhzUcJE6Azt0nitZef6VxS3D7F97MSKB YUj0MGqhMrco2j4K4W9A7YOaMS3U5WNiuCloujkMhodp7tLctr67Xc+z3yn6NFJLeSuW +V0g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AGRZ1gKN1G/QDFlTh9ctt2fikt+JuZlb8oEBUPaqxD1MpNugf/3tv/wG EVyRGKmur2sawBHseUR0irLY1mxE4zZkWT7sCbb8hE1gphJdcXvT+isGwh//kdgRu+LidoJtQEH +uHkvUGLM9bhj3QiL2AMmzMAeL8wsd0Wvrx9SHdz44DF6d8CuO/ESELR21cEYpNlZmA== X-Received: by 2002:a62:5793:: with SMTP id i19-v6mr1088247pfj.158.1540385493168; Wed, 24 Oct 2018 05:51:33 -0700 (PDT) X-Google-Smtp-Source: AJdET5fPjk5IN7RYptNuaK3MHX4SaQhxE6CN7eIUzJc9QtvNdBxHDEnj+cr1ZTKz0iZA8lili2tq X-Received: by 2002:a62:5793:: with SMTP id i19-v6mr1088174pfj.158.1540385491880; Wed, 24 Oct 2018 05:51:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540385491; cv=none; d=google.com; s=arc-20160816; b=BKKmDPbi7xDm4P8MZ8kBNacDyjllCJ4lU+OCxf68QS0txm0MrOEWVtX9+S1EV8dDWl T1MvlCSaGQQf2ly1JpV26T9vr3JFbvESzq6r7KFLxnKVk5rD6pMrT+CXns4OyK3+yfNj t9HUNlKSuAz2BgXzNtBMle8No/g4/vWzXmaUSvG5oSGNkPwtawWVAcM6jnNHJbPHefhx lUeoLB2ZQ7gzh7NhzVlfY//at47APAECTtiD7iy+GVz9mczM01F/IltsBXcbpf5auA29 NE0puzGFvqF3Aj1eCF/qlyGnVJWKYC0Ohrw95kaVLfVTMPZaXYGbl21xPJu0LB/QdrxP Dg9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=AYUuyP9PdaBVHaAxknChsL0Vk2q3XWwxuZ+Oqe+taS8=; b=e5uAYJOMgL49aqMN4GlvK6FkzArPBNV2VODYOm/plEpK/lapoM0pnhlZQ9QFxwu4pY 4nW2EejljzuJZNE285oOmB5BbFItZM5mhw3yPhhxPxx58+JRbyzfKCrpuSKmPz0JXa3U hP0dMd15KuoQNbXIBx9Bh1g+6L2qKuvQtaaBr6JuauszUdOpKZG7QFfWDlki7ihJsF0+ /l+Fq7IqcwiIFZ3YMAl1HYyzZLJuxra1Vpx9kKfT9W/42nYFWzK8JrWKdkIUpbZ77col BUfeKq+bLewFnxjiPAZVMFKOwP1jajQXOprzPiWo4+CxGUaGC23NLbo4U4oFvjiHFxk8 ATQQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga05.intel.com (mga05.intel.com. [192.55.52.43]) by mx.google.com with ESMTPS id f2-v6si4429309plr.153.2018.10.24.05.51.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Oct 2018 05:51:31 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.43 as permitted sender) client-ip=192.55.52.43; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Oct 2018 05:51:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,420,1534834800"; d="scan'208";a="81221311" Received: from black.fi.intel.com ([10.237.72.28]) by fmsmga008.fm.intel.com with ESMTP; 24 Oct 2018 05:51:27 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id B39F17D; Wed, 24 Oct 2018 15:51:26 +0300 (EEST) From: "Kirill A. Shutemov" To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org Cc: boris.ostrovsky@oracle.com, jgross@suse.com, bhe@redhat.com, willy@infradead.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCHv2 1/2] x86/mm: Move LDT remap out of KASLR region on 5-level paging Date: Wed, 24 Oct 2018 15:51:11 +0300 Message-Id: <20181024125112.55999-2-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181024125112.55999-1-kirill.shutemov@linux.intel.com> References: <20181024125112.55999-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP On 5-level paging LDT remap area is placed in the middle of KASLR randomization region and it can overlap with direct mapping, vmalloc or vmap area. Let's move LDT just before direct mapping which makes it safe for KASLR. This also allows us to unify layout between 4- and 5-level paging. We don't touch 4 pgd slot gap just before the direct mapping reserved for a hypervisor, but move direct mapping by one slot instead. The LDT mapping is per-mm, so we cannot move it into P4D page table next to CPU_ENTRY_AREA without complicating PGD table allocation for 5-level paging. Signed-off-by: Kirill A. Shutemov Fixes: f55f0501cbf6 ("x86/pti: Put the LDT in its own PGD if PTI is on") --- Documentation/x86/x86_64/mm.txt | 34 +++++++++++++------------ arch/x86/include/asm/page_64_types.h | 12 +++++---- arch/x86/include/asm/pgtable_64_types.h | 4 +-- arch/x86/xen/mmu_pv.c | 6 ++--- 4 files changed, 29 insertions(+), 27 deletions(-) diff --git a/Documentation/x86/x86_64/mm.txt b/Documentation/x86/x86_64/mm.txt index 702898633b00..75bff98928a8 100644 --- a/Documentation/x86/x86_64/mm.txt +++ b/Documentation/x86/x86_64/mm.txt @@ -34,23 +34,24 @@ __________________|____________|__________________|_________|___________________ ____________________________________________________________|___________________________________________________________ | | | | ffff800000000000 | -128 TB | ffff87ffffffffff | 8 TB | ... guard hole, also reserved for hypervisor - ffff880000000000 | -120 TB | ffffc7ffffffffff | 64 TB | direct mapping of all physical memory (page_offset_base) - ffffc80000000000 | -56 TB | ffffc8ffffffffff | 1 TB | ... unused hole + ffff880000000000 | -120 TB | ffff887fffffffff | 0.5 TB | LDT remap for PTI + ffff888000000000 | -119.5 TB | ffffc87fffffffff | 64 TB | direct mapping of all physical memory (page_offset_base) + ffffc88000000000 | -55.5 TB | ffffc8ffffffffff | 0.5 TB | ... unused hole ffffc90000000000 | -55 TB | ffffe8ffffffffff | 32 TB | vmalloc/ioremap space (vmalloc_base) ffffe90000000000 | -23 TB | ffffe9ffffffffff | 1 TB | ... unused hole ffffea0000000000 | -22 TB | ffffeaffffffffff | 1 TB | virtual memory map (vmemmap_base) ffffeb0000000000 | -21 TB | ffffebffffffffff | 1 TB | ... unused hole ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory - fffffc0000000000 | -4 TB | fffffdffffffffff | 2 TB | ... unused hole - | | | | vaddr_end for KASLR - fffffe0000000000 | -2 TB | fffffe7fffffffff | 0.5 TB | cpu_entry_area mapping - fffffe8000000000 | -1.5 TB | fffffeffffffffff | 0.5 TB | LDT remap for PTI - ffffff0000000000 | -1 TB | ffffff7fffffffff | 0.5 TB | %esp fixup stacks __________________|____________|__________________|_________|____________________________________________________________ | - | Identical layout to the 47-bit one from here on: + | Identical layout to the 56-bit one from here on: ____________________________________________________________|____________________________________________________________ | | | | + fffffc0000000000 | -4 TB | fffffdffffffffff | 2 TB | ... unused hole + | | | | vaddr_end for KASLR + fffffe0000000000 | -2 TB | fffffe7fffffffff | 0.5 TB | cpu_entry_area mapping + fffffe8000000000 | -1.5 TB | fffffeffffffffff | 0.5 TB | ... unused hole + ffffff0000000000 | -1 TB | ffffff7fffffffff | 0.5 TB | %esp fixup stacks ffffff8000000000 | -512 GB | ffffffeeffffffff | 444 GB | ... unused hole ffffffef00000000 | -68 GB | fffffffeffffffff | 64 GB | EFI region mapping space ffffffff00000000 | -4 GB | ffffffff7fffffff | 2 GB | ... unused hole @@ -83,7 +84,7 @@ Notes: __________________|____________|__________________|_________|___________________________________________________________ | | | | 0000800000000000 | +64 PB | ffff7fffffffffff | ~16K PB | ... huge, still almost 64 bits wide hole of non-canonical - | | | | virtual memory addresses up to the -128 TB + | | | | virtual memory addresses up to the -64 PB | | | | starting offset of kernel mappings. __________________|____________|__________________|_________|___________________________________________________________ | @@ -91,23 +92,24 @@ __________________|____________|__________________|_________|___________________ ____________________________________________________________|___________________________________________________________ | | | | ff00000000000000 | -64 PB | ff0fffffffffffff | 4 PB | ... guard hole, also reserved for hypervisor - ff10000000000000 | -60 PB | ff8fffffffffffff | 32 PB | direct mapping of all physical memory (page_offset_base) - ff90000000000000 | -28 PB | ff9fffffffffffff | 4 PB | LDT remap for PTI + ff10000000000000 | -60 PB | ff10ffffffffffff | 0.25 PB | LDT remap for PTI + ff11000000000000 | -59.75 PB | ff90ffffffffffff | 32 PB | direct mapping of all physical memory (page_offset_base) + ff91000000000000 | -27.75 PB | ff9fffffffffffff | 3.75 PB | ... unused hole ffa0000000000000 | -24 PB | ffd1ffffffffffff | 12.5 PB | vmalloc/ioremap space (vmalloc_base) ffd2000000000000 | -11.5 PB | ffd3ffffffffffff | 0.5 PB | ... unused hole ffd4000000000000 | -11 PB | ffd5ffffffffffff | 0.5 PB | virtual memory map (vmemmap_base) ffd6000000000000 | -10.5 PB | ffdeffffffffffff | 2.25 PB | ... unused hole ffdf000000000000 | -8.25 PB | fffffdffffffffff | ~8 PB | KASAN shadow memory - fffffc0000000000 | -4 TB | fffffdffffffffff | 2 TB | ... unused hole - | | | | vaddr_end for KASLR - fffffe0000000000 | -2 TB | fffffe7fffffffff | 0.5 TB | cpu_entry_area mapping - fffffe8000000000 | -1.5 TB | fffffeffffffffff | 0.5 TB | ... unused hole - ffffff0000000000 | -1 TB | ffffff7fffffffff | 0.5 TB | %esp fixup stacks __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 47-bit one from here on: ____________________________________________________________|____________________________________________________________ | | | | + fffffc0000000000 | -4 TB | fffffdffffffffff | 2 TB | ... unused hole + | | | | vaddr_end for KASLR + fffffe0000000000 | -2 TB | fffffe7fffffffff | 0.5 TB | cpu_entry_area mapping + fffffe8000000000 | -1.5 TB | fffffeffffffffff | 0.5 TB | ... unused hole + ffffff0000000000 | -1 TB | ffffff7fffffffff | 0.5 TB | %esp fixup stacks ffffff8000000000 | -512 GB | ffffffeeffffffff | 444 GB | ... unused hole ffffffef00000000 | -68 GB | fffffffeffffffff | 64 GB | EFI region mapping space ffffffff00000000 | -4 GB | ffffffff7fffffff | 2 GB | ... unused hole diff --git a/arch/x86/include/asm/page_64_types.h b/arch/x86/include/asm/page_64_types.h index cd0cf1c568b4..8f657286d599 100644 --- a/arch/x86/include/asm/page_64_types.h +++ b/arch/x86/include/asm/page_64_types.h @@ -33,12 +33,14 @@ /* * Set __PAGE_OFFSET to the most negative possible address + - * PGDIR_SIZE*16 (pgd slot 272). The gap is to allow a space for a - * hypervisor to fit. Choosing 16 slots here is arbitrary, but it's - * what Xen requires. + * PGDIR_SIZE*17 (pgd slot 273). + * + * The gap is to allow a space for LDT remap for PTI (1 pgd slot) and space for + * a hypervisor (16 slots). Choosing 16 slots for a hypervisor is arbitrary, + * but it's what Xen requires. */ -#define __PAGE_OFFSET_BASE_L5 _AC(0xff10000000000000, UL) -#define __PAGE_OFFSET_BASE_L4 _AC(0xffff880000000000, UL) +#define __PAGE_OFFSET_BASE_L5 _AC(0xff11000000000000, UL) +#define __PAGE_OFFSET_BASE_L4 _AC(0xffff888000000000, UL) #ifdef CONFIG_DYNAMIC_MEMORY_LAYOUT #define __PAGE_OFFSET page_offset_base diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm/pgtable_64_types.h index 04edd2d58211..84bd9bdc1987 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -111,9 +111,7 @@ extern unsigned int ptrs_per_p4d; */ #define MAXMEM (1UL << MAX_PHYSMEM_BITS) -#define LDT_PGD_ENTRY_L4 -3UL -#define LDT_PGD_ENTRY_L5 -112UL -#define LDT_PGD_ENTRY (pgtable_l5_enabled() ? LDT_PGD_ENTRY_L5 : LDT_PGD_ENTRY_L4) +#define LDT_PGD_ENTRY -240UL #define LDT_BASE_ADDR (LDT_PGD_ENTRY << PGDIR_SHIFT) #define LDT_END_ADDR (LDT_BASE_ADDR + PGDIR_SIZE) diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c index 70ea598a37d2..7a2a74c2dd30 100644 --- a/arch/x86/xen/mmu_pv.c +++ b/arch/x86/xen/mmu_pv.c @@ -1905,7 +1905,7 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn) init_top_pgt[0] = __pgd(0); /* Pre-constructed entries are in pfn, so convert to mfn */ - /* L4[272] -> level3_ident_pgt */ + /* L4[273] -> level3_ident_pgt */ /* L4[511] -> level3_kernel_pgt */ convert_pfn_mfn(init_top_pgt); @@ -1925,8 +1925,8 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn) addr[0] = (unsigned long)pgd; addr[1] = (unsigned long)l3; addr[2] = (unsigned long)l2; - /* Graft it onto L4[272][0]. Note that we creating an aliasing problem: - * Both L4[272][0] and L4[511][510] have entries that point to the same + /* Graft it onto L4[273][0]. Note that we creating an aliasing problem: + * Both L4[273][0] and L4[511][510] have entries that point to the same * L2 (PMD) tables. Meaning that if you modify it in __va space * it will be also modified in the __ka space! (But if you just * modify the PMD table to point to other PTE's or none, then you From patchwork Wed Oct 24 12:51:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A . Shutemov" X-Patchwork-Id: 10654621 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 700BA17E2 for ; Wed, 24 Oct 2018 12:51:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F45E2A77C for ; Wed, 24 Oct 2018 12:51:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5210E2A76C; Wed, 24 Oct 2018 12:51:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B77832A77C for ; Wed, 24 Oct 2018 12:51:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B40136B0288; Wed, 24 Oct 2018 08:51:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AED346B028B; Wed, 24 Oct 2018 08:51:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B4126B028C; Wed, 24 Oct 2018 08:51:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 5C7116B0288 for ; Wed, 24 Oct 2018 08:51:33 -0400 (EDT) Received: by mail-pf1-f200.google.com with SMTP id z10-v6so3275496pfd.5 for ; Wed, 24 Oct 2018 05:51:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=6P5mpgfHnxoW8AFvc5Fz1NrbxyV8PgnIyF9q7AJ04yA=; b=Yaq7XOkwhQddZbTUGX2xtn88VpwhMQTPfHl+NXdokKAdxn+tFODx7AbhKawObfku0t gt+66bkln+Hk+U6lzx+8LwE55fzi2hGkzgJ9szPpj94nfm4jNHWq9Y+6ixVmModNFNmx ZweYY8vB1GrZ7a2IKpxJK0KxvX7Rdm69El0I5rbxg+hSRtBZRGOc8snSjkS2z1kWpplO 9DeNIZZOCzGp8Xxwh8DeFkS+7CFMGBFPD665Ohz6kpclQDlQXKn6V6jthMtes6bh5/bV uTMbhYuAjaZEx5nNnpgSschJzDTjGRa0DIV0Vh+tK9rlZaWPIaJfiH7DQnzLfbyyQDqd Eflg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AGRZ1gLwMqDYB3rdhqCqmSGiQf8Y5xsHimCQm1+QuXQTIGdh3BaWtauS V3BJW2PJJc59WnhooEp2i9XP7uE4nel7N5arrYG5pf6NgmRjd1XPQOUmCA4GjuSQgRaAVZBFoaX TiRLiNCyTGUUr7al7iDuPxJJMw0Tz36N7eOxVgDRE4jkzO+1W46WLFDjaZL+4bULiKQ== X-Received: by 2002:a65:4103:: with SMTP id w3-v6mr2467392pgp.284.1540385493026; Wed, 24 Oct 2018 05:51:33 -0700 (PDT) X-Google-Smtp-Source: AJdET5fXtSSd4Vdh8heNumdFiKHx46S34PEbL8nhCHPBR8LyNkOC3dmNuJ7/7BfFsCIJSMCPqwfC X-Received: by 2002:a65:4103:: with SMTP id w3-v6mr2467344pgp.284.1540385491891; Wed, 24 Oct 2018 05:51:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540385491; cv=none; d=google.com; s=arc-20160816; b=CM0vj8EEkzKj/Fr7WZq+pBPV1hpBXjxi85y8Pycp1QeYglXE4jvpTT4YFRIZZUjuTe 5njKkCvQvNN1uyPyopvk3jDAsM7/+ljZXTjSyPpfyIsk0eEzT8rVmCSS2kcvhjLk743B tBBT+qTjV0FYEU8RyfMQd+lMNu9Kny7eKEnlsh9s8h2wHUiBdvuQ9C+MYfKxyiqJK/F9 caFWeLcw0Y5JKAtA7qgKNuGXhZZPAfWf32EtCgcVch2BDX2Q5n3iQXix7S7LA9xIYf7L lJ6TYsEm1W3b/CInZDfPNlwJhLi7jZjbs95UkQyYMic5CcC2iWIyJotf02S+dOcSpxKv KwgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=6P5mpgfHnxoW8AFvc5Fz1NrbxyV8PgnIyF9q7AJ04yA=; b=B8HxyWDbgjaiG1SkGsli0TwK4oDADOPy4GeHWqLTn2tfBZGcFLyHQcl7pYKyyHRqO0 y5iSwGbJQZ51evfwRlv4PxfE4p6Rahl2U6G1FimrRh2to62WDV3deMIQ3KNLD/MqC/U8 oK6m/ax8IMnccykZ0qx1oIqXnaResJD+pswpRib06SjQ05pTelA3/kwlfPcl16ksw7j+ zo+w4qpVlYqqRJ3me8TtEU+/Y82GKEkA4rO6IwdsQZDbvIlzTgi6Gd2ryZr3r9PpRWIh Fo5OrWCWnLSRoc+naXCHB/rykkLU7auTX8yTFkdJRFQJ8F13XthpWLiVQjkmtzilXaoe j3hQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id m9-v6si4621867pge.326.2018.10.24.05.51.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Oct 2018 05:51:31 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Oct 2018 05:51:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,420,1534834800"; d="scan'208";a="274050048" Received: from black.fi.intel.com ([10.237.72.28]) by fmsmga005.fm.intel.com with ESMTP; 24 Oct 2018 05:51:27 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id C0193133; Wed, 24 Oct 2018 15:51:26 +0300 (EEST) From: "Kirill A. Shutemov" To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org Cc: boris.ostrovsky@oracle.com, jgross@suse.com, bhe@redhat.com, willy@infradead.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCHv2 2/2] x86/ldt: Unmap PTEs for the slot before freeing LDT pages Date: Wed, 24 Oct 2018 15:51:12 +0300 Message-Id: <20181024125112.55999-3-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181024125112.55999-1-kirill.shutemov@linux.intel.com> References: <20181024125112.55999-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP modify_ldt(2) leaves old LDT mapped after we switch over to the new one. Memory for the old LDT gets freed and the pages can be re-used. Leaving the mapping in place can have security implications. The mapping is present in userspace copy of page tables and Meltdown-like attack can read these freed and possibly reused pages. It's relatively simple to fix: just unmap the old LDT and flush TLB before freeing LDT memory. We can now avoid flushing TLB on map_ldt_struct() as the slot is unmapped and flushed by unmap_ldt_struct() (or never mapped in the first place). The overhead of the change should be negligible. It shouldn't be a particularly hot path anyway. Signed-off-by: Kirill A. Shutemov Fixes: f55f0501cbf6 ("x86/pti: Put the LDT in its own PGD if PTI is on") --- arch/x86/kernel/ldt.c | 59 ++++++++++++++++++++++++++++--------------- 1 file changed, 38 insertions(+), 21 deletions(-) diff --git a/arch/x86/kernel/ldt.c b/arch/x86/kernel/ldt.c index ab18e0884dc6..5dc8ed202fa8 100644 --- a/arch/x86/kernel/ldt.c +++ b/arch/x86/kernel/ldt.c @@ -199,14 +199,6 @@ static void sanity_check_ldt_mapping(struct mm_struct *mm) /* * If PTI is enabled, this maps the LDT into the kernelmode and * usermode tables for the given mm. - * - * There is no corresponding unmap function. Even if the LDT is freed, we - * leave the PTEs around until the slot is reused or the mm is destroyed. - * This is harmless: the LDT is always in ordinary memory, and no one will - * access the freed slot. - * - * If we wanted to unmap freed LDTs, we'd also need to do a flush to make - * it useful, and the flush would slow down modify_ldt(). */ static int map_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt, int slot) @@ -214,8 +206,7 @@ map_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt, int slot) unsigned long va; bool is_vmalloc; spinlock_t *ptl; - pgd_t *pgd; - int i; + int i, nr_pages; if (!static_cpu_has(X86_FEATURE_PTI)) return 0; @@ -229,16 +220,10 @@ map_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt, int slot) /* Check if the current mappings are sane */ sanity_check_ldt_mapping(mm); - /* - * Did we already have the top level entry allocated? We can't - * use pgd_none() for this because it doens't do anything on - * 4-level page table kernels. - */ - pgd = pgd_offset(mm, LDT_BASE_ADDR); - is_vmalloc = is_vmalloc_addr(ldt->entries); - for (i = 0; i * PAGE_SIZE < ldt->nr_entries * LDT_ENTRY_SIZE; i++) { + nr_pages = DIV_ROUND_UP(ldt->nr_entries * LDT_ENTRY_SIZE, PAGE_SIZE); + for (i = 0; i < nr_pages; i++) { unsigned long offset = i << PAGE_SHIFT; const void *src = (char *)ldt->entries + offset; unsigned long pfn; @@ -272,13 +257,39 @@ map_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt, int slot) /* Propagate LDT mapping to the user page-table */ map_ldt_struct_to_user(mm); - va = (unsigned long)ldt_slot_va(slot); - flush_tlb_mm_range(mm, va, va + LDT_SLOT_STRIDE, PAGE_SHIFT, false); - ldt->slot = slot; return 0; } +static void +unmap_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt) +{ + unsigned long va; + int i, nr_pages; + + if (!ldt) + return; + + /* LDT map/unmap is only required for PTI */ + if (!static_cpu_has(X86_FEATURE_PTI)) + return; + + nr_pages = DIV_ROUND_UP(ldt->nr_entries * LDT_ENTRY_SIZE, PAGE_SIZE); + for (i = 0; i < nr_pages; i++) { + unsigned long offset = i << PAGE_SHIFT; + pte_t *ptep; + spinlock_t *ptl; + + va = (unsigned long)ldt_slot_va(ldt->slot) + offset; + ptep = get_locked_pte(mm, va, &ptl); + pte_clear(mm, va, ptep); + pte_unmap_unlock(ptep, ptl); + } + + va = (unsigned long)ldt_slot_va(ldt->slot); + flush_tlb_mm_range(mm, va, va + nr_pages * PAGE_SIZE, 0, false); +} + #else /* !CONFIG_PAGE_TABLE_ISOLATION */ static int @@ -286,6 +297,11 @@ map_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt, int slot) { return 0; } + +static void +unmap_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt) +{ +} #endif /* CONFIG_PAGE_TABLE_ISOLATION */ static void free_ldt_pgtables(struct mm_struct *mm) @@ -524,6 +540,7 @@ static int write_ldt(void __user *ptr, unsigned long bytecount, int oldmode) } install_ldt(mm, new_ldt); + unmap_ldt_struct(mm, old_ldt); free_ldt_struct(old_ldt); error = 0;