From patchwork Tue Jun 5 09:33:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jun Yao X-Patchwork-Id: 10447947 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D35A96024A for ; Tue, 5 Jun 2018 09:34:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CCBA028DDD for ; Tue, 5 Jun 2018 09:34:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C17FE28E17; Tue, 5 Jun 2018 09:34:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id D72F828DDD for ; Tue, 5 Jun 2018 09:34:41 +0000 (UTC) Received: (qmail 11870 invoked by uid 550); 5 Jun 2018 09:34:27 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 11670 invoked from network); 5 Jun 2018 09:34:25 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=WIuP6AGbYoc6QktONJieyyJ+iMwhik3c9QBZFHCDUc8=; b=bLnv6bukVQN4YMEsskImgsQzAhaovQqrnehH3Fn8PzaRpAZ30181KGMpcdSqWSAKoF 6jDOvDnX8yPJYjTKOyX/AvjRtmkSAFLnMgvXOJUFP4b/Gago+TjEusQIs5iH8+E6FrR6 PWrsxnz0WXmG0DVIo3VFcKfHZBUmL+A/dLm1obYhDtsljy9lDv9pdk9EQdnokfmqI4Tn wKh6Pi4fkldoTleRnUpAhAD+JNkn4DPo+Y1QpPxWCIhXn4lc5lfdNg13eLqDR6kT7R4O C8B+uYcMfI6RuH7hOc9YQJ0CYgBkU0fMafS3rqckG7K/c00UCJldSzsJX7tiLFZTwDRk vleg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=WIuP6AGbYoc6QktONJieyyJ+iMwhik3c9QBZFHCDUc8=; b=fTAI5l4y1AH8c7e1qbKBHr/hJC8/R5CCPoutNthO1oH7brwhdNMqHchE5O3+erapWQ 1l3wZyu6bo7G93Y5BhNRIEjTsV1PKXCuYz6EqC0+B08eOWLD7ZRfr4xVRTiW639xY4Xk zEU4S4lDOLd2uiLtHsEYIcjNBHKXAnZpnNHHUUFQekt35Ux34VBUu8mxAUL/EzSnWC2Q xEy1VEEVABQbFMHUte2p4AAOF5kzD+9vtvvy6aOlJhuPRLgX1oL9OUhhciwdAI4fuSyu 8F5dmyO5+W6Ag2O0NPcmmhYQ7hKjVeU0ipO/r3c5nIgH8PKyUfIwE3EyhuqNQUDm34G/ RIng== X-Gm-Message-State: ALKqPwfUvOv29UAb/Gy+SreYpnjQbP8NseCwEfJSlAzAjv8S6CRHlePs No2yyqZlhV5ZWrp5wBCIqO0= X-Google-Smtp-Source: ADUXVKKPMzpHIyPXyh0F25ynjabPVtcFa6yILwDg1ih07qBb1e8b0qGVa0PVoKaySIgbBateGykN7g== X-Received: by 2002:a62:f713:: with SMTP id h19-v6mr24517954pfi.165.1528191254062; Tue, 05 Jun 2018 02:34:14 -0700 (PDT) From: Jun Yao To: linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, james.morse@arm.com, robin.murphy@arm.com, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com Subject: [PATCH v2 3/3] arm64/mm: migrate swapper_pg_dir and tramp_pg_dir Date: Tue, 5 Jun 2018 17:33:50 +0800 Message-Id: <20180605093350.24504-4-yaojun8558363@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180605093350.24504-1-yaojun8558363@gmail.com> References: <20180605093350.24504-1-yaojun8558363@gmail.com> X-Virus-Scanned: ClamAV using ClamSMTP Migrate swapper_pg_dir and tramp_pg_dir. And their virtual addresses do not correlate with kernel's address. Signed-off-by: Jun Yao --- arch/arm64/mm/mmu.c | 75 ++++++++++++++++++++++++++++----------------- 1 file changed, 47 insertions(+), 28 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index fcb425b0fcab..3ffcff2d7dcd 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -58,6 +58,9 @@ EXPORT_SYMBOL(kimage_voffset); volatile phys_addr_t __section(".mmuoff.data.read") __pa_swapper_pg_dir; pgd_t *new_swapper_pg_dir = swapper_pg_dir; +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +pgd_t *new_tramp_pg_dir; +#endif /* * Empty_zero_page is a special page that is used for zero-initialized data @@ -81,19 +84,14 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, } EXPORT_SYMBOL(phys_mem_access_prot); -static phys_addr_t __init early_pgtable_alloc(void) +static void __init clear_page_phys(phys_addr_t phys) { - phys_addr_t phys; - void *ptr; - - phys = memblock_alloc(PAGE_SIZE, PAGE_SIZE); - /* * The FIX_{PGD,PUD,PMD} slots may be in active use, but the FIX_PTE * slot will be free, so we can (ab)use the FIX_PTE slot to initialise * any level of table. */ - ptr = pte_set_fixmap(phys); + void *ptr = pte_set_fixmap(phys); memset(ptr, 0, PAGE_SIZE); @@ -102,6 +100,14 @@ static phys_addr_t __init early_pgtable_alloc(void) * table walker */ pte_clear_fixmap(); +} + +static phys_addr_t __init early_pgtable_alloc(void) +{ + phys_addr_t phys; + + phys = memblock_alloc(PAGE_SIZE, PAGE_SIZE); + clear_page_phys(phys); return phys; } @@ -555,6 +561,10 @@ static int __init map_entry_trampoline(void) __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, PAGE_SIZE, prot, pgd_pgtable_alloc, 0); + memcpy(new_tramp_pg_dir, tramp_pg_dir, PGD_SIZE); + memblock_free(__pa_symbol(tramp_pg_dir), + __pa_symbol(swapper_pg_dir) - __pa_symbol(tramp_pg_dir)); + /* Map both the text and data into the kernel page table */ __set_fixmap(FIX_ENTRY_TRAMP_TEXT, pa_start, prot); if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) { @@ -632,38 +642,47 @@ static void __init map_kernel(pgd_t *pgdp) */ void __init paging_init(void) { - phys_addr_t pgd_phys = early_pgtable_alloc(); - pgd_t *pgdp = pgd_set_fixmap(pgd_phys); + phys_addr_t pgd_phys; + pgd_t *pgdp; + phys_addr_t mem_size; + + mem_size = __pa_symbol(swapper_pg_dir) + PAGE_SIZE + - (__pa_symbol(idmap_pg_dir) + IDMAP_DIR_SIZE); + + if (mem_size == PAGE_SIZE) { + pgd_phys = early_pgtable_alloc(); + __pa_swapper_pg_dir = pgd_phys; + } else { + phys_addr_t p; + + pgd_phys = memblock_alloc(mem_size, PAGE_SIZE); + + for (p = pgd_phys; p < pgd_phys + mem_size; p += PAGE_SIZE) + clear_page_phys(p); + + #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + new_tramp_pg_dir = __va(pgd_phys); + #endif + __pa_swapper_pg_dir = pgd_phys + mem_size - PAGE_SIZE; + } - __pa_swapper_pg_dir = __pa_symbol(swapper_pg_dir); __flush_dcache_area((void *)&__pa_swapper_pg_dir, sizeof(__pa_swapper_pg_dir)); + new_swapper_pg_dir = __va(__pa_swapper_pg_dir); + + pgdp = pgd_set_fixmap(__pa_swapper_pg_dir); + map_kernel(pgdp); map_mem(pgdp); - /* - * We want to reuse the original swapper_pg_dir so we don't have to - * communicate the new address to non-coherent secondaries in - * secondary_entry, and so cpu_switch_mm can generate the address with - * adrp+add rather than a load from some global variable. - * - * To do this we need to go via a temporary pgd. - */ - cpu_replace_ttbr1(pgd_phys); - memcpy(swapper_pg_dir, pgdp, PGD_SIZE); cpu_replace_ttbr1(__pa_swapper_pg_dir); + init_mm.pgd = new_swapper_pg_dir; pgd_clear_fixmap(); - memblock_free(pgd_phys, PAGE_SIZE); - /* - * We only reuse the PGD from the swapper_pg_dir, not the pud + pmd - * allocated with it. - */ - memblock_free(__pa_symbol(swapper_pg_dir) + PAGE_SIZE, - __pa_symbol(swapper_pg_end) - __pa_symbol(swapper_pg_dir) - - PAGE_SIZE); + memblock_free(__pa_symbol(swapper_pg_dir), + __pa_symbol(swapper_pg_end) - __pa_symbol(swapper_pg_dir)); } /*