From patchwork Fri Jun 1 08:09:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jun Yao X-Patchwork-Id: 10442669 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2ADFC602BD for ; Fri, 1 Jun 2018 08:09:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1D88528D2E for ; Fri, 1 Jun 2018 08:09:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 10F4628FE0; Fri, 1 Jun 2018 08:09:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 27D9528D2E for ; Fri, 1 Jun 2018 08:09:39 +0000 (UTC) Received: (qmail 20392 invoked by uid 550); 1 Jun 2018 08:09:38 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 20317 invoked from network); 1 Jun 2018 08:09:37 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=vk1f7aNQMMywQ9wT4QCSsx1CMe6qXi0eNMv3jlDaUIc=; b=XETFxDdjoP/Kf1WidG2eWvkdC9c/zJyaC/6oErWXaldkGvTQrEVgIevDCusTY0xMpd L2SwckZwxH6TJzOS8Y9ckZRvC6hID8VHk+1M/M5jTgvIfRe+JlZa/idWVEm9e+EXS/Lc BlU+cKnKTQEZOXdCi1UFdNrNAR+Uc4Xb4jjskZLUhhwSo77bfnmjVk2CNabW/N6KEW1M DizI+LIJw0nkOr/gCVcdbVRjRKfZFJHFuIznVSsFCBy5ReFmBSiagzUbgpp+6LanHLi0 IiTVbzNEf9fxa6+E8aVl2voI9QALWLVZk/CT90YFz//+X6QR4O5jkOSkygPq2AllpAwE xE5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=vk1f7aNQMMywQ9wT4QCSsx1CMe6qXi0eNMv3jlDaUIc=; b=f3LdonJnmxpsQvmCtOVGP86zyTqE2tU6LRCTIoSiy0kVh4dyZtt1eAzHeh/4F1oF3c lhMjvj5N3kiWWvCM4GhiJ+KdSccFzLZHGjoW/sKkrk7++BaKeHEHTFckomju24O2MpEj k1jJA3CYbgAERSAZjI1qQJoUh7Dhfpyu3mBctJZNpD6w7sj+AlCkgC7OieZlx/v29La8 GMWCVyRKcmQ1MM/IhWbVFJ4vKUmxnFwxAD2q+9Ev65Op+NloX0DUxbr3YeecgHdIvfYW p6jthfMIBbWm3ba0+SVq68TAVfvatd0ZrD54jO94ZpYIxgEPjyF6RcGUm+O9Y1YB0fKS CL8Q== X-Gm-Message-State: ALKqPwdxgjtc9Ek+TPCWGE4Wj1IAyL7c8fF6iGsXL5Q/KmLLzCwP61n9 44qrPY0FQ1tLweIXoBPLBfQ= X-Google-Smtp-Source: ADUXVKKZu9BZgNI6apOgzhdfMNDnLl+VgbtwK7NZLWh3BOHRCcOVKnwyI/baMMWKVQjg9c1fbOpi4A== X-Received: by 2002:a62:f619:: with SMTP id x25-v6mr9885100pfh.106.1527840565700; Fri, 01 Jun 2018 01:09:25 -0700 (PDT) From: Jun Yao To: linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, greg@kroah.com Subject: [PATCH 4/4] arm64/mm: migrate swapper_pg_dir and tramp_pg_dir Date: Fri, 1 Jun 2018 16:09:18 +0800 Message-Id: <20180601080918.11918-1-yaojun8558363@gmail.com> X-Mailer: git-send-email 2.17.0 X-Virus-Scanned: ClamAV using ClamSMTP Migrate swapper_pg_dir and tramp_pg_dir. And their virtual addresses do not correlate with kernel's address. Signed-off-by: Jun Yao --- arch/arm64/mm/mmu.c | 70 +++++++++++++++++++++++++++------------------ 1 file changed, 42 insertions(+), 28 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 26ba3e70a91c..5baae59479d8 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -57,6 +57,9 @@ EXPORT_SYMBOL(kimage_voffset); phys_addr_t __pa_swapper_pg_dir; pgd_t *new_swapper_pg_dir = swapper_pg_dir; +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +pgd_t *new_tramp_pg_dir; +#endif /* * Empty_zero_page is a special page that is used for zero-initialized data @@ -80,19 +83,14 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, } EXPORT_SYMBOL(phys_mem_access_prot); -static phys_addr_t __init early_pgtable_alloc(void) +static void __init clear_page_phys(phys_addr_t phys) { - phys_addr_t phys; - void *ptr; - - phys = memblock_alloc(PAGE_SIZE, PAGE_SIZE); - /* * The FIX_{PGD,PUD,PMD} slots may be in active use, but the FIX_PTE * slot will be free, so we can (ab)use the FIX_PTE slot to initialise * any level of table. */ - ptr = pte_set_fixmap(phys); + void *ptr = pte_set_fixmap(phys); memset(ptr, 0, PAGE_SIZE); @@ -101,6 +99,14 @@ static phys_addr_t __init early_pgtable_alloc(void) * table walker */ pte_clear_fixmap(); +} + +static phys_addr_t __init early_pgtable_alloc(void) +{ + phys_addr_t phys; + + phys = memblock_alloc(PAGE_SIZE, PAGE_SIZE); + clear_page_phys(phys); return phys; } @@ -554,6 +560,10 @@ static int __init map_entry_trampoline(void) __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, PAGE_SIZE, prot, pgd_pgtable_alloc, 0); + memcpy(new_tramp_pg_dir, tramp_pg_dir, PGD_SIZE); + memblock_free(__pa_symbol(tramp_pg_dir), + __pa_symbol(swapper_pg_dir) - __pa_symbol(tramp_pg_dir)); + /* Map both the text and data into the kernel page table */ __set_fixmap(FIX_ENTRY_TRAMP_TEXT, pa_start, prot); if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) { @@ -631,36 +641,40 @@ static void __init map_kernel(pgd_t *pgdp) */ void __init paging_init(void) { - phys_addr_t pgd_phys = early_pgtable_alloc(); - pgd_t *pgdp = pgd_set_fixmap(pgd_phys); + phys_addr_t pgd_phys; + pgd_t *pgdp; + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + phys_addr_t mem_size; + phys_addr_t p; - __pa_swapper_pg_dir = __pa_symbol(swapper_pg_dir); + mem_size = __pa_symbol(swapper_pg_dir) + PAGE_SIZE + - __pa_symbol(tramp_pg_dir); + pgd_phys = memblock_alloc(mem_size, PAGE_SIZE); + + for (p = pgd_phys; p < pgd_phys + mem_size; p += PAGE_SIZE) + clear_page_phys(p); + + new_tramp_pg_dir = __va(pgd_phys); + __pa_swapper_pg_dir = pgd_phys + PAGE_SIZE; +#else + pgd_phys = early_pgtable_alloc(); + __pa_swapper_pg_dir = pgd_phys; +#endif + new_swapper_pg_dir = __va(__pa_swapper_pg_dir); + + pgdp = pgd_set_fixmap(__pa_swapper_pg_dir); map_kernel(pgdp); map_mem(pgdp); - /* - * We want to reuse the original swapper_pg_dir so we don't have to - * communicate the new address to non-coherent secondaries in - * secondary_entry, and so cpu_switch_mm can generate the address with - * adrp+add rather than a load from some global variable. - * - * To do this we need to go via a temporary pgd. - */ - cpu_replace_ttbr1(pgd_phys); - memcpy(swapper_pg_dir, pgdp, PGD_SIZE); cpu_replace_ttbr1(__pa_swapper_pg_dir); + init_mm.pgd = new_swapper_pg_dir; pgd_clear_fixmap(); - memblock_free(pgd_phys, PAGE_SIZE); - /* - * We only reuse the PGD from the swapper_pg_dir, not the pud + pmd - * allocated with it. - */ - memblock_free(__pa_symbol(swapper_pg_dir) + PAGE_SIZE, - __pa_symbol(swapper_pg_end) - __pa_symbol(swapper_pg_dir) - - PAGE_SIZE); + memblock_free(__pa_symbol(swapper_pg_dir), + __pa_symbol(swapper_pg_end) - __pa_symbol(swapper_pg_dir)); } /*