From patchwork Wed May 30 09:12:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jun Yao X-Patchwork-Id: 10437993 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 17EFC601D3 for ; Wed, 30 May 2018 09:14:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 068EB288E8 for ; Wed, 30 May 2018 09:14:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EFA26288ED; Wed, 30 May 2018 09:14:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 017B22874E for ; Wed, 30 May 2018 09:14:11 +0000 (UTC) Received: (qmail 17928 invoked by uid 550); 30 May 2018 09:13:42 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 17841 invoked from network); 30 May 2018 09:13:38 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ZETQVYzfSjGf4llusVqsxE3ITuJT6eXYaWaQ/LGspdk=; b=OD41wYdtwF01dhuIlnIodXsMJ9DIEfNFZ90oTVkppiwUlUedrsdAllC8UpZWG+yhW2 cwcpZuMZAvCtswHE56URVDXHHOjQ62cFaNXIrqb/NiobWxUlwY733ya9EzH8wMZPXk8T PUqGmHdLL+j1U1bOv31Ew+P5gdk2sA+MLW2Vuoq2r1WvGDaqbzsdwrV6cx8n+EaDI6+i LVRK9VEEpdt+IayUg5v4mDLmqsH/OjWMZjdi7/wtlROpuilmJt+/xF7aQnRIzLl6/dvW nydPX1M0ntM8iYTD79jyWOqy4HlXFL7DMBSt38P7r5QewJSuyogT91YW4J0Yzzn+LTGs x1NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ZETQVYzfSjGf4llusVqsxE3ITuJT6eXYaWaQ/LGspdk=; b=ScQzBhQw2mjayYlBzIcpgmX4XNPbZMRa7+CMkQwFI9jEIFCuambLM03qVi786z0Vci Fjv07uxnwJhZATb0LeEMBOVTF2yPzgIfC1CYwKIGek2ftZJ8jxtns0c0RlqifT4ZrU9K RjmXqfoTJI31OXGNeMs3ipvAvqeKD6NGJzi9ZLLmyMEh8cF2kCWtiTs4MF7VMZU6pn10 mrihIMXRRGKMuLtxh3z2ZTLIHo47w3gMUlXHC5NrrUuhQ/Lli91Qo2E8n1fNtdEEw+FD 6INLUg4n3j6t0a+PulAzPe1ZkBswbqkZqJaMTkrxug6hcYEjLLIiKZ27Wy1kGiiLwoEj 4BVg== X-Gm-Message-State: ALKqPwfG9sMU5m1jOw18DgYHDleVhiG7dlV2O23OAfzdSUGmo3FvPiPe 3/zkc39a3iIjK78dPU3E/FSE5wKz X-Google-Smtp-Source: ADUXVKIYkg1XePl03K+GiXrjZhrvlzjch2CG9VClmAlbA8vZGg7Xcs87eceQ3a2eJCcIWIoZ3f3PBg== X-Received: by 2002:a63:7a4a:: with SMTP id j10-v6mr1586253pgn.421.1527671606194; Wed, 30 May 2018 02:13:26 -0700 (PDT) From: YaoJun To: kernel-hardening@lists.openwall.com Cc: catalin.marinas@arm.com, will.deacon@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, mark.rutland@arm.com Subject: [PATCH 4/4] arm64/mm: migrate swapper_pg_dir Date: Wed, 30 May 2018 17:12:59 +0800 Message-Id: <20180530091259.9386-5-yaojun8558363@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180530091259.9386-1-yaojun8558363@gmail.com> References: <20180530091259.9386-1-yaojun8558363@gmail.com> X-Virus-Scanned: ClamAV using ClamSMTP Migrate swapper_pg_dir and tramp_pg_dir. And its placement in the virtual address space does not correlate with the placement of the kernel. Signed-off-by: YaoJun --- arch/arm64/mm/mmu.c | 67 +++++++++++++++++++++++++++++++-------------- 1 file changed, 46 insertions(+), 21 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 26ba3e70a91c..b508de2cc6c4 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -57,6 +57,9 @@ EXPORT_SYMBOL(kimage_voffset); phys_addr_t __pa_swapper_pg_dir; pgd_t *new_swapper_pg_dir = swapper_pg_dir; +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +pgd_t *new_tramp_pg_dir; +#endif /* * Empty_zero_page is a special page that is used for zero-initialized data @@ -105,6 +108,25 @@ static phys_addr_t __init early_pgtable_alloc(void) return phys; } +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +static phys_addr_t __init early_pgtables_alloc(int num) +{ + int i; + phys_addr_t phys; + void *ptr; + + phys = memblock_alloc(PAGE_SIZE * num, PAGE_SIZE); + + for (i = 0; i < num; i++) { + ptr = pte_set_fixmap(phys + i * PAGE_SIZE); + memset(ptr, 0, PAGE_SIZE); + pte_clear_fixmap(); + } + + return phys; +} +#endif + static bool pgattr_change_is_safe(u64 old, u64 new) { /* @@ -554,6 +576,10 @@ static int __init map_entry_trampoline(void) __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, PAGE_SIZE, prot, pgd_pgtable_alloc, 0); + memcpy(new_tramp_pg_dir, tramp_pg_dir, PGD_SIZE); + memblock_free(__pa_symbol(tramp_pg_dir), + __pa_symbol(swapper_pg_dir) - __pa_symbol(tramp_pg_dir)); + /* Map both the text and data into the kernel page table */ __set_fixmap(FIX_ENTRY_TRAMP_TEXT, pa_start, prot); if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) { @@ -631,36 +657,35 @@ static void __init map_kernel(pgd_t *pgdp) */ void __init paging_init(void) { - phys_addr_t pgd_phys = early_pgtable_alloc(); - pgd_t *pgdp = pgd_set_fixmap(pgd_phys); + phys_addr_t pgd_phys; + pgd_t *pgdp; + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + int pages; + + pages = (__pa_symbol(swapper_pg_dir) - __pa_symbol(tramp_pg_dir) + + PAGE_SIZE) >> PAGE_SHIFT; + pgd_phys = early_pgtables_alloc(pages); + new_tramp_pg_dir = __va(pgd_phys); + __pa_swapper_pg_dir = pgd_phys + PAGE_SIZE; +#else + pgd_phys = early_pgtable_alloc(); + __pa_swapper_pg_dir = pgd_phys; +#endif + new_swapper_pg_dir = __va(__pa_swapper_pg_dir); - __pa_swapper_pg_dir = __pa_symbol(swapper_pg_dir); + pgdp = pgd_set_fixmap(__pa_swapper_pg_dir); map_kernel(pgdp); map_mem(pgdp); - /* - * We want to reuse the original swapper_pg_dir so we don't have to - * communicate the new address to non-coherent secondaries in - * secondary_entry, and so cpu_switch_mm can generate the address with - * adrp+add rather than a load from some global variable. - * - * To do this we need to go via a temporary pgd. - */ - cpu_replace_ttbr1(pgd_phys); - memcpy(swapper_pg_dir, pgdp, PGD_SIZE); cpu_replace_ttbr1(__pa_swapper_pg_dir); + init_mm.pgd = new_swapper_pg_dir; pgd_clear_fixmap(); - memblock_free(pgd_phys, PAGE_SIZE); - /* - * We only reuse the PGD from the swapper_pg_dir, not the pud + pmd - * allocated with it. - */ - memblock_free(__pa_symbol(swapper_pg_dir) + PAGE_SIZE, - __pa_symbol(swapper_pg_end) - __pa_symbol(swapper_pg_dir) - - PAGE_SIZE); + memblock_free(__pa_symbol(swapper_pg_dir), + __pa_symbol(swapper_pg_end) - __pa_symbol(swapper_pg_dir)); } /*