From patchwork Thu Jan 25 10:31:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 10183863 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6574260388 for ; Thu, 25 Jan 2018 10:32:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 522CB28A34 for ; Thu, 25 Jan 2018 10:32:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4505D28A38; Thu, 25 Jan 2018 10:32:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 90D4828A34 for ; Thu, 25 Jan 2018 10:32:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=zH/tuhTQXt1owp3vinkn/PerpjL+50nJXPao0UWzGl8=; b=GmOrnStAI611COJjivUPALAUGQ yUNbLknFivQ0vM3rVxVQC443ClP9bRMNKSNTtVRqJDhN5zRxCjXb81iCSS4Az1PSM40FT2i8YGPIh OjR0l2XV2Ziydoiz8Wy5Co83+TWUeFf/UzODB315/OEgWvuP8Pa2jPdkGHeKQ/mdOFALz/rhlTFOZ YrXLISZA6YZ0o9DFeUqyXFT+xCYWZV1choaf9VS6x6LChtNNJJjknOuGWGWNQtzdWp4V2hClugRGC qwCDK5iHloxAPs4crRkztYCVjtFVtU8QVGIDz12vmqmqbg7h07TG6H6cH4dhlvOc1tmDPviBf+Uz3 v6KDhDtg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1eeepX-00009S-UR; Thu, 25 Jan 2018 10:32:27 +0000 Received: from mail-wr0-x241.google.com ([2a00:1450:400c:c0c::241]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eeep3-00085H-EZ for linux-arm-kernel@lists.infradead.org; Thu, 25 Jan 2018 10:31:59 +0000 Received: by mail-wr0-x241.google.com with SMTP id 36so7133052wrh.1 for ; Thu, 25 Jan 2018 02:31:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=oxpKvnWnu3AzKIA2gQBJX2ShOJ2rqfrO9V0GrV28NxA=; b=ZvRTSsxkPG+Q60tHUITQujanRucBkC9Un2d6alaSTAKgKCnsA7VQDZuAI7Q56e6dzN dcH4GHTRk5dOnhjXbYKp8BZhkHTOIYHpfCGYoeUAysgQV9AfUqsJjWQYcc39QMGoCfen QtUGwZHQOKvEsmKV3SRP3R2RextP3xhVudabs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=oxpKvnWnu3AzKIA2gQBJX2ShOJ2rqfrO9V0GrV28NxA=; b=NWJM8jTy2EspaUoFwToCpt1UStQAKK+5OlgkrDEy6U4ph2CznVAHSlWaiKjYCS7YPQ OZAVxwtnW+aSff/TbGfW51e9sSh0pRVKYLnOHUUXL8bBs/18B+Pq92CVqM4ahA3fB1E2 mvGXZK97wDyw09ugQjh6r2ye7KyAnENrtgCLRt6oKRXEWyzZRxYfW4avo8YaLCtBuL6d dsVsxWsZdqGfF8pUAwuKazRX/5zbytQyrJUFvyVdrPT0f9WKQCAN6RZqWZQdD+Rk9ZEV qavOqMk8aDnmefLVnW5gg+RhSBwlkMpc7+gkueH+BrAClYa8I7WStCemEUGe955bmbKb JkPw== X-Gm-Message-State: AKwxytea8q5q02puSn514apsxhPRw1UZkIEM61Lrbsj8XW9wJ9Aw8n2Z jeNthg6ZjGephQ2K4VJmymT7tQ== X-Google-Smtp-Source: AH8x2272yvp73ZrGMaQsFJWP+8ltjQHNyWcozxU0mIK+7iDWlwxqm3U2xmu5IBoWKWZaRqWDDblPtQ== X-Received: by 10.223.135.187 with SMTP id b56mr8429409wrb.164.1516876305409; Thu, 25 Jan 2018 02:31:45 -0800 (PST) Received: from localhost.localdomain ([160.167.127.168]) by smtp.gmail.com with ESMTPSA id j77sm1199964wmf.37.2018.01.25.02.31.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Jan 2018 02:31:44 -0800 (PST) From: Ard Biesheuvel To: linux-efi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, will.deacon@arm.com, catalin.marinas@arm.com, mark.rutland@arm.com, marc.zyngier@arm.com Subject: [PATCH 2/4] efi/arm64: map the stack and entry wrapper into the UEFI page tables Date: Thu, 25 Jan 2018 10:31:29 +0000 Message-Id: <20180125103131.19168-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180125103131.19168-1-ard.biesheuvel@linaro.org> References: <20180125103131.19168-1-ard.biesheuvel@linaro.org> X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: joakim.bech@linaro.org, graeme.gregory@linaro.org, leif.lindholm@linaro.org, Ard Biesheuvel MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP As a preparatory step towards unmapping the kernel entirely while executing UEFI runtime services, move the stack and the entry wrapper routine mappings into the EFI page tables. Also, create a vector table that overrides the main one while executing in the firmware so we will be able to remap/unmap the kernel while taking interrupts. Signed-off-by: Ard Biesheuvel --- arch/arm/include/asm/efi.h | 5 ++ arch/arm64/include/asm/efi.h | 23 ++++++++- arch/arm64/include/asm/stacktrace.h | 4 ++ arch/arm64/kernel/efi-rt-wrapper.S | 51 +++++++++++++++++++- arch/arm64/kernel/efi.c | 24 +++++++++ arch/arm64/kernel/entry.S | 1 + drivers/firmware/efi/arm-runtime.c | 2 + 7 files changed, 107 insertions(+), 3 deletions(-) diff --git a/arch/arm/include/asm/efi.h b/arch/arm/include/asm/efi.h index 17f1f1a814ff..3a63e7cc1dfa 100644 --- a/arch/arm/include/asm/efi.h +++ b/arch/arm/include/asm/efi.h @@ -99,4 +99,9 @@ static inline unsigned long efi_get_max_initrd_addr(unsigned long dram_base, return dram_base + SZ_512M; } +static inline int efi_allocate_runtime_regions(struct mm_struct *mm) +{ + return 0; +} + #endif /* _ASM_ARM_EFI_H */ diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index 192d791f1103..b9b09a734719 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -2,11 +2,13 @@ #ifndef _ASM_EFI_H #define _ASM_EFI_H +#include + +#ifndef __ASSEMBLY__ #include #include #include #include -#include #include #include #include @@ -30,8 +32,9 @@ int efi_set_mapping_permissions(struct mm_struct *mm, efi_memory_desc_t *md); #define arch_efi_call_virt(p, f, args...) \ ({ \ efi_##f##_t *__f; \ + typeof(__efi_rt_asm_wrapper) *__wrap = (void *)EFI_CODE_BASE; \ __f = p->f; \ - __efi_rt_asm_wrapper(__f, #f, args); \ + __wrap(__f, #f, args); \ }) #define arch_efi_call_virt_teardown() \ @@ -146,4 +149,20 @@ static inline void efi_set_pgd(struct mm_struct *mm) void efi_virtmap_load(void); void efi_virtmap_unload(void); +int __init efi_allocate_runtime_regions(struct mm_struct *mm); + +#endif /* __ASSEMBLY__ */ + +/* + * When running with vmap'ed stacks, we need the base of the stack to be aligned + * appropriately, where the exact alignment depends on the page size. Let's just + * put the stack at address 0x0, which is guaranteed to be free and aligned. + */ +#define EFI_STACK_BASE 0x0 +#define EFI_STACK_SIZE THREAD_SIZE + +/* where to map the pivot code in the UEFI page tables */ +#define EFI_CODE_BASE 0x200000 +#define EFI_CODE_SIZE PAGE_SIZE + #endif /* _ASM_EFI_H */ diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index 472ef944e932..b1212b3b3df5 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -72,6 +72,8 @@ static inline bool on_overflow_stack(unsigned long sp) static inline bool on_overflow_stack(unsigned long sp) { return false; } #endif +bool on_efi_stack(unsigned long sp); + /* * We can only safely access per-cpu stacks from current in a non-preemptible * context. @@ -88,6 +90,8 @@ static inline bool on_accessible_stack(struct task_struct *tsk, unsigned long sp return true; if (on_sdei_stack(sp)) return true; + if (on_efi_stack(sp)) + return true; return false; } diff --git a/arch/arm64/kernel/efi-rt-wrapper.S b/arch/arm64/kernel/efi-rt-wrapper.S index 05235ebb336d..09e77e5edd94 100644 --- a/arch/arm64/kernel/efi-rt-wrapper.S +++ b/arch/arm64/kernel/efi-rt-wrapper.S @@ -7,7 +7,10 @@ */ #include +#include + .section ".rodata", "a" + .align PAGE_SHIFT ENTRY(__efi_rt_asm_wrapper) stp x29, x30, [sp, #-32]! mov x29, sp @@ -19,6 +22,12 @@ ENTRY(__efi_rt_asm_wrapper) */ stp x1, x18, [sp, #16] + /* switch to the EFI runtime stack and vector table */ + mov sp, #EFI_STACK_BASE + EFI_STACK_SIZE + adr x1, __efi_rt_vectors + msr vbar_el1, x1 + isb + /* * We are lucky enough that no EFI runtime services take more than * 5 arguments, so all are passed in registers rather than via the @@ -32,10 +41,50 @@ ENTRY(__efi_rt_asm_wrapper) mov x4, x6 blr x8 + /* switch back to the task stack and primary vector table */ + mov sp, x29 + ldr x1, 2f + msr vbar_el1, x1 + isb + ldp x1, x2, [sp, #16] cmp x2, x18 ldp x29, x30, [sp], #32 b.ne 0f ret -0: b efi_handle_corrupted_x18 // tail call +0: ldr x8, 1f + br x8 // tail call ENDPROC(__efi_rt_asm_wrapper) + .align 3 +1: .quad efi_handle_corrupted_x18 +2: .quad vectors + + .macro ventry + .align 7 +.Lv\@ : stp x29, x30, [sp, #-16]! // preserve x29 and x30 + mrs x29, elr_el1 // preserve ELR + adr x30, .Lret // take return address + msr elr_el1, x30 // set ELR to return address + ldr x30, 2b // take address of 'vectors' + msr vbar_el1, x30 // set VBAR to 'vectors' + isb + add x30, x30, #.Lv\@ - __efi_rt_vectors + br x30 + .endm + +.Lret: msr elr_el1, x29 + adr x30, __efi_rt_vectors + msr vbar_el1, x30 + isb + ldp x29, x30, [sp], #16 + eret + + .align 11 +__efi_rt_vectors: + .rept 8 + ventry + .endr + /* + * EFI runtime services never drop to EL0, so the + * remaining vector table entries are not needed. + */ diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c index af4f943cffac..68c920b2f4f0 100644 --- a/arch/arm64/kernel/efi.c +++ b/arch/arm64/kernel/efi.c @@ -130,3 +130,27 @@ asmlinkage efi_status_t efi_handle_corrupted_x18(efi_status_t s, const char *f) pr_err_ratelimited(FW_BUG "register x18 corrupted by EFI %s\n", f); return s; } + +bool on_efi_stack(unsigned long sp) +{ + return sp >= EFI_STACK_BASE && sp < (EFI_STACK_BASE + EFI_STACK_SIZE); +} + +int __init efi_allocate_runtime_regions(struct mm_struct *mm) +{ + static u8 stack[EFI_STACK_SIZE] __page_aligned_bss; + + /* map the stack */ + create_pgd_mapping(mm, __pa_symbol(stack), + EFI_STACK_BASE, EFI_STACK_SIZE, + __pgprot(pgprot_val(PAGE_KERNEL) | PTE_NG), + false); + + /* map the runtime wrapper pivot function */ + create_pgd_mapping(mm, __pa_symbol(__efi_rt_asm_wrapper), + EFI_CODE_BASE, EFI_CODE_SIZE, + __pgprot(pgprot_val(PAGE_KERNEL_ROX) | PTE_NG), + false); + + return 0; +} diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index b34e717d7597..3bab6c60a12b 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -204,6 +204,7 @@ alternative_if ARM64_HAS_PAN alternative_else_nop_endif .if \el != 0 + tbz x21, #63, 1f // skip if TTBR0 covers the stack mrs x21, ttbr0_el1 tst x21, #TTBR_ASID_MASK // Check for the reserved ASID orr x23, x23, #PSR_PAN_BIT // Set the emulated PAN in the saved SPSR diff --git a/drivers/firmware/efi/arm-runtime.c b/drivers/firmware/efi/arm-runtime.c index 1cc41c3d6315..e84f4d961de2 100644 --- a/drivers/firmware/efi/arm-runtime.c +++ b/drivers/firmware/efi/arm-runtime.c @@ -107,6 +107,8 @@ static bool __init efi_virtmap_init(void) if (efi_memattr_apply_permissions(&efi_mm, efi_set_mapping_permissions)) return false; + efi_allocate_runtime_regions(&efi_mm); + return true; }