From patchwork Wed Feb 28 03:56:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 10246735 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 85ED960362 for ; Wed, 28 Feb 2018 04:04:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 757F628CD4 for ; Wed, 28 Feb 2018 04:04:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6904828CD6; Wed, 28 Feb 2018 04:04:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D358528CD4 for ; Wed, 28 Feb 2018 04:04:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ngFdBr6Uzvi47WtN+nimfSQkpj3fnDpM6Zpa2LvLJfg=; b=PeUpN4NJ1QVMfc9xujzaPkHPx+ kDvTUarLtABjvxVb4VfQf15k4u3DSmgG5y3lLHueg1R2E5TrWPLpJREuI7dRIzRh7AYLjHoDhfrF+ UpstzcUoxZKR70tHGNWdCv4jJYKpoJ3/KPjvsO/5Vaw4CUaim/mK8nB8Lc9xStiZ5TsNCyaiU0Z0f VO4jHV0+Hb1KFSp8Xzt9LSCw6xPJXDjU7rfS47+1rqIDtMxCFIE9LNt6POvrGIMyv+ZoxeQGhZjI5 0duDFE3aZjRHnLgW+RTpuyjscgSz6wnMgcnP5LS/ZrthuUbzjHl4g2EEu1V6k5Sk0uzEyN9E+0Ga0 XpFnCPmw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1eqsyc-0003EM-01; Wed, 28 Feb 2018 04:04:22 +0000 Received: from merlin.infradead.org ([205.233.59.134]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eqstz-0007jc-N4 for linux-arm-kernel@bombadil.infradead.org; Wed, 28 Feb 2018 03:59:35 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=mw1ugMQkrRom83ynaVYoq/Tb8k4WnH3naAwpWK3t9VY=; b=ST28vu5bVkWWhq5WG/1xk18qc 9gihNuzoh2dyp4Wn7mFsGQeDSqmHGWYIIT4yhz2AUnVxe98btATDDQdBPPvlJtDbcXl8glZM68Lzm rx4s8X6itx6N3nQHl8LKYGqJRVTIRCZbCNm0xYLDtsvPhbsUYtRiVbSR1UaP4TsyluK4iSZSCjYA5 59hl8uSHP249pDCITNJ0uKecuNNKniNP6DmuVtfxBqb2tsb7wnSONSf2q1amfd9KwW01L/xIZaYpF aOAHQJqTHsdH0q4Z0pF/wUj1o1fA5aV+QXnD+oYEdMdd/2Dzja1HwjUVPtThZkcFkjpSS56nZs3ct gAiyHg96A==; Received: from mail-pg0-x241.google.com ([2607:f8b0:400e:c05::241]) by merlin.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eqstv-0004ZJ-Eg for linux-arm-kernel@lists.infradead.org; Wed, 28 Feb 2018 03:59:33 +0000 Received: by mail-pg0-x241.google.com with SMTP id l4so477068pgp.11 for ; Tue, 27 Feb 2018 19:59:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mw1ugMQkrRom83ynaVYoq/Tb8k4WnH3naAwpWK3t9VY=; b=WjFaltI08pTdm+kzBYOFtt4iDK7PidHLpFDoJAu4/JwHJxT17Wu1aphabty+nVIbvv VMcTErx8cc9VbtJVVk9seKpXKIWi4y+rfDC9opqCrxthXo+2bltaoZBqt38wN85P0EC9 r1+KxGwQbCZ2hVwAmrfjgUkwsDG6ejDyfBU5I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mw1ugMQkrRom83ynaVYoq/Tb8k4WnH3naAwpWK3t9VY=; b=dF9ZlS7BI75Jj5YTV0pIXUUrMRuMs4wYaZK0NMjyTvOSIBnmF4km3prZLUicU0Fm84 Prpgq1wx/u22GwYm/Prvq+IS+rVlEa/o1guYz/8gDVdSsrX3gt6HxRLVMroHblu2o1it w/v3OFtHA/dj+xH/Nn5zjKJavmv6nCo6dSzYWwQfwjq2vnDOZrU2lfwoFBFzaiSwk9o3 GZgq/Tv42zqSxVFBh9BHvoaR6bmLdksJnXd2VX9CZ+FxJLsf78Wk4eK0cWPlImVQQQVk ieMcpJgEaiAww/QdnaCP3ReyVKnrjsHLSOu7J0c5EkK8rfrDifzAeKoXzcb4OKmGbgUz HJrQ== X-Gm-Message-State: APf1xPA8HHnNj031EHgKpNssYaiGbJJacUPn4ipnhIrbWajkUwo3OnQ2 3vnx/unPmMMC0pp0qvYQA4EsrK3OFpg= X-Google-Smtp-Source: AH8x226C74xe+B4CBDFNQydomQ3CshQ1T2zF/CHfkj9cOt+gs9R0vgvUxTgcDChT5L+ktk5naBQCdA== X-Received: by 10.99.9.195 with SMTP id 186mr12972734pgj.152.1519790356397; Tue, 27 Feb 2018 19:59:16 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.59.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:59:15 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 14/29] arm64: entry: Add fake CPU feature for unmapping the kernel at EL0 Date: Wed, 28 Feb 2018 11:56:36 +0800 Message-Id: <1519790211-16582-15-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180227_225931_504546_313A2C1C X-CRM114-Status: GOOD ( 21.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Shi MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Will Deacon commit ea1e3de85e94 upstream. Allow explicit disabling of the entry trampoline on the kernel command line (kpti=off) by adding a fake CPU feature (ARM64_UNMAP_KERNEL_AT_EL0) that can be used to toggle the alternative sequences in our entry code and avoid use of the trampoline altogether if desired. This also allows us to make use of a static key in arm64_kernel_unmapped_at_el0(). Reviewed-by: Mark Rutland Tested-by: Laura Abbott Tested-by: Shanker Donthineni Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: skip non enabled cpu features in arch/arm64/include/asm/cpucaps.h and arch/arm64/kernel/cpufeature.c using cpus_have_cap instead of cpus_have_const_cap in arch/arm64/include/asm/mmu.h --- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/include/asm/mmu.h | 3 ++- arch/arm64/kernel/cpufeature.c | 41 ++++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/entry.S | 9 +++++---- 4 files changed, 50 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 87b4465..7ddf233 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -34,7 +34,8 @@ #define ARM64_HAS_32BIT_EL0 13 #define ARM64_HYP_OFFSET_LOW 14 #define ARM64_MISMATCHED_CACHE_LINE_SIZE 15 +#define ARM64_UNMAP_KERNEL_AT_EL0 16 -#define ARM64_NCAPS 16 +#define ARM64_NCAPS 17 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 279e75b..a813edf 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -34,7 +34,8 @@ typedef struct { static inline bool arm64_kernel_unmapped_at_el0(void) { - return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0); + return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0) && + cpus_have_cap(ARM64_UNMAP_KERNEL_AT_EL0); } extern void paging_init(void); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 3a129d4..74b168c 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -746,6 +746,40 @@ static bool hyp_offset_low(const struct arm64_cpu_capabilities *entry, return idmap_addr > GENMASK(VA_BITS - 2, 0) && !is_kernel_in_hyp_mode(); } +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */ + +static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, + int __unused) +{ + /* Forced on command line? */ + if (__kpti_forced) { + pr_info_once("kernel page table isolation forced %s by command line option\n", + __kpti_forced > 0 ? "ON" : "OFF"); + return __kpti_forced > 0; + } + + /* Useful for KASLR robustness */ + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) + return true; + + return false; +} + +static int __init parse_kpti(char *str) +{ + bool enabled; + int ret = strtobool(str, &enabled); + + if (ret) + return ret; + + __kpti_forced = enabled ? 1 : -1; + return 0; +} +__setup("kpti=", parse_kpti); +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ + static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "GIC system register CPU interface", @@ -829,6 +863,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .def_scope = SCOPE_SYSTEM, .matches = hyp_offset_low, }, +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + { + .capability = ARM64_UNMAP_KERNEL_AT_EL0, + .def_scope = SCOPE_SYSTEM, + .matches = unmap_kernel_at_el0, + }, +#endif {}, }; diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index eccb6d8..54f35cc 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -73,6 +73,7 @@ .macro kernel_ventry, el, label, regsize = 64 .align 7 #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +alternative_if ARM64_UNMAP_KERNEL_AT_EL0 .if \el == 0 .if \regsize == 64 mrs x30, tpidrro_el0 @@ -81,6 +82,7 @@ mov x30, xzr .endif .endif +alternative_else_nop_endif #endif sub sp, sp, #S_FRAME_SIZE @@ -208,10 +210,9 @@ alternative_else_nop_endif ldr lr, [sp, #S_LR] add sp, sp, #S_FRAME_SIZE // restore sp -#ifndef CONFIG_UNMAP_KERNEL_AT_EL0 - eret -#else .if \el == 0 +alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 bne 4f msr far_el1, x30 tramp_alias x30, tramp_exit_native @@ -219,10 +220,10 @@ alternative_else_nop_endif 4: tramp_alias x30, tramp_exit_compat br x30 +#endif .else eret .endif -#endif .endm .macro get_thread_info, rd