From patchwork Mon Feb 26 08:20:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 10241687 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 62DAA60208 for ; Mon, 26 Feb 2018 08:52:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5A76F29307 for ; Mon, 26 Feb 2018 08:52:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4E8302936E; Mon, 26 Feb 2018 08:52:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CCEC229307 for ; Mon, 26 Feb 2018 08:52:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Un3eEPFDQEqz4mDM50gS1it2rLOxoNpqM+QrqlfaVpM=; b=avryM6fqymSLii bIuor2xv06nBnr26+qHzrR2gyH60U0aRbYJlBIo/HGKCxj/+3FKU7+gdSbcLDq/ZEZBFqNIwnjv02 wYBzfh3pZ3ONi2dbN3D2IqZrBqOAWonxQK2FRQpO7QCph40TrBI3hkPnONDwAHiyg4sJwuUvv4NYy A71LbYdKv2QvRfVmNH+AQb5nZAM5nA0rXJtw0Y/AYEx2UFKFYCjhgoUVpFYd9q2C1JRR9xmpcfwIo NflM0njJnCdZ+J6mOSdaKd+rjyrEcrFaxtss9IstSGUANOBLHyPuNLzeBidk3CG6uvYVtnEwaiPox GkNs1VFxzgM261irK2zg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1eqEWa-0007zX-HL; Mon, 26 Feb 2018 08:52:44 +0000 Received: from mail-pf0-x244.google.com ([2607:f8b0:400e:c00::244]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eqE8G-0004qj-ES for linux-arm-kernel@lists.infradead.org; Mon, 26 Feb 2018 08:27:59 +0000 Received: by mail-pf0-x244.google.com with SMTP id z14so6221000pfe.10 for ; Mon, 26 Feb 2018 00:27:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=c+m8Xx/gdzZk3oAgyKLtXnF31YiTGhZDX4r8l10YuK4=; b=Zh8hg4a58XLRH9ZUZZAbbTsxHhtmh4//H1Y1YjpNB5gPV7cEHDnDNISEwBOdXrdb++ yMUjG/wRHSN96UZxm3t2TswSbmQKyc3A5m/TgPbAvf8Kw9ovue+yVKjFR9UWJGwsPI/B 8mlmlRb1dvoiUnyus68owrxZSqzPkk+61tpa0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=c+m8Xx/gdzZk3oAgyKLtXnF31YiTGhZDX4r8l10YuK4=; b=TzFVyi5ITMH8hAsPuQc89vMUQ8gjcyS4vsZZtfBl/dXBZ8gDRqIV50YHOh5HWe0YbW 5N8dyVH6+v0Mi1BXZrkMVF4GJvTHE+uioGaMXJN/Vzb9AUJb71jVy80lRiD12Tcu2fcB P+RjRFp2CUy6R9ePcZiOegxRrP6sPuZw5YoPp7SgK7XiYTYJwiiKQ7HyQpN19JbY+rov NTYaWXZU6XNCQ1Se5kr///+pjEmgIXCG9J3fnnmRGGsoMPp6A4u1NAQvMWppCu0w6k/H zLYwyl/KXpXikff2serQoRbdtYD2dNyHpN+yFyZDu+0pjqi6LO+ZrNc4aTcqgTSvJeku d/wg== X-Gm-Message-State: APf1xPCE5Ez0VuHE9DYIISKmCNwK6erxdIS7FV8bBIoAnJSbQQyuIPTK 2ZN7hSvOuSD8rNOQolT4QYztUQ== X-Google-Smtp-Source: AH8x2255/5/eH6Qdijae6xa52x5Zrm4idxv02oAYA4e6AFZdisFT769QBMhacDcEOw2hfDtvKryb2Q== X-Received: by 10.98.227.10 with SMTP id g10mr9837192pfh.200.1519633645577; Mon, 26 Feb 2018 00:27:25 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.27.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:27:25 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Christoffer Dall , Russell King , kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 48/52] arm: KVM: Invalidate BTB on guest exit for Cortex-A12/A17 Date: Mon, 26 Feb 2018 16:20:22 +0800 Message-Id: <1519633227-29832-49-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180226_002737_374449_1113FDB5 X-CRM114-Status: GOOD ( 20.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Marc Zyngier ** Not yet queued for inclusion in mainline ** In order to avoid aliasing attacks against the branch predictor, let's invalidate the BTB on guest exit. This is made complicated by the fact that we cannot take a branch before invalidating the BTB. We only apply this to A12 and A17, which are the only two ARM cores on which this useful. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: no hvc stub in hyp_hvc in arch/arm/kvm/hyp/hyp-entry.S --- arch/arm/include/asm/kvm_asm.h | 2 -- arch/arm/include/asm/kvm_mmu.h | 18 ++++++++++- arch/arm/kvm/hyp/hyp-entry.S | 71 ++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 86 insertions(+), 5 deletions(-) diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h index 8ef0538..24f3ec7 100644 --- a/arch/arm/include/asm/kvm_asm.h +++ b/arch/arm/include/asm/kvm_asm.h @@ -61,8 +61,6 @@ struct kvm_vcpu; extern char __kvm_hyp_init[]; extern char __kvm_hyp_init_end[]; -extern char __kvm_hyp_vector[]; - extern void __kvm_flush_vm_context(void); extern void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa); extern void __kvm_tlb_flush_vmid(struct kvm *kvm); diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h index d10e362..2887129 100644 --- a/arch/arm/include/asm/kvm_mmu.h +++ b/arch/arm/include/asm/kvm_mmu.h @@ -37,6 +37,7 @@ #include #include +#include #include #include @@ -225,7 +226,22 @@ static inline unsigned int kvm_get_vmid_bits(void) static inline void *kvm_get_hyp_vector(void) { - return kvm_ksym_ref(__kvm_hyp_vector); + switch(read_cpuid_part()) { +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR + case ARM_CPU_PART_CORTEX_A12: + case ARM_CPU_PART_CORTEX_A17: + { + extern char __kvm_hyp_vector_bp_inv[]; + return kvm_ksym_ref(__kvm_hyp_vector_bp_inv); + } + +#endif + default: + { + extern char __kvm_hyp_vector[]; + return kvm_ksym_ref(__kvm_hyp_vector); + } + } } static inline int kvm_map_vectors(void) diff --git a/arch/arm/kvm/hyp/hyp-entry.S b/arch/arm/kvm/hyp/hyp-entry.S index 96beb53..b6b8cb1 100644 --- a/arch/arm/kvm/hyp/hyp-entry.S +++ b/arch/arm/kvm/hyp/hyp-entry.S @@ -71,6 +71,66 @@ __kvm_hyp_vector: W(b) hyp_irq W(b) hyp_fiq +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR + .align 5 +__kvm_hyp_vector_bp_inv: + .global __kvm_hyp_vector_bp_inv + + /* + * We encode the exception entry in the bottom 3 bits of + * SP, and we have to guarantee to be 8 bytes aligned. + */ + W(add) sp, sp, #1 /* Reset 7 */ + W(add) sp, sp, #1 /* Undef 6 */ + W(add) sp, sp, #1 /* Syscall 5 */ + W(add) sp, sp, #1 /* Prefetch abort 4 */ + W(add) sp, sp, #1 /* Data abort 3 */ + W(add) sp, sp, #1 /* HVC 2 */ + W(add) sp, sp, #1 /* IRQ 1 */ + W(nop) /* FIQ 0 */ + + mcr p15, 0, r0, c7, c5, 6 /* BPIALL */ + isb + +#ifdef CONFIG_THUMB2_KERNEL + /* + * Yet another silly hack: Use VPIDR as a temp register. + * Thumb2 is really a pain, as SP cannot be used with most + * of the bitwise instructions. The vect_br macro ensures + * things gets cleaned-up. + */ + mcr p15, 4, r0, c0, c0, 0 /* VPIDR */ + mov r0, sp + and r0, r0, #7 + sub sp, sp, r0 + push {r1, r2} + mov r1, r0 + mrc p15, 4, r0, c0, c0, 0 /* VPIDR */ + mrc p15, 0, r2, c0, c0, 0 /* MIDR */ + mcr p15, 4, r2, c0, c0, 0 /* VPIDR */ +#endif + +.macro vect_br val, targ +ARM( eor sp, sp, #\val ) +ARM( tst sp, #7 ) +ARM( eorne sp, sp, #\val ) + +THUMB( cmp r1, #\val ) +THUMB( popeq {r1, r2} ) + + beq \targ +.endm + + vect_br 0, hyp_fiq + vect_br 1, hyp_irq + vect_br 2, hyp_hvc + vect_br 3, hyp_dabt + vect_br 4, hyp_pabt + vect_br 5, hyp_svc + vect_br 6, hyp_undef + vect_br 7, hyp_reset +#endif + .macro invalid_vector label, cause .align \label: mov r0, #\cause @@ -131,7 +191,14 @@ hyp_hvc: mrceq p15, 4, r0, c12, c0, 0 @ get HVBAR beq 1f - push {lr} + /* + * Pushing r2 here is just a way of keeping the stack aligned to + * 8 bytes on any path that can trigger a HYP exception. Here, + * we may well be about to jump into the guest, and the guest + * exit would otherwise be badly decoded by our fancy + * "decode-exception-without-a-branch" code... + */ + push {r2, lr} mov lr, r0 mov r0, r1 @@ -141,7 +208,7 @@ hyp_hvc: THUMB( orr lr, #1) blx lr @ Call the HYP function - pop {lr} + pop {r2, lr} 1: eret guest_trap: