From patchwork Thu Mar 1 15:55:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 10251771 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A41E960211 for ; Thu, 1 Mar 2018 15:56:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9493228622 for ; Thu, 1 Mar 2018 15:56:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 88A2828630; Thu, 1 Mar 2018 15:56:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2ABD428622 for ; Thu, 1 Mar 2018 15:56:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1032763AbeCAP43 (ORCPT ); Thu, 1 Mar 2018 10:56:29 -0500 Received: from foss.arm.com ([217.140.101.70]:40770 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1032710AbeCAP4Z (ORCPT ); Thu, 1 Mar 2018 10:56:25 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E60F91AFA; Thu, 1 Mar 2018 07:56:24 -0800 (PST) Received: from approximate.cambridge.arm.com (approximate.cambridge.arm.com [10.1.207.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 263993F25C; Thu, 1 Mar 2018 07:56:23 -0800 (PST) From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Christoffer Dall , Mark Rutland , Catalin Marinas , Will Deacon , James Morse , Steve Capper , Peter Maydell Subject: [PATCH v5 19/23] arm64: KVM: Allow far branches from vector slots to the main vectors Date: Thu, 1 Mar 2018 15:55:34 +0000 Message-Id: <20180301155538.26860-20-marc.zyngier@arm.com> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20180301155538.26860-1-marc.zyngier@arm.com> References: <20180301155538.26860-1-marc.zyngier@arm.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP So far, the branch from the vector slots to the main vectors can at most be 4GB from the main vectors (the reach of ADRP), and this distance is known at compile time. If we were to remap the slots to an unrelated VA, things would break badly. A way to achieve VA independence would be to load the absolute address of the vectors (__kvm_hyp_vector), either using a constant pool or a series of movs, followed by an indirect branch. This patches implements the latter solution, using another instance of a patching callback. Signed-off-by: Marc Zyngier Acked-by: Catalin Marinas --- arch/arm64/kernel/bpi.S | 11 ++++++++++- arch/arm64/kvm/va_layout.c | 27 +++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S index e000cb390618..e8d997788ad0 100644 --- a/arch/arm64/kernel/bpi.S +++ b/arch/arm64/kernel/bpi.S @@ -19,6 +19,9 @@ #include #include +#include +#include + .macro hyp_ventry offset .align 7 .rept 29 @@ -64,9 +67,15 @@ ENTRY(__bp_harden_hyp_vecs_start) .endr __kvm_enter_vectors: +alternative_cb kvm_patch_vector_branch + movz x1, #0 + movk x1, #0, lsl #16 + movk x1, #0, lsl #32 + movk x1, #0, lsl #48 +alternative_cb_end - adr_l x1, __kvm_hyp_vector add x0, x1, x0 + kern_hyp_va x0 br x0 ENTRY(__bp_harden_hyp_vecs_end) diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c index a73e47804972..7ef3d920c8d4 100644 --- a/arch/arm64/kvm/va_layout.c +++ b/arch/arm64/kvm/va_layout.c @@ -152,3 +152,30 @@ void __init kvm_update_va_mask(struct alt_instr *alt, updptr[i] = cpu_to_le32(insn); } } + +void kvm_patch_vector_branch(struct alt_instr *alt, + __le32 *origptr, __le32 *updptr, int nr_inst) +{ + enum aarch64_insn_movewide_type type; + u64 addr; + u32 oinsn, rd; + int s; + + BUG_ON(nr_inst != 4); + + addr = (uintptr_t)kvm_ksym_ref(__kvm_hyp_vector); + oinsn = le32_to_cpu(origptr[0]); + rd = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RD, oinsn); + + type = AARCH64_INSN_MOVEWIDE_ZERO; + for (s = 0; nr_inst--; s += 16) { + u32 insn = aarch64_insn_gen_movewide(rd, + (u16)(addr >> s), + s, + AARCH64_INSN_VARIANT_64BIT, + type); + *updptr++ = cpu_to_le32(insn); + type = AARCH64_INSN_MOVEWIDE_KEEP; + } + +}