From patchwork Fri Jan 12 12:07:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 10160659 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4AC6560327 for ; Fri, 12 Jan 2018 12:28:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 32AAD289CC for ; Fri, 12 Jan 2018 12:28:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 27793289D3; Fri, 12 Jan 2018 12:28:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7BA88289CC for ; Fri, 12 Jan 2018 12:28:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Sd2YbhlOANIAkC24W+e9B9wLa8a8+gwgRHy/+2av4j0=; b=XiwQ4K+1fyOaqVq/VxFyNbEM91 nPstJf6yJH44WoDqCrpTzEOxE61foweRyXqnXlAUaqD6nPE0uKqcxMImfugRDfUuhAWC4+wLtLPIo vdJuWCgAuuf+4o9WhaKNm75HrhMl7MooZ3HCeuTgn5gPhI5UGJgoSOAfo9bwO40aMOU+rvJM3WP/s LxRQvquu9phfDEkALn8axfV86vRVYkfwZ4k+fGr4fAsgvt64J5Dga3Zz5albqHTlgU4IDK/Siabzj Y7MUQ+1HG28ktumMZOHKFhXKvkozl2hu/n09z8HhkSj/934RWyKj6ZWxgpbyOsZKO2hbaGrmlxgdX C5M/eC6Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1eZyRL-0005X8-UL; Fri, 12 Jan 2018 12:28:07 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eZyNv-0007Z3-5c for linux-arm-kernel@bombadil.infradead.org; Fri, 12 Jan 2018 12:24:35 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=1YOwrJceyUykWRrtvgxaXsbPodAgpGeK7jogU4b3Dd4=; b=DBR5EIqYYMUZdmntuR7GW2MF2 YLLwc4h+2BSKEnWjKA2IbCdINDGjjZujSYPwaSsDa11tuDYeeA4Pbm9/bhXcO4X1ue8FoLt9BdEeJ rV46lJ03VKwHJNNG4DHDTTftG4K+r2rrNa0U6GwhwUjrDAfFVpsmohVtaRwn8Pdi6gher0SjNB1/d 4sXL62RRNdOteQvQJGtYEWPTj32XtNSYgXoWYxv0LOt51ZUGq2C8FpxHmKWeO0FJw3hBHHSC8knYv Ocp3ddyynlxJ/zv4oHIjmEwD6oiHbmdhDx0cJl61kO7n6pEq2jneeksHcC/fRoLNgsxi+UevoIKhI GSiZvGE8Q==; Received: from mail-wm0-x241.google.com ([2a00:1450:400c:c09::241]) by casper.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eZy8Z-00039P-4y for linux-arm-kernel@lists.infradead.org; Fri, 12 Jan 2018 12:08:47 +0000 Received: by mail-wm0-x241.google.com with SMTP id 141so11315065wme.3 for ; Fri, 12 Jan 2018 04:08:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1YOwrJceyUykWRrtvgxaXsbPodAgpGeK7jogU4b3Dd4=; b=hVHmWIlnAmUCN40p5jIT7BC8jkVk3QPxul5QjpL7wjAY1LRSgTmyB6LmsXR5D5cyUj 9SqCzF7kgBPGU3fstvBCOJ5DMKGN4Wxoh7PYygIu+1EDnb65f3fKAKyBWmEXal/TR5SN S95zGLNE/GmRv6hCMG4kws2B2+JvLd5fpsunY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1YOwrJceyUykWRrtvgxaXsbPodAgpGeK7jogU4b3Dd4=; b=N0Wpu5AhrGjho7BSB7Tucpw4Y9YT2QJqyX1SfPa7LqcJkhwBkMAFhN40f2PBZgaSUK eom2venPOR4RIlftj+fnX9fN+eWad1WQ/hxnOfxUgi7gNWhLR6p9wNj5lsmXnur1RswW kz85Im08eWbnbWjJNtEKcnBc2O5vAqp6dS2uOPHh2R0jJE/F5uwLeV71LVKye5HaVD0N F848ZkwQXcMzuF1y+jC/SvArInyxmzLR7E2EKFn1f9Qtrfr2/h7TTkud+4q1YDz+dVtA eRq2pnDQnHQeQ9dG9pC+t6Fa0XEMWMrOy6YRnCCAB/FoTZ9MAD3l2jv74dNWozhWnQfY jB0w== X-Gm-Message-State: AKGB3mI34csyq3TAdkps9ECCNL0C3JveWffA9xnVvJrPykV4pvv9huVH J4G2RPZI8hmPBwvrTlVyrAtadg== X-Google-Smtp-Source: ACJfBov92wknzObAr3siIPoL19pcNDGREs6IYXAKQTvN2RNuLrPOEvdCICZPBG5vmcp5FD5QyweEpw== X-Received: by 10.80.181.37 with SMTP id y34mr35294676edd.277.1515758911296; Fri, 12 Jan 2018 04:08:31 -0800 (PST) Received: from localhost.localdomain (x50d2404e.cust.hiper.dk. [80.210.64.78]) by smtp.gmail.com with ESMTPSA id f16sm13489705edj.65.2018.01.12.04.08.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 12 Jan 2018 04:08:29 -0800 (PST) From: Christoffer Dall To: kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 26/41] KVM: arm64: Introduce framework for accessing deferred sysregs Date: Fri, 12 Jan 2018 13:07:32 +0100 Message-Id: <20180112120747.27999-27-christoffer.dall@linaro.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20180112120747.27999-1-christoffer.dall@linaro.org> References: <20180112120747.27999-1-christoffer.dall@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180112_120843_298248_C118E625 X-CRM114-Status: GOOD ( 24.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marc Zyngier , Andrew Jones , Christoffer Dall , Shih-Wei Li , kvm@vger.kernel.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP We are about to defer saving and restoring some groups of system registers to vcpu_put and vcpu_load on supported systems. This means that we need some infrastructure to access system registes which supports either accessing the memory backing of the register or directly accessing the system registers, depending on the state of the system when we access the register. We do this by defining a set of read/write accessors for each system register, and letting each system register be defined as "immediate" or "deferrable". Immediate registers are always saved/restored in the world-switch path, but deferrable registers are only saved/restored in vcpu_put/vcpu_load when supported and sysregs_loaded_on_cpu will be set in that case. Not that we don't use the deferred mechanism yet in this patch, but only introduce infrastructure. This is to improve convenience of review in the subsequent patches where it is clear which registers become deferred. [ Most of this logic was contributed by Marc Zyngier ] Signed-off-by: Marc Zyngier Signed-off-by: Christoffer Dall Reviewed-by: Julien Thierry --- arch/arm64/include/asm/kvm_host.h | 8 +- arch/arm64/kvm/sys_regs.c | 160 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 166 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 91272c35cc36..4b5ef82f6bdb 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -281,6 +281,10 @@ struct kvm_vcpu_arch { /* Detect first run of a vcpu */ bool has_run_once; + + /* True when deferrable sysregs are loaded on the physical CPU, + * see kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs. */ + bool sysregs_loaded_on_cpu; }; #define vcpu_gp_regs(v) (&(v)->arch.ctxt.gp_regs) @@ -293,8 +297,8 @@ struct kvm_vcpu_arch { */ #define __vcpu_sys_reg(v,r) ((v)->arch.ctxt.sys_regs[(r)]) -#define vcpu_read_sys_reg(v,r) __vcpu_sys_reg(v,r) -#define vcpu_write_sys_reg(v,r,n) do { __vcpu_sys_reg(v,r) = n; } while (0) +u64 vcpu_read_sys_reg(struct kvm_vcpu *vcpu, int reg); +void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, int reg, u64 val); /* * CP14 and CP15 live in the same array, as they are backed by the diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 96398d53b462..9d353a6a55c9 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -76,6 +77,165 @@ static bool write_to_read_only(struct kvm_vcpu *vcpu, return false; } +struct sys_reg_accessor { + u64 (*rdsr)(struct kvm_vcpu *, int); + void (*wrsr)(struct kvm_vcpu *, int, u64); +}; + +#define DECLARE_IMMEDIATE_SR(i) \ + static u64 __##i##_read(struct kvm_vcpu *vcpu, int r) \ + { \ + return __vcpu_sys_reg(vcpu, r); \ + } \ + \ + static void __##i##_write(struct kvm_vcpu *vcpu, int r, u64 v) \ + { \ + __vcpu_sys_reg(vcpu, r) = v; \ + } \ + +#define DECLARE_DEFERRABLE_SR(i, s) \ + static u64 __##i##_read(struct kvm_vcpu *vcpu, int r) \ + { \ + if (vcpu->arch.sysregs_loaded_on_cpu) { \ + WARN_ON(kvm_arm_get_running_vcpu() != vcpu); \ + return read_sysreg_s((s)); \ + } \ + return __vcpu_sys_reg(vcpu, r); \ + } \ + \ + static void __##i##_write(struct kvm_vcpu *vcpu, int r, u64 v) \ + { \ + if (vcpu->arch.sysregs_loaded_on_cpu) { \ + WARN_ON(kvm_arm_get_running_vcpu() != vcpu); \ + write_sysreg_s(v, (s)); \ + } else { \ + __vcpu_sys_reg(vcpu, r) = v; \ + } \ + } \ + + +#define SR_HANDLER_RANGE(i,e) \ + [i ... e] = (struct sys_reg_accessor) { \ + .rdsr = __##i##_read, \ + .wrsr = __##i##_write, \ + } + +#define SR_HANDLER(i) SR_HANDLER_RANGE(i, i) + +static void bad_sys_reg(int reg) +{ + WARN_ONCE(1, "Bad system register access %d\n", reg); +} + +static u64 __default_read_sys_reg(struct kvm_vcpu *vcpu, int reg) +{ + bad_sys_reg(reg); + return 0; +} + +static void __default_write_sys_reg(struct kvm_vcpu *vcpu, int reg, u64 val) +{ + bad_sys_reg(reg); +} + +/* Ordered as in enum vcpu_sysreg */ +DECLARE_IMMEDIATE_SR(MPIDR_EL1); +DECLARE_IMMEDIATE_SR(CSSELR_EL1); +DECLARE_IMMEDIATE_SR(SCTLR_EL1); +DECLARE_IMMEDIATE_SR(ACTLR_EL1); +DECLARE_IMMEDIATE_SR(CPACR_EL1); +DECLARE_IMMEDIATE_SR(TTBR0_EL1); +DECLARE_IMMEDIATE_SR(TTBR1_EL1); +DECLARE_IMMEDIATE_SR(TCR_EL1); +DECLARE_IMMEDIATE_SR(ESR_EL1); +DECLARE_IMMEDIATE_SR(AFSR0_EL1); +DECLARE_IMMEDIATE_SR(AFSR1_EL1); +DECLARE_IMMEDIATE_SR(FAR_EL1); +DECLARE_IMMEDIATE_SR(MAIR_EL1); +DECLARE_IMMEDIATE_SR(VBAR_EL1); +DECLARE_IMMEDIATE_SR(CONTEXTIDR_EL1); +DECLARE_IMMEDIATE_SR(TPIDR_EL0); +DECLARE_IMMEDIATE_SR(TPIDRRO_EL0); +DECLARE_IMMEDIATE_SR(TPIDR_EL1); +DECLARE_IMMEDIATE_SR(AMAIR_EL1); +DECLARE_IMMEDIATE_SR(CNTKCTL_EL1); +DECLARE_IMMEDIATE_SR(PAR_EL1); +DECLARE_IMMEDIATE_SR(MDSCR_EL1); +DECLARE_IMMEDIATE_SR(MDCCINT_EL1); +DECLARE_IMMEDIATE_SR(PMCR_EL0); +DECLARE_IMMEDIATE_SR(PMSELR_EL0); +DECLARE_IMMEDIATE_SR(PMEVCNTR0_EL0); +/* PMEVCNTR30_EL0 */ +DECLARE_IMMEDIATE_SR(PMCCNTR_EL0); +DECLARE_IMMEDIATE_SR(PMEVTYPER0_EL0); +/* PMEVTYPER30_EL0 */ +DECLARE_IMMEDIATE_SR(PMCCFILTR_EL0); +DECLARE_IMMEDIATE_SR(PMCNTENSET_EL0); +DECLARE_IMMEDIATE_SR(PMINTENSET_EL1); +DECLARE_IMMEDIATE_SR(PMOVSSET_EL0); +DECLARE_IMMEDIATE_SR(PMSWINC_EL0); +DECLARE_IMMEDIATE_SR(PMUSERENR_EL0); +DECLARE_IMMEDIATE_SR(DACR32_EL2); +DECLARE_IMMEDIATE_SR(IFSR32_EL2); +DECLARE_IMMEDIATE_SR(FPEXC32_EL2); +DECLARE_IMMEDIATE_SR(DBGVCR32_EL2); + +static const struct sys_reg_accessor sys_reg_accessors[NR_SYS_REGS] = { + [0 ... NR_SYS_REGS - 1] = { + .rdsr = __default_read_sys_reg, + .wrsr = __default_write_sys_reg, + }, + + SR_HANDLER(MPIDR_EL1), + SR_HANDLER(CSSELR_EL1), + SR_HANDLER(SCTLR_EL1), + SR_HANDLER(ACTLR_EL1), + SR_HANDLER(CPACR_EL1), + SR_HANDLER(TTBR0_EL1), + SR_HANDLER(TTBR1_EL1), + SR_HANDLER(TCR_EL1), + SR_HANDLER(ESR_EL1), + SR_HANDLER(AFSR0_EL1), + SR_HANDLER(AFSR1_EL1), + SR_HANDLER(FAR_EL1), + SR_HANDLER(MAIR_EL1), + SR_HANDLER(VBAR_EL1), + SR_HANDLER(CONTEXTIDR_EL1), + SR_HANDLER(TPIDR_EL0), + SR_HANDLER(TPIDRRO_EL0), + SR_HANDLER(TPIDR_EL1), + SR_HANDLER(AMAIR_EL1), + SR_HANDLER(CNTKCTL_EL1), + SR_HANDLER(PAR_EL1), + SR_HANDLER(MDSCR_EL1), + SR_HANDLER(MDCCINT_EL1), + SR_HANDLER(PMCR_EL0), + SR_HANDLER(PMSELR_EL0), + SR_HANDLER_RANGE(PMEVCNTR0_EL0, PMEVCNTR30_EL0), + SR_HANDLER(PMCCNTR_EL0), + SR_HANDLER_RANGE(PMEVTYPER0_EL0, PMEVTYPER30_EL0), + SR_HANDLER(PMCCFILTR_EL0), + SR_HANDLER(PMCNTENSET_EL0), + SR_HANDLER(PMINTENSET_EL1), + SR_HANDLER(PMOVSSET_EL0), + SR_HANDLER(PMSWINC_EL0), + SR_HANDLER(PMUSERENR_EL0), + SR_HANDLER(DACR32_EL2), + SR_HANDLER(IFSR32_EL2), + SR_HANDLER(FPEXC32_EL2), + SR_HANDLER(DBGVCR32_EL2), +}; + +u64 vcpu_read_sys_reg(struct kvm_vcpu *vcpu, int reg) +{ + return sys_reg_accessors[reg].rdsr(vcpu, reg); +} + +void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, int reg, u64 val) +{ + sys_reg_accessors[reg].wrsr(vcpu, reg, val); +} + /* 3 bits per cache level, as per CLIDR, but non-existent caches always 0 */ static u32 cache_levels;