From patchwork Fri Nov 22 22:58:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11258573 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BAAFC1390 for ; Fri, 22 Nov 2019 22:58:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9AA6020721 for ; Fri, 22 Nov 2019 22:58:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726992AbfKVW6f (ORCPT ); Fri, 22 Nov 2019 17:58:35 -0500 Received: from mga06.intel.com ([134.134.136.31]:15520 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726089AbfKVW6e (ORCPT ); Fri, 22 Nov 2019 17:58:34 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Nov 2019 14:58:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,231,1571727600"; d="scan'208";a="382219000" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2019 14:58:33 -0800 From: Sean Christopherson To: Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/3] KVM: x86: Add EMULTYPE_PF when emulation is triggered by a page fault Date: Fri, 22 Nov 2019 14:58:30 -0800 Message-Id: <20191122225832.26684-2-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191122225832.26684-1-sean.j.christopherson@intel.com> References: <20191122225832.26684-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a new emulation type flag to explicitly mark emulation related to a page fault. Use the new flag, EMULTYPE_PF, to move the propagation of the GPA into the emulator context from the page fault handler into x86_emulate_instruction. Similarly, to help clarify how cr2 is used, don't propagate cr2 into the exception.address when it's *not* valid and instead explicitly zero out exception.address. Functionally this is a nop, as all non-EMULTYPE_PF callers pass zero for cr2, i.e. it's effectively a documentation update. Zeroing exception.address is probably unnecessary as it shouldn't be exposed to the guest for non-#PF exceptions, but keep the current behavior to avoid introducing an unintentional function change. Append _PF to EMULTYPE_ALLOW_RETRY to make it clear retrying is only intended for use when emulating after a #PF, and add WARNs to enforce that notion. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 12 +++++++++--- arch/x86/kvm/mmu/mmu.c | 10 ++-------- arch/x86/kvm/x86.c | 25 +++++++++++++++++++------ 3 files changed, 30 insertions(+), 17 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index b79cd6aa4075..6311fffbc2f0 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1347,8 +1347,9 @@ extern u64 kvm_mce_cap_supported; * decode the instruction length. For use *only* by * kvm_x86_ops->skip_emulated_instruction() implementations. * - * EMULTYPE_ALLOW_RETRY - Set when the emulator should resume the guest to - * retry native execution under certain conditions. + * EMULTYPE_ALLOW_RETRY_PF - Set when the emulator should resume the guest to + * retry native execution under certain conditions, + * Can only be set in conjunction with EMULTYPE_PF. * * EMULTYPE_TRAP_UD_FORCED - Set when emulating an intercepted #UD that was * triggered by KVM's magic "force emulation" prefix, @@ -1361,13 +1362,18 @@ extern u64 kvm_mce_cap_supported; * backdoor emulation, which is opt in via module param. * VMware backoor emulation handles select instructions * and reinjects the #GP for all other cases. + * + * EMULTYPE_PF - Set when emulating MMIO by way of an intercepted #PF, in which + * case the CR2/GPA value pass on the stack is valid. */ #define EMULTYPE_NO_DECODE (1 << 0) #define EMULTYPE_TRAP_UD (1 << 1) #define EMULTYPE_SKIP (1 << 2) -#define EMULTYPE_ALLOW_RETRY (1 << 3) +#define EMULTYPE_ALLOW_RETRY_PF (1 << 3) #define EMULTYPE_TRAP_UD_FORCED (1 << 4) #define EMULTYPE_VMWARE_GP (1 << 5) +#define EMULTYPE_PF (1 << 6) + int kvm_emulate_instruction(struct kvm_vcpu *vcpu, int emulation_type); int kvm_emulate_instruction_from_buffer(struct kvm_vcpu *vcpu, void *insn, int insn_len); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 6f92b40d798c..fa49d9971b59 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5519,15 +5519,9 @@ static int make_mmu_pages_available(struct kvm_vcpu *vcpu) int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code, void *insn, int insn_len) { - int r, emulation_type = 0; + int r, emulation_type = EMULTYPE_PF; bool direct = vcpu->arch.mmu->direct_map; - /* With shadow page tables, fault_address contains a GVA or nGPA. */ - if (vcpu->arch.mmu->direct_map) { - vcpu->arch.gpa_available = true; - vcpu->arch.gpa_val = cr2; - } - r = RET_PF_INVALID; if (unlikely(error_code & PFERR_RSVD_MASK)) { r = handle_mmio_page_fault(vcpu, cr2, direct); @@ -5572,7 +5566,7 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code, * for L1 isn't going to magically fix whatever issue cause L2 to fail. */ if (!mmio_info_in_cache(vcpu, cr2, direct) && !is_guest_mode(vcpu)) - emulation_type = EMULTYPE_ALLOW_RETRY; + emulation_type |= EMULTYPE_ALLOW_RETRY_PF; emulate: /* * On AMD platforms, under certain conditions insn_len may be zero on #NPF. diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index a256e09f321a..848e4ec21c5e 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6393,10 +6393,11 @@ static bool reexecute_instruction(struct kvm_vcpu *vcpu, gva_t cr2, gpa_t gpa = cr2; kvm_pfn_t pfn; - if (!(emulation_type & EMULTYPE_ALLOW_RETRY)) + if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; - if (WARN_ON_ONCE(is_guest_mode(vcpu))) + if (WARN_ON_ONCE(is_guest_mode(vcpu)) || + WARN_ON_ONCE(!(emulation_type & EMULTYPE_PF))) return false; if (!vcpu->arch.mmu->direct_map) { @@ -6484,10 +6485,11 @@ static bool retry_instruction(struct x86_emulate_ctxt *ctxt, */ vcpu->arch.last_retry_eip = vcpu->arch.last_retry_addr = 0; - if (!(emulation_type & EMULTYPE_ALLOW_RETRY)) + if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; - if (WARN_ON_ONCE(is_guest_mode(vcpu))) + if (WARN_ON_ONCE(is_guest_mode(vcpu)) || + WARN_ON_ONCE(!(emulation_type & EMULTYPE_PF))) return false; if (x86_page_table_writing_insn(ctxt)) @@ -6742,8 +6744,19 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, } restart: - /* Save the faulting GPA (cr2) in the address field */ - ctxt->exception.address = cr2; + if (emulation_type & EMULTYPE_PF) { + /* Save the faulting GPA (cr2) in the address field */ + ctxt->exception.address = cr2; + + /* With shadow page tables, cr2 contains a GVA or nGPA. */ + if (vcpu->arch.mmu->direct_map) { + vcpu->arch.gpa_available = true; + vcpu->arch.gpa_val = cr2; + } + } else { + /* Sanitize the address out of an abundance of paranoia. */ + ctxt->exception.address = 0; + } r = x86_emulate_insn(ctxt); From patchwork Fri Nov 22 22:58:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11258579 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9650A14DB for ; Fri, 22 Nov 2019 22:58:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7EFC620721 for ; Fri, 22 Nov 2019 22:58:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726880AbfKVW6e (ORCPT ); Fri, 22 Nov 2019 17:58:34 -0500 Received: from mga06.intel.com ([134.134.136.31]:15520 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726089AbfKVW6d (ORCPT ); Fri, 22 Nov 2019 17:58:33 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Nov 2019 14:58:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,231,1571727600"; d="scan'208";a="382219002" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2019 14:58:33 -0800 From: Sean Christopherson To: Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/3] KVM: x86: Move gpa_val and gpa_available into the emulator context Date: Fri, 22 Nov 2019 14:58:31 -0800 Message-Id: <20191122225832.26684-3-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191122225832.26684-1-sean.j.christopherson@intel.com> References: <20191122225832.26684-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Move the GPA tracking into the emulator context to prevent it from being abused in the standard page fault flow. All paths into the emulator go through init_emulate_ctxt(), so there's no need to clear gpa_available after every VM-Exit. The only partial exception is EMULTYPE_NO_DECODE, but in that case the emulator is simply being re-entered, i.e. retaining gpa_available is correct and consistent with current behavior. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_emulate.h | 4 ++++ arch/x86/include/asm/kvm_host.h | 4 ---- arch/x86/kvm/x86.c | 13 ++++++------- 3 files changed, 10 insertions(+), 11 deletions(-) diff --git a/arch/x86/include/asm/kvm_emulate.h b/arch/x86/include/asm/kvm_emulate.h index 77cf6c11f66b..e81658a4ab9d 100644 --- a/arch/x86/include/asm/kvm_emulate.h +++ b/arch/x86/include/asm/kvm_emulate.h @@ -307,6 +307,10 @@ struct x86_emulate_ctxt { bool have_exception; struct x86_exception exception; + /* GPA available */ + bool gpa_available; + gpa_t gpa_val; + /* * decode cache */ diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 6311fffbc2f0..e434f4cfecd1 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -791,10 +791,6 @@ struct kvm_vcpu_arch { int pending_ioapic_eoi; int pending_external_vector; - /* GPA available */ - bool gpa_available; - gpa_t gpa_val; - /* be preempted when it's in kernel-mode(cpl=0) */ bool preempted_in_kernel; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 848e4ec21c5e..9a8adfdf1e0a 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5664,10 +5664,9 @@ static int emulator_read_write_onepage(unsigned long addr, void *val, * operation using rep will only have the initial GPA from the NPF * occurred. */ - if (vcpu->arch.gpa_available && - emulator_can_use_gpa(ctxt) && - (addr & ~PAGE_MASK) == (vcpu->arch.gpa_val & ~PAGE_MASK)) { - gpa = vcpu->arch.gpa_val; + if (ctxt->gpa_available && emulator_can_use_gpa(ctxt) && + (addr & ~PAGE_MASK) == (ctxt->gpa_val & ~PAGE_MASK)) { + gpa = ctxt->gpa_val; ret = vcpu_is_mmio_gpa(vcpu, addr, gpa, write); } else { ret = vcpu_mmio_gva_to_gpa(vcpu, addr, &gpa, exception, write); @@ -6318,6 +6317,7 @@ static void init_emulate_ctxt(struct kvm_vcpu *vcpu) kvm_x86_ops->get_cs_db_l_bits(vcpu, &cs_db, &cs_l); + ctxt->gpa_available = false; ctxt->eflags = kvm_get_rflags(vcpu); ctxt->tf = (ctxt->eflags & X86_EFLAGS_TF) != 0; @@ -6750,8 +6750,8 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, /* With shadow page tables, cr2 contains a GVA or nGPA. */ if (vcpu->arch.mmu->direct_map) { - vcpu->arch.gpa_available = true; - vcpu->arch.gpa_val = cr2; + ctxt->gpa_available = true; + ctxt->gpa_val = cr2; } } else { /* Sanitize the address out of an abundance of paranoia. */ @@ -8306,7 +8306,6 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (vcpu->arch.apic_attention) kvm_lapic_sync_from_vapic(vcpu); - vcpu->arch.gpa_available = false; r = kvm_x86_ops->handle_exit(vcpu); return r; From patchwork Fri Nov 22 22:58:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11258575 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1CA8B1390 for ; Fri, 22 Nov 2019 22:58:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 060D020721 for ; Fri, 22 Nov 2019 22:58:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726967AbfKVW6e (ORCPT ); Fri, 22 Nov 2019 17:58:34 -0500 Received: from mga06.intel.com ([134.134.136.31]:15520 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726937AbfKVW6e (ORCPT ); Fri, 22 Nov 2019 17:58:34 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Nov 2019 14:58:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,231,1571727600"; d="scan'208";a="382219004" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2019 14:58:33 -0800 From: Sean Christopherson To: Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] KVM: x86: Move #PF retry tracking variables into emulation context Date: Fri, 22 Nov 2019 14:58:32 -0800 Message-Id: <20191122225832.26684-4-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191122225832.26684-1-sean.j.christopherson@intel.com> References: <20191122225832.26684-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Move last_retry_eip and last_retry_addr into the emulation context as they are specific to retrying an instruction after emulation failure. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_emulate.h | 4 ++++ arch/x86/include/asm/kvm_host.h | 3 --- arch/x86/kvm/x86.c | 11 ++++++----- 3 files changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/kvm_emulate.h b/arch/x86/include/asm/kvm_emulate.h index e81658a4ab9d..9c5db3b4120e 100644 --- a/arch/x86/include/asm/kvm_emulate.h +++ b/arch/x86/include/asm/kvm_emulate.h @@ -311,6 +311,10 @@ struct x86_emulate_ctxt { bool gpa_available; gpa_t gpa_val; + /* Track EIP and CR2/GPA when retrying faulting instruction on #PF. */ + unsigned long last_retry_eip; + unsigned long last_retry_addr; + /* * decode cache */ diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index e434f4cfecd1..6c8bfebabc31 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -745,9 +745,6 @@ struct kvm_vcpu_arch { cpumask_var_t wbinvd_dirty_mask; - unsigned long last_retry_eip; - unsigned long last_retry_addr; - struct { bool halted; gfn_t gfns[roundup_pow_of_two(ASYNC_PF_PER_VCPU)]; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9a8adfdf1e0a..3aa2d7d98779 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6317,6 +6317,7 @@ static void init_emulate_ctxt(struct kvm_vcpu *vcpu) kvm_x86_ops->get_cs_db_l_bits(vcpu, &cs_db, &cs_l); + /* last_retry_{eip,addr} are persistent and must not be init'd here. */ ctxt->gpa_available = false; ctxt->eflags = kvm_get_rflags(vcpu); ctxt->tf = (ctxt->eflags & X86_EFLAGS_TF) != 0; @@ -6467,8 +6468,8 @@ static bool retry_instruction(struct x86_emulate_ctxt *ctxt, struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt); unsigned long last_retry_eip, last_retry_addr, gpa = cr2; - last_retry_eip = vcpu->arch.last_retry_eip; - last_retry_addr = vcpu->arch.last_retry_addr; + last_retry_eip = ctxt->last_retry_eip; + last_retry_addr = ctxt->last_retry_addr; /* * If the emulation is caused by #PF and it is non-page_table @@ -6483,7 +6484,7 @@ static bool retry_instruction(struct x86_emulate_ctxt *ctxt, * and the address again, we can break out of the potential infinite * loop. */ - vcpu->arch.last_retry_eip = vcpu->arch.last_retry_addr = 0; + ctxt->last_retry_eip = ctxt->last_retry_addr = 0; if (!(emulation_type & EMULTYPE_ALLOW_RETRY_PF)) return false; @@ -6498,8 +6499,8 @@ static bool retry_instruction(struct x86_emulate_ctxt *ctxt, if (ctxt->eip == last_retry_eip && last_retry_addr == cr2) return false; - vcpu->arch.last_retry_eip = ctxt->eip; - vcpu->arch.last_retry_addr = cr2; + ctxt->last_retry_eip = ctxt->eip; + ctxt->last_retry_addr = cr2; if (!vcpu->arch.mmu->direct_map) gpa = kvm_mmu_gva_to_gpa_write(vcpu, cr2, NULL);