From patchwork Wed Jun 9 23:42:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D815C48BCD for ; Wed, 9 Jun 2021 23:43:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4B4F8613EF for ; Wed, 9 Jun 2021 23:43:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229865AbhFIXo7 (ORCPT ); Wed, 9 Jun 2021 19:44:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229507AbhFIXo4 (ORCPT ); Wed, 9 Jun 2021 19:44:56 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45FE8C061574 for ; Wed, 9 Jun 2021 16:42:45 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id u7-20020a259b470000b02904dca50820c2so33538153ybo.11 for ; Wed, 09 Jun 2021 16:42:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=VEMIxquREMiHOnk/Jwp5zKxKYIE/fmUnZrtCHKTS9JE=; b=cbLQOldlrEJpcz3ehwNa0GO97G4YQJTH0mM5y5ltEc8E+CH1uqML8Bj64K0EVu30JJ lQin+4Y57CJR9RBI/i6RMN2srl3OySvH7RVzRiYUXSqYZJ7/2rUS7VBGCUFDX6eVOGGV gorWsAPUaEBxLAmCAr0nMY/SJik9PG7v4ogEteRbF+T+iq3TpHt0ifU6AY2Hd+2QSAtr wwmzNLmP+dWZqu/w6oaNsUN8maJbimA0pnE8qGCoDj6PUjz5CWqnR2UuqdRQqXhH+UWi 9Wzi29AG3Yrug992Nbg8VDpxOjSnIoR00AXzvMuZg4FabugdtEYLObZEeBtqjJbh5MmE HGew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=VEMIxquREMiHOnk/Jwp5zKxKYIE/fmUnZrtCHKTS9JE=; b=N+A43LTnqGJHU8nQ0eoAeRyeGNHcxGsmXUHNgfJpjThoplLG8vBMgI694KZd6ZiJro 0bDkW+4czr6xNyZTQSUyAZzW4AZv0iuOSM5tga6z+0Xs6VU4LhKWwA5pD/jqkIs/4RwI D0Y/xysJWQjaXCHAiE6Nk9eUEHVaFZx5JioSthTdNIkbfOwNNWmFxzVnFuxPcp37Ipfr Fj49CKiURXjVdcIJeqYj6hKXR35k/EUhQsPfGCpg+AgiL9+dspACUl/NVfJQDWJfjuG3 st4QgpvJMDN6/Vv7wpD1GXKH76LRdQ4e5h66rKndswa+iqeIMkhV8ifcxtVTAisr6o4t tUTQ== X-Gm-Message-State: AOAM530NL6Gi6RhXjmtSZIPDJsfz8C81aLz5bGVHcYqnlod6c1eVEk9k PPQZiA5yUX6zS/ke7QdgT2wOhnR8fmc= X-Google-Smtp-Source: ABdhPJzQvihd2dKUwtYo1j9tsKMwkQ02mHyj3HmBM1skP2T35iB6n8uIriD5vHGZbWoTiEXqps95CbyFTp8= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:a25:4fc4:: with SMTP id d187mr3417113ybb.245.1623282164446; Wed, 09 Jun 2021 16:42:44 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:21 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-2-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 01/15] KVM: nVMX: Sync all PGDs on nested transition with shadow paging From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Trigger a full TLB flush on behalf of the guest on nested VM-Enter and VM-Exit when VPID is disabled for L2. kvm_mmu_new_pgd() syncs only the current PGD, which can theoretically leave stale, unsync'd entries in a previous guest PGD, which could be consumed if L2 is allowed to load CR3 with PCID_NOFLUSH=1. Rename KVM_REQ_HV_TLB_FLUSH to KVM_REQ_TLB_FLUSH_GUEST so that it can be utilized for its obvious purpose of emulating a guest TLB flush. Note, there is no change the actual TLB flush executed by KVM, even though the fast PGD switch uses KVM_REQ_TLB_FLUSH_CURRENT. When VPID is disabled for L2, vpid02 is guaranteed to be '0', and thus nested_get_vpid02() will return the VPID that is shared by L1 and L2. Generate the request outside of kvm_mmu_new_pgd(), as getting the common helper to correctly identify which requested is needed is quite painful. E.g. using KVM_REQ_TLB_FLUSH_GUEST when nested EPT is in play is wrong as a TLB flush from the L1 kernel's perspective does not invalidate EPT mappings. And, by using KVM_REQ_TLB_FLUSH_GUEST, nVMX can do future simplification by moving the logic into nested_vmx_transition_tlb_flush(). Fixes: 41fab65e7c44 ("KVM: nVMX: Skip MMU sync on nested VMX transition when possible") Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/hyperv.c | 2 +- arch/x86/kvm/vmx/nested.c | 17 ++++++++++++----- arch/x86/kvm/x86.c | 2 +- 4 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 9c7ced0e3171..6652e51a86fd 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -85,7 +85,7 @@ #define KVM_REQ_APICV_UPDATE \ KVM_ARCH_REQ_FLAGS(25, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) #define KVM_REQ_TLB_FLUSH_CURRENT KVM_ARCH_REQ(26) -#define KVM_REQ_HV_TLB_FLUSH \ +#define KVM_REQ_TLB_FLUSH_GUEST \ KVM_ARCH_REQ_FLAGS(27, KVM_REQUEST_NO_WAKEUP) #define KVM_REQ_APF_READY KVM_ARCH_REQ(28) #define KVM_REQ_MSR_FILTER_CHANGED KVM_ARCH_REQ(29) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index f00830e5202f..fdd1eca717fd 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -1704,7 +1704,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, u64 ingpa, u16 rep_cnt, bool * vcpu->arch.cr3 may not be up-to-date for running vCPUs so we can't * analyze it here, flush TLB regardless of the specified address space. */ - kvm_make_vcpus_request_mask(kvm, KVM_REQ_HV_TLB_FLUSH, + kvm_make_vcpus_request_mask(kvm, KVM_REQ_TLB_FLUSH_GUEST, NULL, vcpu_mask, &hv_vcpu->tlb_flush); ret_success: diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 6058a65a6ede..1c243758dd2c 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -1127,12 +1127,19 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, bool ne /* * Unconditionally skip the TLB flush on fast CR3 switch, all TLB - * flushes are handled by nested_vmx_transition_tlb_flush(). See - * nested_vmx_transition_mmu_sync for details on skipping the MMU sync. + * flushes are handled by nested_vmx_transition_tlb_flush(). */ - if (!nested_ept) - kvm_mmu_new_pgd(vcpu, cr3, true, - !nested_vmx_transition_mmu_sync(vcpu)); + if (!nested_ept) { + kvm_mmu_new_pgd(vcpu, cr3, true, true); + + /* + * A TLB flush on VM-Enter/VM-Exit flushes all linear mappings + * across all PCIDs, i.e. all PGDs need to be synchronized. + * See nested_vmx_transition_mmu_sync() for more details. + */ + if (nested_vmx_transition_mmu_sync(vcpu)) + kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); + } vcpu->arch.cr3 = cr3; kvm_register_mark_available(vcpu, VCPU_EXREG_CR3); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9dd23bdfc6cc..905de6854efa 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9167,7 +9167,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) } if (kvm_check_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu)) kvm_vcpu_flush_tlb_current(vcpu); - if (kvm_check_request(KVM_REQ_HV_TLB_FLUSH, vcpu)) + if (kvm_check_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu)) kvm_vcpu_flush_tlb_guest(vcpu); if (kvm_check_request(KVM_REQ_REPORT_TPR_ACCESS, vcpu)) { From patchwork Wed Jun 9 23:42:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311471 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 096F8C48BCD for ; Wed, 9 Jun 2021 23:44:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E509261249 for ; Wed, 9 Jun 2021 23:44:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230336AbhFIXp5 (ORCPT ); Wed, 9 Jun 2021 19:45:57 -0400 Received: from mail-qt1-f202.google.com ([209.85.160.202]:43552 "EHLO mail-qt1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230334AbhFIXpz (ORCPT ); Wed, 9 Jun 2021 19:45:55 -0400 Received: by mail-qt1-f202.google.com with SMTP id z20-20020ac845540000b0290248cbf50215so4901907qtn.10 for ; Wed, 09 Jun 2021 16:43:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=LN0JaAN3T5+nq6SFVxphdupyoLpcoEoPHxPPBl9bv9U=; b=Tla+YY0L+7EOHZs/dedTljFCSPp26cpMfl30zgXQ+OLUmSM/PUJMr+izjPlZ5UJBMu ii9xWBIsKqjss/9j8fA5GoeSocniFYkln61UJap8MPcxp8XXdZNYdjT+DktrczynrTXk 4HvgjORzpc4CzCRf9zsyU+RFcAUMMJ+8LxkVAxs+a6CgiwkBddTyvEvlApbvbFofThvp rrt4mM3//Ck/2FGMMZAQwtCbbly+YjzzPuIH3ZlPjGVcZdM4CpmpO/ampQMKhX9XmR70 WQJyu/ntDcu4dKxhl0wNDj2ePL+0rNI1SIf02fEGC8Z+Zp/DNuvjBwXhnAk4lKc+eLDz zpvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=LN0JaAN3T5+nq6SFVxphdupyoLpcoEoPHxPPBl9bv9U=; b=Z765bhY0x+ZcrHg3DuIjuL3Zvf3lWqIf3V8i5Tnq4JV+2jlGLKFiW1C4Mk31tMZRro h0tZiLm3oj6FKg4KxRCRRyJHwGbdW3Bet//h8wg9deArEOG+lCc6TjNXcVe4uirpvxV3 uAZBSIhJOieYmP9ALEjDf1MWlTEYTk9Tu6dBuI6PuHk5YFQunGrq/tF/ktgYp7PaK2lq JAgBuviu9uvjiFp+SG6kZCipEF1puHFLxeMcdxXLOhEd4Y2sepKwAe8OByTmFQ0YB/AQ tmtWqvUa5Vcrg45cI+ms4zHfx09mUI+o6O6huRqgyXzEul+5MSnI/UyVCwCfkd/a/N1G b3PA== X-Gm-Message-State: AOAM533r4SJqkCtVxyHaP7VZLPE1XGIoRwRDeUFnsaHXLFUNyICPOEq5 a0BzwFdRl+RyhHaHQ4Tb0di9RO0xNhk= X-Google-Smtp-Source: ABdhPJy1wdnajK8auCAaIPjf4hKWJO3bnez97krGV/zeyw/JEJzMfsrA/ZOHXieVRy2GU1OrEiucYXPEmVQ= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:ad4:4c0c:: with SMTP id bz12mr2346916qvb.21.1623282166720; Wed, 09 Jun 2021 16:42:46 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:22 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-3-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 02/15] KVM: nVMX: Ensure 64-bit shift when checking VMFUNC bitmap From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use BIT_ULL() instead of an open-coded shift to check whether or not a function is enabled in L1's VMFUNC bitmap. This is a benign bug as KVM supports only bit 0, and will fail VM-Enter if any other bits are set, i.e. bits 63:32 are guaranteed to be zero. Note, "function" is bounded by hardware as VMFUNC will #UD before taking a VM-Exit if the function is greater than 63. Before: if ((vmcs12->vm_function_control & (1 << function)) == 0) 0x000000000001a916 <+118>: mov $0x1,%eax 0x000000000001a91b <+123>: shl %cl,%eax 0x000000000001a91d <+125>: cltq 0x000000000001a91f <+127>: and 0x128(%rbx),%rax After: if (!(vmcs12->vm_function_control & BIT_ULL(function & 63))) 0x000000000001a955 <+117>: mov 0x128(%rbx),%rdx 0x000000000001a95c <+124>: bt %rax,%rdx Fixes: 27c42a1bb867 ("KVM: nVMX: Enable VMFUNC for the L1 hypervisor") Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 1c243758dd2c..c3624109ffeb 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -5540,7 +5540,7 @@ static int handle_vmfunc(struct kvm_vcpu *vcpu) } vmcs12 = get_vmcs12(vcpu); - if ((vmcs12->vm_function_control & (1 << function)) == 0) + if (!(vmcs12->vm_function_control & BIT_ULL(function))) goto fail; switch (function) { From patchwork Wed Jun 9 23:42:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEBE1C48BDF for ; Wed, 9 Jun 2021 23:43:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AE9F961249 for ; Wed, 9 Jun 2021 23:43:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229976AbhFIXpC (ORCPT ); Wed, 9 Jun 2021 19:45:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48358 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229947AbhFIXpB (ORCPT ); Wed, 9 Jun 2021 19:45:01 -0400 Received: from mail-qt1-x849.google.com (mail-qt1-x849.google.com [IPv6:2607:f8b0:4864:20::849]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B4B7C0617A8 for ; Wed, 9 Jun 2021 16:42:50 -0700 (PDT) Received: by mail-qt1-x849.google.com with SMTP id c29-20020ac86e9d0000b0290247b267c8e4so5903285qtv.22 for ; Wed, 09 Jun 2021 16:42:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=zb4BgBHrj3z234wVPkDQ98HLS1UUgQ6DIJZgVdXDBds=; b=g/HXga9GcQ4nONrdSa0q1CxhKd1rennpVPxoeipm+a5+/DzEA9X2i5YtsuWEXG25t3 DnnDVM2jong448xcz0BXsCL7LMlZR+ZRN6Px3RPGIoNK1CyDjsS8Ob1ml/97DORJzTln i4ZkNF86Luqpqza7MDEIUFYqLFrF57CK6B8qmjhr+WUmuxF7kYN0qibLRf8EPy2rt2C4 0T/PJ2PtXbyTfm/VvXdLAjO3iF9WrfEmIQr3qScTeHscTyn5htdTSOm8aWzvu3h3rVEO Pg34qcOTy02QviOOM4AXNOzw0FxNF0KvoG8F5ZIIJpjqfuKx6U6x37xYZgH0Mpe33pmo GgFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=zb4BgBHrj3z234wVPkDQ98HLS1UUgQ6DIJZgVdXDBds=; b=ZDLRFO0itzM+sNRwQaBEoz2cRYcf1HMkay/GVTKChlv8PnGHHTXaLWbJjjymErFgYA x5yesRt4aPSSq3V6X60hyzWdhxivKQLe/NRLpAbQfKG+GPmvBFdSpO6/GJa1RaMEXFD4 yiNqVbzqkVjpm0eKI37xeJxAONUgnedpcc3wY8+nUmKiGof+84HayZpPGWZthQLIpyCz UVjuOl0JdaveHyZXtCvnQV65U+UZufUNjDIeKlDRKTjB4bGmqZOAmXGaB7rHN+FIlbqD 2CrZmsjOBo+RfMwgY2FPyzWA4yqsBIVLHFG0NaqvlcNlI5LLNnk5s3F6gNmIkmGurt4j FeEg== X-Gm-Message-State: AOAM53192hExNv7UGecescA8A5U2qCsz91mNsDaKVdauZOU+9w7mxWLt Bp7rjAuUYxCZc4Aw0r8477GnT8bDiUI= X-Google-Smtp-Source: ABdhPJzLiGRqYKhQpiJw4Ox9Mpuyg4GqiBUdrg+GEm8Q6sIJdKTPuIUixfOa55zZay0YzPthMxaoysV9S9E= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:a0c:ea83:: with SMTP id d3mr2600334qvp.25.1623282169148; Wed, 09 Jun 2021 16:42:49 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:23 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-4-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 03/15] KVM: nVMX: Don't clobber nested MMU's A/D status on EPTP switch From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Drop bogus logic that incorrectly clobbers the accessed/dirty enabling status of the nested MMU on an EPTP switch. When nested EPT is enabled, walk_mmu points at L2's _legacy_ page tables, not L1's EPT for L2. This is likely a benign bug, as mmu->ept_ad is never consumed (since the MMU is not a nested EPT MMU), and stuffing mmu_role.base.ad_disabled will never propagate into future shadow pages since the nested MMU isn't used to map anything, just to walk L2's page tables. Note, KVM also does a full MMU reload, i.e. the guest_mmu will be recreated using the new EPTP, and thus any change in A/D enabling will be properly recognized in the relevant MMU. Fixes: 41ab93727467 ("KVM: nVMX: Emulate EPTP switching for the L1 hypervisor") Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index c3624109ffeb..e102a5c10a83 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -5488,8 +5488,6 @@ static int nested_vmx_eptp_switching(struct kvm_vcpu *vcpu, { u32 index = kvm_rcx_read(vcpu); u64 new_eptp; - bool accessed_dirty; - struct kvm_mmu *mmu = vcpu->arch.walk_mmu; if (!nested_cpu_has_eptp_switching(vmcs12) || !nested_cpu_has_ept(vmcs12)) @@ -5498,13 +5496,10 @@ static int nested_vmx_eptp_switching(struct kvm_vcpu *vcpu, if (index >= VMFUNC_EPTP_ENTRIES) return 1; - if (kvm_vcpu_read_guest_page(vcpu, vmcs12->eptp_list_address >> PAGE_SHIFT, &new_eptp, index * 8, 8)) return 1; - accessed_dirty = !!(new_eptp & VMX_EPTP_AD_ENABLE_BIT); - /* * If the (L2) guest does a vmfunc to the currently * active ept pointer, we don't have to do anything else @@ -5513,8 +5508,6 @@ static int nested_vmx_eptp_switching(struct kvm_vcpu *vcpu, if (!nested_vmx_check_eptp(vcpu, new_eptp)) return 1; - mmu->ept_ad = accessed_dirty; - mmu->mmu_role.base.ad_disabled = !accessed_dirty; vmcs12->ept_pointer = new_eptp; kvm_make_request(KVM_REQ_MMU_RELOAD, vcpu); From patchwork Wed Jun 9 23:42:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF434C48BCF for ; Wed, 9 Jun 2021 23:44:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B8304613F0 for ; Wed, 9 Jun 2021 23:44:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230396AbhFIXqF (ORCPT ); Wed, 9 Jun 2021 19:46:05 -0400 Received: from mail-yb1-f201.google.com ([209.85.219.201]:40763 "EHLO mail-yb1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230346AbhFIXqC (ORCPT ); Wed, 9 Jun 2021 19:46:02 -0400 Received: by mail-yb1-f201.google.com with SMTP id 67-20020a2514460000b029053a9edba2a6so33544623ybu.7 for ; Wed, 09 Jun 2021 16:43:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=InBM03qDuJbydmZqwKqzqlUqaOkn74aCL0GR1gl20pw=; b=OK+CIFOEMmoa4Kw3KkKINDvQzlgD8gX0zYTY8FBWyC46kD/n+9gxplMvqumOjgY1CE Wo5O/YldyO4iNeEpEb7uNc9k6P6jNei2jjK0Qii28yktq7sUHNQGuFqOHg6RkNOVFaKu E5RD1Pca5/CZxyWCI6bI1u10LwqNiMOQP4D0GqDcBIHq/AOUg+aIFWJeu+Hw+3J3DZi2 YWgzA/y5SeMn3aPXL/3xID0uh/w9hF1HyaK6TIKKLjNS23UYqlWLBGjkxbOylGUEuHX1 IerVw+DBVdvOkBghKaf9PTu/hgzzdtjnDoADh+g8++2oqHVt37xIdX5+TF2JPTt1L7bL VwJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=InBM03qDuJbydmZqwKqzqlUqaOkn74aCL0GR1gl20pw=; b=r6ew572mS9ZWwc5nOb4RayrUVVqS9CEM69q5cFygesmt9YqdTbvygbczXvAIfhpVwf BeW0QGNirp/Ahh0f3JxtzY/o+TPht9WV4XgjHMBCGM+DoQb5LJ6IGwK7jSo8V3SKhtCn SfmR1L/DPThTncNTzB+IlPjtXUq0uatuE6gkOHQBNajJo0+rSjvEHLrwTcrlUiL5ItRz agS5KrNYfD190Ht4/Gh2wmIWiTGj5GDs9PsiWCoEdklCic28K3N7uJsv7LJ+26qiDX5H 3bzrJWsNo5KI9IcRwGz/CE7i29h8HvDFDlwMtXPW9pC/1lCpnGENvM8b9vBAPdjIT9WS rWgw== X-Gm-Message-State: AOAM533UkdH0rU0y/8LTPl7SyJyZ4DDFo0a7ahl3EjfElD5tQPOpGD9/ stPuskyQqcy+O2nSdcWqSQBpiZi5+zk= X-Google-Smtp-Source: ABdhPJwGy0x6e/I2p/w7EqDxykx8RTBdU/hpX+eMYgqPqRSpSwFjGlBUKKapmirqU2vewWECZ2SoYSpawm0= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:a25:8709:: with SMTP id a9mr3770356ybl.395.1623282171518; Wed, 09 Jun 2021 16:42:51 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:24 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-5-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 04/15] KVM: x86: Invalidate all PGDs for the current PCID on MOV CR3 w/ flush From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Flush and sync all PGDs for the current/target PCID on MOV CR3 with a TLB flush, i.e. without PCID_NOFLUSH set. Paraphrasing Intel's SDM regarding the behavior of MOV to CR3: - If CR4.PCIDE = 0, invalidates all TLB entries associated with PCID 000H and all entries in all paging-structure caches associated with PCID 000H. - If CR4.PCIDE = 1 and NOFLUSH=0, invalidates all TLB entries associated with the PCID specified in bits 11:0, and all entries in all paging-structure caches associated with that PCID. It is not required to invalidate entries in the TLBs and paging-structure caches that are associated with other PCIDs. - If CR4.PCIDE=1 and NOFLUSH=1, is not required to invalidate any TLB entries or entries in paging-structure caches. Extract and reuse the logic for INVPCID(single) which is effectively the same flow and works even if CR4.PCIDE=0, as the current PCID will be '0' in that case, thus honoring the requirement of flushing PCID=0. Continue passing skip_tlb_flush to kvm_mmu_new_pgd() even though it _should_ be redundant; the clean up will be done in a future patch. The overhead of an unnecessary nop sync is minimal (especially compared to the actual sync), and the TLB flush is handled via request. Avoiding the the negligible overhead is not worth the risk of breaking kernels that backport the fix. Fixes: 956bf3531fba ("kvm: x86: Skip shadow page resync on CR3 switch when indicated by guest") Cc: Junaid Shahid Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 69 ++++++++++++++++++++++++++++------------------ 1 file changed, 42 insertions(+), 27 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 905de6854efa..e2f6d6a1ba54 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1084,25 +1084,45 @@ int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) } EXPORT_SYMBOL_GPL(kvm_set_cr4); +static void kvm_invalidate_pcid(struct kvm_vcpu *vcpu, unsigned long pcid) +{ + struct kvm_mmu *mmu = vcpu->arch.mmu; + unsigned long roots_to_free = 0; + int i; + + /* + * If neither the current CR3 nor any of the prev_roots use the given + * PCID, then nothing needs to be done here because a resync will + * happen anyway before switching to any other CR3. + */ + if (kvm_get_active_pcid(vcpu) == pcid) { + kvm_mmu_sync_roots(vcpu); + kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu); + } + + for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) + if (kvm_get_pcid(vcpu, mmu->prev_roots[i].pgd) == pcid) + roots_to_free |= KVM_MMU_ROOT_PREVIOUS(i); + + kvm_mmu_free_roots(vcpu, mmu, roots_to_free); +} + int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) { bool skip_tlb_flush = false; + unsigned long pcid = 0; #ifdef CONFIG_X86_64 bool pcid_enabled = kvm_read_cr4_bits(vcpu, X86_CR4_PCIDE); if (pcid_enabled) { skip_tlb_flush = cr3 & X86_CR3_PCID_NOFLUSH; cr3 &= ~X86_CR3_PCID_NOFLUSH; + pcid = cr3 & X86_CR3_PCID_MASK; } #endif - if (cr3 == kvm_read_cr3(vcpu) && !pdptrs_changed(vcpu)) { - if (!skip_tlb_flush) { - kvm_mmu_sync_roots(vcpu); - kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu); - } - return 0; - } + if (cr3 == kvm_read_cr3(vcpu) && !pdptrs_changed(vcpu)) + goto handle_tlb_flush; /* * Do not condition the GPA check on long mode, this helper is used to @@ -1115,10 +1135,23 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) if (is_pae_paging(vcpu) && !load_pdptrs(vcpu, vcpu->arch.walk_mmu, cr3)) return 1; - kvm_mmu_new_pgd(vcpu, cr3, skip_tlb_flush, skip_tlb_flush); + if (cr3 != kvm_read_cr3(vcpu)) + kvm_mmu_new_pgd(vcpu, cr3, skip_tlb_flush, skip_tlb_flush); + vcpu->arch.cr3 = cr3; kvm_register_mark_available(vcpu, VCPU_EXREG_CR3); +handle_tlb_flush: + /* + * A load of CR3 that flushes the TLB flushes only the current PCID, + * even if PCID is disabled, in which case PCID=0 is flushed. It's a + * moot point in the end because _disabling_ PCID will flush all PCIDs, + * and it's impossible to use a non-zero PCID when PCID is disabled, + * i.e. only PCID=0 can be relevant. + */ + if (!skip_tlb_flush) + kvm_invalidate_pcid(vcpu, pcid); + return 0; } EXPORT_SYMBOL_GPL(kvm_set_cr3); @@ -11697,8 +11730,6 @@ int kvm_handle_invpcid(struct kvm_vcpu *vcpu, unsigned long type, gva_t gva) { bool pcid_enabled; struct x86_exception e; - unsigned i; - unsigned long roots_to_free = 0; struct { u64 pcid; u64 gla; @@ -11732,23 +11763,7 @@ int kvm_handle_invpcid(struct kvm_vcpu *vcpu, unsigned long type, gva_t gva) return 1; } - if (kvm_get_active_pcid(vcpu) == operand.pcid) { - kvm_mmu_sync_roots(vcpu); - kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu); - } - - for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) - if (kvm_get_pcid(vcpu, vcpu->arch.mmu->prev_roots[i].pgd) - == operand.pcid) - roots_to_free |= KVM_MMU_ROOT_PREVIOUS(i); - - kvm_mmu_free_roots(vcpu, vcpu->arch.mmu, roots_to_free); - /* - * If neither the current cr3 nor any of the prev_roots use the - * given PCID, then nothing needs to be done here because a - * resync will happen anyway before switching to any other CR3. - */ - + kvm_invalidate_pcid(vcpu, operand.pcid); return kvm_skip_emulated_instruction(vcpu); case INVPCID_TYPE_ALL_NON_GLOBAL: From patchwork Wed Jun 9 23:42:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311467 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2EF9C48BCF for ; Wed, 9 Jun 2021 23:43:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 85ED9613EF for ; Wed, 9 Jun 2021 23:43:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230287AbhFIXpt (ORCPT ); Wed, 9 Jun 2021 19:45:49 -0400 Received: from mail-qt1-f201.google.com ([209.85.160.201]:46939 "EHLO mail-qt1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230245AbhFIXpt (ORCPT ); Wed, 9 Jun 2021 19:45:49 -0400 Received: by mail-qt1-f201.google.com with SMTP id h20-20020ac87d540000b0290249d0777b80so2376528qtb.13 for ; Wed, 09 Jun 2021 16:43:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=TSyhgxkAM87zPQDaBTB/xYIGoMe0SaHqhMUIzhv8SIc=; b=uAMj2WkqpBB4NV2SMlBv+LrSjvGsZgAFPCUKGdnlt4go5X6wG+GmFSR0KLzuX/PVbZ OAKh5QDgt5Q1OGS7e2dDtyN08kHG5SeRB4uBeduWhaq+63jDEA3KvST5qi77s/+BZUC1 Aeowf1/E6dAnI1Gg9ibHAijPlDTQHgxpPsl9duyrM2ymFI8Xzt2caAriDboHDXm6ma3y e22v2GJ6swXY+gSKwr58QSKwO4FzW99NFvVYwz+Jt4CH00t+zeJy36tN6gq6enzNhYmI W5P6+ACmEzBZJIvnIAKvNdrfKeO2L09H6zUMSPlnLbELvLFKWoc3hiPzJjUeGyIlXLbL soiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=TSyhgxkAM87zPQDaBTB/xYIGoMe0SaHqhMUIzhv8SIc=; b=uoMiVWFYIl+fCcyilGIiecu7OB3IuD7nnmemHM4CxxCRqyDSRKi1MwaId2wBsRgK5L s+wrpPMLdA8dQr6LO50eiycwxhEi8EB+XrCemJisAKUqzqKaCC6pI2puwVd1eMhNv7Uh o8Yn4pI3/Hk+Bbgg2EhsdOg4SToBQf5TFqyIu/KxUzmGTKtNca/Kh0LK6FgX2t0wl32q +Dy0zMjNhFo89DsfsaNte4dmIG9gccln708EV1Al+j3UTLth7MnaqmQDG5M+qDVXUTAG u+ip6QqVE9aIu/DURZDqMj5cbFZO9QOvpvNIWTsQWfiq8HqBH3Xy35vJ0Fq0nAt57lKo pmHQ== X-Gm-Message-State: AOAM531ex06VsbEP55Lg2rqwwgz9E6/lp3rW5w6LLnku9BfY39hPPtou WqKao3G5V7QZ4zkSS6AGjS6QpY3jb/M= X-Google-Smtp-Source: ABdhPJxt9s6tpqX+2vXnuajtY+tmND2r+hdonH3bjIK3Ank/XmwdAQ1TFT/vKoKqFPLk2R1fpLNcv5HUggQ= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:ad4:4dcb:: with SMTP id cw11mr2243546qvb.54.1623282173808; Wed, 09 Jun 2021 16:42:53 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:25 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-6-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 05/15] KVM: x86: Uncondtionally skip MMU sync/TLB flush in MOV CR3's PGD switch From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Stop leveraging the MMU sync and TLB flush requested by the fast PGD switch helper now that kvm_set_cr3() manually handles the necessary sync, frees, and TLB flush. This will allow dropping the params from the fast PGD helpers since nested SVM is now the odd blob out. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index e2f6d6a1ba54..02ceb1f606f4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1136,7 +1136,7 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) return 1; if (cr3 != kvm_read_cr3(vcpu)) - kvm_mmu_new_pgd(vcpu, cr3, skip_tlb_flush, skip_tlb_flush); + kvm_mmu_new_pgd(vcpu, cr3, true, true); vcpu->arch.cr3 = cr3; kvm_register_mark_available(vcpu, VCPU_EXREG_CR3); From patchwork Wed Jun 9 23:42:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C47E5C48BD1 for ; Wed, 9 Jun 2021 23:43:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A976761249 for ; Wed, 9 Jun 2021 23:43:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230118AbhFIXpT (ORCPT ); Wed, 9 Jun 2021 19:45:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230130AbhFIXpJ (ORCPT ); Wed, 9 Jun 2021 19:45:09 -0400 Received: from mail-qt1-x849.google.com (mail-qt1-x849.google.com [IPv6:2607:f8b0:4864:20::849]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 00BEAC0617AF for ; Wed, 9 Jun 2021 16:42:56 -0700 (PDT) Received: by mail-qt1-x849.google.com with SMTP id m18-20020a05622a1192b0290247a89f63bcso5938214qtk.20 for ; Wed, 09 Jun 2021 16:42:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=IaHvriPR1ilnz00LgSKYZNDwy+t0L+MzDKmguXPVTIU=; b=MHgOPpSpKZ+onNecVS/T1TDwOCedi+KOQmXbTeAm/KuGD3YTLADXeur6XvPWjGKp9a 0DALtfjZ94/elE2wLmLX8UfS7MtgLC39VFTYx3b4DL5EcMhpeCENrvOGnc6hwVtgDB8t 8bMwaCu28N7N9SulkZtEe459ldZZPgoIrh3m/e6kzaCjZHHPahFgJjHpjhdHFzGwoC6V htNM/xWI2HlYCcZ85s1ynnDjTTBqfsIbV6clg9oBvpjB/ecGnft6AneNKDN27HaE/hNx ASu+EalLdUJu6ZmxbFS/QUtd1wQ/tknQeXvj4PGYlex4+uq6/LCQ+j/f+MuxLXAjI5Di B6rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=IaHvriPR1ilnz00LgSKYZNDwy+t0L+MzDKmguXPVTIU=; b=R1LMJM42zsJFKpTEnZaTK8VWf/rLGBCaPC6IVPIFzu8YU1yvmlikeDbAyGRdz1VY4L LrR1O4OxuQBuNgN0LjwPrc2QShRVijlz1qA/qUWPclFD25ScPnuOfImSGr1vUpU/wGre VhmTLZenNAhk5Yi7WBuht084Ebjzb0/QTgm3Tg0p29SUi8r4wRgmUYjS92aehi4D1ckS pYha1Jc4V8byCb04lNtN1QX2oWsIuC5T8oFw0tw88+kaJ6zh+Kn/3Z9pa+9nkljnuzyW vTL3KXmzo7r89jZFnhAGd56JTq1gw3UJGN3tzdM6Jo8ryRGRzv0OVpnSboFaC2/555ru 1Nvw== X-Gm-Message-State: AOAM533F8qRO0oDG0vSwN0sL5VWOQjv0RgojanZuisI2+8MFbf0QwYN2 ie/AHNVF1beDWF2cblnJLO2wywTy8e4= X-Google-Smtp-Source: ABdhPJz+QZrNtj4Rku4gjD033qxRenTviSAlZ1fQ8xlfDY3NGP6a/NZJNOML9KwqS+Vleoo+POLZt8kZENs= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:a0c:f582:: with SMTP id k2mr2424200qvm.51.1623282176141; Wed, 09 Jun 2021 16:42:56 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:26 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-7-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 06/15] KVM: nSVM: Move TLB flushing logic (or lack thereof) to dedicated helper From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Introduce nested_svm_transition_tlb_flush() and use it force an MMU sync and TLB flush on nSVM VM-Enter and VM-Exit instead of sneaking the logic into the __kvm_mmu_new_pgd() call sites. Add a partial todo list to document issues that need to be addressed before the unconditional sync and flush can be modified to look more like nVMX's logic. In addition to making nSVM's forced flushing more overt (guess who keeps losing track of it), the new helper brings further convergence between nSVM and nVMX, and also sets the stage for dropping the "skip" params from __kvm_mmu_new_pgd(). Cc: Maxim Levitsky Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/svm/nested.c | 38 +++++++++++++++++++++++++++++--------- 2 files changed, 30 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0144c40d09c7..d7f29bf94ca3 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4648,7 +4648,7 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, u32 cr0, u32 cr4, u32 efer, struct kvm_mmu *context = &vcpu->arch.guest_mmu; union kvm_mmu_role new_role = kvm_calc_shadow_npt_root_page_role(vcpu); - __kvm_mmu_new_pgd(vcpu, nested_cr3, new_role.base, false, false); + __kvm_mmu_new_pgd(vcpu, nested_cr3, new_role.base, true, true); if (new_role.as_u64 != context->mmu_role.as_u64) { shadow_mmu_init_context(vcpu, context, cr0, cr4, efer, new_role); diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index 5e8d8443154e..fe2705557960 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -380,6 +380,25 @@ static inline bool nested_npt_enabled(struct vcpu_svm *svm) return svm->nested.ctl.nested_ctl & SVM_NESTED_CTL_NP_ENABLE; } +static void nested_svm_transition_tlb_flush(struct kvm_vcpu *vcpu) +{ + /* + * TODO: optimize unconditional TLB flush/MMU sync. A partial list of + * things to fix before this can be conditional: + * + * - Flush TLBs for both L1 and L2 remote TLB flush + * - Honor L1's request to flush an ASID on nested VMRUN + * - Sync nested NPT MMU on VMRUN that flushes L2's ASID[*] + * - Don't crush a pending TLB flush in vmcb02 on nested VMRUN + * - Flush L1's ASID on KVM_REQ_TLB_FLUSH_GUEST + * + * [*] Unlike nested EPT, SVM's ASID management can invalidate nested + * NPT guest-physical mappings on VMRUN. + */ + kvm_make_request(KVM_REQ_MMU_SYNC, vcpu); + kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu); +} + /* * Load guest's/host's cr3 on nested vmentry or vmexit. @nested_npt is true * if we are emulating VM-Entry into a guest with NPT enabled. @@ -396,12 +415,8 @@ static int nested_svm_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, return -EINVAL; } - /* - * TODO: optimize unconditional TLB flush/MMU sync here and in - * kvm_init_shadow_npt_mmu(). - */ if (!nested_npt) - kvm_mmu_new_pgd(vcpu, cr3, false, false); + kvm_mmu_new_pgd(vcpu, cr3, true, true); vcpu->arch.cr3 = cr3; kvm_register_mark_available(vcpu, VCPU_EXREG_CR3); @@ -481,6 +496,7 @@ static void nested_vmcb02_prepare_save(struct vcpu_svm *svm, struct vmcb *vmcb12 static void nested_vmcb02_prepare_control(struct vcpu_svm *svm) { const u32 mask = V_INTR_MASKING_MASK | V_GIF_ENABLE_MASK | V_GIF_MASK; + struct kvm_vcpu *vcpu = &svm->vcpu; /* * Filled at exit: exit_code, exit_code_hi, exit_info_1, exit_info_2, @@ -505,10 +521,10 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm) /* nested_cr3. */ if (nested_npt_enabled(svm)) - nested_svm_init_mmu_context(&svm->vcpu); + nested_svm_init_mmu_context(vcpu); - svm->vmcb->control.tsc_offset = svm->vcpu.arch.tsc_offset = - svm->vcpu.arch.l1_tsc_offset + svm->nested.ctl.tsc_offset; + svm->vmcb->control.tsc_offset = vcpu->arch.tsc_offset = + vcpu->arch.l1_tsc_offset + svm->nested.ctl.tsc_offset; svm->vmcb->control.int_ctl = (svm->nested.ctl.int_ctl & ~mask) | @@ -523,8 +539,10 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm) svm->vmcb->control.pause_filter_count = svm->nested.ctl.pause_filter_count; svm->vmcb->control.pause_filter_thresh = svm->nested.ctl.pause_filter_thresh; + nested_svm_transition_tlb_flush(vcpu); + /* Enter Guest-Mode */ - enter_guest_mode(&svm->vcpu); + enter_guest_mode(vcpu); /* * Merge guest and host intercepts - must be called with vcpu in @@ -803,6 +821,8 @@ int nested_svm_vmexit(struct vcpu_svm *svm) kvm_vcpu_unmap(vcpu, &map, true); + nested_svm_transition_tlb_flush(vcpu); + nested_svm_uninit_mmu_context(vcpu); rc = nested_svm_load_cr3(vcpu, svm->vmcb->save.cr3, false); From patchwork Wed Jun 9 23:42:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-23.5 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50D91C48BCF for ; Wed, 9 Jun 2021 23:44:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2364B613EC for ; Wed, 9 Jun 2021 23:44:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230136AbhFIXp6 (ORCPT ); Wed, 9 Jun 2021 19:45:58 -0400 Received: from mail-qt1-f201.google.com ([209.85.160.201]:53039 "EHLO mail-qt1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230329AbhFIXpy (ORCPT ); Wed, 9 Jun 2021 19:45:54 -0400 Received: by mail-qt1-f201.google.com with SMTP id z17-20020ac86b910000b0290244cba55754so9535964qts.19 for ; Wed, 09 Jun 2021 16:43:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=K2ifrSGaCEQ+14CO+xHrLrRJvzgN5omyn9Y8rpPGs5U=; b=vBSSvL3SzM5RYdD8hznFl+2Av5Oh0maOv15Y8aCTl9T/9vVBRBd+QE0GfOUzE7OkQA 1Pxidjlh5ytQSdf9DP3nFulJ/Tl2evzv5DXArI+cZGg8KxZvMEB+vvydaSXCHdglTSt9 w4CE2swL95FjJyZXfXFhw5XfyTt/vgSzozO5UyyZU/8A83fxHR7NdSXyMx9y5SvIayu+ Bj/ybC9GKy856/X4iHGQ0Ho8o5BxuozB42MmWa14cnO3nDfIyu7DcZFz345EEjxa6bVk kk/ktEY4SWLwrKPzqLFBvsUOlpY5GV22g570kcNlPzBzQajKik6IcanQI0V86yCdXAP2 yvMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=K2ifrSGaCEQ+14CO+xHrLrRJvzgN5omyn9Y8rpPGs5U=; b=LGancHcpS0kIdyUYtcW7UA9tXO2N/fhADoue4jq14325coDO34tdwQ0ZdBIcJPO/80 j8nW3gquqEJ6ey5U2iZbLrQ/KfFZoOpnyCBP7d+sX6CUjk2HVSZE3O5Xl8uIFYWDAiwS kxpeqDawxNqjimVzElMf7Ai0fc/P0ci5hEFVk5UFjfgC7k2cGAK9s7P7zQnBiZpyO0Xg t4OGr8VeMTdRfZtvwDbt9f3Esvop8SnZbUuZ3IjKwFSxHACni7bbZb9dK5CeG/JTHuzO h4nvqWDUNidF+bkJqqpxCmom6hD+8Ll++4s0iI8fI7b27Ldqxb+mcNCUqzpJo52s0evt 5jUw== X-Gm-Message-State: AOAM532+zcbcAAl8FzvBZBF/C0ugENlvMekfBwbGdCreKm08L2+gz9jw KBerCnQ6UnhZMgBw1XJZPyz5Rym7CSA= X-Google-Smtp-Source: ABdhPJwXRZ4esdSqe6zj+H+ek2FyB5tVwuNcFmAOeZ9qQ6Ek9sDSd6Y17SrCPhl137vn3P4uPM1Rw/8+XDc= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:ad4:4b71:: with SMTP id m17mr2374885qvx.45.1623282178456; Wed, 09 Jun 2021 16:42:58 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:27 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-8-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 07/15] KVM: x86: Drop skip MMU sync and TLB flush params from "new PGD" helpers From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Drop skip_mmu_sync and skip_tlb_flush from __kvm_mmu_new_pgd() now that all call sites unconditionally skip both the sync and flush. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 3 +-- arch/x86/kvm/mmu/mmu.c | 17 +++++++---------- arch/x86/kvm/svm/nested.c | 2 +- arch/x86/kvm/vmx/nested.c | 6 +----- arch/x86/kvm/x86.c | 2 +- 5 files changed, 11 insertions(+), 19 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 6652e51a86fd..c05448d3beff 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1675,8 +1675,7 @@ void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva); void kvm_mmu_invalidate_gva(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, gva_t gva, hpa_t root_hpa); void kvm_mmu_invpcid_gva(struct kvm_vcpu *vcpu, gva_t gva, unsigned long pcid); -void kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new_pgd, bool skip_tlb_flush, - bool skip_mmu_sync); +void kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new_pgd); void kvm_configure_mmu(bool enable_tdp, int tdp_max_root_level, int tdp_huge_page_level); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index d7f29bf94ca3..a832e0fedf32 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3913,8 +3913,7 @@ static bool fast_pgd_switch(struct kvm_vcpu *vcpu, gpa_t new_pgd, } static void __kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new_pgd, - union kvm_mmu_page_role new_role, - bool skip_tlb_flush, bool skip_mmu_sync) + union kvm_mmu_page_role new_role) { if (!fast_pgd_switch(vcpu, new_pgd, new_role)) { kvm_mmu_free_roots(vcpu, vcpu->arch.mmu, KVM_MMU_ROOT_CURRENT); @@ -3929,10 +3928,10 @@ static void __kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new_pgd, */ kvm_make_request(KVM_REQ_LOAD_MMU_PGD, vcpu); - if (!skip_mmu_sync || force_flush_and_sync_on_reuse) + if (force_flush_and_sync_on_reuse) { kvm_make_request(KVM_REQ_MMU_SYNC, vcpu); - if (!skip_tlb_flush || force_flush_and_sync_on_reuse) kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu); + } /* * The last MMIO access's GVA and GPA are cached in the VCPU. When @@ -3951,11 +3950,9 @@ static void __kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new_pgd, to_shadow_page(vcpu->arch.mmu->root_hpa)); } -void kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new_pgd, bool skip_tlb_flush, - bool skip_mmu_sync) +void kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new_pgd) { - __kvm_mmu_new_pgd(vcpu, new_pgd, kvm_mmu_calc_root_page_role(vcpu), - skip_tlb_flush, skip_mmu_sync); + __kvm_mmu_new_pgd(vcpu, new_pgd, kvm_mmu_calc_root_page_role(vcpu)); } EXPORT_SYMBOL_GPL(kvm_mmu_new_pgd); @@ -4648,7 +4645,7 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, u32 cr0, u32 cr4, u32 efer, struct kvm_mmu *context = &vcpu->arch.guest_mmu; union kvm_mmu_role new_role = kvm_calc_shadow_npt_root_page_role(vcpu); - __kvm_mmu_new_pgd(vcpu, nested_cr3, new_role.base, true, true); + __kvm_mmu_new_pgd(vcpu, nested_cr3, new_role.base); if (new_role.as_u64 != context->mmu_role.as_u64) { shadow_mmu_init_context(vcpu, context, cr0, cr4, efer, new_role); @@ -4700,7 +4697,7 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, bool execonly, kvm_calc_shadow_ept_root_page_role(vcpu, accessed_dirty, execonly, level); - __kvm_mmu_new_pgd(vcpu, new_eptp, new_role.base, true, true); + __kvm_mmu_new_pgd(vcpu, new_eptp, new_role.base); if (new_role.as_u64 == context->mmu_role.as_u64) return; diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index fe2705557960..ccd90ea93acd 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -416,7 +416,7 @@ static int nested_svm_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, } if (!nested_npt) - kvm_mmu_new_pgd(vcpu, cr3, true, true); + kvm_mmu_new_pgd(vcpu, cr3); vcpu->arch.cr3 = cr3; kvm_register_mark_available(vcpu, VCPU_EXREG_CR3); diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index e102a5c10a83..3fb87e5aead4 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -1125,12 +1125,8 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, bool ne } } - /* - * Unconditionally skip the TLB flush on fast CR3 switch, all TLB - * flushes are handled by nested_vmx_transition_tlb_flush(). - */ if (!nested_ept) { - kvm_mmu_new_pgd(vcpu, cr3, true, true); + kvm_mmu_new_pgd(vcpu, cr3); /* * A TLB flush on VM-Enter/VM-Exit flushes all linear mappings diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 02ceb1f606f4..117acfbc7ba9 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1136,7 +1136,7 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) return 1; if (cr3 != kvm_read_cr3(vcpu)) - kvm_mmu_new_pgd(vcpu, cr3, true, true); + kvm_mmu_new_pgd(vcpu, cr3); vcpu->arch.cr3 = cr3; kvm_register_mark_available(vcpu, VCPU_EXREG_CR3); From patchwork Wed Jun 9 23:42:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3451FC48BCF for ; Wed, 9 Jun 2021 23:43:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 19735613EC for ; Wed, 9 Jun 2021 23:43:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229957AbhFIXpB (ORCPT ); Wed, 9 Jun 2021 19:45:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48410 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229814AbhFIXo6 (ORCPT ); Wed, 9 Jun 2021 19:44:58 -0400 Received: from mail-qv1-xf4a.google.com (mail-qv1-xf4a.google.com [IPv6:2607:f8b0:4864:20::f4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 807E0C061574 for ; Wed, 9 Jun 2021 16:43:02 -0700 (PDT) Received: by mail-qv1-xf4a.google.com with SMTP id ca13-20020ad4560d0000b029023ebd662003so1202205qvb.17 for ; Wed, 09 Jun 2021 16:43:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=ITsTEXIXZfR4GZglAO2blyali5OWfQZ4wyKMhZGD1zU=; b=q4yijGpdjTGlevoC5SnN9mqDG878mTfvCHQ5l56bZpLHvufpGtXexLHoR8+Z7j2DXh rGnIqdVKv9m5owlAzRpgopaIIWUMYRJEQoOqQoPEPsVl9k0BtA4qBGi9P6smrzQmHsGv nyAacDgxBgVoeYAh/20lXc2M0HVdkGr7lMcfPdifYKDq4kj9rYHN4WYRK5l5vMJr6NDo uRU0fpdAYLlG+rPDphY66uIIZOpcBMJOdmJpWtE7Tgk+v5ouVFJ+m7kmkyD2Rvr1keV2 QoyyicDtX0sR5MfxxPoh3SX4IXix9X2yZHN+LQgnDDoVJolOgi6WNNUW9AwAKDO9Sq4T EX6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=ITsTEXIXZfR4GZglAO2blyali5OWfQZ4wyKMhZGD1zU=; b=l7Oj8f3IxjW8YDUxQZZ5AHFlX2EIHtUO4Bik6jhtpWMkvWUN0acw2ojxHvGncgp/Iz kkSCgR8MEvOmzLcXHDwooM3UhtymjHq+7cNVKnsm2bN1CVWUnQmbXo3S/uszv67NbNbX t3Xnr5iyUu9XFTlLUDFcUv776+DCTWmfWV7SwA3EXTgibQ2b+yQPRKCSUd7Vc9qxf3jI ya6h9yFqsfTG10SOH3vYnJQE1oFKqIkdo/bWKtrPpeqJQu0khsrDnWVy5hlTMIbl0Pzm IUvS5fis5qmGhjpixAZueiQhErqhsBmfeq0gkcbOTK7UmtG/+FT0Gamj4yxX06XYp2tN xroQ== X-Gm-Message-State: AOAM530RKgeTIx1INxnw4NxzH3HlOpqskgI1xiOacrnngXCl6cfKharo 2q/XWH6OlwL632EytbTi9Ix6MSG9ct8= X-Google-Smtp-Source: ABdhPJwTG/9fZnINaggj2dmoIUsmhPVmm1t+BG2C4LqlRv1t+8rbK0VeKZxOWz9+KKZa0Zqfvb+RGI/EmsI= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:a0c:fec3:: with SMTP id z3mr2517850qvs.57.1623282180796; Wed, 09 Jun 2021 16:43:00 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:28 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-9-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 08/15] KVM: nVMX: Consolidate VM-Enter/VM-Exit TLB flush and MMU sync logic From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Drop the dedicated nested_vmx_transition_mmu_sync() now that the MMU sync is handled via KVM_REQ_TLB_FLUSH_GUEST, and fold that flush into the all-encompassing nested_vmx_transition_tlb_flush(). Opportunistically add a comment explaning why nested EPT never needs to sync the MMU on VM-Enter. Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 91 +++++++++++---------------------------- 1 file changed, 25 insertions(+), 66 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 3fb87e5aead4..6d7c368c92e7 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -1057,48 +1057,6 @@ static void prepare_vmx_msr_autostore_list(struct kvm_vcpu *vcpu, } } -/* - * Returns true if the MMU needs to be sync'd on nested VM-Enter/VM-Exit. - * tl;dr: the MMU needs a sync if L0 is using shadow paging and L1 didn't - * enable VPID for L2 (implying it expects a TLB flush on VMX transitions). - * Here's why. - * - * If EPT is enabled by L0 a sync is never needed: - * - if it is disabled by L1, then L0 is not shadowing L1 or L2 PTEs, there - * cannot be unsync'd SPTEs for either L1 or L2. - * - * - if it is also enabled by L1, then L0 doesn't need to sync on VM-Enter - * VM-Enter as VM-Enter isn't required to invalidate guest-physical mappings - * (irrespective of VPID), i.e. L1 can't rely on the (virtual) CPU to flush - * stale guest-physical mappings for L2 from the TLB. And as above, L0 isn't - * shadowing L1 PTEs so there are no unsync'd SPTEs to sync on VM-Exit. - * - * If EPT is disabled by L0: - * - if VPID is enabled by L1 (for L2), the situation is similar to when L1 - * enables EPT: L0 doesn't need to sync as VM-Enter and VM-Exit aren't - * required to invalidate linear mappings (EPT is disabled so there are - * no combined or guest-physical mappings), i.e. L1 can't rely on the - * (virtual) CPU to flush stale linear mappings for either L2 or itself (L1). - * - * - however if VPID is disabled by L1, then a sync is needed as L1 expects all - * linear mappings (EPT is disabled so there are no combined or guest-physical - * mappings) to be invalidated on both VM-Enter and VM-Exit. - * - * Note, this logic is subtly different than nested_has_guest_tlb_tag(), which - * additionally checks that L2 has been assigned a VPID (when EPT is disabled). - * Whether or not L2 has been assigned a VPID by L0 is irrelevant with respect - * to L1's expectations, e.g. L0 needs to invalidate hardware TLB entries if L2 - * doesn't have a unique VPID to prevent reusing L1's entries (assuming L1 has - * been assigned a VPID), but L0 doesn't need to do a MMU sync because L1 - * doesn't expect stale (virtual) TLB entries to be flushed, i.e. L1 doesn't - * know that L0 will flush the TLB and so L1 will do INVVPID as needed to flush - * stale TLB entries, at which point L0 will sync L2's MMU. - */ -static bool nested_vmx_transition_mmu_sync(struct kvm_vcpu *vcpu) -{ - return !enable_ept && !nested_cpu_has_vpid(get_vmcs12(vcpu)); -} - /* * Load guest's/host's cr3 at nested entry/exit. @nested_ept is true if we are * emulating VM-Entry into a guest with EPT enabled. On failure, the expected @@ -1125,18 +1083,9 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, bool ne } } - if (!nested_ept) { + if (!nested_ept) kvm_mmu_new_pgd(vcpu, cr3); - /* - * A TLB flush on VM-Enter/VM-Exit flushes all linear mappings - * across all PCIDs, i.e. all PGDs need to be synchronized. - * See nested_vmx_transition_mmu_sync() for more details. - */ - if (nested_vmx_transition_mmu_sync(vcpu)) - kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); - } - vcpu->arch.cr3 = cr3; kvm_register_mark_available(vcpu, VCPU_EXREG_CR3); @@ -1172,18 +1121,29 @@ static void nested_vmx_transition_tlb_flush(struct kvm_vcpu *vcpu, { struct vcpu_vmx *vmx = to_vmx(vcpu); - /* - * If VPID is disabled, linear and combined mappings are flushed on - * VM-Enter/VM-Exit, and guest-physical mappings are valid only for - * their associated EPTP. - */ - if (!enable_vpid) - return; - /* * If vmcs12 doesn't use VPID, L1 expects linear and combined mappings - * for *all* contexts to be flushed on VM-Enter/VM-Exit. + * for *all* contexts to be flushed on VM-Enter/VM-Exit, i.e. it's a + * full TLB flush from the guest's perspective. This is required even + * if VPID is disabled in the host as KVM may need to synchronize the + * MMU in response to the guest TLB flush. * + * Note, using TLB_FLUSH_GUEST is correct even if nested EPT is in use. + * EPT is a special snowflake, as guest-physical mappings aren't + * flushed on VPID invalidations, including VM-Enter or VM-Exit with + * VPID disabled. As a result, KVM _never_ needs to sync nEPT + * entries on VM-Enter because L1 can't rely on VM-Enter to flush + * those mappings. + */ + if (!nested_cpu_has_vpid(vmcs12)) { + kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); + return; + } + + /* L2 should never have a VPID if VPID is disabled. */ + WARN_ON(!enable_vpid); + + /* * If VPID is enabled and used by vmc12, but L2 does not have a unique * TLB tag (ASID), i.e. EPT is disabled and KVM was unable to allocate * a VPID for L2, flush the current context as the effective ASID is @@ -1195,13 +1155,12 @@ static void nested_vmx_transition_tlb_flush(struct kvm_vcpu *vcpu, * * If a TLB flush isn't required due to any of the above, and vpid12 is * changing then the new "virtual" VPID (vpid12) will reuse the same - * "real" VPID (vpid02), and so needs to be sync'd. There is no direct + * "real" VPID (vpid02), and so needs to be flushed. There's no direct * mapping between vpid02 and vpid12, vpid02 is per-vCPU and reused for - * all nested vCPUs. + * all nested vCPUs. Remember, a flush on VM-Enter does not invalidate + * guest-physical mappings, so there is no need to sync the nEPT MMU. */ - if (!nested_cpu_has_vpid(vmcs12)) { - kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu); - } else if (!nested_has_guest_tlb_tag(vcpu)) { + if (!nested_has_guest_tlb_tag(vcpu)) { kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu); } else if (is_vmenter && vmcs12->virtual_processor_id != vmx->nested.last_vpid) { From patchwork Wed Jun 9 23:42:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 579FBC48BCD for ; Wed, 9 Jun 2021 23:44:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3AED7613F0 for ; Wed, 9 Jun 2021 23:44:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230344AbhFIXqC (ORCPT ); Wed, 9 Jun 2021 19:46:02 -0400 Received: from mail-yb1-f202.google.com ([209.85.219.202]:54856 "EHLO mail-yb1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230370AbhFIXp7 (ORCPT ); Wed, 9 Jun 2021 19:45:59 -0400 Received: by mail-yb1-f202.google.com with SMTP id n129-20020a2527870000b02904ed02e1aab5so33463915ybn.21 for ; Wed, 09 Jun 2021 16:44:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=OUQR2M6ORqp4R4Q1XNMmsY3Jpp0VsWP0Wlsf/XP786M=; b=e573bk0IhJWH1KrhuNXg6S0B40xlHphSQjHX5zez/NH5JKvS0U3LKHjxi/W9ZTj/iX Djq6KWyvSQXrTczghntGdbOzeGs+E9PeM3yRyExQxONDN/xpYJQCOI24BG49/qXGrRYs 4WgaxK/lGhTaZZ4dcwZvjnV2o8Jwi1ZOR/GZiN52WW+UD23ESJIZDl3IN6JMWwS062Q4 ll9U61n+qf8cpAMoY5VqAiJ+kZKXx2egKGOC3s21sz+/YDofPbZG0npmdP/uNxAvnwgb PzIK200AyWJhul+KIPaP6FJQwpBzFfv9oVcotyAPAH0SXPpRCUzy0XIeOaOq8qw11fZw uENw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=OUQR2M6ORqp4R4Q1XNMmsY3Jpp0VsWP0Wlsf/XP786M=; b=GmZQ3OtsUXsfCA2ebeYBCN0tgrPaKf5VFrt8OJmS91uPh0ptMEmJbvlpWyZWaGPIWX Ey4ozj/mgj1OnoI+ee4OKTcp20Um185lx9iot7tOA7yCPIgRBnj8qVm1oIl9G8U+135S +dCgG16gAlScwWsHkJBsjRAqKwnaiZBMWlfYt8Fmh5w8fhIT0o6WsYG+nk0EiuCWO/sh ZUV9/NLhIp3EP3nxv4FmBD7rpUshG6VcNQPNUjffGT1I5wiOurDMzVz8RvG5dlj0mdT8 gxN03OgVi/LA0lmACgGhYzw02o6rKypXxA3CHKsNMt54d6JauJ4hYM6MVY6STXs95bBC i6JA== X-Gm-Message-State: AOAM531kfON2uJVArAOX1KAG/xbE5Fbqo2R63pboz68XjaqmA9lQKzJt gdgqsyF8RRVB6w1ZpwmIo7kAMnPX0cQ= X-Google-Smtp-Source: ABdhPJxQ9nCGPraYT+eSs9eaqwy1FxGg/921tA/4pDpTqwKUUebnfytfLTqxWPsgFjSVFeHSWQcbdkxv3GM= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:a25:c7c1:: with SMTP id w184mr3529192ybe.204.1623282183334; Wed, 09 Jun 2021 16:43:03 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:29 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-10-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 09/15] KVM: nVMX: Free only guest_mode (L2) roots on INVVPID w/o EPT From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When emulating INVVPID for L1, free only L2+ roots, using the guest_mode tag in the MMU role to identify L2+ roots. From L1's perspective, its own TLB entries use VPID=0, and INVVPID is not requied to invalidate such entries. Per Intel's SDM, INVVPID _may_ invalidate entries with VPID=0, but it is not required to do so. Cc: Lai Jiangshan Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 27 +++++++++++++++++++++++++++ arch/x86/kvm/vmx/nested.c | 7 +++---- 3 files changed, 31 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index c05448d3beff..05c5ca047c53 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1650,6 +1650,7 @@ int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gfn); void __kvm_mmu_free_some_pages(struct kvm_vcpu *vcpu); void kvm_mmu_free_roots(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, ulong roots_to_free); +void kvm_mmu_free_guest_mode_roots(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu); gpa_t translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, u32 access, struct x86_exception *exception); gpa_t kvm_mmu_gva_to_gpa_read(struct kvm_vcpu *vcpu, gva_t gva, diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a832e0fedf32..f987f2ea4a01 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3180,6 +3180,33 @@ void kvm_mmu_free_roots(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, } EXPORT_SYMBOL_GPL(kvm_mmu_free_roots); +void kvm_mmu_free_guest_mode_roots(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu) +{ + unsigned long roots_to_free = 0; + hpa_t root_hpa; + int i; + + /* + * This should not be called while L2 is active, L2 can't invalidate + * _only_ its own roots, e.g. INVVPID unconditionally exits. + */ + WARN_ON_ONCE(mmu->mmu_role.base.guest_mode); + + for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) { + root_hpa = mmu->prev_roots[i].hpa; + if (!VALID_PAGE(root_hpa)) + continue; + + if (!to_shadow_page(root_hpa) || + to_shadow_page(root_hpa)->role.guest_mode) + roots_to_free |= KVM_MMU_ROOT_PREVIOUS(i); + } + + kvm_mmu_free_roots(vcpu, mmu, roots_to_free); +} +EXPORT_SYMBOL_GPL(kvm_mmu_free_guest_mode_roots); + + static int mmu_check_root(struct kvm_vcpu *vcpu, gfn_t root_gfn) { int ret = 0; diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 6d7c368c92e7..2a881afc1fd0 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -5423,8 +5423,8 @@ static int handle_invvpid(struct kvm_vcpu *vcpu) /* * Sync the shadow page tables if EPT is disabled, L1 is invalidating - * linear mappings for L2 (tagged with L2's VPID). Free all roots as - * VPIDs are not tracked in the MMU role. + * linear mappings for L2 (tagged with L2's VPID). Free all guest + * roots as VPIDs are not tracked in the MMU role. * * Note, this operates on root_mmu, not guest_mmu, as L1 and L2 share * an MMU when EPT is disabled. @@ -5432,8 +5432,7 @@ static int handle_invvpid(struct kvm_vcpu *vcpu) * TODO: sync only the affected SPTEs for INVDIVIDUAL_ADDR. */ if (!enable_ept) - kvm_mmu_free_roots(vcpu, &vcpu->arch.root_mmu, - KVM_MMU_ROOTS_ALL); + kvm_mmu_free_guest_mode_roots(vcpu, &vcpu->arch.root_mmu); return nested_vmx_succeed(vcpu); } From patchwork Wed Jun 9 23:42:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34D74C48BCD for ; Wed, 9 Jun 2021 23:44:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1E669613EF for ; Wed, 9 Jun 2021 23:44:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230416AbhFIXqa (ORCPT ); Wed, 9 Jun 2021 19:46:30 -0400 Received: from mail-yb1-f202.google.com ([209.85.219.202]:39428 "EHLO mail-yb1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230417AbhFIXqP (ORCPT ); Wed, 9 Jun 2021 19:46:15 -0400 Received: by mail-yb1-f202.google.com with SMTP id r5-20020a2582850000b02905381b1b616eso33580891ybk.6 for ; Wed, 09 Jun 2021 16:44:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=TUFUQ1j7f0KidBMcnav4RlAHSJzXFRaH+3nAG9epvWY=; b=vuUcivnIlKiksHf5E7Skc7ygl7tdHKj97QOsLpmVDT2r5vAldHZDQfDZ5d0f8A0rQb X0+XQWIqMtmrbDQUGGwYj5zI06RpeL9gK8pf+9uSaxSTCYgpjKmarjBFk7ZNDyVRhS54 ZtPPT0vfxfhRul9wwY/fQqDmS8rWbchKT8ZlRzmnebA1Bv97Ydt1sRVlsubNkreq91DI GpIP5aLsRXXr6k8mslPOd5+gLPYZ3q8XHBayPZYtit3kJu28HAovlISFF16Y10vye5se r4H7jniF6dFZY2IFaTnpGqxATVJ+Or2u3XKa67DkuWUQUsnmsnAQAR+jxMJC+TTo3a58 HL9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=TUFUQ1j7f0KidBMcnav4RlAHSJzXFRaH+3nAG9epvWY=; b=n39jMF7sq9xIn+/dY3P3jR5tRGdVcT/wkgfRRJ6OT2OSJ+g74Cb/fmqcXPp8yd5xcm J6rlBImNZgDYk00HkEpPQUfmleqn2UBLKP7TeFDqa9aYI/vDkLNlD+NiCMOTN5LuXx9C t0UJ85ouSPGm0e7P1vNJKEVLUi/L00lvcE40FTyupPb3VU/uejYXvOUlbT6d6HyPjhzJ Ky8D7VeB6EFAEeLVx81OdlUO40T+PSGl8pVmrW4tod2NYWll5oLW89/wG/FSd48Q3PhR RnKXT9RZEO0EkUcIIzqpLCGgkABTMa2/BJqX86Yhj+681aY6tctEJyEo3I7iURAyH4j+ rVRA== X-Gm-Message-State: AOAM530/Hp9YeWoMLqvXhvI7lbAd2XxYjuCxxyQeaQTe7wh63azu+tGX GsUqrN5NtoizET5+5MQi0A0T6yaD4ss= X-Google-Smtp-Source: ABdhPJzLzGmIn1+LCDpkY+ny6vWC8ch9yN7MkPjoJ1hLDKBLYRlvN68idXIRLbm+4yMdYAr5xOT5ljpdDNM= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:a25:868d:: with SMTP id z13mr3614585ybk.270.1623282185838; Wed, 09 Jun 2021 16:43:05 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:30 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-11-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 10/15] KVM: x86: Use KVM_REQ_TLB_FLUSH_GUEST to handle INVPCID(ALL) emulation From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use KVM_REQ_TLB_FLUSH_GUEST instead of KVM_REQ_MMU_RELOAD when emulating INVPCID of all contexts. In the current code, this is a glorified nop as TLB_FLUSH_GUEST becomes kvm_mmu_unload(), same as MMU_RELOAD, when TDP is disabled, which is the only time INVPCID is only intercepted+emulated. In the future, reusing TLB_FLUSH_GUEST will simplify optimizing paths that emulate a guest TLB flush, e.g. by synchronizing as needed instead of completely unloading all MMUs. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 117acfbc7ba9..9620d8936dc4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11776,7 +11776,7 @@ int kvm_handle_invpcid(struct kvm_vcpu *vcpu, unsigned long type, gva_t gva) fallthrough; case INVPCID_TYPE_ALL_INCL_GLOBAL: - kvm_make_request(KVM_REQ_MMU_RELOAD, vcpu); + kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); return kvm_skip_emulated_instruction(vcpu); default: From patchwork Wed Jun 9 23:42:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D93CC48BD1 for ; Wed, 9 Jun 2021 23:43:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EC904613F2 for ; Wed, 9 Jun 2021 23:43:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230259AbhFIXp1 (ORCPT ); Wed, 9 Jun 2021 19:45:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230197AbhFIXpR (ORCPT ); Wed, 9 Jun 2021 19:45:17 -0400 Received: from mail-qt1-x84a.google.com (mail-qt1-x84a.google.com [IPv6:2607:f8b0:4864:20::84a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 038BEC0617A6 for ; Wed, 9 Jun 2021 16:43:09 -0700 (PDT) Received: by mail-qt1-x84a.google.com with SMTP id f17-20020ac87f110000b02901e117339ea7so12478015qtk.16 for ; Wed, 09 Jun 2021 16:43:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=fb8S+8E9GBSwAg++L6gtt8Nn1MrMQ2oNQg5Qhp48JOI=; b=Ree6kn2WyBZamEhL9hgHMOjzUSQOjH+eIYk+p+gjbUP0mikIvxbVSfgE+0Z4N0XyX3 87WO6x8RsY9ZBtZWyDeL5/88a7ayzvS+7KY4JTduCKwezoHGwdZgZ0pU0rHqTqZv+OKi UIIc+DG/Q1BEE9lE0BXj0eZb2OgKg8dPMbxDBHVGuJ2KSX0t8560Ddrc7UWcbDMNF7EJ ms3Jy09sEfyNNhdjPYTyywwtkofXcpIlrWoW4axvzniVFb6EjNwQ0D3Oc7bek0wTkJdD r/ix5fhoF4SKxEHoyVn3zdEzmMna9bRXGApvBJkg/G9FzSC9x795tDrrM3DtrJKIYi+O lq5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=fb8S+8E9GBSwAg++L6gtt8Nn1MrMQ2oNQg5Qhp48JOI=; b=pksTkLWgSnZdQY8qX0ymMI+xlfXT0Dgyi05sY1ZhHOfKrnM9nxg27gitTw0tQ0Tk+y cqUdPF2egUBhvT6zGv6aor1Dby0X+1eg+mTILOcr0V5yEVtfND1eIuxuGvx5rcB11v8w an38F9SjDJUHwORSXgaFId8zrEhZSvHyu8Exk28CeXheDfS2vnGZ/L9udc++dLqnxSrW KjyBFRIoBpxuqF8mFiARGhHQwiJCSDWuHUSgTjicWRy5HKD6iKL8JiE3BBcSx9XEwvkk cpykZVPv+NMoZnjJOYqFLMAYzEHdrhXrI8IJ8FP4hmCTWliJ0D7vgUkmV42gxf+N7lAB iYKA== X-Gm-Message-State: AOAM533eizFSeA2hHSaUuPC5kRlvkhzlTjMfZiIMQiB8ZMQMnlL1eTcs h7CkK5LEVocjuvUIoYYobPSXH04vYLQ= X-Google-Smtp-Source: ABdhPJzhbO4/J4uB9VkFdBImQ4cEiICxjk+UoMkmyRx/RSskI7a7LuITuCfxoTsTCavmVeYe2LqcU+RZ83s= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:ad4:50c6:: with SMTP id e6mr2458346qvq.6.1623282188110; Wed, 09 Jun 2021 16:43:08 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:31 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-12-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 11/15] KVM: nVMX: Use fast PGD switch when emulating VMFUNC[EPTP_SWITCH] From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use __kvm_mmu_new_pgd() via kvm_init_shadow_ept_mmu() to emulate VMFUNC[EPTP_SWITCH] instead of nuking all MMUs. EPTP_SWITCH is the EPT equivalent of MOV to CR3, i.e. is a perfect fit for the common PGD flow, the only hiccup being that A/D enabling is buried in the EPTP. But, that is easily handled by bouncing through kvm_init_shadow_ept_mmu(). Explicitly request a guest TLB flush if VPID is disabled. Per Intel's SDM, if VPID is disabled, "an EPTP-switching VMFUNC invalidates combined mappings associated with VPID 0000H (for all PCIDs and for all EP4TA values, where EP4TA is the value of bits 51:12 of EPTP)". Note, this technically is a very bizarre bug fix of sorts if L2 is using PAE paging, as avoiding the full MMU reload also avoids incorrectly reloading the PDPTEs, which the SDM explicitly states are not touched: If PAE paging is in use, an EPTP-switching VMFUNC does not load the four page-directory-pointer-table entries (PDPTEs) from the guest-physical address in CR3. The logical processor continues to use the four guest-physical addresses already present in the PDPTEs. The guest-physical address in CR3 is not translated through the new EPT paging structures (until some operation that would load the PDPTEs). In addition to optimizing L2's MMU shenanigans, avoiding the full reload also optimizes L1's MMU as KVM_REQ_MMU_RELOAD wipes out all roots in both root_mmu and guest_mmu. Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 2a881afc1fd0..4b8f5dca49ac 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -346,16 +346,21 @@ static void nested_ept_inject_page_fault(struct kvm_vcpu *vcpu, vmcs12->guest_physical_address = fault->address; } +static void nested_ept_new_eptp(struct kvm_vcpu *vcpu) +{ + kvm_init_shadow_ept_mmu(vcpu, + to_vmx(vcpu)->nested.msrs.ept_caps & + VMX_EPT_EXECUTE_ONLY_BIT, + nested_ept_ad_enabled(vcpu), + nested_ept_get_eptp(vcpu)); +} + static void nested_ept_init_mmu_context(struct kvm_vcpu *vcpu) { WARN_ON(mmu_is_nested(vcpu)); vcpu->arch.mmu = &vcpu->arch.guest_mmu; - kvm_init_shadow_ept_mmu(vcpu, - to_vmx(vcpu)->nested.msrs.ept_caps & - VMX_EPT_EXECUTE_ONLY_BIT, - nested_ept_ad_enabled(vcpu), - nested_ept_get_eptp(vcpu)); + nested_ept_new_eptp(vcpu); vcpu->arch.mmu->get_guest_pgd = nested_ept_get_eptp; vcpu->arch.mmu->inject_page_fault = nested_ept_inject_page_fault; vcpu->arch.mmu->get_pdptr = kvm_pdptr_read; @@ -5463,8 +5468,10 @@ static int nested_vmx_eptp_switching(struct kvm_vcpu *vcpu, return 1; vmcs12->ept_pointer = new_eptp; + nested_ept_new_eptp(vcpu); - kvm_make_request(KVM_REQ_MMU_RELOAD, vcpu); + if (!nested_cpu_has_vpid(vmcs12)) + kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); } return 0; From patchwork Wed Jun 9 23:42:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 192C8C48BCF for ; Wed, 9 Jun 2021 23:43:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0389761249 for ; Wed, 9 Jun 2021 23:43:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230251AbhFIXp0 (ORCPT ); Wed, 9 Jun 2021 19:45:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230196AbhFIXpR (ORCPT ); Wed, 9 Jun 2021 19:45:17 -0400 Received: from mail-qk1-x749.google.com (mail-qk1-x749.google.com [IPv6:2607:f8b0:4864:20::749]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47B54C061574 for ; Wed, 9 Jun 2021 16:43:11 -0700 (PDT) Received: by mail-qk1-x749.google.com with SMTP id k16-20020ae9f1100000b02903aa0311ef7bso15095389qkg.0 for ; Wed, 09 Jun 2021 16:43:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=ActxyiNIAcKHyidC7LFkIvgoRqVhbyRGA66pjNXcgQE=; b=m07n+P+gFAsAJyhYlFLQ9ZQrL75NF7caFSlvPHloWwX3zHtXO4cbiRRCpbNQ0CVclM Cx5o6D+MnSCtYDGyj8nYX4pcYiYc5I9DceQ2rAA4KmVGM5o02u+qlo9IuiS+LTQ5NQN9 J7/Sy+zysap40wxcRdjNFLQs40StLgmMm5U4OjYIGZF5rMyYp7UcvaQUeVK6ma1h3pk5 iE5z3JUX0ZWsaLac5W/G7x1ieWfyUeCY7ymkW4rtzvWmNBlYJ+aARI1a+ujs2PhEIvg/ JVkDI5PXrVqVJjwDq/ZVqMr7jglH3bUMKpTLUb+idz2IpfjW2fs0pmFhDFBqD2WPUUYU 5XkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=ActxyiNIAcKHyidC7LFkIvgoRqVhbyRGA66pjNXcgQE=; b=EPSySAH42/76NB7oXIz7C3kcw+lhR5w/Mu2c0HS0lhaBJa9Bd2cIX5wDFXVme71ZGc oUIBqkDsXi1WIeIJYovmT5NKKWBLUiFHTuM208yyuIvtj7H3EOBNBCSlBVisQCW1f9tv UoaL1o7QXiiAOCkjkIwkfiFSFviMT+iWbvd9gZekk4jlv5jQP467E4qJ4Gm6hMYWAd+1 69E6S0Vxaz0dhuiIy9MGq/653CdBSmDMWpWXzT2J44Pabu4jGBtvF+sKPZsOQ+juH8t2 /dq70rdO0Xo7zoqu0sYBOI4Imx7NUXhdop4e+tM/w78hx4Kx6WmmVbNuQwjMMKFNwAld EE3w== X-Gm-Message-State: AOAM533cz85PCm8lVEsfInr5cWFPG8jaGjTxeLCD5RHU+amgw8dW46ip yqieWBb4znrvPeEatu1Jz2r982v4MCI= X-Google-Smtp-Source: ABdhPJzmrF3mElvNIB2Fkk21bCN8biEs9twT/HY2vEYwJLImwQzdCDYt5p3QSFOLneMFNnnfPZroTu4kslw= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:a0c:eda5:: with SMTP id h5mr2604134qvr.26.1623282190465; Wed, 09 Jun 2021 16:43:10 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:32 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-13-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 12/15] KVM: x86: Defer MMU sync on PCID invalidation From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Defer the MMU sync on PCID invalidation so that multiple sync requests in a single VM-Exit are batched. This is a very minor optimization as checking for unsync'd children is quite cheap. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9620d8936dc4..d3a2a3375541 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1096,7 +1096,7 @@ static void kvm_invalidate_pcid(struct kvm_vcpu *vcpu, unsigned long pcid) * happen anyway before switching to any other CR3. */ if (kvm_get_active_pcid(vcpu) == pcid) { - kvm_mmu_sync_roots(vcpu); + kvm_make_request(KVM_REQ_MMU_SYNC, vcpu); kvm_make_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu); } From patchwork Wed Jun 9 23:42:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53934C48BCD for ; Wed, 9 Jun 2021 23:44:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 371A5613EC for ; Wed, 9 Jun 2021 23:44:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230409AbhFIXqP (ORCPT ); Wed, 9 Jun 2021 19:46:15 -0400 Received: from mail-yb1-f202.google.com ([209.85.219.202]:41720 "EHLO mail-yb1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230017AbhFIXqI (ORCPT ); Wed, 9 Jun 2021 19:46:08 -0400 Received: by mail-yb1-f202.google.com with SMTP id j7-20020a258b870000b029052360b1e3e2so33425455ybl.8 for ; Wed, 09 Jun 2021 16:44:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=nHEr3fDpE6Im0i88yYJ1tQIvyNGCyZi5EB4ETLvhbfY=; b=ea22XXYvukJm5qYdUPMhFg2EwDHUYSLhllnrvu6LWOdE4ige4l3f5SSF7zSTUtjFZ0 k7V92WLCSNLDNx4JiPgFUIYt6oq3lorgTCYxAU7INZZv0Q7UO7o0r8eyI3WXESUulGyA JMRa8YmO6EUqcxlp5sDD4ejzd836NR1zRmC60Q8bmKx7cH61K330+GujZHcXCPobaqZ3 S4BWkCbgItpkoT/xDTKrR+va7IZVWsr8l6YKXj4f7Do85enlEGckmVfgWscxr34tiCYc UwY5M/SPB2uZwhYpZgAZXDUfk333JhV24AFJ6a8yg4CLmTaR4gdarmffZ2Wpu9BAnAnR 0Kag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=nHEr3fDpE6Im0i88yYJ1tQIvyNGCyZi5EB4ETLvhbfY=; b=GFnvgJr+IVcfviR60yfpspZE8E/giuWFMC3D4JHXV+HqVOMENZGe/xssouhJKQJ9KD Q6na/xpNI0rVszSdDqk8qO+XCfFYwWjv2mezeuq8ldqblY71JfmadLhC41Cl6Tk47PAS E+z9vYW/XqG30pLp0m3rw0XmkbMYpnbW4eO4QZJPAAflUM2QOkfrJZL/Su8Rol32LF++ XnSfPC1oV0xVQ8z2Z66IP7WqnjHhH7bb1xd6IkIju8KGfrxUlTSt0LGXhpC13mKDzuf/ wpCK8WtcyVTCrgPLsKLzj6E1pp6f6jzwEm9ddzGnIRBts1ge6MBWLkOvsB5TQ75BhY+g /JIQ== X-Gm-Message-State: AOAM533XLMJLgfciDdrjSbgAiffKJHimm0tBeEm5/aEGcIj7N7AKdqkP jDuPsP4A27pby4HgWXgt3EMPg/LhB8g= X-Google-Smtp-Source: ABdhPJysM8J+3gM4vv0/ZiTCwqxEs/+tlef/R/iat1CDlz/ELLXR0cKWvayn0RNoduTPVcuYXNlNvduHaNQ= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:a5b:d0e:: with SMTP id y14mr3691106ybp.207.1623282192818; Wed, 09 Jun 2021 16:43:12 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:33 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-14-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 13/15] KVM: x86: Drop pointless @reset_roots from kvm_init_mmu() From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Remove the @reset_roots param from kvm_init_mmu(), the one user, kvm_mmu_reset_context() has already unloaded the MMU and thus freed and invalidated all roots. This also happens to be why the reset_roots=true paths doesn't leak roots; they're already invalid. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu.h | 2 +- arch/x86/kvm/mmu/mmu.c | 13 ++----------- arch/x86/kvm/svm/nested.c | 2 +- arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/x86.c | 2 +- 5 files changed, 6 insertions(+), 15 deletions(-) diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 88d0ed5225a4..63b49725fb24 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -65,7 +65,7 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only); void reset_shadow_zero_bits_mask(struct kvm_vcpu *vcpu, struct kvm_mmu *context); -void kvm_init_mmu(struct kvm_vcpu *vcpu, bool reset_roots); +void kvm_init_mmu(struct kvm_vcpu *vcpu); void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, u32 cr0, u32 cr4, u32 efer, gpa_t nested_cr3); void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, bool execonly, diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f987f2ea4a01..b4fa8ec8afce 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4817,17 +4817,8 @@ static void init_kvm_nested_mmu(struct kvm_vcpu *vcpu) update_last_nonleaf_level(vcpu, g_context); } -void kvm_init_mmu(struct kvm_vcpu *vcpu, bool reset_roots) +void kvm_init_mmu(struct kvm_vcpu *vcpu) { - if (reset_roots) { - uint i; - - vcpu->arch.mmu->root_hpa = INVALID_PAGE; - - for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) - vcpu->arch.mmu->prev_roots[i] = KVM_MMU_ROOT_INFO_INVALID; - } - if (mmu_is_nested(vcpu)) init_kvm_nested_mmu(vcpu); else if (tdp_enabled) @@ -4853,7 +4844,7 @@ kvm_mmu_calc_root_page_role(struct kvm_vcpu *vcpu) void kvm_mmu_reset_context(struct kvm_vcpu *vcpu) { kvm_mmu_unload(vcpu); - kvm_init_mmu(vcpu, true); + kvm_init_mmu(vcpu); } EXPORT_SYMBOL_GPL(kvm_mmu_reset_context); diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index ccd90ea93acd..8a4276d8753d 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -421,7 +421,7 @@ static int nested_svm_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, vcpu->arch.cr3 = cr3; kvm_register_mark_available(vcpu, VCPU_EXREG_CR3); - kvm_init_mmu(vcpu, false); + kvm_init_mmu(vcpu); return 0; } diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 4b8f5dca49ac..f686618d9ede 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -1094,7 +1094,7 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, bool ne vcpu->arch.cr3 = cr3; kvm_register_mark_available(vcpu, VCPU_EXREG_CR3); - kvm_init_mmu(vcpu, false); + kvm_init_mmu(vcpu); return 0; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index d3a2a3375541..32e93492273f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10409,7 +10409,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) kvm_vcpu_mtrr_init(vcpu); vcpu_load(vcpu); kvm_vcpu_reset(vcpu, false); - kvm_init_mmu(vcpu, false); + kvm_init_mmu(vcpu); vcpu_put(vcpu); return 0; From patchwork Wed Jun 9 23:42:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94359C48BCD for ; Wed, 9 Jun 2021 23:43:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7D2EA613FF for ; Wed, 9 Jun 2021 23:43:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230239AbhFIXpb (ORCPT ); Wed, 9 Jun 2021 19:45:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230169AbhFIXpX (ORCPT ); Wed, 9 Jun 2021 19:45:23 -0400 Received: from mail-qv1-xf49.google.com (mail-qv1-xf49.google.com [IPv6:2607:f8b0:4864:20::f49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA2DDC0613A4 for ; Wed, 9 Jun 2021 16:43:15 -0700 (PDT) Received: by mail-qv1-xf49.google.com with SMTP id k12-20020a0cfd6c0000b029020df9543019so16756157qvs.14 for ; Wed, 09 Jun 2021 16:43:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=AvlzpSEIa9aM5YGGjLq2zoHFMvduDK875CtYDd8dUL4=; b=ifbBgdC2nQ37DNJxiu2U2mgYs0uHPbXMIH2n+cwuFFAH3iKBe0qhAO5/HRlTdd/4gs weRLHdWGD9UbIn2pAQUz8rHljyeUxKHvtCbS+xVYvJCSnDe3dxpRkMXraJCLh/gyvhm5 5yrTTOtc7Mny/qg/Gj7Vg4/nOw02qcWhgnsgPkIwueGb2ZabKjxzssvPCtXkp60CfKFE PmoQ45l64W4zQkJrONpN53AUoSBmF7Je3yLzxcZ5RUwEZrYHZLNwDeyGtEeFMYv0WR6g HNk4IRb6Qsrddzx8dVSpTL71KipzEdzcFYuM1HCXzJ5Hg6kSObAN4k3pBurFUMN3geqe de7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=AvlzpSEIa9aM5YGGjLq2zoHFMvduDK875CtYDd8dUL4=; b=IZhovDBABZMcgmcTDeDeJ0+3MB+qrRLKoJxD9W0T3H6ISq5cRQYkz/5rLYBUGjIr5q wVVOgI+LBWuIVo6ujl/h+F0F2Ljpv01r+QgzGGyLogRScpCgwchjld5ZkRwq3fB/O6q2 1/GHqgnUk5hreL3tqreimy6Bgbd25Bn3b3ZETzVZbKde32b5g50xeFM9Wynicby9HTIT AeBaG46Z4Ep+fjKrk6Sf20k2ekcXMMUg7k7O5ky45V+vdwLp8PEiV0XpYZHi58UtGL81 +1Ug/vcViWJpnbZNpmtFTNVefNajkKbvqFjNRvMsKSVebrBSTdgESHcBLG2yGag3BghL QGaw== X-Gm-Message-State: AOAM530L4583VKHMroWCuLmw5SZy9TgTfS/BLjLQYsbIcj9VE71ZD7GG SL3/5Z+oLV6sdMwNmnz4XASXMmUMqNs= X-Google-Smtp-Source: ABdhPJx232hNBWou88XddItzp4JVduww7ZRV8rUjKiIgDLbiXVcQWPYtOjEGIKR1VhkY6bLEHvW6mLXOC/U= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:ad4:576e:: with SMTP id r14mr2569384qvx.61.1623282195011; Wed, 09 Jun 2021 16:43:15 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:34 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-15-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 14/15] KVM: nVMX: WARN if subtly-impossible VMFUNC conditions occur From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org WARN and inject #UD when emulating VMFUNC for L2 if the function is out-of-bounds or if VMFUNC is not enabled in vmcs12. Neither condition should occur in practice, as the CPU is supposed to prioritize the #UD over VM-Exit for out-of-bounds input and KVM is supposed to enable VMFUNC in vmcs02 if and only if it's enabled in vmcs12, but neither of those dependencies is obvious. Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index f686618d9ede..0075d3f0f8fa 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -5494,6 +5494,16 @@ static int handle_vmfunc(struct kvm_vcpu *vcpu) } vmcs12 = get_vmcs12(vcpu); + + /* + * #UD on out-of-bounds function has priority over VM-Exit, and VMFUNC + * is enabled in vmcs02 if and only if it's enabled in vmcs12. + */ + if (WARN_ON_ONCE((function > 63) || !nested_cpu_has_vmfunc(vmcs12))) { + kvm_queue_exception(vcpu, UD_VECTOR); + return 1; + } + if (!(vmcs12->vm_function_control & BIT_ULL(function))) goto fail; From patchwork Wed Jun 9 23:42:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12311459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C461C48BCD for ; Wed, 9 Jun 2021 23:43:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 271D5613F2 for ; Wed, 9 Jun 2021 23:43:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230148AbhFIXpV (ORCPT ); Wed, 9 Jun 2021 19:45:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230084AbhFIXpN (ORCPT ); Wed, 9 Jun 2021 19:45:13 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C908C0613A2 for ; Wed, 9 Jun 2021 16:43:18 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id q63-20020a25d9420000b0290532e824f77cso33532500ybg.10 for ; Wed, 09 Jun 2021 16:43:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=JW6PtqG5EBBMLWfIzHJuPDSiGDz3QYcEpSpJG7r7k4g=; b=Wbx/J0n9iESf85d0G1L74LLD2eLDijf8SWts0pPqgG2Sh6Qt30OBZcAeexQRuv2Eic AdDdnF7Ylc2SOHlJdoC3UVNxMt/X7lyHmAFrIxYmvsssBjq2lE/qjMVC5Ywox/b4dmbK 6Or2DNy9yEyhVAPfwLdxCSxfRL9wl/nJtSKM2e0/naTxCLKa7n1DZDteuLHsWHbSEP7k ywPmFqd7HCMdpm/KUzBcwG+HN20+uUH2hHynJtjnXjLJVRXqGxt9qgntherm675NlU2o tRNazhW5B1ioLVhGBf9yD6NY4i0tH3UR7T73lP+Sbzl2JFxhoSF4dnkXP/Slxro3GFgI eN+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=JW6PtqG5EBBMLWfIzHJuPDSiGDz3QYcEpSpJG7r7k4g=; b=tOezK+YmPKcfC7U9Pfy/cnJy6X5KFEIiqbiC07jEBIcdW4GpcZZ8mr6fNIEBGB1NYD OzUqNgstwJbrH+6TBz7P6ohY3c7ecvPGbeLvzey7QP7ECJ/R44eb9nJ8XGIrM2CXIyV5 Zm6bK1Iq4zEmaQ1JzRRThoJPLLvpZ+7CW/g4I3QynQMrl3Ph/on1pSeU6t0aXJTJeH0O 0EGf749xLlzhsnot3PGCKEJYSDuD3Vus4kAsRnKsQMJ2ZcfGtMqCtZVZPNmUvSHDlldg H5hHpKtgIWCDt/8ggKaf8NWbXxQ8VBNV239eMIT+lQxlcCFAWBtTCjS87kGhHcHoazz9 +iEA== X-Gm-Message-State: AOAM533Eehb3JT0cJ80OstV57oS0h3gqtPQFuiiZtQHBfRcsbsey1clu RF5prOrKb4YDdiaPsR1Pe/R5EAyuoBo= X-Google-Smtp-Source: ABdhPJz/9YQGh8Mu2InFaNSyU2MwVQXomSR7fbxj6idTc8vdJENDrCVczbkCOzHIWvle3h2EcYAz7iWtxKw= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:8daf:e5e:ae50:4f28]) (user=seanjc job=sendgmr) by 2002:a25:aa53:: with SMTP id s77mr3153575ybi.89.1623282197388; Wed, 09 Jun 2021 16:43:17 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 9 Jun 2021 16:42:35 -0700 In-Reply-To: <20210609234235.1244004-1-seanjc@google.com> Message-Id: <20210609234235.1244004-16-seanjc@google.com> Mime-Version: 1.0 References: <20210609234235.1244004-1-seanjc@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 15/15] KVM: nVMX: Drop redundant checks on vmcs12 in EPTP switching emulation From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Junaid Shahid , Maxim Levitsky , Lai Jiangshan Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Drop the explicit checks on EPTP switching and EPT itself being enabled. The EPTP switching check is handled in the generic VMFUNC function check, the underlying VMFUNC enablement check is done by hardware and redone by generic VMFUNC emulation, and the vmcs12 EPT check is handled by KVM at VM-Enter in the form of a consistency check. Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 0075d3f0f8fa..479ec9378609 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -5448,10 +5448,6 @@ static int nested_vmx_eptp_switching(struct kvm_vcpu *vcpu, u32 index = kvm_rcx_read(vcpu); u64 new_eptp; - if (!nested_cpu_has_eptp_switching(vmcs12) || - !nested_cpu_has_ept(vmcs12)) - return 1; - if (index >= VMFUNC_EPTP_ENTRIES) return 1;