From patchwork Tue Jun 12 22:52:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Junaid Shahid X-Patchwork-Id: 10461317 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 19EA86020F for ; Tue, 12 Jun 2018 22:53:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0C9CC28535 for ; Tue, 12 Jun 2018 22:53:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0129D2877F; Tue, 12 Jun 2018 22:53:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 95ACF28535 for ; Tue, 12 Jun 2018 22:53:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935027AbeFLWxe (ORCPT ); Tue, 12 Jun 2018 18:53:34 -0400 Received: from mail-ua0-f202.google.com ([209.85.217.202]:48107 "EHLO mail-ua0-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S935003AbeFLWx1 (ORCPT ); Tue, 12 Jun 2018 18:53:27 -0400 Received: by mail-ua0-f202.google.com with SMTP id u45-v6so220175uau.14 for ; Tue, 12 Jun 2018 15:53:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:date:in-reply-to:message-id:references:subject:from:to :cc; bh=VgxZvHoFhfWZqW7vmlqWna8Vr/HfUawBjAz2h9CJinw=; b=nl+Z7pk0RGoI7S8mCV+p9q7uhWkcasvWQ8nMv1fsaXmaVV/O8E19ze5Vi4CD+knxP5 g9kLb9YIwm3tZJaZLRnHKRarIJhSKqwAFmM+Mi6HXDjrHK9UJ6RPV/AZetNUsIfGMM32 rCHDL5aH3QXQK6rgeUZibyFYGXJecF4mlYLu01oc+YOHgszqXoKIJfnlolFNpINY/4Vf 0XhVadBTdEraOCZNUy28qJmpN75ImeRtAlFFlk7ZUnuFXs5jseLHmdbxvmnT0dns8y3O WRC7x2jpLmEXxXe7BGmHZlX75pimG+Xe6U7GvwqiwNoirnILDy9BzDXmaDKMihDTUfHX 1EiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:date:in-reply-to:message-id :references:subject:from:to:cc; bh=VgxZvHoFhfWZqW7vmlqWna8Vr/HfUawBjAz2h9CJinw=; b=Iejof+mkTDMiM9+pxtbR6KRJ4Uv7ZCjTetkDIDvd8HlIy/CfNvwxBpTXvs1wNDzoyy 8klXq+lDVwQ1doisTNgQv9ulKHeNDLH2IqqN4hhQKvNQBtNIZfJ6AYGA9SNa+KXzMvjh X3ttvh0/HPelrLzaBRrEE28MyHJ/bauYfVOaE6ScMBVUcwi6kQM5JYDc29qmGjhAfRoA vlZauddfiyA41cDC5acbXPF7bYd6oyGXVZAq0/yACRrqMxUErJL+gC7C4s7FOBFHqzsr ssapKq7qHCrCECSAkeg0rxZuc8XsW1hT8VKL564d3pu7FOjNljCUOGPbinRbc1101pzA sZIw== X-Gm-Message-State: APt69E1xzQcR/6NwpgKNe97GEJ58bpBSZAA6qWsozZKm78viFm63rbp5 xcfuYy0V2tCedNXa57CtVw9R4w8vaXfK X-Google-Smtp-Source: ADUXVKJFWWJZLdj8DF6EqOAqu9H9+WdaJtwc1hXfQLfa6qqCwyYLLyav0L10j7lrbFYBCA43k26kvufEOfNP MIME-Version: 1.0 X-Received: by 2002:ab0:2516:: with SMTP id j22-v6mr1103089uan.121.1528844007212; Tue, 12 Jun 2018 15:53:27 -0700 (PDT) Date: Tue, 12 Jun 2018 15:52:35 -0700 In-Reply-To: <20180612225244.71856-1-junaids@google.com> Message-Id: <20180612225244.71856-9-junaids@google.com> References: <20180612225244.71856-1-junaids@google.com> X-Mailer: git-send-email 2.18.0.rc1.242.g61856ae69a-goog Subject: [PATCH v2 08/17] kvm: x86: Use fast CR3 switch for nested VMX From: Junaid Shahid To: pbonzini@redhat.com Cc: kvm@vger.kernel.org, andreslc@google.com, jmattson@google.com, liran.alon@oracle.com Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Use the fast CR3 switch mechanism to locklessly change the MMU root page when switching between L1 and L2. The switch from L2 to L1 should always go through the fast path, while the switch from L1 to L2 should go through the fast path if L1's CR3/EPTP for L2 hasn't changed since the last time. Signed-off-by: Junaid Shahid --- arch/x86/kvm/mmu.c | 3 ++- arch/x86/kvm/vmx.c | 16 +++++++++++----- virt/kvm/kvm_main.c | 14 ++++++++++---- 3 files changed, 23 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index 845ca328cf38..94d4cf4cb743 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -4043,7 +4043,7 @@ static bool fast_cr3_switch(struct kvm_vcpu *vcpu, gpa_t new_cr3, return false; swap(mmu->root_hpa, mmu->prev_root.hpa); - mmu->prev_root.cr3 = kvm_read_cr3(vcpu); + mmu->prev_root.cr3 = mmu->get_cr3(vcpu); if (new_cr3 == prev_cr3 && VALID_PAGE(mmu->root_hpa) && @@ -4075,6 +4075,7 @@ void kvm_mmu_new_cr3(struct kvm_vcpu *vcpu, gpa_t new_cr3, if (!fast_cr3_switch(vcpu, new_cr3, new_role)) kvm_mmu_free_roots(vcpu, false); } +EXPORT_SYMBOL_GPL(kvm_mmu_new_cr3); static unsigned long get_cr3(struct kvm_vcpu *vcpu) { diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 48989f78be60..9c85c6249280 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -10564,7 +10564,9 @@ static int nested_ept_init_mmu_context(struct kvm_vcpu *vcpu) if (!valid_ept_address(vcpu, nested_ept_get_cr3(vcpu))) return 1; - kvm_mmu_unload(vcpu); + kvm_mmu_new_cr3(vcpu, nested_ept_get_cr3(vcpu), + kvm_mmu_calc_shadow_ept_root_page_role(vcpu, + nested_ept_ad_enabled(vcpu))); kvm_init_shadow_ept_mmu(vcpu, to_vmx(vcpu)->nested.msrs.ept_caps & VMX_EPT_EXECUTE_ONLY_BIT, @@ -11119,12 +11121,16 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, bool ne return 1; } } - - vcpu->arch.cr3 = cr3; - __set_bit(VCPU_EXREG_CR3, (ulong *)&vcpu->arch.regs_avail); } - kvm_mmu_reset_context(vcpu); + if (!nested_ept) + kvm_mmu_new_cr3(vcpu, cr3, kvm_mmu_calc_root_page_role(vcpu)); + + vcpu->arch.cr3 = cr3; + __set_bit(VCPU_EXREG_CR3, (ulong *)&vcpu->arch.regs_avail); + + kvm_init_mmu(vcpu, false); + return 0; } diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index aa7da1d8ece2..e4ee76bee3dd 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2122,16 +2122,22 @@ static void shrink_halt_poll_ns(struct kvm_vcpu *vcpu) static int kvm_vcpu_check_block(struct kvm_vcpu *vcpu) { + int ret = -EINTR; + int idx = srcu_read_lock(&vcpu->kvm->srcu); + if (kvm_arch_vcpu_runnable(vcpu)) { kvm_make_request(KVM_REQ_UNHALT, vcpu); - return -EINTR; + goto out; } if (kvm_cpu_has_pending_timer(vcpu)) - return -EINTR; + goto out; if (signal_pending(current)) - return -EINTR; + goto out; - return 0; + ret = 0; +out: + srcu_read_unlock(&vcpu->kvm->srcu, idx); + return ret; } /*