From patchwork Tue Mar 2 18:45:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12113055 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 716D7C43142 for ; Wed, 3 Mar 2021 07:23:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 607AF64F02 for ; Wed, 3 Mar 2021 07:23:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1380390AbhCCHSW (ORCPT ); Wed, 3 Mar 2021 02:18:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39286 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1581624AbhCBTAq (ORCPT ); Tue, 2 Mar 2021 14:00:46 -0500 Received: from mail-qv1-xf4a.google.com (mail-qv1-xf4a.google.com [IPv6:2607:f8b0:4864:20::f4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45B69C061788 for ; Tue, 2 Mar 2021 10:45:52 -0800 (PST) Received: by mail-qv1-xf4a.google.com with SMTP id s16so11878668qvw.3 for ; Tue, 02 Mar 2021 10:45:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=wHB5rTht2wbUwkXtbWaUHLq8QDHf2xz7pZGkxQoWdEU=; b=QVLW01bcTBWnkqf2e6KLYMQTgLvLIp5fsHf51XhYWbunjl+jSctBGZ/3ZheeFQhk/d vhvYoPOahtlj2T5a1NXp7DivbiwyfHpAzbH41UKn7Z9+yRWYAsqlsfo6Nqnqhuv1ifYJ 9Nnldr2Zn0/KzvOKX+hrA6OLQa3KX1zPsQfBPp0wvDD+T+FiRUrzRcTWYhiQpL8QBZN/ bZK3ECs205YmFI88fDj/OWowFdZlvkD1jmJD3XYkASeE8R+DZD4DwCWVmX6bAMDqYNJp 0ICX1yjazabfXs2dqOe2q/Hd2THTDUrQ3eNXZ6Vf0qwkRi0rzlzJV7U7TS9iuuBhpaz/ fqOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=wHB5rTht2wbUwkXtbWaUHLq8QDHf2xz7pZGkxQoWdEU=; b=Pkjo76G0PnvKTfTWqkD3+K4KV0SjFE6xmAMQtVpmP/bX7hSgDJhyg81PeHBSSdF958 ljU7LLQRy85xYBwErhPnfyEVdCyW6YTVcKjHOAVyhNmsHHJsA70OJoTE/c3icrSnEc3p dxSBZY0m9JCsA/CqnudqvkcJ/WAPKHI1TIcnHiw1cv+TvJmzQasiJAMr81PZPJYgIYRa jDg71qTWRk/i2SIMbg5/tNFy6BRGhqmf6nm9GDXT5E8RlbTKAKHyGC9LI3B/8aAdmMNX UtewYzOGZGd2M9PQeWbt/AIJbrTts6wjcplUPO8CSvk8+YVN4FeeMBqgbdP/6/PnUJOD cNkA== X-Gm-Message-State: AOAM5308c7eKbVOalxr3srsNRq+mbOJExAPbrb0mI4TRYlDUV6GahygJ UrrCuACWuaZPByFW5CPLljjfazuBheQ= X-Google-Smtp-Source: ABdhPJxddJibr9G6EZKclLhSikto3yLEmV7Ap8MHT9rMpmd+rJT07TmFbLzbRXqnYqJ3Nhm6sPuAlFSzpEY= Sender: "seanjc via sendgmr" X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:805d:6324:3372:6183]) (user=seanjc job=sendgmr) by 2002:a0c:c78c:: with SMTP id k12mr4967337qvj.47.1614710751429; Tue, 02 Mar 2021 10:45:51 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 2 Mar 2021 10:45:27 -0800 In-Reply-To: <20210302184540.2829328-1-seanjc@google.com> Message-Id: <20210302184540.2829328-3-seanjc@google.com> Mime-Version: 1.0 References: <20210302184540.2829328-1-seanjc@google.com> X-Mailer: git-send-email 2.30.1.766.gb4fecdf3b7-goog Subject: [PATCH 02/15] KVM: x86/mmu: Alloc page for PDPTEs when shadowing 32-bit NPT with 64-bit From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ben Gardon , Brijesh Singh , Tom Lendacky Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Allocate the so called pae_root page on-demand, along with the lm_root page, when shadowing 32-bit NPT with 64-bit NPT, i.e. when running a 32-bit L1. KVM currently only allocates the page when NPT is disabled, or when L0 is 32-bit (using PAE paging). Note, there is an existing memory leak involving the MMU roots, as KVM fails to free the PAE roots on failure. This will be addressed in a future commit. Fixes: ee6268ba3a68 ("KVM: x86: Skip pae_root shadow allocation if tdp enabled") Fixes: b6b80c78af83 ("KVM: x86/mmu: Allocate PAE root array when using SVM's 32-bit NPT") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson Reviewed-by: Ben Gardon --- arch/x86/kvm/mmu/mmu.c | 44 ++++++++++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 15 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0987cc1d53eb..2ed3fac1244e 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3187,14 +3187,14 @@ void kvm_mmu_free_roots(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, if (mmu->shadow_root_level >= PT64_ROOT_4LEVEL && (mmu->root_level >= PT64_ROOT_4LEVEL || mmu->direct_map)) { mmu_free_root_page(kvm, &mmu->root_hpa, &invalid_list); - } else { + } else if (mmu->pae_root) { for (i = 0; i < 4; ++i) if (mmu->pae_root[i] != 0) mmu_free_root_page(kvm, &mmu->pae_root[i], &invalid_list); - mmu->root_hpa = INVALID_PAGE; } + mmu->root_hpa = INVALID_PAGE; mmu->root_pgd = 0; } @@ -3306,9 +3306,23 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) * the shadow page table may be a PAE or a long mode page table. */ pm_mask = PT_PRESENT_MASK; - if (vcpu->arch.mmu->shadow_root_level == PT64_ROOT_4LEVEL) + if (vcpu->arch.mmu->shadow_root_level == PT64_ROOT_4LEVEL) { pm_mask |= PT_ACCESSED_MASK | PT_WRITABLE_MASK | PT_USER_MASK; + /* + * Allocate the page for the PDPTEs when shadowing 32-bit NPT + * with 64-bit only when needed. Unlike 32-bit NPT, it doesn't + * need to be in low mem. See also lm_root below. + */ + if (!vcpu->arch.mmu->pae_root) { + WARN_ON_ONCE(!tdp_enabled); + + vcpu->arch.mmu->pae_root = (void *)get_zeroed_page(GFP_KERNEL_ACCOUNT); + if (!vcpu->arch.mmu->pae_root) + return -ENOMEM; + } + } + for (i = 0; i < 4; ++i) { MMU_WARN_ON(VALID_PAGE(vcpu->arch.mmu->pae_root[i])); if (vcpu->arch.mmu->root_level == PT32E_ROOT_LEVEL) { @@ -3331,21 +3345,19 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) vcpu->arch.mmu->root_hpa = __pa(vcpu->arch.mmu->pae_root); /* - * If we shadow a 32 bit page table with a long mode page - * table we enter this path. + * When shadowing 32-bit or PAE NPT with 64-bit NPT, the PML4 and PDP + * tables are allocated and initialized at MMU creation as there is no + * equivalent level in the guest's NPT to shadow. Allocate the tables + * on demand, as running a 32-bit L1 VMM is very rare. The PDP is + * handled above (to share logic with PAE), deal with the PML4 here. */ if (vcpu->arch.mmu->shadow_root_level == PT64_ROOT_4LEVEL) { if (vcpu->arch.mmu->lm_root == NULL) { - /* - * The additional page necessary for this is only - * allocated on demand. - */ - u64 *lm_root; lm_root = (void*)get_zeroed_page(GFP_KERNEL_ACCOUNT); - if (lm_root == NULL) - return 1; + if (!lm_root) + return -ENOMEM; lm_root[0] = __pa(vcpu->arch.mmu->pae_root) | pm_mask; @@ -5248,9 +5260,11 @@ static int __kvm_mmu_create(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu) * while the PDP table is a per-vCPU construct that's allocated at MMU * creation. When emulating 32-bit mode, cr3 is only 32 bits even on * x86_64. Therefore we need to allocate the PDP table in the first - * 4GB of memory, which happens to fit the DMA32 zone. Except for - * SVM's 32-bit NPT support, TDP paging doesn't use PAE paging and can - * skip allocating the PDP table. + * 4GB of memory, which happens to fit the DMA32 zone. TDP paging + * generally doesn't use PAE paging and can skip allocating the PDP + * table. The main exception, handled here, is SVM's 32-bit NPT. The + * other exception is for shadowing L1's 32-bit or PAE NPT on 64-bit + * KVM; that horror is handled on-demand by mmu_alloc_shadow_roots(). */ if (tdp_enabled && kvm_mmu_get_tdp_level(vcpu) > PT32E_ROOT_LEVEL) return 0;