From patchwork Tue Sep 21 08:10:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Stevens X-Patchwork-Id: 12507369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7224C433FE for ; Tue, 21 Sep 2021 08:10:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AD00A61181 for ; Tue, 21 Sep 2021 08:10:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230516AbhIUIMF (ORCPT ); Tue, 21 Sep 2021 04:12:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230503AbhIUIME (ORCPT ); Tue, 21 Sep 2021 04:12:04 -0400 Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A37C0C061574 for ; Tue, 21 Sep 2021 01:10:36 -0700 (PDT) Received: by mail-pf1-x42c.google.com with SMTP id 145so4817393pfz.11 for ; Tue, 21 Sep 2021 01:10:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gqSb6j4QRhVbcfEJejcSoBsQvuv4/cNZIL9rAN4Q0hA=; b=PELG7kS3lN8UtXOhNNLdA4nqJ60qmLHh7+zrQsC6JizwO6CT+YPqFBafj62bp7SZqj t0dkM6lqQETxe3vXhlMjoTQ6gsFbPGw50UtjDmL652mw7ovatrVJiAddT3xcjTFascjj hK5fO3Fy/tpm5EYWb/YSD23E2SMl6sjoWWDrQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gqSb6j4QRhVbcfEJejcSoBsQvuv4/cNZIL9rAN4Q0hA=; b=3C/Fs1chXuTzvzjLSFsfh5emkm52vft0NNw4cbJl/TW01x7N/m1oc9bSDBx00eGHUw dTQ0kFvi1LylT7xz8V9wzVdX7wb/Jpxdt9UzHEfJwziKIFL4kVTY41rAJDyTYe0AnCfS /d1QHTJM2H+gIZw1T5eH5fW9QVKPmui12cEfx5HiXyQH7f2H4sjeoAdV/tRXVc5K4hP6 gyDtZNZ046tqQ3/M2PDkn5zZyx9Iu7coiq3Zrqbebpjp64J6IexJTaSx8AgMV3aahzeM XlSfai8z+JFhEOBL5zyberLRUYWkZLHECx0bqVDfmxC+Mm6bB0z0vF9tbqnvHvAKyg6N Ss9w== X-Gm-Message-State: AOAM531r/X11LnlcL4rhaltcMKHojs2bCm6OkY/9fMoPxrGBLAewPsbU X7Baf+NZmdB1M6HPXAEujBZy8NKj2SxCbA== X-Google-Smtp-Source: ABdhPJzEf69drbQ3iEQj+oBbjYexjFbvcsB9OmJbkWU0ykpWZgd3uHqpEwhVmCovht8coMxX6SMWnQ== X-Received: by 2002:a62:1717:0:b0:440:527f:6664 with SMTP id 23-20020a621717000000b00440527f6664mr27914710pfx.73.1632211836048; Tue, 21 Sep 2021 01:10:36 -0700 (PDT) Received: from localhost ([2401:fa00:8f:203:2b35:ed47:4cb5:84b]) by smtp.gmail.com with UTF8SMTPSA id w22sm12947071pgc.56.2021.09.21.01.10.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 21 Sep 2021 01:10:35 -0700 (PDT) From: David Stevens X-Google-Original-From: David Stevens To: kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , David Stevens Subject: [PATCH 1/3] KVM: x86: add config for non-kvm users of page tracking Date: Tue, 21 Sep 2021 17:10:08 +0900 Message-Id: <20210921081010.457591-2-stevensd@google.com> X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog In-Reply-To: <20210921081010.457591-1-stevensd@google.com> References: <20210921081010.457591-1-stevensd@google.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: David Stevens Add a config option that allows kvm to determine whether or not there are any external users of page tracking. Signed-off-by: David Stevens --- arch/x86/kvm/Kconfig | 3 +++ drivers/gpu/drm/i915/Kconfig | 1 + 2 files changed, 4 insertions(+) diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index ac69894eab88..619186138176 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -129,4 +129,7 @@ config KVM_MMU_AUDIT This option adds a R/W kVM module parameter 'mmu_audit', which allows auditing of KVM MMU events at runtime. +config KVM_EXTERNAL_WRITE_TRACKING + bool + endif # VIRTUALIZATION diff --git a/drivers/gpu/drm/i915/Kconfig b/drivers/gpu/drm/i915/Kconfig index f960f5d7664e..107762427648 100644 --- a/drivers/gpu/drm/i915/Kconfig +++ b/drivers/gpu/drm/i915/Kconfig @@ -126,6 +126,7 @@ config DRM_I915_GVT_KVMGT depends on DRM_I915_GVT depends on KVM depends on VFIO_MDEV + select KVM_EXTERNAL_WRITE_TRACKING default n help Choose this option if you want to enable KVMGT support for From patchwork Tue Sep 21 08:10:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Stevens X-Patchwork-Id: 12507371 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61793C433EF for ; Tue, 21 Sep 2021 08:10:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 44F7C611ED for ; Tue, 21 Sep 2021 08:10:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231133AbhIUIMI (ORCPT ); Tue, 21 Sep 2021 04:12:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231124AbhIUIMH (ORCPT ); Tue, 21 Sep 2021 04:12:07 -0400 Received: from mail-pf1-x430.google.com (mail-pf1-x430.google.com [IPv6:2607:f8b0:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F87EC061574 for ; Tue, 21 Sep 2021 01:10:40 -0700 (PDT) Received: by mail-pf1-x430.google.com with SMTP id k17so4393088pff.8 for ; Tue, 21 Sep 2021 01:10:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+30wEd0qPykkQFIQZ4HkTpt+CXX50j+PYHWtyQl6vRc=; b=bqzAhp5WeQKf1XBVbKjDSt4Wf/ujjv81U8vpoVXiEwFjgbExKZ9+KoTyVQLN5gVDHh MdPt83xwB9gNP7d+F0uMmr99mZUjUlwl5ua3kRXhHUTFnQChLadyQV897rblhbZXyMLn 1+3gWUz5RXWH/ioW11uu4ExfCh+vXoiSsB6bE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+30wEd0qPykkQFIQZ4HkTpt+CXX50j+PYHWtyQl6vRc=; b=6cUmzWEOJjKmt0nvUHpILUrJgx7nHMbUQhrNf2HfUE510dlUyMskmkN/0CZ7muduqi YQjM33cFhTnb7qS5Z3UnBbKNuJ9nVuKSu3t6dw5Nk9fem55KaTtLt/6xt2ok2ePni0ig qGBY0yDzVriPrnY0UCimCiQP8ArGEbJZWBA9uTv2Lr9+vLIkXpvARHC455xMCTWXbVfC ebzLIOkAk7u7MlaKc24A+JnIxUFbpbhOKSZi+SyFPgiyhJ8emsZYqQWAiu09ZEH1jPQ5 ySi+idQCyr3+9yLwIg6ZKV9aYlgs/G/OHZZM3zzroCpX1QLAH7U1dqgfZgIOxAJPweYt zTcQ== X-Gm-Message-State: AOAM531benvX+Tuwz7fc9Oix2/80bGI7R+PXXS8cI6Acha7qC2kcaWsg rVefi+O7mFO2jZv7E/QX6d7N3XJcpyDlrQ== X-Google-Smtp-Source: ABdhPJyz20FY1BueYn2d92a4fnYRdZSoX8b5fqYSOKSgk3xMifwWcVMIjY9BWK/ot5bwACPJwfC4uQ== X-Received: by 2002:a63:6e02:: with SMTP id j2mr27490957pgc.157.1632211839398; Tue, 21 Sep 2021 01:10:39 -0700 (PDT) Received: from localhost ([2401:fa00:8f:203:2b35:ed47:4cb5:84b]) by smtp.gmail.com with UTF8SMTPSA id y15sm17291404pfq.32.2021.09.21.01.10.37 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 21 Sep 2021 01:10:39 -0700 (PDT) From: David Stevens X-Google-Original-From: David Stevens To: kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , David Stevens Subject: [PATCH 2/3] KVM: x86/mmu: skip page tracking when possible Date: Tue, 21 Sep 2021 17:10:09 +0900 Message-Id: <20210921081010.457591-3-stevensd@google.com> X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog In-Reply-To: <20210921081010.457591-1-stevensd@google.com> References: <20210921081010.457591-1-stevensd@google.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: David Stevens Skip allocating gfn_track arrays if nothing will use them, to save 2 bytes per 4kb of guest memory. This can be done when there are no external users of page tracking (i.e. GVT-g is disabled), KVM is using tdp, and the guest doesn't support nested virtualization. Signed-off-by: David Stevens --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/include/asm/kvm_page_track.h | 5 +++-- arch/x86/kvm/mmu/page_track.c | 25 +++++++++++++++++++++---- arch/x86/kvm/svm/svm.c | 6 ++++++ arch/x86/kvm/vmx/vmx.c | 6 ++++++ arch/x86/kvm/x86.c | 5 +++-- 7 files changed, 42 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index cefe1d81e2e8..99b0254305ea 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -18,6 +18,7 @@ KVM_X86_OP_NULL(hardware_unsetup) KVM_X86_OP_NULL(cpu_has_accelerated_tpr) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) +KVM_X86_OP(nested_enabled) KVM_X86_OP(vm_init) KVM_X86_OP_NULL(vm_destroy) KVM_X86_OP(vcpu_create) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index f8f48a7ec577..ba29f64d16d3 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1049,6 +1049,7 @@ struct kvm_arch { struct list_head lpage_disallowed_mmu_pages; struct kvm_page_track_notifier_node mmu_sp_tracker; struct kvm_page_track_notifier_head track_notifier_head; + bool mmu_uses_page_tracking; /* * Protects marking pages unsync during page faults, as TDP MMU page * faults only take mmu_lock for read. For simplicity, the unsync @@ -1302,6 +1303,7 @@ struct kvm_x86_ops { bool (*cpu_has_accelerated_tpr)(void); bool (*has_emulated_msr)(struct kvm *kvm, u32 index); void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); + bool (*nested_enabled)(void); unsigned int vm_size; int (*vm_init)(struct kvm *kvm); diff --git a/arch/x86/include/asm/kvm_page_track.h b/arch/x86/include/asm/kvm_page_track.h index 87bd6025d91d..92e9e2c74219 100644 --- a/arch/x86/include/asm/kvm_page_track.h +++ b/arch/x86/include/asm/kvm_page_track.h @@ -46,11 +46,12 @@ struct kvm_page_track_notifier_node { struct kvm_page_track_notifier_node *node); }; -void kvm_page_track_init(struct kvm *kvm); +void kvm_page_track_init(struct kvm *kvm, bool mmu_uses_page_tracking); void kvm_page_track_cleanup(struct kvm *kvm); void kvm_page_track_free_memslot(struct kvm_memory_slot *slot); -int kvm_page_track_create_memslot(struct kvm_memory_slot *slot, +int kvm_page_track_create_memslot(struct kvm *kvm, + struct kvm_memory_slot *slot, unsigned long npages); void kvm_slot_page_track_add_page(struct kvm *kvm, diff --git a/arch/x86/kvm/mmu/page_track.c b/arch/x86/kvm/mmu/page_track.c index 269f11f92fd0..c553bc09d50a 100644 --- a/arch/x86/kvm/mmu/page_track.c +++ b/arch/x86/kvm/mmu/page_track.c @@ -29,11 +29,16 @@ void kvm_page_track_free_memslot(struct kvm_memory_slot *slot) } } -int kvm_page_track_create_memslot(struct kvm_memory_slot *slot, +int kvm_page_track_create_memslot(struct kvm *kvm, + struct kvm_memory_slot *slot, unsigned long npages) { int i; + if (!IS_ENABLED(CONFIG_KVM_EXTERNAL_WRITE_TRACKING) && + !kvm->arch.mmu_uses_page_tracking) + return 0; + for (i = 0; i < KVM_PAGE_TRACK_MAX; i++) { slot->arch.gfn_track[i] = kvcalloc(npages, sizeof(*slot->arch.gfn_track[i]), @@ -61,10 +66,15 @@ static void update_gfn_track(struct kvm_memory_slot *slot, gfn_t gfn, enum kvm_page_track_mode mode, short count) { int index, val; + unsigned short *arr; + + arr = slot->arch.gfn_track[mode]; + if (WARN_ON(!arr)) + return; index = gfn_to_index(gfn, slot->base_gfn, PG_LEVEL_4K); - val = slot->arch.gfn_track[mode][index]; + val = arr[index]; if (WARN_ON(val + count < 0 || val + count > USHRT_MAX)) return; @@ -143,6 +153,7 @@ bool kvm_page_track_is_active(struct kvm_vcpu *vcpu, gfn_t gfn, enum kvm_page_track_mode mode) { struct kvm_memory_slot *slot; + unsigned short *arr; int index; if (WARN_ON(!page_track_mode_is_valid(mode))) @@ -152,8 +163,12 @@ bool kvm_page_track_is_active(struct kvm_vcpu *vcpu, gfn_t gfn, if (!slot) return false; + arr = slot->arch.gfn_track[mode]; + if (!arr) + return false; + index = gfn_to_index(gfn, slot->base_gfn, PG_LEVEL_4K); - return !!READ_ONCE(slot->arch.gfn_track[mode][index]); + return !!READ_ONCE(arr[index]); } void kvm_page_track_cleanup(struct kvm *kvm) @@ -164,13 +179,15 @@ void kvm_page_track_cleanup(struct kvm *kvm) cleanup_srcu_struct(&head->track_srcu); } -void kvm_page_track_init(struct kvm *kvm) +void kvm_page_track_init(struct kvm *kvm, bool mmu_uses_page_tracking) { struct kvm_page_track_notifier_head *head; head = &kvm->arch.track_notifier_head; init_srcu_struct(&head->track_srcu); INIT_HLIST_HEAD(&head->track_notifier_list); + + kvm->arch.mmu_uses_page_tracking = mmu_uses_page_tracking; } /* diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 1a70e11f0487..bcc8f887d3bd 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4049,6 +4049,11 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) } } +static bool svm_nested_enabled(void) +{ + return nested; +} + static bool svm_has_wbinvd_exit(void) { return true; @@ -4590,6 +4595,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .get_exit_info = svm_get_exit_info, .vcpu_after_set_cpuid = svm_vcpu_after_set_cpuid, + .nested_enabled = svm_nested_enabled, .has_wbinvd_exit = svm_has_wbinvd_exit, diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 0c2c0d5ae873..1b21ed01e837 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7225,6 +7225,11 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) vmx_update_exception_bitmap(vcpu); } +static bool vmx_nested_enabled(void) +{ + return nested; +} + static __init void vmx_set_cpu_caps(void) { kvm_set_cpu_caps(); @@ -7637,6 +7642,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .get_exit_info = vmx_get_exit_info, .vcpu_after_set_cpuid = vmx_vcpu_after_set_cpuid, + .nested_enabled = vmx_nested_enabled, .has_wbinvd_exit = cpu_has_vmx_wbinvd_exit, diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 28ef14155726..cc47571a148a 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11174,7 +11174,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) kvm_apicv_init(kvm); kvm_hv_init_vm(kvm); - kvm_page_track_init(kvm); + kvm_page_track_init(kvm, !tdp_enabled || + static_call(kvm_x86_nested_enabled)); kvm_mmu_init_vm(kvm); kvm_xen_init_vm(kvm); @@ -11474,7 +11475,7 @@ static int kvm_alloc_memslot_metadata(struct kvm *kvm, } } - if (kvm_page_track_create_memslot(slot, npages)) + if (kvm_page_track_create_memslot(kvm, slot, npages)) goto out_free; return 0; From patchwork Tue Sep 21 08:10:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Stevens X-Patchwork-Id: 12507373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F06BEC433F5 for ; Tue, 21 Sep 2021 08:10:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DA05D611ED for ; Tue, 21 Sep 2021 08:10:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230527AbhIUIMM (ORCPT ); Tue, 21 Sep 2021 04:12:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34350 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231148AbhIUIML (ORCPT ); Tue, 21 Sep 2021 04:12:11 -0400 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F4B4C061574 for ; Tue, 21 Sep 2021 01:10:43 -0700 (PDT) Received: by mail-pl1-x62f.google.com with SMTP id 5so12832627plo.5 for ; Tue, 21 Sep 2021 01:10:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hZaPD0afvShGe4v0fn38fl1H66ICVahlmkFLFvCamco=; b=hX0/JMqYSNri5hBXw6Te8PjCPqs/ww76NjT1aZxKhu/zWgVUMiXFq/ZSSKPUI4JBsK JxWBOlhNxhCG+4jWWYPnbhqavZb6V3oKOTzxCwLJ0Rnxvu3mqei8NtKYZIzSekfqVD6A rtasEPkfSIgIuSAi4s2SrInR1EPZbnnHXGVWQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hZaPD0afvShGe4v0fn38fl1H66ICVahlmkFLFvCamco=; b=AGe9a9ZEkC/4sDqev+w2HOx9lM3vvdexcCUotLHU1j3pG2QZVf9lX3wltmxQpHkbTG erVp+XON1Syn0OD9lQ6J6smPbDId/wN1Tvh9Rv1sijUVEJOeIMI69C9IRbsZ68FlRkC/ QqLBk60NIcuvRCKIxREvQqANUaV+jhtTW6dnh6M/ilIoFYfbo2yDdepwudADXn/ITnP7 WaqDafR0DLYSEewMtHZYWLRPI5jhSAyCWOQhfm9oBk+8pnYsSHkV/+E79i25rqlCbVmD WrGl/z9PmPra57JIvZoPrEGVdULQv1QimKBQkUGjCX23Ev2WzSjbdQ9I7VlLjojd3LIE XhTw== X-Gm-Message-State: AOAM530GrQwKqXBjJrh11oFRbmuSDgy5eqbtcW3MCYfHzofDVi9hnANW rnN7nQt3Jd+DWm9A/kwbDqoY+kTXVA57iQ== X-Google-Smtp-Source: ABdhPJwYSKopMqh8ifvMSn7UNd8mYqRfM63o0v0U97ggPCfrhjqX8F4oYavGdUmD66p0nEspiST1LQ== X-Received: by 2002:a17:903:1207:b0:138:e2f9:6c98 with SMTP id l7-20020a170903120700b00138e2f96c98mr27149217plh.11.1632211842917; Tue, 21 Sep 2021 01:10:42 -0700 (PDT) Received: from localhost ([2401:fa00:8f:203:2b35:ed47:4cb5:84b]) by smtp.gmail.com with UTF8SMTPSA id o20sm1552630pgd.31.2021.09.21.01.10.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 21 Sep 2021 01:10:42 -0700 (PDT) From: David Stevens X-Google-Original-From: David Stevens To: kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , David Stevens Subject: [PATCH 3/3] KVM: VMX: skip page tracking based on cpuid Date: Tue, 21 Sep 2021 17:10:10 +0900 Message-Id: <20210921081010.457591-4-stevensd@google.com> X-Mailer: git-send-email 2.33.0.464.g1972c5931b-goog In-Reply-To: <20210921081010.457591-1-stevensd@google.com> References: <20210921081010.457591-1-stevensd@google.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: David Stevens Consider X86_FEATURE_VMX when determining whether or not page tracking is necessary, since that flag can be used to control whether or not a particular guest supports nested virtualization. When the flag is toggled, it is necessary to free/allocate gfn_track arrays for any already existing slots. If cpuid is heterogeneous or is modified after KVM_RUN, page tracking may not work properly. This may cause guest instability, as per the caveat on KVM_SET_CPUID{,2}. Host stability should not be affected. Signed-off-by: David Stevens --- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/include/asm/kvm_page_track.h | 2 + arch/x86/kvm/cpuid.c | 55 +++++++++++++++++++++------ arch/x86/kvm/mmu/page_track.c | 49 ++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 4 +- arch/x86/kvm/vmx/vmx.c | 7 +++- 6 files changed, 104 insertions(+), 15 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index ba29f64d16d3..dc732ac0eb56 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1302,7 +1302,7 @@ struct kvm_x86_ops { void (*hardware_unsetup)(void); bool (*cpu_has_accelerated_tpr)(void); bool (*has_emulated_msr)(struct kvm *kvm, u32 index); - void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); + int (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); bool (*nested_enabled)(void); unsigned int vm_size; diff --git a/arch/x86/include/asm/kvm_page_track.h b/arch/x86/include/asm/kvm_page_track.h index 92e9e2c74219..f917cea1b4b1 100644 --- a/arch/x86/include/asm/kvm_page_track.h +++ b/arch/x86/include/asm/kvm_page_track.h @@ -49,6 +49,8 @@ struct kvm_page_track_notifier_node { void kvm_page_track_init(struct kvm *kvm, bool mmu_uses_page_tracking); void kvm_page_track_cleanup(struct kvm *kvm); +int kvm_page_tracking_mmu_enable(struct kvm *kvm, bool enable); + void kvm_page_track_free_memslot(struct kvm_memory_slot *slot); int kvm_page_track_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot, diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 739be5da3bca..fb011b0e76e6 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -150,10 +150,11 @@ void kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu) } EXPORT_SYMBOL_GPL(kvm_update_cpuid_runtime); -static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) +static int kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) { struct kvm_lapic *apic = vcpu->arch.apic; struct kvm_cpuid_entry2 *best; + int ret; best = kvm_find_cpuid_entry(vcpu, 1, 0); if (best && apic) { @@ -198,14 +199,20 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) kvm_hv_set_cpuid(vcpu); - /* Invoke the vendor callback only after the above state is updated. */ - static_call(kvm_x86_vcpu_after_set_cpuid)(vcpu); + /* + * Invoke the vendor callback only after the above state is updated. If + * it fails, continue the function and let the caller roll back to the + * previous cpuid. + */ + ret = static_call(kvm_x86_vcpu_after_set_cpuid)(vcpu); /* * Except for the MMU, which needs to do its thing any vendor specific * adjustments to the reserved GPA bits. */ kvm_mmu_after_set_cpuid(vcpu); + + return ret; } static int is_efer_nx(void) @@ -261,9 +268,9 @@ int kvm_vcpu_ioctl_set_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid *cpuid, struct kvm_cpuid_entry __user *entries) { - int r, i; + int r, i, orig_nent; struct kvm_cpuid_entry *e = NULL; - struct kvm_cpuid_entry2 *e2 = NULL; + struct kvm_cpuid_entry2 *e2 = NULL, *orig_e2; if (cpuid->nent > KVM_MAX_CPUID_ENTRIES) return -E2BIG; @@ -298,13 +305,25 @@ int kvm_vcpu_ioctl_set_cpuid(struct kvm_vcpu *vcpu, goto out_free_cpuid; } - kvfree(vcpu->arch.cpuid_entries); + orig_e2 = vcpu->arch.cpuid_entries; + orig_nent = vcpu->arch.cpuid_nent; + vcpu->arch.cpuid_entries = e2; vcpu->arch.cpuid_nent = cpuid->nent; cpuid_fix_nx_cap(vcpu); kvm_update_cpuid_runtime(vcpu); - kvm_vcpu_after_set_cpuid(vcpu); + r = kvm_vcpu_after_set_cpuid(vcpu); + + if (!r) { + kvfree(orig_e2); + } else { + kvfree(e2); + vcpu->arch.cpuid_entries = orig_e2; + vcpu->arch.cpuid_nent = orig_nent; + kvm_update_cpuid_runtime(vcpu); + kvm_vcpu_after_set_cpuid(vcpu); + } out_free_cpuid: kvfree(e); @@ -316,8 +335,8 @@ int kvm_vcpu_ioctl_set_cpuid2(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid, struct kvm_cpuid_entry2 __user *entries) { - struct kvm_cpuid_entry2 *e2 = NULL; - int r; + struct kvm_cpuid_entry2 *e2 = NULL, *orig_e2; + int r, orig_nent; if (cpuid->nent > KVM_MAX_CPUID_ENTRIES) return -E2BIG; @@ -334,14 +353,26 @@ int kvm_vcpu_ioctl_set_cpuid2(struct kvm_vcpu *vcpu, return r; } - kvfree(vcpu->arch.cpuid_entries); + orig_e2 = vcpu->arch.cpuid_entries; + orig_nent = vcpu->arch.cpuid_nent; + vcpu->arch.cpuid_entries = e2; vcpu->arch.cpuid_nent = cpuid->nent; kvm_update_cpuid_runtime(vcpu); - kvm_vcpu_after_set_cpuid(vcpu); + r = kvm_vcpu_after_set_cpuid(vcpu); - return 0; + if (!r) { + kvfree(orig_e2); + } else { + kvfree(e2); + vcpu->arch.cpuid_entries = orig_e2; + vcpu->arch.cpuid_nent = orig_nent; + kvm_update_cpuid_runtime(vcpu); + kvm_vcpu_after_set_cpuid(vcpu); + } + + return r; } int kvm_vcpu_ioctl_get_cpuid2(struct kvm_vcpu *vcpu, diff --git a/arch/x86/kvm/mmu/page_track.c b/arch/x86/kvm/mmu/page_track.c index c553bc09d50a..e4d0e6ad2178 100644 --- a/arch/x86/kvm/mmu/page_track.c +++ b/arch/x86/kvm/mmu/page_track.c @@ -19,6 +19,55 @@ #include "mmu.h" #include "mmu_internal.h" +int kvm_page_tracking_mmu_enable(struct kvm *kvm, bool enable) +{ + struct kvm_memslots *slots; + struct kvm_memory_slot *s; + unsigned short **gfn_track, *buf; + int i, ret = 0; + + if (IS_ENABLED(CONFIG_KVM_EXTERNAL_WRITE_TRACKING)) + return 0; + + mutex_lock(&kvm->slots_lock); + + if (enable == kvm->arch.mmu_uses_page_tracking) { + mutex_unlock(&kvm->slots_lock); + return 0; + } + + for (i = 0; enable && i < KVM_ADDRESS_SPACE_NUM && !ret; i++) { + slots = __kvm_memslots(kvm, i); + kvm_for_each_memslot(s, slots) { + gfn_track = s->arch.gfn_track + KVM_PAGE_TRACK_WRITE; + *gfn_track = kvcalloc(s->npages, sizeof(*gfn_track), + GFP_KERNEL_ACCOUNT); + if (!*gfn_track) { + ret = -ENOMEM; + break; + } + } + } + + for (i = 0; (!enable || !ret) && i < KVM_ADDRESS_SPACE_NUM; i++) { + slots = __kvm_memslots(kvm, i); + kvm_for_each_memslot(s, slots) { + gfn_track = s->arch.gfn_track + KVM_PAGE_TRACK_WRITE; + buf = *gfn_track; + *gfn_track = NULL; + synchronize_srcu(&kvm->srcu); + kvfree(buf); + } + } + + if (!ret) + kvm->arch.mmu_uses_page_tracking = enable; + + mutex_unlock(&kvm->slots_lock); + + return ret; +} + void kvm_page_track_free_memslot(struct kvm_memory_slot *slot) { int i; diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index bcc8f887d3bd..af904e9f4be9 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -3981,7 +3981,7 @@ static u64 svm_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) return 0; } -static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) +static int svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm = to_svm(vcpu); struct kvm_cpuid_entry2 *best; @@ -4047,6 +4047,8 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) set_msr_interception(vcpu, svm->msrpm, MSR_IA32_SYSENTER_EIP, 1, 1); set_msr_interception(vcpu, svm->msrpm, MSR_IA32_SYSENTER_ESP, 1, 1); } + + return 0; } static bool svm_nested_enabled(void) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 1b21ed01e837..6455759907d1 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7166,7 +7166,7 @@ static void update_intel_pt_cfg(struct kvm_vcpu *vcpu) vmx->pt_desc.ctl_bitmask &= ~(0xfULL << (32 + i * 4)); } -static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) +static int vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -7223,6 +7223,11 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) /* Refresh #PF interception to account for MAXPHYADDR changes. */ vmx_update_exception_bitmap(vcpu); + + if (tdp_enabled) + return kvm_page_tracking_mmu_enable(vcpu->kvm, + nested_vmx_allowed(vcpu)); + return 0; } static bool vmx_nested_enabled(void)