From patchwork Fri Jul 15 23:00:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12919897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC037C433EF for ; Fri, 15 Jul 2022 23:00:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230218AbiGOXAZ (ORCPT ); Fri, 15 Jul 2022 19:00:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59648 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230047AbiGOXAX (ORCPT ); Fri, 15 Jul 2022 19:00:23 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BAC7F3C144 for ; Fri, 15 Jul 2022 16:00:21 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id c18-20020a17090a8d1200b001ef85196fb4so5895463pjo.4 for ; Fri, 15 Jul 2022 16:00:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=4mRUe9k6N29Y8TY671ys1SOzt46CXKbNlnzYh7VBMGg=; b=BY+ofGKbdhxI2uO5Nyqd5dE57NYG6j6ZefdgDGiAMMy15vVMXVMCrAxkz/HmhpImK+ Grv+x99LKg2/2a7c8R/phTxEvCikiuo0xcEF+jCO7cRYnO7Jz86w6MRWzcHiUpAcmpse 0iTfrjC1O28pwL1VrBCVYAXfp8Eo9G1UJBI8jOC23NXXn0vDG7ZLCMOn05PaeAjhfClh wO0w2V7+MZ1Bl/hlLPcPW45z8QlYCjNcKvOAYamGbWsL9h7BLrpWxBRhmOYPC/bkhtLM qUA7RMiIQSituv4YUDa3XRxArvCtVaN2kUHvfSpAQ6HYdnbb4MjeiIsma9YMnf6/n2C4 1sNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=4mRUe9k6N29Y8TY671ys1SOzt46CXKbNlnzYh7VBMGg=; b=WFxlVkW3f2BJlrfjlU9Vhb36h/ah4RlooGLFwyyIXxl0uwhrELn64TJ+xjmt6FWz8T yt7y24cQT160/LurP5xq4Oz7jGzN4/RkFjTzyaQbrScactNLNHO7eNj8W9nLnGfvG96I 7HRgy2JYgBTQNucrahZ4X/EB/4hrSeNzm7AnoSp1/8qhfyD6DNkfGitvQvYqI9OKGS0X DAea0Xl2FaKY/gWcR3fPj0AkojnaEkwoVw9/LQ2ULDepDnSWUJ7ijTkVl/KxrjWXeqAJ SnbpEi+ffuPqzyoo6KCmUh4BXxbvPnpt0Fr0OivpgrvLr1kZv2+M7ZpjgZ2oQqaEWc45 Xb6w== X-Gm-Message-State: AJIora/NK0JlxUtQWs8uuIEfXnXDf6KfkWZo2N9IbsGgeyB+GfObFgAZ 4Gj3Y1fioxRt9EQPniuvIMiKIRVdyqE= X-Google-Smtp-Source: AGRyM1smbVwsZvafhg6YAk0nSxXgCOlfpSnfKN9p9HtlEhf/jQcGocWbnbcv1Ozn+bbDrtBXCdl//YT/YPo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:d54b:b0:16b:eea4:77da with SMTP id z11-20020a170902d54b00b0016beea477damr15632782plf.45.1657926021275; Fri, 15 Jul 2022 16:00:21 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 15 Jul 2022 23:00:13 +0000 In-Reply-To: <20220715230016.3762909-1-seanjc@google.com> Message-Id: <20220715230016.3762909-2-seanjc@google.com> Mime-Version: 1.0 References: <20220715230016.3762909-1-seanjc@google.com> X-Mailer: git-send-email 2.37.0.170.g444d1eabd0-goog Subject: [PATCH 1/4] KVM: x86: Reject loading KVM if host.PAT[0] != WB From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Reject KVM if entry '0' in the host's IA32_PAT MSR is not programmed to writeback (WB) memtype. KVM subtly relies on IA32_PAT entry '0' to be programmed to WB by leaving the PAT bits in shadow paging and NPT SPTEs as '0'. If something other than WB is in PAT[0], at _best_ guests will suffer very poor performance, and at worst KVM will crash the system by breaking cache-coherency expecations (e.g. using WC for guest memory). Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/x86.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index f389691d8c04..12199c40f2bc 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9141,6 +9141,7 @@ static struct notifier_block pvclock_gtod_notifier = { int kvm_arch_init(void *opaque) { struct kvm_x86_init_ops *ops = opaque; + u64 host_pat; int r; if (kvm_x86_ops.hardware_enable) { @@ -9179,6 +9180,20 @@ int kvm_arch_init(void *opaque) goto out; } + /* + * KVM assumes that PAT entry '0' encodes WB memtype and simply zeroes + * the PAT bits in SPTEs. Bail if PAT[0] is programmed to something + * other than WB. Note, EPT doesn't utilize the PAT, but don't bother + * with an exception. PAT[0] is set to WB on RESET and also by the + * kernel, i.e. failure indicates a kernel bug or broken firmware. + */ + if (rdmsrl_safe(MSR_IA32_CR_PAT, &host_pat) || + (host_pat & GENMASK(2, 0)) != 6) { + pr_err("kvm: host PAT[0] is not WB\n"); + r = -EIO; + goto out; + } + r = -ENOMEM; x86_emulator_cache = kvm_alloc_emulator_cache(); From patchwork Fri Jul 15 23:00:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12919898 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EF11C433EF for ; Fri, 15 Jul 2022 23:00:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230240AbiGOXAv (ORCPT ); Fri, 15 Jul 2022 19:00:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230120AbiGOXAY (ORCPT ); Fri, 15 Jul 2022 19:00:24 -0400 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44D594AD4F for ; Fri, 15 Jul 2022 16:00:23 -0700 (PDT) Received: by mail-pj1-x1049.google.com with SMTP id q8-20020a17090a304800b001ef82a71a9eso3611513pjl.3 for ; Fri, 15 Jul 2022 16:00:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=V+ijd0IpTmqqqLjto4/xJJjuTl9mPDgkHleQw8xA900=; b=g2f2U6dAMh2svtMNa0Gg+5Wo0SupEJFUQDZb4iOR1xe/N8YGAQMOLjhFPelZfKgo/c ey5P2g+Qwc12BJCqJLeZZ27863S5UOGpryHde4e5Z1+5/gk0cH07cYblwVoMeGKubPug Q5SWn8SZOfKaTQKlEECe/t/qWODjgKTW49O97PDpDPhy/9oa1bTWWSYNTjALE9yfqGFH D1aT+2F0D63s/iwgfYnUdeFxR+XZXlEIlp0EgfgjQsue3DRFxWn70QkWMilm+nOsOXu+ 67VYZ/MdkbCDsj6eJon214a52MGGbx3XX2e2rZwMTAy0CE4OHw8PNeudcRN+Bf2KWINS Y24w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=V+ijd0IpTmqqqLjto4/xJJjuTl9mPDgkHleQw8xA900=; b=AfSEKS3rOTHG1aoMFwbD2YCbDCMcPGbrXCXzeNXtttAtpoIab8t2DnnMzfbUn04q/1 fIS4UxUVP0ZcticnWcy431vG6Awj/SmxDe9OEkWCr9I2HKrhQhhxrZQCtPkd5QzxdlKk 6e+uT74lhKUBekAFpD781OYwRxcH75db2ngsFwY8PnKTrLYem3wmOPTa8XMGGEAjYmCl IBpdgvt5Ahk4dxxgEwBZtikoauhrv3P87aHevdgkbQQ5XOyLm3iiQY9BPt9YouLZ5Jf2 LgtgUAspXK9AnPQwAapwLmrX8zTFUqwrQ/AY0KmvccbZ8bwlRCcpi9RZzgrgBkcboc63 TnHw== X-Gm-Message-State: AJIora+mhcPFQg+iBX7SjhPzmRSmHdN4rUtFLphZCcMnaPptjObNe3oT qBps21tFZcyxioOx/j1xII0gYQAbyL0= X-Google-Smtp-Source: AGRyM1tWkRhKexMwNka8pMZLwhRDlIvKyPaqRa57ONwmleNE0JM3aPXeIWSQNInSCOIPyAw8GygZDNqnLQU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a62:6d05:0:b0:528:99a2:b10 with SMTP id i5-20020a626d05000000b0052899a20b10mr16101370pfc.72.1657926022849; Fri, 15 Jul 2022 16:00:22 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 15 Jul 2022 23:00:14 +0000 In-Reply-To: <20220715230016.3762909-1-seanjc@google.com> Message-Id: <20220715230016.3762909-3-seanjc@google.com> Mime-Version: 1.0 References: <20220715230016.3762909-1-seanjc@google.com> X-Mailer: git-send-email 2.37.0.170.g444d1eabd0-goog Subject: [PATCH 2/4] KVM: x86: Drop unnecessary goto+label in kvm_arch_init() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Return directly if kvm_arch_init() detects an error before doing any real work, jumping through a label obfuscates what's happening and carries the unnecessary risk of leaving 'r' uninitialized. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/x86.c | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 12199c40f2bc..41aa3137665c 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9146,21 +9146,18 @@ int kvm_arch_init(void *opaque) if (kvm_x86_ops.hardware_enable) { pr_err("kvm: already loaded vendor module '%s'\n", kvm_x86_ops.name); - r = -EEXIST; - goto out; + return -EEXIST; } if (!ops->cpu_has_kvm_support()) { pr_err_ratelimited("kvm: no hardware support for '%s'\n", ops->runtime_ops->name); - r = -EOPNOTSUPP; - goto out; + return -EOPNOTSUPP; } if (ops->disabled_by_bios()) { pr_err_ratelimited("kvm: support for '%s' disabled by bios\n", ops->runtime_ops->name); - r = -EOPNOTSUPP; - goto out; + return -EOPNOTSUPP; } /* @@ -9170,14 +9167,12 @@ int kvm_arch_init(void *opaque) */ if (!boot_cpu_has(X86_FEATURE_FPU) || !boot_cpu_has(X86_FEATURE_FXSR)) { printk(KERN_ERR "kvm: inadequate fpu\n"); - r = -EOPNOTSUPP; - goto out; + return -EOPNOTSUPP; } if (IS_ENABLED(CONFIG_PREEMPT_RT) && !boot_cpu_has(X86_FEATURE_CONSTANT_TSC)) { pr_err("RT requires X86_FEATURE_CONSTANT_TSC\n"); - r = -EOPNOTSUPP; - goto out; + return -EOPNOTSUPP; } /* @@ -9190,21 +9185,19 @@ int kvm_arch_init(void *opaque) if (rdmsrl_safe(MSR_IA32_CR_PAT, &host_pat) || (host_pat & GENMASK(2, 0)) != 6) { pr_err("kvm: host PAT[0] is not WB\n"); - r = -EIO; - goto out; + return -EIO; } - r = -ENOMEM; - x86_emulator_cache = kvm_alloc_emulator_cache(); if (!x86_emulator_cache) { pr_err("kvm: failed to allocate cache for x86 emulator\n"); - goto out; + return -ENOMEM; } user_return_msrs = alloc_percpu(struct kvm_user_return_msrs); if (!user_return_msrs) { printk(KERN_ERR "kvm: failed to allocate percpu kvm_user_return_msrs\n"); + r = -ENOMEM; goto out_free_x86_emulator_cache; } kvm_nr_uret_msrs = 0; @@ -9235,7 +9228,6 @@ int kvm_arch_init(void *opaque) free_percpu(user_return_msrs); out_free_x86_emulator_cache: kmem_cache_destroy(x86_emulator_cache); -out: return r; } From patchwork Fri Jul 15 23:00:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12919899 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6993FC433EF for ; Fri, 15 Jul 2022 23:00:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231210AbiGOXAy (ORCPT ); Fri, 15 Jul 2022 19:00:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230254AbiGOXA1 (ORCPT ); Fri, 15 Jul 2022 19:00:27 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 698A84E631 for ; Fri, 15 Jul 2022 16:00:25 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-317f6128c86so49667897b3.22 for ; Fri, 15 Jul 2022 16:00:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=yUjqGaXxNRIkWFGwl+2pUdgstDXS94lxvTblB9yzYuE=; b=d0HtqcngCGEWckrmfolh8CY2i0XJnzrFb5U6PGDeVR1h0nBWTx0oQovQksQedcqaw5 rNIEvehYP/e2WByw7UXCle8v5tnX9eUF2ixPX/LeaN32FU0RS43PhyHsktjm7mRl9kCx doOlYb7OJsbiz3CvdzNyLZJfaxmHMCvjtfJF3wTxPbYjHHoz9N2QJslDF/oOxCp1ysxU sEAKUl41bbj251CIbRBdBNFCB3nPXrHNA1IXyyN8rh9ycMtnsuGKxk9kPqTN/nUgyl6t ibjUEGvETaptRy5VB8ovSHwrKh1+6LxooBefTD6Tv8D2FL9GLC7UBbdFtd3ghJwAqq47 0eAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=yUjqGaXxNRIkWFGwl+2pUdgstDXS94lxvTblB9yzYuE=; b=7L16VZpXRrx98ckRnQ0UaZ4rhyrcD8WjFjD4MA/QHtIm0xsJZ5bbmH0t/7Uubmc6V/ ofDkjxABcgIyfmzvncG6z8r1GM+M70+bzwGkriqCCcbO7cpqeq6qaJYnQG/AWRoQPjfu ZztqIdziH0GcY+IgsjFW9QDZ0oam1kB0G3xTKpl+yG/WwUnCQplrYLwJxqobHUJAcWuu oR8LJ5JdTnx+2CxY2225jtwQihRtuRgNynIwA9bOtW1hqm0cyPHOQ4itlMZn2jNqNjk3 sTIgcyKX6vBsZ5KhuoqqnEfBLGWBNb/PDY3ttLLgT1cwMR2mu/OzV4B26mG4zhzHbkT0 xwfw== X-Gm-Message-State: AJIora+g3rwtJHtDBPvZppSHiYEr+ClUOWE6TV4Kj42awyd9Q3BiWzmw M8xObqdN2X05FexnUtzg32c5cZM33i0= X-Google-Smtp-Source: AGRyM1tq+cKZgY4d++EmVCawqmZ05GuVHpsnrWVt/0hYjlpfSVIk4z9NrsOkzJTeXpY7o67QjtJoJD4vVbc= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a81:13d7:0:b0:31c:c22b:4727 with SMTP id 206-20020a8113d7000000b0031cc22b4727mr19236561ywt.38.1657926024502; Fri, 15 Jul 2022 16:00:24 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 15 Jul 2022 23:00:15 +0000 In-Reply-To: <20220715230016.3762909-1-seanjc@google.com> Message-Id: <20220715230016.3762909-4-seanjc@google.com> Mime-Version: 1.0 References: <20220715230016.3762909-1-seanjc@google.com> X-Mailer: git-send-email 2.37.0.170.g444d1eabd0-goog Subject: [PATCH 3/4] KVM: x86/mmu: Add shadow mask for effective host MTRR memtype From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add shadow_memtype_mask to capture that EPT needs a non-zero memtype mask instead of relying on TDP being enabled, as NPT doesn't need a non-zero mask. This is a glorified nop as kvm_x86_ops.get_mt_mask() returns zero for NPT anyways. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/mmu/spte.c | 21 ++++++++++++++++++--- arch/x86/kvm/mmu/spte.h | 1 + 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index fb1f17504138..7314d27d57a4 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -33,6 +33,7 @@ u64 __read_mostly shadow_mmio_value; u64 __read_mostly shadow_mmio_mask; u64 __read_mostly shadow_mmio_access_mask; u64 __read_mostly shadow_present_mask; +u64 __read_mostly shadow_memtype_mask; u64 __read_mostly shadow_me_value; u64 __read_mostly shadow_me_mask; u64 __read_mostly shadow_acc_track_mask; @@ -161,10 +162,10 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, if (level > PG_LEVEL_4K) spte |= PT_PAGE_SIZE_MASK; - if (tdp_enabled) + + if (shadow_memtype_mask) spte |= static_call(kvm_x86_get_mt_mask)(vcpu, gfn, - kvm_is_mmio_pfn(pfn)); - + kvm_is_mmio_pfn(pfn)); if (host_writable) spte |= shadow_host_writable_mask; else @@ -391,6 +392,13 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only) shadow_nx_mask = 0ull; shadow_x_mask = VMX_EPT_EXECUTABLE_MASK; shadow_present_mask = has_exec_only ? 0ull : VMX_EPT_READABLE_MASK; + /* + * EPT overrides the host MTRRs, and so KVM must program the desired + * memtype directly into the SPTEs. Note, this mask is just the mask + * of all bits that factor into the memtype, the actual memtype must be + * dynamically calculated, e.g. to ensure host MMIO is mapped UC. + */ + shadow_memtype_mask = VMX_EPT_MT_MASK | VMX_EPT_IPAT_BIT; shadow_acc_track_mask = VMX_EPT_RWX_MASK; shadow_host_writable_mask = EPT_SPTE_HOST_WRITABLE; shadow_mmu_writable_mask = EPT_SPTE_MMU_WRITABLE; @@ -441,6 +449,13 @@ void kvm_mmu_reset_all_pte_masks(void) shadow_nx_mask = PT64_NX_MASK; shadow_x_mask = 0; shadow_present_mask = PT_PRESENT_MASK; + + /* + * For shadow paging and NPT, KVM uses PAT entry '0' to encode WB + * memtype in the SPTEs, i.e. relies on host MTRRs to provide the + * correct memtype (WB is the "weakest" memtype). + */ + shadow_memtype_mask = 0; shadow_acc_track_mask = 0; shadow_me_mask = 0; shadow_me_value = 0; diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index ba3dccb202bc..cabe3fbb4f39 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -147,6 +147,7 @@ extern u64 __read_mostly shadow_mmio_value; extern u64 __read_mostly shadow_mmio_mask; extern u64 __read_mostly shadow_mmio_access_mask; extern u64 __read_mostly shadow_present_mask; +extern u64 __read_mostly shadow_memtype_mask; extern u64 __read_mostly shadow_me_value; extern u64 __read_mostly shadow_me_mask; From patchwork Fri Jul 15 23:00:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12919900 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A692C433EF for ; Fri, 15 Jul 2022 23:00:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231425AbiGOXAz (ORCPT ); Fri, 15 Jul 2022 19:00:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59800 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230350AbiGOXA2 (ORCPT ); Fri, 15 Jul 2022 19:00:28 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6AFF44E858 for ; Fri, 15 Jul 2022 16:00:27 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id i15-20020a17090a2a0f00b001ef826b921dso3604339pjd.5 for ; Fri, 15 Jul 2022 16:00:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=qyJeBdXwF5dNNWMBDELyZ0rPk0zOo8cNBsAR84NI3Pc=; b=s9nFvsxyVGJKS1GkkPlQocaGQPmzqZ8uonf6tpQY6iPa8ARVd585TI3w+2MAYJChA6 RL8d3MEoa4I/hstCMnfcU73XTiSG9gtZ4upagsnWDVM6B5KevwWwwSPCGqYRrtcSdRnw Foh0IHfWfS2P5NERSQOrvWXjXllKqbvD5mjBoawU6eGxdc194RqjecJKFmBAl8JQuMmA uj6mJb3MhGsGETgB2WJn5DSmZkZv1p8LRakMXhYPd6QzzyvZOyQts7GIuwB6/jKgCdtI JcTfbAOBThkzGrTnkEaSsVcPmzYsTXvyNEvGauh13geubQFqttp8ZimYmHjPdSvIDVDZ x6mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=qyJeBdXwF5dNNWMBDELyZ0rPk0zOo8cNBsAR84NI3Pc=; b=q4ASv5dSaOpdxXWcQ5TEt4LpzgDiVw9XEMv7YpYSy+m0u6XQond+WnFCo6T2asPyAl JckwyjwJuw5MNIwM7PEjuKDaTgFcWBA3o1chQBjQZga94b8ZHRWfGm7LQr2M3dx9vi9l svpzZOnm1pxOpx2SQv7bOS6UT6l4rvf9HxfhMR+I60kCNaJIV38qPw5bvh2lvo+M2oK6 03+WDCgJxBcyFnCbR7qPhu22yp1EQNU9cqYZj7YZ3H9vWx8tXDNKKL2W9OyEneKhUuNp oEYEE2PgGC+LMXdFtV+IzORC5Rb5lRTTK3jet0R5POGDg2L64JWBJxmPZFIvO6YPZsXb jE+Q== X-Gm-Message-State: AJIora/3GvzSiKOLP7YUvIy99TjXymA7jzAFJ7MQ3J4cnd3n5CwTWzvR c7zhssNnRMRn6SC4LXzvMdgP9P55dwU= X-Google-Smtp-Source: AGRyM1sOQH8hQ8hf4xM2w/u1OrR0ibxbHFPEKSoDDrbPxn2n+mONs5jxdLyEZrumJUwzh4RWPB/CyyZIFsk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90b:181:b0:1ef:c348:6835 with SMTP id t1-20020a17090b018100b001efc3486835mr913262pjs.1.1657926026078; Fri, 15 Jul 2022 16:00:26 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 15 Jul 2022 23:00:16 +0000 In-Reply-To: <20220715230016.3762909-1-seanjc@google.com> Message-Id: <20220715230016.3762909-5-seanjc@google.com> Mime-Version: 1.0 References: <20220715230016.3762909-1-seanjc@google.com> X-Mailer: git-send-email 2.37.0.170.g444d1eabd0-goog Subject: [PATCH 4/4] KVM: x86/mmu: Restrict mapping level based on guest MTRR iff they're used From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Restrict the mapping level for SPTEs based on the guest MTRRs if and only if KVM may actually use the guest MTRRs to compute the "real" memtype. For all forms of paging, guest MTRRs are purely virtual in the sense that they are completely ignored by hardware, i.e. they affect the memtype only if software manually consumes them. The only scenario where KVM consumes the guest MTRRs is when shadow_memtype_mask is non-zero and the guest has non-coherent DMA, in all other cases KVM simply leaves the PAT field in SPTEs as '0' to encode WB memtype. Note, KVM may still ultimately ignore guest MTRRs, e.g. if the backing pfn is host MMIO, but false positives are ok as they only cause a slight performance blip (unless the guest is doing weird things with its MTRRs, which is extremely unlikely). Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/mmu/mmu.c | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 52664c3caaab..82f38af06f5c 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4295,14 +4295,26 @@ EXPORT_SYMBOL_GPL(kvm_handle_page_fault); int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { - while (fault->max_level > PG_LEVEL_4K) { - int page_num = KVM_PAGES_PER_HPAGE(fault->max_level); - gfn_t base = (fault->addr >> PAGE_SHIFT) & ~(page_num - 1); + /* + * If the guest's MTRRs may be used to compute the "real" memtype, + * restrict the mapping level to ensure KVM uses a consistent memtype + * across the entire mapping. If the host MTRRs are ignored by TDP + * (shadow_memtype_mask is non-zero), and the VM has non-coherent DMA + * (DMA doesn't snoop CPU caches), KVM's ABI is to honor the memtype + * from the guest's MTRRs so that guest accesses to memory that is + * DMA'd aren't cached against the guest's wishes. + * + * Note, KVM may still ultimately ignore guest MTRRs for certain PFNs, + * e.g. KVM will force UC memtype for host MMIO. + */ + if (shadow_memtype_mask && kvm_arch_has_noncoherent_dma(vcpu->kvm)) { + for ( ; fault->max_level > PG_LEVEL_4K; --fault->max_level) { + int page_num = KVM_PAGES_PER_HPAGE(fault->max_level); + gfn_t base = (fault->addr >> PAGE_SHIFT) & ~(page_num - 1); - if (kvm_mtrr_check_gfn_range_consistency(vcpu, base, page_num)) - break; - - --fault->max_level; + if (kvm_mtrr_check_gfn_range_consistency(vcpu, base, page_num)) + break; + } } return direct_page_fault(vcpu, fault);