From patchwork Tue May 4 17:17:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238341 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12FC9C433ED for ; Tue, 4 May 2021 17:17:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EB615613BC for ; Tue, 4 May 2021 17:17:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232082AbhEDRSn (ORCPT ); Tue, 4 May 2021 13:18:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39548 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232048AbhEDRSk (ORCPT ); Tue, 4 May 2021 13:18:40 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 88C7FC061574 for ; Tue, 4 May 2021 10:17:45 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id n129-20020a2527870000b02904ed02e1aab5so12566066ybn.21 for ; Tue, 04 May 2021 10:17:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=GQeYPNBasLogl534ND9Q7MqGb65bHRch8YH0hNcQgdE=; b=Ojd4yjrIuWqLjcbV/Te/47XOZv/fcPnbpUUqoPh30f3y+iIiXxDPItFNyUrNyDYDh9 0iNgVWksjbSECQHZUKcdEhJPFncNrqDVrdGJwBXHUia2e34lFnP5Ip4hsPAmdJvd+RA+ 2i9yloFDGIZVo10i5Pp3hn/Z7G8jh/W6zNmh3+bco+SFOrOYqegQ3BQirXlgUSWXaW+X KitiQ0UW+swuGRkFbepxHXx2yzzXEaGvPsnRuUKF/cZqVWvSVD+ezisiIXbDPLDLVDjB r0t5vcEHZhRSGqZJja1qf7r11kzkptlCj6VgLav0iBk8LHrDVhmu54dicOvxlqKCstL5 /Vfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=GQeYPNBasLogl534ND9Q7MqGb65bHRch8YH0hNcQgdE=; b=JdQk4Oz1itdNfI36NJXoYVWRvL2HywpUjKaipFHdWvGnwFIfTnJztnI/jjeAgYlGNE cSRSqcnElxcfp+Z3MU5wonRh1MXf6StIsyU0qmFmgjkyU7mtAg4YRo4Tvd5LCxDz0D+R e1s7LbdFhqIbaKopwdaEeOq2lJEC/7HfJf26eqWEdFFoN7qZ4R/6PRBGvdjQ26gZpNBD xXtz38n6DoUt7M4OiRhfrLcetZ8ycHMGJPvt16bTAWTlZFNnrACImKhn7t+YMjSJnVC3 iW+VCfA2+hAiWYwDynFBQPg8GoDXvlNNICATmolrROl04sh8yQU8m3LLaR7mlyEXVo5Y HH0w== X-Gm-Message-State: AOAM53153Q0RWdWOVfCQF6V17uwB4sFaSUEIao+P5iQX3t8pZyJsxMGL WfFXLMvGPDih92xwnWKUrPKAklddk0Q= X-Google-Smtp-Source: ABdhPJxlSOlcX5fvKMS9Z8Y/32xdFNS3BueN6Hhz6HUJ3ejXJluoaat8wBgsB528Ps/FO0WpSewliWx1IYg= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a5b:c52:: with SMTP id d18mr34112608ybr.401.1620148664836; Tue, 04 May 2021 10:17:44 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:20 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-2-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 01/15] KVM: VMX: Do not adverise RDPID if ENABLE_RDTSCP control is unsupported From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Clear KVM's RDPID capability if the ENABLE_RDTSCP secondary exec control is unsupported. Despite being enumerated in a separate CPUID flag, RDPID is bundled under the same VMCS control as RDTSCP and will #UD in VMX non-root if ENABLE_RDTSCP is not enabled. Fixes: 41cd02c6f7f6 ("kvm: x86: Expose RDPID in KVM_GET_SUPPORTED_CPUID") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson Reviewed-by: Jim Mattson Reviewed-by: Reiji Watanabe Reviewed-by: Maxim Levitsky --- arch/x86/kvm/vmx/vmx.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 10b610fc7bbc..82404ee2520e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7377,9 +7377,11 @@ static __init void vmx_set_cpu_caps(void) if (!cpu_has_vmx_xsaves()) kvm_cpu_cap_clear(X86_FEATURE_XSAVES); - /* CPUID 0x80000001 */ - if (!cpu_has_vmx_rdtscp()) + /* CPUID 0x80000001 and 0x7 (RDPID) */ + if (!cpu_has_vmx_rdtscp()) { kvm_cpu_cap_clear(X86_FEATURE_RDTSCP); + kvm_cpu_cap_clear(X86_FEATURE_RDPID); + } if (cpu_has_vmx_waitpkg()) kvm_cpu_cap_check_and_set(X86_FEATURE_WAITPKG); From patchwork Tue May 4 17:17:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238343 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0332CC43470 for ; Tue, 4 May 2021 17:17:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D8CA5613C9 for ; Tue, 4 May 2021 17:17:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232102AbhEDRSq (ORCPT ); Tue, 4 May 2021 13:18:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232080AbhEDRSn (ORCPT ); Tue, 4 May 2021 13:18:43 -0400 Received: from mail-qt1-x84a.google.com (mail-qt1-x84a.google.com [IPv6:2607:f8b0:4864:20::84a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA722C06138B for ; Tue, 4 May 2021 10:17:47 -0700 (PDT) Received: by mail-qt1-x84a.google.com with SMTP id r20-20020ac85c940000b02901bac34fa2eeso4007617qta.11 for ; Tue, 04 May 2021 10:17:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=zyk/CnPUPX4IZt1E/UCYCEZ+CYodKlkrWWKieBP0u40=; b=sXmep3CNmV6ncUmh4Bwt9ZAeJiSDP0JoBEhefkkGoDExZ3KWPbvsIU53JnIsEc8zfT 2CLrK3Ev6uYm//ZTV8x3LBbl0pu76DCNtR4bhbBmeIoP1Yxz394XylT340mUTD2ikJIm +1cv0jsiXrLQd9DYbrRc3EM3FU169gR0ontu9WZBcWU/oftdv3RNOpv39sWk79rVQmhu B/zjoWAHBa3MpJXy4gwirrxumf1PQnWBGtr/HgGs/qyv9/i2JD56atdndo7MZm/gbzY5 MFZ6Abd/WzdR+F5CHGnUmZQzWy1xnjK6u/Qywe7tLexPOd7WjMK9OLibvAMFXhgOgPAB HnXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=zyk/CnPUPX4IZt1E/UCYCEZ+CYodKlkrWWKieBP0u40=; b=FsKiS/E1q9zzYEGLRhSBVg7fBTEPvrYZjYWMqpA7BrbtdaJOZJrSpWFKB1tQvB/C5o ts92jBAPl+BDFA7BPmEmxYUJ43LF5Oe3E+OTfWtIhkE0cIE3Mp50qBLE00N0DMn2a+1h QItdiBj5mgtxO0SySgU+iT+wu6361kqD7GkqPbOXtRWN+MytAGSK7tZWXLaGdKq5Ge3e knTGRrMpkV0/hO9S1QyBcy+8HvX5rqAvShxFkbZoGjjSZIFZsEO9vJ61B6dROilRA+ef NyfCzWr5PV8YL3BlLwvuvunJpT2u/gdqTZi3I/ensuVJ4AapAFX5/OcJQmT+Ot5+yRhf A1uw== X-Gm-Message-State: AOAM530xmdgSz3YjSEmRn3IjkiZ9Rgico+mzgCxF2Vy7D/HfFpWDjdHh clUuM3kN7sfLkkzQsmKCGYJKGYLL9Pc= X-Google-Smtp-Source: ABdhPJxwWT2eyWq3SHAnHcamuoyl+SuIC6Dyz9Xd5AV959ifFtMmBZB15+K8e6dS33p4FmM7eJNyadnHMuA= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a05:6214:c2d:: with SMTP id a13mr18106571qvd.37.1620148666938; Tue, 04 May 2021 10:17:46 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:21 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-3-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 02/15] KVM: x86: Emulate RDPID only if RDTSCP is supported From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Do not advertise emulation support for RDPID if RDTSCP is unsupported. RDPID emulation subtly relies on MSR_TSC_AUX to exist in hardware, as both vmx_get_msr() and svm_get_msr() will return an error if the MSR is unsupported, i.e. ctxt->ops->get_msr() will fail and the emulator will inject a #UD. Note, RDPID emulation also relies on RDTSCP being enabled in the guest, but this is a KVM bug and will eventually be fixed. Fixes: fb6d4d340e05 ("KVM: x86: emulate RDPID") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/cpuid.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index f765bf7a529c..c96f79c9fff2 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -637,7 +637,8 @@ static int __do_cpuid_func_emulated(struct kvm_cpuid_array *array, u32 func) case 7: entry->flags |= KVM_CPUID_FLAG_SIGNIFCANT_INDEX; entry->eax = 0; - entry->ecx = F(RDPID); + if (kvm_cpu_cap_has(X86_FEATURE_RDTSCP)) + entry->ecx = F(RDPID); ++array->nent; default: break; From patchwork Tue May 4 17:17:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D30BC433ED for ; Tue, 4 May 2021 17:17:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 83BF3613C6 for ; Tue, 4 May 2021 17:17:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232113AbhEDRSr (ORCPT ); Tue, 4 May 2021 13:18:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39580 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232064AbhEDRSp (ORCPT ); Tue, 4 May 2021 13:18:45 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2DBB2C06174A for ; Tue, 4 May 2021 10:17:50 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id v12-20020a25848c0000b02904f30b36aebfso12767676ybk.1 for ; Tue, 04 May 2021 10:17:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=j0hBscA0403vKyQ6SIWuUyRenydWaHr9yEWTv+u3OUs=; b=ViGRM4AJOU518/+fjRX9OlnzW/83aCiR1KgihpFq073sjX7H1DBYYHGZqylYb5Y7Q2 Uy/gi/efm0nM4Dc+A9bK5CQp8VswRZeS/3eOIoB/umJuYp/pxlaKPQ0qUPtfPdg4BPyu VR/9ZeJTs0uJ2ZYBElFK2p5be4z5mcspDhD+eai1dmzNGh9xlgZzEP6q0lCUmX4jfrGN CbOUQlh9p82oQfrxYXtz7dtIOi4BMLqhiECO5oENxRneRyHKxSCkyZeGHq0AGq3YXHKB 9rybb4wVz7MsGHzg+JHNFfgmol3G3HWFHN7OE/ioX13Q5FuXARrTtE6NxnlmRGOlEVvb qycw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=j0hBscA0403vKyQ6SIWuUyRenydWaHr9yEWTv+u3OUs=; b=bLX1M/NpDSero0m1xIillBqTcmZoQwV9EA4s44CCH/Ly119mjbe2QQFou414u6k5Pw +1r6f1Z6OnBgqQy04f/5M8wIsWtZPF1oCBVUwc/V9RxDup0Mu18HJcB+N4AuVJ9zz0AP lakehwCmWOQWJmsatYoY0XMi21swTlNySR43g/Qq2PS9wJhsAY3YKeWP1Tyq5FuQENEV aDz8/ibh3HBUnh1jgoIu0Sb2UhQDLdhv5BIO/FzARvAzQOiyuH7LeSsJXBue2AYM0vud 65shlwFJYY8FWTH66z83/ftfTAFABWXpOAjzJxz9mlpf/UbcXZBMn8q4hViBeFO3J5Kt 0/tw== X-Gm-Message-State: AOAM5315d7rsENiwYUSA3TVw6eoobyngSviRwKb7XvXeZb0dkTQkeTSb kygi2W9yO26pcNSVqqoKed6txmN4Mm8= X-Google-Smtp-Source: ABdhPJwry1A6z/xjHiq7OfswGhOeDlb9iJLLC5CJCuD7yr/gYHfb7E3S+/lNHAndjUQOSs8wYttrWfkEfes= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a05:6902:1349:: with SMTP id g9mr18387086ybu.30.1620148669369; Tue, 04 May 2021 10:17:49 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:22 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-4-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 03/15] KVM: SVM: Inject #UD on RDTSCP when it should be disabled in the guest From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Intercept RDTSCP to inject #UD if RDTSC is disabled in the guest. Note, SVM does not support intercepting RDPID. Unlike VMX's ENABLE_RDTSCP control, RDTSCP interception does not apply to RDPID. This is a benign virtualization hole as the host kernel (incorrectly) sets MSR_TSC_AUX if RDTSCP is supported, and KVM loads the guest's MSR_TSC_AUX into hardware if RDTSCP is supported in the host, i.e. KVM will not leak the host's MSR_TSC_AUX to the guest. But, when the kernel bug is fixed, KVM will start leaking the host's MSR_TSC_AUX if RDPID is supported in hardware, but RDTSCP isn't available for whatever reason. This leak will be remedied in a future commit. Fixes: 46896c73c1a4 ("KVM: svm: add support for RDTSCP") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson Reviewed-by: Jim Mattson Reviewed-by: Reiji Watanabe Reviewed-by: Maxim Levitsky --- arch/x86/kvm/svm/svm.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index a7271f31df47..8f2b184270c0 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -1100,7 +1100,9 @@ static u64 svm_write_l1_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) return svm->vmcb->control.tsc_offset; } -static void svm_check_invpcid(struct vcpu_svm *svm) +/* Evaluate instruction intercepts that depend on guest CPUID features. */ +static void svm_recalc_instruction_intercepts(struct kvm_vcpu *vcpu, + struct vcpu_svm *svm) { /* * Intercept INVPCID if shadow paging is enabled to sync/free shadow @@ -1113,6 +1115,13 @@ static void svm_check_invpcid(struct vcpu_svm *svm) else svm_clr_intercept(svm, INTERCEPT_INVPCID); } + + if (kvm_cpu_cap_has(X86_FEATURE_RDTSCP)) { + if (guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP)) + svm_clr_intercept(svm, INTERCEPT_RDTSCP); + else + svm_set_intercept(svm, INTERCEPT_RDTSCP); + } } static void init_vmcb(struct kvm_vcpu *vcpu) @@ -1248,7 +1257,7 @@ static void init_vmcb(struct kvm_vcpu *vcpu) svm_clr_intercept(svm, INTERCEPT_PAUSE); } - svm_check_invpcid(svm); + svm_recalc_instruction_intercepts(vcpu, svm); /* * If the host supports V_SPEC_CTRL then disable the interception @@ -3084,6 +3093,7 @@ static int (*const svm_exit_handlers[])(struct kvm_vcpu *vcpu) = { [SVM_EXIT_STGI] = stgi_interception, [SVM_EXIT_CLGI] = clgi_interception, [SVM_EXIT_SKINIT] = skinit_interception, + [SVM_EXIT_RDTSCP] = kvm_handle_invalid_op, [SVM_EXIT_WBINVD] = kvm_emulate_wbinvd, [SVM_EXIT_MONITOR] = kvm_emulate_monitor, [SVM_EXIT_MWAIT] = kvm_emulate_mwait, @@ -4007,8 +4017,7 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) svm->nrips_enabled = kvm_cpu_cap_has(X86_FEATURE_NRIPS) && guest_cpuid_has(vcpu, X86_FEATURE_NRIPS); - /* Check again if INVPCID interception if required */ - svm_check_invpcid(svm); + svm_recalc_instruction_intercepts(vcpu, svm); /* For sev guests, the memory encryption bit is not reserved in CR3. */ if (sev_guest(vcpu->kvm)) { From patchwork Tue May 4 17:17:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62C2DC43460 for ; Tue, 4 May 2021 17:17:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4ABEE613AA for ; Tue, 4 May 2021 17:17:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232136AbhEDRSu (ORCPT ); Tue, 4 May 2021 13:18:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39594 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230463AbhEDRSs (ORCPT ); Tue, 4 May 2021 13:18:48 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 696A8C06174A for ; Tue, 4 May 2021 10:17:52 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id m68-20020a6326470000b029020f37ad2901so5674695pgm.7 for ; Tue, 04 May 2021 10:17:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=x9ju8Y95QmZw6Siui/FK/R18zMsSmb8w0Q5nDcsgnqg=; b=v/A3HzwLp+vZV56pMiyGse3+Y7Ns38lqueNgFzl39BqNJ8/nNkwrQwLE7jj0HL6n3b bTksTsCS/rwy4KFIpqe8pt2TgSLjkyG/8jr8S2+csy2GFj3kf+Vk+b2KJ26ri0osQnAE 7zOM779YwH3NA+nudqbsxSiBW7gUOdYB7+pZA91l7wUgytk0IAsvHC0yI4FxxC1QaTRT EjKP0xq8zK8bmp7mNLxzwGD66AvagO8ZGcuyxa2AVrDT1g0uouUrsyQULAtR4oy9IMWQ hvXGsaW34984PFebHKAzJblE8fgtliDC98prZJTZwDWp655aymPS2gaOd3APbn2NyhSc m39w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=x9ju8Y95QmZw6Siui/FK/R18zMsSmb8w0Q5nDcsgnqg=; b=oXO6DKgpQ7xYNtdlbUH8BkzZEIXnC1bdt8Zf1JeWgo8HIvuRjukA2WTi/S13zR0VMQ eN+1Wd6wwEpdcdpCiQqo6ViMSOi50PTE+f2c9XOu40bVsygYe9a6fbjaCJck2X0rAJUj YXqKypnb8Y/ajK175xw76p3D2RKI55gpmh2SFles/Zr4DsF5vCjycF3L3WN9tOWvo+2N 97lrmWPo7WwLmztLRkv5XDzEQCj/JDevfhNAWJpsEn6URbjDMHgvCd0IbR8XGiYeqiV7 YvE7jfFFBbyfrmxZ7HJA6saIsdairB9LfELw3rkxFxAuON0gW545mbpvUzOghV2G2Mzp YS1A== X-Gm-Message-State: AOAM530oSGqOoweihCPaT+6QbJsmzWuoBuJuIVw6qq9zvyssq3pt1PhG U1R9QX7ILNjGEYWoq/1/eZ5mYEqdmr8= X-Google-Smtp-Source: ABdhPJyMfi4QaGrLYdrW6ssUNtQ+SG89NJapbP1MqWBMxR/nCoes9dndo5SZcIvi2xFF+IwubJiTNFdt5nk= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a17:90a:f690:: with SMTP id cl16mr29862523pjb.207.1620148671954; Tue, 04 May 2021 10:17:51 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:23 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-5-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 04/15] KVM: x86: Move RDPID emulation intercept to its own enum From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a dedicated intercept enum for RDPID instead of piggybacking RDTSCP. Unlike VMX's ENABLE_RDTSCP, RDPID is not bound to SVM's RDTSCP intercept. Fixes: fb6d4d340e05 ("KVM: x86: emulate RDPID") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson Reviewed-by: Jim Mattson --- arch/x86/kvm/emulate.c | 2 +- arch/x86/kvm/kvm_emulate.h | 1 + arch/x86/kvm/vmx/vmx.c | 3 ++- 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c index abd9a4db11a8..8fc71e70857d 100644 --- a/arch/x86/kvm/emulate.c +++ b/arch/x86/kvm/emulate.c @@ -4502,7 +4502,7 @@ static const struct opcode group8[] = { * from the register case of group9. */ static const struct gprefix pfx_0f_c7_7 = { - N, N, N, II(DstMem | ModRM | Op3264 | EmulateOnUD, em_rdpid, rdtscp), + N, N, N, II(DstMem | ModRM | Op3264 | EmulateOnUD, em_rdpid, rdpid), }; diff --git a/arch/x86/kvm/kvm_emulate.h b/arch/x86/kvm/kvm_emulate.h index 0d359115429a..f016838faedd 100644 --- a/arch/x86/kvm/kvm_emulate.h +++ b/arch/x86/kvm/kvm_emulate.h @@ -468,6 +468,7 @@ enum x86_intercept { x86_intercept_clgi, x86_intercept_skinit, x86_intercept_rdtscp, + x86_intercept_rdpid, x86_intercept_icebp, x86_intercept_wbinvd, x86_intercept_monitor, diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 82404ee2520e..99591e523b47 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7437,8 +7437,9 @@ static int vmx_check_intercept(struct kvm_vcpu *vcpu, /* * RDPID causes #UD if disabled through secondary execution controls. * Because it is marked as EmulateOnUD, we need to intercept it here. + * Note, RDPID is hidden behind ENABLE_RDTSCP. */ - case x86_intercept_rdtscp: + case x86_intercept_rdpid: if (!nested_cpu_has2(vmcs12, SECONDARY_EXEC_ENABLE_RDTSCP)) { exception->vector = UD_VECTOR; exception->error_code_valid = false; From patchwork Tue May 4 17:17:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0010EC43460 for ; Tue, 4 May 2021 17:18:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CFC8E613C5 for ; Tue, 4 May 2021 17:18:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232169AbhEDRSy (ORCPT ); Tue, 4 May 2021 13:18:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232139AbhEDRSu (ORCPT ); Tue, 4 May 2021 13:18:50 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 372AFC061574 for ; Tue, 4 May 2021 10:17:55 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id o186-20020a2528c30000b02904f824478356so2062193ybo.4 for ; Tue, 04 May 2021 10:17:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=Mu+gEWNFD64rfAygjoRzLbXHjILdR4aSiYoIxX4BVaU=; b=vc6WfaDNhGXnxvzcIjsY5cNjkGyx7dmFjddNsCJRbyGdRPrl3THfl2KHLG8BUXv01Z NS/TtAghiP1EJwfN0gWaU4pT2Wyzko5S4/KXldiq7EKJKzM1g3b7qyn+DqsHSWtAmfss 09AW1NzRufPSnjBL5eVsky5IxIr6t8562okUKZI1ldnSsZjp0WbGwzTO9iiJR01LmkQU dg9HlBwkuIB9I4JwcOWvTOnx1pDLH1/gRTMml5QGa8ZxxPvRKtFUB7ETic0tP8HKinKa Tgdat3v7IKJ7gr95sLIXPPIatl3gAVBLSALMyuUfBYgGRRxZpyv4K5hbLcp0+cuztjid V25g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=Mu+gEWNFD64rfAygjoRzLbXHjILdR4aSiYoIxX4BVaU=; b=bttYvU+5OIkr5ZYJEFBvuiTluh7k952CZPMxwMoWm/qBnYNVp6mDRTBOw8FqX2fHLK LRAN54cMg3Ihwa5BRTrs9n67ZUx5+Ku6FtzEfFYHVDfn/FtE6pIXXf3So/rvsbpuj8dr Ej0seRkfvOwQj145foUaClcVhhBsJxY15hBU4cgLSrEhVIQ/Z+GuEm+exb5BVmy0BRng nMsUmCNuueESuKS4iggW1fcq3Op1WU2J35lLCZ0DNQ6UqXWQGv9/BfgGMLqAjKYKYrok 4A9BRRLauIUC8UV/8WqJVwRiHyq/XF8DXCeDkNhD6sZIbRE0/36qKTcGiPXZw1dHekOV hINQ== X-Gm-Message-State: AOAM531ZCB5ect6jO1jKc9yuCfuK4BTjUE5AJYj2CX80KfDnRuRemvpr 3uTN952Ld0gc1KDGhyPShIQ/9ryreqM= X-Google-Smtp-Source: ABdhPJwlIQBBBVZtWF2iJlUZEqIfnqJVgYceDZL5CJzNPL1GoAgGUfkgW/iYbkTyzo7VpPPUFM1H019rjwk= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a25:e6d4:: with SMTP id d203mr4165572ybh.226.1620148674458; Tue, 04 May 2021 10:17:54 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:24 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-6-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 05/15] KVM: VMX: Disable preemption when probing user return MSRs From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Disable preemption when probing a user return MSR via RDSMR/WRMSR. If the MSR holds a different value per logical CPU, the WRMSR could corrupt the host's value if KVM is preempted between the RDMSR and WRMSR, and then rescheduled on a different CPU. Opportunistically land the helper in common x86, SVM will use the helper in a future commit. Fixes: 4be534102624 ("KVM: VMX: Initialize vmx->guest_msrs[] right after allocation") Cc: stable@vger.kernel.org Cc: Xiaoyao Li Signed-off-by: Sean Christopherson Reviewed-by: Jim Mattson Reviewed-by: Maxim Levitsky --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/vmx.c | 5 +---- arch/x86/kvm/x86.c | 16 ++++++++++++++++ 3 files changed, 18 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 3e5fc80a35c8..a02c9bf3f7f1 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1778,6 +1778,7 @@ int kvm_pv_send_ipi(struct kvm *kvm, unsigned long ipi_bitmap_low, unsigned long icr, int op_64_bit); void kvm_define_user_return_msr(unsigned index, u32 msr); +int kvm_probe_user_return_msr(u32 msr); int kvm_set_user_return_msr(unsigned index, u64 val, u64 mask); u64 kvm_scale_tsc(struct kvm_vcpu *vcpu, u64 tsc); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 99591e523b47..990ee339a05f 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6914,12 +6914,9 @@ static int vmx_create_vcpu(struct kvm_vcpu *vcpu) for (i = 0; i < ARRAY_SIZE(vmx_uret_msrs_list); ++i) { u32 index = vmx_uret_msrs_list[i]; - u32 data_low, data_high; int j = vmx->nr_uret_msrs; - if (rdmsr_safe(index, &data_low, &data_high) < 0) - continue; - if (wrmsr_safe(index, data_low, data_high) < 0) + if (kvm_probe_user_return_msr(index)) continue; vmx->guest_uret_msrs[j].slot = i; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 3bf52ba5f2bb..e304447be42d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -339,6 +339,22 @@ static void kvm_on_user_return(struct user_return_notifier *urn) } } +int kvm_probe_user_return_msr(u32 msr) +{ + u64 val; + int ret; + + preempt_disable(); + ret = rdmsrl_safe(msr, &val); + if (ret) + goto out; + ret = wrmsrl_safe(msr, val); +out: + preempt_enable(); + return ret; +} +EXPORT_SYMBOL_GPL(kvm_probe_user_return_msr); + void kvm_define_user_return_msr(unsigned slot, u32 msr) { BUG_ON(slot >= KVM_MAX_NR_USER_RETURN_MSRS); From patchwork Tue May 4 17:17:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF177C43462 for ; Tue, 4 May 2021 17:18:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AC86B613D0 for ; Tue, 4 May 2021 17:18:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231824AbhEDRS4 (ORCPT ); Tue, 4 May 2021 13:18:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39630 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232151AbhEDRSw (ORCPT ); Tue, 4 May 2021 13:18:52 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F2B4C061763 for ; Tue, 4 May 2021 10:17:57 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id j63-20020a25d2420000b02904d9818b80e8so8157430ybg.14 for ; Tue, 04 May 2021 10:17:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=x+NCaDzYaJB+NMadtQ9JUPGNk0j6WFjhbI1X+ndrCYA=; b=GYxeu56pESpHnymBBR37hcoL+erzIi1Cs2Fcie4S/JXxyz05HzWd8tfYRy2rUHxVD6 eJtObUcSxC1FczqzDMUZNwHyfxJ21cyb6kmOP30GXeH7chJrTDIyg1/pQ8A4ErT237jt 2e2nygFpJqozYZiDrgVQKjorHBF3s7zyrtYOS40HX12yiEZsf7gbjcIrrnB8RRb2Sb/1 IreqbcFdIOSmU8aPz9ZhCeTCCRo0nmbb1RMCNevhOVOLeKFsE+rMuzbbGGOlpMIq7REm Vdn5utrZ4eXb4F5kpCx95zET+f3Sbv23kgnex9h7n5Y4dcZuPqjKfCZY9I6OvmHqqdb1 /eGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=x+NCaDzYaJB+NMadtQ9JUPGNk0j6WFjhbI1X+ndrCYA=; b=k5XFKFU/ZTrrbMah7PCkiVaLxbTVF5WIAsC2pssuFhilTv4KYcKF6tmrOxlp9o3QCR DOS9llTncOM+rVo9URHS1J6hp0kYyjeebnmDLt6yePAAS1n8k0Uk3aR3SDVUi3nXDKuO CerPfSrW/1saF0xQtQf/DdWNLLDGmPbmMzhiWfeUT7wn3dLvUOXb57SAMZy4H46/nBjq XDvp4lhhhGur0WsEblRKubu0g/vQ4RAVJI37dwpxbgtXCPytR6mLkkNdFv3Lv1nEQfmd XlG6ha/sbhT+WEcUBxks1cN4KOmTAeF0REdzXtiKV6hAu3M/o1nFWfs32xeUW4X0TRNs fvug== X-Gm-Message-State: AOAM5320uBrgXvw2yrb8xGjvUY4WFAJC0WCb7QX/rNJnahJDYL0i8a/v C10kUIsnSPN9spdfDgMbpAIAAup6bhY= X-Google-Smtp-Source: ABdhPJx6D43kn5MV0I4+HM7Y77hYMTE0j8mAoTlwNGlkhP8mTP6+0FnyQn2hkMtruWcNfQViYps2vJMB3uo= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a25:3103:: with SMTP id x3mr34207809ybx.8.1620148676838; Tue, 04 May 2021 10:17:56 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:25 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-7-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 06/15] KVM: SVM: Probe and load MSR_TSC_AUX regardless of RDTSCP support in host From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Probe MSR_TSC_AUX whether or not RDTSCP is supported in the host, and if probing succeeds, load the guest's MSR_TSC_AUX into hardware prior to VMRUN. Because SVM doesn't support interception of RDPID, RDPID cannot be disallowed in the guest (without resorting to binary translation). Leaving the host's MSR_TSC_AUX in hardware would leak the host's value to the guest if RDTSCP is not supported. Note, there is also a kernel bug that prevents leaking the host's value. The host kernel initializes MSR_TSC_AUX if and only if RDTSCP is supported, even though the vDSO usage consumes MSR_TSC_AUX via RDPID. I.e. if RDTSCP is not supported, there is no host value to leak. But, if/when the host kernel bug is fixed, KVM would start leaking MSR_TSC_AUX in the case where hardware supports RDPID but RDTSCP is unavailable for whatever reason. Probing MSR_TSC_AUX will also allow consolidating the probe and define logic in common x86, and will make it simpler to condition the existence of MSR_TSX_AUX (from the guest's perspective) on RDTSCP *or* RDPID. Fixes: AMD CPUs Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/svm/svm.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 8f2b184270c0..b3153d40cc4d 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -212,7 +212,7 @@ DEFINE_PER_CPU(struct svm_cpu_data *, svm_data); * RDTSCP and RDPID are not used in the kernel, specifically to allow KVM to * defer the restoration of TSC_AUX until the CPU returns to userspace. */ -#define TSC_AUX_URET_SLOT 0 +static int tsc_aux_uret_slot __read_mostly = -1; static const u32 msrpm_ranges[] = {0, 0xc0000000, 0xc0010000}; @@ -959,8 +959,10 @@ static __init int svm_hardware_setup(void) kvm_tsc_scaling_ratio_frac_bits = 32; } - if (boot_cpu_has(X86_FEATURE_RDTSCP)) - kvm_define_user_return_msr(TSC_AUX_URET_SLOT, MSR_TSC_AUX); + if (!kvm_probe_user_return_msr(MSR_TSC_AUX)) { + tsc_aux_uret_slot = 0; + kvm_define_user_return_msr(tsc_aux_uret_slot, MSR_TSC_AUX); + } /* Check for pause filtering support */ if (!boot_cpu_has(X86_FEATURE_PAUSEFILTER)) { @@ -1454,8 +1456,8 @@ static void svm_prepare_guest_switch(struct kvm_vcpu *vcpu) } } - if (static_cpu_has(X86_FEATURE_RDTSCP)) - kvm_set_user_return_msr(TSC_AUX_URET_SLOT, svm->tsc_aux, -1ull); + if (likely(tsc_aux_uret_slot >= 0)) + kvm_set_user_return_msr(tsc_aux_uret_slot, svm->tsc_aux, -1ull); svm->guest_state_loaded = true; } @@ -2664,7 +2666,7 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) msr_info->data |= (u64)svm->sysenter_esp_hi << 32; break; case MSR_TSC_AUX: - if (!boot_cpu_has(X86_FEATURE_RDTSCP)) + if (tsc_aux_uret_slot < 0) return 1; if (!msr_info->host_initiated && !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP)) @@ -2885,7 +2887,7 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) svm->sysenter_esp_hi = guest_cpuid_is_intel(vcpu) ? (data >> 32) : 0; break; case MSR_TSC_AUX: - if (!boot_cpu_has(X86_FEATURE_RDTSCP)) + if (tsc_aux_uret_slot < 0) return 1; if (!msr->host_initiated && @@ -2908,7 +2910,7 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) * guest via direct_access_msrs, and switch it via user return. */ preempt_disable(); - r = kvm_set_user_return_msr(TSC_AUX_URET_SLOT, data, -1ull); + r = kvm_set_user_return_msr(tsc_aux_uret_slot, data, -1ull); preempt_enable(); if (r) return 1; From patchwork Tue May 4 17:17:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1BF4C433ED for ; Tue, 4 May 2021 17:18:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 87711613C5 for ; Tue, 4 May 2021 17:18:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232154AbhEDRTD (ORCPT ); Tue, 4 May 2021 13:19:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39634 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232157AbhEDRSz (ORCPT ); Tue, 4 May 2021 13:18:55 -0400 Received: from mail-qt1-x84a.google.com (mail-qt1-x84a.google.com [IPv6:2607:f8b0:4864:20::84a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F448C06174A for ; Tue, 4 May 2021 10:18:00 -0700 (PDT) Received: by mail-qt1-x84a.google.com with SMTP id d16-20020ac811900000b02901bbebf64663so4019459qtj.14 for ; Tue, 04 May 2021 10:18:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=BGCQSILYAU01VvgA8TaO/AwM6Dcj3M5kpkKiJ9GwIVo=; b=VoB1ZQZUQTZDbAWIb7z6YLsP1EcMmTZaJ21TOhoo/VBB8DHbA/WYo7YmRUtOh+XWCL JHH+uUsd40UADuqDaFksTVfYQXRNeuI4xpnKVKdCjrsCWjSuUczwY6PkZMyaLmFMJt4p r9AXFSCo3QBnt2KcWe/W6aHmzAKDQl/I3vDLGVJT/HxrAwj5sgyIP2ptWLQWgXHQGU3E IZX8NGOJp4EBjwcMDGh964qDz/CTANPH1FckHRPrhLRkchsqqXBoWKAhbisk7iqXuUFC NtGAe0bVUIP024/Lo487nX7s+aMm0dtH3OVvl7VW6NKxgP/fNmzSOLn+VYLYU1ZxX+Cf fVjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=BGCQSILYAU01VvgA8TaO/AwM6Dcj3M5kpkKiJ9GwIVo=; b=ScDrpHTKDtgULD0O1TlfFVOSJ8rkflITNwa4mzBZkjc1jBY+OZQ9SNd3MBJYHFibty 8pMThdQxricLw7Sxj9cOpZGDGwu9gwdxWzexHmPVTR4v2QjWR5/S6WbWdPzUnZ2zw5dT QRDgCKO0Fo1GXn2R5BtiLAi+ODqq95M/PmSoKVUeip7jnn18vc1vGwwzxCualRtNdh7t m5JMNUHA3lfH2DsHykdk8wDC5eXH/U+PmTSPZ9CdEcnF18b3wl/JkV6VFCBDbaYlffvi 3BGjYIVXStmKnfXgIwDUbTLZhBNxFrAK7Rv/z8FB614uHUV7PIwjN1FAZiQTHROpXgeS 91yg== X-Gm-Message-State: AOAM5300q28XuH9ub7AmdsjHQ0s5u2gpcFcwoKoThsL2WVIFbYmcrFDh /Scgho00ar9G3zI5LzxZvLihwUgVNxM= X-Google-Smtp-Source: ABdhPJzYd4vDWCMOY7/UC0i1nOs93WMze7d08OhFnwiOaXeQ9VuyOuNTqKK6zWp1RgoaybpJN7R/mxKxMS4= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:ad4:5fc6:: with SMTP id jq6mr5885999qvb.43.1620148679153; Tue, 04 May 2021 10:17:59 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:26 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-8-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 07/15] KVM: x86: Add support for RDPID without RDTSCP From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Allow userspace to enable RDPID for a guest without also enabling RDTSCP. Aside from checking for RDPID support in the obvious flows, VMX also needs to set ENABLE_RDTSCP=1 when RDPID is exposed. For the record, there is no known scenario where enabling RDPID without RDTSCP is desirable. But, both AMD and Intel architectures allow for the condition, i.e. this is purely to make KVM more architecturally accurate. Fixes: 41cd02c6f7f6 ("kvm: x86: Expose RDPID in KVM_GET_SUPPORTED_CPUID") Cc: stable@vger.kernel.org Reported-by: Reiji Watanabe Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/svm.c | 6 ++++-- arch/x86/kvm/vmx/vmx.c | 27 +++++++++++++++++++++++---- arch/x86/kvm/x86.c | 3 ++- 3 files changed, 29 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index b3153d40cc4d..231b9650d864 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -2669,7 +2669,8 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) if (tsc_aux_uret_slot < 0) return 1; if (!msr_info->host_initiated && - !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP)) + !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP) && + !guest_cpuid_has(vcpu, X86_FEATURE_RDPID)) return 1; msr_info->data = svm->tsc_aux; break; @@ -2891,7 +2892,8 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) return 1; if (!msr->host_initiated && - !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP)) + !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP) && + !guest_cpuid_has(vcpu, X86_FEATURE_RDPID)) return 1; /* diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 990ee339a05f..42e4bbaa299a 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1788,7 +1788,8 @@ static void setup_msrs(struct vcpu_vmx *vmx) if (update_transition_efer(vmx)) vmx_setup_uret_msr(vmx, MSR_EFER); - if (guest_cpuid_has(&vmx->vcpu, X86_FEATURE_RDTSCP)) + if (guest_cpuid_has(&vmx->vcpu, X86_FEATURE_RDTSCP) || + guest_cpuid_has(&vmx->vcpu, X86_FEATURE_RDPID)) vmx_setup_uret_msr(vmx, MSR_TSC_AUX); vmx_setup_uret_msr(vmx, MSR_IA32_TSX_CTRL); @@ -1994,7 +1995,8 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) break; case MSR_TSC_AUX: if (!msr_info->host_initiated && - !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP)) + !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP) && + !guest_cpuid_has(vcpu, X86_FEATURE_RDPID)) return 1; goto find_uret_msr; case MSR_IA32_DEBUGCTLMSR: @@ -2314,7 +2316,8 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) break; case MSR_TSC_AUX: if (!msr_info->host_initiated && - !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP)) + !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP) && + !guest_cpuid_has(vcpu, X86_FEATURE_RDPID)) return 1; /* Check reserved bit, higher 32 bits should be zero */ if ((data >> 32) != 0) @@ -4368,7 +4371,23 @@ static void vmx_compute_secondary_exec_control(struct vcpu_vmx *vmx) xsaves_enabled, false); } - vmx_adjust_sec_exec_feature(vmx, &exec_control, rdtscp, RDTSCP); + /* + * RDPID is also gated by ENABLE_RDTSCP, turn on the control if either + * feature is exposed to the guest. This creates a virtualization hole + * if both are supported in hardware but only one is exposed to the + * guest, but letting the guest execute RDTSCP or RDPID when either one + * is advertised is preferable to emulating the advertised instruction + * in KVM on #UD, and obviously better than incorrectly injecting #UD. + */ + if (cpu_has_vmx_rdtscp()) { + bool rdpid_or_rdtscp_enabled = + guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP) || + guest_cpuid_has(vcpu, X86_FEATURE_RDPID); + + vmx_adjust_secondary_exec_control(vmx, &exec_control, + SECONDARY_EXEC_ENABLE_RDTSCP, + rdpid_or_rdtscp_enabled, false); + } vmx_adjust_sec_exec_feature(vmx, &exec_control, invpcid, INVPCID); vmx_adjust_sec_exec_exiting(vmx, &exec_control, rdrand, RDRAND); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index e304447be42d..b4516d303413 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5978,7 +5978,8 @@ static void kvm_init_msr_list(void) continue; break; case MSR_TSC_AUX: - if (!kvm_cpu_cap_has(X86_FEATURE_RDTSCP)) + if (!kvm_cpu_cap_has(X86_FEATURE_RDTSCP) && + !kvm_cpu_cap_has(X86_FEATURE_RDPID)) continue; break; case MSR_IA32_UMWAIT_CONTROL: From patchwork Tue May 4 17:17:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 650ACC433ED for ; Tue, 4 May 2021 17:18:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4199D613C5 for ; Tue, 4 May 2021 17:18:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232263AbhEDRTH (ORCPT ); Tue, 4 May 2021 13:19:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39662 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232184AbhEDRS5 (ORCPT ); Tue, 4 May 2021 13:18:57 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80B0FC06138D for ; Tue, 4 May 2021 10:18:02 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id u3-20020a2509430000b02904e7f1a30cffso12571922ybm.8 for ; Tue, 04 May 2021 10:18:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=BuFmFez9jUwRrnImsJ+azB1vrZijd6DUJ5M+cWbALfs=; b=PIfalVHC3CmdnOo5mB3aYhbwxb1wXrQpggHTflDbp6Uh83tVVd+KOzlWt1oSBY493K 9FnzGeEcc9Y9Ro58Pc567FMkCbJYMGyMy0Z7EYWWrenzyg5qKNzMevOGakrh7kFzJEXx 9pCf8KAH845t6dr/gtKuc7T5Ct6jH6xiu/BEhzgd2wOPw6acO9Chl0Mw+ZElTupOYZmm pBccjs7Knx/aXfVGK2S1pYOL+rSYV0xi1jndG2/aSrRJ6mWrkPwsGxRxL8Vi/6hJhg0R l7wFb1vxrSi5NtojsrOULz6I4wCiWaIAImJA+yclYjN5OFuBIo+QSVYozmmVXrB5TroK E+BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=BuFmFez9jUwRrnImsJ+azB1vrZijd6DUJ5M+cWbALfs=; b=XjsxBWFHubRx91++pMGAL7F4/FggUeJg1qOqwnZJRFKUN55mLbNGdbIjs3+nhDpB6S 6x0NrgGL8dKC6bmNsH0RJbKJq0MYUIXvgysEvzC/18GflYzM1A1K+Y9iIQRQ3wFhZq/o sp8pt3fjl6m7LvSYy3UlgKmF6y1IM1scOkQeDH0sPmTQebUan77EtQ32dOgq6H5TU7rn WlN1apL6TeHR7lDZGGsrkT26cjtEORDvJXwYjRb4jIDED4ScQiUuvvYjXA5rYMNKx6iY uy8rDt2+cE6FbqjZ9JuFbE0fIoZM3s03RITrSPc/PLkLNJj5SgMR4FjOaMiznMyhhizI kR6A== X-Gm-Message-State: AOAM531w4PYXOllzILWK5w0siUQ5SBKtSDwnSSna2G0ar8gWx8BVHyUE Ps1WkTbhSdqhFcyoa7dxF3Ut8lo0Xl0= X-Google-Smtp-Source: ABdhPJxM3Dh5e52Zged/McORpx1LmcgL4PsGFgtMVfeMjkfKzXR2ggkHVyGPixQ/p2PxcnWIM9V1rgaNfDU= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a5b:802:: with SMTP id x2mr35901294ybp.28.1620148681694; Tue, 04 May 2021 10:18:01 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:27 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-9-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 08/15] KVM: VMX: Configure list of user return MSRs at module init From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Configure the list of user return MSRs that are actually supported at module init instead of reprobing the list of possible MSRs every time a vCPU is created. Curating the list on a per-vCPU basis is pointless; KVM is completely hosed if the set of supported MSRs changes after module init, or if the set of MSRs differs per physical PCU. The per-vCPU lists also increase complexity (see __vmx_find_uret_msr()) and creates corner cases that _should_ be impossible, but theoretically exist in KVM, e.g. advertising RDTSCP to userspace without actually being able to virtualize RDTSCP if probing MSR_TSC_AUX fails. Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/vmx/vmx.c | 61 ++++++++++++++++++++++++++++-------------- arch/x86/kvm/vmx/vmx.h | 10 ++++++- 2 files changed, 50 insertions(+), 21 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 42e4bbaa299a..68454b0de2b1 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -461,7 +461,7 @@ static unsigned long host_idt_base; * support this emulation, IA32_STAR must always be included in * vmx_uret_msrs_list[], even in i386 builds. */ -static const u32 vmx_uret_msrs_list[] = { +static u32 vmx_uret_msrs_list[] = { #ifdef CONFIG_X86_64 MSR_SYSCALL_MASK, MSR_LSTAR, MSR_CSTAR, #endif @@ -469,6 +469,12 @@ static const u32 vmx_uret_msrs_list[] = { MSR_IA32_TSX_CTRL, }; +/* + * Number of user return MSRs that are actually supported in hardware. + * vmx_uret_msrs_list is modified when KVM is loaded to drop unsupported MSRs. + */ +static int vmx_nr_uret_msrs; + #if IS_ENABLED(CONFIG_HYPERV) static bool __read_mostly enlightened_vmcs = true; module_param(enlightened_vmcs, bool, 0444); @@ -700,9 +706,16 @@ static inline int __vmx_find_uret_msr(struct vcpu_vmx *vmx, u32 msr) { int i; - for (i = 0; i < vmx->nr_uret_msrs; ++i) + /* + * Note, vmx->guest_uret_msrs is the same size as vmx_uret_msrs_list, + * but is ordered differently. The MSR is matched against the list of + * supported uret MSRs using "slot", but the index that is returned is + * the index into guest_uret_msrs. + */ + for (i = 0; i < vmx_nr_uret_msrs; ++i) { if (vmx_uret_msrs_list[vmx->guest_uret_msrs[i].slot] == msr) return i; + } return -1; } @@ -6929,18 +6942,10 @@ static int vmx_create_vcpu(struct kvm_vcpu *vcpu) goto free_vpid; } - BUILD_BUG_ON(ARRAY_SIZE(vmx_uret_msrs_list) != MAX_NR_USER_RETURN_MSRS); + for (i = 0; i < vmx_nr_uret_msrs; ++i) { + vmx->guest_uret_msrs[i].data = 0; - for (i = 0; i < ARRAY_SIZE(vmx_uret_msrs_list); ++i) { - u32 index = vmx_uret_msrs_list[i]; - int j = vmx->nr_uret_msrs; - - if (kvm_probe_user_return_msr(index)) - continue; - - vmx->guest_uret_msrs[j].slot = i; - vmx->guest_uret_msrs[j].data = 0; - switch (index) { + switch (vmx_uret_msrs_list[i]) { case MSR_IA32_TSX_CTRL: /* * TSX_CTRL_CPUID_CLEAR is handled in the CPUID @@ -6954,15 +6959,14 @@ static int vmx_create_vcpu(struct kvm_vcpu *vcpu) * host so that TSX remains always disabled. */ if (boot_cpu_has(X86_FEATURE_RTM)) - vmx->guest_uret_msrs[j].mask = ~(u64)TSX_CTRL_CPUID_CLEAR; + vmx->guest_uret_msrs[i].mask = ~(u64)TSX_CTRL_CPUID_CLEAR; else - vmx->guest_uret_msrs[j].mask = 0; + vmx->guest_uret_msrs[i].mask = 0; break; default: - vmx->guest_uret_msrs[j].mask = -1ull; + vmx->guest_uret_msrs[i].mask = -1ull; break; } - ++vmx->nr_uret_msrs; } err = alloc_loaded_vmcs(&vmx->vmcs01); @@ -7821,17 +7825,34 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, }; +static __init void vmx_setup_user_return_msrs(void) +{ + u32 msr; + int i; + + BUILD_BUG_ON(ARRAY_SIZE(vmx_uret_msrs_list) != MAX_NR_USER_RETURN_MSRS); + + for (i = 0; i < ARRAY_SIZE(vmx_uret_msrs_list); ++i) { + msr = vmx_uret_msrs_list[i]; + + if (kvm_probe_user_return_msr(msr)) + continue; + + kvm_define_user_return_msr(vmx_nr_uret_msrs, msr); + vmx_uret_msrs_list[vmx_nr_uret_msrs++] = msr; + } +} + static __init int hardware_setup(void) { unsigned long host_bndcfgs; struct desc_ptr dt; - int r, i, ept_lpage_level; + int r, ept_lpage_level; store_idt(&dt); host_idt_base = dt.address; - for (i = 0; i < ARRAY_SIZE(vmx_uret_msrs_list); ++i) - kvm_define_user_return_msr(i, vmx_uret_msrs_list[i]); + vmx_setup_user_return_msrs(); if (setup_vmcs_config(&vmcs_config, &vmx_capability) < 0) return -EIO; diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 008cb87ff088..d71ed8b425c5 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -245,8 +245,16 @@ struct vcpu_vmx { u32 idt_vectoring_info; ulong rflags; + /* + * User return MSRs are always emulated when enabled in the guest, but + * only loaded into hardware when necessary, e.g. SYSCALL #UDs outside + * of 64-bit mode or if EFER.SCE=1, thus the SYSCALL MSRs don't need to + * be loaded into hardware if those conditions aren't met. + * nr_active_uret_msrs tracks the number of MSRs that need to be loaded + * into hardware when running the guest. guest_uret_msrs[] is resorted + * whenever the number of "active" uret MSRs is modified. + */ struct vmx_uret_msr guest_uret_msrs[MAX_NR_USER_RETURN_MSRS]; - int nr_uret_msrs; int nr_active_uret_msrs; bool guest_uret_msrs_loaded; #ifdef CONFIG_X86_64 From patchwork Tue May 4 17:17:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A66AC43462 for ; Tue, 4 May 2021 17:18:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4729F613B4 for ; Tue, 4 May 2021 17:18:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232369AbhEDRTT (ORCPT ); Tue, 4 May 2021 13:19:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232231AbhEDRTD (ORCPT ); Tue, 4 May 2021 13:19:03 -0400 Received: from mail-qv1-xf4a.google.com (mail-qv1-xf4a.google.com [IPv6:2607:f8b0:4864:20::f4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CAF51C061343 for ; Tue, 4 May 2021 10:18:04 -0700 (PDT) Received: by mail-qv1-xf4a.google.com with SMTP id x6-20020a0cda060000b02901c4b3f7d3d9so8169873qvj.0 for ; Tue, 04 May 2021 10:18:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=rLxHu6ur1nnGAfhjZTDq/vOPrU25QwD4j6qC952iens=; b=XLTGauuasYQDrfDo0C+Tj+Gx4uHJ3NazgHV23+Kc3qBnAQcTkbb/mNSz0siFv0jp4d AKXvXaovhVqyrLLsFlq4XB0WIgTTcw6W4RwefPhbI9cSmxi9QuUxOtkiset6gQsi0Cly gmQf9quYOsxwPVCmeKfZHPI7r+W2XiOt4u+Yp9oyNvwtEZ6JxtlFvpiHR2HFjsyEFwAf kKatzGYF+mm/8caQJioDiIYpvERIFCRglo3986Y8naW/nYnw6KG+qHvRKle7nb/bqMvS Ir4TSNkEzIujxrI53+O2j7VlSh0kmH7ysLlmszmn8bxndd/Pu+24H9fZSxWIeg6JORIF l4Pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=rLxHu6ur1nnGAfhjZTDq/vOPrU25QwD4j6qC952iens=; b=Gpsf/R/hOI7Sm9Ksj31BZc7oW33E1qV/E/jSMovHPrjAiAj2i/Wy5zm4S7IPfc0gI3 6x9h5tLtCQthMK3FrhEOI1+oYTMZ/0GRWwxlOjGXJ9s7tNDDAZQyo2lQtLnvtDE3CDu2 +dNiQrtfEKp10Kle38D7GlcdjufGslwarcDISQ+J0AaKH2lXxSTgnTQfInLl1uUB0AOS 9bUt1DOFdCvSuKXVHJnjSmJnBRaP92CiTXQlHPxTqMp6VevGoK/zvASpEv7IZc0dj25x Mxp2CCf+bYf0SEBfBx5LwlyBREw0CeKwgHNFR1RlJGzWqPe4f8ODoQ1msLxKki/XIy+J +duQ== X-Gm-Message-State: AOAM532zLsbA2kCiL3C1DZz8nvcZWtKqSuo4BG2t3J3Br6+g1tSfLfCg aNBIHB2IAOaQFfkzDz8V+aDcrGKF4FM= X-Google-Smtp-Source: ABdhPJyTEKiVJfglk9hr9Q5KTagcSToFUaeeof7MhHoT10+7kKLUl9DID6mdMDzkS9DwPiuPmiqvxYfHwNU= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a0c:df08:: with SMTP id g8mr27360414qvl.12.1620148683932; Tue, 04 May 2021 10:18:03 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:28 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-10-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 09/15] KVM: VMX: Use flag to indicate "active" uret MSRs instead of sorting list From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Explicitly flag a uret MSR as needing to be loaded into hardware instead of resorting the list of "active" MSRs and tracking how many MSRs in total need to be loaded. The only benefit to sorting the list is that the loop to load MSRs during vmx_prepare_switch_to_guest() doesn't need to iterate over all supported uret MRS, only those that are active. But that is a pointless optimization, as the most common case, running a 64-bit guest, will load the vast majority of MSRs. Not to mention that a single WRMSR is far more expensive than iterating over the list. Providing a stable list order obviates the need to track a given MSR's "slot" in the per-CPU list of user return MSRs; all lists simply use the same ordering. Future patches will take advantage of the stable order to further simplify the related code. Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/vmx/vmx.c | 80 ++++++++++++++++++++++-------------------- arch/x86/kvm/vmx/vmx.h | 2 +- 2 files changed, 42 insertions(+), 40 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 68454b0de2b1..6caabcd5037e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -458,8 +458,9 @@ static unsigned long host_idt_base; * Though SYSCALL is only supported in 64-bit mode on Intel CPUs, kvm * will emulate SYSCALL in legacy mode if the vendor string in guest * CPUID.0:{EBX,ECX,EDX} is "AuthenticAMD" or "AMDisbetter!" To - * support this emulation, IA32_STAR must always be included in - * vmx_uret_msrs_list[], even in i386 builds. + * support this emulation, MSR_STAR is included in the list for i386, + * but is never loaded into hardware. MSR_CSTAR is also never loaded + * into hardware and is here purely for emulation purposes. */ static u32 vmx_uret_msrs_list[] = { #ifdef CONFIG_X86_64 @@ -702,18 +703,12 @@ static bool is_valid_passthrough_msr(u32 msr) return r; } -static inline int __vmx_find_uret_msr(struct vcpu_vmx *vmx, u32 msr) +static inline int __vmx_find_uret_msr(u32 msr) { int i; - /* - * Note, vmx->guest_uret_msrs is the same size as vmx_uret_msrs_list, - * but is ordered differently. The MSR is matched against the list of - * supported uret MSRs using "slot", but the index that is returned is - * the index into guest_uret_msrs. - */ for (i = 0; i < vmx_nr_uret_msrs; ++i) { - if (vmx_uret_msrs_list[vmx->guest_uret_msrs[i].slot] == msr) + if (vmx_uret_msrs_list[i] == msr) return i; } return -1; @@ -723,7 +718,7 @@ struct vmx_uret_msr *vmx_find_uret_msr(struct vcpu_vmx *vmx, u32 msr) { int i; - i = __vmx_find_uret_msr(vmx, msr); + i = __vmx_find_uret_msr(msr); if (i >= 0) return &vmx->guest_uret_msrs[i]; return NULL; @@ -732,13 +727,14 @@ struct vmx_uret_msr *vmx_find_uret_msr(struct vcpu_vmx *vmx, u32 msr) static int vmx_set_guest_uret_msr(struct vcpu_vmx *vmx, struct vmx_uret_msr *msr, u64 data) { + unsigned int slot = msr - vmx->guest_uret_msrs; int ret = 0; u64 old_msr_data = msr->data; msr->data = data; - if (msr - vmx->guest_uret_msrs < vmx->nr_active_uret_msrs) { + if (msr->load_into_hardware) { preempt_disable(); - ret = kvm_set_user_return_msr(msr->slot, msr->data, msr->mask); + ret = kvm_set_user_return_msr(slot, msr->data, msr->mask); preempt_enable(); if (ret) msr->data = old_msr_data; @@ -1090,7 +1086,7 @@ static bool update_transition_efer(struct vcpu_vmx *vmx) return false; } - i = __vmx_find_uret_msr(vmx, MSR_EFER); + i = __vmx_find_uret_msr(MSR_EFER); if (i < 0) return false; @@ -1252,11 +1248,14 @@ void vmx_prepare_switch_to_guest(struct kvm_vcpu *vcpu) */ if (!vmx->guest_uret_msrs_loaded) { vmx->guest_uret_msrs_loaded = true; - for (i = 0; i < vmx->nr_active_uret_msrs; ++i) - kvm_set_user_return_msr(vmx->guest_uret_msrs[i].slot, + for (i = 0; i < vmx_nr_uret_msrs; ++i) { + if (!vmx->guest_uret_msrs[i].load_into_hardware) + continue; + + kvm_set_user_return_msr(i, vmx->guest_uret_msrs[i].data, vmx->guest_uret_msrs[i].mask); - + } } if (vmx->nested.need_vmcs12_to_shadow_sync) @@ -1763,19 +1762,16 @@ static void vmx_queue_exception(struct kvm_vcpu *vcpu) vmx_clear_hlt(vcpu); } -static void vmx_setup_uret_msr(struct vcpu_vmx *vmx, unsigned int msr) +static void vmx_setup_uret_msr(struct vcpu_vmx *vmx, unsigned int msr, + bool load_into_hardware) { - struct vmx_uret_msr tmp; - int from, to; + struct vmx_uret_msr *uret_msr; - from = __vmx_find_uret_msr(vmx, msr); - if (from < 0) + uret_msr = vmx_find_uret_msr(vmx, msr); + if (!uret_msr) return; - to = vmx->nr_active_uret_msrs++; - tmp = vmx->guest_uret_msrs[to]; - vmx->guest_uret_msrs[to] = vmx->guest_uret_msrs[from]; - vmx->guest_uret_msrs[from] = tmp; + uret_msr->load_into_hardware = load_into_hardware; } /* @@ -1785,30 +1781,36 @@ static void vmx_setup_uret_msr(struct vcpu_vmx *vmx, unsigned int msr) */ static void setup_msrs(struct vcpu_vmx *vmx) { - vmx->guest_uret_msrs_loaded = false; - vmx->nr_active_uret_msrs = 0; #ifdef CONFIG_X86_64 + bool load_syscall_msrs; + /* * The SYSCALL MSRs are only needed on long mode guests, and only * when EFER.SCE is set. */ - if (is_long_mode(&vmx->vcpu) && (vmx->vcpu.arch.efer & EFER_SCE)) { - vmx_setup_uret_msr(vmx, MSR_STAR); - vmx_setup_uret_msr(vmx, MSR_LSTAR); - vmx_setup_uret_msr(vmx, MSR_SYSCALL_MASK); - } + load_syscall_msrs = is_long_mode(&vmx->vcpu) && + (vmx->vcpu.arch.efer & EFER_SCE); + + vmx_setup_uret_msr(vmx, MSR_STAR, load_syscall_msrs); + vmx_setup_uret_msr(vmx, MSR_LSTAR, load_syscall_msrs); + vmx_setup_uret_msr(vmx, MSR_SYSCALL_MASK, load_syscall_msrs); #endif - if (update_transition_efer(vmx)) - vmx_setup_uret_msr(vmx, MSR_EFER); + vmx_setup_uret_msr(vmx, MSR_EFER, update_transition_efer(vmx)); - if (guest_cpuid_has(&vmx->vcpu, X86_FEATURE_RDTSCP) || - guest_cpuid_has(&vmx->vcpu, X86_FEATURE_RDPID)) - vmx_setup_uret_msr(vmx, MSR_TSC_AUX); + vmx_setup_uret_msr(vmx, MSR_TSC_AUX, + guest_cpuid_has(&vmx->vcpu, X86_FEATURE_RDTSCP) || + guest_cpuid_has(&vmx->vcpu, X86_FEATURE_RDPID)); - vmx_setup_uret_msr(vmx, MSR_IA32_TSX_CTRL); + vmx_setup_uret_msr(vmx, MSR_IA32_TSX_CTRL, true); if (cpu_has_vmx_msr_bitmap()) vmx_update_msr_bitmap(&vmx->vcpu); + + /* + * The set of MSRs to load may have changed, reload MSRs before the + * next VM-Enter. + */ + vmx->guest_uret_msrs_loaded = false; } static u64 vmx_write_l1_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index d71ed8b425c5..16e4e457ba23 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -36,7 +36,7 @@ struct vmx_msrs { }; struct vmx_uret_msr { - unsigned int slot; /* The MSR's slot in kvm_user_return_msrs. */ + bool load_into_hardware; u64 data; u64 mask; }; From patchwork Tue May 4 17:17:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4467C433B4 for ; Tue, 4 May 2021 17:18:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8C8F1613C5 for ; Tue, 4 May 2021 17:18:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231175AbhEDRTP (ORCPT ); Tue, 4 May 2021 13:19:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232220AbhEDRTC (ORCPT ); Tue, 4 May 2021 13:19:02 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13A53C0613ED for ; Tue, 4 May 2021 10:18:07 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id u7-20020a259b470000b02904dca50820c2so12588479ybo.11 for ; Tue, 04 May 2021 10:18:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=ui+JXTkoxDdvwTCRiewZGYRxlMDy8e9kXT2BE3xqXtU=; b=S3hEGvhbPEekD+aCXuLlUiynkTyC45PVD7hjC6JATa85ouheGVuvmafHDWknN9bH/s XukGjXIT/clrheeK5PifQDXlfpG6fG7rBlb8UktuxQkUsPW3PiadJTVSrLxKAmTrnUev WpUzkfX12zPAXP9fxSSbmpNL3sCjYqndEI4ySMMBgbsGDHoXLvjhnXsZH4SZKdmqkKff n0/jgK+xmFOM6n/pSEW5O+1ChTwgCiqrq05+Ccc2xBl1DDfnQi32k+IeDkPMST0TP1SO tSyJtMoHzKCOf2zqouTt8k8ASBnRwqUhuACf6M71tEjt9bVEYhz358TzJ6IdPeoZQN9I QKFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=ui+JXTkoxDdvwTCRiewZGYRxlMDy8e9kXT2BE3xqXtU=; b=GMJXsPEZpaBV4MsakR8XgZtRCfmKeN8jVLN6Uyf+HRxbtYApbaKAlUXpRHyDAv0a/N QzkmbsUiJFPmdUo+37V2abJQBX7fVk5YHBh6lzpXa+Rv65nyVsPc9eZxHO6dVhfVEu/J Ysr4wuc2Gq5BGFqA7WmZUDPXb1P5Ezd9pGn4brorzXmfzVyuhog1SGKWUq4d9XzGLmUh if4rGqcwt3mx3163L5dmdLtBQBlffWwaOnqaVq7FV4Va4aFvrX7ly6FMD5e6BhcuFnAQ jYl5Ydc8f3c4vQI3NhwFKo7WQWpMI8T6hp5tsfD/fin3Nt2SmESWtM/wfUB1pvA0zCFq Kfwg== X-Gm-Message-State: AOAM5311abItF/HH4NLQOvNx+aMW4sAzJWKEV3ZViOHtSKaxNe5sfoys 0fOy/GBqftNdjKtrlLmuC9tfTf9ha8U= X-Google-Smtp-Source: ABdhPJypHZ1VkL2AY0aogZQv1fR5oFfAY2i3dt5Xcgm+D9QA2gWxFmOgsndFRP9+IHwaAAJoB/Y0iGLTguI= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a25:c801:: with SMTP id y1mr37699490ybf.250.1620148686337; Tue, 04 May 2021 10:18:06 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:29 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-11-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 10/15] KVM: VMX: Use common x86's uret MSR list as the one true list From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Drop VMX's global list of user return MSRs now that VMX doesn't resort said list to isolate "active" MSRs, i.e. now that VMX's list and x86's list have the same MSRs in the same order. In addition to eliminating the redundant list, this will also allow moving more of the list management into common x86. Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/vmx.c | 97 ++++++++++++++------------------- arch/x86/kvm/x86.c | 12 ++++ 3 files changed, 53 insertions(+), 57 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index a02c9bf3f7f1..c9452472ed55 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1778,6 +1778,7 @@ int kvm_pv_send_ipi(struct kvm *kvm, unsigned long ipi_bitmap_low, unsigned long icr, int op_64_bit); void kvm_define_user_return_msr(unsigned index, u32 msr); +int kvm_find_user_return_msr(u32 msr); int kvm_probe_user_return_msr(u32 msr); int kvm_set_user_return_msr(unsigned index, u64 val, u64 mask); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 6caabcd5037e..4b432d2bbd06 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -454,26 +454,7 @@ static inline void vmx_segment_cache_clear(struct vcpu_vmx *vmx) static unsigned long host_idt_base; -/* - * Though SYSCALL is only supported in 64-bit mode on Intel CPUs, kvm - * will emulate SYSCALL in legacy mode if the vendor string in guest - * CPUID.0:{EBX,ECX,EDX} is "AuthenticAMD" or "AMDisbetter!" To - * support this emulation, MSR_STAR is included in the list for i386, - * but is never loaded into hardware. MSR_CSTAR is also never loaded - * into hardware and is here purely for emulation purposes. - */ -static u32 vmx_uret_msrs_list[] = { -#ifdef CONFIG_X86_64 - MSR_SYSCALL_MASK, MSR_LSTAR, MSR_CSTAR, -#endif - MSR_EFER, MSR_TSC_AUX, MSR_STAR, - MSR_IA32_TSX_CTRL, -}; - -/* - * Number of user return MSRs that are actually supported in hardware. - * vmx_uret_msrs_list is modified when KVM is loaded to drop unsupported MSRs. - */ +/* Number of user return MSRs that are actually supported in hardware. */ static int vmx_nr_uret_msrs; #if IS_ENABLED(CONFIG_HYPERV) @@ -703,22 +684,11 @@ static bool is_valid_passthrough_msr(u32 msr) return r; } -static inline int __vmx_find_uret_msr(u32 msr) -{ - int i; - - for (i = 0; i < vmx_nr_uret_msrs; ++i) { - if (vmx_uret_msrs_list[i] == msr) - return i; - } - return -1; -} - struct vmx_uret_msr *vmx_find_uret_msr(struct vcpu_vmx *vmx, u32 msr) { int i; - i = __vmx_find_uret_msr(msr); + i = kvm_find_user_return_msr(msr); if (i >= 0) return &vmx->guest_uret_msrs[i]; return NULL; @@ -1086,7 +1056,7 @@ static bool update_transition_efer(struct vcpu_vmx *vmx) return false; } - i = __vmx_find_uret_msr(MSR_EFER); + i = kvm_find_user_return_msr(MSR_EFER); if (i < 0) return false; @@ -6922,6 +6892,7 @@ static void vmx_free_vcpu(struct kvm_vcpu *vcpu) static int vmx_create_vcpu(struct kvm_vcpu *vcpu) { + struct vmx_uret_msr *tsx_ctrl; struct vcpu_vmx *vmx; int i, cpu, err; @@ -6946,29 +6917,25 @@ static int vmx_create_vcpu(struct kvm_vcpu *vcpu) for (i = 0; i < vmx_nr_uret_msrs; ++i) { vmx->guest_uret_msrs[i].data = 0; - - switch (vmx_uret_msrs_list[i]) { - case MSR_IA32_TSX_CTRL: - /* - * TSX_CTRL_CPUID_CLEAR is handled in the CPUID - * interception. Keep the host value unchanged to avoid - * changing CPUID bits under the host kernel's feet. - * - * hle=0, rtm=0, tsx_ctrl=1 can be found with some - * combinations of new kernel and old userspace. If - * those guests run on a tsx=off host, do allow guests - * to use TSX_CTRL, but do not change the value on the - * host so that TSX remains always disabled. - */ - if (boot_cpu_has(X86_FEATURE_RTM)) - vmx->guest_uret_msrs[i].mask = ~(u64)TSX_CTRL_CPUID_CLEAR; - else - vmx->guest_uret_msrs[i].mask = 0; - break; - default: - vmx->guest_uret_msrs[i].mask = -1ull; - break; - } + vmx->guest_uret_msrs[i].mask = -1ull; + } + tsx_ctrl = vmx_find_uret_msr(vmx, MSR_IA32_TSX_CTRL); + if (tsx_ctrl) { + /* + * TSX_CTRL_CPUID_CLEAR is handled in the CPUID interception. + * Keep the host value unchanged to avoid changing CPUID bits + * under the host kernel's feet. + * + * hle=0, rtm=0, tsx_ctrl=1 can be found with some combinations + * of new kernel and old userspace. If those guests run on a + * tsx=off host, do allow guests to use TSX_CTRL, but do not + * change the value on the host so that TSX remains always + * disabled. + */ + if (boot_cpu_has(X86_FEATURE_RTM)) + vmx->guest_uret_msrs[i].mask = ~(u64)TSX_CTRL_CPUID_CLEAR; + else + vmx->guest_uret_msrs[i].mask = 0; } err = alloc_loaded_vmcs(&vmx->vmcs01); @@ -7829,6 +7796,22 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { static __init void vmx_setup_user_return_msrs(void) { + + /* + * Though SYSCALL is only supported in 64-bit mode on Intel CPUs, kvm + * will emulate SYSCALL in legacy mode if the vendor string in guest + * CPUID.0:{EBX,ECX,EDX} is "AuthenticAMD" or "AMDisbetter!" To + * support this emulation, MSR_STAR is included in the list for i386, + * but is never loaded into hardware. MSR_CSTAR is also never loaded + * into hardware and is here purely for emulation purposes. + */ + const u32 vmx_uret_msrs_list[] = { + #ifdef CONFIG_X86_64 + MSR_SYSCALL_MASK, MSR_LSTAR, MSR_CSTAR, + #endif + MSR_EFER, MSR_TSC_AUX, MSR_STAR, + MSR_IA32_TSX_CTRL, + }; u32 msr; int i; @@ -7841,7 +7824,7 @@ static __init void vmx_setup_user_return_msrs(void) continue; kvm_define_user_return_msr(vmx_nr_uret_msrs, msr); - vmx_uret_msrs_list[vmx_nr_uret_msrs++] = msr; + vmx_nr_uret_msrs++; } } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b4516d303413..90ef340565a4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -364,6 +364,18 @@ void kvm_define_user_return_msr(unsigned slot, u32 msr) } EXPORT_SYMBOL_GPL(kvm_define_user_return_msr); +int kvm_find_user_return_msr(u32 msr) +{ + int i; + + for (i = 0; i < user_return_msrs_global.nr; ++i) { + if (user_return_msrs_global.msrs[i] == msr) + return i; + } + return -1; +} +EXPORT_SYMBOL_GPL(kvm_find_user_return_msr); + static void kvm_user_return_msr_cpu_online(void) { unsigned int cpu = smp_processor_id(); From patchwork Tue May 4 17:17:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81C16C433ED for ; Tue, 4 May 2021 17:18:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 66104613C5 for ; Tue, 4 May 2021 17:18:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232259AbhEDRTV (ORCPT ); Tue, 4 May 2021 13:19:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231842AbhEDRTF (ORCPT ); Tue, 4 May 2021 13:19:05 -0400 Received: from mail-qv1-xf4a.google.com (mail-qv1-xf4a.google.com [IPv6:2607:f8b0:4864:20::f4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 27AC3C06174A for ; Tue, 4 May 2021 10:18:09 -0700 (PDT) Received: by mail-qv1-xf4a.google.com with SMTP id d11-20020a0cdb0b0000b02901c0da4391d5so8167849qvk.12 for ; Tue, 04 May 2021 10:18:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=OfxrzbCyayDhIsgc+QsgzNeUTDdIWPqYIy1TjIvLC5Y=; b=QJ3Hkp2+Ym0T9YKmjqMWJvgu3kPTtTxOSvakW/z/eVcGcwWHYaOjwvTKYbuvwTfaIF qBwf/FVLAnsMogpo9M59vB2p0NsyV4Ho3A1HW2P/t2OxEG/jq4mYRrcyhj0FYG5nzu7T FDUlut38RLct/pFsq/vDamxiVrg6s1IQksMtj/5QnP9+OHZ3iOhoz7sN2sptcSpo0Izi V16YnJ9OrgXKaz7caMvCC41x4/+ajPUOJXU7tS8c2qbwLlONrJPEct8BUgsovVUTWCDm HB0FGx9dHdGIQGcxQ2EnOMoy3ZBUzhdNm6V5zjebIi1o3CHHh9F5vA6o+t/NyRG//dO/ GlIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=OfxrzbCyayDhIsgc+QsgzNeUTDdIWPqYIy1TjIvLC5Y=; b=B2M0HNrfh+DX/2/cOEme46RWLbUmXfbv5tJOgpgVqsb0HWqlIVaixgcTxv1ApuoQO1 LCSFQi+kREnxLDk+kHNhQP43J8anpVtd9ryC4J+/mo0ejCnChaA+Sk8767BFRzo92L+7 oOaSxF89lgviK1jsJFFYMPKc5ZMoRnfXlOqW41GaTr5KajlxnbjMVAzjydOHCreantU/ moUIAyOHrVMzpA13AcsxWr492Q7xuBrmShFJA2t00OPFJ+VI0+IkQ68jmaVs5PzPjOjo bHNqaGWkoEwPN3HIAbBCp5EXzn6Eno1spfBv0kSG2aYZ+ggNiDijSX++e53YD3DFrhN0 3gGQ== X-Gm-Message-State: AOAM532mrXJ7BOyQ3//jZYOJHIvsUBZzthYoyLuN6e/sTr3yVTG8A1ie 6yFcIQoM0PNmQzH34ACb+gNCgEAs+Vg= X-Google-Smtp-Source: ABdhPJyOGmfzlqBR28ALLywAnRsDHPLL4pecU+CxalZu57YMVP1A5uIdGv3FvOoGtTUMYKqm6+SYvig3Gws= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a0c:bd2b:: with SMTP id m43mr26873037qvg.52.1620148688313; Tue, 04 May 2021 10:18:08 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:30 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-12-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 11/15] KVM: VMX: Disable loading of TSX_CTRL MSR the more conventional way From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Tag TSX_CTRL as not needing to be loaded when RTM isn't supported in the host. Crushing the write mask to '0' has the same effect, but requires more mental gymnastics to understand. Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/vmx/vmx.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 4b432d2bbd06..7a53568b34fc 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1771,7 +1771,13 @@ static void setup_msrs(struct vcpu_vmx *vmx) guest_cpuid_has(&vmx->vcpu, X86_FEATURE_RDTSCP) || guest_cpuid_has(&vmx->vcpu, X86_FEATURE_RDPID)); - vmx_setup_uret_msr(vmx, MSR_IA32_TSX_CTRL, true); + /* + * hle=0, rtm=0, tsx_ctrl=1 can be found with some combinations of new + * kernel and old userspace. If those guests run on a tsx=off host, do + * allow guests to use TSX_CTRL, but don't change the value in hardware + * so that TSX remains always disabled. + */ + vmx_setup_uret_msr(vmx, MSR_IA32_TSX_CTRL, boot_cpu_has(X86_FEATURE_RTM)); if (cpu_has_vmx_msr_bitmap()) vmx_update_msr_bitmap(&vmx->vcpu); @@ -6919,23 +6925,15 @@ static int vmx_create_vcpu(struct kvm_vcpu *vcpu) vmx->guest_uret_msrs[i].data = 0; vmx->guest_uret_msrs[i].mask = -1ull; } - tsx_ctrl = vmx_find_uret_msr(vmx, MSR_IA32_TSX_CTRL); - if (tsx_ctrl) { + if (boot_cpu_has(X86_FEATURE_RTM)) { /* * TSX_CTRL_CPUID_CLEAR is handled in the CPUID interception. * Keep the host value unchanged to avoid changing CPUID bits * under the host kernel's feet. - * - * hle=0, rtm=0, tsx_ctrl=1 can be found with some combinations - * of new kernel and old userspace. If those guests run on a - * tsx=off host, do allow guests to use TSX_CTRL, but do not - * change the value on the host so that TSX remains always - * disabled. */ - if (boot_cpu_has(X86_FEATURE_RTM)) + tsx_ctrl = vmx_find_uret_msr(vmx, MSR_IA32_TSX_CTRL); + if (tsx_ctrl) vmx->guest_uret_msrs[i].mask = ~(u64)TSX_CTRL_CPUID_CLEAR; - else - vmx->guest_uret_msrs[i].mask = 0; } err = alloc_loaded_vmcs(&vmx->vmcs01); From patchwork Tue May 4 17:17:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 507C6C433B4 for ; Tue, 4 May 2021 17:18:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3795A613AA for ; Tue, 4 May 2021 17:18:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232229AbhEDRT2 (ORCPT ); Tue, 4 May 2021 13:19:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232306AbhEDRTK (ORCPT ); Tue, 4 May 2021 13:19:10 -0400 Received: from mail-qk1-x749.google.com (mail-qk1-x749.google.com [IPv6:2607:f8b0:4864:20::749]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89864C06134B for ; Tue, 4 May 2021 10:18:11 -0700 (PDT) Received: by mail-qk1-x749.google.com with SMTP id d15-20020a05620a136fb02902e9e93c69c8so4024663qkl.23 for ; Tue, 04 May 2021 10:18:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=EJVLUy9Wu1XMVquYiQb1OEBFSqdA144hYS1qNH/p6ok=; b=GaRByPbJVmtgHWnYspoS8Dyjsh95eo63qw5bARG4QFcCgvqBFPaPZgkMwhIudspGV3 XxsQJfbuP47evZwpOW+90yLQ54UiDMP0KpyTMEay5IS3EmorT3z2HvoZ8/5HY+58ZT7D kJ1Q9SUEAN74G/9VOxqFRd6SHA+CU6zjz/rfMNmqlbPWQ3VkFDZQnYgRzUciw+sZiZRW QFVzh75tN13ns60wE+2IufukKRoZ2SteQiTMAvH8QOhuRZ86Q7W4fN6vNVaPCs6TWYY+ gDXEUjdS1PRaApRNuzzRf8V8/kPKZjXAdFlxHvSdFGggSKInJtBXv+eGNm05UHzp+n+t TkJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=EJVLUy9Wu1XMVquYiQb1OEBFSqdA144hYS1qNH/p6ok=; b=IzaIvDToHjFz6iXncEYyBHxEk7cwA3wWQYeUeLHWbD7QHB/SOloxrP70qhBjEt3PVl rW2E9AQRhTDj6DBEOktJXQvHCpYEogatpdNkaPlodb0uDwU9YiK2iriWhfsi4LSfa6sI QwaEeQuqUtUXYMLxnEfo4rO7nqa9hfYUR8IsNs135CcI12vNfFIJWAwGu2QsTQ0zUMIi 2zgWmgXp4sLiUJj4uq6/uF1SnjbgXByrGodcgBStjhlRcQfp06NEUgVbHPCpQfeDTbDL oqOQLH9F64uapzQTlWhqSxFQWs5SDcqYH8X4pyNzYsLb/fsUTWrW7k7jZGGXPGM0Slb8 xNMg== X-Gm-Message-State: AOAM530wAMV8X4NQyqNMxdjTR1DS9qwAMkHbBaHQSa5AGGLee2ht3lk/ hJeemAjga4afFHff/Nw8hDzZU95UUXE= X-Google-Smtp-Source: ABdhPJyhVg/bmFC4ZCuJps+AFIdNNG3bxydLRShSlFwsS1FMASCi+Phzl7sFcSPLPYREl0r4GH/DbMxTvP4= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a0c:e486:: with SMTP id n6mr24367135qvl.21.1620148690729; Tue, 04 May 2021 10:18:10 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:31 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-13-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 12/15] KVM: x86: Export the number of uret MSRs to vendor modules From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Split out and export the number of configured user return MSRs so that VMX can iterate over the set of MSRs without having to do its own tracking. Keep the list itself internal to x86 so that vendor code still has to go through the "official" APIs to add/modify entries. Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 29 +++++++++++++---------------- 2 files changed, 14 insertions(+), 16 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index c9452472ed55..10663610f105 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1419,6 +1419,7 @@ struct kvm_arch_async_pf { bool direct_map; }; +extern u32 __read_mostly kvm_nr_uret_msrs; extern u64 __read_mostly host_efer; extern bool __read_mostly allow_smaller_maxphyaddr; extern struct kvm_x86_ops kvm_x86_ops; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 90ef340565a4..2fd46e917666 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -184,11 +184,6 @@ module_param(pi_inject_timer, bint, S_IRUGO | S_IWUSR); */ #define KVM_MAX_NR_USER_RETURN_MSRS 16 -struct kvm_user_return_msrs_global { - int nr; - u32 msrs[KVM_MAX_NR_USER_RETURN_MSRS]; -}; - struct kvm_user_return_msrs { struct user_return_notifier urn; bool registered; @@ -198,7 +193,9 @@ struct kvm_user_return_msrs { } values[KVM_MAX_NR_USER_RETURN_MSRS]; }; -static struct kvm_user_return_msrs_global __read_mostly user_return_msrs_global; +u32 __read_mostly kvm_nr_uret_msrs; +EXPORT_SYMBOL_GPL(kvm_nr_uret_msrs); +static u32 __read_mostly kvm_uret_msrs_list[KVM_MAX_NR_USER_RETURN_MSRS]; static struct kvm_user_return_msrs __percpu *user_return_msrs; #define KVM_SUPPORTED_XCR0 (XFEATURE_MASK_FP | XFEATURE_MASK_SSE \ @@ -330,10 +327,10 @@ static void kvm_on_user_return(struct user_return_notifier *urn) user_return_notifier_unregister(urn); } local_irq_restore(flags); - for (slot = 0; slot < user_return_msrs_global.nr; ++slot) { + for (slot = 0; slot < kvm_nr_uret_msrs; ++slot) { values = &msrs->values[slot]; if (values->host != values->curr) { - wrmsrl(user_return_msrs_global.msrs[slot], values->host); + wrmsrl(kvm_uret_msrs_list[slot], values->host); values->curr = values->host; } } @@ -358,9 +355,9 @@ EXPORT_SYMBOL_GPL(kvm_probe_user_return_msr); void kvm_define_user_return_msr(unsigned slot, u32 msr) { BUG_ON(slot >= KVM_MAX_NR_USER_RETURN_MSRS); - user_return_msrs_global.msrs[slot] = msr; - if (slot >= user_return_msrs_global.nr) - user_return_msrs_global.nr = slot + 1; + kvm_uret_msrs_list[slot] = msr; + if (slot >= kvm_nr_uret_msrs) + kvm_nr_uret_msrs = slot + 1; } EXPORT_SYMBOL_GPL(kvm_define_user_return_msr); @@ -368,8 +365,8 @@ int kvm_find_user_return_msr(u32 msr) { int i; - for (i = 0; i < user_return_msrs_global.nr; ++i) { - if (user_return_msrs_global.msrs[i] == msr) + for (i = 0; i < kvm_nr_uret_msrs; ++i) { + if (kvm_uret_msrs_list[i] == msr) return i; } return -1; @@ -383,8 +380,8 @@ static void kvm_user_return_msr_cpu_online(void) u64 value; int i; - for (i = 0; i < user_return_msrs_global.nr; ++i) { - rdmsrl_safe(user_return_msrs_global.msrs[i], &value); + for (i = 0; i < kvm_nr_uret_msrs; ++i) { + rdmsrl_safe(kvm_uret_msrs_list[i], &value); msrs->values[i].host = value; msrs->values[i].curr = value; } @@ -399,7 +396,7 @@ int kvm_set_user_return_msr(unsigned slot, u64 value, u64 mask) value = (value & mask) | (msrs->values[slot].host & ~mask); if (value == msrs->values[slot].curr) return 0; - err = wrmsrl_safe(user_return_msrs_global.msrs[slot], value); + err = wrmsrl_safe(kvm_uret_msrs_list[slot], value); if (err) return 1; From patchwork Tue May 4 17:17:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81114C433B4 for ; Tue, 4 May 2021 17:18:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 63F65613AA for ; Tue, 4 May 2021 17:18:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232479AbhEDRTd (ORCPT ); Tue, 4 May 2021 13:19:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232314AbhEDRTK (ORCPT ); Tue, 4 May 2021 13:19:10 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A24D3C06134D for ; Tue, 4 May 2021 10:18:13 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id y5-20020a2586050000b02904ee36d3b170so12617034ybk.10 for ; Tue, 04 May 2021 10:18:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=I0YgTTuADCddU7gmQFiimvdU+bEYx4FI76LT0CJf50k=; b=WsX+UOIow9SVeSekUUUEP42Bla3aG09c7lCeheves/gmCrujgJGG87UyJ4m7cJeJN1 wnUUM1dANdZ8mQ8om84Y12/x6dn4Kft3RKNNuvRu33si3QODEkp3DOq2RNjlC5ovblVE G3UryiQJrlaUDYIO2ZXr1SBeQmN8dYgc5KqNUlnwDLTCPk5tVsLbqC1Qa01MlU0fdwGs pHDrf5VQX/feY0r67svoSFElzASM+ocj2nDrLeUZI7xnei7/Bb+xpVZNlIaQtHVfnpeH QjTkjqkPW1dz07H+PZG/yukqHvMIVDzrJt8+7yc29RIAYgH47VA8e6FdEcE7ZSItBlv1 roqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=I0YgTTuADCddU7gmQFiimvdU+bEYx4FI76LT0CJf50k=; b=Caga8XWxsB2ZVL2fjKUp17T4DVL9Dbb0HzADq9M7BWqZzCyGNFaA4/BcjYV6vP2A+2 VbVxLrmFoK3aWrP1FfFhEUCZKuSS45xYg8fSyAzbrk8PwPBKiEdICvCmhMNjOl6SI1xc cOHC6bT2sSmUuOfd67E/pKi48nNvK4fy6LO4dKYCLa123VWNwztLuESwagR5MdOdbIzN 3ZysZKXZ6A+YYTbIX5hcx0dU2i0A11TVQdcs8JMHO9eYX9vKgTuG/zjmrvoFto5Bgl+d J4lPjTySWor5yC3oMFKG1+eCrsCpgX8La3r/e1Q3ISS2O2uA/LrVMpZn1FdyaUZXRTmu ebVA== X-Gm-Message-State: AOAM530H9SnLaOnyNFl2VJuVNT5Pu1RxV3J/mE/E+YEdj8qkpP+5IcK8 fASpGrN/qvF2Qc9Lfn8hlwd2mpU91kU= X-Google-Smtp-Source: ABdhPJzteeOdR219s7VgR2XpDpavvwOQHfDWKk4e7CibhAkBQS4q/EgxJG4LPRotVa6mT3PVcXfHbqXxObs= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a25:b993:: with SMTP id r19mr37069140ybg.445.1620148692865; Tue, 04 May 2021 10:18:12 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:32 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-14-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 13/15] KVM: x86: Move uret MSR slot management to common x86 From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Now that SVM and VMX both probe MSRs before "defining" user return slots for them, consolidate the code for probe+define into common x86 and eliminate the odd behavior of having the vendor code define the slot for a given MSR. Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/include/asm/kvm_host.h | 3 +-- arch/x86/kvm/svm/svm.c | 5 +---- arch/x86/kvm/vmx/vmx.c | 19 ++++--------------- arch/x86/kvm/x86.c | 19 +++++++++++-------- 4 files changed, 17 insertions(+), 29 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 10663610f105..a4b912f7e427 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1778,9 +1778,8 @@ int kvm_pv_send_ipi(struct kvm *kvm, unsigned long ipi_bitmap_low, unsigned long ipi_bitmap_high, u32 min, unsigned long icr, int op_64_bit); -void kvm_define_user_return_msr(unsigned index, u32 msr); +int kvm_add_user_return_msr(u32 msr); int kvm_find_user_return_msr(u32 msr); -int kvm_probe_user_return_msr(u32 msr); int kvm_set_user_return_msr(unsigned index, u64 val, u64 mask); u64 kvm_scale_tsc(struct kvm_vcpu *vcpu, u64 tsc); diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 231b9650d864..de921935e8de 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -959,10 +959,7 @@ static __init int svm_hardware_setup(void) kvm_tsc_scaling_ratio_frac_bits = 32; } - if (!kvm_probe_user_return_msr(MSR_TSC_AUX)) { - tsc_aux_uret_slot = 0; - kvm_define_user_return_msr(tsc_aux_uret_slot, MSR_TSC_AUX); - } + tsc_aux_uret_slot = kvm_add_user_return_msr(MSR_TSC_AUX); /* Check for pause filtering support */ if (!boot_cpu_has(X86_FEATURE_PAUSEFILTER)) { diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 7a53568b34fc..26f82f302391 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -454,9 +454,6 @@ static inline void vmx_segment_cache_clear(struct vcpu_vmx *vmx) static unsigned long host_idt_base; -/* Number of user return MSRs that are actually supported in hardware. */ -static int vmx_nr_uret_msrs; - #if IS_ENABLED(CONFIG_HYPERV) static bool __read_mostly enlightened_vmcs = true; module_param(enlightened_vmcs, bool, 0444); @@ -1218,7 +1215,7 @@ void vmx_prepare_switch_to_guest(struct kvm_vcpu *vcpu) */ if (!vmx->guest_uret_msrs_loaded) { vmx->guest_uret_msrs_loaded = true; - for (i = 0; i < vmx_nr_uret_msrs; ++i) { + for (i = 0; i < kvm_nr_uret_msrs; ++i) { if (!vmx->guest_uret_msrs[i].load_into_hardware) continue; @@ -6921,7 +6918,7 @@ static int vmx_create_vcpu(struct kvm_vcpu *vcpu) goto free_vpid; } - for (i = 0; i < vmx_nr_uret_msrs; ++i) { + for (i = 0; i < kvm_nr_uret_msrs; ++i) { vmx->guest_uret_msrs[i].data = 0; vmx->guest_uret_msrs[i].mask = -1ull; } @@ -7810,20 +7807,12 @@ static __init void vmx_setup_user_return_msrs(void) MSR_EFER, MSR_TSC_AUX, MSR_STAR, MSR_IA32_TSX_CTRL, }; - u32 msr; int i; BUILD_BUG_ON(ARRAY_SIZE(vmx_uret_msrs_list) != MAX_NR_USER_RETURN_MSRS); - for (i = 0; i < ARRAY_SIZE(vmx_uret_msrs_list); ++i) { - msr = vmx_uret_msrs_list[i]; - - if (kvm_probe_user_return_msr(msr)) - continue; - - kvm_define_user_return_msr(vmx_nr_uret_msrs, msr); - vmx_nr_uret_msrs++; - } + for (i = 0; i < ARRAY_SIZE(vmx_uret_msrs_list); ++i) + kvm_add_user_return_msr(vmx_uret_msrs_list[i]); } static __init int hardware_setup(void) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 2fd46e917666..adca491d3b4b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -336,7 +336,7 @@ static void kvm_on_user_return(struct user_return_notifier *urn) } } -int kvm_probe_user_return_msr(u32 msr) +static int kvm_probe_user_return_msr(u32 msr) { u64 val; int ret; @@ -350,16 +350,18 @@ int kvm_probe_user_return_msr(u32 msr) preempt_enable(); return ret; } -EXPORT_SYMBOL_GPL(kvm_probe_user_return_msr); -void kvm_define_user_return_msr(unsigned slot, u32 msr) +int kvm_add_user_return_msr(u32 msr) { - BUG_ON(slot >= KVM_MAX_NR_USER_RETURN_MSRS); - kvm_uret_msrs_list[slot] = msr; - if (slot >= kvm_nr_uret_msrs) - kvm_nr_uret_msrs = slot + 1; + BUG_ON(kvm_nr_uret_msrs >= KVM_MAX_NR_USER_RETURN_MSRS); + + if (kvm_probe_user_return_msr(msr)) + return -1; + + kvm_uret_msrs_list[kvm_nr_uret_msrs] = msr; + return kvm_nr_uret_msrs++; } -EXPORT_SYMBOL_GPL(kvm_define_user_return_msr); +EXPORT_SYMBOL_GPL(kvm_add_user_return_msr); int kvm_find_user_return_msr(u32 msr) { @@ -8169,6 +8171,7 @@ int kvm_arch_init(void *opaque) printk(KERN_ERR "kvm: failed to allocate percpu kvm_user_return_msrs\n"); goto out_free_x86_emulator_cache; } + kvm_nr_uret_msrs = 0; r = kvm_mmu_module_init(); if (r) From patchwork Tue May 4 17:17:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238367 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 625A8C433B4 for ; Tue, 4 May 2021 17:18:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 40903613AA for ; Tue, 4 May 2021 17:18:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232345AbhEDRTk (ORCPT ); Tue, 4 May 2021 13:19:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232350AbhEDRTR (ORCPT ); Tue, 4 May 2021 13:19:17 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F584C061342 for ; Tue, 4 May 2021 10:18:16 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id i201-20020a25d1d20000b02904ed4c01f82bso12703545ybg.20 for ; Tue, 04 May 2021 10:18:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=n3WdNBU3wLg2BveFZOVMyvP1egZ3CBjYg+W/tOrWu7w=; b=P9lOY7moXKyYkZluKXbmMMkDQlfFbAkPP2wDNkED27PIj1b4i6FFISiWtL5WAVl5nY 6cfelvMxZHAGVOq9hsc3gCalvxujh70dTTF2l6iBQUQ7ParF5ztKn/aG/tdxpff2EnQK 8AMY7ns0OnqE/0AkxjclOd9+OQE4MorLT20iLimXbUJJP1Gyt4+CVRimOw5jtoqWjklg KRWsyYxKk1/olUpTRHkmQAT67E0+wnieg288etP1YKMF+Jzu1umOBzN6MB5i5xbcqMkm iPMYj2u5Kl5wgZN533a0sIk+OQHiHS3nnHcM8ydPOc+hCUsw7zg2xK9F0C3ODE6bQ8cJ tXJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=n3WdNBU3wLg2BveFZOVMyvP1egZ3CBjYg+W/tOrWu7w=; b=FnpwB47Fqj6UOT4NmqBHW4A4+0D12fSDwmyDOfw3jXO/6aEfzvJPGksa9ynERVOM4o BQgQ5V+9TlPdZT81npddxhI7dRfxb7vy0StYTPm5ysN8bT9tmHTvXUo9+w+aUe563Dtb iPR0GIJJcQD+2srpjMwVBJnG1yrMl2rj0NtKVRyMy0yQqqkDM/3Q9pcI09WiWDhAYke1 azd93c7MWnC4mmDHZggM5mQgMafTULE4BWq+gcqXX5ZUrg6Flj8ZC+AvgYC/awHey/FO 4XXZdmPt7OVdeoFWr0XkaXcNdnMPn0kXsXT63gLk4P8GnD4IUXJVDcIFYpVUdwzAb4nv gfpw== X-Gm-Message-State: AOAM533+JN5gagRj995JiJIIRQrd3EXhXWrb8RwESlYgP9eaMyh4uRro maRqmFoUnmnO+Gy1wi3V9EkhtTrrYfQ= X-Google-Smtp-Source: ABdhPJwyh5X6WR35hSEdU0UOSUQJq4myiesLVifDWYcIH6ej+A+EgNY0bGrDFWfx929hgZ7bgpzF1NRuIGs= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a25:ef51:: with SMTP id w17mr35753680ybm.520.1620148695580; Tue, 04 May 2021 10:18:15 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:33 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-15-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 14/15] KVM: x86: Tie Intel and AMD behavior for MSR_TSC_AUX to guest CPU model From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Squish the Intel and AMD emulation of MSR_TSC_AUX together and tie it to the guest CPU model instead of the host CPU behavior. While not strictly necessary to avoid guest breakage, emulating cross-vendor "architecture" will provide consistent behavior for the guest, e.g. WRMSR fault behavior won't change if the vCPU is migrated to a host with divergent behavior. Note, the "new" kvm_is_supported_user_return_msr() checks do not add new functionality on either SVM or VMX. On SVM, the equivalent was "tsc_aux_uret_slot < 0", and on VMX the check was buried in the vmx_find_uret_msr() call at the find_uret_msr label. Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/include/asm/kvm_host.h | 5 +++++ arch/x86/kvm/svm/svm.c | 24 ---------------------- arch/x86/kvm/vmx/vmx.c | 15 -------------- arch/x86/kvm/x86.c | 36 +++++++++++++++++++++++++++++++++ 4 files changed, 41 insertions(+), 39 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index a4b912f7e427..00fb9efb9984 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1782,6 +1782,11 @@ int kvm_add_user_return_msr(u32 msr); int kvm_find_user_return_msr(u32 msr); int kvm_set_user_return_msr(unsigned index, u64 val, u64 mask); +static inline bool kvm_is_supported_user_return_msr(u32 msr) +{ + return kvm_find_user_return_msr(msr) >= 0; +} + u64 kvm_scale_tsc(struct kvm_vcpu *vcpu, u64 tsc); u64 kvm_read_l1_tsc(struct kvm_vcpu *vcpu, u64 host_tsc); diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index de921935e8de..6c7c6a303cc5 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -2663,12 +2663,6 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) msr_info->data |= (u64)svm->sysenter_esp_hi << 32; break; case MSR_TSC_AUX: - if (tsc_aux_uret_slot < 0) - return 1; - if (!msr_info->host_initiated && - !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP) && - !guest_cpuid_has(vcpu, X86_FEATURE_RDPID)) - return 1; msr_info->data = svm->tsc_aux; break; /* @@ -2885,24 +2879,6 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) svm->sysenter_esp_hi = guest_cpuid_is_intel(vcpu) ? (data >> 32) : 0; break; case MSR_TSC_AUX: - if (tsc_aux_uret_slot < 0) - return 1; - - if (!msr->host_initiated && - !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP) && - !guest_cpuid_has(vcpu, X86_FEATURE_RDPID)) - return 1; - - /* - * Per Intel's SDM, bits 63:32 are reserved, but AMD's APM has - * incomplete and conflicting architectural behavior. Current - * AMD CPUs completely ignore bits 63:32, i.e. they aren't - * reserved and always read as zeros. Emulate AMD CPU behavior - * to avoid explosions if the vCPU is migrated from an AMD host - * to an Intel host. - */ - data = (u32)data; - /* * TSC_AUX is usually changed only during boot and never read * directly. Intercept TSC_AUX instead of exposing it to the diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 26f82f302391..d85ac5876982 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1981,12 +1981,6 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) else msr_info->data = vmx->pt_desc.guest.addr_a[index / 2]; break; - case MSR_TSC_AUX: - if (!msr_info->host_initiated && - !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP) && - !guest_cpuid_has(vcpu, X86_FEATURE_RDPID)) - return 1; - goto find_uret_msr; case MSR_IA32_DEBUGCTLMSR: msr_info->data = vmcs_read64(GUEST_IA32_DEBUGCTL); break; @@ -2302,15 +2296,6 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) else vmx->pt_desc.guest.addr_a[index / 2] = data; break; - case MSR_TSC_AUX: - if (!msr_info->host_initiated && - !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP) && - !guest_cpuid_has(vcpu, X86_FEATURE_RDPID)) - return 1; - /* Check reserved bit, higher 32 bits should be zero */ - if ((data >> 32) != 0) - return 1; - goto find_uret_msr; case MSR_IA32_PERF_CAPABILITIES: if (data && !vcpu_to_pmu(vcpu)->version) return 1; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index adca491d3b4b..896127ea4d4f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1642,6 +1642,30 @@ static int __kvm_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data, * invokes 64-bit SYSENTER. */ data = get_canonical(data, vcpu_virt_addr_bits(vcpu)); + break; + case MSR_TSC_AUX: + if (!kvm_is_supported_user_return_msr(MSR_TSC_AUX)) + return 1; + + if (!host_initiated && + !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP) && + !guest_cpuid_has(vcpu, X86_FEATURE_RDPID)) + return 1; + + /* + * Per Intel's SDM, bits 63:32 are reserved, but AMD's APM has + * incomplete and conflicting architectural behavior. Current + * AMD CPUs completely ignore bits 63:32, i.e. they aren't + * reserved and always read as zeros. Enforce Intel's reserved + * bits check if and only if the guest CPU is Intel, and clear + * the bits in all other cases. This ensures cross-vendor + * migration will provide consistent behavior for the guest. + */ + if (guest_cpuid_is_intel(vcpu) && (data >> 32) != 0) + return 1; + + data = (u32)data; + break; } msr.data = data; @@ -1678,6 +1702,18 @@ int __kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data, if (!host_initiated && !kvm_msr_allowed(vcpu, index, KVM_MSR_FILTER_READ)) return KVM_MSR_RET_FILTERED; + switch (index) { + case MSR_TSC_AUX: + if (!kvm_is_supported_user_return_msr(MSR_TSC_AUX)) + return 1; + + if (!host_initiated && + !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP) && + !guest_cpuid_has(vcpu, X86_FEATURE_RDPID)) + return 1; + break; + } + msr.index = index; msr.host_initiated = host_initiated; From patchwork Tue May 4 17:17:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 12238369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 245A2C433B4 for ; Tue, 4 May 2021 17:18:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0A4F6613BC for ; Tue, 4 May 2021 17:18:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231351AbhEDRTr (ORCPT ); Tue, 4 May 2021 13:19:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232157AbhEDRTU (ORCPT ); Tue, 4 May 2021 13:19:20 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7D86C061352 for ; Tue, 4 May 2021 10:18:18 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id y5-20020a2586050000b02904ee36d3b170so12617284ybk.10 for ; Tue, 04 May 2021 10:18:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=la9oY1bxuLYI2LhjbikVYGCKhsj27XYDlkUaBPeOjcE=; b=jn7iGmXk2tM9WVcHBi3kVdO4RzzftrZfQjnK+KwLY8ldx7zkeKX1Ji8siS6FGY4sOb hweay/CtK/y3qJ+aMkl1CKVsa8O4blH0Z4ObvgtaA2EAlo2umn8s6Yh6lEjq/vkfVOmF AK9du+htAn4roZgYlRNomur8QSNZN5lnsPUhd8pyzyLcBrUpSHk5yIC2W8XPaQ8UU0Wa MyUHUKhTVHjSOjjUBrcay/jqEBB7LOLLdxPljAg5aA0jtOfrTs3W+zClQqkcPIj//3SO rJJBBQCpm9MsJldnED0rOfmLYLzH9G9mmBS54TnM4bqZvRA+SMbAbru+RuE520OCxPQ7 5mHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=la9oY1bxuLYI2LhjbikVYGCKhsj27XYDlkUaBPeOjcE=; b=SVtyR88RFQrVVJsyKdmjlFMTVLUnfe5un5AEGBSWtTGfwjNkLGbbSY5cZ0OCTlRcJU MmeLpIEL8ueUSJg070y6cfXU0dtJq7XfK7S2bukUWAKehvLqYlRibYyaVZOPMvB1T3HD CKdWRaqqUmaqz0eKh8HIVQinh2XgpN/ypiDO5gzAy4cw392VxS2r8T0n98TU7mMUfnlJ h2+70y7Fy3Urt6mONYbk1snqmXdjjpjwWSyZn30FZUrt+EYKF7swsBWdSYFS7uOgG9Pc HR1DUolSbgr+OgPKxam3a6jcPJlqir1CNZ7ro8V4PnXD0yGC+XwwPrn0nTvjEKK1n++2 ktGg== X-Gm-Message-State: AOAM53055vYC6pY4IZvy6UZ4Uqm8jS9mStazUO7kf3VVFSb74NyStmaJ Ft350ClS3fKimumlwfKbJ4126P/8dtw= X-Google-Smtp-Source: ABdhPJwwHw598d//FTS6A9v+yJFz6d3uPtTa/zkCcvmZHBETLX5230DxHwzzjkufCXdGKaG2q1I+ZkEAJBc= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:df57:48cb:ea33:a156]) (user=seanjc job=sendgmr) by 2002:a05:6902:120a:: with SMTP id s10mr34885283ybu.91.1620148698033; Tue, 04 May 2021 10:18:18 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 4 May 2021 10:17:34 -0700 In-Reply-To: <20210504171734.1434054-1-seanjc@google.com> Message-Id: <20210504171734.1434054-16-seanjc@google.com> Mime-Version: 1.0 References: <20210504171734.1434054-1-seanjc@google.com> X-Mailer: git-send-email 2.31.1.527.g47e6f16901-goog Subject: [PATCH 15/15] KVM: x86: Hide RDTSCP and RDPID if MSR_TSC_AUX probing failed From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Xiaoyao Li , Reiji Watanabe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org If probing MSR_TSC_AUX failed, hide RDTSCP and RDPID, and WARN if either feature was reported as supported. In theory, such a scenario should never happen as both Intel and AMD state that MSR_TSC_AUX is available if RDTSCP or RDPID is supported. But, KVM injects #GP on MSR_TSC_AUX accesses if probing failed, faults on WRMSR(MSR_TSC_AUX) may be fatal to the guest (because they happen during early CPU bringup), and KVM itself has effectively misreported RDPID support in the past. Note, this also has the happy side effect of omitting MSR_TSC_AUX from the list of MSRs that are exposed to userspace if probing the MSR fails. Signed-off-by: Sean Christopherson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/cpuid.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index c96f79c9fff2..bf0f74ce4974 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -567,6 +567,21 @@ void kvm_set_cpu_caps(void) F(ACE2) | F(ACE2_EN) | F(PHE) | F(PHE_EN) | F(PMM) | F(PMM_EN) ); + + /* + * Hide RDTSCP and RDPID if either feature is reported as supported but + * probing MSR_TSC_AUX failed. This is purely a sanity check and + * should never happen, but the guest will likely crash if RDTSCP or + * RDPID is misreported, and KVM has botched MSR_TSC_AUX emulation in + * the past, e.g. the sanity check may fire if this instance of KVM is + * running as L1 on top of an older, broken KVM. + */ + if (WARN_ON((kvm_cpu_cap_has(X86_FEATURE_RDTSCP) || + kvm_cpu_cap_has(X86_FEATURE_RDPID)) && + !kvm_is_supported_user_return_msr(MSR_TSC_AUX))) { + kvm_cpu_cap_clear(X86_FEATURE_RDTSCP); + kvm_cpu_cap_clear(X86_FEATURE_RDPID); + } } EXPORT_SYMBOL_GPL(kvm_set_cpu_caps);