From patchwork Tue Apr 17 16:26:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 10345855 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4B72960233 for ; Tue, 17 Apr 2018 16:26:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 37C5D1FFE4 for ; Tue, 17 Apr 2018 16:26:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2AEA227B2F; Tue, 17 Apr 2018 16:26:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 539F11FFE4 for ; Tue, 17 Apr 2018 16:26:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755344AbeDQQ0S (ORCPT ); Tue, 17 Apr 2018 12:26:18 -0400 Received: from mga17.intel.com ([192.55.52.151]:59886 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754432AbeDQQ0P (ORCPT ); Tue, 17 Apr 2018 12:26:15 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Apr 2018 09:26:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,464,1517904000"; d="scan'208";a="32657935" Received: from orsmsx105.amr.corp.intel.com ([10.22.225.132]) by fmsmga007.fm.intel.com with ESMTP; 17 Apr 2018 09:26:14 -0700 Received: from orsmsx154.amr.corp.intel.com (10.22.226.12) by ORSMSX105.amr.corp.intel.com (10.22.225.132) with Microsoft SMTP Server (TLS) id 14.3.319.2; Tue, 17 Apr 2018 09:26:14 -0700 Received: from orsmsx114.amr.corp.intel.com ([169.254.8.92]) by ORSMSX154.amr.corp.intel.com ([169.254.11.191]) with mapi id 14.03.0319.002; Tue, 17 Apr 2018 09:26:14 -0700 From: "Christopherson, Sean J" To: Paolo Bonzini , Zdenek Kaspar , "kvm@vger.kernel.org" Subject: RE: 4.16 kernel: vmwrite error: reg 401e value 2021 (err 12) Thread-Topic: 4.16 kernel: vmwrite error: reg 401e value 2021 (err 12) Thread-Index: AQHT1h0iTynj0ZDLlkC6qEHZL6ljfqQFEk8AgACCZgD//4xMQA== Date: Tue, 17 Apr 2018 16:26:13 +0000 Message-ID: <37306EFA9975BE469F115FDE982C075BCE93CC3B@ORSMSX114.amr.corp.intel.com> References: <3e32aef9-3c06-7ba4-6284-3b10daa28136@gmail.com> <37306EFA9975BE469F115FDE982C075BCE93CBEA@ORSMSX114.amr.corp.intel.com> <254709eb-f59a-4d0a-dfa0-55f0b58e0ff9@redhat.com> In-Reply-To: <254709eb-f59a-4d0a-dfa0-55f0b58e0ff9@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiY2QwMWE4OTYtMTVhMy00YjE1LTk2OWItOGRkNzgzZTNiYzcxIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjIuNS4xOCIsIlRydXN0ZWRMYWJlbEhhc2giOiJkT0VTeWVjUXltYk1xSlVTRjRqd0ZEVWQ0RzkwM3BtYzVpUEIzeSs2YVc2Z25nWHpLeFQzQ2FvQXNvelJiYmdFIn0= dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [10.22.254.138] MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Tue, 2018-04-17, Paolo Bonzini wrote: > On 17/04/2018 17:46, Christopherson, Sean J wrote: > > On Tue, 2018-04-17, Zdenek Kaspar wrote: > >> Hello, I did quick test with latest stable kernel (4.16.2) and got tons > >> of vmwrite errors immediately when starting VM: > > > > Code related to UMIP emulation is effectively doing an unconditional > > RMW on SECONDARY_VM_EXEC_CONTROL, which isn't guaranteed to exist on > > older processors. KVM already ensures it only advertises UMIP (via > > emulation) when SECONDARY_EXEC_DESC can be set, i.e. KVM is already > > implicitly checking for SECONDARY_VM_EXEC_CONTROL, so fixing the bug > > is just a matter of omitting the unneeded VMREAD/VMWRITE sequence. > > Thanks for the report! > > This should be a fix: > > diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c > index aa66ccd6ed6c..c5dd185825c7 100644 > --- a/arch/x86/kvm/vmx.c > +++ b/arch/x86/kvm/vmx.c > @@ -4767,14 +4767,16 @@ static int vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) > else > hw_cr4 |= KVM_PMODE_VM_CR4_ALWAYS_ON; > > - if ((cr4 & X86_CR4_UMIP) && !boot_cpu_has(X86_FEATURE_UMIP)) { > - vmcs_set_bits(SECONDARY_VM_EXEC_CONTROL, > - SECONDARY_EXEC_DESC); > - hw_cr4 &= ~X86_CR4_UMIP; > - } else if (!is_guest_mode(vcpu) || > - !nested_cpu_has2(get_vmcs12(vcpu), SECONDARY_EXEC_DESC)) > - vmcs_clear_bits(SECONDARY_VM_EXEC_CONTROL, > - SECONDARY_EXEC_DESC); > + if (!boot_cpu_has(X86_FEATURE_UMIP) && vmx_umip_emulated()) { > + if (cr4 & X86_CR4_UMIP) { > + vmcs_set_bits(SECONDARY_VM_EXEC_CONTROL, > + SECONDARY_EXEC_DESC); > + hw_cr4 &= ~X86_CR4_UMIP; > + } else if (!is_guest_mode(vcpu) || > + !nested_cpu_has2(get_vmcs12(vcpu), SECONDARY_EXEC_DESC)) > + vmcs_clear_bits(SECONDARY_VM_EXEC_CONTROL, > + SECONDARY_EXEC_DESC); > + } > > if (cr4 & X86_CR4_VMXE) { > /* > > I'll test it and send the patch more formally. Below is what I was thinking for a patch. We should avoid the VMREAD/VMWRITE when possible even when we're emulating UMIP. From: Sean Christopherson Subject: [PATCH] KVM: vmx: update SECONDARY_EXEC_DESC only if CR4.UMIP changes Update SECONDARY_EXEC_DESC in SECONDARY_VM_EXEC_CONTROL for UMIP emulation if and only if CR4.UMIP is being modified and UMIP is not supported by hardware, i.e. we're emulating UMIP. If CR4.UMIP is not being changed then it's safe to assume that the previous invocation of vmx_set_cr4() correctly set SECONDARY_EXEC_DESC, i.e. the desired value is already the current value. This avoids unnecessary VMREAD/VMWRITE to SECONDARY_VM_EXEC_CONTROL, which is critical as not all processors support SECONDARY_VM_EXEC_CONTROL. WARN once and signal a fault if CR4.UMIP is changing and UMIP can't be emulated, i.e. SECONDARY_EXEC_DESC can't be set. Prior checks should prevent setting UMIP if it can't be emulated, i.e. UMIP shouldn't have been advertised to the guest if it can't be emulated, regardless of whether or not UMIP is supported in bare metal. Fixes: 0367f205a3b7 ("KVM: vmx: add support for emulating UMIP") Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx.c | 34 ++++++++++++++++++++-------------- 1 file changed, 20 insertions(+), 14 deletions(-) -- > > Thanks, > > Paolo diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index aafcc9881e88..31b36b9801bb 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -1494,6 +1494,12 @@ static inline bool cpu_has_vmx_vmfunc(void) SECONDARY_EXEC_ENABLE_VMFUNC; } +static bool vmx_umip_emulated(void) +{ + return vmcs_config.cpu_based_2nd_exec_ctrl & + SECONDARY_EXEC_DESC; +} + static inline bool report_flexpriority(void) { return flexpriority_enabled; @@ -4776,14 +4782,20 @@ static int vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) else hw_cr4 |= KVM_PMODE_VM_CR4_ALWAYS_ON; - if ((cr4 & X86_CR4_UMIP) && !boot_cpu_has(X86_FEATURE_UMIP)) { - vmcs_set_bits(SECONDARY_VM_EXEC_CONTROL, - SECONDARY_EXEC_DESC); - hw_cr4 &= ~X86_CR4_UMIP; - } else if (!is_guest_mode(vcpu) || - !nested_cpu_has2(get_vmcs12(vcpu), SECONDARY_EXEC_DESC)) - vmcs_clear_bits(SECONDARY_VM_EXEC_CONTROL, - SECONDARY_EXEC_DESC); + if (((cr4 ^ kvm_read_cr4(vcpu)) & X86_CR4_UMIP) && + !boot_cpu_has(X86_FEATURE_UMIP)) { + if (WARN_ON_ONCE(!vmx_umip_emulated())) + return 1; + + if (cr4 & X86_CR4_UMIP) { + vmcs_set_bits(SECONDARY_VM_EXEC_CONTROL, + SECONDARY_EXEC_DESC); + hw_cr4 &= ~X86_CR4_UMIP; + } else if (!is_guest_mode(vcpu) || + !nested_cpu_has2(get_vmcs12(vcpu), SECONDARY_EXEC_DESC)) + vmcs_clear_bits(SECONDARY_VM_EXEC_CONTROL, + SECONDARY_EXEC_DESC); + } if (cr4 & X86_CR4_VMXE) { /* @@ -9512,12 +9524,6 @@ static bool vmx_xsaves_supported(void) SECONDARY_EXEC_XSAVES; } -static bool vmx_umip_emulated(void) -{ - return vmcs_config.cpu_based_2nd_exec_ctrl & - SECONDARY_EXEC_DESC; -} - static void vmx_recover_nmi_blocking(struct vcpu_vmx *vmx) { u32 exit_intr_info;