From patchwork Fri Sep 6 04:34:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13793204 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B384F4594A for ; Fri, 6 Sep 2024 04:34:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597261; cv=none; b=d5ivpAd0TjSU9ZVU9H/mPmmVE4xbCqTjHH3piLUpy5KcEoyXOTHANv/gD6rAxBDm5D7wzDGa4eMDLemxKq30pNnNlix8SZH9NVKSHFKrxEgCImSyxkBdZoMfq4Y34RlcFBX4ahJyQePk/XC7CWENHHttUrISUw1KAcM+rWosLTM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597261; c=relaxed/simple; bh=IvNHY4s2sMwGIf0YtwnB+tocpqjPSUFN5NVhawuBXGY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RpctkBdSELp4BYhdtZ+7i+h/cb+Tjs3LsRsMOU1ziOOGkHKEkns8nueCUclQfsROlf9SQNkFrScPbjk6w/zKeQ6ag6E4oQdYGhRXdMnYuk48lsJfF7vC6dSTuaBg9TRMC/Oia5RO7HKnsSPmhQNzbHAjJ6tI12k9XP97odx8hgk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=GSRoYNGE; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GSRoYNGE" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6c3982a0c65so70077697b3.1 for ; Thu, 05 Sep 2024 21:34:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725597258; x=1726202058; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=mCaD0TEOBVOQwGb6+w3Vfrkr+Z/WBNGokOq9Bgi0c+0=; b=GSRoYNGEPKl/6E7LBNLR3g7V2gfPZG2wSWUEuMD+ZLeZpvpBBoRLHVx7xDraG1yfJH HH/um61pKQehkoU60FWYxt+91VhyGNalT5PkBXodSgO4EDIT4Bbn5ZNWuJMOzyVbUByN DuJXRuJEpTxALSodktKG5QPNjWtKk4FlJXN9AwLUkeH3PTunAQrirViLmQ/Z7/EBh0mu BndDgRh/ANP81IkGHfE9LsM52BW39zVMQbrWsNJGLpl5xZX0ZDa9cStj37AtdksvifFB nvC1/SmgbEdUNYd8PY+5S6M7sD9fdGocmERaIaPTiHN+6pS6y9L4sjJuwDmmmI/eLWoe gQcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725597258; x=1726202058; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=mCaD0TEOBVOQwGb6+w3Vfrkr+Z/WBNGokOq9Bgi0c+0=; b=lF0vXZcqTOfWY7lCANbScy64tnGiHE9jWY23hITCjcoNils1+PpDRVO8OT6+Q88KCg ZiUejNFbWdL9Nu7Ue+/IKFvpbDF6/fzxt+TK2ZGswysBtVJQYZjKEYbY66KdvSBwOSdi +yO5QCh+BUP36pWom5+Ip+kZsOsdjNqrDWjpA9eKSBSY1PGy3YD8Mtim2MB97vL5WMf7 bb/DFJtIcmQfaLZhNI5RJrjNnPz1BRNte5zlGb6tt0gcaFE//kdKvEiOvhTjd4p6GAAU dEHtGahq8/zpDnbK9O8W0yqwElJ0Nzvr+wvpYAg8TYuZiKLUroTEVz0YK6BneQ15fr+D 4JaQ== X-Gm-Message-State: AOJu0Yw0bZ6k44djPL2HPIGs9FOOyAJwdcCuTWyxoVyty4O4GSJuNJ97 8gqtQpJIG9s7OzXafSEOj3jmCumvNszD3s9XiuxdHR6zQFrZHOsqzMyv76x8ZWJFVoii7KgS3gV QvQ== X-Google-Smtp-Source: AGHT+IGCDgcglWhh/eqririlfxwRjfWcrURdZg+FkjDnBxo0+i6VcRR3gAjFm5Rm67GVxheJV7Gf0Avr2mM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:690c:fcf:b0:627:a671:8805 with SMTP id 00721157ae682-6db4512db7bmr1018207b3.3.1725597258694; Thu, 05 Sep 2024 21:34:18 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 5 Sep 2024 21:34:07 -0700 In-Reply-To: <20240906043413.1049633-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240906043413.1049633-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240906043413.1049633-2-seanjc@google.com> Subject: [PATCH v2 1/7] KVM: x86: Move "ack" phase of local APIC IRQ delivery to separate API From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Nathan Chancellor , Chao Gao , Zeng Guang Split the "ack" phase, i.e. the movement of an interrupt from IRR=>ISR, out of kvm_get_apic_interrupt() and into a separate API so that nested VMX can acknowledge a specific interrupt _after_ emulating a VM-Exit from L2 to L1. To correctly emulate nested posted interrupts while APICv is active, KVM must: 1. find the highest pending interrupt. 2. check if that IRQ is L2's notification vector 3. emulate VM-Exit if the IRQ is NOT the notification vector 4. ACK the IRQ in L1 _after_ VM-Exit When APICv is active, the process of moving the IRQ from the IRR to the ISR also requires a VMWRITE to update vmcs01.GUEST_INTERRUPT_STATUS.SVI, and so acknowledging the interrupt before switching to vmcs01 would result in marking the IRQ as in-service in the wrong VMCS. KVM currently fudges around this issue by doing kvm_get_apic_interrupt() smack dab in the middle of emulating VM-Exit, but that hack doesn't play nice with nested posted interrupts, as notification vector IRQs don't trigger a VM-Exit in the first place. Cc: Nathan Chancellor Signed-off-by: Sean Christopherson --- arch/x86/kvm/lapic.c | 17 +++++++++++++---- arch/x86/kvm/lapic.h | 1 + 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index a7172ba59ad2..ff63ef8163a9 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2924,14 +2924,13 @@ void kvm_inject_apic_timer_irqs(struct kvm_vcpu *vcpu) } } -int kvm_get_apic_interrupt(struct kvm_vcpu *vcpu) +void kvm_apic_ack_interrupt(struct kvm_vcpu *vcpu, int vector) { - int vector = kvm_apic_has_interrupt(vcpu); struct kvm_lapic *apic = vcpu->arch.apic; u32 ppr; - if (vector == -1) - return -1; + if (WARN_ON_ONCE(vector < 0 || !apic)) + return; /* * We get here even with APIC virtualization enabled, if doing @@ -2959,6 +2958,16 @@ int kvm_get_apic_interrupt(struct kvm_vcpu *vcpu) __apic_update_ppr(apic, &ppr); } +} +EXPORT_SYMBOL_GPL(kvm_apic_ack_interrupt); + +int kvm_get_apic_interrupt(struct kvm_vcpu *vcpu) +{ + int vector = kvm_apic_has_interrupt(vcpu); + + if (vector != -1) + kvm_apic_ack_interrupt(vcpu, vector); + return vector; } diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index 7ef8ae73e82d..db80a2965b9c 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -88,6 +88,7 @@ int kvm_create_lapic(struct kvm_vcpu *vcpu); void kvm_free_lapic(struct kvm_vcpu *vcpu); int kvm_apic_has_interrupt(struct kvm_vcpu *vcpu); +void kvm_apic_ack_interrupt(struct kvm_vcpu *vcpu, int vector); int kvm_apic_accept_pic_intr(struct kvm_vcpu *vcpu); int kvm_get_apic_interrupt(struct kvm_vcpu *vcpu); int kvm_apic_accept_events(struct kvm_vcpu *vcpu); From patchwork Fri Sep 6 04:34:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13793205 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EE6A54656 for ; Fri, 6 Sep 2024 04:34:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597264; cv=none; b=ENFVND1+jOfZZs9yAyRL86Z4sYE9tNp+FbWL4GDQjxxF6sGhji4Fd1zLP0YW1AYLSzClMqn54kfKz+rmvrx92QcKcSEr+gVwuSj2C6XMyM5nG1GpB16YzYfObRz89vvxx5FtT6Whyvj+CyWuXKBw51Tjda0HzIcG/ba/u7Sv6Bo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597264; c=relaxed/simple; bh=eDNwSm0+gbgy/kZmZL0w/jpknN55gI59zurj7zb2dhI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ppWCzpIV/c2O9D7w6FyzfAnqDd2fSmmexMnuMbQMteBLkbKXRoklNLU31fceJD6wrffVW6xTeqFgQljsK1IgAXzIlEev5lcLTUBI+UU0xpeqYK3ygA5ewKiH/WuFutj9PTlVSYjgjP9Rqi+54g+VTiCAD+b2tc5+ql8Hr5w83qM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=z6qUfcVF; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="z6qUfcVF" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6d4bd76f5a8so52557087b3.0 for ; Thu, 05 Sep 2024 21:34:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725597260; x=1726202060; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=mJlh6I1bqg+VJjot9mgJ443ha3usj4WF5fwSb0yKCRc=; b=z6qUfcVFty3w5sjiH0ssZ+6fQW4EL0rVjaHewPvndsjgjgDuMkM2lUUkPu45JY0yuA o2a1AUITRMKPopdeiKTpHqLoIMHEX7Fx5PE6KmQaPgirjXZ97RKx4AofdbRTHi8Lap3R h0B66iT6z0Bsmvy2VnYfnnXNHQwDxiQJN/ymw/eVZAxuc1rFwQrtJBjk13z4LfqicGof cio6B8Y+nMgBOymaWEfXedKSdEQQmMceZ0K9uLEnLPe2IWTreR3iRfVBTyuQsIdIjwzG eN8S8Byc/Hga+FosILUEvGL9Ku/pyGxwsP//RmAnAN7EDuOFIjQChZjDeuTFbfu+F9pT /Ykw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725597260; x=1726202060; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=mJlh6I1bqg+VJjot9mgJ443ha3usj4WF5fwSb0yKCRc=; b=oUnWvEZ+WldVBFJQSl/NRNc2ejP1dQQ6n3ey6l4jKnjNA5d7rrB3Wwzd4RnatVHrgs P4px8g507LYnEkW4iyfX1ND9X09xOR2OiUjAPYr62svizvvnEVojyZniNIOgQVkPqpfm IEd1otIJAY9peDYnuyhzTGs2hemyXuw+pn16fYRdurueOdspN8iZFR2fHDIsOk237WnQ B71TuYtSOtiuNIx2Whmykzf5L1DscDRyItXyy3uk0+y2rb6z3bNeRkZ6mXs8UO7wH3aR XRLFeRD6WGnq8fY6gXCW5cSrS57FV8kRc862zVqfZOraUzvTkLF+FElh5Nr6PMMz5UOy LQ3A== X-Gm-Message-State: AOJu0Yz8w1FFeXjueFCHejH/p0kPA82wGzBbZzW8Sutk/uPeh2OEVScr 07T6rk2waxmar4VlN9R9eDP4AV7qLKpTyZMfXIAgJJf8GZO+AWhCChwFyJcQwYt9z8WPG7cAgb4 wOQ== X-Google-Smtp-Source: AGHT+IEPohibJVK7CmOK3wntKFvrjLv1A44TW/i8I3yEYDsA5G0xDa+I83WX8upp1fWDeFMg/T0IzHbDDpU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:690c:2c11:b0:690:8ad7:55f9 with SMTP id 00721157ae682-6db44d6c217mr285207b3.2.1725597260254; Thu, 05 Sep 2024 21:34:20 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 5 Sep 2024 21:34:08 -0700 In-Reply-To: <20240906043413.1049633-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240906043413.1049633-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240906043413.1049633-3-seanjc@google.com> Subject: [PATCH v2 2/7] KVM: nVMX: Get to-be-acknowledge IRQ for nested VM-Exit at injection site From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Nathan Chancellor , Chao Gao , Zeng Guang Move the logic to get the to-be-acknowledge IRQ for a nested VM-Exit from nested_vmx_vmexit() to vmx_check_nested_events(), which is subtly the one and only path where KVM invokes nested_vmx_vmexit() with EXIT_REASON_EXTERNAL_INTERRUPT. A future fix will perform a last-minute check on L2's nested posted interrupt notification vector, just before injecting a nested VM-Exit. To handle that scenario correctly, KVM needs to get the interrupt _before_ injecting VM-Exit, as simply querying the highest priority interrupt, via kvm_cpu_has_interrupt(), would result in TOCTOU bug, as a new, higher priority interrupt could arrive between kvm_cpu_has_interrupt() and kvm_cpu_get_interrupt(). Unfortunately, simply moving the call to kvm_cpu_get_interrupt() doesn't suffice, as a VMWRITE to GUEST_INTERRUPT_STATUS.SVI is hiding in kvm_get_apic_interrupt(), and acknowledging the interrupt before nested VM-Exit would cause the VMWRITE to hit vmcs02 instead of vmcs01. Open code a rough equivalent to kvm_cpu_get_interrupt() so that the IRQ is acknowledged after emulating VM-Exit, taking care to avoid the TOCTOU issue described above. Opportunistically convert the WARN_ON() to a WARN_ON_ONCE(). If KVM has a bug that results in a false positive from kvm_cpu_has_interrupt(), spamming dmesg won't help the situation. Note, nested_vmx_reflect_vmexit() can never reflect external interrupts as they are always "wanted" by L0. Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/irq.c | 3 ++- arch/x86/kvm/vmx/nested.c | 36 ++++++++++++++++++++++++--------- 3 files changed, 30 insertions(+), 10 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 950a03e0181e..d66d562c0ab3 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2251,6 +2251,7 @@ int kvm_cpu_has_injectable_intr(struct kvm_vcpu *v); int kvm_cpu_has_interrupt(struct kvm_vcpu *vcpu); int kvm_cpu_has_extint(struct kvm_vcpu *v); int kvm_arch_interrupt_allowed(struct kvm_vcpu *vcpu); +int kvm_cpu_get_extint(struct kvm_vcpu *v); int kvm_cpu_get_interrupt(struct kvm_vcpu *v); void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event); diff --git a/arch/x86/kvm/irq.c b/arch/x86/kvm/irq.c index 3d7eb11d0e45..810da99ff7ed 100644 --- a/arch/x86/kvm/irq.c +++ b/arch/x86/kvm/irq.c @@ -108,7 +108,7 @@ EXPORT_SYMBOL_GPL(kvm_cpu_has_interrupt); * Read pending interrupt(from non-APIC source) * vector and intack. */ -static int kvm_cpu_get_extint(struct kvm_vcpu *v) +int kvm_cpu_get_extint(struct kvm_vcpu *v) { if (!kvm_cpu_has_extint(v)) { WARN_ON(!lapic_in_kernel(v)); @@ -131,6 +131,7 @@ static int kvm_cpu_get_extint(struct kvm_vcpu *v) } else return kvm_pic_read_irq(v->kvm); /* PIC */ } +EXPORT_SYMBOL_GPL(kvm_cpu_get_extint); /* * Read pending interrupt vector and intack. diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 2392a7ef254d..e6af5f1d3b61 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -4284,11 +4284,37 @@ static int vmx_check_nested_events(struct kvm_vcpu *vcpu) } if (kvm_cpu_has_interrupt(vcpu) && !vmx_interrupt_blocked(vcpu)) { + int irq; + if (block_nested_events) return -EBUSY; if (!nested_exit_on_intr(vcpu)) goto no_vmexit; - nested_vmx_vmexit(vcpu, EXIT_REASON_EXTERNAL_INTERRUPT, 0, 0); + + if (!nested_exit_intr_ack_set(vcpu)) { + nested_vmx_vmexit(vcpu, EXIT_REASON_EXTERNAL_INTERRUPT, 0, 0); + return 0; + } + + irq = kvm_cpu_get_extint(vcpu); + if (irq != -1) { + nested_vmx_vmexit(vcpu, EXIT_REASON_EXTERNAL_INTERRUPT, + INTR_INFO_VALID_MASK | INTR_TYPE_EXT_INTR | irq, 0); + return 0; + } + + irq = kvm_apic_has_interrupt(vcpu); + WARN_ON_ONCE(irq < 0); + + nested_vmx_vmexit(vcpu, EXIT_REASON_EXTERNAL_INTERRUPT, + INTR_INFO_VALID_MASK | INTR_TYPE_EXT_INTR | irq, 0); + + /* + * ACK the interrupt _after_ emulating VM-Exit, as the IRQ must + * be marked as in-service in vmcs01.GUEST_INTERRUPT_STATUS.SVI + * if APICv is active. + */ + kvm_apic_ack_interrupt(vcpu, irq); return 0; } @@ -4969,14 +4995,6 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; if (likely(!vmx->fail)) { - if ((u16)vm_exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT && - nested_exit_intr_ack_set(vcpu)) { - int irq = kvm_cpu_get_interrupt(vcpu); - WARN_ON(irq < 0); - vmcs12->vm_exit_intr_info = irq | - INTR_INFO_VALID_MASK | INTR_TYPE_EXT_INTR; - } - if (vm_exit_reason != -1) trace_kvm_nested_vmexit_inject(vmcs12->vm_exit_reason, vmcs12->exit_qualification, From patchwork Fri Sep 6 04:34:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13793206 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5403B84A51 for ; Fri, 6 Sep 2024 04:34:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597264; cv=none; b=P3S/zVQY5W4p4cY4JyHZdrOitzcwr8hTGULhImE6q3RonlhMnfBjQWM0uQ59jubFdGs8OmEJBm1Ej77DgQAOmagXw1Xqkyda6rMxqJgEmwoCMFvWm4pp4FZaEaRxpSLLvXiWfm+tSpXuK2KhJznJtW1lLQ76AO9/RNv6zZ5Ryt8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597264; c=relaxed/simple; bh=vUO7vGmYGYbyr6uF29dDmJSIjGj15xrKsxlgZD+fbfg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=i3j2W891NqIjPCEVXMb0s4G6KUlR9ym/aLmSkxFIMG+zZK7nyKZOxLVizvAxY/TLuKKpk0tGgOWgGUlEpLYVBWoeTJtdt0lUinDB7SNu21oEjSPX2VsbTcYQOgrHDGjll0AcXoXUcQIH7iSqnTyEHPJgbVPf1/gau4VnKy+eids= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FUPefJP/; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FUPefJP/" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2053f49d0c9so32426985ad.1 for ; Thu, 05 Sep 2024 21:34:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725597263; x=1726202063; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=vgikLRROm580u43MLI6MsYbRQEvUit5RnsfmT63jtwE=; b=FUPefJP/mkwt7ASgqON3SD6fgvjFdLDCw9mLfc8r2DxQjNQT3Y0uu/K+G+KTwUsijd qCues+6mUfE5EtMqwGhcGBnj4oMKTYKTrgQTJiK3YcFC3QX/sW76yM0FsjHZcn73es1I 1osHDH++r5teDW1cB4uOXvpZDZ02NdblZMixcbyPg6muPZenSDfjF9WpyiaOi8D3iJsw Gp7p7+0l2fPQ26TTTSQdsbwVGeoi2tw4+KeahFw2/YophgQgjIQzAL6G1OcKLt6zdBAQ vWMJRLaVAbDh7DrxWX4o4Hvtd+NXuAnUP19ZKNLgLPDFHS/kIdLvjTGNijavviUkdQK+ vbAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725597263; x=1726202063; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vgikLRROm580u43MLI6MsYbRQEvUit5RnsfmT63jtwE=; b=gzm8QtAMuUwsPzdNrIX1U0X+w9lel/txl4yO4MFuloo8wZlmJ84FAgel8YxiMjXVZX adi8+RmAqlCh1aluMVb2rkEzxFg81U+RxTbLUg9gb8GivtUELNmPmD+AcHKEEUjcZi9P EmLc9SGDiAd1DjtTUFaMuOCTxiVHI9Suc9A89ZHlEuiiXQ59294wuqnraB1R7guQfuVu tIDKpcUKnQiKyODf3UTd3I70NSIO/flmjBYVR33lC5ldEY6bxTUomXUaBwL6Taz35gyx Lu0MYxdxAyyxRxNRHK5k/7l4bk5rzL14ZOw/UOgpXbMj9whwPnmgdJxjorDYI+D0vrlB Wfow== X-Gm-Message-State: AOJu0YzNq4W0g9UEuXFIoFHTPlXbKePpDGx5n4juqWkzlHXkl1B0s4Hw ZEWBSfk5fmgbiI06ufVsnqrjkDVz+lJ2PAAuUaW19qNNnyHSM9msY/BrxXXP9D1Uo4iGRoatPDv yaw== X-Google-Smtp-Source: AGHT+IEyEFfjoRDNJyVIEZ+IVKjmpudL0fgn44x8C5JkT3Kr/DFcFMcxqut3wtjuTeyUVblYq49w0HSFXXU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:1c5:b0:206:8dca:1b1d with SMTP id d9443c01a7336-206eeba5a3dmr964925ad.4.1725597262568; Thu, 05 Sep 2024 21:34:22 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 5 Sep 2024 21:34:09 -0700 In-Reply-To: <20240906043413.1049633-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240906043413.1049633-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240906043413.1049633-4-seanjc@google.com> Subject: [PATCH v2 3/7] KVM: nVMX: Suppress external interrupt VM-Exit injection if there's no IRQ From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Nathan Chancellor , Chao Gao , Zeng Guang In the should-be-impossible scenario that kvm_cpu_get_interrupt() doesn't return a valid vector after checking kvm_cpu_has_interrupt(), skip VM-Exit injection to reduce the probability of crashing/confusing L1. Now that KVM gets the IRQ _before_ calling nested_vmx_vmexit(), squashing the VM-Exit injection is trivial since there are no actions that need to be undone. Reviewed-by: Chao Gao Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index e6af5f1d3b61..6b7e0ab0e45e 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -4304,7 +4304,8 @@ static int vmx_check_nested_events(struct kvm_vcpu *vcpu) } irq = kvm_apic_has_interrupt(vcpu); - WARN_ON_ONCE(irq < 0); + if (WARN_ON_ONCE(irq < 0)) + goto no_vmexit; nested_vmx_vmexit(vcpu, EXIT_REASON_EXTERNAL_INTERRUPT, INTR_INFO_VALID_MASK | INTR_TYPE_EXT_INTR | irq, 0); From patchwork Fri Sep 6 04:34:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13793207 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D7D2139D03 for ; Fri, 6 Sep 2024 04:34:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597267; cv=none; b=YGw1uBKXCF5mChq0u64IlTryUWmcSdR3xfKJcBY/wOU+NVQHCw0wjFKEubutghzQSjNMJuODXv9Hk+w4J6UVhM2ImPtEWXwmyn41lNBuHL48IrCR5z8B3xJkkxUDvoqZ+pifOU+aAWS1ny/0mk5Hmf9CfUk2nYJIV4A4eG8UPeY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597267; c=relaxed/simple; bh=4MxbT7g3aljkXYP/fW+/iJXROczzlwmWrxw5houVUTk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=DD4JhNSaOEZCAkAA6Ts7R7RRtmY6i2ilGC/NkcgZ6nNFpCFda37lGKBpjVhCHwadJ2wi+p4ImziTo2CbwliFlTUrcfSkU2bzDi4FB6nGOo2MHcH0TbLB5dCkkbEazekELRnWu9abUBuMt4eC7I3zzAniNx4ZOqwLS6kSVvxnxuc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=21Ht/jGS; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="21Ht/jGS" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7d4fc1a2bb7so2223005a12.1 for ; Thu, 05 Sep 2024 21:34:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725597265; x=1726202065; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=uJLB+hmA2IZBXvwBsppVhdrlAWLw+G5oD+o2j3BJ3I4=; b=21Ht/jGSmKnW6fbOPcXoEZImpHcwhBDCzwC8KNC53hqCtes7lW461AbgeVAA1VvBfa qkCj9+2Db9Bnlobh3vKKVl8P9LJgRapazgBmPtM2Pzqakw/Rz9qAAYxcdd8428h34WCk tvEs9d0geclRNyBXKxU/KtpkReDECOtnf8XBpwXBPntuBCNrouu4LnC+REuCR9NEw2Ib wvj0RDXc8/irtd0/zrxOMuJzub1NxB7580WvpjQyPRF/Ly7a0vHa7wq7LtTOMEsIMcGp B56M/+T+feZ7w+/Z09ywbFMc7TKy5gz4z2L5pMDYHQxej3K8cMoVkrF/5P0a2PAg1z5G zB0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725597265; x=1726202065; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uJLB+hmA2IZBXvwBsppVhdrlAWLw+G5oD+o2j3BJ3I4=; b=Htw194ODaUCZsGoRXkuYM0OZgSlVakmhpHnGtqReGAnHO61kxiVQjtWOdjZbnsLPTY LfL/7VffTtAR9Gl+5R+YwhPVnNywNtxRR2MqnM58CP+vqyoky8y7Ziy8IKLxtFimWjOu M81jv02b3mOPIg2EBnmhPU2NNt9SkTrxF0m4D7vYlrrIABP96HUJIo05C3o+5cmEzXxk 9htQ5VF4FP96jpaeooZVtzJN2VPF98S3SORwkNYsfqrejhRwxEtmW10xRx2YvJ0msEBZ 3/axmoIRYrlMjJztyr0F7yawyQ/iTfk1VPG+ata5UED4h0i+fExG2c+QKdADIjKxOk0W UAHA== X-Gm-Message-State: AOJu0Yy9UlXkSWpvGxfwF7CVdFKzXOhOKpCRTEtanTdOC9rOemQXuO0G upd+oV6z89S94IbNSXfRWY2p3t4GNbNhRX66cG5FEfagCFudiU+xvlkxhYUqXrGA7oUxV8zkA+u r7g== X-Google-Smtp-Source: AGHT+IFNTtl3T3l+R72HDbqkIu5QcO5e1kVK+nJKPwOJDH0/aHbtdFACRdueBtk9SPJKGw5qbvPZQZGqRD0= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:1ca:b0:206:aa07:b62 with SMTP id d9443c01a7336-206f05148abmr1064025ad.5.1725597264616; Thu, 05 Sep 2024 21:34:24 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 5 Sep 2024 21:34:10 -0700 In-Reply-To: <20240906043413.1049633-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240906043413.1049633-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240906043413.1049633-5-seanjc@google.com> Subject: [PATCH v2 4/7] KVM: nVMX: Detect nested posted interrupt NV at nested VM-Exit injection From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Nathan Chancellor , Chao Gao , Zeng Guang When synthensizing a nested VM-Exit due to an external interrupt, pend a nested posted interrupt if the external interrupt vector matches L2's PI notification vector, i.e. if the interrupt is a PI notification for L2. This fixes a bug where KVM will incorrectly inject VM-Exit instead of processing nested posted interrupt when IPI virtualization is enabled. Per the SDM, detection of the notification vector doesn't occur until the interrupt is acknowledge and deliver to the CPU core. If the external-interrupt exiting VM-execution control is 1, any unmasked external interrupt causes a VM exit (see Section 26.2). If the "process posted interrupts" VM-execution control is also 1, this behavior is changed and the processor handles an external interrupt as follows: 1. The local APIC is acknowledged; this provides the processor core with an interrupt vector, called here the physical vector. 2. If the physical vector equals the posted-interrupt notification vector, the logical processor continues to the next step. Otherwise, a VM exit occurs as it would normally due to an external interrupt; the vector is saved in the VM-exit interruption-information field. For the most part, KVM has avoided problems because a PI NV for L2 that arrives will L2 is active will be processed by hardware, and KVM checks for a pending notification vector during nested VM-Enter. Thus, to hit the bug, the PI NV interrupt needs to sneak its way into L1's vIRR while L2 is active. Without IPI virtualization, the scenario is practically impossible to hit, modulo L1 doing weird things (see below), as the ordering between vmx_deliver_posted_interrupt() and nested VM-Enter effectively guarantees that either the sender will see the vCPU as being in_guest_mode(), or the receiver will see the interrupt in its vIRR. With IPI virtualization, introduced by commit d588bb9be1da ("KVM: VMX: enable IPI virtualization"), the sending CPU effectively implements a rough equivalent of vmx_deliver_posted_interrupt(), sans the nested PI NV check. If the target vCPU has a valid PID, the CPU will send a PI NV interrupt based on _L1's_ PID, as the sender's because IPIv table points at L1 PIDs. PIR := 32 bytes at PID_ADDR; // under lock PIR[V] := 1; store PIR at PID_ADDR; // release lock NotifyInfo := 8 bytes at PID_ADDR + 32; // under lock IF NotifyInfo.ON = 0 AND NotifyInfo.SN = 0; THEN NotifyInfo.ON := 1; SendNotify := 1; ELSE SendNotify := 0; FI; store NotifyInfo at PID_ADDR + 32; // release lock IF SendNotify = 1; THEN send an IPI specified by NotifyInfo.NDST and NotifyInfo.NV; FI; As a result, the target vCPU ends up receiving an interrupt on KVM's POSTED_INTR_VECTOR while L2 is running, with an interrupt in L1's PIR for L2's nested PI NV. The POSTED_INTR_VECTOR interrupt triggers a VM-Exit from L2 to L0, KVM moves the interrupt from L1's PIR to vIRR, triggers a KVM_REQ_EVENT prior to re-entry to L2, and calls vmx_check_nested_events(), effectively bypassing all of KVM's "early" checks on nested PI NV. Without IPI virtualization, the bug can likely be hit only if L1 programs an assigned device to _post_ an interrupt to L2's notification vector, by way of L1's PID.PIR. Doing so would allow the interrupt to get into L1's vIRR without KVM checking vmcs12's NV. Which is architecturally allowed, but unlikely behavior for a hypervisor. Cc: Zeng Guang Reviewed-by: Chao Gao Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 6b7e0ab0e45e..238c26155c2a 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -4307,6 +4307,20 @@ static int vmx_check_nested_events(struct kvm_vcpu *vcpu) if (WARN_ON_ONCE(irq < 0)) goto no_vmexit; + /* + * If the IRQ is L2's PI notification vector, process posted + * interrupts for L2 instead of injecting VM-Exit, as the + * detection/morphing architecturally occurs when the IRQ is + * delivered to the CPU. Note, only interrupts that are routed + * through the local APIC trigger posted interrupt processing, + * and enabling posted interrupts requires ACK-on-exit. + */ + if (irq == vmx->nested.posted_intr_nv) { + vmx->nested.pi_pending = true; + kvm_apic_clear_irr(vcpu, irq); + goto no_vmexit; + } + nested_vmx_vmexit(vcpu, EXIT_REASON_EXTERNAL_INTERRUPT, INTR_INFO_VALID_MASK | INTR_TYPE_EXT_INTR | irq, 0); From patchwork Fri Sep 6 04:34:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13793208 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9D6813D600 for ; Fri, 6 Sep 2024 04:34:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597270; cv=none; b=MjrvR1vT4+ptVTwuyVa1psdwDa8jjopRqZ20qBCBn3xDRVC9ATl22U0Mdbdf3BxSqQPWkZ5K8YUCsjqfNJ3/Shv/FifvmWzbBD27VQHBGvnsvKvLq+dBe2auTyJGZ6gedIXt8J5H6xbeBIngy6yQJqpQI0F6MJHsJY5460tWEKw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597270; c=relaxed/simple; bh=16BMCLeOVpdv0K/WH9ozgUVcrDZ/XLhLK3PWi4tm9l0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=YXDmgKNT+wfSud0DPJTHjxT5Yx6NfFGCC9OjCoPLlfTdcRULKajnNo8bbwbKf/a/fndY6mD5erTwWCju173IwpFs3WOt0YkKJI2/GoEkpLxJJ2Hj4ABs/T0up1SN3iiH2O6C3cIlllXIi4d+2gEnG8qgWewgaHAzX7R8Jvo9k5k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lnUKM2Le; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lnUKM2Le" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6d6b2b9a534so62743627b3.2 for ; Thu, 05 Sep 2024 21:34:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725597268; x=1726202068; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=uzxku0+LxyHsHOYX6rcGXjDtKiKbrlhkjaeoxb7nWcQ=; b=lnUKM2Le9gtS7mKy1NFJU7llV3HsuTaVEo9Ge9k3I/lAvyaMJbqIWUgf2aG4SVjgq6 0O86BuIKEnFPCusK0FD58aRa7/VfxwxWMcvN8OsH/Actv38uJyx7lU8T4P0ECfY7YQDO wNerNKjpeSS7qpKqOxYb9RQulZ2u+2sFi0y13ie7ROOPWqHRvtcd87LWiu07MFmRFMmP ImXJozsi4HExJiYXYeK7R0Jw7x85In8LnXT3IvQgYN8+1cHeAoIod64VuAM10JeoHMFQ o2QdhwDLvnyw+VZdSBqhHbrXMvJ2VoiGkve1Nr8DL3N9YvkSC4k20YxqjvHBiOBRNdnz qOfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725597268; x=1726202068; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uzxku0+LxyHsHOYX6rcGXjDtKiKbrlhkjaeoxb7nWcQ=; b=n+cEPrivZy9oEG9EEvVHjhnMIfFSaqQmaVuXhRP44gs5pYOvuqdM+AxgcWyzj5R6rj 7v54yOJeC2ah+eqspzOJcVq25PdvxQOwup+iNk0Os6xOZA6pdvI0uzUixpTquSjHq0gG a2Pwmrxk+waPgelRClJ3wVJuGX3BPpIfAHQKZu9kN//bNf9/XdE9uyriKGrwHz0Qb8zX Va8T/hmx4EEbRh2nB5/tOb2Ub3mBxUdjuLomq0sKF45QW1XhxaDFL7cpI0HHFnpTosyv W/pIg691XY8Ed5yzhACVL5PUtVeteFWmaKqtchCEHAywekN0mUAtn7y6GBXfgDCczCcr il/A== X-Gm-Message-State: AOJu0YypDSJt8XKOKdeikilDu8JU39mk6OJErxMLJVKWTb4E7sdR3aeF p6nfZj0J5Ctk8TRirjWV4FvYx5vovNYlmJ+YODiqBTEf/flrmC6fbM4CieAjIm1KoqkeAwdz95M jYg== X-Google-Smtp-Source: AGHT+IHnO4jNQ69g0TR8pRoeAENrswe//rGX8Vs4B2T2hgTvjDXJ14AKHraOC49CTLTBDc+mdxX6LOuY5UY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a0d:e9c4:0:b0:6d9:c367:54a5 with SMTP id 00721157ae682-6db45036a46mr9767b3.4.1725597266843; Thu, 05 Sep 2024 21:34:26 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 5 Sep 2024 21:34:11 -0700 In-Reply-To: <20240906043413.1049633-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240906043413.1049633-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240906043413.1049633-6-seanjc@google.com> Subject: [PATCH v2 5/7] KVM: x86: Fold kvm_get_apic_interrupt() into kvm_cpu_get_interrupt() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Nathan Chancellor , Chao Gao , Zeng Guang Fold kvm_get_apic_interrupt() into kvm_cpu_get_interrupt() now that nVMX essentially open codes kvm_get_apic_interrupt() in order to correctly emulate nested posted interrupts. Opportunistically stop exporting kvm_cpu_get_interrupt(), as the aforementioned nVMX flow was the only user in vendor code. Signed-off-by: Sean Christopherson --- arch/x86/kvm/irq.c | 7 +++++-- arch/x86/kvm/lapic.c | 10 ---------- arch/x86/kvm/lapic.h | 1 - 3 files changed, 5 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/irq.c b/arch/x86/kvm/irq.c index 810da99ff7ed..63f66c51975a 100644 --- a/arch/x86/kvm/irq.c +++ b/arch/x86/kvm/irq.c @@ -142,9 +142,12 @@ int kvm_cpu_get_interrupt(struct kvm_vcpu *v) if (vector != -1) return vector; /* PIC */ - return kvm_get_apic_interrupt(v); /* APIC */ + vector = kvm_apic_has_interrupt(v); /* APIC */ + if (vector != -1) + kvm_apic_ack_interrupt(v, vector); + + return vector; } -EXPORT_SYMBOL_GPL(kvm_cpu_get_interrupt); void kvm_inject_pending_timer_irqs(struct kvm_vcpu *vcpu) { diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index ff63ef8163a9..5c820e3f06a8 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2961,16 +2961,6 @@ void kvm_apic_ack_interrupt(struct kvm_vcpu *vcpu, int vector) } EXPORT_SYMBOL_GPL(kvm_apic_ack_interrupt); -int kvm_get_apic_interrupt(struct kvm_vcpu *vcpu) -{ - int vector = kvm_apic_has_interrupt(vcpu); - - if (vector != -1) - kvm_apic_ack_interrupt(vcpu, vector); - - return vector; -} - static int kvm_apic_state_fixup(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s, bool set) { diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index db80a2965b9c..8310ff74be29 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -90,7 +90,6 @@ void kvm_free_lapic(struct kvm_vcpu *vcpu); int kvm_apic_has_interrupt(struct kvm_vcpu *vcpu); void kvm_apic_ack_interrupt(struct kvm_vcpu *vcpu, int vector); int kvm_apic_accept_pic_intr(struct kvm_vcpu *vcpu); -int kvm_get_apic_interrupt(struct kvm_vcpu *vcpu); int kvm_apic_accept_events(struct kvm_vcpu *vcpu); void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event); u64 kvm_lapic_get_cr8(struct kvm_vcpu *vcpu); From patchwork Fri Sep 6 04:34:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13793209 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5126013DB9F for ; Fri, 6 Sep 2024 04:34:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597271; cv=none; b=NWjBZEYDt0z78idf5iB1V0iwVhdtJlPjNuD0EhBfurt09iPIQhx4eIPyJBWsOgZR1YXRm5X1wxxei++ag+J9O8iFtUpmHUb33w0siMCKbiH9v2BIv23j6IDHA5zqqbvDaLiDH1MeCom3YOXia27fWLh6L4c4zCyEbqOhxImFHa8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597271; c=relaxed/simple; bh=KOuZWdc9JPRpBXVKGqAv89lMSGg+kGyVbTW1GAwZ1wU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=T/H7uE8EBcQ9Wn9zuHw+FvcdmTurFQaYbX/WQDdOh1owK1a+p6VirjqtZJcdyJwfxPJxc5PwGpgQZPDHpjLhKPWR+7A1+wCCXYpKh538lmw4+8mETvczN9QpS7FxRJpYz1ihSIBcprU8wpw7t7qxwpmhYBVqXRM86WR6++MmF8M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=OwcH8E/j; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="OwcH8E/j" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7d4fc1a2bb7so2223059a12.1 for ; Thu, 05 Sep 2024 21:34:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725597269; x=1726202069; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=YYgMGUi3oxiD0kO1ugWhSXeqKUt6BlZh0E2W4sJdCss=; b=OwcH8E/jVBgRIUhB739yDghDvswotReeUXNvXUbZ009xWuvcWj8LsFNC09WINLr7xK qWI9zyB8lQ7nz5x1jjHecDtqWoNbcSmOPTv2Mh6QvomyvrWiZsRMJyPOTH0me0bfRYVj WkYeJg1q3R7KdwthmQxeUyVyY6rCiHgPCySjdI4AyqtO8v0Y5MdEJHoC95sCUNJmo/OC YWuruJAZsTp8nZdTfVKgN+OQmSMZsTwus4Wq+LAC1rTfUdKNEBEECIGcuf+iqQUZ8qKF FtrYYjWXoUzTQHnNmIJ/LWeS+jpHd+HnYv933F7NK+Y7+lidx7v4yKREWadSIRKRlMyp a8uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725597269; x=1726202069; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YYgMGUi3oxiD0kO1ugWhSXeqKUt6BlZh0E2W4sJdCss=; b=K9Sn2wCO9j4ym1QgpXipMAZAN18yLIt/fyiwy1FjRTrc1xgpHqEKC706aS5ZuL91Op ENIuzxjXD5faT9mO3+pwZ/eRPoLTSBxpvVhgHRFTCr6sq6mldvSp5CoTT+4jvgJHuz+U iQFHSWISoTAbYfxdUZdDj4dd/nYh05TXh2v0ND9X4WpHLe+NqtyYNo+CuxV2rYXCM47B WSK2eC2SRbA9JRc/Il5Vqc1bMTfhZo2ux9ndn5HRGisAStn+HXRf+4ivdb6wrRB8Y2Tb IEog+8uZH+IbCtkUxPzrjL3mOPagSSrabY+3tfvoCu14WeAtqt6ni2tUIwKD/ErW6S9X 0hyw== X-Gm-Message-State: AOJu0YxqnpEupHyohJVdUJVrbsW8oApUO2mWC94mLA0DjHNyFwODLWmB uYBwkc7p3LIh9WoVVenkDltH3ZZsegB2nKFN0f3Lbx6k1q0B8H8/SG3ybDdoMsHwNTAnGo81PR/ M/Q== X-Google-Smtp-Source: AGHT+IGIG/2AqeejW/EiNPsvj7vZAMKQLNqM6dC/eSPaVhlrS/jIejMZxR/29yan4Y6HwPe+cS1JZoSR8O8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:ea09:b0:205:560d:947d with SMTP id d9443c01a7336-206f0603526mr1127025ad.7.1725597269489; Thu, 05 Sep 2024 21:34:29 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 5 Sep 2024 21:34:12 -0700 In-Reply-To: <20240906043413.1049633-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240906043413.1049633-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240906043413.1049633-7-seanjc@google.com> Subject: [PATCH v2 6/7] KVM: nVMX: Explicitly invalidate posted_intr_nv if PI is disabled at VM-Enter From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Nathan Chancellor , Chao Gao , Zeng Guang Explicitly invalidate posted_intr_nv when emulating nested VM-Enter and posted interrupts are disabled to make it clear that posted_intr_nv is valid if and only if nested posted interrupts are enabled, and as a cheap way to harden against KVM bugs. KVM initializes posted_intr_nv to -1 at vCPU creation and resets it to -1 when unloading vmcs12 and/or leaving nested mode, i.e. this is not a bug fix (or at least, it's not intended to be a bug fix). Note, tracking nested.posted_intr_nv as a u16 subtly adds a measure of safety, as it prevents unintentionally matching KVM's informal "no IRQ" vector of -1, stored as a signed int. Because a u16 can be always be represented as a signed int, the effective "invalid" value of posted_intr_nv, 65535, will be preserved as-is when comparing against an int, i.e. will be zero-extended, not sign-extended, and thus won't get a false positive if KVM is buggy and compares posted_intr_nv against -1. Opportunistically add a comment in vmx_deliver_nested_posted_interrupt() to call out that it must check vmx->nested.posted_intr_nv, not the vector in vmcs12, which is presumably the _entire_ reason nested.posted_intr_nv exists. E.g. vmcs12 is a KVM-controlled snapshot, so there are no TOCTOU races to worry about, the only potential badness is if the vCPU leaves nested and frees vmcs12 between the sender checking is_guest_mode() and dereferencing the vmcs12 pointer. Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 6 ++++-- arch/x86/kvm/vmx/vmx.c | 7 +++++++ 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 238c26155c2a..7e8a646e2851 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -2317,10 +2317,12 @@ static void prepare_vmcs02_early(struct vcpu_vmx *vmx, struct loaded_vmcs *vmcs0 /* Posted interrupts setting is only taken from vmcs12. */ vmx->nested.pi_pending = false; - if (nested_cpu_has_posted_intr(vmcs12)) + if (nested_cpu_has_posted_intr(vmcs12)) { vmx->nested.posted_intr_nv = vmcs12->posted_intr_nv; - else + } else { + vmx->nested.posted_intr_nv = -1; exec_control &= ~PIN_BASED_POSTED_INTR; + } pin_controls_set(vmx, exec_control); /* diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index f18c2d8c7476..63d656032384 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -4219,6 +4219,13 @@ static int vmx_deliver_nested_posted_interrupt(struct kvm_vcpu *vcpu, { struct vcpu_vmx *vmx = to_vmx(vcpu); + /* + * DO NOT query the vCPU's vmcs12, as vmcs12 is dynamically allocated + * and freed, and must not be accessed outside of vcpu->mutex. The + * vCPU's cached PI NV is valid if and only if posted interrupts + * enabled in its vmcs12, i.e. checking the vector also checks that + * L1 has enabled posted interrupts for L2. + */ if (is_guest_mode(vcpu) && vector == vmx->nested.posted_intr_nv) { /* From patchwork Fri Sep 6 04:34:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13793210 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E1CC1474CE for ; Fri, 6 Sep 2024 04:34:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597273; cv=none; b=d+3TXMP90yNsckKtmF6LocnCJtFPWX+6B4WJkrP8JqDE7Kvn9yhz/trfHE6A/MMnWSnFxSdEj7i/vpYirA2x+uTJPWsruFBiKI3St67U+5TwMFWphblGpff2SON4dxaAhWwegDYGqoELVd7ccYQx8+dnWn6qdybAC7GIflaGIfE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725597273; c=relaxed/simple; bh=fYasJcFsQHDkQBLbQydtD9whZiU5icAeuGX4p36bSTw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=pZ+o1PvHBcV3flbOuncTCeLAHCyFRK0b4r0JKgxHlqYDSEHkxRgZw8IXalV+7PMVfUdux7N2Zl9hZavEYKQRG3HjK9dwLG2BVIbCsPVUgLjZc2XL/QutZS7p2LFozv8BZzb/cxqua07nG44WxI3r3HnxZh702/bdgAtS/765eMo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=hpcYi11N; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hpcYi11N" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-7179469744dso1665707b3a.2 for ; Thu, 05 Sep 2024 21:34:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725597271; x=1726202071; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Ng8EbVRlXMuE/cCrzi7LkSpAYLCcIcl00WaW0vWSlZE=; b=hpcYi11NChzmDdeZ61nRL10CYvSRl2C3aC8Kol16DE/XWomxj0t+jAFKPJk/ogslXr 7CZeP41WeT24ZxcdmkfLspenSsVfxtXJ4zYCP0+y4Efc9/UnizDWKM4/K5bl3Pg+6AnC KUHPOrz639zxsrdnBX5EMwVd991w9OfnflSr62cq+W7RaH6rIhtePYWBBhzC7GYbl1yF Wj05Pxjq7L3T5pcTPDUdIxlsV2MnMYJSKQGoZHtmETH47n6+PJvKh9/gPzoArG2mjgxH NjDdM6VzK5VfUlQHu7ZDu3LKZSrPBpTN37qe+YC240msPJKnHF9TIOt2jPkzp/sZtKyB L7oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725597271; x=1726202071; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Ng8EbVRlXMuE/cCrzi7LkSpAYLCcIcl00WaW0vWSlZE=; b=SHj0jblmWVFR2aS5nOV64ZYM0mN/Zuz0lD10IrAqGBDzovP7TYptEuKh6P2zc6WPYI OqwsqdS0boUxxKkHOnub11GydLe44Za1q/QZ2CzHzAaoPll/yxmYlBKibKt7DFJuUtbF IxtOH/XD1e8L2nHfGmikZ4s+ZBZ/KIpQPqMcPnfziM0PIpZ8uPH9UH9cEyxontajQDxu HyY1xkWuLYDAkBvX1xeSQsE3ZV/tbfbFzIXXjxOKKEsZAP8J7NSqkYtGQjBvb/2+OfYt DUTgXC2yi/OqOoFgpXM+xZqvtPYXn8yunBE8mu487QZXXv191IbxQRposvjzITZwlrj4 hDTg== X-Gm-Message-State: AOJu0Ywb6SoQzeGxUJ62O6OfRRUu/c3YdwIq+SiSSs3o0IoJea5YQPLP kL+AusfO3xJeUEgCQXU7A8ZcBP7VvCY5hvSG6MIS0hsCOJETdL0x70R1FKm1lK00idA9UaHXPqb CLA== X-Google-Smtp-Source: AGHT+IGnip38uUNcVH5/E6tvK0YEJByz8PD5gVFdSt7Dd8HEI5fzxeWwOi6izTL6TuI+wk0frCtzr+Zv8jU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:aa7:8d96:0:b0:717:92f2:d50b with SMTP id d2e1a72fcca58-718d5c30082mr3607b3a.0.1725597271485; Thu, 05 Sep 2024 21:34:31 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 5 Sep 2024 21:34:13 -0700 In-Reply-To: <20240906043413.1049633-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240906043413.1049633-1-seanjc@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240906043413.1049633-8-seanjc@google.com> Subject: [PATCH v2 7/7] KVM: nVMX: Assert that vcpu->mutex is held when accessing secondary VMCSes From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Nathan Chancellor , Chao Gao , Zeng Guang Add lockdep assertions in get_vmcs12() and get_shadow_vmcs12() to verify the vCPU's mutex is held, as the returned VMCS objects are dynamically allocated/freed when nested VMX is turned on/off, i.e. accessing vmcs12 structures without holding vcpu->mutex is susceptible to use-after-free. Waive the assertion if the VM is being destroyed, as KVM currently forces a nested VM-Exit when freeing the vCPU. If/when that wart is fixed, the assertion can/should be converted to an unqualified lockdep assertion. See also https://lore.kernel.org/all/Zsd0TqCeY3B5Sb5b@google.com. Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/kvm/vmx/nested.h b/arch/x86/kvm/vmx/nested.h index cce4e2aa30fb..668b6c83a373 100644 --- a/arch/x86/kvm/vmx/nested.h +++ b/arch/x86/kvm/vmx/nested.h @@ -39,11 +39,17 @@ bool nested_vmx_check_io_bitmaps(struct kvm_vcpu *vcpu, unsigned int port, static inline struct vmcs12 *get_vmcs12(struct kvm_vcpu *vcpu) { + lockdep_assert_once(lockdep_is_held(&vcpu->mutex) || + !refcount_read(&vcpu->kvm->users_count)); + return to_vmx(vcpu)->nested.cached_vmcs12; } static inline struct vmcs12 *get_shadow_vmcs12(struct kvm_vcpu *vcpu) { + lockdep_assert_once(lockdep_is_held(&vcpu->mutex) || + !refcount_read(&vcpu->kvm->users_count)); + return to_vmx(vcpu)->nested.cached_shadow_vmcs12; }