From patchwork Fri May 7 13:06:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Tosatti X-Patchwork-Id: 12244549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B865C433ED for ; Fri, 7 May 2021 13:10:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D968561466 for ; Fri, 7 May 2021 13:10:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236743AbhEGNLW (ORCPT ); Fri, 7 May 2021 09:11:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44833 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236434AbhEGNLT (ORCPT ); Fri, 7 May 2021 09:11:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620393019; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=yZmXVCCtQd+G9YWj0HVU+KdVOSHrhwSnoYMJJQ0B04w=; b=BUWYgA7sTe5A+ytt2V2DA2EspZ4ierxMWWeOklolacJrE+Sa54J6XAqJ08pofUVjOg61oT eKOLbbOLQ967FGr6NXr/PHzQxXIlqymNCEOfE9jWMBZTgFwQYYyGCoOZ2dgw4Jf6NPEO7R te1HDVzFq00rIXR+w2Gr+QE74vqatFE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-442-nqxt_hWAMvKorCU_9QPP1Q-1; Fri, 07 May 2021 09:10:17 -0400 X-MC-Unique: nqxt_hWAMvKorCU_9QPP1Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DF4A3107ACC7; Fri, 7 May 2021 13:10:16 +0000 (UTC) Received: from fuller.cnet (ovpn-112-4.gru2.redhat.com [10.97.112.4]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D8EE536E0; Fri, 7 May 2021 13:10:12 +0000 (UTC) Received: by fuller.cnet (Postfix, from userid 1000) id A01B941486FD; Fri, 7 May 2021 10:10:07 -0300 (-03) Message-ID: <20210507130923.438255076@redhat.com> User-Agent: quilt/0.66 Date: Fri, 07 May 2021 10:06:10 -0300 From: Marcelo Tosatti To: kvm@vger.kernel.org Cc: Paolo Bonzini , Alex Williamson , Sean Christopherson , Marcelo Tosatti Subject: [patch 1/4] KVM: x86: add start_assignment hook to kvm_x86_ops References: <20210507130609.269153197@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a start_assignment hook to kvm_x86_ops, which is called when kvm_arch_start_assignment is done. The hook is required to update the wakeup vector of a sleeping vCPU when a device is assigned to the guest. Signed-off-by: Marcelo Tosatti Index: kvm/arch/x86/include/asm/kvm_host.h =================================================================== --- kvm.orig/arch/x86/include/asm/kvm_host.h +++ kvm/arch/x86/include/asm/kvm_host.h @@ -1322,6 +1322,7 @@ struct kvm_x86_ops { int (*update_pi_irte)(struct kvm *kvm, unsigned int host_irq, uint32_t guest_irq, bool set); + void (*start_assignment)(struct kvm *kvm, int device_count); void (*apicv_post_state_restore)(struct kvm_vcpu *vcpu); bool (*dy_apicv_has_pending_interrupt)(struct kvm_vcpu *vcpu); Index: kvm/arch/x86/kvm/svm/svm.c =================================================================== --- kvm.orig/arch/x86/kvm/svm/svm.c +++ kvm/arch/x86/kvm/svm/svm.c @@ -4601,6 +4601,7 @@ static struct kvm_x86_ops svm_x86_ops __ .deliver_posted_interrupt = svm_deliver_avic_intr, .dy_apicv_has_pending_interrupt = svm_dy_apicv_has_pending_interrupt, .update_pi_irte = svm_update_pi_irte, + .start_assignment = NULL, .setup_mce = svm_setup_mce, .smi_allowed = svm_smi_allowed, Index: kvm/arch/x86/kvm/vmx/vmx.c =================================================================== --- kvm.orig/arch/x86/kvm/vmx/vmx.c +++ kvm/arch/x86/kvm/vmx/vmx.c @@ -7732,6 +7732,7 @@ static struct kvm_x86_ops vmx_x86_ops __ .nested_ops = &vmx_nested_ops, .update_pi_irte = pi_update_irte, + .start_assignment = NULL, #ifdef CONFIG_X86_64 .set_hv_timer = vmx_set_hv_timer, Index: kvm/arch/x86/kvm/x86.c =================================================================== --- kvm.orig/arch/x86/kvm/x86.c +++ kvm/arch/x86/kvm/x86.c @@ -11295,7 +11295,10 @@ bool kvm_arch_can_dequeue_async_page_pre void kvm_arch_start_assignment(struct kvm *kvm) { - atomic_inc(&kvm->arch.assigned_device_count); + int ret; + + ret = atomic_inc_return(&kvm->arch.assigned_device_count); + static_call_cond(kvm_x86_start_assignment)(kvm, ret); } EXPORT_SYMBOL_GPL(kvm_arch_start_assignment); Index: kvm/arch/x86/include/asm/kvm-x86-ops.h =================================================================== --- kvm.orig/arch/x86/include/asm/kvm-x86-ops.h +++ kvm/arch/x86/include/asm/kvm-x86-ops.h @@ -99,6 +99,7 @@ KVM_X86_OP_NULL(post_block) KVM_X86_OP_NULL(vcpu_blocking) KVM_X86_OP_NULL(vcpu_unblocking) KVM_X86_OP_NULL(update_pi_irte) +KVM_X86_OP_NULL(start_assignment) KVM_X86_OP_NULL(apicv_post_state_restore) KVM_X86_OP_NULL(dy_apicv_has_pending_interrupt) KVM_X86_OP_NULL(set_hv_timer) From patchwork Fri May 7 13:06:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Tosatti X-Patchwork-Id: 12244547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94F2BC433B4 for ; Fri, 7 May 2021 13:10:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4E80461464 for ; Fri, 7 May 2021 13:10:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236726AbhEGNLV (ORCPT ); Fri, 7 May 2021 09:11:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:35598 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232140AbhEGNLT (ORCPT ); Fri, 7 May 2021 09:11:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620393019; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=3szDupmhb5P3gzH2roiv5CqgjhCLQjU+Y1Q5biEIpRs=; b=SSLvvI0fK5jIbF5ALwRwj8SUF4ZPhKeuXRQ18RvkcVRy86IY30h45e/wfT1uFuektmTdNZ ebcqx+4qkjb6Nrh9f4UEW/hRo1D6NmAcYuuFwwAZhjCYtIpxZnGC4nTc/n0mSgPHpVyYHa 0kFy0JFHPdLSuIVOldkEwnDrXSvaeUI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-242-Xa6uYFsgMjuSOm6-ga3jVQ-1; Fri, 07 May 2021 09:10:17 -0400 X-MC-Unique: Xa6uYFsgMjuSOm6-ga3jVQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DED0A64149; Fri, 7 May 2021 13:10:16 +0000 (UTC) Received: from fuller.cnet (ovpn-112-4.gru2.redhat.com [10.97.112.4]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F128B13487; Fri, 7 May 2021 13:10:12 +0000 (UTC) Received: by fuller.cnet (Postfix, from userid 1000) id A29D640E6DFB; Fri, 7 May 2021 10:10:07 -0300 (-03) Message-ID: <20210507130923.468254904@redhat.com> User-Agent: quilt/0.66 Date: Fri, 07 May 2021 10:06:11 -0300 From: Marcelo Tosatti To: kvm@vger.kernel.org Cc: Paolo Bonzini , Alex Williamson , Sean Christopherson , Marcelo Tosatti Subject: [patch 2/4] KVM: add arch specific vcpu_check_block callback References: <20210507130609.269153197@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add callback in kvm_vcpu_check_block, so that architectures can direct a vcpu to exit the vcpu block loop without requiring events that would unhalt it. Signed-off-by: Marcelo Tosatti Index: kvm/include/linux/kvm_host.h =================================================================== --- kvm.orig/include/linux/kvm_host.h +++ kvm/include/linux/kvm_host.h @@ -971,6 +971,13 @@ static inline int kvm_arch_flush_remote_ } #endif +#ifndef __KVM_HAVE_ARCH_VCPU_CHECK_BLOCK +static inline int kvm_arch_vcpu_check_block(struct kvm_vcpu *vcpu) +{ + return 0; +} +#endif + #ifdef __KVM_HAVE_ARCH_NONCOHERENT_DMA void kvm_arch_register_noncoherent_dma(struct kvm *kvm); void kvm_arch_unregister_noncoherent_dma(struct kvm *kvm); Index: kvm/virt/kvm/kvm_main.c =================================================================== --- kvm.orig/virt/kvm/kvm_main.c +++ kvm/virt/kvm/kvm_main.c @@ -2794,6 +2794,8 @@ static int kvm_vcpu_check_block(struct k goto out; if (signal_pending(current)) goto out; + if (kvm_arch_vcpu_check_block(vcpu)) + goto out; ret = 0; out: From patchwork Fri May 7 13:06:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Tosatti X-Patchwork-Id: 12244551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 672C2C43462 for ; Fri, 7 May 2021 13:10:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 27C8F6145D for ; Fri, 7 May 2021 13:10:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236821AbhEGNLX (ORCPT ); Fri, 7 May 2021 09:11:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:40708 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236480AbhEGNLV (ORCPT ); Fri, 7 May 2021 09:11:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620393021; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=0FISU66+bSzKWpxBjhGrO1Tlhi9U0QwtDOlzrc6O7DQ=; b=CNABv+0g3KaGtuL62LCbV6WFzQHgonfCeYHGciRH90bO0QRgUnro1Anmtlcg4SOpEdKdim stl9XiiVyRBXSz+I+GpOtupval/Aap8UwQSjHJmckVZmlE8fwNKZUZvq8owXY64ak4uaJc x296V2+D/JUndBXCJXXG6dp9n//FPGo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-401-ZErEkDUlNwqGgWd_f8nj1g-1; Fri, 07 May 2021 09:10:17 -0400 X-MC-Unique: ZErEkDUlNwqGgWd_f8nj1g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E4DAF10066E6; Fri, 7 May 2021 13:10:16 +0000 (UTC) Received: from fuller.cnet (ovpn-112-4.gru2.redhat.com [10.97.112.4]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E6595100164A; Fri, 7 May 2021 13:10:12 +0000 (UTC) Received: by fuller.cnet (Postfix, from userid 1000) id A776F40B1EB1; Fri, 7 May 2021 10:10:07 -0300 (-03) Message-ID: <20210507130923.498207045@redhat.com> User-Agent: quilt/0.66 Date: Fri, 07 May 2021 10:06:12 -0300 From: Marcelo Tosatti To: kvm@vger.kernel.org Cc: Paolo Bonzini , Alex Williamson , Sean Christopherson , Marcelo Tosatti Subject: [patch 3/4] KVM: x86: implement kvm_arch_vcpu_check_block callback References: <20210507130609.269153197@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement kvm_arch_vcpu_check_block for x86. Next patch will add implementation of kvm_x86_ops.vcpu_check_block for VMX. Signed-off-by: Marcelo Tosatti Index: kvm/arch/x86/include/asm/kvm_host.h =================================================================== --- kvm.orig/arch/x86/include/asm/kvm_host.h +++ kvm/arch/x86/include/asm/kvm_host.h @@ -1320,6 +1320,8 @@ struct kvm_x86_ops { void (*vcpu_blocking)(struct kvm_vcpu *vcpu); void (*vcpu_unblocking)(struct kvm_vcpu *vcpu); + int (*vcpu_check_block)(struct kvm_vcpu *vcpu); + int (*update_pi_irte)(struct kvm *kvm, unsigned int host_irq, uint32_t guest_irq, bool set); void (*start_assignment)(struct kvm *kvm, int device_count); @@ -1801,6 +1803,15 @@ static inline bool kvm_irq_is_postable(s irq->delivery_mode == APIC_DM_LOWEST); } +#define __KVM_HAVE_ARCH_VCPU_CHECK_BLOCK +static inline int kvm_arch_vcpu_check_block(struct kvm_vcpu *vcpu) +{ + if (kvm_x86_ops.vcpu_check_block) + return static_call(kvm_x86_vcpu_check_block)(vcpu); + + return 0; +} + static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) { static_call_cond(kvm_x86_vcpu_blocking)(vcpu); Index: kvm/arch/x86/kvm/vmx/vmx.c =================================================================== --- kvm.orig/arch/x86/kvm/vmx/vmx.c +++ kvm/arch/x86/kvm/vmx/vmx.c @@ -7727,6 +7727,7 @@ static struct kvm_x86_ops vmx_x86_ops __ .pre_block = vmx_pre_block, .post_block = vmx_post_block, + .vcpu_check_block = NULL, .pmu_ops = &intel_pmu_ops, .nested_ops = &vmx_nested_ops, Index: kvm/arch/x86/include/asm/kvm-x86-ops.h =================================================================== --- kvm.orig/arch/x86/include/asm/kvm-x86-ops.h +++ kvm/arch/x86/include/asm/kvm-x86-ops.h @@ -98,6 +98,7 @@ KVM_X86_OP_NULL(pre_block) KVM_X86_OP_NULL(post_block) KVM_X86_OP_NULL(vcpu_blocking) KVM_X86_OP_NULL(vcpu_unblocking) +KVM_X86_OP_NULL(vcpu_check_block) KVM_X86_OP_NULL(update_pi_irte) KVM_X86_OP_NULL(start_assignment) KVM_X86_OP_NULL(apicv_post_state_restore) Index: kvm/arch/x86/kvm/svm/svm.c =================================================================== --- kvm.orig/arch/x86/kvm/svm/svm.c +++ kvm/arch/x86/kvm/svm/svm.c @@ -4517,6 +4517,7 @@ static struct kvm_x86_ops svm_x86_ops __ .vcpu_put = svm_vcpu_put, .vcpu_blocking = svm_vcpu_blocking, .vcpu_unblocking = svm_vcpu_unblocking, + .vcpu_check_block = NULL, .update_exception_bitmap = svm_update_exception_bitmap, .get_msr_feature = svm_get_msr_feature, From patchwork Fri May 7 13:06:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Tosatti X-Patchwork-Id: 12244555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAF18C433B4 for ; Fri, 7 May 2021 13:10:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 932C66145A for ; Fri, 7 May 2021 13:10:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237126AbhEGNLY (ORCPT ); Fri, 7 May 2021 09:11:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:21140 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236544AbhEGNLV (ORCPT ); Fri, 7 May 2021 09:11:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620393021; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=OzNZ8fsZP5SyHIdVZ/cRlR2nzmpijUfIpMsfTWcEav0=; b=U0xy1KI9k+El0CQgvgd11hgL1nAtrBMnoWxjztu4S0C/0vThF89kWH7p9djiwEtnUKiaWb cFThFdj1cl9e3iSiNfouL58xmQJUudRW4byen8+l/8kjc15iBgoWuf4sMoehqF1xMF2OuW SQy74EatT0cD8TBtaOiEpwbhy0nIzmI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-163-scAvktjpMAqONfvlLHsVMw-1; Fri, 07 May 2021 09:10:20 -0400 X-MC-Unique: scAvktjpMAqONfvlLHsVMw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 188D0107ACC7; Fri, 7 May 2021 13:10:19 +0000 (UTC) Received: from fuller.cnet (ovpn-112-4.gru2.redhat.com [10.97.112.4]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1611B19D61; Fri, 7 May 2021 13:10:13 +0000 (UTC) Received: by fuller.cnet (Postfix, from userid 1000) id ABE5540B1EB2; Fri, 7 May 2021 10:10:07 -0300 (-03) Message-ID: <20210507130923.528132061@redhat.com> User-Agent: quilt/0.66 Date: Fri, 07 May 2021 10:06:13 -0300 From: Marcelo Tosatti To: kvm@vger.kernel.org Cc: Paolo Bonzini , Alex Williamson , Sean Christopherson , Pei Zhang , Marcelo Tosatti Subject: [patch 4/4] KVM: VMX: update vcpu posted-interrupt descriptor when assigning device References: <20210507130609.269153197@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org For VMX, when a vcpu enters HLT emulation, pi_post_block will: 1) Add vcpu to per-cpu list of blocked vcpus. 2) Program the posted-interrupt descriptor "notification vector" to POSTED_INTR_WAKEUP_VECTOR With interrupt remapping, an interrupt will set the PIR bit for the vector programmed for the device on the CPU, test-and-set the ON bit on the posted interrupt descriptor, and if the ON bit is clear generate an interrupt for the notification vector. This way, the target CPU wakes upon a device interrupt and wakes up the target vcpu. Problem is that pi_post_block only programs the notification vector if kvm_arch_has_assigned_device() is true. Its possible for the following to happen: 1) vcpu V HLTs on pcpu P, kvm_arch_has_assigned_device is false, notification vector is not programmed 2) device is assigned to VM 3) device interrupts vcpu V, sets ON bit (notification vector not programmed, so pcpu P remains in idle) 4) vcpu 0 IPIs vcpu V (in guest), but since pi descriptor ON bit is set, kvm_vcpu_kick is skipped 5) vcpu 0 busy spins on vcpu V's response for several seconds, until RCU watchdog NMIs all vCPUs. To fix this, use the start_assignment kvm_x86_ops callback to kick vcpus out of the halt loop, so the notification vector is properly reprogrammed to the wakeup vector. Reported-by: Pei Zhang Signed-off-by: Marcelo Tosatti --- v2: add vmx_pi_start_assignment to vmx's kvm_x86_ops Index: kvm/arch/x86/kvm/vmx/posted_intr.c =================================================================== --- kvm.orig/arch/x86/kvm/vmx/posted_intr.c +++ kvm/arch/x86/kvm/vmx/posted_intr.c @@ -203,6 +203,25 @@ void pi_post_block(struct kvm_vcpu *vcpu local_irq_enable(); } +int vmx_vcpu_check_block(struct kvm_vcpu *vcpu) +{ + struct pi_desc *pi_desc = vcpu_to_pi_desc(vcpu); + + if (!irq_remapping_cap(IRQ_POSTING_CAP)) + return 0; + + if (!kvm_vcpu_apicv_active(vcpu)) + return 0; + + if (!kvm_arch_has_assigned_device(vcpu->kvm)) + return 0; + + if (pi_desc->nv == POSTED_INTR_WAKEUP_VECTOR) + return 0; + + return 1; +} + /* * Handler for POSTED_INTERRUPT_WAKEUP_VECTOR. */ @@ -236,6 +255,26 @@ bool pi_has_pending_interrupt(struct kvm (pi_test_sn(pi_desc) && !pi_is_pir_empty(pi_desc)); } +void vmx_pi_start_assignment(struct kvm *kvm, int device_count) +{ + struct kvm_vcpu *vcpu; + int i; + + if (!irq_remapping_cap(IRQ_POSTING_CAP)) + return; + + /* only care about first device assignment */ + if (device_count != 1) + return; + + /* Update wakeup vector and add vcpu to blocked_vcpu_list */ + kvm_for_each_vcpu(i, vcpu, kvm) { + if (!kvm_vcpu_apicv_active(vcpu)) + continue; + + kvm_vcpu_kick(vcpu); + } +} /* * pi_update_irte - set IRTE for Posted-Interrupts Index: kvm/arch/x86/kvm/vmx/posted_intr.h =================================================================== --- kvm.orig/arch/x86/kvm/vmx/posted_intr.h +++ kvm/arch/x86/kvm/vmx/posted_intr.h @@ -95,5 +95,7 @@ void __init pi_init_cpu(int cpu); bool pi_has_pending_interrupt(struct kvm_vcpu *vcpu); int pi_update_irte(struct kvm *kvm, unsigned int host_irq, uint32_t guest_irq, bool set); +void vmx_pi_start_assignment(struct kvm *kvm, int device_count); +int vmx_vcpu_check_block(struct kvm_vcpu *vcpu); #endif /* __KVM_X86_VMX_POSTED_INTR_H */ Index: kvm/arch/x86/kvm/vmx/vmx.c =================================================================== --- kvm.orig/arch/x86/kvm/vmx/vmx.c +++ kvm/arch/x86/kvm/vmx/vmx.c @@ -7727,13 +7727,13 @@ static struct kvm_x86_ops vmx_x86_ops __ .pre_block = vmx_pre_block, .post_block = vmx_post_block, - .vcpu_check_block = NULL, + .vcpu_check_block = vmx_vcpu_check_block, .pmu_ops = &intel_pmu_ops, .nested_ops = &vmx_nested_ops, .update_pi_irte = pi_update_irte, - .start_assignment = NULL, + .start_assignment = vmx_pi_start_assignment, #ifdef CONFIG_X86_64 .set_hv_timer = vmx_set_hv_timer,