From patchwork Mon Dec 11 07:21:08 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ladi Prosek X-Patchwork-Id: 10104343 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9637F605D2 for ; Mon, 11 Dec 2017 07:22:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6E963206E2 for ; Mon, 11 Dec 2017 07:22:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6390228565; Mon, 11 Dec 2017 07:22:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 03EF628451 for ; Mon, 11 Dec 2017 07:22:33 +0000 (UTC) Received: from localhost ([::1]:51121 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eOIQ4-0001WQ-D5 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 11 Dec 2017 02:22:32 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33046) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eOIOu-0000Aq-T1 for qemu-devel@nongnu.org; Mon, 11 Dec 2017 02:21:21 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eOIOt-0003k6-JL for qemu-devel@nongnu.org; Mon, 11 Dec 2017 02:21:20 -0500 Received: from mx1.redhat.com ([209.132.183.28]:57374) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eOIOt-0003j0-BR for qemu-devel@nongnu.org; Mon, 11 Dec 2017 02:21:19 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 922D7C0587FA; Mon, 11 Dec 2017 07:21:18 +0000 (UTC) Received: from dhcp-1-107.brq.redhat.com (unknown [10.43.2.157]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2BF795C6C5; Mon, 11 Dec 2017 07:21:17 +0000 (UTC) From: Ladi Prosek To: qemu-devel@nongnu.org Date: Mon, 11 Dec 2017 08:21:08 +0100 Message-Id: <20171211072110.9058-3-lprosek@redhat.com> In-Reply-To: <20171211072110.9058-1-lprosek@redhat.com> References: <20171211072110.9058-1-lprosek@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Mon, 11 Dec 2017 07:21:18 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v4 2/4] ivshmem: Always remove irqfd notifiers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: geoff@hostfission.com, pbonzini@redhat.com, armbru@redhat.com, marcandre.lureau@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP As of commit 660c97eef6f8 ("ivshmem: use kvm irqfd for msi notifications"), QEMU crashes with: ivshmem: msix_set_vector_notifiers failed msix_unset_vector_notifiers: Assertion `dev->msix_vector_use_notifier && dev->msix_vector_release_notifier' failed. if MSI-X is repeatedly enabled and disabled on the ivshmem device, for example by loading and unloading the Windows ivshmem driver. This is because msix_unset_vector_notifiers() doesn't call any of the release notifier callbacks since MSI-X is already disabled at that point (msix_enabled() returning false is how this transition is detected in the first place). Thus ivshmem_vector_mask() doesn't run and when MSI-X is subsequently enabled again ivshmem_vector_unmask() fails. This is fixed by keeping track of unmasked vectors and making sure that ivshmem_vector_mask() always runs on MSI-X disable. Fixes: 660c97eef6f8 ("ivshmem: use kvm irqfd for msi notifications") Signed-off-by: Ladi Prosek Reviewed-by: Markus Armbruster --- hw/misc/ivshmem.c | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/hw/misc/ivshmem.c b/hw/misc/ivshmem.c index 6e46669744..91364d8364 100644 --- a/hw/misc/ivshmem.c +++ b/hw/misc/ivshmem.c @@ -77,6 +77,7 @@ typedef struct Peer { typedef struct MSIVector { PCIDevice *pdev; int virq; + bool unmasked; } MSIVector; typedef struct IVShmemState { @@ -321,6 +322,7 @@ static int ivshmem_vector_unmask(PCIDevice *dev, unsigned vector, error_report("ivshmem: vector %d route does not exist", vector); return -EINVAL; } + assert(!v->unmasked); ret = kvm_irqchip_update_msi_route(kvm_state, v->virq, msg, dev); if (ret < 0) { @@ -328,7 +330,13 @@ static int ivshmem_vector_unmask(PCIDevice *dev, unsigned vector, } kvm_irqchip_commit_routes(kvm_state); - return kvm_irqchip_add_irqfd_notifier_gsi(kvm_state, n, NULL, v->virq); + ret = kvm_irqchip_add_irqfd_notifier_gsi(kvm_state, n, NULL, v->virq); + if (ret < 0) { + return ret; + } + v->unmasked = true; + + return 0; } static void ivshmem_vector_mask(PCIDevice *dev, unsigned vector) @@ -343,11 +351,14 @@ static void ivshmem_vector_mask(PCIDevice *dev, unsigned vector) error_report("ivshmem: vector %d route does not exist", vector); return; } + assert(v->unmasked); ret = kvm_irqchip_remove_irqfd_notifier_gsi(kvm_state, n, v->virq); - if (ret != 0) { + if (ret < 0) { error_report("remove_irqfd_notifier_gsi failed"); + return; } + v->unmasked = false; } static void ivshmem_vector_poll(PCIDevice *dev, @@ -817,11 +828,20 @@ static void ivshmem_disable_irqfd(IVShmemState *s) PCIDevice *pdev = PCI_DEVICE(s); int i; - for (i = 0; i < s->peers[s->vm_id].nb_eventfds; i++) { - ivshmem_remove_kvm_msi_virq(s, i); - } - msix_unset_vector_notifiers(pdev); + + for (i = 0; i < s->peers[s->vm_id].nb_eventfds; i++) { + /* + * MSI-X is already disabled here so msix_unset_vector_notifiers() + * didn't call our release notifier. Do it now to keep our masks and + * unmasks balanced. + */ + if (s->msi_vectors[i].unmasked) { + ivshmem_vector_mask(pdev, i); + } + ivshmem_remove_kvm_msi_virq(s, i); + } + } static void ivshmem_write_config(PCIDevice *pdev, uint32_t address,