From patchwork Thu Nov 21 00:56:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Gibson X-Patchwork-Id: 11255093 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0B8D414DB for ; Thu, 21 Nov 2019 00:58:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E1422208A1 for ; Thu, 21 Nov 2019 00:58:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.b="IcxKWzfs" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726614AbfKUA6r (ORCPT ); Wed, 20 Nov 2019 19:58:47 -0500 Received: from bilbo.ozlabs.org ([203.11.71.1]:44437 "EHLO ozlabs.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726333AbfKUA6q (ORCPT ); Wed, 20 Nov 2019 19:58:46 -0500 Received: by ozlabs.org (Postfix, from userid 1007) id 47JLm41Hqpz9sNx; Thu, 21 Nov 2019 11:58:44 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1574297924; bh=fREJuuqgAEuiDzU2O79CZ0GSodqvDi8dVSOI0miZGuE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IcxKWzfsNp+TDMm/FcmYOF/G6BK8Fwdx7sj3AbbpJBecSmOD3EvejoKZXJ9Hdo5X4 GBJcG9PY7fmZVX3w/91Q8nXcqBSmyoGG35fPguQUP2szAqbFxrua9emPjVKUaoeYJL b8W+cjT9+2k1AAe70PVOMNqijScMEkmCl0xnVIm4= From: David Gibson To: Alex Williamson , clg@kaod.org Cc: groug@kaod.org, philmd@redhat.com, qemu-ppc@nongnu.org, Paolo Bonzini , Jason Wang , Laurent Vivier , David Gibson , kvm@vger.kernel.org, qemu-devel@nongnu.org, Riku Voipio , =?utf-8?q?Marc-And?= =?utf-8?q?r=C3=A9_Lureau?= , Alexey Kardashevskiy Subject: [PATCH 1/5] kvm: Introduce KVM irqchip change notifier Date: Thu, 21 Nov 2019 11:56:03 +1100 Message-Id: <20191121005607.274347-2-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191121005607.274347-1-david@gibson.dropbear.id.au> References: <20191121005607.274347-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Awareness of an in kernel irqchip is usually local to the machine and its top-level interrupt controller. However, in a few cases other things need to know about it. In particular vfio devices need this in order to accelerate interrupt delivery. If interrupt routing is changed, such devices may need to readjust their connection to the KVM irqchip. pci_bus_fire_intx_routing_notifier() exists to do just this. However, for the pseries machine type we have a situation where the routing remains constant but the top-level irq chip itself is changed. This occurs because of PAPR feature negotiation which allows the guest to decide between the older XICS and newer XIVE irq chip models (both of which are paravirtualized). To allow devices like vfio to adjust to this change, introduce a new notifier for the purpose kvm_irqchip_change_notify(). Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson --- accel/kvm/kvm-all.c | 18 ++++++++++++++++++ accel/stubs/kvm-stub.c | 12 ++++++++++++ include/sysemu/kvm.h | 5 +++++ 3 files changed, 35 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 140b0bd8f6..ca00daa2f5 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -149,6 +149,9 @@ static const KVMCapabilityInfo kvm_required_capabilites[] = { KVM_CAP_LAST_INFO }; +static NotifierList kvm_irqchip_change_notifiers = + NOTIFIER_LIST_INITIALIZER(kvm_irqchip_change_notifiers); + #define kvm_slots_lock(kml) qemu_mutex_lock(&(kml)->slots_lock) #define kvm_slots_unlock(kml) qemu_mutex_unlock(&(kml)->slots_lock) @@ -1396,6 +1399,21 @@ void kvm_irqchip_release_virq(KVMState *s, int virq) trace_kvm_irqchip_release_virq(virq); } +void kvm_irqchip_add_change_notifier(Notifier *n) +{ + notifier_list_add(&kvm_irqchip_change_notifiers, n); +} + +void kvm_irqchip_remove_change_notifier(Notifier *n) +{ + notifier_remove(n); +} + +void kvm_irqchip_change_notify(void) +{ + notifier_list_notify(&kvm_irqchip_change_notifiers, NULL); +} + static unsigned int kvm_hash_msi(uint32_t data) { /* This is optimized for IA32 MSI layout. However, no other arch shall diff --git a/accel/stubs/kvm-stub.c b/accel/stubs/kvm-stub.c index 6feb66ed80..82f118d2df 100644 --- a/accel/stubs/kvm-stub.c +++ b/accel/stubs/kvm-stub.c @@ -138,6 +138,18 @@ void kvm_irqchip_commit_routes(KVMState *s) { } +void kvm_irqchip_add_change_notifier(Notifier *n) +{ +} + +void kvm_irqchip_remove_change_notifier(Notifier *n) +{ +} + +void kvm_irqchip_change_notify(void) +{ +} + int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter) { return -ENOSYS; diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index 9d143282bc..9fe233b9bf 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -201,6 +201,7 @@ typedef struct KVMCapabilityInfo { struct KVMState; typedef struct KVMState KVMState; extern KVMState *kvm_state; +typedef struct Notifier Notifier; /* external API */ @@ -401,6 +402,10 @@ int kvm_irqchip_send_msi(KVMState *s, MSIMessage msg); void kvm_irqchip_add_irq_route(KVMState *s, int gsi, int irqchip, int pin); +void kvm_irqchip_add_change_notifier(Notifier *n); +void kvm_irqchip_remove_change_notifier(Notifier *n); +void kvm_irqchip_change_notify(void); + void kvm_get_apic_state(DeviceState *d, struct kvm_lapic_state *kapic); struct kvm_guest_debug; From patchwork Thu Nov 21 00:56:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Gibson X-Patchwork-Id: 11255101 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 691E1109A for ; Thu, 21 Nov 2019 00:58:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 405AC20878 for ; Thu, 21 Nov 2019 00:58:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.b="IrCBuvyd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726757AbfKUA6t (ORCPT ); Wed, 20 Nov 2019 19:58:49 -0500 Received: from ozlabs.org ([203.11.71.1]:35763 "EHLO ozlabs.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726343AbfKUA6r (ORCPT ); Wed, 20 Nov 2019 19:58:47 -0500 Received: by ozlabs.org (Postfix, from userid 1007) id 47JLm42L1Gz9sPW; Thu, 21 Nov 2019 11:58:44 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1574297924; bh=M0mcg5/1en/4FPfwraZYSINmLiks1lDcs9ZPZyf4HFY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IrCBuvydoJarKwc/AqXBCCfhRHgtOv2SIHcrg7RMzyu44qkv46jqKR1N3sGuqc4mh I9yv01OBJiKsd+pNQA58bD4EXpoXcEVNT14kxVqV0ZDbKeXx1WZqgw8B5syMI78vvk FKKxC/uAqg99z7l1n69qh3cmU4omIQGoacX/vSQ4= From: David Gibson To: Alex Williamson , clg@kaod.org Cc: groug@kaod.org, philmd@redhat.com, qemu-ppc@nongnu.org, Paolo Bonzini , Jason Wang , Laurent Vivier , David Gibson , kvm@vger.kernel.org, qemu-devel@nongnu.org, Riku Voipio , =?utf-8?q?Marc-And?= =?utf-8?q?r=C3=A9_Lureau?= , Alexey Kardashevskiy Subject: [PATCH 2/5] vfio/pci: Split vfio_intx_update() Date: Thu, 21 Nov 2019 11:56:04 +1100 Message-Id: <20191121005607.274347-3-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191121005607.274347-1-david@gibson.dropbear.id.au> References: <20191121005607.274347-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This splits the vfio_intx_update() function into one part doing the actual reconnection with the KVM irqchip (vfio_intx_update(), now taking an argument with the new routing) and vfio_intx_routing_notifier() which handles calls to the pci device intx routing notifier and calling vfio_intx_update() when necessary. This will make adding support for the irqchip change notifier easier. Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson --- hw/vfio/pci.c | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 0c55883bba..521289aa7d 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -216,30 +216,18 @@ static void vfio_intx_disable_kvm(VFIOPCIDevice *vdev) #endif } -static void vfio_intx_update(PCIDevice *pdev) +static void vfio_intx_update(VFIOPCIDevice *vdev, PCIINTxRoute *route) { - VFIOPCIDevice *vdev = PCI_VFIO(pdev); - PCIINTxRoute route; Error *err = NULL; - if (vdev->interrupt != VFIO_INT_INTx) { - return; - } - - route = pci_device_route_intx_to_irq(&vdev->pdev, vdev->intx.pin); - - if (!pci_intx_route_changed(&vdev->intx.route, &route)) { - return; /* Nothing changed */ - } - trace_vfio_intx_update(vdev->vbasedev.name, - vdev->intx.route.irq, route.irq); + vdev->intx.route.irq, route->irq); vfio_intx_disable_kvm(vdev); - vdev->intx.route = route; + vdev->intx.route = *route; - if (route.mode != PCI_INTX_ENABLED) { + if (route->mode != PCI_INTX_ENABLED) { return; } @@ -252,6 +240,22 @@ static void vfio_intx_update(PCIDevice *pdev) vfio_intx_eoi(&vdev->vbasedev); } +static void vfio_intx_routing_notifier(PCIDevice *pdev) +{ + VFIOPCIDevice *vdev = PCI_VFIO(pdev); + PCIINTxRoute route; + + if (vdev->interrupt != VFIO_INT_INTx) { + return; + } + + route = pci_device_route_intx_to_irq(&vdev->pdev, vdev->intx.pin); + + if (pci_intx_route_changed(&vdev->intx.route, &route)) { + vfio_intx_update(vdev, &route); + } +} + static int vfio_intx_enable(VFIOPCIDevice *vdev, Error **errp) { uint8_t pin = vfio_pci_read_config(&vdev->pdev, PCI_INTERRUPT_PIN, 1); @@ -2967,7 +2971,8 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) if (vfio_pci_read_config(&vdev->pdev, PCI_INTERRUPT_PIN, 1)) { vdev->intx.mmap_timer = timer_new_ms(QEMU_CLOCK_VIRTUAL, vfio_intx_mmap_enable, vdev); - pci_device_set_intx_routing_notifier(&vdev->pdev, vfio_intx_update); + pci_device_set_intx_routing_notifier(&vdev->pdev, + vfio_intx_routing_notifier); ret = vfio_intx_enable(vdev, errp); if (ret) { goto out_teardown; From patchwork Thu Nov 21 00:56:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Gibson X-Patchwork-Id: 11255095 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 922E2109A for ; Thu, 21 Nov 2019 00:58:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6FFBB208A1 for ; Thu, 21 Nov 2019 00:58:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.b="HTZ8RZOP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726685AbfKUA6r (ORCPT ); Wed, 20 Nov 2019 19:58:47 -0500 Received: from ozlabs.org ([203.11.71.1]:52665 "EHLO ozlabs.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726346AbfKUA6r (ORCPT ); Wed, 20 Nov 2019 19:58:47 -0500 Received: by ozlabs.org (Postfix, from userid 1007) id 47JLm434w0z9sPn; Thu, 21 Nov 2019 11:58:44 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1574297924; bh=Q2CHP0Ke6mkaX6t4hswofWQZGdJTg0Yt2bzdTOJZj8s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HTZ8RZOPsHWsPlug7qeJ77TO3NZIzw84Ege2mnZE9FQXPBQNOMFKManr/FpA/JEVX JpUbc+NncdPM4Lln8rWPT5XCKio3EgYMEr4M73X4n8sjX4u6pacwgYbfxPOAc+45r2 /nFqLBv+EQ3+WLKVvQgA3zAbTm5KsbKFNpAU9tfs= From: David Gibson To: Alex Williamson , clg@kaod.org Cc: groug@kaod.org, philmd@redhat.com, qemu-ppc@nongnu.org, Paolo Bonzini , Jason Wang , Laurent Vivier , David Gibson , kvm@vger.kernel.org, qemu-devel@nongnu.org, Riku Voipio , =?utf-8?q?Marc-And?= =?utf-8?q?r=C3=A9_Lureau?= , Alexey Kardashevskiy Subject: [PATCH 3/5] vfio/pci: Respond to KVM irqchip change notifier Date: Thu, 21 Nov 2019 11:56:05 +1100 Message-Id: <20191121005607.274347-4-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191121005607.274347-1-david@gibson.dropbear.id.au> References: <20191121005607.274347-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org VFIO PCI devices already respond to the pci intx routing notifier, in order to update kernel irqchip mappings when routing is updated. However this won't handle the case where the irqchip itself is replaced by a different model while retaining the same routing. This case can happen on the pseries machine type due to PAPR feature negotiation. To handle that case, add a handler for the irqchip change notifier, which does much the same thing as the routing notifier, but is unconditional, rather than being a no-op when the routing hasn't changed. Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson --- hw/vfio/pci.c | 23 ++++++++++++++++++----- hw/vfio/pci.h | 1 + 2 files changed, 19 insertions(+), 5 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 521289aa7d..95478c2c55 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -256,6 +256,14 @@ static void vfio_intx_routing_notifier(PCIDevice *pdev) } } +static void vfio_irqchip_change(Notifier *notify, void *data) +{ + VFIOPCIDevice *vdev = container_of(notify, VFIOPCIDevice, + irqchip_change_notifier); + + vfio_intx_update(vdev, &vdev->intx.route); +} + static int vfio_intx_enable(VFIOPCIDevice *vdev, Error **errp) { uint8_t pin = vfio_pci_read_config(&vdev->pdev, PCI_INTERRUPT_PIN, 1); @@ -2973,16 +2981,18 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) vfio_intx_mmap_enable, vdev); pci_device_set_intx_routing_notifier(&vdev->pdev, vfio_intx_routing_notifier); + vdev->irqchip_change_notifier.notify = vfio_irqchip_change; + kvm_irqchip_add_change_notifier(&vdev->irqchip_change_notifier); ret = vfio_intx_enable(vdev, errp); if (ret) { - goto out_teardown; + goto out_deregister; } } if (vdev->display != ON_OFF_AUTO_OFF) { ret = vfio_display_probe(vdev, errp); if (ret) { - goto out_teardown; + goto out_deregister; } } if (vdev->enable_ramfb && vdev->dpy == NULL) { @@ -2992,11 +3002,11 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) if (vdev->display_xres || vdev->display_yres) { if (vdev->dpy == NULL) { error_setg(errp, "xres and yres properties require display=on"); - goto out_teardown; + goto out_deregister; } if (vdev->dpy->edid_regs == NULL) { error_setg(errp, "xres and yres properties need edid support"); - goto out_teardown; + goto out_deregister; } } @@ -3020,8 +3030,10 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) return; -out_teardown: +out_deregister: pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); + kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); +out_teardown: vfio_teardown_msi(vdev); vfio_bars_exit(vdev); error: @@ -3064,6 +3076,7 @@ static void vfio_exitfn(PCIDevice *pdev) vfio_unregister_req_notifier(vdev); vfio_unregister_err_notifier(vdev); pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); + kvm_irqchip_remove_change_notifier(&vdev->irqchip_change_notifier); vfio_disable_interrupts(vdev); if (vdev->intx.mmap_timer) { timer_free(vdev->intx.mmap_timer); diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index b329d50338..35626cd63e 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -169,6 +169,7 @@ typedef struct VFIOPCIDevice { bool enable_ramfb; VFIODisplay *dpy; Error *migration_blocker; + Notifier irqchip_change_notifier; } VFIOPCIDevice; uint32_t vfio_pci_read_config(PCIDevice *pdev, uint32_t addr, int len); From patchwork Thu Nov 21 00:56:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Gibson X-Patchwork-Id: 11255097 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5354E14C0 for ; Thu, 21 Nov 2019 00:58:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2A8DE2089D for ; Thu, 21 Nov 2019 00:58:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.b="GH/ey9I1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726701AbfKUA6s (ORCPT ); Wed, 20 Nov 2019 19:58:48 -0500 Received: from ozlabs.org ([203.11.71.1]:34531 "EHLO ozlabs.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726568AbfKUA6s (ORCPT ); Wed, 20 Nov 2019 19:58:48 -0500 Received: by ozlabs.org (Postfix, from userid 1007) id 47JLm44S9lz9sR2; Thu, 21 Nov 2019 11:58:44 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1574297924; bh=uy06rYtNsupoZ3y3vcdSdik4ZpqVFHNbtPoj6Ji6YFU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GH/ey9I17WPcqhxnfntIjzp89v+GocZFviNoVj5tc/u1wRv9nZRm70g0d6H7E4huG Ko5wf/5H1AmADYpIy9Xxx7C+drHdYhuzBZcJJdM9B/L1KHZaBNVIE79jAeczsA/ns9 js14uu6SHHwCWZDkUy0PdzSRpEXyti5fXNxUYejc= From: David Gibson To: Alex Williamson , clg@kaod.org Cc: groug@kaod.org, philmd@redhat.com, qemu-ppc@nongnu.org, Paolo Bonzini , Jason Wang , Laurent Vivier , David Gibson , kvm@vger.kernel.org, qemu-devel@nongnu.org, Riku Voipio , =?utf-8?q?Marc-And?= =?utf-8?q?r=C3=A9_Lureau?= , Alexey Kardashevskiy Subject: [PATCH 4/5] spapr: Handle irq backend changes with VFIO PCI devices Date: Thu, 21 Nov 2019 11:56:06 +1100 Message-Id: <20191121005607.274347-5-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191121005607.274347-1-david@gibson.dropbear.id.au> References: <20191121005607.274347-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org pseries machine type can have one of two different interrupt controllers in use depending on feature negotiation with the guest. Usually this is invisible to devices, because they route to a common set of qemu_irqs which in turn dispatch to the correct back end. VFIO passthrough devices, however, wire themselves up directly to the KVM irqchip for performance, which means they are affected by this change in interrupt controller. To get them to adjust correctly for the change in irqchip, we need to fire the kvm irqchip change notifier. Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson Reviewed-by: Cédric Le Goater --- hw/ppc/spapr_irq.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index 168044be85..1d27034962 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -508,6 +508,12 @@ static void set_active_intc(SpaprMachineState *spapr, } spapr->active_intc = new_intc; + + /* + * We've changed the kernel irqchip, let VFIO devices know they + * need to readjust. + */ + kvm_irqchip_change_notify(); } void spapr_irq_update_active_intc(SpaprMachineState *spapr) From patchwork Thu Nov 21 00:56:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Gibson X-Patchwork-Id: 11255091 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF26914C0 for ; Thu, 21 Nov 2019 00:58:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B967420885 for ; Thu, 21 Nov 2019 00:58:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.b="d5Yl8X8j" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726548AbfKUA6q (ORCPT ); Wed, 20 Nov 2019 19:58:46 -0500 Received: from bilbo.ozlabs.org ([203.11.71.1]:43295 "EHLO ozlabs.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726351AbfKUA6q (ORCPT ); Wed, 20 Nov 2019 19:58:46 -0500 Received: by ozlabs.org (Postfix, from userid 1007) id 47JLm43p1Sz9sQp; Thu, 21 Nov 2019 11:58:44 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gibson.dropbear.id.au; s=201602; t=1574297924; bh=kjLCHlKyKt7jP0NFe7tI8vqbJnysCqs9Js2O7NlIAAQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d5Yl8X8jt5f6X3VYPrSZIeC4Sr8ahHtnfT9r4tRoBX95XiR92/bUGI6IPEAELp+p/ gj4PkS/PiYEpGg2AlbgHsnkezWv9MlYEin1RnA0QOdA8ICddY5MvLGDgMhrae1DmfQ tN1PBz0gCD0y8oOnF1ZySK7+BH5GU5e+myP++TYw= From: David Gibson To: Alex Williamson , clg@kaod.org Cc: groug@kaod.org, philmd@redhat.com, qemu-ppc@nongnu.org, Paolo Bonzini , Jason Wang , Laurent Vivier , David Gibson , kvm@vger.kernel.org, qemu-devel@nongnu.org, Riku Voipio , =?utf-8?q?Marc-And?= =?utf-8?q?r=C3=A9_Lureau?= , Alexey Kardashevskiy Subject: [PATCH 5/5] spapr: Work around spurious warnings from vfio INTx initialization Date: Thu, 21 Nov 2019 11:56:07 +1100 Message-Id: <20191121005607.274347-6-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191121005607.274347-1-david@gibson.dropbear.id.au> References: <20191121005607.274347-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Traditional PCI INTx for vfio devices can only perform well if using an in-kernel irqchip. Therefore, vfio_intx_update() issues a warning if an in kernel irqchip is not available. We usually do have an in-kernel irqchip available for pseries machines on POWER hosts. However, because the platform allows feature negotiation of what interrupt controller model to use, we don't currently initialize it until machine reset. vfio_intx_update() is called (first) from vfio_realize() before that, so it can issue a spurious warning, even if we will have an in kernel irqchip by the time we need it. To workaround this, make a call to spapr_irq_update_active_intc() from spapr_irq_init() which is called at machine realize time, before the vfio realize. This call will be pretty much obsoleted by the later call at reset time, but it serves to suppress the spurious warning from VFIO. Cc: Alex Williamson Cc: Alexey Kardashevskiy Signed-off-by: David Gibson Reviewed-by: Cédric Le Goater --- hw/ppc/spapr_irq.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index 1d27034962..d6bb7fd2d6 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -373,6 +373,14 @@ void spapr_irq_init(SpaprMachineState *spapr, Error **errp) spapr->qirqs = qemu_allocate_irqs(spapr_set_irq, spapr, smc->nr_xirqs + SPAPR_XIRQ_BASE); + + /* + * Mostly we don't actually need this until reset, except that not + * having this set up can cause VFIO devices to issue a + * false-positive warning during realize(), because they don't yet + * have an in-kernel irq chip. + */ + spapr_irq_update_active_intc(spapr); } int spapr_irq_claim(SpaprMachineState *spapr, int irq, bool lsi, Error **errp) @@ -528,7 +536,8 @@ void spapr_irq_update_active_intc(SpaprMachineState *spapr) * this. */ new_intc = SPAPR_INTC(spapr->xive); - } else if (spapr_ovec_test(spapr->ov5_cas, OV5_XIVE_EXPLOIT)) { + } else if (spapr->ov5_cas + && spapr_ovec_test(spapr->ov5_cas, OV5_XIVE_EXPLOIT)) { new_intc = SPAPR_INTC(spapr->xive); } else { new_intc = SPAPR_INTC(spapr->ics);