[v2,0/3] ivshmem: MSI bug fixes

Commit Message

Ladi Prosek Nov. 20, 2017, 12:49 p.m. UTC

On Mon, Nov 20, 2017 at 10:07 AM, Ladi Prosek <[e-mail]> wrote:
> On Sun, Nov 19, 2017 at 9:39 PM,  <[e-mail]> wrote:
>> I just updated to the latest build and applied this patch set, now on VM
>> reset the qemu crashes with the following assert:
>>
>> ivshmem.c:467: ivshmem_add_kvm_msi_virq: Assertion
>> `!s->msi_vectors[vector].pdev' failed.
>
> I see asserts too. Even with v1 on top of QEMU v2.10.0 so I must have
> missed something.
>
> Looking. And, needless to say, these patches should not be applied just yet :)

Ok, here goes it.

1)
ivshmem.c:467: ivshmem_add_kvm_msi_virq: Assertion
`!s->msi_vectors[vector].pdev' failed.

Is caused by the ivshmem device not undoing the effects of
ivshmem_enable_irqfd() on reset.

This fix works for me:

Then either no further changes to this patchset are necessary. Or, if
relying on unmasks/masks (or
msix_vector_use_notifier/msix_vector_release_notifier as it's called
in msix.c) always being balanced is not recommended, the assert will
simply change into an if.

> Thanks!
> Ladi
>
>> On 2017-11-15 18:31, Ladi Prosek wrote:
>>>
>>> Fixes bugs in the ivshmem device implementation uncovered with the new
>>> Windows ivshmem driver:
>>>
>>> https://github.com/virtio-win/kvm-guest-drivers-windows/tree/master/ivshmem
>>>
>>> v1->v2:
>>> * Patch 1 - added reproducer info to commit message (Markus)
>>> * Patch 2 - restructured conditionals, fixed comment formatting (Markus)
>>> * Patch 3 - added reproducer info to commit message (Markus)
>>>
>>> Ladi Prosek (3):
>>>   ivshmem: Don't update non-existent MSI routes
>>>   ivshmem: Always remove irqfd notifiers
>>>   ivshmem: Improve MSI irqfd error handling
>>>
>>>  hw/misc/ivshmem.c | 77
>>> +++++++++++++++++++++++++++++++++++++++++--------------
>>>  1 file changed, 58 insertions(+), 19 deletions(-)
>>
>>

Comments

Ladi Prosek Dec. 8, 2017, 7:48 a.m. UTC | #1

On Mon, Nov 20, 2017 at 1:49 PM, Ladi Prosek <[e-mail]> wrote:
> On Mon, Nov 20, 2017 at 10:07 AM, Ladi Prosek <[e-mail]> wrote:
>> On Sun, Nov 19, 2017 at 9:39 PM,  <[e-mail]> wrote:
>>> I just updated to the latest build and applied this patch set, now on VM
>>> reset the qemu crashes with the following assert:
>>>
>>> ivshmem.c:467: ivshmem_add_kvm_msi_virq: Assertion
>>> `!s->msi_vectors[vector].pdev' failed.
>>
>> I see asserts too. Even with v1 on top of QEMU v2.10.0 so I must have
>> missed something.
>>
>> Looking. And, needless to say, these patches should not be applied just yet :)
>
> Ok, here goes it.
>
> 1)
> ivshmem.c:467: ivshmem_add_kvm_msi_virq: Assertion
> `!s->msi_vectors[vector].pdev' failed.
>
> Is caused by the ivshmem device not undoing the effects of
> ivshmem_enable_irqfd() on reset.
>
> This fix works for me:
>
> --- a/hw/misc/ivshmem.c
> +++ b/hw/misc/ivshmem.c
> @@ -758,10 +758,15 @@ static void ivshmem_msix_vector_use(IVShmemState *s)
>      }
>  }
>
> +
> +static void ivshmem_disable_irqfd(IVShmemState *s);
> +
>  static void ivshmem_reset(DeviceState *d)
>  {
>      IVShmemState *s = IVSHMEM_COMMON(d);
>
> +    ivshmem_disable_irqfd(s);
> +
>      s->intrstatus = 0;
>      s->intrmask = 0;
>      if (ivshmem_has_feature(s, IVSHMEM_MSI)) {

I have added this to v3 as patch 4.

> 2)
> ivshmem.c:354: ivshmem_vector_mask: Assertion `v->unmasked' failed.
>
> which I've been also getting after I enabled Driver Verifier and
> Windows started crashing
> (https://github.com/virtio-win/kvm-guest-drivers-windows/pull/199), is
> caused by the MSI-X code masking already masked vectors on reset. I'm
> going to post a patch similar to this:
>
> --- a/hw/pci/msix.c
> +++ b/hw/pci/msix.c
> @@ -500,11 +500,12 @@ void msix_reset(PCIDevice *dev)
>          return;
>      }
>      msix_clear_all_vectors(dev);
> +    msix_mask_all(dev, dev->msix_entries_nr);
>      dev->config[dev->msix_cap + MSIX_CONTROL_OFFSET] &=
>             ~dev->wmask[dev->msix_cap + MSIX_CONTROL_OFFSET];
>      memset(dev->msix_table, 0, dev->msix_entries_nr * PCI_MSIX_ENTRY_SIZE);
>      memset(dev->msix_pba, 0, QEMU_ALIGN_UP(dev->msix_entries_nr, 64) / 8);
> -    msix_mask_all(dev, dev->msix_entries_nr);
> +    msix_update_function_masked(dev);
>  }
>
>
> Then either no further changes to this patchset are necessary. Or, if
> relying on unmasks/masks (or
> msix_vector_use_notifier/msix_vector_release_notifier as it's called
> in msix.c) always being balanced is not recommended, the assert will
> simply change into an if.

This is fixed in "msix: don't mask already masked vectors on reset":
https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg01362.html

Patch

--- a/hw/misc/ivshmem.c
+++ b/hw/misc/ivshmem.c
@@ -758,10 +758,15 @@  static void ivshmem_msix_vector_use(IVShmemState *s)
     }
 }

+
+static void ivshmem_disable_irqfd(IVShmemState *s);
+
 static void ivshmem_reset(DeviceState *d)
 {
     IVShmemState *s = IVSHMEM_COMMON(d);

+    ivshmem_disable_irqfd(s);
+
     s->intrstatus = 0;
     s->intrmask = 0;
     if (ivshmem_has_feature(s, IVSHMEM_MSI)) {


2)
ivshmem.c:354: ivshmem_vector_mask: Assertion `v->unmasked' failed.

which I've been also getting after I enabled Driver Verifier and
Windows started crashing
(https://github.com/virtio-win/kvm-guest-drivers-windows/pull/199), is
caused by the MSI-X code masking already masked vectors on reset. I'm
going to post a patch similar to this:

--- a/hw/pci/msix.c
+++ b/hw/pci/msix.c
@@ -500,11 +500,12 @@  void msix_reset(PCIDevice *dev)
         return;
     }
     msix_clear_all_vectors(dev);
+    msix_mask_all(dev, dev->msix_entries_nr);
     dev->config[dev->msix_cap + MSIX_CONTROL_OFFSET] &=
            ~dev->wmask[dev->msix_cap + MSIX_CONTROL_OFFSET];
     memset(dev->msix_table, 0, dev->msix_entries_nr * PCI_MSIX_ENTRY_SIZE);
     memset(dev->msix_pba, 0, QEMU_ALIGN_UP(dev->msix_entries_nr, 64) / 8);
-    msix_mask_all(dev, dev->msix_entries_nr);
+    msix_update_function_masked(dev);
 }