| Message ID | 20230330164944.75481-1-viktor@daynix.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [RFC] vhost: enable IOMMU_NOTIFIER_UNMAP events handling when device-iotlb=on | expand |
On Thu, Mar 30, 2023 at 7:49 PM Viktor Prutyanov <viktor@daynix.com> wrote: > > Even if Device-TLB and PCI ATS is enabled, the guest can reject to use > it. For example, this situation appears when Windows Server 2022 is > running with intel-iommu with device-iotlb=on and virtio-net-pci with > vhost=on. The guest implies that no address translation info cached in > device IOTLB and doesn't send device IOTLB invalidation commands. So, > it leads to irrelevant address translations in vhost-net in the host > kernel. Therefore network frames from the guest in host tap interface > contains wrong payload data. > > This patch enables IOTLB unmap events (IOMMU_NOTIFIER_UNMAP) along with > Device-TLB unmap events (IOMMU_NOTIFIER_DEVIOTLB_UNMAP) handling for > proper vhost IOTLB unmapping when the guest isn't aware of Device-TLB. > > Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2001312 > Signed-off-by: Viktor Prutyanov <viktor@daynix.com> > --- > > Tested on Windows Server 2022 and Fedora guests with > -device virtio-net-pci,bus=pci.3,netdev=nd0,iommu_platform=on,ats=on > -netdev tap,id=nd0,ifname=tap1,script=no,downscript=no,vhost=on > -device intel-iommu,intremap=on,eim=on,device-iotlb=on/off > > hw/virtio/vhost.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c > index a266396576..968ca18fce 100644 > --- a/hw/virtio/vhost.c > +++ b/hw/virtio/vhost.c > @@ -796,7 +796,7 @@ static void vhost_iommu_region_add(MemoryListener *listener, > iommu_idx = memory_region_iommu_attrs_to_index(iommu_mr, > MEMTXATTRS_UNSPECIFIED); > iommu_notifier_init(&iommu->n, vhost_iommu_unmap_notify, > - IOMMU_NOTIFIER_DEVIOTLB_UNMAP, > + IOMMU_NOTIFIER_DEVIOTLB_UNMAP | IOMMU_NOTIFIER_UNMAP, > section->offset_within_region, > int128_get64(end), > iommu_idx); > -- > 2.35.1 > ping
On Thu, Apr 13, 2023 at 2:24 PM Viktor Prutyanov <viktor@daynix.com> wrote: > > On Thu, Mar 30, 2023 at 7:49 PM Viktor Prutyanov <viktor@daynix.com> wrote: > > > > Even if Device-TLB and PCI ATS is enabled, the guest can reject to use > > it. For example, this situation appears when Windows Server 2022 is > > running with intel-iommu with device-iotlb=on and virtio-net-pci with > > vhost=on. The guest implies that no address translation info cached in > > device IOTLB and doesn't send device IOTLB invalidation commands. So, > > it leads to irrelevant address translations in vhost-net in the host > > kernel. Therefore network frames from the guest in host tap interface > > contains wrong payload data. > > > > This patch enables IOTLB unmap events (IOMMU_NOTIFIER_UNMAP) along with > > Device-TLB unmap events (IOMMU_NOTIFIER_DEVIOTLB_UNMAP) handling for > > proper vhost IOTLB unmapping when the guest isn't aware of Device-TLB. > > > > Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2001312 > > Signed-off-by: Viktor Prutyanov <viktor@daynix.com> > > --- > > > > Tested on Windows Server 2022 and Fedora guests with > > -device virtio-net-pci,bus=pci.3,netdev=nd0,iommu_platform=on,ats=on > > -netdev tap,id=nd0,ifname=tap1,script=no,downscript=no,vhost=on > > -device intel-iommu,intremap=on,eim=on,device-iotlb=on/off > > > > hw/virtio/vhost.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c > > index a266396576..968ca18fce 100644 > > --- a/hw/virtio/vhost.c > > +++ b/hw/virtio/vhost.c > > @@ -796,7 +796,7 @@ static void vhost_iommu_region_add(MemoryListener *listener, > > iommu_idx = memory_region_iommu_attrs_to_index(iommu_mr, > > MEMTXATTRS_UNSPECIFIED); > > iommu_notifier_init(&iommu->n, vhost_iommu_unmap_notify, > > - IOMMU_NOTIFIER_DEVIOTLB_UNMAP, > > + IOMMU_NOTIFIER_DEVIOTLB_UNMAP | IOMMU_NOTIFIER_UNMAP, > > section->offset_within_region, > > int128_get64(end), > > iommu_idx); This looks kind of tricky. E.g when ATS is enabled, there could be unnecessary flues. Or at least it requires some comment to explain. I think the correct fix is: 1) introduce a transport specific method for querying if it supports device IOTLB 2) in the pci implementation hook it to some like pice_ats_enabled() 3) so we can decide if device IOTLB is enabled, go for DEVIOTLB_UNMAP otherwise a simple UNMAP. Thanks > > -- > > 2.35.1 > > > > ping >
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index a266396576..968ca18fce 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -796,7 +796,7 @@ static void vhost_iommu_region_add(MemoryListener *listener, iommu_idx = memory_region_iommu_attrs_to_index(iommu_mr, MEMTXATTRS_UNSPECIFIED); iommu_notifier_init(&iommu->n, vhost_iommu_unmap_notify, - IOMMU_NOTIFIER_DEVIOTLB_UNMAP, + IOMMU_NOTIFIER_DEVIOTLB_UNMAP | IOMMU_NOTIFIER_UNMAP, section->offset_within_region, int128_get64(end), iommu_idx);
Even if Device-TLB and PCI ATS is enabled, the guest can reject to use it. For example, this situation appears when Windows Server 2022 is running with intel-iommu with device-iotlb=on and virtio-net-pci with vhost=on. The guest implies that no address translation info cached in device IOTLB and doesn't send device IOTLB invalidation commands. So, it leads to irrelevant address translations in vhost-net in the host kernel. Therefore network frames from the guest in host tap interface contains wrong payload data. This patch enables IOTLB unmap events (IOMMU_NOTIFIER_UNMAP) along with Device-TLB unmap events (IOMMU_NOTIFIER_DEVIOTLB_UNMAP) handling for proper vhost IOTLB unmapping when the guest isn't aware of Device-TLB. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2001312 Signed-off-by: Viktor Prutyanov <viktor@daynix.com> --- Tested on Windows Server 2022 and Fedora guests with -device virtio-net-pci,bus=pci.3,netdev=nd0,iommu_platform=on,ats=on -netdev tap,id=nd0,ifname=tap1,script=no,downscript=no,vhost=on -device intel-iommu,intremap=on,eim=on,device-iotlb=on/off hw/virtio/vhost.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)