| Message ID | 4-v3-2cf356649677+a32-intel_no_snoop_jgg@nvidia.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Make the iommu driver no-snoop block feature consistent | expand |
Hi, We encounter a issue with the patch: our platform is ARM64, and we run DPDK with smmu disable on VM (without iommu=smmuv3 etc), so we use noiommu mode with enable_unsafe_noiommu_mode=1 to passthrough the device to VM with following steps (those steps are on VM) : insmod vfio.ko enable_unsafe_noiommu_mode=1 insmod vfio_virqfd.ko insmod vfio-pci-core.ko insmdo vfio-pci.ko insmod vfio_iommu_type1.ko echo vfio-pci > /sys/bus/pci/devices/0000:00:02.0/driver_override echo 0000:00:02.0 > /sys/bus/pci/drivers_probe ------------------ failed I find that vfio-pci device is not probed because of the additional check. It works well without this patch. Do we need to skip the check if enable_unsafe_noiommu_mode=1? Best regards, Xiang Chen 在 2022/4/11 23:16, Jason Gunthorpe 写道: > IOMMU_CACHE means that normal DMAs do not require any additional coherency > mechanism and is the basic uAPI that VFIO exposes to userspace. For > instance VFIO applications like DPDK will not work if additional coherency > operations are required. > > Therefore check IOMMU_CAP_CACHE_COHERENCY like vdpa & usnic do before > allowing an IOMMU backed VFIO device to be created. > > Reviewed-by: Kevin Tian <kevin.tian@intel.com> > Acked-by: Alex Williamson <alex.williamson@redhat.com> > Acked-by: Robin Murphy <robin.murphy@arm.com> > Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> > --- > drivers/vfio/vfio.c | 7 +++++++ > 1 file changed, 7 insertions(+) > > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c > index a4555014bd1e72..9edad767cfdad3 100644 > --- a/drivers/vfio/vfio.c > +++ b/drivers/vfio/vfio.c > @@ -815,6 +815,13 @@ static int __vfio_register_dev(struct vfio_device *device, > > int vfio_register_group_dev(struct vfio_device *device) > { > + /* > + * VFIO always sets IOMMU_CACHE because we offer no way for userspace to > + * restore cache coherency. > + */ > + if (!iommu_capable(device->dev->bus, IOMMU_CAP_CACHE_COHERENCY)) > + return -EINVAL; > + > return __vfio_register_dev(device, > vfio_group_find_or_alloc(device->dev)); > }
On Mon, Jul 04, 2022 at 09:21:26PM +0800, chenxiang (M) wrote: > Hi, > > We encounter a issue with the patch: our platform is ARM64, and we run DPDK > with smmu disable on VM (without iommu=smmuv3 etc), > > so we use noiommu mode with enable_unsafe_noiommu_mode=1 to passthrough the > device to VM with following steps (those steps are on VM) : > > insmod vfio.ko enable_unsafe_noiommu_mode=1 > insmod vfio_virqfd.ko > insmod vfio-pci-core.ko > insmdo vfio-pci.ko > insmod vfio_iommu_type1.ko > > echo vfio-pci > /sys/bus/pci/devices/0000:00:02.0/driver_override > echo 0000:00:02.0 > /sys/bus/pci/drivers_probe ------------------ failed > > I find that vfio-pci device is not probed because of the additional check. > It works well without this patch. > > Do we need to skip the check if enable_unsafe_noiommu_mode=1? Yes, that is definately an unintended mistake. Thanks, Jason
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index a4555014bd1e72..9edad767cfdad3 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -815,6 +815,13 @@ static int __vfio_register_dev(struct vfio_device *device, int vfio_register_group_dev(struct vfio_device *device) { + /* + * VFIO always sets IOMMU_CACHE because we offer no way for userspace to + * restore cache coherency. + */ + if (!iommu_capable(device->dev->bus, IOMMU_CAP_CACHE_COHERENCY)) + return -EINVAL; + return __vfio_register_dev(device, vfio_group_find_or_alloc(device->dev)); }