| Message ID | 20210611185500.343525-1-peterx@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | docs: Add '-device intel-iommu' entry | expand |
On Fri, Jun 11, 2021 at 02:55:00PM -0400, Peter Xu wrote: > The parameters of intel-iommu device are non-trivial to understand. Add an > entry for it so that people can reference to it when using. > > There're actually a few more options there, but I hide them explicitly because > they shouldn't be used by normal QEMU users. > > Cc: Chao Yang <chayang@redhat.com> > Cc: Lei Yang <leiyang@redhat.com> > Cc: Jing Zhao <jinzhao@redhat.com> > Cc: Jason Wang <jasowang@redhat.com> > Cc: Michael S. Tsirkin <mst@redhat.com> > Cc: Alex Williamson <alex.williamson@redhat.com> > Signed-off-by: Peter Xu <peterx@redhat.com> > --- > qemu-options.hx | 32 ++++++++++++++++++++++++++++++++ > 1 file changed, 32 insertions(+) Ping..
在 2021/6/12 上午2:55, Peter Xu 写道: > The parameters of intel-iommu device are non-trivial to understand. Add an > entry for it so that people can reference to it when using. > > There're actually a few more options there, but I hide them explicitly because > they shouldn't be used by normal QEMU users. > > Cc: Chao Yang <chayang@redhat.com> > Cc: Lei Yang <leiyang@redhat.com> > Cc: Jing Zhao <jinzhao@redhat.com> > Cc: Jason Wang <jasowang@redhat.com> > Cc: Michael S. Tsirkin <mst@redhat.com> > Cc: Alex Williamson <alex.williamson@redhat.com> > Signed-off-by: Peter Xu <peterx@redhat.com> > --- > qemu-options.hx | 32 ++++++++++++++++++++++++++++++++ > 1 file changed, 32 insertions(+) > > diff --git a/qemu-options.hx b/qemu-options.hx > index 14258784b3a..4bb04243907 100644 > --- a/qemu-options.hx > +++ b/qemu-options.hx > @@ -926,6 +926,38 @@ SRST > > ``-device pci-ipmi-bt,bmc=id`` > Like the KCS interface, but defines a BT interface on the PCI bus. > + > +``-device intel-iommu[,option=...]`` > + This is only supported by ``-machine q35``, which will enable Intel VT-d > + emulation within the guest. It supports below options: > + > + ``intremap=on|off`` (default: auto) > + This enables interrupt remapping feature in the guest. I think we don't need "in the guest" here. Other than this: Reviewed-by: Jason Wang <jasowang@redhat.com> Thanks > It's required > + to enable complete x2apic. Currently it only supports kvm > + kernel-irqchip modes ``off`` or ``split``. Full kernel-irqchip is not > + yet supported. > + > + ``caching-mode=on|off`` (default: off) > + This enables caching mode for the VT-d emulated device. When > + caching-mode is enabled, each guest DMA buffer mapping will generate an > + IOTLB invalidation from the guest IOMMU driver to the vIOMMU device in > + a synchronous way. It is required for ``-device vfio-pci`` to work > + with the VT-d device, because host assigned devices requires to setup > + the DMA mapping on the host before guest DMA starts. > + > + ``device-iotlb=on|off`` (default: off) > + This enables device-iotlb capability for the emulated VT-d device. So > + far virtio/vhost should be the only real user for this parameter, > + paired with ats=on configured for the device. > + > + ``aw-bits=39|48`` (default: 39) > + This decides the address width of IOVA address space. The address > + space has 39 bits width for 3-level IOMMU page tables, and 48 bits for > + 4-level IOMMU page tables. > + > + Please also refer to the wiki page for general scenarios of VT-d > + emulation in QEMU: https://wiki.qemu.org/Features/VT-d. > + > ERST > > DEF("name", HAS_ARG, QEMU_OPTION_name,
Hi Peter, On 6/11/21 8:55 PM, Peter Xu wrote: > The parameters of intel-iommu device are non-trivial to understand. Add an > entry for it so that people can reference to it when using. > > There're actually a few more options there, but I hide them explicitly because > they shouldn't be used by normal QEMU users. > > Cc: Chao Yang <chayang@redhat.com> > Cc: Lei Yang <leiyang@redhat.com> > Cc: Jing Zhao <jinzhao@redhat.com> > Cc: Jason Wang <jasowang@redhat.com> > Cc: Michael S. Tsirkin <mst@redhat.com> > Cc: Alex Williamson <alex.williamson@redhat.com> > Signed-off-by: Peter Xu <peterx@redhat.com> > --- > qemu-options.hx | 32 ++++++++++++++++++++++++++++++++ > 1 file changed, 32 insertions(+) > > diff --git a/qemu-options.hx b/qemu-options.hx > index 14258784b3a..4bb04243907 100644 > --- a/qemu-options.hx > +++ b/qemu-options.hx > @@ -926,6 +926,38 @@ SRST > > ``-device pci-ipmi-bt,bmc=id`` > Like the KCS interface, but defines a BT interface on the PCI bus. > + > +``-device intel-iommu[,option=...]`` > + This is only supported by ``-machine q35``, which will enable Intel VT-d > + emulation within the guest. It supports below options: > + > + ``intremap=on|off`` (default: auto) > + This enables interrupt remapping feature in the guest. It's required > + to enable complete x2apic. Currently it only supports kvm > + kernel-irqchip modes ``off`` or ``split``. Full kernel-irqchip is not > + yet supported. maybe explain how the default is chosen (based on kernel-irqhcip mode?) > + > + ``caching-mode=on|off`` (default: off) > + This enables caching mode for the VT-d emulated device. When > + caching-mode is enabled, each guest DMA buffer mapping will generate an > + IOTLB invalidation from the guest IOMMU driver to the vIOMMU device in > + a synchronous way. It is required for ``-device vfio-pci`` to work > + with the VT-d device, because host assigned devices requires to setup > + the DMA mapping on the host before guest DMA starts. > + > + ``device-iotlb=on|off`` (default: off) > + This enables device-iotlb capability for the emulated VT-d device. So > + far virtio/vhost should be the only real user for this parameter, I thought if was needed for vhost. What is the usage for virtio? > + paired with ats=on configured for the device. What is not obvious to me is whether there is a kind of diagnosis whether device-iotlb and caching mode settings are consistent with vfio/vhost usage. Does qemu exit if there is inconsistency? > + > + ``aw-bits=39|48`` (default: 39) > + This decides the address width of IOVA address space. The address > + space has 39 bits width for 3-level IOMMU page tables, and 48 bits for > + 4-level IOMMU page tables. > + > + Please also refer to the wiki page for general scenarios of VT-d > + emulation in QEMU: https://wiki.qemu.org/Features/VT-d. > + > ERST > > DEF("name", HAS_ARG, QEMU_OPTION_name, > Besides, it is quite useful. Thanks Eric
> From: Peter Xu < peterx@redhat.com > > Sent: Saturday, June 12, 2021 2:55 AM > > The parameters of intel-iommu device are non-trivial to understand. Add > an > entry for it so that people can reference to it when using. > > There're actually a few more options there, but I hide them explicitly > because > they shouldn't be used by normal QEMU users. yes, it's a good start. Looks good to me. Reviewed-by: Liu Yi L <yi.l.liu@intel.com> Regards, Yi Liu > > Cc: Chao Yang <chayang@redhat.com> > Cc: Lei Yang <leiyang@redhat.com> > Cc: Jing Zhao <jinzhao@redhat.com> > Cc: Jason Wang <jasowang@redhat.com> > Cc: Michael S. Tsirkin <mst@redhat.com> > Cc: Alex Williamson <alex.williamson@redhat.com> > Signed-off-by: Peter Xu <peterx@redhat.com> > --- > qemu-options.hx | 32 ++++++++++++++++++++++++++++++++ > 1 file changed, 32 insertions(+) > > diff --git a/qemu-options.hx b/qemu-options.hx > index 14258784b3a..4bb04243907 100644 > --- a/qemu-options.hx > +++ b/qemu-options.hx > @@ -926,6 +926,38 @@ SRST > > ``-device pci-ipmi-bt,bmc=id`` > Like the KCS interface, but defines a BT interface on the PCI bus. > + > +``-device intel-iommu[,option=...]`` > + This is only supported by ``-machine q35``, which will enable Intel VT-d > + emulation within the guest. It supports below options: > + > + ``intremap=on|off`` (default: auto) > + This enables interrupt remapping feature in the guest. It's required > + to enable complete x2apic. Currently it only supports kvm > + kernel-irqchip modes ``off`` or ``split``. Full kernel-irqchip is not > + yet supported. > + > + ``caching-mode=on|off`` (default: off) > + This enables caching mode for the VT-d emulated device. When > + caching-mode is enabled, each guest DMA buffer mapping will > generate an > + IOTLB invalidation from the guest IOMMU driver to the vIOMMU > device in > + a synchronous way. It is required for ``-device vfio-pci`` to work > + with the VT-d device, because host assigned devices requires to setup > + the DMA mapping on the host before guest DMA starts. > + > + ``device-iotlb=on|off`` (default: off) > + This enables device-iotlb capability for the emulated VT-d device. So > + far virtio/vhost should be the only real user for this parameter, > + paired with ats=on configured for the device. > + > + ``aw-bits=39|48`` (default: 39) > + This decides the address width of IOVA address space. The address > + space has 39 bits width for 3-level IOMMU page tables, and 48 bits for > + 4-level IOMMU page tables. > + > + Please also refer to the wiki page for general scenarios of VT-d > + emulation in QEMU: https://wiki.qemu.org/Features/VT-d. > + > ERST > > DEF("name", HAS_ARG, QEMU_OPTION_name, > -- > 2.31.1 >
On Wed, Jul 07, 2021 at 11:35:43AM +0800, Jason Wang wrote: > > 在 2021/6/12 上午2:55, Peter Xu 写道: > > The parameters of intel-iommu device are non-trivial to understand. Add an > > entry for it so that people can reference to it when using. > > > > There're actually a few more options there, but I hide them explicitly because > > they shouldn't be used by normal QEMU users. > > > > Cc: Chao Yang <chayang@redhat.com> > > Cc: Lei Yang <leiyang@redhat.com> > > Cc: Jing Zhao <jinzhao@redhat.com> > > Cc: Jason Wang <jasowang@redhat.com> > > Cc: Michael S. Tsirkin <mst@redhat.com> > > Cc: Alex Williamson <alex.williamson@redhat.com> > > Signed-off-by: Peter Xu <peterx@redhat.com> > > --- > > qemu-options.hx | 32 ++++++++++++++++++++++++++++++++ > > 1 file changed, 32 insertions(+) > > > > diff --git a/qemu-options.hx b/qemu-options.hx > > index 14258784b3a..4bb04243907 100644 > > --- a/qemu-options.hx > > +++ b/qemu-options.hx > > @@ -926,6 +926,38 @@ SRST > > ``-device pci-ipmi-bt,bmc=id`` > > Like the KCS interface, but defines a BT interface on the PCI bus. > > + > > +``-device intel-iommu[,option=...]`` > > + This is only supported by ``-machine q35``, which will enable Intel VT-d > > + emulation within the guest. It supports below options: > > + > > + ``intremap=on|off`` (default: auto) > > + This enables interrupt remapping feature in the guest. > > > I think we don't need "in the guest" here. Sure. > > Other than this: > > Reviewed-by: Jason Wang <jasowang@redhat.com> Thanks,
On Wed, Jul 07, 2021 at 11:29:47AM +0200, Eric Auger wrote: > Hi Peter, Hi, Eric, > > On 6/11/21 8:55 PM, Peter Xu wrote: > > The parameters of intel-iommu device are non-trivial to understand. Add an > > entry for it so that people can reference to it when using. > > > > There're actually a few more options there, but I hide them explicitly because > > they shouldn't be used by normal QEMU users. > > > > Cc: Chao Yang <chayang@redhat.com> > > Cc: Lei Yang <leiyang@redhat.com> > > Cc: Jing Zhao <jinzhao@redhat.com> > > Cc: Jason Wang <jasowang@redhat.com> > > Cc: Michael S. Tsirkin <mst@redhat.com> > > Cc: Alex Williamson <alex.williamson@redhat.com> > > Signed-off-by: Peter Xu <peterx@redhat.com> > > --- > > qemu-options.hx | 32 ++++++++++++++++++++++++++++++++ > > 1 file changed, 32 insertions(+) > > > > diff --git a/qemu-options.hx b/qemu-options.hx > > index 14258784b3a..4bb04243907 100644 > > --- a/qemu-options.hx > > +++ b/qemu-options.hx > > @@ -926,6 +926,38 @@ SRST > > > > ``-device pci-ipmi-bt,bmc=id`` > > Like the KCS interface, but defines a BT interface on the PCI bus. > > + > > +``-device intel-iommu[,option=...]`` > > + This is only supported by ``-machine q35``, which will enable Intel VT-d > > + emulation within the guest. It supports below options: > > + > > + ``intremap=on|off`` (default: auto) > > + This enables interrupt remapping feature in the guest. It's required > > + to enable complete x2apic. Currently it only supports kvm > > + kernel-irqchip modes ``off`` or ``split``. Full kernel-irqchip is not > > + yet supported. > maybe explain how the default is chosen (based on kernel-irqhcip mode?) Will do. > > + > > + ``caching-mode=on|off`` (default: off) > > + This enables caching mode for the VT-d emulated device. When > > + caching-mode is enabled, each guest DMA buffer mapping will generate an > > + IOTLB invalidation from the guest IOMMU driver to the vIOMMU device in > > + a synchronous way. It is required for ``-device vfio-pci`` to work > > + with the VT-d device, because host assigned devices requires to setup > > + the DMA mapping on the host before guest DMA starts. > > + > > + ``device-iotlb=on|off`` (default: off) > > + This enables device-iotlb capability for the emulated VT-d device. So > > + far virtio/vhost should be the only real user for this parameter, > I thought if was needed for vhost. What is the usage for virtio? I used "virtio/vhost" because vhost is only one type of virtio backend, so normally they should be used in pair. Also, the "ats=on" parameter should be an option to virtio frontend rather than vhost, hence I mentioned both. > > + paired with ats=on configured for the device. > What is not obvious to me is whether there is a kind of diagnosis > whether device-iotlb and caching mode settings are consistent with > vfio/vhost usage. Does qemu exit if there is inconsistency? Caching mode check is done in vtd_machine_done_notify_one(). I think there's no such check for vhost, however iirc vhost should work with/without ats=on; I did't check deeper. > > + > > + ``aw-bits=39|48`` (default: 39) > > + This decides the address width of IOVA address space. The address > > + space has 39 bits width for 3-level IOMMU page tables, and 48 bits for > > + 4-level IOMMU page tables. > > + > > + Please also refer to the wiki page for general scenarios of VT-d > > + emulation in QEMU: https://wiki.qemu.org/Features/VT-d. > > + > > ERST > > > > DEF("name", HAS_ARG, QEMU_OPTION_name, > > > Besides, it is quite useful. Thanks for taking a look!
Hi Peter, On 7/7/21 5:35 PM, Peter Xu wrote: > On Wed, Jul 07, 2021 at 11:29:47AM +0200, Eric Auger wrote: >> Hi Peter, > > Hi, Eric, > >> >> On 6/11/21 8:55 PM, Peter Xu wrote: >>> The parameters of intel-iommu device are non-trivial to understand. Add an >>> entry for it so that people can reference to it when using. >>> >>> There're actually a few more options there, but I hide them explicitly because >>> they shouldn't be used by normal QEMU users. >>> >>> Cc: Chao Yang <chayang@redhat.com> >>> Cc: Lei Yang <leiyang@redhat.com> >>> Cc: Jing Zhao <jinzhao@redhat.com> >>> Cc: Jason Wang <jasowang@redhat.com> >>> Cc: Michael S. Tsirkin <mst@redhat.com> >>> Cc: Alex Williamson <alex.williamson@redhat.com> >>> Signed-off-by: Peter Xu <peterx@redhat.com> >>> --- >>> qemu-options.hx | 32 ++++++++++++++++++++++++++++++++ >>> 1 file changed, 32 insertions(+) >>> >>> diff --git a/qemu-options.hx b/qemu-options.hx >>> index 14258784b3a..4bb04243907 100644 >>> --- a/qemu-options.hx >>> +++ b/qemu-options.hx >>> @@ -926,6 +926,38 @@ SRST >>> >>> ``-device pci-ipmi-bt,bmc=id`` >>> Like the KCS interface, but defines a BT interface on the PCI bus. >>> + >>> +``-device intel-iommu[,option=...]`` >>> + This is only supported by ``-machine q35``, which will enable Intel VT-d >>> + emulation within the guest. It supports below options: >>> + >>> + ``intremap=on|off`` (default: auto) >>> + This enables interrupt remapping feature in the guest. It's required >>> + to enable complete x2apic. Currently it only supports kvm >>> + kernel-irqchip modes ``off`` or ``split``. Full kernel-irqchip is not >>> + yet supported. >> maybe explain how the default is chosen (based on kernel-irqhcip mode?) > > Will do. > >>> + >>> + ``caching-mode=on|off`` (default: off) >>> + This enables caching mode for the VT-d emulated device. When >>> + caching-mode is enabled, each guest DMA buffer mapping will generate an >>> + IOTLB invalidation from the guest IOMMU driver to the vIOMMU device in >>> + a synchronous way. It is required for ``-device vfio-pci`` to work >>> + with the VT-d device, because host assigned devices requires to setup >>> + the DMA mapping on the host before guest DMA starts. >>> + >>> + ``device-iotlb=on|off`` (default: off) >>> + This enables device-iotlb capability for the emulated VT-d device. So >>> + far virtio/vhost should be the only real user for this parameter, >> I thought if was needed for vhost. What is the usage for virtio? > > I used "virtio/vhost" because vhost is only one type of virtio backend, so > normally they should be used in pair. Also, the "ats=on" parameter should be > an option to virtio frontend rather than vhost, hence I mentioned both. OK makes sense. > >>> + paired with ats=on configured for the device. >> What is not obvious to me is whether there is a kind of diagnosis >> whether device-iotlb and caching mode settings are consistent with >> vfio/vhost usage. Does qemu exit if there is inconsistency? > > Caching mode check is done in vtd_machine_done_notify_one(). > > I think there's no such check for vhost, however iirc vhost should work > with/without ats=on; I did't check deeper. OK > >>> + >>> + ``aw-bits=39|48`` (default: 39) >>> + This decides the address width of IOVA address space. The address >>> + space has 39 bits width for 3-level IOMMU page tables, and 48 bits for >>> + 4-level IOMMU page tables. >>> + >>> + Please also refer to the wiki page for general scenarios of VT-d >>> + emulation in QEMU: https://wiki.qemu.org/Features/VT-d. >>> + >>> ERST >>> >>> DEF("name", HAS_ARG, QEMU_OPTION_name, >>> >> Besides, it is quite useful. > > Thanks for taking a look! > Besides, Reviewed-by: Eric Auger <eric.auger@redhat.com> Eric
diff --git a/qemu-options.hx b/qemu-options.hx index 14258784b3a..4bb04243907 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -926,6 +926,38 @@ SRST ``-device pci-ipmi-bt,bmc=id`` Like the KCS interface, but defines a BT interface on the PCI bus. + +``-device intel-iommu[,option=...]`` + This is only supported by ``-machine q35``, which will enable Intel VT-d + emulation within the guest. It supports below options: + + ``intremap=on|off`` (default: auto) + This enables interrupt remapping feature in the guest. It's required + to enable complete x2apic. Currently it only supports kvm + kernel-irqchip modes ``off`` or ``split``. Full kernel-irqchip is not + yet supported. + + ``caching-mode=on|off`` (default: off) + This enables caching mode for the VT-d emulated device. When + caching-mode is enabled, each guest DMA buffer mapping will generate an + IOTLB invalidation from the guest IOMMU driver to the vIOMMU device in + a synchronous way. It is required for ``-device vfio-pci`` to work + with the VT-d device, because host assigned devices requires to setup + the DMA mapping on the host before guest DMA starts. + + ``device-iotlb=on|off`` (default: off) + This enables device-iotlb capability for the emulated VT-d device. So + far virtio/vhost should be the only real user for this parameter, + paired with ats=on configured for the device. + + ``aw-bits=39|48`` (default: 39) + This decides the address width of IOVA address space. The address + space has 39 bits width for 3-level IOMMU page tables, and 48 bits for + 4-level IOMMU page tables. + + Please also refer to the wiki page for general scenarios of VT-d + emulation in QEMU: https://wiki.qemu.org/Features/VT-d. + ERST DEF("name", HAS_ARG, QEMU_OPTION_name,
The parameters of intel-iommu device are non-trivial to understand. Add an entry for it so that people can reference to it when using. There're actually a few more options there, but I hide them explicitly because they shouldn't be used by normal QEMU users. Cc: Chao Yang <chayang@redhat.com> Cc: Lei Yang <leiyang@redhat.com> Cc: Jing Zhao <jinzhao@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> --- qemu-options.hx | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)