[RFC,00/12] iommu/vt-d: Remove superfluous IOMMU IOTLB invalidations

Message

Zhang, Tina Oct. 17, 2023, 3:20 a.m. UTC

This series based on "Share sva" patch-set[1], aims to remove superfluous
IOMMU IOTLB invalidations in VT-d driver.

In the current VT-d driver, IOMMU IOTLB invalidation commands and
device-TLB commands are performed per device, which leads to superfluous
IOTLB invalidations. For example, if there are four devices behind a IOMMU
are attached to one sva domain (which could be a common case in
virtualization scenarios where one virtual IOMMU working for all the
virtual devices), four IOTLB invalidation commands and four device-IOTLB
invalidation commands will be issued. However, only one IOTLB invalidation
command and four device-IOTLB invalidation commands are necessary.
Superfluous IOMMU IOTLB invalidations impact run-time performance.

Although the goal could be built straight-forwardly in the current VT-d
driver, some refactoring works are considered necessary before landing the
one solving the problem of redundant IOTLB invalidations:

1) VT-d driver uses different structures to keep attached device info.
For default domain, it uses struct dev_pasids_info and some related fields
of struct dmar_domain. For sva domain, it uses struct intel_svm and
struct intel_svm_dev. The reason of it is because previously the sva
domain is not shared among devices behind different IOMMUs and
therefore dmar_domain and its fields cannot be used globally to keep
all attached device info. After the patch-set[1] gets acceptance, sva
domain is global to the attached devices. Thus, retiring struct
intel_svm/intel_svm_dev is the main refactoring work of this patch-set.

2) Most logic of intel_svm_set_dev_pasid() can be covered by
intel_iommu_set_dev_pasid(). Refactoring both intel_svm_set_dev_pasid()
and intel_iommu_set_dev_pasid() and let the former call the latter for
set_dev_pasid operation to avoid duplicating code.

3) Last but not least, struct mmu_notifier is proposed to iommu_domain.
This is a change to IOMMU core, which helps to centralize info required by
sva to sva domain and therefore can help cleanup the code in IOMMU drivers.

This patchset is on github per-iommu_IOTLB_invalidation branch[2].

[1]: https://lore.kernel.org/linux-iommu/[e-mail]/
[2]: https://github.com/TinaZhangZW/linux/tree/per-iommu_IOTLB_invalidation

Tina Zhang (12):
  iommu/vt-d: Retire the treatment for revoking PASIDs with pending
    pgfaults
  iommu/vt-d: Remove initialization for dynamically heap-allocated
    rcu_head
  iommu/vt-d: Retire intel_svm_bind_mm()
  iommu/vt-d: Make dev_to_intel_iommu() helper global
  iommu/vt-d: Retire struct intel_svm_dev
  iommu: Add mmu_notifier to sva domain
  iommu/vt-d: Retire struct intel_svm
  iommu/vt-d: Use RCU for dev_pasids list updates in
    set/remove_dev_pasid()
  iommu/vt-d: Refactor intel_iommu_set_dev_pasid()
  iommu/vt-d: Refactor intel_iommu_remove_dev_pasid()
  iommu/vt-d: Use intel_iommu_set_dev_pasid() for sva domain
  iommu/vt-d: Remove superfluous IOMMU IOTLB invalidations

 drivers/iommu/intel/iommu.c |  91 +++++++++----
 drivers/iommu/intel/iommu.h |  30 ++---
 drivers/iommu/intel/svm.c   | 261 ++++++++----------------------------
 include/linux/iommu.h       |   2 +
 4 files changed, 128 insertions(+), 256 deletions(-)