[RFC,0/5] KVM/ARM64 Add support for pinned VMIDs

Message

Shameerali Kolothum Thodi Feb. 22, 2021, 3:53 p.m. UTC

On an ARM64 system with a SMMUv3 implementation that fully supports
Broadcast TLB Maintenance(BTM) feature as part of the Distributed
Virtual Memory(DVM) protocol, the CPU TLB invalidate instructions are
received by SMMUv3. This is very useful when the SMMUv3 shares the
page tables with the CPU(eg: Guest SVA use case). For this to work,
the SMMU must use the same VMID that is allocated by KVM to configure
the stage 2 translations. At present KVM VMID allocations are recycled
on rollover and may change as a result. This will create issues if we
have to share the KVM VMID with SMMU.
 
Please see the discussion here,
https://lore.kernel.org/linux-iommu/20200522101755.GA3453945@myrica/

This series proposes a way to share the VMID between KVM and IOMMU
driver by,

1. Splitting the KVM VMID space into two equal halves based on the
   command line option "kvm-arm.pinned_vmid_enable".
2. First half of the VMID space follows the normal recycle on rollover
   policy.
3. Second half of the VMID space doesn't roll over and is used to
   allocate pinned VMIDs.
4. Provides helper function to retrieve the KVM instance associated
   with a device(if it is part of a vfio group).
5. Introduces generic interfaces to get/put pinned KVM VMIDs.

Open Items:
1. I couldn't figure out a way to determine whether a platform actually
   fully supports DVM/BTM or not. Not sure we can take a call based on
   SMMUv3 BTM feature bit alone. Probably we can get it from firmware
   via IORT?
2. The current splitting of VMID space is only one way to do this and
   probably not the best. Maybe we can follow the pinned ASID method used
   in SVA code. Suggestions welcome here.
3. The detach_pasid_table() interface is not very clear to me as the current
   Qemu prototype is not  using that. This requires fixing from my side.
 
This is based on Jean-Philippe's SVA series[1] and Eric's SMMUv3 dual-stage
support series[2].

The branch with the whole vSVA + BTM solution is here,
https://github.com/hisilicon/kernel-dev/tree/5.10-rc4-2stage-v13-vsva-btm-rfc

This is lightly tested on a HiSilicon D06 platform with uacce/zip dev test tool,
./zip_sva_per -k tlb

Thanks,
Shameer

1. https://github.com/Linaro/linux-kernel-uadk/commits/uacce-devel-5.10
2. https://lore.kernel.org/linux-iommu/[e-mail]/T/

Shameer Kolothum (5):
  vfio: Add a helper to retrieve kvm instance from a dev
  KVM: Add generic infrastructure to support pinned VMIDs
  KVM: ARM64: Add support for pinned VMIDs
  iommu/arm-smmu-v3: Use pinned VMID for NESTED stage with BTM
  KVM: arm64: Make sure pinned vmid is released on VM exit

 arch/arm64/include/asm/kvm_host.h           |   2 +
 arch/arm64/kvm/Kconfig                      |   1 +
 arch/arm64/kvm/arm.c                        | 116 +++++++++++++++++++-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  49 ++++++++-
 drivers/vfio/vfio.c                         |  12 ++
 include/linux/kvm_host.h                    |  17 +++
 include/linux/vfio.h                        |   1 +
 virt/kvm/Kconfig                            |   2 +
 virt/kvm/kvm_main.c                         |  25 +++++
 9 files changed, 220 insertions(+), 5 deletions(-)