[RFC,v2,0/2] hw/arm/virt-acpi-build: Add IORT RMR regions to handle MSI nested binding

Message

Eric Auger Oct. 5, 2021, 8:53 a.m. UTC

To handle SMMUv3 nested stage support it is practical to
expose the guest with reserved memory regions (RMRs)
covering the IOVAs used by the host kernel to map
physical MSI doorbells.

Those IOVAs belong to [0x8000000, 0x8100000] matching
MSI_IOVA_BASE and MSI_IOVA_LENGTH definitions in kernel
arm-smmu-v3 driver. This is the window used to allocate
IOVAs matching physical MSI doorbells.

With those RMRs, the guest is forced to use a flat mapping
for this range. Hence the assigned device is programmed
with one IOVA from this range. Stage 1, owned by the guest
has a flat mapping for this IOVA. Stage2, owned by the VMM
then enforces a mapping from this IOVA to the physical
MSI doorbell.

At IORT table level, due to the single mapping flag being
set on the ID mapping, 256 IORT RMR nodes need to be
created per bus. This looks awkward from a specification
and implementation point of view.

This may also produce a warning at execution time:
qemu-system-aarch64: warning: ACPI table size 114709 exceeds
65536 bytes, migration may not work
(here with 5 pcie root ports, ie. 256 * 6 = 1536 RMR nodes!).

The creation of those RMR nodes only is relevant if nested
stage SMMU is in use, along with VFIO. As VFIO devices can be
hotplugged, all RMRs need to be created in advance. Hence
the patch introduces a new arm virt "nested-smmuv3" iommu type.

ARM DEN 0049E.b IORT specification also mandates that when
RMRs are present, the OS must preserve PCIe configuration
performed by the boot FW. So along with the RMR IORT nodes,
a _DSM function #5, as defined by PCI FIRMWARE SPECIFICATION
EVISION 3.3, chapter 4.6.5 is added to PCIe host bridge
and PCIe expander bridge objects.

The series applies on top of Igor's
[1] [PATCH v4 00/35] acpi: refactor error prone build_header() and
packed structures usage in ACPI tables and
[2] [PATCH v2 0/3] hw/arm/virt_acpi_build: Upgrate the IORT table up
  to revision E.b

The guest can use RMRs with Shameer's series:
[3] [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node

The latest IORT specification (ARM DEN 0049E.b) can be found at
IO Remapping Table - Platform Design Document
https://developer.arm.com/documentation/den0049/latest/

This series and its dependency can be found at
https://github.com/eauger/qemu.git
branch: igor_acpi_refactoring_v4_dbg2_v3_rmr_v2

History:
v1 -> v2:
- add DSM #5

Eric Auger (2):
  hw/pci-host/gpex: Allow to generate preserve boot config DSM #5
  hw/arm/virt-acpi-build: Add IORT RMR regions to handle MSI nested
    binding

 include/hw/arm/virt.h      |  7 ++++
 include/hw/pci-host/gpex.h |  1 +
 hw/arm/virt-acpi-build.c   | 84 ++++++++++++++++++++++++++++++++------
 hw/arm/virt.c              |  7 +++-
 hw/pci-host/gpex-acpi.c    | 12 ++++++
 5 files changed, 98 insertions(+), 13 deletions(-)