mbox series

[00/13] Linux RISC-V IOMMU Support

Message ID cover.1689792825.git.tjeznach@rivosinc.com (mailing list archive)
Headers show
Series Linux RISC-V IOMMU Support | expand

Message

Tomasz Jeznach July 19, 2023, 7:33 p.m. UTC 
The RISC-V IOMMU specification is now ratified as-per the RISC-V international
process [1]. The latest frozen specifcation can be found at:
https://github.com/riscv-non-isa/riscv-iommu/releases/download/v1.0/riscv-iommu.pdf

At a high-level, the RISC-V IOMMU specification defines:
1) Memory-mapped programming interface
   - Mandatory and optional registers layout and description.
   - Software guidelines for device initialization and capabilities discovery.
2) In-memory queue interface
   - A command-queue used by software to queue commands to the IOMMU.
   - A fault/event queue used to bring faults and events to software’s attention.
   - A page-request queue used to report “Page Request” messages received from
     PCIe devices.
   - Message-signalled and wire-signaled interrupt mechanism.
3) In-memory data structures
   - Device-context: used to associate a device with an address space and to hold
     other per-device parameters used by the IOMMU to perform address translations.
   - Process-contexts: used to associate a different virtual address space based on
     device provided process identification number.
   - MSI page table configuration used to direct an MSI to a guest interrupt file
     in an IMSIC. The MSI page table formats are defined by the Advanced Interrupt
     Architecture specification [2].

This series introduces complete single-level translation support, including shared
virtual address (SVA), ATS/PRI interfaces in the kernel driver. Patches adding MSI
identity remapping and G-Stage translation (GPA to SPA) are added only to excercise
hardware interfaces, to be complemented with AIA/KVM bindings in follow-up series.

This series is a logical regrouping of series of incremental patches based on
RISC-V International IOMMU Task Group discussions and specification development
process. Original series can be found at the maintainer's repository branch [3].

These patches can also be found in the riscv_iommu_v1 branch at:
https://github.com/tjeznach/linux/tree/riscv_iommu_v1

To test this series, use QEMU/OpenSBI with RISC-V IOMMU implementation available in
the riscv_iommu_v1 branch at:
https://github.com/tjeznach/qemu/tree/riscv_iommu_v1

References:
[1] - https://wiki.riscv.org/display/HOME/Specification+Status
[2] - https://github.com/riscv/riscv-aia/releases/download/1.0/riscv-interrupts-1.0.pdf
[3] - https://github.com/tjeznach/qemu/tree/tjeznach/riscv-iommu-20230719


Anup Patel (1):
  dt-bindings: Add RISC-V IOMMU bindings

Tomasz Jeznach (10):
  RISC-V: drivers/iommu: Add RISC-V IOMMU - Ziommu support.
  RISC-V: arch/riscv/config: enable RISC-V IOMMU support
  MAINTAINERS: Add myself for RISC-V IOMMU driver
  RISC-V: drivers/iommu/riscv: Add sysfs interface
  RISC-V: drivers/iommu/riscv: Add command, fault, page-req queues
  RISC-V: drivers/iommu/riscv: Add device context support
  RISC-V: drivers/iommu/riscv: Add page table support
  RISC-V: drivers/iommu/riscv: Add SVA with PASID/ATS/PRI support.
  RISC-V: drivers/iommu/riscv: Add MSI identity remapping
  RISC-V: drivers/iommu/riscv: Add G-Stage translation support

 .../bindings/iommu/riscv,iommu.yaml           |  146 ++
 MAINTAINERS                                   |    7 +
 arch/riscv/configs/defconfig                  |    1 +
 drivers/iommu/Kconfig                         |    1 +
 drivers/iommu/Makefile                        |    2 +-
 drivers/iommu/io-pgtable.c                    |    3 +
 drivers/iommu/riscv/Kconfig                   |   22 +
 drivers/iommu/riscv/Makefile                  |    1 +
 drivers/iommu/riscv/io_pgtable.c              |  266 ++
 drivers/iommu/riscv/iommu-bits.h              |  704 ++++++
 drivers/iommu/riscv/iommu-pci.c               |  206 ++
 drivers/iommu/riscv/iommu-platform.c          |  160 ++
 drivers/iommu/riscv/iommu-sysfs.c             |  183 ++
 drivers/iommu/riscv/iommu.c                   | 2130 +++++++++++++++++
 drivers/iommu/riscv/iommu.h                   |  165 ++
 include/linux/io-pgtable.h                    |    2 +
 16 files changed, 3998 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/iommu/riscv,iommu.yaml
 create mode 100644 drivers/iommu/riscv/Kconfig
 create mode 100644 drivers/iommu/riscv/Makefile
 create mode 100644 drivers/iommu/riscv/io_pgtable.c
 create mode 100644 drivers/iommu/riscv/iommu-bits.h
 create mode 100644 drivers/iommu/riscv/iommu-pci.c
 create mode 100644 drivers/iommu/riscv/iommu-platform.c
 create mode 100644 drivers/iommu/riscv/iommu-sysfs.c
 create mode 100644 drivers/iommu/riscv/iommu.c
 create mode 100644 drivers/iommu/riscv/iommu.h

Comments

Zong Li Feb. 23, 2024, 2:04 p.m. UTC | #1 
>
> The RISC-V IOMMU specification is now ratified as-per the RISC-V international
> process [1]. The latest frozen specifcation can be found at:
> https://github.com/riscv-non-isa/riscv-iommu/releases/download/v1.0/riscv-iommu.pdf
>
> At a high-level, the RISC-V IOMMU specification defines:
> 1) Memory-mapped programming interface
>    - Mandatory and optional registers layout and description.
>    - Software guidelines for device initialization and capabilities discovery.
> 2) In-memory queue interface
>    - A command-queue used by software to queue commands to the IOMMU.
>    - A fault/event queue used to bring faults and events to software’s attention.
>    - A page-request queue used to report “Page Request” messages received from
>      PCIe devices.
>    - Message-signalled and wire-signaled interrupt mechanism.
> 3) In-memory data structures
>    - Device-context: used to associate a device with an address space and to hold
>      other per-device parameters used by the IOMMU to perform address translations.
>    - Process-contexts: used to associate a different virtual address space based on
>      device provided process identification number.
>    - MSI page table configuration used to direct an MSI to a guest interrupt file
>      in an IMSIC. The MSI page table formats are defined by the Advanced Interrupt
>      Architecture specification [2].
>
> This series introduces complete single-level translation support, including shared
> virtual address (SVA), ATS/PRI interfaces in the kernel driver. Patches adding MSI
> identity remapping and G-Stage translation (GPA to SPA) are added only to excercise
> hardware interfaces, to be complemented with AIA/KVM bindings in follow-up series.
>
> This series is a logical regrouping of series of incremental patches based on
> RISC-V International IOMMU Task Group discussions and specification development
> process. Original series can be found at the maintainer's repository branch [3].
>
> These patches can also be found in the riscv_iommu_v1 branch at:
> https://github.com/tjeznach/linux/tree/riscv_iommu_v1
>
> To test this series, use QEMU/OpenSBI with RISC-V IOMMU implementation available in
> the riscv_iommu_v1 branch at:
> https://github.com/tjeznach/qemu/tree/riscv_iommu_v1
>
> References:
> [1] - https://wiki.riscv.org/display/HOME/Specification+Status
> [2] - https://github.com/riscv/riscv-aia/releases/download/1.0/riscv-interrupts-1.0.pdf
> [3] - https://github.com/tjeznach/qemu/tree/tjeznach/riscv-iommu-20230719
>
>
> Anup Patel (1):
>   dt-bindings: Add RISC-V IOMMU bindings
>
> Tomasz Jeznach (10):
>   RISC-V: drivers/iommu: Add RISC-V IOMMU - Ziommu support.
>   RISC-V: arch/riscv/config: enable RISC-V IOMMU support
>   MAINTAINERS: Add myself for RISC-V IOMMU driver
>   RISC-V: drivers/iommu/riscv: Add sysfs interface
>   RISC-V: drivers/iommu/riscv: Add command, fault, page-req queues
>   RISC-V: drivers/iommu/riscv: Add device context support
>   RISC-V: drivers/iommu/riscv: Add page table support
>   RISC-V: drivers/iommu/riscv: Add SVA with PASID/ATS/PRI support.
>   RISC-V: drivers/iommu/riscv: Add MSI identity remapping
>   RISC-V: drivers/iommu/riscv: Add G-Stage translation support
>
>  .../bindings/iommu/riscv,iommu.yaml           |  146 ++
>  MAINTAINERS                                   |    7 +
>  arch/riscv/configs/defconfig                  |    1 +
>  drivers/iommu/Kconfig                         |    1 +
>  drivers/iommu/Makefile                        |    2 +-
>  drivers/iommu/io-pgtable.c                    |    3 +
>  drivers/iommu/riscv/Kconfig                   |   22 +
>  drivers/iommu/riscv/Makefile                  |    1 +
>  drivers/iommu/riscv/io_pgtable.c              |  266 ++
>  drivers/iommu/riscv/iommu-bits.h              |  704 ++++++
>  drivers/iommu/riscv/iommu-pci.c               |  206 ++
>  drivers/iommu/riscv/iommu-platform.c          |  160 ++
>  drivers/iommu/riscv/iommu-sysfs.c             |  183 ++
>  drivers/iommu/riscv/iommu.c                   | 2130 +++++++++++++++++
>  drivers/iommu/riscv/iommu.h                   |  165 ++
>  include/linux/io-pgtable.h                    |    2 +
>  16 files changed, 3998 insertions(+), 1 deletion(-)
>  create mode 100644 Documentation/devicetree/bindings/iommu/riscv,iommu.yaml
>  create mode 100644 drivers/iommu/riscv/Kconfig
>  create mode 100644 drivers/iommu/riscv/Makefile
>  create mode 100644 drivers/iommu/riscv/io_pgtable.c
>  create mode 100644 drivers/iommu/riscv/iommu-bits.h
>  create mode 100644 drivers/iommu/riscv/iommu-pci.c
>  create mode 100644 drivers/iommu/riscv/iommu-platform.c
>  create mode 100644 drivers/iommu/riscv/iommu-sysfs.c
>  create mode 100644 drivers/iommu/riscv/iommu.c
>  create mode 100644 drivers/iommu/riscv/iommu.h
>
> --
> 2.34.1
>
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv

Hi Tomasz,
Could I know if you have a plan for the next version and if you have
any estimates for when the v2 patch will be ready? We have some
patches based on top of your old implementation, and it would be great
if we can rebase them onto your next version. Thanks.
Tomasz Jeznach April 4, 2024, 5:37 p.m. UTC | #2 
On Fri, Feb 23, 2024 at 6:04 AM Zong Li <zong.li@sifive.com> wrote:
>
> >
> > The RISC-V IOMMU specification is now ratified as-per the RISC-V international
> > process [1]. The latest frozen specifcation can be found at:
> > https://github.com/riscv-non-isa/riscv-iommu/releases/download/v1.0/riscv-iommu.pdf
> >
> > At a high-level, the RISC-V IOMMU specification defines:
> > 1) Memory-mapped programming interface
> >    - Mandatory and optional registers layout and description.
> >    - Software guidelines for device initialization and capabilities discovery.
> > 2) In-memory queue interface
> >    - A command-queue used by software to queue commands to the IOMMU.
> >    - A fault/event queue used to bring faults and events to software’s attention.
> >    - A page-request queue used to report “Page Request” messages received from
> >      PCIe devices.
> >    - Message-signalled and wire-signaled interrupt mechanism.
> > 3) In-memory data structures
> >    - Device-context: used to associate a device with an address space and to hold
> >      other per-device parameters used by the IOMMU to perform address translations.
> >    - Process-contexts: used to associate a different virtual address space based on
> >      device provided process identification number.
> >    - MSI page table configuration used to direct an MSI to a guest interrupt file
> >      in an IMSIC. The MSI page table formats are defined by the Advanced Interrupt
> >      Architecture specification [2].
> >
> > This series introduces complete single-level translation support, including shared
> > virtual address (SVA), ATS/PRI interfaces in the kernel driver. Patches adding MSI
> > identity remapping and G-Stage translation (GPA to SPA) are added only to excercise
> > hardware interfaces, to be complemented with AIA/KVM bindings in follow-up series.
> >
> > This series is a logical regrouping of series of incremental patches based on
> > RISC-V International IOMMU Task Group discussions and specification development
> > process. Original series can be found at the maintainer's repository branch [3].
> >
> > These patches can also be found in the riscv_iommu_v1 branch at:
> > https://github.com/tjeznach/linux/tree/riscv_iommu_v1
> >
> > To test this series, use QEMU/OpenSBI with RISC-V IOMMU implementation available in
> > the riscv_iommu_v1 branch at:
> > https://github.com/tjeznach/qemu/tree/riscv_iommu_v1
> >
> > References:
> > [1] - https://wiki.riscv.org/display/HOME/Specification+Status
> > [2] - https://github.com/riscv/riscv-aia/releases/download/1.0/riscv-interrupts-1.0.pdf
> > [3] - https://github.com/tjeznach/qemu/tree/tjeznach/riscv-iommu-20230719
> >
> >
> > Anup Patel (1):
> >   dt-bindings: Add RISC-V IOMMU bindings
> >
> > Tomasz Jeznach (10):
> >   RISC-V: drivers/iommu: Add RISC-V IOMMU - Ziommu support.
> >   RISC-V: arch/riscv/config: enable RISC-V IOMMU support
> >   MAINTAINERS: Add myself for RISC-V IOMMU driver
> >   RISC-V: drivers/iommu/riscv: Add sysfs interface
> >   RISC-V: drivers/iommu/riscv: Add command, fault, page-req queues
> >   RISC-V: drivers/iommu/riscv: Add device context support
> >   RISC-V: drivers/iommu/riscv: Add page table support
> >   RISC-V: drivers/iommu/riscv: Add SVA with PASID/ATS/PRI support.
> >   RISC-V: drivers/iommu/riscv: Add MSI identity remapping
> >   RISC-V: drivers/iommu/riscv: Add G-Stage translation support
> >
> >  .../bindings/iommu/riscv,iommu.yaml           |  146 ++
> >  MAINTAINERS                                   |    7 +
> >  arch/riscv/configs/defconfig                  |    1 +
> >  drivers/iommu/Kconfig                         |    1 +
> >  drivers/iommu/Makefile                        |    2 +-
> >  drivers/iommu/io-pgtable.c                    |    3 +
> >  drivers/iommu/riscv/Kconfig                   |   22 +
> >  drivers/iommu/riscv/Makefile                  |    1 +
> >  drivers/iommu/riscv/io_pgtable.c              |  266 ++
> >  drivers/iommu/riscv/iommu-bits.h              |  704 ++++++
> >  drivers/iommu/riscv/iommu-pci.c               |  206 ++
> >  drivers/iommu/riscv/iommu-platform.c          |  160 ++
> >  drivers/iommu/riscv/iommu-sysfs.c             |  183 ++
> >  drivers/iommu/riscv/iommu.c                   | 2130 +++++++++++++++++
> >  drivers/iommu/riscv/iommu.h                   |  165 ++
> >  include/linux/io-pgtable.h                    |    2 +
> >  16 files changed, 3998 insertions(+), 1 deletion(-)
> >  create mode 100644 Documentation/devicetree/bindings/iommu/riscv,iommu.yaml
> >  create mode 100644 drivers/iommu/riscv/Kconfig
> >  create mode 100644 drivers/iommu/riscv/Makefile
> >  create mode 100644 drivers/iommu/riscv/io_pgtable.c
> >  create mode 100644 drivers/iommu/riscv/iommu-bits.h
> >  create mode 100644 drivers/iommu/riscv/iommu-pci.c
> >  create mode 100644 drivers/iommu/riscv/iommu-platform.c
> >  create mode 100644 drivers/iommu/riscv/iommu-sysfs.c
> >  create mode 100644 drivers/iommu/riscv/iommu.c
> >  create mode 100644 drivers/iommu/riscv/iommu.h
> >
> > --
> > 2.34.1
> >
> >
> > _______________________________________________
> > linux-riscv mailing list
> > linux-riscv@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-riscv
>
> Hi Tomasz,
> Could I know if you have a plan for the next version and if you have
> any estimates for when the v2 patch will be ready? We have some
> patches based on top of your old implementation, and it would be great
> if we can rebase them onto your next version. Thanks.

Hi Zong,

Thank you for your interest. Next version of the iommu/riscv is almost ready to
be sent in next few days.
There is a number of bug fixes and design changes based on the testing and
great feedback after v1 was published.
Upcoming patch set will be smaller, with core functionality only, hopefully to
make the review easier. Functionality related to the MSI remapping, shared
virtual addressing, nested translations will be moved to separate patch sets.

Complete, up to date revision is always available at
https://github.com/tjeznach/linux/

regards,
- Tomasz
Zong Li April 10, 2024, 5:38 a.m. UTC | #3 
On Fri, Apr 5, 2024 at 1:37 AM Tomasz Jeznach <tjeznach@rivosinc.com> wrote:
>
> On Fri, Feb 23, 2024 at 6:04 AM Zong Li <zong.li@sifive.com> wrote:
> >
> > >
> > > The RISC-V IOMMU specification is now ratified as-per the RISC-V international
> > > process [1]. The latest frozen specifcation can be found at:
> > > https://github.com/riscv-non-isa/riscv-iommu/releases/download/v1.0/riscv-iommu.pdf
> > >
> > > At a high-level, the RISC-V IOMMU specification defines:
> > > 1) Memory-mapped programming interface
> > >    - Mandatory and optional registers layout and description.
> > >    - Software guidelines for device initialization and capabilities discovery.
> > > 2) In-memory queue interface
> > >    - A command-queue used by software to queue commands to the IOMMU.
> > >    - A fault/event queue used to bring faults and events to software’s attention.
> > >    - A page-request queue used to report “Page Request” messages received from
> > >      PCIe devices.
> > >    - Message-signalled and wire-signaled interrupt mechanism.
> > > 3) In-memory data structures
> > >    - Device-context: used to associate a device with an address space and to hold
> > >      other per-device parameters used by the IOMMU to perform address translations.
> > >    - Process-contexts: used to associate a different virtual address space based on
> > >      device provided process identification number.
> > >    - MSI page table configuration used to direct an MSI to a guest interrupt file
> > >      in an IMSIC. The MSI page table formats are defined by the Advanced Interrupt
> > >      Architecture specification [2].
> > >
> > > This series introduces complete single-level translation support, including shared
> > > virtual address (SVA), ATS/PRI interfaces in the kernel driver. Patches adding MSI
> > > identity remapping and G-Stage translation (GPA to SPA) are added only to excercise
> > > hardware interfaces, to be complemented with AIA/KVM bindings in follow-up series.
> > >
> > > This series is a logical regrouping of series of incremental patches based on
> > > RISC-V International IOMMU Task Group discussions and specification development
> > > process. Original series can be found at the maintainer's repository branch [3].
> > >
> > > These patches can also be found in the riscv_iommu_v1 branch at:
> > > https://github.com/tjeznach/linux/tree/riscv_iommu_v1
> > >
> > > To test this series, use QEMU/OpenSBI with RISC-V IOMMU implementation available in
> > > the riscv_iommu_v1 branch at:
> > > https://github.com/tjeznach/qemu/tree/riscv_iommu_v1
> > >
> > > References:
> > > [1] - https://wiki.riscv.org/display/HOME/Specification+Status
> > > [2] - https://github.com/riscv/riscv-aia/releases/download/1.0/riscv-interrupts-1.0.pdf
> > > [3] - https://github.com/tjeznach/qemu/tree/tjeznach/riscv-iommu-20230719
> > >
> > >
> > > Anup Patel (1):
> > >   dt-bindings: Add RISC-V IOMMU bindings
> > >
> > > Tomasz Jeznach (10):
> > >   RISC-V: drivers/iommu: Add RISC-V IOMMU - Ziommu support.
> > >   RISC-V: arch/riscv/config: enable RISC-V IOMMU support
> > >   MAINTAINERS: Add myself for RISC-V IOMMU driver
> > >   RISC-V: drivers/iommu/riscv: Add sysfs interface
> > >   RISC-V: drivers/iommu/riscv: Add command, fault, page-req queues
> > >   RISC-V: drivers/iommu/riscv: Add device context support
> > >   RISC-V: drivers/iommu/riscv: Add page table support
> > >   RISC-V: drivers/iommu/riscv: Add SVA with PASID/ATS/PRI support.
> > >   RISC-V: drivers/iommu/riscv: Add MSI identity remapping
> > >   RISC-V: drivers/iommu/riscv: Add G-Stage translation support
> > >
> > >  .../bindings/iommu/riscv,iommu.yaml           |  146 ++
> > >  MAINTAINERS                                   |    7 +
> > >  arch/riscv/configs/defconfig                  |    1 +
> > >  drivers/iommu/Kconfig                         |    1 +
> > >  drivers/iommu/Makefile                        |    2 +-
> > >  drivers/iommu/io-pgtable.c                    |    3 +
> > >  drivers/iommu/riscv/Kconfig                   |   22 +
> > >  drivers/iommu/riscv/Makefile                  |    1 +
> > >  drivers/iommu/riscv/io_pgtable.c              |  266 ++
> > >  drivers/iommu/riscv/iommu-bits.h              |  704 ++++++
> > >  drivers/iommu/riscv/iommu-pci.c               |  206 ++
> > >  drivers/iommu/riscv/iommu-platform.c          |  160 ++
> > >  drivers/iommu/riscv/iommu-sysfs.c             |  183 ++
> > >  drivers/iommu/riscv/iommu.c                   | 2130 +++++++++++++++++
> > >  drivers/iommu/riscv/iommu.h                   |  165 ++
> > >  include/linux/io-pgtable.h                    |    2 +
> > >  16 files changed, 3998 insertions(+), 1 deletion(-)
> > >  create mode 100644 Documentation/devicetree/bindings/iommu/riscv,iommu.yaml
> > >  create mode 100644 drivers/iommu/riscv/Kconfig
> > >  create mode 100644 drivers/iommu/riscv/Makefile
> > >  create mode 100644 drivers/iommu/riscv/io_pgtable.c
> > >  create mode 100644 drivers/iommu/riscv/iommu-bits.h
> > >  create mode 100644 drivers/iommu/riscv/iommu-pci.c
> > >  create mode 100644 drivers/iommu/riscv/iommu-platform.c
> > >  create mode 100644 drivers/iommu/riscv/iommu-sysfs.c
> > >  create mode 100644 drivers/iommu/riscv/iommu.c
> > >  create mode 100644 drivers/iommu/riscv/iommu.h
> > >
> > > --
> > > 2.34.1
> > >
> > >
> > > _______________________________________________
> > > linux-riscv mailing list
> > > linux-riscv@lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/linux-riscv
> >
> > Hi Tomasz,
> > Could I know if you have a plan for the next version and if you have
> > any estimates for when the v2 patch will be ready? We have some
> > patches based on top of your old implementation, and it would be great
> > if we can rebase them onto your next version. Thanks.
>
> Hi Zong,
>
> Thank you for your interest. Next version of the iommu/riscv is almost ready to
> be sent in next few days.

Hi Tomasz,
Thanks you for the update, I would help to review the v2 series as well.

> There is a number of bug fixes and design changes based on the testing and
> great feedback after v1 was published.
> Upcoming patch set will be smaller, with core functionality only, hopefully to
> make the review easier. Functionality related to the MSI remapping, shared
> virtual addressing, nested translations will be moved to separate patch sets.
>
> Complete, up to date revision is always available at
> https://github.com/tjeznach/linux/
>
> regards,
> - Tomasz