[v4,00/23] iommu: Further abstract iommu-pages

Message

Jason Gunthorpe April 8, 2025, 4:53 p.m. UTC

This is part of the consolidated iommu page table work, it brings the
allocator I previously sketched to all drivers.

iommu-pages is a small abstraction for allocating page table pages that
iommu drivers use. It has a few properties that distinguish it from the
other allocators in the kernel:

 - Allocations are always power of two, and always physically aligned to
   their size
 - Allocations can be threaded on a list, in atomic contexts without
   memory allocations. The list is only used for freeing batch of pages
   (ie after IOTLB flush as the mm does)
 - Allocations are accounted for in the secondary page table counters
 - Allocations can sometimes be less than a full CPU page, 1/4 and 1/16 are
   some common sub page allocation sizes.
 - Allocations can sometimes be multiple CPU pages
 - In future I'd like atomic-safe RCU free of the page lists, as the mm does

Make the API tighter and leak fewer internal details to the
callers. Particularly this series aims to remove all struct page usage
related to iommu-pages memory from all drivers, this is things such as:

 struct page
 virt_to_page()
 page_to_virt()
 page_to_pfn()
 pfn_to_page()
 dma_map_page()
 page_address()
 page->lru

Once drivers no longer use struct page convert iommu-pages to folios and
use a private memory descriptor. This should help prepare iommu for
Matthew's memdesc project and clears the way to using more space in the
struct page for iommu-pages features in future.

Improve the API to work directly on sizes instead of order, the drivers
generally have HW specs and code paths that already have specific sizes.
Pass those sizes down into the allocator to remove some boiler plate
get_order() in drivers. This is cleanup to be ready for a possible sub
page allocator some day.

This is on github: https://github.com/jgunthorpe/linux/commits/iommu_pages

v4:
 - Rebase to v6.15-rc1
 - Don't let ARM page table size fall below 64 bytes due to HW alignment
v3: https://patch.msgid.link/r/[e-mail]
 - Fix comments
 - Rename __iommu_free_page() to __iommu_free_desc()
 - Retain the max IMR table size comment in vt-d
v2: https://patch.msgid.link/r/[e-mail]
 - Use struct tegra_pd instead of u32 *
 - Use dma_unmap_single() instead of unmap_page in tegra
 - Fix Tegra SIZE_PT typo
 - Use iommu_free_pages() as the free function name instead of
   iommu_free_page()
 - Remove numa_node_id() and use the NUMA_NO_NODE path with numa_mem_id()
 - Make all the allocation APIs use size only, fully remove order and lg2
   versions
 - Round up the riscv queue size, use SZ_4K in riscv not PAGE_SIZE
 - Convert AMD to use kvalloc for CPU memory and fix the device table to
   round up to 4K
 - Use PAGE_ALIGN instead of get_order in AMD iommu_alloc_4k_pages()
v1: https://patch.msgid.link/r/[e-mail]

Jason Gunthorpe (23):
  iommu/terga: Do not use struct page as the handle for as->pd memory
  iommu/tegra: Do not use struct page as the handle for pts
  iommu/pages: Remove __iommu_alloc_pages()/__iommu_free_pages()
  iommu/pages: Make iommu_put_pages_list() work with high order
    allocations
  iommu/pages: Remove the order argument to iommu_free_pages()
  iommu/pages: Remove iommu_free_page()
  iommu/pages: De-inline the substantial functions
  iommu/pages: Formalize the freelist API
  iommu/riscv: Convert to use struct iommu_pages_list
  iommu/amd: Convert to use struct iommu_pages_list
  iommu: Change iommu_iotlb_gather to use iommu_page_list
  iommu/pages: Remove iommu_put_pages_list_old and the _Generic
  iommu/pages: Move from struct page to struct ioptdesc and folio
  iommu/pages: Move the __GFP_HIGHMEM checks into the common code
  iommu/pages: Allow sub page sizes to be passed into the allocator
  iommu/amd: Change rlookup, irq_lookup, and alias to use kvalloc()
  iommu/amd: Use roundup_pow_two() instead of get_order()
  iommu/riscv: Update to use iommu_alloc_pages_node_lg2()
  iommu: Update various drivers to pass in lg2sz instead of order to
    iommu pages
  iommu/pages: Remove iommu_alloc_page/pages()
  iommu/pages: Remove iommu_alloc_page_node()
  iommu/amd: Use iommu_alloc_pages_node_sz() for the IRT
  iommu/vtd: Remove iommu_alloc_pages_node()

 drivers/iommu/Makefile              |   1 +
 drivers/iommu/amd/amd_iommu_types.h |   9 +-
 drivers/iommu/amd/init.c            |  91 +++++-------
 drivers/iommu/amd/io_pgtable.c      |  38 +++--
 drivers/iommu/amd/io_pgtable_v2.c   |  12 +-
 drivers/iommu/amd/iommu.c           |  16 +--
 drivers/iommu/amd/ppr.c             |   2 +-
 drivers/iommu/dma-iommu.c           |   9 +-
 drivers/iommu/exynos-iommu.c        |  12 +-
 drivers/iommu/intel/dmar.c          |  10 +-
 drivers/iommu/intel/iommu.c         |  49 ++++---
 drivers/iommu/intel/iommu.h         |   7 +-
 drivers/iommu/intel/irq_remapping.c |  12 +-
 drivers/iommu/intel/pasid.c         |  13 +-
 drivers/iommu/intel/pasid.h         |   1 -
 drivers/iommu/intel/prq.c           |   7 +-
 drivers/iommu/io-pgtable-arm.c      |  18 ++-
 drivers/iommu/io-pgtable-dart.c     |  23 +--
 drivers/iommu/iommu-pages.c         | 117 +++++++++++++++
 drivers/iommu/iommu-pages.h         | 211 +++++++++-------------------
 drivers/iommu/riscv/iommu.c         |  43 +++---
 drivers/iommu/rockchip-iommu.c      |  14 +-
 drivers/iommu/sun50i-iommu.c        |   6 +-
 drivers/iommu/tegra-smmu.c          | 111 ++++++++-------
 include/linux/iommu.h               |  16 ++-
 25 files changed, 438 insertions(+), 410 deletions(-)
 create mode 100644 drivers/iommu/iommu-pages.c


base-commit: 0af2f6be1b4281385b618cb86ad946eded089ac8

Comments

Joerg Roedel April 17, 2025, 2:25 p.m. UTC | #1

> Jason Gunthorpe (23):
>   iommu/terga: Do not use struct page as the handle for as->pd memory
>   iommu/tegra: Do not use struct page as the handle for pts
>   iommu/pages: Remove __iommu_alloc_pages()/__iommu_free_pages()
>   iommu/pages: Make iommu_put_pages_list() work with high order
>     allocations
>   iommu/pages: Remove the order argument to iommu_free_pages()
>   iommu/pages: Remove iommu_free_page()
>   iommu/pages: De-inline the substantial functions
>   iommu/pages: Formalize the freelist API
>   iommu/riscv: Convert to use struct iommu_pages_list
>   iommu/amd: Convert to use struct iommu_pages_list
>   iommu: Change iommu_iotlb_gather to use iommu_page_list
>   iommu/pages: Remove iommu_put_pages_list_old and the _Generic
>   iommu/pages: Move from struct page to struct ioptdesc and folio
>   iommu/pages: Move the __GFP_HIGHMEM checks into the common code
>   iommu/pages: Allow sub page sizes to be passed into the allocator
>   iommu/amd: Change rlookup, irq_lookup, and alias to use kvalloc()
>   iommu/amd: Use roundup_pow_two() instead of get_order()
>   iommu/riscv: Update to use iommu_alloc_pages_node_lg2()
>   iommu: Update various drivers to pass in lg2sz instead of order to
>     iommu pages
>   iommu/pages: Remove iommu_alloc_page/pages()
>   iommu/pages: Remove iommu_alloc_page_node()
>   iommu/amd: Use iommu_alloc_pages_node_sz() for the IRT
>   iommu/vtd: Remove iommu_alloc_pages_node()

Applied to the new iommu-pages branch. I think this has the potential to
become a conflict nightmare, let's see how it goes in this cycle.

 	Joerg

Jason Gunthorpe April 17, 2025, 2:27 p.m. UTC | #2

On Thu, Apr 17, 2025 at 04:25:29PM +0200, Joerg Roedel wrote:
> > Jason Gunthorpe (23):
> >   iommu/terga: Do not use struct page as the handle for as->pd memory
> >   iommu/tegra: Do not use struct page as the handle for pts
> >   iommu/pages: Remove __iommu_alloc_pages()/__iommu_free_pages()
> >   iommu/pages: Make iommu_put_pages_list() work with high order
> >     allocations
> >   iommu/pages: Remove the order argument to iommu_free_pages()
> >   iommu/pages: Remove iommu_free_page()
> >   iommu/pages: De-inline the substantial functions
> >   iommu/pages: Formalize the freelist API
> >   iommu/riscv: Convert to use struct iommu_pages_list
> >   iommu/amd: Convert to use struct iommu_pages_list
> >   iommu: Change iommu_iotlb_gather to use iommu_page_list
> >   iommu/pages: Remove iommu_put_pages_list_old and the _Generic
> >   iommu/pages: Move from struct page to struct ioptdesc and folio
> >   iommu/pages: Move the __GFP_HIGHMEM checks into the common code
> >   iommu/pages: Allow sub page sizes to be passed into the allocator
> >   iommu/amd: Change rlookup, irq_lookup, and alias to use kvalloc()
> >   iommu/amd: Use roundup_pow_two() instead of get_order()
> >   iommu/riscv: Update to use iommu_alloc_pages_node_lg2()
> >   iommu: Update various drivers to pass in lg2sz instead of order to
> >     iommu pages
> >   iommu/pages: Remove iommu_alloc_page/pages()
> >   iommu/pages: Remove iommu_alloc_page_node()
> >   iommu/amd: Use iommu_alloc_pages_node_sz() for the IRT
> >   iommu/vtd: Remove iommu_alloc_pages_node()
> 
> Applied to the new iommu-pages branch. I think this has the potential to
> become a conflict nightmare, let's see how it goes in this cycle.

I haven't had much trouble rebasing it, so fingers crossed, thanks

Jason

Joerg Roedel April 17, 2025, 2:40 p.m. UTC | #3

On Thu, Apr 17, 2025 at 11:27:52AM -0300, Jason Gunthorpe wrote:
> I haven't had much trouble rebasing it, so fingers crossed, thanks

Forget the iommu-pages branch, this just makes it worse. I put it
directly into the core branch now for everyone to rebase on in case of
conflicts. This should avoid some of the potential trouble :)

Regards,

	Joerg