| Message ID | 20240529053250.91284-1-baolu.lu@linux.intel.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | iommu: Refactoring domain allocation interface | expand |
On 2024/5/29 13:32, Lu Baolu wrote: > The IOMMU subsystem has undergone some changes, including the removal > of iommu_ops from the bus structure. Consequently, the existing domain > allocation interface, which relies on a bus type argument, is no longer > relevant: > > struct iommu_domain *iommu_domain_alloc(struct bus_type *bus) > > This series is designed to refactor the use of this interface. It > proposes two new interfaces to replace iommu_domain_alloc(): > > - iommu_user_domain_alloc(): This interface is intended for allocating > iommu domains managed by userspace for device passthrough scenarios, > such as those used by iommufd, vfio, and vdpa. It clearly indicates > that the domain is for user-managed device DMA. user paging domain? It looks to me user domain includes the nested domains as well. > If an IOMMU driver does not implement iommu_ops->domain_alloc_user, > this interface will rollback to the generic paging domain allocation. > > - iommu_paging_domain_alloc(): This interface is for allocating iommu > domains managed by kernel drivers for kernel DMA purposes. It takes a > device pointer as a parameter, which better reflects the current > design of the IOMMU subsystem. > > The majority of device drivers currently using iommu_domain_alloc() do > so to allocate a domain for a specific device and then attach that > domain to the device. These cases can be straightforwardly migrated to > the new interfaces. > > However, there are some drivers with more complex use cases that do > not fit neatly into this new scheme. For example: > > $ git grep "= iommu_domain_alloc" > arch/arm/mm/dma-mapping.c: mapping->domain = iommu_domain_alloc(bus); > drivers/gpu/drm/rockchip/rockchip_drm_drv.c: private->domain = iommu_domain_alloc(private->iommu_dev->bus); > drivers/gpu/drm/tegra/drm.c: tegra->domain = iommu_domain_alloc(&platform_bus_type); > drivers/infiniband/hw/usnic/usnic_uiom.c: pd->domain = domain = iommu_domain_alloc(dev->bus); > > This series leave those cases unchanged and keep iommu_domain_alloc() > for their usage. But new drivers should not use it anymore. does it mean there is still domains allocated via iommu_domain_alloc() on VT-d platform? > The whole series is also available on GitHub: > https://github.com/LuBaolu/intel-iommu/commits/iommu-domain-allocation-refactor-v1 > > Lu Baolu (20): > iommu: Add iommu_user_domain_alloc() interface > iommufd: Use iommu_user_domain_alloc() > vfio/type1: Use iommu_paging_domain_alloc() > vhost-vdpa: Use iommu_user_domain_alloc() > iommu: Add iommu_paging_domain_alloc() interface > drm/msm: Use iommu_paging_domain_alloc() > drm/nouveau/tegra: Use iommu_paging_domain_alloc() > gpu: host1x: Use iommu_paging_domain_alloc() > media: nvidia: tegra: Use iommu_paging_domain_alloc() > media: venus: firmware: Use iommu_paging_domain_alloc() > ath10k: Use iommu_paging_domain_alloc() > wifi: ath11k: Use iommu_paging_domain_alloc() > remoteproc: Use iommu_paging_domain_alloc() > soc/fsl/qbman: Use iommu_paging_domain_alloc() > iommu/vt-d: Add helper to allocate paging domain > iommu/vt-d: Add domain_alloc_paging support > iommu/vt-d: Simplify compatibility check for identity domain > iommu/vt-d: Enhance compatibility check for paging domain attach > iommu/vt-d: Remove domain_update_iommu_cap() > iommu/vt-d: Remove domain_update_iommu_superpage() > > include/linux/iommu.h | 12 + > drivers/gpu/drm/msm/msm_iommu.c | 8 +- > .../drm/nouveau/nvkm/engine/device/tegra.c | 4 +- > drivers/gpu/host1x/dev.c | 6 +- > drivers/iommu/intel/iommu.c | 319 ++++++++---------- > drivers/iommu/intel/pasid.c | 28 +- > drivers/iommu/iommu.c | 62 ++++ > drivers/iommu/iommufd/hw_pagetable.c | 20 +- > .../media/platform/nvidia/tegra-vde/iommu.c | 6 +- > drivers/media/platform/qcom/venus/firmware.c | 6 +- > drivers/net/wireless/ath/ath10k/snoc.c | 6 +- > drivers/net/wireless/ath/ath11k/ahb.c | 6 +- > drivers/remoteproc/remoteproc_core.c | 6 +- > drivers/soc/fsl/qbman/qman_portal.c | 4 +- > drivers/vfio/vfio_iommu_type1.c | 7 +- > drivers/vhost/vdpa.c | 11 +- > 16 files changed, 253 insertions(+), 258 deletions(-) >
On 2024/5/29 17:03, Yi Liu wrote: > On 2024/5/29 13:32, Lu Baolu wrote: >> The IOMMU subsystem has undergone some changes, including the removal >> of iommu_ops from the bus structure. Consequently, the existing domain >> allocation interface, which relies on a bus type argument, is no longer >> relevant: >> >> struct iommu_domain *iommu_domain_alloc(struct bus_type *bus) >> >> This series is designed to refactor the use of this interface. It >> proposes two new interfaces to replace iommu_domain_alloc(): >> >> - iommu_user_domain_alloc(): This interface is intended for allocating >> iommu domains managed by userspace for device passthrough scenarios, >> such as those used by iommufd, vfio, and vdpa. It clearly indicates >> that the domain is for user-managed device DMA. > > user paging domain? It looks to me user domain includes the nested domains > as well. Yes, nested domain is a user domain. The iommu driver should implement iommu_ops->domain_alloc_user for nested domain allocation. > >> If an IOMMU driver does not implement iommu_ops->domain_alloc_user, >> this interface will rollback to the generic paging domain allocation. >> >> - iommu_paging_domain_alloc(): This interface is for allocating iommu >> domains managed by kernel drivers for kernel DMA purposes. It takes a >> device pointer as a parameter, which better reflects the current >> design of the IOMMU subsystem. >> >> The majority of device drivers currently using iommu_domain_alloc() do >> so to allocate a domain for a specific device and then attach that >> domain to the device. These cases can be straightforwardly migrated to >> the new interfaces. >> >> However, there are some drivers with more complex use cases that do >> not fit neatly into this new scheme. For example: >> >> $ git grep "= iommu_domain_alloc" >> arch/arm/mm/dma-mapping.c: mapping->domain = >> iommu_domain_alloc(bus); >> drivers/gpu/drm/rockchip/rockchip_drm_drv.c: private->domain = >> iommu_domain_alloc(private->iommu_dev->bus); >> drivers/gpu/drm/tegra/drm.c: tegra->domain = >> iommu_domain_alloc(&platform_bus_type); >> drivers/infiniband/hw/usnic/usnic_uiom.c: pd->domain = domain = >> iommu_domain_alloc(dev->bus); >> >> This series leave those cases unchanged and keep iommu_domain_alloc() >> for their usage. But new drivers should not use it anymore. > > does it mean there is still domains allocated via iommu_domain_alloc() > on VT-d platform? I think the drivers mentioned above do not run on x86 platforms, or do they? Best regards, baolu
On 29/05/2024 6:32 am, Lu Baolu wrote: > The IOMMU subsystem has undergone some changes, including the removal > of iommu_ops from the bus structure. Consequently, the existing domain > allocation interface, which relies on a bus type argument, is no longer > relevant: > > struct iommu_domain *iommu_domain_alloc(struct bus_type *bus) > > This series is designed to refactor the use of this interface. It > proposes two new interfaces to replace iommu_domain_alloc(): > > - iommu_user_domain_alloc(): This interface is intended for allocating > iommu domains managed by userspace for device passthrough scenarios, > such as those used by iommufd, vfio, and vdpa. It clearly indicates > that the domain is for user-managed device DMA. > > If an IOMMU driver does not implement iommu_ops->domain_alloc_user, > this interface will rollback to the generic paging domain allocation. > > - iommu_paging_domain_alloc(): This interface is for allocating iommu > domains managed by kernel drivers for kernel DMA purposes. It takes a > device pointer as a parameter, which better reflects the current > design of the IOMMU subsystem. > > The majority of device drivers currently using iommu_domain_alloc() do > so to allocate a domain for a specific device and then attach that > domain to the device. These cases can be straightforwardly migrated to > the new interfaces. Ooh, nice! This was rising back up my to-do list as well, but I concur it's rather more straightforward than my version that did devious things to keep the iommu_domain_alloc() name... > However, there are some drivers with more complex use cases that do > not fit neatly into this new scheme. For example: > > $ git grep "= iommu_domain_alloc" > arch/arm/mm/dma-mapping.c: mapping->domain = iommu_domain_alloc(bus); This one's simple enough, the refactor just needs to go one step deeper. I've just rebased and pushed my old patch for that, if you'd like it [1]. > drivers/gpu/drm/rockchip/rockchip_drm_drv.c: private->domain = iommu_domain_alloc(private->iommu_dev->bus); Both this one and usnic_uiom_alloc_pd() should be OK - back when I did all the figuring out to clean up iommu_present(), I specifically reworked them into "dev->bus" style as a reminder that it *is* supposed to be the right device for doing this with, even if the attach is a bit more distant. > drivers/gpu/drm/tegra/drm.c: tegra->domain = iommu_domain_alloc(&platform_bus_type); This is the tricky one, where the device to hand may *not* be the right device for IOMMU API use [2]. FWIW my plan was to pull the "walk the platform bus to find any IOMMU-mapped device" trick into this code and use it both to remove the final iommu_present() and for a device-based domain allocation. > drivers/infiniband/hw/usnic/usnic_uiom.c: pd->domain = domain = iommu_domain_alloc(dev->bus); > > This series leave those cases unchanged and keep iommu_domain_alloc() > for their usage. But new drivers should not use it anymore. I'd certainly be keen for it to be gone ASAP, since I'm seeing increasing demand for supporting multiple IOMMU drivers, and this is the last bus-based thing standing in the way of that. Thanks, Robin. [1] https://gitlab.arm.com/linux-arm/linux-rm/-/commit/f048cc6a323d8641898025ca96071df7cbe8bd52 [2] https://lore.kernel.org/linux-iommu/add31812-50d5-6cb0-3908-143c523abd37@collabora.com/ > The whole series is also available on GitHub: > https://github.com/LuBaolu/intel-iommu/commits/iommu-domain-allocation-refactor-v1 > > Lu Baolu (20): > iommu: Add iommu_user_domain_alloc() interface > iommufd: Use iommu_user_domain_alloc() > vfio/type1: Use iommu_paging_domain_alloc() > vhost-vdpa: Use iommu_user_domain_alloc() > iommu: Add iommu_paging_domain_alloc() interface > drm/msm: Use iommu_paging_domain_alloc() > drm/nouveau/tegra: Use iommu_paging_domain_alloc() > gpu: host1x: Use iommu_paging_domain_alloc() > media: nvidia: tegra: Use iommu_paging_domain_alloc() > media: venus: firmware: Use iommu_paging_domain_alloc() > ath10k: Use iommu_paging_domain_alloc() > wifi: ath11k: Use iommu_paging_domain_alloc() > remoteproc: Use iommu_paging_domain_alloc() > soc/fsl/qbman: Use iommu_paging_domain_alloc() > iommu/vt-d: Add helper to allocate paging domain > iommu/vt-d: Add domain_alloc_paging support > iommu/vt-d: Simplify compatibility check for identity domain > iommu/vt-d: Enhance compatibility check for paging domain attach > iommu/vt-d: Remove domain_update_iommu_cap() > iommu/vt-d: Remove domain_update_iommu_superpage() > > include/linux/iommu.h | 12 + > drivers/gpu/drm/msm/msm_iommu.c | 8 +- > .../drm/nouveau/nvkm/engine/device/tegra.c | 4 +- > drivers/gpu/host1x/dev.c | 6 +- > drivers/iommu/intel/iommu.c | 319 ++++++++---------- > drivers/iommu/intel/pasid.c | 28 +- > drivers/iommu/iommu.c | 62 ++++ > drivers/iommu/iommufd/hw_pagetable.c | 20 +- > .../media/platform/nvidia/tegra-vde/iommu.c | 6 +- > drivers/media/platform/qcom/venus/firmware.c | 6 +- > drivers/net/wireless/ath/ath10k/snoc.c | 6 +- > drivers/net/wireless/ath/ath11k/ahb.c | 6 +- > drivers/remoteproc/remoteproc_core.c | 6 +- > drivers/soc/fsl/qbman/qman_portal.c | 4 +- > drivers/vfio/vfio_iommu_type1.c | 7 +- > drivers/vhost/vdpa.c | 11 +- > 16 files changed, 253 insertions(+), 258 deletions(-) >
On 5/31/24 1:59 AM, Robin Murphy wrote: > On 29/05/2024 6:32 am, Lu Baolu wrote: >> The IOMMU subsystem has undergone some changes, including the removal >> of iommu_ops from the bus structure. Consequently, the existing domain >> allocation interface, which relies on a bus type argument, is no longer >> relevant: >> >> struct iommu_domain *iommu_domain_alloc(struct bus_type *bus) >> >> This series is designed to refactor the use of this interface. It >> proposes two new interfaces to replace iommu_domain_alloc(): >> >> - iommu_user_domain_alloc(): This interface is intended for allocating >> iommu domains managed by userspace for device passthrough scenarios, >> such as those used by iommufd, vfio, and vdpa. It clearly indicates >> that the domain is for user-managed device DMA. >> >> If an IOMMU driver does not implement iommu_ops->domain_alloc_user, >> this interface will rollback to the generic paging domain allocation. >> >> - iommu_paging_domain_alloc(): This interface is for allocating iommu >> domains managed by kernel drivers for kernel DMA purposes. It takes a >> device pointer as a parameter, which better reflects the current >> design of the IOMMU subsystem. >> >> The majority of device drivers currently using iommu_domain_alloc() do >> so to allocate a domain for a specific device and then attach that >> domain to the device. These cases can be straightforwardly migrated to >> the new interfaces. > > Ooh, nice! This was rising back up my to-do list as well, but I concur > it's rather more straightforward than my version that did devious things > to keep the iommu_domain_alloc() name... > >> However, there are some drivers with more complex use cases that do >> not fit neatly into this new scheme. For example: >> >> $ git grep "= iommu_domain_alloc" >> arch/arm/mm/dma-mapping.c: mapping->domain = >> iommu_domain_alloc(bus); > > This one's simple enough, the refactor just needs to go one step deeper. > I've just rebased and pushed my old patch for that, if you'd like it [1]. Great! With this change, we can safely replace iommu_domain_alloc(). diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c index 52f9c56cc3cb..88c2d68a69c9 100644 --- a/arch/arm/mm/dma-mapping.c +++ b/arch/arm/mm/dma-mapping.c @@ -1585,9 +1585,11 @@ arm_iommu_create_mapping(struct device *dev, dma_addr_t base, u64 size) spin_lock_init(&mapping->lock); - mapping->domain = iommu_domain_alloc(dev->bus); - if (!mapping->domain) + mapping->domain = iommu_paging_domain_alloc(dev); + if (IS_ERR(mapping->domain)) { + err = PTR_ERR(mapping->domain); goto err4; + } kref_init(&mapping->kref); return mapping; > >> drivers/gpu/drm/rockchip/rockchip_drm_drv.c: private->domain = >> iommu_domain_alloc(private->iommu_dev->bus); > > Both this one and usnic_uiom_alloc_pd() should be OK - back when I did > all the figuring out to clean up iommu_present(), I specifically > reworked them into "dev->bus" style as a reminder that it *is* supposed > to be the right device for doing this with, even if the attach is a bit > more distant. Yeah! I will cleanup these two in the next version. > >> drivers/gpu/drm/tegra/drm.c: tegra->domain = >> iommu_domain_alloc(&platform_bus_type); > > This is the tricky one, where the device to hand may *not* be the right > device for IOMMU API use [2]. FWIW my plan was to pull the "walk the > platform bus to find any IOMMU-mapped device" trick into this code and > use it both to remove the final iommu_present() and for a device-based > domain allocation. I am not familiar with this driver, so the solution you mentioned above is the best option I can think of for now. I will incorporate this into the next version. > >> drivers/infiniband/hw/usnic/usnic_uiom.c: pd->domain = domain = >> iommu_domain_alloc(dev->bus); >> >> This series leave those cases unchanged and keep iommu_domain_alloc() >> for their usage. But new drivers should not use it anymore. > > I'd certainly be keen for it to be gone ASAP, since I'm seeing > increasing demand for supporting multiple IOMMU drivers, and this is the > last bus-based thing standing in the way of that. Agreed. With all iommu_domain_alloc() removed, iommu_domain_alloc() could be dropped. > > Thanks, > Robin. > > [1] > https://gitlab.arm.com/linux-arm/linux-rm/-/commit/f048cc6a323d8641898025ca96071df7cbe8bd52 > [2] > https://lore.kernel.org/linux-iommu/add31812-50d5-6cb0-3908-143c523abd37@collabora.com/ Best regards, baolu
On 2024/5/29 20:02, Baolu Lu wrote: > On 2024/5/29 17:03, Yi Liu wrote: >> On 2024/5/29 13:32, Lu Baolu wrote: >>> The IOMMU subsystem has undergone some changes, including the removal >>> of iommu_ops from the bus structure. Consequently, the existing domain >>> allocation interface, which relies on a bus type argument, is no longer >>> relevant: >>> >>> struct iommu_domain *iommu_domain_alloc(struct bus_type *bus) >>> >>> This series is designed to refactor the use of this interface. It >>> proposes two new interfaces to replace iommu_domain_alloc(): >>> >>> - iommu_user_domain_alloc(): This interface is intended for allocating >>> iommu domains managed by userspace for device passthrough scenarios, >>> such as those used by iommufd, vfio, and vdpa. It clearly indicates >>> that the domain is for user-managed device DMA. >> >> user paging domain? It looks to me user domain includes the nested domains >> as well. > > Yes, nested domain is a user domain. The iommu driver should implement > iommu_ops->domain_alloc_user for nested domain allocation. will it be more clear to name iommu_user_domain_alloc() be iommu_user_paging_domain_alloc() as it is mainly for paging domain allocation? >> >>> If an IOMMU driver does not implement iommu_ops->domain_alloc_user, >>> this interface will rollback to the generic paging domain allocation. >>> >>> - iommu_paging_domain_alloc(): This interface is for allocating iommu >>> domains managed by kernel drivers for kernel DMA purposes. It takes a >>> device pointer as a parameter, which better reflects the current >>> design of the IOMMU subsystem. >>> >>> The majority of device drivers currently using iommu_domain_alloc() do >>> so to allocate a domain for a specific device and then attach that >>> domain to the device. These cases can be straightforwardly migrated to >>> the new interfaces. >>> >>> However, there are some drivers with more complex use cases that do >>> not fit neatly into this new scheme. For example: >>> >>> $ git grep "= iommu_domain_alloc" >>> arch/arm/mm/dma-mapping.c: mapping->domain = iommu_domain_alloc(bus); >>> drivers/gpu/drm/rockchip/rockchip_drm_drv.c: private->domain = >>> iommu_domain_alloc(private->iommu_dev->bus); >>> drivers/gpu/drm/tegra/drm.c: tegra->domain = >>> iommu_domain_alloc(&platform_bus_type); >>> drivers/infiniband/hw/usnic/usnic_uiom.c: pd->domain = domain = >>> iommu_domain_alloc(dev->bus); >>> >>> This series leave those cases unchanged and keep iommu_domain_alloc() >>> for their usage. But new drivers should not use it anymore. >> >> does it mean there is still domains allocated via iommu_domain_alloc() >> on VT-d platform? > > I think the drivers mentioned above do not run on x86 platforms, or do > they? cool. BTW. I know out-of-tree drivers are not counted in upstream review. Just out of curious, is there a formal way to let such drivers know it is no longer allowed to use iommu_domain_alloc() on VT-d?
On 5/31/24 11:16 AM, Yi Liu wrote: > On 2024/5/29 20:02, Baolu Lu wrote: >> On 2024/5/29 17:03, Yi Liu wrote: >>> On 2024/5/29 13:32, Lu Baolu wrote: >>>> The IOMMU subsystem has undergone some changes, including the removal >>>> of iommu_ops from the bus structure. Consequently, the existing domain >>>> allocation interface, which relies on a bus type argument, is no longer >>>> relevant: >>>> >>>> struct iommu_domain *iommu_domain_alloc(struct bus_type *bus) >>>> >>>> This series is designed to refactor the use of this interface. It >>>> proposes two new interfaces to replace iommu_domain_alloc(): >>>> >>>> - iommu_user_domain_alloc(): This interface is intended for allocating >>>> iommu domains managed by userspace for device passthrough scenarios, >>>> such as those used by iommufd, vfio, and vdpa. It clearly indicates >>>> that the domain is for user-managed device DMA. >>> >>> user paging domain? It looks to me user domain includes the nested >>> domains >>> as well. >> >> Yes, nested domain is a user domain. The iommu driver should implement >> iommu_ops->domain_alloc_user for nested domain allocation. > > will it be more clear to name iommu_user_domain_alloc() be > iommu_user_paging_domain_alloc() as it is mainly for paging domain > allocation? That might be better; let's wait and see if there's another option. > >>> >>>> If an IOMMU driver does not implement iommu_ops->domain_alloc_user, >>>> this interface will rollback to the generic paging domain >>>> allocation. >>>> >>>> - iommu_paging_domain_alloc(): This interface is for allocating iommu >>>> domains managed by kernel drivers for kernel DMA purposes. It >>>> takes a >>>> device pointer as a parameter, which better reflects the current >>>> design of the IOMMU subsystem. >>>> >>>> The majority of device drivers currently using iommu_domain_alloc() do >>>> so to allocate a domain for a specific device and then attach that >>>> domain to the device. These cases can be straightforwardly migrated to >>>> the new interfaces. >>>> >>>> However, there are some drivers with more complex use cases that do >>>> not fit neatly into this new scheme. For example: >>>> >>>> $ git grep "= iommu_domain_alloc" >>>> arch/arm/mm/dma-mapping.c: mapping->domain = >>>> iommu_domain_alloc(bus); >>>> drivers/gpu/drm/rockchip/rockchip_drm_drv.c: private->domain = >>>> iommu_domain_alloc(private->iommu_dev->bus); >>>> drivers/gpu/drm/tegra/drm.c: tegra->domain = >>>> iommu_domain_alloc(&platform_bus_type); >>>> drivers/infiniband/hw/usnic/usnic_uiom.c: pd->domain = domain >>>> = iommu_domain_alloc(dev->bus); >>>> >>>> This series leave those cases unchanged and keep iommu_domain_alloc() >>>> for their usage. But new drivers should not use it anymore. >>> >>> does it mean there is still domains allocated via iommu_domain_alloc() >>> on VT-d platform? >> >> I think the drivers mentioned above do not run on x86 platforms, or do >> they? > > cool. BTW. I know out-of-tree drivers are not counted in upstream review. > Just out of curious, is there a formal way to let such drivers know it is > no longer allowed to use iommu_domain_alloc() on VT-d? As Robin suggested, we should try to remove iommu_domain_alloc() from the tree in this series. Best regards, baolu
On 2024/5/31 14:00, Baolu Lu wrote: > On 5/31/24 11:16 AM, Yi Liu wrote: >> On 2024/5/29 20:02, Baolu Lu wrote: >>> On 2024/5/29 17:03, Yi Liu wrote: >>>> On 2024/5/29 13:32, Lu Baolu wrote: >>>>> The IOMMU subsystem has undergone some changes, including the removal >>>>> of iommu_ops from the bus structure. Consequently, the existing domain >>>>> allocation interface, which relies on a bus type argument, is no longer >>>>> relevant: >>>>> >>>>> struct iommu_domain *iommu_domain_alloc(struct bus_type *bus) >>>>> >>>>> This series is designed to refactor the use of this interface. It >>>>> proposes two new interfaces to replace iommu_domain_alloc(): >>>>> >>>>> - iommu_user_domain_alloc(): This interface is intended for allocating >>>>> iommu domains managed by userspace for device passthrough scenarios, >>>>> such as those used by iommufd, vfio, and vdpa. It clearly indicates >>>>> that the domain is for user-managed device DMA. >>>> >>>> user paging domain? It looks to me user domain includes the nested domains >>>> as well. >>> >>> Yes, nested domain is a user domain. The iommu driver should implement >>> iommu_ops->domain_alloc_user for nested domain allocation. >> >> will it be more clear to name iommu_user_domain_alloc() be >> iommu_user_paging_domain_alloc() as it is mainly for paging domain >> allocation? > > That might be better; let's wait and see if there's another option. > >> >>>> >>>>> If an IOMMU driver does not implement iommu_ops->domain_alloc_user, >>>>> this interface will rollback to the generic paging domain allocation. >>>>> >>>>> - iommu_paging_domain_alloc(): This interface is for allocating iommu >>>>> domains managed by kernel drivers for kernel DMA purposes. It takes a >>>>> device pointer as a parameter, which better reflects the current >>>>> design of the IOMMU subsystem. >>>>> >>>>> The majority of device drivers currently using iommu_domain_alloc() do >>>>> so to allocate a domain for a specific device and then attach that >>>>> domain to the device. These cases can be straightforwardly migrated to >>>>> the new interfaces. >>>>> >>>>> However, there are some drivers with more complex use cases that do >>>>> not fit neatly into this new scheme. For example: >>>>> >>>>> $ git grep "= iommu_domain_alloc" >>>>> arch/arm/mm/dma-mapping.c: mapping->domain = >>>>> iommu_domain_alloc(bus); >>>>> drivers/gpu/drm/rockchip/rockchip_drm_drv.c: private->domain = >>>>> iommu_domain_alloc(private->iommu_dev->bus); >>>>> drivers/gpu/drm/tegra/drm.c: tegra->domain = >>>>> iommu_domain_alloc(&platform_bus_type); >>>>> drivers/infiniband/hw/usnic/usnic_uiom.c: pd->domain = domain = >>>>> iommu_domain_alloc(dev->bus); >>>>> >>>>> This series leave those cases unchanged and keep iommu_domain_alloc() >>>>> for their usage. But new drivers should not use it anymore. >>>> >>>> does it mean there is still domains allocated via iommu_domain_alloc() >>>> on VT-d platform? >>> >>> I think the drivers mentioned above do not run on x86 platforms, or do >>> they? >> >> cool. BTW. I know out-of-tree drivers are not counted in upstream review. >> Just out of curious, is there a formal way to let such drivers know it is >> no longer allowed to use iommu_domain_alloc() on VT-d? > > As Robin suggested, we should try to remove iommu_domain_alloc() from > the tree in this series. If it's completely dropped, that's fine. OOT drivers should fail in the time of compiling.
On Wed, May 29, 2024 at 08:02:12PM +0800, Baolu Lu wrote: > > > drivers/infiniband/hw/usnic/usnic_uiom.c: pd->domain = domain > > > = iommu_domain_alloc(dev->bus); > > > > > > This series leave those cases unchanged and keep iommu_domain_alloc() > > > for their usage. But new drivers should not use it anymore. > > > > does it mean there is still domains allocated via iommu_domain_alloc() > > on VT-d platform? > > I think the drivers mentioned above do not run on x86 platforms, or do > they? usnic does.. What was preventing converting it? Jason
On 6/3/24 9:35 PM, Jason Gunthorpe wrote: > On Wed, May 29, 2024 at 08:02:12PM +0800, Baolu Lu wrote: >>>> drivers/infiniband/hw/usnic/usnic_uiom.c: pd->domain = domain >>>> = iommu_domain_alloc(dev->bus); >>>> >>>> This series leave those cases unchanged and keep iommu_domain_alloc() >>>> for their usage. But new drivers should not use it anymore. >>> >>> does it mean there is still domains allocated via iommu_domain_alloc() >>> on VT-d platform? >> >> I think the drivers mentioned above do not run on x86 platforms, or do >> they? > > usnic does.. What was preventing converting it? Nothing, just because it wasn't that obvious. :-) I will convert it in the next version. Best regards, baolu
The IOMMU subsystem has undergone some changes, including the removal of iommu_ops from the bus structure. Consequently, the existing domain allocation interface, which relies on a bus type argument, is no longer relevant: struct iommu_domain *iommu_domain_alloc(struct bus_type *bus) This series is designed to refactor the use of this interface. It proposes two new interfaces to replace iommu_domain_alloc(): - iommu_user_domain_alloc(): This interface is intended for allocating iommu domains managed by userspace for device passthrough scenarios, such as those used by iommufd, vfio, and vdpa. It clearly indicates that the domain is for user-managed device DMA. If an IOMMU driver does not implement iommu_ops->domain_alloc_user, this interface will rollback to the generic paging domain allocation. - iommu_paging_domain_alloc(): This interface is for allocating iommu domains managed by kernel drivers for kernel DMA purposes. It takes a device pointer as a parameter, which better reflects the current design of the IOMMU subsystem. The majority of device drivers currently using iommu_domain_alloc() do so to allocate a domain for a specific device and then attach that domain to the device. These cases can be straightforwardly migrated to the new interfaces. However, there are some drivers with more complex use cases that do not fit neatly into this new scheme. For example: $ git grep "= iommu_domain_alloc" arch/arm/mm/dma-mapping.c: mapping->domain = iommu_domain_alloc(bus); drivers/gpu/drm/rockchip/rockchip_drm_drv.c: private->domain = iommu_domain_alloc(private->iommu_dev->bus); drivers/gpu/drm/tegra/drm.c: tegra->domain = iommu_domain_alloc(&platform_bus_type); drivers/infiniband/hw/usnic/usnic_uiom.c: pd->domain = domain = iommu_domain_alloc(dev->bus); This series leave those cases unchanged and keep iommu_domain_alloc() for their usage. But new drivers should not use it anymore. The whole series is also available on GitHub: https://github.com/LuBaolu/intel-iommu/commits/iommu-domain-allocation-refactor-v1 Lu Baolu (20): iommu: Add iommu_user_domain_alloc() interface iommufd: Use iommu_user_domain_alloc() vfio/type1: Use iommu_paging_domain_alloc() vhost-vdpa: Use iommu_user_domain_alloc() iommu: Add iommu_paging_domain_alloc() interface drm/msm: Use iommu_paging_domain_alloc() drm/nouveau/tegra: Use iommu_paging_domain_alloc() gpu: host1x: Use iommu_paging_domain_alloc() media: nvidia: tegra: Use iommu_paging_domain_alloc() media: venus: firmware: Use iommu_paging_domain_alloc() ath10k: Use iommu_paging_domain_alloc() wifi: ath11k: Use iommu_paging_domain_alloc() remoteproc: Use iommu_paging_domain_alloc() soc/fsl/qbman: Use iommu_paging_domain_alloc() iommu/vt-d: Add helper to allocate paging domain iommu/vt-d: Add domain_alloc_paging support iommu/vt-d: Simplify compatibility check for identity domain iommu/vt-d: Enhance compatibility check for paging domain attach iommu/vt-d: Remove domain_update_iommu_cap() iommu/vt-d: Remove domain_update_iommu_superpage() include/linux/iommu.h | 12 + drivers/gpu/drm/msm/msm_iommu.c | 8 +- .../drm/nouveau/nvkm/engine/device/tegra.c | 4 +- drivers/gpu/host1x/dev.c | 6 +- drivers/iommu/intel/iommu.c | 319 ++++++++---------- drivers/iommu/intel/pasid.c | 28 +- drivers/iommu/iommu.c | 62 ++++ drivers/iommu/iommufd/hw_pagetable.c | 20 +- .../media/platform/nvidia/tegra-vde/iommu.c | 6 +- drivers/media/platform/qcom/venus/firmware.c | 6 +- drivers/net/wireless/ath/ath10k/snoc.c | 6 +- drivers/net/wireless/ath/ath11k/ahb.c | 6 +- drivers/remoteproc/remoteproc_core.c | 6 +- drivers/soc/fsl/qbman/qman_portal.c | 4 +- drivers/vfio/vfio_iommu_type1.c | 7 +- drivers/vhost/vdpa.c | 11 +- 16 files changed, 253 insertions(+), 258 deletions(-)