| Message ID | 20200711064846.16007-19-yong.wu@mediatek.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | MT8192 IOMMU support | expand |
On Sat, Jul 11, 2020 at 02:48:43PM +0800, Yong Wu wrote: > Some HW IP(ex: CCU) require the special iova range. That means the > iova got from dma_alloc_attrs for that devices must locate in his > special range. In this patch, we allocate a special iova_range for > each a special requirement and create each a iommu domain for each > a iova_range. > > meanwhile we still use one pagetable which support 16GB iova. > > After this patch, If the iova range of a master is over 4G, the master > should: > a) Declare its special dma_ranges in its dtsi node. For example, If we > preassign the iova 4G-8G for vcodec, then the vcodec dtsi node should: > dma-ranges = <0x1 0x0 0x1 0x0 0x1 0x0>; /* 4G ~ 8G */ BTW, dma-ranges should be in the parent node of the vcodec. > b) Update the dma_mask: > dma_set_mask_and_coherent(dev, DMA_BIT_MASK(33)); This should happen for you automatically. The DMA PFN offset should also be 4GB here. > > Signed-off-by: Yong Wu <yong.wu@mediatek.com> > --- > drivers/iommu/mtk_iommu.c | 49 ++++++++++++++++++++++++++++++++------- > drivers/iommu/mtk_iommu.h | 3 ++- > 2 files changed, 42 insertions(+), 10 deletions(-)
On Thu, 2020-07-23 at 14:47 -0600, Rob Herring wrote: > On Sat, Jul 11, 2020 at 02:48:43PM +0800, Yong Wu wrote: > > Some HW IP(ex: CCU) require the special iova range. That means the > > iova got from dma_alloc_attrs for that devices must locate in his > > special range. In this patch, we allocate a special iova_range for > > each a special requirement and create each a iommu domain for each > > a iova_range. > > > > meanwhile we still use one pagetable which support 16GB iova. > > > > After this patch, If the iova range of a master is over 4G, the master > > should: > > a) Declare its special dma_ranges in its dtsi node. For example, If we > > preassign the iova 4G-8G for vcodec, then the vcodec dtsi node should: > > dma-ranges = <0x1 0x0 0x1 0x0 0x1 0x0>; /* 4G ~ 8G */ > > BTW, dma-ranges should be in the parent node of the vcodec. But the vcodec doesn't have its special parent node. Currently the vcodec/display dtsi like this: soc { ovl:{ /* display */ /*No dma-ranges property. defaultly it is 0-4G iova range. */ } vcodec_dec: { /* decode */ dma-ranges = <0x1 0x0 0x1 0x0 0x1 0x0>; /* 4G ~ 8G*/ }; vcodec_enc: { /* encode */ dma-ranges = <0x1 0x0 0x1 0x0 0x1 0x0>; /* 4G ~ 8G*/ }; camera: { dma-ranges = <0x2 0x0 0x2 0x0 0x1 0x0>; /* 8G ~ 12G */ }; } If we add the parent node for vcodec, the vcodec driver flow will be changed, and it may be incompatible with the previous dtb. Here we don't have the actual bus concept. currently we support 16GB dma_addr(iova) ranges. we only preassign 4-8G for vcodec, 8G-12G for camera. If the usage of dma-ranges here is different from the common one. then how should I do here? Thanks. > > > b) Update the dma_mask: > > dma_set_mask_and_coherent(dev, DMA_BIT_MASK(33)); > > This should happen for you automatically. The DMA PFN offset > should also be 4GB here. I may not follow here. If the iova start at 0x1_0000_0000, phys address start at 0x4000_0000. Do you means the dma-ranges should be <0x1 0 0x0 0x40000000 0x1 0x0>? then dma_pfn_offset = PFN_DOWN(paddr - dma_addr) = 0xffffffff40000. this is also ok for us. we don't call the macro regarding this "dev->dma_pfn_offset" The purpose that I call it here is for updating the dev->coherent_dma_mask[1], then we could get the iova over 4GB. [1] https://elixir.bootlin.com/linux/v5.8-rc1/source/drivers/iommu/dma-iommu.c#L619 > > > > > Signed-off-by: Yong Wu <yong.wu@mediatek.com> > > --- > > drivers/iommu/mtk_iommu.c | 49 ++++++++++++++++++++++++++++++++------- > > drivers/iommu/mtk_iommu.h | 3 ++- > > 2 files changed, 42 insertions(+), 10 deletions(-)
diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c index 766c9e73d541..7dfd8071a858 100644 --- a/drivers/iommu/mtk_iommu.c +++ b/drivers/iommu/mtk_iommu.c @@ -361,6 +361,14 @@ static int mtk_iommu_domain_finalise(struct mtk_iommu_domain *dom) { struct mtk_iommu_data *data = mtk_iommu_get_m4u_data(); + /* Use the exist domain as there is one m4u pgtable here. */ + if (data->m4u_dom) { + dom->iop = data->m4u_dom->iop; + dom->cfg = data->m4u_dom->cfg; + dom->domain.pgsize_bitmap = data->m4u_dom->cfg.pgsize_bitmap; + return 0; + } + dom->cfg = (struct io_pgtable_cfg) { .quirks = IO_PGTABLE_QUIRK_ARM_NS | IO_PGTABLE_QUIRK_NO_PERMS | @@ -386,6 +394,8 @@ static int mtk_iommu_domain_finalise(struct mtk_iommu_domain *dom) static struct iommu_domain *mtk_iommu_domain_alloc(unsigned type) { + struct mtk_iommu_data *data = mtk_iommu_get_m4u_data(); + const struct mtk_iommu_iova_region *region; struct mtk_iommu_domain *dom; if (type != IOMMU_DOMAIN_DMA) @@ -401,8 +411,10 @@ static struct iommu_domain *mtk_iommu_domain_alloc(unsigned type) if (mtk_iommu_domain_finalise(dom)) goto put_dma_cookie; - dom->domain.geometry.aperture_start = 0; - dom->domain.geometry.aperture_end = DMA_BIT_MASK(32); + region = data->plat_data->iova_region + data->cur_domid; + dom->domain.geometry.aperture_start = region->iova_base; + dom->domain.geometry.aperture_end = region->iova_base + + region->size - 1; dom->domain.geometry.force_aperture = true; return &dom->domain; @@ -540,19 +552,31 @@ static void mtk_iommu_release_device(struct device *dev) static struct iommu_group *mtk_iommu_device_group(struct device *dev) { struct mtk_iommu_data *data = mtk_iommu_get_m4u_data(); + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct iommu_group *group; + int domid; if (!data) return ERR_PTR(-ENODEV); - /* All the client devices are in the same m4u iommu-group */ - if (!data->m4u_group) { - data->m4u_group = iommu_group_alloc(); - if (IS_ERR(data->m4u_group)) + domid = MTK_M4U_TO_DOM(fwspec->ids[0]); + if (domid >= data->plat_data->iova_region_nr) { + dev_err(dev, "domain id(%d/%d) is error.\n", domid, + data->plat_data->iova_region_nr); + return ERR_PTR(-EINVAL); + } + + group = data->m4u_group[domid]; + if (!group) { + group = iommu_group_alloc(); + if (IS_ERR(group)) dev_err(dev, "Failed to allocate M4U IOMMU group\n"); + data->m4u_group[domid] = group; } else { - iommu_group_ref_get(data->m4u_group); + iommu_group_ref_get(group); } - return data->m4u_group; + data->cur_domid = domid; + return group; } static int mtk_iommu_of_xlate(struct device *dev, struct of_phandle_args *args) @@ -581,14 +605,21 @@ static void mtk_iommu_get_resv_regions(struct device *dev, struct list_head *head) { struct mtk_iommu_data *data = dev_iommu_priv_get(dev); - const struct mtk_iommu_iova_region *resv; + const struct mtk_iommu_iova_region *resv, *curdom; struct iommu_resv_region *region; int prot = IOMMU_WRITE | IOMMU_READ; unsigned int i; + curdom = data->plat_data->iova_region + data->cur_domid; for (i = 0; i < data->plat_data->iova_region_nr; i++) { resv = data->plat_data->iova_region + i; + /* Only reserve when the region is in the current domain */ + if (resv->iova_base <= curdom->iova_base || + resv->iova_base + resv->size >= + curdom->iova_base + curdom->size) + continue; + region = iommu_alloc_resv_region(resv->iova_base, resv->size, prot, IOMMU_RESV_RESERVED); if (!region) diff --git a/drivers/iommu/mtk_iommu.h b/drivers/iommu/mtk_iommu.h index bb929b875d8c..11795b8d82ff 100644 --- a/drivers/iommu/mtk_iommu.h +++ b/drivers/iommu/mtk_iommu.h @@ -65,7 +65,7 @@ struct mtk_iommu_data { phys_addr_t protect_base; /* protect memory base */ struct mtk_iommu_suspend_reg reg; struct mtk_iommu_domain *m4u_dom; - struct iommu_group *m4u_group; + struct iommu_group *m4u_group[MTK_M4U_DOM_NR_MAX]; bool enable_4GB; spinlock_t tlb_lock; /* lock for tlb range flush */ @@ -73,6 +73,7 @@ struct mtk_iommu_data { const struct mtk_iommu_plat_data *plat_data; struct device *smicomm_dev; + unsigned int cur_domid; struct list_head list; struct mtk_smi_larb_iommu larb_imu[MTK_LARB_NR_MAX]; };
Some HW IP(ex: CCU) require the special iova range. That means the iova got from dma_alloc_attrs for that devices must locate in his special range. In this patch, we allocate a special iova_range for each a special requirement and create each a iommu domain for each a iova_range. meanwhile we still use one pagetable which support 16GB iova. After this patch, If the iova range of a master is over 4G, the master should: a) Declare its special dma_ranges in its dtsi node. For example, If we preassign the iova 4G-8G for vcodec, then the vcodec dtsi node should: dma-ranges = <0x1 0x0 0x1 0x0 0x1 0x0>; /* 4G ~ 8G */ b) Update the dma_mask: dma_set_mask_and_coherent(dev, DMA_BIT_MASK(33)); Signed-off-by: Yong Wu <yong.wu@mediatek.com> --- drivers/iommu/mtk_iommu.c | 49 ++++++++++++++++++++++++++++++++------- drivers/iommu/mtk_iommu.h | 3 ++- 2 files changed, 42 insertions(+), 10 deletions(-)