| Message ID | 20181111090341.120786-4-drinkcat@chromium.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | iommu/io-pgtable-arm-v7s: Use DMA32 zone for page tables | expand |
On Sun, Nov 11, 2018 at 05:03:41PM +0800, Nicolas Boichat wrote: > For level 1/2 pages, ensure GFP_DMA32 is used if CONFIG_ZONE_DMA32 > is defined (e.g. on arm64 platforms). > > For level 2 pages, allocate a slab cache in SLAB_CACHE_DMA32. > > Also, print an error when the physical address does not fit in > 32-bit, to make debugging easier in the future. > > Fixes: ad67f5a6545f ("arm64: replace ZONE_DMA with ZONE_DMA32") > Signed-off-by: Nicolas Boichat <drinkcat@chromium.org> > --- > > Changes since v1: > - Changed approach to use SLAB_CACHE_DMA32 added by the previous > commit. > - Use DMA or DMA32 depending on the architecture (DMA for arm, > DMA32 for arm64). > > drivers/iommu/io-pgtable-arm-v7s.c | 20 ++++++++++++++++---- > 1 file changed, 16 insertions(+), 4 deletions(-) > > diff --git a/drivers/iommu/io-pgtable-arm-v7s.c b/drivers/iommu/io-pgtable-arm-v7s.c > index 445c3bde04800c..996f7b6d00b44a 100644 > --- a/drivers/iommu/io-pgtable-arm-v7s.c > +++ b/drivers/iommu/io-pgtable-arm-v7s.c > @@ -161,6 +161,14 @@ > > #define ARM_V7S_TCR_PD1 BIT(5) > > +#ifdef CONFIG_ZONE_DMA32 > +#define ARM_V7S_TABLE_GFP_DMA GFP_DMA32 > +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA32 > +#else > +#define ARM_V7S_TABLE_GFP_DMA GFP_DMA > +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA > +#endif It's a bit grotty that GFP_DMA32 doesn't just map to GFP_DMA on 32-bit architectures, since then we wouldn't need this #ifdeffery afaict. Will
On Wed, 21 Nov 2018, Will Deacon wrote:
> > +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA32
SLAB_CACHE_DMA32??? WTH is going on here? We are trying to get rid of
the dma slab array.
On 21/11/2018 17:38, Christopher Lameter wrote: > On Wed, 21 Nov 2018, Will Deacon wrote: > >>> +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA32 > > SLAB_CACHE_DMA32??? WTH is going on here? We are trying to get rid of > the dma slab array. See the previous two patches in this series. If there's already a (better) way to have a kmem_cache which allocates its backing pages with GFP_DMA32, please do let us know. Robin.
On Wed 21-11-18 16:46:38, Will Deacon wrote: > On Sun, Nov 11, 2018 at 05:03:41PM +0800, Nicolas Boichat wrote: > > For level 1/2 pages, ensure GFP_DMA32 is used if CONFIG_ZONE_DMA32 > > is defined (e.g. on arm64 platforms). > > > > For level 2 pages, allocate a slab cache in SLAB_CACHE_DMA32. > > > > Also, print an error when the physical address does not fit in > > 32-bit, to make debugging easier in the future. > > > > Fixes: ad67f5a6545f ("arm64: replace ZONE_DMA with ZONE_DMA32") > > Signed-off-by: Nicolas Boichat <drinkcat@chromium.org> > > --- > > > > Changes since v1: > > - Changed approach to use SLAB_CACHE_DMA32 added by the previous > > commit. > > - Use DMA or DMA32 depending on the architecture (DMA for arm, > > DMA32 for arm64). > > > > drivers/iommu/io-pgtable-arm-v7s.c | 20 ++++++++++++++++---- > > 1 file changed, 16 insertions(+), 4 deletions(-) > > > > diff --git a/drivers/iommu/io-pgtable-arm-v7s.c b/drivers/iommu/io-pgtable-arm-v7s.c > > index 445c3bde04800c..996f7b6d00b44a 100644 > > --- a/drivers/iommu/io-pgtable-arm-v7s.c > > +++ b/drivers/iommu/io-pgtable-arm-v7s.c > > @@ -161,6 +161,14 @@ > > > > #define ARM_V7S_TCR_PD1 BIT(5) > > > > +#ifdef CONFIG_ZONE_DMA32 > > +#define ARM_V7S_TABLE_GFP_DMA GFP_DMA32 > > +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA32 > > +#else > > +#define ARM_V7S_TABLE_GFP_DMA GFP_DMA > > +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA > > +#endif > > It's a bit grotty that GFP_DMA32 doesn't just map to GFP_DMA on 32-bit > architectures, since then we wouldn't need this #ifdeffery afaict. But GFP_DMA32 should map to GFP_KERNEL on 32b, no? Or what exactly is going on in here?
On Wed, 21 Nov 2018, Robin Murphy wrote: > On 21/11/2018 17:38, Christopher Lameter wrote: > > On Wed, 21 Nov 2018, Will Deacon wrote: > > > > > > +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA32 > > > > SLAB_CACHE_DMA32??? WTH is going on here? We are trying to get rid of > > the dma slab array. > > See the previous two patches in this series. If there's already a (better) way > to have a kmem_cache which allocates its backing pages with GFP_DMA32, please > do let us know. Was not cced on the whole patchset. Trying to find it. Its best to allocate DMA memory through the page based allocation functions. dma_alloc_coherent() and friends.
On Thu, Nov 22, 2018 at 2:02 AM Michal Hocko <mhocko@kernel.org> wrote: > > On Wed 21-11-18 16:46:38, Will Deacon wrote: > > On Sun, Nov 11, 2018 at 05:03:41PM +0800, Nicolas Boichat wrote: > > > For level 1/2 pages, ensure GFP_DMA32 is used if CONFIG_ZONE_DMA32 > > > is defined (e.g. on arm64 platforms). > > > > > > For level 2 pages, allocate a slab cache in SLAB_CACHE_DMA32. > > > > > > Also, print an error when the physical address does not fit in > > > 32-bit, to make debugging easier in the future. > > > > > > Fixes: ad67f5a6545f ("arm64: replace ZONE_DMA with ZONE_DMA32") > > > Signed-off-by: Nicolas Boichat <drinkcat@chromium.org> > > > --- > > > > > > Changes since v1: > > > - Changed approach to use SLAB_CACHE_DMA32 added by the previous > > > commit. > > > - Use DMA or DMA32 depending on the architecture (DMA for arm, > > > DMA32 for arm64). > > > > > > drivers/iommu/io-pgtable-arm-v7s.c | 20 ++++++++++++++++---- > > > 1 file changed, 16 insertions(+), 4 deletions(-) > > > > > > diff --git a/drivers/iommu/io-pgtable-arm-v7s.c b/drivers/iommu/io-pgtable-arm-v7s.c > > > index 445c3bde04800c..996f7b6d00b44a 100644 > > > --- a/drivers/iommu/io-pgtable-arm-v7s.c > > > +++ b/drivers/iommu/io-pgtable-arm-v7s.c > > > @@ -161,6 +161,14 @@ > > > > > > #define ARM_V7S_TCR_PD1 BIT(5) > > > > > > +#ifdef CONFIG_ZONE_DMA32 > > > +#define ARM_V7S_TABLE_GFP_DMA GFP_DMA32 > > > +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA32 > > > +#else > > > +#define ARM_V7S_TABLE_GFP_DMA GFP_DMA > > > +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA > > > +#endif > > > > It's a bit grotty that GFP_DMA32 doesn't just map to GFP_DMA on 32-bit > > architectures, since then we wouldn't need this #ifdeffery afaict. > > But GFP_DMA32 should map to GFP_KERNEL on 32b, no? Or what exactly is > going on in here? GFP_DMA32 will fail due to check_slab_flags (aka GFP_SLAB_BUG_MASK before patch 1/3 of this series)... But yes, it may be neater if there was transparent remapping of GFP_DMA32/SLAB_CACHE_DMA32 to GFP_DMA/SLAB_CACHE_DMA on 32-bit arch... > -- > Michal Hocko > SUSE Labs
On 11/22/18 2:20 AM, Nicolas Boichat wrote: > On Thu, Nov 22, 2018 at 2:02 AM Michal Hocko <mhocko@kernel.org> wrote: >> >> On Wed 21-11-18 16:46:38, Will Deacon wrote: >>> On Sun, Nov 11, 2018 at 05:03:41PM +0800, Nicolas Boichat wrote: >>> >>> It's a bit grotty that GFP_DMA32 doesn't just map to GFP_DMA on 32-bit >>> architectures, since then we wouldn't need this #ifdeffery afaict. >> >> But GFP_DMA32 should map to GFP_KERNEL on 32b, no? Or what exactly is >> going on in here? > > GFP_DMA32 will fail due to check_slab_flags (aka GFP_SLAB_BUG_MASK > before patch 1/3 of this series)... But yes, it may be neater if there > was transparent remapping of GFP_DMA32/SLAB_CACHE_DMA32 to > GFP_DMA/SLAB_CACHE_DMA on 32-bit arch... I don't know about ARM, but AFAIK on x86 DMA means within first 4MB of physical memory, and DMA32 means within first 4GB. It doesn't matter if the CPU is running in 32bit or 64bit mode. But, when it runs 32bit, the kernel can direct map less than 4GB anyway, which means it doesn't need the extra DMA32 zone, i.e. GFP_KERNEL can only get you memory that's also acceptable for GFP_DMA32. But, DMA is still DMA, i.e. first 4MB. Remapping GFP_DMA32 to GFP_DMA on x86 wouldn't work, as the GFP_DMA32 allocations would then only use those 4MB and exhaust it very fast. >> -- >> Michal Hocko >> SUSE Labs
diff --git a/drivers/iommu/io-pgtable-arm-v7s.c b/drivers/iommu/io-pgtable-arm-v7s.c index 445c3bde04800c..996f7b6d00b44a 100644 --- a/drivers/iommu/io-pgtable-arm-v7s.c +++ b/drivers/iommu/io-pgtable-arm-v7s.c @@ -161,6 +161,14 @@ #define ARM_V7S_TCR_PD1 BIT(5) +#ifdef CONFIG_ZONE_DMA32 +#define ARM_V7S_TABLE_GFP_DMA GFP_DMA32 +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA32 +#else +#define ARM_V7S_TABLE_GFP_DMA GFP_DMA +#define ARM_V7S_TABLE_SLAB_CACHE SLAB_CACHE_DMA +#endif + typedef u32 arm_v7s_iopte; static bool selftest_running; @@ -198,13 +206,17 @@ static void *__arm_v7s_alloc_table(int lvl, gfp_t gfp, void *table = NULL; if (lvl == 1) - table = (void *)__get_dma_pages(__GFP_ZERO, get_order(size)); + table = (void *)__get_free_pages( + __GFP_ZERO | ARM_V7S_TABLE_GFP_DMA, get_order(size)); else if (lvl == 2) - table = kmem_cache_zalloc(data->l2_tables, gfp | GFP_DMA); + table = kmem_cache_zalloc(data->l2_tables, + gfp | ARM_V7S_TABLE_GFP_DMA); phys = virt_to_phys(table); - if (phys != (arm_v7s_iopte)phys) + if (phys != (arm_v7s_iopte)phys) { /* Doesn't fit in PTE */ + dev_err(dev, "Page table does not fit in PTE: %pa", &phys); goto out_free; + } if (table && !(cfg->quirks & IO_PGTABLE_QUIRK_NO_DMA)) { dma = dma_map_single(dev, table, size, DMA_TO_DEVICE); if (dma_mapping_error(dev, dma)) @@ -737,7 +749,7 @@ static struct io_pgtable *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg, data->l2_tables = kmem_cache_create("io-pgtable_armv7s_l2", ARM_V7S_TABLE_SIZE(2), ARM_V7S_TABLE_SIZE(2), - SLAB_CACHE_DMA, NULL); + ARM_V7S_TABLE_SLAB_CACHE, NULL); if (!data->l2_tables) goto out_free_data;
For level 1/2 pages, ensure GFP_DMA32 is used if CONFIG_ZONE_DMA32 is defined (e.g. on arm64 platforms). For level 2 pages, allocate a slab cache in SLAB_CACHE_DMA32. Also, print an error when the physical address does not fit in 32-bit, to make debugging easier in the future. Fixes: ad67f5a6545f ("arm64: replace ZONE_DMA with ZONE_DMA32") Signed-off-by: Nicolas Boichat <drinkcat@chromium.org> --- Changes since v1: - Changed approach to use SLAB_CACHE_DMA32 added by the previous commit. - Use DMA or DMA32 depending on the architecture (DMA for arm, DMA32 for arm64). drivers/iommu/io-pgtable-arm-v7s.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-)