| Message ID | 20230609131740.7496-2-linyunsheng@huawei.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
| Series | [net-next,v3,1/4] page_pool: frag API support for 32-bit arch with 64-bit DMA | expand |
On 09/06/2023 15.17, Yunsheng Lin wrote: > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > index a7c526ee5024..cd4ac378cc63 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > @@ -832,6 +832,15 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params, > /* Create a page_pool and register it with rxq */ > struct page_pool_params pp_params = { 0 }; > > + /* Return error here to aoivd writing to page->pp_frag_count in ^^^^^ Typo > + * mlx5e_page_release_fragmented() for page->pp_frag_count is not > + * usable for arch with PAGE_POOL_DMA_USE_PP_FRAG_COUNT being true. > + */ > + if (PAGE_POOL_DMA_USE_PP_FRAG_COUNT) { > + err = -EINVAL; > + goto err_free_by_rq_type; > + } > + > pp_params.order = 0; > pp_params.flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV | PP_FLAG_PAGE_FRAG; > pp_params.pool_size = pool_size; > diff --git a/include/net/page_pool.h b/include/net/page_pool.h > index 126f9e294389..5c7f7501f300 100644 > --- a/include/net/page_pool.h > +++ b/include/net/page_pool.h > @@ -33,6 +33,7 @@ > #include <linux/mm.h> /* Needed by ptr_ring */ > #include <linux/ptr_ring.h> > #include <linux/dma-direction.h> > +#include <linux/dma-mapping.h> > > #define PP_FLAG_DMA_MAP BIT(0) /* Should page_pool do the DMA > * map/unmap > @@ -50,6 +51,9 @@ > PP_FLAG_DMA_SYNC_DEV |\ > PP_FLAG_PAGE_FRAG) > > +#define PAGE_POOL_DMA_USE_PP_FRAG_COUNT \ > + (sizeof(dma_addr_t) > sizeof(unsigned long)) > + I have a problem with the name PAGE_POOL_DMA_USE_PP_FRAG_COUNT because it is confusing to read in an if-statement. Proposals rename to: DMA_OVERLAP_PP_FRAG_COUNT Or: MM_DMA_OVERLAP_PP_FRAG_COUNT Or: DMA_ADDR_OVERLAP_PP_FRAG_COUNT Notice how I also removed the prefix PAGE_POOL_ because this is a MM-layer constraint and not a property of page_pool. --Jesper
On 2023/6/9 23:02, Jesper Dangaard Brouer wrote: ... >> PP_FLAG_DMA_SYNC_DEV |\ >> PP_FLAG_PAGE_FRAG) >> +#define PAGE_POOL_DMA_USE_PP_FRAG_COUNT \ >> + (sizeof(dma_addr_t) > sizeof(unsigned long)) >> + > > I have a problem with the name PAGE_POOL_DMA_USE_PP_FRAG_COUNT > because it is confusing to read in an if-statement. Actually, it is already in an if-statement before this patch:) Maybe starting to use it in the driver is confusing to you? If not, maybe we can keep it that for now, and change it when we come up with a better name. > > Proposals rename to: DMA_OVERLAP_PP_FRAG_COUNT > Or: MM_DMA_OVERLAP_PP_FRAG_COUNT > Or: DMA_ADDR_OVERLAP_PP_FRAG_COUNT It seems DMA_ADDR_OVERLAP_PP_FRAG_COUNT is better, and DMA_ADDR_UPPER_OVERLAP_PP_FRAG_COUNT seems more accurate if a longer macro name is not an issue here. > > Notice how I also removed the prefix PAGE_POOL_ because this is a MM-layer constraint and not a property of page_pool. I am not sure if it is a MM-layer constraint yet. Do you mean 'MM-layer constraint' as 'struct page' not having enough space for page pool with 32-bit arch with 64-bit DMA? If that is the case, we may need a more generic name for that constraint instead of 'DMA_ADDR_OVERLAP_PP_FRAG_COUNT'? And a more generic name seems confusing for page pool too, as it doesn't tell that we only have that problem for 32-bit arch with 64-bit DMA. So if the above makes sense, it seems we may need to keep the PAGE_POOL_ prefix, which would be 'PAGE_POOL_DMA_ADDR_UPPER_OVERLAP_PP_FRAG_COUNT' if the long name is not issue here. Anyway, naming is hard, we may need a seperate patch to explain it, which is not really related to this patchset IHMO, so I'd rather keep it as before if we can not come up with a name which is not confusing to most people. > > > --Jesper > >
On 10/06/2023 15.13, Yunsheng Lin wrote: > On 2023/6/9 23:02, Jesper Dangaard Brouer wrote: > ... > >>> PP_FLAG_DMA_SYNC_DEV |\ >>> PP_FLAG_PAGE_FRAG) >>> +#define PAGE_POOL_DMA_USE_PP_FRAG_COUNT \ >>> + (sizeof(dma_addr_t) > sizeof(unsigned long)) >>> + >> >> I have a problem with the name PAGE_POOL_DMA_USE_PP_FRAG_COUNT >> because it is confusing to read in an if-statement. > > Actually, it is already in an if-statement before this patch:) I did notice, but I've had a problem with this name for a while. (see later, why this might be long in separate patch) > Maybe starting to use it in the driver is confusing to you? > If not, maybe we can keep it that for now, and change it when > we come up with a better name. > >> >> Proposals rename to: DMA_OVERLAP_PP_FRAG_COUNT >> Or: MM_DMA_OVERLAP_PP_FRAG_COUNT >> Or: DMA_ADDR_OVERLAP_PP_FRAG_COUNT > > It seems DMA_ADDR_OVERLAP_PP_FRAG_COUNT is better, > and DMA_ADDR_UPPER_OVERLAP_PP_FRAG_COUNT seems more accurate if a > longer macro name is not an issue here. > I like the shorter DMA_ADDR_OVERLAP_PP_FRAG_COUNT variant best. >> >> Notice how I also removed the prefix PAGE_POOL_ because this is a >> MM-layer constraint and not a property of page_pool. > > I am not sure if it is a MM-layer constraint yet. > Do you mean 'MM-layer constraint' as 'struct page' not having > enough space for page pool with 32-bit arch with 64-bit DMA? Yes. > If that is the case, we may need a more generic name for that > constraint instead of 'DMA_ADDR_OVERLAP_PP_FRAG_COUNT'? > I think this name is clear enough; the dma_addr_t is overlapping the pp_frag_count. > And a more generic name seems confusing for page pool too, as > it doesn't tell that we only have that problem for 32-bit arch > with 64-bit DMA. > > So if the above makes sense, it seems we may need to keep the > PAGE_POOL_ prefix, which would be > 'PAGE_POOL_DMA_ADDR_UPPER_OVERLAP_PP_FRAG_COUNT' if the long > name is not issue here. > I think it gets too long now. Also I still disagree with PAGE_POOL_ prefix, if anything it is a property of 'struct page'. Thus a prefix with PAGE_ make more sense to me, but it also gets too long (for my taste). > Anyway, naming is hard, we may need a seperate patch to explain > it, which is not really related to this patchset IHMO, so I'd > rather keep it as before if we can not come up with a name which > is not confusing to most people. > Okay, lets do the (re)naming in another patch then. --Jesper
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index a7c526ee5024..cd4ac378cc63 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -832,6 +832,15 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params, /* Create a page_pool and register it with rxq */ struct page_pool_params pp_params = { 0 }; + /* Return error here to aoivd writing to page->pp_frag_count in + * mlx5e_page_release_fragmented() for page->pp_frag_count is not + * usable for arch with PAGE_POOL_DMA_USE_PP_FRAG_COUNT being true. + */ + if (PAGE_POOL_DMA_USE_PP_FRAG_COUNT) { + err = -EINVAL; + goto err_free_by_rq_type; + } + pp_params.order = 0; pp_params.flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV | PP_FLAG_PAGE_FRAG; pp_params.pool_size = pool_size; diff --git a/include/net/page_pool.h b/include/net/page_pool.h index 126f9e294389..5c7f7501f300 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -33,6 +33,7 @@ #include <linux/mm.h> /* Needed by ptr_ring */ #include <linux/ptr_ring.h> #include <linux/dma-direction.h> +#include <linux/dma-mapping.h> #define PP_FLAG_DMA_MAP BIT(0) /* Should page_pool do the DMA * map/unmap @@ -50,6 +51,9 @@ PP_FLAG_DMA_SYNC_DEV |\ PP_FLAG_PAGE_FRAG) +#define PAGE_POOL_DMA_USE_PP_FRAG_COUNT \ + (sizeof(dma_addr_t) > sizeof(unsigned long)) + /* * Fast allocation side cache array/stack * @@ -219,8 +223,33 @@ static inline struct page *page_pool_dev_alloc_pages(struct page_pool *pool) return page_pool_alloc_pages(pool, gfp); } -struct page *page_pool_alloc_frag(struct page_pool *pool, unsigned int *offset, - unsigned int size, gfp_t gfp); +struct page *__page_pool_alloc_frag(struct page_pool *pool, + unsigned int *offset, unsigned int size, + gfp_t gfp); + +static inline struct page *page_pool_alloc_frag(struct page_pool *pool, + unsigned int *offset, + unsigned int size, gfp_t gfp) +{ + unsigned int max_size = PAGE_SIZE << pool->p.order; + + size = ALIGN(size, dma_get_cache_alignment()); + + if (WARN_ON(!(pool->p.flags & PP_FLAG_PAGE_FRAG) || + size > max_size)) + return NULL; + + /* Don't allow page splitting and allocate one big frag + * for 32-bit arch with 64-bit DMA, corresponding to + * the checking in page_pool_is_last_frag(). + */ + if (PAGE_POOL_DMA_USE_PP_FRAG_COUNT) { + *offset = 0; + return page_pool_alloc_pages(pool, gfp); + } + + return __page_pool_alloc_frag(pool, offset, size, gfp); +} static inline struct page *page_pool_dev_alloc_frag(struct page_pool *pool, unsigned int *offset, @@ -322,8 +351,14 @@ static inline long page_pool_defrag_page(struct page *page, long nr) static inline bool page_pool_is_last_frag(struct page_pool *pool, struct page *page) { - /* If fragments aren't enabled or count is 0 we were the last user */ + /* We assume we are the last frag user that is still holding + * on to the page if: + * 1. Fragments aren't enabled. + * 2. We are running in 32-bit arch with 64-bit DMA. + * 3. page_pool_defrag_page() indicate we are the last user. + */ return !(pool->p.flags & PP_FLAG_PAGE_FRAG) || + PAGE_POOL_DMA_USE_PP_FRAG_COUNT || (page_pool_defrag_page(page, 1) == 0); } @@ -357,9 +392,6 @@ static inline void page_pool_recycle_direct(struct page_pool *pool, page_pool_put_full_page(pool, page, true); } -#define PAGE_POOL_DMA_USE_PP_FRAG_COUNT \ - (sizeof(dma_addr_t) > sizeof(unsigned long)) - static inline dma_addr_t page_pool_get_dma_addr(struct page *page) { dma_addr_t ret = page->dma_addr; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index a3e12a61d456..9c4118c62997 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -14,7 +14,6 @@ #include <net/xdp.h> #include <linux/dma-direction.h> -#include <linux/dma-mapping.h> #include <linux/page-flags.h> #include <linux/mm.h> /* for put_page() */ #include <linux/poison.h> @@ -200,10 +199,6 @@ static int page_pool_init(struct page_pool *pool, */ } - if (PAGE_POOL_DMA_USE_PP_FRAG_COUNT && - pool->p.flags & PP_FLAG_PAGE_FRAG) - return -EINVAL; - #ifdef CONFIG_PAGE_POOL_STATS pool->recycle_stats = alloc_percpu(struct page_pool_recycle_stats); if (!pool->recycle_stats) @@ -715,18 +710,13 @@ static void page_pool_free_frag(struct page_pool *pool) page_pool_return_page(pool, page); } -struct page *page_pool_alloc_frag(struct page_pool *pool, - unsigned int *offset, - unsigned int size, gfp_t gfp) +struct page *__page_pool_alloc_frag(struct page_pool *pool, + unsigned int *offset, + unsigned int size, gfp_t gfp) { unsigned int max_size = PAGE_SIZE << pool->p.order; struct page *page = pool->frag_page; - if (WARN_ON(!(pool->p.flags & PP_FLAG_PAGE_FRAG) || - size > max_size)) - return NULL; - - size = ALIGN(size, dma_get_cache_alignment()); *offset = pool->frag_offset; if (page && *offset + size > max_size) { @@ -759,7 +749,7 @@ struct page *page_pool_alloc_frag(struct page_pool *pool, alloc_stat_inc(pool, fast); return page; } -EXPORT_SYMBOL(page_pool_alloc_frag); +EXPORT_SYMBOL(__page_pool_alloc_frag); static void page_pool_empty_ring(struct page_pool *pool) {
Currently page_pool_alloc_frag() is not supported in 32-bit arch with 64-bit DMA, which seems to be quite common, see [1], which means driver may need to handle it when using page_pool_alloc_frag() API. In order to simplify the driver's work for supporting page frag, this patch allows page_pool_alloc_frag() to call page_pool_alloc_pages() to return a big page frag without page splitting because of overlap issue between pp_frag_count and dma_addr_upper in 'struct page' for those arches. As page_pool_create() with PP_FLAG_PAGE_FRAG is supported in 32-bit arch with 64-bit DMA now, mlx5 calls page_pool_create() with PP_FLAG_PAGE_FRAG and manipulate the page->pp_frag_count directly using the page_pool_defrag_page(), so add a checking for it to aoivd writing to page->pp_frag_count that may not exist in some arch. Note that it may aggravate truesize underestimate problem for skb as there is no page splitting for those pages, if driver need a accuate truesize, it may calculate that according to frag size, page order and PAGE_POOL_DMA_USE_PP_FRAG_COUNT being true or not. And we may provide a helper for that if it turns out to be helpful. 1. https://lore.kernel.org/all/20211117075652.58299-1-linyunsheng@huawei.com/ Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com> CC: Lorenzo Bianconi <lorenzo@kernel.org> CC: Alexander Duyck <alexander.duyck@gmail.com> --- .../net/ethernet/mellanox/mlx5/core/en_main.c | 9 ++++ include/net/page_pool.h | 44 ++++++++++++++++--- net/core/page_pool.c | 18 ++------ 3 files changed, 51 insertions(+), 20 deletions(-)