From patchwork Mon Nov 13 13:00:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13453962 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AFB511DA3F for ; Mon, 13 Nov 2023 13:00:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=none Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 917E419B5; Mon, 13 Nov 2023 05:00:33 -0800 (PST) Received: from dggpemm500005.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4STTzv4QsTzWhJn; Mon, 13 Nov 2023 21:00:11 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by dggpemm500005.china.huawei.com (7.185.36.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 13 Nov 2023 21:00:31 +0800 From: Yunsheng Lin To: , , CC: , , Mina Almasry , Jesper Dangaard Brouer , Ilias Apalodimas , Eric Dumazet Subject: [PATCH RFC 1/8] net: page_pool: factor out releasing DMA from releasing the page Date: Mon, 13 Nov 2023 21:00:33 +0800 Message-ID: <20231113130041.58124-2-linyunsheng@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20231113130041.58124-1-linyunsheng@huawei.com> References: <20231113130041.58124-1-linyunsheng@huawei.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500005.china.huawei.com (7.185.36.74) X-CFilter-Loop: Reflected X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Jakub Kicinski Releasing the DMA mapping will be useful for other types of pages, so factor it out. Make sure compiler inlines it, to avoid any regressions. Signed-off-by: Jakub Kicinski Signed-off-by: Mina Almasry --- net/core/page_pool.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 5e409b98aba0..578b6f2eeb46 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -514,21 +514,16 @@ static s32 page_pool_inflight(struct page_pool *pool) return inflight; } -/* Disconnects a page (from a page_pool). API users can have a need - * to disconnect a page (from a page_pool), to allow it to be used as - * a regular page (that will eventually be returned to the normal - * page-allocator via put_page). - */ -static void page_pool_return_page(struct page_pool *pool, struct page *page) +static __always_inline +void __page_pool_release_page_dma(struct page_pool *pool, struct page *page) { dma_addr_t dma; - int count; if (!(pool->p.flags & PP_FLAG_DMA_MAP)) /* Always account for inflight pages, even if we didn't * map them */ - goto skip_dma_unmap; + return; dma = page_pool_get_dma_addr(page); @@ -537,7 +532,19 @@ static void page_pool_return_page(struct page_pool *pool, struct page *page) PAGE_SIZE << pool->p.order, pool->p.dma_dir, DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_WEAK_ORDERING); page_pool_set_dma_addr(page, 0); -skip_dma_unmap: +} + +/* Disconnects a page (from a page_pool). API users can have a need + * to disconnect a page (from a page_pool), to allow it to be used as + * a regular page (that will eventually be returned to the normal + * page-allocator via put_page). + */ +void page_pool_return_page(struct page_pool *pool, struct page *page) +{ + int count; + + __page_pool_release_page_dma(pool, page); + page_pool_clear_pp_info(page); /* This may be the last page returned, releasing the pool, so From patchwork Mon Nov 13 13:00:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13453965 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D93CE1DA53 for ; Mon, 13 Nov 2023 13:00:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=none Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC0E2171A; Mon, 13 Nov 2023 05:00:35 -0800 (PST) Received: from dggpemm500005.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4STV026PRSzvQQN; Mon, 13 Nov 2023 21:00:18 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by dggpemm500005.china.huawei.com (7.185.36.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 13 Nov 2023 21:00:33 +0800 From: Yunsheng Lin To: , , CC: , , Mina Almasry , Yunsheng Lin , Jesper Dangaard Brouer , Ilias Apalodimas , Eric Dumazet Subject: [PATCH RFC 2/8] net: page_pool: create hooks for custom page providers Date: Mon, 13 Nov 2023 21:00:34 +0800 Message-ID: <20231113130041.58124-3-linyunsheng@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20231113130041.58124-1-linyunsheng@huawei.com> References: <20231113130041.58124-1-linyunsheng@huawei.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500005.china.huawei.com (7.185.36.74) X-CFilter-Loop: Reflected X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Jakub Kicinski The page providers which try to reuse the same pages will need to hold onto the ref, even if page gets released from the pool - as in releasing the page from the pp just transfers the "ownership" reference from pp to the provider, and provider will wait for other references to be gone before feeding this page back into the pool. Signed-off-by: Jakub Kicinski Signed-off-by: Mina Almasry Signed-off-by: Yunsheng Lin --- include/net/page_pool/types.h | 19 ++++++++ net/core/page_pool.c | 81 +++++++++++++++++++++++++++++------ 2 files changed, 87 insertions(+), 13 deletions(-) diff --git a/include/net/page_pool/types.h b/include/net/page_pool/types.h index 6fc5134095ed..5e4fcd45ba50 100644 --- a/include/net/page_pool/types.h +++ b/include/net/page_pool/types.h @@ -60,6 +60,8 @@ struct page_pool_params { int nid; struct device *dev; struct napi_struct *napi; + u8 memory_provider; + void *mp_priv; enum dma_data_direction dma_dir; unsigned int max_len; unsigned int offset; @@ -118,6 +120,20 @@ struct page_pool_stats { }; #endif +struct mem_provider; + +enum pp_memory_provider_type { + __PP_MP_NONE, /* Use system allocator directly */ +}; + +struct pp_memory_provider_ops { + int (*init)(struct page_pool *pool); + void (*destroy)(struct page_pool *pool); + struct page *(*alloc_pages)(struct page_pool *pool, gfp_t gfp); + void (*release_page)(struct page_pool *pool, struct page *page); + void (*free_pages)(struct page_pool *pool, struct page *page); +}; + struct page_pool { struct page_pool_params p; @@ -165,6 +181,9 @@ struct page_pool { */ struct ptr_ring ring; + const struct pp_memory_provider_ops *mp_ops; + void *mp_priv; + #ifdef CONFIG_PAGE_POOL_STATS /* recycle stats are per-cpu to avoid locking */ struct page_pool_recycle_stats __percpu *recycle_stats; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 578b6f2eeb46..6c502bea842b 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -23,6 +23,8 @@ #include +static DEFINE_STATIC_KEY_FALSE(page_pool_mem_providers); + #define DEFER_TIME (msecs_to_jiffies(1000)) #define DEFER_WARN_INTERVAL (60 * HZ) @@ -172,6 +174,7 @@ static int page_pool_init(struct page_pool *pool, const struct page_pool_params *params) { unsigned int ring_qsize = 1024; /* Default */ + int err; memcpy(&pool->p, params, sizeof(pool->p)); @@ -225,10 +228,34 @@ static int page_pool_init(struct page_pool *pool, /* Driver calling page_pool_create() also call page_pool_destroy() */ refcount_set(&pool->user_cnt, 1); + switch (pool->p.memory_provider) { + case __PP_MP_NONE: + break; + default: + err = -EINVAL; + goto free_ptr_ring; + } + + pool->mp_priv = pool->p.mp_priv; + if (pool->mp_ops) { + err = pool->mp_ops->init(pool); + if (err) { + pr_warn("%s() mem-provider init failed %d\n", + __func__, err); + goto free_ptr_ring; + } + + static_branch_inc(&page_pool_mem_providers); + } + if (pool->p.flags & PP_FLAG_DMA_MAP) get_device(pool->p.dev); return 0; + +free_ptr_ring: + ptr_ring_cleanup(&pool->ring, NULL); + return err; } /** @@ -257,18 +284,13 @@ EXPORT_SYMBOL(page_pool_create); static void page_pool_return_page(struct page_pool *pool, struct page *page); -noinline -static struct page *page_pool_refill_alloc_cache(struct page_pool *pool) +static bool page_pool_page_in_pref_node(struct page_pool *pool, struct page *page) { - struct ptr_ring *r = &pool->ring; - struct page *page; int pref_nid; /* preferred NUMA node */ - /* Quicker fallback, avoid locks when ring is empty */ - if (__ptr_ring_empty(r)) { - alloc_stat_inc(pool, empty); - return NULL; - } + /* Always assume page is in pref node for mem providers */ + if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_ops) + return true; /* Softirq guarantee CPU and thus NUMA node is stable. This, * assumes CPU refilling driver RX-ring will also run RX-NAPI. @@ -280,13 +302,31 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool) pref_nid = numa_mem_id(); /* will be zero like page_to_nid() */ #endif + if (page_to_nid(page) == pref_nid) + return true; + + return false; +} + +noinline +static struct page *page_pool_refill_alloc_cache(struct page_pool *pool) +{ + struct ptr_ring *r = &pool->ring; + struct page *page; + + /* Quicker fallback, avoid locks when ring is empty */ + if (__ptr_ring_empty(r)) { + alloc_stat_inc(pool, empty); + return NULL; + } + /* Refill alloc array, but only if NUMA match */ do { page = __ptr_ring_consume(r); if (unlikely(!page)) break; - if (likely(page_to_nid(page) == pref_nid)) { + if (likely(page_pool_page_in_pref_node(pool, page))) { pool->alloc.cache[pool->alloc.count++] = page; } else { /* NUMA mismatch; @@ -490,7 +530,10 @@ struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp) return page; /* Slow-path: cache empty, do real allocation */ - page = __page_pool_alloc_pages_slow(pool, gfp); + if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_ops) + page = pool->mp_ops->alloc_pages(pool, gfp); + else + page = __page_pool_alloc_pages_slow(pool, gfp); return page; } EXPORT_SYMBOL(page_pool_alloc_pages); @@ -543,7 +586,10 @@ void page_pool_return_page(struct page_pool *pool, struct page *page) { int count; - __page_pool_release_page_dma(pool, page); + if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_ops) + pool->mp_ops->release_page(pool, page); + else + __page_pool_release_page_dma(pool, page); page_pool_clear_pp_info(page); @@ -553,7 +599,11 @@ void page_pool_return_page(struct page_pool *pool, struct page *page) count = atomic_inc_return_relaxed(&pool->pages_state_release_cnt); trace_page_pool_state_release(pool, page, count); - put_page(page); + if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_ops) + pool->mp_ops->free_pages(pool, page); + else + put_page(page); + /* An optimization would be to call __free_pages(page, pool->p.order) * knowing page is not part of page-cache (thus avoiding a * __page_cache_release() call). @@ -821,6 +871,11 @@ static void __page_pool_destroy(struct page_pool *pool) if (pool->disconnect) pool->disconnect(pool); + if (pool->mp_ops) { + pool->mp_ops->destroy(pool); + static_branch_dec(&page_pool_mem_providers); + } + ptr_ring_cleanup(&pool->ring, NULL); if (pool->p.flags & PP_FLAG_DMA_MAP) From patchwork Mon Nov 13 13:00:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13453967 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D32201DDE9 for ; Mon, 13 Nov 2023 13:00:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=none Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5BA91981; Mon, 13 Nov 2023 05:00:36 -0800 (PST) Received: from dggpemm500005.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4STTzy2H7vzWhL1; Mon, 13 Nov 2023 21:00:14 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by dggpemm500005.china.huawei.com (7.185.36.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 13 Nov 2023 21:00:34 +0800 From: Yunsheng Lin To: , , CC: , , Mina Almasry , Willem de Bruijn , Kaiyuan Zhang , Yunsheng Lin , Jesper Dangaard Brouer , Ilias Apalodimas , Eric Dumazet Subject: [PATCH RFC 3/8] memory-provider: dmabuf devmem memory provider Date: Mon, 13 Nov 2023 21:00:35 +0800 Message-ID: <20231113130041.58124-4-linyunsheng@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20231113130041.58124-1-linyunsheng@huawei.com> References: <20231113130041.58124-1-linyunsheng@huawei.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500005.china.huawei.com (7.185.36.74) X-CFilter-Loop: Reflected X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Mina Almasry Implement a memory provider that allocates dmabuf devmem page_pool_iovs. Support of PP_FLAG_DMA_MAP and PP_FLAG_DMA_SYNC_DEV is omitted for simplicity. The provider receives a reference to the struct netdev_dmabuf_binding via the pool->mp_priv pointer. The driver needs to set this pointer for the provider in the page_pool_params. The provider obtains a reference on the netdev_dmabuf_binding which guarantees the binding and the underlying mapping remains alive until the provider is destroyed. Signed-off-by: Willem de Bruijn Signed-off-by: Kaiyuan Zhang Signed-off-by: Mina Almasry Signed-off-by: Yunsheng Lin --- include/net/page_pool/types.h | 28 +++++++++++ net/core/page_pool.c | 93 +++++++++++++++++++++++++++++++++++ 2 files changed, 121 insertions(+) diff --git a/include/net/page_pool/types.h b/include/net/page_pool/types.h index 5e4fcd45ba50..52e4cf98ebc6 100644 --- a/include/net/page_pool/types.h +++ b/include/net/page_pool/types.h @@ -124,6 +124,7 @@ struct mem_provider; enum pp_memory_provider_type { __PP_MP_NONE, /* Use system allocator directly */ + PP_MP_DMABUF_DEVMEM, /* dmabuf devmem provider */ }; struct pp_memory_provider_ops { @@ -134,6 +135,33 @@ struct pp_memory_provider_ops { void (*free_pages)(struct page_pool *pool, struct page *page); }; +extern const struct pp_memory_provider_ops dmabuf_devmem_ops; + +struct page_pool_iov { + unsigned long res0; + unsigned long pp_magic; + struct page_pool *pp; + struct page *page; /* dmabuf memory provider specific field */ + unsigned long dma_addr; + atomic_long_t pp_frag_count; + unsigned int res1; + refcount_t _refcount; +}; + +#define PAGE_POOL_MATCH(pg, iov) \ + static_assert(offsetof(struct page, pg) == \ + offsetof(struct page_pool_iov, iov)) +PAGE_POOL_MATCH(flags, res0); +PAGE_POOL_MATCH(pp_magic, pp_magic); +PAGE_POOL_MATCH(pp, pp); +PAGE_POOL_MATCH(_pp_mapping_pad, page); +PAGE_POOL_MATCH(dma_addr, dma_addr); +PAGE_POOL_MATCH(pp_frag_count, pp_frag_count); +PAGE_POOL_MATCH(_mapcount, res1); +PAGE_POOL_MATCH(_refcount, _refcount); +#undef PAGE_POOL_MATCH +static_assert(sizeof(struct page_pool_iov) <= sizeof(struct page)); + struct page_pool { struct page_pool_params p; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 6c502bea842b..1bd7a2306f09 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -231,6 +231,9 @@ static int page_pool_init(struct page_pool *pool, switch (pool->p.memory_provider) { case __PP_MP_NONE: break; + case PP_MP_DMABUF_DEVMEM: + pool->mp_ops = &dmabuf_devmem_ops; + break; default: err = -EINVAL; goto free_ptr_ring; @@ -1010,3 +1013,93 @@ void page_pool_update_nid(struct page_pool *pool, int new_nid) } } EXPORT_SYMBOL(page_pool_update_nid); + +/*** "Dmabuf devmem memory provider" ***/ + +static int mp_dmabuf_devmem_init(struct page_pool *pool) +{ + if (pool->p.flags & PP_FLAG_DMA_MAP || + pool->p.flags & PP_FLAG_DMA_SYNC_DEV) + return -EOPNOTSUPP; + return 0; +} + +static struct page *mp_dmabuf_devmem_alloc_pages(struct page_pool *pool, + gfp_t gfp) +{ + struct page_pool_iov *ppiov; + struct page *page; + dma_addr_t dma; + + ppiov = kvmalloc(sizeof(*ppiov), gfp | __GFP_ZERO); + if (!ppiov) + return NULL; + + page = alloc_pages_node(pool->p.nid, gfp, pool->p.order); + if (!page) { + kvfree(ppiov); + return NULL; + } + + dma = dma_map_page_attrs(pool->p.dev, page, 0, + (PAGE_SIZE << pool->p.order), + pool->p.dma_dir, DMA_ATTR_SKIP_CPU_SYNC | + DMA_ATTR_WEAK_ORDERING); + if (dma_mapping_error(pool->p.dev, dma)) { + put_page(page); + kvfree(ppiov); + return NULL; + } + + ppiov->pp = pool; + ppiov->pp_magic = PP_SIGNATURE; + ppiov->page = page; + refcount_set(&ppiov->_refcount, 1); + page_pool_fragment_page((struct page *)ppiov, 1); + page_pool_set_dma_addr((struct page *)ppiov, dma); + pool->pages_state_hold_cnt++; + trace_page_pool_state_hold(pool, (struct page *)ppiov, + pool->pages_state_hold_cnt); + return (struct page *)ppiov; +} + +static void mp_dmabuf_devmem_destroy(struct page_pool *pool) +{ +} + +static void mp_dmabuf_devmem_release_page(struct page_pool *pool, + struct page *page) +{ + struct page_pool_iov *ppiov = (struct page_pool_iov *)page; + dma_addr_t dma; + + dma = page_pool_get_dma_addr(page); + + /* When page is unmapped, it cannot be returned to our pool */ + dma_unmap_page_attrs(pool->p.dev, dma, + PAGE_SIZE << pool->p.order, pool->p.dma_dir, + DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_WEAK_ORDERING); + page_pool_set_dma_addr(page, 0); + + put_page(ppiov->page); +} + +static void mp_dmabuf_devmem_free_pages(struct page_pool *pool, + struct page *page) +{ + int count; + + count = atomic_inc_return_relaxed(&pool->pages_state_release_cnt); + trace_page_pool_state_release(pool, page, count); + + kvfree(page); +} + +const struct pp_memory_provider_ops dmabuf_devmem_ops = { + .init = mp_dmabuf_devmem_init, + .destroy = mp_dmabuf_devmem_destroy, + .alloc_pages = mp_dmabuf_devmem_alloc_pages, + .release_page = mp_dmabuf_devmem_release_page, + .free_pages = mp_dmabuf_devmem_free_pages, +}; +EXPORT_SYMBOL(dmabuf_devmem_ops); From patchwork Mon Nov 13 13:00:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13453966 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D0A6F1DDE0 for ; Mon, 13 Nov 2023 13:00:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=none Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9DC701984; Mon, 13 Nov 2023 05:00:38 -0800 (PST) Received: from dggpemm500005.china.huawei.com (unknown [172.30.72.55]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4STTwZ5Mryz1P8HW; Mon, 13 Nov 2023 20:57:18 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by dggpemm500005.china.huawei.com (7.185.36.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 13 Nov 2023 21:00:35 +0800 From: Yunsheng Lin To: , , CC: , , Yunsheng Lin Subject: [PATCH RFC 4/8] skbuff: explicitize the semantics of skb_frag_fill_page_desc() Date: Mon, 13 Nov 2023 21:00:36 +0800 Message-ID: <20231113130041.58124-5-linyunsheng@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20231113130041.58124-1-linyunsheng@huawei.com> References: <20231113130041.58124-1-linyunsheng@huawei.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500005.china.huawei.com (7.185.36.74) X-CFilter-Loop: Reflected X-Patchwork-State: RFC As we have ensured that the page and the offset to the page for the skb frag is passed togetherly in one function call, see [1], make it explicit that skb_frag_fill_page_desc() expect a base page or head page for a compound page, so that we can avoid the compound_head() in the net stack. Log a warning if the caller passes a tail page of compoud page and adjust 'page' to point to it's head page and 'offset' to point to start of it's head page. The warning can be removed if we have ensured all the caller is passing non tail page to skb_frag_fill_page_desc() in the future. 1. https://lore.kernel.org/all/20230511011213.59091-2-linyunsheng@huawei.com/ Signed-off-by: Yunsheng Lin --- include/linux/skbuff.h | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 27998f73183e..3e2f806c8ed8 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -2423,6 +2423,14 @@ static inline void skb_frag_fill_page_desc(skb_frag_t *frag, struct page *page, int off, int size) { + /* expect head page for compound page */ + if (WARN_ON_ONCE(PageTail(page))) { + struct page *head = compound_head(page); + + off += (page_to_pfn(page) - page_to_pfn(head)) * PAGE_SIZE; + page = head; + } + frag->bv_page = page; frag->bv_offset = off; skb_frag_size_set(frag, size); From patchwork Mon Nov 13 13:00:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13453964 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9C4A1DA39 for ; Mon, 13 Nov 2023 13:00:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=none Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3726519A7; Mon, 13 Nov 2023 05:00:39 -0800 (PST) Received: from dggpemm500005.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4STTvY6l89zPnY6; Mon, 13 Nov 2023 20:56:25 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by dggpemm500005.china.huawei.com (7.185.36.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 13 Nov 2023 21:00:37 +0800 From: Yunsheng Lin To: , , CC: , , Yunsheng Lin , Eric Dumazet Subject: [PATCH RFC 5/8] skbuff: remove compound_head() related function calling Date: Mon, 13 Nov 2023 21:00:37 +0800 Message-ID: <20231113130041.58124-6-linyunsheng@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20231113130041.58124-1-linyunsheng@huawei.com> References: <20231113130041.58124-1-linyunsheng@huawei.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500005.china.huawei.com (7.185.36.74) X-CFilter-Loop: Reflected X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC As we have ensured that the page for the skb frag is always a base page or head page for a compound page, we can remove some compound_head() related function. Signed-off-by: Yunsheng Lin --- include/linux/skbuff.h | 8 +++++--- net/core/skbuff.c | 4 +--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 3e2f806c8ed8..1889b0968be0 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -2473,13 +2473,15 @@ static inline void skb_len_add(struct sk_buff *skb, int delta) static inline void __skb_fill_page_desc(struct sk_buff *skb, int i, struct page *page, int off, int size) { - __skb_fill_page_desc_noacc(skb_shinfo(skb), i, page, off, size); + skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; + + skb_frag_fill_page_desc(frag, page, off, size); /* Propagate page pfmemalloc to the skb if we can. The problem is * that not all callers have unique ownership of the page but rely * on page_is_pfmemalloc doing the right thing(tm). */ - page = compound_head(page); + page = frag->bv_page; if (page_is_pfmemalloc(page)) skb->pfmemalloc = true; } @@ -3429,7 +3431,7 @@ static inline struct page *skb_frag_page(const skb_frag_t *frag) */ static inline void __skb_frag_ref(skb_frag_t *frag) { - get_page(skb_frag_page(frag)); + page_ref_inc(skb_frag_page(frag)); } /** diff --git a/net/core/skbuff.c b/net/core/skbuff.c index b157efea5dea..ada3da4fe221 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -896,8 +896,6 @@ bool napi_pp_put_page(struct page *page, bool napi_safe) bool allow_direct = false; struct page_pool *pp; - page = compound_head(page); - /* page->pp_magic is OR'ed with PP_SIGNATURE after the allocation * in order to preserve any existing bits, such as bit 0 for the * head page of compound page and bit 1 for pfmemalloc page, so @@ -939,7 +937,7 @@ static bool skb_pp_recycle(struct sk_buff *skb, void *data, bool napi_safe) { if (!IS_ENABLED(CONFIG_PAGE_POOL) || !skb->pp_recycle) return false; - return napi_pp_put_page(virt_to_page(data), napi_safe); + return napi_pp_put_page(virt_to_head_page(data), napi_safe); } static void skb_kfree_head(void *head, unsigned int end_offset) From patchwork Mon Nov 13 13:00:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13453968 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D31B1F613 for ; Mon, 13 Nov 2023 13:00:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=none Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C6CD173E; Mon, 13 Nov 2023 05:00:41 -0800 (PST) Received: from dggpemm500005.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4STV040g0tzWhLB; Mon, 13 Nov 2023 21:00:20 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by dggpemm500005.china.huawei.com (7.185.36.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 13 Nov 2023 21:00:40 +0800 From: Yunsheng Lin To: , , CC: , , Yunsheng Lin , Ayush Sawal , Eric Dumazet , Jesper Dangaard Brouer , Ilias Apalodimas , Boris Pismenny , John Fastabend Subject: [PATCH RFC 6/8] skbuff: always try to do page pool frag reference counting Date: Mon, 13 Nov 2023 21:00:38 +0800 Message-ID: <20231113130041.58124-7-linyunsheng@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20231113130041.58124-1-linyunsheng@huawei.com> References: <20231113130041.58124-1-linyunsheng@huawei.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500005.china.huawei.com (7.185.36.74) X-CFilter-Loop: Reflected X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC As we have unified the frag_count handling in [1], we can do frag reference counting instead of native page reference counting whenever possible in net stack, in order to enable the best possibility of recycling as we assume other subsystem other than page pool is also hodling on to a page if page_ref_count(page) != 1, so we stop the recycling and release the page from page pool. As the page from the devmem, we reuse most of the fields in 'struct page' for devmem in order to have unified handling for both normal memory and devmem, but the meta data is not really tied to mm, so we can't really use page->_refcount as reference counting for devmem, instead it relies on page pool's frag reference counting. After this patch, pp_frag_count is not appropriate name as we don't ensure only one user holding on to a frag, we may rename it to frag_refcount in the future to avoid confusion. 1. https://lore.kernel.org/all/20231020095952.11055-1-linyunsheng@huawei.com/ Signed-off-by: Yunsheng Lin --- .../chelsio/inline_crypto/ch_ktls/chcr_ktls.c | 2 +- drivers/net/ethernet/sun/cassini.c | 4 ++-- drivers/net/veth.c | 2 +- include/linux/skbuff.h | 12 +++++++++--- include/net/page_pool/types.h | 13 +++++++++++++ net/core/skbuff.c | 6 +++--- net/tls/tls_device_fallback.c | 2 +- 7 files changed, 30 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c b/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c index 6482728794dd..87fda3e9d7d1 100644 --- a/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c +++ b/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c @@ -1658,7 +1658,7 @@ static void chcr_ktls_copy_record_in_skb(struct sk_buff *nskb, for (i = 0; i < record->num_frags; i++) { skb_shinfo(nskb)->frags[i] = record->frags[i]; /* increase the frag ref count */ - __skb_frag_ref(&skb_shinfo(nskb)->frags[i]); + skb_frag_ref(nskb, i); } skb_shinfo(nskb)->nr_frags = record->num_frags; diff --git a/drivers/net/ethernet/sun/cassini.c b/drivers/net/ethernet/sun/cassini.c index b317b9486455..8b818809c721 100644 --- a/drivers/net/ethernet/sun/cassini.c +++ b/drivers/net/ethernet/sun/cassini.c @@ -1999,7 +1999,7 @@ static int cas_rx_process_pkt(struct cas *cp, struct cas_rx_comp *rxc, skb->len += hlen - swivel; skb_frag_fill_page_desc(frag, page->buffer, off, hlen - swivel); - __skb_frag_ref(frag); + __skb_frag_ref(frag, skb->pp_recycle); /* any more data? */ if ((words[0] & RX_COMP1_SPLIT_PKT) && ((dlen -= hlen) > 0)) { @@ -2023,7 +2023,7 @@ static int cas_rx_process_pkt(struct cas *cp, struct cas_rx_comp *rxc, frag++; skb_frag_fill_page_desc(frag, page->buffer, 0, hlen); - __skb_frag_ref(frag); + __skb_frag_ref(frag, skb->pp_recycle); RX_USED_ADD(page, hlen + cp->crc_size); } diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 9980517ed8b0..8ee323a1184b 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -724,7 +724,7 @@ static void veth_xdp_get(struct xdp_buff *xdp) return; for (i = 0; i < sinfo->nr_frags; i++) - __skb_frag_ref(&sinfo->frags[i]); + __skb_frag_ref(&sinfo->frags[i], false); } static int veth_convert_skb_to_xdp_buff(struct veth_rq *rq, diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 1889b0968be0..b77043a2b446 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -37,6 +37,7 @@ #endif #include #include +#include /** * DOC: skb checksums @@ -3429,9 +3430,14 @@ static inline struct page *skb_frag_page(const skb_frag_t *frag) * * Takes an additional reference on the paged fragment @frag. */ -static inline void __skb_frag_ref(skb_frag_t *frag) +static inline void __skb_frag_ref(skb_frag_t *frag, bool recycle) { - page_ref_inc(skb_frag_page(frag)); + struct page *page = skb_frag_page(frag); + + if (page_pool_frag_ref(page, recycle)) + return; + + page_ref_inc(page); } /** @@ -3443,7 +3449,7 @@ static inline void __skb_frag_ref(skb_frag_t *frag) */ static inline void skb_frag_ref(struct sk_buff *skb, int f) { - __skb_frag_ref(&skb_shinfo(skb)->frags[f]); + __skb_frag_ref(&skb_shinfo(skb)->frags[f], skb->pp_recycle); } bool napi_pp_put_page(struct page *page, bool napi_safe); diff --git a/include/net/page_pool/types.h b/include/net/page_pool/types.h index 52e4cf98ebc6..ea4e654168bd 100644 --- a/include/net/page_pool/types.h +++ b/include/net/page_pool/types.h @@ -275,6 +275,19 @@ static inline bool is_page_pool_compiled_in(void) #endif } +static inline bool page_pool_frag_ref(struct page *page, bool recycle) +{ + if (recycle && (page->pp_magic & ~0x3UL) == PP_SIGNATURE) { + atomic_long_inc(&page->pp_frag_count); + return true; + } + + BUG_ON((page->pp_magic & ~0x3UL) == PP_SIGNATURE && + page->pp->mp_ops); + + return false; +} + /* Caller must provide appropriate safe context, e.g. NAPI. */ void page_pool_update_nid(struct page_pool *pool, int new_nid); diff --git a/net/core/skbuff.c b/net/core/skbuff.c index ada3da4fe221..a33df6e49694 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -4046,7 +4046,7 @@ int skb_shift(struct sk_buff *tgt, struct sk_buff *skb, int shiftlen) to++; } else { - __skb_frag_ref(fragfrom); + __skb_frag_ref(fragfrom, skb->pp_recycle); skb_frag_page_copy(fragto, fragfrom); skb_frag_off_copy(fragto, fragfrom); skb_frag_size_set(fragto, todo); @@ -4695,7 +4695,7 @@ struct sk_buff *skb_segment(struct sk_buff *head_skb, } *nskb_frag = (i < 0) ? skb_head_frag_to_page_desc(frag_skb) : *frag; - __skb_frag_ref(nskb_frag); + __skb_frag_ref(nskb_frag, nskb->pp_recycle); size = skb_frag_size(nskb_frag); if (pos < offset) { @@ -5830,7 +5830,7 @@ bool skb_try_coalesce(struct sk_buff *to, struct sk_buff *from, * since we set nr_frags to 0. */ for (i = 0; i < from_shinfo->nr_frags; i++) - __skb_frag_ref(&from_shinfo->frags[i]); + __skb_frag_ref(&from_shinfo->frags[i], from->pp_recycle); to->truesize += delta; to->len += len; diff --git a/net/tls/tls_device_fallback.c b/net/tls/tls_device_fallback.c index 4e7228f275fa..d4000b4a1f7d 100644 --- a/net/tls/tls_device_fallback.c +++ b/net/tls/tls_device_fallback.c @@ -277,7 +277,7 @@ static int fill_sg_in(struct scatterlist *sg_in, for (i = 0; remaining > 0; i++) { skb_frag_t *frag = &record->frags[i]; - __skb_frag_ref(frag); + __skb_frag_ref(frag, false); sg_set_page(sg_in + i, skb_frag_page(frag), skb_frag_size(frag), skb_frag_off(frag)); From patchwork Mon Nov 13 13:00:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13453969 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13D831DA53 for ; Mon, 13 Nov 2023 13:00:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=none Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C624C172D; Mon, 13 Nov 2023 05:00:44 -0800 (PST) Received: from dggpemm500005.china.huawei.com (unknown [172.30.72.53]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4STTvD12YMzMmpm; Mon, 13 Nov 2023 20:56:08 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by dggpemm500005.china.huawei.com (7.185.36.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 13 Nov 2023 21:00:41 +0800 From: Yunsheng Lin To: , , CC: , , Yunsheng Lin , Yisen Zhuang , Salil Mehta , Eric Dumazet Subject: [PATCH RFC 7/8] net: hns3: temp hack for hns3 to use dmabuf memory provider Date: Mon, 13 Nov 2023 21:00:39 +0800 Message-ID: <20231113130041.58124-8-linyunsheng@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20231113130041.58124-1-linyunsheng@huawei.com> References: <20231113130041.58124-1-linyunsheng@huawei.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500005.china.huawei.com (7.185.36.74) X-CFilter-Loop: Reflected X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Signed-off-by: Yunsheng Lin --- drivers/net/ethernet/hisilicon/hns3/hns3_enet.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c index 06117502001f..43454aa6a279 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c @@ -3371,6 +3371,8 @@ static int hns3_alloc_buffer(struct hns3_enet_ring *ring, struct page *p; if (ring->page_pool) { + struct page_pool_iov *ppiov; + p = page_pool_dev_alloc_frag(ring->page_pool, &cb->page_offset, hns3_buf_size(ring)); @@ -3378,7 +3380,8 @@ static int hns3_alloc_buffer(struct hns3_enet_ring *ring, return -ENOMEM; cb->priv = p; - cb->buf = page_address(p); + ppiov = cb->priv; + cb->buf = page_address(ppiov->page); cb->dma = page_pool_get_dma_addr(p); cb->type = DESC_TYPE_PP_FRAG; cb->reuse_flag = 0; @@ -4940,7 +4943,7 @@ static void hns3_put_ring_config(struct hns3_nic_priv *priv) static void hns3_alloc_page_pool(struct hns3_enet_ring *ring) { struct page_pool_params pp_params = { - .flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV, + .flags = 0,//PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV, .order = hns3_page_order(ring), .pool_size = ring->desc_num * hns3_buf_size(ring) / (PAGE_SIZE << hns3_page_order(ring)), @@ -4949,6 +4952,8 @@ static void hns3_alloc_page_pool(struct hns3_enet_ring *ring) .dma_dir = DMA_FROM_DEVICE, .offset = 0, .max_len = PAGE_SIZE << hns3_page_order(ring), + .memory_provider = PP_MP_DMABUF_DEVMEM, + .mp_priv = &dmabuf_devmem_ops, }; ring->page_pool = page_pool_create(&pp_params); From patchwork Mon Nov 13 13:00:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13453970 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D14C1DA53 for ; Mon, 13 Nov 2023 13:00:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=none Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC7FF1727; Mon, 13 Nov 2023 05:00:51 -0800 (PST) Received: from dggpemm500005.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4STV0G0LFJzWhLY; Mon, 13 Nov 2023 21:00:30 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by dggpemm500005.china.huawei.com (7.185.36.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 13 Nov 2023 21:00:50 +0800 From: Yunsheng Lin To: , , CC: , , Yunsheng Lin , Eric Dumazet Subject: [PATCH RFC 8/8] net: temp hack for dmabuf page in __skb_datagram_iter() Date: Mon, 13 Nov 2023 21:00:40 +0800 Message-ID: <20231113130041.58124-9-linyunsheng@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20231113130041.58124-1-linyunsheng@huawei.com> References: <20231113130041.58124-1-linyunsheng@huawei.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500005.china.huawei.com (7.185.36.74) X-CFilter-Loop: Reflected X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Signed-off-by: Yunsheng Lin --- net/core/datagram.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/net/core/datagram.c b/net/core/datagram.c index 103d46fa0eeb..5556782ac658 100644 --- a/net/core/datagram.c +++ b/net/core/datagram.c @@ -436,7 +436,15 @@ static int __skb_datagram_iter(const struct sk_buff *skb, int offset, end = start + skb_frag_size(frag); if ((copy = end - offset) > 0) { struct page *page = skb_frag_page(frag); - u8 *vaddr = kmap(page); + u8 *vaddr; + + if ((page->pp_magic & ~0x3UL) == PP_SIGNATURE) { + struct page_pool_iov *ppiov = (struct page_pool_iov *)page; + + page = ppiov->page; + } + + vaddr = kmap(page); if (copy > len) copy = len;