[net-next,0/2] page_pool: allow direct bulk recycling

Message

Alexander Lobakin March 29, 2024, 4:55 p.m. UTC

Previously, there was no reliable way to check whether it's safe to use
direct PP cache. The drivers were passing @allow_direct to the PP
recycling functions and that was it. Bulk recycling is used by
xdp_return_frame_bulk() on .ndo_xdp_xmit() frames completion where
the page origin is unknown, thus the direct recycling has never been
tried.
Now that we have at least 2 ways of checking if we're allowed to perform
direct recycling -- pool->p.napi (Jakub) and pool->cpuid (Lorenzo), we
can use them when doing bulk recycling as well. Just move that logic
from the skb core to the PP core and call it before
__page_pool_put_page() every time @allow_direct is false.
Under high .ndo_xdp_xmit() traffic load, the win is 2-3% Pps assuming
the sending driver uses xdp_return_frame_bulk() on Tx completion.

Alexander Lobakin (2):
  page_pool: check for PP direct cache locality later
  page_pool: try direct bulk recycling

 include/linux/skbuff.h | 12 ++++----
 net/core/page_pool.c   | 38 ++++++++++++++++++++---
 net/core/skbuff.c      | 70 +++++++++++++-----------------------------
 net/ipv4/esp4.c        |  2 +-
 net/ipv6/esp6.c        |  2 +-
 5 files changed, 63 insertions(+), 61 deletions(-)

Comments

[e-mail] April 3, 2024, 1:40 a.m. UTC | #1

Hello:

This series was applied to netdev/net-next.git (main)
by Jakub Kicinski <[e-mail]>:

On Fri, 29 Mar 2024 17:55:05 +0100 you wrote:
> Previously, there was no reliable way to check whether it's safe to use
> direct PP cache. The drivers were passing @allow_direct to the PP
> recycling functions and that was it. Bulk recycling is used by
> xdp_return_frame_bulk() on .ndo_xdp_xmit() frames completion where
> the page origin is unknown, thus the direct recycling has never been
> tried.
> Now that we have at least 2 ways of checking if we're allowed to perform
> direct recycling -- pool->p.napi (Jakub) and pool->cpuid (Lorenzo), we
> can use them when doing bulk recycling as well. Just move that logic
> from the skb core to the PP core and call it before
> __page_pool_put_page() every time @allow_direct is false.
> Under high .ndo_xdp_xmit() traffic load, the win is 2-3% Pps assuming
> the sending driver uses xdp_return_frame_bulk() on Tx completion.
> 
> [...]

Here is the summary with links:
  - [net-next,1/2] page_pool: check for PP direct cache locality later
    https://git.kernel.org/netdev/net-next/c/4a96a4e807c3
  - [net-next,2/2] page_pool: try direct bulk recycling
    https://git.kernel.org/netdev/net-next/c/39806b96c89a

You are awesome, thank you!