[net-next,v2,00/10] page_pool: Add page_pool stat counters

Message

Joe Damato Jan. 29, 2022, 11:38 p.m. UTC

Greetings:

This series adds some stat counters for the page_pool allocation path which
help to track:

	- fast path allocations
	- slow path order-0 allocations
	- slow path high order allocations
	- refills which failed due to an empty ptr ring, forcing a slow
	  path allocation
	- allocations fulfilled via successful refill
	- pages which cannot be added to the cache because of numa mismatch
	  (i.e. waived)

This v2 series includes some major changes from the original, namely:

	1. A new kernel config option has been added, which defaults to N,
	   preventing this code from being compiled in by default
	2. The stats structure has been converted to a per-cpu structure
	3. The stats are now exported via proc (/proc/net/page_pool_stat)

The main advantage of the v2 over the original approach is that no
modifications to drivers are required and no new external APIs are
introduced.

I benchmarked the code with the stats enabled and again with them disabled
using the netoptimizer/prototype-kernel benchmark programs [1], as Jesper
suggested.

Test system:
	- 2x Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
	- 2 NUMA zones, with 18 cores per zone and 2 threads per core

bench_page_pool_simple results:
test name			stats enabled		stats disabled
				cycles	nanosec		cycles	nanosec

for_loop			0	0.337		0	0.334
atomic_inc 			13	6.021		13	6.022
lock				31	13.545		31	13.846

no-softirq-page_pool01		45	20.040		44	19.388
no-softirq-page_pool02		46	20.295		43	19.073
no-softirq-page_pool03		110	48.053		122	53.405

tasklet_page_pool01_fast_path	14	6.126		12	5.640
tasklet_page_pool02_ptr_ring	42	18.334		40	17.695
tasklet_page_pool03_slow	109	47.854		108	47.355

bench_page_pool_cross_cpu results:
test name			stats enabled		stats disabled
				cycles	nanosec		cycles	nanosec

page_pool_cross_cpu CPU(0)	2246	979.007		2136	931.075
page_pool_cross_cpu CPU(1)	2240	976.289		2145	934.924
page_pool_cross_cpu CPU(2)	1123	489.511		1072	467.474

page_pool_cross_cpu average	1886	-		1784	-

Thanks.

[1]:
https://github.com/netoptimizer/prototype-kernel/tree/master/kernel/lib

Joe Damato (10):
  page_pool: kconfig: Add flag for page pool stats
  page_pool: Add per-cpu page_pool_stats struct
  page_pool: Add a macro for incrementing stats
  page_pool: Add stat tracking fast path allocations
  page_pool: Add slow path order 0 allocation stat
  page_pool: Add slow path high order allocation stat
  page_pool: Add stat tracking empty ring
  page_pool: Add stat tracking cache refill
  page_pool: Add a stat tracking waived pages
  net-procfs: Show page pool stats in proc

 include/net/page_pool.h | 20 ++++++++++++++
 net/Kconfig             | 12 +++++++++
 net/core/net-procfs.c   | 69 ++++++++++++++++++++++++++++++++++++++++++++++++-
 net/core/page_pool.c    | 28 +++++++++++++++++---
 4 files changed, 125 insertions(+), 4 deletions(-)