[RFC,v2,net-next,0/2] Use per-task storage for XDP-redirects on PREEMPT_RT

Message

Sebastian Andrzej Siewior Feb. 29, 2024, 6:17 p.m. UTC

In [0] I introduced explicit locking for resources which are otherwise
locked implicit locked by local_bh_disable() and this protections goes
away if the lock in local_bh_disable() is removed on PREEMPT_RT.

There was a big complained once it came to the locking of XDP related
resources during XDP-redirect because it was mostly per-packet and the
locking, even not contended, was seen as a problem [1]. Another XDP
related problem was that I also touched every NIC-driver using XDP. This
complicated the "how to write a NIC driver" further.

To solve both problems I was thinking about an alternative and ended up
with moving the data structure from per-CPU to per-task on PREEMPT_RT.
Instead of adding it to task_struct, I added a pointer there and setup
the struct on stack. In my debug build on x86-64 the struct
bpf_xdp_storage has 112 bytes and collected the per-CPU ressources.

I've been benchmark the outcome on !RT. I meassured 14,880,077 pkt/s
with pktgen + "xdp-bench drop" on a ixgbe which is the interface limit.
I then lowered the clockspeed of the CPU (on the RX side) and received
approx between 12,156,279 and 12,476,759 pkt/s. The results were not
consistent and varied between runs mostly in that range with and without
the patches.

v1…v2 https://lore.kernel.org/all/[e-mail]
  - Renamed the container struct from xdp_storage to bpf_net_context. Suggested
    by Toke Høiland-Jørgensen.
  - Use the container struct also on !PREEMPT_RT builds. Store the pointer to
    the on-stack struct in a per-CPU variable. Suggested by Toke
    Høiland-Jørgensen.

I'm posting here only two patches which replace the XDP redirect part
(patches 14 to 24) from the original series.

[0] https://lore.kernel.org/all/[e-mail]/
[1] https://lore.kernel.org/all/[e-mail]/
[2] https://lore.kernel.org/all/[e-mail]

Sebastian