| Message ID | 20240205210453.11301-3-jdamato@fastly.com (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | Per epoll context busy poll support | expand |
On Mon, 5 Feb 2024 21:04:47 +0000 Joe Damato wrote: > When using epoll-based busy poll, the packet budget is hardcoded to > BUSY_POLL_BUDGET (8). Users may desire larger busy poll budgets, which > can potentially increase throughput when busy polling under high network > load. > > Other busy poll methods allow setting the busy poll budget via > SO_BUSY_POLL_BUDGET, but epoll-based busy polling uses a hardcoded > value. > > Fix this edge case by adding support for a per-epoll context busy poll > packet budget. If not specified, the default value (BUSY_POLL_BUDGET) is > used. Reviewed-by: Jakub Kicinski <kuba@kernel.org>
On Wed, Feb 7, 2024 at 8:04 PM Jakub Kicinski <kuba@kernel.org> wrote: > > On Mon, 5 Feb 2024 21:04:47 +0000 Joe Damato wrote: > > When using epoll-based busy poll, the packet budget is hardcoded to > > BUSY_POLL_BUDGET (8). Users may desire larger busy poll budgets, which > > can potentially increase throughput when busy polling under high network > > load. > > > > Other busy poll methods allow setting the busy poll budget via > > SO_BUSY_POLL_BUDGET, but epoll-based busy polling uses a hardcoded > > value. > > > > Fix this edge case by adding support for a per-epoll context busy poll > > packet budget. If not specified, the default value (BUSY_POLL_BUDGET) is > > used. > > Reviewed-by: Jakub Kicinski <kuba@kernel.org> Reviewed-by: Eric Dumazet <edumazet@google.com>
diff --git a/fs/eventpoll.c b/fs/eventpoll.c index ce75189d46df..3985434df527 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -229,6 +229,8 @@ struct eventpoll { unsigned int napi_id; /* busy poll timeout */ u64 busy_poll_usecs; + /* busy poll packet budget */ + u16 busy_poll_budget; #endif #ifdef CONFIG_DEBUG_LOCK_ALLOC @@ -437,10 +439,14 @@ static bool ep_busy_loop_end(void *p, unsigned long start_time) static bool ep_busy_loop(struct eventpoll *ep, int nonblock) { unsigned int napi_id = READ_ONCE(ep->napi_id); + u16 budget = READ_ONCE(ep->busy_poll_budget); + + if (!budget) + budget = BUSY_POLL_BUDGET; if ((napi_id >= MIN_NAPI_ID) && ep_busy_loop_on(ep)) { napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, false, - BUSY_POLL_BUDGET); + budget); if (ep_events_available(ep)) return true; /* @@ -2098,6 +2104,7 @@ static int do_epoll_create(int flags) } #ifdef CONFIG_NET_RX_BUSY_POLL ep->busy_poll_usecs = 0; + ep->busy_poll_budget = 0; #endif ep->file = file; fd_install(fd, file);
When using epoll-based busy poll, the packet budget is hardcoded to BUSY_POLL_BUDGET (8). Users may desire larger busy poll budgets, which can potentially increase throughput when busy polling under high network load. Other busy poll methods allow setting the busy poll budget via SO_BUSY_POLL_BUDGET, but epoll-based busy polling uses a hardcoded value. Fix this edge case by adding support for a per-epoll context busy poll packet budget. If not specified, the default value (BUSY_POLL_BUDGET) is used. Signed-off-by: Joe Damato <jdamato@fastly.com> --- fs/eventpoll.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)