From patchwork Wed Oct 28 13:34:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11864077 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AE6AC56202 for ; Wed, 28 Oct 2020 23:26:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B2491207BC for ; Wed, 28 Oct 2020 23:26:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Un01M6pT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390358AbgJ1XZ4 (ORCPT ); Wed, 28 Oct 2020 19:25:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387571AbgJ1XZy (ORCPT ); Wed, 28 Oct 2020 19:25:54 -0400 Received: from mail-io1-xd43.google.com (mail-io1-xd43.google.com [IPv6:2607:f8b0:4864:20::d43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53B53C0613CF; Wed, 28 Oct 2020 16:25:54 -0700 (PDT) Received: by mail-io1-xd43.google.com with SMTP id q25so1388569ioh.4; Wed, 28 Oct 2020 16:25:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EldFY16BxtvGb7fdIx/6iAPYxXdUeuIFxuwoe4Wvm20=; b=Un01M6pTcPg2IQ37QqHcx4HvGjenSIcRcZf/MDO0H3x1cUi1d9LRajo2owOwlRSI2N ItnuwaqNPqFNkHa7CxUCHpcYEmNPWDAKKuYV+smsBYW6HgoenTxT4Qe6KrGKrYJ7njxm q7rrcjthGmokAlsUWE2Qm6KMVs3vbt9HwaN89Lv4SyqK+zKDlLHNr235pcUKKKGvtTXz 3KsORgfCo7G/iisyJjrNVGfK1qiwBER0ahyx76HOJUs0xK7FAmm+TkDr2t7PQhWvEw7M ImWPFkwqa8SnVqGHr04KUrtHv0TwWf+QxQgX5OtHVwDTLLXYgrTw1mLcQDwsZWg4EVFl uRxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EldFY16BxtvGb7fdIx/6iAPYxXdUeuIFxuwoe4Wvm20=; b=lT0Z+80bShyb1+vEjVUNNX3KnrKFz2i6uOu70XuxBlrz/oB2jPWaDljd9DWKwhVflm dLqhoGhujVi0mzjxiVyhVjXH2C9KQMtdA+ESAvbTs4+qm+hQsShJ0NxI6Icp5OByNG16 miWsCrgruBsl5hB8QXgMurPNtcWfzh0EPy/wtbMphPASO2kUTqkTcItFstHbcT81IL8H Hufs0z5PGsXttrSWVyOl7E0XGpfTtLaEmspphiyK6ZrkgND/0ZeBk7vbn7VrMgVRZpgs hq+41jQIALLWhbXtFenKz0Rre3VkW1peaxxPTB39fRanI8wPuLTXIl9z5eyTPADny4M9 spkA== X-Gm-Message-State: AOAM530g7BRt3okXWjKXeP22ItSuJp+VUD/Fcm8r1f0tyH7hKwJcRoAx yk3usPlcILLBG6XXA2UJZjxE3p4QXpyBz3pY X-Google-Smtp-Source: ABdhPJy+S90MJjSHPYTZmqtsGVHrfsyandTfzipF3FoHF4WnOPbv3h+YZpedav1EFBmxNIjyaP6ayQ== X-Received: by 2002:a63:530c:: with SMTP id h12mr6684388pgb.424.1603892109532; Wed, 28 Oct 2020 06:35:09 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q14sm5935393pjp.43.2020.10.28.06.35.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2020 06:35:08 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, ast@kernel.org, daniel@iogearbox.net, maciej.fijalkowski@intel.com, sridhar.samudrala@intel.com, jesse.brandeburg@intel.com, qi.z.zhang@intel.com, kuba@kernel.org, edumazet@google.com, intel-wired-lan@lists.osuosl.org, jonathan.lemon@gmail.com Subject: [RFC PATCH bpf-next 1/9] net: introduce biased busy-polling Date: Wed, 28 Oct 2020 14:34:29 +0100 Message-Id: <20201028133437.212503-2-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028133437.212503-1-bjorn.topel@gmail.com> References: <20201028133437.212503-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Björn Töpel This change adds a new NAPI mode, called biased busy-polling, which is an extension to the existing busy-polling mode. The new mode is enabled on the socket layer, where a socket setting this option "promisies" to busy-poll the NAPI context via a system call. When this mode is enabled, the NAPI context will operate in a mode with interrupts disabled. The kernel monitors that the busy-polling promise is fulfilled by an internal watchdog. If the socket fail/stop performing the busy-polling, the mode will be disabled. The watchdog is currently 200 ms. Biased busy-polling follows the same mechanism as the existing busy-poll; The napi_id is reported to the socket via the skbuff. Later commits will extend napi_id reporting to XDP, in order to work correctly with XDP sockets. Let us walk through a flow of execution: 1. A socket sets the new SO_BIAS_BUSY_POLL socket option to true. The socket now shows an intent of doing busy-polling. No data has been received to the socket, so the napi_id of the socket is still 0 (non-valid). As usual for busy-polling, the SO_BUSY_POLL option also has to be non-zero for biased busy-polling. 2. Data is received on the socket changing the napi_id to non-zero. 3. The socket does a system call that has the busy-polling logic wired up, e.g. recvfrom() for UDP sockets. The NAPI context is now marked as biased busy-poll. The kernel watchdog is armed. If the NAPI context is already running, it will try to finish as soon as possible and move to busy-polling. If the NAPI context is not running, it will execute the NAPI poll function for the corresponding napi_id. 4. Goto 3, or wait until the watchdog timeout. Given the nature of busy-polling, this mode only make sense for non-blocking sockets. When the NAPI context is in biased busy-polling mode, it will not allow a NAPI to be scheduled using the napi_schedule_prep()/napi_scheduleXXX() combo. Signed-off-by: Björn Töpel --- arch/alpha/include/uapi/asm/socket.h | 2 + arch/mips/include/uapi/asm/socket.h | 2 + arch/parisc/include/uapi/asm/socket.h | 2 + arch/sparc/include/uapi/asm/socket.h | 2 + include/linux/netdevice.h | 33 +++++----- include/net/busy_poll.h | 17 ++++- include/net/sock.h | 3 + include/uapi/asm-generic/socket.h | 2 + net/core/dev.c | 89 +++++++++++++++++++++++++-- net/core/sock.c | 9 +++ 10 files changed, 140 insertions(+), 21 deletions(-) diff --git a/arch/alpha/include/uapi/asm/socket.h b/arch/alpha/include/uapi/asm/socket.h index de6c4df61082..0f776668fb09 100644 --- a/arch/alpha/include/uapi/asm/socket.h +++ b/arch/alpha/include/uapi/asm/socket.h @@ -124,6 +124,8 @@ #define SO_DETACH_REUSEPORT_BPF 68 +#define SO_BIAS_BUSY_POLL 69 + #if !defined(__KERNEL__) #if __BITS_PER_LONG == 64 diff --git a/arch/mips/include/uapi/asm/socket.h b/arch/mips/include/uapi/asm/socket.h index d0a9ed2ca2d6..d23984731504 100644 --- a/arch/mips/include/uapi/asm/socket.h +++ b/arch/mips/include/uapi/asm/socket.h @@ -135,6 +135,8 @@ #define SO_DETACH_REUSEPORT_BPF 68 +#define SO_BIAS_BUSY_POLL 69 + #if !defined(__KERNEL__) #if __BITS_PER_LONG == 64 diff --git a/arch/parisc/include/uapi/asm/socket.h b/arch/parisc/include/uapi/asm/socket.h index 10173c32195e..49469713ed2a 100644 --- a/arch/parisc/include/uapi/asm/socket.h +++ b/arch/parisc/include/uapi/asm/socket.h @@ -116,6 +116,8 @@ #define SO_DETACH_REUSEPORT_BPF 0x4042 +#define SO_BIAS_BUSY_POLL 0x4043 + #if !defined(__KERNEL__) #if __BITS_PER_LONG == 64 diff --git a/arch/sparc/include/uapi/asm/socket.h b/arch/sparc/include/uapi/asm/socket.h index 8029b681fc7c..009aba6f7a54 100644 --- a/arch/sparc/include/uapi/asm/socket.h +++ b/arch/sparc/include/uapi/asm/socket.h @@ -117,6 +117,8 @@ #define SO_DETACH_REUSEPORT_BPF 0x0047 +#define SO_BIAS_BUSY_POLL 0x0048 + #if !defined(__KERNEL__) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 964b494b0e8d..9bdc84d3d6b8 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -344,29 +344,32 @@ struct napi_struct { struct list_head rx_list; /* Pending GRO_NORMAL skbs */ int rx_count; /* length of rx_list */ struct hrtimer timer; + struct hrtimer bp_watchdog; struct list_head dev_list; struct hlist_node napi_hash_node; unsigned int napi_id; }; enum { - NAPI_STATE_SCHED, /* Poll is scheduled */ - NAPI_STATE_MISSED, /* reschedule a napi */ - NAPI_STATE_DISABLE, /* Disable pending */ - NAPI_STATE_NPSVC, /* Netpoll - don't dequeue from poll_list */ - NAPI_STATE_LISTED, /* NAPI added to system lists */ - NAPI_STATE_NO_BUSY_POLL,/* Do not add in napi_hash, no busy polling */ - NAPI_STATE_IN_BUSY_POLL,/* sk_busy_loop() owns this NAPI */ + NAPI_STATE_SCHED, /* Poll is scheduled */ + NAPI_STATE_MISSED, /* reschedule a napi */ + NAPI_STATE_DISABLE, /* Disable pending */ + NAPI_STATE_NPSVC, /* Netpoll - don't dequeue from poll_list */ + NAPI_STATE_LISTED, /* NAPI added to system lists */ + NAPI_STATE_NO_BUSY_POLL, /* Do not add in napi_hash, no busy polling */ + NAPI_STATE_IN_BUSY_POLL, /* sk_busy_loop() owns this NAPI */ + NAPI_STATE_BIAS_BUSY_POLL, /* biased busy-polling */ }; enum { - NAPIF_STATE_SCHED = BIT(NAPI_STATE_SCHED), - NAPIF_STATE_MISSED = BIT(NAPI_STATE_MISSED), - NAPIF_STATE_DISABLE = BIT(NAPI_STATE_DISABLE), - NAPIF_STATE_NPSVC = BIT(NAPI_STATE_NPSVC), - NAPIF_STATE_LISTED = BIT(NAPI_STATE_LISTED), - NAPIF_STATE_NO_BUSY_POLL = BIT(NAPI_STATE_NO_BUSY_POLL), - NAPIF_STATE_IN_BUSY_POLL = BIT(NAPI_STATE_IN_BUSY_POLL), + NAPIF_STATE_SCHED = BIT(NAPI_STATE_SCHED), + NAPIF_STATE_MISSED = BIT(NAPI_STATE_MISSED), + NAPIF_STATE_DISABLE = BIT(NAPI_STATE_DISABLE), + NAPIF_STATE_NPSVC = BIT(NAPI_STATE_NPSVC), + NAPIF_STATE_LISTED = BIT(NAPI_STATE_LISTED), + NAPIF_STATE_NO_BUSY_POLL = BIT(NAPI_STATE_NO_BUSY_POLL), + NAPIF_STATE_IN_BUSY_POLL = BIT(NAPI_STATE_IN_BUSY_POLL), + NAPIF_STATE_BIAS_BUSY_POLL = BIT(NAPI_STATE_BIAS_BUSY_POLL), }; enum gro_result { @@ -555,6 +558,8 @@ static inline bool napi_if_scheduled_mark_missed(struct napi_struct *n) return true; } +void napi_bias_busy_poll(unsigned int napi_id); + enum netdev_queue_state_t { __QUEUE_STATE_DRV_XOFF, __QUEUE_STATE_STACK_XOFF, diff --git a/include/net/busy_poll.h b/include/net/busy_poll.h index b001fa91c14e..9738923ed17b 100644 --- a/include/net/busy_poll.h +++ b/include/net/busy_poll.h @@ -23,6 +23,9 @@ */ #define MIN_NAPI_ID ((unsigned int)(NR_CPUS + 1)) +/* Biased busy-poll watchdog timeout in ms */ +#define BIASED_BUSY_POLL_TIMEOUT 200 + #ifdef CONFIG_NET_RX_BUSY_POLL struct napi_struct; @@ -99,13 +102,25 @@ static inline bool sk_busy_loop_timeout(struct sock *sk, return true; } +#ifdef CONFIG_NET_RX_BUSY_POLL +static inline void __sk_bias_busy_poll(struct sock *sk, unsigned int napi_id) +{ + if (likely(!READ_ONCE(sk->sk_bias_busy_poll))) + return; + + napi_bias_busy_poll(napi_id); +} +#endif + static inline void sk_busy_loop(struct sock *sk, int nonblock) { #ifdef CONFIG_NET_RX_BUSY_POLL unsigned int napi_id = READ_ONCE(sk->sk_napi_id); - if (napi_id >= MIN_NAPI_ID) + if (napi_id >= MIN_NAPI_ID) { + __sk_bias_busy_poll(sk, napi_id); napi_busy_loop(napi_id, nonblock ? NULL : sk_busy_loop_end, sk); + } #endif } diff --git a/include/net/sock.h b/include/net/sock.h index a5c6ae78df77..cf71834fb601 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -479,6 +479,9 @@ struct sock { u32 sk_ack_backlog; u32 sk_max_ack_backlog; kuid_t sk_uid; +#ifdef CONFIG_NET_RX_BUSY_POLL + u8 sk_bias_busy_poll; +#endif struct pid *sk_peer_pid; const struct cred *sk_peer_cred; long sk_rcvtimeo; diff --git a/include/uapi/asm-generic/socket.h b/include/uapi/asm-generic/socket.h index 77f7c1638eb1..8a2b37ccd9d5 100644 --- a/include/uapi/asm-generic/socket.h +++ b/include/uapi/asm-generic/socket.h @@ -119,6 +119,8 @@ #define SO_DETACH_REUSEPORT_BPF 68 +#define SO_BIAS_BUSY_POLL 69 + #if !defined(__KERNEL__) #if __BITS_PER_LONG == 64 || (defined(__x86_64__) && defined(__ILP32__)) diff --git a/net/core/dev.c b/net/core/dev.c index 9499a414d67e..a29e4c4a35f6 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6378,6 +6378,9 @@ bool napi_schedule_prep(struct napi_struct *n) val = READ_ONCE(n->state); if (unlikely(val & NAPIF_STATE_DISABLE)) return false; + if (unlikely(val & NAPIF_STATE_BIAS_BUSY_POLL)) + return false; + new = val | NAPIF_STATE_SCHED; /* Sets STATE_MISSED bit if STATE_SCHED was already set @@ -6458,12 +6461,14 @@ bool napi_complete_done(struct napi_struct *n, int work_done) /* If STATE_MISSED was set, leave STATE_SCHED set, * because we will call napi->poll() one more time. - * This C code was suggested by Alexander Duyck to help gcc. */ - new |= (val & NAPIF_STATE_MISSED) / NAPIF_STATE_MISSED * - NAPIF_STATE_SCHED; + if (val & NAPIF_STATE_MISSED && !(val & NAPIF_STATE_BIAS_BUSY_POLL)) + new |= NAPIF_STATE_SCHED; } while (cmpxchg(&n->state, val, new) != val); + if (unlikely(val & NAPIF_STATE_BIAS_BUSY_POLL)) + return false; + if (unlikely(val & NAPIF_STATE_MISSED)) { __napi_schedule(n); return false; @@ -6497,6 +6502,20 @@ static void busy_poll_stop(struct napi_struct *napi, void *have_poll_lock) { int rc; + clear_bit(NAPI_STATE_IN_BUSY_POLL, &napi->state); + + local_bh_disable(); + /* If we're biased towards busy poll, clear the sched flags, + * so that we can enter again. + */ + if (READ_ONCE(napi->state) & NAPIF_STATE_BIAS_BUSY_POLL) { + netpoll_poll_unlock(have_poll_lock); + napi_complete(napi); + __kfree_skb_flush(); + local_bh_enable(); + return; + } + /* Busy polling means there is a high chance device driver hard irq * could not grab NAPI_STATE_SCHED, and that NAPI_STATE_MISSED was * set in napi_schedule_prep(). @@ -6507,9 +6526,6 @@ static void busy_poll_stop(struct napi_struct *napi, void *have_poll_lock) * to perform these two clear_bit() */ clear_bit(NAPI_STATE_MISSED, &napi->state); - clear_bit(NAPI_STATE_IN_BUSY_POLL, &napi->state); - - local_bh_disable(); /* All we really want here is to re-enable device interrupts. * Ideally, a new ndo_busy_poll_stop() could avoid another round. @@ -6569,6 +6585,11 @@ void napi_busy_loop(unsigned int napi_id, goto count; have_poll_lock = netpoll_poll_lock(napi); napi_poll = napi->poll; + if (val & NAPIF_STATE_BIAS_BUSY_POLL) { + hrtimer_start(&napi->bp_watchdog, + ms_to_ktime(BIASED_BUSY_POLL_TIMEOUT), + HRTIMER_MODE_REL_PINNED); + } } work = napi_poll(napi, BUSY_POLL_BUDGET); trace_napi_poll(napi, work, BUSY_POLL_BUDGET); @@ -6652,6 +6673,53 @@ static enum hrtimer_restart napi_watchdog(struct hrtimer *timer) return HRTIMER_NORESTART; } +static enum hrtimer_restart napi_biased_busy_poll_watchdog(struct hrtimer *timer) +{ + struct napi_struct *napi; + unsigned long val, new; + + napi = container_of(timer, struct napi_struct, bp_watchdog); + + do { + val = READ_ONCE(napi->state); + if (WARN_ON_ONCE(!(val & NAPIF_STATE_BIAS_BUSY_POLL))) + return HRTIMER_NORESTART; + + new = val & ~NAPIF_STATE_BIAS_BUSY_POLL; + } while (cmpxchg(&napi->state, val, new) != val); + + if (!napi_disable_pending(napi) && + !test_and_set_bit(NAPI_STATE_SCHED, &napi->state)) + __napi_schedule_irqoff(napi); + + return HRTIMER_NORESTART; +} + +void napi_bias_busy_poll(unsigned int napi_id) +{ +#ifdef CONFIG_NET_RX_BUSY_POLL + struct napi_struct *napi; + unsigned long val, new; + + napi = napi_by_id(napi_id); + if (!napi) + return; + + do { + val = READ_ONCE(napi->state); + if (val & NAPIF_STATE_BIAS_BUSY_POLL) + return; + + new = val | NAPIF_STATE_BIAS_BUSY_POLL; + } while (cmpxchg(&napi->state, val, new) != val); + + hrtimer_start(&napi->bp_watchdog, ms_to_ktime(BIASED_BUSY_POLL_TIMEOUT), + HRTIMER_MODE_REL_PINNED); +#endif +} +EXPORT_SYMBOL(napi_bias_busy_poll); + + static void init_gro_hash(struct napi_struct *napi) { int i; @@ -6673,6 +6741,8 @@ void netif_napi_add(struct net_device *dev, struct napi_struct *napi, INIT_HLIST_NODE(&napi->napi_hash_node); hrtimer_init(&napi->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_PINNED); napi->timer.function = napi_watchdog; + hrtimer_init(&napi->bp_watchdog, CLOCK_MONOTONIC, HRTIMER_MODE_REL_PINNED); + napi->bp_watchdog.function = napi_biased_busy_poll_watchdog; init_gro_hash(napi); napi->skb = NULL; INIT_LIST_HEAD(&napi->rx_list); @@ -6704,7 +6774,9 @@ void napi_disable(struct napi_struct *n) msleep(1); hrtimer_cancel(&n->timer); + hrtimer_cancel(&n->bp_watchdog); + clear_bit(NAPI_STATE_BIAS_BUSY_POLL, &n->state); clear_bit(NAPI_STATE_DISABLE, &n->state); } EXPORT_SYMBOL(napi_disable); @@ -6767,6 +6839,11 @@ static int napi_poll(struct napi_struct *n, struct list_head *repoll) if (likely(work < weight)) goto out_unlock; + if (unlikely(n->state & NAPIF_STATE_BIAS_BUSY_POLL)) { + napi_complete(n); + goto out_unlock; + } + /* Drivers must not modify the NAPI state if they * consume the entire weight. In such cases this code * still "owns" the NAPI instance and therefore can diff --git a/net/core/sock.c b/net/core/sock.c index 727ea1cc633c..686eb5549b79 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -1159,6 +1159,12 @@ int sock_setsockopt(struct socket *sock, int level, int optname, sk->sk_ll_usec = val; } break; + case SO_BIAS_BUSY_POLL: + if (valbool && !capable(CAP_NET_ADMIN)) + ret = -EPERM; + else + sk->sk_bias_busy_poll = valbool; + break; #endif case SO_MAX_PACING_RATE: @@ -1523,6 +1529,9 @@ int sock_getsockopt(struct socket *sock, int level, int optname, case SO_BUSY_POLL: v.val = sk->sk_ll_usec; break; + case SO_BIAS_BUSY_POLL: + v.val = sk->sk_bias_busy_poll; + break; #endif case SO_MAX_PACING_RATE: From patchwork Wed Oct 28 13:34:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11864009 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3CBCC55179 for ; Wed, 28 Oct 2020 23:00:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8AE4920757 for ; Wed, 28 Oct 2020 23:00:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PLrzjJzw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389944AbgJ1XAT (ORCPT ); Wed, 28 Oct 2020 19:00:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60788 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387732AbgJ1XAR (ORCPT ); Wed, 28 Oct 2020 19:00:17 -0400 Received: from mail-vs1-xe44.google.com (mail-vs1-xe44.google.com [IPv6:2607:f8b0:4864:20::e44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A59FC0613CF; Wed, 28 Oct 2020 16:00:17 -0700 (PDT) Received: by mail-vs1-xe44.google.com with SMTP id w25so475554vsk.9; Wed, 28 Oct 2020 16:00:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ctARgd5IeefW/xDEFZzb1NY2kFiOmkaVN6jb4VQWnGY=; b=PLrzjJzwRFNojRi8jy55qv7BTPweCnOMSMk+Ol7VV2YvxManvupJmNNvMgGaujesOZ HRKRAOl+mxj3YnZ1ORrIZ/62yhyrBDRiCG2LY7JDf8SihsNrdcO7MEyYbLjD6QRO7AQq KziySVPDyskFCbXY+X7qwOPqVg3ynAcZ0BVimmg8nNPHdhsLd43olWNA3v6ck4SAoi+z DLby4KSb1LwgcE0o4KQ0uoRMVr6HYDGGlnwu9aCOB+kbpDf+dvw1bhCJ9lwoMmI6cvEf JTCF7m0Myf6inaqlKyjLHDjIB0Yg6MHpB4KzY5NcXF+++ScoO9twHW+C9WziI5d9mSse 60Mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ctARgd5IeefW/xDEFZzb1NY2kFiOmkaVN6jb4VQWnGY=; b=a+VvTThXg2ZUmRRwZhcRin6rCwPINlVHxVJYjWl3P0YxF2FSXSYNYYXRPXZ33clMlh q5xW+irohfhXrf3IcX0x7lQu2hlnOP7p8De8CZSVU/UefkF/JHzLTXmfbzrGOqid1CTY /38+QRBggND2vm1NM2gSlP4KQAgYm94XzE4/xAvBPl7lmAnLH6j277TZuz6zqYWHH6xv K+iD5ONxsVf13tzMf+P+fI8T2CpPxzcf46uUaMLNdLk0Nkzedm4llgbE6xxvrZJw6ZPh GoBijnlv9ezQzEVoO81eKvLmPG9eHHyvYEnuLXNiuI26pNlx7Z9CvZCdrWc12DBQO109 7Ixg== X-Gm-Message-State: AOAM530dBSVHcEtFOIwOMfuR0nj52mvo+xSYV4T8vxHw9uvvJGh8sBaC foVhFaTsfn2o6VLCRkBq1hhqHi+I/e7tr+i/ X-Google-Smtp-Source: ABdhPJy4JsZnLSuXFGGfGf5z9wYwEIBbyRNvXY0p7TXn6SDylWXjbgKrQDAWfIugPhofnBPsR2d//A== X-Received: by 2002:a62:7a8f:0:b029:163:d0b3:ac18 with SMTP id v137-20020a627a8f0000b0290163d0b3ac18mr7777212pfc.9.1603892115932; Wed, 28 Oct 2020 06:35:15 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q14sm5935393pjp.43.2020.10.28.06.35.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2020 06:35:15 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, ast@kernel.org, daniel@iogearbox.net, maciej.fijalkowski@intel.com, sridhar.samudrala@intel.com, jesse.brandeburg@intel.com, qi.z.zhang@intel.com, kuba@kernel.org, edumazet@google.com, intel-wired-lan@lists.osuosl.org, jonathan.lemon@gmail.com Subject: [RFC PATCH bpf-next 2/9] net: add SO_BUSY_POLL_BUDGET socket option Date: Wed, 28 Oct 2020 14:34:30 +0100 Message-Id: <20201028133437.212503-3-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028133437.212503-1-bjorn.topel@gmail.com> References: <20201028133437.212503-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Björn Töpel This option lets a user set a per socket NAPI budget for busy-polling. If the options is not set, it will use the default of 8. Signed-off-by: Björn Töpel --- arch/alpha/include/uapi/asm/socket.h | 1 + arch/mips/include/uapi/asm/socket.h | 1 + arch/parisc/include/uapi/asm/socket.h | 1 + arch/sparc/include/uapi/asm/socket.h | 1 + fs/eventpoll.c | 3 ++- include/net/busy_poll.h | 6 ++++-- include/net/sock.h | 1 + include/uapi/asm-generic/socket.h | 1 + net/core/dev.c | 20 +++++++++----------- net/core/sock.c | 10 ++++++++++ 10 files changed, 31 insertions(+), 14 deletions(-) diff --git a/arch/alpha/include/uapi/asm/socket.h b/arch/alpha/include/uapi/asm/socket.h index 0f776668fb09..4ea972b7b711 100644 --- a/arch/alpha/include/uapi/asm/socket.h +++ b/arch/alpha/include/uapi/asm/socket.h @@ -125,6 +125,7 @@ #define SO_DETACH_REUSEPORT_BPF 68 #define SO_BIAS_BUSY_POLL 69 +#define SO_BUSY_POLL_BUDGET 70 #if !defined(__KERNEL__) diff --git a/arch/mips/include/uapi/asm/socket.h b/arch/mips/include/uapi/asm/socket.h index d23984731504..13eaffbfbe50 100644 --- a/arch/mips/include/uapi/asm/socket.h +++ b/arch/mips/include/uapi/asm/socket.h @@ -136,6 +136,7 @@ #define SO_DETACH_REUSEPORT_BPF 68 #define SO_BIAS_BUSY_POLL 69 +#define SO_BUSY_POLL_BUDGET 70 #if !defined(__KERNEL__) diff --git a/arch/parisc/include/uapi/asm/socket.h b/arch/parisc/include/uapi/asm/socket.h index 49469713ed2a..036e42dac6b3 100644 --- a/arch/parisc/include/uapi/asm/socket.h +++ b/arch/parisc/include/uapi/asm/socket.h @@ -117,6 +117,7 @@ #define SO_DETACH_REUSEPORT_BPF 0x4042 #define SO_BIAS_BUSY_POLL 0x4043 +#define SO_BUSY_POLL_BUDGET 0x4044 #if !defined(__KERNEL__) diff --git a/arch/sparc/include/uapi/asm/socket.h b/arch/sparc/include/uapi/asm/socket.h index 009aba6f7a54..bc482dc93bd4 100644 --- a/arch/sparc/include/uapi/asm/socket.h +++ b/arch/sparc/include/uapi/asm/socket.h @@ -118,6 +118,7 @@ #define SO_DETACH_REUSEPORT_BPF 0x0047 #define SO_BIAS_BUSY_POLL 0x0048 +#define SO_BUSY_POLL_BUDGET 0x0049 #if !defined(__KERNEL__) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 4df61129566d..fa00a0640264 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -397,7 +397,8 @@ static void ep_busy_loop(struct eventpoll *ep, int nonblock) unsigned int napi_id = READ_ONCE(ep->napi_id); if ((napi_id >= MIN_NAPI_ID) && net_busy_loop_on()) - napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep); + napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, + BUSY_POLL_BUDGET); } static inline void ep_reset_busy_poll_napi_id(struct eventpoll *ep) diff --git a/include/net/busy_poll.h b/include/net/busy_poll.h index 9738923ed17b..c6c413d3824d 100644 --- a/include/net/busy_poll.h +++ b/include/net/busy_poll.h @@ -25,6 +25,7 @@ /* Biased busy-poll watchdog timeout in ms */ #define BIASED_BUSY_POLL_TIMEOUT 200 +#define BUSY_POLL_BUDGET 8 #ifdef CONFIG_NET_RX_BUSY_POLL @@ -46,7 +47,7 @@ bool sk_busy_loop_end(void *p, unsigned long start_time); void napi_busy_loop(unsigned int napi_id, bool (*loop_end)(void *, unsigned long), - void *loop_end_arg); + void *loop_end_arg, u16 budget); #else /* CONFIG_NET_RX_BUSY_POLL */ static inline unsigned long net_busy_loop_on(void) @@ -119,7 +120,8 @@ static inline void sk_busy_loop(struct sock *sk, int nonblock) if (napi_id >= MIN_NAPI_ID) { __sk_bias_busy_poll(sk, napi_id); - napi_busy_loop(napi_id, nonblock ? NULL : sk_busy_loop_end, sk); + napi_busy_loop(napi_id, nonblock ? NULL : sk_busy_loop_end, sk, + sk->sk_busy_poll_budget ?: BUSY_POLL_BUDGET); } #endif } diff --git a/include/net/sock.h b/include/net/sock.h index cf71834fb601..3caf53b6bd71 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -481,6 +481,7 @@ struct sock { kuid_t sk_uid; #ifdef CONFIG_NET_RX_BUSY_POLL u8 sk_bias_busy_poll; + u16 sk_busy_poll_budget; #endif struct pid *sk_peer_pid; const struct cred *sk_peer_cred; diff --git a/include/uapi/asm-generic/socket.h b/include/uapi/asm-generic/socket.h index 8a2b37ccd9d5..9dc1f35fe77f 100644 --- a/include/uapi/asm-generic/socket.h +++ b/include/uapi/asm-generic/socket.h @@ -120,6 +120,7 @@ #define SO_DETACH_REUSEPORT_BPF 68 #define SO_BIAS_BUSY_POLL 69 +#define SO_BUSY_POLL_BUDGET 70 #if !defined(__KERNEL__) diff --git a/net/core/dev.c b/net/core/dev.c index a29e4c4a35f6..b34520acaa7f 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6496,9 +6496,7 @@ static struct napi_struct *napi_by_id(unsigned int napi_id) #if defined(CONFIG_NET_RX_BUSY_POLL) -#define BUSY_POLL_BUDGET 8 - -static void busy_poll_stop(struct napi_struct *napi, void *have_poll_lock) +static void busy_poll_stop(struct napi_struct *napi, void *have_poll_lock, u16 budget) { int rc; @@ -6530,14 +6528,14 @@ static void busy_poll_stop(struct napi_struct *napi, void *have_poll_lock) /* All we really want here is to re-enable device interrupts. * Ideally, a new ndo_busy_poll_stop() could avoid another round. */ - rc = napi->poll(napi, BUSY_POLL_BUDGET); + rc = napi->poll(napi, budget); /* We can't gro_normal_list() here, because napi->poll() might have * rearmed the napi (napi_complete_done()) in which case it could * already be running on another CPU. */ - trace_napi_poll(napi, rc, BUSY_POLL_BUDGET); + trace_napi_poll(napi, rc, budget); netpoll_poll_unlock(have_poll_lock); - if (rc == BUSY_POLL_BUDGET) { + if (rc == budget) { /* As the whole budget was spent, we still own the napi so can * safely handle the rx_list. */ @@ -6549,7 +6547,7 @@ static void busy_poll_stop(struct napi_struct *napi, void *have_poll_lock) void napi_busy_loop(unsigned int napi_id, bool (*loop_end)(void *, unsigned long), - void *loop_end_arg) + void *loop_end_arg, u16 budget) { unsigned long start_time = loop_end ? busy_loop_current_time() : 0; int (*napi_poll)(struct napi_struct *napi, int budget); @@ -6591,8 +6589,8 @@ void napi_busy_loop(unsigned int napi_id, HRTIMER_MODE_REL_PINNED); } } - work = napi_poll(napi, BUSY_POLL_BUDGET); - trace_napi_poll(napi, work, BUSY_POLL_BUDGET); + work = napi_poll(napi, budget); + trace_napi_poll(napi, work, budget); gro_normal_list(napi); count: if (work > 0) @@ -6605,7 +6603,7 @@ void napi_busy_loop(unsigned int napi_id, if (unlikely(need_resched())) { if (napi_poll) - busy_poll_stop(napi, have_poll_lock); + busy_poll_stop(napi, have_poll_lock, budget); preempt_enable(); rcu_read_unlock(); cond_resched(); @@ -6616,7 +6614,7 @@ void napi_busy_loop(unsigned int napi_id, cpu_relax(); } if (napi_poll) - busy_poll_stop(napi, have_poll_lock); + busy_poll_stop(napi, have_poll_lock, budget); preempt_enable(); out: rcu_read_unlock(); diff --git a/net/core/sock.c b/net/core/sock.c index 686eb5549b79..799125de4add 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -1165,6 +1165,16 @@ int sock_setsockopt(struct socket *sock, int level, int optname, else sk->sk_bias_busy_poll = valbool; break; + case SO_BUSY_POLL_BUDGET: + if ((val > sk->sk_busy_poll_budget) && !capable(CAP_NET_ADMIN)) + ret = -EPERM; + else { + if (val < 0) + ret = -EINVAL; + else + sk->sk_busy_poll_budget = val; + } + break; #endif case SO_MAX_PACING_RATE: From patchwork Wed Oct 28 13:34:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11863673 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C832C55179 for ; Wed, 28 Oct 2020 22:28:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 269C120725 for ; Wed, 28 Oct 2020 22:28:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OgykjbeB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733036AbgJ1W2R (ORCPT ); Wed, 28 Oct 2020 18:28:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387422AbgJ1W2K (ORCPT ); Wed, 28 Oct 2020 18:28:10 -0400 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73ACFC0613CF; Wed, 28 Oct 2020 15:28:10 -0700 (PDT) Received: by mail-pg1-x541.google.com with SMTP id f38so700562pgm.2; Wed, 28 Oct 2020 15:28:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=p+7zqDEUsInVWiiKbv2mU/avNMxBK0HvJzz4eBBZxRg=; b=OgykjbeBYTcwSjAJLswfBmSpu1YsZF27KG6D/MRCQVqUmHCuYYMJytcrH+RQXviJFv OFpc2lZtT/zrN9NPlVxVbV/cKdTUEdxu4p0k2/w7cHLBOU2jd3TtFC2kWLi0pdWeAgB8 nOAUZXdFSQa+qnNaSJvy918NPqfJKi7c39qDSfg0hzSTcU5cMRHqvEKqihRR8WH1dMqs tFg4hqxvUSCWie8BCN6/OTuxV9FhjoGrm8UboeP5HMBKHqNsGRQnOTpzhtpzWs2fRCDT FuO65G8Meu2jt9aKFfT4cdZsji/kNeB6X5seMYVfWtcnZN2eKTC0/4qcDOUZ0766e7A+ iGuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=p+7zqDEUsInVWiiKbv2mU/avNMxBK0HvJzz4eBBZxRg=; b=gwzLz8s7/wgt1nlMKW6ayOGadLF+VJfMuHxHKoslrSHVXcciKVTyiL3lSKr8uSxnN1 jESTQaa10AvGM3lDMPHc4uIkt2SepeH66kyy77BwDbeHHSGDMbI6pJEv4FZw/XrfP5wc rKxoNFDGy+rI79Ne4ILYqCl849TXuMeKAFQAFzYYo+1bJ5FVEbNwtB7CEOYop+Sxs1AS MTSq8csjYSMqkZmtG3E5HQNDuxKG4wu6UnhwX8nzD3ywfNW18awyUQRdOI4dW9Y94Z59 vG9KRotIo8H9oFfUelzGBw1eqevD0ztphxCVoEmRkWYpFwLDFIUS91rfNYuNfbW44486 MpgQ== X-Gm-Message-State: AOAM533XvqMVP0+3mXu2lnwWMezbYLPDZc2DZABBCoO4ng7lfUmyYcl5 vDczOeKqA9rIqcg/fG7Y2hkAw4TbBBJpzlLo X-Google-Smtp-Source: ABdhPJyg91YP5G0ly1+wJndgmQ+pQQEXBSh2NKhveMXIaGJ3NqHZZSc+XmBwH86rU6ST9iPb0aIa7w== X-Received: by 2002:aa7:954b:0:b029:154:8ee9:e4d with SMTP id w11-20020aa7954b0000b02901548ee90e4dmr7751812pfq.11.1603892121903; Wed, 28 Oct 2020 06:35:21 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q14sm5935393pjp.43.2020.10.28.06.35.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2020 06:35:21 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, ast@kernel.org, daniel@iogearbox.net, maciej.fijalkowski@intel.com, sridhar.samudrala@intel.com, jesse.brandeburg@intel.com, qi.z.zhang@intel.com, kuba@kernel.org, edumazet@google.com, intel-wired-lan@lists.osuosl.org, jonathan.lemon@gmail.com Subject: [RFC PATCH bpf-next 3/9] xsk: add support for recvmsg() Date: Wed, 28 Oct 2020 14:34:31 +0100 Message-Id: <20201028133437.212503-4-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028133437.212503-1-bjorn.topel@gmail.com> References: <20201028133437.212503-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Björn Töpel Add support for non-blocking recvmsg() to XDP sockets. Previously, only sendmsg() was supported by XDP socket. Now, for symmetry and the upcoming busy-polling support, recvmsg() is added. Signed-off-by: Björn Töpel --- net/xdp/xsk.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index b71a32eeae65..17d51d1a5752 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -474,6 +474,26 @@ static int xsk_sendmsg(struct socket *sock, struct msghdr *m, size_t total_len) return __xsk_sendmsg(sk); } +static int xsk_recvmsg(struct socket *sock, struct msghdr *m, size_t len, int flags) +{ + bool need_wait = !(flags & MSG_DONTWAIT); + struct sock *sk = sock->sk; + struct xdp_sock *xs = xdp_sk(sk); + + if (unlikely(!(xs->dev->flags & IFF_UP))) + return -ENETDOWN; + if (unlikely(!xs->rx)) + return -ENOBUFS; + if (unlikely(!xsk_is_bound(xs))) + return -ENXIO; + if (unlikely(need_wait)) + return -EOPNOTSUPP; + + if (xs->pool->cached_need_wakeup & XDP_WAKEUP_RX && xs->zc) + return xsk_wakeup(xs, XDP_WAKEUP_RX); + return 0; +} + static __poll_t xsk_poll(struct file *file, struct socket *sock, struct poll_table_struct *wait) { @@ -1134,7 +1154,7 @@ static const struct proto_ops xsk_proto_ops = { .setsockopt = xsk_setsockopt, .getsockopt = xsk_getsockopt, .sendmsg = xsk_sendmsg, - .recvmsg = sock_no_recvmsg, + .recvmsg = xsk_recvmsg, .mmap = xsk_mmap, .sendpage = sock_no_sendpage, }; From patchwork Wed Oct 28 13:34:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11864063 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 003A7C4363A for ; Wed, 28 Oct 2020 23:23:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9CB9C207BC for ; Wed, 28 Oct 2020 23:23:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q9pVa6xI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390116AbgJ1XXX (ORCPT ); Wed, 28 Oct 2020 19:23:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389991AbgJ1XWm (ORCPT ); Wed, 28 Oct 2020 19:22:42 -0400 Received: from mail-oi1-x241.google.com (mail-oi1-x241.google.com [IPv6:2607:f8b0:4864:20::241]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 792FCC0613CF; Wed, 28 Oct 2020 16:22:42 -0700 (PDT) Received: by mail-oi1-x241.google.com with SMTP id u127so1362844oib.6; Wed, 28 Oct 2020 16:22:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Y6HWVZiVbe0DlTeVilFZZF5KBmHXDNnXyxJw5KGNE18=; b=Q9pVa6xIH+HoZ+K1VGiQHxhDmSJCKl7x7+ZbKUN+1xgaFlZPQuHBPSYv5HYuTA0sjG LaBrJYUGQdfYPMrxeuLTCcoLcIdvfoyLKCgD7ctg9GhjENImdx+SP4vUIz3jxNzJax00 KzTrt0WGUUcaVDXm2LnN12lN30pJptzbO0zN4RL/FSLHYsv/H9aUFTnj4kX35RAW+xzI 2kQ0qioK0hLQ7eZDqfUs3BDOnpbhE3mkNc//l4MlMpMe0/e86tfXaORAtZ5ZT7v/WJUH LxhfRoWvRFmbHww3nS6kAmBDTl5Gc5meNz4ZAo/GWNGbR/0ivPMwYTrnoTIqxcUZ4uSV mC1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Y6HWVZiVbe0DlTeVilFZZF5KBmHXDNnXyxJw5KGNE18=; b=m6/kUnJ+AqPYQ2ZgncTLXYzEX4RI5WDWSbx3AgseD8/+BpOuscxi11of5Sspi257tu CK3vWN5f/oVwoxALf8ZRjWGKmFdtq2f0J6KrmKUY5qUrx23ubr08qOUCPtNr9/8Vj9kt ueixVsZGaqVJAAdrWITbKsWSPmxVZ3nhxS3UTuQnxWnACfsBhR5pzbAZ6DD0Ih1CNotj g5EMRlyzWvDZvv8RcKpfCWnihZNbAm9L8ERp/Rzx6UD8xf9JxQ92njTe85Phez/vkrdR RxoOajhXwYzzNAbNXw2ZQZge77TTgEn2SxEUNMOoHA2Z/U0CfUYuMuSDEdOkunpvl3LS PlhQ== X-Gm-Message-State: AOAM533LBdxZ1OEswZhCUIri2oAQ9CIQM6hV7df9JBgMLYDCbQqs2TAV b9BJp4ONtDrvD6rrPk2JurIPW0gKuClv8eiW X-Google-Smtp-Source: ABdhPJyLrY/Dmb/oakRjXGVfkc2hQL0ea2nhABJdqF8y6eKgUaX7nxSXT40wSwQ545JBnNZK8RbSXQ== X-Received: by 2002:a17:90b:204:: with SMTP id fy4mr7400019pjb.156.1603892127374; Wed, 28 Oct 2020 06:35:27 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q14sm5935393pjp.43.2020.10.28.06.35.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2020 06:35:26 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, ast@kernel.org, daniel@iogearbox.net, maciej.fijalkowski@intel.com, sridhar.samudrala@intel.com, jesse.brandeburg@intel.com, qi.z.zhang@intel.com, kuba@kernel.org, edumazet@google.com, intel-wired-lan@lists.osuosl.org, jonathan.lemon@gmail.com Subject: [RFC PATCH bpf-next 4/9] xsk: check need wakeup flag in sendmsg() Date: Wed, 28 Oct 2020 14:34:32 +0100 Message-Id: <20201028133437.212503-5-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028133437.212503-1-bjorn.topel@gmail.com> References: <20201028133437.212503-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Björn Töpel Add a check for need wake up in sendmsg(), so that if a user calls sendmsg() when no wakeup is needed, do not trigger a wakeup. To simplify the need wakeup check in the syscall, unconditionally enable the need wakeup flag for Tx. This has a side-effect for poll(); If poll() is called for a socket without enabled need wakeup, a Tx wakeup is unconditionally performed. The wakeup matrix for AF_XDP now looks like: need wakeup | poll() | sendmsg() | recvmsg() ------------+------------------------+---------------------+--------------------- disabled | wake Tx | wake Tx | nop enabled | check flag; wake Tx/Rx | check flag; wake Tx | check flag; wake Rx Signed-off-by: Björn Töpel --- net/xdp/xsk.c | 6 +++++- net/xdp/xsk_buff_pool.c | 13 ++++++------- 2 files changed, 11 insertions(+), 8 deletions(-) diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index 17d51d1a5752..2e5b9f27c7a3 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -465,13 +465,17 @@ static int xsk_sendmsg(struct socket *sock, struct msghdr *m, size_t total_len) bool need_wait = !(m->msg_flags & MSG_DONTWAIT); struct sock *sk = sock->sk; struct xdp_sock *xs = xdp_sk(sk); + struct xsk_buff_pool *pool; if (unlikely(!xsk_is_bound(xs))) return -ENXIO; if (unlikely(need_wait)) return -EOPNOTSUPP; - return __xsk_sendmsg(sk); + pool = xs->pool; + if (pool->cached_need_wakeup & XDP_WAKEUP_TX) + return __xsk_sendmsg(sk); + return 0; } static int xsk_recvmsg(struct socket *sock, struct msghdr *m, size_t len, int flags) diff --git a/net/xdp/xsk_buff_pool.c b/net/xdp/xsk_buff_pool.c index 64c9e55d4d4e..a4acb5e9576f 100644 --- a/net/xdp/xsk_buff_pool.c +++ b/net/xdp/xsk_buff_pool.c @@ -144,14 +144,13 @@ static int __xp_assign_dev(struct xsk_buff_pool *pool, if (err) return err; - if (flags & XDP_USE_NEED_WAKEUP) { + if (flags & XDP_USE_NEED_WAKEUP) pool->uses_need_wakeup = true; - /* Tx needs to be explicitly woken up the first time. - * Also for supporting drivers that do not implement this - * feature. They will always have to call sendto(). - */ - pool->cached_need_wakeup = XDP_WAKEUP_TX; - } + /* Tx needs to be explicitly woken up the first time. Also + * for supporting drivers that do not implement this + * feature. They will always have to call sendto() or poll(). + */ + pool->cached_need_wakeup = XDP_WAKEUP_TX; dev_hold(netdev); From patchwork Wed Oct 28 13:34:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11863457 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAAE0C55179 for ; Wed, 28 Oct 2020 22:05:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8378B24754 for ; Wed, 28 Oct 2020 22:05:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ThThPoDI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729972AbgJ1WFO (ORCPT ); Wed, 28 Oct 2020 18:05:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51334 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729966AbgJ1WFM (ORCPT ); Wed, 28 Oct 2020 18:05:12 -0400 Received: from mail-vs1-xe43.google.com (mail-vs1-xe43.google.com [IPv6:2607:f8b0:4864:20::e43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45EB2C0613D1; Wed, 28 Oct 2020 15:05:12 -0700 (PDT) Received: by mail-vs1-xe43.google.com with SMTP id w25so409733vsk.9; Wed, 28 Oct 2020 15:05:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sBa77MmCFN0T1+VtUx1HDnNMT2/Xv5NhXqq896ClIPA=; b=ThThPoDIPRt59MZkfI1qQFRnAS2OlQaGLzpGPN2cxTSQfDtCdz08ENqwNG9U8jzOSq s34Q276RHnCcq8eK/nTv0czP0MtEL51wsFyO+l0/EF/aZx+dKou4D7B6eVgK94JVOkJy gNfNJgZzXGP//V7CiqZ3l8ybUm24uPVJ0zBOsk3MEzh+BQ/TucW/kMC61IEEHLyMgwnz 6TaxNwP8l906lqpO77tS0TEDJFzAltKegGSigo6TgM+iVrTnY7ITYI4zxmNp2cKEmth2 8o4dgHKzSDhquPjaHN+aQt5e/yANa4V1Nlnmnor+iDQiKXFMgUOhuOG8XxBJGAdtrER9 PYdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sBa77MmCFN0T1+VtUx1HDnNMT2/Xv5NhXqq896ClIPA=; b=CpanmAL7sESG0s+5JOewhFV3rblxzAaHakFJx7sD3yAuJQj50LYmJsEr50r8ID3Kor +wMVb3xQRm5P4ST60RnF83fnRwVaZ/TTy00x8QP6mGCEdSZytCCyxSROha3uiOwf4FP8 EyZrLbidrcpQNdR+TWnarUpjvSt0aHTIixD6n2d29jBDJY3x40Ks8ZTM+s3WBq6xQHy9 2oUjMJMM+enQan+OXcYUe/4ag2w7wVU4MlHjQsvyJMrkbXRq4eFCs2avtixKEFFDmqud 83ywZiI9f0j1MWAwWY3PSW9DqEa3CDGK689U2PJTnxkATzaxYB5r2IufkbhT3UPwTh1Y DTsw== X-Gm-Message-State: AOAM530JWLoBBhpQBvGrbdA9cMJVxWaWL/vfmGl7Qegzoanmt5VeeZuK XTiHrxZOTNkFqI252gaa5k7iL5rjk/Bho8E7 X-Google-Smtp-Source: ABdhPJx1sVLTojg+U9YZMBvRLhAg0GBPcu5nq1fPysti/X+d+Gmy//kTyoSjwtScwRHBrv8BGUyEYA== X-Received: by 2002:a62:ea0c:0:b029:164:3789:547b with SMTP id t12-20020a62ea0c0000b02901643789547bmr7074341pfh.27.1603892133273; Wed, 28 Oct 2020 06:35:33 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q14sm5935393pjp.43.2020.10.28.06.35.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2020 06:35:32 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, ast@kernel.org, daniel@iogearbox.net, maciej.fijalkowski@intel.com, sridhar.samudrala@intel.com, jesse.brandeburg@intel.com, qi.z.zhang@intel.com, kuba@kernel.org, edumazet@google.com, intel-wired-lan@lists.osuosl.org, jonathan.lemon@gmail.com Subject: [RFC PATCH bpf-next 5/9] xsk: add busy-poll support for {recv,send}msg() Date: Wed, 28 Oct 2020 14:34:33 +0100 Message-Id: <20201028133437.212503-6-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028133437.212503-1-bjorn.topel@gmail.com> References: <20201028133437.212503-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Björn Töpel Wire-up XDP socket busy-poll support for recvmsg() and sendmsg(). Signed-off-by: Björn Töpel --- net/xdp/xsk.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index 2e5b9f27c7a3..da649b4f377c 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include "xsk_queue.h" @@ -472,6 +473,9 @@ static int xsk_sendmsg(struct socket *sock, struct msghdr *m, size_t total_len) if (unlikely(need_wait)) return -EOPNOTSUPP; + if (sk_can_busy_loop(sk)) + sk_busy_loop(sk, 1); /* only support non-blocking sockets */ + pool = xs->pool; if (pool->cached_need_wakeup & XDP_WAKEUP_TX) return __xsk_sendmsg(sk); @@ -493,6 +497,9 @@ static int xsk_recvmsg(struct socket *sock, struct msghdr *m, size_t len, int fl if (unlikely(need_wait)) return -EOPNOTSUPP; + if (sk_can_busy_loop(sk)) + sk_busy_loop(sk, 1); /* only support non-blocking sockets */ + if (xs->pool->cached_need_wakeup & XDP_WAKEUP_RX && xs->zc) return xsk_wakeup(xs, XDP_WAKEUP_RX); return 0; From patchwork Wed Oct 28 13:34:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11863997 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E25B7C388F7 for ; Wed, 28 Oct 2020 22:58:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7D1BA20757 for ; Wed, 28 Oct 2020 22:58:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jllmkngj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389704AbgJ1W4K (ORCPT ); Wed, 28 Oct 2020 18:56:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732878AbgJ1WxQ (ORCPT ); Wed, 28 Oct 2020 18:53:16 -0400 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FBC3C0613CF; Wed, 28 Oct 2020 15:53:16 -0700 (PDT) Received: by mail-pg1-x543.google.com with SMTP id o3so716566pgr.11; Wed, 28 Oct 2020 15:53:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sqsN722ssb+yJiDbsT28nLzbB0jvmz4/+1jmYycos9s=; b=jllmkngjeH0ITVZX0ppk4FLiUJa/QLhGrx4t40SXgyj35xlIxOxxNiiHBnuISZyHPH fZCMuyqiPoConmFas95ZyV2wUNQj62wynryoAnuPpXGR8HFfhz2wGprXh3blfGx8q/Qu U/D7XQ4n4swnfN+0xl4hLAqQ3GTfxnXOtwv9ASV8DCMp5LIs7qmnwtER0DuqYEM+QaQ8 b5fda0H4Wr5DcNiBQPK3TGdg9RkBvHA48+vTXmNVTMUkcMq9JZdCInvmTZc7Zd+suyFu Se4ftntO2XKVLFt+bZ2u2voWzflP3ObzvZqFObwAJiZAcQt+filaJaPsLX8oDeKwgODo c1Jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sqsN722ssb+yJiDbsT28nLzbB0jvmz4/+1jmYycos9s=; b=m3bUb4lShy9Q1cHrjXpdvYJLD1FLg4KlVSVqNwErLK+FtxbBMpDSW5MmqPbHG5usZJ +ZEYTBBJvb1ayfNOhM7plXWUKOuzZdvgVskCMw7j557XDKv9WujFj6cX36um1O7Ayenm wiyZkEYxUzfnU99ww1RbfM0Dk/o+feXbCg8LAhCKiSE52jxmevo8Ch4cFXmlrhI848pA IJ3JbFwfNSyx7KROeljt5xdY+OBoKL+usdR5L4V3i1NPCvNJWFdYEuvr5BPGSwEj7nQ7 las1fxGCUV+fHye5cv0r+ahxGertkbh4z45QhFgyJNPsZ8qGUzrJjkHJYaj2V3ks5aTa xaBA== X-Gm-Message-State: AOAM530NZYwYJR3mvRODLlrSngHYBekpIhcxXatCzgmcQx8kuKpbLfXu ecJr9U8hRJZHGPwxXoFX509cM/jIL6pgiw2o X-Google-Smtp-Source: ABdhPJwu5YC2c6TAcUXj2ofNJUwXTX0HC4+qPGQ5Im0p0vaf/TekcrX1Lm37LtaH9cf6ZJdY6lQIew== X-Received: by 2002:a63:4383:: with SMTP id q125mr6591975pga.30.1603892139093; Wed, 28 Oct 2020 06:35:39 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q14sm5935393pjp.43.2020.10.28.06.35.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2020 06:35:38 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, ast@kernel.org, daniel@iogearbox.net, maciej.fijalkowski@intel.com, sridhar.samudrala@intel.com, jesse.brandeburg@intel.com, qi.z.zhang@intel.com, kuba@kernel.org, edumazet@google.com, intel-wired-lan@lists.osuosl.org, jonathan.lemon@gmail.com Subject: [RFC PATCH bpf-next 6/9] xsk: propagate napi_id to XDP socket Rx path Date: Wed, 28 Oct 2020 14:34:34 +0100 Message-Id: <20201028133437.212503-7-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028133437.212503-1-bjorn.topel@gmail.com> References: <20201028133437.212503-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Björn Töpel Add napi_id to the xdp_rxq_info structure, and make sure the XDP socket pick up the napi_id in the Rx path. The napi_id is used to find the corresponding NAPI structure for socket busy polling. TODO: Only verified by for the Intel drivers. I'll reach out to the driver authors for a potential non-RFC version. Signed-off-by: Björn Töpel --- drivers/net/ethernet/amazon/ena/ena_netdev.c | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 2 +- .../ethernet/cavium/thunder/nicvf_queues.c | 2 +- .../net/ethernet/freescale/dpaa2/dpaa2-eth.c | 2 +- drivers/net/ethernet/intel/i40e/i40e_txrx.c | 2 +- drivers/net/ethernet/intel/ice/ice_base.c | 4 ++-- drivers/net/ethernet/intel/ice/ice_txrx.c | 2 +- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 2 +- drivers/net/ethernet/marvell/mvneta.c | 2 +- .../net/ethernet/marvell/mvpp2/mvpp2_main.c | 4 ++-- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 2 +- .../ethernet/netronome/nfp/nfp_net_common.c | 2 +- drivers/net/ethernet/qlogic/qede/qede_main.c | 2 +- drivers/net/ethernet/sfc/rx_common.c | 2 +- drivers/net/ethernet/socionext/netsec.c | 2 +- drivers/net/ethernet/ti/cpsw_priv.c | 2 +- drivers/net/hyperv/netvsc.c | 2 +- drivers/net/tun.c | 2 +- drivers/net/veth.c | 2 +- drivers/net/virtio_net.c | 2 +- drivers/net/xen-netfront.c | 2 +- include/net/busy_poll.h | 19 +++++++++++++++---- include/net/xdp.h | 3 ++- net/core/dev.c | 2 +- net/core/xdp.c | 3 ++- net/xdp/xsk.c | 1 + 26 files changed, 44 insertions(+), 30 deletions(-) diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index e8131dadc22c..6ad59f0068f6 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -416,7 +416,7 @@ static int ena_xdp_register_rxq_info(struct ena_ring *rx_ring) { int rc; - rc = xdp_rxq_info_reg(&rx_ring->xdp_rxq, rx_ring->netdev, rx_ring->qid); + rc = xdp_rxq_info_reg(&rx_ring->xdp_rxq, rx_ring->netdev, rx_ring->qid, 0); if (rc) { netif_err(rx_ring->adapter, ifup, rx_ring->netdev, diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index fa147865e33f..5df13387ab74 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -2894,7 +2894,7 @@ static int bnxt_alloc_rx_rings(struct bnxt *bp) if (rc) return rc; - rc = xdp_rxq_info_reg(&rxr->xdp_rxq, bp->dev, i); + rc = xdp_rxq_info_reg(&rxr->xdp_rxq, bp->dev, i, 0); if (rc < 0) return rc; diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c index 7a141ce32e86..f782e6af45e9 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c +++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c @@ -770,7 +770,7 @@ static void nicvf_rcv_queue_config(struct nicvf *nic, struct queue_set *qs, rq->caching = 1; /* Driver have no proper error path for failed XDP RX-queue info reg */ - WARN_ON(xdp_rxq_info_reg(&rq->xdp_rxq, nic->netdev, qidx) < 0); + WARN_ON(xdp_rxq_info_reg(&rq->xdp_rxq, nic->netdev, qidx, 0) < 0); /* Send a mailbox msg to PF to config RQ */ mbx.rq.msg = NIC_MBOX_MSG_RQ_CFG; diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c index cf9400a9886d..40953980e846 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c @@ -3334,7 +3334,7 @@ static int dpaa2_eth_setup_rx_flow(struct dpaa2_eth_priv *priv, return 0; err = xdp_rxq_info_reg(&fq->channel->xdp_rxq, priv->net_dev, - fq->flowid); + fq->flowid, 0); if (err) { dev_err(dev, "xdp_rxq_info_reg failed\n"); return err; diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c index d43ce13a93c9..a3d5bdaca2f5 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c @@ -1436,7 +1436,7 @@ int i40e_setup_rx_descriptors(struct i40e_ring *rx_ring) /* XDP RX-queue info only needed for RX rings exposed to XDP */ if (rx_ring->vsi->type == I40E_VSI_MAIN) { err = xdp_rxq_info_reg(&rx_ring->xdp_rxq, rx_ring->netdev, - rx_ring->queue_index); + rx_ring->queue_index, rx_ring->q_vector->napi.napi_id); if (err < 0) return err; } diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c index fe4320e2d1f2..3124a3bf519a 100644 --- a/drivers/net/ethernet/intel/ice/ice_base.c +++ b/drivers/net/ethernet/intel/ice/ice_base.c @@ -306,7 +306,7 @@ int ice_setup_rx_ctx(struct ice_ring *ring) if (!xdp_rxq_info_is_reg(&ring->xdp_rxq)) /* coverity[check_return] */ xdp_rxq_info_reg(&ring->xdp_rxq, ring->netdev, - ring->q_index); + ring->q_index, ring->q_vector->napi.napi_id); ring->xsk_pool = ice_xsk_pool(ring); if (ring->xsk_pool) { @@ -333,7 +333,7 @@ int ice_setup_rx_ctx(struct ice_ring *ring) /* coverity[check_return] */ xdp_rxq_info_reg(&ring->xdp_rxq, ring->netdev, - ring->q_index); + ring->q_index, ring->q_vector->napi.napi_id); err = xdp_rxq_info_reg_mem_model(&ring->xdp_rxq, MEM_TYPE_PAGE_SHARED, diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c index eae75260fe20..77d5eae6b4c2 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c @@ -483,7 +483,7 @@ int ice_setup_rx_ring(struct ice_ring *rx_ring) if (rx_ring->vsi->type == ICE_VSI_PF && !xdp_rxq_info_is_reg(&rx_ring->xdp_rxq)) if (xdp_rxq_info_reg(&rx_ring->xdp_rxq, rx_ring->netdev, - rx_ring->q_index)) + rx_ring->q_index, rx_ring->q_vector->napi.napi_id)) goto err; return 0; diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index 45ae33e15303..50e6b8b6ba7b 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -6577,7 +6577,7 @@ int ixgbe_setup_rx_resources(struct ixgbe_adapter *adapter, /* XDP RX-queue info */ if (xdp_rxq_info_reg(&rx_ring->xdp_rxq, adapter->netdev, - rx_ring->queue_index) < 0) + rx_ring->queue_index, rx_ring->q_vector->napi.napi_id) < 0) goto err; rx_ring->xdp_prog = adapter->xdp_prog; diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c index 54b0bf574c05..7d0098f4ef9d 100644 --- a/drivers/net/ethernet/marvell/mvneta.c +++ b/drivers/net/ethernet/marvell/mvneta.c @@ -3219,7 +3219,7 @@ static int mvneta_create_page_pool(struct mvneta_port *pp, return err; } - err = xdp_rxq_info_reg(&rxq->xdp_rxq, pp->dev, rxq->id); + err = xdp_rxq_info_reg(&rxq->xdp_rxq, pp->dev, rxq->id, 0); if (err < 0) goto err_free_pp; diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c index f6616c8933ca..ff8729b6c414 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c @@ -2606,11 +2606,11 @@ static int mvpp2_rxq_init(struct mvpp2_port *port, mvpp2_rxq_status_update(port, rxq->id, 0, rxq->size); if (priv->percpu_pools) { - err = xdp_rxq_info_reg(&rxq->xdp_rxq_short, port->dev, rxq->id); + err = xdp_rxq_info_reg(&rxq->xdp_rxq_short, port->dev, rxq->id, 0); if (err < 0) goto err_free_dma; - err = xdp_rxq_info_reg(&rxq->xdp_rxq_long, port->dev, rxq->id); + err = xdp_rxq_info_reg(&rxq->xdp_rxq_long, port->dev, rxq->id, 0); if (err < 0) goto err_unregister_rxq_short; diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c index 502d1b97855c..f561979e5731 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -283,7 +283,7 @@ int mlx4_en_create_rx_ring(struct mlx4_en_priv *priv, ring->log_stride = ffs(ring->stride) - 1; ring->buf_size = ring->size * ring->stride + TXBB_SIZE; - if (xdp_rxq_info_reg(&ring->xdp_rxq, priv->dev, queue_index) < 0) + if (xdp_rxq_info_reg(&ring->xdp_rxq, priv->dev, queue_index, 0) < 0) goto err_ring; tmp = size * roundup_pow_of_two(MLX4_EN_MAX_RX_FRAGS * diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index b150da43adb2..b4acf2f41e84 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -2533,7 +2533,7 @@ nfp_net_rx_ring_alloc(struct nfp_net_dp *dp, struct nfp_net_rx_ring *rx_ring) if (dp->netdev) { err = xdp_rxq_info_reg(&rx_ring->xdp_rxq, dp->netdev, - rx_ring->idx); + rx_ring->idx, rx_ring->r_vec->napi.napi_id); if (err < 0) return err; } diff --git a/drivers/net/ethernet/qlogic/qede/qede_main.c b/drivers/net/ethernet/qlogic/qede/qede_main.c index 05e3a3b60269..b73e95329acd 100644 --- a/drivers/net/ethernet/qlogic/qede/qede_main.c +++ b/drivers/net/ethernet/qlogic/qede/qede_main.c @@ -1762,7 +1762,7 @@ static void qede_init_fp(struct qede_dev *edev) /* Driver have no error path from here */ WARN_ON(xdp_rxq_info_reg(&fp->rxq->xdp_rxq, edev->ndev, - fp->rxq->rxq_id) < 0); + fp->rxq->rxq_id. 0) < 0); if (xdp_rxq_info_reg_mem_model(&fp->rxq->xdp_rxq, MEM_TYPE_PAGE_ORDER0, diff --git a/drivers/net/ethernet/sfc/rx_common.c b/drivers/net/ethernet/sfc/rx_common.c index 19cf7cac1e6e..68fc7d317693 100644 --- a/drivers/net/ethernet/sfc/rx_common.c +++ b/drivers/net/ethernet/sfc/rx_common.c @@ -262,7 +262,7 @@ void efx_init_rx_queue(struct efx_rx_queue *rx_queue) /* Initialise XDP queue information */ rc = xdp_rxq_info_reg(&rx_queue->xdp_rxq_info, efx->net_dev, - rx_queue->core_index); + rx_queue->core_index, 0); if (rc) { netif_err(efx, rx_err, efx->net_dev, diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c index 1503cc9ec6e2..80ab24658e87 100644 --- a/drivers/net/ethernet/socionext/netsec.c +++ b/drivers/net/ethernet/socionext/netsec.c @@ -1304,7 +1304,7 @@ static int netsec_setup_rx_dring(struct netsec_priv *priv) goto err_out; } - err = xdp_rxq_info_reg(&dring->xdp_rxq, priv->ndev, 0); + err = xdp_rxq_info_reg(&dring->xdp_rxq, priv->ndev, 0, 0); if (err) goto err_out; diff --git a/drivers/net/ethernet/ti/cpsw_priv.c b/drivers/net/ethernet/ti/cpsw_priv.c index 51cc29f39038..d8f287c88d77 100644 --- a/drivers/net/ethernet/ti/cpsw_priv.c +++ b/drivers/net/ethernet/ti/cpsw_priv.c @@ -1189,7 +1189,7 @@ static int cpsw_ndev_create_xdp_rxq(struct cpsw_priv *priv, int ch) pool = cpsw->page_pool[ch]; rxq = &priv->xdp_rxq[ch]; - ret = xdp_rxq_info_reg(rxq, priv->ndev, ch); + ret = xdp_rxq_info_reg(rxq, priv->ndev, ch, 0); if (ret) return ret; diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c index 0c3de94b5178..fa8341f8359a 100644 --- a/drivers/net/hyperv/netvsc.c +++ b/drivers/net/hyperv/netvsc.c @@ -1499,7 +1499,7 @@ struct netvsc_device *netvsc_device_add(struct hv_device *device, u64_stats_init(&nvchan->tx_stats.syncp); u64_stats_init(&nvchan->rx_stats.syncp); - ret = xdp_rxq_info_reg(&nvchan->xdp_rxq, ndev, i); + ret = xdp_rxq_info_reg(&nvchan->xdp_rxq, ndev, i, 0); if (ret) { netdev_err(ndev, "xdp_rxq_info_reg fail: %d\n", ret); diff --git a/drivers/net/tun.c b/drivers/net/tun.c index be69d272052f..f2541d645707 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -791,7 +791,7 @@ static int tun_attach(struct tun_struct *tun, struct file *file, } else { /* Setup XDP RX-queue info, for new tfile getting attached */ err = xdp_rxq_info_reg(&tfile->xdp_rxq, - tun->dev, tfile->queue_index); + tun->dev, tfile->queue_index, 0); if (err < 0) goto out; err = xdp_rxq_info_reg_mem_model(&tfile->xdp_rxq, diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 8c737668008a..04d20e9d8431 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -926,7 +926,7 @@ static int veth_enable_xdp(struct net_device *dev) for (i = 0; i < dev->real_num_rx_queues; i++) { struct veth_rq *rq = &priv->rq[i]; - err = xdp_rxq_info_reg(&rq->xdp_rxq, dev, i); + err = xdp_rxq_info_reg(&rq->xdp_rxq, dev, i, 0); if (err < 0) goto err_rxq_reg; diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 21b71148c532..d71fe41595b7 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -1485,7 +1485,7 @@ static int virtnet_open(struct net_device *dev) if (!try_fill_recv(vi, &vi->rq[i], GFP_KERNEL)) schedule_delayed_work(&vi->refill, 0); - err = xdp_rxq_info_reg(&vi->rq[i].xdp_rxq, dev, i); + err = xdp_rxq_info_reg(&vi->rq[i].xdp_rxq, dev, i, 0); if (err < 0) return err; diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c index 3e9895bec15f..28714a48f5d0 100644 --- a/drivers/net/xen-netfront.c +++ b/drivers/net/xen-netfront.c @@ -2014,7 +2014,7 @@ static int xennet_create_page_pool(struct netfront_queue *queue) } err = xdp_rxq_info_reg(&queue->xdp_rxq, queue->info->netdev, - queue->id); + queue->id, 0); if (err) { netdev_err(queue->info->netdev, "xdp_rxq_info_reg failed\n"); goto err_free_pp; diff --git a/include/net/busy_poll.h b/include/net/busy_poll.h index c6c413d3824d..262f60065355 100644 --- a/include/net/busy_poll.h +++ b/include/net/busy_poll.h @@ -148,14 +148,25 @@ static inline void sk_mark_napi_id(struct sock *sk, const struct sk_buff *skb) sk_rx_queue_set(sk, skb); } -/* variant used for unconnected sockets */ -static inline void sk_mark_napi_id_once(struct sock *sk, - const struct sk_buff *skb) +static inline void __sk_mark_napi_id_once_xdp(struct sock *sk, unsigned int napi_id) { #ifdef CONFIG_NET_RX_BUSY_POLL if (!READ_ONCE(sk->sk_napi_id)) - WRITE_ONCE(sk->sk_napi_id, skb->napi_id); + WRITE_ONCE(sk->sk_napi_id, napi_id); #endif } +/* variant used for unconnected sockets */ +static inline void sk_mark_napi_id_once(struct sock *sk, + const struct sk_buff *skb) +{ + __sk_mark_napi_id_once_xdp(sk, skb->napi_id); +} + +static inline void sk_mark_napi_id_once_xdp(struct sock *sk, + const struct xdp_buff *xdp) +{ + __sk_mark_napi_id_once_xdp(sk, xdp->rxq->napi_id); +} + #endif /* _LINUX_NET_BUSY_POLL_H */ diff --git a/include/net/xdp.h b/include/net/xdp.h index 3814fb631d52..4d4255a94773 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -59,6 +59,7 @@ struct xdp_rxq_info { u32 queue_index; u32 reg_state; struct xdp_mem_info mem; + unsigned int napi_id; } ____cacheline_aligned; /* perf critical, avoid false-sharing */ struct xdp_txq_info { @@ -211,7 +212,7 @@ static inline void xdp_release_frame(struct xdp_frame *xdpf) } int xdp_rxq_info_reg(struct xdp_rxq_info *xdp_rxq, - struct net_device *dev, u32 queue_index); + struct net_device *dev, u32 queue_index, unsigned int napi_id); void xdp_rxq_info_unreg(struct xdp_rxq_info *xdp_rxq); void xdp_rxq_info_unused(struct xdp_rxq_info *xdp_rxq); bool xdp_rxq_info_is_reg(struct xdp_rxq_info *xdp_rxq); diff --git a/net/core/dev.c b/net/core/dev.c index b34520acaa7f..ad3261be5e21 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -9834,7 +9834,7 @@ static int netif_alloc_rx_queues(struct net_device *dev) rx[i].dev = dev; /* XDP RX-queue setup */ - err = xdp_rxq_info_reg(&rx[i].xdp_rxq, dev, i); + err = xdp_rxq_info_reg(&rx[i].xdp_rxq, dev, i, 0); if (err < 0) goto err_rxq_info; } diff --git a/net/core/xdp.c b/net/core/xdp.c index 48aba933a5a8..7cca7cb5b65f 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -158,7 +158,7 @@ static void xdp_rxq_info_init(struct xdp_rxq_info *xdp_rxq) /* Returns 0 on success, negative on failure */ int xdp_rxq_info_reg(struct xdp_rxq_info *xdp_rxq, - struct net_device *dev, u32 queue_index) + struct net_device *dev, u32 queue_index, unsigned int napi_id) { if (xdp_rxq->reg_state == REG_STATE_UNUSED) { WARN(1, "Driver promised not to register this"); @@ -179,6 +179,7 @@ int xdp_rxq_info_reg(struct xdp_rxq_info *xdp_rxq, xdp_rxq_info_init(xdp_rxq); xdp_rxq->dev = dev; xdp_rxq->queue_index = queue_index; + xdp_rxq->napi_id = napi_id; xdp_rxq->reg_state = REG_STATE_REGISTERED; return 0; diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index da649b4f377c..0b825612d895 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -233,6 +233,7 @@ static int xsk_rcv(struct xdp_sock *xs, struct xdp_buff *xdp, if (xs->dev != xdp->rxq->dev || xs->queue_id != xdp->rxq->queue_index) return -EINVAL; + sk_mark_napi_id_once_xdp(&xs->sk, xdp); len = xdp->data_end - xdp->data; return xdp->rxq->mem.type == MEM_TYPE_XSK_BUFF_POOL ? From patchwork Wed Oct 28 13:34:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11864013 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8455BC55179 for ; Wed, 28 Oct 2020 23:02:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2A23820756 for ; Wed, 28 Oct 2020 23:02:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GxjsphOX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389990AbgJ1XCZ (ORCPT ); Wed, 28 Oct 2020 19:02:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389508AbgJ1XBo (ORCPT ); Wed, 28 Oct 2020 19:01:44 -0400 Received: from mail-io1-xd43.google.com (mail-io1-xd43.google.com [IPv6:2607:f8b0:4864:20::d43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52114C0613CF; Wed, 28 Oct 2020 16:01:44 -0700 (PDT) Received: by mail-io1-xd43.google.com with SMTP id q25so1325282ioh.4; Wed, 28 Oct 2020 16:01:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pjXiaJJaFcfh/d5Hg7T1ivvJjPLwr7C0xxhr2qpyt5M=; b=GxjsphOX2zEbZqqdQjCwAyYsCORJvqE7NNfEH29SRF58gKtpndwV3xboOfeL6smVhr 4g9HGrTfQyhg0GU/TbXK7hJt3NIiNdA2V8nV9goYxAa4mJST3cVcz/KhnqPh8EqWynDC sriF1cbKq5FLWJnxDmMDZg+khJmAWWy+d1pAz51gd8d4ihLk9EF2itAqhARQNy8Jq0ii SkL+OfhtU1GE//eVEggFflv5XZg0HQvYBN4ZTWsrISGG2pRxnVmvt9J1hvabHy0olmuR 915k6mYOzmj3DXKD4SYLXVGkIPhldX8Acsy8lWnmYbC/VTdrJNcf9BwKVJw9n8v8xx+g 5u5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pjXiaJJaFcfh/d5Hg7T1ivvJjPLwr7C0xxhr2qpyt5M=; b=YJy7437eVW9YzTJ0IPFEUl/kL4YDYmd4d2Nuee5nC0suu8uRW2DA14fUcEFcEsHSgv G+pm4qjyq7TxWm+kkXNrrXk5DfcwOE2z3dMc5FCf0mQMpgR0rjq9uzmuIuyqr6qU7JdG yUrlnYeBWiT0B8OE1g8W/MCeZVzk7vXG80vOeFOZAiQMI3NKpMA+qUr+6nngP5UFFEtj I/CuS1RIFphwgBgInYZJcR4ZTL7a8M3647Vxk/x0nQSbOxb+mJScZyohMWYjJJM7EDPd P0XJ2Y1iNcX9xBIS7QrDy3kQa0KQ7vzhEj0oR01e5t0+vCnb/QZ7Ex29jViHNwwn0665 /O/g== X-Gm-Message-State: AOAM533MEGDv7unlDVHTr9eMU3A32cwzLocfvNG3TESA6NbCF0E4bnYP zO47KQPrfysLJhLWcClmdHQpY17k+wpiAbW4 X-Google-Smtp-Source: ABdhPJyIdK45Sv7Xjy21+qnn5zK3LkhfJoLsJXWYGmLi0HlZiH+Y7k2D41upxa+5dTuxo4y475uc9g== X-Received: by 2002:a63:c211:: with SMTP id b17mr6356557pgd.195.1603892144863; Wed, 28 Oct 2020 06:35:44 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q14sm5935393pjp.43.2020.10.28.06.35.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2020 06:35:44 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, ast@kernel.org, daniel@iogearbox.net, maciej.fijalkowski@intel.com, sridhar.samudrala@intel.com, jesse.brandeburg@intel.com, qi.z.zhang@intel.com, kuba@kernel.org, edumazet@google.com, intel-wired-lan@lists.osuosl.org, jonathan.lemon@gmail.com Subject: [RFC PATCH bpf-next 7/9] samples/bpf: use recvfrom() in xdpsock Date: Wed, 28 Oct 2020 14:34:35 +0100 Message-Id: <20201028133437.212503-8-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028133437.212503-1-bjorn.topel@gmail.com> References: <20201028133437.212503-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Björn Töpel Start using recvfrom() the rxdrop scenario. Signed-off-by: Björn Töpel --- samples/bpf/xdpsock_user.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c index 1149e94ca32f..96d0b6482ac4 100644 --- a/samples/bpf/xdpsock_user.c +++ b/samples/bpf/xdpsock_user.c @@ -1172,7 +1172,7 @@ static inline void complete_tx_only(struct xsk_socket_info *xsk, } } -static void rx_drop(struct xsk_socket_info *xsk, struct pollfd *fds) +static void rx_drop(struct xsk_socket_info *xsk) { unsigned int rcvd, i; u32 idx_rx = 0, idx_fq = 0; @@ -1182,7 +1182,7 @@ static void rx_drop(struct xsk_socket_info *xsk, struct pollfd *fds) if (!rcvd) { if (xsk_ring_prod__needs_wakeup(&xsk->umem->fq)) { xsk->app_stats.rx_empty_polls++; - ret = poll(fds, num_socks, opt_timeout); + recvfrom(xsk_socket__fd(xsk->xsk), NULL, 0, MSG_DONTWAIT, NULL, NULL); } return; } @@ -1193,7 +1193,7 @@ static void rx_drop(struct xsk_socket_info *xsk, struct pollfd *fds) exit_with_error(-ret); if (xsk_ring_prod__needs_wakeup(&xsk->umem->fq)) { xsk->app_stats.fill_fail_polls++; - ret = poll(fds, num_socks, opt_timeout); + recvfrom(xsk_socket__fd(xsk->xsk), NULL, 0, MSG_DONTWAIT, NULL, NULL); } ret = xsk_ring_prod__reserve(&xsk->umem->fq, rcvd, &idx_fq); } @@ -1235,7 +1235,7 @@ static void rx_drop_all(void) } for (i = 0; i < num_socks; i++) - rx_drop(xsks[i], fds); + rx_drop(xsks[i]); if (benchmark_done) break; From patchwork Wed Oct 28 13:34:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11863853 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B6ADC56202 for ; Wed, 28 Oct 2020 22:36:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AA3C420759 for ; Wed, 28 Oct 2020 22:36:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UYBUdFIv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388349AbgJ1WgP (ORCPT ); Wed, 28 Oct 2020 18:36:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387836AbgJ1WgN (ORCPT ); Wed, 28 Oct 2020 18:36:13 -0400 Received: from mail-pl1-x641.google.com (mail-pl1-x641.google.com [IPv6:2607:f8b0:4864:20::641]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4259C0613D1; Wed, 28 Oct 2020 15:36:12 -0700 (PDT) Received: by mail-pl1-x641.google.com with SMTP id f21so343978plr.5; Wed, 28 Oct 2020 15:36:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wUafzXmN1fiYZSw7cbOBZF12NArhlfVl1gsXYHPJ9hM=; b=UYBUdFIvxBj29SgLEr+PGFX5gVUE8FB3CJJRd/bfzpIQQ5KYJtggQpbY18+Pv5Cp9U GETiiq9f9VgN6/0CGEKAuvp5kSsWipRguLnJ8NAhl0vX8PWSPjbxjcbxJBoCoHvFafwf W23JBMb9GkgESvHW/T5YT+zmhkBYllOpcFPAMyYGUWadanyIQZ3LpPrVWAW+fIzBF9lo 0kj8lQoCHvovgnZmqayUr/fUgssea4HlPXT4VbSIgJwyvU5Fj6YscXekqqCglCv5NoiE JzavnYIycuqqz9tI8hlAoIZdaUKxNEH9tmqgEBH1OefZ3RvsPIECbr+xvUzf5vWy/UOP zyuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wUafzXmN1fiYZSw7cbOBZF12NArhlfVl1gsXYHPJ9hM=; b=pOGP/IATimki/PXuv+nIzwUKt0EeSalHEGzZCEo1+drjQzRZOi5yRji9ba+LCt28zJ ++glzABmosFkajXnd9aBUqPYQE7SeXfNEs6w2McAOwiTyQo1EKBM/AL1kXMmX6besh29 y6ErutnKF6mlCI0pvgk4GKrWAMXJKdFC4U7RZmqu5cfOIIp9S41Xh5/xYWKgQrS8mNCh WWaK9Cg+TFY4G6zntBtO8zGg1qXH20L/ARG5DCbziOg96sWIG9aAB3rsOYN3+zQDSnci Q0eUb5YKMf7/aNkvax9d420T9IVoOu8nAV3Il/jJ/F9IYYrqqVIRwOtUBhLRXjTnoMRF QpAA== X-Gm-Message-State: AOAM533TpTFNAmh+rpnBCqPsw0nsgT9UfrzNknQVPFzyBhHk4IOi2wGq 8+N9ThxhkoY9WLbIO3mIf6jQLng9/yz0YIw7 X-Google-Smtp-Source: ABdhPJzErfbq9Axb2rnwLp5dHWjy9HVcmEllOuyy9jBLW7mTYF4ZduBV2MEQaTfs1jwmX5emGT7rtA== X-Received: by 2002:aa7:9d03:0:b029:164:2981:2331 with SMTP id k3-20020aa79d030000b029016429812331mr7939350pfp.0.1603892150737; Wed, 28 Oct 2020 06:35:50 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q14sm5935393pjp.43.2020.10.28.06.35.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2020 06:35:50 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, ast@kernel.org, daniel@iogearbox.net, maciej.fijalkowski@intel.com, sridhar.samudrala@intel.com, jesse.brandeburg@intel.com, qi.z.zhang@intel.com, kuba@kernel.org, edumazet@google.com, intel-wired-lan@lists.osuosl.org, jonathan.lemon@gmail.com Subject: [RFC PATCH bpf-next 8/9] samples/bpf: add busy-poll support to xdpsock Date: Wed, 28 Oct 2020 14:34:36 +0100 Message-Id: <20201028133437.212503-9-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028133437.212503-1-bjorn.topel@gmail.com> References: <20201028133437.212503-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Björn Töpel Add a new option to xdpsock, 'B', for busy-polling. This option will also set the batching size, 'b' option, to the busy-poll budget. Signed-off-by: Björn Töpel --- samples/bpf/xdpsock_user.c | 40 +++++++++++++++++++++++++++++++------- 1 file changed, 33 insertions(+), 7 deletions(-) diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c index 96d0b6482ac4..7ef2c01a1094 100644 --- a/samples/bpf/xdpsock_user.c +++ b/samples/bpf/xdpsock_user.c @@ -95,6 +95,7 @@ static int opt_timeout = 1000; static bool opt_need_wakeup = true; static u32 opt_num_xsks = 1; static u32 prog_id; +static bool opt_busy_poll; struct xsk_ring_stats { unsigned long rx_npkts; @@ -911,6 +912,7 @@ static struct option long_options[] = { {"quiet", no_argument, 0, 'Q'}, {"app-stats", no_argument, 0, 'a'}, {"irq-string", no_argument, 0, 'I'}, + {"busy-poll", no_argument, 0, 'B'}, {0, 0, 0, 0} }; @@ -949,6 +951,7 @@ static void usage(const char *prog) " -Q, --quiet Do not display any stats.\n" " -a, --app-stats Display application (syscall) statistics.\n" " -I, --irq-string Display driver interrupt statistics for interface associated with irq-string.\n" + " -B, --busy-poll Busy poll.\n" "\n"; fprintf(stderr, str, prog, XSK_UMEM__DEFAULT_FRAME_SIZE, opt_batch_size, MIN_PKT_SIZE, MIN_PKT_SIZE, @@ -964,7 +967,7 @@ static void parse_command_line(int argc, char **argv) opterr = 0; for (;;) { - c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:xQaI:", + c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:xQaI:B", long_options, &option_index); if (c == -1) break; @@ -1062,7 +1065,9 @@ static void parse_command_line(int argc, char **argv) fprintf(stderr, "ERROR: Failed to get irqs for %s\n", opt_irq_str); usage(basename(argv[0])); } - + break; + case 'B': + opt_busy_poll = 1; break; default: usage(basename(argv[0])); @@ -1132,7 +1137,7 @@ static inline void complete_tx_l2fwd(struct xsk_socket_info *xsk, while (ret != rcvd) { if (ret < 0) exit_with_error(-ret); - if (xsk_ring_prod__needs_wakeup(&umem->fq)) { + if (opt_busy_poll || xsk_ring_prod__needs_wakeup(&umem->fq)) { xsk->app_stats.fill_fail_polls++; ret = poll(fds, num_socks, opt_timeout); } @@ -1180,7 +1185,7 @@ static void rx_drop(struct xsk_socket_info *xsk) rcvd = xsk_ring_cons__peek(&xsk->rx, opt_batch_size, &idx_rx); if (!rcvd) { - if (xsk_ring_prod__needs_wakeup(&xsk->umem->fq)) { + if (opt_busy_poll || xsk_ring_prod__needs_wakeup(&xsk->umem->fq)) { xsk->app_stats.rx_empty_polls++; recvfrom(xsk_socket__fd(xsk->xsk), NULL, 0, MSG_DONTWAIT, NULL, NULL); } @@ -1191,7 +1196,7 @@ static void rx_drop(struct xsk_socket_info *xsk) while (ret != rcvd) { if (ret < 0) exit_with_error(-ret); - if (xsk_ring_prod__needs_wakeup(&xsk->umem->fq)) { + if (opt_busy_poll || xsk_ring_prod__needs_wakeup(&xsk->umem->fq)) { xsk->app_stats.fill_fail_polls++; recvfrom(xsk_socket__fd(xsk->xsk), NULL, 0, MSG_DONTWAIT, NULL, NULL); } @@ -1342,7 +1347,7 @@ static void l2fwd(struct xsk_socket_info *xsk, struct pollfd *fds) rcvd = xsk_ring_cons__peek(&xsk->rx, opt_batch_size, &idx_rx); if (!rcvd) { - if (xsk_ring_prod__needs_wakeup(&xsk->umem->fq)) { + if (opt_busy_poll || xsk_ring_prod__needs_wakeup(&xsk->umem->fq)) { xsk->app_stats.rx_empty_polls++; ret = poll(fds, num_socks, opt_timeout); } @@ -1354,7 +1359,7 @@ static void l2fwd(struct xsk_socket_info *xsk, struct pollfd *fds) if (ret < 0) exit_with_error(-ret); complete_tx_l2fwd(xsk, fds); - if (xsk_ring_prod__needs_wakeup(&xsk->tx)) { + if (opt_busy_poll || xsk_ring_prod__needs_wakeup(&xsk->tx)) { xsk->app_stats.tx_wakeup_sendtos++; kick_tx(xsk); } @@ -1461,6 +1466,24 @@ static void enter_xsks_into_map(struct bpf_object *obj) } } +static void apply_setsockopt(struct xsk_socket_info *xsk) +{ + int sock_opt; + + if (!opt_busy_poll) + return; + + sock_opt = 1; + if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_BIAS_BUSY_POLL, + (void *)&sock_opt, sizeof(sock_opt)) < 0) + exit_with_error(errno); + + sock_opt = 20; // randomly picked :-P + if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_BUSY_POLL, + (void *)&sock_opt, sizeof(sock_opt)) < 0) + exit_with_error(errno); +} + int main(int argc, char **argv) { struct rlimit r = {RLIM_INFINITY, RLIM_INFINITY}; @@ -1502,6 +1525,9 @@ int main(int argc, char **argv) for (i = 0; i < opt_num_xsks; i++) xsks[num_socks++] = xsk_configure_socket(umem, rx, tx); + for (i = 0; i < opt_num_xsks; i++) + apply_setsockopt(xsks[i]); + if (opt_bench == BENCH_TXONLY) { gen_eth_hdr_data(); From patchwork Wed Oct 28 13:34:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 11863879 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09559C388F7 for ; Wed, 28 Oct 2020 22:38:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8D9A02076A for ; Wed, 28 Oct 2020 22:38:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="E8ql5xGf" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388953AbgJ1Wil (ORCPT ); Wed, 28 Oct 2020 18:38:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57144 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388951AbgJ1Wib (ORCPT ); Wed, 28 Oct 2020 18:38:31 -0400 Received: from mail-oi1-x243.google.com (mail-oi1-x243.google.com [IPv6:2607:f8b0:4864:20::243]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F609C0613CF; Wed, 28 Oct 2020 15:38:31 -0700 (PDT) Received: by mail-oi1-x243.google.com with SMTP id x203so1236078oia.10; Wed, 28 Oct 2020 15:38:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oRd9D9KAGEvSUVDHFx51Z3pe4W3JS4rlMi+zj5tnu1o=; b=E8ql5xGfRHpoTKr5i0U4XM9uzxC6u2e9Cg3/yWktFZDdSuRO6Udnjb7bA3rCe9EnUG 8cIQMmKu7BUC4a0CJ8oFGmBVSKHR0QRNlTOVJQG4a4IhAJW2NHV1grRhMQUmzccVLcC9 onsJs2Ihyxv3p5mJNwmpmmsx9jMdvIbYUgH1mvLb2cogBg8iLX58dBUz+r82kCKHVvnb XumzNtq+C05tnupu1y3h8jPs/EIisyHYrMzfPJ+h0052DweivHzgByZDQi8JNFRKysny FHHfLpjyjztIKOBRKGcDOuk6qtN2e+9lTmDCOG764V61f0M6J7B3i0hqr0ttd07PAhbv 8LMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oRd9D9KAGEvSUVDHFx51Z3pe4W3JS4rlMi+zj5tnu1o=; b=qVh3tAo0s2weUv26rBSWrKJXKFS0KIK9QXJgu1sEQfRg1Bza+gJ84GIh/kbsx9oJfS KUCxWJ3YanUEC9HYtMVe8AV6pYcnOQAz3FrmJJB6Hf8hCp/K/QWbHOVGdr867XFuap7y kZFGBvgkcOztNEinfs8yNY0eS6yHuHcGKt4nLGfh61z8cqb2y1K+KNKe/Zq5k8xHzuGR D8bRk/5yu7xLBwveJoYl4wQ8WRqtQU4XMng7bdSon/H0mGu0DJmZax3OgU8WQhFUptI/ fH/UJNS263LLUWdJI5Ao59mRAaxsWkh56YFtyd6gM7DXu3NV4g6zghFzfzzm4RqwU/qz i2Iw== X-Gm-Message-State: AOAM532XU2g56jSzPmCWHxYCKFfObNFTVl2No6024+DUU8jfLpj1GPjb Z2g+nDf2afvUXgo/5QNUAjIRvVhyeNlFYbj8 X-Google-Smtp-Source: ABdhPJw08anLcn91mXmmfsPzN+W5dpD39hXzuhPgRbUE4/T21wkwPxdeaE55dA/3l8zuJN2Sj8xr2A== X-Received: by 2002:a17:90b:23c2:: with SMTP id md2mr2718827pjb.205.1603892156901; Wed, 28 Oct 2020 06:35:56 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q14sm5935393pjp.43.2020.10.28.06.35.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Oct 2020 06:35:56 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, ast@kernel.org, daniel@iogearbox.net, maciej.fijalkowski@intel.com, sridhar.samudrala@intel.com, jesse.brandeburg@intel.com, qi.z.zhang@intel.com, kuba@kernel.org, edumazet@google.com, intel-wired-lan@lists.osuosl.org, jonathan.lemon@gmail.com Subject: [RFC PATCH bpf-next 9/9] samples/bpf: add option to set the busy-poll budget Date: Wed, 28 Oct 2020 14:34:37 +0100 Message-Id: <20201028133437.212503-10-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028133437.212503-1-bjorn.topel@gmail.com> References: <20201028133437.212503-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC From: Björn Töpel Support for the SO_BUSY_POLL_BUDGET setsockopt, via the batching option ('b'). Signed-off-by: Björn Töpel --- samples/bpf/xdpsock_user.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c index 7ef2c01a1094..948faada96d5 100644 --- a/samples/bpf/xdpsock_user.c +++ b/samples/bpf/xdpsock_user.c @@ -1482,6 +1482,11 @@ static void apply_setsockopt(struct xsk_socket_info *xsk) if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_BUSY_POLL, (void *)&sock_opt, sizeof(sock_opt)) < 0) exit_with_error(errno); + + sock_opt = opt_batch_size; + if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_BUSY_POLL_BUDGET, + (void *)&sock_opt, sizeof(sock_opt)) < 0) + exit_with_error(errno); } int main(int argc, char **argv)