From patchwork Wed Apr 13 15:30:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812143 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBC87C4332F for ; Wed, 13 Apr 2022 15:30:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236442AbiDMPcz (ORCPT ); Wed, 13 Apr 2022 11:32:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232010AbiDMPcy (ORCPT ); Wed, 13 Apr 2022 11:32:54 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBC94377FC; Wed, 13 Apr 2022 08:30:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863831; x=1681399831; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hwRtOeN1pipEIxhLjSgAdaALV7fhjpXgexFd9B2XHYU=; b=cy3a0xy9zU1KQl7KpizsUio71EVI/VKXUCd9zcNytcb6i4IQkwDcw+Ji OKEqj/JMIxIJp80hFYAzOHIaY9GnH7AMX0pWg6m97shH97areRvPC9RTU BQcSXNUTkoPP3uckvFmsdyVgfarCeClZW/Q0BCgaz0qeym2vWL5ClraZU Gp+67rzbvM8SA6wi1xRpGVqwE5eE94XcRoI6u2iSwSZWzwIyTsqU8VL1a +ftCvpImQqxc9xwUgf2ks+aSHTcg5oLRj++b1S7m+hiL2PJnZOJOgLFcw B1A0385gmw+nbe7huTNzxWI+y0cmTTOYd9FrsA36cE3uEWqK9UyHkS5It g==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544157" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544157" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:30:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318190" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:30:28 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Jesper Dangaard Brouer , Maciej Fijalkowski Subject: [PATCH v2 bpf-next 01/14] xsk: improve xdp_do_redirect() error codes Date: Wed, 13 Apr 2022 17:30:02 +0200 Message-Id: <20220413153015.453864-2-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Björn Töpel The error codes returned by xdp_do_redirect() when redirecting a frame to an AF_XDP socket has not been very useful. A driver could not distinguish between different errors. Prior this change the following codes where used: Socket not bound or incorrect queue/netdev: EINVAL XDP frame/AF_XDP buffer size mismatch: ENOSPC Could not allocate buffer (copy mode): ENOSPC AF_XDP Rx buffer full: ENOSPC After this change: Socket not bound or incorrect queue/netdev: EINVAL XDP frame/AF_XDP buffer size mismatch: ENOSPC Could not allocate buffer (copy mode): ENOMEM AF_XDP Rx buffer full: ENOBUFS An AF_XDP zero-copy driver can now potentially determine if the failure was due to a full Rx buffer, and if so stop processing more frames, yielding to the userland AF_XDP application. Acked-by: Jesper Dangaard Brouer Signed-off-by: Björn Töpel Signed-off-by: Maciej Fijalkowski --- net/xdp/xsk.c | 2 +- net/xdp/xsk_queue.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index 2c34caee0fd1..f75e121073e7 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -184,7 +184,7 @@ static int __xsk_rcv(struct xdp_sock *xs, struct xdp_buff *xdp) xsk_xdp = xsk_buff_alloc(xs->pool); if (!xsk_xdp) { xs->rx_dropped++; - return -ENOSPC; + return -ENOMEM; } xsk_copy_xdp(xsk_xdp, xdp, len); diff --git a/net/xdp/xsk_queue.h b/net/xdp/xsk_queue.h index 801cda5d1938..644479e65578 100644 --- a/net/xdp/xsk_queue.h +++ b/net/xdp/xsk_queue.h @@ -382,7 +382,7 @@ static inline int xskq_prod_reserve_desc(struct xsk_queue *q, u32 idx; if (xskq_prod_is_full(q)) - return -ENOSPC; + return -ENOBUFS; /* A, matches D */ idx = q->cached_prod++ & q->ring_mask; From patchwork Wed Apr 13 15:30:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812145 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DE3CC433F5 for ; Wed, 13 Apr 2022 15:30:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236513AbiDMPc5 (ORCPT ); Wed, 13 Apr 2022 11:32:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232010AbiDMPcz (ORCPT ); Wed, 13 Apr 2022 11:32:55 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9907F377F9; Wed, 13 Apr 2022 08:30:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863834; x=1681399834; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=s2NRM68uI3Xf1aRNp8SErdefczkOuptyPRLP2SdbgSY=; b=E6O9HCPm0mQiaCw5obIhPcz3AMKwJG9eM1ZR93Bh2lCOTH1uPrGXAoP8 XI5PzS/hLLSlCB5VwbLLP9eqCYVESx5qQUSyaG04olJjSxwnQofNHZXJc HSK0kdncU+rphfgHWZGR+01fUcDo+mPDRTP62x08hh5c8kNz87VGflHx/ 78/TYg6rAVoL+eoRg2brGkBIGUCouw1Cdky3jcVpK97bAeX1tC4HUWxWo zqQdWKPUflFj5DvbwF5RMM5dB+nnpZ/1IH4QIaiuy4/qawma40GzVjyRw T/XtpSk/H52FkYvORjAmh47NgxLFwPHBbO40HwBiotpN0vj1GXPHvxceB g==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544175" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544175" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:30:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318202" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:30:31 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski , Jesper Dangaard Brouer Subject: [PATCH v2 bpf-next 02/14] xsk: diversify return codes in xsk_rcv_check() Date: Wed, 13 Apr 2022 17:30:03 +0200 Message-Id: <20220413153015.453864-3-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Inspired by patch that made xdp_do_redirect() return values for XSKMAP more meaningful, return -ENXIO instead of -EINVAL for socket being unbound in xsk_rcv_check() as this is the usual value that is returned for such event. In turn, it is now possible to easily distinguish what went wrong, which is a bit harder when for both cases checked, -EINVAL was returned. Return codes can be counted in a nice way via bpftrace oneliner that Jesper has shown: bpftrace -e 'tracepoint:xdp:xdp_redirect* {@err[-args->err] = count();}' Acked-by: Jesper Dangaard Brouer Signed-off-by: Maciej Fijalkowski --- net/xdp/xsk.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index f75e121073e7..040c73345b7c 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -217,7 +217,7 @@ static bool xsk_is_bound(struct xdp_sock *xs) static int xsk_rcv_check(struct xdp_sock *xs, struct xdp_buff *xdp) { if (!xsk_is_bound(xs)) - return -EINVAL; + return -ENXIO; if (xs->dev != xdp->rxq->dev || xs->queue_id != xdp->rxq->queue_index) return -EINVAL; From patchwork Wed Apr 13 15:30:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812146 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B133C433FE for ; Wed, 13 Apr 2022 15:30:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236522AbiDMPdA (ORCPT ); Wed, 13 Apr 2022 11:33:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232010AbiDMPc6 (ORCPT ); Wed, 13 Apr 2022 11:32:58 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6333337A03; Wed, 13 Apr 2022 08:30:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863837; x=1681399837; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hLCIV0xDKVUO54k59dWJqyAkxQMhB+VY6VyVkCjIRwo=; b=iopa6syL2Jl+eTVQnSKNx5iAYCX7krRBa0SqYAqO/W4WPominObYdxmr awBBW6aptFylIT6X0v6ee0vSYXJbqTuLVfI8KcK6zylBypo1xt7I+aNZD Z+fWN9werTVOOfGT4G8c3LZbVaIRPAfxiPc3EHi4O8s0mHRlI+1OWabfo alUT24oSUIhUa40Xk+L33RL/Hd/48tCKAps3wCTDtkZFdQEcaATysF047 DnE077e99ZmMviil/7u6GDkxD8ADraClBxLbeXyy03YxxBbdl1R0chKX2 xaX4ySzpI2y+HB+82Z6r8Zmlo3XyF7VPot0/OirgNAvfY8C9Pw42JexmN g==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544193" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544193" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:30:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318227" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:30:34 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski , Jesper Dangaard Brouer Subject: [PATCH v2 bpf-next 03/14] ice: xsk: decorate ICE_XDP_REDIR with likely() Date: Wed, 13 Apr 2022 17:30:04 +0200 Message-Id: <20220413153015.453864-4-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net ice_run_xdp_zc() suggests to compiler that XDP_REDIRECT is the most probable action returned from BPF program that AF_XDP has in its pipeline. Let's also bring this suggestion up to the callsite of ice_run_xdp_zc() so that compiler will be able to generate more optimized code which in turn will make branch predictor happy. Suggested-by: Jesper Dangaard Brouer Signed-off-by: Maciej Fijalkowski --- drivers/net/ethernet/intel/ice/ice_xsk.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c index 866ee4df9671..e9ff05de0084 100644 --- a/drivers/net/ethernet/intel/ice/ice_xsk.c +++ b/drivers/net/ethernet/intel/ice/ice_xsk.c @@ -629,18 +629,19 @@ int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring, int budget) xsk_buff_dma_sync_for_cpu(xdp, rx_ring->xsk_pool); xdp_res = ice_run_xdp_zc(rx_ring, xdp, xdp_prog, xdp_ring); - if (xdp_res) { - if (xdp_res & (ICE_XDP_TX | ICE_XDP_REDIR)) - xdp_xmit |= xdp_res; - else - xsk_buff_free(xdp); + if (likely(xdp_res & (ICE_XDP_TX | ICE_XDP_REDIR))) + xdp_xmit |= xdp_res; + else if (xdp_res == ICE_XDP_CONSUMED) + xsk_buff_free(xdp); + else + goto construct_skb; - total_rx_bytes += size; - total_rx_packets++; + total_rx_bytes += size; + total_rx_packets++; + + ice_bump_ntc(rx_ring); + continue; - ice_bump_ntc(rx_ring); - continue; - } construct_skb: /* XDP_PASS path */ skb = ice_construct_skb_zc(rx_ring, xdp); From patchwork Wed Apr 13 15:30:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812147 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B737C433EF for ; Wed, 13 Apr 2022 15:30:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236518AbiDMPdF (ORCPT ); Wed, 13 Apr 2022 11:33:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236524AbiDMPdE (ORCPT ); Wed, 13 Apr 2022 11:33:04 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DFEA37A2F; Wed, 13 Apr 2022 08:30:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863840; x=1681399840; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=x0o6u3BnYrQT4VVNj0XRk9xfs3RfOLQKuz53r0geu0U=; b=RVnIcgjU3UB4CPUJhl9NITSB7Bewnxba4wgBabneH80mWntnreCpjWUh fJ4xOkDOwn/txmnVl/fcF0VsApMW4dzLmGrdEGt7m1sdwwaWqKtkbPMiH s8X5XpRSbhzVbwlUpotvBHnhSHCHTdH7AXYw4vYFdViU9NlwNKJoGXEVs v3NzCTp0SVuv3/ycTBacKGznHqzFB5Q8gm5/NPbW5CqcgopIVOZs+5sRA b0zOvJ1kD4fnuNBdh3+vwHi5DaM1+h4C2iXcKGepyDMf239r6lNe7SnA7 PkHCNjZAV3gHa2M7cJWBhajgUS8PIX/+qa1z6PBEyA6fREJ1+imxqMqK3 g==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544219" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544219" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:30:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318249" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:30:37 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski , Jesper Dangaard Brouer Subject: [PATCH v2 bpf-next 04/14] ixgbe: xsk: decorate IXGBE_XDP_REDIR with likely() Date: Wed, 13 Apr 2022 17:30:05 +0200 Message-Id: <20220413153015.453864-5-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net ixgbe_run_xdp_zc() suggests to compiler that XDP_REDIRECT is the most probable action returned from BPF program that AF_XDP has in its pipeline. Let's also bring this suggestion up to the callsite of ixgbe_run_xdp_zc() so that compiler will be able to generate more optimized code which in turn will make branch predictor happy. Suggested-by: Jesper Dangaard Brouer Signed-off-by: Maciej Fijalkowski --- drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 25 ++++++++++---------- 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c index dd7ff66d422f..85497bf10624 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c @@ -303,21 +303,22 @@ int ixgbe_clean_rx_irq_zc(struct ixgbe_q_vector *q_vector, xsk_buff_dma_sync_for_cpu(bi->xdp, rx_ring->xsk_pool); xdp_res = ixgbe_run_xdp_zc(adapter, rx_ring, bi->xdp); - if (xdp_res) { - if (xdp_res & (IXGBE_XDP_TX | IXGBE_XDP_REDIR)) - xdp_xmit |= xdp_res; - else - xsk_buff_free(bi->xdp); + if (likely(xdp_res & (IXGBE_XDP_TX | IXGBE_XDP_REDIR))) + xdp_xmit |= xdp_res; + else if (xdp_res == IXGBE_XDP_CONSUMED) + xsk_buff_free(bi->xdp); + else + goto construct_skb; - bi->xdp = NULL; - total_rx_packets++; - total_rx_bytes += size; + bi->xdp = NULL; + total_rx_packets++; + total_rx_bytes += size; - cleaned_count++; - ixgbe_inc_ntc(rx_ring); - continue; - } + cleaned_count++; + ixgbe_inc_ntc(rx_ring); + continue; +construct_skb: /* XDP_PASS path */ skb = ixgbe_construct_skb_zc(rx_ring, bi->xdp); if (!skb) { From patchwork Wed Apr 13 15:30:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812148 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D6D1C433FE for ; Wed, 13 Apr 2022 15:30:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236528AbiDMPdG (ORCPT ); Wed, 13 Apr 2022 11:33:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236527AbiDMPdE (ORCPT ); Wed, 13 Apr 2022 11:33:04 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD22037A87; Wed, 13 Apr 2022 08:30:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863842; x=1681399842; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DViUqkwcqzhXJl9eAtXWVX72kqmsj+d+jOdh0DLymh8=; b=RZmkCpvBwrap79z83HeEsVMyml2t4S2vrSTnOve6pQo9q5nu2oDGpJzj MfprrB679z25mWHoCAJXu4jP+OrH+POl5sQrMFA5z9wMsacLlvrRyFsnE utJJ71xkNTrS2u8Dd8YyueLej3G5E3z3FT4ezOaAsGxuwR5++C9YAjZc1 qxyCKNe/ZZ4XUwD5SzJgxTHV2tVihviPPz3YLKXQNU6K7Y5hDJrvI8blt MUVix03D0mvAJhZEg3QsUq2gIxLz6lIurhpCRpGvXSqYzgoCxrPlICXwB 8thFrZ0aiNmz353GR2IackjtQIfRehsPYn7IdsiTcHxA/rBU5CX5l5lGF A==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544240" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544240" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:30:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318275" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:30:39 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski Subject: [PATCH v2 bpf-next 05/14] ice: xsk: terminate Rx side of NAPI when XSK Rx queue gets full Date: Wed, 13 Apr 2022 17:30:06 +0200 Message-Id: <20220413153015.453864-6-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net When XSK pool uses need_wakeup feature, correlate -ENOBUFS that was returned from xdp_do_redirect() with a XSK Rx queue being full. In such case, terminate the Rx processing that is being done on the current HW Rx ring and let the user space consume descriptors from XSK Rx queue so that there is room that driver can use later on. Introduce new internal return code ICE_XDP_EXIT that will indicate case described above. Note that it does not affect Tx processing that is bound to the same NAPI context, nor the other Rx rings. Signed-off-by: Maciej Fijalkowski --- drivers/net/ethernet/intel/ice/ice_txrx.h | 1 + drivers/net/ethernet/intel/ice/ice_xsk.c | 29 +++++++++++++++-------- 2 files changed, 20 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.h b/drivers/net/ethernet/intel/ice/ice_txrx.h index cead3eb149bd..f5a906c03669 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.h +++ b/drivers/net/ethernet/intel/ice/ice_txrx.h @@ -133,6 +133,7 @@ static inline int ice_skb_pad(void) #define ICE_XDP_CONSUMED BIT(0) #define ICE_XDP_TX BIT(1) #define ICE_XDP_REDIR BIT(2) +#define ICE_XDP_EXIT BIT(3) #define ICE_RX_DMA_ATTR \ (DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_WEAK_ORDERING) diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c index e9ff05de0084..99bfa21c3938 100644 --- a/drivers/net/ethernet/intel/ice/ice_xsk.c +++ b/drivers/net/ethernet/intel/ice/ice_xsk.c @@ -540,9 +540,13 @@ ice_run_xdp_zc(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp, if (likely(act == XDP_REDIRECT)) { err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); - if (err) - goto out_failure; - return ICE_XDP_REDIR; + if (!err) + return ICE_XDP_REDIR; + if (xsk_uses_need_wakeup(rx_ring->xsk_pool) && err == -ENOBUFS) + result = ICE_XDP_EXIT; + else + result = ICE_XDP_CONSUMED; + goto out_failure; } switch (act) { @@ -553,15 +557,16 @@ ice_run_xdp_zc(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp, if (result == ICE_XDP_CONSUMED) goto out_failure; break; + case XDP_DROP: + result = ICE_XDP_CONSUMED; + break; default: bpf_warn_invalid_xdp_action(rx_ring->netdev, xdp_prog, act); fallthrough; case XDP_ABORTED: + result = ICE_XDP_CONSUMED; out_failure: trace_xdp_exception(rx_ring->netdev, xdp_prog, act); - fallthrough; - case XDP_DROP: - result = ICE_XDP_CONSUMED; break; } @@ -629,12 +634,16 @@ int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring, int budget) xsk_buff_dma_sync_for_cpu(xdp, rx_ring->xsk_pool); xdp_res = ice_run_xdp_zc(rx_ring, xdp, xdp_prog, xdp_ring); - if (likely(xdp_res & (ICE_XDP_TX | ICE_XDP_REDIR))) + if (likely(xdp_res & (ICE_XDP_TX | ICE_XDP_REDIR))) { xdp_xmit |= xdp_res; - else if (xdp_res == ICE_XDP_CONSUMED) + } else if (xdp_res == ICE_XDP_EXIT) { + failure = true; + break; + } else if (xdp_res == ICE_XDP_CONSUMED) { xsk_buff_free(xdp); - else + } else if (xdp_res == ICE_XDP_PASS) { goto construct_skb; + } total_rx_bytes += size; total_rx_packets++; @@ -669,7 +678,7 @@ int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring, int budget) ice_receive_skb(rx_ring, skb, vlan_tag); } - failure = !ice_alloc_rx_bufs_zc(rx_ring, ICE_DESC_UNUSED(rx_ring)); + failure |= !ice_alloc_rx_bufs_zc(rx_ring, ICE_DESC_UNUSED(rx_ring)); ice_finalize_xdp_rx(xdp_ring, xdp_xmit); ice_update_rx_ring_stats(rx_ring, total_rx_packets, total_rx_bytes); From patchwork Wed Apr 13 15:30:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812149 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24A54C433EF for ; Wed, 13 Apr 2022 15:30:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232010AbiDMPdQ (ORCPT ); Wed, 13 Apr 2022 11:33:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236527AbiDMPdJ (ORCPT ); Wed, 13 Apr 2022 11:33:09 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22D8837A26; Wed, 13 Apr 2022 08:30:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863847; x=1681399847; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iuX1I9I5/OrcahEg2jn+B/BkoU3Ra9dfauKmPu7YwyQ=; b=V3nGn2Do8uCx4d6Ca13nm4MWi738ddAbKOC90SAJzbVPa/6TGmzpUDa2 hR9AucLiTPJGDv92jK3gEhEBVV9vF7ZY7TD+fWKyqMUKBrfXAlBHgFujW Vh2FO4zosolV6AZJclmXTv6+s5rbxyrzNtaCDvNdyDQqh5gRB7qO6Upoa 2HjSM1bHCWeZ/pIUMTTjdRasg/7U5P7lKMSEvuppYT5naCJzacBBhjGVN ITluuEmY0KOEyAZGuFskKI2WY2woNB270wjaEStYgPg57vBpzhi0cihba 4R/A2QdcfH6a4kchsWLjZ8olEnmadOK5J3s3ABWAS87kgIwvn20Tz+I/r Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544268" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544268" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:30:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318309" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:30:42 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski Subject: [PATCH v2 bpf-next 06/14] i40e: xsk: terminate Rx side of NAPI when XSK Rx queue gets full Date: Wed, 13 Apr 2022 17:30:07 +0200 Message-Id: <20220413153015.453864-7-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net When XSK pool uses need_wakeup feature, correlate -ENOBUFS that was returned from xdp_do_redirect() with a XSK Rx queue being full. In such case, terminate the Rx processing that is being done on the current HW Rx ring and let the user space consume descriptors from XSK Rx queue so that there is room that driver can use later on. Introduce new internal return code I40E_XDP_EXIT that will indicate case described above. Note that it does not affect Tx processing that is bound to the same NAPI context, nor the other Rx rings. Signed-off-by: Maciej Fijalkowski --- .../ethernet/intel/i40e/i40e_txrx_common.h | 1 + drivers/net/ethernet/intel/i40e/i40e_xsk.c | 32 +++++++++++++------ 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx_common.h b/drivers/net/ethernet/intel/i40e/i40e_txrx_common.h index 19da3b22160f..8c5118c8baaf 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx_common.h +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx_common.h @@ -20,6 +20,7 @@ void i40e_release_rx_desc(struct i40e_ring *rx_ring, u32 val); #define I40E_XDP_CONSUMED BIT(0) #define I40E_XDP_TX BIT(1) #define I40E_XDP_REDIR BIT(2) +#define I40E_XDP_EXIT BIT(3) /* * build_ctob - Builds the Tx descriptor (cmd, offset and type) qword diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c index c1d25b0b0ca2..94ea8288859a 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c +++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c @@ -161,9 +161,13 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) if (likely(act == XDP_REDIRECT)) { err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); - if (err) - goto out_failure; - return I40E_XDP_REDIR; + if (!err) + return I40E_XDP_REDIR; + if (xsk_uses_need_wakeup(rx_ring->xsk_pool) && err == -ENOBUFS) + result = I40E_XDP_EXIT; + else + result = I40E_XDP_CONSUMED; + goto out_failure; } switch (act) { @@ -175,16 +179,17 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) if (result == I40E_XDP_CONSUMED) goto out_failure; break; + case XDP_DROP: + result = I40E_XDP_CONSUMED; + break; default: bpf_warn_invalid_xdp_action(rx_ring->netdev, xdp_prog, act); fallthrough; case XDP_ABORTED: + result = I40E_XDP_CONSUMED; out_failure: trace_xdp_exception(rx_ring->netdev, xdp_prog, act); fallthrough; /* handle aborts by dropping packet */ - case XDP_DROP: - result = I40E_XDP_CONSUMED; - break; } return result; } @@ -271,7 +276,8 @@ static void i40e_handle_xdp_result_zc(struct i40e_ring *rx_ring, unsigned int *rx_packets, unsigned int *rx_bytes, unsigned int size, - unsigned int xdp_res) + unsigned int xdp_res, + bool *failure) { struct sk_buff *skb; @@ -281,11 +287,15 @@ static void i40e_handle_xdp_result_zc(struct i40e_ring *rx_ring, if (likely(xdp_res == I40E_XDP_REDIR) || xdp_res == I40E_XDP_TX) return; + if (xdp_res == I40E_XDP_EXIT) { + *failure = true; + return; + } + if (xdp_res == I40E_XDP_CONSUMED) { xsk_buff_free(xdp_buff); return; } - if (xdp_res == I40E_XDP_PASS) { /* NB! We are not checking for errors using * i40e_test_staterr with @@ -371,7 +381,9 @@ int i40e_clean_rx_irq_zc(struct i40e_ring *rx_ring, int budget) xdp_res = i40e_run_xdp_zc(rx_ring, bi); i40e_handle_xdp_result_zc(rx_ring, bi, rx_desc, &rx_packets, - &rx_bytes, size, xdp_res); + &rx_bytes, size, xdp_res, &failure); + if (failure) + break; total_rx_packets += rx_packets; total_rx_bytes += rx_bytes; xdp_xmit |= xdp_res & (I40E_XDP_TX | I40E_XDP_REDIR); @@ -382,7 +394,7 @@ int i40e_clean_rx_irq_zc(struct i40e_ring *rx_ring, int budget) cleaned_count = (next_to_clean - rx_ring->next_to_use - 1) & count_mask; if (cleaned_count >= I40E_RX_BUFFER_WRITE) - failure = !i40e_alloc_rx_buffers_zc(rx_ring, cleaned_count); + failure |= !i40e_alloc_rx_buffers_zc(rx_ring, cleaned_count); i40e_finalize_xdp_rx(rx_ring, xdp_xmit); i40e_update_rx_stats(rx_ring, total_rx_bytes, total_rx_packets); From patchwork Wed Apr 13 15:30:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812150 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9560EC433FE for ; Wed, 13 Apr 2022 15:30:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236531AbiDMPdR (ORCPT ); Wed, 13 Apr 2022 11:33:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57346 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236530AbiDMPdL (ORCPT ); Wed, 13 Apr 2022 11:33:11 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4483D37A87; Wed, 13 Apr 2022 08:30:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863850; x=1681399850; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jgo3evBwcKnrCtnEZCg2TuPoWoUfHaUdZYgSBqSFEXE=; b=U4GfUpiHjEw4mI1w54IsOXmGwSAoC8kpLREAZQdDFR7L7fFFQqvz7A76 rJK5ue0vPsL06L7CRInoHi8dRais/zcaqJbL9qlpHVUznWmGp20nE5zYC S70MzGPnRrHNAp0nFZJ+Kt3b3SgZi9QAPeQRuZeSqedtqMuOy3Y4rnV0/ 6I9MsToNqxu/mPLkTDNx94pRKmwNjiJ1Lt5fyaI73pljX/j6dWdSB39FW HdylDtfRovukCr20NruYBZs4echyqDmIycLKYhAYkWWyFrImWBLsXwynX ziw/sqdZj0qd1Ajncb1qFT53FTmizTpIcxHYaJ5vw3aLZGqboX7TnE5aF g==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544278" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544278" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:30:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318330" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:30:47 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski Subject: [PATCH v2 bpf-next 07/14] ixgbe: xsk: terminate Rx side of NAPI when XSK Rx queue gets full Date: Wed, 13 Apr 2022 17:30:08 +0200 Message-Id: <20220413153015.453864-8-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net When XSK pool uses need_wakeup feature, correlate -ENOBUFS that was returned from xdp_do_redirect() with a XSK Rx queue being full. In such case, terminate the Rx processing that is being done on the current HW Rx ring and let the user space consume descriptors from XSK Rx queue so that there is room that driver can use later on. Introduce new internal return code IXGBE_XDP_EXIT that will indicate case described above. Note that it does not affect Tx processing that is bound to the same NAPI context, nor the other Rx rings. Signed-off-by: Maciej Fijalkowski --- .../ethernet/intel/ixgbe/ixgbe_txrx_common.h | 1 + drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 27 ++++++++++++------- 2 files changed, 19 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_txrx_common.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_txrx_common.h index bba3feaf3318..f1f69ce67420 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_txrx_common.h +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_txrx_common.h @@ -8,6 +8,7 @@ #define IXGBE_XDP_CONSUMED BIT(0) #define IXGBE_XDP_TX BIT(1) #define IXGBE_XDP_REDIR BIT(2) +#define IXGBE_XDP_EXIT BIT(3) #define IXGBE_TXD_CMD (IXGBE_TXD_CMD_EOP | \ IXGBE_TXD_CMD_RS) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c index 85497bf10624..bdd70b85a787 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c @@ -109,9 +109,13 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter, if (likely(act == XDP_REDIRECT)) { err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); - if (err) - goto out_failure; - return IXGBE_XDP_REDIR; + if (!err) + return IXGBE_XDP_REDIR; + if (xsk_uses_need_wakeup(rx_ring->xsk_pool) && err == -ENOBUFS) + result = IXGBE_XDP_EXIT; + else + result = IXGBE_XDP_CONSUMED; + goto out_failure; } switch (act) { @@ -130,16 +134,17 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter, if (result == IXGBE_XDP_CONSUMED) goto out_failure; break; + case XDP_DROP: + result = IXGBE_XDP_CONSUMED; + break; default: bpf_warn_invalid_xdp_action(rx_ring->netdev, xdp_prog, act); fallthrough; case XDP_ABORTED: + result = IXGBE_XDP_CONSUMED; out_failure: trace_xdp_exception(rx_ring->netdev, xdp_prog, act); fallthrough; /* handle aborts by dropping packet */ - case XDP_DROP: - result = IXGBE_XDP_CONSUMED; - break; } return result; } @@ -303,12 +308,16 @@ int ixgbe_clean_rx_irq_zc(struct ixgbe_q_vector *q_vector, xsk_buff_dma_sync_for_cpu(bi->xdp, rx_ring->xsk_pool); xdp_res = ixgbe_run_xdp_zc(adapter, rx_ring, bi->xdp); - if (likely(xdp_res & (IXGBE_XDP_TX | IXGBE_XDP_REDIR))) + if (likely(xdp_res & (IXGBE_XDP_TX | IXGBE_XDP_REDIR))) { xdp_xmit |= xdp_res; - else if (xdp_res == IXGBE_XDP_CONSUMED) + } else if (xdp_res == IXGBE_XDP_EXIT) { + failure = true; + break; + } else if (xdp_res == IXGBE_XDP_CONSUMED) { xsk_buff_free(bi->xdp); - else + } else if (xdp_res == IXGBE_XDP_PASS) { goto construct_skb; + } bi->xdp = NULL; total_rx_packets++; From patchwork Wed Apr 13 15:30:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812151 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B43E2C4332F for ; Wed, 13 Apr 2022 15:31:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236539AbiDMPdT (ORCPT ); Wed, 13 Apr 2022 11:33:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236533AbiDMPdO (ORCPT ); Wed, 13 Apr 2022 11:33:14 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F80B37A8D; Wed, 13 Apr 2022 08:30:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863852; x=1681399852; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=u2udKhUqzP8jSQEG0gLDXOHv/lNHu3lWJ70Ks7JBhfQ=; b=dfZLuusJU/iNC8Ue2r+AozpcK6PheYV0Y+o86kHXyyTVClGvXP98SqXu qS2K4YFSRZ09AOCMtYIHvMDoSVFl+Z2rbFBrxinviBhA9PHigcDRS1r49 qeGqf0guQ/ZOPWuEV00Jwt10YcvJo01pnf/YiS33AoWjHgLWaaMwP0Jjd WMkPemfRDPKNPYAcmZ1nWJ3yk3k7e4Fq1SBkYEtZcVbkClGP0BiFks4jn M0pF6W49PU2HJdpKBSog+saE0MDjOX5GB8h48ivU+NUmdatFuM6yd+MGm LO9NyHFp3yqFEDzses4dIzVYHYDD34+BfgOefuIhG0vDozQkJE9lyGTGi w==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544286" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544286" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:30:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318345" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:30:49 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski Subject: [PATCH v2 bpf-next 08/14] ice: xsk: diversify return values from xsk_wakeup call paths Date: Wed, 13 Apr 2022 17:30:09 +0200 Message-Id: <20220413153015.453864-9-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Currently, when debugging AF_XDP workloads, one can correlate the -ENXIO return code as the case that XSK is not in the bound state. Returning same code from ndo_xsk_wakeup can be misleading and simply makes it harder to follow what is going on. Change ENXIOs in ice's ndo_xsk_wakeup() implementation to EINVALs, so that when probing it is clear that something is wrong on the driver side, not the xsk_{recv,send}msg. There is a -ENETDOWN that can happen from both kernel/driver sides though, but I don't have a correct replacement for this on one of the sides, so let's keep it that way. Signed-off-by: Maciej Fijalkowski --- drivers/net/ethernet/intel/ice/ice_xsk.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c index 99bfa21c3938..53c299eebd99 100644 --- a/drivers/net/ethernet/intel/ice/ice_xsk.c +++ b/drivers/net/ethernet/intel/ice/ice_xsk.c @@ -934,13 +934,13 @@ ice_xsk_wakeup(struct net_device *netdev, u32 queue_id, return -ENETDOWN; if (!ice_is_xdp_ena_vsi(vsi)) - return -ENXIO; + return -EINVAL; if (queue_id >= vsi->num_txq) - return -ENXIO; + return -EINVAL; if (!vsi->xdp_rings[queue_id]->xsk_pool) - return -ENXIO; + return -EINVAL; ring = vsi->xdp_rings[queue_id]; From patchwork Wed Apr 13 15:30:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812152 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1746FC433EF for ; Wed, 13 Apr 2022 15:31:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236519AbiDMPdV (ORCPT ); Wed, 13 Apr 2022 11:33:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57680 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236543AbiDMPdQ (ORCPT ); Wed, 13 Apr 2022 11:33:16 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 202A237AB2; Wed, 13 Apr 2022 08:30:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863855; x=1681399855; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GEOFI2L+Sr82ytthmiJvWAK5wJHfsuRxDbyttCm5JNQ=; b=lAMLez76tb38e3No8/dDRuloWRnBhecJ2ng1UNkCg4qQ6xpi+sGzVIWw CcQBU8/cHZUCyeZt/b1eSXK9GG8vIo/szO4IGRSes2lF2jFGdCT0A14Ev qUaRPnE2RVqPUeaGtIJOKc/S7XFWQhTrd6cdI7YSWSaKFA04gxvutLvZl 5He2H3inOFLMs6VaVVtljMdNyrcq8oY5WWQR5CvBd4OjpEbUfCh4enQCK gc3dmQ6T+PQD4n/Bf82PSz/N4qNSGTH8Jjm+HHfvpwziLkRgeW+AvUwda jOToB2RYssgvGiOTuBvLQUTpdcZ+Sf150w9e1JazFF44ab31b5FkzhRBN g==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544303" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544303" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:30:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318367" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:30:52 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski Subject: [PATCH v2 bpf-next 09/14] i40e: xsk: diversify return values from xsk_wakeup call paths Date: Wed, 13 Apr 2022 17:30:10 +0200 Message-Id: <20220413153015.453864-10-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Currently, when debugging AF_XDP workloads, one can correlate the -ENXIO return code as the case that XSK is not in the bound state. Returning same code from ndo_xsk_wakeup can be misleading and simply makes it harder to follow what is going on. Change ENXIOs in i40e's ndo_xsk_wakeup() implementation to EINVALs, so that when probing it is clear that something is wrong on the driver side, not the xsk_{recv,send}msg. There is a -ENETDOWN that can happen from both kernel/driver sides though, but I don't have a correct replacement for this on one of the sides, so let's keep it that way. Signed-off-by: Maciej Fijalkowski --- drivers/net/ethernet/intel/i40e/i40e_xsk.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c index 94ea8288859a..050280fd10c1 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c +++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c @@ -606,13 +606,13 @@ int i40e_xsk_wakeup(struct net_device *dev, u32 queue_id, u32 flags) return -ENETDOWN; if (!i40e_enabled_xdp_vsi(vsi)) - return -ENXIO; + return -EINVAL; if (queue_id >= vsi->num_queue_pairs) - return -ENXIO; + return -EINVAL; if (!vsi->xdp_rings[queue_id]->xsk_pool) - return -ENXIO; + return -EINVAL; ring = vsi->xdp_rings[queue_id]; From patchwork Wed Apr 13 15:30:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812153 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 056D9C433FE for ; Wed, 13 Apr 2022 15:31:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236538AbiDMPd1 (ORCPT ); Wed, 13 Apr 2022 11:33:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236535AbiDMPdZ (ORCPT ); Wed, 13 Apr 2022 11:33:25 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D834D37A9F; Wed, 13 Apr 2022 08:30:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863857; x=1681399857; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zR2LLs75XT0kCmEEP/LYE5KhyXYEesKqdXnC7IQSGqI=; b=DR0ebTJYeTq1WNSck7iIH4yu/07XmtE56XTrAB7yJoAPHIIMHDnciYtM 6Vc82J+z6KgakKDSWMk3VWKWSKzvnr8CBtUn4KfvR4kxISAkY+6/tjfl4 MO6g+8A0/Mm28v6ir8McW4S1qRK+sTeCyfrHXYye7aLRozKxisJqAhJMm YxgkFZw/DmUtij3CpzsEmSwO6A4jSgjcozZuCvqiFmVWNQL4mymRWEtVv XsIocRvFj51IAh1dhoM7Y/a9DFUKcNb4EuchF2ervJM2nPC/TlZI10kHm mZCxBouTjYNb97M40Jt52x8RBA8FaeTofjY3vsGHtdpctRw2W51IwtjRy w==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544309" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544309" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:30:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318385" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:30:55 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski Subject: [PATCH v2 bpf-next 10/14] ixgbe: xsk: diversify return values from xsk_wakeup call paths Date: Wed, 13 Apr 2022 17:30:11 +0200 Message-Id: <20220413153015.453864-11-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Currently, when debugging AF_XDP workloads, one can correlate the -ENXIO return code as the case that XSK is not in the bound state. Returning same code from ndo_xsk_wakeup can be misleading and simply makes it harder to follow what is going on. Change ENXIOs in ixgbe's ndo_xsk_wakeup() implementation to EINVALs, so that when probing it is clear that something is wrong on the driver side, not the xsk_{recv,send}msg. There is a -ENETDOWN that can happen from both kernel/driver sides though, but I don't have a correct replacement for this on one of the sides, so let's keep it that way. Signed-off-by: Maciej Fijalkowski --- drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c index bdd70b85a787..68532cffd453 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c @@ -526,10 +526,10 @@ int ixgbe_xsk_wakeup(struct net_device *dev, u32 qid, u32 flags) return -ENETDOWN; if (!READ_ONCE(adapter->xdp_prog)) - return -ENXIO; + return -EINVAL; if (qid >= adapter->num_xdp_queues) - return -ENXIO; + return -EINVAL; ring = adapter->xdp_ring[qid]; @@ -537,7 +537,7 @@ int ixgbe_xsk_wakeup(struct net_device *dev, u32 qid, u32 flags) return -ENETDOWN; if (!ring->xsk_pool) - return -ENXIO; + return -EINVAL; if (!napi_if_scheduled_mark_missed(&ring->q_vector->napi)) { u64 eics = BIT_ULL(ring->q_vector->v_idx); From patchwork Wed Apr 13 15:30:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812154 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 205D9C433EF for ; Wed, 13 Apr 2022 15:31:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236552AbiDMPd2 (ORCPT ); Wed, 13 Apr 2022 11:33:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58424 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236549AbiDMPd0 (ORCPT ); Wed, 13 Apr 2022 11:33:26 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C3BE37AB0; Wed, 13 Apr 2022 08:31:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863860; x=1681399860; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=W0ivcRo/CNJCiHsXXiZIy0nwpm+Mc6B1Iwk6iOajJno=; b=BBjEnQ50sRuDbeD1+YRY6b8+h8LCR0t5PyXTaeSIMrLfUGqz3dbPUBSn JkFY3aKv/102/bRuXLyvydYeHdCWuutwf+ByjcQexyFWtNcogeATxBo2z AkrHfoNwW2FH+yFSgrKZmrxca9Rcx4XPMr5QmRIH6CwVgdiAdJcrkd/qT kQEjaEmaOId35jZHzDY9oEdfCb3SxFidmGIgk+r+BISpFPq/nTXFDxCk4 jtEXqn3xjZq5pm2IfGUtMWtJl/ttqk3tKphRv4G0s48oOvqiNjqljA4tw Enp6xweA13RbkcxoV1wiD/fAaojznDSZ12SNNm75QYxPQs2LuTwIv0P1L w==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544320" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544320" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:30:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318405" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:30:57 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski Subject: [PATCH v2 bpf-next 11/14] mlx5: xsk: diversify return values from xsk_wakeup call paths Date: Wed, 13 Apr 2022 17:30:12 +0200 Message-Id: <20220413153015.453864-12-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Currently, when debugging AF_XDP workloads, one can correlate the -ENXIO return code as the case that XSK is not in the bound state. Returning same code from ndo_xsk_wakeup can be misleading and simply makes it harder to follow what is going on. Change ENXIO in mlx5's ndo_xsk_wakeup() implementation to EINVAL, so that when probing it is clear that something is wrong on the driver side, not the xsk_{recv,send}msg. There is a -ENETDOWN that can happen from both kernel/driver sides though, but I don't have a correct replacement for this on one of the sides, so let's keep it that way. Signed-off-by: Maciej Fijalkowski --- drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c index 3ec0c17db010..4902ef74fedf 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c @@ -23,7 +23,7 @@ int mlx5e_xsk_wakeup(struct net_device *dev, u32 qid, u32 flags) c = priv->channels.c[ix]; if (unlikely(!test_bit(MLX5E_CHANNEL_STATE_XSK, c->state))) - return -ENXIO; + return -EINVAL; if (!napi_if_scheduled_mark_missed(&c->napi)) { /* To avoid WQE overrun, don't post a NOP if async_icosq is not From patchwork Wed Apr 13 15:30:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812155 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E5F8C433F5 for ; Wed, 13 Apr 2022 15:31:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236557AbiDMPdd (ORCPT ); Wed, 13 Apr 2022 11:33:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236553AbiDMPd0 (ORCPT ); Wed, 13 Apr 2022 11:33:26 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1D8137BD0; Wed, 13 Apr 2022 08:31:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863862; x=1681399862; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=g4dm2m2hBU8t34U2sYB3MEK8uSxXcD6yVdkrodWYRhk=; b=Wn0TIdppgJbAcXtOilLT+vZta3CF9SpOYaEteEhYiAFvui5wm7YJxhsn yDLGKh3JWP2613ckMOyvc0mNgvhZt6S2TpkM9wc/9haTn/1NMFGekOCtj ep16RtzHfmE7qXjcO0kf7hsz46VYScBrdjh/kvyUdqA6ALcpk4REqPq9M qxtlmwPwvOhS549j4XJPABrzNErNE6wkxlJxMPsyx4p1cFyAlXuQHXVNz lmvsb5GQ2lLr0eQiGaCAbhvBFUiXf2yCVcLjzGPqHTty16X4iAFtRVrV8 17pEpupFYrg5TT27FjwUmlOIgz4xFzFDqP8SzFYm66NHkiLaowKdG8TmB Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544330" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544330" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:31:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318419" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:30:59 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski Subject: [PATCH v2 bpf-next 12/14] stmmac: xsk: diversify return values from xsk_wakeup call paths Date: Wed, 13 Apr 2022 17:30:13 +0200 Message-Id: <20220413153015.453864-13-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Currently, when debugging AF_XDP workloads, one can correlate the -ENXIO return code as the case that XSK is not in the bound state. Returning same code from ndo_xsk_wakeup can be misleading and simply makes it harder to follow what is going on. Change ENXIOs in stmmac's ndo_xsk_wakeup() implementation to EINVALs, so that when probing it is clear that something is wrong on the driver side, not the xsk_{recv,send}msg. There is a -ENETDOWN that can happen from both kernel/driver sides though, but I don't have a correct replacement for this on one of the sides, so let's keep it that way. Signed-off-by: Maciej Fijalkowski --- drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index 4a4b3651ab3e..c9e077b2a56e 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -6565,7 +6565,7 @@ int stmmac_xsk_wakeup(struct net_device *dev, u32 queue, u32 flags) return -ENETDOWN; if (!stmmac_xdp_is_enabled(priv)) - return -ENXIO; + return -EINVAL; if (queue >= priv->plat->rx_queues_to_use || queue >= priv->plat->tx_queues_to_use) @@ -6576,7 +6576,7 @@ int stmmac_xsk_wakeup(struct net_device *dev, u32 queue, u32 flags) ch = &priv->channel[queue]; if (!rx_q->xsk_pool && !tx_q->xsk_pool) - return -ENXIO; + return -EINVAL; if (!napi_if_scheduled_mark_missed(&ch->rxtx_napi)) { /* EQoS does not have per-DMA channel SW interrupt, From patchwork Wed Apr 13 15:30:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812156 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E82C0C43217 for ; Wed, 13 Apr 2022 15:31:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236558AbiDMPde (ORCPT ); Wed, 13 Apr 2022 11:33:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236544AbiDMPd1 (ORCPT ); Wed, 13 Apr 2022 11:33:27 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 826B837BEF; Wed, 13 Apr 2022 08:31:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863865; x=1681399865; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jYH2RJS/d0A3UuePhevxDHsE0BibcVkMT3K9xBY9ZEY=; b=hYTW6Y73SKvIuZ+d7eanPxmsVfQwoOPhCIp7HEf5K+8iYX9wEQs2Aa91 Xph5kU2gH0hx3EnAIkkPcDNeIZAg9WcJe9JBRvIlTKUoRP2bcp+0wpb4H S7JZtRG4eecoXDyIS+ruXQhOdV+56H0i3jPHKLu1J37Ht8hE312NolA/2 pZmvG6A56eAfKMRsuFukoBjn8ULc6r3LnjBlc9bK2dNeI7M7/uZUMblJz YBTkMy0tmHLOKA5zyfSm7w8V847VtJrxZ4Hcm/dboJavgnvL/n3MdeXqy BeijZb1ICpOopR8GWQXzQmta4c3Fv2WLetHdsscSpnRgUzmoq0NNMVMZY A==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544344" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544344" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:31:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318427" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:31:02 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski Subject: [PATCH v2 bpf-next 13/14] ice: xsk: avoid refilling single Rx descriptors Date: Wed, 13 Apr 2022 17:30:14 +0200 Message-Id: <20220413153015.453864-14-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Call alloc Rx routine for ZC driver only when the amount of unallocated descriptors exceeds given threshold. Signed-off-by: Maciej Fijalkowski --- drivers/net/ethernet/intel/ice/ice_xsk.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c index 53c299eebd99..c26b408ba419 100644 --- a/drivers/net/ethernet/intel/ice/ice_xsk.c +++ b/drivers/net/ethernet/intel/ice/ice_xsk.c @@ -587,6 +587,7 @@ int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring, int budget) unsigned int xdp_xmit = 0; struct bpf_prog *xdp_prog; bool failure = false; + int entries_to_alloc; /* ZC patch is enabled only when XDP program is set, * so here it can not be NULL @@ -678,7 +679,9 @@ int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring, int budget) ice_receive_skb(rx_ring, skb, vlan_tag); } - failure |= !ice_alloc_rx_bufs_zc(rx_ring, ICE_DESC_UNUSED(rx_ring)); + entries_to_alloc = ICE_DESC_UNUSED(rx_ring); + if (entries_to_alloc > ICE_RING_QUARTER(rx_ring)) + failure |= !ice_alloc_rx_bufs_zc(rx_ring, entries_to_alloc); ice_finalize_xdp_rx(xdp_ring, xdp_xmit); ice_update_rx_ring_stats(rx_ring, total_rx_packets, total_rx_bytes); From patchwork Wed Apr 13 15:30:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 12812157 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66144C433FE for ; Wed, 13 Apr 2022 15:31:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236554AbiDMPdl (ORCPT ); Wed, 13 Apr 2022 11:33:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236551AbiDMPd2 (ORCPT ); Wed, 13 Apr 2022 11:33:28 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94D6F381A5; Wed, 13 Apr 2022 08:31:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649863867; x=1681399867; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1ZZv1T3pCpwRQJnAcBaHCxkSUHlK085zX9k9Xlvk0xE=; b=VCkhNqT7cxDutkkBJO3PG0p/57Q4/6R1QbhoFT2Hd3CfxKWdYWjb66Yg FW3X4XytTeCbK60hklYZENRzmGa0c/7kCFGuncRJO9mCSmKB1oCoBarH3 LYv1yHh+HMC2cU4ZmpWg/KAnxqXH6PgdNRmri9NSN+ywpHTpqsg722/Jn oUK13GDEAGAf26xLS9yZ2AJYDCNuCxwafDj7aQ2+2YXWxquQrhzX443V/ fA1kwMzF+fsXga9flsuudsvqbL81xlAnHwdrBDTQXh+/pIFjRhBa6QPCd 6FiYTvNmKF657E9+w3Kf7GeP2bstxmCY47FPR3xwx7EjTZUok2FeDZYPW g==; X-IronPort-AV: E=McAfee;i="6400,9594,10316"; a="261544354" X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="261544354" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2022 08:31:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,257,1643702400"; d="scan'208";a="573318442" Received: from boxer.igk.intel.com ([10.102.20.173]) by orsmga008.jf.intel.com with ESMTP; 13 Apr 2022 08:31:05 -0700 From: Maciej Fijalkowski To: bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net Cc: netdev@vger.kernel.org, magnus.karlsson@intel.com, alexandr.lobakin@intel.com, maximmi@nvidia.com, kuba@kernel.org, bjorn@kernel.org, Maciej Fijalkowski Subject: [PATCH v2 bpf-next 14/14] xsk: drop ternary operator from xskq_cons_has_entries Date: Wed, 13 Apr 2022 17:30:15 +0200 Message-Id: <20220413153015.453864-15-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220413153015.453864-1-maciej.fijalkowski@intel.com> References: <20220413153015.453864-1-maciej.fijalkowski@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Simplify the mentioned helper function by removing ternary operator. The expression that is there outputs the boolean value by itself. This helper might be used in the hot path so this simplification can also be considered as micro optimization. Signed-off-by: Maciej Fijalkowski --- net/xdp/xsk_queue.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/xdp/xsk_queue.h b/net/xdp/xsk_queue.h index 644479e65578..a794410989cc 100644 --- a/net/xdp/xsk_queue.h +++ b/net/xdp/xsk_queue.h @@ -263,7 +263,7 @@ static inline u32 xskq_cons_nb_entries(struct xsk_queue *q, u32 max) static inline bool xskq_cons_has_entries(struct xsk_queue *q, u32 cnt) { - return xskq_cons_nb_entries(q, cnt) >= cnt ? true : false; + return xskq_cons_nb_entries(q, cnt) >= cnt; } static inline bool xskq_cons_peek_addr_unchecked(struct xsk_queue *q, u64 *addr)