From patchwork Sat Apr 2 00:21:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 12798870 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72A95C433F5 for ; Sat, 2 Apr 2022 00:21:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353532AbiDBAXS (ORCPT ); Fri, 1 Apr 2022 20:23:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237710AbiDBAXQ (ORCPT ); Fri, 1 Apr 2022 20:23:16 -0400 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0851F1AC42C for ; Fri, 1 Apr 2022 17:21:24 -0700 (PDT) Received: by mail-pl1-x633.google.com with SMTP id e5so3787706pls.4 for ; Fri, 01 Apr 2022 17:21:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1CaNTaWcYpGFRBja4KUhAw6tzNKG58s97BBX+XuLKF4=; b=Nf2KfWBkZip2AO5doZiGtf4DmEigfyrct7dobSai5n03psGDstEWZ2dlFGwPa1mo1E dhkFNt9NdFdmLNUnL0KwER4e3PXy/KpRPaNRYssXDsRkvqgHYu2vJ6MS6OpcYszZnbL6 vtuI11MBxhvMHr7EEDDy4MIYGiNUPznD/9zew= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1CaNTaWcYpGFRBja4KUhAw6tzNKG58s97BBX+XuLKF4=; b=gGIhMBeFka6c05t4i75Z5nEKm9ABXN2aDm9mPCtIQkhBBopv0/ZikKtP56pDykK9DI qIOQCsyplpYjb/ssJ/Jtxd6JajxXcO5XLUhJyZPk1yBimwWzdP/sSXN0C4Uvs4aopsXA L0EtNUGwGbNXgy1CWK1WPnydCzoiofdMort6kgvTSDT7a9zAakDVT4K+VFDbPqoa0F5a YE5dTgPYYeAkurvjc3sb2b0SZvWeY/IMOAv7MuKx9v3LMfYs4RoPDJVxpQMtx50HUACk 7hy3ABG5XvmwPdZYOo4lqxLyeZRN0VZ+9m3NfQdguo4HuFe8LBJZiLaZssM3Hw0vxi15 5p+g== X-Gm-Message-State: AOAM532fORFkuJSQFXz/cWVFqYzOWUSv/gUqPU4X90TKaonS+G+lL/hD fh84JWcR6h7WmYPMGcDpPCECuE1Ab29m+Q== X-Google-Smtp-Source: ABdhPJwiQwH4Rdc+RGJGv1UIKT0HCsHEo/0YbS3o5z3n2dh6C2sjhrue5yBz8l2NRgLm9V8eCvfSgA== X-Received: by 2002:a17:90b:4c92:b0:1c7:a9a3:6274 with SMTP id my18-20020a17090b4c9200b001c7a9a36274mr14305734pjb.148.1648858883842; Fri, 01 Apr 2022 17:21:23 -0700 (PDT) Received: from localhost.swdvt.lab.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id k10-20020a056a00168a00b004f7e2a550ccsm4295050pfc.78.2022.04.01.17.21.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Apr 2022 17:21:23 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, kuba@kernel.org, gospo@broadcom.com, bpf@vger.kernel.org, Pavan Chebbi Subject: [PATCH net 1/3] bnxt_en: Synchronize tx when xdp redirects happen on same ring Date: Fri, 1 Apr 2022 20:21:10 -0400 Message-Id: <1648858872-14682-2-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1648858872-14682-1-git-send-email-michael.chan@broadcom.com> References: <1648858872-14682-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Pavan Chebbi If there are more CPUs than the number of TX XDP rings, multiple XDP redirects can select the same TX ring based on the CPU on which XDP redirect is called. Add locking when needed and use static key to decide whether to take the lock. Fixes: f18c2b77b2e4 ("bnxt_en: optimized XDP_REDIRECT support") Signed-off-by: Pavan Chebbi Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 7 +++++++ drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 ++ drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 8 ++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h | 2 ++ 4 files changed, 19 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 1c28495875cf..874fad0a5cf8 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -3253,6 +3253,7 @@ static int bnxt_alloc_tx_rings(struct bnxt *bp) } qidx = bp->tc_to_qidx[j]; ring->queue_id = bp->q_info[qidx].queue_id; + spin_lock_init(&txr->xdp_tx_lock); if (i < bp->tx_nr_rings_xdp) continue; if (i % bp->tx_nr_rings_per_tc == (bp->tx_nr_rings_per_tc - 1)) @@ -10338,6 +10339,12 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init) if (irq_re_init) udp_tunnel_nic_reset_ntf(bp->dev); + if (bp->tx_nr_rings_xdp < num_possible_cpus()) { + if (!static_key_enabled(&bnxt_xdp_locking_key)) + static_branch_enable(&bnxt_xdp_locking_key); + } else if (static_key_enabled(&bnxt_xdp_locking_key)) { + static_branch_disable(&bnxt_xdp_locking_key); + } set_bit(BNXT_STATE_OPEN, &bp->state); bnxt_enable_int(bp); /* Enable TX queues */ diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 61aa3e8c5952..b4d3d051463b 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -800,6 +800,8 @@ struct bnxt_tx_ring_info { u32 dev_state; struct bnxt_ring_struct tx_ring_struct; + /* Synchronize simultaneous xdp_xmit on same ring */ + spinlock_t xdp_tx_lock; }; #define BNXT_LEGACY_COAL_CMPL_PARAMS \ diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c index 52fad0fdeacf..c0541ff00ac8 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c @@ -20,6 +20,8 @@ #include "bnxt.h" #include "bnxt_xdp.h" +DEFINE_STATIC_KEY_FALSE(bnxt_xdp_locking_key); + struct bnxt_sw_tx_bd *bnxt_xmit_bd(struct bnxt *bp, struct bnxt_tx_ring_info *txr, dma_addr_t mapping, u32 len) @@ -227,6 +229,9 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames, ring = smp_processor_id() % bp->tx_nr_rings_xdp; txr = &bp->tx_ring[ring]; + if (static_branch_unlikely(&bnxt_xdp_locking_key)) + spin_lock(&txr->xdp_tx_lock); + for (i = 0; i < num_frames; i++) { struct xdp_frame *xdp = frames[i]; @@ -250,6 +255,9 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames, bnxt_db_write(bp, &txr->tx_db, txr->tx_prod); } + if (static_branch_unlikely(&bnxt_xdp_locking_key)) + spin_unlock(&txr->xdp_tx_lock); + return nxmit; } diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h index 0df40c3beb05..067bb5e821f5 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h @@ -10,6 +10,8 @@ #ifndef BNXT_XDP_H #define BNXT_XDP_H +DECLARE_STATIC_KEY_FALSE(bnxt_xdp_locking_key); + struct bnxt_sw_tx_bd *bnxt_xmit_bd(struct bnxt *bp, struct bnxt_tx_ring_info *txr, dma_addr_t mapping, u32 len); From patchwork Sat Apr 2 00:21:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 12798872 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5BB4C43217 for ; Sat, 2 Apr 2022 00:21:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353567AbiDBAXT (ORCPT ); Fri, 1 Apr 2022 20:23:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234770AbiDBAXQ (ORCPT ); Fri, 1 Apr 2022 20:23:16 -0400 Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03E241AC430 for ; Fri, 1 Apr 2022 17:21:25 -0700 (PDT) Received: by mail-pg1-x52e.google.com with SMTP id t4so3561757pgc.1 for ; Fri, 01 Apr 2022 17:21:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UoCzq94MXVcOlUN/6wORFWX01+cATfQvzmGgy4cwnmY=; b=TIA3nvjrMWNJo+pdvAb16CeuUfx069MT93AIKLOMWSLD4SHdnvOAeDlo9+WDVmB78V NAHS9Nvs1acA6HDuKaUV5T+VBCgCXJBJCvYtfjoSk9dEwC6woMlxzuG/uH0AnicX+UaQ JngovlfJicueozbdWXN4tRLxnOCvj64TcF78s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UoCzq94MXVcOlUN/6wORFWX01+cATfQvzmGgy4cwnmY=; b=shnR1QzFIFzMrmAwLa+r0GNe3X2XQVTVDBHUVGjMHmXS7eaByULg1agdMX4aGq8L1/ HUVTtJ7FQDcpINnt9bdOSumWMw39DCeSC/5EhRzsZOBuzQDE9Wuwrv6+tNjvl8LSmjcy 3fiKqYajht41nuFQG/c0LID77TccYdVzCcKMr+OrbiV9dBH/s8V8yePxvw5wecZXxI6s v36feWAjPxPG2t5Mobyt65b3sTXYbWC1ib1Dw6yzsUBN2GzlF+nubY38tebeKbR1kWDq 0DJkMrPwpE4cEffto7fBm0uFW577qp8ajEyejy7Wl2Mu2gQHM7M0OMtm01hHMzc13cjk 7y1w== X-Gm-Message-State: AOAM532IMrvnK1vbx+Mp0DSPRqGuKndyhNEs25XEOesNt4gFKFKrg6Jl VPtIMwTy3tvQ9+K5K1g2g1KxdQ== X-Google-Smtp-Source: ABdhPJzsUqEu+w84DLdBr69wJOlXHGu7hB6DSAgppcb/GktobFXVx2JLqnIajZ8F6jsIntOdJCyFfg== X-Received: by 2002:a63:ba07:0:b0:382:4739:8941 with SMTP id k7-20020a63ba07000000b0038247398941mr17181809pgf.293.1648858884927; Fri, 01 Apr 2022 17:21:24 -0700 (PDT) Received: from localhost.swdvt.lab.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id k10-20020a056a00168a00b004f7e2a550ccsm4295050pfc.78.2022.04.01.17.21.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Apr 2022 17:21:24 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, kuba@kernel.org, gospo@broadcom.com, bpf@vger.kernel.org Subject: [PATCH net 2/3] bnxt_en: reserve space inside receive page for skb_shared_info Date: Fri, 1 Apr 2022 20:21:11 -0400 Message-Id: <1648858872-14682-3-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1648858872-14682-1-git-send-email-michael.chan@broadcom.com> References: <1648858872-14682-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Andy Gospodarek Insufficient space was being reserved in the page used for packet reception, so the interface MTU could be set too large to still have room for the contents of the packet when doing XDP redirect. This resulted in the following message when redirecting a packet between 3520 and 3822 bytes with an MTU of 3822: [311815.561880] XDP_WARN: xdp_update_frame_from_buff(line:200): Driver BUG: missing reserved tailroom Fixes: f18c2b77b2e4 ("bnxt_en: optimized XDP_REDIRECT support") Reviewed-by: Somnath Kotur Reviewed-by: Pavan Chebbi Signed-off-by: Andy Gospodarek Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index b4d3d051463b..98453a78cbd0 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -593,7 +593,8 @@ struct nqe_cn { #define BNXT_MAX_MTU 9500 #define BNXT_MAX_PAGE_MODE_MTU \ ((unsigned int)PAGE_SIZE - VLAN_ETH_HLEN - NET_IP_ALIGN - \ - XDP_PACKET_HEADROOM) + XDP_PACKET_HEADROOM - \ + SKB_DATA_ALIGN((unsigned int)sizeof(struct skb_shared_info))) #define BNXT_MIN_PKT_SIZE 52 From patchwork Sat Apr 2 00:21:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 12798871 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4197FC43219 for ; Sat, 2 Apr 2022 00:21:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353579AbiDBAXU (ORCPT ); Fri, 1 Apr 2022 20:23:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353528AbiDBAXS (ORCPT ); Fri, 1 Apr 2022 20:23:18 -0400 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 991311AC42A for ; Fri, 1 Apr 2022 17:21:27 -0700 (PDT) Received: by mail-pj1-x102a.google.com with SMTP id x14so533007pjf.2 for ; Fri, 01 Apr 2022 17:21:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rUCoJwRG6uYy4sxxy0GpZ+5krDGq6Oa0W/tZrkMg2+I=; b=dIY7efO1ay0LTT2jrqB1uDDpVHPeAGrvb/yj8+vFUajlaWKByE/BahEqsKffnjl4ka TgEd9wmxs0ZqfHpAHlRBP+eG48N/pF+GrHlxK9ZKhCRvd/weUyR9jBJMTuqESq3raCOI AVBqSVZ9xaltwt3BiBB41j4oB/D/6Faj2prPg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=rUCoJwRG6uYy4sxxy0GpZ+5krDGq6Oa0W/tZrkMg2+I=; b=WblmMINzVj35XO9dWa+ETq008TwV4zNclK09gdGC33wRQZAs/HO7t2XSbXBzi29JaR KDyNa568TRAl14gjAxJLFNGCYw4QraFpUo5T2Z4qdVtKN7Hhl8SvXu/wvVy35e5FlBUY nDA6ILtRS6WoR0yaNK2Q3wczlqP5OsNA2Yy3HHSnSg0FtQNX4luGWro0ipAWE1Ttj6uL BklrRshykHMBOfq0l6sh11pPB31NycM4DmzX/Z8fFctHTRJhUL1qoSk2aKDUGpvPnq04 UZnfyro7bAqio+bYE7L6KA5nAVlaoKpUiZ8q6kVBXRqZ7p2QXwugkku0xsImt59qsL61 8TkQ== X-Gm-Message-State: AOAM531eqhHyHBgojKbc9p7NU7M2iknGekbmcdnXBCVhDYsZJDui5vbm e3RebEJN8b5sTcL6kwKmXEVnzw== X-Google-Smtp-Source: ABdhPJxO8BFF9MOpu8VKJ0T0ybqrIQFfG9BYI+gHRClBXgGDwMfpFGKWVDYUV5RSTAIuDQBgGOEBtQ== X-Received: by 2002:a17:90b:3447:b0:1c6:fe01:675c with SMTP id lj7-20020a17090b344700b001c6fe01675cmr14294126pjb.59.1648858886266; Fri, 01 Apr 2022 17:21:26 -0700 (PDT) Received: from localhost.swdvt.lab.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id k10-20020a056a00168a00b004f7e2a550ccsm4295050pfc.78.2022.04.01.17.21.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Apr 2022 17:21:25 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, kuba@kernel.org, gospo@broadcom.com, bpf@vger.kernel.org, Ray Jui Subject: [PATCH net 3/3] bnxt_en: Prevent XDP redirect from running when stopping TX queue Date: Fri, 1 Apr 2022 20:21:12 -0400 Message-Id: <1648858872-14682-4-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1648858872-14682-1-git-send-email-michael.chan@broadcom.com> References: <1648858872-14682-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Ray Jui Add checks in the XDP redirect callback to prevent XDP from running when the TX ring is undergoing shutdown. Also remove redundant checks in the XDP redirect callback to validate the txr and the flag that indicates the ring supports XDP. The modulo arithmetic on 'tx_nr_rings_xdp' already guarantees the derived TX ring is an XDP ring. txr is also guaranteed to be valid after checking BNXT_STATE_OPEN and within RCU grace period. Fixes: f18c2b77b2e4 ("bnxt_en: optimized XDP_REDIRECT support") Reviewed-by: Vladimir Olovyannikov Signed-off-by: Ray Jui Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c index c0541ff00ac8..03b1d6c04504 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c @@ -229,14 +229,16 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames, ring = smp_processor_id() % bp->tx_nr_rings_xdp; txr = &bp->tx_ring[ring]; + if (READ_ONCE(txr->dev_state) == BNXT_DEV_STATE_CLOSING) + return -EINVAL; + if (static_branch_unlikely(&bnxt_xdp_locking_key)) spin_lock(&txr->xdp_tx_lock); for (i = 0; i < num_frames; i++) { struct xdp_frame *xdp = frames[i]; - if (!txr || !bnxt_tx_avail(bp, txr) || - !(bp->bnapi[ring]->flags & BNXT_NAPI_FLAG_XDP)) + if (!bnxt_tx_avail(bp, txr)) break; mapping = dma_map_single(&pdev->dev, xdp->data, xdp->len,