From patchwork Tue Apr 30 22:44:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13650005 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD7F22C853 for ; Tue, 30 Apr 2024 22:46:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517175; cv=none; b=kdNXbATSrGDrUXWa18AlLYqcUiVHeDPLi4JtVSeegEGG4OVGRhGBZWFg3u7n1u+kjxK3RWdBmfoGGk1k2gARWKG2xu7VzP0kdG/dkDqHknDN9r9Fl3sdwBYgP2w8JIT8Qqao7zpFuENVhAYZ/hpCWhUlEMe9gBzgtOdp2IXuz8s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517175; c=relaxed/simple; bh=6mVPS7TncWc8rZeUJrQnSatBetIYzPKAWstvnb9Wqmg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=N6dBydkbNQixTJFzUr0U4aqmuOGjCSK43/yeP9k21X6B2cdfA6f/5a0jHdxBhiBdHcroCk8xvdjeeVsOcBKkeEcfBMBqHY9zB8GdClObebDd31Z2FoBC2W0IInxKoKQSuN3aVqSossEErQlTMN0AFMbfLFtShgjrEHdaqXrJfx4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=gfbl6+FV; arc=none smtp.client-ip=209.85.215.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="gfbl6+FV" Received: by mail-pg1-f171.google.com with SMTP id 41be03b00d2f7-5e152c757a5so3960509a12.2 for ; Tue, 30 Apr 2024 15:46:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1714517173; x=1715121973; darn=vger.kernel.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=L4aiW0tCtnnxe4pUVwZynZF9OlmD9SNkaQDmahQztks=; b=gfbl6+FVEOmE9KPNsmY7ebb05vrtL+kB+ClYj35rpMWQZ+72Jtu27xkRPQO+LOhwtn cr533J3XCWBeM87NrUfE8+1cfT63Y+2IJavpAs1+s2GPDXJnqugkLcik6PxuaEAgepS2 RdbtiCVw5VTbAVRSt00JQmgP1XiEWZYpHJxGE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714517173; x=1715121973; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=L4aiW0tCtnnxe4pUVwZynZF9OlmD9SNkaQDmahQztks=; b=fBQ3nmfysMtdQaYDIs41MaB/lSI2QMGXNUs7gzpslOC5wwqnMD4+F4zu6E/BcHdQrQ OYoh+zi9Z9Z9lHiIjv6lPd07OgYNaNqiwTpBb+pOQaY1rPTuRBs6PYoXgXvY6EmxD8Sr EYGktPC2CLfVn2JW2qbvYmoQFE4TB4CoKzVRD4OOlBR8W4INju2Wxkv8qkXmPQYbC6jL v0q3KlVBqz43zYmJ0hNLG/dXP3x9rc9lWhetg7Ve/AAMBzCioNn39JE8ikuiRKcS0/wx YOrHDlG+NU2G0aDlnmGBZWJNc1Z1GbULv1dS0+rh6bdLUGkL0UJ8WE7A3PpEvXjsxn+y eTFg== X-Gm-Message-State: AOJu0YyYDyc858juXz7XCX7NQQhnCllLJFdxg8Bvc66g2yGutjTp5k58 19U+ySdU32CWOaXDpqZ46qxsj19ABkNGTrQq48TBdtvust46tYQSbSlJC/dOJw== X-Google-Smtp-Source: AGHT+IELbP68HhD34YUDfFjQwmkD0S83nH8oCLTpg70viNuifIf3Z8hp/lxMgOlCW1exJJI/dhBAcQ== X-Received: by 2002:a17:90a:bb89:b0:2a2:97f3:83b3 with SMTP id v9-20020a17090abb8900b002a297f383b3mr789203pjr.48.1714517172165; Tue, 30 Apr 2024 15:46:12 -0700 (PDT) Received: from lvnvda5233.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id s11-20020a170902a50b00b001eb2fb28eabsm7836476plq.227.2024.04.30.15.46.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Apr 2024 15:46:10 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew.gospodarek@broadcom.com, David Wei , Pavan Chebbi , Ajit Khaparde Subject: [PATCH net-next 1/7] bnxt_en: Fix and simplify bnxt_get_avail_msix() calls Date: Tue, 30 Apr 2024 15:44:32 -0700 Message-Id: <20240430224438.91494-2-michael.chan@broadcom.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20240430224438.91494-1-michael.chan@broadcom.com> References: <20240430224438.91494-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org David reported this issue of the driver not initializing on an older chip not running in the new resource manager mode (!BNXT_NEW_RM()). Sample dmesg: bnxt_en 0000:02:00.0 (unnamed net_device) (uninitialized): Able to reserve only 0 out of 9 requested RX rings bnxt_en 0000:02:00.0 (unnamed net_device) (uninitialized): Unable to reserve tx rings bnxt_en 0000:02:00.0 (unnamed net_device) (uninitialized): 2nd rings reservation failed. bnxt_en 0000:02:00.0 (unnamed net_device) (uninitialized): Not enough rings available. bnxt_en 0000:02:00.0: probe with driver bnxt_en failed with error -12 This is a regression caused by a recent commit that adds a call to bnxt_get_avail_msix() before MSIX is initialized: bnxt_set_dflt_rings() __bnxt_reserve_rings() bnxt_get_avail_msix() bnxt_get_avail_msix() returns a negative number if !BNXT_NEW_RM() and when MSIX is not initialized. This causes __bnxt_reserve_rings() to fail in this call sequence and the driver aborts initialization. Before this commit in 2022: 303432211324 ("bnxt_en: Remove runtime interrupt vector allocation") MSIX allocation for RoCE was dynamic and bnxt_get_avail_msix() was used to determine the available run-time MSIX available for RoCE. Today, bnxt_get_avail_msix() is only used to reserve some available MSIX ahead of time to be ready when the RoCE driver loads. It only needs to be called when BNXT_NEW_RM() is true because older chips do not require reservations for MSIX. Simplify bnxt_get_avail_msix() to only consider the BNXT_NEW_RM() case and only make this call if BNXT_NEW_RM() is true. Also change bnxt_get_avail_msix() to static since it is only used in bnxt.c. Reported-by: David Wei Fixes: d630624ebd70 ("bnxt_en: Utilize ulp client resources if RoCE is not registered") Reviewed-by: Pavan Chebbi Reviewed-by: Andy Gospodarek Reviewed-by: Ajit Khaparde Signed-off-by: Michael Chan --- Cc: David Wei --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 20 ++++++++------------ drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 - 2 files changed, 8 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index be96bb494ae6..0eb880766012 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -7459,6 +7459,8 @@ static bool bnxt_rings_ok(struct bnxt *bp, struct bnxt_hw_rings *hwr) hwr->stat && (hwr->cp_p5 || !(bp->flags & BNXT_FLAG_CHIP_P5_PLUS)); } +static int bnxt_get_avail_msix(struct bnxt *bp, int num); + static int __bnxt_reserve_rings(struct bnxt *bp) { struct bnxt_hw_rings hwr = {0}; @@ -7471,7 +7473,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp) if (!bnxt_need_reserve_rings(bp)) return 0; - if (!bnxt_ulp_registered(bp->edev)) { + if (!bnxt_ulp_registered(bp->edev) && BNXT_NEW_RM(bp)) { ulp_msix = bnxt_get_avail_msix(bp, bp->ulp_num_msix_want); if (!ulp_msix) bnxt_set_ulp_stat_ctxs(bp, 0); @@ -10474,19 +10476,13 @@ unsigned int bnxt_get_avail_stat_ctxs_for_en(struct bnxt *bp) return bnxt_get_max_func_stat_ctxs(bp) - bnxt_get_func_stat_ctxs(bp); } -int bnxt_get_avail_msix(struct bnxt *bp, int num) +/* Only called if BNXT_NEW_RM() is true to get the available MSIX to + * reserve ahead of time before RoCE is registered. + */ +static int bnxt_get_avail_msix(struct bnxt *bp, int num) { - int max_cp = bnxt_get_max_func_cp_rings(bp); int max_irq = bnxt_get_max_func_irqs(bp); int total_req = bp->cp_nr_rings + num; - int max_idx, avail_msix; - - max_idx = bp->total_irqs; - if (!(bp->flags & BNXT_FLAG_CHIP_P5_PLUS)) - max_idx = min_t(int, bp->total_irqs, max_cp); - avail_msix = max_idx - bp->cp_nr_rings; - if (!BNXT_NEW_RM(bp) || avail_msix >= num) - return avail_msix; if (max_irq < total_req) { num = max_irq - bp->cp_nr_rings; @@ -10619,7 +10615,7 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init) if (!bnxt_need_reserve_rings(bp)) return 0; - if (!bnxt_ulp_registered(bp->edev)) { + if (!bnxt_ulp_registered(bp->edev) && BNXT_NEW_RM(bp)) { int ulp_msix = bnxt_get_avail_msix(bp, bp->ulp_num_msix_want); if (ulp_msix > bp->ulp_num_msix_want) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index ad57ef051798..0c680032ab66 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -2736,7 +2736,6 @@ unsigned int bnxt_get_max_func_stat_ctxs(struct bnxt *bp); unsigned int bnxt_get_avail_stat_ctxs_for_en(struct bnxt *bp); unsigned int bnxt_get_max_func_cp_rings(struct bnxt *bp); unsigned int bnxt_get_avail_cp_rings_for_en(struct bnxt *bp); -int bnxt_get_avail_msix(struct bnxt *bp, int num); int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init); void bnxt_tx_disable(struct bnxt *bp); void bnxt_tx_enable(struct bnxt *bp); From patchwork Tue Apr 30 22:44:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13650006 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 82E411CD21 for ; Tue, 30 Apr 2024 22:46:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517176; cv=none; b=tkQb0QB5Ke4ZhNtlX05Y0cjeXgzPrGdVdYWZ/xSuBa0Kts5Th5VLJX6D3vXFR7C2G+8hmd9fhnV8CfzHvzsYJcx3NdVA40HtH8JoPnjHo1pVkAcoXcMsPZqu4P7eYmZmU9FlOxdUUS2hu24lGdxaBRV3e8x7xVZghR0S91YsOzk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517176; c=relaxed/simple; bh=b9xN8kwZWqeZgpQmlkl0t17iScBMtGRr7qoWQ16qxIQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=hGqK7GXu8hMIu7ziGQfvu6KemMFxIhmspa8rZg7zpnJtiT1GtcReEmtOVL7eF+nu/i16TGwhcNOrXUE2bXiq5ZDjdjObK0gnYqNy7g3X3ySO3omTbnQwpILmR4dViKHtot/huTNM1bTaK2HbayKhGYMDCJfBMR5eb+dGZMlWlN0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=ahPulP1H; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="ahPulP1H" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-1e5c7d087e1so54314815ad.0 for ; Tue, 30 Apr 2024 15:46:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1714517174; x=1715121974; darn=vger.kernel.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=firrJvki98oSDb8LiFa5e2fqHhjUz+5CIjU/D7FJC9I=; b=ahPulP1HdOvoSA8FsoMRwfELRXiuzLXf/3vmi/sT66Iv1esJtbziuCocdPjPAM7aW+ lbjOOUD+Gv2yTUM61ruLW9uwY071U/Jm/L1IHZV3fW1JUNgjgXJPsCwZvGhVSCL+Lz/3 0gVYDOSEuEJpuBcxBOilV7vyLol7xMIfTcU7o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714517174; x=1715121974; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=firrJvki98oSDb8LiFa5e2fqHhjUz+5CIjU/D7FJC9I=; b=jQPo7qp14WEJ6JP3EHiiLvCle1XltzEj4tAemOU4MSJUMjN7pbBRHZa5hNZ/dvxsxn Gdceu/KSeaGWtwvE3Zq2MgcSteZOsUcnJTUe+gtbufv3p1vD55mk1dbJV8qDlhVT0G73 KcYQOa22snNHpr0o+2/+pLTyRTNeDoW9+pw9halp46Ac94oF28zhbp0NT77ZzJpFnDBK brz+zNATefqM4jXPVITWVHpJndutfLtSGItqelbz3BKmPYrMwKp2Gtc32iGsYQAT2KFa 3Yj5xIcEQixEej3YXvTP82S3ttyc48AjMuTX8FAIytDIRTdn31xo9XaJ8W+uRkNyvmjT ZfRg== X-Gm-Message-State: AOJu0Yw6/Tae7ijMhm/SL9nL5y6OQGsKT3m4Ma/hlU5lndyTmvawJ9j2 4uDrLB1/mpxuMDbI4rfPl1nW/Xk1ShBCioUpIETUr2j50RYaZC7cu0iBHm+tYw== X-Google-Smtp-Source: AGHT+IE8WnWL/zt0HUpP+sFwaoeamR7xWaJZXK2ME3IFamrs23oUGXt4Kt2Pvpe266mOFuS0198vpg== X-Received: by 2002:a17:902:d54d:b0:1eb:dae:714f with SMTP id z13-20020a170902d54d00b001eb0dae714fmr992904plf.9.1714517173359; Tue, 30 Apr 2024 15:46:13 -0700 (PDT) Received: from lvnvda5233.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id s11-20020a170902a50b00b001eb2fb28eabsm7836476plq.227.2024.04.30.15.46.12 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Apr 2024 15:46:12 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew.gospodarek@broadcom.com, Edwin Peer Subject: [PATCH net-next 2/7] bnxt_en: share NQ ring sw_stats memory with subrings Date: Tue, 30 Apr 2024 15:44:33 -0700 Message-Id: <20240430224438.91494-3-michael.chan@broadcom.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20240430224438.91494-1-michael.chan@broadcom.com> References: <20240430224438.91494-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Edwin Peer On P5_PLUS chips and later, the NQ rings have subrings for RX and TX completions respectively. These subrings are passed to the poll function instead of the base NQ, but each ring carries its own copy of the software ring statistics. For stats to be conveniently accessible in __bnxt_poll_work(), the statistics memory should either be shared between the NQ and its subrings or the subrings need to be included in the ethtool stats aggregation logic. This patch opts for the former, because it's more efficient and less confusing having the software statistics for a ring exist in a single place. Before this patch, the counter will not be displayed if the "wrong" cpr->sw_stats was used to increment a counter. Link: https://lore.kernel.org/netdev/CACKFLikEhVAJA+osD7UjQNotdGte+fth7zOy7yDdLkTyFk9Pyw@mail.gmail.com/ Signed-off-by: Edwin Peer Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 40 +++++++++++-------- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 +- .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 4 +- 3 files changed, 27 insertions(+), 19 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 0eb880766012..38134b995478 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -1811,7 +1811,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp, skb = bnxt_copy_skb(bnapi, data_ptr, len, mapping); if (!skb) { bnxt_abort_tpa(cpr, idx, agg_bufs); - cpr->bnapi->cp_ring.sw_stats.rx.rx_oom_discards += 1; + cpr->sw_stats->rx.rx_oom_discards += 1; return NULL; } } else { @@ -1821,7 +1821,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp, new_data = __bnxt_alloc_rx_frag(bp, &new_mapping, GFP_ATOMIC); if (!new_data) { bnxt_abort_tpa(cpr, idx, agg_bufs); - cpr->bnapi->cp_ring.sw_stats.rx.rx_oom_discards += 1; + cpr->sw_stats->rx.rx_oom_discards += 1; return NULL; } @@ -1837,7 +1837,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp, if (!skb) { skb_free_frag(data); bnxt_abort_tpa(cpr, idx, agg_bufs); - cpr->bnapi->cp_ring.sw_stats.rx.rx_oom_discards += 1; + cpr->sw_stats->rx.rx_oom_discards += 1; return NULL; } skb_reserve(skb, bp->rx_offset); @@ -1848,7 +1848,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp, skb = bnxt_rx_agg_pages_skb(bp, cpr, skb, idx, agg_bufs, true); if (!skb) { /* Page reuse already handled by bnxt_rx_pages(). */ - cpr->bnapi->cp_ring.sw_stats.rx.rx_oom_discards += 1; + cpr->sw_stats->rx.rx_oom_discards += 1; return NULL; } } @@ -2106,7 +2106,7 @@ static int bnxt_rx_pkt(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, rc = -EIO; if (rx_err & RX_CMPL_ERRORS_BUFFER_ERROR_MASK) { - bnapi->cp_ring.sw_stats.rx.rx_buf_errors++; + bnapi->cp_ring.sw_stats->rx.rx_buf_errors++; if (!(bp->flags & BNXT_FLAG_CHIP_P5_PLUS) && !(bp->fw_cap & BNXT_FW_CAP_RING_MONITOR)) { netdev_warn_once(bp->dev, "RX buffer error %x\n", @@ -2222,7 +2222,7 @@ static int bnxt_rx_pkt(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, } else { if (rxcmp1->rx_cmp_cfa_code_errors_v2 & RX_CMP_L4_CS_ERR_BITS) { if (dev->features & NETIF_F_RXCSUM) - bnapi->cp_ring.sw_stats.rx.rx_l4_csum_errors++; + bnapi->cp_ring.sw_stats->rx.rx_l4_csum_errors++; } } @@ -2259,7 +2259,7 @@ static int bnxt_rx_pkt(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, return rc; oom_next_rx: - cpr->bnapi->cp_ring.sw_stats.rx.rx_oom_discards += 1; + cpr->sw_stats->rx.rx_oom_discards += 1; rc = -ENOMEM; goto next_rx; } @@ -2308,7 +2308,7 @@ static int bnxt_force_rx_discard(struct bnxt *bp, } rc = bnxt_rx_pkt(bp, cpr, raw_cons, event); if (rc && rc != -EBUSY) - cpr->bnapi->cp_ring.sw_stats.rx.rx_netpoll_discards += 1; + cpr->sw_stats->rx.rx_netpoll_discards += 1; return rc; } @@ -3951,6 +3951,7 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp) if (rc) return rc; cpr2->bnapi = bnapi; + cpr2->sw_stats = cpr->sw_stats; cpr2->cp_idx = k; if (!k && rx) { bp->rx_ring[i].rx_cpr = cpr2; @@ -4792,6 +4793,9 @@ static void bnxt_free_ring_stats(struct bnxt *bp) struct bnxt_cp_ring_info *cpr = &bnapi->cp_ring; bnxt_free_stats_mem(bp, &cpr->stats); + + kfree(cpr->sw_stats); + cpr->sw_stats = NULL; } } @@ -4806,6 +4810,10 @@ static int bnxt_alloc_stats(struct bnxt *bp) struct bnxt_napi *bnapi = bp->bnapi[i]; struct bnxt_cp_ring_info *cpr = &bnapi->cp_ring; + cpr->sw_stats = kzalloc(sizeof(*cpr->sw_stats), GFP_KERNEL); + if (!cpr->sw_stats) + return -ENOMEM; + cpr->stats.len = size; rc = bnxt_alloc_stats_mem(bp, &cpr->stats, !i); if (rc) @@ -10807,9 +10815,9 @@ static void bnxt_disable_napi(struct bnxt *bp) cpr = &bnapi->cp_ring; if (bnapi->tx_fault) - cpr->sw_stats.tx.tx_resets++; + cpr->sw_stats->tx.tx_resets++; if (bnapi->in_reset) - cpr->sw_stats.rx.rx_resets++; + cpr->sw_stats->rx.rx_resets++; napi_disable(&bnapi->napi); if (bnapi->rx_ring) cancel_work_sync(&cpr->dim.work); @@ -12334,8 +12342,8 @@ static void bnxt_get_ring_stats(struct bnxt *bp, stats->tx_dropped += BNXT_GET_RING_STATS64(sw, tx_error_pkts); stats->rx_dropped += - cpr->sw_stats.rx.rx_netpoll_discards + - cpr->sw_stats.rx.rx_oom_discards; + cpr->sw_stats->rx.rx_netpoll_discards + + cpr->sw_stats->rx.rx_oom_discards; } } @@ -12402,7 +12410,7 @@ static void bnxt_get_one_ring_err_stats(struct bnxt *bp, struct bnxt_total_ring_err_stats *stats, struct bnxt_cp_ring_info *cpr) { - struct bnxt_sw_stats *sw_stats = &cpr->sw_stats; + struct bnxt_sw_stats *sw_stats = cpr->sw_stats; u64 *hw_stats = cpr->stats.sw_stats; stats->rx_total_l4_csum_errors += sw_stats->rx.rx_l4_csum_errors; @@ -13245,7 +13253,7 @@ static void bnxt_rx_ring_reset(struct bnxt *bp) rxr->bnapi->in_reset = false; bnxt_alloc_one_rx_ring(bp, i); cpr = &rxr->bnapi->cp_ring; - cpr->sw_stats.rx.rx_resets++; + cpr->sw_stats->rx.rx_resets++; if (bp->flags & BNXT_FLAG_AGG_RINGS) bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod); bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod); @@ -13457,7 +13465,7 @@ static void bnxt_chk_missed_irq(struct bnxt *bp) bnxt_dbg_hwrm_ring_info_get(bp, DBG_RING_INFO_GET_REQ_RING_TYPE_L2_CMPL, fw_ring_id, &val[0], &val[1]); - cpr->sw_stats.cmn.missed_irqs++; + cpr->sw_stats->cmn.missed_irqs++; } } } @@ -14765,7 +14773,7 @@ static void bnxt_get_queue_stats_rx(struct net_device *dev, int i, stats->bytes += BNXT_GET_RING_STATS64(sw, rx_mcast_bytes); stats->bytes += BNXT_GET_RING_STATS64(sw, rx_bcast_bytes); - stats->alloc_fail = cpr->sw_stats.rx.rx_oom_discards; + stats->alloc_fail = cpr->sw_stats->rx.rx_oom_discards; } static void bnxt_get_queue_stats_tx(struct net_device *dev, int i, diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 0c680032ab66..003c00b229f2 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -1152,7 +1152,7 @@ struct bnxt_cp_ring_info { struct bnxt_stats_mem stats; u32 hw_stats_ctx_id; - struct bnxt_sw_stats sw_stats; + struct bnxt_sw_stats *sw_stats; struct bnxt_ring_struct cp_ring_struct; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c index 68444234b268..6de3cfcea90f 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c @@ -631,13 +631,13 @@ static void bnxt_get_ethtool_stats(struct net_device *dev, buf[j] = sw_stats[k]; skip_tpa_ring_stats: - sw = (u64 *)&cpr->sw_stats.rx; + sw = (u64 *)&cpr->sw_stats->rx; if (is_rx_ring(bp, i)) { for (k = 0; k < NUM_RING_RX_SW_STATS; j++, k++) buf[j] = sw[k]; } - sw = (u64 *)&cpr->sw_stats.cmn; + sw = (u64 *)&cpr->sw_stats->cmn; for (k = 0; k < NUM_RING_CMN_SW_STATS; j++, k++) buf[j] = sw[k]; } From patchwork Tue Apr 30 22:44:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13650007 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9DE332C853 for ; Tue, 30 Apr 2024 22:46:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517178; cv=none; b=FHnLDHX/hcsVsD3VO6y+gerkXRGAU9afCBLw9eOoj11fYeoJCYNZRtSzuT5Piqq8gPfY6xAiLWXz16OHOh74l0lJBNWVWcrn6d8+uPcRV0PPO64/8cK5SeAsyj+K4jK+9bn+Fsp8IJBApXSZM2FZvZfFtAHzL9cWyKBMqvqA+UA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517178; c=relaxed/simple; bh=tkGXFQXtAWmsicJg0LJ2zyBkyzztbvR2v4gqgvSJNgY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=EssGHwgVSUy49QLfoj6yB1xoTpGz4RElXcAsWbIoXPkEcAPvUUP5zzJ2dQ3FSApmUWxdlJH/aGdGQ2kYCDtxBxIWcBCgoy4Bp3ClhZTxW8jNFPPeTJJgFhHLK7vEd4yNvPDLCy9T/CeWyhquRi0uLhlgCs+dbuQeTPTefUZDZzQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=dPv37e/b; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="dPv37e/b" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1ec41d82b8bso12335675ad.2 for ; Tue, 30 Apr 2024 15:46:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1714517176; x=1715121976; darn=vger.kernel.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=0FwS7qe6zFjFa4NLwF+MQne8+Z4JoQ0VM2BJTZ+bUHM=; b=dPv37e/bmA1/Wu3SPNdnKxrwqX47OkKPaKddwO8sNuFmVmKW53TAS5e9d2ux+iG+eC ttad2SuNWGGkJDCZHcewZNcX+BKcgsSPnAtZwz+Ukl6H0y37Pj0QeG4bOq6vxxtz3xoP 1WQsYizYbMSsOD68gTmKYXG/u40p019SX8zNs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714517176; x=1715121976; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0FwS7qe6zFjFa4NLwF+MQne8+Z4JoQ0VM2BJTZ+bUHM=; b=HxOVJPk35cKXKY+fFXTnu3sxM7gILUy6kRvZSCLrysJ7sin1NhD8yQ++ThI7Z0fqeO GfvWUdXhC0ZCdDfm6WfIe6kD4uDQkx4I+0P4/DDzOYZzygLXKUd/Yj2MYUSP06UaLKmr hUwzpZzvyaxrtJXTjrmtX175DpfKNeQpnWRj0UskFUjcA6VkKVc/1NKvvzdJ0ZyrLdgc o9AbfySygQNU+ZiKT8MilYOmmAIuYdMgdtqwC23Ga3wlqtXovl3wz8evkGNawVJ+Xrln oQJ2g1cf1Bz8UvDMepy0kbA9YKWb3P4ft/VRcFe4c8VqHeRd5/iG1cgg0VK5Au68l5JJ A9Dw== X-Gm-Message-State: AOJu0YxVumJK4HhFXWCxFu399DtmhnnJz7qKpjXDreqswps75efo7Hb4 cZYcNhZPm49hokFI4IL2yt1nbLF8+cvE78EXl8nmgQ1RCPS0FLKpksyVDO71ZA== X-Google-Smtp-Source: AGHT+IE62mEZCwNX4fpGfvu61VW5wrI/m3QlSrHvbTFZtxogrx/kR4hkkwwvzXnj9g9cnzYf4gLvAw== X-Received: by 2002:a17:902:f682:b0:1e7:d482:9e07 with SMTP id l2-20020a170902f68200b001e7d4829e07mr859876plg.12.1714517175343; Tue, 30 Apr 2024 15:46:15 -0700 (PDT) Received: from lvnvda5233.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id s11-20020a170902a50b00b001eb2fb28eabsm7836476plq.227.2024.04.30.15.46.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Apr 2024 15:46:14 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew.gospodarek@broadcom.com, Kalesh AP , Selvin Thyparampil Xavier , Vikas Gupta , Pavan Chebbi Subject: [PATCH net-next 3/7] bnxt_en: Don't support offline self test when RoCE driver is loaded Date: Tue, 30 Apr 2024 15:44:34 -0700 Message-Id: <20240430224438.91494-4-michael.chan@broadcom.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20240430224438.91494-1-michael.chan@broadcom.com> References: <20240430224438.91494-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Kalesh AP Offline self test is a very disruptive operation for RoCE and requires all active QPs to be destroyed. With a large number of QPs, it can take a long time to destroy all the QPs and can timeout. Do not allow ethtool offline self test if the RoCE driver is registered on the device. Reviewed-by: Selvin Thyparampil Xavier Reviewed-by: Vikas Gupta Reviewed-by: Pavan Chebbi Signed-off-by: Kalesh AP Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c index 6de3cfcea90f..8763f8a01457 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c @@ -4820,6 +4820,14 @@ static void bnxt_self_test(struct net_device *dev, struct ethtool_test *etest, if (!bp->num_tests || !BNXT_PF(bp)) return; + + if (etest->flags & ETH_TEST_FL_OFFLINE && + bnxt_ulp_registered(bp->edev)) { + etest->flags |= ETH_TEST_FL_FAILED; + netdev_warn(dev, "Offline tests cannot be run with RoCE driver loaded\n"); + return; + } + memset(buf, 0, sizeof(u64) * bp->num_tests); if (!netif_running(dev)) { etest->flags |= ETH_TEST_FL_FAILED; @@ -4850,7 +4858,6 @@ static void bnxt_self_test(struct net_device *dev, struct ethtool_test *etest, if (!offline) { bnxt_run_fw_tests(bp, test_mask, &test_results); } else { - bnxt_ulp_stop(bp); bnxt_close_nic(bp, true, false); bnxt_run_fw_tests(bp, test_mask, &test_results); @@ -4861,7 +4868,6 @@ static void bnxt_self_test(struct net_device *dev, struct ethtool_test *etest, if (rc) { bnxt_hwrm_mac_loopback(bp, false); etest->flags |= ETH_TEST_FL_FAILED; - bnxt_ulp_start(bp, rc); return; } if (bnxt_run_loopback(bp)) @@ -4888,7 +4894,6 @@ static void bnxt_self_test(struct net_device *dev, struct ethtool_test *etest, bnxt_hwrm_phy_loopback(bp, false, false); bnxt_half_close_nic(bp); rc = bnxt_open_nic(bp, true, true); - bnxt_ulp_start(bp, rc); } if (rc || bnxt_test_irq(bp)) { buf[BNXT_IRQ_TEST_IDX] = 1; From patchwork Tue Apr 30 22:44:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13650008 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E3881CD21 for ; Tue, 30 Apr 2024 22:46:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517178; cv=none; b=pyy3VshZU55oCU76P4zM/SOXsYqYUIek/VEdQjcWJxTETVwXeZiwIn4fi+HWfAjAvIKEYwSoQrjY/gmrcbwOqP/EIWivXKcF17s6TAPq0FyfuCDD2feCtlGfx5YUqoVv34XxvqdiTrIY8/seMkPIB5wokZ1ED5k49sVDKpWUDg0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517178; c=relaxed/simple; bh=gRJidVXNUR5dQZuMIxt2C+ncIYoQB8UKXg3x2O8bM4I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=M8rdD3UM1HP1O0yiuvHXFT6oRc6/mTrkJNvfaMdejov4j5IJzgfU0bvT/bApMPpz1s5xIgIYW9b/vHQn3DtACNtDiF0j7+wHlKybU9k0C2g2tDi8pBL38sVl5Xr34mj3hQvTdXELXS39D13FQwJFxs1C+yEvjCnjhzUakUf13uo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=Rn423Ypw; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="Rn423Ypw" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-1e5c7d087e1so54315075ad.0 for ; Tue, 30 Apr 2024 15:46:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1714517177; x=1715121977; darn=vger.kernel.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=sUDAzuO3n68gKGxKRGquuyLNv4X9EOcJtergvgTofqI=; b=Rn423YpwqE+rIFL/gDwMLSMoK30f+i69Cuz8FNQTIhobZvhKaCQf3EdIlApP8FwS3v O/EeI2kzmzoMK+OcOx/ugYKqtPg/rvgyDzqjqWCTDUQXN7Na/kwt6GoeP+30YA//IUc+ akenbg1fSTeMaoAFYWxAgHExcMff/9ZSEmiRY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714517177; x=1715121977; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sUDAzuO3n68gKGxKRGquuyLNv4X9EOcJtergvgTofqI=; b=WwzmHS8kSy/nU3s0wGD/6ysTh0W79o7LoaD/ep8Sv+fOjUd6SeBbgdsXVN9OBZW1IG XLb/3Mok4fWLrRyv+UZzghDwjoA3wye6gn93vPjj228A/cvEGDzCIRS9n1aPxD9dpYaU WrwkdeMCCncl6FXJ3sbfaBQgcFrk4nUb6bhzrzPGYFN0wgL6/PWFoZHPcKPhUh29Lfgu AvwRhE891W2vqE6kBRWAq/hiiu1BVRw5N6nvHPwQjJDDjaKatrMu2veQ/YSnVe12z3yn A3u5tf5m7cmlGKY3J6jh33aenlgKNnQ7ahYYwV1J7/auk8qJ6J66wAL8qEX0dLHslK4M G67w== X-Gm-Message-State: AOJu0YzSK0ey9R+VA7RjRpw0uXAShpJjgb1UKPkrz6EEmWm0CgwAwfxm v+6IwjqZjCL3q/6N0dLoZKi7+OfkPGkf7BLcZZVZ6XfYR9UqtPi5i9bhD1hB3g== X-Google-Smtp-Source: AGHT+IEyjlaNLWPgHd9/gYd1mP4YhicFNT6tyh56ozk8YgDw2q4KcqPCo+9wPIy+lc34JO9X3zB8CQ== X-Received: by 2002:a17:902:b201:b0:1e4:4000:a576 with SMTP id t1-20020a170902b20100b001e44000a576mr810658plr.43.1714517176522; Tue, 30 Apr 2024 15:46:16 -0700 (PDT) Received: from lvnvda5233.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id s11-20020a170902a50b00b001eb2fb28eabsm7836476plq.227.2024.04.30.15.46.15 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Apr 2024 15:46:16 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew.gospodarek@broadcom.com, Pavan Chebbi Subject: [PATCH net-next 4/7] bnxt_en: Don't call ULP_STOP/ULP_START during L2 reset Date: Tue, 30 Apr 2024 15:44:35 -0700 Message-Id: <20240430224438.91494-5-michael.chan@broadcom.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20240430224438.91494-1-michael.chan@broadcom.com> References: <20240430224438.91494-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org There is no need to call ULP_STOP and ULP_START before and after the L2 reset in bnxt_reset_task(). This L2 reset is done after detecting TX timeout, RX ring errors, or VF config changes. The L2 reset does not affect RoCE since the firmware is not reset and the backing store is left alone. Reviewed-by: Andy Gospodarek Reviewed-by: Pavan Chebbi Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 38134b995478..4d7b4eabe0af 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -13085,17 +13085,8 @@ static void bnxt_reset_task(struct bnxt *bp, bool silent) if (!silent) bnxt_dbg_dump_states(bp); if (netif_running(bp->dev)) { - int rc; - - if (silent) { - bnxt_close_nic(bp, false, false); - bnxt_open_nic(bp, false, false); - } else { - bnxt_ulp_stop(bp); - bnxt_close_nic(bp, true, false); - rc = bnxt_open_nic(bp, true, false); - bnxt_ulp_start(bp, rc); - } + bnxt_close_nic(bp, !silent, false); + bnxt_open_nic(bp, !silent, false); } } From patchwork Tue Apr 30 22:44:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13650009 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF0381E86E for ; Tue, 30 Apr 2024 22:46:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517181; cv=none; b=K+gCWyITm8Wv2Z5IgcMWUIe5SKQhhDtlpDRqStfMr5OV3UjPV+PDU250axgiOsqwCiNcOIUMOaikcIr12dUdYuW8dbibOnnXKe0CykZp5lxQM1GHtZRI0vl2J+Mc1mcEWOEa8oizPLUCWf9ugw9rAukvqPbV7V/GdjvX3ZjxZSI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517181; c=relaxed/simple; bh=kdzlwUF/UHFb2w0rbYfNxBu/C4MVgbSz5wt6LIrJALY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=LJ9nk0d34kNEHGUxt42HvmlH0B0Is7sAA1saJjzvlJi+7UIqSf+s5wMaW+HMDpAxGGZ766dHADlQA0vobe/CdKoTVUHFnjIcoAJcqYQCKa3PEmnKVun6CPRvAm6B7yM9gpzwG6M0it/jbtoxqFWkAoMnnmvMQrsJs7q7vrJeUMw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=TCiLGaSu; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="TCiLGaSu" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1ec41d82b8bso12336055ad.2 for ; Tue, 30 Apr 2024 15:46:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1714517179; x=1715121979; darn=vger.kernel.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=ow6G6TnDEsRyRmPppeXnL6X2ewoO6C6MophJ7XX5+0M=; b=TCiLGaSuc0NQ0CoDUFwauAauIUaBhBw4EvCFd2liX0EWIC3EQXPBbRTuA0LgEC1bYP kp1YFs5AJqT9PupaLsvpbFcx/8qiFgqhZn7Q+FmJoeyLRuCZswQLH4x9iXMyIY7n4Dse 7EFhhD0A+m7ynumBeU0pxMrLbFhJCnd3fij2g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714517179; x=1715121979; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ow6G6TnDEsRyRmPppeXnL6X2ewoO6C6MophJ7XX5+0M=; b=srSIvePd49OERfYRyT2LUytxW/jiv0ycE/lqEeIQLB9kX4bmpynGzcyVeQTSgcfe1u Bgd3eaW2Yw6Vb5HAZGC1lVAq7ZMcMESLk2oECR5HN7JOw3pU1yYBJihUfJs2MsJAz6zm PMRiniEhR4oTJYQjuyqYpLUDn6H/9tiyS4t48/P5vVT6lMedDCiTD6BLTut/RpfQty7B 7J71OJBpCWfNW3UW9h5WOjFBLe4V/6eveGTd2WUjurADJcXK/nJRyNvH+4izqXaO6U44 4PfUk3O0Mc0LPqy2ua9H95rKrevzYyU+5FQcGn6tnaCkuP7uV5hXvnMjzJzoF+d+Qezc ZYCg== X-Gm-Message-State: AOJu0Yzul6QgRMGWPLPGDMowVXZ+EJmBEBoX+ZTd0a0zRylzv8MgUZA7 9aYQ631Sm/4H2n57BS/r5OVI42pSCf4DAw7HcvUh9YosEeOxYWbFKG4cZ6czqQ== X-Google-Smtp-Source: AGHT+IHQErMnaGaL2b5tBUz4PWQL1B5EpRf0oBVkCmpLE8AvF6U1yvnOYxmeLMvxQs739UyciKemHg== X-Received: by 2002:a17:902:c403:b0:1e0:ca47:4d96 with SMTP id k3-20020a170902c40300b001e0ca474d96mr811625plk.3.1714517178668; Tue, 30 Apr 2024 15:46:18 -0700 (PDT) Received: from lvnvda5233.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id s11-20020a170902a50b00b001eb2fb28eabsm7836476plq.227.2024.04.30.15.46.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Apr 2024 15:46:17 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew.gospodarek@broadcom.com, Kalesh AP , Selvin Thyparampil Xavier , Vikas Gupta , Pavan Chebbi Subject: [PATCH net-next 5/7] bnxt_en: Add a mutex to synchronize ULP operations Date: Tue, 30 Apr 2024 15:44:36 -0700 Message-Id: <20240430224438.91494-6-michael.chan@broadcom.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20240430224438.91494-1-michael.chan@broadcom.com> References: <20240430224438.91494-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Kalesh AP The current scheme relies heavily on the RTNL lock for all ULP operations between the L2 and the RoCE driver. Add a new en_dev_lock mutex so that the asynchronous ULP_STOP and ULP_START operations can be serialized with bnxt_register_dev() and bnxt_unregister_dev() calls without relying on the RTNL lock. The next patch will remove the RTNL lock from the ULP_STOP and ULP_START calls. Reviewed-by: Selvin Thyparampil Xavier Reviewed-by: Vikas Gupta Reviewed-by: Pavan Chebbi Signed-off-by: Kalesh AP Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 20 ++++++++++++++++++- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h | 3 +++ 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index d8927838f1cf..ba3fa1c2e5d9 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -113,6 +113,7 @@ int bnxt_register_dev(struct bnxt_en_dev *edev, int rc = 0; rtnl_lock(); + mutex_lock(&edev->en_dev_lock); if (!bp->irq_tbl) { rc = -ENODEV; goto exit; @@ -136,6 +137,7 @@ int bnxt_register_dev(struct bnxt_en_dev *edev, bnxt_fill_msix_vecs(bp, bp->edev->msix_entries); edev->flags |= BNXT_EN_FLAG_MSIX_REQUESTED; exit: + mutex_unlock(&edev->en_dev_lock); rtnl_unlock(); return rc; } @@ -150,6 +152,7 @@ void bnxt_unregister_dev(struct bnxt_en_dev *edev) ulp = edev->ulp_tbl; rtnl_lock(); + mutex_lock(&edev->en_dev_lock); if (ulp->msix_requested) edev->flags &= ~BNXT_EN_FLAG_MSIX_REQUESTED; edev->ulp_tbl->msix_requested = 0; @@ -165,6 +168,7 @@ void bnxt_unregister_dev(struct bnxt_en_dev *edev) msleep(100); i++; } + mutex_unlock(&edev->en_dev_lock); rtnl_unlock(); return; } @@ -223,6 +227,12 @@ void bnxt_ulp_stop(struct bnxt *bp) if (!edev) return; + mutex_lock(&edev->en_dev_lock); + if (!bnxt_ulp_registered(edev)) { + mutex_unlock(&edev->en_dev_lock); + return; + } + edev->flags |= BNXT_EN_FLAG_ULP_STOPPED; if (aux_priv) { struct auxiliary_device *adev; @@ -237,6 +247,7 @@ void bnxt_ulp_stop(struct bnxt *bp) adrv->suspend(adev, pm); } } + mutex_unlock(&edev->en_dev_lock); } void bnxt_ulp_start(struct bnxt *bp, int err) @@ -252,6 +263,12 @@ void bnxt_ulp_start(struct bnxt *bp, int err) if (err) return; + mutex_lock(&edev->en_dev_lock); + if (!bnxt_ulp_registered(edev)) { + mutex_unlock(&edev->en_dev_lock); + return; + } + if (edev->ulp_tbl->msix_requested) bnxt_fill_msix_vecs(bp, edev->msix_entries); @@ -267,7 +284,7 @@ void bnxt_ulp_start(struct bnxt *bp, int err) adrv->resume(adev); } } - + mutex_unlock(&edev->en_dev_lock); } void bnxt_ulp_irq_stop(struct bnxt *bp) @@ -383,6 +400,7 @@ static void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp) edev->l2_db_size = bp->db_size; edev->l2_db_size_nc = bp->db_size; edev->l2_db_offset = bp->db_offset; + mutex_init(&edev->en_dev_lock); if (bp->flags & BNXT_FLAG_ROCEV1_CAP) edev->flags |= BNXT_EN_FLAG_ROCEV1_CAP; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index b86baf901a5d..4eafe6ec0abf 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -88,6 +88,9 @@ struct bnxt_en_dev { u16 ulp_num_msix_vec; u16 ulp_num_ctxs; + + /* serialize ulp operations */ + struct mutex en_dev_lock; }; static inline bool bnxt_ulp_registered(struct bnxt_en_dev *edev) From patchwork Tue Apr 30 22:44:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13650010 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65592219FF for ; Tue, 30 Apr 2024 22:46:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517183; cv=none; b=jEdj6sn8JtbfgWXJpCw7nCI5sds3lsKlxDUizXpsDf4jDRqR7JuXIO4J1p22EEBkt81G1fgMUkj41BrMLdZ7PCcbAxKXD7uN5YFv1S0AagFWbqsVsm7VUi8iI1aUVChca6fFucLK2dUbqRqDS1dU4yCIPdjx1/TXP+esZGW6ytc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517183; c=relaxed/simple; bh=77zKzFjG2CTQIAkel9LrKBEyMLeh7H2qYAPCNTN663U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=BRxl6KNSH6QP/uqFx0ZX3tyyK1IOzTYvMW8y15oDOAgcHdF5CqYi1ZLXK15hywTT6LZskyunkLFjbXtrml6pafwUB1YbLvBI+p+CB6uHiYPhB6rGKKpN2McusSXemwbW/2wtpfYsGyfTUg0acQoLT67e+2xqv8CAJzW+RqHRHoE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=P/hKvwk7; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="P/hKvwk7" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-1ec69e3dbe5so7780255ad.0 for ; Tue, 30 Apr 2024 15:46:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1714517180; x=1715121980; darn=vger.kernel.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=9egGFzew9U8bi31/23yU44dJ+Tw23C/cwmi30lF79/s=; b=P/hKvwk7qPKGhe+rcUxFx5EIodDXNNvv/9Q6XUogLDqBYQ8ZhREB0lWSUWvqC6n/PS FbROrNgavhrvk8jZ06qhbpqy6VfiUJhNT8A+T506uVtVxIYAR6Bpif1UqNkxsg9ufrnu 4SBxuLN1MA766vcMWs5wZFIfTch3kFYfg3GCw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714517180; x=1715121980; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9egGFzew9U8bi31/23yU44dJ+Tw23C/cwmi30lF79/s=; b=p7PnkEqX71JXlpbnftLoAa8LbFvDXTjb4rAtr/usg8+KYNzGATzRtVXMIe6qWTV8s1 aTHTV+nm4pZG32sDq87cTjZdqM5aowAWZsGM1/eD9/ngAkDGQi/t5thcTgOeI29dRTbC 6aacl41dxQJ080y/W/27o40sftmLCog3Yl3zPGBheimPUk6v3ZUZHU6KAR0Oifg/126B d0oco4pohSt/7Li9ZLC8HFiHDkrug8U7DLCHRxaRNPbpPq/ksCOnLPb2b+qqRVIycVDF cqm1R+JPHh8NA2uZ1ipPXD/WimNOcX9je3w+9WzvylsnkRYJkJtSC8GwPHJROk4+hROF 9m2Q== X-Gm-Message-State: AOJu0YyT1ZVk+QRIvdEzQkFVl4Xbl+O6NunFk6s97tflgp3+BS1PR5ph VR3zTrPuXqe00t+2NZJF+WPpPn2IYIyCh9Pq9JoFotzalSkSCJ/jn2Nr9F0UtQ== X-Google-Smtp-Source: AGHT+IEjomUpI/KmMGFi7D4k7mjJLlp4GHtilnNB2HKmKxw9d/eCefGYBEXyf6FNB+U4/ETElIzJgA== X-Received: by 2002:a17:903:234e:b0:1e2:3e1e:3d9 with SMTP id c14-20020a170903234e00b001e23e1e03d9mr920536plh.63.1714517180388; Tue, 30 Apr 2024 15:46:20 -0700 (PDT) Received: from lvnvda5233.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id s11-20020a170902a50b00b001eb2fb28eabsm7836476plq.227.2024.04.30.15.46.18 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Apr 2024 15:46:19 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew.gospodarek@broadcom.com, Kalesh AP , Selvin Thyparampil Xavier , Vikas Gupta , Pavan Chebbi Subject: [PATCH net-next 6/7] bnxt_en: Optimize recovery path ULP locking in the driver Date: Tue, 30 Apr 2024 15:44:37 -0700 Message-Id: <20240430224438.91494-7-michael.chan@broadcom.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20240430224438.91494-1-michael.chan@broadcom.com> References: <20240430224438.91494-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Kalesh AP In the error recovery path (AER, firmware recovery, etc), the driver notifies the RoCE driver via ULP_STOP before the reset and via ULP_START after the reset, all under RTNL_LOCK. The RoCE driver can take a long time if there are a lot of QPs to destroy, so it is not ideal to hold the global RTNL lock. Rely on the new en_dev_lock mutex instead for ULP_STOP and ULP_START. For the most part, we move the ULP_STOP call before we take the RTNL lock and move the ULP_START after RTNL unlock. Note that SRIOV re-enablement must be done after ULP_START or RoCE on the VFs will not resume properly after reset. The one scenario in bnxt_hwrm_if_change() where the RTNL lock is already taken in the .ndo_open() context requires the ULP restart to be deferred to the bnxt_sp_task() workqueue. Reviewed-by: Selvin Thyparampil Xavier Reviewed-by: Vikas Gupta Reviewed-by: Pavan Chebbi Signed-off-by: Kalesh AP Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 62 ++++++++++++------- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + .../net/ethernet/broadcom/bnxt/bnxt_devlink.c | 7 ++- 3 files changed, 44 insertions(+), 26 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 4d7b4eabe0af..67294835e10e 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -11552,7 +11552,7 @@ static int bnxt_hwrm_if_change(struct bnxt *bp, bool up) if (fw_reset) { set_bit(BNXT_STATE_FW_RESET_DET, &bp->state); if (!test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) - bnxt_ulp_stop(bp); + bnxt_ulp_irq_stop(bp); bnxt_free_ctx_mem(bp); bnxt_dcb_free(bp); rc = bnxt_fw_init_one(bp); @@ -12107,10 +12107,9 @@ static int bnxt_open(struct net_device *dev) bnxt_hwrm_if_change(bp, false); } else { if (test_and_clear_bit(BNXT_STATE_FW_RESET_DET, &bp->state)) { - if (!test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) { - bnxt_ulp_start(bp, 0); - bnxt_reenable_sriov(bp); - } + if (!test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) + bnxt_queue_sp_work(bp, + BNXT_RESTART_ULP_SP_EVENT); } } @@ -13266,7 +13265,6 @@ static void bnxt_fw_fatal_close(struct bnxt *bp) static void bnxt_fw_reset_close(struct bnxt *bp) { - bnxt_ulp_stop(bp); /* When firmware is in fatal state, quiesce device and disable * bus master to prevent any potential bad DMAs before freeing * kernel memory. @@ -13347,6 +13345,7 @@ void bnxt_fw_exception(struct bnxt *bp) { netdev_warn(bp->dev, "Detected firmware fatal condition, initiating reset\n"); set_bit(BNXT_STATE_FW_FATAL_COND, &bp->state); + bnxt_ulp_stop(bp); bnxt_rtnl_lock_sp(bp); bnxt_force_fw_reset(bp); bnxt_rtnl_unlock_sp(bp); @@ -13378,6 +13377,7 @@ static int bnxt_get_registered_vfs(struct bnxt *bp) void bnxt_fw_reset(struct bnxt *bp) { + bnxt_ulp_stop(bp); bnxt_rtnl_lock_sp(bp); if (test_bit(BNXT_STATE_OPEN, &bp->state) && !test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) { @@ -13502,6 +13502,12 @@ static void bnxt_fw_echo_reply(struct bnxt *bp) hwrm_req_send(bp, req); } +static void bnxt_ulp_restart(struct bnxt *bp) +{ + bnxt_ulp_stop(bp); + bnxt_ulp_start(bp, 0); +} + static void bnxt_sp_task(struct work_struct *work) { struct bnxt *bp = container_of(work, struct bnxt, sp_task); @@ -13513,6 +13519,11 @@ static void bnxt_sp_task(struct work_struct *work) return; } + if (test_and_clear_bit(BNXT_RESTART_ULP_SP_EVENT, &bp->sp_event)) { + bnxt_ulp_restart(bp); + bnxt_reenable_sriov(bp); + } + if (test_and_clear_bit(BNXT_RX_MASK_SP_EVENT, &bp->sp_event)) bnxt_cfg_rx_mode(bp); @@ -13969,10 +13980,8 @@ static bool bnxt_fw_reset_timeout(struct bnxt *bp) static void bnxt_fw_reset_abort(struct bnxt *bp, int rc) { clear_bit(BNXT_STATE_IN_FW_RESET, &bp->state); - if (bp->fw_reset_state != BNXT_FW_RESET_STATE_POLL_VF) { - bnxt_ulp_start(bp, rc); + if (bp->fw_reset_state != BNXT_FW_RESET_STATE_POLL_VF) bnxt_dl_health_fw_status_update(bp, false); - } bp->fw_reset_state = 0; dev_close(bp->dev); } @@ -14003,7 +14012,7 @@ static void bnxt_fw_reset_task(struct work_struct *work) bp->fw_reset_state = 0; netdev_err(bp->dev, "Firmware reset aborted, bnxt_get_registered_vfs() returns %d\n", n); - return; + goto ulp_start; } bnxt_queue_fw_reset_work(bp, HZ / 10); return; @@ -14013,7 +14022,7 @@ static void bnxt_fw_reset_task(struct work_struct *work) if (test_bit(BNXT_STATE_ABORT_ERR, &bp->state)) { bnxt_fw_reset_abort(bp, rc); rtnl_unlock(); - return; + goto ulp_start; } bnxt_fw_reset_close(bp); if (bp->fw_cap & BNXT_FW_CAP_ERR_RECOVER_RELOAD) { @@ -14106,7 +14115,7 @@ static void bnxt_fw_reset_task(struct work_struct *work) netdev_err(bp->dev, "bnxt_open() failed during FW reset\n"); bnxt_fw_reset_abort(bp, rc); rtnl_unlock(); - return; + goto ulp_start; } if ((bp->fw_cap & BNXT_FW_CAP_ERROR_RECOVERY) && @@ -14118,10 +14127,6 @@ static void bnxt_fw_reset_task(struct work_struct *work) /* Make sure fw_reset_state is 0 before clearing the flag */ smp_mb__before_atomic(); clear_bit(BNXT_STATE_IN_FW_RESET, &bp->state); - bnxt_ulp_start(bp, 0); - bnxt_reenable_sriov(bp); - bnxt_vf_reps_alloc(bp); - bnxt_vf_reps_open(bp); bnxt_ptp_reapply_pps(bp); clear_bit(BNXT_STATE_FW_ACTIVATE, &bp->state); if (test_and_clear_bit(BNXT_STATE_RECOVER, &bp->state)) { @@ -14129,6 +14134,12 @@ static void bnxt_fw_reset_task(struct work_struct *work) bnxt_dl_health_fw_status_update(bp, true); } rtnl_unlock(); + bnxt_ulp_start(bp, 0); + bnxt_reenable_sriov(bp); + rtnl_lock(); + bnxt_vf_reps_alloc(bp); + bnxt_vf_reps_open(bp); + rtnl_unlock(); break; } return; @@ -14144,6 +14155,8 @@ static void bnxt_fw_reset_task(struct work_struct *work) rtnl_lock(); bnxt_fw_reset_abort(bp, rc); rtnl_unlock(); +ulp_start: + bnxt_ulp_start(bp, rc); } static int bnxt_init_board(struct pci_dev *pdev, struct net_device *dev) @@ -15530,8 +15543,9 @@ static int bnxt_suspend(struct device *device) struct bnxt *bp = netdev_priv(dev); int rc = 0; - rtnl_lock(); bnxt_ulp_stop(bp); + + rtnl_lock(); if (netif_running(dev)) { netif_device_detach(dev); rc = bnxt_close(dev); @@ -15586,10 +15600,10 @@ static int bnxt_resume(struct device *device) } resume_exit: + rtnl_unlock(); bnxt_ulp_start(bp, rc); if (!rc) bnxt_reenable_sriov(bp); - rtnl_unlock(); return rc; } @@ -15619,11 +15633,11 @@ static pci_ers_result_t bnxt_io_error_detected(struct pci_dev *pdev, netdev_info(netdev, "PCI I/O error detected\n"); + bnxt_ulp_stop(bp); + rtnl_lock(); netif_device_detach(netdev); - bnxt_ulp_stop(bp); - if (test_and_set_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) { netdev_err(bp->dev, "Firmware reset already in progress\n"); abort = true; @@ -15759,13 +15773,13 @@ static void bnxt_io_resume(struct pci_dev *pdev) if (!err && netif_running(netdev)) err = bnxt_open(netdev); - bnxt_ulp_start(bp, err); - if (!err) { - bnxt_reenable_sriov(bp); + if (!err) netif_device_attach(netdev); - } rtnl_unlock(); + bnxt_ulp_start(bp, err); + if (!err) + bnxt_reenable_sriov(bp); } static const struct pci_error_handlers bnxt_err_handler = { diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 003c00b229f2..c62bdbb7249f 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -2440,6 +2440,7 @@ struct bnxt { #define BNXT_LINK_CFG_CHANGE_SP_EVENT 21 #define BNXT_THERMAL_THRESHOLD_SP_EVENT 22 #define BNXT_FW_ECHO_REQUEST_SP_EVENT 23 +#define BNXT_RESTART_ULP_SP_EVENT 24 struct delayed_work fw_reset_task; int fw_reset_state; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c index d9ea6fa23923..4cb0fabf977e 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c @@ -437,18 +437,20 @@ static int bnxt_dl_reload_down(struct devlink *dl, bool netns_change, switch (action) { case DEVLINK_RELOAD_ACTION_DRIVER_REINIT: { + bnxt_ulp_stop(bp); rtnl_lock(); if (bnxt_sriov_cfg(bp)) { NL_SET_ERR_MSG_MOD(extack, "reload is unsupported while VFs are allocated or being configured"); rtnl_unlock(); + bnxt_ulp_start(bp, 0); return -EOPNOTSUPP; } if (bp->dev->reg_state == NETREG_UNREGISTERED) { rtnl_unlock(); + bnxt_ulp_start(bp, 0); return -ENODEV; } - bnxt_ulp_stop(bp); if (netif_running(bp->dev)) bnxt_close_nic(bp, true, true); bnxt_vf_reps_free(bp); @@ -516,7 +518,6 @@ static int bnxt_dl_reload_up(struct devlink *dl, enum devlink_reload_action acti bnxt_vf_reps_alloc(bp); if (netif_running(bp->dev)) rc = bnxt_open_nic(bp, true, true); - bnxt_ulp_start(bp, rc); if (!rc) { bnxt_reenable_sriov(bp); bnxt_ptp_reapply_pps(bp); @@ -570,6 +571,8 @@ static int bnxt_dl_reload_up(struct devlink *dl, enum devlink_reload_action acti dev_close(bp->dev); } rtnl_unlock(); + if (action == DEVLINK_RELOAD_ACTION_DRIVER_REINIT) + bnxt_ulp_start(bp, rc); return rc; } From patchwork Tue Apr 30 22:44:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13650011 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 114961E86E for ; Tue, 30 Apr 2024 22:46:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517184; cv=none; b=TfnRyi+1FMEFuYV5zaYFLl29W+y3JUvJ9aAr/4gbLCjR0RVdXF+FbvxY884wpCLixcGfH2OV8W2rRMSfgRV+5G11fqfRGiWafXhQsj8DacT2dfP9tQFiiZ4QRnblvjIC/KslFLkrZhwZ9EVr98xR8T864LtgcTGdlhS+xFL3EQE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517184; c=relaxed/simple; bh=REEi6FCvPYgY8s901rpqmat1FA1f7KjS3zghBxLTepY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=ivhutjycHZK5WTL0yy/3IlQKv1wuj4ZkbQ/fE/hzlLJ3i+QJKicxZy6NcEfoW3fjzXd0jFer4VDTI3rn1DH6cB8eyfCGJuRVa51xjG6Dj+OHXGAYgy++fw2/1HJMj2Fe6RFR5nq2Lxo+bzUm8Da5u3VPf9TqX6mboEHLf2x+GdI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=cpw8qlZZ; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="cpw8qlZZ" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1e4f341330fso58285275ad.0 for ; Tue, 30 Apr 2024 15:46:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1714517182; x=1715121982; darn=vger.kernel.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=qKl26w8FEt7WfcZ7WONzzntRM1njhxf1a8zSMePoihk=; b=cpw8qlZZD5AeUPof4MQuRVrdHlc/zRvjoMruHnhQbzcNtDlcOlAN+2O7n1l+jW8ifl 1KNFBx0Qm48RAI9+O4MruoJbA258tmpMVeM5KvhwgThvO1+HCuY2wX9dgzlFkTFr226f V3SbZ59FnD4lQ8DmUtcEw27ceG0Q9v2YAwh9w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714517182; x=1715121982; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=qKl26w8FEt7WfcZ7WONzzntRM1njhxf1a8zSMePoihk=; b=hdX2JpepLP+xcgDsgNC6aDNrnAg1beibcDsSQ2eHrFVvM1pWrMagTYbJQPl+4f/ahI PA4YgNHU0mdK6IOrcmysXw50CnDLOETHyqSk5c88lePcBk5926p9VLMgcBoyuTNnGUu8 0f3HMFujCis7uyVKAsCay16cpl5jKy4CDR2rqhMqM4QyhD9pJ2h2Xs6nvQm/YBmA9Ale O/pLHLk1rosjeyExgcYgIWvgfOb649IBxpWwhY8R807KhWYabjthaN4zFptkqFfHYB/K qhO24oY61TCFEE5g+LOfwgveDQ0mMsz39Blw35QFxhpLMAL7YuZ4ymfGcPNS+xqV6oYz IYEg== X-Gm-Message-State: AOJu0YyEDdPMGaD7BbWb49UZoB/pI46F2jidtm9nwNtR3ztcCSFOa1EU MtDZgG9V+LRcRloHt8y74MJU99R4p9YbOWIVs3HT5F79FJXiTqKl0dOkQyb9ww== X-Google-Smtp-Source: AGHT+IEwU0Wbwv0Eh004rBBBcgcwF5yh9ge8PYPAx0YSwY4m1WhnMGMCwMd+fbiOCv8EAChG4/lBGA== X-Received: by 2002:a17:902:d38d:b0:1e2:9ddc:f72d with SMTP id e13-20020a170902d38d00b001e29ddcf72dmr839580pld.26.1714517182110; Tue, 30 Apr 2024 15:46:22 -0700 (PDT) Received: from lvnvda5233.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id s11-20020a170902a50b00b001eb2fb28eabsm7836476plq.227.2024.04.30.15.46.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Apr 2024 15:46:21 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew.gospodarek@broadcom.com, Ajit Khaparde , Selvin Thyparampil Xavier Subject: [PATCH net-next 7/7] bnxt_en: Add VF PCI ID for 5760X (P7) chips Date: Tue, 30 Apr 2024 15:44:38 -0700 Message-Id: <20240430224438.91494-8-michael.chan@broadcom.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20240430224438.91494-1-michael.chan@broadcom.com> References: <20240430224438.91494-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Ajit Khaparde No driver logic changes are required to support the VFs, so just add the VF PCI ID. Reviewed-by: Selvin Thyparampil Xavier Signed-off-by: Ajit Khaparde Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 4 +++- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 67294835e10e..2c9ced024250 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -137,6 +137,7 @@ static const struct { [NETXTREME_E_VF_HV] = { "Broadcom NetXtreme-E Virtual Function for Hyper-V" }, [NETXTREME_E_P5_VF] = { "Broadcom BCM5750X NetXtreme-E Ethernet Virtual Function" }, [NETXTREME_E_P5_VF_HV] = { "Broadcom BCM5750X NetXtreme-E Virtual Function for Hyper-V" }, + [NETXTREME_E_P7_VF] = { "Broadcom BCM5760X Virtual Function" }, }; static const struct pci_device_id bnxt_pci_tbl[] = { @@ -211,6 +212,7 @@ static const struct pci_device_id bnxt_pci_tbl[] = { { PCI_VDEVICE(BROADCOM, 0x1807), .driver_data = NETXTREME_E_P5_VF }, { PCI_VDEVICE(BROADCOM, 0x1808), .driver_data = NETXTREME_E_P5_VF_HV }, { PCI_VDEVICE(BROADCOM, 0x1809), .driver_data = NETXTREME_E_P5_VF_HV }, + { PCI_VDEVICE(BROADCOM, 0x1819), .driver_data = NETXTREME_E_P7_VF }, { PCI_VDEVICE(BROADCOM, 0xd800), .driver_data = NETXTREME_S_VF }, #endif { 0 } @@ -294,7 +296,7 @@ static bool bnxt_vf_pciid(enum board_idx idx) return (idx == NETXTREME_C_VF || idx == NETXTREME_E_VF || idx == NETXTREME_S_VF || idx == NETXTREME_C_VF_HV || idx == NETXTREME_E_VF_HV || idx == NETXTREME_E_P5_VF || - idx == NETXTREME_E_P5_VF_HV); + idx == NETXTREME_E_P5_VF_HV || idx == NETXTREME_E_P7_VF); } #define DB_CP_REARM_FLAGS (DB_KEY_CP | DB_IDX_VALID) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index c62bdbb7249f..656ab81c0272 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -2013,6 +2013,7 @@ enum board_idx { NETXTREME_E_VF_HV, NETXTREME_E_P5_VF, NETXTREME_E_P5_VF_HV, + NETXTREME_E_P7_VF, }; struct bnxt {