From patchwork Tue Apr 30 22:44:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13650005 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD7F22C853 for ; Tue, 30 Apr 2024 22:46:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517175; cv=none; b=kdNXbATSrGDrUXWa18AlLYqcUiVHeDPLi4JtVSeegEGG4OVGRhGBZWFg3u7n1u+kjxK3RWdBmfoGGk1k2gARWKG2xu7VzP0kdG/dkDqHknDN9r9Fl3sdwBYgP2w8JIT8Qqao7zpFuENVhAYZ/hpCWhUlEMe9gBzgtOdp2IXuz8s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714517175; c=relaxed/simple; bh=6mVPS7TncWc8rZeUJrQnSatBetIYzPKAWstvnb9Wqmg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=N6dBydkbNQixTJFzUr0U4aqmuOGjCSK43/yeP9k21X6B2cdfA6f/5a0jHdxBhiBdHcroCk8xvdjeeVsOcBKkeEcfBMBqHY9zB8GdClObebDd31Z2FoBC2W0IInxKoKQSuN3aVqSossEErQlTMN0AFMbfLFtShgjrEHdaqXrJfx4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=gfbl6+FV; arc=none smtp.client-ip=209.85.215.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="gfbl6+FV" Received: by mail-pg1-f171.google.com with SMTP id 41be03b00d2f7-5e152c757a5so3960509a12.2 for ; Tue, 30 Apr 2024 15:46:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1714517173; x=1715121973; darn=vger.kernel.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=L4aiW0tCtnnxe4pUVwZynZF9OlmD9SNkaQDmahQztks=; b=gfbl6+FVEOmE9KPNsmY7ebb05vrtL+kB+ClYj35rpMWQZ+72Jtu27xkRPQO+LOhwtn cr533J3XCWBeM87NrUfE8+1cfT63Y+2IJavpAs1+s2GPDXJnqugkLcik6PxuaEAgepS2 RdbtiCVw5VTbAVRSt00JQmgP1XiEWZYpHJxGE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714517173; x=1715121973; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=L4aiW0tCtnnxe4pUVwZynZF9OlmD9SNkaQDmahQztks=; b=fBQ3nmfysMtdQaYDIs41MaB/lSI2QMGXNUs7gzpslOC5wwqnMD4+F4zu6E/BcHdQrQ OYoh+zi9Z9Z9lHiIjv6lPd07OgYNaNqiwTpBb+pOQaY1rPTuRBs6PYoXgXvY6EmxD8Sr EYGktPC2CLfVn2JW2qbvYmoQFE4TB4CoKzVRD4OOlBR8W4INju2Wxkv8qkXmPQYbC6jL v0q3KlVBqz43zYmJ0hNLG/dXP3x9rc9lWhetg7Ve/AAMBzCioNn39JE8ikuiRKcS0/wx YOrHDlG+NU2G0aDlnmGBZWJNc1Z1GbULv1dS0+rh6bdLUGkL0UJ8WE7A3PpEvXjsxn+y eTFg== X-Gm-Message-State: AOJu0YyYDyc858juXz7XCX7NQQhnCllLJFdxg8Bvc66g2yGutjTp5k58 19U+ySdU32CWOaXDpqZ46qxsj19ABkNGTrQq48TBdtvust46tYQSbSlJC/dOJw== X-Google-Smtp-Source: AGHT+IELbP68HhD34YUDfFjQwmkD0S83nH8oCLTpg70viNuifIf3Z8hp/lxMgOlCW1exJJI/dhBAcQ== X-Received: by 2002:a17:90a:bb89:b0:2a2:97f3:83b3 with SMTP id v9-20020a17090abb8900b002a297f383b3mr789203pjr.48.1714517172165; Tue, 30 Apr 2024 15:46:12 -0700 (PDT) Received: from lvnvda5233.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id s11-20020a170902a50b00b001eb2fb28eabsm7836476plq.227.2024.04.30.15.46.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Apr 2024 15:46:10 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew.gospodarek@broadcom.com, David Wei , Pavan Chebbi , Ajit Khaparde Subject: [PATCH net-next 1/7] bnxt_en: Fix and simplify bnxt_get_avail_msix() calls Date: Tue, 30 Apr 2024 15:44:32 -0700 Message-Id: <20240430224438.91494-2-michael.chan@broadcom.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20240430224438.91494-1-michael.chan@broadcom.com> References: <20240430224438.91494-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org David reported this issue of the driver not initializing on an older chip not running in the new resource manager mode (!BNXT_NEW_RM()). Sample dmesg: bnxt_en 0000:02:00.0 (unnamed net_device) (uninitialized): Able to reserve only 0 out of 9 requested RX rings bnxt_en 0000:02:00.0 (unnamed net_device) (uninitialized): Unable to reserve tx rings bnxt_en 0000:02:00.0 (unnamed net_device) (uninitialized): 2nd rings reservation failed. bnxt_en 0000:02:00.0 (unnamed net_device) (uninitialized): Not enough rings available. bnxt_en 0000:02:00.0: probe with driver bnxt_en failed with error -12 This is a regression caused by a recent commit that adds a call to bnxt_get_avail_msix() before MSIX is initialized: bnxt_set_dflt_rings() __bnxt_reserve_rings() bnxt_get_avail_msix() bnxt_get_avail_msix() returns a negative number if !BNXT_NEW_RM() and when MSIX is not initialized. This causes __bnxt_reserve_rings() to fail in this call sequence and the driver aborts initialization. Before this commit in 2022: 303432211324 ("bnxt_en: Remove runtime interrupt vector allocation") MSIX allocation for RoCE was dynamic and bnxt_get_avail_msix() was used to determine the available run-time MSIX available for RoCE. Today, bnxt_get_avail_msix() is only used to reserve some available MSIX ahead of time to be ready when the RoCE driver loads. It only needs to be called when BNXT_NEW_RM() is true because older chips do not require reservations for MSIX. Simplify bnxt_get_avail_msix() to only consider the BNXT_NEW_RM() case and only make this call if BNXT_NEW_RM() is true. Also change bnxt_get_avail_msix() to static since it is only used in bnxt.c. Reported-by: David Wei Fixes: d630624ebd70 ("bnxt_en: Utilize ulp client resources if RoCE is not registered") Reviewed-by: Pavan Chebbi Reviewed-by: Andy Gospodarek Reviewed-by: Ajit Khaparde Signed-off-by: Michael Chan --- Cc: David Wei --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 20 ++++++++------------ drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 - 2 files changed, 8 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index be96bb494ae6..0eb880766012 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -7459,6 +7459,8 @@ static bool bnxt_rings_ok(struct bnxt *bp, struct bnxt_hw_rings *hwr) hwr->stat && (hwr->cp_p5 || !(bp->flags & BNXT_FLAG_CHIP_P5_PLUS)); } +static int bnxt_get_avail_msix(struct bnxt *bp, int num); + static int __bnxt_reserve_rings(struct bnxt *bp) { struct bnxt_hw_rings hwr = {0}; @@ -7471,7 +7473,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp) if (!bnxt_need_reserve_rings(bp)) return 0; - if (!bnxt_ulp_registered(bp->edev)) { + if (!bnxt_ulp_registered(bp->edev) && BNXT_NEW_RM(bp)) { ulp_msix = bnxt_get_avail_msix(bp, bp->ulp_num_msix_want); if (!ulp_msix) bnxt_set_ulp_stat_ctxs(bp, 0); @@ -10474,19 +10476,13 @@ unsigned int bnxt_get_avail_stat_ctxs_for_en(struct bnxt *bp) return bnxt_get_max_func_stat_ctxs(bp) - bnxt_get_func_stat_ctxs(bp); } -int bnxt_get_avail_msix(struct bnxt *bp, int num) +/* Only called if BNXT_NEW_RM() is true to get the available MSIX to + * reserve ahead of time before RoCE is registered. + */ +static int bnxt_get_avail_msix(struct bnxt *bp, int num) { - int max_cp = bnxt_get_max_func_cp_rings(bp); int max_irq = bnxt_get_max_func_irqs(bp); int total_req = bp->cp_nr_rings + num; - int max_idx, avail_msix; - - max_idx = bp->total_irqs; - if (!(bp->flags & BNXT_FLAG_CHIP_P5_PLUS)) - max_idx = min_t(int, bp->total_irqs, max_cp); - avail_msix = max_idx - bp->cp_nr_rings; - if (!BNXT_NEW_RM(bp) || avail_msix >= num) - return avail_msix; if (max_irq < total_req) { num = max_irq - bp->cp_nr_rings; @@ -10619,7 +10615,7 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init) if (!bnxt_need_reserve_rings(bp)) return 0; - if (!bnxt_ulp_registered(bp->edev)) { + if (!bnxt_ulp_registered(bp->edev) && BNXT_NEW_RM(bp)) { int ulp_msix = bnxt_get_avail_msix(bp, bp->ulp_num_msix_want); if (ulp_msix > bp->ulp_num_msix_want) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index ad57ef051798..0c680032ab66 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -2736,7 +2736,6 @@ unsigned int bnxt_get_max_func_stat_ctxs(struct bnxt *bp); unsigned int bnxt_get_avail_stat_ctxs_for_en(struct bnxt *bp); unsigned int bnxt_get_max_func_cp_rings(struct bnxt *bp); unsigned int bnxt_get_avail_cp_rings_for_en(struct bnxt *bp); -int bnxt_get_avail_msix(struct bnxt *bp, int num); int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init); void bnxt_tx_disable(struct bnxt *bp); void bnxt_tx_enable(struct bnxt *bp);