From patchwork Sat Mar 4 02:43:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13159773 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27C44C61DA4 for ; Sat, 4 Mar 2023 02:44:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229509AbjCDCoY (ORCPT ); Fri, 3 Mar 2023 21:44:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52114 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229447AbjCDCoW (ORCPT ); Fri, 3 Mar 2023 21:44:22 -0500 Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com [IPv6:2607:f8b0:4864:20::82e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4E2FA619F for ; Fri, 3 Mar 2023 18:44:21 -0800 (PST) Received: by mail-qt1-x82e.google.com with SMTP id c3so5062620qtc.8 for ; Fri, 03 Mar 2023 18:44:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1677897860; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=u6HPuD5v/lw3q/tDMYG3Ltfk0jbxgGzylQ1BtW0dqug=; b=Kswc5gQswjnXQ1h+aEFSn6/gfzkunXziXvmeO5n1dt44DF9rrJpwwh7fMrFWjb6BtS asIqNTApWSsUC6euFyk3Dn2UsGoTxizIlch72aYKSY9/3vwEBSnFlulxXUwZup0l7kTA IZl0Q0yNlQ7PLDHYVY6UBDyAvR3XbGvPIK8rc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677897860; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=u6HPuD5v/lw3q/tDMYG3Ltfk0jbxgGzylQ1BtW0dqug=; b=pzLT23U7eq1P4as8vMc8E0rXPDAktoKZzejnRKvy1zGzQDxru8fLEvCKNm1K9+b59G lI1GQSBzkHmpk29sEOWGZDPJmXG/Erz51/WHokFO3E1Z5uv26d3kVre+wm6Y3C/eN+CS ZQO/rhjpiBAtprEInq2L2zAtXRQ3FL/+q5kJFs9O6jvV4cz3LRXDSAcTqJ+P09aqkEo+ hxkmBbU40WbbVpBMmdIHlxiqQopJBHjNMjhzoXhxoJBU9Plj5PscSMaUkpazzqDct5OA SJt3QgiPK9KdKe0P9G3z/5G1mYMyxg67qoHgfn0HBFxUz91dLMYsxGBAKOO59WG1PqtX Ki7w== X-Gm-Message-State: AO0yUKU1b4iq4UaSYNud/L36vlF5Ba8BmHEeS8nCpb5cGYcIoygpB3Au uNMaVS58rkoPbzkNaxWp07WtQg== X-Google-Smtp-Source: AK7set8GKwdN/n5IyXzPwI5QFnMpnmKOAGb0gYjTL6ULUGbf0Zb5MUcnEfVfj6IkgVd6pFAKOoCV3w== X-Received: by 2002:a05:622a:1106:b0:3ba:36f2:c207 with SMTP id e6-20020a05622a110600b003ba36f2c207mr7429995qty.39.1677897859771; Fri, 03 Mar 2023 18:44:19 -0800 (PST) Received: from localhost.attlocal.net (108-196-84-92.lightspeed.irvnca.sbcglobal.net. [108.196.84.92]) by smtp.gmail.com with ESMTPSA id h20-20020ac846d4000000b003b64f1b1f40sm2963845qto.40.2023.03.03.18.44.18 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Mar 2023 18:44:19 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, kuba@kernel.org, edumazet@google.com, pabeni@redhat.com, gospo@broadcom.com Subject: [PATCH net 1/2] bnxt_en: Avoid order-5 memory allocation for TPA data Date: Fri, 3 Mar 2023 18:43:57 -0800 Message-Id: <1677897838-23562-2-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1677897838-23562-1-git-send-email-michael.chan@broadcom.com> References: <1677897838-23562-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org The driver needs to keep track of all the possible concurrent TPA (GRO/LRO) completions on the aggregation ring. On P5 chips, the maximum number of concurrent TPA is 256 and the amount of memory we allocate is order-5 on systems using 4K pages. Memory allocation failure has been reported: NetworkManager: page allocation failure: order:5, mode:0x40dc0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null),cpuset=/,mems_allowed=0-1 CPU: 15 PID: 2995 Comm: NetworkManager Kdump: loaded Not tainted 5.10.156 #1 Hardware name: Dell Inc. PowerEdge R660/0M1CC5, BIOS 0.2.25 08/12/2022 Call Trace: dump_stack+0x57/0x6e warn_alloc.cold.120+0x7b/0xdd ? _cond_resched+0x15/0x30 ? __alloc_pages_direct_compact+0x15f/0x170 __alloc_pages_slowpath.constprop.108+0xc58/0xc70 __alloc_pages_nodemask+0x2d0/0x300 kmalloc_order+0x24/0xe0 kmalloc_order_trace+0x19/0x80 bnxt_alloc_mem+0x1150/0x15c0 [bnxt_en] ? bnxt_get_func_stat_ctxs+0x13/0x60 [bnxt_en] __bnxt_open_nic+0x12e/0x780 [bnxt_en] bnxt_open+0x10b/0x240 [bnxt_en] __dev_open+0xe9/0x180 __dev_change_flags+0x1af/0x220 dev_change_flags+0x21/0x60 do_setlink+0x35c/0x1100 Instead of allocating this big chunk of memory and dividing it up for the concurrent TPA instances, allocate each small chunk separately for each TPA instance. This will reduce it to order-0 allocations. Fixes: 79632e9ba386 ("bnxt_en: Expand bnxt_tpa_info struct to support 57500 chips.") Reviewed-by: Somnath Kotur Reviewed-by: Damodharam Ammepalli Reviewed-by: Pavan Chebbi Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 5d4b1f2ebeac..f96d539b469c 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -3145,7 +3145,7 @@ static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem) static void bnxt_free_tpa_info(struct bnxt *bp) { - int i; + int i, j; for (i = 0; i < bp->rx_nr_rings; i++) { struct bnxt_rx_ring_info *rxr = &bp->rx_ring[i]; @@ -3153,8 +3153,10 @@ static void bnxt_free_tpa_info(struct bnxt *bp) kfree(rxr->rx_tpa_idx_map); rxr->rx_tpa_idx_map = NULL; if (rxr->rx_tpa) { - kfree(rxr->rx_tpa[0].agg_arr); - rxr->rx_tpa[0].agg_arr = NULL; + for (j = 0; j < bp->max_tpa; j++) { + kfree(rxr->rx_tpa[j].agg_arr); + rxr->rx_tpa[j].agg_arr = NULL; + } } kfree(rxr->rx_tpa); rxr->rx_tpa = NULL; @@ -3163,14 +3165,13 @@ static void bnxt_free_tpa_info(struct bnxt *bp) static int bnxt_alloc_tpa_info(struct bnxt *bp) { - int i, j, total_aggs = 0; + int i, j; bp->max_tpa = MAX_TPA; if (bp->flags & BNXT_FLAG_CHIP_P5) { if (!bp->max_tpa_v2) return 0; bp->max_tpa = max_t(u16, bp->max_tpa_v2, MAX_TPA_P5); - total_aggs = bp->max_tpa * MAX_SKB_FRAGS; } for (i = 0; i < bp->rx_nr_rings; i++) { @@ -3184,12 +3185,12 @@ static int bnxt_alloc_tpa_info(struct bnxt *bp) if (!(bp->flags & BNXT_FLAG_CHIP_P5)) continue; - agg = kcalloc(total_aggs, sizeof(*agg), GFP_KERNEL); - rxr->rx_tpa[0].agg_arr = agg; - if (!agg) - return -ENOMEM; - for (j = 1; j < bp->max_tpa; j++) - rxr->rx_tpa[j].agg_arr = agg + j * MAX_SKB_FRAGS; + for (j = 0; j < bp->max_tpa; j++) { + agg = kcalloc(MAX_SKB_FRAGS, sizeof(*agg), GFP_KERNEL); + if (!agg) + return -ENOMEM; + rxr->rx_tpa[j].agg_arr = agg; + } rxr->rx_tpa_idx_map = kzalloc(sizeof(*rxr->rx_tpa_idx_map), GFP_KERNEL); if (!rxr->rx_tpa_idx_map)