From patchwork Wed Jul 26 14:51:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13328221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A024BC001DC for ; Wed, 26 Jul 2023 15:03:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234639AbjGZPDa (ORCPT ); Wed, 26 Jul 2023 11:03:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46268 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234637AbjGZPDX (ORCPT ); Wed, 26 Jul 2023 11:03:23 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 904D11BFA for ; Wed, 26 Jul 2023 08:03:14 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-666e6541c98so6376937b3a.2 for ; Wed, 26 Jul 2023 08:03:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1690383794; x=1690988594; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=TxtIiQ2UTv36FdDseVProJr/0okxfhbglYJvGFAJX6Q=; b=O6AdN6th1jWJQoBKInxMkRxxnDdIilkERNyLgHS0TKqfVkEVQqsZ+PO6lPxF11i4TH QzDMZ+hsSrQQxP7aTh8P9hnIJMozduMD2PHWM14Q30GHJY+flf6/QA2gLtNaYWVozm0B R1xq/FKlE15Js12VZs7xwx2OznG8pHq+pGhdI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690383794; x=1690988594; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=TxtIiQ2UTv36FdDseVProJr/0okxfhbglYJvGFAJX6Q=; b=Jlrmy99/BNyMRwGznII53nd8oK3ooRJfjdHmWxVqqrhZVcKL9rWV1Um5Vy80JbNiMW ioJHXYU65aHqwquJcOvk2dR8hbxuReZNX0kKfShmmmAvXSs3GXSuJOutpCs+/5nVGUhg /wQTOWtu4Ow/1Jz7Tr16nEQYJLFQ2f0TuRo8geoIiu/plEMaHELGdtbnWXZ6PKFSpYLv 5wiUEfDTEcp5ixNCnmBaKqNBE8gDSBt0agX0aFGQ8sSKA34rz0D5GC4tx+ZOV0NEDTO8 lpVz5a+Srug4jYS9dUz5YLwLYYURW2PDkwg1DEXNHd0/JVGbeS/P66u8yY765G1y4lVy ogeQ== X-Gm-Message-State: ABy/qLaPOpRUS0KXRI2HmdMjNkxhZzNTc5fRU1s6EdCGbkWo/fnBiuZI jeHg49fKx0nudMvrWG5ZOFs3gQ== X-Google-Smtp-Source: APBJJlEdyFeusgE9zPGHM5U6BDIBPJ71VEcCMZW0vPi8oqQIGMp1d4DqNP2l5loQSa1juvAJNmOh2w== X-Received: by 2002:a05:6a00:a06:b0:64c:4f2f:a235 with SMTP id p6-20020a056a000a0600b0064c4f2fa235mr3423336pfh.30.1690383793750; Wed, 26 Jul 2023 08:03:13 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id y4-20020a63ad44000000b0055fd10306a2sm12772846pgo.75.2023.07.26.08.03.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Jul 2023 08:03:12 -0700 (PDT) From: Selvin Xavier To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, Chandramohan Akula , Selvin Xavier Subject: [PATCH for-next 4/4] bnxt_re: Update the debug counters for doorbell pacing Date: Wed, 26 Jul 2023 07:51:21 -0700 Message-Id: <1690383081-15033-5-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1690383081-15033-1-git-send-email-selvin.xavier@broadcom.com> References: <1690383081-15033-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chandramohan Akula Add debug counters to track the Doorbell pacing events and report the doorbell pacing debug stats. Signed-off-by: Chandramohan Akula Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/hw_counters.c | 18 ++++++++++++++++++ drivers/infiniband/hw/bnxt_re/hw_counters.h | 11 +++++++++++ drivers/infiniband/hw/bnxt_re/main.c | 3 +++ 3 files changed, 32 insertions(+) diff --git a/drivers/infiniband/hw/bnxt_re/hw_counters.c b/drivers/infiniband/hw/bnxt_re/hw_counters.c index e50a1cb..9357240 100644 --- a/drivers/infiniband/hw/bnxt_re/hw_counters.c +++ b/drivers/infiniband/hw/bnxt_re/hw_counters.c @@ -146,6 +146,10 @@ static const struct rdma_stat_desc bnxt_re_stat_descs[] = { [BNXT_RE_TX_CNP].name = "tx_cnp_pkts", [BNXT_RE_RX_CNP].name = "rx_cnp_pkts", [BNXT_RE_RX_ECN].name = "rx_ecn_marked_pkts", + [BNXT_RE_PACING_RESCHED].name = "pacing_reschedule", + [BNXT_RE_PACING_CMPL].name = "pacing_complete", + [BNXT_RE_PACING_ALERT].name = "pacing_alerts", + [BNXT_RE_DB_FIFO_REG].name = "db_fifo_register", }; static void bnxt_re_copy_ext_stats(struct bnxt_re_dev *rdev, @@ -278,6 +282,18 @@ static void bnxt_re_copy_err_stats(struct bnxt_re_dev *rdev, err_s->res_oos_drop_count; } +static void bnxt_re_copy_db_pacing_stats(struct bnxt_re_dev *rdev, + struct rdma_hw_stats *stats) +{ + struct bnxt_re_db_pacing_stats *pacing_s = &rdev->stats.pacing; + + stats->value[BNXT_RE_PACING_RESCHED] = pacing_s->resched; + stats->value[BNXT_RE_PACING_CMPL] = pacing_s->complete; + stats->value[BNXT_RE_PACING_ALERT] = pacing_s->alerts; + stats->value[BNXT_RE_DB_FIFO_REG] = + readl(rdev->en_dev->bar0 + rdev->pacing.dbr_db_fifo_reg_off); +} + int bnxt_re_ib_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats, u32 port, int index) @@ -350,6 +366,8 @@ int bnxt_re_ib_get_hw_stats(struct ib_device *ibdev, goto done; } } + if (rdev->pacing.dbr_pacing) + bnxt_re_copy_db_pacing_stats(rdev, stats); } done: diff --git a/drivers/infiniband/hw/bnxt_re/hw_counters.h b/drivers/infiniband/hw/bnxt_re/hw_counters.h index f3c4e35..e541b6f 100644 --- a/drivers/infiniband/hw/bnxt_re/hw_counters.h +++ b/drivers/infiniband/hw/bnxt_re/hw_counters.h @@ -129,11 +129,21 @@ enum bnxt_re_hw_stats { BNXT_RE_TX_CNP, BNXT_RE_RX_CNP, BNXT_RE_RX_ECN, + BNXT_RE_PACING_RESCHED, + BNXT_RE_PACING_CMPL, + BNXT_RE_PACING_ALERT, + BNXT_RE_DB_FIFO_REG, BNXT_RE_NUM_EXT_COUNTERS }; #define BNXT_RE_NUM_STD_COUNTERS (BNXT_RE_OUT_OF_SEQ_ERR + 1) +struct bnxt_re_db_pacing_stats { + u64 resched; + u64 complete; + u64 alerts; +}; + struct bnxt_re_res_cntrs { atomic_t qp_count; atomic_t rc_qp_count; @@ -164,6 +174,7 @@ struct bnxt_re_rstat { struct bnxt_re_stats { struct bnxt_re_rstat rstat; struct bnxt_re_res_cntrs res; + struct bnxt_re_db_pacing_stats pacing; }; struct rdma_hw_stats *bnxt_re_ib_alloc_hw_port_stats(struct ib_device *ibdev, diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 91efa04..87960ac 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -533,6 +533,7 @@ static void bnxt_re_db_fifo_check(struct work_struct *work) pacing_data->pacing_th * BNXT_RE_PACING_ALARM_TH_MULTIPLE; schedule_delayed_work(&rdev->dbq_pacing_work, msecs_to_jiffies(rdev->pacing.dbq_pacing_time)); + rdev->stats.pacing.alerts++; mutex_unlock(&rdev->pacing.dbq_lock); } @@ -563,12 +564,14 @@ static void bnxt_re_pacing_timer_exp(struct work_struct *work) pacing_data->do_pacing = max_t(u32, rdev->pacing.dbr_def_do_pacing, pacing_data->do_pacing); if (pacing_data->do_pacing <= rdev->pacing.dbr_def_do_pacing) { bnxt_re_set_default_pacing_data(rdev); + rdev->stats.pacing.complete++; goto dbq_unlock; } restart_timer: schedule_delayed_work(&rdev->dbq_pacing_work, msecs_to_jiffies(rdev->pacing.dbq_pacing_time)); + rdev->stats.pacing.resched++; dbq_unlock: rdev->pacing.do_pacing_save = pacing_data->do_pacing; mutex_unlock(&rdev->pacing.dbq_lock);