From patchwork Wed Jul 19 05:02:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13318146 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B811DEB64DA for ; Wed, 19 Jul 2023 05:17:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229528AbjGSFRU (ORCPT ); Wed, 19 Jul 2023 01:17:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229454AbjGSFRT (ORCPT ); Wed, 19 Jul 2023 01:17:19 -0400 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 188811BF2 for ; Tue, 18 Jul 2023 22:17:18 -0700 (PDT) Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1b8b2b60731so36588255ad.2 for ; Tue, 18 Jul 2023 22:17:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1689743837; x=1692335837; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=KouEAsGFwzAwDcajTw1d0AyrhbQafSedR0kwWYC3+Gw=; b=SUHft9sisOLh6xRYnKwNtdSQn5LN06EMt7vgqLaOES3ea+uWU1SxB7kK1OAfYG5/3G OUJqszttCKfTJDBIGJ9xc7Cq9ZJY398kHDada+j/UxnMtoLHOSeBRCSxwIqt/MMUKnhQ Oj+4cEBKbkZ9/MbsFFa17AQzuwm2AWkZ66XIU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689743837; x=1692335837; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KouEAsGFwzAwDcajTw1d0AyrhbQafSedR0kwWYC3+Gw=; b=iGL/kjZBfHx+WjSEWlmVT8lj2jHzimkstJzPTYBSRP+eAQ9Ferbx2yGIEi3HGvrpNJ u51WnZ9F77IMWViHbar3C3z4UUCRouhc19EgxddVGPDClR+IkumlYDtw7EISZnD22wFE 0qhvmyed25qsVWlMIVLS9RgAJwnoPSPmMEcavnL/En2nVbp36WIIZyTFpqjK/NJ5fbDY Rl+jQw4uke6ebo2jixO+TVprC4GDyRLPMhWiV4KoB/C1N55/Aesj0hQKQj79ealQV4Es c/oqGKFNCOEUKS5BP5ufxWF/ZT4IaWIeCKUxVag5FtxJGLwSAqMgIC8hBOhP9a3phQXN emQg== X-Gm-Message-State: ABy/qLbWdGPnk8p0Pr9A7JKm1PSxZpGaCf/+5IOVGWEnCf21dREObOOn dUVWjstEQSpXmCx6sVr037tIVq+XFO2bHGwnIcY= X-Google-Smtp-Source: APBJJlH6mdR6yZ8NaECsdvcwpMFtdJNQ/BHg5a+NiScxR5vstYH+Bp1oQ1OW3q2OE5rBkqICHL1Wiw== X-Received: by 2002:a17:902:788d:b0:1a9:40d5:b0ae with SMTP id q13-20020a170902788d00b001a940d5b0aemr13782730pll.12.1689743837501; Tue, 18 Jul 2023 22:17:17 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id t8-20020a170902e84800b001b9de8fbd78sm2816424plg.212.2023.07.18.22.17.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Jul 2023 22:17:15 -0700 (PDT) From: Selvin Xavier To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, michael.chan@broadcom.com, Chandramohan Akula , Selvin Xavier Subject: [PATCH for-next v3 1/7] bnxt_en: Update HW interface headers Date: Tue, 18 Jul 2023 22:02:51 -0700 Message-Id: <1689742977-9128-2-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> References: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chandramohan Akula Updating the HW structures for the doorbell pacing related information. Newly added interface structures will be used in the followup patches. CC: Michael Chan Signed-off-by: Chandramohan Akula Signed-off-by: Selvin Xavier --- drivers/net/ethernet/broadcom/bnxt/bnxt_hsi.h | 54 +++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_hsi.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_hsi.h index b31de4c..a2d3a80 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_hsi.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_hsi.h @@ -3721,6 +3721,60 @@ struct hwrm_func_backing_store_qcaps_v2_output { u8 valid; }; +/* hwrm_func_dbr_pacing_qcfg_input (size:128b/16B) */ +struct hwrm_func_dbr_pacing_qcfg_input { + __le16 req_type; + __le16 cmpl_ring; + __le16 seq_id; + __le16 target_id; + __le64 resp_addr; +}; + +/* hwrm_func_dbr_pacing_qcfg_output (size:512b/64B) */ +struct hwrm_func_dbr_pacing_qcfg_output { + __le16 error_code; + __le16 req_type; + __le16 seq_id; + __le16 resp_len; + u8 flags; +#define FUNC_DBR_PACING_QCFG_RESP_FLAGS_DBR_NQ_EVENT_ENABLED 0x1UL + u8 unused_0[7]; + __le32 dbr_stat_db_fifo_reg; +#define FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_MASK 0x3UL +#define FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_SFT 0 +#define FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_PCIE_CFG 0x0UL +#define FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_GRC 0x1UL +#define FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_BAR0 0x2UL +#define FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_BAR1 0x3UL +#define FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_LAST \ + FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_BAR1 +#define FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_MASK 0xfffffffcUL +#define FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_SFT 2 + __le32 dbr_stat_db_fifo_reg_watermark_mask; + u8 dbr_stat_db_fifo_reg_watermark_shift; + u8 unused_1[3]; + __le32 dbr_stat_db_fifo_reg_fifo_room_mask; + u8 dbr_stat_db_fifo_reg_fifo_room_shift; + u8 unused_2[3]; + __le32 dbr_throttling_aeq_arm_reg; +#define FUNC_DBR_PACING_QCFG_RESP_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_MASK 0x3UL +#define FUNC_DBR_PACING_QCFG_RESP_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_SFT 0 +#define FUNC_DBR_PACING_QCFG_RESP_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_PCIE_CFG 0x0UL +#define FUNC_DBR_PACING_QCFG_RESP_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_GRC 0x1UL +#define FUNC_DBR_PACING_QCFG_RESP_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_BAR0 0x2UL +#define FUNC_DBR_PACING_QCFG_RESP_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_BAR1 0x3UL +#define FUNC_DBR_PACING_QCFG_RESP_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_LAST \ + FUNC_DBR_PACING_QCFG_RESP_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SPACE_BAR1 +#define FUNC_DBR_PACING_QCFG_RESP_DBR_THROTTLING_AEQ_ARM_REG_ADDR_MASK 0xfffffffcUL +#define FUNC_DBR_PACING_QCFG_RESP_DBR_THROTTLING_AEQ_ARM_REG_ADDR_SFT 2 + u8 dbr_throttling_aeq_arm_reg_val; + u8 unused_3[7]; + __le32 primary_nq_id; + __le32 pacing_threshold; + u8 unused_4[7]; + u8 valid; +}; + /* hwrm_func_drv_if_change_input (size:192b/24B) */ struct hwrm_func_drv_if_change_input { __le16 req_type; From patchwork Wed Jul 19 05:02:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13318147 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66B08C0015E for ; Wed, 19 Jul 2023 05:17:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229454AbjGSFRX (ORCPT ); Wed, 19 Jul 2023 01:17:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230063AbjGSFRW (ORCPT ); Wed, 19 Jul 2023 01:17:22 -0400 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E8DF1BF3 for ; Tue, 18 Jul 2023 22:17:21 -0700 (PDT) Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1b8b318c5cfso51478565ad.1 for ; Tue, 18 Jul 2023 22:17:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1689743841; x=1692335841; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=Jr8OTTTb/MRT26Be5IIEhl6v64ZsBGtwxI7/5nEM/WU=; b=dkRSQr+pPmuKSxhMAIipTt7w2KHXFKbLsKyvYLMmqCc526PBUfnROmLSYDD40ZeDHR q2deVoFI7ZbIeqctEgHjqtAR71ZiZQriDRCPFEukpnaHI8I0xH/AP+o1TzTHEiWacvga KI505OirkYnbJ8sDoQgdUziFk2FlAhG9fc9pw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689743841; x=1692335841; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Jr8OTTTb/MRT26Be5IIEhl6v64ZsBGtwxI7/5nEM/WU=; b=PvqsnsDOcCLcb/PL4LIrhWVtUiZUQbOlf61/2DeiQSGh9zKFTwKrGcuE4QqjxnN7od azuEBmAcKqqpFgzgVBDnYnKrgLrfpt0wutvAMtYtrFA/x2p+IATQm7inJdMTPksS1mGT f4LOzXVFwlpOZfkbJtDPoWme9SAsIVNi0Ie5On6uw+wwmbVa8BkNFVY9+Qp7jULwidEN hRLuNqX4QhnyqN/bvvTvXDqeMmdFFygEz7l8qDK80oc4NVyMdyvDwtBHlvfYVrwDSQe2 Xk4fky8zbxDx4q2JiVJ812qzq0U4GHnSccDAZcrG+o/CuTtDUNU7HkTdE/olV6MvH+Pr AnpQ== X-Gm-Message-State: ABy/qLYWF1Qoa7KVFqR3YiYbO4N01NFLmMFosM2VfWzEpdMIcAumGTZ1 qgjoRU7xcGtHSmwvET5FLcRB1Q== X-Google-Smtp-Source: APBJJlGCb03zparj8fX0ADIGYfkL+DHPKdLI1G4BJWO+YPhp6qPbgefoJYAlKLy4uS2gXDgHn0QrTQ== X-Received: by 2002:a17:902:d504:b0:1b6:76ee:190b with SMTP id b4-20020a170902d50400b001b676ee190bmr20269648plg.35.1689743840996; Tue, 18 Jul 2023 22:17:20 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id t8-20020a170902e84800b001b9de8fbd78sm2816424plg.212.2023.07.18.22.17.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Jul 2023 22:17:19 -0700 (PDT) From: Selvin Xavier To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, michael.chan@broadcom.com, Chandramohan Akula , Selvin Xavier Subject: [PATCH for-next v3 2/7] bnxt_en: Share the bar0 address with the RoCE driver Date: Tue, 18 Jul 2023 22:02:52 -0700 Message-Id: <1689742977-9128-3-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> References: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chandramohan Akula Add a parameter in the bnxt_en_dev structure to share the bar0 address with RoCE driver. CC: Michael Chan Signed-off-by: Chandramohan Akula Signed-off-by: Selvin Xavier --- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index 852eb44..6ba2b93 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -345,7 +345,7 @@ static void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp) edev->hw_ring_stats_size = bp->hw_ring_stats_size; edev->pf_port_id = bp->pf.port_id; edev->en_state = bp->state; - + edev->bar0 = bp->bar0; edev->ulp_tbl->msix_requested = bnxt_get_ulp_msix_num(bp); } diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index 80cbc4b..6ff77f0 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -81,6 +81,7 @@ struct bnxt_en_dev { * mode only. Will be * updated in resume. */ + void __iomem *bar0; }; static inline bool bnxt_ulp_registered(struct bnxt_en_dev *edev) From patchwork Wed Jul 19 05:02:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13318148 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B2D2EB64DA for ; Wed, 19 Jul 2023 05:17:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230076AbjGSFR3 (ORCPT ); Wed, 19 Jul 2023 01:17:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51300 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230163AbjGSFR1 (ORCPT ); Wed, 19 Jul 2023 01:17:27 -0400 Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B74D1FC0 for ; Tue, 18 Jul 2023 22:17:26 -0700 (PDT) Received: by mail-pl1-x629.google.com with SMTP id d9443c01a7336-1b8a44ee159so40223095ad.3 for ; Tue, 18 Jul 2023 22:17:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1689743845; x=1692335845; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=BOo91a4BXQHGtZ0GJDA9EHUfdGncVJC41jxNg72T98s=; b=QKG3xFSHBvf3POSvvtmC+TyYiUA2qyK4I5/sEFRr7kk1NEVd+ePego0Ja8HY6X/Dtg +qquvkfwkb499hsJkS6n9qSWAKkvQir/DX66dPUcNPJb0pRNCCv0kGfTXwzXDVHwPvdd 9DS66Cr2AXvbcm/v7vOKzEl75tD7IUKRRaexY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689743845; x=1692335845; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=BOo91a4BXQHGtZ0GJDA9EHUfdGncVJC41jxNg72T98s=; b=GV6Y5Nh4azPzPGbg9Ptn6SlZnaM02UC05PpMV0NObr38rE/nDSIx63Aun/lSwGiCXz /2Dd1PmIiGT1aBrByRpDHxhZRDbxnGGs+xzq63h5AGmrhywz4d6HPLPiMz8N6XwBkHmW paRCTOkfTTz/L2tfQM9dsvCa1/eE3RiJddiOJ/IIiYYBIzQmgoFvHUeieA9xKVmIzCHh LsXD7tQAi0T5spiF34P5rQsCw5XtudBZTsMu+C5UpNX856j8R3ChUxL7X6TP64tLv6sR yEbsB3fgpoheiKpk8wuu3DH0r5D3+sNGnhhLDtt6XJ/55TeNz85I+VkAsT/ZuhCgivZV 4xuw== X-Gm-Message-State: ABy/qLYmn8JSoXYTsXlgB351mD/FSKqnpfDQyPAKzbyTTtlUUITa80dL LYldsHlHu1SZxX2WKANuuVJ0dw== X-Google-Smtp-Source: APBJJlFZIHBuRx7q7kvToAY+T7HpC+jjwC+6B7KvoLiMNmH/+XkYC4ThsSdxt+nPf0HItbqYzltBbw== X-Received: by 2002:a17:902:b285:b0:1b8:af5e:853f with SMTP id u5-20020a170902b28500b001b8af5e853fmr14388787plr.24.1689743845302; Tue, 18 Jul 2023 22:17:25 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id t8-20020a170902e84800b001b9de8fbd78sm2816424plg.212.2023.07.18.22.17.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Jul 2023 22:17:23 -0700 (PDT) From: Selvin Xavier To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, michael.chan@broadcom.com, Chandramohan Akula , Selvin Xavier Subject: [PATCH for-next v3 3/7] RDMA/bnxt_re: Initialize Doorbell pacing feature Date: Tue, 18 Jul 2023 22:02:53 -0700 Message-Id: <1689742977-9128-4-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> References: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chandramohan Akula Checks for pacing feature capability and get the doorbell pacing configuration using FW commands. Allocate a page and initialize the pacing parameters for the applications. Cleanup the page and de-initialize the pacing during device removal. Signed-off-by: Chandramohan Akula Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/bnxt_re.h | 22 +++++++ drivers/infiniband/hw/bnxt_re/main.c | 96 +++++++++++++++++++++++++++++++ drivers/infiniband/hw/bnxt_re/qplib_res.h | 19 ++++++ 3 files changed, 137 insertions(+) diff --git a/drivers/infiniband/hw/bnxt_re/bnxt_re.h b/drivers/infiniband/hw/bnxt_re/bnxt_re.h index ea81b24..1543f80 100644 --- a/drivers/infiniband/hw/bnxt_re/bnxt_re.h +++ b/drivers/infiniband/hw/bnxt_re/bnxt_re.h @@ -112,6 +112,27 @@ struct bnxt_re_gsi_context { #define BNXT_RE_NQ_IDX 1 #define BNXT_RE_GEN_P5_MAX_VF 64 +struct bnxt_re_pacing { + u64 dbr_db_fifo_reg_off; + void *dbr_page; + u64 dbr_bar_addr; + u32 pacing_algo_th; + u32 do_pacing_save; + u32 dbq_pacing_time; /* ms */ + u32 dbr_def_do_pacing; + bool dbr_pacing; +}; + +#define BNXT_RE_DBR_PACING_TIME 5 /* ms */ +#define BNXT_RE_PACING_ALGO_THRESHOLD 250 /* Entries in DB FIFO */ +#define BNXT_RE_PACING_ALARM_TH_MULTIPLE 2 /* Multiple of pacing algo threshold */ +/* Default do_pacing value when there is no congestion */ +#define BNXT_RE_DBR_DO_PACING_NO_CONGESTION 0x7F /* 1 in 512 probability */ +#define BNXT_RE_DB_FIFO_ROOM_MASK 0x1FFF8000 +#define BNXT_RE_MAX_FIFO_DEPTH 0x2c00 +#define BNXT_RE_DB_FIFO_ROOM_SHIFT 15 +#define BNXT_RE_GRC_FIFO_REG_BASE 0x2000 + struct bnxt_re_dev { struct ib_device ibdev; struct list_head list; @@ -171,6 +192,7 @@ struct bnxt_re_dev { atomic_t nq_alloc_cnt; u32 is_virtfn; u32 num_vfs; + struct bnxt_re_pacing pacing; }; #define to_bnxt_re_dev(ptr, member) \ diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index b42166f..13cd84d 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -432,9 +432,92 @@ int bnxt_re_hwrm_qcaps(struct bnxt_re_dev *rdev) return rc; cctx->modes.db_push = le32_to_cpu(resp.flags) & FUNC_QCAPS_RESP_FLAGS_WCB_PUSH_MODE; + cctx->modes.dbr_pacing = + le32_to_cpu(resp.flags_ext2) & FUNC_QCAPS_RESP_FLAGS_EXT2_DBR_PACING_EXT_SUPPORTED ? + true : false; return 0; } +static int bnxt_re_hwrm_dbr_pacing_qcfg(struct bnxt_re_dev *rdev) +{ + struct hwrm_func_dbr_pacing_qcfg_output resp = {}; + struct hwrm_func_dbr_pacing_qcfg_input req = {}; + struct bnxt_en_dev *en_dev = rdev->en_dev; + struct bnxt_qplib_chip_ctx *cctx; + struct bnxt_fw_msg fw_msg = {}; + int rc; + + cctx = rdev->chip_ctx; + bnxt_re_init_hwrm_hdr((void *)&req, HWRM_FUNC_DBR_PACING_QCFG); + bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, + sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); + rc = bnxt_send_msg(en_dev, &fw_msg); + if (rc) + return rc; + + if ((le32_to_cpu(resp.dbr_stat_db_fifo_reg) & + FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_MASK) == + FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_GRC) + cctx->dbr_stat_db_fifo = + le32_to_cpu(resp.dbr_stat_db_fifo_reg) & + ~FUNC_DBR_PACING_QCFG_RESP_DBR_STAT_DB_FIFO_REG_ADDR_SPACE_MASK; + return 0; +} + +/* Update the pacing tunable parameters to the default values */ +static void bnxt_re_set_default_pacing_data(struct bnxt_re_dev *rdev) +{ + struct bnxt_qplib_db_pacing_data *pacing_data = rdev->qplib_res.pacing_data; + + pacing_data->do_pacing = rdev->pacing.dbr_def_do_pacing; + pacing_data->pacing_th = rdev->pacing.pacing_algo_th; + pacing_data->alarm_th = + pacing_data->pacing_th * BNXT_RE_PACING_ALARM_TH_MULTIPLE; +} + +static int bnxt_re_initialize_dbr_pacing(struct bnxt_re_dev *rdev) +{ + if (bnxt_re_hwrm_dbr_pacing_qcfg(rdev)) + return -EIO; + + /* Allocate a page for app use */ + rdev->pacing.dbr_page = (void *)__get_free_page(GFP_KERNEL); + if (!rdev->pacing.dbr_page) + return -ENOMEM; + + memset((u8 *)rdev->pacing.dbr_page, 0, PAGE_SIZE); + rdev->qplib_res.pacing_data = (struct bnxt_qplib_db_pacing_data *)rdev->pacing.dbr_page; + + /* MAP HW window 2 for reading db fifo depth */ + writel(rdev->chip_ctx->dbr_stat_db_fifo & BNXT_GRC_BASE_MASK, + rdev->en_dev->bar0 + BNXT_GRCPF_REG_WINDOW_BASE_OUT + 4); + rdev->pacing.dbr_db_fifo_reg_off = + (rdev->chip_ctx->dbr_stat_db_fifo & BNXT_GRC_OFFSET_MASK) + + BNXT_RE_GRC_FIFO_REG_BASE; + rdev->pacing.dbr_bar_addr = + pci_resource_start(rdev->qplib_res.pdev, 0) + rdev->pacing.dbr_db_fifo_reg_off; + + rdev->pacing.pacing_algo_th = BNXT_RE_PACING_ALGO_THRESHOLD; + rdev->pacing.dbq_pacing_time = BNXT_RE_DBR_PACING_TIME; + rdev->pacing.dbr_def_do_pacing = BNXT_RE_DBR_DO_PACING_NO_CONGESTION; + rdev->pacing.do_pacing_save = rdev->pacing.dbr_def_do_pacing; + rdev->qplib_res.pacing_data->fifo_max_depth = BNXT_RE_MAX_FIFO_DEPTH; + rdev->qplib_res.pacing_data->fifo_room_mask = BNXT_RE_DB_FIFO_ROOM_MASK; + rdev->qplib_res.pacing_data->fifo_room_shift = BNXT_RE_DB_FIFO_ROOM_SHIFT; + rdev->qplib_res.pacing_data->grc_reg_offset = rdev->pacing.dbr_db_fifo_reg_off; + bnxt_re_set_default_pacing_data(rdev); + return 0; +} + +static void bnxt_re_deinitialize_dbr_pacing(struct bnxt_re_dev *rdev) +{ + if (rdev->pacing.dbr_page) + free_page((u64)rdev->pacing.dbr_page); + + rdev->pacing.dbr_page = NULL; + rdev->pacing.dbr_pacing = false; +} + static int bnxt_re_net_ring_free(struct bnxt_re_dev *rdev, u16 fw_ring_id, int type) { @@ -1217,6 +1300,9 @@ static void bnxt_re_dev_uninit(struct bnxt_re_dev *rdev) if (test_and_clear_bit(BNXT_RE_FLAG_GOT_MSIX, &rdev->flags)) rdev->num_msix = 0; + if (rdev->pacing.dbr_pacing) + bnxt_re_deinitialize_dbr_pacing(rdev); + bnxt_re_destroy_chip_ctx(rdev); if (test_and_clear_bit(BNXT_RE_FLAG_NETDEV_REGISTERED, &rdev->flags)) bnxt_unregister_dev(rdev->en_dev); @@ -1309,6 +1395,16 @@ static int bnxt_re_dev_init(struct bnxt_re_dev *rdev, u8 wqe_mode) goto free_ring; } + if (bnxt_qplib_dbr_pacing_en(rdev->chip_ctx)) { + rc = bnxt_re_initialize_dbr_pacing(rdev); + if (!rc) { + rdev->pacing.dbr_pacing = true; + } else { + ibdev_err(&rdev->ibdev, + "DBR pacing disabled with error : %d\n", rc); + rdev->pacing.dbr_pacing = false; + } + } rc = bnxt_qplib_get_dev_attr(&rdev->rcfw, &rdev->dev_attr, rdev->is_virtfn); if (rc) diff --git a/drivers/infiniband/hw/bnxt_re/qplib_res.h b/drivers/infiniband/hw/bnxt_re/qplib_res.h index d850a55..57161d3 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_res.h +++ b/drivers/infiniband/hw/bnxt_re/qplib_res.h @@ -48,6 +48,7 @@ extern const struct bnxt_qplib_gid bnxt_qplib_gid_zero; struct bnxt_qplib_drv_modes { u8 wqe_mode; bool db_push; + bool dbr_pacing; }; struct bnxt_qplib_chip_ctx { @@ -58,6 +59,17 @@ struct bnxt_qplib_chip_ctx { u16 hwrm_cmd_max_timeout; struct bnxt_qplib_drv_modes modes; u64 hwrm_intf_ver; + u32 dbr_stat_db_fifo; +}; + +struct bnxt_qplib_db_pacing_data { + u32 do_pacing; + u32 pacing_th; + u32 alarm_th; + u32 fifo_max_depth; + u32 fifo_room_mask; + u32 fifo_room_shift; + u32 grc_reg_offset; }; #define BNXT_QPLIB_DBR_PF_DB_OFFSET 0x10000 @@ -271,6 +283,7 @@ struct bnxt_qplib_res { struct mutex dpi_tbl_lock; bool prio; bool is_vf; + struct bnxt_qplib_db_pacing_data *pacing_data; }; static inline bool bnxt_qplib_is_chip_gen_p5(struct bnxt_qplib_chip_ctx *cctx) @@ -467,4 +480,10 @@ static inline bool _is_ext_stats_supported(u16 dev_cap_flags) return dev_cap_flags & CREQ_QUERY_FUNC_RESP_SB_EXT_STATS; } + +static inline u8 bnxt_qplib_dbr_pacing_en(struct bnxt_qplib_chip_ctx *cctx) +{ + return cctx->modes.dbr_pacing; +} + #endif /* __BNXT_QPLIB_RES_H__ */ From patchwork Wed Jul 19 05:02:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13318149 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9183C001DF for ; Wed, 19 Jul 2023 05:17:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230102AbjGSFRb (ORCPT ); Wed, 19 Jul 2023 01:17:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51300 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230136AbjGSFRb (ORCPT ); Wed, 19 Jul 2023 01:17:31 -0400 Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2AFB11BFC for ; Tue, 18 Jul 2023 22:17:29 -0700 (PDT) Received: by mail-pl1-x629.google.com with SMTP id d9443c01a7336-1bb2468257fso22006405ad.0 for ; Tue, 18 Jul 2023 22:17:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1689743849; x=1692335849; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=RmVKRmNjZud7L0IVo1jPlNTaqNlqkl9lPP0kNAaxJXI=; b=H2lzY+rzCE0+EqEF8ekPQIPckW4A/Xy1h2tXdfRs0mBjl0crt9mAktRvrQpHEcy+WC +tgiZ+1S9j4K1Oxz/ToH+/dofzp47m9a2HOmuupxbzS+1CKgOX2I+a4jd7LtiswUmKyM qOBqighQHR7si5rTvjq+gsltHFpZ6PPxV/Lfs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689743849; x=1692335849; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=RmVKRmNjZud7L0IVo1jPlNTaqNlqkl9lPP0kNAaxJXI=; b=CUEa61OEeep7cwKoLFXcRuYhf7Wj2Yu1akIRWmdSCSFnu+GJe3FKxeX1JaoB5/Y1iG Ghc4nExJoWSnR4DVKrK1sQiO9maWUbUVYBSTeAnv+licIHmC8x7RX5Lj2Im9HbUMknaF VisaJ6S3t3hMTsdMD6Zb3MSOhBnEOFOLCWQ1JLxewqka6f3iwWUhEg0el2/6F2G8fl70 zX6WgW0cgbK/LV6zfBbXfslDA7Av2IXfy2DY9BFZVxO6vTFpdvxJFWKVkKZdCTXFTHc/ 1OyBxqlfJh16TCrG4kY7BQlRyYj6VbulH/UObjenisOAXDTj+TfBdWfED9iD8D9ZZbw/ gchg== X-Gm-Message-State: ABy/qLbYvfj5tocRjyhdMW98hbWHSI3b+3yZBxQmZbF2HErQcUIDlTB5 cS5qbDdkdfaCUXXjGKaE+FRv4Q== X-Google-Smtp-Source: APBJJlF0TQcmIfQheChmkmoybJlAsDbrV1CaCn5zImEFP996UiShK8ezI3a/xGqvahpcQXWyJys04A== X-Received: by 2002:a17:903:48c:b0:1b8:7483:d47a with SMTP id jj12-20020a170903048c00b001b87483d47amr13807022plb.37.1689743849391; Tue, 18 Jul 2023 22:17:29 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id t8-20020a170902e84800b001b9de8fbd78sm2816424plg.212.2023.07.18.22.17.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Jul 2023 22:17:27 -0700 (PDT) From: Selvin Xavier To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, michael.chan@broadcom.com, Chandramohan Akula , Selvin Xavier Subject: [PATCH for-next v3 4/7] RDMA/bnxt_re: Enable pacing support for the user apps Date: Tue, 18 Jul 2023 22:02:54 -0700 Message-Id: <1689742977-9128-5-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> References: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chandramohan Akula Report the pacing capability to the user applications. Signed-off-by: Chandramohan Akula Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 2 ++ include/uapi/rdma/bnxt_re-abi.h | 1 + 2 files changed, 3 insertions(+) diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index abef0b8..86b71c7 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -4075,6 +4075,8 @@ int bnxt_re_alloc_ucontext(struct ib_ucontext *ctx, struct ib_udata *udata) goto cfail; } uctx->shpage_mmap = &entry->rdma_entry; + if (rdev->pacing.dbr_pacing) + resp.comp_mask |= BNXT_RE_UCNTX_CMASK_DBR_PACING_ENABLED; rc = ib_copy_to_udata(udata, &resp, min(udata->outlen, sizeof(resp))); if (rc) { diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h index 8a2a1d4..060bf1d 100644 --- a/include/uapi/rdma/bnxt_re-abi.h +++ b/include/uapi/rdma/bnxt_re-abi.h @@ -53,6 +53,7 @@ enum { BNXT_RE_UCNTX_CMASK_HAVE_CCTX = 0x1ULL, BNXT_RE_UCNTX_CMASK_HAVE_MODE = 0x02ULL, BNXT_RE_UCNTX_CMASK_WC_DPI_ENABLED = 0x04ULL, + BNXT_RE_UCNTX_CMASK_DBR_PACING_ENABLED = 0x08ULL, }; enum bnxt_re_wqe_mode { From patchwork Wed Jul 19 05:02:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13318150 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6C64C0015E for ; Wed, 19 Jul 2023 05:17:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230310AbjGSFRp (ORCPT ); Wed, 19 Jul 2023 01:17:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230234AbjGSFRl (ORCPT ); Wed, 19 Jul 2023 01:17:41 -0400 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D45381FCC for ; Tue, 18 Jul 2023 22:17:33 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1b8b318c5a7so50987025ad.3 for ; Tue, 18 Jul 2023 22:17:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1689743853; x=1692335853; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=yI1RVHR74RzIRXe42ygwfTiNqgFA7oOJZNGh7bE3s8c=; b=Nx5ZGgz42zyn9YKLN6nHD7tyswqI8wkX0fclV9dJgG3+9aWVBuEi/KaZFfoGRAOF5Y Znbdt9GOudjAgnb1KKMTGEXys6gyn7I2FYpsbxkgGA86+2fM0fIJ5w7+wJ/R4pR4YUxV /u/Qa/hr7DXKL1aIM+E9Fp/lssOY1nobkNs+g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689743853; x=1692335853; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yI1RVHR74RzIRXe42ygwfTiNqgFA7oOJZNGh7bE3s8c=; b=J3n2ESYD4GT7f63iVl9e83yv13T1/gjvG4T7ntPw4lpAmYn5BIa+NicoQLvKZAKy8g HsdJBDMR2z/26DE0ymIU6VtNCA3DPOi9f0B2/lVSfmi50sSFt4kNjGxxNleshpdki2np 3s7BImy7Wu+GfOBeSmA/ZxAPhy/lZivgGlVzaxVElGCZ0ldwunJXRQOyWl9KoDCTBH1o MFCod7duL51FJNbzIyMsK37jAN6meYOeDQme+Kcj1Gw8OQJY4iViXFgIMiorn/pqmGuo CoJzOf+yNAO0YRLihY4oOnb1WynS9SW6kMHEjHBYLu3UL6D83zYoIY2Nlv0y86zoP4UL Y0Iw== X-Gm-Message-State: ABy/qLayV6Jw2Rkj55cYDzfeaboMmwZDk5gZ98Opbe7q3uZZTfuFg9Q7 u5LInUx2++8P5+6pYJW85YEMXg== X-Google-Smtp-Source: APBJJlER6daGIYJ5z1fZWHAczFI81M4tgs6Sb8JYQ8D4RIaaj8Oh45P1mD+NAwEJpjYX9ZZIyaw3bQ== X-Received: by 2002:a17:902:bb96:b0:1b6:6bf0:eb8f with SMTP id m22-20020a170902bb9600b001b66bf0eb8fmr15150115pls.38.1689743853102; Tue, 18 Jul 2023 22:17:33 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id t8-20020a170902e84800b001b9de8fbd78sm2816424plg.212.2023.07.18.22.17.29 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Jul 2023 22:17:31 -0700 (PDT) From: Selvin Xavier To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, michael.chan@broadcom.com, Chandramohan Akula , Selvin Xavier Subject: [PATCH for-next v3 5/7] RDMA/bnxt_re: Update alloc_page uapi for pacing Date: Tue, 18 Jul 2023 22:02:55 -0700 Message-Id: <1689742977-9128-6-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> References: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chandramohan Akula Update the alloc_page uapi functionality for handling the mapping of doorbell pacing shared page and bar address. Signed-off-by: Chandramohan Akula Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 35 +++++++++++++++++++++++++++++--- drivers/infiniband/hw/bnxt_re/ib_verbs.h | 2 ++ include/uapi/rdma/bnxt_re-abi.h | 2 ++ 3 files changed, 36 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index 86b71c7..0703163 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -565,6 +565,8 @@ bnxt_re_mmap_entry_insert(struct bnxt_re_ucontext *uctx, u64 mem_offset, break; case BNXT_RE_MMAP_UC_DB: case BNXT_RE_MMAP_WC_DB: + case BNXT_RE_MMAP_DBR_BAR: + case BNXT_RE_MMAP_DBR_PAGE: ret = rdma_user_mmap_entry_insert(&uctx->ib_uctx, &entry->rdma_entry, PAGE_SIZE); break; @@ -4149,6 +4151,19 @@ int bnxt_re_mmap(struct ib_ucontext *ib_uctx, struct vm_area_struct *vma) case BNXT_RE_MMAP_SH_PAGE: ret = vm_insert_page(vma, vma->vm_start, virt_to_page(uctx->shpg)); break; + case BNXT_RE_MMAP_DBR_BAR: + pfn = bnxt_entry->mem_offset >> PAGE_SHIFT; + ret = rdma_user_mmap_io(ib_uctx, vma, pfn, PAGE_SIZE, + pgprot_noncached(vma->vm_page_prot), + rdma_entry); + break; + case BNXT_RE_MMAP_DBR_PAGE: + /* Driver doesn't expect write access for user space */ + if (vma->vm_flags & VM_WRITE) + return -EFAULT; + ret = vm_insert_page(vma, vma->vm_start, + virt_to_page((void *)bnxt_entry->mem_offset)); + break; default: ret = -EINVAL; break; @@ -4180,7 +4195,7 @@ static int UVERBS_HANDLER(BNXT_RE_METHOD_ALLOC_PAGE)(struct uverbs_attr_bundle * u64 mmap_offset; u32 length; u32 dpi; - u64 dbr; + u64 addr; int err; uctx = container_of(ib_uverbs_get_ucontext(attrs), struct bnxt_re_ucontext, ib_uctx); @@ -4202,19 +4217,30 @@ static int UVERBS_HANDLER(BNXT_RE_METHOD_ALLOC_PAGE)(struct uverbs_attr_bundle * return -ENOMEM; length = PAGE_SIZE; dpi = uctx->wcdpi.dpi; - dbr = (u64)uctx->wcdpi.umdbr; + addr = (u64)uctx->wcdpi.umdbr; mmap_flag = BNXT_RE_MMAP_WC_DB; } else { return -EINVAL; } break; + case BNXT_RE_ALLOC_DBR_BAR_PAGE: + length = PAGE_SIZE; + addr = (u64)rdev->pacing.dbr_bar_addr; + mmap_flag = BNXT_RE_MMAP_DBR_BAR; + break; + + case BNXT_RE_ALLOC_DBR_PAGE: + length = PAGE_SIZE; + addr = (u64)rdev->pacing.dbr_page; + mmap_flag = BNXT_RE_MMAP_DBR_PAGE; + break; default: return -EOPNOTSUPP; } - entry = bnxt_re_mmap_entry_insert(uctx, dbr, mmap_flag, &mmap_offset); + entry = bnxt_re_mmap_entry_insert(uctx, addr, mmap_flag, &mmap_offset); if (!entry) return -ENOMEM; @@ -4254,6 +4280,9 @@ static int alloc_page_obj_cleanup(struct ib_uobject *uobject, uctx->wcdpi.dbr = NULL; } break; + case BNXT_RE_MMAP_DBR_BAR: + case BNXT_RE_MMAP_DBR_PAGE: + break; default: goto exit; } diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h index 32d9e9d..f392a09 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h @@ -146,6 +146,8 @@ enum bnxt_re_mmap_flag { BNXT_RE_MMAP_SH_PAGE, BNXT_RE_MMAP_UC_DB, BNXT_RE_MMAP_WC_DB, + BNXT_RE_MMAP_DBR_PAGE, + BNXT_RE_MMAP_DBR_BAR, }; struct bnxt_re_user_mmap_entry { diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h index 060bf1d..78a324f 100644 --- a/include/uapi/rdma/bnxt_re-abi.h +++ b/include/uapi/rdma/bnxt_re-abi.h @@ -136,6 +136,8 @@ enum bnxt_re_objects { enum bnxt_re_alloc_page_type { BNXT_RE_ALLOC_WC_PAGE = 0, + BNXT_RE_ALLOC_DBR_BAR_PAGE, + BNXT_RE_ALLOC_DBR_PAGE, }; enum bnxt_re_var_alloc_page_attrs { From patchwork Wed Jul 19 05:02:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13318151 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB673C0015E for ; Wed, 19 Jul 2023 05:17:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230136AbjGSFRx (ORCPT ); Wed, 19 Jul 2023 01:17:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51598 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230319AbjGSFRq (ORCPT ); Wed, 19 Jul 2023 01:17:46 -0400 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A368F1FF6 for ; Tue, 18 Jul 2023 22:17:37 -0700 (PDT) Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-262cc036fa4so3258674a91.3 for ; Tue, 18 Jul 2023 22:17:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1689743857; x=1692335857; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=gPNBwLfxchRPAfG9conFfdMAmud5Umz1VaRSS53m1A0=; b=JT+WbOJrBtGJG1Pl6qVNZSqRBpFg33ayCB1wLsvo8t9M/FNzQxLkp9iu8FRE1ApWNo Klk7Kz0iL+3/NxCuN7OPP+jl2MZFz+BTxmUL1xunFxHCoSgtK0ier9vZK5nt0sK8AY4S 0r6ttj35kWP1x2yKxFFAqFMUeLh3o2jUpznN0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689743857; x=1692335857; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gPNBwLfxchRPAfG9conFfdMAmud5Umz1VaRSS53m1A0=; b=SqT7uNfyE1RSfeOueLTIvLvFDfhnhKEPghJdTDkug9FzuSe1+UdLR899hnK4RPYR7a kslLL4GnRlrnpxzsB9Xk2X1cbCk8g4//54+B/lUnoTA/o92bePOy4ywdgrxqgV+2fVZh tbNWc2IoSxrPbl4u7ii9AkjPeT3c0OCX+aynVSeFFYbEs7CTkl3z1qmZnYIP7HiOiGZi Lu3F7ooA2p0qd1TYjcfLVYy3RhiEulUrDwI3mR4krGqVgmO+d0NfNnP60LMrIqR00pLb KwkJpZei41guvkBT23f0w3tWBtDjcgvY2B9jIPl68Mbj17eWp2WxHNWPU9tjlqJyddf2 bmJA== X-Gm-Message-State: ABy/qLYZL7cnJPLSqYEcEM4Yh65mJajESbkT4bJH2DAUkn6kgVYT3+zA /jrTSVjjpMpWU5RVFAzJapSr0g== X-Google-Smtp-Source: APBJJlHBELBs2lcg+x4ZMRI4eoZ/blYYTClN1JqxqUUU5eKr/TlGHaZk6I0xQ4d2LOmrG2AE2E3rcw== X-Received: by 2002:a17:90a:6e09:b0:263:9329:19df with SMTP id b9-20020a17090a6e0900b00263932919dfmr12120451pjk.36.1689743856792; Tue, 18 Jul 2023 22:17:36 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id t8-20020a170902e84800b001b9de8fbd78sm2816424plg.212.2023.07.18.22.17.33 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Jul 2023 22:17:35 -0700 (PDT) From: Selvin Xavier To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, michael.chan@broadcom.com, Chandramohan Akula , Selvin Xavier Subject: [PATCH for-next v3 6/7] RDMA/bnxt_re: Implement doorbell pacing algorithm Date: Tue, 18 Jul 2023 22:02:56 -0700 Message-Id: <1689742977-9128-7-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> References: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chandramohan Akula User applications alert the driver when the Doorbell FIFO reaches the alarm threshold. The driver updates the pacing parameters in the shared page to do the maximum pacing by the application till the DB FIFO congestion reduces to pacing threshold. Driver keeps checking the DB FIFO depth at the pacing interval and gradually adjusts the pacing level. Once the pacing level reaches default values (no congestion in the FIFO) pacing gets completed. Signed-off-by: Chandramohan Akula Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/bnxt_re.h | 5 ++ drivers/infiniband/hw/bnxt_re/main.c | 124 ++++++++++++++++++++++++++++++++ 2 files changed, 129 insertions(+) diff --git a/drivers/infiniband/hw/bnxt_re/bnxt_re.h b/drivers/infiniband/hw/bnxt_re/bnxt_re.h index 1543f80..2175103 100644 --- a/drivers/infiniband/hw/bnxt_re/bnxt_re.h +++ b/drivers/infiniband/hw/bnxt_re/bnxt_re.h @@ -121,8 +121,10 @@ struct bnxt_re_pacing { u32 dbq_pacing_time; /* ms */ u32 dbr_def_do_pacing; bool dbr_pacing; + struct mutex dbq_lock; /* synchronize db pacing algo */ }; +#define BNXT_RE_MAX_DBR_DO_PACING 0xFFFF #define BNXT_RE_DBR_PACING_TIME 5 /* ms */ #define BNXT_RE_PACING_ALGO_THRESHOLD 250 /* Entries in DB FIFO */ #define BNXT_RE_PACING_ALARM_TH_MULTIPLE 2 /* Multiple of pacing algo threshold */ @@ -193,6 +195,8 @@ struct bnxt_re_dev { u32 is_virtfn; u32 num_vfs; struct bnxt_re_pacing pacing; + struct work_struct dbq_fifo_check_work; + struct delayed_work dbq_pacing_work; }; #define to_bnxt_re_dev(ptr, member) \ @@ -203,6 +207,7 @@ struct bnxt_re_dev { #define BNXT_RE_ROCEV2_IPV6_PACKET 3 #define BNXT_RE_CHECK_RC(x) ((x) && ((x) != -ETIMEDOUT)) +void bnxt_re_pacing_alert(struct bnxt_re_dev *rdev); static inline struct device *rdev_to_dev(struct bnxt_re_dev *rdev) { diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 13cd84d..6469811 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -475,6 +475,125 @@ static void bnxt_re_set_default_pacing_data(struct bnxt_re_dev *rdev) pacing_data->pacing_th * BNXT_RE_PACING_ALARM_TH_MULTIPLE; } +static void __wait_for_fifo_occupancy_below_th(struct bnxt_re_dev *rdev) +{ + u32 read_val, fifo_occup; + + /* loop shouldn't run infintely as the occupancy usually goes + * below pacing algo threshold as soon as pacing kicks in. + */ + while (1) { + read_val = readl(rdev->en_dev->bar0 + rdev->pacing.dbr_db_fifo_reg_off); + fifo_occup = BNXT_RE_MAX_FIFO_DEPTH - + ((read_val & BNXT_RE_DB_FIFO_ROOM_MASK) >> + BNXT_RE_DB_FIFO_ROOM_SHIFT); + /* Fifo occupancy cannot be greater the MAX FIFO depth */ + if (fifo_occup > BNXT_RE_MAX_FIFO_DEPTH) + break; + + if (fifo_occup < rdev->qplib_res.pacing_data->pacing_th) + break; + } +} + +static void bnxt_re_db_fifo_check(struct work_struct *work) +{ + struct bnxt_re_dev *rdev = container_of(work, struct bnxt_re_dev, + dbq_fifo_check_work); + struct bnxt_qplib_db_pacing_data *pacing_data; + u32 pacing_save; + + if (!mutex_trylock(&rdev->pacing.dbq_lock)) + return; + pacing_data = rdev->qplib_res.pacing_data; + pacing_save = rdev->pacing.do_pacing_save; + __wait_for_fifo_occupancy_below_th(rdev); + cancel_delayed_work_sync(&rdev->dbq_pacing_work); + if (pacing_save > rdev->pacing.dbr_def_do_pacing) { + /* Double the do_pacing value during the congestion */ + pacing_save = pacing_save << 1; + } else { + /* + * when a new congestion is detected increase the do_pacing + * by 8 times. And also increase the pacing_th by 4 times. The + * reason to increase pacing_th is to give more space for the + * queue to oscillate down without getting empty, but also more + * room for the queue to increase without causing another alarm. + */ + pacing_save = pacing_save << 3; + pacing_data->pacing_th = rdev->pacing.pacing_algo_th * 4; + } + + if (pacing_save > BNXT_RE_MAX_DBR_DO_PACING) + pacing_save = BNXT_RE_MAX_DBR_DO_PACING; + + pacing_data->do_pacing = pacing_save; + rdev->pacing.do_pacing_save = pacing_data->do_pacing; + pacing_data->alarm_th = + pacing_data->pacing_th * BNXT_RE_PACING_ALARM_TH_MULTIPLE; + schedule_delayed_work(&rdev->dbq_pacing_work, + msecs_to_jiffies(rdev->pacing.dbq_pacing_time)); + mutex_unlock(&rdev->pacing.dbq_lock); +} + +static void bnxt_re_pacing_timer_exp(struct work_struct *work) +{ + struct bnxt_re_dev *rdev = container_of(work, struct bnxt_re_dev, + dbq_pacing_work.work); + struct bnxt_qplib_db_pacing_data *pacing_data; + u32 read_val, fifo_occup; + + if (!mutex_trylock(&rdev->pacing.dbq_lock)) + return; + + pacing_data = rdev->qplib_res.pacing_data; + read_val = readl(rdev->en_dev->bar0 + rdev->pacing.dbr_db_fifo_reg_off); + fifo_occup = BNXT_RE_MAX_FIFO_DEPTH - + ((read_val & BNXT_RE_DB_FIFO_ROOM_MASK) >> + BNXT_RE_DB_FIFO_ROOM_SHIFT); + + if (fifo_occup > pacing_data->pacing_th) + goto restart_timer; + + /* + * Instead of immediately going back to the default do_pacing + * reduce it by 1/8 times and restart the timer. + */ + pacing_data->do_pacing = pacing_data->do_pacing - (pacing_data->do_pacing >> 3); + pacing_data->do_pacing = max_t(u32, rdev->pacing.dbr_def_do_pacing, pacing_data->do_pacing); + if (pacing_data->do_pacing <= rdev->pacing.dbr_def_do_pacing) { + bnxt_re_set_default_pacing_data(rdev); + goto dbq_unlock; + } + +restart_timer: + schedule_delayed_work(&rdev->dbq_pacing_work, + msecs_to_jiffies(rdev->pacing.dbq_pacing_time)); +dbq_unlock: + rdev->pacing.do_pacing_save = pacing_data->do_pacing; + mutex_unlock(&rdev->pacing.dbq_lock); +} + +void bnxt_re_pacing_alert(struct bnxt_re_dev *rdev) +{ + struct bnxt_qplib_db_pacing_data *pacing_data; + + if (!rdev->pacing.dbr_pacing) + return; + mutex_lock(&rdev->pacing.dbq_lock); + pacing_data = rdev->qplib_res.pacing_data; + + /* + * Increase the alarm_th to max so that other user lib instances do not + * keep alerting the driver. + */ + pacing_data->alarm_th = BNXT_RE_MAX_FIFO_DEPTH; + pacing_data->do_pacing = BNXT_RE_MAX_DBR_DO_PACING; + cancel_work_sync(&rdev->dbq_fifo_check_work); + schedule_work(&rdev->dbq_fifo_check_work); + mutex_unlock(&rdev->pacing.dbq_lock); +} + static int bnxt_re_initialize_dbr_pacing(struct bnxt_re_dev *rdev) { if (bnxt_re_hwrm_dbr_pacing_qcfg(rdev)) @@ -506,11 +625,16 @@ static int bnxt_re_initialize_dbr_pacing(struct bnxt_re_dev *rdev) rdev->qplib_res.pacing_data->fifo_room_shift = BNXT_RE_DB_FIFO_ROOM_SHIFT; rdev->qplib_res.pacing_data->grc_reg_offset = rdev->pacing.dbr_db_fifo_reg_off; bnxt_re_set_default_pacing_data(rdev); + /* Initialize worker for DBR Pacing */ + INIT_WORK(&rdev->dbq_fifo_check_work, bnxt_re_db_fifo_check); + INIT_DELAYED_WORK(&rdev->dbq_pacing_work, bnxt_re_pacing_timer_exp); return 0; } static void bnxt_re_deinitialize_dbr_pacing(struct bnxt_re_dev *rdev) { + cancel_work_sync(&rdev->dbq_fifo_check_work); + cancel_delayed_work_sync(&rdev->dbq_pacing_work); if (rdev->pacing.dbr_page) free_page((u64)rdev->pacing.dbr_page); From patchwork Wed Jul 19 05:02:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13318152 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B95DEB64DA for ; Wed, 19 Jul 2023 05:18:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230053AbjGSFR6 (ORCPT ); Wed, 19 Jul 2023 01:17:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51666 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230211AbjGSFRx (ORCPT ); Wed, 19 Jul 2023 01:17:53 -0400 Received: from mail-pg1-x534.google.com (mail-pg1-x534.google.com [IPv6:2607:f8b0:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E4C52112 for ; Tue, 18 Jul 2023 22:17:42 -0700 (PDT) Received: by mail-pg1-x534.google.com with SMTP id 41be03b00d2f7-55ae51a45deso4040095a12.3 for ; Tue, 18 Jul 2023 22:17:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1689743861; x=1692335861; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=kWllv783cfJ3hPP3B+Y2l8sJxbXv9rA4AB5lr5Fz+nU=; b=fM1DW7f3dM5Hmf5Fsvn6aunwaMpSusLxEvvtAl8VB4/n3rZe0F4VcfIDs4gyVsMLYw bszjncDkvftr0gONVoJ+Cx7s/zkDVhxsrG3pkn+LG1QNh5sSqL3xrU/GRjF/2Bu+A0Bc zPttXY6tYh5sP1P1ZUBvBw58kJ+iQz6IUWPqs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689743861; x=1692335861; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kWllv783cfJ3hPP3B+Y2l8sJxbXv9rA4AB5lr5Fz+nU=; b=OmpGhBRbQKt7ssSmTA21bHuEYfSrXodobK76PrMs/QXava117Gc1N/65jhew5n/Nyb 5Cj2Hpxrv+CQBE8kTwbnF2vVAfPZNwYbG7QKKyoN5jR46876oJvkFR3A8h1//84Q6Xn8 J4jsIXTpeX6CniGLvsO9STEhzpVJ/kKbKbS0p1HpwgSGaiptgBhenGBE+9YTDnTJ0JAs BWGe+fk0sSg5wGbOGl4nLysAvxlKzB53JVZz5J0GEHQ4g4iMK1sGp6+M4RYChinnq0Kh 3v0GjJZXaAPPt59pvydRmpWUUPYnU+96m0VfvKYl58UjddKEkfwmSbFc18yZB8gpGPvG bOHA== X-Gm-Message-State: ABy/qLYAhOcGU/xcuT3vXYaShoefxg2lXCb90hxY2NamROTDxb+fDrwU dF4YDiNCKuqa7QLlb9bE6dNHSg== X-Google-Smtp-Source: APBJJlGbHfMZAQok5XmElgRvb6WHVZnWicXzXpN2Wa/Zs7ym46bbvst/W9aPvaCLYz/FT7/c4nGTTg== X-Received: by 2002:a17:903:41cb:b0:1b8:2ba0:c9c0 with SMTP id u11-20020a17090341cb00b001b82ba0c9c0mr1525220ple.59.1689743861233; Tue, 18 Jul 2023 22:17:41 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id t8-20020a170902e84800b001b9de8fbd78sm2816424plg.212.2023.07.18.22.17.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Jul 2023 22:17:39 -0700 (PDT) From: Selvin Xavier To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, michael.chan@broadcom.com, Chandramohan Akula , Selvin Xavier Subject: [PATCH for-next v3 7/7] RDMA/bnxt_re: Add a new uapi for driver notification Date: Tue, 18 Jul 2023 22:02:57 -0700 Message-Id: <1689742977-9128-8-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> References: <1689742977-9128-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chandramohan Akula Add driver notify uapi for application notifying the driver about the doorbell FIFO congestion. Signed-off-by: Chandramohan Akula Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 15 +++++++++++++++ include/uapi/rdma/bnxt_re-abi.h | 4 ++++ 2 files changed, 19 insertions(+) diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index 0703163..ec4d163 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -4183,6 +4183,15 @@ void bnxt_re_mmap_free(struct rdma_user_mmap_entry *rdma_entry) kfree(bnxt_entry); } +static int UVERBS_HANDLER(BNXT_RE_METHOD_NOTIFY_DRV)(struct uverbs_attr_bundle *attrs) +{ + struct bnxt_re_ucontext *uctx; + + uctx = container_of(ib_uverbs_get_ucontext(attrs), struct bnxt_re_ucontext, ib_uctx); + bnxt_re_pacing_alert(uctx->rdev); + return 0; +} + static int UVERBS_HANDLER(BNXT_RE_METHOD_ALLOC_PAGE)(struct uverbs_attr_bundle *attrs) { struct ib_uobject *uobj = uverbs_attr_get_uobject(attrs, BNXT_RE_ALLOC_PAGE_HANDLE); @@ -4320,7 +4329,13 @@ DECLARE_UVERBS_NAMED_OBJECT(BNXT_RE_OBJECT_ALLOC_PAGE, &UVERBS_METHOD(BNXT_RE_METHOD_ALLOC_PAGE), &UVERBS_METHOD(BNXT_RE_METHOD_DESTROY_PAGE)); +DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_NOTIFY_DRV); + +DECLARE_UVERBS_GLOBAL_METHODS(BNXT_RE_OBJECT_NOTIFY_DRV, + &UVERBS_METHOD(BNXT_RE_METHOD_NOTIFY_DRV)); + const struct uapi_definition bnxt_re_uapi_defs[] = { UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_ALLOC_PAGE), + UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_NOTIFY_DRV), {} }; diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h index 78a324f..6e7c67a 100644 --- a/include/uapi/rdma/bnxt_re-abi.h +++ b/include/uapi/rdma/bnxt_re-abi.h @@ -132,6 +132,7 @@ enum bnxt_re_shpg_offt { enum bnxt_re_objects { BNXT_RE_OBJECT_ALLOC_PAGE = (1U << UVERBS_ID_NS_SHIFT), + BNXT_RE_OBJECT_NOTIFY_DRV, }; enum bnxt_re_alloc_page_type { @@ -157,4 +158,7 @@ enum bnxt_re_alloc_page_methods { BNXT_RE_METHOD_DESTROY_PAGE, }; +enum bnxt_re_notify_drv_methods { + BNXT_RE_METHOD_NOTIFY_DRV = (1U << UVERBS_ID_NS_SHIFT), +}; #endif /* __BNXT_RE_UVERBS_ABI_H__*/