From patchwork Wed Feb 1 20:44:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 13124989 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABCAEC636D4 for ; Wed, 1 Feb 2023 20:46:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231896AbjBAUqo (ORCPT ); Wed, 1 Feb 2023 15:46:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231552AbjBAUpn (ORCPT ); Wed, 1 Feb 2023 15:45:43 -0500 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 655CD7BBE1 for ; Wed, 1 Feb 2023 12:45:21 -0800 (PST) Received: by mail-pj1-x1032.google.com with SMTP id j5so18467525pjn.5 for ; Wed, 01 Feb 2023 12:45:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=aOE6cjkLUg0RtALmTJgRgj8bg8VL/peNPaqDm4MPBsc=; b=NEu6A6vZtlk+6Jbx7LMzfn+URCg3WF6k9xJ8rq+uCEi1Ma+ftSQh8XVztIPnAdPKP7 R7aBx8+ajYRZCcqhrgSYDfnIs1QlG1qYHaIoXrZAtBSWOQViE3KYqipP8fM09rlf6aPB 9WygnheQwWWZOZ4JzIIaL9JX9iDFDDAxYhNlc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=aOE6cjkLUg0RtALmTJgRgj8bg8VL/peNPaqDm4MPBsc=; b=Tz4lpZHuWCvYRXujCYja3iQ+q+lalRxh9H8T17ONJ4AEKT7OAbS2jG5oNa/4Su4Ih6 CRpYAVnXDTynOlJAha0zldvy84SY5vE0XuOKzcSqwE3EKjFssl1+ZJlwOFVjb8p7bbXt NmaR5gpHVHuT4AjhRNmjp91BVL6jh3YeC0i/sKNhzMlVaMuFQthFrq12q4q+ADLgKPxo nPpzjgjiiJ9bGKEPJ/STEhWI/pbSNzCd6TKxtXVqypczMLRgPvTQPyyqZPfw9UcpTwCO dpjI7WEBJCE30wb4oj6U70rlGM+JyNfEyJYKVx7nCg6FjF0KUvmI4hPrtKF7iucOalFZ o6cQ== X-Gm-Message-State: AO0yUKW5d3Hfr12bim7oSUlH2+MD0TKSr2wP6yjzEOVmj32n+xkec9PD Iktyt0OO1nFEfgkqV/B3lA3eVw== X-Google-Smtp-Source: AK7set+ndOk7OajJnHTbIEOspYm1n3LxXldRHLzi8+S0EdMWwXVqylXpleo+Azxc0rztL41ZGYNGEg== X-Received: by 2002:a17:902:ca91:b0:196:77e1:3da4 with SMTP id v17-20020a170902ca9100b0019677e13da4mr3318405pld.12.1675284312114; Wed, 01 Feb 2023 12:45:12 -0800 (PST) Received: from C02GC2QQMD6T.wifi.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id w9-20020a1709027b8900b0019682e27995sm6485795pll.223.2023.02.01.12.45.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 12:45:11 -0800 (PST) From: Ajit Khaparde To: ajit.khaparde@broadcom.com Cc: andrew.gospodarek@broadcom.com, davem@davemloft.net, edumazet@google.com, jgg@ziepe.ca, kuba@kernel.org, leon@kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, michael.chan@broadcom.com, netdev@vger.kernel.org, pabeni@redhat.com, selvin.xavier@broadcom.com, gregkh@linuxfoundation.org, Leon Romanovsky Subject: [PATCH net-next v10 4/8] bnxt_en: Use direct API instead of indirection Date: Wed, 1 Feb 2023 12:44:56 -0800 Message-Id: <20230201204500.19420-5-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20230201204500.19420-1-ajit.khaparde@broadcom.com> References: <20230201204500.19420-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org For a single ULP user there is no need for complicating function indirection calls. Remove all this complexity in favour of direct function calls exported by the bnxt_en driver. This allows to simplify the code greatly. Also remove unused ulp_async_notifier. Suggested-by: Leon Romanovsky Signed-off-by: Ajit Khaparde Reviewed-by: Andy Gospodarek Reviewed-by: Selvin Xavier Reviewed-by: Leon Romanovsky --- drivers/infiniband/hw/bnxt_re/main.c | 71 ++++-------------- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 1 - drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 75 +++++-------------- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h | 25 +++---- 4 files changed, 45 insertions(+), 127 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 5bf3dc067484..fa64fe9887bf 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -350,23 +350,6 @@ static struct bnxt_ulp_ops bnxt_re_ulp_ops = { /* RoCE -> Net driver */ -/* Driver registration routines used to let the networking driver (bnxt_en) - * to know that the RoCE driver is now installed - */ -static int bnxt_re_unregister_netdev(struct bnxt_re_dev *rdev) -{ - struct bnxt_en_dev *en_dev; - int rc; - - if (!rdev) - return -EINVAL; - - en_dev = rdev->en_dev; - - rc = en_dev->en_ops->bnxt_unregister_device(rdev->en_dev); - return rc; -} - static int bnxt_re_register_netdev(struct bnxt_re_dev *rdev) { struct bnxt_en_dev *en_dev; @@ -374,26 +357,12 @@ static int bnxt_re_register_netdev(struct bnxt_re_dev *rdev) en_dev = rdev->en_dev; - rc = en_dev->en_ops->bnxt_register_device(en_dev, - &bnxt_re_ulp_ops, rdev); - rdev->qplib_res.pdev = rdev->en_dev->pdev; + rc = bnxt_register_dev(en_dev, &bnxt_re_ulp_ops, rdev); + if (!rc) + rdev->qplib_res.pdev = rdev->en_dev->pdev; return rc; } -static int bnxt_re_free_msix(struct bnxt_re_dev *rdev) -{ - struct bnxt_en_dev *en_dev; - - if (!rdev) - return -EINVAL; - - en_dev = rdev->en_dev; - - en_dev->en_ops->bnxt_free_msix(rdev->en_dev); - - return 0; -} - static int bnxt_re_request_msix(struct bnxt_re_dev *rdev) { int rc = 0, num_msix_want = BNXT_RE_MAX_MSIX, num_msix_got; @@ -403,9 +372,9 @@ static int bnxt_re_request_msix(struct bnxt_re_dev *rdev) num_msix_want = min_t(u32, BNXT_RE_MAX_MSIX, num_online_cpus()); - num_msix_got = en_dev->en_ops->bnxt_request_msix(en_dev, - rdev->msix_entries, - num_msix_want); + num_msix_got = bnxt_req_msix_vecs(en_dev, + rdev->msix_entries, + num_msix_want); if (num_msix_got < BNXT_RE_MIN_MSIX) { rc = -EINVAL; goto done; @@ -466,7 +435,7 @@ static int bnxt_re_net_ring_free(struct bnxt_re_dev *rdev, req.ring_id = cpu_to_le16(fw_ring_id); bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); + rc = bnxt_send_msg(en_dev, &fw_msg); if (rc) ibdev_err(&rdev->ibdev, "Failed to free HW ring:%d :%#x", req.ring_id, rc); @@ -503,7 +472,7 @@ static int bnxt_re_net_ring_alloc(struct bnxt_re_dev *rdev, req.int_mode = ring_attr->mode; bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); + rc = bnxt_send_msg(en_dev, &fw_msg); if (!rc) *fw_ring_id = le16_to_cpu(resp.ring_id); @@ -531,7 +500,7 @@ static int bnxt_re_net_stats_ctx_free(struct bnxt_re_dev *rdev, req.stat_ctx_id = cpu_to_le32(fw_stats_ctx_id); bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); + rc = bnxt_send_msg(en_dev, &fw_msg); if (rc) ibdev_err(&rdev->ibdev, "Failed to free HW stats context %#x", rc); @@ -564,7 +533,7 @@ static int bnxt_re_net_stats_ctx_alloc(struct bnxt_re_dev *rdev, req.stat_ctx_flags = STAT_CTX_ALLOC_REQ_STAT_CTX_FLAGS_ROCE; bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); + rc = bnxt_send_msg(en_dev, &fw_msg); if (!rc) *fw_stats_ctx_id = le32_to_cpu(resp.stat_ctx_id); @@ -1050,7 +1019,7 @@ static int bnxt_re_query_hwrm_pri2cos(struct bnxt_re_dev *rdev, u8 dir, bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); + rc = bnxt_send_msg(en_dev, &fw_msg); if (rc) return rc; @@ -1233,7 +1202,7 @@ static void bnxt_re_query_hwrm_intf_version(struct bnxt_re_dev *rdev) req.hwrm_intf_upd = HWRM_VERSION_UPDATE; bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); + rc = bnxt_send_msg(en_dev, &fw_msg); if (rc) { ibdev_err(&rdev->ibdev, "Failed to query HW version, rc = 0x%x", rc); @@ -1296,20 +1265,12 @@ static void bnxt_re_dev_uninit(struct bnxt_re_dev *rdev) bnxt_re_net_ring_free(rdev, rdev->rcfw.creq.ring_id, type); bnxt_qplib_free_rcfw_channel(&rdev->rcfw); } - if (test_and_clear_bit(BNXT_RE_FLAG_GOT_MSIX, &rdev->flags)) { - rc = bnxt_re_free_msix(rdev); - if (rc) - ibdev_warn(&rdev->ibdev, - "Failed to free MSI-X vectors: %#x", rc); - } + if (test_and_clear_bit(BNXT_RE_FLAG_GOT_MSIX, &rdev->flags)) + bnxt_free_msix_vecs(rdev->en_dev); bnxt_re_destroy_chip_ctx(rdev); - if (test_and_clear_bit(BNXT_RE_FLAG_NETDEV_REGISTERED, &rdev->flags)) { - rc = bnxt_re_unregister_netdev(rdev); - if (rc) - ibdev_warn(&rdev->ibdev, - "Failed to unregister with netdev: %#x", rc); - } + if (test_and_clear_bit(BNXT_RE_FLAG_NETDEV_REGISTERED, &rdev->flags)) + bnxt_unregister_dev(rdev->en_dev); } /* worker thread for polling periodic events. Now used for QoS programming*/ diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 2540f1b980fb..fd6b5862f74e 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -2414,7 +2414,6 @@ static int bnxt_async_event_process(struct bnxt *bp, } bnxt_queue_sp_work(bp); async_event_process_exit: - bnxt_ulp_async_events(bp, cmpl); return 0; } diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index a88dfe139bad..e53f2f9157ff 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -28,9 +28,9 @@ static DEFINE_IDA(bnxt_aux_dev_ids); -static int bnxt_register_dev(struct bnxt_en_dev *edev, - struct bnxt_ulp_ops *ulp_ops, - void *handle) +int bnxt_register_dev(struct bnxt_en_dev *edev, + struct bnxt_ulp_ops *ulp_ops, + void *handle) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); @@ -55,8 +55,9 @@ static int bnxt_register_dev(struct bnxt_en_dev *edev, return 0; } +EXPORT_SYMBOL(bnxt_register_dev); -static int bnxt_unregister_dev(struct bnxt_en_dev *edev) +void bnxt_unregister_dev(struct bnxt_en_dev *edev) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); @@ -65,7 +66,7 @@ static int bnxt_unregister_dev(struct bnxt_en_dev *edev) ulp = edev->ulp_tbl; if (ulp->msix_requested) - edev->en_ops->bnxt_free_msix(edev); + bnxt_free_msix_vecs(edev); if (ulp->max_async_event_id) bnxt_hwrm_func_drv_rgtr(bp, NULL, 0, true); @@ -80,8 +81,9 @@ static int bnxt_unregister_dev(struct bnxt_en_dev *edev) } kfree(ulp); edev->ulp_tbl = NULL; - return 0; + return; } +EXPORT_SYMBOL(bnxt_unregister_dev); static void bnxt_fill_msix_vecs(struct bnxt *bp, struct bnxt_msix_entry *ent) { @@ -103,7 +105,7 @@ static void bnxt_fill_msix_vecs(struct bnxt *bp, struct bnxt_msix_entry *ent) } } -static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, +int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, struct bnxt_msix_entry *ent, int num_msix) { @@ -165,8 +167,9 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, edev->flags |= BNXT_EN_FLAG_MSIX_REQUESTED; return avail_msix; } +EXPORT_SYMBOL(bnxt_req_msix_vecs); -static void bnxt_free_msix_vecs(struct bnxt_en_dev *edev) +void bnxt_free_msix_vecs(struct bnxt_en_dev *edev) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); @@ -185,6 +188,7 @@ static void bnxt_free_msix_vecs(struct bnxt_en_dev *edev) return; } +EXPORT_SYMBOL(bnxt_free_msix_vecs); int bnxt_get_ulp_msix_num(struct bnxt *bp) { @@ -219,7 +223,7 @@ int bnxt_get_ulp_stat_ctxs(struct bnxt *bp) return 0; } -static int bnxt_send_msg(struct bnxt_en_dev *edev, +int bnxt_send_msg(struct bnxt_en_dev *edev, struct bnxt_fw_msg *fw_msg) { struct net_device *dev = edev->net; @@ -253,6 +257,7 @@ static int bnxt_send_msg(struct bnxt_en_dev *edev, hwrm_req_drop(bp, req); return rc; } +EXPORT_SYMBOL(bnxt_send_msg); static void bnxt_ulp_get(struct bnxt_ulp *ulp) { @@ -312,14 +317,11 @@ void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs) return; ulp = edev->ulp_tbl; - rcu_read_lock(); ops = rcu_dereference(ulp->ulp_ops); - if (!ops || !ops->ulp_sriov_config) { - rcu_read_unlock(); + if (!ops || !ops->ulp_sriov_config) return; - } + bnxt_ulp_get(ulp); - rcu_read_unlock(); ops->ulp_sriov_config(ulp->handle, num_vfs); bnxt_ulp_put(ulp); } @@ -376,37 +378,9 @@ void bnxt_ulp_irq_restart(struct bnxt *bp, int err) } } -void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *cmpl) -{ - u16 event_id = le16_to_cpu(cmpl->event_id); - struct bnxt_en_dev *edev = bp->edev; - struct bnxt_ulp_ops *ops; - struct bnxt_ulp *ulp; - - if (!bnxt_ulp_registered(edev)) - return; - - ulp = edev->ulp_tbl; - - rcu_read_lock(); - - ops = rcu_dereference(ulp->ulp_ops); - if (!ops || !ops->ulp_async_notifier) - goto exit; - if (!ulp->async_events_bmap || event_id > ulp->max_async_event_id) - goto exit; - - /* Read max_async_event_id first before testing the bitmap. */ - smp_rmb(); - if (test_bit(event_id, ulp->async_events_bmap)) - ops->ulp_async_notifier(ulp->handle, cmpl); -exit: - rcu_read_unlock(); -} - -static int bnxt_register_async_events(struct bnxt_en_dev *edev, - unsigned long *events_bmap, - u16 max_id) +int bnxt_register_async_events(struct bnxt_en_dev *edev, + unsigned long *events_bmap, + u16 max_id) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); @@ -420,15 +394,7 @@ static int bnxt_register_async_events(struct bnxt_en_dev *edev, bnxt_hwrm_func_drv_rgtr(bp, events_bmap, max_id + 1, true); return 0; } - -static const struct bnxt_en_ops bnxt_en_ops_tbl = { - .bnxt_register_device = bnxt_register_dev, - .bnxt_unregister_device = bnxt_unregister_dev, - .bnxt_request_msix = bnxt_req_msix_vecs, - .bnxt_free_msix = bnxt_free_msix_vecs, - .bnxt_send_fw_msg = bnxt_send_msg, - .bnxt_register_fw_async_events = bnxt_register_async_events, -}; +EXPORT_SYMBOL(bnxt_register_async_events); void bnxt_rdma_aux_device_uninit(struct bnxt *bp) { @@ -458,7 +424,6 @@ static void bnxt_aux_dev_release(struct device *dev) static void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp) { - edev->en_ops = &bnxt_en_ops_tbl; edev->net = bp->dev; edev->pdev = bp->pdev; edev->l2_db_size = bp->db_size; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index aa4e57f670d8..6ce2a1893bcc 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -26,8 +26,6 @@ struct bnxt_msix_entry { }; struct bnxt_ulp_ops { - /* async_notifier() cannot sleep (in BH context) */ - void (*ulp_async_notifier)(void *, struct hwrm_async_event_cmpl *); void (*ulp_stop)(void *); void (*ulp_start)(void *); void (*ulp_sriov_config)(void *, int); @@ -63,7 +61,6 @@ struct bnxt_en_dev { BNXT_EN_FLAG_ROCEV2_CAP) #define BNXT_EN_FLAG_MSIX_REQUESTED 0x4 #define BNXT_EN_FLAG_ULP_STOPPED 0x8 - const struct bnxt_en_ops *en_ops; struct bnxt_ulp *ulp_tbl; int l2_db_size; /* Doorbell BAR size in * bytes mapped by L2 @@ -75,19 +72,6 @@ struct bnxt_en_dev { */ }; -struct bnxt_en_ops { - int (*bnxt_register_device)(struct bnxt_en_dev *edev, - struct bnxt_ulp_ops *ulp_ops, void *handle); - int (*bnxt_unregister_device)(struct bnxt_en_dev *edev); - int (*bnxt_request_msix)(struct bnxt_en_dev *edev, - struct bnxt_msix_entry *ent, int num_msix); - void (*bnxt_free_msix)(struct bnxt_en_dev *edev); - int (*bnxt_send_fw_msg)(struct bnxt_en_dev *edev, - struct bnxt_fw_msg *fw_msg); - int (*bnxt_register_fw_async_events)(struct bnxt_en_dev *edev, - unsigned long *events_bmap, u16 max_id); -}; - static inline bool bnxt_ulp_registered(struct bnxt_en_dev *edev) { if (edev && edev->ulp_tbl) @@ -106,4 +90,13 @@ void bnxt_ulp_irq_restart(struct bnxt *bp, int err); void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *cmpl); void bnxt_rdma_aux_device_uninit(struct bnxt *bp); void bnxt_rdma_aux_device_init(struct bnxt *bp); +int bnxt_register_dev(struct bnxt_en_dev *edev, struct bnxt_ulp_ops *ulp_ops, + void *handle); +void bnxt_unregister_dev(struct bnxt_en_dev *edev); +int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, struct bnxt_msix_entry *ent, + int num_msix); +void bnxt_free_msix_vecs(struct bnxt_en_dev *edev); +int bnxt_send_msg(struct bnxt_en_dev *edev, struct bnxt_fw_msg *fw_msg); +int bnxt_register_async_events(struct bnxt_en_dev *edev, + unsigned long *events_bmap, u16 max_id); #endif