From patchwork Mon Sep 13 13:28:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 12489407 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C42FC433F5 for ; Mon, 13 Sep 2021 13:35:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3208261221 for ; Mon, 13 Sep 2021 13:35:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241481AbhIMNgS (ORCPT ); Mon, 13 Sep 2021 09:36:18 -0400 Received: from mail-bn8nam12on2100.outbound.protection.outlook.com ([40.107.237.100]:65407 "EHLO NAM12-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S242611AbhIMN3q (ORCPT ); Mon, 13 Sep 2021 09:29:46 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ukw0Lomvl+oZjD5Xd0tWkXlGBm9MMi9cKwmYcXrpLdwk+nyvpzTJ+lUnpBSxhScJypr71J9Qu2/n4igC01keTvybc9ae7slG9bDY5+Czf5jRmJIi2iQBmPcb6PEMtvIfaSDuyDQFhOYyjVt6V+9h+CE5skdq8IwEhcEzKMLedMAAjdSsvh0wEHwFGK2uheiXX6nVVEQODeJzayIvDszUWe48at9KZUS9wD0gHu4bt3KmJQ4Xak7OtmShJcTalIo1pS9qSx445N43lepfdWo0RMc2FVoX1kIWjqXjDqzZ2K9Aw3nZB/3RbEXdSb89QZ1j28343hp2Fs+STvVoZcduuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=gtnNN7MYW/KYbpedMrTYpuT1KuhqzmVc8W4r92+535U=; b=a0xVJObZIbMNLTrzeyVJc1qNc2NDM9DvH+p9wWZA5LR3CVbMrFDayOmhhBqVgEI06ChlRohVuZmZqCUIcQzC8XRPCH9Lcv1u5ix1gPd6q5Cohn5tcDBokn1Hl9Rl/d9IJEojbWk5BH5KYv2dyIOk+ZOpM3i2cb7i6oQDKeBz4/91fG/qG5zB4iEeF8oZOsqlDtplnK502RyxyCIfSkEnW/Qt0s50uSY+NqNUpXJdIl9VfG7/FkNumWwjZkEgkPJz80/2fwAq5eLHoVLgt3uHiXVhfzZvlxG8kLqgerYAl1APBGorcR+iITkokh9wWP+gdEwb5GzRgs+f8r9Sqsgtjg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=cornelisnetworks.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gtnNN7MYW/KYbpedMrTYpuT1KuhqzmVc8W4r92+535U=; b=dTMgvKGN0dvJAg5VtViw9Fj7n0EAn8a7qK9woZRj3BsayjfDdq/LVx8IbLopnuFr7KuoiYfxYSdiMhE12DEQ16+wCRSzDZuR91W7yhT0IkrtCIOJvKR1OdBGp/Wj6TdqMk/SCrA8zbkypn4N65wysYH3TF8D1TN+Kv2S13AU4qhkocp0Ou8TISkw0tyPBt11KLptKY2vdqNA18i68oEv21grDu6UzEX1vHIj2TwUmfAma8Nk0x9Qdl/DlWmh5PjybKmyBgKkdceGF7y+EIcMFnjRijyOko2tWBY3s5b5Sw2dn2SH5svxiV5FkxiFTJSjCTh302cfQZSUtPTzz3XTbA== Received: from MWHPR20CA0013.namprd20.prod.outlook.com (2603:10b6:300:13d::23) by DM8PR01MB7109.prod.exchangelabs.com (2603:10b6:8:a::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15; Mon, 13 Sep 2021 13:28:28 +0000 Received: from CO1NAM11FT014.eop-nam11.prod.protection.outlook.com (2603:10b6:300:13d:cafe::63) by MWHPR20CA0013.outlook.office365.com (2603:10b6:300:13d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15 via Frontend Transport; Mon, 13 Sep 2021 13:28:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; cornelisnetworks.com; dkim=none (message not signed) header.d=none;cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-01.cornelisnetworks.com; Received: from awfm-01.cornelisnetworks.com (208.255.156.42) by CO1NAM11FT014.mail.protection.outlook.com (10.13.175.99) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 13:28:27 +0000 Received: from awfm-01.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-01.cornelisnetworks.com (8.14.7/8.14.7) with ESMTP id 18DDSQBf146867; Mon, 13 Sep 2021 09:28:26 -0400 Subject: [PATCH for-next 1/6] IB/hfi1: Remove cache and embed txreq in ring From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Mike Marciniszyn Date: Mon, 13 Sep 2021 09:28:26 -0400 Message-ID: <20210913132826.131370.4397.stgit@awfm-01.cornelisnetworks.com> In-Reply-To: <20210913132317.131370.54825.stgit@awfm-01.cornelisnetworks.com> References: <20210913132317.131370.54825.stgit@awfm-01.cornelisnetworks.com> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e5c51c60-e5e2-49b2-7f9c-08d976ba5eea X-MS-TrafficTypeDiagnostic: DM8PR01MB7109: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /+ayscpl3SJVkA4BXBb2mO+sIvYbzqvJph+jHhY5jLafkLh5OqUd2E0kwWlYWPL0M5Meuzz6IeLYNYQzaU4mFUZLSGDhxYn4O12xTLJptTsc4CvOPaPL5Pl8cqq55CBeOciDos0LLs91O1AjyTQVrq+o92J6eUgWYxFx2gpofU/0/nSSsibc+n294kVwxTBnFgcbyKHqvqB2qGg0wBpyxOX2SRbaOHdN2kbx+DaEy9FDLU5SpX7XFfE6nHPwL/8wmNeDLNedluFibdsKf6qhbLxU8uM5RM+TYs3kwq8QRgErhkaaLp0ei2leV2DXbrWDy4KiLV1up+6w/Tb5KVuCwE3lbosK+DkM/8vrTbP1DzgzlkXUigZzTfqxFb8XaLi1yy3DzOCXG3Fi54Vl4vyI41nt4kPP5DY2uwHct13yO0NHiTUUIAePOxUZiLI39fVCbXDhZmf+4xWQ/dTitlmc9DfhVjHd84+1K2ha/OBwPIlyvcH+pToJaSxA42qLGdDnrcj8De/XTQAIxJrZZddTtF528+2ouwdP+BMzkzAPi4e4ewttbTP8crIAVZxcp+sjqCyuzNks/EvOh8W90jT/ivH6dzTI8MA5Cpv90xKcpwa8XNamaexoUEa13/AFj6F6ahvzx7uS/AxHW3lpzhknL+DCMyvhb3du06FRMaOlPoDdF3wpisI+kcPQhuVVCd4D65O7SVyOFvDER3A3i6YJN1AdV9OinxlPgDUyuS57yUA= X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-01.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(4326008)(8676002)(55016002)(2906002)(186003)(356005)(70586007)(83380400001)(47076005)(30864003)(86362001)(36860700001)(44832011)(7126003)(508600001)(316002)(103116003)(5660300002)(7696005)(8936002)(426003)(107886003)(82310400003)(1076003)(70206006)(336012)(26005)(36906005)(81166007)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 13:28:27.6147 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e5c51c60-e5e2-49b2-7f9c-08d976ba5eea X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-01.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT014.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR01MB7109 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mike Marciniszyn This patch removes kmem cache allocation and deallocation in favor of having the ipoib_txreq in the ring. The consumer is now the packet sending side allocating tx descriptors from ring and the producer is the napi interrupt handling freeing tx descriptors. The locks are now eliminated because the napi tx lock insures a single consumer and the napi handling insures a single producer. The napi poll is converted to memory poll looking for items that have been marked completed. Fixes: d99dc602e2a5 ("IB/hfi1: Add functions to transmit datagram ipoib packets") Signed-off-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/ipoib.h | 17 +-- drivers/infiniband/hw/hfi1/ipoib_tx.c | 212 +++++++++++++++------------------ 2 files changed, 101 insertions(+), 128 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/ipoib.h b/drivers/infiniband/hw/hfi1/ipoib.h index 2cff38b..4e91b70 100644 --- a/drivers/infiniband/hw/hfi1/ipoib.h +++ b/drivers/infiniband/hw/hfi1/ipoib.h @@ -45,21 +45,19 @@ /** * struct hfi1_ipoib_circ_buf - List of items to be processed - * @items: ring of items + * @items: ring of items each a power of two size * @head: ring head * @tail: ring tail * @max_items: max items + 1 that the ring can contain - * @producer_lock: producer sync lock - * @consumer_lock: consumer sync lock + * @shift: log2 of size for getting txreq */ struct ipoib_txreq; struct hfi1_ipoib_circ_buf { - struct ipoib_txreq **items; - unsigned long head; - unsigned long tail; - unsigned long max_items; - spinlock_t producer_lock; /* head sync lock */ - spinlock_t consumer_lock; /* tail sync lock */ + void *items; + u32 head; + u32 tail; + u32 max_items; + u32 shift; }; /** @@ -102,7 +100,6 @@ struct hfi1_ipoib_dev_priv { struct net_device *netdev; struct ib_device *device; struct hfi1_ipoib_txq *txqs; - struct kmem_cache *txreq_cache; struct napi_struct *tx_napis; u16 pkey; u16 pkey_index; diff --git a/drivers/infiniband/hw/hfi1/ipoib_tx.c b/drivers/infiniband/hw/hfi1/ipoib_tx.c index e74ddbe..0a5d327 100644 --- a/drivers/infiniband/hw/hfi1/ipoib_tx.c +++ b/drivers/infiniband/hw/hfi1/ipoib_tx.c @@ -27,6 +27,7 @@ * @txreq: sdma transmit request * @sdma_hdr: 9b ib headers * @sdma_status: status returned by sdma engine + * @complete: non-zero implies complete * @priv: ipoib netdev private data * @txq: txq on which skb was output * @skb: skb to send @@ -35,6 +36,7 @@ struct ipoib_txreq { struct sdma_txreq txreq; struct hfi1_sdma_header sdma_hdr; int sdma_status; + int complete; struct hfi1_ipoib_dev_priv *priv; struct hfi1_ipoib_txq *txq; struct sk_buff *skb; @@ -51,7 +53,13 @@ struct ipoib_txparms { u8 entropy; }; -static u64 hfi1_ipoib_txreqs(const u64 sent, const u64 completed) +static struct ipoib_txreq * +hfi1_txreq_from_idx(struct hfi1_ipoib_circ_buf *r, u32 idx) +{ + return (struct ipoib_txreq *)(r->items + (idx << r->shift)); +} + +static u32 hfi1_ipoib_txreqs(const u64 sent, const u64 completed) { return sent - completed; } @@ -139,51 +147,55 @@ static void hfi1_ipoib_free_tx(struct ipoib_txreq *tx, int budget) } napi_consume_skb(tx->skb, budget); + tx->skb = NULL; sdma_txclean(priv->dd, &tx->txreq); - kmem_cache_free(priv->txreq_cache, tx); } -static int hfi1_ipoib_drain_tx_ring(struct hfi1_ipoib_txq *txq, int budget) +static void hfi1_ipoib_drain_tx_ring(struct hfi1_ipoib_txq *txq) { struct hfi1_ipoib_circ_buf *tx_ring = &txq->tx_ring; - unsigned long head; - unsigned long tail; - unsigned int max_tx; - int work_done; - int tx_count; - - spin_lock_bh(&tx_ring->consumer_lock); - - /* Read index before reading contents at that index. */ - head = smp_load_acquire(&tx_ring->head); - tail = tx_ring->tail; - max_tx = tx_ring->max_items; - - work_done = min_t(int, CIRC_CNT(head, tail, max_tx), budget); + int i; + struct ipoib_txreq *tx; - for (tx_count = work_done; tx_count; tx_count--) { - hfi1_ipoib_free_tx(tx_ring->items[tail], budget); - tail = CIRC_NEXT(tail, max_tx); + for (i = 0; i < tx_ring->max_items; i++) { + tx = hfi1_txreq_from_idx(tx_ring, i); + tx->complete = 0; + dev_kfree_skb_any(tx->skb); + tx->skb = NULL; + sdma_txclean(txq->priv->dd, &tx->txreq); } - - atomic64_add(work_done, &txq->complete_txreqs); - - /* Finished freeing tx items so store the tail value. */ - smp_store_release(&tx_ring->tail, tail); - - spin_unlock_bh(&tx_ring->consumer_lock); - - hfi1_ipoib_check_queue_stopped(txq); - - return work_done; + tx_ring->head = 0; + tx_ring->tail = 0; + atomic64_set(&txq->complete_txreqs, 0); + txq->sent_txreqs = 0; } -static int hfi1_ipoib_process_tx_ring(struct napi_struct *napi, int budget) +static int hfi1_ipoib_poll_tx_ring(struct napi_struct *napi, int budget) { struct hfi1_ipoib_dev_priv *priv = hfi1_ipoib_priv(napi->dev); struct hfi1_ipoib_txq *txq = &priv->txqs[napi - priv->tx_napis]; + struct hfi1_ipoib_circ_buf *tx_ring = &txq->tx_ring; + u32 head = tx_ring->head; + u32 max_tx = tx_ring->max_items; + int work_done; + struct ipoib_txreq *tx = hfi1_txreq_from_idx(tx_ring, head); - int work_done = hfi1_ipoib_drain_tx_ring(txq, budget); + trace_hfi1_txq_poll(txq); + for (work_done = 0; work_done < budget; work_done++) { + /* See hfi1_ipoib_sdma_complete() */ + if (!smp_load_acquire(&tx->complete)) + break; + tx->complete = 0; + hfi1_ipoib_free_tx(tx, budget); + head = CIRC_NEXT(head, max_tx); + tx = hfi1_txreq_from_idx(tx_ring, head); + } + atomic64_add(work_done, &txq->complete_txreqs); + + /* Finished freeing tx items so store the head value. */ + smp_store_release(&tx_ring->head, head); + + hfi1_ipoib_check_queue_stopped(txq); if (work_done < budget) napi_complete_done(napi, work_done); @@ -191,45 +203,15 @@ static int hfi1_ipoib_process_tx_ring(struct napi_struct *napi, int budget) return work_done; } -static void hfi1_ipoib_add_tx(struct ipoib_txreq *tx) -{ - struct hfi1_ipoib_circ_buf *tx_ring = &tx->txq->tx_ring; - unsigned long head; - unsigned long tail; - size_t max_tx; - - spin_lock(&tx_ring->producer_lock); - - head = tx_ring->head; - tail = READ_ONCE(tx_ring->tail); - max_tx = tx_ring->max_items; - - if (likely(CIRC_SPACE(head, tail, max_tx))) { - tx_ring->items[head] = tx; - - /* Finish storing txreq before incrementing head. */ - smp_store_release(&tx_ring->head, CIRC_ADD(head, 1, max_tx)); - napi_schedule_irqoff(tx->txq->napi); - } else { - struct hfi1_ipoib_txq *txq = tx->txq; - struct hfi1_ipoib_dev_priv *priv = tx->priv; - - /* Ring was full */ - hfi1_ipoib_free_tx(tx, 0); - atomic64_inc(&txq->complete_txreqs); - dd_dev_dbg(priv->dd, "txq %d full.\n", txq->q_idx); - } - - spin_unlock(&tx_ring->producer_lock); -} - static void hfi1_ipoib_sdma_complete(struct sdma_txreq *txreq, int status) { struct ipoib_txreq *tx = container_of(txreq, struct ipoib_txreq, txreq); + trace_hfi1_txq_complete(tx->txq); tx->sdma_status = status; - - hfi1_ipoib_add_tx(tx); + /* see hfi1_ipoib_poll_tx_ring */ + smp_store_release(&tx->complete, 1); + napi_schedule_irqoff(tx->txq->napi); } static int hfi1_ipoib_build_ulp_payload(struct ipoib_txreq *tx, @@ -385,19 +367,24 @@ static struct ipoib_txreq *hfi1_ipoib_send_dma_common(struct net_device *dev, struct ipoib_txparms *txp) { struct hfi1_ipoib_dev_priv *priv = hfi1_ipoib_priv(dev); + struct hfi1_ipoib_txq *txq = txp->txq; struct ipoib_txreq *tx; + struct hfi1_ipoib_circ_buf *tx_ring; + u32 tail; int ret; - tx = kmem_cache_alloc_node(priv->txreq_cache, - GFP_ATOMIC, - priv->dd->node); - if (unlikely(!tx)) + if (unlikely(hfi1_ipoib_used(txq) >= hfi1_ipoib_ring_hwat(txq))) + /* This shouldn't happen with a stopped queue */ return ERR_PTR(-ENOMEM); + tx_ring = &txq->tx_ring; + tail = tx_ring->tail; + tx = hfi1_txreq_from_idx(tx_ring, tx_ring->tail); + trace_hfi1_txq_alloc_tx(txq); /* so that we can test if the sdma descriptors are there */ tx->txreq.num_desc = 0; tx->priv = priv; - tx->txq = txp->txq; + tx->txq = txq; tx->skb = skb; INIT_LIST_HEAD(&tx->txreq.list); @@ -405,21 +392,20 @@ static struct ipoib_txreq *hfi1_ipoib_send_dma_common(struct net_device *dev, ret = hfi1_ipoib_build_tx_desc(tx, txp); if (likely(!ret)) { - if (txp->txq->flow.as_int != txp->flow.as_int) { - txp->txq->flow.tx_queue = txp->flow.tx_queue; - txp->txq->flow.sc5 = txp->flow.sc5; - txp->txq->sde = + if (txq->flow.as_int != txp->flow.as_int) { + txq->flow.tx_queue = txp->flow.tx_queue; + txq->flow.sc5 = txp->flow.sc5; + txq->sde = sdma_select_engine_sc(priv->dd, txp->flow.tx_queue, txp->flow.sc5); - trace_hfi1_flow_switch(txp->txq); + trace_hfi1_flow_switch(txq); } return tx; } sdma_txclean(priv->dd, &tx->txreq); - kmem_cache_free(priv->txreq_cache, tx); return ERR_PTR(ret); } @@ -480,8 +466,8 @@ static int hfi1_ipoib_send_dma_single(struct net_device *dev, struct sk_buff *skb, struct ipoib_txparms *txp) { - struct hfi1_ipoib_dev_priv *priv = hfi1_ipoib_priv(dev); struct hfi1_ipoib_txq *txq = txp->txq; + struct hfi1_ipoib_circ_buf *tx_ring; struct ipoib_txreq *tx; int ret; @@ -499,6 +485,9 @@ static int hfi1_ipoib_send_dma_single(struct net_device *dev, return NETDEV_TX_OK; } + tx_ring = &txq->tx_ring; + /* consume tx */ + smp_store_release(&tx_ring->tail, CIRC_NEXT(tx_ring->tail, tx_ring->max_items)); ret = hfi1_ipoib_submit_tx(txq, tx); if (likely(!ret)) { tx_ok: @@ -514,9 +503,10 @@ static int hfi1_ipoib_send_dma_single(struct net_device *dev, if (ret == -EBUSY || ret == -ECOMM) goto tx_ok; - sdma_txclean(priv->dd, &tx->txreq); - dev_kfree_skb_any(skb); - kmem_cache_free(priv->txreq_cache, tx); + /* mark complete and kick napi tx */ + smp_store_release(&tx->complete, 1); + napi_schedule(tx->txq->napi); + ++dev->stats.tx_carrier_errors; return NETDEV_TX_OK; @@ -527,6 +517,7 @@ static int hfi1_ipoib_send_dma_list(struct net_device *dev, struct ipoib_txparms *txp) { struct hfi1_ipoib_txq *txq = txp->txq; + struct hfi1_ipoib_circ_buf *tx_ring; struct ipoib_txreq *tx; /* Has the flow change ? */ @@ -556,6 +547,9 @@ static int hfi1_ipoib_send_dma_list(struct net_device *dev, return NETDEV_TX_OK; } + tx_ring = &txq->tx_ring; + /* consume tx */ + smp_store_release(&tx_ring->tail, CIRC_NEXT(tx_ring->tail, tx_ring->max_items)); list_add_tail(&tx->txreq.list, &txq->tx_list); hfi1_ipoib_check_queue_depth(txq); @@ -696,31 +690,22 @@ static void hfi1_ipoib_flush_txq(struct work_struct *work) int hfi1_ipoib_txreq_init(struct hfi1_ipoib_dev_priv *priv) { struct net_device *dev = priv->netdev; - char buf[HFI1_IPOIB_TXREQ_NAME_LEN]; - unsigned long tx_ring_size; + u32 tx_ring_size, tx_item_size; int i; - /* - * Ring holds 1 less than tx_ring_size - * Round up to next power of 2 in order to hold at least tx_queue_len - */ - tx_ring_size = roundup_pow_of_two((unsigned long)dev->tx_queue_len + 1); - - snprintf(buf, sizeof(buf), "hfi1_%u_ipoib_txreq_cache", priv->dd->unit); - priv->txreq_cache = kmem_cache_create(buf, - sizeof(struct ipoib_txreq), - 0, - 0, - NULL); - if (!priv->txreq_cache) - return -ENOMEM; - priv->tx_napis = kcalloc_node(dev->num_tx_queues, sizeof(struct napi_struct), GFP_KERNEL, priv->dd->node); if (!priv->tx_napis) - goto free_txreq_cache; + return -ENOMEM; + + /* + * Ring holds 1 less than tx_ring_size + * Round up to next power of 2 in order to hold at least tx_queue_len + */ + tx_ring_size = roundup_pow_of_two(dev->tx_queue_len + 1); + tx_item_size = roundup_pow_of_two(sizeof(struct ipoib_txreq)); priv->txqs = kcalloc_node(dev->num_tx_queues, sizeof(struct hfi1_ipoib_txq), @@ -756,19 +741,17 @@ int hfi1_ipoib_txreq_init(struct hfi1_ipoib_dev_priv *priv) priv->dd->node); txq->tx_ring.items = - kcalloc_node(tx_ring_size, - sizeof(struct ipoib_txreq *), + kcalloc_node(tx_ring_size, tx_item_size, GFP_KERNEL, priv->dd->node); if (!txq->tx_ring.items) goto free_txqs; - spin_lock_init(&txq->tx_ring.producer_lock); - spin_lock_init(&txq->tx_ring.consumer_lock); txq->tx_ring.max_items = tx_ring_size; + txq->tx_ring.shift = ilog2(tx_ring_size); txq->napi = &priv->tx_napis[i]; netif_tx_napi_add(dev, txq->napi, - hfi1_ipoib_process_tx_ring, + hfi1_ipoib_poll_tx_ring, NAPI_POLL_WEIGHT); } @@ -788,10 +771,6 @@ int hfi1_ipoib_txreq_init(struct hfi1_ipoib_dev_priv *priv) free_tx_napis: kfree(priv->tx_napis); priv->tx_napis = NULL; - -free_txreq_cache: - kmem_cache_destroy(priv->txreq_cache); - priv->txreq_cache = NULL; return -ENOMEM; } @@ -808,13 +787,13 @@ static void hfi1_ipoib_drain_tx_list(struct hfi1_ipoib_txq *txq) list_del(&txreq->list); sdma_txclean(txq->priv->dd, &tx->txreq); dev_kfree_skb_any(tx->skb); - kmem_cache_free(txq->priv->txreq_cache, tx); + tx->skb = NULL; atomic64_inc(complete_txreqs); } if (hfi1_ipoib_used(txq)) dd_dev_warn(txq->priv->dd, - "txq %d not empty found %llu requests\n", + "txq %d not empty found %u requests\n", txq->q_idx, hfi1_ipoib_txreqs(txq->sent_txreqs, atomic64_read(complete_txreqs))); @@ -831,7 +810,7 @@ void hfi1_ipoib_txreq_deinit(struct hfi1_ipoib_dev_priv *priv) iowait_sdma_drain(&txq->wait); hfi1_ipoib_drain_tx_list(txq); netif_napi_del(txq->napi); - (void)hfi1_ipoib_drain_tx_ring(txq, txq->tx_ring.max_items); + hfi1_ipoib_drain_tx_ring(txq); kfree(txq->tx_ring.items); } @@ -840,9 +819,6 @@ void hfi1_ipoib_txreq_deinit(struct hfi1_ipoib_dev_priv *priv) kfree(priv->tx_napis); priv->tx_napis = NULL; - - kmem_cache_destroy(priv->txreq_cache); - priv->txreq_cache = NULL; } void hfi1_ipoib_napi_tx_enable(struct net_device *dev) @@ -866,7 +842,7 @@ void hfi1_ipoib_napi_tx_disable(struct net_device *dev) struct hfi1_ipoib_txq *txq = &priv->txqs[i]; napi_disable(txq->napi); - (void)hfi1_ipoib_drain_tx_ring(txq, txq->tx_ring.max_items); + hfi1_ipoib_drain_tx_ring(txq); } } @@ -888,9 +864,9 @@ void hfi1_ipoib_tx_timeout(struct net_device *dev, unsigned int q) dd_dev_info(priv->dd, "flow %x\n", txq->flow.as_int); dd_dev_info(priv->dd, "sent %llu completed %llu used %llu\n", txq->sent_txreqs, completed, hfi1_ipoib_used(txq)); - dd_dev_info(priv->dd, "tx_queue_len %u max_items %lu\n", + dd_dev_info(priv->dd, "tx_queue_len %u max_items %u\n", dev->tx_queue_len, txq->tx_ring.max_items); - dd_dev_info(priv->dd, "head %lu tail %lu\n", + dd_dev_info(priv->dd, "head %u tail %u\n", txq->tx_ring.head, txq->tx_ring.tail); dd_dev_info(priv->dd, "wait queued %u\n", !list_empty(&txq->wait.list)); From patchwork Mon Sep 13 13:28:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 12489391 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55410C433F5 for ; Mon, 13 Sep 2021 13:32:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3B79061362 for ; Mon, 13 Sep 2021 13:32:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241558AbhIMNdx (ORCPT ); Mon, 13 Sep 2021 09:33:53 -0400 Received: from mail-bn8nam12on2130.outbound.protection.outlook.com ([40.107.237.130]:22881 "EHLO NAM12-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S242642AbhIMN3t (ORCPT ); Mon, 13 Sep 2021 09:29:49 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MJqZDUUy7PUHn35R3XNvGOTKf//CzdbHKHFmRoIKrMVWTkh3Fa91wczIkWoYc8DvnG/zbjunFmuqL7TiDH+hh0Rlm1V44UtD31mxBZs2BHH70noKS9HRK/ScNGH09NZH0v8B8N9o5tiEtS2LshjrXm3YKtpzzM4Aox2h4qKjofTPI11o4SxFKZeN9dAiA3pN5KMlQ6MjVXZ5ZoIS5x4fQg7zvjYLBo69OZ6F+t8naB8YAu6cFhLU9d/TXG1bCUsrfYGPAHF7QPMvWoZ/tjbzRZ6jdQJ1pJspziCvP7xsaRByc0Z1Jb7MJdFYURtHPE4bq767uf8LtEnx74oCRV4hxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=qzId785fVI2WA82cwYMS1wfD6SqtSAmEPbM7nvR65jc=; b=WEC7c2Jnh2aZDB74CHOjHH4BptslJOWu9c7RrTtkEXlJkqWEs09oI6A6kPFdGDX2rjLMHcnpjCaYs7EDdxG1EPLaZ1w8KCoYn7lClbqUHfknFmbCvvIHtCmF+7GCtCFrckwSb847kudlDkbJmAjJCmeL0qlZ1FDHETUzAj00aRnd1sesK44lgQGboVm//lXcHHOHFnBZf+SFVXMe2W9gTC2KgtwdQMkE527Pn2H60kZkLXoLE6/xk/GiqqK67ep1WoLl3iRy1bVEv+qiNPW6sNUwCNRV6DWhyg0z0H7QVIdiospWDQ08hPBMC0xmVtsIEy7+kI1IndI9war2EaOmsw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=cornelisnetworks.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qzId785fVI2WA82cwYMS1wfD6SqtSAmEPbM7nvR65jc=; b=h0EZ5wd32vPxJ0GYR0pA/EFpyWJcu2s6rpV2a0PszflII2W30in0n3SnRigNfXepq8GllWa8cBL68IcwFQraF8bHXd4mbyTdX/dF3ntA1Deju4t6FDL4nrl7/WrD4LHL2k57ECpfs6IyYF8Q8oyU+toAGbxOTTTzuZ8f70YCuAFtSpQDWs9myxDVSvuBie/JEHpdWXijXqKY7xrKXN58R1FEfgeZng805MZYBCMFaPsNFdEOVQKmVM36dwd5LaKuFx02IKZiu0tIprXWn9A1Gcp7upEEDuMk2F5dgfIFpfgpWwteFHzNlZA0sSGB4cU0+Y78Q78JW3Lo5XCW1M3l4w== Received: from BN6PR2001CA0016.namprd20.prod.outlook.com (2603:10b6:404:b4::26) by BL0PR01MB4708.prod.exchangelabs.com (2603:10b6:208:30::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Mon, 13 Sep 2021 13:28:32 +0000 Received: from BN8NAM11FT031.eop-nam11.prod.protection.outlook.com (2603:10b6:404:b4:cafe::b) by BN6PR2001CA0016.outlook.office365.com (2603:10b6:404:b4::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 13:28:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; cornelisnetworks.com; dkim=none (message not signed) header.d=none;cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-01.cornelisnetworks.com; Received: from awfm-01.cornelisnetworks.com (208.255.156.42) by BN8NAM11FT031.mail.protection.outlook.com (10.13.177.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 13:28:32 +0000 Received: from awfm-01.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-01.cornelisnetworks.com (8.14.7/8.14.7) with ESMTP id 18DDSVld147117; Mon, 13 Sep 2021 09:28:31 -0400 Subject: [PATCH for-next 2/6] IB/hfi1: Get rid of hot path divide From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Mike Marciniszyn Date: Mon, 13 Sep 2021 09:28:31 -0400 Message-ID: <20210913132831.131370.3993.stgit@awfm-01.cornelisnetworks.com> In-Reply-To: <20210913132317.131370.54825.stgit@awfm-01.cornelisnetworks.com> References: <20210913132317.131370.54825.stgit@awfm-01.cornelisnetworks.com> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c088b009-0f50-4d55-61b8-08d976ba619e X-MS-TrafficTypeDiagnostic: BL0PR01MB4708: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Eh4acXfgVnunQmhNIQNvt+1Sh9tAuzAzNQOtG2HwOxIY/ktszCKtb6Z5xkvtcDCvmG5/fzz2N/r8KtuRjtFUCEk970BFcUNR+vKcug8Q4SNyO6MCVAEVASRBgLtqZg84I1M1UD9WHE/i3gTCRoDk7qkjDfHfeQad0H+U1lTl5A8dlhTvglr4+OrXLK8rhxBRXX0FG2dysbbpat724kfMzqQO6mqFEM2F21mBzHwiOpBDBOmekvO7GdlWJiLQuc/WYLATUU001jM+YghWwo9FD5WE7w773r/xmlXswQ1i92whsacDYJgfDLgam8qQBTRyClDqieohyQX0oEq2z9HIfTB2gmlhMIPx3ag7bq8nRZxMmJ3VmWgEZA+Ah0QmXsZ3CORrCexH4uzUZpewvjJI7G/fm9/RgfNCi4UuGiu9dLp6hdEXn12j03ukqnB0nqRhVR6NQvPUMyhWr+/Ge1l/q1Roswyuz5gcb3ujD/BJuadpic2en0luWLGnW71O0vUJs+476TISLg7fXpmoxcG1Y18cBJqFrQIudTeOYpVzdKD3kp99yud7e8Qwz8HO74bR9FdrR+7v2IDrJF0dHyooLoKKFDjK+FpHuAb2IeYwpv95xHRJtoef4iTBuMweCV/+tmOiwwpUM5K58WmEnCtZXzXQLgQgkzgefsqjjkkS6WLDW53UtVRb0LUq61g811Lu7yqBeXG0GkigHmggFIWiJvBFPWVrBt5z+r/bDdYXqsmhboJyqq0eu1qGh/fBBCj0baznz2KPwFv1T67TyTK4jw== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-01.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(356005)(26005)(70586007)(2906002)(55016002)(47076005)(83380400001)(36860700001)(82310400003)(7696005)(508600001)(7126003)(107886003)(8676002)(103116003)(86362001)(1076003)(336012)(186003)(426003)(316002)(8936002)(5660300002)(44832011)(81166007)(4326008)(36906005)(70206006)(26583001)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 13:28:32.3000 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c088b009-0f50-4d55-61b8-08d976ba619e X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-01.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT031.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR01MB4708 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mike Marciniszyn The pointer math in this statemet does a divide; struct hfi1_ipoib_txq *txq = &priv->txqs[napi - priv->tx_napis]; Elminate the divide by embedding the struct napi_strut in the txq and getting the txq with a container_of() using the newly embedded napi. Fixes: d99dc602e2a5 ("IB/hfi1: Add functions to transmit datagram ipoib packets") Reviewed-by: Dennis Dalessandro Signed-off-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/ipoib.h | 3 +-- drivers/infiniband/hw/hfi1/ipoib_tx.c | 35 +++++++++------------------------ 2 files changed, 11 insertions(+), 27 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/ipoib.h b/drivers/infiniband/hw/hfi1/ipoib.h index 4e91b70..71b102d 100644 --- a/drivers/infiniband/hw/hfi1/ipoib.h +++ b/drivers/infiniband/hw/hfi1/ipoib.h @@ -78,6 +78,7 @@ struct hfi1_ipoib_circ_buf { * @tx_ring: ring of ipoib txreqs to be reaped by napi callback */ struct hfi1_ipoib_txq { + struct napi_struct napi; struct hfi1_ipoib_dev_priv *priv; struct sdma_engine *sde; struct list_head tx_list; @@ -91,7 +92,6 @@ struct hfi1_ipoib_txq { struct iowait wait; atomic64_t ____cacheline_aligned_in_smp complete_txreqs; - struct napi_struct *napi; struct hfi1_ipoib_circ_buf tx_ring; }; @@ -100,7 +100,6 @@ struct hfi1_ipoib_dev_priv { struct net_device *netdev; struct ib_device *device; struct hfi1_ipoib_txq *txqs; - struct napi_struct *tx_napis; u16 pkey; u16 pkey_index; u32 qkey; diff --git a/drivers/infiniband/hw/hfi1/ipoib_tx.c b/drivers/infiniband/hw/hfi1/ipoib_tx.c index 0a5d327..053eb43 100644 --- a/drivers/infiniband/hw/hfi1/ipoib_tx.c +++ b/drivers/infiniband/hw/hfi1/ipoib_tx.c @@ -172,8 +172,8 @@ static void hfi1_ipoib_drain_tx_ring(struct hfi1_ipoib_txq *txq) static int hfi1_ipoib_poll_tx_ring(struct napi_struct *napi, int budget) { - struct hfi1_ipoib_dev_priv *priv = hfi1_ipoib_priv(napi->dev); - struct hfi1_ipoib_txq *txq = &priv->txqs[napi - priv->tx_napis]; + struct hfi1_ipoib_txq *txq = + container_of(napi, struct hfi1_ipoib_txq, napi); struct hfi1_ipoib_circ_buf *tx_ring = &txq->tx_ring; u32 head = tx_ring->head; u32 max_tx = tx_ring->max_items; @@ -211,7 +211,7 @@ static void hfi1_ipoib_sdma_complete(struct sdma_txreq *txreq, int status) tx->sdma_status = status; /* see hfi1_ipoib_poll_tx_ring */ smp_store_release(&tx->complete, 1); - napi_schedule_irqoff(tx->txq->napi); + napi_schedule_irqoff(&tx->txq->napi); } static int hfi1_ipoib_build_ulp_payload(struct ipoib_txreq *tx, @@ -505,7 +505,7 @@ static int hfi1_ipoib_send_dma_single(struct net_device *dev, /* mark complete and kick napi tx */ smp_store_release(&tx->complete, 1); - napi_schedule(tx->txq->napi); + napi_schedule(&tx->txq->napi); ++dev->stats.tx_carrier_errors; @@ -693,13 +693,6 @@ int hfi1_ipoib_txreq_init(struct hfi1_ipoib_dev_priv *priv) u32 tx_ring_size, tx_item_size; int i; - priv->tx_napis = kcalloc_node(dev->num_tx_queues, - sizeof(struct napi_struct), - GFP_KERNEL, - priv->dd->node); - if (!priv->tx_napis) - return -ENOMEM; - /* * Ring holds 1 less than tx_ring_size * Round up to next power of 2 in order to hold at least tx_queue_len @@ -712,7 +705,7 @@ int hfi1_ipoib_txreq_init(struct hfi1_ipoib_dev_priv *priv) GFP_KERNEL, priv->dd->node); if (!priv->txqs) - goto free_tx_napis; + return -ENOMEM; for (i = 0; i < dev->num_tx_queues; i++) { struct hfi1_ipoib_txq *txq = &priv->txqs[i]; @@ -749,8 +742,7 @@ int hfi1_ipoib_txreq_init(struct hfi1_ipoib_dev_priv *priv) txq->tx_ring.max_items = tx_ring_size; txq->tx_ring.shift = ilog2(tx_ring_size); - txq->napi = &priv->tx_napis[i]; - netif_tx_napi_add(dev, txq->napi, + netif_tx_napi_add(dev, &txq->napi, hfi1_ipoib_poll_tx_ring, NAPI_POLL_WEIGHT); } @@ -761,16 +753,12 @@ int hfi1_ipoib_txreq_init(struct hfi1_ipoib_dev_priv *priv) for (i--; i >= 0; i--) { struct hfi1_ipoib_txq *txq = &priv->txqs[i]; - netif_napi_del(txq->napi); + netif_napi_del(&txq->napi); kfree(txq->tx_ring.items); } kfree(priv->txqs); priv->txqs = NULL; - -free_tx_napis: - kfree(priv->tx_napis); - priv->tx_napis = NULL; return -ENOMEM; } @@ -809,16 +797,13 @@ void hfi1_ipoib_txreq_deinit(struct hfi1_ipoib_dev_priv *priv) iowait_cancel_work(&txq->wait); iowait_sdma_drain(&txq->wait); hfi1_ipoib_drain_tx_list(txq); - netif_napi_del(txq->napi); + netif_napi_del(&txq->napi); hfi1_ipoib_drain_tx_ring(txq); kfree(txq->tx_ring.items); } kfree(priv->txqs); priv->txqs = NULL; - - kfree(priv->tx_napis); - priv->tx_napis = NULL; } void hfi1_ipoib_napi_tx_enable(struct net_device *dev) @@ -829,7 +814,7 @@ void hfi1_ipoib_napi_tx_enable(struct net_device *dev) for (i = 0; i < dev->num_tx_queues; i++) { struct hfi1_ipoib_txq *txq = &priv->txqs[i]; - napi_enable(txq->napi); + napi_enable(&txq->napi); } } @@ -841,7 +826,7 @@ void hfi1_ipoib_napi_tx_disable(struct net_device *dev) for (i = 0; i < dev->num_tx_queues; i++) { struct hfi1_ipoib_txq *txq = &priv->txqs[i]; - napi_disable(txq->napi); + napi_disable(&txq->napi); hfi1_ipoib_drain_tx_ring(txq); } } From patchwork Mon Sep 13 13:28:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 12489389 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3453C433FE for ; Mon, 13 Sep 2021 13:32:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 85FBA61507 for ; Mon, 13 Sep 2021 13:32:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240471AbhIMNds (ORCPT ); Mon, 13 Sep 2021 09:33:48 -0400 Received: from mail-mw2nam12on2114.outbound.protection.outlook.com ([40.107.244.114]:33536 "EHLO NAM12-MW2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S242697AbhIMN3z (ORCPT ); Mon, 13 Sep 2021 09:29:55 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HAwLRjp0eaN8v0MxvsVnJAEfbU8D8STF+nv1ztIVO51Xipl0RbfbM+kR5buVZuJFvTYfeEiFGjU+L3XGZigklti5GK10Yca5SZhyxflbpTfDyw6DFXQ246+HmX/iHqkzSGZg1uB4Pg8fxPus518Q3HsVVwLVh7PLXed6/+WdZrZcp5uKTH9R3oM9g5H0VSqWRwzCyV20Ed5hfEgTVtdcSHXjdvpEl6PGPWI/mi07ZzlN4+3Jod8YKCYfoRZCA7Y+uDvnx5qHuRmzQnsbTbG04exjEbTc+fw9+h9EJcsXWyBJMAvGgLR5tvgpgngManp4b2R3oeSCInLxf7RQLvwiiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=azbwXoyBPw4CFbHMQiU41wRCJyZPh/BW/PFkgRFDf10=; b=AGbaxQNVR0gXFPrL5ltnX4brYJ8j2N7CXN5Q4/q77V0l33lq7zxqfXtxVfAmvTelpfpnwJX8pHBrxwnrFw4gaLAl+U1qvYldbfRMQSLzAic4qohiZjhvRJgd/DYmL22vGEQ2lb3AappF5vFD3U6PKYU1ekBrxEHRVOQVBpO0ZxoeJ7uheZLUQyLL93D293EOW53kxSm0O92xjsFBHKqIhsNIzLyPzYsrsVNBQ99dhwxg6v6v2/0ETX/nSZyui0XUeG+0eaxD6q7eW7dnaxY97N6WKsDlx2jQenuxBqFSeetJ/A3iT9mp4DDJ6/ru+WGs5HBfbfgMM/fWjbB/bVTAbA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=cornelisnetworks.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=azbwXoyBPw4CFbHMQiU41wRCJyZPh/BW/PFkgRFDf10=; b=SNpR+WBgb/f9sVVQMmNhsGAHfVom4HLJfT/oUVcXPnwRjBcrguOlMahZXsEyTcr7nHJGCkdmIcQj8C7a+CUv0QWX37Bg6PfLIfiaAYqTZ3bJukEUee6VOY5XqB6g2BXM71D6KQAaIA8O/KDQwIxo4KueIWy82WpEbuVjnUmv9nDT7pnEVjeK0HFItC2XPeUcy0x8jQ5UP0/CdYCvWg7myLfoeuIRVNXkI6PR5GYRMC3NsvosRthouDnjqeZG1m4gtBnVcq5EbBikmkyhr2cFYGipHdX8Z9HvWmdJySxCbfEJQQxf03iVq3r+7O/C0Lq+n1wDYYSU+OjL0Y+ndlyYtw== Received: from BN6PR22CA0054.namprd22.prod.outlook.com (2603:10b6:404:ca::16) by PH0PR01MB6261.prod.exchangelabs.com (2603:10b6:510:8::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Mon, 13 Sep 2021 13:28:37 +0000 Received: from BN8NAM11FT033.eop-nam11.prod.protection.outlook.com (2603:10b6:404:ca:cafe::b4) by BN6PR22CA0054.outlook.office365.com (2603:10b6:404:ca::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 13:28:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; cornelisnetworks.com; dkim=none (message not signed) header.d=none;cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-01.cornelisnetworks.com; Received: from awfm-01.cornelisnetworks.com (208.255.156.42) by BN8NAM11FT033.mail.protection.outlook.com (10.13.177.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 13:28:37 +0000 Received: from awfm-01.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-01.cornelisnetworks.com (8.14.7/8.14.7) with ESMTP id 18DDSbtJ147395; Mon, 13 Sep 2021 09:28:37 -0400 Subject: [PATCH for-next 3/6] IB/hfi1: Get rid of tx priv backpointer From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Mike Marciniszyn Date: Mon, 13 Sep 2021 09:28:37 -0400 Message-ID: <20210913132836.131370.89704.stgit@awfm-01.cornelisnetworks.com> In-Reply-To: <20210913132317.131370.54825.stgit@awfm-01.cornelisnetworks.com> References: <20210913132317.131370.54825.stgit@awfm-01.cornelisnetworks.com> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ae925640-62b0-41bf-c6f7-08d976ba64c0 X-MS-TrafficTypeDiagnostic: PH0PR01MB6261: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FpsAbvrHXCc+KQORvP0o40QFYeitAkbKu/ErRhjvNb6xySpA8BwqWrbE895ifSgFsBDctI5H819Y/2E6tNms85z/uCkP1P+OdY8rp5xl3A9l17oIhitDebdp+67H7rTE9xtu2XYl5yNA4MN3RPin5Cp9C60K5ZPyHzRwHsW26kJshMlRgOOz54eSjBfaDsinBcBriAJN9MV1QJ4ZuvfKNqkpW1glQxEniCjN5ZY3I7eWMXDoHQRoQc/UtybsR0sbjmdMcT7Wcj+VPC/1lVWXj2Ofscz3ZrtbjDmyWPndNT8JpzQn5Sq1OWmp7K9ANyW5m0omCeSEPK0DkCP9po2zrnCumbWDTC87fNxGNZxyQDZhefWiNI5x8MiByBoToo4Hg7Vu6x2AHPcBAsPkl6qDeAPPBpBQkaoTJ9tTzKM+//DTqf7LbX90g/On2JFjAZzWgXKGG1aPUrmGITVc9e+GZXZLIKL2c2VtQ/VYZH2X05Hzz2O7vOuPavhLAWmsblukNc2JAuHznO5vBoeYgaQ8OlN/lf87sjWkXRvtDKy2wktG8xM9PgjfAaUpvUu7yPonfHMiBmXsi2+e/aQ1F+bIZpRX+Dq9ZnkyWUvHnYZBQVQh42feDhcB/2P19QiZ/yXil+Vy1X8W2msX+SqztC76UspyyxltCzlBIuzq5howyP781faDIBjdZysTp3fHj3/u31SEcbc/ZFDTtgeYQs/GjlJBIfrZ1EuiX7mJXUhpWg+d2S6sACSHBGAh0mrd+Lpz X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-01.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(376002)(346002)(396003)(136003)(39830400003)(36840700001)(46966006)(336012)(36906005)(103116003)(426003)(8936002)(44832011)(107886003)(8676002)(82310400003)(186003)(81166007)(7126003)(316002)(86362001)(2906002)(26005)(4326008)(478600001)(83380400001)(47076005)(5660300002)(36860700001)(55016002)(356005)(70206006)(70586007)(7696005)(1076003)(26583001)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 13:28:37.5606 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ae925640-62b0-41bf-c6f7-08d976ba64c0 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-01.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT033.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR01MB6261 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mike Marciniszyn The txq has the backpointer, so this is a micro optimization for the tx path. Fixes: d99dc602e2a5 ("IB/hfi1: Add functions to transmit datagram ipoib packets") Reviewed-by: Dennis Dalessandro Signed-off-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/ipoib_tx.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/ipoib_tx.c b/drivers/infiniband/hw/hfi1/ipoib_tx.c index 053eb43..734b91d 100644 --- a/drivers/infiniband/hw/hfi1/ipoib_tx.c +++ b/drivers/infiniband/hw/hfi1/ipoib_tx.c @@ -133,7 +133,7 @@ static void hfi1_ipoib_check_queue_stopped(struct hfi1_ipoib_txq *txq) static void hfi1_ipoib_free_tx(struct ipoib_txreq *tx, int budget) { - struct hfi1_ipoib_dev_priv *priv = tx->priv; + struct hfi1_ipoib_dev_priv *priv = tx->txq->priv; if (likely(!tx->sdma_status)) { dev_sw_netstats_tx_add(priv->netdev, 1, tx->skb->len); @@ -273,7 +273,7 @@ static int hfi1_ipoib_build_tx_desc(struct ipoib_txreq *tx, static void hfi1_ipoib_build_ib_tx_headers(struct ipoib_txreq *tx, struct ipoib_txparms *txp) { - struct hfi1_ipoib_dev_priv *priv = tx->priv; + struct hfi1_ipoib_dev_priv *priv = tx->txq->priv; struct hfi1_sdma_header *sdma_hdr = &tx->sdma_hdr; struct sk_buff *skb = tx->skb; struct hfi1_pportdata *ppd = ppd_from_ibp(txp->ibp); @@ -383,7 +383,6 @@ static struct ipoib_txreq *hfi1_ipoib_send_dma_common(struct net_device *dev, /* so that we can test if the sdma descriptors are there */ tx->txreq.num_desc = 0; - tx->priv = priv; tx->txq = txq; tx->skb = skb; INIT_LIST_HEAD(&tx->txreq.list); @@ -491,7 +490,7 @@ static int hfi1_ipoib_send_dma_single(struct net_device *dev, ret = hfi1_ipoib_submit_tx(txq, tx); if (likely(!ret)) { tx_ok: - trace_sdma_output_ibhdr(tx->priv->dd, + trace_sdma_output_ibhdr(txq->priv->dd, &tx->sdma_hdr.hdr, ib_is_sc5(txp->flow.sc5)); hfi1_ipoib_check_queue_depth(txq); @@ -554,7 +553,7 @@ static int hfi1_ipoib_send_dma_list(struct net_device *dev, hfi1_ipoib_check_queue_depth(txq); - trace_sdma_output_ibhdr(tx->priv->dd, + trace_sdma_output_ibhdr(txq->priv->dd, &tx->sdma_hdr.hdr, ib_is_sc5(txp->flow.sc5)); From patchwork Mon Sep 13 13:28:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 12489387 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9F7CC433F5 for ; Mon, 13 Sep 2021 13:32:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4078610F7 for ; Mon, 13 Sep 2021 13:32:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241928AbhIMNdb (ORCPT ); Mon, 13 Sep 2021 09:33:31 -0400 Received: from mail-dm6nam12on2109.outbound.protection.outlook.com ([40.107.243.109]:7905 "EHLO NAM12-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S242750AbhIMNaA (ORCPT ); Mon, 13 Sep 2021 09:30:00 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lW2kVev2I3ixFCkkpeRF010lofVC3fpIdu8LIeqGRgxwhVcaL+4VAP0GVumLj7CbPSWSxdOFhjldysTDXTAcgvOVU1/snlUuoY9PNx6vQh1n9bHiMOY7hhZDC42fTrlmmLACQdXuhPqcC+j4kCk7DJ3oD5VM0oDLKiOkykD6Jfu+1LS33AXGU3ya1pK8q72DEMo1YLTCpOlKt18gvj+CvK0BD0TU60JqDlFURiCxc3/iOEhFg9ntmPCXV3n6DIqLI4NKfDNkOU/dm753gbj18a1OMMi6e5bsWdsY8/oveMMf5fFcxkHFNNPmuTkloE6nom/yfZ8FP6Qq7uPwXqFt4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Gywo0aLQRFh+oZ0t4VwUFyfLN/KzV7ZNJuTCFiaomao=; b=lQyHsldpTk70Oqx7Vk1PuvILmDE3/2boci4MNvXTcc9nZQTkgsuPsy+5klrp4vEAcBCzlJDztHQtPfBDfqh2DdLAnMPvoK9Z3/qhoGPCV4qWzOluKTihPd+2RWIXlKFefrRkvDoaCgi0ddVzWfEZI4y5ZZ35cF4OjtNIS8YNlBHCR8AOtxkDzSdCCnuo1p/VBrF4mFOwKPPyGP0xrGZv0hZuCxFFNKokldfiyOKF1IiWH0dAG4PUgXr6LA8MPLu0T8/+UHr9LeG8iX2cELnOufUVTEazSpLfBLf1Ru5eBcMGp2C0WjMJgQfDXRHtEj86KlkQ0Z9XIOR8m+e6a+ZgVw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=cornelisnetworks.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Gywo0aLQRFh+oZ0t4VwUFyfLN/KzV7ZNJuTCFiaomao=; b=jDc/tBuy3i11x2cctm8asQao6ucf5G4z3Fp/5JdLpofiSkUQlZrvcZe+s7CCmn65i7bKgLXOLWUFJYWsKfsIVhox2Ran+rs9y4A4TAywq5cw4di9sVSAdRHGuaT7HW5X2hufsWmd44WhWVczJyavNGEG3ud+iodWG0845r+T66IauMFWPtz3CNVICFJwJye1YNj2Ig74G0Y5jYTiPhYXkKjZQIQzM08w4I4Yc31G0iPd3bB+FK2k6SvLzF9qQrpSaMWJeZY89NnSkgw9Mb87/TAls5V01+G/qwK5i6SLr6fzefUysP93zpDAWD+ozh1TS3ME4lQHOwBg0py1xjX1Gw== Received: from BN9P221CA0013.NAMP221.PROD.OUTLOOK.COM (2603:10b6:408:10a::6) by SA1PR01MB6797.prod.exchangelabs.com (2603:10b6:806:1a6::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Mon, 13 Sep 2021 13:28:42 +0000 Received: from BN8NAM11FT068.eop-nam11.prod.protection.outlook.com (2603:10b6:408:10a:cafe::8b) by BN9P221CA0013.outlook.office365.com (2603:10b6:408:10a::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 13:28:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; cornelisnetworks.com; dkim=none (message not signed) header.d=none;cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-01.cornelisnetworks.com; Received: from awfm-01.cornelisnetworks.com (208.255.156.42) by BN8NAM11FT068.mail.protection.outlook.com (10.13.177.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 13:28:42 +0000 Received: from awfm-01.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-01.cornelisnetworks.com (8.14.7/8.14.7) with ESMTP id 18DDSg6R147684; Mon, 13 Sep 2021 09:28:42 -0400 Subject: [PATCH for-next 4/6] IB/hfi1: Tune netdev xmit cachelines From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Mike Marciniszyn Date: Mon, 13 Sep 2021 09:28:42 -0400 Message-ID: <20210913132842.131370.15636.stgit@awfm-01.cornelisnetworks.com> In-Reply-To: <20210913132317.131370.54825.stgit@awfm-01.cornelisnetworks.com> References: <20210913132317.131370.54825.stgit@awfm-01.cornelisnetworks.com> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6c9e75ab-3098-4d25-7926-08d976ba67da X-MS-TrafficTypeDiagnostic: SA1PR01MB6797: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: G+MngaAN7yf9Au+bZOXhog0PyatD9NACNaucKMftnK2y2YWVc8eg2kyJkk5VEWA9jV9KmDg/jm9TUO5+4p2rY6e+QUef9NXm73EYC0S0eWIk/eaymlvbTZhx3zmp+n0vXKEpCAzKXhEFbfHzmRshZGAiCJIgZJ5gwRKY4KESCz8Ofmzf0B81PlRT5w2Eq+zY8tLih1VrObzJve+UfqCnIYaTvX3ORuOt6XoGgQupTuJgpNMYRdtCOHilf1zxjZEILuZppuMgNwhHzQXzFERWQaq4NTIUBhDIYStAWdrHSzFJs+Lly2SjQOF8kWCmgs7nmusriFnaFqzkwAHiTuLTsTwYvA4xR3rBQblHuGtOIw0IPO7G0418bkvXxBczxUFfg2KNDDiTRa1aAwQDuspuQo3zmUwQcUmrMws6PfhVQzS/Pi0hw1konU7OkuFjJDMacPsoyGUonvjW5mD0uLXnHS1IKAO3jcOAqI0XbDIAFlIzkh4j3Ul2ht7GmCZPKzqg8N25G70RThhE8/rsZEqVV3L0GARDSDj2CPBoI++mN/sqG/gx/l5MN09xWT9nwqqLh/wX7r9t3XGAGvFLKQ2MmmtFYltELdkKKpubbU5PRoLN0ZgYFtgJwYwYdH4dpEEN+ahwEIaAaky2K14r0aM/nx+luHdAonoQsMuI1IK2iSWwDp5dpNbYOfovAfd5lWfIUpeDzFSqlCKBvNmohYvvcJ7YCnPKg5lCnTV4dxBRAVo= X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-01.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(39830400003)(136003)(396003)(346002)(376002)(46966006)(36840700001)(7696005)(2906002)(36860700001)(70586007)(44832011)(70206006)(86362001)(1076003)(426003)(336012)(107886003)(55016002)(103116003)(186003)(26005)(8676002)(30864003)(82310400003)(5660300002)(316002)(4326008)(36906005)(478600001)(8936002)(356005)(81166007)(47076005)(7126003)(83380400001)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 13:28:42.7611 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6c9e75ab-3098-4d25-7926-08d976ba67da X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-01.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT068.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR01MB6797 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mike Marciniszyn This patch moves fields in the ring and creates a line for the producer and the consumer. The adds a consumer side variable that tracks the ring avail so that the code doesn't have the read the other cacheline to get a count for every packet. A read now only occurs when the avail is at 0. Fixes: d99dc602e2a5 ("IB/hfi1: Add functions to transmit datagram ipoib packets") Reviewed-by: Dennis Dalessandro Signed-off-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/ipoib.h | 39 ++++++++++-------- drivers/infiniband/hw/hfi1/ipoib_tx.c | 71 +++++++++++++++++++-------------- drivers/infiniband/hw/hfi1/trace_tx.h | 8 ++-- 3 files changed, 66 insertions(+), 52 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/ipoib.h b/drivers/infiniband/hw/hfi1/ipoib.h index 71b102d..8d9a03a 100644 --- a/drivers/infiniband/hw/hfi1/ipoib.h +++ b/drivers/infiniband/hw/hfi1/ipoib.h @@ -46,18 +46,31 @@ /** * struct hfi1_ipoib_circ_buf - List of items to be processed * @items: ring of items each a power of two size - * @head: ring head - * @tail: ring tail * @max_items: max items + 1 that the ring can contain * @shift: log2 of size for getting txreq + * @sent_txreqs: count of txreqs posted to sdma + * @tail: ring tail + * @stops: count of stops of queue + * @ring_full: ring has been filled + * @no_desc: descriptor shortage seen + * @complete_txreqs: count of txreqs completed by sdma + * @head: ring head */ struct ipoib_txreq; struct hfi1_ipoib_circ_buf { void *items; - u32 head; - u32 tail; u32 max_items; u32 shift; + /* consumer cache line */ + u64 ____cacheline_aligned_in_smp sent_txreqs; + u32 avail; + u32 tail; + atomic_t stops; + atomic_t ring_full; + atomic_t no_desc; + /* producer cache line */ + atomic64_t ____cacheline_aligned_in_smp complete_txreqs; + u32 head; }; /** @@ -66,14 +79,10 @@ struct hfi1_ipoib_circ_buf { * @sde: sdma engine * @tx_list: tx request list * @sent_txreqs: count of txreqs posted to sdma - * @stops: count of stops of queue - * @ring_full: ring has been filled - * @no_desc: descriptor shortage seen * @flow: tracks when list needs to be flushed for a flow change * @q_idx: ipoib Tx queue index * @pkts_sent: indicator packets have been sent from this queue * @wait: iowait structure - * @complete_txreqs: count of txreqs completed by sdma * @napi: pointer to tx napi interface * @tx_ring: ring of ipoib txreqs to be reaped by napi callback */ @@ -82,17 +91,12 @@ struct hfi1_ipoib_txq { struct hfi1_ipoib_dev_priv *priv; struct sdma_engine *sde; struct list_head tx_list; - u64 sent_txreqs; - atomic_t stops; - atomic_t ring_full; - atomic_t no_desc; union hfi1_ipoib_flow flow; u8 q_idx; bool pkts_sent; struct iowait wait; - atomic64_t ____cacheline_aligned_in_smp complete_txreqs; - struct hfi1_ipoib_circ_buf tx_ring; + struct hfi1_ipoib_circ_buf ____cacheline_aligned_in_smp tx_ring; }; struct hfi1_ipoib_dev_priv { @@ -100,13 +104,12 @@ struct hfi1_ipoib_dev_priv { struct net_device *netdev; struct ib_device *device; struct hfi1_ipoib_txq *txqs; + const struct net_device_ops *netdev_ops; + struct rvt_qp *qp; + u32 qkey; u16 pkey; u16 pkey_index; - u32 qkey; u8 port_num; - - const struct net_device_ops *netdev_ops; - struct rvt_qp *qp; }; /* hfi1 ipoib rdma netdev's private data structure */ diff --git a/drivers/infiniband/hw/hfi1/ipoib_tx.c b/drivers/infiniband/hw/hfi1/ipoib_tx.c index 734b91d..c3e43da 100644 --- a/drivers/infiniband/hw/hfi1/ipoib_tx.c +++ b/drivers/infiniband/hw/hfi1/ipoib_tx.c @@ -66,21 +66,21 @@ static u32 hfi1_ipoib_txreqs(const u64 sent, const u64 completed) static u64 hfi1_ipoib_used(struct hfi1_ipoib_txq *txq) { - return hfi1_ipoib_txreqs(txq->sent_txreqs, - atomic64_read(&txq->complete_txreqs)); + return hfi1_ipoib_txreqs(txq->tx_ring.sent_txreqs, + atomic64_read(&txq->tx_ring.complete_txreqs)); } static void hfi1_ipoib_stop_txq(struct hfi1_ipoib_txq *txq) { trace_hfi1_txq_stop(txq); - if (atomic_inc_return(&txq->stops) == 1) + if (atomic_inc_return(&txq->tx_ring.stops) == 1) netif_stop_subqueue(txq->priv->netdev, txq->q_idx); } static void hfi1_ipoib_wake_txq(struct hfi1_ipoib_txq *txq) { trace_hfi1_txq_wake(txq); - if (atomic_dec_and_test(&txq->stops)) + if (atomic_dec_and_test(&txq->tx_ring.stops)) netif_wake_subqueue(txq->priv->netdev, txq->q_idx); } @@ -98,9 +98,9 @@ static uint hfi1_ipoib_ring_lwat(struct hfi1_ipoib_txq *txq) static void hfi1_ipoib_check_queue_depth(struct hfi1_ipoib_txq *txq) { - ++txq->sent_txreqs; + ++txq->tx_ring.sent_txreqs; if (hfi1_ipoib_used(txq) >= hfi1_ipoib_ring_hwat(txq) && - !atomic_xchg(&txq->ring_full, 1)) { + !atomic_xchg(&txq->tx_ring.ring_full, 1)) { trace_hfi1_txq_full(txq); hfi1_ipoib_stop_txq(txq); } @@ -125,7 +125,7 @@ static void hfi1_ipoib_check_queue_stopped(struct hfi1_ipoib_txq *txq) * to protect against ring overflow. */ if (hfi1_ipoib_used(txq) < hfi1_ipoib_ring_lwat(txq) && - atomic_xchg(&txq->ring_full, 0)) { + atomic_xchg(&txq->tx_ring.ring_full, 0)) { trace_hfi1_txq_xmit_unstopped(txq); hfi1_ipoib_wake_txq(txq); } @@ -168,6 +168,7 @@ static void hfi1_ipoib_drain_tx_ring(struct hfi1_ipoib_txq *txq) tx_ring->tail = 0; atomic64_set(&txq->complete_txreqs, 0); txq->sent_txreqs = 0; + tx_ring->avail = hfi1_ipoib_ring_hwat(txq); } static int hfi1_ipoib_poll_tx_ring(struct napi_struct *napi, int budget) @@ -190,7 +191,7 @@ static int hfi1_ipoib_poll_tx_ring(struct napi_struct *napi, int budget) head = CIRC_NEXT(head, max_tx); tx = hfi1_txreq_from_idx(tx_ring, head); } - atomic64_add(work_done, &txq->complete_txreqs); + atomic64_add(work_done, &txq->tx_ring.complete_txreqs); /* Finished freeing tx items so store the head value. */ smp_store_release(&tx_ring->head, head); @@ -344,7 +345,7 @@ static void hfi1_ipoib_build_ib_tx_headers(struct ipoib_txreq *tx, ohdr->bth[0] = cpu_to_be32(bth0); ohdr->bth[1] = cpu_to_be32(txp->dqpn); - ohdr->bth[2] = cpu_to_be32(mask_psn((u32)txp->txq->sent_txreqs)); + ohdr->bth[2] = cpu_to_be32(mask_psn((u32)txp->txq->tx_ring.sent_txreqs)); /* Build the deth */ ohdr->u.ud.deth[0] = cpu_to_be32(priv->qkey); @@ -369,16 +370,25 @@ static struct ipoib_txreq *hfi1_ipoib_send_dma_common(struct net_device *dev, struct hfi1_ipoib_dev_priv *priv = hfi1_ipoib_priv(dev); struct hfi1_ipoib_txq *txq = txp->txq; struct ipoib_txreq *tx; - struct hfi1_ipoib_circ_buf *tx_ring; - u32 tail; + struct hfi1_ipoib_circ_buf *tx_ring = &txq->tx_ring; + u32 tail = tx_ring->tail; int ret; - if (unlikely(hfi1_ipoib_used(txq) >= hfi1_ipoib_ring_hwat(txq))) - /* This shouldn't happen with a stopped queue */ - return ERR_PTR(-ENOMEM); - tx_ring = &txq->tx_ring; - tail = tx_ring->tail; - tx = hfi1_txreq_from_idx(tx_ring, tx_ring->tail); + if (unlikely(!tx_ring->avail)) { + u32 head; + + if (hfi1_ipoib_used(txq) >= hfi1_ipoib_ring_hwat(txq)) + /* This shouldn't happen with a stopped queue */ + return ERR_PTR(-ENOMEM); + /* See hfi1_ipoib_poll_tx_ring() */ + head = smp_load_acquire(&tx_ring->head); + tx_ring->avail = + min_t(u32, hfi1_ipoib_ring_hwat(txq), + CIRC_CNT(head, tail, tx_ring->max_items)); + } else { + tx_ring->avail--; + } + tx = hfi1_txreq_from_idx(tx_ring, tail); trace_hfi1_txq_alloc_tx(txq); /* so that we can test if the sdma descriptors are there */ @@ -639,7 +649,7 @@ static int hfi1_ipoib_sdma_sleep(struct sdma_engine *sde, if (list_empty(&txq->wait.list)) { struct hfi1_ibport *ibp = &sde->ppd->ibport_data; - if (!atomic_xchg(&txq->no_desc, 1)) { + if (!atomic_xchg(&txq->tx_ring.no_desc, 1)) { trace_hfi1_txq_queued(txq); hfi1_ipoib_stop_txq(txq); } @@ -682,7 +692,7 @@ static void hfi1_ipoib_flush_txq(struct work_struct *work) if (likely(dev->reg_state == NETREG_REGISTERED) && likely(!hfi1_ipoib_flush_tx_list(dev, txq))) - if (atomic_xchg(&txq->no_desc, 0)) + if (atomic_xchg(&txq->tx_ring.no_desc, 0)) hfi1_ipoib_wake_txq(txq); } @@ -720,10 +730,10 @@ int hfi1_ipoib_txreq_init(struct hfi1_ipoib_dev_priv *priv) txq->priv = priv; txq->sde = NULL; INIT_LIST_HEAD(&txq->tx_list); - atomic64_set(&txq->complete_txreqs, 0); - atomic_set(&txq->stops, 0); - atomic_set(&txq->ring_full, 0); - atomic_set(&txq->no_desc, 0); + atomic64_set(&txq->tx_ring.complete_txreqs, 0); + atomic_set(&txq->tx_ring.stops, 0); + atomic_set(&txq->tx_ring.ring_full, 0); + atomic_set(&txq->tx_ring.no_desc, 0); txq->q_idx = i; txq->flow.tx_queue = 0xff; txq->flow.sc5 = 0xff; @@ -740,6 +750,7 @@ int hfi1_ipoib_txreq_init(struct hfi1_ipoib_dev_priv *priv) txq->tx_ring.max_items = tx_ring_size; txq->tx_ring.shift = ilog2(tx_ring_size); + txq->tx_ring.avail = hfi1_ipoib_ring_hwat(txq); netif_tx_napi_add(dev, &txq->napi, hfi1_ipoib_poll_tx_ring, @@ -765,7 +776,7 @@ static void hfi1_ipoib_drain_tx_list(struct hfi1_ipoib_txq *txq) { struct sdma_txreq *txreq; struct sdma_txreq *txreq_tmp; - atomic64_t *complete_txreqs = &txq->complete_txreqs; + atomic64_t *complete_txreqs = &txq->tx_ring.complete_txreqs; list_for_each_entry_safe(txreq, txreq_tmp, &txq->tx_list, list) { struct ipoib_txreq *tx = @@ -782,7 +793,7 @@ static void hfi1_ipoib_drain_tx_list(struct hfi1_ipoib_txq *txq) dd_dev_warn(txq->priv->dd, "txq %d not empty found %u requests\n", txq->q_idx, - hfi1_ipoib_txreqs(txq->sent_txreqs, + hfi1_ipoib_txreqs(txq->tx_ring.sent_txreqs, atomic64_read(complete_txreqs))); } @@ -834,20 +845,20 @@ void hfi1_ipoib_tx_timeout(struct net_device *dev, unsigned int q) { struct hfi1_ipoib_dev_priv *priv = hfi1_ipoib_priv(dev); struct hfi1_ipoib_txq *txq = &priv->txqs[q]; - u64 completed = atomic64_read(&txq->complete_txreqs); + u64 completed = atomic64_read(&txq->tx_ring.complete_txreqs); dd_dev_info(priv->dd, "timeout txq %llx q %u stopped %u stops %d no_desc %d ring_full %d\n", (unsigned long long)txq, q, __netif_subqueue_stopped(dev, txq->q_idx), - atomic_read(&txq->stops), - atomic_read(&txq->no_desc), - atomic_read(&txq->ring_full)); + atomic_read(&txq->tx_ring.stops), + atomic_read(&txq->tx_ring.no_desc), + atomic_read(&txq->tx_ring.ring_full)); dd_dev_info(priv->dd, "sde %llx engine %u\n", (unsigned long long)txq->sde, txq->sde ? txq->sde->this_idx : 0); dd_dev_info(priv->dd, "flow %x\n", txq->flow.as_int); dd_dev_info(priv->dd, "sent %llu completed %llu used %llu\n", - txq->sent_txreqs, completed, hfi1_ipoib_used(txq)); + txq->tx_ring.sent_txreqs, completed, hfi1_ipoib_used(txq)); dd_dev_info(priv->dd, "tx_queue_len %u max_items %u\n", dev->tx_queue_len, txq->tx_ring.max_items); dd_dev_info(priv->dd, "head %u tail %u\n", diff --git a/drivers/infiniband/hw/hfi1/trace_tx.h b/drivers/infiniband/hw/hfi1/trace_tx.h index 7318aa6..c9b1cd0 100644 --- a/drivers/infiniband/hw/hfi1/trace_tx.h +++ b/drivers/infiniband/hw/hfi1/trace_tx.h @@ -917,11 +917,11 @@ __entry->tail = txq->tx_ring.tail; __entry->idx = txq->q_idx; __entry->used = - txq->sent_txreqs - - atomic64_read(&txq->complete_txreqs); + txq->tx_ring.sent_txreqs - + atomic64_read(&txq->tx_ring.complete_txreqs); __entry->flow = txq->flow.as_int; - __entry->stops = atomic_read(&txq->stops); - __entry->no_desc = atomic_read(&txq->no_desc); + __entry->stops = atomic_read(&txq->tx_ring.stops); + __entry->no_desc = atomic_read(&txq->tx_ring.no_desc); __entry->stopped = __netif_subqueue_stopped(txq->priv->netdev, txq->q_idx); ), From patchwork Mon Sep 13 13:28:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 12489383 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6536BC433EF for ; Mon, 13 Sep 2021 13:32:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5333E61106 for ; Mon, 13 Sep 2021 13:32:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239037AbhIMNdR (ORCPT ); Mon, 13 Sep 2021 09:33:17 -0400 Received: from mail-mw2nam10on2120.outbound.protection.outlook.com ([40.107.94.120]:31776 "EHLO NAM10-MW2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S242819AbhIMNaG (ORCPT ); Mon, 13 Sep 2021 09:30:06 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QdbvnEwHF5X+5c/92z0zhRAJd3vgC0cS35jS3eZ/PKMje35BRTu7daPZiiShWRD/fHL5FFUU6c1jHgwnvmiYdqxH8+D1EgUTnCy6YOLbvJbyhbL42fxbyaNuPfXyqfHpEft4dIE58ItJhiU0RJS5yuFonJSegLXMeB9jBffql8SIqEpB6a/LuDQSa7+GbdxddcGzvyNuixe2WHoQvib4sNDQO+iqArNBqBLg6Lx7+wxlpHS3IK+q6J0CjZrZLRGQvnBH3Jz+WjhtCh+YwPvZM+e7RfbXW1eD132vur3VHeghuKOKBU+Wf8/bBeQJduVQzrvJR2xRYTOqTIfw1EI4/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=NoO4+i9amwtGx1SOyAciQ+Xvn/ADiv2xAncIe73D52U=; b=nTY9cMPbruuqAP8trMd4klYsbT+Z/yZzS+T0GcmaXXNzm7LNhqfUcyrSbbC4ZMh4kZx+4XGwiamq1OTX6Z87NqtMSnYgz4H6pCok/QxvyjQaAoJZqROJ55+Xt5i8NeUMAG65uUFg6woDD/xRnm2/rfsusiUs/qLYZvuo5yWgCNESmYyamwxU5aCF0K8D1MeIoo2P2+wKcw7gR+yZW1fKp/lgu3QuJ03AYp7CijkLOf1OmRRASzPdwpQFnEi7PJMFL+5KFdEF4yR3QCzSOUgIeR4+3VEAMUbLwDQC13wLes8xKXXTEjjo9KxE5bLmBwFMl9B3E1Mb6dJ4Is+DGHzziQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=cornelisnetworks.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NoO4+i9amwtGx1SOyAciQ+Xvn/ADiv2xAncIe73D52U=; b=OooZa/+IHpD2lGmQopRzbrgwDr9LUNj2Iv7+lHbRkgF4KDykHjkYw8Lfj+c0GqL/JvuhW3wrI3gecmVaJlNn4C6XLcIetf834B8B3IfrP0Cuv35IxhntpYxEdP1A4v4u4r57k8IciSOgRk4rBPpR88Y6VDFPPgpdc9+qBIzs2QdQYyoYMIEqx58J8DCIfPFHEGKXxeq+9OTVn9/DFzlHA39fsKoraHgQmKb+gxTEzMuWPqgGZDAFT/+3mg1MVr1vEQsDXUl2FrSEBPjxb+bDVbYh0EZODr2ZpZqW/RWHYHbMTdYH8Vxf/j0IfoX78RU0vOa77dZxDQoAj5Bp/6HJew== Received: from BN6PR13CA0044.namprd13.prod.outlook.com (2603:10b6:404:13e::30) by SN6PR01MB4669.prod.exchangelabs.com (2603:10b6:805:da::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep 2021 13:28:48 +0000 Received: from BN8NAM11FT025.eop-nam11.prod.protection.outlook.com (2603:10b6:404:13e:cafe::b) by BN6PR13CA0044.outlook.office365.com (2603:10b6:404:13e::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.8 via Frontend Transport; Mon, 13 Sep 2021 13:28:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; cornelisnetworks.com; dkim=none (message not signed) header.d=none;cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-01.cornelisnetworks.com; Received: from awfm-01.cornelisnetworks.com (208.255.156.42) by BN8NAM11FT025.mail.protection.outlook.com (10.13.177.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 13:28:47 +0000 Received: from awfm-01.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-01.cornelisnetworks.com (8.14.7/8.14.7) with ESMTP id 18DDSlEa147959; Mon, 13 Sep 2021 09:28:47 -0400 Subject: [PATCH for-next 5/6] IB/hfi1: Remove atomic completion count From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Mike Marciniszyn Date: Mon, 13 Sep 2021 09:28:47 -0400 Message-ID: <20210913132847.131370.54250.stgit@awfm-01.cornelisnetworks.com> In-Reply-To: <20210913132317.131370.54825.stgit@awfm-01.cornelisnetworks.com> References: <20210913132317.131370.54825.stgit@awfm-01.cornelisnetworks.com> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a699d84f-a4a8-46ae-adaf-08d976ba6af8 X-MS-TrafficTypeDiagnostic: SN6PR01MB4669: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2399; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mqMkcKzN3YRAHqnzXpf8hJ2EfE4P/EqR+VQ7Y2lxP6f8T1dirvv0RNqiSeVcbM0iWHFR6jPgge0qu53ygjobhVfFTOzFUHCZdxUT24l0VuvnTFE3hs+HgJJ3nxeBSqiYNaZnXS+JlqrGF3Gm75rkwt0XXIkUCiwDOV9iHEsSvf+VwsNNPP+qM0iGDS+pccY+LdC/igql1W9zwlqWAmcWJ34cRX7BvrwL2wP5mRxraL4YmrVfoPrfRLJ4U+Mfh5hpLyfCe5aSqZ0QFTu611VSXvMvi7gKONbZGXh8uIPDIqMj9438EpXCU7r58O3RkuG7zZDZdoW7b5YCyk0HvUfw9WdsaD0loejXN9Qwqh+P91DIEdAAoJMrlGRa4I4CCClykDDHHp2kVyBfgyv/wj4784bpjLNg6iCrsw6v7cE2jWKWNlhUcxRLOSDgopECqPCiY7COOYEDmfpfhcCbPKcwGie+haugAA12CUgivc29EwTze+l2xeewZhpLdYABwr92XIvcIw39Uc2hD5xUkcS16vZV9B6OKuO+aj/ixNu6W0AvuADtZ0kijXFyKlyua1CuRXpAx4v4SdYSgkUsv5bWe320do1Yccm70+/gfA6vuTIBtjbut02wGIfmdqsA4b/m3FMbIc82wUwtPB7Xr0vh92pb+AUHU0mGpIwW385fNMoHT9JolBSRU+NRYplkUcVb8uUBH1jshIXL8J/Ti4RZ0g3ow9KzFNWhdKqIJSFtNpM= X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-01.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(396003)(136003)(39830400003)(346002)(376002)(36840700001)(46966006)(7126003)(4326008)(5660300002)(47076005)(86362001)(82310400003)(478600001)(1076003)(44832011)(107886003)(70206006)(8676002)(70586007)(2906002)(55016002)(316002)(36906005)(36860700001)(356005)(7696005)(336012)(81166007)(426003)(83380400001)(8936002)(103116003)(26005)(186003)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 13:28:47.9912 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a699d84f-a4a8-46ae-adaf-08d976ba6af8 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-01.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT025.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR01MB4669 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mike Marciniszyn The atomic is not needed. Fixes: d99dc602e2a5 ("IB/hfi1: Add functions to transmit datagram ipoib packets") Reviewed-by: Dennis Dalessandro Signed-off-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/ipoib.h | 2 +- drivers/infiniband/hw/hfi1/ipoib_tx.c | 18 ++++++++---------- drivers/infiniband/hw/hfi1/trace_tx.h | 2 +- 3 files changed, 10 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/ipoib.h b/drivers/infiniband/hw/hfi1/ipoib.h index 8d9a03a..eb5c251 100644 --- a/drivers/infiniband/hw/hfi1/ipoib.h +++ b/drivers/infiniband/hw/hfi1/ipoib.h @@ -69,7 +69,7 @@ struct hfi1_ipoib_circ_buf { atomic_t ring_full; atomic_t no_desc; /* producer cache line */ - atomic64_t ____cacheline_aligned_in_smp complete_txreqs; + u64 ____cacheline_aligned_in_smp complete_txreqs; u32 head; }; diff --git a/drivers/infiniband/hw/hfi1/ipoib_tx.c b/drivers/infiniband/hw/hfi1/ipoib_tx.c index c3e43da..1a7a837 100644 --- a/drivers/infiniband/hw/hfi1/ipoib_tx.c +++ b/drivers/infiniband/hw/hfi1/ipoib_tx.c @@ -67,7 +67,7 @@ static u32 hfi1_ipoib_txreqs(const u64 sent, const u64 completed) static u64 hfi1_ipoib_used(struct hfi1_ipoib_txq *txq) { return hfi1_ipoib_txreqs(txq->tx_ring.sent_txreqs, - atomic64_read(&txq->tx_ring.complete_txreqs)); + txq->tx_ring.complete_txreqs); } static void hfi1_ipoib_stop_txq(struct hfi1_ipoib_txq *txq) @@ -166,8 +166,8 @@ static void hfi1_ipoib_drain_tx_ring(struct hfi1_ipoib_txq *txq) } tx_ring->head = 0; tx_ring->tail = 0; - atomic64_set(&txq->complete_txreqs, 0); - txq->sent_txreqs = 0; + tx_ring->complete_txreqs = 0; + tx_ring->sent_txreqs = 0; tx_ring->avail = hfi1_ipoib_ring_hwat(txq); } @@ -191,7 +191,7 @@ static int hfi1_ipoib_poll_tx_ring(struct napi_struct *napi, int budget) head = CIRC_NEXT(head, max_tx); tx = hfi1_txreq_from_idx(tx_ring, head); } - atomic64_add(work_done, &txq->tx_ring.complete_txreqs); + tx_ring->complete_txreqs += work_done; /* Finished freeing tx items so store the head value. */ smp_store_release(&tx_ring->head, head); @@ -730,7 +730,6 @@ int hfi1_ipoib_txreq_init(struct hfi1_ipoib_dev_priv *priv) txq->priv = priv; txq->sde = NULL; INIT_LIST_HEAD(&txq->tx_list); - atomic64_set(&txq->tx_ring.complete_txreqs, 0); atomic_set(&txq->tx_ring.stops, 0); atomic_set(&txq->tx_ring.ring_full, 0); atomic_set(&txq->tx_ring.no_desc, 0); @@ -776,7 +775,6 @@ static void hfi1_ipoib_drain_tx_list(struct hfi1_ipoib_txq *txq) { struct sdma_txreq *txreq; struct sdma_txreq *txreq_tmp; - atomic64_t *complete_txreqs = &txq->tx_ring.complete_txreqs; list_for_each_entry_safe(txreq, txreq_tmp, &txq->tx_list, list) { struct ipoib_txreq *tx = @@ -786,7 +784,7 @@ static void hfi1_ipoib_drain_tx_list(struct hfi1_ipoib_txq *txq) sdma_txclean(txq->priv->dd, &tx->txreq); dev_kfree_skb_any(tx->skb); tx->skb = NULL; - atomic64_inc(complete_txreqs); + txq->tx_ring.complete_txreqs++; } if (hfi1_ipoib_used(txq)) @@ -794,7 +792,7 @@ static void hfi1_ipoib_drain_tx_list(struct hfi1_ipoib_txq *txq) "txq %d not empty found %u requests\n", txq->q_idx, hfi1_ipoib_txreqs(txq->tx_ring.sent_txreqs, - atomic64_read(complete_txreqs))); + txq->tx_ring.complete_txreqs)); } void hfi1_ipoib_txreq_deinit(struct hfi1_ipoib_dev_priv *priv) @@ -845,7 +843,6 @@ void hfi1_ipoib_tx_timeout(struct net_device *dev, unsigned int q) { struct hfi1_ipoib_dev_priv *priv = hfi1_ipoib_priv(dev); struct hfi1_ipoib_txq *txq = &priv->txqs[q]; - u64 completed = atomic64_read(&txq->tx_ring.complete_txreqs); dd_dev_info(priv->dd, "timeout txq %llx q %u stopped %u stops %d no_desc %d ring_full %d\n", (unsigned long long)txq, q, @@ -858,7 +855,8 @@ void hfi1_ipoib_tx_timeout(struct net_device *dev, unsigned int q) txq->sde ? txq->sde->this_idx : 0); dd_dev_info(priv->dd, "flow %x\n", txq->flow.as_int); dd_dev_info(priv->dd, "sent %llu completed %llu used %llu\n", - txq->tx_ring.sent_txreqs, completed, hfi1_ipoib_used(txq)); + txq->tx_ring.sent_txreqs, txq->tx_ring.complete_txreqs, + hfi1_ipoib_used(txq)); dd_dev_info(priv->dd, "tx_queue_len %u max_items %u\n", dev->tx_queue_len, txq->tx_ring.max_items); dd_dev_info(priv->dd, "head %u tail %u\n", diff --git a/drivers/infiniband/hw/hfi1/trace_tx.h b/drivers/infiniband/hw/hfi1/trace_tx.h index c9b1cd0..f00696f 100644 --- a/drivers/infiniband/hw/hfi1/trace_tx.h +++ b/drivers/infiniband/hw/hfi1/trace_tx.h @@ -918,7 +918,7 @@ __entry->idx = txq->q_idx; __entry->used = txq->tx_ring.sent_txreqs - - atomic64_read(&txq->tx_ring.complete_txreqs); + txq->tx_ring.complete_txreqs; __entry->flow = txq->flow.as_int; __entry->stops = atomic_read(&txq->tx_ring.stops); __entry->no_desc = atomic_read(&txq->tx_ring.no_desc); From patchwork Mon Sep 13 13:28:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 12489385 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73EB3C433EF for ; Mon, 13 Sep 2021 13:32:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5C47B61106 for ; Mon, 13 Sep 2021 13:32:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240196AbhIMNdS (ORCPT ); Mon, 13 Sep 2021 09:33:18 -0400 Received: from mail-bn8nam12on2124.outbound.protection.outlook.com ([40.107.237.124]:39182 "EHLO NAM12-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241571AbhIMNaL (ORCPT ); Mon, 13 Sep 2021 09:30:11 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AtOd1WFAsXkOvKdDg76BBUtEZTxkXDL3LbJTFqIwRRFZnW9KPARoVxjuL0zBRTiQ7m9hlo/3jORE9wH7I2jv8f/6UX414ERMGfufXf7NLhuRyth+8XswXQTRMoGVdWU43iVOHIR2YDqoNb27EDYechgysB5y0fVnQpK7IfeE5XHH/6EQtBGDjpYFZuFfLLwwtab+99oZyWmosDrJ9pUx1bxiRF0gcrYb28KOl/KzNn/GWxweVQaPKGwp+nPxAHLeiY4KhiJhd6YvIC0miyGscued8n4hcY0jcaLKJ0fA90ppnL4XHwF+ASYxkJ9osSgX9RGLZuMq3ekjk0pqhe3MrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=yeF8qyubKW7wuoh+cLRFOtKzzl4X7h3KTcNR2AlGHYE=; b=IRL8wLxs926HkzL2Y7C6AWUV0vnFjGER5Kz6XxQI7QOPx56K1hhnzFvt5YAmIyYQ6ZM9YtrIej+8SG1Dyo1Lvq7EIX8SvDvIV/Ymc2/10Jr1dx4YEAfnhcFqiuV5zb6kPdg5pGDabX9QolyM3F40O/w1b5Es/KaBoLuXVj5sU7YMZHtM3aGOZ3IGEjIk4blULg2Y7ZlYTWLIcblWXc76fT2Z9ojhLpD+OL7TZ/CqMg4pvtU+joQ4BX8yS5IYDidGhp4DQ4xlCpkejAIJtU62O0eWVAF412E6OlZIfeku1vXfBkcr03tx7//7K742COAimGvX4YTFYI/eG43N5WRsjA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=cornelisnetworks.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yeF8qyubKW7wuoh+cLRFOtKzzl4X7h3KTcNR2AlGHYE=; b=A1EHpRKYu9ezyyY4+de1Z3AaqQmSZcxzea/9pHkMctMTRFxmYw26IgIHILvLA83TH54BiTDFm4PZptuOuWRVVgRpmzn5ElYp3fUhvi+0BCGtfCmESY2NicgV1UH276iX4Be4v4J/rHWcLbmEjCSIns2MzfUiNfYtxrPwW3Ys62OsyosyrUp5F+ZoBMKxEjDRCrDNfX4RFPYgFI23mGXC7p1SQcAFylqTmI27nf9lu5nZxNnrYsf+08FksnKEo3TdVOZLyCtEWaynd2t/EBJGtABkBsD6hg7Jbc/v29pvx1+xK02gZUUFVMqP0qOFB4Qn3x4Nls4mwpHTmB4n/cr4Bw== Received: from BN9PR03CA0605.namprd03.prod.outlook.com (2603:10b6:408:106::10) by BL0PR0102MB3491.prod.exchangelabs.com (2603:10b6:207:38::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15; Mon, 13 Sep 2021 13:28:53 +0000 Received: from BN8NAM11FT026.eop-nam11.prod.protection.outlook.com (2603:10b6:408:106:cafe::44) by BN9PR03CA0605.outlook.office365.com (2603:10b6:408:106::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16 via Frontend Transport; Mon, 13 Sep 2021 13:28:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; cornelisnetworks.com; dkim=none (message not signed) header.d=none;cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-01.cornelisnetworks.com; Received: from awfm-01.cornelisnetworks.com (208.255.156.42) by BN8NAM11FT026.mail.protection.outlook.com (10.13.177.51) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 13:28:53 +0000 Received: from awfm-01.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-01.cornelisnetworks.com (8.14.7/8.14.7) with ESMTP id 18DDSq6U148226; Mon, 13 Sep 2021 09:28:52 -0400 Subject: [PATCH for-next 6/6] IB/hfi1: Add ring consumer and producers traces From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Mike Marciniszyn Date: Mon, 13 Sep 2021 09:28:52 -0400 Message-ID: <20210913132852.131370.9664.stgit@awfm-01.cornelisnetworks.com> In-Reply-To: <20210913132317.131370.54825.stgit@awfm-01.cornelisnetworks.com> References: <20210913132317.131370.54825.stgit@awfm-01.cornelisnetworks.com> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b2e6ec2b-5330-4ee7-16b9-08d976ba6e11 X-MS-TrafficTypeDiagnostic: BL0PR0102MB3491: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:403; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qRuVpsGpIZ+zfQf4XkePC9hbVAD4w8CGDB5iLPoYU6pWTBRcLyhKACzPumFYcZSEejGHzOmeTDFDnIsqgyLvhJAqCecd7h9toZC0GASTaVnPRI4MovgjftVd7nj08Z6lCnmdltMumYjQLe09RHTyqOaQfUgv8vn9R3skXa5d36+1zXShvfPLBHd4WCZXCkKg37K6/+fBQzQtyiVHEXxgxgOBslD2FT4S1LdsnBS0UZBTDS4rJb3vpiWZJg8u09LKcaVp5zjEh2KgqTP8auSZYVW6RMObnjWmDsIjVDmOVxNFZ9IYjtG66ObD5np72om9ubAxZxydrsqeMiHX3ZpbjbQachjhsIZxafMz+W1CTniV5beKwRrn1wQN46iyy4KjpXoUp+rJ3Ii8BWsIrHSiFuZHr/YoCzB3a1Bal0eYw1m33/7yhXEYWYV7IkmLzuzycX8yqIx1QI4Oguik2l8LevvT9+5n3IeN2N9iUc0SLaQQX4JbRoAhy7MGsPi7oWxTCciWfKUzttz2YfUgcDzygo7xxrvD7YeGOA5b5NeOzK9HrQ6i8XqUSSKna8xZFQq9KkbOlkLioASp4fMybqENEv/40tJlxh2gVArObAUWb+OT7Uq3YALdfb5z0UiLBlsajKg9F4sp0mGHkjokha9VjuZlo+kTkdIV/FMJfmbmGHtrjf76THpQaaKe0024bYsCnZXzlDbBcdJIC4JOkdjzMFnh3divMsxvyKFl+8RLwdw= X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-01.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(136003)(39830400003)(346002)(376002)(396003)(46966006)(36840700001)(86362001)(26005)(36860700001)(82310400003)(7696005)(426003)(1076003)(44832011)(186003)(336012)(478600001)(47076005)(4326008)(81166007)(356005)(36906005)(5660300002)(55016002)(83380400001)(316002)(7126003)(107886003)(70586007)(70206006)(103116003)(8676002)(8936002)(2906002)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 13:28:53.1864 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b2e6ec2b-5330-4ee7-16b9-08d976ba6e11 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-01.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT026.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR0102MB3491 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mike Marciniszyn These traces are used to debugging ring issues. The ipoib_txreq needed to be moved to a header file to allow access from the trace header file. The trace changes include: - new producer/consumer traces - new allocation deallocation traces - additional fidelity for SDMA engine prints Fixes: 4bd00b55c978 ("IB/hfi1: Add AIP tx traces") Signed-off-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/ipoib.h | 21 ++++++++++- drivers/infiniband/hw/hfi1/ipoib_tx.c | 23 ++---------- drivers/infiniband/hw/hfi1/trace_tx.h | 63 ++++++++++++++++++++++++++++++++- 3 files changed, 85 insertions(+), 22 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/ipoib.h b/drivers/infiniband/hw/hfi1/ipoib.h index eb5c251..9091229 100644 --- a/drivers/infiniband/hw/hfi1/ipoib.h +++ b/drivers/infiniband/hw/hfi1/ipoib.h @@ -44,6 +44,26 @@ }; /** + * struct ipoib_txreq - IPOIB transmit descriptor + * @txreq: sdma transmit request + * @sdma_hdr: 9b ib headers + * @sdma_status: status returned by sdma engine + * @complete: non-zero implies complete + * @priv: ipoib netdev private data + * @txq: txq on which skb was output + * @skb: skb to send + */ +struct ipoib_txreq { + struct sdma_txreq txreq; + struct hfi1_sdma_header sdma_hdr; + int sdma_status; + int complete; + struct hfi1_ipoib_dev_priv *priv; + struct hfi1_ipoib_txq *txq; + struct sk_buff *skb; +}; + +/** * struct hfi1_ipoib_circ_buf - List of items to be processed * @items: ring of items each a power of two size * @max_items: max items + 1 that the ring can contain @@ -56,7 +76,6 @@ * @complete_txreqs: count of txreqs completed by sdma * @head: ring head */ -struct ipoib_txreq; struct hfi1_ipoib_circ_buf { void *items; u32 max_items; diff --git a/drivers/infiniband/hw/hfi1/ipoib_tx.c b/drivers/infiniband/hw/hfi1/ipoib_tx.c index 1a7a837..d1c2cf5 100644 --- a/drivers/infiniband/hw/hfi1/ipoib_tx.c +++ b/drivers/infiniband/hw/hfi1/ipoib_tx.c @@ -22,26 +22,6 @@ #define CIRC_NEXT(val, size) CIRC_ADD(val, 1, size) #define CIRC_PREV(val, size) CIRC_ADD(val, -1, size) -/** - * struct ipoib_txreq - IPOIB transmit descriptor - * @txreq: sdma transmit request - * @sdma_hdr: 9b ib headers - * @sdma_status: status returned by sdma engine - * @complete: non-zero implies complete - * @priv: ipoib netdev private data - * @txq: txq on which skb was output - * @skb: skb to send - */ -struct ipoib_txreq { - struct sdma_txreq txreq; - struct hfi1_sdma_header sdma_hdr; - int sdma_status; - int complete; - struct hfi1_ipoib_dev_priv *priv; - struct hfi1_ipoib_txq *txq; - struct sk_buff *skb; -}; - struct ipoib_txparms { struct hfi1_devdata *dd; struct rdma_ah_attr *ah_attr; @@ -187,6 +167,7 @@ static int hfi1_ipoib_poll_tx_ring(struct napi_struct *napi, int budget) if (!smp_load_acquire(&tx->complete)) break; tx->complete = 0; + trace_hfi1_tx_produce(tx, head); hfi1_ipoib_free_tx(tx, budget); head = CIRC_NEXT(head, max_tx); tx = hfi1_txreq_from_idx(tx_ring, head); @@ -495,6 +476,7 @@ static int hfi1_ipoib_send_dma_single(struct net_device *dev, } tx_ring = &txq->tx_ring; + trace_hfi1_tx_consume(tx, tx_ring->tail); /* consume tx */ smp_store_release(&tx_ring->tail, CIRC_NEXT(tx_ring->tail, tx_ring->max_items)); ret = hfi1_ipoib_submit_tx(txq, tx); @@ -557,6 +539,7 @@ static int hfi1_ipoib_send_dma_list(struct net_device *dev, } tx_ring = &txq->tx_ring; + trace_hfi1_tx_consume(tx, tx_ring->tail); /* consume tx */ smp_store_release(&tx_ring->tail, CIRC_NEXT(tx_ring->tail, tx_ring->max_items)); list_add_tail(&tx->txreq.list, &txq->tx_list); diff --git a/drivers/infiniband/hw/hfi1/trace_tx.h b/drivers/infiniband/hw/hfi1/trace_tx.h index f00696f..ed1b9e1 100644 --- a/drivers/infiniband/hw/hfi1/trace_tx.h +++ b/drivers/infiniband/hw/hfi1/trace_tx.h @@ -926,11 +926,13 @@ __netif_subqueue_stopped(txq->priv->netdev, txq->q_idx); ), TP_printk(/* print */ - "[%s] txq %llx idx %u sde %llx head %lx tail %lx flow %x used %u stops %d no_desc %d stopped %u", + "[%s] txq %llx idx %u sde %llx:%u cpu %d head %lx tail %lx flow %x used %u stops %d no_desc %d stopped %u", __get_str(dev), (unsigned long long)__entry->txq, __entry->idx, (unsigned long long)__entry->sde, + __entry->sde ? __entry->sde->this_idx : 0, + __entry->sde ? __entry->sde->cpu : 0, __entry->head, __entry->tail, __entry->flow, @@ -995,6 +997,65 @@ TP_ARGS(txq) ); +DECLARE_EVENT_CLASS(/* AIP */ + hfi1_ipoib_tx_template, + TP_PROTO(struct ipoib_txreq *tx, u32 idx), + TP_ARGS(tx, idx), + TP_STRUCT__entry(/* entry */ + DD_DEV_ENTRY(tx->txq->priv->dd) + __field(struct ipoib_txreq *, tx) + __field(struct hfi1_ipoib_txq *, txq) + __field(struct sk_buff *, skb) + __field(ulong, idx) + ), + TP_fast_assign(/* assign */ + DD_DEV_ASSIGN(tx->txq->priv->dd); + __entry->tx = tx; + __entry->skb = tx->skb; + __entry->txq = tx->txq; + __entry->idx = idx; + ), + TP_printk(/* print */ + "[%s] tx %llx txq %llx,%u skb %llx idx %lu", + __get_str(dev), + (unsigned long long)__entry->tx, + (unsigned long long)__entry->txq, + __entry->txq ? __entry->txq->q_idx : 0, + (unsigned long long)__entry->skb, + __entry->idx + ) +); + +DEFINE_EVENT(/* produce */ + hfi1_ipoib_tx_template, hfi1_tx_produce, + TP_PROTO(struct ipoib_txreq *tx, u32 idx), + TP_ARGS(tx, idx) +); + +DEFINE_EVENT(/* consume */ + hfi1_ipoib_tx_template, hfi1_tx_consume, + TP_PROTO(struct ipoib_txreq *tx, u32 idx), + TP_ARGS(tx, idx) +); + +DEFINE_EVENT(/* alloc_tx */ + hfi1_ipoib_txq_template, hfi1_txq_alloc_tx, + TP_PROTO(struct hfi1_ipoib_txq *txq), + TP_ARGS(txq) +); + +DEFINE_EVENT(/* poll */ + hfi1_ipoib_txq_template, hfi1_txq_poll, + TP_PROTO(struct hfi1_ipoib_txq *txq), + TP_ARGS(txq) +); + +DEFINE_EVENT(/* complete */ + hfi1_ipoib_txq_template, hfi1_txq_complete, + TP_PROTO(struct hfi1_ipoib_txq *txq), + TP_ARGS(txq) +); + #endif /* __HFI1_TRACE_TX_H */ #undef TRACE_INCLUDE_PATH