From patchwork Mon Apr 13 12:57:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sriram R X-Patchwork-Id: 11485551 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0116114B4 for ; Mon, 13 Apr 2020 12:58:31 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D1E912073E for ; Mon, 13 Apr 2020 12:58:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="r5ip504P"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=mg.codeaurora.org header.i=@mg.codeaurora.org header.b="ZJdHv40R" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D1E912073E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=ath11k-bounces+patchwork-ath11k=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=h2vLY7nYD2r1JN2N+/LuB9QAwe3HannUPQBousn/uI8=; b=r5i p504PgSMkVFotYt3qBLWM9fsII+u7iISA/wn5vIWEOd8MK+/i22I5KEzORNVlvB5SsSecDZuUICdK WWa7DJFmdW7LI3wMsw73xtg6TAHlg6Zk2hkGl2swawbssfnYL2zEhiq+H3XkJATIvg3KHjIfvz4fZ X3RfoGJHmQXAXuNZK2sCaOOUJ9Z628ikdgKoBCR+F+WbM/uzl6COVKVDQy9bwQG0AjfCcianlIkuu AB7oATAed3ESLIK0BtIH8TPvNn6ANElP8e1iCXtZP3V+FGlzQ7PIwPqE4mSXi2MQUaHp6KqD/jMvC KURcqiKz1IRBbFlHBp93A2aAYx/JeuA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jNyfV-0000VB-Ar; Mon, 13 Apr 2020 12:58:29 +0000 Received: from mail27.static.mailgun.info ([104.130.122.27]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jNyfP-0000Uc-Fb for ath11k@lists.infradead.org; Mon, 13 Apr 2020 12:58:27 +0000 DKIM-Signature: a=rsa-sha256; v=1; c=relaxed/relaxed; d=mg.codeaurora.org; q=dns/txt; s=smtp; t=1586782705; h=Message-Id: Date: Subject: Cc: To: From: Sender; bh=4+mo7/Pvr13OOsfsq2l2XFs4m3v1lIOe1HuATB7CG4M=; b=ZJdHv40RLK4hcvusjiZwgoYgPrsvhnGz1wkD7BMRoA3R/2NhFqHXQb9ToVokvtKCNXJdYbLD 23wUrxeyRGmEBX2UJ6FCqZ/lwQz5DpCQahSixkJqn/TRUX1hC+HGssZJ1+40+fvF+EftGgDt j4gtVAF3jyYW2/vTfhQpHCjewNc= X-Mailgun-Sending-Ip: 104.130.122.27 X-Mailgun-Sid: WyJmOGQ2ZiIsICJhdGgxMWtAbGlzdHMuaW5mcmFkZWFkLm9yZyIsICJiZTllNGEiXQ== Received: from smtp.codeaurora.org (ec2-35-166-182-171.us-west-2.compute.amazonaws.com [35.166.182.171]) by mxa.mailgun.org with ESMTP id 5e9461e4.7f021a4caa78-smtp-out-n05; Mon, 13 Apr 2020 12:58:12 -0000 (UTC) Received: by smtp.codeaurora.org (Postfix, from userid 1001) id 7CFACC433BA; Mon, 13 Apr 2020 12:58:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-caf-mail-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=2.0 tests=ALL_TRUSTED,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.0 Received: from checstp253621-lin.qualcomm.com (blr-c-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.19.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: srirrama) by smtp.codeaurora.org (Postfix) with ESMTPSA id 510CFC433F2; Mon, 13 Apr 2020 12:58:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 510CFC433F2 Authentication-Results: aws-us-west-2-caf-mail-1.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: aws-us-west-2-caf-mail-1.web.codeaurora.org; spf=none smtp.mailfrom=srirrama@codeaurora.org From: Sriram R To: ath11k@lists.infradead.org Subject: [PATCH] ath11k: Add dynamic tcl ring selection logic with retry mechanism Date: Mon, 13 Apr 2020 18:27:02 +0530 Message-Id: <1586782622-22570-1-git-send-email-srirrama@codeaurora.org> X-Mailer: git-send-email 2.7.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200413_055826_081497_CAC4368A X-CRM114-Status: GOOD ( 18.50 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [104.130.122.27 listed in list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-BeenThere: ath11k@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-wireless@vger.kernel.org, Sriram R MIME-Version: 1.0 Sender: "ath11k" Errors-To: ath11k-bounces+patchwork-ath11k=patchwork.kernel.org@lists.infradead.org IPQ8074 HW supports three TCL rings for tx. Currently these rings are mapped based on the Access categories, viz. VO, VI, BE, BK. In case, one of the traffic type dominates, then it could stress the same tcl rings. Rather, it would be optimal to make use of all the rings in a round robin fashion irrespective of the traffic type so that the load could be evenly distributed among all the rings. Also, in case the selected ring is busy or full, a retry mechanism is used to ensure other available ring is selected without dropping the packet. In SMP systems, this change avoids a single CPU from getting hogged when heavy traffic of same category is transmitted. The tx completion interrupts corresponding to the used tcl ring would be more which causes the assigned CPU to get hogged. Distribution of tx packets to different tcl rings helps balance this load. Signed-off-by: Sriram R --- drivers/net/wireless/ath/ath11k/core.h | 3 +++ drivers/net/wireless/ath/ath11k/dp_tx.c | 46 +++++++++++++++++++++++++++------ 2 files changed, 41 insertions(+), 8 deletions(-) diff --git a/drivers/net/wireless/ath/ath11k/core.h b/drivers/net/wireless/ath/ath11k/core.h index b4c3e041..2e20faa 100644 --- a/drivers/net/wireless/ath/ath11k/core.h +++ b/drivers/net/wireless/ath/ath11k/core.h @@ -656,6 +656,9 @@ struct ath11k_base { u32 fw_crash_counter; } stats; u32 pktlog_defs_checksum; + + /* Round robbin based TCL ring selector */ + atomic_t tcl_ring_selector; }; struct ath11k_fw_stats_pdev { diff --git a/drivers/net/wireless/ath/ath11k/dp_tx.c b/drivers/net/wireless/ath/ath11k/dp_tx.c index 7aac4b0..e9a436e 100644 --- a/drivers/net/wireless/ath/ath11k/dp_tx.c +++ b/drivers/net/wireless/ath/ath11k/dp_tx.c @@ -9,10 +9,6 @@ #include "hw.h" #include "peer.h" -/* NOTE: Any of the mapped ring id value must not exceed DP_TCL_NUM_RING_MAX */ -static const u8 -ath11k_txq_tcl_ring_map[ATH11K_HW_MAX_QUEUES] = { 0x0, 0x1, 0x2, 0x2 }; - static enum hal_tcl_encap_type ath11k_dp_tx_get_encap_type(struct ath11k_vif *arvif, struct sk_buff *skb) { @@ -84,6 +80,8 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif, u8 pool_id; u8 hal_ring_id; int ret; + u8 ring_selector = 0, ring_map = 0; + bool tcl_ring_retry; if (test_bit(ATH11K_FLAG_CRASH_FLUSH, &ar->ab->dev_flags)) return -ESHUTDOWN; @@ -92,7 +90,20 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif, return -ENOTSUPP; pool_id = skb_get_queue_mapping(skb) & (ATH11K_HW_MAX_QUEUES - 1); - ti.ring_id = ath11k_txq_tcl_ring_map[pool_id]; + + /* Let the default ring selection be based on a round robin + * fashion where one of the 3 tcl rings are selected based on + * the tcl_ring_selector counter. In case that ring + * is full/busy, we resort to other available rings. + * If all rings are full, we drop the packet. + * //TODO Add throttling logic when all rings are full + */ + ring_selector = atomic_inc_return(&ab->tcl_ring_selector); + +tcl_ring_sel: + tcl_ring_retry = false; + ti.ring_id = ring_selector % DP_TCL_NUM_RING_MAX; + ring_map |= BIT(ti.ring_id); tx_ring = &dp->tx_ring[ti.ring_id]; @@ -101,8 +112,14 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif, DP_TX_IDR_SIZE - 1, GFP_ATOMIC); spin_unlock_bh(&tx_ring->tx_idr_lock); - if (ret < 0) - return -ENOSPC; + if (ret < 0) { + if (ring_map == (BIT(DP_TCL_NUM_RING_MAX) - 1)) + return -ENOSPC; + + /* Check if the next ring is available */ + ring_selector++; + goto tcl_ring_sel; + } ti.desc_id = FIELD_PREP(DP_TX_DESC_ID_MAC_ID, ar->pdev_idx) | FIELD_PREP(DP_TX_DESC_ID_MSDU_ID, ret) | @@ -178,11 +195,21 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif, if (!hal_tcl_desc) { /* NOTE: It is highly unlikely we'll be running out of tcl_ring * desc because the desc is directly enqueued onto hw queue. - * So add tx packet throttling logic in future if required. */ ath11k_hal_srng_access_end(ab, tcl_ring); spin_unlock_bh(&tcl_ring->lock); ret = -ENOMEM; + + /* Checking for available tcl descritors in another ring in + * case of failure due to full tcl ring now, is better than + * checking this ring earlier for each pkt tx. + * Restart ring selection if some rings are not checked yet. + */ + if (ring_map != (BIT(DP_TCL_NUM_RING_MAX) - 1)) { + tcl_ring_retry = true; + ring_selector++; + } + goto fail_unmap_dma; } @@ -206,6 +233,9 @@ int ath11k_dp_tx(struct ath11k *ar, struct ath11k_vif *arvif, FIELD_GET(DP_TX_DESC_ID_MSDU_ID, ti.desc_id)); spin_unlock_bh(&tx_ring->tx_idr_lock); + if (tcl_ring_retry) + goto tcl_ring_sel; + return ret; }