From patchwork Mon Sep 16 13:09:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yibo Zhao X-Patchwork-Id: 11147017 X-Patchwork-Delegate: johannes@sipsolutions.net Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3F96C14DB for ; Mon, 16 Sep 2019 13:10:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 004EE21670 for ; Mon, 16 Sep 2019 13:10:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="key not found in DNS" (0-bit key) header.d=codeaurora.org header.i=@codeaurora.org header.b="WAEX8gyc"; dkim=fail reason="key not found in DNS" (0-bit key) header.d=codeaurora.org header.i=@codeaurora.org header.b="aX+034+c" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728004AbfIPNKJ (ORCPT ); Mon, 16 Sep 2019 09:10:09 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:43668 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727935AbfIPNKJ (ORCPT ); Mon, 16 Sep 2019 09:10:09 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id D7664611FA; Mon, 16 Sep 2019 13:10:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1568639407; bh=N1/FCPeJHzQuYeEVyXwjYaJ9UWHutZ+Js8Xa50mX66A=; h=From:To:Cc:Subject:Date:From; b=WAEX8gycEl88RYwVOCGcrVSoyYMvPWMBA/l5+392ANubRQrIucXSxdYB20ZpTeT9u OmXAkx/5K/4vYRPiOflPO4AOZM+YUMf1Pr1KQ77bardMDc2ofhb1GhbCbMQuQn9pns RUL5Rj9VmLgOmsSYBhR1va/Dp4Gka/X25AB2uYCI= X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on pdx-caf-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=ALL_TRUSTED,BAYES_00, DKIM_INVALID,DKIM_SIGNED,SPF_NONE autolearn=no autolearn_force=no version=3.4.0 Received: from ybzhao-HP-Z230-SFF-Workstation.ap.qualcomm.com (unknown [180.166.53.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: yiboz@codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id E5D2060A60; Mon, 16 Sep 2019 13:10:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1568639406; bh=N1/FCPeJHzQuYeEVyXwjYaJ9UWHutZ+Js8Xa50mX66A=; h=From:To:Cc:Subject:Date:From; b=aX+034+cOc2ftyrbotBHRiYx2xcaNSlpp7UKgK9Su9HKZMEJNHFnO6t+JiQ4ms5YD aSI1pt1kowjGM5jVDEPD3nvo1dOuX8j6MJ/9U3ZTteQJ3wA3Nj0k71ELRNOjWJKqMC jgVT696y418n9AVHWUbN7TgQi5D0qLJfMfOG8UTw= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org E5D2060A60 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=yiboz@codeaurora.org From: Yibo Zhao To: ath10k@lists.infradead.org Cc: linux-wireless@vger.kernel.org, =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgens?= =?utf-8?q?en?= Subject: [PATCH 1/4] mac80211: Switch to a virtual time-based airtime scheduler Date: Mon, 16 Sep 2019 21:09:45 +0800 Message-Id: <1568639388-27291-1-git-send-email-yiboz@codeaurora.org> X-Mailer: git-send-email 1.9.1 MIME-Version: 1.0 Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org From: Toke Høiland-Jørgensen This switches the airtime scheduler in mac80211 to use a virtual time-based scheduler instead of the round-robin scheduler used before. This has a couple of advantages: - No need to sync up the round-robin scheduler in firmware/hardware with the round-robin airtime scheduler. - If several stations are eligible for transmission we can schedule both of them; no need to hard-block the scheduling rotation until the head of the queue has used up its quantum. - The check of whether a station is eligible for transmission becomes simpler (in ieee80211_txq_may_transmit()). The drawback is that scheduling becomes slightly more expensive, as we need to maintain an rbtree of TXQs sorted by virtual time. This means that ieee80211_register_airtime() becomes O(logN) in the number of currently scheduled TXQs. However, hopefully this number rarely grows too big (it's only TXQs currently backlogged, not all associated stations), so it shouldn't be too big of an issue. Signed-off-by: Toke Høiland-Jørgensen --- net/mac80211/debugfs.c | 48 +++++++++- net/mac80211/debugfs_sta.c | 16 ++-- net/mac80211/ieee80211_i.h | 14 ++- net/mac80211/main.c | 2 +- net/mac80211/sta_info.c | 19 +++- net/mac80211/sta_info.h | 3 +- net/mac80211/tx.c | 217 +++++++++++++++++++++++++++++---------------- 7 files changed, 223 insertions(+), 96 deletions(-) diff --git a/net/mac80211/debugfs.c b/net/mac80211/debugfs.c index 2d43bc1..4847168 100644 --- a/net/mac80211/debugfs.c +++ b/net/mac80211/debugfs.c @@ -150,6 +150,46 @@ static ssize_t aqm_write(struct file *file, .llseek = default_llseek, }; +static ssize_t airtime_read(struct file *file, + char __user *user_buf, + size_t count, + loff_t *ppos) +{ + struct ieee80211_local *local = file->private_data; + char buf[200]; + u64 v_t[IEEE80211_NUM_ACS]; + u64 wt[IEEE80211_NUM_ACS]; + int len = 0, ac; + + for (ac = 0; ac < IEEE80211_NUM_ACS; ac++) { + spin_lock_bh(&local->active_txq_lock[ac]); + v_t[ac] = local->airtime_v_t[ac]; + wt[ac] = local->airtime_weight_sum[ac]; + spin_unlock_bh(&local->active_txq_lock[ac]); + } + len = scnprintf(buf, sizeof(buf), + "\tVO VI BE BK\n" + "Virt-t\t%-10llu %-10llu %-10llu %-10llu\n" + "Weight\t%-10llu %-10llu %-10llu %-10llu\n", + v_t[0], + v_t[1], + v_t[2], + v_t[3], + wt[0], + wt[1], + wt[2], + wt[3]); + + return simple_read_from_buffer(user_buf, count, ppos, + buf, len); +} + +static const struct file_operations airtime_ops = { + .read = airtime_read, + .open = simple_open, + .llseek = default_llseek, +}; + #ifdef CONFIG_PM static ssize_t reset_write(struct file *file, const char __user *user_buf, size_t count, loff_t *ppos) @@ -386,8 +426,12 @@ void debugfs_hw_add(struct ieee80211_local *local) if (local->ops->wake_tx_queue) DEBUGFS_ADD_MODE(aqm, 0600); - debugfs_create_u16("airtime_flags", 0600, - phyd, &local->airtime_flags); + if (wiphy_ext_feature_isset(local->hw.wiphy, + NL80211_EXT_FEATURE_AIRTIME_FAIRNESS)) { + DEBUGFS_ADD_MODE(airtime, 0600); + debugfs_create_u16("airtime_flags", 0600, + phyd, &local->airtime_flags); + } statsd = debugfs_create_dir("statistics", phyd); diff --git a/net/mac80211/debugfs_sta.c b/net/mac80211/debugfs_sta.c index 3aa618d..80028da 100644 --- a/net/mac80211/debugfs_sta.c +++ b/net/mac80211/debugfs_sta.c @@ -203,7 +203,7 @@ static ssize_t sta_airtime_read(struct file *file, char __user *userbuf, size_t bufsz = 200; char *buf = kzalloc(bufsz, GFP_KERNEL), *p = buf; u64 rx_airtime = 0, tx_airtime = 0; - s64 deficit[IEEE80211_NUM_ACS]; + u64 v_t[IEEE80211_NUM_ACS]; ssize_t rv; int ac; @@ -214,20 +214,20 @@ static ssize_t sta_airtime_read(struct file *file, char __user *userbuf, spin_lock_bh(&local->active_txq_lock[ac]); rx_airtime += sta->airtime[ac].rx_airtime; tx_airtime += sta->airtime[ac].tx_airtime; - deficit[ac] = sta->airtime[ac].deficit; + v_t[ac] = sta->airtime[ac].v_t; spin_unlock_bh(&local->active_txq_lock[ac]); } p += scnprintf(p, bufsz + buf - p, "RX: %llu us\nTX: %llu us\nWeight: %u\n" - "Deficit: VO: %lld us VI: %lld us BE: %lld us BK: %lld us\n", + "Virt-T: VO: %lld us VI: %lld us BE: %lld us BK: %lld us\n", rx_airtime, tx_airtime, sta->airtime_weight, - deficit[0], - deficit[1], - deficit[2], - deficit[3]); + v_t[0], + v_t[1], + v_t[2], + v_t[3]); rv = simple_read_from_buffer(userbuf, count, ppos, buf, p - buf); kfree(buf); @@ -245,7 +245,7 @@ static ssize_t sta_airtime_write(struct file *file, const char __user *userbuf, spin_lock_bh(&local->active_txq_lock[ac]); sta->airtime[ac].rx_airtime = 0; sta->airtime[ac].tx_airtime = 0; - sta->airtime[ac].deficit = sta->airtime_weight; + sta->airtime[ac].v_t = 0; spin_unlock_bh(&local->active_txq_lock[ac]); } diff --git a/net/mac80211/ieee80211_i.h b/net/mac80211/ieee80211_i.h index e170f98..a4556f9 100644 --- a/net/mac80211/ieee80211_i.h +++ b/net/mac80211/ieee80211_i.h @@ -846,8 +846,7 @@ struct txq_info { struct codel_vars def_cvars; struct codel_stats cstats; struct sk_buff_head frags; - struct list_head schedule_order; - u16 schedule_round; + struct rb_node schedule_order; unsigned long flags; /* keep last! */ @@ -1141,8 +1140,10 @@ struct ieee80211_local { /* protects active_txqs and txqi->schedule_order */ spinlock_t active_txq_lock[IEEE80211_NUM_ACS]; - struct list_head active_txqs[IEEE80211_NUM_ACS]; - u16 schedule_round[IEEE80211_NUM_ACS]; + struct rb_root_cached active_txqs[IEEE80211_NUM_ACS]; + struct rb_node *schedule_pos[IEEE80211_NUM_ACS]; + u64 airtime_v_t[IEEE80211_NUM_ACS]; + u64 airtime_weight_sum[IEEE80211_NUM_ACS]; u16 airtime_flags; @@ -1779,6 +1780,11 @@ int ieee80211_tx_control_port(struct wiphy *wiphy, struct net_device *dev, const u8 *buf, size_t len, const u8 *dest, __be16 proto, bool unencrypted); +void ieee80211_resort_txq(struct ieee80211_hw *hw, + struct ieee80211_txq *txq); +void ieee80211_unschedule_txq(struct ieee80211_hw *hw, + struct ieee80211_txq *txq); + /* HT */ void ieee80211_apply_htcap_overrides(struct ieee80211_sub_if_data *sdata, struct ieee80211_sta_ht_cap *ht_cap); diff --git a/net/mac80211/main.c b/net/mac80211/main.c index 5055aeb..e9ffa8e 100644 --- a/net/mac80211/main.c +++ b/net/mac80211/main.c @@ -666,7 +666,7 @@ struct ieee80211_hw *ieee80211_alloc_hw_nm(size_t priv_data_len, spin_lock_init(&local->queue_stop_reason_lock); for (i = 0; i < IEEE80211_NUM_ACS; i++) { - INIT_LIST_HEAD(&local->active_txqs[i]); + local->active_txqs[i] = RB_ROOT_CACHED; spin_lock_init(&local->active_txq_lock[i]); } local->airtime_flags = AIRTIME_USE_TX | AIRTIME_USE_RX; diff --git a/net/mac80211/sta_info.c b/net/mac80211/sta_info.c index 11f0589..9d01fdd 100644 --- a/net/mac80211/sta_info.c +++ b/net/mac80211/sta_info.c @@ -389,7 +389,6 @@ struct sta_info *sta_info_alloc(struct ieee80211_sub_if_data *sdata, for (i = 0; i < IEEE80211_NUM_ACS; i++) { skb_queue_head_init(&sta->ps_tx_buf[i]); skb_queue_head_init(&sta->tx_filtered[i]); - sta->airtime[i].deficit = sta->airtime_weight; } for (i = 0; i < IEEE80211_NUM_TIDS; i++) @@ -1831,18 +1830,32 @@ void ieee80211_sta_register_airtime(struct ieee80211_sta *pubsta, u8 tid, { struct sta_info *sta = container_of(pubsta, struct sta_info, sta); struct ieee80211_local *local = sta->sdata->local; + struct ieee80211_txq *txq = sta->sta.txq[tid]; u8 ac = ieee80211_ac_from_tid(tid); - u32 airtime = 0; + u64 airtime = 0, weight_sum; + + if (!txq) + return; if (sta->local->airtime_flags & AIRTIME_USE_TX) airtime += tx_airtime; if (sta->local->airtime_flags & AIRTIME_USE_RX) airtime += rx_airtime; + /* Weights scale so the unit weight is 256 */ + airtime <<= 8; + spin_lock_bh(&local->active_txq_lock[ac]); + sta->airtime[ac].tx_airtime += tx_airtime; sta->airtime[ac].rx_airtime += rx_airtime; - sta->airtime[ac].deficit -= airtime; + + weight_sum = local->airtime_weight_sum[ac] ?: sta->airtime_weight; + + local->airtime_v_t[ac] += airtime / weight_sum; + sta->airtime[ac].v_t += airtime / sta->airtime_weight; + ieee80211_resort_txq(&local->hw, txq); + spin_unlock_bh(&local->active_txq_lock[ac]); } EXPORT_SYMBOL(ieee80211_sta_register_airtime); diff --git a/net/mac80211/sta_info.h b/net/mac80211/sta_info.h index 71f7e49..5c1cac9 100644 --- a/net/mac80211/sta_info.h +++ b/net/mac80211/sta_info.h @@ -130,11 +130,12 @@ enum ieee80211_agg_stop_reason { /* Debugfs flags to enable/disable use of RX/TX airtime in scheduler */ #define AIRTIME_USE_TX BIT(0) #define AIRTIME_USE_RX BIT(1) +#define AIRTIME_GRACE 500 /* usec of grace period before reset */ struct airtime_info { u64 rx_airtime; u64 tx_airtime; - s64 deficit; + u64 v_t; }; struct sta_info; diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c index 61c7ea9..d00baaa 100644 --- a/net/mac80211/tx.c +++ b/net/mac80211/tx.c @@ -1449,7 +1449,7 @@ void ieee80211_txq_init(struct ieee80211_sub_if_data *sdata, codel_vars_init(&txqi->def_cvars); codel_stats_init(&txqi->cstats); __skb_queue_head_init(&txqi->frags); - INIT_LIST_HEAD(&txqi->schedule_order); + RB_CLEAR_NODE(&txqi->schedule_order); txqi->txq.vif = &sdata->vif; @@ -1493,9 +1493,7 @@ void ieee80211_txq_purge(struct ieee80211_local *local, ieee80211_purge_tx_queue(&local->hw, &txqi->frags); spin_unlock_bh(&fq->lock); - spin_lock_bh(&local->active_txq_lock[txqi->txq.ac]); - list_del_init(&txqi->schedule_order); - spin_unlock_bh(&local->active_txq_lock[txqi->txq.ac]); + ieee80211_unschedule_txq(&local->hw, &txqi->txq); } void ieee80211_txq_set_params(struct ieee80211_local *local) @@ -3640,126 +3638,191 @@ struct sk_buff *ieee80211_tx_dequeue(struct ieee80211_hw *hw, struct ieee80211_txq *ieee80211_next_txq(struct ieee80211_hw *hw, u8 ac) { struct ieee80211_local *local = hw_to_local(hw); + struct rb_node *node = local->schedule_pos[ac]; struct txq_info *txqi = NULL; + bool first = false; lockdep_assert_held(&local->active_txq_lock[ac]); - begin: - txqi = list_first_entry_or_null(&local->active_txqs[ac], - struct txq_info, - schedule_order); - if (!txqi) + if (!node) { + node = rb_first_cached(&local->active_txqs[ac]); + first = true; + } else { + node = rb_next(node); + } + + if (!node) return NULL; + txqi = container_of(node, struct txq_info, schedule_order); + if (txqi->txq.sta) { struct sta_info *sta = container_of(txqi->txq.sta, struct sta_info, sta); - if (sta->airtime[txqi->txq.ac].deficit < 0) { - sta->airtime[txqi->txq.ac].deficit += - sta->airtime_weight; - list_move_tail(&txqi->schedule_order, - &local->active_txqs[txqi->txq.ac]); - goto begin; + if (sta->airtime[ac].v_t > local->airtime_v_t[ac]) { + if (first) + local->airtime_v_t[ac] = sta->airtime[ac].v_t; + else + return NULL; } } - if (txqi->schedule_round == local->schedule_round[ac]) - return NULL; - - list_del_init(&txqi->schedule_order); - txqi->schedule_round = local->schedule_round[ac]; + local->schedule_pos[ac] = node; return &txqi->txq; } EXPORT_SYMBOL(ieee80211_next_txq); -void ieee80211_return_txq(struct ieee80211_hw *hw, +static void __ieee80211_insert_txq(struct rb_root_cached *root, + struct txq_info *txqi, u8 ac) +{ + struct rb_node **new = &root->rb_root.rb_node; + struct rb_node *parent = NULL; + struct txq_info *__txqi; + bool leftmost = true; + + while (*new) { + parent = *new; + __txqi = rb_entry(parent, struct txq_info, schedule_order); + + if (!txqi->txq.sta) { + /* new txqi has no sta - insert to the left */ + new = &parent->rb_left; + } else if (!__txqi->txq.sta) { + /* existing txqi has no sta - insert to the right */ + new = &parent->rb_right; + leftmost = false; + } else { + struct sta_info *old_sta = container_of(__txqi->txq.sta, + struct sta_info, + sta); + struct sta_info *new_sta = container_of(txqi->txq.sta, + struct sta_info, + sta); + + if (new_sta->airtime[ac].v_t <= old_sta->airtime[ac].v_t) { + new = &parent->rb_left; + } else { + new = &parent->rb_right; + leftmost = false; + } + } + } + + rb_link_node(&txqi->schedule_order, parent, new); + rb_insert_color_cached(&txqi->schedule_order, root, leftmost); +} + +void ieee80211_schedule_txq(struct ieee80211_hw *hw, + struct ieee80211_txq *txq) + __acquires(txq_lock) __releases(txq_lock) +{ + struct ieee80211_local *local = hw_to_local(hw); + struct txq_info *txqi = to_txq_info(txq); + u8 ac = txq->ac; + + spin_lock_bh(&local->active_txq_lock[ac]); + + if (!RB_EMPTY_NODE(&txqi->schedule_order)) + goto out; + + if (txq->sta) { + struct sta_info *sta = container_of(txq->sta, + struct sta_info, sta); + + local->airtime_weight_sum[ac] += sta->airtime_weight; + if (local->airtime_v_t[ac] > AIRTIME_GRACE) + sta->airtime[ac].v_t = max(local->airtime_v_t[ac] - AIRTIME_GRACE, + sta->airtime[ac].v_t); + } + + __ieee80211_insert_txq(&local->active_txqs[ac], txqi, ac); + + out: + spin_unlock_bh(&local->active_txq_lock[ac]); +} +EXPORT_SYMBOL(ieee80211_schedule_txq); + +void ieee80211_resort_txq(struct ieee80211_hw *hw, struct ieee80211_txq *txq) { struct ieee80211_local *local = hw_to_local(hw); struct txq_info *txqi = to_txq_info(txq); + u8 ac = txq->ac; + + if (!RB_EMPTY_NODE(&txqi->schedule_order)) { + rb_erase_cached(&txqi->schedule_order, + &local->active_txqs[ac]); + RB_CLEAR_NODE(&txqi->schedule_order); + __ieee80211_insert_txq(&local->active_txqs[ac], txqi, ac); + } +} + +static void __ieee80211_unschedule_txq(struct ieee80211_hw *hw, + struct ieee80211_txq *txq) +{ + struct ieee80211_local *local = hw_to_local(hw); + struct txq_info *txqi = to_txq_info(txq); + u8 ac = txq->ac; lockdep_assert_held(&local->active_txq_lock[txq->ac]); - if (list_empty(&txqi->schedule_order) && - (!skb_queue_empty(&txqi->frags) || txqi->tin.backlog_packets)) { - /* If airtime accounting is active, always enqueue STAs at the - * head of the list to ensure that they only get moved to the - * back by the airtime DRR scheduler once they have a negative - * deficit. A station that already has a negative deficit will - * get immediately moved to the back of the list on the next - * call to ieee80211_next_txq(). - */ - if (txqi->txq.sta && - wiphy_ext_feature_isset(local->hw.wiphy, - NL80211_EXT_FEATURE_AIRTIME_FAIRNESS)) - list_add(&txqi->schedule_order, - &local->active_txqs[txq->ac]); - else - list_add_tail(&txqi->schedule_order, - &local->active_txqs[txq->ac]); + if (RB_EMPTY_NODE(&txqi->schedule_order)) + return; + + if (txq->sta) { + struct sta_info *sta = container_of(txq->sta, + struct sta_info, sta); + + local->airtime_weight_sum[ac] -= sta->airtime_weight; } + + rb_erase_cached(&txqi->schedule_order, + &local->active_txqs[txq->ac]); + RB_CLEAR_NODE(&txqi->schedule_order); } -EXPORT_SYMBOL(ieee80211_return_txq); -void ieee80211_schedule_txq(struct ieee80211_hw *hw, - struct ieee80211_txq *txq) +void ieee80211_unschedule_txq(struct ieee80211_hw *hw, + struct ieee80211_txq *txq) __acquires(txq_lock) __releases(txq_lock) { struct ieee80211_local *local = hw_to_local(hw); spin_lock_bh(&local->active_txq_lock[txq->ac]); - ieee80211_return_txq(hw, txq); + __ieee80211_unschedule_txq(hw, txq); spin_unlock_bh(&local->active_txq_lock[txq->ac]); } -EXPORT_SYMBOL(ieee80211_schedule_txq); + +void ieee80211_return_txq(struct ieee80211_hw *hw, + struct ieee80211_txq *txq) +{ + struct ieee80211_local *local = hw_to_local(hw); + struct txq_info *txqi = to_txq_info(txq); + + lockdep_assert_held(&local->active_txq_lock[txq->ac]); + + if (!RB_EMPTY_NODE(&txqi->schedule_order) && + (skb_queue_empty(&txqi->frags) && !txqi->tin.backlog_packets)) + __ieee80211_unschedule_txq(hw, txq); +} +EXPORT_SYMBOL(ieee80211_return_txq); bool ieee80211_txq_may_transmit(struct ieee80211_hw *hw, struct ieee80211_txq *txq) { struct ieee80211_local *local = hw_to_local(hw); - struct txq_info *iter, *tmp, *txqi = to_txq_info(txq); + struct txq_info *txqi = to_txq_info(txq); struct sta_info *sta; u8 ac = txq->ac; lockdep_assert_held(&local->active_txq_lock[ac]); if (!txqi->txq.sta) - goto out; - - if (list_empty(&txqi->schedule_order)) - goto out; - - list_for_each_entry_safe(iter, tmp, &local->active_txqs[ac], - schedule_order) { - if (iter == txqi) - break; - - if (!iter->txq.sta) { - list_move_tail(&iter->schedule_order, - &local->active_txqs[ac]); - continue; - } - sta = container_of(iter->txq.sta, struct sta_info, sta); - if (sta->airtime[ac].deficit < 0) - sta->airtime[ac].deficit += sta->airtime_weight; - list_move_tail(&iter->schedule_order, &local->active_txqs[ac]); - } + return true; sta = container_of(txqi->txq.sta, struct sta_info, sta); - if (sta->airtime[ac].deficit >= 0) - goto out; - - sta->airtime[ac].deficit += sta->airtime_weight; - list_move_tail(&txqi->schedule_order, &local->active_txqs[ac]); - - return false; -out: - if (!list_empty(&txqi->schedule_order)) - list_del_init(&txqi->schedule_order); - - return true; + return (sta->airtime[ac].v_t <= local->airtime_v_t[ac]); } EXPORT_SYMBOL(ieee80211_txq_may_transmit); @@ -3769,7 +3832,6 @@ void ieee80211_txq_schedule_start(struct ieee80211_hw *hw, u8 ac) struct ieee80211_local *local = hw_to_local(hw); spin_lock_bh(&local->active_txq_lock[ac]); - local->schedule_round[ac]++; } EXPORT_SYMBOL(ieee80211_txq_schedule_start); @@ -3778,6 +3840,7 @@ void ieee80211_txq_schedule_end(struct ieee80211_hw *hw, u8 ac) { struct ieee80211_local *local = hw_to_local(hw); + local->schedule_pos[ac] = NULL; spin_unlock_bh(&local->active_txq_lock[ac]); } EXPORT_SYMBOL(ieee80211_txq_schedule_end); From patchwork Mon Sep 16 13:09:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yibo Zhao X-Patchwork-Id: 11147019 X-Patchwork-Delegate: johannes@sipsolutions.net Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CAC8A14DB for ; Mon, 16 Sep 2019 13:10:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9E34B20678 for ; Mon, 16 Sep 2019 13:10:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="key not found in DNS" (0-bit key) header.d=codeaurora.org header.i=@codeaurora.org header.b="eFeL7J8c"; dkim=fail reason="key not found in DNS" (0-bit key) header.d=codeaurora.org header.i=@codeaurora.org header.b="WmTPcQSF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728008AbfIPNKP (ORCPT ); Mon, 16 Sep 2019 09:10:15 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:43748 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727882AbfIPNKO (ORCPT ); Mon, 16 Sep 2019 09:10:14 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 0CF5860A60; Mon, 16 Sep 2019 13:10:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1568639413; bh=An9+PkJ418+vo52vNn4j9yFdZoSyrVtzihR23XPqoJg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eFeL7J8cjXJ4WVh7uV6TMMnye2ssxOU/g3dzqcyESl5wUrFRurGxj6c68ES09hDH4 rZlScRTo2U8msDOpx0EV/P2Y05L78i0qgiCrfDrplBtP3sgkRW8ThdyOdezrcAFHga 7efNxYRpGHIG9HLI1Zsz74N8w+UDE7zLTTf7IKSE= X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on pdx-caf-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=ALL_TRUSTED,BAYES_00, DKIM_INVALID,DKIM_SIGNED,SPF_NONE autolearn=no autolearn_force=no version=3.4.0 Received: from ybzhao-HP-Z230-SFF-Workstation.ap.qualcomm.com (unknown [180.166.53.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: yiboz@codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 961C960A60; Mon, 16 Sep 2019 13:10:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1568639411; bh=An9+PkJ418+vo52vNn4j9yFdZoSyrVtzihR23XPqoJg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WmTPcQSF1kcPg6B8ej/OeXWz9co0JaEjFGwC2qfpWsvEP2aAOsjEL7ns0mRS2dqZZ DkKi40J3gtnvZyDH9FJsnHd7FeDhgbEKpwPKI/WnHsvsvq6WvNMSvbikO393p2uNeb +S2vGFg9XYsbhNnsLcJeCvEKsEbk7L/xN1XgedkQ= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 961C960A60 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=yiboz@codeaurora.org From: Yibo Zhao To: ath10k@lists.infradead.org Cc: linux-wireless@vger.kernel.org, Yibo Zhao , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH 2/4] mac80211: defer txqs removal from rbtree Date: Mon, 16 Sep 2019 21:09:46 +0800 Message-Id: <1568639388-27291-2-git-send-email-yiboz@codeaurora.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1568639388-27291-1-git-send-email-yiboz@codeaurora.org> References: <1568639388-27291-1-git-send-email-yiboz@codeaurora.org> MIME-Version: 1.0 Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org In a loop txqs dequeue scenario, if the first txq in the rbtree gets removed from rbtree immediately in the ieee80211_return_txq(), the loop will break soon in the ieee80211_next_txq() due to schedule_pos not leading to the second txq in the rbtree. Thus, defering the removal right before the end of this schedule round. Co-developed-by: Yibo Zhao Signed-off-by: Yibo Zhao Signed-off-by: Toke Høiland-Jørgensen --- include/net/mac80211.h | 16 ++++++++++-- net/mac80211/ieee80211_i.h | 3 +++ net/mac80211/main.c | 6 +++++ net/mac80211/tx.c | 63 +++++++++++++++++++++++++++++++++++++++++++--- 4 files changed, 83 insertions(+), 5 deletions(-) diff --git a/include/net/mac80211.h b/include/net/mac80211.h index ac2ed8e..ba5a345 100644 --- a/include/net/mac80211.h +++ b/include/net/mac80211.h @@ -925,6 +925,8 @@ struct ieee80211_tx_rate { #define IEEE80211_MAX_TX_RETRY 31 +#define IEEE80211_AIRTIME_TXQ_RM_CHK_INTV_IN_MS 100 + static inline void ieee80211_rate_set_vht(struct ieee80211_tx_rate *rate, u8 mcs, u8 nss) { @@ -6232,7 +6234,8 @@ struct sk_buff *ieee80211_tx_dequeue(struct ieee80211_hw *hw, * @ac: AC number to return packets from. * * Should only be called between calls to ieee80211_txq_schedule_start() - * and ieee80211_txq_schedule_end(). + * and ieee80211_txq_schedule_end(). If the txq is empty, it will be added + * to a remove list and get removed later. * Returns the next txq if successful, %NULL if no queue is eligible. If a txq * is returned, it should be returned with ieee80211_return_txq() after the * driver has finished scheduling it. @@ -6268,7 +6271,8 @@ void ieee80211_txq_schedule_start(struct ieee80211_hw *hw, u8 ac) * @hw: pointer as obtained from ieee80211_alloc_hw() * @ac: AC number to acquire locks for * - * Release locks previously acquired by ieee80211_txq_schedule_end(). + * Release locks previously acquired by ieee80211_txq_schedule_end(). Check + * and remove the empty txq from rb-tree. */ void ieee80211_txq_schedule_end(struct ieee80211_hw *hw, u8 ac) __releases(txq_lock); @@ -6287,6 +6291,14 @@ void ieee80211_schedule_txq(struct ieee80211_hw *hw, struct ieee80211_txq *txq) __acquires(txq_lock) __releases(txq_lock); /** + * ieee80211_txqs_check - Check txqs waiting for removal + * + * @tmr: pointer as obtained from local + * + */ +void ieee80211_txqs_check(struct timer_list *tmr); + +/** * ieee80211_txq_may_transmit - check whether TXQ is allowed to transmit * * This function is used to check whether given txq is allowed to transmit by diff --git a/net/mac80211/ieee80211_i.h b/net/mac80211/ieee80211_i.h index a4556f9..49aa143e 100644 --- a/net/mac80211/ieee80211_i.h +++ b/net/mac80211/ieee80211_i.h @@ -847,6 +847,7 @@ struct txq_info { struct codel_stats cstats; struct sk_buff_head frags; struct rb_node schedule_order; + struct list_head candidate; unsigned long flags; /* keep last! */ @@ -1145,6 +1146,8 @@ struct ieee80211_local { u64 airtime_v_t[IEEE80211_NUM_ACS]; u64 airtime_weight_sum[IEEE80211_NUM_ACS]; + struct list_head remove_list[IEEE80211_NUM_ACS]; + struct timer_list remove_timer; u16 airtime_flags; const struct ieee80211_ops *ops; diff --git a/net/mac80211/main.c b/net/mac80211/main.c index e9ffa8e..78fe24a 100644 --- a/net/mac80211/main.c +++ b/net/mac80211/main.c @@ -667,10 +667,15 @@ struct ieee80211_hw *ieee80211_alloc_hw_nm(size_t priv_data_len, for (i = 0; i < IEEE80211_NUM_ACS; i++) { local->active_txqs[i] = RB_ROOT_CACHED; + INIT_LIST_HEAD(&local->remove_list[i]); spin_lock_init(&local->active_txq_lock[i]); } local->airtime_flags = AIRTIME_USE_TX | AIRTIME_USE_RX; + timer_setup(&local->remove_timer, ieee80211_txqs_check, 0); + mod_timer(&local->remove_timer, + jiffies + msecs_to_jiffies(IEEE80211_AIRTIME_TXQ_RM_CHK_INTV_IN_MS)); + INIT_LIST_HEAD(&local->chanctx_list); mutex_init(&local->chanctx_mtx); @@ -1305,6 +1310,7 @@ void ieee80211_unregister_hw(struct ieee80211_hw *hw) tasklet_kill(&local->tx_pending_tasklet); tasklet_kill(&local->tasklet); + del_timer_sync(&local->remove_timer); #ifdef CONFIG_INET unregister_inetaddr_notifier(&local->ifa_notifier); #endif diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c index d00baaa..42ca010 100644 --- a/net/mac80211/tx.c +++ b/net/mac80211/tx.c @@ -1450,6 +1450,7 @@ void ieee80211_txq_init(struct ieee80211_sub_if_data *sdata, codel_stats_init(&txqi->cstats); __skb_queue_head_init(&txqi->frags); RB_CLEAR_NODE(&txqi->schedule_order); + INIT_LIST_HEAD(&txqi->candidate); txqi->txq.vif = &sdata->vif; @@ -3724,6 +3725,9 @@ void ieee80211_schedule_txq(struct ieee80211_hw *hw, spin_lock_bh(&local->active_txq_lock[ac]); + if (!list_empty(&txqi->candidate)) + list_del_init(&txqi->candidate); + if (!RB_EMPTY_NODE(&txqi->schedule_order)) goto out; @@ -3783,6 +3787,20 @@ static void __ieee80211_unschedule_txq(struct ieee80211_hw *hw, RB_CLEAR_NODE(&txqi->schedule_order); } +void ieee80211_remove_txq(struct ieee80211_hw *hw, + struct ieee80211_txq *txq) +{ + struct ieee80211_local *local = hw_to_local(hw); + struct txq_info *txqi = to_txq_info(txq); + + lockdep_assert_held(&local->active_txq_lock[txq->ac]); + + if (!RB_EMPTY_NODE(&txqi->schedule_order)) { + __ieee80211_unschedule_txq(hw, txq); + list_del_init(&txqi->candidate); + } +} + void ieee80211_unschedule_txq(struct ieee80211_hw *hw, struct ieee80211_txq *txq) __acquires(txq_lock) __releases(txq_lock) @@ -3790,7 +3808,7 @@ void ieee80211_unschedule_txq(struct ieee80211_hw *hw, struct ieee80211_local *local = hw_to_local(hw); spin_lock_bh(&local->active_txq_lock[txq->ac]); - __ieee80211_unschedule_txq(hw, txq); + ieee80211_remove_txq(hw, txq); spin_unlock_bh(&local->active_txq_lock[txq->ac]); } @@ -3803,11 +3821,48 @@ void ieee80211_return_txq(struct ieee80211_hw *hw, lockdep_assert_held(&local->active_txq_lock[txq->ac]); if (!RB_EMPTY_NODE(&txqi->schedule_order) && - (skb_queue_empty(&txqi->frags) && !txqi->tin.backlog_packets)) - __ieee80211_unschedule_txq(hw, txq); + !txq_has_queue(&txqi->txq) && + list_empty(&txqi->candidate)) + list_add_tail(&txqi->candidate, &local->remove_list[txq->ac]); + } EXPORT_SYMBOL(ieee80211_return_txq); +void __ieee80211_check_txqs(struct ieee80211_local *local, int ac) +{ + struct txq_info *iter, *tmp; + struct sta_info *sta; + + lockdep_assert_held(&local->active_txq_lock[ac]); + + list_for_each_entry_safe(iter, tmp, &local->remove_list[ac], + candidate) { + sta = container_of(iter->txq.sta, struct sta_info, sta); + + if (txq_has_queue(&iter->txq)) + list_del_init(&iter->candidate); + else + ieee80211_remove_txq(&local->hw, &iter->txq); + } +} + +void ieee80211_txqs_check(struct timer_list *t) +{ + struct ieee80211_local *local = from_timer(local, t, remove_timer); + struct txq_info *iter, *tmp; + struct sta_info *sta; + int ac; + + for (ac = 0; ac < IEEE80211_NUM_ACS; ac++) { + spin_lock_bh(&local->active_txq_lock[ac]); + __ieee80211_check_txqs(local, ac); + spin_unlock_bh(&local->active_txq_lock[ac]); + } + + mod_timer(&local->remove_timer, + jiffies + msecs_to_jiffies(IEEE80211_AIRTIME_TXQ_RM_CHK_INTV_IN_MS)); +} + bool ieee80211_txq_may_transmit(struct ieee80211_hw *hw, struct ieee80211_txq *txq) { @@ -3841,6 +3896,8 @@ void ieee80211_txq_schedule_end(struct ieee80211_hw *hw, u8 ac) struct ieee80211_local *local = hw_to_local(hw); local->schedule_pos[ac] = NULL; + __ieee80211_check_txqs(local, ac); + spin_unlock_bh(&local->active_txq_lock[ac]); } EXPORT_SYMBOL(ieee80211_txq_schedule_end); From patchwork Mon Sep 16 13:09:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yibo Zhao X-Patchwork-Id: 11147025 X-Patchwork-Delegate: johannes@sipsolutions.net Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ECD1214DB for ; Mon, 16 Sep 2019 13:10:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CA77C214DE for ; Mon, 16 Sep 2019 13:10:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="key not found in DNS" (0-bit key) header.d=codeaurora.org header.i=@codeaurora.org header.b="eRSc9AkU"; dkim=fail reason="key not found in DNS" (0-bit key) header.d=codeaurora.org header.i=@codeaurora.org header.b="pN7h8XF8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728014AbfIPNKg (ORCPT ); Mon, 16 Sep 2019 09:10:36 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:44308 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728004AbfIPNKg (ORCPT ); Mon, 16 Sep 2019 09:10:36 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id C1DDD61528; Mon, 16 Sep 2019 13:10:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1568639434; bh=5ZC6jmwiMy7decVcU1F89bZb8lrRdZ+ifQlFcir8cq8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eRSc9AkUFFNKdZbxqnI6DWI0DdnM7iYGtLcjPkPnnG65I9DwjAk8mZAdRH5a195Wg VnXHMxVy4+Ihr+UerynjaQ8xIkkzZ7QYhlrrUun4JoaQeuAl/hM47rLpO1a97zCttc BA/5hvknP86RXOnfakR5Aspm9l34PxOjSDltPttA= X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on pdx-caf-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=ALL_TRUSTED,BAYES_00, DKIM_INVALID,DKIM_SIGNED,SPF_NONE autolearn=no autolearn_force=no version=3.4.0 Received: from ybzhao-HP-Z230-SFF-Workstation.ap.qualcomm.com (unknown [180.166.53.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: yiboz@codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id EDDF5611FA; Mon, 16 Sep 2019 13:10:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1568639433; bh=5ZC6jmwiMy7decVcU1F89bZb8lrRdZ+ifQlFcir8cq8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pN7h8XF8UrO7qWE3gKv16d9m80IWGVDqxwMmg/BEL+YyLEpD99X/bLmU/a6O+uCcT pJ3m0EY1bxAThNrxcuiBC6rwNrl77mG4bF7+JUsg1Yned5KyrrARMhbFKNfHyTJVwD s0YflXUnBaPt7W23A+cDz7RdEqqnmdTTg8f/oWjc= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org EDDF5611FA Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=yiboz@codeaurora.org From: Yibo Zhao To: ath10k@lists.infradead.org Cc: linux-wireless@vger.kernel.org, Yibo Zhao , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH 3/4] mac80211: fix low throughput in push pull mode Date: Mon, 16 Sep 2019 21:09:47 +0800 Message-Id: <1568639388-27291-3-git-send-email-yiboz@codeaurora.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1568639388-27291-1-git-send-email-yiboz@codeaurora.org> References: <1568639388-27291-1-git-send-email-yiboz@codeaurora.org> MIME-Version: 1.0 Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org If station is ineligible for transmission in ieee80211_txq_may_transmit(), no packet will be delivered to FW. During the tests in push-pull mode with many clients, after several seconds, not a single station is an eligible candidate for transmission since global time is smaller than all the station's virtual airtime. As a consequence, the Tx has been blocked and throughput is quite low. To avoid this situation to occur in push-pull mode, the new proposal is: - Increase the airtime grace period a little more to reduce the unexpected sync - If global virtual time is less than the virtual airtime of any station, sync it to the airtime of first station in the red-black tree - Round the division result since the process of global virtual time involves the division calculation Co-developed-by: Yibo Zhao Signed-off-by: Yibo Zhao Signed-off-by: Toke Høiland-Jørgensen Signed-off-by: Yibo Zhao Signed-off-by: Toke Høiland-Jørgensen Signed-off-by: B <...> Signed-off-by: A <...> --- net/mac80211/sta_info.c | 3 ++- net/mac80211/sta_info.h | 2 +- net/mac80211/tx.c | 16 +++++++++++++++- 3 files changed, 18 insertions(+), 3 deletions(-) diff --git a/net/mac80211/sta_info.c b/net/mac80211/sta_info.c index 9d01fdd..feac975 100644 --- a/net/mac80211/sta_info.c +++ b/net/mac80211/sta_info.c @@ -1852,7 +1852,8 @@ void ieee80211_sta_register_airtime(struct ieee80211_sta *pubsta, u8 tid, weight_sum = local->airtime_weight_sum[ac] ?: sta->airtime_weight; - local->airtime_v_t[ac] += airtime / weight_sum; + /* Round the calculation of global vt */ + local->airtime_v_t[ac] += (airtime + (weight_sum >> 1)) / weight_sum; sta->airtime[ac].v_t += airtime / sta->airtime_weight; ieee80211_resort_txq(&local->hw, txq); diff --git a/net/mac80211/sta_info.h b/net/mac80211/sta_info.h index 5c1cac9..5055f94 100644 --- a/net/mac80211/sta_info.h +++ b/net/mac80211/sta_info.h @@ -130,7 +130,7 @@ enum ieee80211_agg_stop_reason { /* Debugfs flags to enable/disable use of RX/TX airtime in scheduler */ #define AIRTIME_USE_TX BIT(0) #define AIRTIME_USE_RX BIT(1) -#define AIRTIME_GRACE 500 /* usec of grace period before reset */ +#define AIRTIME_GRACE 2000 /* usec of grace period before reset */ struct airtime_info { u64 rx_airtime; diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c index 42ca010..60cf569 100644 --- a/net/mac80211/tx.c +++ b/net/mac80211/tx.c @@ -3867,15 +3867,29 @@ bool ieee80211_txq_may_transmit(struct ieee80211_hw *hw, struct ieee80211_txq *txq) { struct ieee80211_local *local = hw_to_local(hw); - struct txq_info *txqi = to_txq_info(txq); + struct txq_info *first_txqi, *txqi = to_txq_info(txq); + struct rb_node *node = NULL; struct sta_info *sta; u8 ac = txq->ac; + first_txqi = NULL; lockdep_assert_held(&local->active_txq_lock[ac]); if (!txqi->txq.sta) return true; + node = rb_first_cached(&local->active_txqs[ac]); + if (node) { + first_txqi = container_of(node, struct txq_info, + schedule_order); + if (first_txqi->txq.sta) { + sta = container_of(first_txqi->txq.sta, + struct sta_info, sta); + if (local->airtime_v_t[ac] < sta->airtime[ac].v_t) + local->airtime_v_t[ac] = sta->airtime[ac].v_t; + } + } + sta = container_of(txqi->txq.sta, struct sta_info, sta); return (sta->airtime[ac].v_t <= local->airtime_v_t[ac]); } From patchwork Mon Sep 16 13:09:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yibo Zhao X-Patchwork-Id: 11147031 X-Patchwork-Delegate: johannes@sipsolutions.net Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D044D1746 for ; Mon, 16 Sep 2019 13:10:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A419A214DE for ; Mon, 16 Sep 2019 13:10:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="key not found in DNS" (0-bit key) header.d=codeaurora.org header.i=@codeaurora.org header.b="OfsHqwBY"; dkim=fail reason="key not found in DNS" (0-bit key) header.d=codeaurora.org header.i=@codeaurora.org header.b="QbpohOGB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732944AbfIPNKm (ORCPT ); Mon, 16 Sep 2019 09:10:42 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:44478 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728018AbfIPNKl (ORCPT ); Mon, 16 Sep 2019 09:10:41 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 467306156A; Mon, 16 Sep 2019 13:10:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1568639440; bh=2pcWV4LuaUy4I49QsvNxKMfZukjy0lVa2EcX3VOVBBc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OfsHqwBYeEf4U2A/2/IhB7dGDeRJBp0suZqcBu25Yvm2BidYMq3oF1zJdZQRrrbjp LTvHhUvQMMq6/nmsTntORj4HWRF4TZzLLdBD2BKnURHGLeeOyXGNhWOzJ1xEmt3eW4 9lwDg5+7JkOhQPwTEsV8uA8yidioUE2t8PRTPCu4= X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on pdx-caf-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=ALL_TRUSTED,BAYES_00, DKIM_INVALID,DKIM_SIGNED,SPF_NONE autolearn=no autolearn_force=no version=3.4.0 Received: from ybzhao-HP-Z230-SFF-Workstation.ap.qualcomm.com (unknown [180.166.53.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: yiboz@codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 3CDE9611FA; Mon, 16 Sep 2019 13:10:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1568639439; bh=2pcWV4LuaUy4I49QsvNxKMfZukjy0lVa2EcX3VOVBBc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QbpohOGBE5rewDBx1Di7W3AEwNRF6hRMbdN2eHECvsZLaLJ3bmCuMu6m++f8niiTn 2bH3xu33xX+kLwC3DBHnUnrMlmp4TTTR6ulUnpH/ACe1jBZWjBpySWlBeoVBOvH9hI tIY4QPuU03R38ijLvrM9+iotGOteUGToC+8a/cQ8= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 3CDE9611FA Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=yiboz@codeaurora.org From: Yibo Zhao To: ath10k@lists.infradead.org Cc: linux-wireless@vger.kernel.org, Yibo Zhao Subject: [PATCH 4/4] mac80211: Sync airtime weight sum with per AC synced sta airtime weight together Date: Mon, 16 Sep 2019 21:09:48 +0800 Message-Id: <1568639388-27291-4-git-send-email-yiboz@codeaurora.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1568639388-27291-1-git-send-email-yiboz@codeaurora.org> References: <1568639388-27291-1-git-send-email-yiboz@codeaurora.org> Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org Global airtime weight sum is updated only when txq is added/removed from rbtree. If upper layer configures sta weight during high load, airtime weight sum will not be updated since txq is most likely on the tree. It could a little late for upper layer to reconfigure sta weight when txq is already in the rbtree. And thus, incorrect airtime weight sum will lead to incorrect global virtual time calculation as well as global airtime weight sum overflow of airtime weight sum during txq removed. Hence, need to update airtime weight sum upon receiving event for configuring sta weight once sta's txq is on the rbtree. Besides, if airtime weight sum of ACs and sta weight is synced under the same per AC lock protection, there can be a very short window causing incorrct airtime weight sum calculation as below: active_txq_lock_VO . VO weight sum is syncd . sta airtime weight sum is synced . active_txq_unlock_VO . . . active_txq_lock_VI . VI weight sum is syncd . sta airtime weight sum active_txq_lock_BE active_txq_unlock_VI Remove txq and thus sum . is calculated with synced . sta airtime weight . active_txq_unlock_BE So introduce a per ac synced station airtime weight synced with per AC synced weight sum together. And the per-AC station airtime weight is used to calculate weight sum. Signed-off-by: Yibo Zhao --- net/mac80211/cfg.c | 27 +++++++++++++++++++++++++-- net/mac80211/sta_info.c | 6 ++++-- net/mac80211/sta_info.h | 3 +++ net/mac80211/tx.c | 4 ++-- 4 files changed, 34 insertions(+), 6 deletions(-) diff --git a/net/mac80211/cfg.c b/net/mac80211/cfg.c index d65aa01..4b420bb 100644 --- a/net/mac80211/cfg.c +++ b/net/mac80211/cfg.c @@ -1284,7 +1284,8 @@ static int sta_apply_parameters(struct ieee80211_local *local, int ret = 0; struct ieee80211_supported_band *sband; struct ieee80211_sub_if_data *sdata = sta->sdata; - u32 mask, set; + u32 mask, set, tid, ac; + struct txq_info *txqi; sband = ieee80211_get_sband(sdata); if (!sband) @@ -1452,8 +1453,30 @@ static int sta_apply_parameters(struct ieee80211_local *local, if (ieee80211_vif_is_mesh(&sdata->vif)) sta_apply_mesh_params(local, sta, params); - if (params->airtime_weight) + if (params->airtime_weight && + params->airtime_weight != sta->airtime_weight) { + for (ac = 0; ac < IEEE80211_NUM_ACS; ac++) { + spin_lock_bh(&local->active_txq_lock[ac]); + for (tid = 0; tid < IEEE80211_NUM_TIDS + 1; tid++) { + if (!sta->sta.txq[tid] || + ac != ieee80211_ac_from_tid(tid)) + continue; + + sta->airtime_weight_synced[ac] = + params->airtime_weight; + + txqi = to_txq_info(sta->sta.txq[tid]); + if (RB_EMPTY_NODE(&txqi->schedule_order)) + continue; + + local->airtime_weight_sum[ac] = local->airtime_weight_sum[ac] + + params->airtime_weight - + sta->airtime_weight; + } + spin_unlock_bh(&local->active_txq_lock[ac]); + } sta->airtime_weight = params->airtime_weight; + } /* set the STA state after all sta info from usermode has been set */ if (test_sta_flag(sta, WLAN_STA_TDLS_PEER) || diff --git a/net/mac80211/sta_info.c b/net/mac80211/sta_info.c index feac975..b00812f 100644 --- a/net/mac80211/sta_info.c +++ b/net/mac80211/sta_info.c @@ -389,6 +389,7 @@ struct sta_info *sta_info_alloc(struct ieee80211_sub_if_data *sdata, for (i = 0; i < IEEE80211_NUM_ACS; i++) { skb_queue_head_init(&sta->ps_tx_buf[i]); skb_queue_head_init(&sta->tx_filtered[i]); + sta->airtime_weight_synced[i] = sta->airtime_weight; } for (i = 0; i < IEEE80211_NUM_TIDS; i++) @@ -1850,11 +1851,12 @@ void ieee80211_sta_register_airtime(struct ieee80211_sta *pubsta, u8 tid, sta->airtime[ac].tx_airtime += tx_airtime; sta->airtime[ac].rx_airtime += rx_airtime; - weight_sum = local->airtime_weight_sum[ac] ?: sta->airtime_weight; + weight_sum = local->airtime_weight_sum[ac] ? + : sta->airtime_weight_synced[ac]; /* Round the calculation of global vt */ local->airtime_v_t[ac] += (airtime + (weight_sum >> 1)) / weight_sum; - sta->airtime[ac].v_t += airtime / sta->airtime_weight; + sta->airtime[ac].v_t += airtime / sta->airtime_weight_synced[ac]; ieee80211_resort_txq(&local->hw, txq); spin_unlock_bh(&local->active_txq_lock[ac]); diff --git a/net/mac80211/sta_info.h b/net/mac80211/sta_info.h index 5055f94..1298902 100644 --- a/net/mac80211/sta_info.h +++ b/net/mac80211/sta_info.h @@ -476,6 +476,8 @@ struct ieee80211_sta_rx_stats { * @tid_seq: per-TID sequence numbers for sending to this STA * @airtime: per-AC struct airtime_info describing airtime statistics for this * station + * @airtime_weight_synced: station per-AC airtime weight for sync and + * calculation * @airtime_weight: station weight for airtime fairness calculation purposes * @ampdu_mlme: A-MPDU state machine state * @mesh: mesh STA information @@ -602,6 +604,7 @@ struct sta_info { u16 tid_seq[IEEE80211_QOS_CTL_TID_MASK + 1]; struct airtime_info airtime[IEEE80211_NUM_ACS]; + u16 airtime_weight_synced[IEEE80211_NUM_ACS]; u16 airtime_weight; /* diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c index 60cf569..3592d49 100644 --- a/net/mac80211/tx.c +++ b/net/mac80211/tx.c @@ -3735,7 +3735,7 @@ void ieee80211_schedule_txq(struct ieee80211_hw *hw, struct sta_info *sta = container_of(txq->sta, struct sta_info, sta); - local->airtime_weight_sum[ac] += sta->airtime_weight; + local->airtime_weight_sum[ac] += sta->airtime_weight_synced[ac]; if (local->airtime_v_t[ac] > AIRTIME_GRACE) sta->airtime[ac].v_t = max(local->airtime_v_t[ac] - AIRTIME_GRACE, sta->airtime[ac].v_t); @@ -3779,7 +3779,7 @@ static void __ieee80211_unschedule_txq(struct ieee80211_hw *hw, struct sta_info *sta = container_of(txq->sta, struct sta_info, sta); - local->airtime_weight_sum[ac] -= sta->airtime_weight; + local->airtime_weight_sum[ac] -= sta->airtime_weight_synced[ac]; } rb_erase_cached(&txqi->schedule_order,