From patchwork Thu May 20 03:46:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Wang X-Patchwork-Id: 12268901 X-Patchwork-Delegate: nbd@nbd.name Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0C21C433ED for ; Thu, 20 May 2021 03:46:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CB42460E0C for ; Thu, 20 May 2021 03:46:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230298AbhETDsM (ORCPT ); Wed, 19 May 2021 23:48:12 -0400 Received: from mailgw02.mediatek.com ([210.61.82.184]:44331 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S229554AbhETDsL (ORCPT ); Wed, 19 May 2021 23:48:11 -0400 X-UUID: 646cb2251ae7492298d48e1015fafb46-20210520 X-UUID: 646cb2251ae7492298d48e1015fafb46-20210520 Received: from mtkcas07.mediatek.inc [(172.21.101.84)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 103525911; Thu, 20 May 2021 11:46:47 +0800 Received: from mtkcas07.mediatek.inc (172.21.101.84) by mtkmbs01n1.mediatek.inc (172.21.101.68) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 20 May 2021 11:46:46 +0800 Received: from mtkswgap22.mediatek.inc (172.21.77.33) by mtkcas07.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 20 May 2021 11:46:46 +0800 From: To: , CC: , , , , , , , , , , , , , , , , Deren Wu , Leon Yen , YN Chen Subject: [PATCH 2/7] mt76: connac: update BA win size in Rx direction Date: Thu, 20 May 2021 11:46:36 +0800 Message-ID: <1621482401-29025-2-git-send-email-sean.wang@mediatek.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1621482401-29025-1-git-send-email-sean.wang@mediatek.com> References: <1621482401-29025-1-git-send-email-sean.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org From: Deren Wu Update BA size used data transimission in the Rx direction to improve Rx throughput. Reviewed-by: Lorenzo Bianconi Signed-off-by: Deren Wu Signed-off-by: Leon Yen Signed-off-by: YN Chen Signed-off-by: Sean Wang --- drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c index e0d6bdadf5b1..73359defa176 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c @@ -931,8 +931,10 @@ void mt76_connac_mcu_wtbl_ba_tlv(struct mt76_dev *dev, struct sk_buff *skb, ba->rst_ba_sb = 1; } - if (is_mt7921(dev)) + if (is_mt7921(dev)) { + ba->ba_winsize = enable ? cpu_to_le16(params->buf_size) : 0; return; + } if (enable && tx) { u8 ba_range[] = { 4, 8, 12, 24, 36, 48, 54, 64 }; From patchwork Thu May 20 03:46:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Wang X-Patchwork-Id: 12268903 X-Patchwork-Delegate: nbd@nbd.name Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 215C6C433B4 for ; Thu, 20 May 2021 03:46:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ED73F60E0C for ; Thu, 20 May 2021 03:46:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230330AbhETDsP (ORCPT ); Wed, 19 May 2021 23:48:15 -0400 Received: from mailgw02.mediatek.com ([210.61.82.184]:44371 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S229554AbhETDsO (ORCPT ); Wed, 19 May 2021 23:48:14 -0400 X-UUID: 9e049adfae294f568a8797ea30dfcafa-20210520 X-UUID: 9e049adfae294f568a8797ea30dfcafa-20210520 Received: from mtkcas11.mediatek.inc [(172.21.101.40)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 1427075522; Thu, 20 May 2021 11:46:50 +0800 Received: from mtkcas07.mediatek.inc (172.21.101.84) by mtkmbs02n1.mediatek.inc (172.21.101.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 20 May 2021 11:46:48 +0800 Received: from mtkswgap22.mediatek.inc (172.21.77.33) by mtkcas07.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 20 May 2021 11:46:48 +0800 From: To: , CC: , , , , , , , , , , , , , , , , Lorenzo Bianconi Subject: [PATCH 3/7] mt76: mt7921: fix reset under the deep sleep is enabled Date: Thu, 20 May 2021 11:46:37 +0800 Message-ID: <1621482401-29025-3-git-send-email-sean.wang@mediatek.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1621482401-29025-1-git-send-email-sean.wang@mediatek.com> References: <1621482401-29025-1-git-send-email-sean.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org From: Sean Wang To fix possibly the race to access register between the WiFi reset and the other context that is caused by explicitly cancelling ps_work and wake_work to break PM_STATE consistency. Deep sleep would cause the hardware into the inactive state, so we forcely put device drv_own state before we start to reset. The patch also ignore the reset request when the procedure is in progress to avoid the consecutive WiFi resets. localhost ~ # [ 2932.073966] SError Interrupt on CPU7, code 0xbe000011 [ 2932.073967] CPU: 7 PID: 8761 Comm: kworker/u16:2 Not tainted 5.4.112 #30 [ 2932.073968] Hardware name: MediaTek Asurada rev1 board (DT) [ 2932.073968] Workqueue: phy0 ieee80211_reconfig_filter [mac80211] [ 2932.073969] pstate: 80400089 (Nzcv daIf +PAN -UAO) [ 2932.073969] pc : el1_irq+0x78/0x180 [ 2932.073970] lr : mt76_mmio_rmw+0x30/0x5c [mt76] [ 2932.073970] sp : ffffffc01142bad0 [ 2932.073970] x29: ffffffc01142bc00 x28: ffffff8f96fb1e00 [ 2932.073971] x27: ffffffd2cdc12138 x26: ffffffd2cdaeb018 [ 2932.073972] x25: 0000000000000000 x24: ffffff8fa8e14c08 [ 2932.073973] x23: 0000000080c00009 x22: ffffffd2a5603918 [ 2932.073974] x21: ffffffc01142bc10 x20: 0000007fffffffff [ 2932.073975] x19: 0000000000000000 x18: 0000000000000400 [ 2932.073975] x17: 0000000000000400 x16: ffffffd2cd2b87dc [ 2932.073976] x15: 0000000000000000 x14: 0000000000000000 [ 2932.073977] x13: 0000000000000001 x12: 0000000000000001 [ 2932.073978] x11: 0000000000000001 x10: 000000000010e000 [ 2932.073978] x9 : 0000000000000000 x8 : ffffffc013921404 [ 2932.073979] x7 : 000000b2b5593519 x6 : 0000000000300000 [ 2932.073980] x5 : 0000000000000000 x4 : ffffffc01142bbc8 [ 2932.073980] x3 : 00000000000001f0 x2 : 0000000000000000 [ 2932.073981] x1 : 0000000000021404 x0 : ffffff8fa8e12300 [ 2932.073982] Kernel panic - not syncing: Asynchronous SError Interrupt [ 2932.073983] CPU: 7 PID: 8761 Comm: kworker/u16:2 Not tainted 5.4.112 #30 [ 2932.073983] Hardware name: MediaTek Asurada rev1 board (DT) [ 2932.073984] Workqueue: phy0 ieee80211_reconfig_filter [mac80211] [ 2932.073984] Call trace: [ 2932.073985] dump_backtrace+0x0/0x14c [ 2932.073985] show_stack+0x20/0x2c [ 2932.073985] dump_stack+0xa0/0xf8 [ 2932.073986] panic+0x154/0x360 [ 2932.073986] test_taint+0x0/0x44 [ 2932.073986] arm64_serror_panic+0x78/0x84 [ 2932.073987] do_serror+0x0/0x118 [ 2932.073987] do_serror+0xa4/0x118 [ 2932.073987] el1_error+0x84/0xf8 [ 2932.073988] el1_irq+0x78/0x180 [ 2932.073988] mt76_mmio_rr+0x30/0xf0 [mt76] [ 2932.073988] mt76_mmio_rmw+0x30/0x5c [mt76] [ 2932.073989] mt7921_rmw+0x4c/0x5c [mt7921e] [ 2932.073989] mt7921_configure_filter+0x138/0x160 [mt7921e] [ 2932.073990] ieee80211_configure_filter+0x2f0/0x3e0 [mac80211] [ 2932.073990] ieee80211_reconfig_filter+0x1c/0x28 [mac80211] [ 2932.073990] process_one_work+0x208/0x3c8 [ 2932.073991] worker_thread+0x23c/0x3e8 [ 2932.073991] kthread+0x140/0x17c [ 2932.073992] ret_from_fork+0x10/0x18 [ 2932.074071] SMP: stopping secondary CPUs [ 2932.074071] Kernel Offset: 0x12bc800000 from 0xffffffc010000000 [ 2932.074072] PHYS_OFFSET: 0xfffffff180000000 [ 2932.074072] CPU features: 0x080026,2a80aa18 [ 2932.074072] Memory Limit: none Fixes: 87843566e581 ("mt76: mt7921: enable deep sleep at runtime") Co-developed-by: Lorenzo Bianconi Signed-off-by: Lorenzo Bianconi Signed-off-by: Sean Wang --- .../net/wireless/mediatek/mt76/mt7921/mac.c | 21 +++++++++-------- .../net/wireless/mediatek/mt76/mt7921/mcu.c | 23 ++++++++++++++----- .../wireless/mediatek/mt76/mt7921/mt7921.h | 1 + 3 files changed, 30 insertions(+), 15 deletions(-) diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c index e29d4fdab572..ebd365ceb70d 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c @@ -1284,9 +1284,10 @@ mt7921_mac_reset(struct mt7921_dev *dev) mt76_worker_enable(&dev->mt76.tx_worker); clear_bit(MT76_MCU_RESET, &dev->mphy.state); - clear_bit(MT76_STATE_PM, &dev->mphy.state); - mt76_wr(dev, MT_WFDMA0_HOST_INT_ENA, 0); + mt76_wr(dev, MT_WFDMA0_HOST_INT_ENA, + MT_INT_RX_DONE_ALL | MT_INT_TX_DONE_ALL | + MT_INT_MCU_CMD); mt76_wr(dev, MT_PCIE_MAC_INT_ENABLE, 0xff); err = mt7921_run_firmware(dev); @@ -1304,22 +1305,23 @@ mt7921_mac_reset(struct mt7921_dev *dev) /* system error recovery */ void mt7921_mac_reset_work(struct work_struct *work) { - struct ieee80211_hw *hw; - struct mt7921_dev *dev; + struct mt7921_dev *dev = container_of(work, struct mt7921_dev, + reset_work); + struct ieee80211_hw *hw = mt76_hw(dev); + struct mt76_connac_pm *pm = &dev->pm; int i; - dev = container_of(work, struct mt7921_dev, reset_work); - hw = mt76_hw(dev); - dev_err(dev->mt76.dev, "chip reset\n"); ieee80211_stop_queues(hw); cancel_delayed_work_sync(&dev->mphy.mac_work); - cancel_delayed_work_sync(&dev->pm.ps_work); - cancel_work_sync(&dev->pm.wake_work); + cancel_delayed_work_sync(&pm->ps_work); + cancel_work_sync(&pm->wake_work); mutex_lock(&dev->mt76.mutex); for (i = 0; i < 10; i++) { + __mt7921_mcu_drv_pmctrl(dev); + if (!mt7921_mac_reset(dev)) break; } @@ -1340,6 +1342,7 @@ void mt7921_mac_reset_work(struct work_struct *work) ieee80211_iterate_active_interfaces(hw, IEEE80211_IFACE_ITER_RESUME_ALL, mt7921_vif_connect_iter, NULL); + mt76_connac_power_save_sched(&dev->mt76.phy, pm); } void mt7921_reset(struct mt76_dev *mdev) diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c index 5a4c695f73c9..486e5593d99a 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c @@ -1296,17 +1296,12 @@ int mt7921_mcu_sta_add(struct mt7921_dev *dev, struct ieee80211_sta *sta, return mt76_connac_mcu_add_sta_cmd(&dev->mphy, &info); } -int mt7921_mcu_drv_pmctrl(struct mt7921_dev *dev) +int __mt7921_mcu_drv_pmctrl(struct mt7921_dev *dev) { struct mt76_phy *mphy = &dev->mt76.phy; struct mt76_connac_pm *pm = &dev->pm; int i, err = 0; - mutex_lock(&pm->mutex); - - if (!test_bit(MT76_STATE_PM, &mphy->state)) - goto out; - for (i = 0; i < MT7921_DRV_OWN_RETRY_COUNT; i++) { mt76_wr(dev, MT_CONN_ON_LPCTL, PCIE_LPCR_HOST_CLR_OWN); if (mt76_poll_msec(dev, MT_CONN_ON_LPCTL, @@ -1326,6 +1321,22 @@ int mt7921_mcu_drv_pmctrl(struct mt7921_dev *dev) pm->stats.last_wake_event = jiffies; pm->stats.doze_time += pm->stats.last_wake_event - pm->stats.last_doze_event; +out: + return err; +} + +int mt7921_mcu_drv_pmctrl(struct mt7921_dev *dev) +{ + struct mt76_phy *mphy = &dev->mt76.phy; + struct mt76_connac_pm *pm = &dev->pm; + int err = 0; + + mutex_lock(&pm->mutex); + + if (!test_bit(MT76_STATE_PM, &mphy->state)) + goto out; + + err = __mt7921_mcu_drv_pmctrl(dev); out: mutex_unlock(&pm->mutex); diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h b/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h index 59862ea4951c..03bcb210c357 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h +++ b/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h @@ -368,6 +368,7 @@ int mt7921_mcu_uni_bss_bcnft(struct mt7921_dev *dev, struct ieee80211_vif *vif, bool enable); int mt7921_mcu_set_bss_pm(struct mt7921_dev *dev, struct ieee80211_vif *vif, bool enable); +int __mt7921_mcu_drv_pmctrl(struct mt7921_dev *dev); int mt7921_mcu_drv_pmctrl(struct mt7921_dev *dev); int mt7921_mcu_fw_pmctrl(struct mt7921_dev *dev); void mt7921_pm_wake_work(struct work_struct *work); From patchwork Thu May 20 03:46:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Wang X-Patchwork-Id: 12268905 X-Patchwork-Delegate: nbd@nbd.name Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1880C433ED for ; Thu, 20 May 2021 03:46:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A25186108D for ; Thu, 20 May 2021 03:46:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230338AbhETDsS (ORCPT ); Wed, 19 May 2021 23:48:18 -0400 Received: from mailgw01.mediatek.com ([210.61.82.183]:41011 "EHLO mailgw01.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S229554AbhETDsS (ORCPT ); Wed, 19 May 2021 23:48:18 -0400 X-UUID: f836aa21dde044bb930e82c11c20b709-20210520 X-UUID: f836aa21dde044bb930e82c11c20b709-20210520 Received: from mtkcas06.mediatek.inc [(172.21.101.30)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 397042348; Thu, 20 May 2021 11:46:53 +0800 Received: from mtkcas07.mediatek.inc (172.21.101.84) by mtkmbs06n1.mediatek.inc (172.21.101.129) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 20 May 2021 11:46:52 +0800 Received: from mtkswgap22.mediatek.inc (172.21.77.33) by mtkcas07.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 20 May 2021 11:46:52 +0800 From: To: , CC: , , , , , , , , , , , , , , , , Lorenzo Bianconi Subject: [PATCH 4/7] mt76: mt7921: avoid unnecessary consecutive WiFi resets Date: Thu, 20 May 2021 11:46:38 +0800 Message-ID: <1621482401-29025-4-git-send-email-sean.wang@mediatek.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1621482401-29025-1-git-send-email-sean.wang@mediatek.com> References: <1621482401-29025-1-git-send-email-sean.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org From: Sean Wang Avoid unnecessary consecutive WiFi resets by dropping reset request when reset work is working. Co-developed-by: Lorenzo Bianconi Signed-off-by: Lorenzo Bianconi Signed-off-by: Sean Wang --- drivers/net/wireless/mediatek/mt76/mt7921/mac.c | 5 ++++- drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c index ebd365ceb70d..fdc69f306f7e 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c @@ -1312,6 +1312,7 @@ void mt7921_mac_reset_work(struct work_struct *work) int i; dev_err(dev->mt76.dev, "chip reset\n"); + dev->hw_full_reset = true; ieee80211_stop_queues(hw); cancel_delayed_work_sync(&dev->mphy.mac_work); @@ -1338,6 +1339,7 @@ void mt7921_mac_reset_work(struct work_struct *work) ieee80211_scan_completed(dev->mphy.hw, &info); } + dev->hw_full_reset = false; ieee80211_wake_queues(hw); ieee80211_iterate_active_interfaces(hw, IEEE80211_IFACE_ITER_RESUME_ALL, @@ -1349,7 +1351,8 @@ void mt7921_reset(struct mt76_dev *mdev) { struct mt7921_dev *dev = container_of(mdev, struct mt7921_dev, mt76); - queue_work(dev->mt76.wq, &dev->reset_work); + if (!dev->hw_full_reset) + queue_work(dev->mt76.wq, &dev->reset_work); } static void diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h b/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h index 03bcb210c357..710ad242fd53 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h +++ b/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h @@ -156,6 +156,7 @@ struct mt7921_dev { u16 chainmask; struct work_struct reset_work; + bool hw_full_reset; struct list_head sta_poll_list; spinlock_t sta_poll_lock; From patchwork Thu May 20 03:46:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Wang X-Patchwork-Id: 12268907 X-Patchwork-Delegate: nbd@nbd.name Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20BB1C433B4 for ; Thu, 20 May 2021 03:47:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EFE95611AE for ; Thu, 20 May 2021 03:47:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230345AbhETDsW (ORCPT ); Wed, 19 May 2021 23:48:22 -0400 Received: from mailgw02.mediatek.com ([210.61.82.184]:44451 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S229554AbhETDsU (ORCPT ); Wed, 19 May 2021 23:48:20 -0400 X-UUID: 0e109272833742f7aac13c37bc65eb3e-20210520 X-UUID: 0e109272833742f7aac13c37bc65eb3e-20210520 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 324472826; Thu, 20 May 2021 11:46:56 +0800 Received: from mtkcas07.mediatek.inc (172.21.101.84) by mtkmbs06n2.mediatek.inc (172.21.101.130) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 20 May 2021 11:46:55 +0800 Received: from mtkswgap22.mediatek.inc (172.21.77.33) by mtkcas07.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 20 May 2021 11:46:54 +0800 From: To: , CC: , , , , , , , , , , , , , , , , Lorenzo Bianconi Subject: [PATCH 5/7] mt76: mt7921: fix invalid register access in wake_work Date: Thu, 20 May 2021 11:46:39 +0800 Message-ID: <1621482401-29025-5-git-send-email-sean.wang@mediatek.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1621482401-29025-1-git-send-email-sean.wang@mediatek.com> References: <1621482401-29025-1-git-send-email-sean.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org From: Sean Wang Make sure mt7921_pm_wake_work wouldn't be scheduled after the driver is in suspend mode to fix the following the kernel crash. [ 3515.390012] mt7921e 0000:01:00.0: calling pci_pm_suspend+0x0/0x22c @ 2869, parent: 0000:00:00.0 [ 3515.390015] mt7921e 0000:01:00.0: mt7921_pci_suspend + [ 3515.396395] anx7625 3-0058: anx7625_suspend+0x0/0x6c returned 0 after 0 usecs [ 3515.405965] mt7921e 0000:01:00.0: mt7921_pci_suspend - [ 3515.411336] usb 1-1.4: usb_dev_suspend+0x0/0x2c returned 0 after 1 usecs [ 3515.411513] SError Interrupt on CPU7, code 0xbe000011 -- SError [ 3515.411515] CPU: 7 PID: 2849 Comm: kworker/u16:27 Not tainted 5.4.114 #44 [ 3515.411516] Hardware name: MediaTek Asurada rev1 board (DT) [ 3515.411517] Workqueue: mt76 mt7921_pm_wake_work [mt7921e] [ 3515.411518] pstate: 80c00009 (Nzcv daif +PAN +UAO) [ 3515.411519] pc : mt76_mmio_rr+0x30/0xf0 [mt76] [ 3515.411520] lr : mt7921_rr+0x38/0x44 [mt7921e] [ 3515.411520] sp : ffffffc015813c50 [ 3515.411521] x29: ffffffc015813c50 x28: 0000000000000402 [ 3515.411522] x27: ffffffe5a2012138 x26: ffffffe5a1eea018 [ 3515.411524] x25: 00000000328be505 x24: 00000000000a0002 [ 3515.411525] x23: 0000000000000006 x22: ffffffbd29b7a300 [ 3515.411527] x21: ffffffbd29b7a300 x20: 00000000000e0010 [ 3515.411528] x19: 00000000eac08f43 x18: 0000000000000000 [ 3515.411529] x17: 0000000000000000 x16: ffffffe5a16b2914 [ 3515.411531] x15: 0000000000000010 x14: 0000000000000010 [ 3515.411532] x13: 00000000003dd3a2 x12: 0000000000010000 [ 3515.411533] x11: ffffffe597abec14 x10: 0000000000000010 [ 3515.411535] x9 : ffffffe597abeba8 x8 : ffffffc013ce0010 [ 3515.411536] x7 : 000000b2b5593519 x6 : 0000000000300000 [ 3515.411537] x5 : 0000000000000000 x4 : 0000000000000032 [ 3515.411539] x3 : 0000000000000000 x2 : 0000000000000004 [ 3515.411540] x1 : 00000000000e0010 x0 : ffffffbd29b7a300 [ 3515.411542] Kernel panic - not syncing: Asynchronous SError Interrupt [ 3515.411543] CPU: 7 PID: 2849 Comm: kworker/u16:27 Not tainted 5.4.114 #44 [ 3515.411544] Hardware name: MediaTek Asurada rev1 board (DT) [ 3515.411544] Workqueue: mt76 mt7921_pm_wake_work [mt7921e] [ 3515.411545] Call trace: [ 3515.411546] dump_backtrace+0x0/0x14c [ 3515.411546] show_stack+0x20/0x2c [ 3515.411547] dump_stack+0xa0/0xfc [ 3515.411548] panic+0x154/0x350 [ 3515.411548] panic+0x0/0x350 [ 3515.411549] arm64_serror_panic+0x78/0x84 [ 3515.411550] do_serror+0x0/0x118 [ 3515.411550] do_serror+0xa4/0x118 [ 3515.411551] el1_error+0x84/0xf8 [ 3515.411552] mt76_mmio_rr+0x30/0xf0 [mt76] [ 3515.411552] mt7921_rr+0x38/0x44 [mt7921e] [ 3515.411553] __mt76_poll_msec+0x5c/0x9c [mt76] [ 3515.411554] __mt7921_mcu_drv_pmctrl+0x50/0x94 [mt7921e] [ 3515.411555] mt7921_mcu_drv_pmctrl+0x38/0xb0 [mt7921e] [ 3515.411555] mt7921_pm_wake_work+0x34/0xd4 [mt7921e] [ 3515.411556] process_one_work+0x208/0x3c8 [ 3515.411557] worker_thread+0x23c/0x3e8 [ 3515.411557] kthread+0x144/0x178 [ 3515.411558] ret_from_fork+0x10/0x18 [ 3515.418831] SMP: stopping secondary CPUs [ 3515.418832] Kernel Offset: 0x2590c00000 from 0xffffffc010000000 [ 3515.418832] PHYS_OFFSET: 0xffffffc400000000 [ 3515.418833] CPU features: 0x080026,2a80aa18 [ 3515.418834] Memory Limit: none [DL] 00000000 00000000 010701 Fixes: 1d8efc741df80 ("mt76: mt7921: introduce Runtime PM support") Co-developed-by: Lorenzo Bianconi Signed-off-by: Lorenzo Bianconi Signed-off-by: Sean Wang --- .../net/wireless/mediatek/mt76/mt76_connac.h | 1 + .../wireless/mediatek/mt76/mt76_connac_mac.c | 6 +++++ .../net/wireless/mediatek/mt76/mt7921/pci.c | 25 +++++++++++++------ 3 files changed, 24 insertions(+), 8 deletions(-) diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac.h b/drivers/net/wireless/mediatek/mt76/mt76_connac.h index 337c5ece7ec3..63c1d1a68a70 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76_connac.h +++ b/drivers/net/wireless/mediatek/mt76/mt76_connac.h @@ -45,6 +45,7 @@ enum { struct mt76_connac_pm { bool enable; + bool suspended; spinlock_t txq_lock; struct { diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c b/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c index 6f180c92d413..5f2705fbd680 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c @@ -17,6 +17,9 @@ int mt76_connac_pm_wake(struct mt76_phy *phy, struct mt76_connac_pm *pm) if (!test_bit(MT76_STATE_PM, &phy->state)) return 0; + if (pm->suspended) + return 0; + queue_work(dev->wq, &pm->wake_work); if (!wait_event_timeout(pm->wait, !test_bit(MT76_STATE_PM, &phy->state), @@ -40,6 +43,9 @@ void mt76_connac_power_save_sched(struct mt76_phy *phy, if (!pm->enable) return; + if (pm->suspended) + return; + pm->last_activity = jiffies; if (!test_bit(MT76_STATE_PM, &phy->state)) { diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/pci.c b/drivers/net/wireless/mediatek/mt76/mt7921/pci.c index fa02d934f0bf..13263f50dc00 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/pci.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/pci.c @@ -188,21 +188,26 @@ static int mt7921_pci_suspend(struct pci_dev *pdev, pm_message_t state) { struct mt76_dev *mdev = pci_get_drvdata(pdev); struct mt7921_dev *dev = container_of(mdev, struct mt7921_dev, mt76); + struct mt76_connac_pm *pm = &dev->pm; bool hif_suspend; int i, err; - err = mt76_connac_pm_wake(&dev->mphy, &dev->pm); + pm->suspended = true; + cancel_delayed_work_sync(&pm->ps_work); + cancel_work_sync(&pm->wake_work); + + err = mt7921_mcu_drv_pmctrl(dev); if (err < 0) - return err; + goto restore_suspend; hif_suspend = !test_bit(MT76_STATE_SUSPEND, &dev->mphy.state); if (hif_suspend) { err = mt76_connac_mcu_set_hif_suspend(mdev, true); if (err) - return err; + goto restore_suspend; } - if (!dev->pm.enable) + if (!pm->enable) mt76_connac_mcu_set_deep_sleep(&dev->mt76, true); napi_disable(&mdev->tx_napi); @@ -231,27 +236,30 @@ static int mt7921_pci_suspend(struct pci_dev *pdev, pm_message_t state) err = mt7921_mcu_fw_pmctrl(dev); if (err) - goto restore; + goto restore_napi; pci_save_state(pdev); err = pci_set_power_state(pdev, pci_choose_state(pdev, state)); if (err) - goto restore; + goto restore_napi; return 0; -restore: +restore_napi: mt76_for_each_q_rx(mdev, i) { napi_enable(&mdev->napi[i]); } napi_enable(&mdev->tx_napi); - if (!dev->pm.enable) + if (!pm->enable) mt76_connac_mcu_set_deep_sleep(&dev->mt76, false); if (hif_suspend) mt76_connac_mcu_set_hif_suspend(mdev, false); +restore_suspend: + pm->suspended = false; + return err; } @@ -261,6 +269,7 @@ static int mt7921_pci_resume(struct pci_dev *pdev) struct mt7921_dev *dev = container_of(mdev, struct mt7921_dev, mt76); int i, err; + dev->pm.suspended = false; err = pci_set_power_state(pdev, PCI_D0); if (err) return err; From patchwork Thu May 20 03:46:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Wang X-Patchwork-Id: 12268909 X-Patchwork-Delegate: nbd@nbd.name Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42CD7C43461 for ; Thu, 20 May 2021 03:47:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 25FBE6108D for ; Thu, 20 May 2021 03:47:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230315AbhETDsY (ORCPT ); Wed, 19 May 2021 23:48:24 -0400 Received: from mailgw01.mediatek.com ([210.61.82.183]:41095 "EHLO mailgw01.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S230339AbhETDsY (ORCPT ); Wed, 19 May 2021 23:48:24 -0400 X-UUID: 6dafb413679a4ccc8ee2a8e4375bc2db-20210520 X-UUID: 6dafb413679a4ccc8ee2a8e4375bc2db-20210520 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1803936245; Thu, 20 May 2021 11:47:01 +0800 Received: from mtkcas07.mediatek.inc (172.21.101.84) by mtkmbs08n1.mediatek.inc (172.21.101.55) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 20 May 2021 11:46:57 +0800 Received: from mtkswgap22.mediatek.inc (172.21.77.33) by mtkcas07.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 20 May 2021 11:46:57 +0800 From: To: , CC: , , , , , , , , , , , , , , , , Deren Wu , YN Chen Subject: [PATCH 6/7] mt76: mt7921: fix OMAC idx usage Date: Thu, 20 May 2021 11:46:40 +0800 Message-ID: <1621482401-29025-6-git-send-email-sean.wang@mediatek.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1621482401-29025-1-git-send-email-sean.wang@mediatek.com> References: <1621482401-29025-1-git-send-email-sean.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org From: Sean Wang OMAC idx have to be same with BSS idx according to firmware usage. Fixes: e0f9fdda81bd ("mt76: mt7921: add ieee80211_ops") Reviewed-by: Lorenzo Bianconi Signed-off-by: Deren Wu Signed-off-by: YN Chen Signed-off-by: Sean Wang --- .../net/wireless/mediatek/mt76/mt7921/main.c | 55 +------------------ 1 file changed, 1 insertion(+), 54 deletions(-) diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/main.c b/drivers/net/wireless/mediatek/mt76/mt7921/main.c index 5dbccbefe047..472afd4ea487 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/main.c @@ -224,54 +224,6 @@ static void mt7921_stop(struct ieee80211_hw *hw) mt7921_mutex_release(dev); } -static inline int get_free_idx(u32 mask, u8 start, u8 end) -{ - return ffs(~mask & GENMASK(end, start)); -} - -static int get_omac_idx(enum nl80211_iftype type, u64 mask) -{ - int i; - - switch (type) { - case NL80211_IFTYPE_STATION: - /* prefer hw bssid slot 1-3 */ - i = get_free_idx(mask, HW_BSSID_1, HW_BSSID_3); - if (i) - return i - 1; - - /* next, try to find a free repeater entry for the sta */ - i = get_free_idx(mask >> REPEATER_BSSID_START, 0, - REPEATER_BSSID_MAX - REPEATER_BSSID_START); - if (i) - return i + 32 - 1; - - i = get_free_idx(mask, EXT_BSSID_1, EXT_BSSID_MAX); - if (i) - return i - 1; - - if (~mask & BIT(HW_BSSID_0)) - return HW_BSSID_0; - - break; - case NL80211_IFTYPE_MONITOR: - /* ap uses hw bssid 0 and ext bssid */ - if (~mask & BIT(HW_BSSID_0)) - return HW_BSSID_0; - - i = get_free_idx(mask, EXT_BSSID_1, EXT_BSSID_MAX); - if (i) - return i - 1; - - break; - default: - WARN_ON(1); - break; - } - - return -1; -} - static int mt7921_add_interface(struct ieee80211_hw *hw, struct ieee80211_vif *vif) { @@ -293,12 +245,7 @@ static int mt7921_add_interface(struct ieee80211_hw *hw, goto out; } - idx = get_omac_idx(vif->type, phy->omac_mask); - if (idx < 0) { - ret = -ENOSPC; - goto out; - } - mvif->mt76.omac_idx = idx; + mvif->mt76.omac_idx = mvif->mt76.idx; mvif->phy = phy; mvif->mt76.band_idx = 0; mvif->mt76.wmm_idx = mvif->mt76.idx % MT7921_MAX_WMM_SETS; From patchwork Thu May 20 03:46:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Wang X-Patchwork-Id: 12268911 X-Patchwork-Delegate: nbd@nbd.name Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B228BC433ED for ; Thu, 20 May 2021 03:47:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 929336108D for ; Thu, 20 May 2021 03:47:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230321AbhETDsa (ORCPT ); Wed, 19 May 2021 23:48:30 -0400 Received: from mailgw01.mediatek.com ([210.61.82.183]:41145 "EHLO mailgw01.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S230232AbhETDsa (ORCPT ); Wed, 19 May 2021 23:48:30 -0400 X-UUID: 344d2a690b5b467f9f35405b00fbae78-20210520 X-UUID: 344d2a690b5b467f9f35405b00fbae78-20210520 Received: from mtkcas10.mediatek.inc [(172.21.101.39)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 132827703; Thu, 20 May 2021 11:47:06 +0800 Received: from mtkcas07.mediatek.inc (172.21.101.84) by mtkmbs01n2.mediatek.inc (172.21.101.79) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 20 May 2021 11:47:04 +0800 Received: from mtkswgap22.mediatek.inc (172.21.77.33) by mtkcas07.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 20 May 2021 11:47:04 +0800 From: To: , CC: , , , , , , , , , , , , , , , , Lorenzo Bianconi Subject: [PATCH 7/7] mt76: mt7921: enable runtime pm by default Date: Thu, 20 May 2021 11:46:41 +0800 Message-ID: <1621482401-29025-7-git-send-email-sean.wang@mediatek.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1621482401-29025-1-git-send-email-sean.wang@mediatek.com> References: <1621482401-29025-1-git-send-email-sean.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org From: Lorenzo Bianconi mt7921 is mainly used in CE/IoT market so enable runtime-pm by default Signed-off-by: Lorenzo Bianconi Signed-off-by: Sean Wang --- drivers/net/wireless/mediatek/mt76/mt7921/init.c | 2 +- drivers/net/wireless/mediatek/mt76/mt7921/mcu.c | 2 -- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/init.c b/drivers/net/wireless/mediatek/mt76/mt7921/init.c index d78eb4ec4b8a..7f5589cd8213 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/init.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/init.c @@ -185,7 +185,6 @@ int mt7921_register_device(struct mt7921_dev *dev) mutex_init(&dev->pm.mutex); init_waitqueue_head(&dev->pm.wait); spin_lock_init(&dev->pm.txq_lock); - set_bit(MT76_STATE_PM, &dev->mphy.state); INIT_LIST_HEAD(&dev->phy.stats_list); INIT_DELAYED_WORK(&dev->mphy.mac_work, mt7921_mac_work); INIT_DELAYED_WORK(&dev->phy.scan_work, mt7921_scan_work); @@ -200,6 +199,7 @@ int mt7921_register_device(struct mt7921_dev *dev) dev->pm.idle_timeout = MT7921_PM_TIMEOUT; dev->pm.stats.last_wake_event = jiffies; dev->pm.stats.last_doze_event = jiffies; + dev->pm.enable = true; ret = mt7921_init_hardware(dev); if (ret) diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c index 486e5593d99a..b7957f4cc321 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c @@ -942,8 +942,6 @@ static int mt7921_load_firmware(struct mt7921_dev *dev) dev->mt76.hw->wiphy->wowlan = &mt76_connac_wowlan_support; #endif /* CONFIG_PM */ - clear_bit(MT76_STATE_PM, &dev->mphy.state); - dev_err(dev->mt76.dev, "Firmware init done\n"); return 0;