From patchwork Wed Jan 8 03:19:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gong X-Patchwork-Id: 11322587 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E9DF0138C for ; Wed, 8 Jan 2020 03:20:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C77BA2077B for ; Wed, 8 Jan 2020 03:20:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="WtIy9tZs"; dkim=fail reason="key not found in DNS" (0-bit key) header.d=mg.codeaurora.org header.i=@mg.codeaurora.org header.b="TBlezwlH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C77BA2077B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=ath10k-bounces+patchwork-ath10k=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+Q6qXd/rXS6dwfZZvbtEee5BO5G5nDQ4bhHE9C7SlNY=; b=WtIy9tZsj5PCrw IpyL6Rc8nAPy4G7GaVmtmhvYMtu0BiD9985b1mdqRlJtd1g/6OoUv/6wZVV9SoofSeBN7s1Mce9E6 1YEmN/0h9083gz3etZ7JX1D3ky5MuxgRd+P2kIXoZmOEzWogF5K/Ldkw8G+82jSZLGcXARrCoIac/ n8aqHW8rSBUD/3myYYSPG7C/fqa13ZqjkY+9Lx9EZ7eyW9rBKT4yJI99IctzFhSA36xNX9FwEFZHJ IuhQGdXz/xyN3Qe+LNezE0PTSvgEoIhErvnFB3+fO+cJkWPihEJ0CajOwEYlmTjuuFISnnRu5oT6/ MQTFO0aq3CCy5WwZxcnw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ip1tI-00048T-Cr; Wed, 08 Jan 2020 03:20:16 +0000 Received: from mail26.static.mailgun.info ([104.130.122.26]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ip1tF-00047j-AJ for ath10k@lists.infradead.org; Wed, 08 Jan 2020 03:20:14 +0000 DKIM-Signature: a=rsa-sha256; v=1; c=relaxed/relaxed; d=mg.codeaurora.org; q=dns/txt; s=smtp; t=1578453612; h=Content-Transfer-Encoding: MIME-Version: References: In-Reply-To: Message-Id: Date: Subject: Cc: To: From: Sender; bh=skGoFpTrGWTxWcKwtO7nTwdJjS9VLcXB/IdxoEcY6yI=; b=TBlezwlHgeFzH9eduivWIDs/noyiFJ8TLpvbNXCCzl2Ce6+F/PWDsLjaYKrRRhKYNxkCUNX6 lXRGgj3nuMK+NqKFa09cmZmlXzHhezUjxM5CPkVH6NpyRc26bjMRi9UNJIBF79WZbSd9CzHt rXMv7z0pZ8E0C1jlVUfCcRXZnmw= X-Mailgun-Sending-Ip: 104.130.122.26 X-Mailgun-Sid: WyJiZDQ3OSIsICJhdGgxMGtAbGlzdHMuaW5mcmFkZWFkLm9yZyIsICJiZTllNGEiXQ== Received: from smtp.codeaurora.org (ec2-35-166-182-171.us-west-2.compute.amazonaws.com [35.166.182.171]) by mxa.mailgun.org with ESMTP id 5e154a69.7ff4e222d0a0-smtp-out-n01; Wed, 08 Jan 2020 03:20:09 -0000 (UTC) Received: by smtp.codeaurora.org (Postfix, from userid 1001) id 2A885C433CB; Wed, 8 Jan 2020 03:20:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-caf-mail-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=2.0 tests=ALL_TRUSTED,SPF_NONE, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from wgong-HP-Z240-SFF-Workstation.qca.qualcomm.com (unknown [180.166.53.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: wgong) by smtp.codeaurora.org (Postfix) with ESMTPSA id 0EADDC43383; Wed, 8 Jan 2020 03:20:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 0EADDC43383 Authentication-Results: aws-us-west-2-caf-mail-1.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: aws-us-west-2-caf-mail-1.web.codeaurora.org; spf=none smtp.mailfrom=wgong@codeaurora.org From: Wen Gong To: ath10k@lists.infradead.org Subject: [PATCH v4 1/2] ath10k: add refcount for ath10k_core_restart Date: Wed, 8 Jan 2020 11:19:56 +0800 Message-Id: <20200108031957.22308-2-wgong@codeaurora.org> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20200108031957.22308-1-wgong@codeaurora.org> References: <20200108031957.22308-1-wgong@codeaurora.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200107_192013_421289_4F2B7FC4 X-CRM114-Status: GOOD ( 12.00 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [104.130.122.26 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: ath10k@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-wireless@vger.kernel.org Sender: "ath10k" Errors-To: ath10k-bounces+patchwork-ath10k=patchwork.kernel.org@lists.infradead.org When it has more than one restart_work queued meanwhile, the 2nd restart_work is very esay to break the 1st restart work and lead recovery fail. Add a ref count to allow only one restart work running untill device successfully recovered. Tested with QCA6174 SDIO with firmware WLAN.RMH.4.4.1-00029. Signed-off-by: Wen Gong --- drivers/net/wireless/ath/ath10k/core.c | 13 +++++++++++++ drivers/net/wireless/ath/ath10k/core.h | 2 ++ drivers/net/wireless/ath/ath10k/mac.c | 1 + 3 files changed, 16 insertions(+) diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c index 91f131b87efc..0e31846e6c89 100644 --- a/drivers/net/wireless/ath/ath10k/core.c +++ b/drivers/net/wireless/ath/ath10k/core.c @@ -2199,6 +2199,14 @@ static void ath10k_core_restart(struct work_struct *work) { struct ath10k *ar = container_of(work, struct ath10k, restart_work); int ret; + int restart_count; + + restart_count = atomic_add_return(1, &ar->restart_count); + if (restart_count > 1) { + ath10k_warn(ar, "can not restart, count: %d\n", restart_count); + atomic_dec(&ar->restart_count); + return; + } set_bit(ATH10K_FLAG_CRASH_FLUSH, &ar->dev_flags); @@ -2231,6 +2239,11 @@ static void ath10k_core_restart(struct work_struct *work) mutex_lock(&ar->conf_mutex); + if (ar->state != ATH10K_STATE_ON) { + ath10k_warn(ar, "state is not on: %d\n", ar->state); + atomic_dec(&ar->restart_count); + } + switch (ar->state) { case ATH10K_STATE_ON: ar->state = ATH10K_STATE_RESTARTING; diff --git a/drivers/net/wireless/ath/ath10k/core.h b/drivers/net/wireless/ath/ath10k/core.h index e57b2e7235e3..810c99f2dc0e 100644 --- a/drivers/net/wireless/ath/ath10k/core.h +++ b/drivers/net/wireless/ath/ath10k/core.h @@ -982,6 +982,8 @@ struct ath10k { /* protected by conf_mutex */ u8 ps_state_enable; + atomic_t restart_count; + bool nlo_enabled; bool p2p; diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c index 3856edba7915..bc1574145e66 100644 --- a/drivers/net/wireless/ath/ath10k/mac.c +++ b/drivers/net/wireless/ath/ath10k/mac.c @@ -7208,6 +7208,7 @@ static void ath10k_reconfig_complete(struct ieee80211_hw *hw, ath10k_info(ar, "device successfully recovered\n"); ar->state = ATH10K_STATE_ON; ieee80211_wake_queues(ar->hw); + atomic_dec(&ar->restart_count); } mutex_unlock(&ar->conf_mutex); From patchwork Wed Jan 8 03:19:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gong X-Patchwork-Id: 11322589 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2C9896C1 for ; Wed, 8 Jan 2020 03:20:22 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 09E1A2077B for ; Wed, 8 Jan 2020 03:20:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="jbTbJloN"; dkim=fail reason="key not found in DNS" (0-bit key) header.d=mg.codeaurora.org header.i=@mg.codeaurora.org header.b="kkY8KP55" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 09E1A2077B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=ath10k-bounces+patchwork-ath10k=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=iIXzz5dJwTDXeIWt3I0MGyBkPHgmlQBuf27q56oNEEQ=; b=jbTbJloNprLd9g 1q7vKmljIEZKuYMLJt5rxZ5mI1H8uclVCmgpZBmlA80nAHM1MTlS7vd+oyQuL4kj4aL3Cztt7T3WV 8V2tOdJppOFzQILp8Ofrgbskk4IQQWR5fCB8z2MIWQ3JEIf1KMdD42cBgFvCcGjXcgTQv6AfABEvI +saYc9i/RF+SFYwBCAZjDtR2XIXYVS22O5Y0DrTpvqVvBx11lMn6N7AvIioe/JIGO12sgNVBULilK g//boJNP7qIWkii5VoxtVIfDsbKrmRCpDclH0Vxsexbd7PIkP4GpoR8gKIpHRogii3MWM3zK53rD/ matwpNnorAMkJVC87tMw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ip1tM-0004Db-Gd; Wed, 08 Jan 2020 03:20:20 +0000 Received: from mail26.static.mailgun.info ([104.130.122.26]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ip1tI-00047j-0C for ath10k@lists.infradead.org; Wed, 08 Jan 2020 03:20:18 +0000 DKIM-Signature: a=rsa-sha256; v=1; c=relaxed/relaxed; d=mg.codeaurora.org; q=dns/txt; s=smtp; t=1578453616; h=Content-Transfer-Encoding: MIME-Version: References: In-Reply-To: Message-Id: Date: Subject: Cc: To: From: Sender; bh=V51ld3pgzq4QL60ZCOiPZgU/SMmYmpxBzwFtvKhoM9s=; b=kkY8KP55L0hI6O8px0l8i6r4lZTt1suZF/hhoiCKwQ5a3VD/6c24DyQ3QupFPMxpHbqJTwzV OOG0HoQD2tB4NBjl1aplNW0rlnEbHoneZkW7OgXe4fxeIYLK/vdTqQXTv50PWVVymdFCKnDa MBnG7DX6Id0Q+1h6+YzLk9F3vQs= X-Mailgun-Sending-Ip: 104.130.122.26 X-Mailgun-Sid: WyJiZDQ3OSIsICJhdGgxMGtAbGlzdHMuaW5mcmFkZWFkLm9yZyIsICJiZTllNGEiXQ== Received: from smtp.codeaurora.org (ec2-35-166-182-171.us-west-2.compute.amazonaws.com [35.166.182.171]) by mxa.mailgun.org with ESMTP id 5e154a6b.7efd96e104c8-smtp-out-n02; Wed, 08 Jan 2020 03:20:11 -0000 (UTC) Received: by smtp.codeaurora.org (Postfix, from userid 1001) id 78367C43383; Wed, 8 Jan 2020 03:20:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-caf-mail-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=2.0 tests=ALL_TRUSTED,SPF_NONE, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from wgong-HP-Z240-SFF-Workstation.qca.qualcomm.com (unknown [180.166.53.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: wgong) by smtp.codeaurora.org (Postfix) with ESMTPSA id 4E6CEC447A1; Wed, 8 Jan 2020 03:20:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 4E6CEC447A1 Authentication-Results: aws-us-west-2-caf-mail-1.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: aws-us-west-2-caf-mail-1.web.codeaurora.org; spf=none smtp.mailfrom=wgong@codeaurora.org From: Wen Gong To: ath10k@lists.infradead.org Subject: [PATCH v4 2/2] ath10k: start recovery process when payload length exceeds max htc length for sdio Date: Wed, 8 Jan 2020 11:19:57 +0800 Message-Id: <20200108031957.22308-3-wgong@codeaurora.org> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20200108031957.22308-1-wgong@codeaurora.org> References: <20200108031957.22308-1-wgong@codeaurora.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200107_192017_138774_DAC318B7 X-CRM114-Status: GOOD ( 14.89 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [104.130.122.26 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: ath10k@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-wireless@vger.kernel.org Sender: "ath10k" Errors-To: ath10k-bounces+patchwork-ath10k=patchwork.kernel.org@lists.infradead.org When simulate random transfer fail for sdio write and read, it happened "payload length exceeds max htc length" and recovery later sometimes. Test steps: 1. Add config and update kernel: CONFIG_FAIL_MMC_REQUEST=y CONFIG_FAULT_INJECTION=y CONFIG_FAULT_INJECTION_DEBUG_FS=y 2. Run simulate fail: cd /sys/kernel/debug/mmc1/fail_mmc_request echo 10 > probability echo 10 > times # repeat until hitting issues 3. It happened payload length exceeds max htc length. [ 199.935506] ath10k_sdio mmc1:0001:1: payload length 57005 exceeds max htc length: 4088 .... [ 264.990191] ath10k_sdio mmc1:0001:1: payload length 57005 exceeds max htc length: 4088 4. after some time, such as 60 seconds, it start recovery which triggered by wmi command timeout for periodic scan. [ 269.229232] ieee80211 phy0: Hardware restart was requested [ 269.734693] ath10k_sdio mmc1:0001:1: device successfully recovered The simulate fail of sdio is not a real sdio transter fail, it only set an error status in mmc_should_fail_request after the transfer end, actually the transfer is success, then sdio_io_rw_ext_helper will return error status and stop transfer the left data. For example, the really RX len is 286 bytes, then it will split to 2 blocks in sdio_io_rw_ext_helper, one is 256 bytes, left is 30 bytes, if the first 256 bytes get an error status by mmc_should_fail_request,then the left 30 bytes will not read in this RX operation. Then when the next RX arrive, the left 30 bytes will be considered as the header of the read, the top 4 bytes of the 30 bytes will be considered as lookaheads, but actually the 4 bytes is not the lookaheads, so the len from this lookaheads is not correct, it exceeds max htc length 4088 sometimes. When happened exceeds, the buffer chain is not matched between firmware and ath10k, then it need to start recovery ASAP. Recently then recovery will be started by wmi command timeout, but it will be long time later, for example, it is 60+ seconds later from the periodic scan, if it does not have periodic scan, it will be longer. Start recovery when it happened "payload length exceeds max htc length" will be reasonable. This patch only effect sdio chips. Tested with QCA6174 SDIO with firmware WLAN.RMH.4.4.1-00029. Signed-off-by: Wen Gong --- drivers/net/wireless/ath/ath10k/sdio.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/net/wireless/ath/ath10k/sdio.c b/drivers/net/wireless/ath/ath10k/sdio.c index 7b894dcaad2e..f48a56906f71 100644 --- a/drivers/net/wireless/ath/ath10k/sdio.c +++ b/drivers/net/wireless/ath/ath10k/sdio.c @@ -557,6 +557,10 @@ static int ath10k_sdio_mbox_rx_alloc(struct ath10k *ar, le16_to_cpu(htc_hdr->len), ATH10K_HTC_MBOX_MAX_PAYLOAD_LENGTH); ret = -ENOMEM; + + queue_work(ar->workqueue, &ar->restart_work); + ath10k_warn(ar, "exceeds length, start recovery\n"); + goto err; }