From patchwork Mon Jun 7 09:29:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 12303001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57BA2C47082 for ; Mon, 7 Jun 2021 09:34:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3A2FC611C0 for ; Mon, 7 Jun 2021 09:34:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230355AbhFGJgH (ORCPT ); Mon, 7 Jun 2021 05:36:07 -0400 Received: from frasgout.his.huawei.com ([185.176.79.56]:3158 "EHLO frasgout.his.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230241AbhFGJgG (ORCPT ); Mon, 7 Jun 2021 05:36:06 -0400 Received: from fraeml734-chm.china.huawei.com (unknown [172.18.147.226]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Fz7Ht4nRTz6G7h5; Mon, 7 Jun 2021 17:24:58 +0800 (CST) Received: from lhreml724-chm.china.huawei.com (10.201.108.75) by fraeml734-chm.china.huawei.com (10.206.15.215) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 7 Jun 2021 11:34:14 +0200 Received: from localhost.localdomain (10.69.192.58) by lhreml724-chm.china.huawei.com (10.201.108.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 7 Jun 2021 10:34:11 +0100 From: John Garry To: , CC: , , , Luo Jiaxing , John Garry Subject: [PATCH 1/5] scsi: hisi_sas: Put a limit of link reset retries Date: Mon, 7 Jun 2021 17:29:35 +0800 Message-ID: <1623058179-80434-2-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1623058179-80434-1-git-send-email-john.garry@huawei.com> References: <1623058179-80434-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To lhreml724-chm.china.huawei.com (10.201.108.75) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Luo Jiaxing If an OOB event is received but the phy still fails to come up, a link reset will be issued repeatedly at an interval of 20s until the phy comes up. Set a limit for link reset issue retries to avoid printing the timeout message endlessly. Signed-off-by: Luo Jiaxing Signed-off-by: John Garry --- drivers/scsi/hisi_sas/hisi_sas.h | 1 + drivers/scsi/hisi_sas/hisi_sas_main.c | 15 +++++++++++++-- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/hisi_sas/hisi_sas.h b/drivers/scsi/hisi_sas/hisi_sas.h index cf879cc59e4c..8e36ede12cf1 100644 --- a/drivers/scsi/hisi_sas/hisi_sas.h +++ b/drivers/scsi/hisi_sas/hisi_sas.h @@ -185,6 +185,7 @@ struct hisi_sas_phy { enum sas_linkrate minimum_linkrate; enum sas_linkrate maximum_linkrate; int enable; + int wait_phyup_cnt; atomic_t down_cnt; /* Trace FIFO */ diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c index 5a204074099c..50420741fc81 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_main.c +++ b/drivers/scsi/hisi_sas/hisi_sas_main.c @@ -857,6 +857,7 @@ static void hisi_sas_phyup_work(struct work_struct *work) struct asd_sas_phy *sas_phy = &phy->sas_phy; int phy_no = sas_phy->id; + phy->wait_phyup_cnt = 0; if (phy->identify.target_port_protocols == SAS_PROTOCOL_SSP) hisi_hba->hw->sl_notify_ssp(hisi_hba, phy_no); hisi_sas_bytes_dmaed(hisi_hba, phy_no, GFP_KERNEL); @@ -899,6 +900,8 @@ static void hisi_sas_wait_phyup_timedout(struct timer_list *t) hisi_sas_notify_phy_event(phy, HISI_PHYE_LINK_RESET); } +#define HISI_SAS_WAIT_PHYUP_RETRIES 10 + void hisi_sas_phy_oob_ready(struct hisi_hba *hisi_hba, int phy_no) { struct hisi_sas_phy *phy = &hisi_hba->phy[phy_no]; @@ -909,8 +912,16 @@ void hisi_sas_phy_oob_ready(struct hisi_hba *hisi_hba, int phy_no) return; if (!timer_pending(&phy->timer)) { - phy->timer.expires = jiffies + HISI_SAS_WAIT_PHYUP_TIMEOUT * HZ; - add_timer(&phy->timer); + if (phy->wait_phyup_cnt < HISI_SAS_WAIT_PHYUP_RETRIES) { + phy->wait_phyup_cnt++; + phy->timer.expires = jiffies + + HISI_SAS_WAIT_PHYUP_TIMEOUT * HZ; + add_timer(&phy->timer); + } else { + dev_warn(dev, "phy%d failed to come up %d times, giving up\n", + phy_no, phy->wait_phyup_cnt); + phy->wait_phyup_cnt = 0; + } } } EXPORT_SYMBOL_GPL(hisi_sas_phy_oob_ready); From patchwork Mon Jun 7 09:29:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 12303003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC7ECC47082 for ; Mon, 7 Jun 2021 09:34:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B0FF3611C1 for ; Mon, 7 Jun 2021 09:34:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230435AbhFGJgK (ORCPT ); Mon, 7 Jun 2021 05:36:10 -0400 Received: from frasgout.his.huawei.com ([185.176.79.56]:3159 "EHLO frasgout.his.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230409AbhFGJgI (ORCPT ); Mon, 7 Jun 2021 05:36:08 -0400 Received: from fraeml714-chm.china.huawei.com (unknown [172.18.147.207]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Fz7Ly23vWz6G7L8; Mon, 7 Jun 2021 17:27:38 +0800 (CST) Received: from lhreml724-chm.china.huawei.com (10.201.108.75) by fraeml714-chm.china.huawei.com (10.206.15.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 7 Jun 2021 11:34:16 +0200 Received: from localhost.localdomain (10.69.192.58) by lhreml724-chm.china.huawei.com (10.201.108.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 7 Jun 2021 10:34:13 +0100 From: John Garry To: , CC: , , , Luo Jiaxing , John Garry Subject: [PATCH 2/5] scsi: hisi_sas: Run I_T nexus resets in parallel for clear nexus reset Date: Mon, 7 Jun 2021 17:29:36 +0800 Message-ID: <1623058179-80434-3-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1623058179-80434-1-git-send-email-john.garry@huawei.com> References: <1623058179-80434-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To lhreml724-chm.china.huawei.com (10.201.108.75) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Luo Jiaxing For a clear nexus reset operation, the I_T nexus resets are executed serially for each device. For devices attached through an expander, this may take 2s per device; so, in total, could take a long time. Reduce the total time by running the I_T nexus resets in parallel through async operations. Signed-off-by: Luo Jiaxing Signed-off-by: John Garry --- drivers/scsi/hisi_sas/hisi_sas.h | 1 + drivers/scsi/hisi_sas/hisi_sas_main.c | 23 +++++++++++++++++------ 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/drivers/scsi/hisi_sas/hisi_sas.h b/drivers/scsi/hisi_sas/hisi_sas.h index 8e36ede12cf1..fbecdf756c77 100644 --- a/drivers/scsi/hisi_sas/hisi_sas.h +++ b/drivers/scsi/hisi_sas/hisi_sas.h @@ -8,6 +8,7 @@ #define _HISI_SAS_H_ #include +#include #include #include #include diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c index 50420741fc81..856cdc1b32d5 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_main.c +++ b/drivers/scsi/hisi_sas/hisi_sas_main.c @@ -1886,12 +1886,24 @@ static int hisi_sas_lu_reset(struct domain_device *device, u8 *lun) return rc; } +static void hisi_sas_async_I_T_nexus_reset(void *data, async_cookie_t cookie) +{ + struct domain_device *device = data; + struct hisi_hba *hisi_hba = dev_to_hisi_hba(device); + int rc; + + rc = hisi_sas_debug_I_T_nexus_reset(device); + if (rc != TMF_RESP_FUNC_COMPLETE) + dev_info(hisi_hba->dev, "I_T_nexus reset fail for dev:%016llx rc=%d\n", + SAS_ADDR(device->sas_addr), rc); +} + static int hisi_sas_clear_nexus_ha(struct sas_ha_struct *sas_ha) { struct hisi_hba *hisi_hba = sas_ha->lldd_ha; - struct device *dev = hisi_hba->dev; HISI_SAS_DECLARE_RST_WORK_ON_STACK(r); - int rc, i; + ASYNC_DOMAIN_EXCLUSIVE(async); + int i; queue_work(hisi_hba->wq, &r.work); wait_for_completion(r.completion); @@ -1906,12 +1918,11 @@ static int hisi_sas_clear_nexus_ha(struct sas_ha_struct *sas_ha) dev_is_expander(device->dev_type)) continue; - rc = hisi_sas_debug_I_T_nexus_reset(device); - if (rc != TMF_RESP_FUNC_COMPLETE) - dev_info(dev, "clear nexus ha: for device[%d] rc=%d\n", - sas_dev->device_id, rc); + async_schedule_domain(hisi_sas_async_I_T_nexus_reset, + device, &async); } + async_synchronize_full_domain(&async); hisi_sas_release_tasks(hisi_hba); return TMF_RESP_FUNC_COMPLETE; From patchwork Mon Jun 7 09:29:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 12303005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 980A2C48BCD for ; Mon, 7 Jun 2021 09:34:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 79023611C2 for ; Mon, 7 Jun 2021 09:34:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230479AbhFGJgM (ORCPT ); Mon, 7 Jun 2021 05:36:12 -0400 Received: from frasgout.his.huawei.com ([185.176.79.56]:3160 "EHLO frasgout.his.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230443AbhFGJgL (ORCPT ); Mon, 7 Jun 2021 05:36:11 -0400 Received: from fraeml713-chm.china.huawei.com (unknown [172.18.147.200]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Fz7M071g1z6G7Kr; Mon, 7 Jun 2021 17:27:40 +0800 (CST) Received: from lhreml724-chm.china.huawei.com (10.201.108.75) by fraeml713-chm.china.huawei.com (10.206.15.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 7 Jun 2021 11:34:19 +0200 Received: from localhost.localdomain (10.69.192.58) by lhreml724-chm.china.huawei.com (10.201.108.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 7 Jun 2021 10:34:16 +0100 From: John Garry To: , CC: , , , Luo Jiaxing , John Garry Subject: [PATCH 3/5] scsi: hisi_sas: Include HZ in timer macros Date: Mon, 7 Jun 2021 17:29:37 +0800 Message-ID: <1623058179-80434-4-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1623058179-80434-1-git-send-email-john.garry@huawei.com> References: <1623058179-80434-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To lhreml724-chm.china.huawei.com (10.201.108.75) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Luo Jiaxing Include HZ in timer macros to make the code more concise. Signed-off-by: Luo Jiaxing Signed-off-by: John Garry --- drivers/scsi/hisi_sas/hisi_sas.h | 4 ++-- drivers/scsi/hisi_sas/hisi_sas_main.c | 17 ++++++++++------- drivers/scsi/hisi_sas/hisi_sas_v2_hw.c | 2 +- drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 2 +- 4 files changed, 14 insertions(+), 11 deletions(-) diff --git a/drivers/scsi/hisi_sas/hisi_sas.h b/drivers/scsi/hisi_sas/hisi_sas.h index fbecdf756c77..8f2492d0d49e 100644 --- a/drivers/scsi/hisi_sas/hisi_sas.h +++ b/drivers/scsi/hisi_sas/hisi_sas.h @@ -91,8 +91,8 @@ #define HISI_SAS_PROT_MASK (HISI_SAS_DIF_PROT_MASK | HISI_SAS_DIX_PROT_MASK) -#define HISI_SAS_WAIT_PHYUP_TIMEOUT 20 -#define CLEAR_ITCT_TIMEOUT 20 +#define HISI_SAS_WAIT_PHYUP_TIMEOUT (20 * HZ) +#define HISI_SAS_CLEAR_ITCT_TIMEOUT (20 * HZ) struct hisi_hba; diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c index 856cdc1b32d5..37ccbc1103b3 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_main.c +++ b/drivers/scsi/hisi_sas/hisi_sas_main.c @@ -915,7 +915,7 @@ void hisi_sas_phy_oob_ready(struct hisi_hba *hisi_hba, int phy_no) if (phy->wait_phyup_cnt < HISI_SAS_WAIT_PHYUP_RETRIES) { phy->wait_phyup_cnt++; phy->timer.expires = jiffies + - HISI_SAS_WAIT_PHYUP_TIMEOUT * HZ; + HISI_SAS_WAIT_PHYUP_TIMEOUT; add_timer(&phy->timer); } else { dev_warn(dev, "phy%d failed to come up %d times, giving up\n", @@ -1193,9 +1193,9 @@ static void hisi_sas_tmf_timedout(struct timer_list *t) complete(&task->slow_task->completion); } -#define TASK_TIMEOUT 20 -#define TASK_RETRY 3 -#define INTERNAL_ABORT_TIMEOUT 6 +#define TASK_TIMEOUT (20 * HZ) +#define TASK_RETRY 3 +#define INTERNAL_ABORT_TIMEOUT (6 * HZ) static int hisi_sas_exec_internal_tmf_task(struct domain_device *device, void *parameter, u32 para_len, struct hisi_sas_tmf_task *tmf) @@ -1223,7 +1223,7 @@ static int hisi_sas_exec_internal_tmf_task(struct domain_device *device, task->task_done = hisi_sas_task_done; task->slow_task->timer.function = hisi_sas_tmf_timedout; - task->slow_task->timer.expires = jiffies + TASK_TIMEOUT * HZ; + task->slow_task->timer.expires = jiffies + TASK_TIMEOUT; add_timer(&task->slow_task->timer); res = hisi_sas_task_exec(task, GFP_KERNEL, 1, tmf); @@ -1761,6 +1761,8 @@ static int hisi_sas_clear_aca(struct domain_device *device, u8 *lun) return rc; } +#define I_T_NEXUS_RESET_PHYUP_TIMEOUT (2 * HZ) + static int hisi_sas_debug_I_T_nexus_reset(struct domain_device *device) { struct sas_phy *local_phy = sas_get_local_phy(device); @@ -1795,7 +1797,8 @@ static int hisi_sas_debug_I_T_nexus_reset(struct domain_device *device) sas_ha->sas_phy[local_phy->number]; struct hisi_sas_phy *phy = container_of(sas_phy, struct hisi_sas_phy, sas_phy); - int ret = wait_for_completion_timeout(&phyreset, 2 * HZ); + int ret = wait_for_completion_timeout(&phyreset, + I_T_NEXUS_RESET_PHYUP_TIMEOUT); unsigned long flags; spin_lock_irqsave(&phy->lock, flags); @@ -2079,7 +2082,7 @@ _hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba, task->task_proto = device->tproto; task->task_done = hisi_sas_task_done; task->slow_task->timer.function = hisi_sas_tmf_timedout; - task->slow_task->timer.expires = jiffies + INTERNAL_ABORT_TIMEOUT * HZ; + task->slow_task->timer.expires = jiffies + INTERNAL_ABORT_TIMEOUT; add_timer(&task->slow_task->timer); res = hisi_sas_internal_abort_task_exec(hisi_hba, sas_dev->device_id, diff --git a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c index 46f60fc2a069..670013bfe333 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c @@ -994,7 +994,7 @@ static int clear_itct_v2_hw(struct hisi_hba *hisi_hba, reg_val = ITCT_CLR_EN_MSK | (dev_id & ITCT_DEV_MSK); hisi_sas_write32(hisi_hba, ITCT_CLR, reg_val); if (!wait_for_completion_timeout(sas_dev->completion, - CLEAR_ITCT_TIMEOUT * HZ)) { + HISI_SAS_CLEAR_ITCT_TIMEOUT)) { dev_warn(dev, "failed to clear ITCT\n"); return -ETIMEDOUT; } diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c index 499c770d405c..85ced22ee6ca 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c @@ -843,7 +843,7 @@ static int clear_itct_v3_hw(struct hisi_hba *hisi_hba, hisi_sas_write32(hisi_hba, ITCT_CLR, reg_val); if (!wait_for_completion_timeout(sas_dev->completion, - CLEAR_ITCT_TIMEOUT * HZ)) { + HISI_SAS_CLEAR_ITCT_TIMEOUT)) { dev_warn(dev, "failed to clear ITCT\n"); return -ETIMEDOUT; } From patchwork Mon Jun 7 09:29:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 12303007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECC93C4743C for ; Mon, 7 Jun 2021 09:34:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D5B3E611C0 for ; Mon, 7 Jun 2021 09:34:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230382AbhFGJgQ (ORCPT ); Mon, 7 Jun 2021 05:36:16 -0400 Received: from frasgout.his.huawei.com ([185.176.79.56]:3161 "EHLO frasgout.his.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230502AbhFGJgN (ORCPT ); Mon, 7 Jun 2021 05:36:13 -0400 Received: from fraeml715-chm.china.huawei.com (unknown [172.18.147.201]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Fz7M30lkZz6G7L6; Mon, 7 Jun 2021 17:27:43 +0800 (CST) Received: from lhreml724-chm.china.huawei.com (10.201.108.75) by fraeml715-chm.china.huawei.com (10.206.15.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 7 Jun 2021 11:34:21 +0200 Received: from localhost.localdomain (10.69.192.58) by lhreml724-chm.china.huawei.com (10.201.108.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 7 Jun 2021 10:34:18 +0100 From: John Garry To: , CC: , , , Luo Jiaxing , John Garry Subject: [PATCH 4/5] scsi: hisi_sas: Reset controller for internal abort timeout Date: Mon, 7 Jun 2021 17:29:38 +0800 Message-ID: <1623058179-80434-5-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1623058179-80434-1-git-send-email-john.garry@huawei.com> References: <1623058179-80434-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To lhreml724-chm.china.huawei.com (10.201.108.75) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Luo Jiaxing If an internal task abort timeout occurs, the controller has developed a fault, and needs to be reset to be recovered. However if a timeout occurs during SCSI error handling, issuing a controller reset immediately may conflict with the error handling. To handle internal abort in these two scenarios, only queue the reset when not in an error handling function. In the case of a timeout during error handling, do nothing and rely on the inevitable ha nexus reset to reset the controller. Signed-off-by: Luo Jiaxing Signed-off-by: John Garry --- drivers/scsi/hisi_sas/hisi_sas_main.c | 40 +++++++++++++++++---------- 1 file changed, 26 insertions(+), 14 deletions(-) diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c index 37ccbc1103b3..0ad861aa5bb6 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_main.c +++ b/drivers/scsi/hisi_sas/hisi_sas_main.c @@ -15,7 +15,7 @@ static int hisi_sas_debug_issue_ssp_tmf(struct domain_device *device, static int hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba, struct domain_device *device, - int abort_flag, int tag); + int abort_flag, int tag, bool rst_to_recover); static int hisi_sas_softreset_ata_disk(struct domain_device *device); static int hisi_sas_control_phy(struct asd_sas_phy *sas_phy, enum phy_func func, void *funcdata); @@ -1074,7 +1074,7 @@ static void hisi_sas_dev_gone(struct domain_device *device) down(&hisi_hba->sem); if (!test_bit(HISI_SAS_RESET_BIT, &hisi_hba->flags)) { hisi_sas_internal_task_abort(hisi_hba, device, - HISI_SAS_INT_ABT_DEV, 0); + HISI_SAS_INT_ABT_DEV, 0, true); hisi_sas_dereg_device(hisi_hba, device); @@ -1516,7 +1516,8 @@ static void hisi_sas_terminate_stp_reject(struct hisi_hba *hisi_hba) continue; rc = hisi_sas_internal_task_abort(hisi_hba, device, - HISI_SAS_INT_ABT_DEV, 0); + HISI_SAS_INT_ABT_DEV, 0, + false); if (rc < 0) dev_err(dev, "STP reject: abort dev failed %d\n", rc); } @@ -1671,7 +1672,8 @@ static int hisi_sas_abort_task(struct sas_task *task) &tmf_task); rc2 = hisi_sas_internal_task_abort(hisi_hba, device, - HISI_SAS_INT_ABT_CMD, tag); + HISI_SAS_INT_ABT_CMD, tag, + false); if (rc2 < 0) { dev_err(dev, "abort task: internal abort (%d)\n", rc2); return TMF_RESP_FUNC_FAILED; @@ -1693,7 +1695,7 @@ static int hisi_sas_abort_task(struct sas_task *task) if (task->dev->dev_type == SAS_SATA_DEV) { rc = hisi_sas_internal_task_abort(hisi_hba, device, HISI_SAS_INT_ABT_DEV, - 0); + 0, false); if (rc < 0) { dev_err(dev, "abort task: internal abort failed\n"); goto out; @@ -1708,7 +1710,8 @@ static int hisi_sas_abort_task(struct sas_task *task) struct hisi_sas_cq *cq = &hisi_hba->cq[slot->dlvry_queue]; rc = hisi_sas_internal_task_abort(hisi_hba, device, - HISI_SAS_INT_ABT_CMD, tag); + HISI_SAS_INT_ABT_CMD, tag, + false); if (((rc < 0) || (rc == TMF_RESP_FUNC_FAILED)) && task->lldd_task) { /* @@ -1734,7 +1737,7 @@ static int hisi_sas_abort_task_set(struct domain_device *device, u8 *lun) int rc; rc = hisi_sas_internal_task_abort(hisi_hba, device, - HISI_SAS_INT_ABT_DEV, 0); + HISI_SAS_INT_ABT_DEV, 0, false); if (rc < 0) { dev_err(dev, "abort task set: internal abort rc=%d\n", rc); return TMF_RESP_FUNC_FAILED; @@ -1828,7 +1831,7 @@ static int hisi_sas_I_T_nexus_reset(struct domain_device *device) int rc; rc = hisi_sas_internal_task_abort(hisi_hba, device, - HISI_SAS_INT_ABT_DEV, 0); + HISI_SAS_INT_ABT_DEV, 0, false); if (rc < 0) { dev_err(dev, "I_T nexus reset: internal abort (%d)\n", rc); return TMF_RESP_FUNC_FAILED; @@ -1858,7 +1861,7 @@ static int hisi_sas_lu_reset(struct domain_device *device, u8 *lun) /* Clear internal IO and then lu reset */ rc = hisi_sas_internal_task_abort(hisi_hba, device, - HISI_SAS_INT_ABT_DEV, 0); + HISI_SAS_INT_ABT_DEV, 0, false); if (rc < 0) { dev_err(dev, "lu_reset: internal abort failed\n"); goto out; @@ -2054,11 +2057,13 @@ hisi_sas_internal_abort_task_exec(struct hisi_hba *hisi_hba, int device_id, * @tag: tag of IO to be aborted (only relevant to single * IO mode) * @dq: delivery queue for this internal abort command + * @rst_to_recover: If rst_to_recover set, queue a controller + * reset if an internal abort times out. */ static int _hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba, struct domain_device *device, int abort_flag, - int tag, struct hisi_sas_dq *dq) + int tag, struct hisi_sas_dq *dq, bool rst_to_recover) { struct sas_task *task; struct hisi_sas_device *sas_dev = device->lldd_dev; @@ -2114,7 +2119,13 @@ _hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba, synchronize_irq(cq->irq_no); slot->task = NULL; } - dev_err(dev, "internal task abort: timeout and not done.\n"); + + if (rst_to_recover) { + dev_err(dev, "internal task abort: timeout and not done. Queuing reset.\n"); + queue_work(hisi_hba->wq, &hisi_hba->rst_work); + } else { + dev_err(dev, "internal task abort: timeout and not done.\n"); + } res = -EIO; goto exit; @@ -2147,7 +2158,7 @@ _hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba, static int hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba, struct domain_device *device, - int abort_flag, int tag) + int abort_flag, int tag, bool rst_to_recover) { struct hisi_sas_slot *slot; struct device *dev = hisi_hba->dev; @@ -2159,7 +2170,8 @@ hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba, slot = &hisi_hba->slot_info[tag]; dq = &hisi_hba->dq[slot->dlvry_queue]; return _hisi_sas_internal_task_abort(hisi_hba, device, - abort_flag, tag, dq); + abort_flag, tag, dq, + rst_to_recover); case HISI_SAS_INT_ABT_DEV: for (i = 0; i < hisi_hba->cq_nvecs; i++) { struct hisi_sas_cq *cq = &hisi_hba->cq[i]; @@ -2170,7 +2182,7 @@ hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba, dq = &hisi_hba->dq[i]; rc = _hisi_sas_internal_task_abort(hisi_hba, device, abort_flag, tag, - dq); + dq, rst_to_recover); if (rc) return rc; } From patchwork Mon Jun 7 09:29:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 12303009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBB9CC47082 for ; Mon, 7 Jun 2021 09:34:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B59C9611C1 for ; Mon, 7 Jun 2021 09:34:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230514AbhFGJgU (ORCPT ); Mon, 7 Jun 2021 05:36:20 -0400 Received: from frasgout.his.huawei.com ([185.176.79.56]:3162 "EHLO frasgout.his.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230516AbhFGJgP (ORCPT ); Mon, 7 Jun 2021 05:36:15 -0400 Received: from fraeml710-chm.china.huawei.com (unknown [172.18.147.226]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Fz7D75Pn4z6H6q6; Mon, 7 Jun 2021 17:21:43 +0800 (CST) Received: from lhreml724-chm.china.huawei.com (10.201.108.75) by fraeml710-chm.china.huawei.com (10.206.15.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 7 Jun 2021 11:34:23 +0200 Received: from localhost.localdomain (10.69.192.58) by lhreml724-chm.china.huawei.com (10.201.108.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 7 Jun 2021 10:34:21 +0100 From: John Garry To: , CC: , , , Luo Jiaxing , John Garry Subject: [PATCH 5/5] scsi: hisi_sas: Speed up error handling when internal abort timeout occurs Date: Mon, 7 Jun 2021 17:29:39 +0800 Message-ID: <1623058179-80434-6-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1623058179-80434-1-git-send-email-john.garry@huawei.com> References: <1623058179-80434-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To lhreml724-chm.china.huawei.com (10.201.108.75) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Luo Jiaxing If an internal task abort timeout occurs, the controller has developed a fault, and needs to be reset to be recovered. When this occurs during error handling, the current policy is to allow error handling to continue, and the inevitable nexus ha reset will handle the required reset. However various steps of error handling need to taken before this happens. These also involve some level of HW interaction, which will also fail with various timeouts. Speed up this process by recording a HW fault bit for an internal abort timeout - when this is set, just automatically error any HW interaction, and essentially go straight to clear nexus ha (to reset the controller). Signed-off-by: Luo Jiaxing Signed-off-by: John Garry --- drivers/scsi/hisi_sas/hisi_sas.h | 1 + drivers/scsi/hisi_sas/hisi_sas_main.c | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/drivers/scsi/hisi_sas/hisi_sas.h b/drivers/scsi/hisi_sas/hisi_sas.h index 8f2492d0d49e..436d174f2194 100644 --- a/drivers/scsi/hisi_sas/hisi_sas.h +++ b/drivers/scsi/hisi_sas/hisi_sas.h @@ -38,6 +38,7 @@ #define HISI_SAS_RESET_BIT 0 #define HISI_SAS_REJECT_CMD_BIT 1 #define HISI_SAS_PM_BIT 2 +#define HISI_SAS_HW_FAULT_BIT 3 #define HISI_SAS_MAX_COMMANDS (HISI_SAS_QUEUE_SLOTS) #define HISI_SAS_RESERVED_IPTT 96 #define HISI_SAS_UNRESERVED_IPTT \ diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c index 0ad861aa5bb6..3a903e8e0384 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_main.c +++ b/drivers/scsi/hisi_sas/hisi_sas_main.c @@ -1616,6 +1616,7 @@ static int hisi_sas_controller_reset(struct hisi_hba *hisi_hba) } hisi_sas_controller_reset_done(hisi_hba); + clear_bit(HISI_SAS_HW_FAULT_BIT, &hisi_hba->flags); dev_info(dev, "controller reset complete\n"); return 0; @@ -2079,6 +2080,9 @@ _hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba, if (!hisi_hba->hw->prep_abort) return TMF_RESP_FUNC_FAILED; + if (test_bit(HISI_SAS_HW_FAULT_BIT, &hisi_hba->flags)) + return -EIO; + task = sas_alloc_slow_task(GFP_KERNEL); if (!task) return -ENOMEM; @@ -2109,6 +2113,8 @@ _hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba, if (!(task->task_state_flags & SAS_TASK_STATE_DONE)) { struct hisi_sas_slot *slot = task->lldd_task; + set_bit(HISI_SAS_HW_FAULT_BIT, &hisi_hba->flags); + if (slot) { struct hisi_sas_cq *cq = &hisi_hba->cq[slot->dlvry_queue];