From patchwork Wed May 10 05:24:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bao D. Nguyen" X-Patchwork-Id: 13236451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9A3FC77B7D for ; Wed, 10 May 2023 05:25:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235126AbjEJFZv (ORCPT ); Wed, 10 May 2023 01:25:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234887AbjEJFZu (ORCPT ); Wed, 10 May 2023 01:25:50 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 752FEE53; Tue, 9 May 2023 22:25:47 -0700 (PDT) Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34A2iO8h025142; Wed, 10 May 2023 05:25:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=uEG9NCAeHkYxw7zM+ehn1ThstYlWo7xMCJyDEpC9A+Y=; b=NsFUKqnVyhOiKXNvp0oG3HEgrK/Jr+6hZ4wbToVcMQlkKsAdaVqBuS5QxqPKnmaMIK0P h9iTgt54YktyVOuFpo00lQDgwerTQJGSCQdaDYGhN4p8D6VZl6eDD1lxyQqZCSbV4+Z8 eKUBiCR2gP1mfkPglvLx4BTWFSOwZBpD8v3FfR52OOPpwa1Gfu0kd6yBOLWEyvon1oyl CoShXggWR/LM6sEVFlEvwIXevMLDoE1aPKjHSn3I2B0Ga/ugGJyB1zc4fA9mQPT+8th5 WdZvNgqIKQPgSCRaqJlMvmF7JP7cBVojHcORo6J86gu7ahjdCBM4+od5sYiC+evqc7/U qA== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3qfsxj95sn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:25:33 +0000 Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 34A5PWtQ021630 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:25:32 GMT Received: from stor-berry.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 9 May 2023 22:25:31 -0700 From: "Bao D. Nguyen" To: , , , , , , , , CC: , "Bao D. Nguyen" , Alim Akhtar , "James E.J. Bottomley" , Eric Biggers , open list Subject: [PATCH v3 1/7] ufs: core: Combine 32-bit command_desc_base_addr_lo/hi Date: Tue, 9 May 2023 22:24:22 -0700 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: BFCH8wCEvVB0KzW8emNXLg_P8Jw3Rva1 X-Proofpoint-GUID: BFCH8wCEvVB0KzW8emNXLg_P8Jw3Rva1 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-10_02,2023-05-05_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 mlxlogscore=999 phishscore=0 spamscore=0 suspectscore=0 clxscore=1015 impostorscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 lowpriorityscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305100042 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The utp command descriptor base address is a 57-bit field in the utp utp transfer request descriptor. Combine the two 32-bit command_desc_base_addr_lo/hi fields into a 64-bit for better handling of this field. Signed-off-by: Bao D. Nguyen Reviewed-by: Bart Van Assche --- drivers/ufs/core/ufshcd.c | 6 ++---- include/ufs/ufshci.h | 6 ++---- 2 files changed, 4 insertions(+), 8 deletions(-) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 9434328..5527d45 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -3875,10 +3875,8 @@ static void ufshcd_host_memory_configure(struct ufs_hba *hba) /* Configure UTRD with command descriptor base address */ cmd_desc_element_addr = (cmd_desc_dma_addr + (cmd_desc_size * i)); - utrdlp[i].command_desc_base_addr_lo = - cpu_to_le32(lower_32_bits(cmd_desc_element_addr)); - utrdlp[i].command_desc_base_addr_hi = - cpu_to_le32(upper_32_bits(cmd_desc_element_addr)); + utrdlp[i].command_desc_base_addr = + cpu_to_le64(cmd_desc_element_addr); /* Response upiu and prdt offset should be in double words */ if (hba->quirks & UFSHCD_QUIRK_PRDT_BYTE_GRAN) { diff --git a/include/ufs/ufshci.h b/include/ufs/ufshci.h index 11424bb..7c5a76b 100644 --- a/include/ufs/ufshci.h +++ b/include/ufs/ufshci.h @@ -503,8 +503,7 @@ struct request_desc_header { /** * struct utp_transfer_req_desc - UTP Transfer Request Descriptor (UTRD) * @header: UTRD header DW-0 to DW-3 - * @command_desc_base_addr_lo: UCD base address low DW-4 - * @command_desc_base_addr_hi: UCD base address high DW-5 + * @command_desc_base_addr: UCD base address DW 4-5 * @response_upiu_length: response UPIU length DW-6 * @response_upiu_offset: response UPIU offset DW-6 * @prd_table_length: Physical region descriptor length DW-7 @@ -516,8 +515,7 @@ struct utp_transfer_req_desc { struct request_desc_header header; /* DW 4-5*/ - __le32 command_desc_base_addr_lo; - __le32 command_desc_base_addr_hi; + __le64 command_desc_base_addr; /* DW 6 */ __le16 response_upiu_length; From patchwork Wed May 10 05:24:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bao D. Nguyen" X-Patchwork-Id: 13236452 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C969BC77B7D for ; Wed, 10 May 2023 05:26:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235735AbjEJF0D (ORCPT ); Wed, 10 May 2023 01:26:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229524AbjEJF0A (ORCPT ); Wed, 10 May 2023 01:26:00 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 165C74EC0; Tue, 9 May 2023 22:25:50 -0700 (PDT) Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34A5LPEC019903; Wed, 10 May 2023 05:25:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=9rljmIBbVbexzgTu0xwlPNsb6fltOx7OyMS8C21bpBY=; b=V6vWBX3X4c4hsCPofOmOtzXZftjPaKRKugb9fX07z0Xkc2u8oRrNqpXrRSEDf6RoR7Eh CZHSwyNPeJfjL7EDo9XPyxo3/oCp4DotY/c6lnRAVjSdfc805AxJwuOiPutTt77F8t1W Qrf2VT3zSyV1sP188/PDA9uSebH8gDCfkSX/hK+hBumEfLr8aAGzl13yWRGReTdT0O6W jtjgOmR2shZXirBtN3YSHzxgbXKQREwsZNede8eJqT3KYcifIcUuy1T1v2o7XTqqFtcS 6B5g6Q/J67aDGXW3wfrnt5uaChKB7ZplBX+ULMEZupzDanN2586fOBBdp1i6sQMUcwOK JQ== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3qfysdrgkh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:25:38 +0000 Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 34A5Pbkf024608 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:25:37 GMT Received: from stor-berry.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 9 May 2023 22:25:36 -0700 From: "Bao D. Nguyen" To: , , , , , , , , CC: , "Bao D. Nguyen" , Alim Akhtar , "James E.J. Bottomley" , open list Subject: [PATCH v3 2/7] ufs: core: Update the ufshcd_clear_cmds() functionality Date: Tue, 9 May 2023 22:24:23 -0700 Message-ID: <6c9990cd13220ceea343b1e5ba463732c4ccffb9.1683688693.git.quic_nguyenb@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: crn19goY7Vxldcqv5TYBf7RO0cOck5nh X-Proofpoint-ORIG-GUID: crn19goY7Vxldcqv5TYBf7RO0cOck5nh X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-10_02,2023-05-05_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0 phishscore=0 malwarescore=0 suspectscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 impostorscore=0 adultscore=0 priorityscore=1501 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305100042 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In the ufshcd_clear_cmds(), the 2nd pamameter would be the bit mask of the command to be cleared in the transfer request door bell register. This bit mask mechanism does not scale well in mcq mode when the queue depth becomes much greater than 64. Change the 2nd parameter to the function to be the task_tag number of the corresponding to the bit to be cleared in the door bell register. By doing so, mcq mode with a large queue depth can reuse this function. Signed-off-by: Bao D. Nguyen Reviewed-by: Bart Van Assche --- drivers/ufs/core/ufshcd.c | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 5527d45..3a7e853 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -3006,13 +3006,15 @@ static int ufshcd_compose_dev_cmd(struct ufs_hba *hba, } /* - * Clear all the requests from the controller for which a bit has been set in - * @mask and wait until the controller confirms that these requests have been - * cleared. + * Clear the pending command in the controller and wait until + * the controller confirms that the command has been cleared. + * @hba: per adapter instance + * @task_tag: The tag number of the command to be cleared. */ -static int ufshcd_clear_cmds(struct ufs_hba *hba, u32 mask) +static int ufshcd_clear_cmds(struct ufs_hba *hba, u32 task_tag) { unsigned long flags; + u32 mask = 1U << task_tag; /* clear outstanding transaction before retry */ spin_lock_irqsave(hba->host->host_lock, flags); @@ -3113,7 +3115,7 @@ static int ufshcd_wait_for_dev_cmd(struct ufs_hba *hba, err = -ETIMEDOUT; dev_dbg(hba->dev, "%s: dev_cmd request timedout, tag %d\n", __func__, lrbp->task_tag); - if (ufshcd_clear_cmds(hba, 1U << lrbp->task_tag) == 0) { + if (ufshcd_clear_cmds(hba, lrbp->task_tag) == 0) { /* successfully cleared the command, retry if needed */ err = -EAGAIN; /* @@ -7286,7 +7288,7 @@ static int ufshcd_eh_device_reset_handler(struct scsi_cmnd *cmd) unsigned long flags, pending_reqs = 0, not_cleared = 0; struct Scsi_Host *host; struct ufs_hba *hba; - u32 pos; + u32 pos, not_cleared_mask = 0; int err; u8 resp = 0xF, lun; @@ -7309,17 +7311,20 @@ static int ufshcd_eh_device_reset_handler(struct scsi_cmnd *cmd) hba->outstanding_reqs &= ~pending_reqs; spin_unlock_irqrestore(&hba->outstanding_lock, flags); - if (ufshcd_clear_cmds(hba, pending_reqs) < 0) { - spin_lock_irqsave(&hba->outstanding_lock, flags); - not_cleared = pending_reqs & - ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); - hba->outstanding_reqs |= not_cleared; - spin_unlock_irqrestore(&hba->outstanding_lock, flags); + for_each_set_bit(pos, &pending_reqs, hba->nutrs) { + if (ufshcd_clear_cmds(hba, pos) < 0) { + spin_lock_irqsave(&hba->outstanding_lock, flags); + not_cleared = 1U << pos & + ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); + hba->outstanding_reqs |= not_cleared; + not_cleared_mask |= not_cleared; + spin_unlock_irqrestore(&hba->outstanding_lock, flags); - dev_err(hba->dev, "%s: failed to clear requests %#lx\n", - __func__, not_cleared); + dev_err(hba->dev, "%s: failed to clear requests %#lx\n", + __func__, not_cleared); + } } - __ufshcd_transfer_req_compl(hba, pending_reqs & ~not_cleared); + __ufshcd_transfer_req_compl(hba, pending_reqs & ~not_cleared_mask); out: hba->req_abort_count = 0; @@ -7416,7 +7421,7 @@ static int ufshcd_try_to_abort_task(struct ufs_hba *hba, int tag) goto out; } - err = ufshcd_clear_cmds(hba, 1U << tag); + err = ufshcd_clear_cmds(hba, tag); if (err) dev_err(hba->dev, "%s: Failed clearing cmd at tag %d, err %d\n", __func__, tag, err); From patchwork Wed May 10 05:24:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bao D. Nguyen" X-Patchwork-Id: 13236453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A718C77B7D for ; Wed, 10 May 2023 05:26:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235006AbjEJF0j (ORCPT ); Wed, 10 May 2023 01:26:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229524AbjEJF0i (ORCPT ); Wed, 10 May 2023 01:26:38 -0400 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3D42BE46; Tue, 9 May 2023 22:26:29 -0700 (PDT) Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34A4U1l6001650; Wed, 10 May 2023 05:26:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=c7bsNq9ZdVyKR02g6lMk4hcNqeHMTnTfKhkL4rmG6hs=; b=MpCi4e83Cgl8nS+U8fV4J1lHi97B7y7msDalj14vZ4PX1/H7RNBuRh6s1/XeVbv5OYE7 dmGjWm1Y/0G9N6LUmp6QKmdWbHof9WfVsrFWhdMxilnnMLU30MIYO8jm2VAQNturS2Dm gNi8BU0LVO0z1GAAThYwnDjr4zPRqg/54XQNKKOoiRTeIdsFU8c3o5hZej0cGYZnrm34 2R7WB88xW/VLLmJoUz3WaLH+wOHA/6JFcmDXyQ188tcrRVPT8lcrW15Wa/hA+70/T+lK EgUFxSavs1zJHSDvXvo6EGOouZuNRYV/VGk7nC5Wf1C3etgjHnF1gbXKbNNDUKzscKw+ zw== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3qfrvm1buv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:26:03 +0000 Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 34A5Q2DO024952 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:26:02 GMT Received: from stor-berry.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 9 May 2023 22:26:02 -0700 From: "Bao D. Nguyen" To: , , , , , , , , CC: , "Bao D. Nguyen" , Alim Akhtar , "James E.J. Bottomley" , Alice Chao , Arthur Simchaev , Yoshihiro Shimoda , Eric Biggers , Kiwoong Kim , open list Subject: [PATCH v3 3/7] ufs: mcq: Add supporting functions for mcq abort Date: Tue, 9 May 2023 22:24:24 -0700 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: b5e5E__dH876xRZHzLpbaU-OcTOy6oYi X-Proofpoint-GUID: b5e5E__dH876xRZHzLpbaU-OcTOy6oYi X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-10_02,2023-05-05_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 bulkscore=0 mlxscore=0 suspectscore=0 lowpriorityscore=0 phishscore=0 adultscore=0 spamscore=0 mlxlogscore=999 clxscore=1011 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305100042 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add supporting functions to handle ufs abort in mcq mode. Signed-off-by: Bao D. Nguyen --- drivers/ufs/core/ufs-mcq.c | 170 +++++++++++++++++++++++++++++++++++++++++ drivers/ufs/core/ufshcd-priv.h | 10 +++ drivers/ufs/core/ufshcd.c | 1 - include/ufs/ufshcd.h | 3 + include/ufs/ufshci.h | 17 +++++ 5 files changed, 200 insertions(+), 1 deletion(-) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 202ff71..cba537c5 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -12,6 +12,10 @@ #include #include #include "ufshcd-priv.h" +#include +#include +#include +#include #define MAX_QUEUE_SUP GENMASK(7, 0) #define UFS_MCQ_MIN_RW_QUEUES 2 @@ -27,6 +31,9 @@ #define MCQ_ENTRY_SIZE_IN_DWORD 8 #define CQE_UCD_BA GENMASK_ULL(63, 7) +/* Max mcq register polling time in microseconds */ +#define MCQ_POLL_US 500000 + static int rw_queue_count_set(const char *val, const struct kernel_param *kp) { return param_set_uint_minmax(val, kp, UFS_MCQ_MIN_RW_QUEUES, @@ -419,6 +426,7 @@ int ufshcd_mcq_init(struct ufs_hba *hba) hwq->max_entries = hba->nutrs; spin_lock_init(&hwq->sq_lock); spin_lock_init(&hwq->cq_lock); + mutex_init(&hwq->sq_mutex); } /* The very first HW queue serves device commands */ @@ -429,3 +437,165 @@ int ufshcd_mcq_init(struct ufs_hba *hba) host->host_tagset = 1; return 0; } + +static int ufshcd_mcq_sq_stop(struct ufs_hba *hba, struct ufs_hw_queue *hwq) +{ + void __iomem *reg; + u32 id = hwq->id, val; + int err; + + writel(SQ_STOP, mcq_opr_base(hba, OPR_SQD, id) + REG_SQRTC); + reg = mcq_opr_base(hba, OPR_SQD, id) + REG_SQRTS; + err = read_poll_timeout(readl, val, val & SQ_STS, 20, + MCQ_POLL_US, false, reg); + if (err) + dev_err(hba->dev, "%s: failed. hwq-id=%d, err=%d\n", + __func__, id, err); + return err; +} + +static int ufshcd_mcq_sq_start(struct ufs_hba *hba, struct ufs_hw_queue *hwq) +{ + void __iomem *reg; + u32 id = hwq->id, val; + int err; + + writel(SQ_START, mcq_opr_base(hba, OPR_SQD, id) + REG_SQRTC); + reg = mcq_opr_base(hba, OPR_SQD, id) + REG_SQRTS; + err = read_poll_timeout(readl, val, !(val & SQ_STS), 20, + MCQ_POLL_US, false, reg); + if (err) + dev_err(hba->dev, "%s: failed. hwq-id=%d, err=%d\n", + __func__, id, err); + return err; +} + +/** + * ufshcd_mcq_sq_cleanup - Clean up submission queue resources + * associated with the pending command. + * @hba - per adapter instance. + * @task_tag - The command's task tag. + * @result - Result of the clean up operation. + * + * Returns 0 and result on completion. Returns error code if + * the operation fails. + */ +int ufshcd_mcq_sq_cleanup(struct ufs_hba *hba, int task_tag, int *result) +{ + struct ufshcd_lrb *lrbp = &hba->lrb[task_tag]; + struct scsi_cmnd *cmd = lrbp->cmd; + struct ufs_hw_queue *hwq; + void __iomem *reg, *opr_sqd_base; + u32 nexus, id, val; + int err; + + if (task_tag != hba->nutrs - UFSHCD_NUM_RESERVED) { + if (!cmd) + return FAILED; + hwq = ufshcd_mcq_req_to_hwq(hba, scsi_cmd_to_rq(cmd)); + } else { + hwq = hba->dev_cmd_queue; + } + + id = hwq->id; + + mutex_lock(&hwq->sq_mutex); + + /* stop the SQ fetching before working on it */ + err = ufshcd_mcq_sq_stop(hba, hwq); + if (err) + goto unlock; + + /* SQCTI = EXT_IID, IID, LUN, Task Tag */ + nexus = lrbp->lun << 8 | task_tag; + opr_sqd_base = mcq_opr_base(hba, OPR_SQD, id); + writel(nexus, opr_sqd_base + REG_SQCTI); + + /* SQRTCy.ICU = 1 */ + writel(SQ_ICU, opr_sqd_base + REG_SQRTC); + + /* Poll SQRTSy.CUS = 1. Return result from SQRTSy.RTC */ + reg = opr_sqd_base + REG_SQRTS; + err = read_poll_timeout(readl, val, val & SQ_CUS, 20, + MCQ_POLL_US, false, reg); + if (err) + dev_err(hba->dev, "%s: failed. hwq=%d, lun=0x%x, tag=%d\n", + __func__, id, lrbp->lun, task_tag); + + *result = FIELD_GET(SQ_ICU_ERR_CODE_MASK, readl(reg)); + + if (ufshcd_mcq_sq_start(hba, hwq)) + err = FAILED; + +unlock: + mutex_unlock(&hwq->sq_mutex); + return err; +} + +/** + * ufshcd_mcq_nullify_sqe - Nullify the submission queue entry. + * Write the sqe's Command Type to 0xF. The host controller will not + * fetch any sqe with Command Type = 0xF. + * + * @utrd - UTP Transfer Request Descriptor to be nullified. + */ +static void ufshcd_mcq_nullify_sqe(struct utp_transfer_req_desc *utrd) +{ + u32 dword_0; + + dword_0 = le32_to_cpu(utrd->header.dword_0); + dword_0 &= ~UPIU_COMMAND_TYPE_MASK; + dword_0 |= FIELD_PREP(UPIU_COMMAND_TYPE_MASK, 0xF); + utrd->header.dword_0 = cpu_to_le32(dword_0); +} + +/** + * ufshcd_mcq_sqe_search - Search for the command in the submission queue + * If the command is in the submission queue and not issued to the device yet, + * nullify the sqe so the host controller will skip fetching the sqe. + * + * @hba - per adapter instance. + * @hwq - Hardware Queue to be searched. + * @task_tag - The command's task tag. + * + * Returns true if the SQE containing the command is present in the SQ + * (not fetched by the controller); returns false if the SQE is not in the SQ. + */ +static bool ufshcd_mcq_sqe_search(struct ufs_hba *hba, + struct ufs_hw_queue *hwq, int task_tag) +{ + struct ufshcd_lrb *lrbp = &hba->lrb[task_tag]; + struct utp_transfer_req_desc *utrd; + u32 mask = hwq->max_entries - 1; + __le64 cmd_desc_base_addr; + bool ret = false; + u64 addr, match; + u32 sq_head_slot; + + mutex_lock(&hwq->sq_mutex); + + ufshcd_mcq_sq_stop(hba, hwq); + sq_head_slot = ufshcd_mcq_get_sq_head_slot(hwq); + if (sq_head_slot == hwq->sq_tail_slot) + goto out; + + cmd_desc_base_addr = lrbp->utr_descriptor_ptr->command_desc_base_addr; + addr = le64_to_cpu(cmd_desc_base_addr) & CQE_UCD_BA; + + while (sq_head_slot != hwq->sq_tail_slot) { + utrd = hwq->sqe_base_addr + + sq_head_slot * sizeof(struct utp_transfer_req_desc); + match = le64_to_cpu(utrd->command_desc_base_addr) & CQE_UCD_BA; + if (addr == match) { + ufshcd_mcq_nullify_sqe(utrd); + ret = true; + goto out; + } + sq_head_slot = (sq_head_slot + 1) & mask; + } + +out: + ufshcd_mcq_sq_start(hba, hwq); + mutex_unlock(&hwq->sq_mutex); + return ret; +} diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index d53b93c2..f8beabb 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -78,6 +78,8 @@ struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, struct ufs_hw_queue *hwq); +int ufshcd_mcq_sq_cleanup(struct ufs_hba *hba, int task_tag, int *result); + #define UFSHCD_MCQ_IO_QUEUE_OFFSET 1 #define SD_ASCII_STD true #define SD_RAW false @@ -404,4 +406,12 @@ static inline struct cq_entry *ufshcd_mcq_cur_cqe(struct ufs_hw_queue *q) return cqe + q->cq_head_slot; } + +static inline u32 ufshcd_mcq_get_sq_head_slot(struct ufs_hw_queue *q) +{ + u32 val = readl(q->mcq_sq_head); + + return val / sizeof(struct utp_transfer_req_desc); +} + #endif /* _UFSHCD_PRIV_H_ */ diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 3a7e853..07a6974 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -173,7 +173,6 @@ EXPORT_SYMBOL_GPL(ufshcd_dump_regs); enum { UFSHCD_MAX_CHANNEL = 0, UFSHCD_MAX_ID = 1, - UFSHCD_NUM_RESERVED = 1, UFSHCD_CMD_PER_LUN = 32 - UFSHCD_NUM_RESERVED, UFSHCD_CAN_QUEUE = 32 - UFSHCD_NUM_RESERVED, }; diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 721ae4c..12518c9 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -1088,6 +1088,7 @@ struct ufs_hba { * @cq_tail_slot: current slot to which CQ tail pointer is pointing * @cq_head_slot: current slot to which CQ head pointer is pointing * @cq_lock: Synchronize between multiple polling instances + * @sq_mutex: prevent submission queue concurrent access */ struct ufs_hw_queue { void __iomem *mcq_sq_head; @@ -1106,6 +1107,8 @@ struct ufs_hw_queue { u32 cq_tail_slot; u32 cq_head_slot; spinlock_t cq_lock; + /* prevent concurrent access to sq hw */ + struct mutex sq_mutex; }; static inline bool is_mcq_enabled(struct ufs_hba *hba) diff --git a/include/ufs/ufshci.h b/include/ufs/ufshci.h index 7c5a76b..9d291ca 100644 --- a/include/ufs/ufshci.h +++ b/include/ufs/ufshci.h @@ -99,6 +99,9 @@ enum { enum { REG_SQHP = 0x0, REG_SQTP = 0x4, + REG_SQRTC = 0x8, + REG_SQCTI = 0xC, + REG_SQRTS = 0x10, }; enum { @@ -111,12 +114,26 @@ enum { REG_CQIE = 0x4, }; +enum { + SQ_START = 0x0, + SQ_STOP = 0x1, + SQ_ICU = 0x2, +}; + +enum { + SQ_STS = 0x1, + SQ_CUS = 0x2, +}; + +#define SQ_ICU_ERR_CODE_MASK GENMASK(7, 4) +#define UPIU_COMMAND_TYPE_MASK GENMASK(31, 28) #define UFS_MASK(mask, offset) ((mask) << (offset)) /* UFS Version 08h */ #define MINOR_VERSION_NUM_MASK UFS_MASK(0xFFFF, 0) #define MAJOR_VERSION_NUM_MASK UFS_MASK(0xFFFF, 16) +#define UFSHCD_NUM_RESERVED 1 /* * Controller UFSHCI version * - 2.x and newer use the following scheme: From patchwork Wed May 10 05:24:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bao D. Nguyen" X-Patchwork-Id: 13236455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A42C0C77B7D for ; Wed, 10 May 2023 05:26:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235541AbjEJF0z (ORCPT ); Wed, 10 May 2023 01:26:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235687AbjEJF0v (ORCPT ); Wed, 10 May 2023 01:26:51 -0400 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ACAE61FEA; Tue, 9 May 2023 22:26:47 -0700 (PDT) Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34A4krAd006126; Wed, 10 May 2023 05:26:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=K4bYyppKwoa4ZdL27Uewmba3TVmcf4ezt2saUpRoh4w=; b=h86iSffUlnFP7ULEJwuAVAe+V3LTLuYXn73uoXSM+8tGrEy6bqpRTnoLWTD3s+tp+6hS ug7bHB3umY3URBNoBFwXp+tx7g7ASPg5ASlY5iN8hIPMfUU7iUisUAjFDtwZiojlSLAx v+uWB82AVZ3D/c6NLHhuQqNCznzvCoPA3PFVSW6EcMR8DctGZl0AyV16gNt8fvtC2ep0 boJpbUetzYp3tIft94NoR1L+b1cd0vImdLOcXb1pO6n7i1Tq96FEi83G7YuJ8XvVwSs4 hyZEYXxNDlZZ4sd9ciPFFtesmZbjBD2zUvblPJ2DyUB0cReFbcN5XGyDrn1VxK2V8upV og== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3qfrvm1bv3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:26:14 +0000 Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 34A5QDes025207 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:26:13 GMT Received: from stor-berry.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 9 May 2023 22:26:12 -0700 From: "Bao D. Nguyen" To: , , , , , , , , CC: , "Bao D. Nguyen" , Alim Akhtar , "James E.J. Bottomley" , Arthur Simchaev , open list Subject: [PATCH v3 4/7] ufs: mcq: Add support for clean up mcq resources Date: Tue, 9 May 2023 22:24:25 -0700 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: mpzDUf4beP644zaPx7tUw5LSptPKob0d X-Proofpoint-GUID: mpzDUf4beP644zaPx7tUw5LSptPKob0d X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-10_02,2023-05-05_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 bulkscore=0 mlxscore=0 suspectscore=0 lowpriorityscore=0 phishscore=0 adultscore=0 spamscore=0 mlxlogscore=999 clxscore=1015 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305100042 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Update ufshcd_clear_cmds() to clean up the mcq resources similar to the function ufshcd_utrl_clear() does for sdb mode. Update ufshcd_try_to_abort_task() to support mcq mode so that this function can be invoked in either mcq or sdb mode. Signed-off-by: Bao D. Nguyen --- drivers/ufs/core/ufshcd-priv.h | 2 +- drivers/ufs/core/ufshcd.c | 56 +++++++++++++++++++++++++++++++++++++++--- 2 files changed, 54 insertions(+), 4 deletions(-) diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index f8beabb..7d2104d 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -77,7 +77,7 @@ struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, struct request *req); unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, struct ufs_hw_queue *hwq); - +bool ufshcd_cmd_inflight(struct scsi_cmnd *cmd); int ufshcd_mcq_sq_cleanup(struct ufs_hba *hba, int task_tag, int *result); #define UFSHCD_MCQ_IO_QUEUE_OFFSET 1 diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 07a6974..834b13ae 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -3005,6 +3005,26 @@ static int ufshcd_compose_dev_cmd(struct ufs_hba *hba, } /* + * Check with the block layer if the command is inflight + * @cmd: command to check. + * + * Returns true if command is inflight; false if not. + */ +bool ufshcd_cmd_inflight(struct scsi_cmnd *cmd) +{ + struct request *rq; + + if (!cmd) + return false; + + rq = scsi_cmd_to_rq(cmd); + if (!rq || !blk_mq_request_started(rq)) + return false; + + return true; +} + +/* * Clear the pending command in the controller and wait until * the controller confirms that the command has been cleared. * @hba: per adapter instance @@ -3013,8 +3033,23 @@ static int ufshcd_compose_dev_cmd(struct ufs_hba *hba, static int ufshcd_clear_cmds(struct ufs_hba *hba, u32 task_tag) { unsigned long flags; + int err, result = FAILED; u32 mask = 1U << task_tag; + if (is_mcq_enabled(hba)) { + /* + * MCQ mode. Clean up the MCQ resources similar to + * what the ufshcd_utrl_clear() does for SDB mode. + */ + err = ufshcd_mcq_sq_cleanup(hba, task_tag, &result); + if (err || result) { + dev_err(hba->dev, "%s: failed tag=%d. err=%d, result=%d\n", + __func__, task_tag, err, result); + return FAILED; + } + return 0; + } + /* clear outstanding transaction before retry */ spin_lock_irqsave(hba->host->host_lock, flags); ufshcd_utrl_clear(hba, mask); @@ -7384,6 +7419,20 @@ static int ufshcd_try_to_abort_task(struct ufs_hba *hba, int tag) */ dev_err(hba->dev, "%s: cmd at tag %d not pending in the device.\n", __func__, tag); + if (is_mcq_enabled(hba)) { + /* MCQ mode */ + if (ufshcd_cmd_inflight(lrbp->cmd)) { + /* sleep for max. 200us same delay as in SDB mode */ + usleep_range(100, 200); + continue; + } + /* command completed already */ + dev_err(hba->dev, "%s: cmd at tag=%d is cleared.\n", + __func__, tag); + goto out; + } + + /* Single Doorbell Mode */ reg = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); if (reg & (1 << tag)) { /* sleep for max. 200us to stabilize */ @@ -7450,8 +7499,8 @@ static int ufshcd_abort(struct scsi_cmnd *cmd) ufshcd_hold(hba, false); reg = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); - /* If command is already aborted/completed, return FAILED. */ - if (!(test_bit(tag, &hba->outstanding_reqs))) { + if (!is_mcq_enabled(hba) && !test_bit(tag, &hba->outstanding_reqs)) { + /* If command is already aborted/completed, return FAILED. */ dev_err(hba->dev, "%s: cmd at tag %d already completed, outstanding=0x%lx, doorbell=0x%x\n", __func__, tag, hba->outstanding_reqs, reg); @@ -7480,7 +7529,8 @@ static int ufshcd_abort(struct scsi_cmnd *cmd) } hba->req_abort_count++; - if (!(reg & (1 << tag))) { + if (!is_mcq_enabled(hba) && !(reg & (1 << tag))) { + /* only execute this code in single doorbell mode */ dev_err(hba->dev, "%s: cmd was completed, but without a notifying intr, tag = %d", __func__, tag); From patchwork Wed May 10 05:24:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bao D. Nguyen" X-Patchwork-Id: 13236454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB7ADC77B7C for ; Wed, 10 May 2023 05:26:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235731AbjEJF0x (ORCPT ); Wed, 10 May 2023 01:26:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235502AbjEJF0o (ORCPT ); Wed, 10 May 2023 01:26:44 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 623E62717; Tue, 9 May 2023 22:26:43 -0700 (PDT) Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34A5PGu8031849; Wed, 10 May 2023 05:26:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=bqSZIJVOXTsS04m4LGY46Jr3ZXdMzAf5b/MEq+LA/wQ=; b=eGKGEsd/vaqt1eYhrBIP+I1Ung2K3bOzAq20kvn8443mHBjR8m3MZkpJLsaxFu3zI+am ttJj0sY+lIAQlEbW3Dx5/3TwFyGGrQwivIq1CkZC3rmRkDhqqqIUpRAXcBgjRasGeydq vL0SjQS+o+SZKv4kvS42dLdnM9s3jg2hf7gzMZaz4g12I5mmGJk3x1w9BElg/eoX2QL8 zV3jUBBEFYZ32jnnwyrnl+WPeffP0BIMTm1DAEIKNZMc4Vpnal5CBEfGqNWMaQlymQ9I AgAhRYyPWJlfr8eku2ypR0M1HsQ/1ZmlY1UGTd3hRcXmFAeYwwpcII3w4MzWdtHZ7Ye1 yQ== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3qfysdrgmu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:26:29 +0000 Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 34A5QSsm022903 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:26:28 GMT Received: from stor-berry.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 9 May 2023 22:26:28 -0700 From: "Bao D. Nguyen" To: , , , , , , , , CC: , "Bao D. Nguyen" , Alim Akhtar , "James E.J. Bottomley" , Alice Chao , Arthur Simchaev , open list Subject: [PATCH v3 5/7] ufs: mcq: Added ufshcd_mcq_abort() Date: Tue, 9 May 2023 22:24:26 -0700 Message-ID: <057f168184b4937cf11fa18c4e094773048e35d9.1683688693.git.quic_nguyenb@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: SDT4GPLaPSf8JPOWBM_rnMQsRjBHvz5D X-Proofpoint-ORIG-GUID: SDT4GPLaPSf8JPOWBM_rnMQsRjBHvz5D X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-10_02,2023-05-05_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0 phishscore=0 malwarescore=0 suspectscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 impostorscore=0 adultscore=0 priorityscore=1501 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305100042 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add ufshcd_mcq_abort() to support ufs abort in mcq mode. Signed-off-by: Bao D. Nguyen --- drivers/ufs/core/ufs-mcq.c | 60 ++++++++++++++++++++++++++++++++++++++++++ drivers/ufs/core/ufshcd-priv.h | 5 +++- drivers/ufs/core/ufshcd.c | 13 ++++++--- 3 files changed, 73 insertions(+), 5 deletions(-) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index cba537c5..8a5385c 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -599,3 +599,63 @@ static bool ufshcd_mcq_sqe_search(struct ufs_hba *hba, mutex_unlock(&hwq->sq_mutex); return ret; } + +/** + * ufshcd_mcq_abort - Abort the command in MCQ. + * @cmd - The command to be aborted. + * + * Returns SUCCESS or FAILED error codes + */ +int ufshcd_mcq_abort(struct scsi_cmnd *cmd) +{ + struct Scsi_Host *host = cmd->device->host; + struct ufs_hba *hba = shost_priv(host); + int tag = scsi_cmd_to_rq(cmd)->tag; + struct ufshcd_lrb *lrbp = &hba->lrb[tag]; + struct ufs_hw_queue *hwq; + int err = FAILED; + + if (!ufshcd_cmd_inflight(lrbp->cmd)) { + dev_err(hba->dev, + "%s: skip abort. cmd at tag %d already completed.\n", + __func__, tag); + goto out; + } + + /* Skip task abort in case previous aborts failed and report failure */ + if (lrbp->req_abort_skip) { + dev_err(hba->dev, "%s: skip abort. tag %d failed earlier\n", + __func__, tag); + goto out; + } + + hwq = ufshcd_mcq_req_to_hwq(hba, scsi_cmd_to_rq(cmd)); + + if (ufshcd_mcq_sqe_search(hba, hwq, tag)) { + /* + * Failure. The command should not be "stuck" in SQ for + * a long time which resulted in command being aborted. + */ + dev_err(hba->dev, "%s: cmd found in sq. hwq=%d, tag=%d\n", + __func__, hwq->id, tag); + goto out; + } + + /* + * The command is not in the submission queue, and it is not + * in the completion queue either. Query the device to see if + * the command is being processed in the device. + */ + if (ufshcd_try_to_abort_task(hba, tag)) { + dev_err(hba->dev, "%s: device abort failed %d\n", __func__, err); + lrbp->req_abort_skip = true; + goto out; + } + + err = SUCCESS; + if (ufshcd_cmd_inflight(lrbp->cmd)) + ufshcd_release_scsi_cmd(hba, lrbp); + +out: + return err; +} diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 7d2104d..0c034d9 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -79,7 +79,10 @@ unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, struct ufs_hw_queue *hwq); bool ufshcd_cmd_inflight(struct scsi_cmnd *cmd); int ufshcd_mcq_sq_cleanup(struct ufs_hba *hba, int task_tag, int *result); - +int ufshcd_mcq_abort(struct scsi_cmnd *cmd); +int ufshcd_try_to_abort_task(struct ufs_hba *hba, int tag); +void ufshcd_release_scsi_cmd(struct ufs_hba *hba, + struct ufshcd_lrb *lrbp); #define UFSHCD_MCQ_IO_QUEUE_OFFSET 1 #define SD_ASCII_STD true #define SD_RAW false diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 834b13ae..64dc2b2 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -300,7 +300,6 @@ static int ufshcd_setup_hba_vreg(struct ufs_hba *hba, bool on); static int ufshcd_setup_vreg(struct ufs_hba *hba, bool on); static inline int ufshcd_config_vreg_hpm(struct ufs_hba *hba, struct ufs_vreg *vreg); -static int ufshcd_try_to_abort_task(struct ufs_hba *hba, int tag); static void ufshcd_wb_toggle_buf_flush_during_h8(struct ufs_hba *hba, bool enable); static void ufshcd_hba_vreg_set_lpm(struct ufs_hba *hba); @@ -5457,8 +5456,8 @@ static irqreturn_t ufshcd_uic_cmd_compl(struct ufs_hba *hba, u32 intr_status) } /* Release the resources allocated for processing a SCSI command. */ -static void ufshcd_release_scsi_cmd(struct ufs_hba *hba, - struct ufshcd_lrb *lrbp) +void ufshcd_release_scsi_cmd(struct ufs_hba *hba, + struct ufshcd_lrb *lrbp) { struct scsi_cmnd *cmd = lrbp->cmd; @@ -7396,7 +7395,7 @@ static void ufshcd_set_req_abort_skip(struct ufs_hba *hba, unsigned long bitmap) * * Returns zero on success, non-zero on failure */ -static int ufshcd_try_to_abort_task(struct ufs_hba *hba, int tag) +int ufshcd_try_to_abort_task(struct ufs_hba *hba, int tag) { struct ufshcd_lrb *lrbp = &hba->lrb[tag]; int err = 0; @@ -7556,6 +7555,12 @@ static int ufshcd_abort(struct scsi_cmnd *cmd) goto release; } + if (is_mcq_enabled(hba)) { + /* MCQ mode. Branch off to handle abort for mcq mode */ + err = ufshcd_mcq_abort(cmd); + goto release; + } + /* Skip task abort in case previous aborts failed and report failure */ if (lrbp->req_abort_skip) { dev_err(hba->dev, "%s: skipping abort\n", __func__); From patchwork Wed May 10 05:24:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bao D. Nguyen" X-Patchwork-Id: 13236456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50761C77B7C for ; Wed, 10 May 2023 05:27:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235850AbjEJF1W (ORCPT ); Wed, 10 May 2023 01:27:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235806AbjEJF1R (ORCPT ); Wed, 10 May 2023 01:27:17 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8892B5B97; Tue, 9 May 2023 22:27:07 -0700 (PDT) Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34A5K1cA012210; Wed, 10 May 2023 05:26:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=OBzrTfBelqa+eLXS81DpdWcoyw7ODaoD9+6BGPsVYvk=; b=G84UxILIdTtkVNIzJVNhY3JV9+sT0oFEOaRv65IlE4cIpEBjI0DHriORgTigD/6Z4Xyp TE6LozaV/PVLdS94YkHPasHuqk4Col+dI0GorbsLZwhI3WdE5XzKcUZ7S9ecrOMgitQd ob2TVRNFPyoilmdF/7kln4oKVLNUFfh90OJAj08opPHino21YcrbPMDCZT5nRAiDKdlf MKWLNTDvYu/fVoj6XhB3EQQr50lnMiTEdqDwOYefETK9P/ohBWxQ8dXOiEHOkVxaQZ6Q KO+7KxdNUv/f1fVmKOTap2urGMFfQ/pwVQ6aYOgPP9PPHNAPCsoXiJl2bJBXZ3ZlIC/7 mg== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3qfr509h1e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:26:52 +0000 Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 34A5QpSG010311 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:26:51 GMT Received: from stor-berry.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 9 May 2023 22:26:50 -0700 From: "Bao D. Nguyen" To: , , , , , , , , CC: , "Bao D. Nguyen" , Alim Akhtar , "James E.J. Bottomley" , Andy Gross , Bjorn Andersson , Konrad Dybcio , Alice Chao , Arthur Simchaev , "open list" , "open list:UNIVERSAL FLASH STORAGE HOST CONTROLLER DRIVER..." Subject: [PATCH v3 6/7] ufs: mcq: Use ufshcd_mcq_poll_cqe_lock() in mcq mode Date: Tue, 9 May 2023 22:24:27 -0700 Message-ID: <0dfd125ec058aae179119672da46e297c4a3e35e.1683688693.git.quic_nguyenb@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 0MamCw7altpvIM7JCjnrdUgYMMnSRDQL X-Proofpoint-ORIG-GUID: 0MamCw7altpvIM7JCjnrdUgYMMnSRDQL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-10_02,2023-05-05_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 bulkscore=0 lowpriorityscore=0 malwarescore=0 clxscore=1011 impostorscore=0 adultscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305100042 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In preparation for adding mcq error handler support, update the mcq code to use the ufshcd_mcq_poll_cqe_lock() in interrupt context instead of using ufshcd_mcq_poll_cqe_nolock(). This is to keep synchronization between mcq interrupt and error handler contexts because both need to access the mcq hardware in separate contexts. Signed-off-by: Bao D. Nguyen --- drivers/ufs/core/ufs-mcq.c | 6 +++--- drivers/ufs/core/ufshcd-priv.h | 2 -- drivers/ufs/core/ufshcd.c | 2 +- drivers/ufs/host/ufs-qcom.c | 2 +- include/ufs/ufshcd.h | 2 +- 5 files changed, 6 insertions(+), 8 deletions(-) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 8a5385c..138e174 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -284,8 +284,8 @@ static void ufshcd_mcq_process_cqe(struct ufs_hba *hba, ufshcd_compl_one_cqe(hba, tag, cqe); } -unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, - struct ufs_hw_queue *hwq) +static unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq) { unsigned long completed_reqs = 0; @@ -301,7 +301,6 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, return completed_reqs; } -EXPORT_SYMBOL_GPL(ufshcd_mcq_poll_cqe_nolock); unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, struct ufs_hw_queue *hwq) @@ -314,6 +313,7 @@ unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, return completed_reqs; } +EXPORT_SYMBOL_GPL(ufshcd_mcq_poll_cqe_lock); void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba) { diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 0c034d9..b73245e 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -71,8 +71,6 @@ void ufshcd_mcq_config_mac(struct ufs_hba *hba, u32 max_active_cmds); void ufshcd_mcq_select_mcq_mode(struct ufs_hba *hba); u32 ufshcd_mcq_read_cqis(struct ufs_hba *hba, int i); void ufshcd_mcq_write_cqis(struct ufs_hba *hba, u32 val, int i); -unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, - struct ufs_hw_queue *hwq); struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, struct request *req); unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 64dc2b2..5e3029ed 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -6811,7 +6811,7 @@ static irqreturn_t ufshcd_handle_mcq_cq_events(struct ufs_hba *hba) ufshcd_mcq_write_cqis(hba, events, i); if (events & UFSHCD_MCQ_CQIS_TAIL_ENT_PUSH_STS) - ufshcd_mcq_poll_cqe_nolock(hba, hwq); + ufshcd_mcq_poll_cqe_lock(hba, hwq); } return IRQ_HANDLED; diff --git a/drivers/ufs/host/ufs-qcom.c b/drivers/ufs/host/ufs-qcom.c index 82d02e7..57f5674 100644 --- a/drivers/ufs/host/ufs-qcom.c +++ b/drivers/ufs/host/ufs-qcom.c @@ -1556,7 +1556,7 @@ static irqreturn_t ufs_qcom_mcq_esi_handler(int irq, void *__hba) struct ufs_hw_queue *hwq = &hba->uhq[id]; ufshcd_mcq_write_cqis(hba, 0x1, id); - ufshcd_mcq_poll_cqe_nolock(hba, hwq); + ufshcd_mcq_poll_cqe_lock(hba, hwq); return IRQ_HANDLED; } diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 12518c9..a6781b0 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -1244,7 +1244,7 @@ void ufshcd_update_evt_hist(struct ufs_hba *hba, u32 id, u32 val); void ufshcd_hba_stop(struct ufs_hba *hba); void ufshcd_schedule_eh_work(struct ufs_hba *hba); void ufshcd_mcq_write_cqis(struct ufs_hba *hba, u32 val, int i); -unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, struct ufs_hw_queue *hwq); void ufshcd_mcq_enable_esi(struct ufs_hba *hba); void ufshcd_mcq_config_esi(struct ufs_hba *hba, struct msi_msg *msg); From patchwork Wed May 10 05:24:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bao D. Nguyen" X-Patchwork-Id: 13236457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62E9AC77B7C for ; Wed, 10 May 2023 05:27:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235883AbjEJF1Z (ORCPT ); Wed, 10 May 2023 01:27:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235606AbjEJF1V (ORCPT ); Wed, 10 May 2023 01:27:21 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 084E81FEF; Tue, 9 May 2023 22:27:10 -0700 (PDT) Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34A2HR7d020674; Wed, 10 May 2023 05:26:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=2eGlXlvn5UzqxcldaXw0YYLt5bMTRsK+KHOAM3hK55Y=; b=M31Dte0CUiCc1VEdtRH/YgXksCmY5GDg/JkWgtd2eC/ZzHKVekpJXp0KhhBJccXMKl6P YU1t1KMxWLAI1iHdGvG3Zma6NPvpgchKajzOHHV+l8T7VvpHRrKUxY1gtfBWsdxV/De7 XqrwhwGQSkBUE/pY8A3YcHUq4cKlGjGzXzXgjcm+Oe2PX/hKRoHJj59Cc6yXuEZH1kjK lrxSLekI4he6CrCjicOtZXTIa7ouER5OAwYZNu+myoYASkw7L/t5ZTk3s/Xu8WU7VJk1 Y8PXbLqu7tJsnvHvD96keEsYdEWm7afJ/kPPOFA05TbtKyPCksNIGhL4Y2iPbDZ4xtqD 7w== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3qfr509h1h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:26:57 +0000 Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 34A5Qudq023106 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 May 2023 05:26:56 GMT Received: from stor-berry.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 9 May 2023 22:26:56 -0700 From: "Bao D. Nguyen" To: , , , , , , , , CC: , "Bao D. Nguyen" , Alim Akhtar , "James E.J. Bottomley" , open list Subject: [PATCH v3 7/7] ufs: core: Add error handling for MCQ mode Date: Tue, 9 May 2023 22:24:28 -0700 Message-ID: <09428bd2ee5f9c832046142c9c6d13efebab8887.1683688693.git.quic_nguyenb@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: yWJmVWtvnAU3Ev08Ury2ajigfNMxVeFT X-Proofpoint-ORIG-GUID: yWJmVWtvnAU3Ev08Ury2ajigfNMxVeFT X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-10_02,2023-05-05_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 bulkscore=0 lowpriorityscore=0 malwarescore=0 clxscore=1015 impostorscore=0 adultscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305100042 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add support for error handling for MCQ mode. Signed-off-by: Bao D. Nguyen --- drivers/ufs/core/ufshcd.c | 85 +++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 74 insertions(+), 11 deletions(-) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 5e3029ed..0c6dec67 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -3148,6 +3148,16 @@ static int ufshcd_wait_for_dev_cmd(struct ufs_hba *hba, err = -ETIMEDOUT; dev_dbg(hba->dev, "%s: dev_cmd request timedout, tag %d\n", __func__, lrbp->task_tag); + + /* MCQ mode */ + if (is_mcq_enabled(hba)) { + err = ufshcd_clear_cmds(hba, lrbp->task_tag); + if (!err) + hba->dev_cmd.complete = NULL; + return err; + } + + /* SDB mode */ if (ufshcd_clear_cmds(hba, lrbp->task_tag) == 0) { /* successfully cleared the command, retry if needed */ err = -EAGAIN; @@ -5581,6 +5591,10 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num) */ static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) { + struct ufshcd_lrb *lrbp; + u32 hwq_num, utag; + int tag; + /* Resetting interrupt aggregation counters first and reading the * DOOR_BELL afterward allows us to handle all the completed requests. * In order to prevent other interrupts starvation the DB is read once @@ -5599,7 +5613,22 @@ static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) * Ignore the ufshcd_poll() return value and return IRQ_HANDLED since we * do not want polling to trigger spurious interrupt complaints. */ - ufshcd_poll(hba->host, UFSHCD_POLL_FROM_INTERRUPT_CONTEXT); + if (!is_mcq_enabled(hba)) { + ufshcd_poll(hba->host, UFSHCD_POLL_FROM_INTERRUPT_CONTEXT); + goto out; + } + + /* MCQ mode */ + for (tag = 0; tag < hba->nutrs; tag++) { + lrbp = &hba->lrb[tag]; + if (ufshcd_cmd_inflight(lrbp->cmd)) { + utag = blk_mq_unique_tag(scsi_cmd_to_rq(lrbp->cmd)); + hwq_num = blk_mq_unique_tag_to_hwq(utag); + ufshcd_poll(hba->host, hwq_num); + } + } + +out: return IRQ_HANDLED; } @@ -6378,18 +6407,36 @@ static bool ufshcd_abort_all(struct ufs_hba *hba) bool needs_reset = false; int tag, ret; - /* Clear pending transfer requests */ - for_each_set_bit(tag, &hba->outstanding_reqs, hba->nutrs) { - ret = ufshcd_try_to_abort_task(hba, tag); - dev_err(hba->dev, "Aborting tag %d / CDB %#02x %s\n", tag, - hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0] : -1, - ret ? "failed" : "succeeded"); - if (ret) { - needs_reset = true; - goto out; + if (is_mcq_enabled(hba)) { + struct ufshcd_lrb *lrbp; + int tag; + + for (tag = 0; tag < hba->nutrs; tag++) { + lrbp = &hba->lrb[tag]; + if (!ufshcd_cmd_inflight(lrbp->cmd)) + continue; + ret = ufshcd_try_to_abort_task(hba, tag); + dev_err(hba->dev, "Aborting tag %d / CDB %#02x %s\n", tag, + hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0] : -1, + ret ? "failed" : "succeeded"); + if (ret) { + needs_reset = true; + goto out; + } + } + } else { + /* Clear pending transfer requests */ + for_each_set_bit(tag, &hba->outstanding_reqs, hba->nutrs) { + ret = ufshcd_try_to_abort_task(hba, tag); + dev_err(hba->dev, "Aborting tag %d / CDB %#02x %s\n", tag, + hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0] : -1, + ret ? "failed" : "succeeded"); + if (ret) { + needs_reset = true; + goto out; + } } } - /* Clear pending task management requests */ for_each_set_bit(tag, &hba->outstanding_tasks, hba->nutmrs) { if (ufshcd_clear_tm_cmd(hba, tag)) { @@ -7321,6 +7368,8 @@ static int ufshcd_eh_device_reset_handler(struct scsi_cmnd *cmd) unsigned long flags, pending_reqs = 0, not_cleared = 0; struct Scsi_Host *host; struct ufs_hba *hba; + struct ufs_hw_queue *hwq; + struct ufshcd_lrb *lrbp; u32 pos, not_cleared_mask = 0; int err; u8 resp = 0xF, lun; @@ -7336,6 +7385,20 @@ static int ufshcd_eh_device_reset_handler(struct scsi_cmnd *cmd) goto out; } + if (is_mcq_enabled(hba)) { + for (pos = 0; pos < hba->nutrs; pos++) { + lrbp = &hba->lrb[pos]; + if (ufshcd_cmd_inflight(lrbp->cmd) && + lrbp->lun == lun) { + ufshcd_clear_cmds(hba, pos); + hwq = ufshcd_mcq_req_to_hwq(hba, scsi_cmd_to_rq(lrbp->cmd)); + ufshcd_mcq_poll_cqe_lock(hba, hwq); + } + } + err = 0; + goto out; + } + /* clear the commands that were pending for corresponding LUN */ spin_lock_irqsave(&hba->outstanding_lock, flags); for_each_set_bit(pos, &hba->outstanding_reqs, hba->nutrs)